the upgrade seems to have been successfully executed - the machine was restarted at ~430pm local time. Projector remains off and diagnostic striptools are on the samsung.

Some ideas Koji suggested:

1. Try approaching the CARM offset zero point "from the other side" - i.e. start with a CARM offset of the opposite sign (I typically use negative CARM offset).
2. With the PRMI locked, try bringing one arm onto resonance while the other arm is held off resonance. 

For the second idea, it is convenient to be able to control the arms in the XARM/YARM basis as opposed to the CARM/DARM basis as we usually do when going through the locking sequence. After some fiddling, I am able to reliably execute this transition, and achieve a state where the FP arm cavities are resonant for the IR with the ALS beat note frequency being the error signal being used. Some important differences:

1. The frequency actuator (ETM) is weaker in this case than in the CARM/DARM basis (where MC2 is the frequency actuator) due to the longer length of the arm cavity (and for ETMX, the higher coil driver series resistance). I had to twiddle the limits of the servo banks to accommodate this. 
2. The ALS error signal is significantly noisier than POX/POY. Hence, the control signal RMS is often in danger of saturating the DAC range. I implemented a partial fix by adding a 1st order Butterworth LPF with 1kHz corner frequency. According to the model, this eats <5 degrees of phase at the desired UGF of ~150 Hz.

Jon and I were surveying the CDS situation so that he can prepare a report for discussion with Rolf/Rich about our upcoming BHD upgrade. In our poking around, we must have bumped something somewhere because the c1ioo machine went offline, and consequently, took all the vertex models out. I rebooted everything with the reboot script, everything seems to have come back smoothly. I took this opportunity to install some saturation counters for the arm servos, as we have for the CARM/DARM loops, because I want to use these for a watch script that catches when the ALS loses lock and shuts stuff off before kicking optics around needlessly. See Attachment #1 for my changes.

The shaking started earlier today than yesterday, at ~9pm local time.

While the IFO is shaking, I thought (as Jan Harms suggested) I'd take a look at the cross-spectra between our seismometer channels at the dominant excitation frequency, which is ~1.135 Hz. Attachment #1 shows the phase of the cross spectrum taken for 10 averages (with 30mHz resolution) during the time period when the shaking was strong yesterday (~1500 seconds with 50% overlap). The logic is that we can use the relative phasing between the seismometer channels to estimate the direction of arrival and hence, the source location. However, I already see some inconsistencies - for example, the relative phase between BS_Z and EX_Z suggests that the signal arrives at the EX seismometer first. But the phasing between EX_Y and BS_Y suggests the opposite. So maybe my thinking about the problem as 3 co-located sensors measuring plane-wave disturbances originating from the same place is too simplistic? Moreover, Koji points out that for two sensors separated by ~40m, for a ground wave velocity of 1.5 km/s, the maximum phase delay we should see between sensors is 30 msec, which corresponds to ~10 degrees of phase. I guess we have to undo the effects of the phasing in the electronics chain.

Does anyone have some code that's already attempted something similar that I can put the data through? I'd like to not get sucked into writing fresh code.

🤞 this means that the shaking is over for today and I get a few hours of locking time later today evening.

Another observarion is that even after the main 1.14 Hz peak dies out, there is elevated seismic acitivity reported by the 1-3 Hz BLRMS band. This unfortunately coincides with some stack resonance, and so the arm cavity transmission reports greater RIN even after the main peak dies out. Today, it seems that all the BLRMS return to their "nominal" nighttime levels ~10 mins after the main 1.14 Hz peak dies out.

Summary:

1. CARM offset was reduced to 0 with the PRMI locked.
2. TRY levels touched ~200 (Recycling gain ~10, IFO is still undercoupled).
3. TRX level never went so high - I suspect QPD issues or clipping in the beampath.

Details:

• Attachment #1 is a StripTool summary of the lock - encouragingly, the PRMI stayed locked for several 10s of minutes even when the CARM offset was brought to 0.
• The MICH signal was pretty glitchy - we increased the gain of the MICH and PRCL loops and thought we saw some improvement, but needs more quantitative investigation.
• Main differences in locking procedure today were:
  • Added some POPDC to the MICH/PRCL trigger elements in addition to POP22
  • Tried adding a DARM offset before doing the final steps of CARM offset reduction, and then zerod the DARM offset too.
• The TRX level never went as high as TRY - even though on the CRT monitors, both arms seemed to saturate somewhat more evenly. Potentially the EX QPD needs a checkout, or there is some clipping in the in-air TRX path. Although, puzzilingly, the POXDC level never goes as high as POYDC either. So maybe the buildup is really lower in the XARM? For the daytime tomorrow.

Next steps:

1. Check the EX QPD / TRX situation.
2. Figure out how to make the PRMI lock more stable as I reduce the CARM offset.
3. Start figuring out the CM board, as we'd want to do the handoff to RF at some point.

I took a look at the TRX/TRY RIN reported in the single arm POX/POY lock, and compared the performance of the two available PDs at each of the two ends. Attachment #1 shows the results. Some remarks:

1. The noise performance of the two QPDs at each end isn't identical - is there some transimpedance gain difference?
2. The lower plot shows the angular motion reported by each QPD when the arm cavity is locked. The EX QPD seems much more sensitive than the EY QPD.
3. I estimate that in this condition, each photodiode is receiving ~20uW of power, corresponding to a shot noise limited RIN of ~10^-7. None of the photodiodes saturate this bound.
4. There are some ND filters placed in front of the QPDs at both ends. Do we really need these ND filters? I estimate that for the highest buildups, we will have ~10kW * 15ppm * 0.5 ~75mW of power incident on the QPD, so ~20mW per segment. Assuming silicon responsivity of 0.2 A/W, a transimpedance of 1kohm would give us 4V of signal. But the schematic shows higher transimpedance. Do we still have the switching capability for this QPD?

Koji and I had noticed that there was some discrepancy between the switchable gain stages of the EX and EY QPDs. Sadly, there was no indication that these switches even exist on the QPD MEDM screen. Yehonathan and I rectified this today. Both EX and EY Transmon QPDs now have some extra info (see Attachment #1). We physically verified the indicated quadrant mapping for the EX QPD (see previous elog in this thread for the details), and I edited the screen accordingly. EY QPD still has to be checked. Note also that there is an ND=0.4 + ND=0.2 filter and some kind of 1064nm light filter installed in series on the EX QPD. The ND filters have a net transmission T~25%.

After making the EX and EY QPDs have the same switchable gain settings (I also reset the trans normalization gains), the angular motion witnessed is much more consistent between the two now - see Attachment #2. The high-frequency noise of the sum channel is somewhat higher for EX than EY - maybe the ND filters are different on the two ends?

Note that there was an extra factor of 40 gain on the EX QPD relative to EY during the lock yesterday. So really, the signals were probably just getting saturated. Now that the gains are consistent between the ends, it'll be interesting to see how balanced the buildup of the two arms is. There still remains the problem that the MICH loop was too unstable, which probably led to to excess arm power fluctuations.

There is a mark on the QPD surface that is probably dirt (since we never have such high power transmitted through the ETM to damage the QPD). I'll try cleaning it up at the next opportunity.

After the QPD fix, both arms report consistent buildup - see Attachment #1. The peak values touch ~250, corresponding to a PRG of ~13. The IFO becomes critically coupled at PRG=15. I am finding that the 3f signal offsets are changing as a function of the CARM offset, and this could be responsible for the lock breaking as I approach 0 CARM offset. I found that I could maintain a more stable and deterministic transition to zero CARM offset by dynamically adjusting the 3f PRCL error signal offset to keep the REFL11 signal approximately at 0. Some shaking seems to have commenced so I am breaking for now.

Note that I find scattered throughout the elog references to a similar problem of the PRMI losing lock as the CARM offset is reduced, e.g. here. But haven't stumbled across what the resolution was, the PRFPMI could be locked pretty easily in 2015 I remember.

In preparation for trying out some high-bandwidth Y arm cavity locking using the CM board, I hooked up the POY11_Q_Mon channel of the POY11 demod board to the IN1 of the CM board (and disconnected the usual REFL11 cable that goes to IN1). The digital phase rotation for usual POY Yarm locking is 106 degrees, so the analog POY11_Q channel contains most of the signal. I then set the IN1 gain of the servo to 0dB, and looked at the CM_Slow signal - I changed the whitening gain of this channel to +18dB (to match that used for POY11_I and POY11_Q), and found that I had to apply a digital gain of 0.5 to get the PDH horns in the usual POY11_I signal and the CM_Slow signal to line up. There was also a sign inversion. Then I was able to use the digital LSC system and lock the Y arm cavity length to the PSL frequency by actuating on ETMY, using CM_Slow as an error signal. A comparison of the in-loop POY11_I ASD when the arm is locked is shown in Attachment #1 - CMslow seems to be dominated by some kind of electroncis noise above ~100 Hz, so possibly needs more whitening (even though the nominal whitening filter is engaged)?

Anyway, now that I have this part of the servo working, the next step is to try and engage the AO path and achieve a higher BW lock of the Y arm cavity to the PSL frequency (= IMC length). Maybe it makes more sense to actuate on MC2 for the slow path.

Summary:

The Y arm cavity length was locked to the PSL frequency with ~26kHz UGF, and 25 degrees phase margin. Slow actuation was done on ETMY using CM_Slow as an error signal, while fast actuation was done on the IMC error point via the IN2 input of the IMC servo board. Attachment #1 shows the comparison of the in-loop error signal spectra with only slow actuation and with the full CM loop engaged.

Details:

1. LSC enable OFF.
2. Configure the CM board for locking:
   • CM board IN1 gain = 25dB.
   • CM_Slow whitening gain = +18dB, make sure the offsets are correctly set. CM_Slow filter bank = -0.015.
   • CM_Slow-->YARM matrix element in LSC input matrix is -2.5.
   • YARM-->ETMY matrix element in LSC Output matrix is 1.
   • AO gain set to +5dB. IMC Servo board IN2 gain starts at -32dB, the path is disabled. The polarity is Plus.
   • Usual YARM FM triggers are set (FM1, FM2, FM3, FM6, FM8), usual YARM servo gain is used (0.01), usual triggering conditions (ON @ TRY>0.3, OFF @ TRY < 0.1), usual power normalization by TRY.
3. Enable LSC mode, wait for the arm to acquire lock.
4. Once the digital boosts are engaged, enable the IMC IN2 path, ramp up the gain to -2 dB. Note that this IN2 path is AC coupled, according to this elog. The corner frequency is 1/2/pi/2e3/6.8uF ~11 Hz. This was confirmed by measurement, see Attachment #3. I couldn't find a 2-pin LEMO-->BNC adaptor so I measured at the BNC connector for the IN2 input, which according to the schematic is shorted to the LEMO (which is what we use for the AO path).
5. Enable the CM board boost.
6. Ramp up the CM board IN1 gain to +31dB. In this config, the CM_Slow signal is ~18,000 cts pk (with the +18dB whitening gain), so not saturating the ADC.
7. Ramp up the IMC IN2 gain to 3dB, engage 2 Super Boosts (can't turn on the third). Limiter is always ON.
8. Use the CM board error point offset adjust to zero the POY11_I error signal average value - there seems to be some offsets when engaging the boosts. The value I used was 0.9 V (this is internally divided by 40 on the CM board).
9. Whiten the CM_Slow signal - this doesn't seem to have any impact on the noise anywhere.

I hypothesize that the high-frequency noise (>100 Hz) is higher for POY than POX in Attachment #1 because I am using the "MON" port of the demod board - this has a gain of 2, and there could also be some flaky components in this path, hence the high frequency noise is a factor of a few greater in the POY spectrum relative to the POX spectrum (which is using the main demodulated output). For REFL11, we have a low noise preamp generating the input signal so I don't think we need to worry about this too much.

The PC Drive RMS didn't look any stranger than it usually does for the duration of the lock.

Attachment #2 shows the OLTF of the locking servo with the final gains / settings, which are in bold. The loop is maybe a bit marginal, could possibly benefit from backing off one of the super boosts. But the arm has stayed locked for >1 hour.

The purpose of this test was to verify the functionality of the CM board and also the IN2 of the IMC servo board in a low-pressure environment. Once I confirm that the modelled OLTF lines up with the measured, I will call this test a success, and move on to looking at REFL11 in the arms on ALS, PRMI on 3f config. I am returning the REFL11 signal to the input of the CM board, but the SR785 remains hooked up.

Unrelated to this work - PMC alignment was tweaked to improve input power to IMC by ~5%.

[KA, GV]

There was no shaking (that disturbed the locking) tonight!

1. REFL165 Demod phase was adjusted from -111deg to -125deg. To minimize coherence b/w MICH and PRCL.
2. MICH 3f loop gain changed to 0.3.
3. If the POP mode shape looks weird, it probably means that the PRM is sligntly misaligned. Tweaking the alignment improves PRMI stability and also makes the arm buildup higher.
4. Ditto for MICH - slightly touching up the BS alignment can lower ASDC.
5. Main finding tonight was that the ALS noise seems to get degraded as a function of the CARM offset! As a result of this, CARM goes through several linewidths, and the arm transmission fluctuates wildly.
   • We suspect some scattered light shenanigans. It is not clear to me why this is happening. Possibilities:
   • Scattered ETM transmission somehow makes it into the fiber coupler and degrades the ALS noise.
   • Sacttered ETM transmission makes it onto the Green PDH photodiode and degrades the ALS noise.
   • Backscatter into the PSL degrades the ALS noise.
   • Shadow sensors of either the ITMs, ETMs, BS, or PRM don't have 1064nm filters and get scatterd light, making the cavity length noise worse.
   • Other possibilities?

The problem is hard to debug because we are feeding back on the ETMs, BS and PRM, and at the low CARM offset (= high PRG), all the DoFs are cross coupled strongly so just by looking at error/control signals, I can't directly determine where the noise is originating. The fact that the ALS CARM spectrum shows a noise increase suggests that the problem has to do with the test masses, PSL, IMC, or end green PDH setups.

My plan is to do a systematic campaign and eliminate some of these possibilities - e.g. install some baffling around the fiber coupler and the end green PDH photodiodes and see if there is any improvement in the situation.

* In attachment #1, the "Ref" traces are when the CARM offset is 0, and the arms are buzzing in and out of resonance. The non-reference traces are for when the CARM offset is ~28kHz (so several linewidths away from resonance).

I re-checked the ALS noise in the following configurations:

• PRM is misaligned.
• Michelson is not locked.
• TRX/TRY is maintained at ~1.

1. Arm lengths are controlled using POX/POY as a sensor, and the ETMs as actuators [orange traces in Attachment #1].
   • EX laser frequency is locked to the arm cavity length using the end PDH servo.
   • ALS beat note frequency fluctuations are read out using the calibrated DFD channels.
   • In this config, the DFD outputs are the out-of-loop sensor.
2. Arm lengths are controlled using the ALS beat frequencies as a sensors [blue traces in Attachment #1]
   • The control is no longer in the XARM/YARM basis, but in the CARM/DARM basis.
   • The CARM actuator is MC2, the DARM actuator is an admixture of the ETMs (equal magnitude of output matrix element, opposite sign).
   • The calibrated POX/POY photodiodes are used as the out-of-loop sensor in this config.

The RMS noise sensed by POX/POY is ~20pm, which is somewhat higher than the best I've seen (maybe the arms are moving more at the time of measurement or the AUX PDH loops need a bit of touching up). But the orange traces in the top row of Attachment #1 are already ~x2 better than the equivalent traces from when we were using the green beams to make the beats. So it's hard to explain the 0-300 fluctuations in the arm powers when the CARM offset is reduced to 0 - i.e. the ALS noise is becoming worse as I reduce the CARM offset (= have more circulating power compared to the conditions of this test). I assume the transmission is Lorentzian, in which case even if we have 5x the CARM linewidth worth of ALS noise, we should see the arm power fluctuate between ~10 and 300. 

* I notice that a big jump in the RMS sensed by POX/POY comes from the 24 Hz peak, which is presumably the Roll mode coupling to length - maybe a ResG can make the situation better. The high frequency noise can also be probably rolled off better.

Summary:

Since we are using the POX and POY photodiodes as out-of-loop sensors for measuring the ALS noise, I decided to double-check their calibrations. I determined the following numbers (for the single arm lock):

POX_I [with 30dB whitening gain]: (8 +/- 1)e-13 m/ct

POY_I [with 18dB whitening gain]: (0.9 +/- 0.1)e-13 m/ct

With this calibration, I measured the in-loop spectra of the XARM and YARM error-points when they are locked - they line up well, see Attachment #1. Note that these numbers are close to what we determined some time ago using the same method (I drove the ITMs then, but yesterday I drove the ETMs, so maybe the more accurate measure of uncertainty is the difference between the two measurements).

Attachment #2 shows the out-of-loop spectra sensed by these photodiodes with this calibration applied, when the arms are under control using ALS beat frequencies as the error signals, and controlled in the CARM/DARM basis. Need to think about why there is such a difference between the two signals.

Methodology:

The procedure used was the same as that outlined here.

• I started by calibrating the AS55_Q output with the free-swinging Michelson.
• Next, I lock the Michelson and calibrate the BS and ITM actuators using the newly calibrated AS55_Q.
• Next, I calibrate the ETM actuator gains by measuring the ratio of response in POX/POY of driving the (unknown) ETMs and the (known) ITMs.
• Finally, I calibrate the POX/POY photodiodes by driving the ETMs by a known amount of meters (at ~310 Hz where the loop gain is negligible because of the sensing matrix measurement notches).

Summary of DC actuator gains:

The quoted values of the DC gain are for counts seen at the output of the LSC filter bank. I've attempted to show that once we account for the different series resistance and some extra gains between the output of the LSC filter bank and the actual coil, things are fairly consistent.

Some remarks:

• I do not understand why we need an extra 12dB of whitening gain on the POX channel to get similar PDH fringe height as the POY channel. The light level on these photodiodes is the same, and the RF transimpedances at 11 MHz are also close according to the wiki (3kohm for POX, 2kohm for POY).
• At night-time, the ALS noise did indeed get reduced compared to what I measured earlier in the evening.
• Even assuming 50% error in the calibration factors, it's hard to explain the swing of TRX/TRY when the CARM offset is brought to zero.
• The increase in (admittedly in-loop) CARM noise as the offset is reduced still seems to me to be correlated with the buildup of IR power in the arm cavities.

Summary:

The ITMX OSEMs report elevated noise in the 10-100 Hz band when we have high circulating power in the arm cavities, see Attachment #1. Since there is no LSC actuation on the ITMs in this state, this could be a radiation presssure effect, or could be scattered 1064nm light entering the OSEMs. The Oplevs don't report any elevated noise however. ITMY has the OSEM whitening broken for two channels, but the other two channels don't report as significant an increase as ITMX, see Attachment #2. I can't find the status of which OSEMs have the 1064nm blocking filters installed. The local damping loops are rolled off by ~100dB at 30 Hz, so the sensing noise re-injection should be attenuated by this factor, so maybe the OSEM sensor noise isn't the likely culprit. But radiation pressure didn't worsen the length noise in the past, even after our mirror cleaning and the increased PRG.

i reconnected the AOM driver to the AOM in the main beam path (it was hijacked for the AOM in the AUX laser path for Anjali's MZ experiment). I also temporarily hooked up the AOM to a CDS channel to facilitate some swept-sine measurements. This was later disconnected. The swept sine will need some hardware to convert the bipolar drive signal from the CDS system to the unipolar input that the AOM driver wants (DTT swept sine wont let me set an offset for the excitation, although awggui can do this).

The old MCL filters are not completely useless - I find a factor of ~2 reduction in the MCL RMS when I turn the FF on. It'd be interesting to see how effective the FF is during the periods of enhanced seismic activity we see. I also wonder if this means the old PRC angular FF filters are also working, it'd help locking, tbc with PRMI carrrier...

Update: The PRC angular FF loops also do some good it seems - though the PIT loop probably needs some retuning.

Summary:

I tried the following minor changes to the locking procedure to see if there were any differences in the ALS noise performance:

1. Actuate DARM only on one ETM (tried both ETMX and ETMY)
2. Enable MCL and PRC seismic feedforward
3. DC couple the ITM Oplevs for better angular stability during the lock acquisition

None of these changes had any effect - the ALS noise still goes up with arm buildup. 

I think a good way to determine if the problem is to do with the IR part of the new ALS system is to resurrect the green beat setup - I expect this to be less invasive than installing attenuators/beam dumps in front of the fiber couplers at the ends. We should at the very least recover the old ALS noise levels and we were able to lock the PRFPMI with that config. If the excess noise persists, we can rule out the problem being IR scatter into the beat-mouth fibers. Does this sound like a reasonable plan?

Attachment #1 - comparison of phase tracker servo angle fluctuations for the green beat vs IR beat.

• To convert to Hz, I used the PT servo calibration detailed here.
• This is only a function of the delay line length and not the signal strength, so shouldn't be affected by the difference in signal strength between the IR and green beats.
• For the green beat - I divided the measured spectra by 2 to convert the green beat frequency fluctuations into equivalent IR frequency fluctuations.
• There is no whitening before digitization. I believe the measured spectra are dominated by ADC noise above ~50 Hz. See this elog for the frequency discriminant as a funtion of signal strength, so 5uV/rtHz ADC noise would be ~2 Hz/rtHz for a -5dBm signal, which is what I expect for the Y beat, and ~0.5 Hz/rtHz for a +5dBm signal, which is what I expect for the X beat. Hence the brown (Green beat, XARM) being lower than the green trace (IR beat, XARM) isn't real, it is just because of my division of 2. So I guess that calibration factor I applied is misleading.
• I did not yet check the noise in the other configuration - arm lengths controlled using ALS, and POX/POY as the OOL sensors. To be tried tonight.

Attachment #2 - RIN of the DCPDs.

• I noticed that over 10s of seconds, the GTRY level was fluctuating by ~5%. 
• This was much more than any drift seen in the GTRX level.
• Measuring the RIN on the DCPDs (Thorlabs PDA36A) supports this observation (spectra were divided by DC value to convert into RIN units).
• There is ~120uW (1.6 VDC, compatible with 30dB gain setting) incident on the GTRX PD, and ~6uW (170 mVDC, compatible with 40dB gain setting) incident on the GTRY PD.
• Not sure what is driving this drift - I don't see any coherence with the IR TRY signal, so doesn't seem like it's the cavity.

Characterization of the green beat setup [past numbers]:

• With some patient alignment effort (usual near-field/far-field matching), I was able to recover the green beat signals.
• Overall, the numbers I measured today are consistent with what was seen in the past when we had the ability to lock using green ALS.
• The mode-matching between the PSL and AUX green beams are still pretty abysmal, ~40-50%. The mode shapes are clearly different, but for now, I don't worry about this.
• I saw some strong AM of the beat signal (for both EX and EY beats) while I was looking at it on a scope, see Attachment #3. This AM is not visible in the IR beat, not sure what to make of it. The frequency of the AM is ~1 MHz, but it's hard to nail this down because the scope doesn't have a very long buffer, and I didn't look at the frequency content on the Agilent (yet).

o BBPD DC output (mV), all measured with Fluke DMM

             XARM   YARM 
 V_DARK:     +1.0    +2.0
 V_PSL:      +8.0    +13.0
 V_ARM:      +157.0  +8.0

o BBPD DC photocurrent (uA)
I_DC = V_DC / R_DC ... R_DC: DC transimpedance (2kOhm)
 I_PSL:       3.5    5.5
 I_ARM:      78.0    3.0

o Expected beat note amplitude
I_beat_full = I1 + I2 + 2 sqrt(e I1 I2) cos(w t) ... e: mode overlap (in power)
I_beat_RF = 2 sqrt(e I1 I2)

V_RF = 2 R sqrt(e I1 I2) ... R: RF transimpedance (2kOhm)

P_RF = V_RF^2/2/50 [Watt]
     = 10 log10(V_RF^2/2/50*1000) [dBm]
     = 10 log10(e I1 I2) + 82.0412 [dBm]
     = 10 log10(e) +10 log10(I1 I2) + 82.0412 [dBm]

for e=1, the expected RF power at the PDs [dBm]
 P_RF:      -13.6  -25.8

o Measured beat note power (measured with oscilloscope, 50 ohm input impedance)      
 P_RF:      -17.95dBm (80 mVpp)  -28.4dBm (24mVpp)   (40MHz and 42MHz)  
    e:        37%                    55  [%]                                             

I also measured the various green powers with the Ophir power meter (filter off): 

o Green light power (uW) [measured just before PD, does not consider reflection off the PD]
 P_PSL:       18    24
 P_ARM:       400     13

The IR beat is not being made at the moment because I blocked the PSL beam entering the fiber.

As part of characterization, I wanted to calibrate the EY uPDH error point monitor into units of Hz. So I thought I'd measure the PDH horn-to-horn voltage with the cable to the laser PZT disconnected. However, I saw no clean PDH fringe while monitoring the signal after the LPF that is immediately downstream of the mixer IF output. I then decided to measure the low pass filter OLTF, and found that it seems to have some complex poles (f0~57kHz, Q~5), that amplify the signal by ~x6 relative to the DC level before beginning to roll-off (see Attachment #1). Is this the desired filter shape? Can't find anything in the elog/wiki about such a filter shape being implemented...

The actual OLTF looks alright to me though, see Attachment #2.

I have been experiencing frequent crashes of DTT on pianosa in the past few weeks. This is pretty annoying to deal with when trying to characterize the interferometer loops. I attach the error log dumped to console. The error has to do with some kind of memory corruption. Recall that we aren't using a GDS version that is packaged with the SL7 lscsoft packages, we are using a pretty ancient (2.15) version that is built from source. I have been unable to build a newer version from source (though I didn't spend much time trying). pianosa is the only usable workstation at the moment, but perhaps someone can make this work on donatella / rossa for general improvement in quality of life.

Summary:

1. While noisier, I was able to control the arm lengths to ~30pm RMS(!) using the green ALS beats as error signals (cf. ~10 pm RMS with the IR ALS system).
2. The PRMI could be locked with a CARM offset applied.
3. When lowering the CARM offset, I saw an increase in the in-loop ALS error signal, just as I had with the IR beat.
4. IR TRX / TRY unsurprisingly did not stabilize in any meaningful way.
5. The noise increase seems to have some periodicity along the frequency axis - need to think about what this means.
6. Since there is no apparent benefit to using the green ALS beats, I restored the IR system. The green PDs should still retain somewhat good alignment if one wishes to do a comparison measurement.
7. While the shadow sensors of the ITMs report elevated noise, it is unlikely to be responsible for the cavity moving by the amount suggested by the elevated ALS error signals because of the digital low-pass filtering and 1/f^2 of the pendulum.
8. I confirmed that the ITM shadow sensors do not report elevated noise when the PRMI is locked such that the carrier is resonant. In this config, there is comparable circulating power in the PRC as to when the CARM offset is reduced to ~0.
9. The fact that the IR and green beats both show similar increase in noise suggestes that the cavity length / laser frequency is in fact being modulated, but I still don't know what the exact mechanism is.

was worth a shot i guess.

Trawling through some elogs, I see that this kind of feature showing up in the ALS CARM is not a new problem, see for example here. But I can't find out what the resolution was.

I'm not sure - maybe it was measurement error on my part, I will double check. Moreover, the EX and EY boxes don't seem to use identical designs, if one believes the schematics drawn on the Pomona boxes. The EY design has a 50ohm input impedance in the stopband, whereas the EX doesn't. Maybe the latter needs a Tee + 50ohm terminator at the input?

Judging by the schematics, the servo inputs to both boxes are driving the non-inverting input of an opamp, so they see high-Z.

Rana and I discussed this alogrythym a bit today - here are some bullet points, I'll work on preparing a notebook. We are still talking about a post-mixer low pass filter.

• We want to filter out the 2f component - attenuation relative to the 1f content and be well below the slew-rate of the first post-mixer opamp (OP27).
• We don't want to lose much phase due to the corner of the LPF, so that we can have a somewhat high UGF - let's shoot for 30kHz.
• What should the order of the filter be such that we achieve these goals?
• We will use a numerical optimization routine, that makes a filter that has
  • yy dB attenuation at high frequencies
  • sufficient stability margin
  • sufficiently small phase lag at 30 kHz so that we can realize ~30kHz UGF with the existing servo electronics.

Here are some loop transfer functions. I basically followed the decomposition of the end PDH loop as was done in the multi-color metrology paper. There is no post-mixer low pass filter at the moment (in my model), but already you can see that the top of the phase bubble is at ~10 kHz. Probably there is still sufficient phase available at 30 kHz, even after we add an LPF. In any case, I'll use this model and set up a cost function minimization problem and see what comes out of it. For the PZT discriminant, I used 5 MHz/V, and for the PDH discriminant, I used 40 uV/Hz, which are numbers that should be close to what's the reality at EY.

(i) Note that there could be some uncertainty in the overall gain (VGA stage in the servo).

(ii) For the cavity pole, I assumed the single pole response, which Rana points out isn't really valid at ~1 MHz, which is close to the next FSR

(ii) The PZT response is approximated as a simple LPF whereas there are likely to be several sharp features which may add/eat phase. 

It was way more annoying without a script and took longer than the 4 minutes it does now.

You can fix the requirement to enter password by changing the sshd settings on the FEs like I did for pianosa.

After running the script, you should verify that there are no red flags in the output to console. Yesterday, some of the settings the script was supposed to reset weren't correctly reset, possibly due to python/EPICS problems on donatella, and this cost me an hour of searching last night because the locking wasn't working. Anyway, best practise is to not crash the FEs.

Trawling through some past elogs, I saw that the ALS noise increase as a function of CARM offset reduction is not really a new thing (see e.g. this elog). In the past, when we were able to lock, when the CARM offset is reduced to zero, the arms would "buzz" through resonance. It just wasn't clear to me how much the buzzing was - in all the plots we presented, we were not looking at the fast 16k output, so it looked like the arm powers had stabilized. But today, looking at the frame data at 16k from back in 2016, it is clear to me that the arm transmission was in fact swinging all the way from 0 to some maximum. Once the IR signal (=REFL11) blending is turned on, we were able to stabilize the arm power somewhat. What this means is that we are in a comparable state as to when we were able to lock in the past (since I'm able to sit at 0 CARM offset with the PRMI locked almost indefinitely).

So, I think what I'll try for the next 3 days is to get this blending going, I think I couldn't enable the CM_slow path because when I was experimenting with the high bandwidth Y arm cavity locking, I had increased the whitening gain of this channel, but REFL11 has much more optical gain (=larger signal) than POY11, and so I'll start from 0dB whitening gain and see if I can turn the magic integrator on. Long term, we should try and compensate the optomechanical plant that changes as our CARM offset gets reduced, as this would further reduce the lock acquisition time and simplify the procedure (no need to fiddle with the integrator, offsets etc). A relevant thread from the past.

Were some cables from the ALS beat setup modified? I can't see the beat on the scope, and this elog doesn't say anything about cable connection rearrangement. At ~2311, I am reverting the setup to as it should be.

• The arm powers could be stabilized somewhat once the CM_SLOW path to MC2 was engaged.
• However, I was never able to get the AO path to do anything good.
• Took a bunch of CM board TFs, need to think about what I need to do differently to get this next bit to work.
• An SR785 is sitting next to the LSC rack hooked up to the CM board. I also borrowed the GPIB unit from the AG4395 to grab data from said SR785.
• One thing I noticed that the CARM_B (=CM_SLOW) and DARM_B (=AS55_Q) signals both had a DC offset, so maybe this is indicative of some DC offset in the PRMI 3f signals? Right now, I lock the PRMI without any offsets, and as I reduce the CARM offset, I can see the DC value of REFL11_I and AS55_Q changing significantly. To be investigated in tonight's locking.

I symlinked the SRmeasure and AGmeasure commands to /usr/bin/ on donatella (as it is done on pianosa) so that these scripts are in $PATH and may be run without having to navigate to the labutils directory.

I tried starting the c1oaf model, but got a DQ error (I want the option of running feedforward during locking even if the filters aren't particularly well tuned yet). Note that this isn't "just a warning light" - some channels are initialized to +/- 1e20, so if you try turning some filters on, you will deliver a massive kick to the optics. Restarting it crashed c1lsc (this is not unexpected behavior - the only way to clear the DQ error is to restart the model, and empirically, the success rate is ~50%). The reboot script brought everything back online smoothly, and the second, time, c1oaf started without any issues.

While looking at the CDS overview screen, I noticed that the c1scy model was reporting frequent RFM errors for the C1:SCY-RFM_ETMY_LSC channel (but none of the others). On the sender model (c1rfm), no errors were being reported. The diag reset button / mxstream restart didn't really work either. See Attachment #1. Just restarting the c1scy model didn't fix the error - I had to reboot the machine and restart the models, and now no errors are being reported.

Attachment #2 shows the current nominal CDS status - the red light on c1lsc is due to some missing c1dnn channels (I'll remove these at the next c1lsc model change because I don't want to un-necessarily reboot the vertex FEs), and the c1omc model is obsolete I guess. c1daf isn't running right now but once I get the new fiber (ordered), I'm gonna restart this model as well.

P.S. The ALS temperature sliders are not SDF-ed. So when the model was restarted, I had to change the sliders back to their old values to get the beat back in the usable range.

In preparation for locking tonight, I re-centered the spots on the Oplev QPDs for the ITMs, BS and PRM after locking and running the dither alignment for the arms and also the PRMI carrier. In the past, DC coupling the ITM Oplevs helped the angular stability a bit, let's see if it still does.

Summary:

I made it to 0 CARM offset, PRMI locked a bunch of times today. However, I could not successfully engage the AO path.

Details:

Much of the procedure is scripted, here is the rough set of steps:

• Transition control of the arms from IR signals to ALS signals.
• DC couple the ITM oplev servos
• Burt-restore the settings for PRMI locking with REFL165I-->PRCL, REFL165Q-->MICH, and then enable the MICH_B / PRCL_B locking servos.
• Add some POPDC to the PRMI triggering (nominally only POP22_I) to let these loops be locked while POP22_I fluctuates wildly when we are near the CARM=0 point.
• Zero the CARM offset.
• Adjust the CARM_A/DARM_A offsets such that CARM_B/DARM_B are fluctuating symmetrically about 0.
• CARM_B gain --> 1.0, to begin the RF blend.
• Prepare to hand the DC control authority to ALS by turning off FM1 in the CARM filter bank, and turning ON an integrator in the CARM_B filter.

As I type this out, I realized that I was incorrectly setting offsets to maximize the arm powers by adjusting CARM/DARM offsets as opposed to CARM_A / DARM_A offsets. Tried another round of locking, but this time, I can't even turn the integrator on to get the arms to click into somewhat stable powers. 

One thing I noticed is that depending on the offsets I put into the 3f locking loops, the mean value of REFL11 and AS55 when the ALS CARM/DARM offsets are zeroed changes quite significantly. What is the correct condition to set these offsets? They are different when locking the PRC without arm cavities, and also seem to change continuously with CARM offset. I am wondering if I have too much offset in one of the vertex locking loops?

[Jordan, gautam]

We did the following:

• Route the fiber from the control room to 1Y2.
• Plug fiber in to FiBox at either end, turned FiBoxes ON.
• Tested the optical connection by driving a 1Vpp 440 Hz sine wave from a function generator - Yehonathan hears it loud and clear in the control room.
• Tested that both CH1 and CH2 work - only CH1 is connected to the speakers in the control room at the moment.
• There is some cross-coupling between the channels - not sure if this is happening in the multi-mode fiber or in the electroncis, but I estimate the isolation to be >30dB.
• Connected CH8 and CH9 of DAC0 in the c1lsc expansion chassis to CH1 and CH2 respectively of the FiBox in 1Y2. 
• Restarted the c1daf model on c1lsc, came up smooth.
• Routed the POY11 error signal through the various matrices in c1daf, and we could 👂 the Y-arm cavity 🔐 😎 
• Channels are muted for now - I'll give this a whirl while doing the PRFPMI locking.

When I was looking at this, the AOM shutdown time was measured to be ~120 ns, and while I wasn't able to do a ringdown measurement with the PMC (it'd just stay locked because at the time i was using the zeroth order beam), the PMC transmission decayed in <200 ns. 

For the IMC servo board, it'd be easiest to copy the wiring scheme for the BIO bits as is configured for the CM board (i.e. copy the grouping of the BIO bits on the individual Acromag units). This will enable us to use the latch code with minimal modifications (it was a pain to debug this the first time around). I don't see any major constraint in the wiring assignment that'd make this difficult.

RTFE. Where did the spares go?

Summary:

There was no light entering the IFO. I worked on a few things to bring the interferometer to a somewhat usable state. The goal is to get back to PRFPMI locking ASAP.

Details:

Problem: All fast models report a "0x4000" DC error. See Attachment #1.

Solution: I think this is a "known" issue that happened last new year too. The fix was to add a hard-coded 1 second offset to the daqd config files. However, incrementing/decreasing this offset by +/- 1 second did not fix the errors for me today. I'll reach out to JH for more troubleshooting tips.

Update 15 Jan 2020 830am: The problem is now fixed. See here.

Problem: c1susaux and c1auxey were unresponsive.

Solution: Keyed c1auxey. Rebooted c1susaux and as usual, manually started the eth0/eth1 subnets. The Acromag crate did not have to be power-cycled. ITMY got stuck in this process - I released it using the usual bias jiggling. Why did c1susaux fail? When did it fail? Was there some un-elogged cable jiggling in that part of the lab?

Problem: IMC autolocker and FSS slow processes aren't running on megatron after the upgrade.

Solution: Since no one bothered to do this, I setup systemd infrastructure for doing this on megatron. To run these, you do:

and to check their status, use:

The systemd setup is currently done in a naive way (using the bash executable to run a series of commands rather than using the systemd infrastructure itself to setup variables etc) but it works. I confirmed that the autolocker can re-acquire IMC lock, and that the FSS loop only runs when the IMC is locked. I also removed the obsolete messages printed to megatron's console (by editing /etc/motd) on ssh-login, advising the usage of initctl - the updated message reflects the above instructions.

In order to do the IMC locking, I changed the DC voltage to the AOM to +1V DC (it was +0.8 V DC). In this setting, the IMC refl level is ~3.6 V DC. When using the undiffracted AOM beam, we had more like +5.6 V DC (so now we have ~65% of the nominal level) from the IMC REFL PD when the IMC was unlocked. IIRC, the diffraction efficiency of the AOM should be somewhat better, at ~85%. Needs investigation, or better yet, let's just go back to the old configuration of using the undiffracted beam.

There was also an UN-ELOGGED change of the nominal value of the PMC servo gain to 12.8, and no transfer function measurement. There needs to be a proper characterization of this loop done to decide what the new nominal value should be.

I'm going to leave the PSL shutter open and let the IMC stay locked for stability investigations. Tomorrow, I'll check the single-arm locking and the ALS system.

Summary:

Every new year (on Dec 31 or Jan 1), all of the realtime models will report a "0x4000" error. This happens due to an offset to the GPStime driver not being updated. Here is how this can be fixed (slightly modified version of what was done at LASTI).

Steps to fix the DC errors:

1. ssh into FB machine. 
2. Edit the file /opt/rtcds/rtscore/release/src/include/drv/spectracomGPS.c:
   • Look for the code block with a text string that reads something like

     /* 2019 had 365 days and no leap seconds */
                  pHardware->gpsOffset += 31536000;

   • Copy and paste the above string for the appropriate number of years of offset you are adding, and edit the comment string appropriately!.
3. Navigate to /opt/rtcds/rtscore/release/src/drv/symmetricom. Run the following commands:

   sudo make
   sudo make install

4. Stop all the daqd processes and reload symmetricom:

   sudo systemctl daqd_* stop
   sudo modprobe -r symmetricom
   sudo modprobe symmetricom

5. Re-start the daqd processes:

   sudo service daqd_* start

Independent of this, there is a 1 second offset between the gpstimes reported by /proc/gps and gpstime. However, this doesn't seem to drift. We had effected a static offset to correct for this in the daqd config files, and it looks like these do not need to be updated on a yearly basis. All the daqd indicators are now green, see Attachment #1.

Single arm locking using POX and POY has been restored. After running the dither alignment servos, the TRX/TRY levels are ~0.7. This is consistent with the IMC transmission being ~11000 counts with the AOM 1st order diffracted beam (c.f. 15000 counts with the undiffracted beam).

I don't think this is an accurate statement. XT1111 modules have sinking digital outputs, while XT1121 modules have sourcing digital outputs. Depending on the requirement, the appropriate units should be used. I believe the XT1111 is the appropriate choice for most of our circuits.

For the ringdowns, I suggest you replicate the setup I had - infrastructurally, this was quite robust, and the main problem I had was that I couldn't extinguish the beam completely. Now that we have the 1st order beam, it should be easy.

Per Yehonathan's request, I removed one PDA10CF from a pickoff of REFL on the AS table (it was being used for the mode spectroscopy project). I placed a razor beam dump where the PD used to be, so that when the PRM is aligned, this pickoff is dumped. This is so that team ringdowns can use a fast PD.

Summary:

1. The ZHL-1010+ gain blocks acquired from MiniCircuits arrived sometime ago.
2. I packaged them in a box prepared  (Attachment #1).
3. Their performance was characterized by me (Attachment #2 and #3).

The measurements are consistent with the specifications, and there is no evidence of compression at any of the power levels we expect to supply to this box (<0dBm).

Details:

These "gain blocks" were acquired for the purpose of amplifying the IR ALS beat signals before transmission to the LSC rack for demodulation. The existing ZHL-3A amplifiers have a little too much gain, since our revamp to use IR light to generate the ALS beat.

Attachment #4: Setups used to measure transfer functions and noise.

For the transfer function measurement, I chose to send the output of the amplifier to a coupler, and measured the coupled port (output port of the coupler was terminated with 50 ohms). This was to avoid saturating the input of the AG4395. The "THRU" calibration feature of the AG4395 was used to remove the effect of cabling, coupler etc, so that the measurement is a true reflection of the transfer function of OUT/IN of this box. Yet, there are some periodic ripples present in the measured gain, though the size of these ripples is smaller than the spec-ed gain flatness of <0.6dB.

For the noise measurement, the plots I've presented in Attachment #3 are scaled by a factor of sqrt(2) since the noise of the ZFL-500-HLN+ and the ZHL-1010+ are nearly identical according to the specification. Note that the output noise measured was divided by the (measured) gain of the ZFL-500-HLN+ and the ZFL-1010+ to get the input referred noise. The trace labelled "Measurement noise floor" was measured with the input to the ZFL-500-HLN+ terminated with 50ohms, while for the other two traces, the inputs of the ZHL-1010+ were terminated with 50ohms.

Raw data in Attachment #5.

I will install these at the next opportunity, so that we can get rid of the many attenuators in this path (the main difficulty will be sourcing the required +12V DC for operation, we only have +15V available near the PSL table).

Summary:

1. The input beam to the PMC cavity was changed back to the zeroth order beam from the AOM. 
2. The PMC was locked and nominal transmission levels were recovered.
3. The AOM driver voltage was set to 0V DC. 
4. A razor beam dump was placed to catch the first (and higher order) beams from the AOM (see Attachment #1), but allow the zeroth order beam to reach the PMC cavity.
5. Some dangling cabling was cleared from the PSL enclosure.

Details

• HEPA turned to 100% while work was going on in the PSL enclosure.
• Input power to the PMC cut from ~1.3 W to ~20 mW using the first available HWP downstream of the laser head, before any realignment work was done.
• Next, the beam dump blocking the undeflected zeroth order beam was removed.
• Triangle wave was applied to the PZT servo board "EXT DC" input to sweep the cavity length to make the alignment easier.
• After some patient alignment, I could see a weak transmitted beam locked to some high order mode, at which point I increased the input power to 200mW, and did the fine alignment by looking at the mode shape of the transmitted beam.
• Once I could lock to a TEM00 mode, I bumped the power back up to the nominal 1.3W, I fine tuned the alignment further by minimizing PMC REFL's DC level. 
• Dialled the power back down (using HWP) for installation of the beam block to catch the AOM's first (and higher order) beams.
• Checked that the reflected beam from the PMC cavity is well centered on the PMC REFL PDH photodiode. The ghost from the AR coating of the high-T beamsplitter is blocked by the iris installed by yehonathan on Friday. 
• The beam was a little low on the PMC REFL CCD camera - I raised the camera by ~1cm.
• With the beam axis well matched to the PMC, I measured 1.33 mW going into the cavity, and 1.1 W transmitted, so . Whatever loss numbers we extract should be consistent with this fact.
• HEPA turned back down to 30% shortly after noon.

Note that for all the alignment work, only the two steering mirrors immediately upstream of the PMC cavity were touched.

For a few days, I've noticed that the PSL overview StripTool panel shows PMC transmission and FSS RMTEMP channels with variation that is too large to be believable. Looking at these signals on an oscilloscope, there was no such fuzziness in the waveform. I ruled out flaky connections, and while these are the only two channels currently being acquired by the temporary Acromag setup underneath the PSL enclosure, the Acromags themselves are not to blame, because once I connected a function generator to the Acromag instead of the PMC transmission photodiode, both channels are well behaved. So the problem seems to be with the PMC transmission photodiode, perhaps a grouding issue? Someone please fix this.

Summary:

The PDH discriminant of the PMC servo was measured to be ~0.064 GV/m. This is ~50 times lower than what is reported here. Perhaps this is a signature of the infamous ERA decay, needs more investigation.

Details:

• Calibration of the error and control points were done using 1 Hz triangle wave injection to the "EXT DC" input of the PMC servo. Two such sweeps are shown in Attachment #1 (measured data as points, fits as solid lines). For the control signal monitor, I've multiplied the signal obtained on the scope by 49.6, which is the voltage divider implemented for this monitor point.
• The PDH discrimiannt was calibrated into physical units knowing the modulation frequency of the PMC, which is 35.5 MHz. The error in this technique due to the free-running NPRO frequency noise is expected to be small since the entire fringe is crossed in <30 ms, in which time the laser frequency is expected to change by < 5 kHz.
• The drive to the PZT was calibrated into physical units using the same technique. This number is within a factor of 2 of the number reported here. 
• Attachment #2 shows the loop OLTF measured using the usual IN1/IN2 prescription (with an SR560). In fact, the 8kHz feature makes the loop unstable. For convenience, I've overlaid the OLTF from March 2017, when things were running smoothly. It is not clear to me why even though the optical gain is now lower, a smaller servo gain results in a larger UGF.

The light level hasn't changed by a factor of 50, leading me to suspect the modulation depth. Recall that the demodulation of the PMC is now done off the servo board using a minicircuits mixer (hence, the "C1:PSL-PMC_LODET" channel isn't a reliable readback of the LO signal strength over time). Although there is a C1:PSL-PMC_MODET channel which looks like it comes from the crystal reference card, and so should still work - this, however, shows no degradation over 1 year.

Somebody had removed the BLP-1.9 that I installed at the I/F output of the mixer to remove the sum frequency component in the demodulated signal, I reinstalled this. I find that there are oscillations in the error signal if the PMC servo gain is increased above 14.5 on the MEDM slider.

I looked into this a little more today.

1. The steering optic used to route PMC REFL to the RFPD is in fact a window (labelled W1-PW-1025-UV-1064-45P), not a High-T beamsplitter.
2. With the PMC unlocked, I measured ~10.70 mW in the stronger of the two beams, 5.39 mW in the weaker one. 
   • The window spec is Tp > 97%. Since we have ~1.3 W incident on the PMC, the primary reflection corresponds to T=99.2%, which is consistent with the spec.
   • There is no spec given for the coating on the back side of this window. But from the measured values, it seems to be R = 100* 5.39e-3 / (1.3*T^2) ~ 0.4%. Seems reasonable.

Currently, the iris is set up such that the stronger beam makes it to the PMC RFPD, while the weaker one is blocked by the iris. As usual, this isn't a new issue - was noted last in 2014, but who knows whether the new window was intalled...

Summary:

I estimate the PMC servo modulation depth to be approximately 50 mrad. This is only 15% lower than what was measured in Jan 2018, and cannot explain the ~x50 reduction of optical gain measured earlier in this thread. Later in the day, I also confirmed that the LO input to the ZAD-6 mixer is +7 dBm. So the crystal is not to blame.

Details:

1. PSL frequency is locked to the IMC length.
2. Arm lengths are locked to the PSL frequency using POX/POY.
3. EX green laser locked to the X arm length using end PDH servo. GTRX was ~0.4 in this measurement, which is the nominal value.
4. The 20dB coupled port of the beat between the EX and PSL lasers was monitored using the AG4395A in "Spectrum" units.
5. The beat was set at ~90 MHz, and a spectrum was taken for ~100 MHz span centered at the beat frequency.
6. The modulation depth is estimated by considering the ratio of power at the beat frequency relative to that 35.5 MHz away. See Attachment #1.

Assuming a finesse of 700 for the PMC, we expect an optical gain of 2*Pin*J0(50e-3)*J1(50e-3)/fp  ~ 1.2e-7 W/Hz (=0.089 GW/m). I can't find a measurement of the PMC RFPD transimpedance to map this onto a V/Hz value. 

upgrade was done

cronjob testing wasn't one by one 😢 

burt snapshots were gone

i brought them back home 🏠 

Summary:

The mixer + LPF combo used to demodulate the PMC PDH error signal seems to work as advertised.

Details:

Measurement setup --- Attachment #1. The IF signal was monitored using the scope in High-Z mode.

Results --- Attachment #2.

So the next step is to characterize the RF transimpedance of the PMC RFPD.