40m QIL Cryo_Lab CTN SUS_Lab TCS_Lab OMC_Lab CRIME_Lab FEA ENG_Labs OptContFac Mariner WBEEShop
  40m Log, Page 28 of 337  Not logged in ELOG logo
ID Date Author Type Category Subject
  15599   Wed Sep 23 08:57:18 2020 gautamUpdateVACTP2 running HOT

The interlocks tripped at ~630am local time. Jordan reported that TP2 was supposedly running at 52 C (!).

V1 was already closed, but TP2 was still running. With him standing by the rack, I remotely exectued the following sequence:

  • VM1 closed (isolates RGA volume).
  • VA6 closed (isolates annuli from being pumped).
  • V7 opened (TP3 now backs TP1, temporarily, until I'm in the lab to check things out further).
  • TP2 turned off.

Jordan confirmed (by hand) that TP2 was indeed hot and this is not just some serial readback issue. I'll do the forensics later.

Attachment 1: Screen_Shot_2020-09-23_at_8.55.39_AM.png
  15598   Tue Sep 22 23:17:51 2020 KojiUpdateGeneralHEPA Inspection

Dimensions / Specs

- HEPA unit dimentions
- HEPA unit manufacturer
- Motor
- Capacitor

Attachment 1: A_HEPA_Dimention.JPG
Attachment 2: B_HEPA_Company.JPG
Attachment 3: C_North_HEPA_Spec.JPG
Attachment 4: D_South_HEPA_Spec.JPG
Attachment 5: E_Motor_Spec.JPG
Attachment 6: F_Cap_Spec.JPG
  15597   Tue Sep 22 23:16:54 2020 KojiUpdateGeneralHEPA Inspection

Gautam reported that the PSL HEPA stopped running (ELOG 15592). So I came in today and started troubleshooting.

It looks like that the AC power reaches the motors. However, both motors do not run. It looks like the problem exists in the capacitors, the motors, or both.
Parts specs can be found in the next ELOG.

Attachment 1 is the connection diagram of the HEPA. The AC power is distributed by the breaker panel. The PSL HEPA is assigned to use M22 breaker (Attachment 2). I checked the breaker switch and it was (and is) ON. The power goes to the junction box above the enclosure (Attachment 3). A couple of wires goes to the HEPA switch (right above the enclosure light switch) and the output goes to the variac. The inside of the junction box looked like this (Attachment 4).

By the way, the wires were just twisted and screwed into a metal threaded (but isolated) caps (Attachment 5). Is this legit? Shouldn't we use stronger crimping? Anyway, there was nothing wrong with the caps w.r.t the connection for now.

I could easily trace the power up to the variac. The variac output was just fine (Attachment 6). The cord goes from the variac to the junction box (and then HEPAs) looked scorched. The connection from the plug to HEPAs was still OK, but this should be eventually replaced. Right now the cable was unplugged after the following tests for the safety reason.

The junction box for each HEPA unit was opened to check the voltage. The supply voltage came to the junction boxes and it was just fine. In Attachments 8 & 9, the voltages look low but this is because I just turned the variac only a little.

At the (main) junction box, the resistances of the HEPAs were checked with the Fluke. As the HEPA units are connected to the AC in parallel, the resistances were individually checked as follows.

South HEPA SW North HEPA SW Resistance
OFF LO 5 Ohm
LO OFF 7 Ohm

The coils were not disconnected (... I wonder if the wiring of South HEPA was flipped? But this is not the main issue right now.)

By removing the pre-filters, the motors were inspected Attachments 10 & 11. At least the north HEPA motor was warm, indicating there was some current before. A capacitor was connected per motor. When the variac was tuned up a bit, one side of the capacitor could see the voltage. I could not judge which has the issue between the capacitor and the motor.

Attachment 1: 0_PSL_HEPA.pdf
Attachment 2: 1_Breaker_Panel.JPG
Attachment 3: 2_Junction_Box.JPG
Attachment 4: 3_Junction_Box_Inside.JPG
Attachment 5: 4_Junction_Box_Inside_2.JPG
Attachment 6: 5_Variac_100%.JPG
Attachment 7: 6_VariAC_to_HEPA.JPG
Attachment 8: 7_North_HEPA_IN.JPG
Attachment 9: 8_South_HEPA_IN.JPG
Attachment 10: 9_North_Prefilter_Removed.JPG
Attachment 11: 10_South_Prefilter_Removed.JPG
  15596   Tue Sep 22 22:38:11 2020 gautamUpdateBHDSensing scheme for homodyne phase - some analytic calcs

I got some feedback from Koji who pointed out that the phase tracker is not required here. This situation is similar to the phase locking of two lasers together, which we frequently do, except in that case, we usually we offset the absolute frequencies of the two lasers by some RF frequency, and we demodulate the resulting RF beatnote to use as an error signal. We can usually acquire the lock by simply engaging an integrator (ignoring the fact that if we actuate on the laser PZT, which is a frequency actuator, just a proportional feedback will be sufficient because of the phase->frequency conversion), the idea being that the error signal is frequently going through a zero-crossing (around which the sinusoidal error signal is approximately linear) and we can just "catch" one of these zero crossings, provided we don't run of actuation range.

So the question here becomes, is the RF44 signal a suitable error signal such that we can close a feedback loop in a similar way? To try and get more insight, I tried to work out the situation analytically. I've attached my thinking as a PDF note. I get some pretty messy complicated expressions for the RF44 signal contributions, so it's likely I've made a mistake (though Mathematica did most of the heavy lifting), it'll benefit from a second set of eyes. 

Anyways, I definitely think there is some additional complications than my simple field cartoon from the preceeding elog would imply - the relative phases of the sidebands seem to have an effect, and I still think the lack of the PRC/SRC make the situation different from what Hang/Teng et. al. outlined for the A+ homodyne phase control analysis. Before the HEPA failed, I had tried closing the feedback loop using one quadrature of the demodulated RF44 signal, but had no success with even a simple integrator as the loop (which the experience with the PLL locking says should be sufficient, and pretty easily closed once we see a sinusoidally oscillating demodulated error signal). But maybe I'm overlooking something basic conceptually?



I don't think the proposed scheme for sensing and controlling the homodyne phase will work without some re-thinking of the scheme. I'll try and explain my thinking here and someone can correct me if I've made a fatal flaw in the reasoning somewhere.

Attachment 1: simpleMich.pdf.zip
  15595   Tue Sep 22 16:29:30 2020 KojiUpdateGeneralControl Room AC setting for continuous running

I came to the lab. The control room AC was off -> Now it is on.

Here is the setting of the AC meant for continuous running

Attachment 1: P_20200922_161125.jpg
  15594   Tue Sep 22 12:14:42 2020 ranaSummaryALSALS noise budget update

This ALS loop is not stable. Its one of those traps that comes from using only the Bode plot to estimate the loop stability. You have to also look at the time domain response - you can look at my feedback lecture for the SURF students for some functions.

  15593   Tue Sep 22 00:14:43 2020 anchalSummaryALSALS noise budget update

This is not a reply to comments given to the last post; Still working on incorporating those suggestions.

Trying out a better filter from scratch

Rana suggested looking first at what needs to be suppressed and then create a filter suited for the noise from scratch. So I discarded all earlier poles and zeros and just kept the resonant gains in the digital filter. With that, I found that all we need is three poles at 1 Hz and a gain of 8.1e5 gives the lowest RMS noise value I could get.

Now there can be some practical reasons unknown to me because of which this filter is not possible, but I just wanted to put it here as I'll add the actual noise spectra into this model now.

Few questions:

  • What anti-aliasing filters are used in ALS?
  • Is the digital delay fixed to a constant upper limit or is it left to change as per filters? I have already used a 470 us delay (modeled with Pade 4th order approximation).
  • I could not find a good place where channel names are listed with corresponding meaning. Where can I find them?
  • Is there a channel which keeps a record of lock status? In short, how do I find the in-lock times
Attachment 1: ALS_NoiseBudgetUpdate.pdf
ALS_NoiseBudgetUpdate.pdf ALS_NoiseBudgetUpdate.pdf ALS_NoiseBudgetUpdate.pdf
Attachment 2: ALS_controls.yaml
# -----------------------------------------------------------------------------
# -----------------------------------------------------------------------------
## Cavity Pole
  p: 1.8883e+04
  k: 1.1865e+05

  z: 0
... 106 more lines ...
  15592   Mon Sep 21 16:40:52 2020 gautamUpdatePSLHEPAs are no longer running

The HEPA filters on the PSL enclosure are no longer running. I tried cycling the power switch on the NW corner of the enclosure, and also turned the dial on the Variac down to zero and back up to maximum, no effect. Judging by the indicator LED on it, the power is making it to the Variac - bypassing the Variac and directly connecting the HEPA to the AC power seems to have no effect either.

I can't be sure, but I'm almost certain I heard them running at max speed an hour ago while I was walking around inside the VEA. Probably any damage that can happen has already been done, but I dialled down the Innolight injection current, and closed its shutter till the issue can be resolved.

  15591   Mon Sep 21 15:57:08 2020 JordanUpdateVACTP3 Forepump Replacement and Vac reset

I removed the forepump (Varian SH-110) for TP3 today to see why it had failed over the weekend. I tested it in the C&B lab and the ultimate pressure was only ~40torr. I checked the tip seals and they were destroyed. The scroll housing also easily pulled off of the motor drive shaft, which is indicative of bad bearings. The excess travel in the bearings likely led to significant increase in tip seal wear. This pump will need to be scrapped, or rebuilt.

I tested the spare Varian SH-110 pump located at the X-end and the ultimate pressure was ~98 mtorr. This pump had tip seals replaced on 11/5/18, and is currently at 55163 operating hours. It has been installed as the TP3 forepump.

Once installed, restarting the pump line occured as follows: V5 Closed, VA6 closed, VASE Closed, VASV closed, VABSSCI closed, VABS closed, VABSSCO closed, VAEV closed, VAEE closed,TP3 was restarted and once at normal operation, valves were opened in same order.

The pressure differential interlock condition for V5 was temporaily changed to 10 torr (by Gautam), so that valves could be opened in a controlled manner. Once, the vacuum system was back to normal state the V5 interlock condition was set back to the nominal 1 torr. Vacuum system is now running normally.

Attachment 1: Screenshot_from_2020-09-21_15-56-04.png
Attachment 2: 20200921_145043.jpg
Attachment 3: 20200921_145040.jpg
  15590   Mon Sep 21 12:40:58 2020 gautamUpdateGeneralWorkable IFO recovery

See Attachment #1.

  • The original ETMY actuation matrix was the naive one so I simply retuned everything by adding the butterfly mode appropriately.
  • The cross coupling between the DOFs has not been characterized, but the local damping, Oplev loops, POX/POY locking, simpel Michelson locking, and ASS dither alignemnt all seem to work so I'm thinking this is good enough for now.
  • A copy of the ETMYaux.db file was made, and the slow bias voltage was redistributed among the three available face OSEMs - note that the magnet polarity has to be taken into account.
  • The series resistance on the slow path was reduced from 430 ohms, 1W to 100 ohms, 3W, to allow DC alignment of the cavity axis to the beam axis.
  • Vertex Oplevs were re-centered on their respective QPDs after locking POX/POY and running the ASS dither alignment.
  • The AS spot was a little low on the camera - presumably the SR2/SR3 got macroscopically misaligned. I re-centered the beam on the camera on the AS table (by moving the camera, the beam path was not disturbed).

The beam spot on ETMY looks weird (looks almost like a TEM10 mode), but the one on ITMY seems fine, see Attachment #2. Wonder what's going on there, maybe just a focusing problem?


We need a vent to fix the suspension, but until then what we can do is to redistribute the POS/PIT/YAW actuations to the three coils.

Attachment 1: IFOrecovery.png
Attachment 2: IMG_5345.JPG
  15589   Sun Sep 20 23:12:13 2020 ranaSummaryALSALS noise budget update

I think the digital loop in the ALS budget is too optimistic. You have to include all the digital delays and anti-aliasing filters to get the real response.

aslo, I recommend grabbing some of the actual spectra from the in-lock times with nds and using the calibrated spectra as inputs to this mode. Although we don't have good models of the stack, you can sort of infer it by using the calibrated seismometer data and the calibrated MC_F or MC_L channels (for IMC) or XARM/YARM signals for those.

  15588   Sun Sep 20 11:41:54 2020 gautamUpdateGeneralIMC re-locked

While I stopped by the lab this morning to pick up some things, I took the opportunity to continue the recovery.

  • IMC suspensions were sufficiently misaligned that the autolocker couldn't re-acqurie the lock. I manually recovered the alignment and now the IMC is locked again.
  • ETMY illuminator was left ON, I turned it off. In the process, I modified the illuminator ON/OFF script to be compatible with python3, but unfortunately, it was written in a way that doesn't permit backward compatibility, so now the illuminators can't be turned ON/OFF via the MEDM screen on pianosa (since the default python is 2.7 on that machine). But it does work on rossa, which I'm using as my primary workstation now (hence the change).
  • ITMX watchdog trip threshold was manually reset to the nominal value - the rampdown script was working, but the threshold was ~1400cts (normally ~200 cts) even at 1130am this morning (>12 hours after Koji's work yesterday evening), so I just accelerated the process.
  • Suspension realignment - using a mix of green and IR beams and the various cameras/photodiodes as diagnostics, I roughly restored the alignment of all the suspensions, except ETMY. I can see IR resonances in the X arm now.

At some point, we should run the suspension eigenmode routine (kick optics, let them ringdown, measure peak locations and Qs) to confirm that the remaining suspensions are okay, will also help in actuation re-allocation efforts on ETMY. But I didn't do this today.

Leaving the lab at 1150.

  15587   Sat Sep 19 23:59:22 2020 anchalSummaryALSALS noise budget update

Setting the record straight

I found out an error I did in copying some control model values from Kiwamu's matlab code. On fixing those, we get a considerably reduced amount of total noise. However, there was still an unstable region around the unity gain frequency because of a very small phase margin. Attachment 3 shows the noise budget, ALS open-loop transfer function, and AUX PDH open-loop transfer function with ALS disengaged. Attachment 4 is the yaml file containing all required zpk values for the control model used. Note that the noise budget shows out-of-loop residual arm length fluctuations with respect to PSL frequency. The RMS curve on this plot is integrated for the shown frequency region.

Trying to fix the unstable region

Adding two more poles at 100 Hz in the ALS digital filter seems to work in making the ALS loop stable everywhere and additionally provides a steeper roll-off after 100 Hz. Attachment 1 shows the noise budget, ALS open-loop transfer function, and AUX PDH open-loop transfer function with ALS disengaged. Attachment 2 is the yaml file containing all required zpk values for the control model used. Note that the noise budget shows out-of-loop residual arm length fluctuations with respect to PSL frequency. The RMS curve on this plot is integrated for the shown frequency region.

But is it really more stable?

  • I tried to think about it from different aspects. One thing is sure that  1+G_{OL} remains greater than 1 in all of the frequency region plotted for. This is also evident in the common-mode to residual noise transfer function which shows no oscillation peaks and is a clean mirror image of the open-loop transfer function (See Attachment 1, page 2).
  • Another way is to look for the phase margin. This is a little controversial way of checking stability. For clarity, the open-loop transfer function I'm plotting does not contain the '-1' feedback in it. So the bad phase value at unity gain frequency is -180 degrees (or 180 degrees) for us. I've taken the difference from the closest side and got 76.2 degrees of phase margin.
  • Another way I checked was by plotting a Nyquist plot for the open-loop transfer function. It is said that if the contour does not encircle the point '-1' in the real axis, then the loop would be stable even if the f_{180} < f_{UGF} where f_{180} is the frequency where phase lag becomes -180 degrees at the lowest frequency. For us, f_{180} is at 1 Hz because of the test mass actuator pole. But I have verified that the Nyquist contour of the open-loop transfer function does not encircle '-1' point. I have not uploaded the Nyquist plot as it is not straight forward to plot. Because of large dc gain, it covers a large region and one needs to zoom in and out to properly follow what the contour is really doing. I didn't get time to make insets for it.

Is this close to reality?

For that, we'll have to take present noise source estimates but Gautum vaguely confirmed that this looked more realistic now 'shape-wise'. If I remember correctly, he mentioned that we currently can achieve 8 pm of residual rms motion in the arm cavity with respect to the PSL frequency. So we might be overestimating our loop's capability or underestimating some noise source. More feedback on this welcome and required.

Additional Info:

The code used to calculate the transfer functions and plot them is in the repo 40m/ALS/noiseBudget

Attachment 5 here shows a block diagram for the control loop model used. Output port 'Res_Disp' is used for referring all the noise sources at the residual arm length fluctuation in the noise budget. The open-loop transfer function for ALS is calculated by -(ALS_DAC->ALS_Out1 / ALS_DAC->ALS_Out2) (removing the -1 negative feedback by putting in the negative sign.) While the AUX PDH open-loop transfer function is calculated by python controls package with simple series cascading of all the loop elements.



Attachment 1: ALS_nb_ExtraPoles.pdf
ALS_nb_ExtraPoles.pdf ALS_nb_ExtraPoles.pdf ALS_nb_ExtraPoles.pdf
Attachment 2: ALS_controls.yaml
# -----------------------------------------------------------------------------
# -----------------------------------------------------------------------------
## Cavity Pole
  p: 1.8883e+04
  k: 1.1865e+05

  z: 0
... 109 more lines ...
Attachment 3: ALS_nb_Kiwamus_Values.pdf
ALS_nb_Kiwamus_Values.pdf ALS_nb_Kiwamus_Values.pdf ALS_nb_Kiwamus_Values.pdf
Attachment 4: ALS_controls_Kiwamus_Values.yaml
# -----------------------------------------------------------------------------
# -----------------------------------------------------------------------------
## Cavity Pole
  p: 1.8883e+04
  k: 1.1865e+05

  z: 0
... 107 more lines ...
Attachment 5: ALS_simulink_model.svg
  15586   Sat Sep 19 19:37:16 2020 not KojiUpdateVACTP3 RP failure

Disconcerting because those tip seals were just replaced [15417]. Maybe they were just defective, but if there is a more serious problem with the pump, there is a spare Varian roughing pump (the old TP2 dry pump) sitting at the X-end.

I reset the interlock error to unfreeze the vac controls (leaving V5 closed).


So the conclusion is that RP for TP3 has failed. Presumably, the tip-seal needs to be replaced.

Right now TP3 was turned off and is ready for the tip-seal replacement. V5 was closed since the watchdog tripped.

  15585   Sat Sep 19 19:14:59 2020 KojiUpdateGeneralITMX released / ETMY UR magnet knocked off

There were two SUSs which didn't look normal.

- ITMX was easily released by the bias slider -> Shake the pitch slider and while all the OSEM values are moving, turn on the damping control (with x10 large watchdog threshold)

- ETMY has UR OSEM 0V output. This means that there is no light. And this didn't change at all with the slider move.
- Went to the Y table and tried to look at the coils. It seems that the UR magnet is detached from the optic and stuck in the OSEM.

We need a vent to fix the suspension, but until then what we can do is to redistribute the POS/PIT/YAW actuations to the three coils.

Attachment 1: IMG_6218.jpeg
  15584   Sat Sep 19 18:46:48 2020 KojiUpdateGeneralHand soap

I supplied a bottle of hand soap. Don't put water in the bottle to dilute it as it makes the soap vulnarable for cotamination.

  15583   Sat Sep 19 18:08:34 2020 ranaUpdateGeneralM4.5 EQ in LA

the EQ was ~14 km south of Caltech and 17 km deep


the seismometers obviously saturated during the EQ, but the accelerometers captured some of it. It looks like there's different saturation levels on different sensors.

Also, it seems the mounting of the MC2 accelerometers is not so good. There's some ~10-20 Hz resonance its mount that's showing up. Either its the MC2 chamber legs or the accelerometers are clamped poorly to the MC2 baseplate.

I'm amazed at how much higher the noise is on the MC2 accelerometer. Is that really how much amplification of the ground motion we're getting? If so, its as if the MC has no vibration isolation from the ground in that band. We should put one set on the ground and make the more direct comparison of the spectra. Also, perhaps do some seismic FF using this sensor - I'm not sure how successful we've been in this band.

Attaching the coherence plot from ldvw.ligo.caltech.edu (apparently it has access to the 40m data, so we can use that as an alternative to dtt or python for remote analysis):

Coherence (GWpy) result, image #: 288304

It would be interesting to see if we can use the ML based FF technology from this summer's SURF project by Nadia to increase the coherence by including some slow IMC alignment channels.

  15582   Sat Sep 19 18:07:35 2020 KojiUpdateVACTP3 RP failure

I came to the campus and Gautam notified that he just had received the alert from the vac watchdog.

I checked the vac status at c1vac. PTP3 went up to 10 torr-ish and this made the diff pressure for TP3 over 1torr. Then the watchdog kicked in.

To check the TP3 functionality, AUX RP was turned on and the manual valve (MV in the figure) was opened to pump the foreline of TP3. This easily made PTP3 <0.2 torr and TP3 happy (I didn't try to open V5 though).

So the conclusion is that RP for TP3 has failed. Presumably, the tip-seal needs to be replaced.

Right now TP3 was turned off and is ready for the tip-seal replacement. V5 was closed since the watchdog tripped.

Attachment 1: vac.png
Attachment 2: Screen_Shot_2020-09-19_at_17.52.40.png
  15581   Sat Sep 19 11:27:04 2020 ranaUpdateGeneralM4.5 EQ in LA

the seismometers obviously saturated during the EQ, but the accelerometers captured some of it. It looks like there's different saturation levels on different sensors.

Also, it seems the mounting of the MC2 accelerometers is not so good. There's some ~10-20 Hz resonance its mount that's showing up. Either its the MC2 chamber legs or the accelerometers are clamped poorly to the MC2 baseplate.

Sun Sep 20 00:02:36 2020 edit: fixed indexing error in plots

* also assuming that the sensors are correctly calibrated in the front end to 1 count = 1 um/s^2 (this is what's used in the summ pages)

Attachment 1: Sep18-EQ.pdf
  15580   Sat Sep 19 01:49:52 2020 KojiUpdateGeneralM4.5 EQ in LA

M4.5 EQ in LA 2020-09-19 06:38:46 (UTC) / -1d 23:38:46 (PDT) https://earthquake.usgs.gov/earthquakes/eventpage/ci38695658/executive

I only checked the watchdogs. All watchdogs were tripped. ITMX and ETMY seemed stuck (or have the OSEM magnet issue). They were left tripped. The watchdogs for the other SUSs were reloaded.

  15579   Fri Sep 18 10:47:48 2020 gautamUpdateBHDSensing scheme for homodyne phase


I don't think the proposed scheme for sensing and controlling the homodyne phase will work without some re-thinking of the scheme. I'll try and explain my thinking here and someone can correct me if I've made a fatal flaw in the reasoning somewhere.

Field spectrum cartoon:

Attachment #1 shows a cartoon of the various field components.

  • The input field is assumed to be purely phase modulated (at 11 MHz and 55 MHz) creating pairs of sidebands that are in quadrature to the main carrier field.
  • The sideband fields are drawn with positive and negative imaginary parts to indicate the relative negative sign between these terms in the Jacobi-Anger expansion.
  • For our air BHD setup, the spectrum of the LO beam will also be the same.
  • At the antisymmetric (= dark) port of the beamsplitter, the differential mode signal field will always be in the phase quadrature.
  • I'm using the simple Michelson as the test setup:
    • The ITMs have real and (nearly) identical reflectivities for all frequency components incident on it.
    • The sideband fields are rotated by 90 degrees due to the i in the Michelson transmission equation.
    • The Schnupp asymmetry preferentially transmits the 55 MHz sideband to the AS port compared to the 11 MHz sideband - note that in the simple Michelson config, I calculate T(11 MHz) = 0.02%, T(55 MHz) = 0.6% (both numbers not accounting for the PRM attenuation).
  • I think the cartoon Hang drew up is for the DRFPMI configuration, with the SRC operated in RSE.
    • The main difference relative to the simple Michelson is that the signal field picks up an additional 90 degrees of phase propagating through the SRC.
  • For completeness, I also draw the case of the DRFPMI where the SRC is operated at nearly the orthogonal tuning.
    • I think the situation is similar to the simple Michelson

So is there a 90 degree relative shift between the signal quadrature in the simple Michelson vs the DRFPMI? But wait, there are more problems...

Closing a feedback loop using the 44 MHz signal:

We still need to sense the 44 MHz signal with a photodiode, acquire the signal into our CDS system, and close a feedback loop.

  • The 44 MHz signal is itself supposed to be generated by the interference between the TEM00 55 MHz sideband from the IFO output with the TEM00 11 MHz sideband from the LO field (let's neglect any mode mismatch, HOMs etc for the moment).
  • By splitting this beat signal photocurrent in two, mixing each part with an electrical 44 MHz signal, and digitizing the IF output of said mixers, we should in principle be able to reconstruct the magnitude and phase of the signal.
  • The problem is that we know from other measurements that this signal is going to go through multiple fringes, and hence, we don't have a signal that is linear in the quantity we would like to control, namely the homodyne phase (either quadrature signal can be a candidate linear signal around a zero crossing, but when the signals are going through multiple fringes, neither signal stays linear).
  • One possible way to get around this problem is to use a phase tracker servo - basically, close a purely digital feedback loop, using one of the demodulated quadratures as an error signal, and changing the demodulation phase digitally such that the signal stays entirely in the orthogonal quadrature. However, such a scheme relies on the signal magnitude remaining constant. If the "error signal" goes to zero for multiple reasons (rotation out of the quadrature being considered, or just that the signal itself goes to zero), then this technique won't work. Of course, the phase tracker doesn't know what the "phase" of the signal is, when it's magnitude is (nearly) zero.
  • It is true that we always expect a "background" level of 44 MHz signal, from the 11 MHz and 55 MHz sidebands in the LO beam directly interfering, but this doesn't contain any useful information, and in fact, it'd only contaminate the phase tracker error signal I think.
  • So we can't rely on the error staying in one quadrature (like we do for the regular IFO PDH signals, where there is no relative phase propagation between the LO and RF sideband optical fields and so once we set the demodulation phase, we can assume the signal will always stay in that quadrature, and hence we can close a feedback loop), and we can't track the quadrature. What to do? I tried to dynamically change the phase tracker servo gain based on the signal magnitude (calculated in the RTCDS code using sqrt(I**2 + Q^2), but this did not yield good results...

Next steps:

I don't have any bright ideas at the moment - anyone has any suggestions?🤔


I wanted to check what kind of signal the photodiode sees when only the LO field is incident on the photodiode. So with the IFO field blocked, I connected the PDA10CF to the Agilent analyzer in "Spectrum" mode, through a DC block. The result is shown in Attachment #2. To calculate the PM/AM ratio, I assumed a modulation depth of 0.2. The RIN was calculated by dividing the spectrum by the DC value of the PDA10CF output, which was ~1V DC. The frequencies are a little bit off from the true modulation frequencies because (i) I didn't sync the AG4395 to a Rb 10 MHz signal, and (ii) the span/BW ratio was set rather coarsely at 3kHz.

I would expect only 44 MHz and 66 MHz peaks, from the interference between the 11 MHz and 55 MHz sideband fields, all other field products are supposed to cancel out (or are in orthogonal quadratures). This is most definitely not what I see - is this level of RIN normal and consistent with past characterization? I've got no history in this particular measurement. 

Attachment 1: fieldQuads.pdf
Attachment 2: PMAMratio.pdf
  15578   Wed Sep 16 17:44:27 2020 gautamUpdateCDSAll vertex FE models restarted

I had to make a CDS change to the c1lsc model in an effort to get a few more signals into the models. Rather than risk requiring hard reboots (typcially my experience if I try to restart a model), I opted for the more deterministic scripted reboot, at the expense of spending ~20mins to get everything back up and running.

Update 2230: this was more complicated than expected - a nuclear reboot was necessary but now everything is back online and functioning as expected. While all the CDS indicators were green when I wrote this up at ~1800, the c1sus model was having frequent CPU overflows (execution time > 60 us). Not sure why this happened, or why a hard power reboot of everything fixed it, but I'm not delving into this.

The point of all this was that I can now simultaneously digitize 4 channels - 2 DCPDs, and 2 demodulated quadratures of an RF signal.

Attachment 1: CDS.png
  15577   Wed Sep 16 12:03:07 2020 JonUpdateVACReplacing pressure gauges

Assembled is the list of dead pressure gauges. Their locations are also circled in Attachment 1.

Gauge Type Location
CC1 Cold cathode Main volume
CC3 Cold cathode Pumpspool 
CC4 Cold cathode RGA chamber
CCMC Cold cathode IMC beamline near MC2
P1b Pirani Main volume
PTP1 Pirani TP1 foreline

For replacements, I recommend we consider the Agilent FRG-700 Pirani Inverted Magnetron Gauge. It uses dual sensing techniques to cover a broad pressure range from 3e-9 torr to atmosphere in a single unit. Although these are more expensive, I think we would net save money by not having to purchase two separate gauges (Pirani + hot/cold cathode) for each location. It would also simplify the digital controls and interlocking to have a streamlined set of pressure readbacks.

For controllers, there are two options with either serial RS232/485 or Ethernet outputs. We probably want the Agilent XGS-600, as it can handle all the gauges in our system (up to 12) in a single controller and no new software development is needed to interface it with the slow controls.

Attachment 1: vac_gauges.png
  15576   Wed Sep 16 09:43:41 2020 gautamUpdateIOOMC F spectrum

This elog suggests that there is uniformly 1 stage engaged across all channels. I didn't look at the board to see what the jumper situation is, but only 1 stage of whitening is compensated digitally for both _F and _L. The Pomona box attached to the NPRO PZT input is also compensated digitally to convert counts to frequency.

I tried the gain re-allocation between VCO gain and FSS COMM (and also compensated for the cts to Hz conversion in MCF), but it doesn't seem to have the desired effect on the MCF SNR in the 5-50Hz band. Since the IMC stays locked, and I had already made the changes to mcup, I'll keep these gains for now. We can revert to the old settings if the IMC locking duty cycle is affected. Explicitly, the changes made were:

VCO gain: +7dB ---> +13 dB

FSS COMM: +6 ddB ---> +0 dB

The mcdown script wasn't modified, so the lock acquisition gains are the same as they've been. 


the "Pentek" whitening board that carries the MC channels has jumpers to enable either 1 or 2 stages of 15:150 whitening. Looks lik MC_F has 2 and MC_L has 1.

Attachment 1: MCF_MCL.pdf
  15575   Tue Sep 15 22:11:52 2020 gautamUpdateBHDMore notes on the RF44 scheme


After more trials, I think the phase tracker part used to provide the error signal for this scheme needs some modification for this servo to work.


Attachment #1 shows a block diagram of the control scheme.

I was using the "standard" phase tracker part used in our ALS model - but unlike the ALS case, the magnitude of the RF signal is squished to (nearly) zero by the servo. But the phase tracker, which is responsible for keeping the error signal in one (demodulated) quadrature (since our servo is a SISO system) has a UGF that is dependent on the magnitude of the RF signal. So, I think what is happening here is that the "plant" we are trying to control is substantially different in the acquisition phase (where the RF signal magnitude is large) and once the lock is enabled (where the RF signal magnitude becomes comparitively tiny).

I believe this can be fixed by dynamically normalizing the gain of the digital phase tracking loop by the magnitude of the signal = sqrt(I^2 + Q^2). I have made a modified CDS block that I think will do the job but am opting against a model reboot tonight - I will try this in the daytime tomorrow. 

I'm also wondering how to confirm that the loop is doing something good - any ideas for an out-of-loop monitor? I suppose I could use the DCPD - once the homodyne phase loop is successfully engaged, I should be able to drive a line in MICH and check for drift by comparing line heights in the DCPD signal and RF signal. This will requrie some modification of the wiring arrangement at 1Y2 but shouldn't be too difficult...

The HEPAs, on the PSL table and near ITMY, were dialled down  / turned off respectively, at ~8pm at the start of this work. They will be returned to their previous states before I leave the lab tonight.

Attachment 1: RF44.pdf
  15574   Tue Sep 15 19:17:30 2020 ranaUpdateIOOMC F spectrum

that's a very curious disconnection

the "Pentek" whitening board that carries the MC channels has jumpers to enable either 1 or 2 stages of 15:150 whitening. Looks lik MC_F has 2 and MC_L has 1.

I guess the MC_F signal is so low because of the high gain on the FSS board.  We could lower the FSS common gain and increase the IMC board's VCO gain to make up for this. Maybe 6 dB would be enough. IF that is risky, we could also up the analog gain on the whitening board.

  15573   Tue Sep 15 17:19:09 2020 gautamUpdateIOOMC F spectrum

There was an abrupt change in the MC_F spectrum between August 4 and August 5, judging by the summary pages - the 1 and 3 Hz resonances are no longer visible in the spectrum. Possibly, this indicates some electronics failure on the MC servo board / whitening board, the CDS settings don't seem to have changed. There is no record of any activity in the elog around those dates that would explain such a change. I'll poke around at 1X2 to see if anything looks different.

Update 1740: I found that the MCL / MCF cables were disconnected. So since August 5, these channels were NOT recording any physical quantity. Because their inputs weren't terminated, I guess this isn't a clean measurement of the whitening + AA noise, but particularly for MC_F, I guess we could use more whitening (see Attachment #1). Probably also means that the wandering ~10-30Hz line in the spectrogram is a electronics feature. The connections have now been restored and things look nominal again.

Attachment 1: MCF_MCL.pdf
  15572   Tue Sep 15 17:04:43 2020 gautamUpdateElectronicsDC adaptors delivered

These were delivered to the 40m today and are on Rana's desk


I'll order a couple of these (5 ordered for delivery on Wednesday) in case there's a hot demand for the jack / plug combo that this one has. 

  15571   Tue Sep 15 12:20:36 2020 gautamUpdateElectronicsSR785 repaired

The unit was repaired and returned to the 40m. Now, with a DMM, I measure a DC offset value that is ~1% of the AC signal amplitude. I measured the TF of a simple 1/20 voltage divider and it looks fine. In FFT mode, the high frequency noise floor levels out around 5-7nV/rtHz when the input is terminated in 50 ohms.

I will upload the repair documents to the wiki.


The "source" output of the SR785 has a DC offset of -6.66 V. I couldn't make this up.

Attachment 1: dividerTF.pdf
  15570   Tue Sep 15 10:57:30 2020 gautamUpdatePSLPMC re-locked, RGA re-enabled

The PMC has been unlocked since September 11 sometime (summary pages are flaky again). I re-locked it just now. I didn't mess with the HEPA settings for now as I'm not using the IFO at the moment, so everything should be running in the configuration reported here. The particulate count numbers (both 0.3um and 0.5um) reported is ~x5-8 of what was reported on Thursday, September 10, after the HEPA filters were turned on. We don't have logging of this information in any automated way so it's hard to correlate things with the conditions in Pasadena. We also don't have a working gauge of the pressure of the vacuum envelope.

The RGA scanning was NOT enabled for whatever reason after the vacuum work. I re-enabled it, and opened VM1 to expose the RGA to the main volume. The unit may still be warming up but this initial scan doesn't look completely crazy compared to the reference trace which is supposedly from a normal time based on my elog scanning (the timestamp is inherited from the c0rga machine whose clock is a bit off).

Update 1500: I checked the particle count on the PSL table and it barely registers on the unit (between 0-20 between multiple trials) so I don't know if we need a better particle coutner or if there is negligible danger of damage to optics due to particulate matter.

Attachment 1: RGAscan.pdf
  15569   Mon Sep 14 07:50:01 2020 YehonathanUpdateBHDMonte Carlo Simulations

Turns out what was causing the instability in the aLIGO plots were the lock commands which I forgot to remove before running the simulation. Removing these also made the simulation much faster.

Other than that I improved other stuff in the simulations:

  • The LO phase in the aPlus simulation is now optimized for the lowest noise at 100Hz.
  • Added RF PDs diagnostics (see attachments 8 for aPlus and 9 for aLIGO). The thresholds (red dashed lines in attachments 8,9) for cutting marginal simulations are set such that roughly 30% of the simulations are discarded.
  • Removed DHARD because it jacks up the RF PD readings in aPlus for some reason.
  • Fixed the sign of laser frequency shift in response to CARM offset.

Still need to do:

  • Incorporate Jon’s noise curves.
  • Add phase noise for LO beam.
  • Add arm reflectivity imbalance.
  • Add mirror curvature imbalance.
  • Include feedback loops using Pytickle.

Feel free to add to the todo list.

Attachment 1: MICH_AplusMCMC.pdf
Attachment 2: PRCL_AplusMCMC.pdf
Attachment 3: SRCL_AplusMCMC(1).pdf
Attachment 4: OMC_Comm_AplusMCMC.pdf
Attachment 5: OMC_Diff_AplusMCMC.pdf
Attachment 6: OMC_Angle_Yaw_AplusMCMC.pdf
Attachment 7: OMC_Angle_Pitch_AplusMCMC.pdf
Attachment 8: Main_Laser_RIN_AplusMCMC.pdf
Attachment 9: aPlus_RF_Diagnostics.pdf
Attachment 10: aLIGO_RF_Diagnostics.pdf
  15568   Thu Sep 10 15:56:08 2020 KojiUpdateGeneralHEPA & Particle Level Status

- PSL HEPA was running at 33% and is now at 100%
- South End HEPA was not on and is now running
- Yarm Portable HEPA was not running and is now running at max speed: the power was taken beneath the ITMY table. It is better to unplug it when one uses the IFO.
- Yend Portable HEPA was not running and is now running (presumably) at max speed

Particle Levels: (Not sure about the unit. The convention here is to multiply x10 of the reading)

Before running the HEPAs at their maximum
9/10/2020 15:30 / 0.3um 292180 / 0.5um 14420

(cf 9/5/2020 / 0.3um 94990 / 0.5um 6210)
After running the HEPAs at their maximum
The number gradually went down and now became constant at about half of the initial values
9/10/2020 19:30 / 0.3um 124400 / 0.5um 7410

  15567   Thu Sep 10 15:43:22 2020 JonUpdateBHDInput noise spectra for A+ BHD modeling

As promised some time ago, I've obtained input noise spectra from the sites calibrated to physical units. They are located in a new subdirectory of the BHD repo: A+/input_noises. I've heavily annotated the notebook that generates them (input_noises.ipynb) with aLOG references, to make it transparent what filters, calibrations, etc. were applied and when the data were taken. Each noise term is stored as a separate HDF5 file, which are all tracked via git LFS.

So far there are measurements of the following sources:

  • L1 SRCL
  • H1 SRCL

These can be used, for example, to make more realistic Hang's bilinear noise modeling [ELOG 15503] and Yehonathan's Monte Carlo simulations [ELOG 15539]. Let me know if there are other specific noises of interest and I will try to acquire them. It's a bit time-consuming to search out individual channel calibrations, so I will have to add them on a case-by-case basis.

  15566   Wed Sep 9 20:52:45 2020 ranaSummaryIOOwandering line in IMC

since the summary pages are working again, I was clicking through and noticed that there's a wandering peak in the whitened IMC spectrogram that goes from 10-30 Hz over the course of a day.


anyone know what this is ?

  15565   Wed Sep 9 00:05:18 2020 gautamUpdateBHDMore notes on the RF44 scheme


  1. With the Michelson locked on a dark fringe, the f2-f1 signal at ~44 MHz does not seem to ever vanish, it seems to bottom out at ~2mV DC. Is this just an electronics offset? Not sure of the implications on using this as a locking signal for the homodyne phase yet.
  2. The inferred relative phase fluctuations between the LO and RF fields using this 44 MHz signal is consistent with that from previous tests.
  3. The laying out of the new, shorter, fiber patch cable seems to have helped to reduce the phase drift over minute time scales.
  4. So far, I have not had any success in using the 44 MHz signal to close a servo loop and keep the homodyne phase locked for more than a few seconds at a time, and even then, the loop shape is sub-optimal as the in-loop error signal is not clean. Maybe some systematic loop shaping will help, but I think the dynamic range requirement on the actuator is too high, and I'm not sure what to make of the fact that the error signal does not vanish.


Attachment #1 shows the optical setup currently being used to send the LO field with RF sidebands on it to the air BHD setup.

  • You can find a video of the large power fluctuations mentioned in my previous elog here. After tightening the collimator in the mount, the arrangement is still rather sensitive, but at least I was able to see some light on the DCPD on the AS table, at which point I could use this signal and tweak the alignment to maximize it.
  • It is well known that the input beam to the IMC drifts during the day, either due to temperature fluctuations / PMC PZT stroke L2A / some other reason (see Attachment #4 for the power drift over ~12 hours, it is not monotonic with temperature). The fact that our collimating setup is so sensitive to the input pointing isn't ideal, but I noticed the power had only degraded by ~5% today compare to yesterday, so maybe the occassional touch up is all that is required.

Attachment #2 shows spectra of the relative phase drift between LO and IFO output field (from the Dark Michelson). 

  • I still haven't overlaid a seismic model. There was some discussion about the TTs having a 1/f roll-off as opposed to 1/f^2, I don't know if there was any characterization at the time of installation, but this SURF report seems to suggest that it should in fact be 1/f^2 because the passive eddy current dampers are mounted to the main suspension cage on springs rather than being rigidly attached. 
  • The noise at ~100 Hz is ~x2 higher if the spectra is collected during the daytime, when the seismic activity is high. Although this shouldn't really matter at 100 Hz? 
  • There are also huge power-line harmonics - I suspect these are making it difficult to close a feedback loop, as I couldn't add a 60 Hz comb which doesn't affect the loop stability for a UGF of ~30-50 Hz. But if they aren't notched out, the control signal RMS is dominated by these frequencies.

Attachment #3 shows the signal magntiude of the signals used to make the spectra in Attachment #2, during the observation time (10 minutes) with which the spectra were computed. The dashed vertical lines denote the 1%, 50% and 99% quantiles.

  • Koji asked me about the 55 MHz signal and why it doesn't vanish - for the dark Michelson, where the ITMs don't apply any relative phase on reflection to the carrier and RF sideband fields, we expect that the upper and lower sidebands cancel, and so there should be no intensity modulation at 55 MHz (just like we don't expect any for a pure phase modulated light field incident on a photodiode).
  • However, from the I/Q demodulated data that is collected, it would appear that while the size of the signal does vary, it doesn't ever completely vanish. This implies some asymmetry in the sidebands (or at least, the transmission of the sidebands by the Michelson). I didn't estimate the effect of the Schnupp asymmetry, or if this asymmetry is coming from elsewhere, but the point is that for the conclusions drawn from Attachment #2 remain valid even though both the amplitude and phase of the 55 MHz signal is changing. 
  • I also plot the corresponding histogram for the 44 MHz signal. You can see that it never goes to 0 (once I fix the x-label ticks). I don't know if this is consistent with some electronics offset.

Attempts to close a feeddback loop to control the homodyne phase:

  • A digital PLL (a.k.a. Phase Tracker) servo was used to keep the demodulated 44 MHz signal in one (demodulated) quadrature, which can then be used as an error signal.
  • Unlike the ALS case, the quantity to be servoed to 0 is the magnitude of the 44 MHz signal, and not its phase, so that's how I've set up the RTCDS model.
  • I played around with the loop shape to try and achieve a stable lock by actuating on the PZT mounted mirror in the LO path - however, I've not yet had any success so far.
Attachment 1: IMG_3397.JPG
Attachment 2: phaseNoisePSD.pdf
Attachment 3: magnitudeHist.pdf
Attachment 4: LOpowerDrift.png
  15564   Tue Sep 8 11:49:04 2020 gautamUpdateCDSSome path changes

I edited /diskless/root.jessie/home/controls/.bashrc so that I don't have to keep doing this every time I do a model recompile.


Where is this variable set and how can I add the new paths to it? 

export RCG_LIB_PATH=/opt/rtcds/userapps/release/isc/c1/models/isc/:/opt/rtcds/userapps/release/isc/c1/models/cds/:/opt/rtcds/userapps/release/isc/c1/models/sus/:$RCG_LIB_PAT
  15563   Tue Sep 8 01:31:43 2020 KojiUpdateBHDA first look at RF44 scheme

- Loose fiber coupler: Sorry about that. I could not detect something was loose there, although some of the locks were not tightened.

- S incident instead of P: Sorry about that too. I completely missed that the IMC takes S-pol.

  15562   Mon Sep 7 23:49:14 2020 gautamUpdateBHDA first look at RF44 scheme


Over the last couple of days, I've been working towards getting the infrastructure ready to test out the scheme of sensing (and eventually, controlling) the homodyne phase using the so-called RF44 scheme. More details will be populated, just quick notes for now before I forget.

  • LO beam with RF sidebands needed to be re-coupled into collimator, it wasn't seated tightly and just touching the fiber completely destroyed the alignment.
  • HWP installed before said collimator - IMC wants s-polarized light whereas the IFO field is p-polarized.
  • After my work, the numbers were: ~1.47mW input to collimator, ~1.07mW out of collimator on AS table, ~1mW making it to the BHD board. All seem like reasonable numbers to me.
  • 44 MHz signal synthesis - for now, I use a Marconi (10 MHz synced to Rb clock), I think we could also use a mixer+SLP50 to mix 11 and 55 MHz signals (which are easily available at the LSC rack) to generate this. I looked at Wenzel quadruplers, the specs don't suggest a quadrupler will do much better.
  • CDS model was modified to accept the phase-tracker output as an error signal for the homodyne phase control servo. Compile and install went smooth but I opted against a model restart tonight, I'll do it tmrw.
  • Some trials were done with the Michelson locked to a dark fringe (as was done for the case of the DC LO field beating with the 55 MHz sideband). While the overall spectrum lines up fairly well with earlier trials, the signal looks somewhat more "discontinuous" in its traversal of I/Q space, and it never quite goes to 0. Some offset? What does this mean for locking? More investigations needed....
  15561   Sun Sep 6 14:17:18 2020 JonUpdateEquipment loanZurich Instruments analyzer

On Friday, I grabbed the Zurich Instruments HF2LI lock-in amplifier and brought it home. As time permits, I will work towards developing a similar readout script as we have for the SR785.

  15560   Sun Sep 6 13:15:44 2020 JonUpdateDAQUPS for framebuilder

Now that the old APC Smart-UPS 2200 is no longer in use by the vacuum system, I looked into whether it can be repurposed for the framebuilder machine. Yes, it can. The max power consumption of the framebuilder (a SunFire X4600) is 1.137kW. With fresh batteries, I estimate this UPS can power the framebuilder for >10 min. and possibly as long as 30 min., depending on the exact load.

@Chub/Jordan, this UPS is ready to be moved to rack 1X6/1X7. It just has to be disconnected from the wall outlet. All of the equipment it was previously powering has been moved to the new UPS. I have ordered a replacement battery (APC #RBC43) which is scheduled to arrive 9/09-11.

  15559   Sat Sep 5 14:28:03 2020 KojiUpdateGeneralLO beam: Fiber coupling work

2PM: Arrived at the 40m. Started the work for the coupling of the RF modulated LO beam into a fiber. -> I left the lab at 10:30 PM.

The fiber coupling setup for the phase-modulated beam was made right next to the PSL injection path. (See attachment 1)

  • For the alignment of the beam, the main PSL path, including the alignment of the 2" PO mirror, has not been touched.
  • There are two PO beams with the optical power of 0.8mW (left) and 1.6mW (right). Both had been blocked but the right one was designed to be used for PSL POS and ANG. For the fiber coupling, the right beam was used.
  • The alignment/mode-matching work has been done with a short (2m?) fiber patch cable from Thorlabs. The fiber is the same as the one used for LO delivery.
  • I tried to have a mode-matching telescope in the LO path. I ended up having no lens for the best result. The resulting transmitted power is 1.21mW out of 1.64mW incident (~74%). These powers were measured with the Ophir power meter. (Note that Thorlabs' fiber power meter indicated 1.0mW transmission.)

Some notes

  • After the PSL activity, the IMC locking was checked to see if I messed up the PSL alignment. It locks fine and looks fine.
    • The input shutter (left closed after Jon's vacuum work?) was opened.
    • The alignment was not optimal and had some pitch misalignment (e.g. TEM03).
    • After some MC SUS alignment, the automatic locking of TEM00 was recovered. Mainly MC3 pitch was moved (+0.17).
    • I've consulted with Gautam and he thinks this is with the level of regular drift. The AS beam was visible.
  • The IMC and MI were moving so much, but this seemed just the usual Saturday night Millikan shake.
  • During the activity, the PSL HEPA was turned up to 100 and it was reverted to 33 after the work.
  • I have been wearing a mask and gloves throughout the work there.
Attachment 1: 20200905212254_IMG_9938.JPG
  15558   Sat Sep 5 12:01:10 2020 JonUpdateVACVac system UPS installation


Yesterday's UPS switchover was mostly a success. The new Tripp Lite 120V UPS is fully installed and is communicating with the slow controls system. The interlocks are configured to trigger a controlled shutdown upon an extended power outage (> ~30 s), and they have been tested. All of the 120V pumpspool equipment (the full c1vac/LAN/Acromag system, pressure gauges, valves, and the two small turbo pumps) has been moved to the new UPS. The only piece of equipment which is not 120V is TP1, which is intended to be powered by a separate 230V UPS. However that unit is still not working, and after more investigation and a call to Tripp Lite, I suspect it may be defective. A detailed account of the changes to the system follow below.

Unfortunately, I think I damaged the Hornet (the only working cathode ionization gauge in the main volume) by inadvertently unplugging it while switching over equipment to the new UPS. The electronics are run from multiple daisy-chained power strips in the bottom of the rack and it is difficult to trace where everything goes. After the switchover, the Hornet repeatedly failed to activate (either remotely or manually) with the error "HV fail." Its compatriot, the Pirani SuperBee, also failed about a year ago under similar circumstances (or at least its remote interface did, making it useless for digital monitoring and control). I think we should replace them both, ideally with ones with some built-in protection against power failures.

New EPICS channels

Four new soft channels per UPS have been created, although the interlocks are currently predicated on only C1:Vac-UPS120V_status.

Channel Type Description Units
C1:Vac-UPS120V_status stringin Operational status -
C1:Vac-UPS120V_battery ai Battery remaining %
C1:Vac-UPS120V_line_volt ai Input line voltage V
C1:Vac-UPS120V_line_freq ai Input line frequency Hz
C1:Vac-UPS240V_status stringin Operational status -
C1:Vac-UPS240V_battery ai Battery remaining %
C1:Vac-UPS240V_line_volt ai Input line voltage V
C1:Vac-UPS240V_line_freq ai Input line frequency Hz

These new readbacks are visible in the MEDM vacuum control/monitor screens, as circled in Attachment 1:

Continuing issues with 230V UPS

Yesterday I brought with me a custom power cable for the 230V UPS. It adapts from a 208/120V three-phase outlet (L21-20R) to a standard outlet receptacle (5-15P) which can mate with the UPS's C14 power cable. I installed the cable and confirmed that, at the UPS end, 208V AC was present split-phase (i.e., two hot wires separated 120 deg in phase, each at 120V relative to ground). This failed to power on the unit. Then Jordan showed up and suggested to try powering it instead from a single-phase 240V outlet (L6-20R). However we found that the voltage present at this outlet was exactly the same as what the adapter cable provides: 208V split-phase.

This UPS nominally requires 230V single-phase. I don't understand well enough how the line-noise-isolation electronics work internally, so I can think of three possible explanations:

  1. 208V AC is insufficient to power the unit.
  2. The unit requires a true neutral wire (i.e., not a split-phase configuration), in which case it is not compatible with the U.S. power grid.
  3. The unit is defective.

I called Tripp Lite technical support. They thought the unit should work as powered in the configuration I described, so this leads me to suspect #3.

@Chub and Jordan: Can you please look into somehow replacing this unit, potentially with a U.S.-specific model? Let's stick with the Tripp Lite brand though, as I already have developed the code to interface those.

UPS-host computer communications

Unlike our older equipment, which communicates serially with the host via RS232/485, the new UPS units can be connected with a USB 3.0 cable. I found a great open-source package for communicating directly with the UPS from within Python, Network UPS Tools (NUT), which eliminates the dependency on Tripp Lite's proprietary GUI. The package is well documented, supports hundreds of power-management devices, and is available in the Debian package manager from Jessie (Debian 8) up. It consists of a large set of low-level, device-specific drivers which communicate with a "server" running as a systemd service. The NUT server can then be queried using a uniform set of programming commands across a huge number of devices.

I document the full set-up procedure below, as we may want to use this with more USB devices in the future.

How to set up

First, install the NUT package and its Python binding:

$ sudo apt install nut python-nut

This automatically creates (and starts) a set of systemd processes which expectedly fail, since we have not yet set up the config. files defining our USB devices. Stop these services, delete their default definitions, and replace them with the modified definitions from the vacuum git repo:

$ sudo systemctl stop nut-*.service
$ sudo rm /lib/systemd/system/nut-*.service
$ sudo cp /opt/target/services/nut-*.service /etc/systemd/system
$ sudo systemctl daemon-reload

Next copy the NUT config. files from the vacuum git repo to the appropriate system location (this will overwrite the existing default ones). Note that the file ups.conf defines the UPS device(s) connected to the system, so for setups other than c1vac it will need to be edited accordingly.

$ sudo cp /opt/target/services/nut/* /etc/nut

Now we are ready to start the NUT server, and then enable it to automatically start after reboots:

$ sudo systemctl start nut-server.service
$ sudo systemctl enable nut-server.service

If it succeeds, the start command will return without printing any output to the terminal. We can test the server by querying all the available UPS parameters with

$ upsc 120v

which will print to the terminal screen something like

battery.charge: 100
battery.runtime: 1215
battery.type: PbAC
battery.voltage: 13.5
battery.voltage.nominal: 12.0
device.mfr: Tripp Lite 
device.model: Tripp Lite UPS 
device.type: ups
driver.name: usbhid-ups
driver.parameter.pollfreq: 30
driver.parameter.pollinterval: 2
driver.parameter.port: auto
driver.parameter.productid: 2010
driver.parameter.vendorid: 09ae
driver.version: 2.7.2
driver.version.data: TrippLite HID 0.81
driver.version.internal: 0.38
input.frequency: 60.1
input.voltage: 120.3
input.voltage.nominal: 120
output.frequency.nominal: 60
output.voltage.nominal: 120
ups.beeper.status: enabled
ups.delay.shutdown: 20
ups.mfr: Tripp Lite 
ups.model: Tripp Lite UPS 
ups.power.nominal: 1000
ups.productid: 2010
ups.status: OL
ups.timer.reboot: 65535
ups.timer.shutdown: 65535
ups.vendorid: 09ae
ups.watchdog.status: 0

Here 120v is the name assigned to the 120V UPS device in the ups.conf file, so it will vary for setups on other systems.

If all succeeds to this point, what we have set up so far is a set of command-line tools for querying (and possibly controlling) the UPS units. To access this functionality from within Python scripts, a set of official Python bindings are provided by the python-nut package. However, at the time of writing, these bindings only exist for Python 2.7. For Python 3 applications (like the vacuum system), I have created a Python 3 translation which is included in the vacuum git repo. Refer to the UPS readout script for an illustration of its usage.

Attachment 1: vac_medm.png
  15557   Fri Sep 4 21:12:51 2020 JonUpdateVACVac system UPS installation

The vac work is completed. All of the vacuum equipment is now running on the new 120V UPS, except for TP1. The 230V TP1 is still running off wall power, as it always has. After talking with Tripp Lite support today, I believe there is a problem with the 230V UPS. I will post a more detailed note in the morning.


The vac controls are going down now to pull and test software changes. Will advise when the work is completed.

  15556   Fri Sep 4 15:26:55 2020 JonUpdateVACVac system UPS installation

The vac controls are going down now to pull and test software changes. Will advise when the work is completed.

  15555   Thu Sep 3 15:55:04 2020 gautamUpdateBHDPhase drift between LO and IFO after fiber replacement


After replacement of the fiber delivering the LO beam to the airBHD setup (some photos here), I repeated the measurement outlined here. There may be some improvement, but overall, conclusions don't change much.


The main addition I made was to implement a digital phase tracker servo (a la ALS), to make sure my arctan2 usage wasn't completely bonkers (the CDS block can be deleted later, or maybe it's useful to keep it, we will see). I didn't measure it today, but the UGF of said servo should be >100 Hz so the attached spectrum should be valid below that (loop has not been done, so above the UGF, the control signal isn't a valid representative of the free running noise). Attachment #1 shows the result. The 1 Hz and 3 Hz suspension resonances are well resolved. Anyways, what this means is that the earlier result was not crazy. I don't know what to make of the high frequency lines, but my guess is that they are electronic pickup from the Sorensens - I'm using clip-mini-grabbers to digitize these signals, and other electronics in that rack (e.g. ALS signals) also show these lines.

It is pretty easy to keep the simple Michelson locked for several minutes. Attachment #2 shows the phase-tracker servo output over several minutes. The y-axis units are degrees. If this is to be believed, the relative phase between the two fields is drifting by 12um ove an hour. This is significantly lower than my previous measurement, while the noise in the ~0.5-10 Hz band is similar, so maybe the shorter fiber patch cable did some good?

I think there is also correlation between the PSL table temperature, but of course, the evidence is weak, and there are certainly other effects at play. At first, I thought the abrupt jumps are artefacts, but they don't actually represent jumps >360 degrees over successive samples, so maybe they are indicative of some real jump in the relative phase? Either fiber slippage or TT suspension jumps? I'll double check with the offline data to make sure it's not some artefact of the phase tracker servo. If you disagree with these conclusions and think there is some meaurement/analysis/interpretation error, I'd love to hear about it.

Next steps:

  1. Budget the offline inferred phase noise spectrum, overlay a seismic noise model, to see if we can disentangle the contributions from the suspensions and that from the LO fiber.
  2. I'll see if I can setup an LO pickup with some RF sidebands on it in parallel to this setup so we can try some of the ideas discussed on the call this week. There are several beams available, but the question is whether I can get this into a fiber without 1 week of optical layout work.

I have left the heterodyne electronics setup at the LSC rack, but it is not powered (because there are some exposed wires). Please leave it as is.

Attachment 1: LOphaseDrift.pdf
Attachment 2: phaseDrift_tempCorr.png
  15554   Thu Sep 3 00:00:57 2020 gautamUpdateBHDNew patch cable installed
  • 10m PM1064 cable was installed. I tried a double shielding approach (photos tmrw here), but I suspect the real weak point is where the fiber is plugged into the collimator - it's hard to imagine we can stabilize this sort of arrangement to better than 100um absolute length over long periods of time, I'd think thermal/mechanical strains in the fiber will modulate the length by ~mm (?). Anyways, let's see what the heterodyne measurement tells us.
  • This work required (i) realignment at the input coupler and (ii) change of position of mode matching lenses in the LO path on the AS table to see any interference with the IFO beam. This indicates something was seriously wrong with the old patch cable, as the collimator should set the mode. The MFD of the two fibers may have been different, but I don't know if that alone can account for it.
  • As of now, I have fringes between the ITM single bounce and the LO, but the fringe pk-pk is only 10% of the theoretical pk-pk based on DC values of the LO and AS beams. So the mode matching can be improved significantly (I preivously had ~60%).

To be continued tomorrow. I think it's a good idea to let the newly installed fiber relax into some sort of stable configuration overnight.

  15553   Wed Sep 2 00:49:47 2020 gautamUpdateBHDSome notes about homodyne phase


Using a heterodyne measurement setup to track both quadratures, I estimated the relative phase fluctuation between the LO field and the interferometer output field. It may be that a single PZT to control the homodyne phase provides insufficient actuation range. I'll also need to think about a good sensing scheme for controlling the homodyne phase, given that it goes through ~3 fringes/sec - I didn't have any success with the double demodulation scheme in my (admittedly limited) trials.

For everything in this elog, the system under study was a simple Michelson (PRM, SRM and ETMs misaligned) locked on the dark fringe.


​This work was mainly motivated by my observation of rapid fringing on the BHD photodiodes with MICH locked on the dark fringe. The seismic-y appearance of these fringes reminded me that there are two tip-tilt suspensions (SR2, SR3), one SOS (SRM) + various steering optics on seismic stacks (6+ steering mirrors) between the dark port of the beamsplitter and the AS table, where the BHD readout resides. These suspensions modulate the phase of the output field of course. So even though the Michelson phase is tightly controlled by our LSC feedback loop, the field seen by the homodyne readout has additional phase noise due to these optics (this will be a problem for in vacuum BHD too, the question is whether we have sufficient actuator range to compensate).

To get a feel for how much relative phase noise there is between the LO field and the interferometer output field (this is the metric of interest), I decided to set up a heterodyne readout so that I can simultaneously monitor two orthogonal quadratures.

  • The idea is that with the Michelson locked, there is no DC carrier field from the interferometer.
  • The field incident on the DCPD from the interferometer should be dominated by the 55 MHz sideband transmitted to the dark port given the Schnupp asymmetry. 
  • The LO field is picked off before any RF sidebands are added to it (the PMC modulation sideband should be suppressed by the cavity transmission).
  • Therefore, the LO field should be dominantly at the carrier frequency.
  • By placing a broadband RFPD (PDA10CF) in place of one of the DCPDs, I can demodulate the optical beat between this 55 MHz sideband, which shares the same output path to the location of the DCPD as the audio-frequency sidebands on the carrier from the dark Michelson, to estimate the relative phase noise between the LO and IFO output fields.
  • The point is that with the heterodyne readout, I can track the fringe wrapping, which is not an option for the BHD readout with two DCPDs, and uncontrolled LO phase.

Attachment #1 shows the detailed measurement setup. I hijacked the ADC channels normally used by the DCPDs (along with the front-end whitening) to record these time-series.

Attachments #2, #3 shows the results in the time domain. The demodulated signal isn't very strong despite my pre-amplification of the PDA10CF output by a ZFL-500-HLN, but I think for the purposes of this measurement, there is sufficient SNR.

This would suggest that there are pretty huge (~200um) relative phase excursions between the LO and IFO fields. I suppose, over minutes, it is reasonable that the fiber length changes by 100um or so? If true, we'd need some actuator that has much more range to control the homodyne phasethan the single PZT we have available right now. Maybe some kind of thermal actuator on the fiber length? If there is some pre-packaged product available, that'd be best, making one from scratch may be a whole project in itself. Attachment #3 is just a zoomed-in version of the time series, showing the fringing more clearly.

Attachment #4 has the same information as Attachment #2, except it is in the frequency domain. The FFT length was 30 seconds. The features between ~1-3 Hz support my hypothesis that the SR2/SR3 suspensions are a dominant source of relative phase noise between LO and IFO fields at those frequencies. I guess we could infer something about the acoustic pickup in the fibers from the other peaks.

Attachment 1: heterodyneMICH.pdf
Attachment 2: unwrappedPhase.pdf
Attachment 3: unwrappedPhase_zoom.pdf
Attachment 4: phaseNoisePSD.pdf
  15552   Tue Sep 1 15:39:04 2020 gautamUpdateElectronicsHV coil driver oscillations fixed


Increasing the compensation capacitance (470 pF now instead of 33 pF) seems to have fixed the oscillation issues associated with this circuit. However, the measured noise is in excess of the model at almost any frequency of relevance. I believe the problem is due to the way the measurement is done, and that we should re-do the measurement once the unit is packaged in a shielded environment.


Attachment #1 shows (schematically) the measurement setup. Main differences from the way I did the last round of testing are:

  1. A 20 ohm series resistor was connected between the high voltage output and ground to simulate the OSEM coil.
  2. The test was done under driven conditions (i.e. some non-zero input voltage) to better simulate conditions under which the circuit will be used.
  3. An Acromag XT1541 DAC was used to provide the input signal, to simulate more realistic operating conditions.
  4. A pomona box filter was used to block out the high voltage DC signal which would otherwise destroy the SR785.

Attachment #2 shows the measurement results:

  • Tests were done at a few different drive levels to check if there was any difference.
  • The difference between "Idrive=0mA" and "Input Grounded" traces is that in the former, the Acromag DAC was connected but putting out 0V, wheras in the latter, I shorted the input to the circuit ground.
  • Because the measurement was done at the output of the PA95, the Johnson noise of 25 kohms (~20 nV/rtHz) was manually summed in quadrature to all the measured traces.
  • The plotted spectra were collected in 3 spans, 0-200 Hz, 200Hz-1.8kHz, and 1.8kHz-14.6kHz. The input range was kept constant, so I'm not sure what to make of the discontinuity around 1.8 kHz. Maybe the comb of lines that were being picked up were distorting the spectra for lower frequencies?
  • The "Model" is only for the electronics noise of the circuit. The low-pass filtered noise of the Acromag should be totally negligible above 10 Hz, see here. The fact that there is little difference between the "Idrive=0mA" and "Input Grounded" traces further supports this claim.
  • The diodes in the Pomona box are also unlikely to be the culprit, because with this Pomona box connected to the SR785 and its input terminated with 50ohms, I don't see the comb of spectral lines.

I didn't capture the data, but viewing the high voltage output on an Oscilloscope threw up no red flags - the oscillations which were previously so evident were nowhere to be seen, so I think the capacitor switch did the trick as far as stability is concerned.

There is a large excess between measurement and model out to a few kHz, if this is really what ends up going to the suspension then this circuit is useless. However, I suspect at least part of the problem is due to close proximity to switching power supplies, judging by the comb of ~10 Hz spaced peaks. This is a frequent problem in coil driver noise measurements - previously, the culprit was a switching power supply to the Prologix GPIB box, but now a Linear AC-DC converter is used (besides, disconnecting it had no visible effect). The bench supplies providing power to the board, however, is a switching supply, maybe that is to blame? I think the KEPCO supplies providing +/-250 V are linear. I tried the usual voodoo of twisting the wires used to receive the signal, moving the SR785 away from the circuit board etc, but these measures had no visible effect either. 


The real requirement of this circuit is that the current noise above 100 Hz be <1pA/rtHz. This measurement suggests a level that is 5x too high. But the problem is likely measurement related. I think we can only make a more informed conclusion after shielding the circuit better and conducting the test in a more electromagnetically quiet environment.

Attachment 1: testSetup.pdf
Attachment 2: HVampNoise_driven.pdf
  15551   Tue Sep 1 01:49:49 2020 KojiUpdateElectronicsTeledyne AP1053 etc were transported

Teledyne AP1053 etc were transported from Rich's office to the 40m. The box is placed on the shelf at the entrance.

My record tells that there are 7 AP1053 in the box. I did not check the number this time.

Attachment 1: 20200831203756_IMG_9931.jpg
Attachment 2: 20200831203826_IMG_9932.jpg
Attachment 3: 20200831205126_IMG_9934.jpg
  15550   Sun Aug 30 11:29:33 2020 ranaUpdateGeneralpower blink?

My power at home winked out for a second this morning, but it looks like either nothing happened in the 40m lab or else it rode it out.

MC is locked - lost lock around 11:25 AM and then relocked.

ELOG V3.1.3-