40m QIL Cryo_Lab CTN SUS_Lab TCS_Lab OMC_Lab CRIME_Lab FEA ENG_Labs OptContFac Mariner WBEEShop
  40m Log, Page 226 of 335  Not logged in ELOG logo
ID Date Authorup Type Category Subject
  15564   Tue Sep 8 11:49:04 2020 gautamUpdateCDSSome path changes

I edited /diskless/root.jessie/home/controls/.bashrc so that I don't have to keep doing this every time I do a model recompile.


Where is this variable set and how can I add the new paths to it? 

export RCG_LIB_PATH=/opt/rtcds/userapps/release/isc/c1/models/isc/:/opt/rtcds/userapps/release/isc/c1/models/cds/:/opt/rtcds/userapps/release/isc/c1/models/sus/:$RCG_LIB_PAT
  15565   Wed Sep 9 00:05:18 2020 gautamUpdateBHDMore notes on the RF44 scheme


  1. With the Michelson locked on a dark fringe, the f2-f1 signal at ~44 MHz does not seem to ever vanish, it seems to bottom out at ~2mV DC. Is this just an electronics offset? Not sure of the implications on using this as a locking signal for the homodyne phase yet.
  2. The inferred relative phase fluctuations between the LO and RF fields using this 44 MHz signal is consistent with that from previous tests.
  3. The laying out of the new, shorter, fiber patch cable seems to have helped to reduce the phase drift over minute time scales.
  4. So far, I have not had any success in using the 44 MHz signal to close a servo loop and keep the homodyne phase locked for more than a few seconds at a time, and even then, the loop shape is sub-optimal as the in-loop error signal is not clean. Maybe some systematic loop shaping will help, but I think the dynamic range requirement on the actuator is too high, and I'm not sure what to make of the fact that the error signal does not vanish.


Attachment #1 shows the optical setup currently being used to send the LO field with RF sidebands on it to the air BHD setup.

  • You can find a video of the large power fluctuations mentioned in my previous elog here. After tightening the collimator in the mount, the arrangement is still rather sensitive, but at least I was able to see some light on the DCPD on the AS table, at which point I could use this signal and tweak the alignment to maximize it.
  • It is well known that the input beam to the IMC drifts during the day, either due to temperature fluctuations / PMC PZT stroke L2A / some other reason (see Attachment #4 for the power drift over ~12 hours, it is not monotonic with temperature). The fact that our collimating setup is so sensitive to the input pointing isn't ideal, but I noticed the power had only degraded by ~5% today compare to yesterday, so maybe the occassional touch up is all that is required.

Attachment #2 shows spectra of the relative phase drift between LO and IFO output field (from the Dark Michelson). 

  • I still haven't overlaid a seismic model. There was some discussion about the TTs having a 1/f roll-off as opposed to 1/f^2, I don't know if there was any characterization at the time of installation, but this SURF report seems to suggest that it should in fact be 1/f^2 because the passive eddy current dampers are mounted to the main suspension cage on springs rather than being rigidly attached. 
  • The noise at ~100 Hz is ~x2 higher if the spectra is collected during the daytime, when the seismic activity is high. Although this shouldn't really matter at 100 Hz? 
  • There are also huge power-line harmonics - I suspect these are making it difficult to close a feedback loop, as I couldn't add a 60 Hz comb which doesn't affect the loop stability for a UGF of ~30-50 Hz. But if they aren't notched out, the control signal RMS is dominated by these frequencies.

Attachment #3 shows the signal magntiude of the signals used to make the spectra in Attachment #2, during the observation time (10 minutes) with which the spectra were computed. The dashed vertical lines denote the 1%, 50% and 99% quantiles.

  • Koji asked me about the 55 MHz signal and why it doesn't vanish - for the dark Michelson, where the ITMs don't apply any relative phase on reflection to the carrier and RF sideband fields, we expect that the upper and lower sidebands cancel, and so there should be no intensity modulation at 55 MHz (just like we don't expect any for a pure phase modulated light field incident on a photodiode).
  • However, from the I/Q demodulated data that is collected, it would appear that while the size of the signal does vary, it doesn't ever completely vanish. This implies some asymmetry in the sidebands (or at least, the transmission of the sidebands by the Michelson). I didn't estimate the effect of the Schnupp asymmetry, or if this asymmetry is coming from elsewhere, but the point is that for the conclusions drawn from Attachment #2 remain valid even though both the amplitude and phase of the 55 MHz signal is changing. 
  • I also plot the corresponding histogram for the 44 MHz signal. You can see that it never goes to 0 (once I fix the x-label ticks). I don't know if this is consistent with some electronics offset.

Attempts to close a feeddback loop to control the homodyne phase:

  • A digital PLL (a.k.a. Phase Tracker) servo was used to keep the demodulated 44 MHz signal in one (demodulated) quadrature, which can then be used as an error signal.
  • Unlike the ALS case, the quantity to be servoed to 0 is the magnitude of the 44 MHz signal, and not its phase, so that's how I've set up the RTCDS model.
  • I played around with the loop shape to try and achieve a stable lock by actuating on the PZT mounted mirror in the LO path - however, I've not yet had any success so far.
Attachment 1: IMG_3397.JPG
Attachment 2: phaseNoisePSD.pdf
Attachment 3: magnitudeHist.pdf
Attachment 4: LOpowerDrift.png
  15570   Tue Sep 15 10:57:30 2020 gautamUpdatePSLPMC re-locked, RGA re-enabled

The PMC has been unlocked since September 11 sometime (summary pages are flaky again). I re-locked it just now. I didn't mess with the HEPA settings for now as I'm not using the IFO at the moment, so everything should be running in the configuration reported here. The particulate count numbers (both 0.3um and 0.5um) reported is ~x5-8 of what was reported on Thursday, September 10, after the HEPA filters were turned on. We don't have logging of this information in any automated way so it's hard to correlate things with the conditions in Pasadena. We also don't have a working gauge of the pressure of the vacuum envelope.

The RGA scanning was NOT enabled for whatever reason after the vacuum work. I re-enabled it, and opened VM1 to expose the RGA to the main volume. The unit may still be warming up but this initial scan doesn't look completely crazy compared to the reference trace which is supposedly from a normal time based on my elog scanning (the timestamp is inherited from the c0rga machine whose clock is a bit off).

Update 1500: I checked the particle count on the PSL table and it barely registers on the unit (between 0-20 between multiple trials) so I don't know if we need a better particle coutner or if there is negligible danger of damage to optics due to particulate matter.

Attachment 1: RGAscan.pdf
  15571   Tue Sep 15 12:20:36 2020 gautamUpdateElectronicsSR785 repaired

The unit was repaired and returned to the 40m. Now, with a DMM, I measure a DC offset value that is ~1% of the AC signal amplitude. I measured the TF of a simple 1/20 voltage divider and it looks fine. In FFT mode, the high frequency noise floor levels out around 5-7nV/rtHz when the input is terminated in 50 ohms.

I will upload the repair documents to the wiki.


The "source" output of the SR785 has a DC offset of -6.66 V. I couldn't make this up.

Attachment 1: dividerTF.pdf
  15572   Tue Sep 15 17:04:43 2020 gautamUpdateElectronicsDC adaptors delivered

These were delivered to the 40m today and are on Rana's desk


I'll order a couple of these (5 ordered for delivery on Wednesday) in case there's a hot demand for the jack / plug combo that this one has. 

  15573   Tue Sep 15 17:19:09 2020 gautamUpdateIOOMC F spectrum

There was an abrupt change in the MC_F spectrum between August 4 and August 5, judging by the summary pages - the 1 and 3 Hz resonances are no longer visible in the spectrum. Possibly, this indicates some electronics failure on the MC servo board / whitening board, the CDS settings don't seem to have changed. There is no record of any activity in the elog around those dates that would explain such a change. I'll poke around at 1X2 to see if anything looks different.

Update 1740: I found that the MCL / MCF cables were disconnected. So since August 5, these channels were NOT recording any physical quantity. Because their inputs weren't terminated, I guess this isn't a clean measurement of the whitening + AA noise, but particularly for MC_F, I guess we could use more whitening (see Attachment #1). Probably also means that the wandering ~10-30Hz line in the spectrogram is a electronics feature. The connections have now been restored and things look nominal again.

Attachment 1: MCF_MCL.pdf
  15575   Tue Sep 15 22:11:52 2020 gautamUpdateBHDMore notes on the RF44 scheme


After more trials, I think the phase tracker part used to provide the error signal for this scheme needs some modification for this servo to work.


Attachment #1 shows a block diagram of the control scheme.

I was using the "standard" phase tracker part used in our ALS model - but unlike the ALS case, the magnitude of the RF signal is squished to (nearly) zero by the servo. But the phase tracker, which is responsible for keeping the error signal in one (demodulated) quadrature (since our servo is a SISO system) has a UGF that is dependent on the magnitude of the RF signal. So, I think what is happening here is that the "plant" we are trying to control is substantially different in the acquisition phase (where the RF signal magnitude is large) and once the lock is enabled (where the RF signal magnitude becomes comparitively tiny).

I believe this can be fixed by dynamically normalizing the gain of the digital phase tracking loop by the magnitude of the signal = sqrt(I^2 + Q^2). I have made a modified CDS block that I think will do the job but am opting against a model reboot tonight - I will try this in the daytime tomorrow. 

I'm also wondering how to confirm that the loop is doing something good - any ideas for an out-of-loop monitor? I suppose I could use the DCPD - once the homodyne phase loop is successfully engaged, I should be able to drive a line in MICH and check for drift by comparing line heights in the DCPD signal and RF signal. This will requrie some modification of the wiring arrangement at 1Y2 but shouldn't be too difficult...

The HEPAs, on the PSL table and near ITMY, were dialled down  / turned off respectively, at ~8pm at the start of this work. They will be returned to their previous states before I leave the lab tonight.

Attachment 1: RF44.pdf
  15576   Wed Sep 16 09:43:41 2020 gautamUpdateIOOMC F spectrum

This elog suggests that there is uniformly 1 stage engaged across all channels. I didn't look at the board to see what the jumper situation is, but only 1 stage of whitening is compensated digitally for both _F and _L. The Pomona box attached to the NPRO PZT input is also compensated digitally to convert counts to frequency.

I tried the gain re-allocation between VCO gain and FSS COMM (and also compensated for the cts to Hz conversion in MCF), but it doesn't seem to have the desired effect on the MCF SNR in the 5-50Hz band. Since the IMC stays locked, and I had already made the changes to mcup, I'll keep these gains for now. We can revert to the old settings if the IMC locking duty cycle is affected. Explicitly, the changes made were:

VCO gain: +7dB ---> +13 dB

FSS COMM: +6 ddB ---> +0 dB

The mcdown script wasn't modified, so the lock acquisition gains are the same as they've been. 


the "Pentek" whitening board that carries the MC channels has jumpers to enable either 1 or 2 stages of 15:150 whitening. Looks lik MC_F has 2 and MC_L has 1.

Attachment 1: MCF_MCL.pdf
  15578   Wed Sep 16 17:44:27 2020 gautamUpdateCDSAll vertex FE models restarted

I had to make a CDS change to the c1lsc model in an effort to get a few more signals into the models. Rather than risk requiring hard reboots (typcially my experience if I try to restart a model), I opted for the more deterministic scripted reboot, at the expense of spending ~20mins to get everything back up and running.

Update 2230: this was more complicated than expected - a nuclear reboot was necessary but now everything is back online and functioning as expected. While all the CDS indicators were green when I wrote this up at ~1800, the c1sus model was having frequent CPU overflows (execution time > 60 us). Not sure why this happened, or why a hard power reboot of everything fixed it, but I'm not delving into this.

The point of all this was that I can now simultaneously digitize 4 channels - 2 DCPDs, and 2 demodulated quadratures of an RF signal.

Attachment 1: CDS.png
  15579   Fri Sep 18 10:47:48 2020 gautamUpdateBHDSensing scheme for homodyne phase


I don't think the proposed scheme for sensing and controlling the homodyne phase will work without some re-thinking of the scheme. I'll try and explain my thinking here and someone can correct me if I've made a fatal flaw in the reasoning somewhere.

Field spectrum cartoon:

Attachment #1 shows a cartoon of the various field components.

  • The input field is assumed to be purely phase modulated (at 11 MHz and 55 MHz) creating pairs of sidebands that are in quadrature to the main carrier field.
  • The sideband fields are drawn with positive and negative imaginary parts to indicate the relative negative sign between these terms in the Jacobi-Anger expansion.
  • For our air BHD setup, the spectrum of the LO beam will also be the same.
  • At the antisymmetric (= dark) port of the beamsplitter, the differential mode signal field will always be in the phase quadrature.
  • I'm using the simple Michelson as the test setup:
    • The ITMs have real and (nearly) identical reflectivities for all frequency components incident on it.
    • The sideband fields are rotated by 90 degrees due to the i in the Michelson transmission equation.
    • The Schnupp asymmetry preferentially transmits the 55 MHz sideband to the AS port compared to the 11 MHz sideband - note that in the simple Michelson config, I calculate T(11 MHz) = 0.02%, T(55 MHz) = 0.6% (both numbers not accounting for the PRM attenuation).
  • I think the cartoon Hang drew up is for the DRFPMI configuration, with the SRC operated in RSE.
    • The main difference relative to the simple Michelson is that the signal field picks up an additional 90 degrees of phase propagating through the SRC.
  • For completeness, I also draw the case of the DRFPMI where the SRC is operated at nearly the orthogonal tuning.
    • I think the situation is similar to the simple Michelson

So is there a 90 degree relative shift between the signal quadrature in the simple Michelson vs the DRFPMI? But wait, there are more problems...

Closing a feedback loop using the 44 MHz signal:

We still need to sense the 44 MHz signal with a photodiode, acquire the signal into our CDS system, and close a feedback loop.

  • The 44 MHz signal is itself supposed to be generated by the interference between the TEM00 55 MHz sideband from the IFO output with the TEM00 11 MHz sideband from the LO field (let's neglect any mode mismatch, HOMs etc for the moment).
  • By splitting this beat signal photocurrent in two, mixing each part with an electrical 44 MHz signal, and digitizing the IF output of said mixers, we should in principle be able to reconstruct the magnitude and phase of the signal.
  • The problem is that we know from other measurements that this signal is going to go through multiple fringes, and hence, we don't have a signal that is linear in the quantity we would like to control, namely the homodyne phase (either quadrature signal can be a candidate linear signal around a zero crossing, but when the signals are going through multiple fringes, neither signal stays linear).
  • One possible way to get around this problem is to use a phase tracker servo - basically, close a purely digital feedback loop, using one of the demodulated quadratures as an error signal, and changing the demodulation phase digitally such that the signal stays entirely in the orthogonal quadrature. However, such a scheme relies on the signal magnitude remaining constant. If the "error signal" goes to zero for multiple reasons (rotation out of the quadrature being considered, or just that the signal itself goes to zero), then this technique won't work. Of course, the phase tracker doesn't know what the "phase" of the signal is, when it's magnitude is (nearly) zero.
  • It is true that we always expect a "background" level of 44 MHz signal, from the 11 MHz and 55 MHz sidebands in the LO beam directly interfering, but this doesn't contain any useful information, and in fact, it'd only contaminate the phase tracker error signal I think.
  • So we can't rely on the error staying in one quadrature (like we do for the regular IFO PDH signals, where there is no relative phase propagation between the LO and RF sideband optical fields and so once we set the demodulation phase, we can assume the signal will always stay in that quadrature, and hence we can close a feedback loop), and we can't track the quadrature. What to do? I tried to dynamically change the phase tracker servo gain based on the signal magnitude (calculated in the RTCDS code using sqrt(I**2 + Q^2), but this did not yield good results...

Next steps:

I don't have any bright ideas at the moment - anyone has any suggestions?🤔


I wanted to check what kind of signal the photodiode sees when only the LO field is incident on the photodiode. So with the IFO field blocked, I connected the PDA10CF to the Agilent analyzer in "Spectrum" mode, through a DC block. The result is shown in Attachment #2. To calculate the PM/AM ratio, I assumed a modulation depth of 0.2. The RIN was calculated by dividing the spectrum by the DC value of the PDA10CF output, which was ~1V DC. The frequencies are a little bit off from the true modulation frequencies because (i) I didn't sync the AG4395 to a Rb 10 MHz signal, and (ii) the span/BW ratio was set rather coarsely at 3kHz.

I would expect only 44 MHz and 66 MHz peaks, from the interference between the 11 MHz and 55 MHz sideband fields, all other field products are supposed to cancel out (or are in orthogonal quadratures). This is most definitely not what I see - is this level of RIN normal and consistent with past characterization? I've got no history in this particular measurement. 

Attachment 1: fieldQuads.pdf
Attachment 2: PMAMratio.pdf
  15588   Sun Sep 20 11:41:54 2020 gautamUpdateGeneralIMC re-locked

While I stopped by the lab this morning to pick up some things, I took the opportunity to continue the recovery.

  • IMC suspensions were sufficiently misaligned that the autolocker couldn't re-acqurie the lock. I manually recovered the alignment and now the IMC is locked again.
  • ETMY illuminator was left ON, I turned it off. In the process, I modified the illuminator ON/OFF script to be compatible with python3, but unfortunately, it was written in a way that doesn't permit backward compatibility, so now the illuminators can't be turned ON/OFF via the MEDM screen on pianosa (since the default python is 2.7 on that machine). But it does work on rossa, which I'm using as my primary workstation now (hence the change).
  • ITMX watchdog trip threshold was manually reset to the nominal value - the rampdown script was working, but the threshold was ~1400cts (normally ~200 cts) even at 1130am this morning (>12 hours after Koji's work yesterday evening), so I just accelerated the process.
  • Suspension realignment - using a mix of green and IR beams and the various cameras/photodiodes as diagnostics, I roughly restored the alignment of all the suspensions, except ETMY. I can see IR resonances in the X arm now.

At some point, we should run the suspension eigenmode routine (kick optics, let them ringdown, measure peak locations and Qs) to confirm that the remaining suspensions are okay, will also help in actuation re-allocation efforts on ETMY. But I didn't do this today.

Leaving the lab at 1150.

  15590   Mon Sep 21 12:40:58 2020 gautamUpdateGeneralWorkable IFO recovery

See Attachment #1.

  • The original ETMY actuation matrix was the naive one so I simply retuned everything by adding the butterfly mode appropriately.
  • The cross coupling between the DOFs has not been characterized, but the local damping, Oplev loops, POX/POY locking, simpel Michelson locking, and ASS dither alignemnt all seem to work so I'm thinking this is good enough for now.
  • A copy of the ETMYaux.db file was made, and the slow bias voltage was redistributed among the three available face OSEMs - note that the magnet polarity has to be taken into account.
  • The series resistance on the slow path was reduced from 430 ohms, 1W to 100 ohms, 3W, to allow DC alignment of the cavity axis to the beam axis.
  • Vertex Oplevs were re-centered on their respective QPDs after locking POX/POY and running the ASS dither alignment.
  • The AS spot was a little low on the camera - presumably the SR2/SR3 got macroscopically misaligned. I re-centered the beam on the camera on the AS table (by moving the camera, the beam path was not disturbed).

The beam spot on ETMY looks weird (looks almost like a TEM10 mode), but the one on ITMY seems fine, see Attachment #2. Wonder what's going on there, maybe just a focusing problem?


We need a vent to fix the suspension, but until then what we can do is to redistribute the POS/PIT/YAW actuations to the three coils.

Attachment 1: IFOrecovery.png
Attachment 2: IMG_5345.JPG
  15592   Mon Sep 21 16:40:52 2020 gautamUpdatePSLHEPAs are no longer running

The HEPA filters on the PSL enclosure are no longer running. I tried cycling the power switch on the NW corner of the enclosure, and also turned the dial on the Variac down to zero and back up to maximum, no effect. Judging by the indicator LED on it, the power is making it to the Variac - bypassing the Variac and directly connecting the HEPA to the AC power seems to have no effect either.

I can't be sure, but I'm almost certain I heard them running at max speed an hour ago while I was walking around inside the VEA. Probably any damage that can happen has already been done, but I dialled down the Innolight injection current, and closed its shutter till the issue can be resolved.

  15596   Tue Sep 22 22:38:11 2020 gautamUpdateBHDSensing scheme for homodyne phase - some analytic calcs

I got some feedback from Koji who pointed out that the phase tracker is not required here. This situation is similar to the phase locking of two lasers together, which we frequently do, except in that case, we usually we offset the absolute frequencies of the two lasers by some RF frequency, and we demodulate the resulting RF beatnote to use as an error signal. We can usually acquire the lock by simply engaging an integrator (ignoring the fact that if we actuate on the laser PZT, which is a frequency actuator, just a proportional feedback will be sufficient because of the phase->frequency conversion), the idea being that the error signal is frequently going through a zero-crossing (around which the sinusoidal error signal is approximately linear) and we can just "catch" one of these zero crossings, provided we don't run of actuation range.

So the question here becomes, is the RF44 signal a suitable error signal such that we can close a feedback loop in a similar way? To try and get more insight, I tried to work out the situation analytically. I've attached my thinking as a PDF note. I get some pretty messy complicated expressions for the RF44 signal contributions, so it's likely I've made a mistake (though Mathematica did most of the heavy lifting), it'll benefit from a second set of eyes. 

Anyways, I definitely think there is some additional complications than my simple field cartoon from the preceeding elog would imply - the relative phases of the sidebands seem to have an effect, and I still think the lack of the PRC/SRC make the situation different from what Hang/Teng et. al. outlined for the A+ homodyne phase control analysis. Before the HEPA failed, I had tried closing the feedback loop using one quadrature of the demodulated RF44 signal, but had no success with even a simple integrator as the loop (which the experience with the PLL locking says should be sufficient, and pretty easily closed once we see a sinusoidally oscillating demodulated error signal). But maybe I'm overlooking something basic conceptually?



I don't think the proposed scheme for sensing and controlling the homodyne phase will work without some re-thinking of the scheme. I'll try and explain my thinking here and someone can correct me if I've made a fatal flaw in the reasoning somewhere.

Attachment 1: simpleMich.pdf.zip
  15599   Wed Sep 23 08:57:18 2020 gautamUpdateVACTP2 running HOT

The interlocks tripped at ~630am local time. Jordan reported that TP2 was supposedly running at 52 C (!).

V1 was already closed, but TP2 was still running. With him standing by the rack, I remotely exectued the following sequence:

  • VM1 closed (isolates RGA volume).
  • VA6 closed (isolates annuli from being pumped).
  • V7 opened (TP3 now backs TP1, temporarily, until I'm in the lab to check things out further).
  • TP2 turned off.

Jordan confirmed (by hand) that TP2 was indeed hot and this is not just some serial readback issue. I'll do the forensics later.

Attachment 1: Screen_Shot_2020-09-23_at_8.55.39_AM.png
  15603   Tue Sep 29 17:07:25 2020 gautamUpdateGeneralLab visit for inventory location

I was in the lab from 1630-1830. I have located and visually inspected all the parts required for the magnet regluing / optic cleaning parts of the planned vent, except the fresh batches of scpectroscopic grade solvents. I was in the cleanroom part of the clean and bake lab from 1630-1700.

  15604   Wed Sep 30 17:12:24 2020 gautamUpdateGeneralHEPA blower startup capacitor replacement

[JV, GV]

The HEPAs work again. After running the HEPAs for ~1 hour, I checked the particle count on the PSL table - the meter registered 0 for both 0.3 um and 0.5 um. So I decided to turn the NPRO back on, at ~1730 local time. The PMC and IMC were readily locke, so the basic interferometer functionality is returned, and we can now go ahead with (i) vent prep (ii) air BHD tests and (iii) IMC debuggin as was discussed on the call today. The earth is shaking, but nothing serious so far, I will resume alignment of the interferometer later in the evening when hopefully things have calmed down a bit more...


  1. Turned off mains switch on the NW corner of the PSL enclosure. Then, disconnected the power cables from mains to Variac and from Variac to HEPAs. Made sure both HEPAs were set to "OFF".
  2. With confidence that no AC power was reaching the motors, I removed the pre-filters, and removed the old startup capacitors. These don't have a polarity, but I marked the cable that was connected to the left terminal of the capacitor when the cap is viewed with the label facing you, in the interest of changing as few things as possible.
  3. The two new capacitors were measured with the LCR meter - the meter registered ~7.5 uF, as expected. Unsurprisingly, the old capacitors that were removed didn't register any reading on the LCR meter. The terminals weren't shorted, but I don't know what the failure mode for this kind of capacitor is.
  4. The two new capacitors were installed. Then, I tested the system by undoing all the changes in bullet #1. We found that the Variac needs to be set to 100% for the motors to startup.
  5. The motor speed was found to vary as the Variac dial was turned. FWIW, at the "nominal" setting of 33% on the Variac (when we run the interferometer), I could see both fan blades were turning, but the flow was low enough that you couldn't hear any wind (at least, neither Jordan nor I could).
  6. Turned off the mains agian, and cleaned up - restored the insulating rubber sleeve on the capacitor leads, and re-installed the pre-filters on the HEPA blowers. Then we turned both blowers back on. 

Note that the many other issues Koji noted in the preceeding elog (e.g. flaky wiring) have not been addressed.

Flow measurements:

Chub kindly provided us with an electronic anemometer. With the meter held directly against the HEPA filter inside the enclosure, we measured ~700 cfm of airflow on each of the two HEPAs, with the Variac set to 100% and the HEPAs themselves set to "High". With the Variac at 50%, the flow drops to ~160 cfm. At the nominal setting of 33%, the meter didn't register any flow. I don't know what the spec'd flow rate is for this combination of blower + filter, but Jordan says similar units in Downs register ~1500 cfm at the "High" setting. The two protable (similarly sized) HEPA units in the 40m, one at ITMY and one at ETMY, register ~900 cfm and ~1100 cfm respectively, when set to high. So we may want to revisit what the "nominal" HEPA setting should be, in case the filters have become clogged over time. 

Some photos of the HEPA blowers with the pre-filters off and the capacitors switched out may be found here.

  15606   Thu Oct 1 17:44:48 2020 gautamUpdateGeneralSome inventory notes

The optomechanics stock in the lab was in a sad state. We have obtained the following from Thorlabs in the last two months:

  1. 6 pcs each of DT12 and DT12B compact translation stages (for lens mode matching).
  2. 3pcs each KM100PM + PM3 cube beamsplitter mounts (for polarization control).
  3. 1 Post / spacer kit for height adjustment.
  4. 3 pcs ea of K6XS + AD11F + F220APC for fiber applications.

I have used some of these for the ari BHD setup. The unused items are stored in the shelves that house the optomechanics ~halfway down the east arm. I'm wondering what's a good setup to document the stock of this stuff so we can always have a healthy stock of optomechanics (at least the non speciality ones like posts, spacers etc). It sucks to realize at 0000hrs that you're missing a 3mm shim or 250mm converging lens or something.

  15607   Fri Oct 2 10:29:49 2020 gautamUpdateOptical LeversETMY, BS and ITMX HeNes degrading

Attachment #1 shows that the ITMX, ETMY and beamsplitter Oplev light levels have decayed significantly from their values when installed. In particular, the ETMY and ITMX sum channels are now only 50% of the values when a new HeNe was installed. ELOG search revealed that ITMY and ETMX HeNes were replaced with newly acquired units in March and September of last year respectively. The ITMX oplev was also replaced in March 2019, but the replacement was a unit that was being used to illuminate our tourist attraction glass fiber at EX.

We should replace these before any vent as they are a useful diagnostic for the DC alignement reference.

Attachment 1: OLsum.png
  15608   Fri Oct 2 12:52:22 2020 gautamUpdateGeneralSpectroscopic grade Isopropanol delivered

2x500 ml bottles of spectroscopic grade isopropanol were delivered. I marked them with today's date and placed them in the solvent cabinet. In the process, my shoulder bumped the laser interlock switch by the door to the VEA in the drill press area, which turned the PSL NPRO off. I turned it back on just now. The other NPROs are not connected to the interlock and so were unaffected.

  15609   Sat Oct 3 16:51:27 2020 gautamUpdateCDSRFM errors

Attachment #1 shows that the c1rfm model isn't able to receive any signals from the front end machines at EX and EY. Attachment #2 shows that the problem appears to have started at ~430am today morning - I certainly wasn't doing anything with the IFO at that time.

I don't know what kind of error this is - what does it mean that the receiving model shows errors but the sender shows no errors? It is not a new kind of error, and the solution in the past has been a series of model reboots, but it'd be nice if we could fix such issues because it eats up a lot of time to reboot all the vertex machines. There is no diagnostic information available in all the places I looked. I'll ask the CDS group for help, but I'm not sure if they'll have anything useful since this RFM technology has been retired at the sites (?).

In the meantime, arm cavity locking in the usual way isn't possible since we don't have the trigger signals from the arm cavity transmission. 

Update 1500 4 Oct: soft reboots of models didn't do the trick so I had to resort to hard reboots of all FEs/expansion chassis. Now the signals seem to be okay.

Attachment 1: RFMstat.png
Attachment 2: RFMerrs.png
  15610   Sun Oct 4 15:32:21 2020 gautamUpdateSUSSuspension health check


After the earthquake on September 19 2020, it looks to me like the only lasting damage to suspensions in vacuum is the ETMY UR magnet being knocked off. 

Suspension ringdown tests:

I did the usual suspension kicking/ringdown test:

  • One difference is that I now kick the suspension "N" times where N is the number of PSD averages desired. 
  • After kicking the suspension, it is allowed to ring down with the damping disabled, for ~1100 seconds so that we can get spectra with 1mHz resolution.
  • We may want to get more e-folding times in, but since the Qs of the modes are a few hundred, I figured this is long enough.
  • I think this kind of approach gives better SNR than letting it ringdown 10,000 seconds (for 10 averages with 10 non overlapping segments of 1000 seconds), and I wanted to test this scheme out, seems to work well.
  • Attachment #1 shows a summary of the results.
  • Attachment #2 has more plots (e.g. transfer function from UL to all other coils), in case anyone is interested in more forensics. The data files are large but if anyone is interested in the times that the suspension was kicked, you can extract it from here.


  1. My cursory scans of the analysis don't throw up any red flags (apart from the known problem of ETMY UR being dislodged) 👌 
  2. The PRM data is weird 
    • I believe this is because the DC bias voltage to the coils was significantly off from what it normally is when the PRC is aligned.
    • In any case, I am able to lock the PRC, so I think the PRM magnets are fine.
  3. The PRC angular FF no longer works turns out this was just a weird interaction with the Oplev loop because the beam was significantly off-centered on the Oplev QPD. Better alignment fixed it, the FF works as it did before.
    • With the PRC locked and the carrier resonant (no ETMs), the old feedforward filters significantly degrade the angular stability to the point that the lock is lost.
    • My best hypothesis is that the earthquake caused a spot shift on PR2/PR3, which changed the TF from seismometer signal to PRC spot motion.
    • Anyways, we can retrain the filter.
    • The fact that the PRC can be locked suggest PR2/PR3 are still suspended and okay.
  4. The SRM data is also questionable, because the DC bias voltage wasn't set to the values for an aligned SRC when the data was collected
    • Nevertheless, the time series shows a clean ringdown, so at least all 5 OSEMs are seeing a signal.
    • Fact that the beam comes out at the AS port suggest SR3/SR2 suspensions are fine 👍 

Attachment #2 also includes info about the matrix diagonalization, and the condition numbers of the resulting matrices are as large as ~30 for some suspensions, but I think this isn't a new feature. 

Attachment 1: combined.pdf
Attachment 2: allPlots.zip
  15611   Mon Oct 5 00:37:19 2020 gautamUpdateBHDSingle bounce interferometer locked


The simple interferometer, composed of a single bounce reflection from ITMY and the LO beam deilvered via fiber to the AS table, can be locked - i.e. the phase of the LO beam can be controlled such that the DC light level on the DCPDs after the two beams are interfered can be stabilized. This test allows us to confirm that various parts of the sensing and actuation chain (e.g. PI PZT for homodyne phase control, Trek amplifier etc etc) are working.

I will post more quantitative analysis tomorrow.

Optical configuration:

  • LO beam is a pickoff of the main PSL beam from just before it goes into the vacuum. The optical power arriving on each DCPD after the various beamsplitters, coupling loss etc is ~200 uW.
  • IFO beam is the single bounce reflection from ITMY. For this test, ETMY, ITMX and ETMY are misaligned. Optical power arriving on each DCPD is ~80uW.
  • The two beams are interfered on a 50-50 beamsplitter. The mode-matching efficiency was estimated to be ~50% which isn't stellar, but should be fine for this test.
  • So, at half-fringe, we expect the signal on each DCPD to be linearly proportional to the phase difference between the two fields, and so we can use that as an error signal.

Servo topology:

Attachment #2 shows the servo topology.

  • For a first attempt to close the feedback loop, we can consider the two blocks labelled "Sensing Chain" and "Actuation chain" to have a flat frequency response. While this isn't true, for a taget loop with ~100 Hz UGF, I think the approximation is reasonable.
  • From the peak-to-peak value (160 cts) of the DCPD signals when the homodyne phase is uncontrolled, I estimate a sensing response (at half-fringe) of approximately 0.3 ct/nm, since this corresponds to 532nm of relative phase between the two beams. 
  • An inverting summing amplifier is used to map the +/- 2^15 ct DAC range to 0-125V on the PI PZT. Assuming the full stroke of the PZT is 10um per the datasheet, and that this voltage range drives half of the full stroke (this is just a guess since all the old PI PZT circuits were designed to work at 0-250 V), we get an actuation coefficient of 0.075 nm/ct.
  • Using these two numbers, we can then design a digital feedback loop that gives an open loop transfer function with ~100 Hz UGF, and sufficient stability margin.
  • From the earlier measurements, we have an estimate for the amount of phase fluctuations caused by (i) seismic disturbances and (ii) fiber phase noise. This is the quantity we wish to suppress, and the suppression factor will be 1/(1+L), where L is the open loop gain.
  • I didn't do this in any systematic way, but the loop in Attachment #3 seemed like a reasonable shape that would suppress the error signal RMS by ~10x, as shown in Attachment #4. So I decided to try this out.

Other notes:

  1. The idea of offloading the DC control voltage to the ITMY suspension seemed to work fine.
  2. It also seems like the relative phase between the two beams doesn't drift by so large an amount in short time scales, at least at night/quiet seismic conditions. So it is possible to maintain the lock for several seconds without having to offload the DC signal to the suspensions.
  3. I didn't bother adapting the FSS Slow PID script to do this offloading in an automated way, seemed like more trouble than was just doing it by hand. But we may want to automate this in the future.
  4. I couldn't make a clean measurement of the loop transfer function using the usual IN1/IN2 method. Introducing a step offset at the error point, the servo is able to track it (I didn't fit the step response time, but it's not as if the loop bandwidth is <1 Hz or something). I have to compare the measured in-loop error signal ASD to the free-running one to get a feel for what the UGF is, I guess, to rule out a weird loop.
  5. Update 1100 Oct 6 2020: I have now added measured, in-loop, error point spectra to Attachment #4. Looks like there might be significant sensing noise re-injection.
    • Initially, I forgot to turn the HEPA on the PSL down for the measurement. So I have the two traces to compare. Looks like with the HEPA turned up to full, there is more noise in the 50-200 Hz range.
    • The trace marked "highGain" was taken with an overall loop gain that was 3dB higher than the nominal value - I could see some oscillations start to appear, and in the spectrum, maybe the feature at ~150 Hz is evidence of some gain peaking?


  1. The PI PZT seems to work just fine.
  2. Need to look into the loop shape. I guess it's not reasonable to expect a UGF much higher than 100-200 Hz, because of the various delays in the system, but maybe the low frequency suppression can be made better.
  3. What are the next steps?? What does this mean for the RF44 sensing scheme?
Attachment 1: simpleHomodyne.png
Attachment 2: singleBounceIFO.pdf
Attachment 3: proposedController.pdf
Attachment 4: freeRunningSuppressed.pdf
  15613   Mon Oct 5 14:01:41 2020 gautamUpdateElectronicsaLIGO demod boards stuffed and delivered

We received 20pcs of stuffed demodulator boards from Screaming Circuits today. Some caveats:

  1. The AP1053 amplifiers weren't stuffed. Note that this part is no longer in standard production, and lead time for a custom run is ~half a year. I recommend stuffing R2 and using a minicircuits amplifier upstream of this board. We have 6 pcs of AP1053 in hand so we can use those for the first AS WFS, but a second WFS will require some workaround.
  2. AD8306ARZ weren't sent to Screaming Circuits. This part is used for the LO and RF signal level detection/monitoring stage, and so aren't crucial to the demodulation operation. @Chub, did we order the correct part now? They are rather pricey so maybe we can just adapt the footprint using some adaptor board?
  3. DQS-10-100 hybrid 90 degree splitters were delivered to us after the lot was sent to Screaming Circuits. We have the pieces in hand, so we can just stuff them as necessary.

I removed 1 from the group to stuff some components that weren't sent to Screaming Circuits and test the functionality on the benchtop, the remaining have been stored in a plastic box for now as shown in Attachment #1. The box has been delivered to Chub who will stuff the remaining 19 boards once I've tested the one piece.

Attachment 1: IMG_8888.JPG
  15618   Thu Oct 8 08:37:15 2020 gautamUpdateComputer Scripts / ProgramsFinesse GUI

This looks cool, we should have something similar, can be really useful.

  15620   Thu Oct 8 15:08:25 2020 gautamUpdateGeneralSome boxes moved from 40m entry hallway
  • UPS batteries
  • 2x HEPA filters
  • VWR chemicals (methanol)

These boxes were moved from the 40m hallway to the inside of the VEA so that we have some space to walk around. You can find some pictures here.

  15623   Tue Oct 13 11:13:54 2020 gautamUpdateBHDInvestigation into RF44 sensing

Attachment #1: spectra of the phase noise between LO and IFO output fields sensed using the RF44 signal.

  • Measurement setup:
    • LO an IFO fields are combined on a beamsplitter, with ~60% mode-matching efficiency.
    • One port of the BS goes to a DCPD.
    • The other port goes to an RF sensing photodiode, PDA10CF. The spec-ed dark noise NEP is ~12 pW/rtHz at 1.6 um, (so let's say 25 pW/rtHz) and transimpedance is 5kohms into a 50 ohm load. We can convert this to an equivalent sensing noise at the error point of this loop, though it's more likely that the electronics (demod, ADC etc) noise downstream dictate the sensing limit, which I measure by blocking light on the photodiode.
  • The demodulation is done on one of the newly received D0902745 boards - this was just a more compact setup than many cascaded minicircuit components. We don't have the hardware to package this into a chassis to shield against electronics noise pickup yet, so I'm using a bench supply to power this for now (via a voltage regulation board, D1000217.
  • "Dark Noise" = ASD with no light incident on the photodiode. "LO field only" = ASD with only the LO field incident on the photodiode.
  • The "Dark noise" trace and "LO field only" traces are converted from cts/rtHz to rad/rtHz by noting that when the Michelson is locked on a dark fringe, the demodulated RF44 quadratures have a pk-pk amplitude of ~160 cts (corresponding to pi radians of phase shift). Since in these conditions the demodulated quadratures do not undergo any fringe wrapping, I converted the spectra by simple multiplication.
  • For the "RF44 open loop" trace:
    • The DC offset in the demodulated signal (due to the RF44 signal from the LO field only) is digitally compensated, so that the fringing has (roughly) zero offset.
    • The Michelson was locked on a dark fringe, and the demodulated RF44 quadratures were monitored for ~5 mins. Then arctangent (specifically, arctan2 to get the correct quadrant in the IQ plane) of the two signals was taken to convert the fringing signals to phase noise.

Closing a feedback loop:

  • Since it seems like we are sensing a signal (below ~1kHz at least), I tried to close a feedback loop (modelled loop shape shown in Attachment #2, it's just a model because I have to guess what the sensing and actuation gains are, and they're both assumed to be flat, digital delays etc aren't accounted for). I've also added the inferred loop gain by taking the ratio of the in loop and unsuppressed ASDs (though of course I don't account for the flat sensing noise at higher frequencies). At least qualitatively, things line up...
  • While I can get the light level on the DCPD to stabilitze somewhat, the loop is not at all stable, and the suppression isn't very good at all.
  • Not sure how meaningful any of the spectra with the loop closed are, but FWIW, I've put in the spectra of the demodulated RF44 signals with the loop engaged (RF44 Q is used as the error signal). A clear problem is evident at ~120 Hz, and the forest of lines isn't helping for sure. Also unclear to me why the I and Q signals don't have the same profile at low frequencies.


  1. What is the reason for the huge forests of lines in the "RF44 open loop" ASD, that are absent in the other two traces? If this were electrical pickup, it should be there in all three traces?
  2. Is the shape of the spectrum reasonable? The roll-off above ~5 Hz doesn't seem quite steep enough to be seismic noise from the suspensions. Can it really be that the Michelson dark field has such high phase noise?
  3. How can we get this scheme to give us cleaner sensing?
  4. The actuation chain was verified to work fine with the single bounce beam from an ITM interfered with the LO field, and using the DC light level as an error signal and locking to the half-fringe point. So the problem is not due to insufficient actuation range. Seems like the error signal is so polluted with these forests of lines that even though there is some suppression of the error signal at low frequencies, the unsuppressed noise is still significant. I can't solve the problem by simply increasing the loop gain...
  5. It is not shown here, but with only the LO field incident on the RFPD, I see a drift of the demodulated signals on the ~5 minute timescale - is this just due to fiber length change? If so, this is potentially problematic, as on long time scales, the true zero of the error point of the servo would be changing on the ~5 minute timescale. This would be true even for the final suspended scheme - if the path length between PR2 and the homodyne BS changes by some microns, we would have to correct this at DC?
Attachment 1: phaseNoisePSD.pdf
Attachment 2: loopTF.pdf
  15624   Tue Oct 13 21:22:29 2020 gautamUpdateGeneralSpace cleared in 1Y1 for new FEs

[JV, GV]

We cleared up some space in the 1Y1 electronics rack to install the 3 new FE machines. I removed the current driver and laser from 1Y1, they are now stored in the E10 cabinet. I will upload some photos to gPhotos soon.

  1. I think it's good to have all these FEs in one rack (at least the new ones) - we should then hook it up to an ethernet power source, so that we can remotely power cycle them. I think we have long enough cables to interface to expansion chasses / dolphin switches, but if not, I think it's still a good idea to have these machines in 1Y1 as it is the least sensitive area in terms of immunity to bumping some cable during setup work and disturbing the rest of the IFO.
  2. We found that the rails that the Supermicros shipped with the servers seem to be just a little too narrow - we mounted these in the rack, but had considerable difficulty sliding the server units in. Once they are in, they don't slide smoothly. Is there some special trick to installing these? 
  3. I spent a few minutes trying to get Debian 8 installed on these machines, so that the rest of the setup work could be done remotely - however, there appear to be some firmware issues and so I'm not gonna dive into this.
    • I couldn't find a disk image for Debian 8.5 which is what the KT wikl recommends, so the OS I tried to install was Debian 8.11.
    • The error that comes up is related to a "stalled CPU" - apparently this is related to some graphics driver issue (there's another forum page that suggests upgrading the BIOS, but I don't think that's the problem here).
    • Anyways, this part of the process is only to install some drivers and do the initial setup - these machines will eventually run a diskless boot from the image on FB, so who knows if there will be some other driver issues/hardware-software incompatibilities there 😱 .
    • We should also make an effort to set these machines up with IPMI, but I think we first need to install an OS and a CLI to setup the IPMI. My cursory browsing of the manual suggests that the initial setup maybe can be done without installing an OS, and then subsequent work, including OS install, can be done remotely. If someone reads more in detail and can provide me a step-by-step, I can follow those instructions (if they aren't available to come into the lab). See here for some brief documentation of how to access the IPMI.
  15627   Wed Oct 14 18:16:27 2020 gautamUpdateGeneralPSL HEPA-->50%

Per Koji's suggestion, I turned the PSL HEPA Variac to 50% just now, so that the power load through the burnt electrical cable is reduced by 75%.

  15628   Thu Oct 15 10:42:39 2020 gautamUpdateBHDMore investigation into RF44 sensing

Summary of discussion between Koji and gautam on 14 Oct:

  1. Koji questioned the accuracy of the "open loop" ASD shown here. While it may not be entirely accurate to compute the free-running (homodyne) phase noise simply by taking the arctangent of the I and Q signals (because the magnitude of the signal is also changing), gautam claims the estimate is probably still close to the true homodyne phase, especially since the ratio of the "in-loop" and free-running ASDs gives something that closely approximates the magnitude of the supposed OLG of the system.
  2. Koji suggested the following tests:
    • Investigate the relative stability of the two RF signal generators involved in this system. Since the 44 MHz electrical LO signal (for demodulation) is generated by a separate IFR from the one used to imprint 11 MHz and 55 MHz phase modulation sidebands on the main PSL beam, we want to investigate what the drift is.
    • Try implementing an analog feedback loop using LB1005 - the idea being we should be able to implement higher bandwidth control, for better suppression of the high frequency noise (which looking at the ASD is not only due to seismic phase modulation of the IFO output field). Maybe some combination of this and the Marconi investigation would suggest why we have these forests of lines in the ASDs of the error signal?
    • Turn off the HEPAs on the PSL enclosure completely as a test, to see if that improves (i) phase noise due to air currents and (ii) mechanical pickup on the fiber producing  phase noise.

I tried all of these last night / overnight, here are my findings.

Analog locking of the homodyne phase:

See Attachment #1

  • RF44_I was used as the error signal.
  • The "C1:OMC-ZETA_IMON_OUT" channel is actually looking at the error signal monitor from the LB1005, and is uncalibrated in this plot.
  • The "monitor" port on the demodulator board provides a convenient location for us to route the demodulated signal to an LB1005 box, while simultaneously digitizing both demodulated quadratures.
  • Empirically, I found settings that could engage the lock. I also found that I couldn't increase the gain much more without destroying the lock. 
  • The time domain signals look much "cleaner" in this analog feedback loop than when I achieved similar stabilization using the digital system. But I will quantify this more when I post some spectra of the in loop error signals.
  • I will do some more characterization (loop TF measurement, error point spectrum in lock etc), but in summary, it looks like we still only have ~100 Hz UGF. So something in the loop is limiting the bandwidth. What could it be?
  • The main problem is that the LB1005 isn't well suited to remote enabling/disabling of the lock, so this isn't such a great system.

Relative stability of two IFR2023s synchronized to the same FS725 Rb standard:

The electrical LO signal for demodulation of the 44 MHz photocurrent is provided by an IFR2023 signal generator. To maintain a fixed phase relation between this signal, and the phase modulation sidebands imprinted on the interferometer light via a separate IFO2023 signal generator, I synchronize both to the same Rb timing standard (a 10 MHz signal from the FS725 is sent to the rear panel frequency standard input on the IFR). We don't have a direct 44 MHz electrical signal available from the main IFO Marconi at the LSC rack (or anywhere else for that matter). So I decided to do this test at 55 MHz. 

  • RF input of the demodulator was driven by 5*11.066209 MHz pickoff from the LSC rack.
  • LO input of the demodulator was driven by 5*11.066209 MHz signal from the IFR2023 used for the RF44 demodulation setup.
  • The outputs were monitored overnight. The RF44_Q channel had a DC level of nearly 0. So this channel is nearly a linear sensor of the phase noise between LO and RF signals.
  • To convert ADC counts to radians, I offset the LO Marconi frequency by 100 Hz, and saw that the two quadratures showed pk-pk variation of ~24000cts. So, at the zero crossing, the conversion is 1/(24000/2) rad/ct ~83urad/ct.
  • The result is shown in Attachment #2. The "measurement noise" trace corresponds to the RF. input of the demodulator being terminated to ground with a 50 ohm terminator.
  • For comparison, I also overlay the phase noise estimate of an individual IFR from Rana. In his investigation, the claim is that the PLL that locks the IFR to the Rb timing standard has ~1kHz UGF, but if my measurement is correct, the relative stability between the two signal generators synchronized to the same timing standard already. degrades at ~1 Hz. Could be just a cts/rad calibration error I guess.
  • In any case, we are far from saturating this limit in the homodyne phase lock.
  • There are several sharp lines in this measurement too - but I don't know what exactly the source is. Of course the two marconis are plugged into separate power strips, so that may explain the 60 Hz lines and harmonics, but what about those that aren't a multiple of 60 Hz?

A look at the time domain signal:

With the Michelson locked on the dark fringe, the RF44 I and Q signals in the time domain are shown in Attachment #3 for a 1 minute stretch.

  • The RF44 signal level bottoms out at ~40 cts. Okay, so this is the offset.
  • However, the maximum value of the RF44 signal amplitude seems to be modulated in time. How can we explain this?
Attachment 1: analogZetaLock.png
Attachment 2: relPhaseNoise.pdf
Attachment 3: sigMagPhase.pdf
  15636   Thu Oct 22 11:14:47 2020 gautamUpdateElectronicsHV coil driver packaged into 2U chassis

I packaged the HV coil driver into a 2U chassis, hoping for better shielding from pickup. There is still considerable excess noise in measurement vs model around 100 Hz, see Attachment #1. The projected displacement noise from this noise contribution is shown in Attachment #2 - I've also plotted the contribution from the 4.5kohm (planned value for fast path series resistance) for comparison. Attachment #3 has some photos of the measurement setup so if someone sees some red flags, please let me know.

  • The noise was measured with the output load connected to a 20ohm load resistor, to simulate an OSEM.
  • The input signal was driven with an Acromag, to try and mimic the actual operating conditions as closely as possible (although the fast path input was left unconnected).
  • The KEPCO switching HV power supplies were used to power the unit.

I've run out of ideas to try and make the measurement cleaner - the presence of the rather prominent power line harmonics suggests that this is still not perfect, but what more shielding can we implement? I have to make the measurement on the circuit side of the 25 kohm series resistor, so I am using some Pomona minigrabbers to clip onto the leg of the wirewound resistor (see photos in Attachment #3), so that's not great maybe, but what's the alternative?

So if this is truly the noise of the circuit, then while it's an improvement on the current situaiton, it's unsatisfying that such a simple circuit can't match the design expectations. But how do we want to proceed?

Attachment 1: HVampNoise_driven_chassis.pdf
Attachment 2: HVampNoise_dispUnits.pdf
Attachment 3: D1900163_measurementSetup.zip
  15639   Thu Oct 22 22:01:53 2020 gautamUpdateElectronicsHV coil driver packaged into 2U chassis

It's not so easy to directly measure this I think, because the filtering is rather aggressive. Attachment #1 shows the measured transfer function (dots) vs the model and Attachment #2 shows the noise. I think this checks out - but I can't definitively rule out some excess noise at 100 Hz from this stage. Because the gain of the HV stage is x31, we'd need a preamp with better than 1nV/rtHz to directly measure the noise I guess. The Acromag noise model in Attachment #2 is based on a measurement I describe here.


what is the noise level before the HV stage? i.e. how well is the acromag noise being filtered?

Attachment 1: DACnoiseFilterGain.pdf
Attachment 2: DACnoiseFilterNoises.pdf
  15644   Mon Oct 26 17:26:26 2020 gautamUpdateIOOExcess laser freq noise investigation

Apart from the questionable wiring on the Acromags, one other important difference is in the way the connections were made between the old VME crates to the Eurocrate backplanes, and how we do it now. The thick cables had their sheilds connected to the eurocrate ground (or at least, there was a dedicated ground lug on those cables which we screwed on to the ground terminals on the Eurocrate backplanes). However, in our current configuration, we interface the Acromag ADCs and DACs to the backplane via these adaptor boards. The shields of the DSUB cables are presumably NOT connected to the Eurocrate grounds. This should also be investigated as one potential cause of the grounding issue - while on some of the Eurocrate modules, the P1/P2 connectors may have either the "A" or "C" row of connectors shorted to ground, some may not, and the TTFSS may suffer from such an issue?

Note that we have this problem in all of the slow machines that were upgraded to Acromag (if this turns out to be the issue). 


In fact, the problem was the grounding issue (presumably on the IOO racks).

  15645   Tue Oct 27 23:47:53 2020 gautamUpdateGeneralISS checkout

I wanted to look into the ISS situation. Some weeks ago, I found the PD that was previously used as the in-loop photodiode. I wanted to use this and measure the open-loop RIN at a few places (to see if there's any variation and also to check its functionality). However, I didn't get very far tonight - for a start, the PD height is 3" (while our beam height is 4" everywhere outside the vacuum), and I needed to put together a circuit to supply the 5V bias and +/- 15 V since the transimpedance is done on the head. I was only able to do a low-level functionality test tonight, checked that the DC voltage output varied linearly with the incident power (calibrated against an NF1611 photodiode, data will be put up later). I didn't get to measuring any noise performance - is an incandescent light bulb still shot noise limited at ~10 Hz < f < 10kHz? Some notes:

  1. The PD is DC coupled, and has a transimpedance of 1 kohm (inverting AD829 does the transimpedance).
  2. Probably a daughter board should be made that supplies the DC power voltages and rotues the output signal to something more convenient like a BNC connector. This daughter board can then also implement a DC coupled path (for monitoring) and AC coupled path (for servoing, fc to be determined).
  3. SR560 based ISS was implemented some years ago but I think the improvement was only seen above 100 Hz, and that too was marginal, the stabilized RIN was 10^-6 (monitored on an out-of-loop photodiode I think, but unsure). We'd probably want to aim for at least an order of magnitude better. Unclear at this point why more suppression wasn't possible back then, was it just insufficient loop gain, or was the sensing noise too high? To be investigated.

Unconnected to this work - this problem reared its ugly head again (i noticed it yesterday morning already actually). I don't have the energy to embark on a fix tonight, Koji is going to be in the lab all day tomorrow and so he will fix it.

  15648   Wed Oct 28 14:07:47 2020 gautamUpdateGeneralISS checkout

Ok I was using the PD in the black mount because Rana recommended it a few weeks ago.

Regarding the M2ISS, I acquired the hardware from QIL some months ago, including a circuit board, and 2 PDs. These had LEMO outputs though (not BNC), and the mounts are not 4". These photodiodes are what I'm using as the airBHD DCPDs right now, and some photos are here - are these the photodiodes you mentioned? Or are there yet more M2ISS photodiodes? I remember Johannes had some custom mounts extruded to make them 4" high, do you mean those? Can I retrieve them his Cryo setup?

BTW, my elog scraping shows only one spectra from Stefan in the ATF elog, and the performance there is more like 1e-7/rtHz @ 100 Hz, and that’s using a dedicated high BW servo circuit, not the SR560. Am I just missing the measurement of 2e-8/rtHz?


that little PD in the black mount was never very good. The AD829 is not a good opamp for transimpedance and especially not good for low frequencies. Stefan Ballmer and I were able to get 2e-8 out of these (@100 Hz) many years ago.

I wonder if we have some of Zach's M2ISS photodetectors around, perhaps in QIL or Cryo. I doubt that any of them are in use now. Those had good performance nad BNC output.

  15651   Thu Oct 29 12:43:35 2020 gautamUpdateGeneralVent Prep
  1. Oplev HeNe at ETMY was replaced, see here for my earlier discussion on this.
    • I thought this is a good idea since we want the Oplev as a coarse reference and it'd not be ideal if this HeNe dies during the time the optic is out of the chamber.
    • New HeNe had 2.8mW of power output as measured with Ophir power meter. This is in line with what is expected from these Lumentum heads.
    • I labelled the head with the power output and today's date, re-aligned the Oplev reflection onto its QPD. 
    • After this work, the Y-arm could be locked without the huge angular fluctuations that was visible earlier, 👍 .
  2. GTRY anomaly
    • I actually judged that there is no anomaly.
    • The GTRY CDS indicator is actually quite useless - the ADC saturates at ~3500 cts (and not 32768 as you would expect from a 16 bit ADC but that's the well-known whitening filter saturation problem). This should be fixed, but this is a task for later.
    • I measured with a DMM the voltage when the TEM00 is locked to the cavity and GTRY is 0.3 (the nominal value these days), the DC voltage was ~5.6 V. The prompt reflection from the ETM registers ~6.5 V DC. So the mode-matching isn't stellar, but this is again a known issue, and can be fixed later.
  3. Other pre-vent checks
    • The Oplevs had drifted significantly, I re-centered ITMs, ETMs and BS after aligning the arm cavities and green beams in the POX/POY lock state. See Attachment #1.
    • I locked the PRMI on carrier and used this config to re-center the PRM Oplev, see Attachment #2.
    • No further action was taken regarding SRM oplev.
    • I checked the ALS noise, see Attachment #3. The X arm ALS has excess noise at ~100 Hz that certainly wasn't there previosuly - sigh. There is nothing I can find about any changes made at EX in the elog.
    • Updated the "DriftMon" values, though I guess we don't even really use this anymore these days?
    • Re-relieved the IMC WFS offsets.
    • Cut the input power to the IMC from 1.007W to 100.1 mW (both numbers measured with Ophir power meter).
    • Replaced the 10% beamsplitter in the MCREFL path on the AS table with a Y1 HR mirror. Note that there is no beam on the IMC WFS in this configuration.
    • Was able to lock the IMC on low power to a TEM00 mode - need to set up the low power autolocker. The IMC autolocker is now set to the low power settings, and I've tested it locks a couple of times. Attachment #5 shows the low power lock in StripTool.
    • Walked around and looked at all the bellows - the jam nuts are up against their stops, and I can't move them with my hands, so I think that's okay.

If everything else looks good, I'll start letting the dry N2 into the main volume after lunch.


Now the green transmissions are visible by the green PDs. Attachment 9 shows the trans and ref of each green beams with and without locking to TEM00. The questionable green TRY was ~0.3. If we compare this with the histrical data (Attachment 10), it is about 1/4 of the value in the past. It's not too crazy but still quite low.

BTW, nice video! @ Koji, How difficult was it to edit it into this form? 

Attachment 1: preVentAlignment.png
Attachment 2: PRMIcarr_PRMalignment.png
Attachment 3: ALS_ool.pdf
Attachment 4: lowPowerMC.png
  15652   Thu Oct 29 19:30:56 2020 gautamUpdateGeneralVent complete
  1. Jam nuts - checked that none of the nuts move by hand, which according to Steve, is sufficient. I recorded photos of all of them here.
  2. Lab particle count - at the SP table, I measured 25,910 / cf @ 0.3 um and 1220 /cf  @ 0.5 um. Steve's guidance is that the latter number should be less than 10,000, so no issues there.
  3. Valves - closed all the annuli off, and also closed VM1, VA6 and V1. The procedure calls for VM3 to be opened, presumably so that the RGA remains pumped, but I see no reason why we can't just leave that RGA volume valved off.
  4. Started letting N2 into the main volume at ~3pm local time, by opening VV1. Attachment #1 shows the valve configuration adopted for this vent. Went up to 25 torr and then switched over to the "Ultra Zero" instrument grade air cylinders.
  5. Aimed for 3-5 torr/min rate of pressure increase locally. The full vent trend can be seen in Attachment #2.
  6. Stopped letting air into the main volume when P1a pressure was 700 torr, at which point I disconnected the cylinders from the main volume, and left VV1 open so that the IFO equilibriates to the lab air pressure. I used 4 full cylinders of the instrument grade air, which is par for the course. 
  7. Since we anticipate opening the ETMY and output optics chambers, I also vented those annuli. The final state of the valves as I am leaving it for the night is shown in Attachment #3.
  8. I re-aligned the IMC mirrors so that I could lock the IMC at low power once again. Indeed, IMC trans of ~1400 cts was realized (see Attachment #4), c.f. ~1500 earlier today, I thought this is fine and didn't optimize further. I think the policy is to not lock the IMC in air unless necessary, so I'm leaving the PSL shutter closed for the night.
  9. The ITMs, ETMs and BS were re-aligned such that the Oplev spots are centered. I can see some higher order resonances of the green beams. The AS spot is fringing (the two ND=0.5 filters on the AS CCD camera were removed for better visibility). So this is fine as far as alignment is concerned.

We are now ready to take the doors off. I've already done the basic prep work (loosened bolts, cleaned chamber, carts for tools, fresh ameristat on portable HEPAs etc).


If everything else looks good, I'll start letting the dry N2 into the main volume after lunch.

Attachment 1: ventValveStatus.png
Attachment 2: vent_Oct2020.jpg
Attachment 3: ventValveStatus_overnight.png
Attachment 4: lowPowerIMC_afterVent.png
  15653   Mon Nov 2 12:44:25 2020 gautamUpdateGeneralETMY and OMC chamber heavy doors off

[koji, rana, gautam]

This morning, we did the following;

  1. Removed the ETMY chamber heavy door. It is stored on the rack at the east end.
  2. Removed OMC chamber heavy door. It is smaller than the other chamber doors, so doesn't sit on the standard size rack we have. So it is laid flat, on a clean sheet of ameristat, on a cart next to the NE corner of the PSL table.
  3. After taking some photos of the chamber and making sure the position of the suspension tower was marked by some stops, we removed the ETMY cage and moved it to the cleanroom area. The optic was EQ stopped during the transport, and the OSEMs were removed. The ETMY Oplev HeNe was turned off and the PSL was shuttered to allow us to work without goggles.
  4. The broken off magnet was retrieved from inside the OSEM. The shadow sensor voltage recovered a value of ~800cts, which means the LED/PD pair on the UR OSEM seems to work fine, and it was in fact the magnet blocking the PD that was the issue.
  5. In the cleanroom area, we removed the optic from the wire loop and placed it on the magnet gluing fixture. The wire is intact (for now), so there is some hope of re-suspending it in the same loop.

The OSEMs remain in the EY vacuum chamber. The next set of steps are:

  1. Clean the EP30-2 residue from the broken magnet joint - this will require some scrubbing with an acetone soaked scrub or similar implement.
  2. Reglue the broken magnet.

We will most likely work on this tomorrow. At ~1615, I briefly opened the PSL shutter and tweaked the IMC alignment. We will almost certainly change the pointing into the IMC when we remove the old OMC and rebalance that table, so care should be taken when working on that...

  1. We are now ready to take the doors off. 
Attachment 1: CB84FD62-C48B-414F-8825-50C9DE0B47CF.jpeg
  15654   Mon Nov 2 16:46:06 2020 gautamUpdateGeneralPlan for OMC chamber

To be a bit more clear about what we are going to do in the OMC chamber, I marked-up some photos, see Attachments #1 and #2.

  1. OM5 will be rotated to bring the IFO AS beam straight out without any splitting to the OMC.
  2. OMMT, OMC, DCPD, DCPD transimpedance amp, and all peripheral optics associated with these components, will be removed. Many of these components are mounted on a breadboard and so removing that breadboard will take care of it. These are marked with pink Xs.

I anticipate that after this work, the only components on the table will be 

  1. IM1, to send the PSL beam to the IMC.
  2. OMs 5 and 6 to bring the IFO AS beam out onto the AP table (in principle, we could try and eliminate both these optics, if the AS beam happens to exit through one of the viewports cleanly, we will not have any intervening objects in the way once the OMC and peripherals are removed).
  3. MMT2 for mode-matching the IMC transmission to the interferometer mode.

Are we in agreement with this plan?

See #15656 for the updated photo

Attachment 1: IMG_2318.JPG
Attachment 2: IMG_2332.JPG
  15656   Mon Nov 2 17:32:05 2020 gautamUpdateGeneralPlan for OMC chamber

Good point - looking back, I also see that I already removed the mirror at the SW corner of the table in 2016. Revised photo in Attachment #1. There is an optic on the east edge of this table whose purpose I'm not sure of, but I'm pretty sure it isn't essential to the main functionality and so can be removed.


I believe the mirror next to IM1 is for the green beams to be delivered to the PSL table. I think we still want to keep it. Otherwise, the plan looks fine.

Attachment 1: IMG_2317.JPG
  15657   Tue Nov 3 09:06:50 2020 gautamUpdateGeneralLab alarm tripped

I got a call from Calum ~830am today saying some facilities people entered the lab, opened the south entrance door, and tripped the alarm in the process. I came to the lab shortly after and was able to reset the alarm by flipping the switch on the alarm box at the south end entrance to "Alarm OFF". Then, I double checked that the door is closed, and re-enabled the alarm. The particle count at the SP table is not unusually high and the lasers (Oplev HeNe and AUX X) were still on, so doesn't look like any lasting damage was done. The facilities people were apparently wearing laser safety goggles.

  15658   Wed Nov 4 00:10:43 2020 gautamUpdateGeneralVent activities today

[koji, gautam]

  1. Glued broken off magnet - curing overnight with lamp to slightly elevate temp for curing.
  2. Remvoed material from OMC chamber as per the plan. This is all sitting wrapped up in foil on a cart for tonight, we should figure out a better storage plan eventually.

The IMC isn't resonant for a TEM00 mode at the time of writing - we are waiting for the stack to relax, at which point if the IMC isn't resonant for a TEM00 mode, we will tweak the input pointing into the IMC (we want to use the suspended cavity as the reference, since it is presumably more reliable than the table from which we removed ~50 kgs of weight and shifted the balance.

Attachment 1: IMG_0184.jpg
Attachment 2: IMG_0185.jpg
Attachment 3: IMG_0187.jpg
Attachment 4: IMG_0207.jpeg
Attachment 5: IMG_0210.jpg
  15659   Wed Nov 4 17:14:49 2020 gautamUpdateCDSc1bhd setup

I am working on the setup of a CDS FE, so please do not attempt any remote login to the IPMI interface of c1bhd until I'm done.

  15660   Thu Nov 5 22:27:34 2020 gautamUpdateGeneralVent activities today

[koji, gautam]

  1. Tweaked last steering mirror before PSL beam is launched into vacuum to get the IMC resonant for the TEM00 mode. Less than 1/8 turn in each Pitch and Yaw was required, and we recovered MC2T ~ 1300 cts (c.f. the expected ~1500 cts when the cavity is well aligned, but we didn't touch the IMC mirrors).
  2. ETMY magnet re-gluing
    • Regluing was successful. Pickle picker came off smoothly. 
    • We performed the razor blade test on all 6 magnets, the integrity of the joints seems uncompromised.
  3. ETMY cleaning
    • Applied F.C. to HR face, HR edge (to remove some residual stains) and AR surface (taking care to go around the magnets).
    • A fresh bottle each of spectroscopic grade acetone and isopropanol was dispensed into clean beakers for the work.
    • Curiously, in the lighting conditions of the cleanroom, both HR and AR faces looked surprisingly clean, even when illuminated with the green flashlight.
    • However, when we took it to the east end and looked at it again with the green flashlight, just before putting it into the chamber, we saw all sorts of stains and markings on the HR side.
    • The central aperture looks fine - the contamination did not happen during the transport so we feel confident not to futz around more with this and since we cleaned it extensively in the cleanroom, we opted not to do another round of cleaning in the chamber.
  4. ETMY re-suspension (on the existing wire loop)
    • The back OSEM-holding plates were removed for access reasons.
    • The optic was rested onto the EQ stops. 
    • Patient nudging of the wire around the optic allowed us to eventually get the wire into the V-grooves in the standoffs.
    • The wire was not damaged! (at least, to eye, free-swinging test is going on now, we will see what the eigenfrequencies and Qs are)...
    • We confirmed that the wire is seated in the v-groove, and took some close up photos.
  5. ETMY pitch balancing
    • A HeNe was brought into the cleanroom and mounted at 5.5" beam height.
    • Level-ness of the beam (i.e. parallel to the optic table) was done coarsely with a spirit level placed on the HeNe holder, and more fine adjustment was done by checking the beam height at the launch point, and again ~2 meters away.
    • All EQ stops were backed off to ensure the optic was really "free-swinging" - no OSEMs either at this point, since all this was done in the cleanroom.
    • The return beam hit the HeNe's output aperture, so we were satisfied that the pitch balancing was good to ~1mrad.
  6. ETMY transport back to EY
    • Since we want to maintain some tension in the wires, the bottom EQ stops were not raised as much as they were when the tower was transported from EY to the cleanroom.
    • As a result, we felt it'd be better for a human (Koji) to carry the cage (rather than cart it, the idea being that there would be fewer vertical impulses due to bumps on the ground etc).
    • The transport went smooth.
  7. EY chamber work
    • Optic was placed back in its original position (marked by some L-clamps).
    • In preparation for alignment work, I undid all the CDS changes I made to facilitate the temporary 3-magnet actuation. This necessitated a reboot of c1auxey VME crate.
    • OSEMs were inserted - best effort to half light as usual, and we tried to replicate the rotation in the mounts as closely as possible to what was the case before (using a photo from the last vent as reference).
    • Coarse alignment was done by making the Oplev return beam hit the QPD.
    • Better alignment of the Y-arm cavity axis was done using the green beam. 
    • IMC was locked - the beam alignment target was used to ensure the IR beam was hitting ETMY at the correct height (by moving TT2), and then the ETMY angular alignment was improved to make the return beam go back roughly collinearly.
  8. Y-arm POY locking
    • Satisfied with the alignment, we returned to the control room (light doors back on EY chamber) - cavity alignment was tweaked further to make the TRY resonances as high as possible.
    • I could lock the Y-arm length to the PSL frequency using POY as a sensor - TRY was ~0.04-0.05 (since IMCT is 1/10th its usual value, we expect it to be 0.1, so this is not bad).
  9. OMC chamber work
    • Tied down the balance weights that were previously placed after the OMC and peripheral optics were removed.
    • Checked the table leveling.
    • Checked that the AS beam is reasonably well centerd on OM5 and OM6. Took some photos.
    • Checked that the IMC could be locked after this work - it could.

So all the primary vent objectives have been achieved 🙌 . The light doors are on the chamber right now. I'm measuring the free-swinging spectra of ETMY overnight. Barring any catastrophic failures and provided all required personnel are available, we will do the final pre-close-up checks, put the heavy doors back on, and pump down starting 10 am Monday, 9 Nov 2020. Some photos here.

  15661   Fri Nov 6 11:36:37 2020 gautamUpdateGeneralETMY suspension eigenmodes

Attachment #1 shows the main result - there are 4 peaks. The frequencies are a little different from what I have on file for ETMY and the Qs are a factor of 3-4 lower (except SIDE) than what they are in vacuum, which is not unreasonable I hypothesize. The fits suggest that the peak shape isn't really Lorentzian, the true shape seems to have narrower tails than a Lorentzian, but around the actual peak, the fit is pretty good. More detailed diagnostic plots (e.g. coil-to-coil TFs) are in the compressed Attachment #2. The condition number of the matrix to diagonalize the sensing matrix (i.e. what we multiply the "naive" OSEM 2 Euler basis matrix by) is ~40, which is large, but I wouldn't read too much into it at this point.

I see no red flags here - the PIT peak is a little less prominent than the others, but looking back through the elog, this kind of variation in peak heights doesn't seem unreasonable to me. If anyone wants to look at the data, the suspension was kicked every ~1100seconds from 1288673974, 15 times.


I'm measuring the free-swinging spectra of ETMY overnight. 

Attachment 1: ETMY_pkFitNaive.pdf
Attachment 2: ETMY.tar.bz2
  15662   Fri Nov 6 14:08:44 2020 gautamUpdatePSLPMC re-locked

The PMC servo railed and so I re-locked it at ~half range. I've been noticing that the diurnal drift of the PZT control voltage has been larger than usual - not sure if it's entirely correlated with temperature on the PSL table. Anyway the cavity is locked again so all is good.

  15663   Fri Nov 6 14:27:16 2020 gautamUpdateCDSc1bhd setup - diskless boot

I was able to boot one of the 3 new Supermicro machines, which I christened c1bhd, in a diskless way (with the boot image hosted on fb, as is the case for all the other realtime FEs in the lab). This is just a first test, but it is reassuring that we can get this custom linux kernel to boot on the new hardware. Some errors about dolphin drivers are thrown at startup but this is to be expected since the server isn't connected to the dolphin network yet. We have the Dolphin adaptor card in hand, but since we have to get another PCIe card (supposedly from LLO according to the BHD spreadsheet), I defer installing this in the server chassis until we have all the necessary hardware on hand.

I also have to figure out the correct BIOS settings for this to really run effectively as a FE (we have to disable all the "un-necessary" system level services) - these machines have BIOS v3.2 as opposed to the older vintages for which there are instructions from K.T. et al.

There may yet be issues with drivers, but this is all the testing that can be done without getting an expansion chassis. After the vent and recovering the IFO, I may try experimenting with the c1ioo chassis, but I'd much prefer if we can do the testing offline on a subnet that doesn't mess with the regular IFO operation (until we need to test the IPC).


I am working on the setup of a CDS FE, so please do not attempt any remote login to the IPMI interface of c1bhd until I'm done.

Attachment 1: Screenshot_2020-11-06_14-26-54.png
  15664   Sun Nov 8 20:22:35 2020 gautamUpdateGeneralVent activities today

Basic IFO alignment checks were done.

  1. IMC could be locked - I tweaked the cavity alignment a little to maximize the MC transmission.
  2. Y arm can be made resonant for a TEM00 mode of the main beam. I can't run ASS successfully in this low power config. I can see that the axis isn't great, the spot is visually off-center on ITMY, but we should have plenty of actuator range to correct for this with TT1/TT2.
  3. X arm shows IR mode flashes in the TRX QPD. The green beam can be made resonant for a TEM00 mode, but that alignment doesn't yield the largest IR resonant peaks in TRX. I suspect it is due to the mis-alignment of the beam axis.
  4. AS beam was aligned onto the CCD. I could see clean Michelson fringes by tweaking the BS alignment. On the AP table, I noticed that the beam on the first steering mirror after the AS beam exits the vacuum is a little high. We can easily resolve this by tweaking OM6 pitch a bit, but even if we don't I don't see any major issues as there is plenty of clearance w.r.t. the viewport when the beam exits the vacuum.
  5. With the PRM aligned, I can see the REFL beam on the CCD, and it doesn't look clipped.
  6. I didn't bother to align the green beams to the arm cavities or re-center the Oplevs - is this necessary? It is a step in the pre-close up checklist, so maybe we should do it... The green transmission does reach the PSL table...

Tomorrow, we should do some visual checks of the chambers / EQ stops on ETMY etc but I don't see any major problems at the moment...


Barring any catastrophic failures and provided all required personnel are available, we will do the final pre-close-up checks, put the heavy doors back on, and pump down starting 10 am Monday, 9 Nov 2020.

Attachment 1: IFOalignment.png
  15666   Mon Nov 9 17:40:17 2020 gautamUpdateGeneralPumpdown

[koji, rana, gautam]

1100 - EY chamber inspected, no issues were found --> EY heavy door on

1200 - OMC chamber was inspected. OM6 was marginally tweaked to bring the beam down a little in pitch, and also a little northwards in Yaw. --> Heavy door on.

1230 - Pumpdown started. Initially, the annuli volume was pumped down. The procedure calls for doing this with the small turbopumps. However, V7 was left open, and hence, in the process, the TP1 foreline pressure (=P2) hit ~30 torr. This caused TP1 to shutdown. We were able to restart it without issue. This case was not caught by the interlock code, which was running at the time. It should be recitified.

1330 - OMC breadboard clean optics and DCPD hardware were wrapped up and packed into tupperware boxes and stored along the south arm. OMC cavity itself, the OMMT, and the breadboard the OMC was sitting on are wrapped in foil/Ameristat and stored in cabinet S13, lower 2 shelves.

1915 - P1a = 0.5 torr pressure reached. Switched over to pumping the main volume with TP1, backed by TP2 and TP3, which themselves are backed by their respective dry pumps and also the AUX drypump for some extra oomph. All cooling fans available in the area were turned on and directed at the turbo pumps. RV2 was used to throttle the flow suitably.

It was at this point that we hit a snag - RV2 has gotten stuck in a partially open position, see Attachment #1. We can see that the thread doesn't move in response to turning the rotary dial. Fortunately, the valve is partially open, so the main volume continues to be pumped - see Attachment #2 for the full history of today's pumping. We are leaving the main volume pumped in this configuration overnight (TP1 pumping main volume backed by TPs 2 and 3, which are in turn backed by their respective drypumps and also the AUX dry pump). I think there is little to no risk of any damage to the turbo pumps, the interlocks should catch any anomalies. The roughing pumps RP1 and RP3 were turned off and that line was disconnected and capped.

What are our options?

  1. The main volume is able to reach the "nominal" pressure of 1e-5 torr, just takes longer.
  2. At some point, we may be able to pump the main volume directly with TP2 and TP3 - it is unclear at this point whether it's better to have the conductance limited TP1, which has a higher pumping capacity, or have the smaller TPs 2 and 3 pump through a larger conductance. 
  3. Depending on how low the ultimate pressure gets, we may be able to run the usual IFO activities until the replacement pressure gauges arrive in ~1 week, at which point we can vent the pumpspool (leaving the main volume isolated) and either repair this valve or replace it with one of the spares we have.

We need some vacuum experts to comment. Why did this happen? Is this an acceptable failure mode of the valve?

KA Ed:
2230 - P1a = 0.025 torr. The pressure is coming down with log-linear scale. x0.1 per 2.5 hours or so.

Attachment 1: IMG_8937.jpg
Attachment 2: pumpdown.png
  15668   Tue Nov 10 11:59:37 2020 gautamUpdateVACStuck RV2

I've uploaded some more photos here. I believe the problem is a worn out thread where the main rotary handle attaches to the shaft that operates the valve.

This morning, I changed the valve config such that TP2 backs TP1 and that combo continues to pump on the main volume through the partially open RV2. TP3 was reconfigured to pump the annuli - initially, I backed it with the AUX drypump but since the load has decreased now, I am turning the AUX drypump off. At some point, if we want to try it, we can try pumping the main volume via the RGA line using TP2/TP3 and see if that allows us to get to a lower pressure, but for now, I think this is a suitable configuration to continue the IFO work.

There was a suggestion at the meeting that the saturation of the main volume pressure at 1mtorr could be due to a leak - to test, I closed V1 for ~5 hours and saw the pressure increased by 1.5 mtorr, which is in line with our estimates from the past. So I think we can discount that possibility.

Attachment 1: damagedThread.001.jpeg
Attachment 2: IFOstatus.png
Attachment 3: P1a_leakTest.png
ELOG V3.1.3-