Based on the observation of the PMC error signal, I started measuring the IMC OLTF. Immediately, it was found that the overall IMC loop gain was too low.
The UGF was ~40kHz, which was really marginal. It had been >100kHz when I have adjusted it about a year ago. (Next entry for the detail)
The first obvious thing was that the SMA cables around the IMC servo have visible degradation (Attatched photos).
I jiggled the signal cable from the demodulator Q_out to the MC servo. The openloop gain seemed fluctuating (increased) based on the cabling.
I decided to repair these cables by adding solder on the shield.
Even after the repair, the open loop TF didn't show any improvement. I checked the LO level and found that it was -16.7dBm.
I traced the problem down to the frequency generator unit (T1000461). The front panel of the unit indicates the output power for the 29.5MHz output is 13dBm,
while measurement showed it was 6~8dBm (fluctuating). The T1000461 document describes that there is only a wenzel oscillator inside. Does this mean the oscillatorwas degraded??? We need to open the box.
I was not sure what was the LO level. I naively assumed the input is 0dBm. Reducing the attenuation of the dial on the AM Stabilizer unit from 12dB attn to 0dB.
This made the LO level -3.3dBm.
Later at home, I thought this nominal LO level of 0dBm could have been wrong.
The demodulator circuit (D990511) has the amplifier ERA-5 (G=~20dB) at the input. Between the input and the ERA-5 there is a pattern for an attenuator.
Assuming we have no attenuator, the ERA-5 has to spit out 20dBm. That is too much for this chip. I need to pull out the box to see how much is the nominal LO for this box using an active probe.
This decrease/increase of the LO level affects the WFS demod too. According to D980233-B, the input stage has the comparator chip AD96687, which can handle differential voltage of 5.5V.
Therefore the effect is minimal.
On the demod board there is a 10 dB attenuator (AT1), which lowers the level to -10 dBm before the ERA-5. Then it should be 10 dBm before going to the rest of the parts. But I guess the ERA-5 chips which come later on in the circuit could be decaying like the ones in the PMC LO board.
Done (Nov 23, 2015) - Check if the attenuator is still there in the input chain
Done (Nov 23, 2015) - Check if the actual LO levels at the 17dBm mixers are reasonable.
- Check if the actual LO levels for the LSC demods are OK too
A tmux session named "MCFreeSwingTest" will run on Rossa. This session is running script scripts/SUS/freeSwingMC.py (also attached) which will trigger at 5:00 am to impart 30000 counts kick to MC1, MC2, and MC3 after shutting PSL shutter and disabling the MC autolocker. It will let them freely swing for 1050 sec and will repeat 15 times to allow some averaging. In the end, it will undo all the changes it does and switches on autolocker on IMC. The script is set to restore any changes in case it fails at any point or a Ctrl-C is detected.
# Initial State
- MC is locked. The PRM monitor shows some oscillations.
- POP monitor shows light flashing once in a while.
- AS monitor shows one beam along with some other flashing beam around it.
- PRM Watchdog is tripped and shutdown. Everything else is normal except for overload on SRM OpLevs.
- Donatella got a mouse promotion
# Reenabling PRM watchdog:
- The custom reEnablePRMWatchdog.py has been deleted.
- Tried enabling the coil outputs manually and switching watchdog to Normal.
- Again saw large fluctuations like yesterday.
- Probably still the same issue of how current calculated actuations to the coils is in range -600 to -900 and gives and impulse to the optics when suddenly turned on.
- Waiting for PRM to damp down a little.
- Today we plan to change the position bias on PRM C1:SUS-PRM_POS_OFFSET instead of changing biases in pitch and yaw.
- Changing C1:SUS-PRM_POS_OFFSET from 0 to +/- 100 without enabling the coils, it seems upper and lower coils are anticorrelated with just changing the position. So going back to changing pitch.
- Changing C1:SUS-PRM_PIT_OFFSET from 0 -> 780. Switched on watchdog to normal.
- PRM damped down. OpLev errors are also within range.
- Enabled both OpLevs.
# Try locking Y-Arm
- IFO>CONFIGURE>YARM>Restore YARM (POY) using Donatella. See a bunch of python error messages in the call complaining about unable to find some python 2 files. Closed it with Ctrl-C after a stuck state.
- Tried running it on Pianosa, the script ran without error but Y-Arm didn't lock.
# Try locking X-Arm
- IFO>CONFIGURE>XARM>Restore XARM (POX) on Donatella. Again a bunch of OSError messages. Donatella is not configured properly to run scripts.
- Tried running it on Piasnosa, the script ran without error but X-Arm didn't lock.
- This might mean that both arms are misaligned or the BS/PRM is misaligned.
- Moving around C1:SUS-PRM_PIT_OFFSET and C1:SUS-PRM_YAW_OFFSET in order to see if the transmitted light is misalgined. Both arms are set to acquire lock if possible. No luck.
# Hypothesis: The Arm cavity is not aligned within itself (ITM-ETM)
- Will try to lock X-Arm with green light while tuning the ETMX. Hopefully the BS and ITM are aligned so that once we align ETMX to get a green lock, the IR will also lock from the other side.
- Running IFO>CONFIGURE>XARM>Restore XARM (ALS) on Pianosa. No lock, moving forward with tunning ETMX pitch and yaw offsets. Nothing changed. Brought back to same values.
[Rana joined, Anchal moved to Rossa from Pianosa]
# Moving on to IMC suspensions characterization:
- Closed the PSL shutter, to our suprise, the MC was still locked. We thought this would take away any light from IMC but it doesn't. Maybe the IFO Overview needs to show the schematic in a way where this doesn't happen: "No light from any laser entering the MC but it still is locked with a resonating field inside."
- Shutting IMCR shutter (hoping that would unlock the IMC), still nothing happend.
- Tried shutting PSL shutter from Rossa, nothing happened to MC lock still.
- Closed shutter IOO>Lock MC> Close PSL and this unlocked the IMC. Found out that this shutter channel is C1:PSL-PSL_ShutterRqst while the one from the sitemap>Shutter>PSL changes C1:AUX-PSL_ShutterRqst. Some clarification on these medm screens would be nice.
- Disabled the MC autolocked from IOO>Lock MC screen (C1:IOO-MC_LOCK_ENABLE).
- Checked the scripts/SUS/freeswing.py to understand how kick is delivered and optic is left to swing freely.
- Next, we are looking at the C1SUS_MC1 screen to understand what channels to read during data acquisition.
- In sensor matrix, we see INMON for each sensor which is probably raw counts data from the OSEMs. Rana mentioned that OSEM data comes out in units of microns. These are C1:SUS-MC1_ULSEN_OUTPUT (and so on for UR, LL, LR, SD).
- In prep for finishing, recovered Autolocker by first opening the PSL mechanical shutter, then re-enabling the Autolocker. The IMC lock didn't immediately recover, and we saw some fuzz on the PSL-FSS_FAST trace, so we closed the shutter again, waited a minute, then re-opened it and MC caught its lock.
Did you fix this issue? It is helpful to post a screenshot of the offending MEDM screen in addition to witticisms. The elog says "sitemap>Shutter>PSL" but I can't find PSL under the dropdown for shutters from Sitemap.
# Moving on to IMC suspensions characterization:
- Closed the PSL shutter, to our suprise, the MC was still locked. We thought this would take away any light from IMC but it doesn't. Maybe the IFO Overview needs to show the schematic in a way where this doesn't happen: "No light from any laser entering the MC but it still is locked with a resonating field inside."
No we didn't fix the issue. We'll post some screenshots tomorrow. From "sitemap>Shutter>PSL" we meant in Shutter medm window, we clicked on the PSL close button. As pointed later, it switches C1:AUX-PSL_ShutterRqst while the PSL shutter switch on Lock MC medm screen switches C1:PSL-PSL_ShutterRqst. We were not sure if this was intentional, so we didn't change anything.
I lowered the (FAST) PZT gain on the IMC/FSS servo today.
I noticed that the MC locks looked unstable a lot of the day, and during lock the PCDRIVE channel is above 1 Vrms (which means the loop is oscillating, ttypically at the PZT/EOM crossover frequency).
I changed the default setting from 22 to 20 19 dB in the PSL Settings screen so the mcup script will use this for now. Feel free to revert if this turns out to be a Fluke (which you would think is a terrible name for a company, but...)
Attachment #1 - In loop error spectra, measured as Koji posted end of last week.
Attachment #2 - OLTF of the IMC loop.
Attachment #3 - Photo courtesy Koji showing the bank of BNC connectors used for these measurements.
Clearly, these measurements were taken in a time when the IMC was "well behaved". How to characterize what's happening when this isn't the case?
Last night, Valera and I looked into two aspects of the IMC:
I will post a more detailed elog about last night's work, but Valera also thought it might be a good idea to try and improve the mode-matching into the IMC. I couldn't find anything on the wiki/elog about the mode matching situation on the PSL table, so I quickly went over yesterday to measure some lengths. From looking at the MCREFL DC levels when the mode cleaner is locked (~0.37V) and unlocked (~5.7V), the current mode matching efficiency seems to be about 88%, so there is definitely some headroom for improvement.
Here is my cartoon of the situation on the PSL table. All lengths are measured in mm, and I would say correct to +/- 5 mm, so there could be considerable error here...
(L1 : f=+200mm. L2: f=-150mm. L3: f=+400mm)
I extracted the lengths from the edge of the PSL table to IM1 and MC1 from (what I think are) the latest CAD drawings on the DCC. I then put all this into an a la mode script [Attachment #5] - I assumed a waist of 370um at the PMC output mirror, and a waist of 1.78mm at MC1. I neglected the passage through the in-vac Faraday, EOM and BS1 (on the sketch above) and the MC1 substrate. I was able to achieve a theoretical mode-matching efficiency of 1 by just moving the positions of L2 and L3.
Given that there are probably errors of the order 0.5cm in the lengths on the PSL table, and also the in-vacuum distance to MC1, I figured it would be ideal to just move one lens and see if we can improve the efficiency. It looks like it may be more effective to move L2 than L3. The plot on the right shows that the sensitivity is approximately equal to the positioning of L2 and L3. Judging by this plot, looks like w.r.t. the coordinates in this plot, we are somewhere around (0.02,-0.02).
It looks like if we want to do this, moving L2 (f = -150mm) may be the best way to go.
%Create a beamPath object
InpPath = beamPath;
%Add components - for a first pass, ignore Faraday and HWPs, so only
%mirrors and lenses..
We implemented the plan outlined in the previous elog. The visibility (Pmax-Pmin)/(Pmax+Pmin) calculated with the MC REFL PD levels with the MC locked/unlocked is now ~96% (up from 88%). The MC REFL DC level in lock is now ~0.12V (compared to 0.4V). Assuming a modulation depth of 0.1 @ 29.5MHz, about 25% of this (i.e. 0.03V) is from sideband light.
The procedure followed was (see sketch in previous elog for various optic labels):
We could probably tweak the fine positioning of L2 and L3 and improve the efficiency a little more, but the primary objective here was to see if there was any effect on the large common mode offset on the WFS demodulated "SUM" output. Unfortunately, we saw no effect.
Here are two photos of the relevant section of the PSL table before (left) and after (right) our work there:
We have increased the output power from the PSL table to 951 mW (it was 96.7 mW).
IMC was recovered including WFS, and both arms are flashing nicely in IR.
We tweaked the alignment of GRX and GRY injection to align them with IR, but it was hard.
Right now IR beams are not centered on TMs. We should center them first.
What we did:
Power increase and IMC recovery
- Replaced a beam splitter which splits the beam into IMC REFL RF PD path and WFS path from R=98% to R=10% one. Reflection goes to RF PD.
- Put a R=98% beam splitter back into WFS path.
- We also tried to put a window in front of IMC REFL camera to recover the arrangement in 40m wiki, but the beam reflected from the window was too weak for us to align. So, we decided not to place a window in front of the camera.
- Attached photos are the IMC REFL path before and after the work.
- Measured the PSL output power as Koji did in elog 40m/16672. It was measured to be 96.7+/- 0.5 mW.
- Rotated the HWP using the Universal Motion Controller (it was not possible for us to do it from the MEDM screen). The position was changed from 73.99 deg to 36.99 deg. Output power was measured to be 951 +/- 1 mW
- IMC locked without any other changes.
- Changed C1:IOO-WFS_TRIGGER_THRESH_ON to 5000 (was 500). IMC WFS also worked.
- After running MC WFS relief script, WFS DC offsets and RF offsets are adjusted following the steps in elog 40m/16835. Below are the results.
C1:IOO-WFS1_SEG1_DC.AOFF => -0.0008882080010759334
C1:IOO-WFS1_SEG2_DC.AOFF => -0.0006527877490346629
C1:IOO-WFS1_SEG3_DC.AOFF => -0.0005847311617496113
C1:IOO-WFS1_SEG4_DC.AOFF => -0.0010395992663688955
C1:IOO-WFS2_SEG1_DC.AOFF => -0.0025944841559976334
C1:IOO-WFS2_SEG2_DC.AOFF => -0.003191715502180159
C1:IOO-WFS2_SEG3_DC.AOFF => -0.0036688060499727726
C1:IOO-WFS2_SEG4_DC.AOFF => -0.004011172490815322
IOO-WFS1_I1 : +1977.7 -> +2250 (Significant change)
IOO-WFS1_I2 : +3785.8 -> +3973.2
IOO-WFS1_I3 : +2014.2 -> +2277.7 (Significant change)
IOO-WFS1_I4 : -208.83 -> +430.96 (Significant change)
IOO-WFS1_Q1 : +2379.5 -> +1517.4 (Significant change)
IOO-WFS1_Q2 : +2260.4 -> +2172.6
IOO-WFS1_Q3 : +588.86 -> +978.98 (Significant change)
IOO-WFS1_Q4 : +1654.8 -> +195.38 (Significant change)
IOO-WFS2_I1 : -1619.9 -> -534.25 (Significant change)
IOO-WFS2_I2 : +1610.4 -> +1619.8
IOO-WFS2_I3 : +1919.6 -> +2179.8 (Significant change)
IOO-WFS2_I4 : +1557 -> +1426.6
IOO-WFS2_Q1 : -62.58 -> +345.56 (Significant change)
IOO-WFS2_Q2 : +777.01 -> +805.41
IOO-WFS2_Q3 : -6183.6 -> -5365.8 (Significant change)
IOO-WFS2_Q4 : +4457.2 -> +4397.
- Aligned both arms using IR. Both arm flashes at the following, which is consistent with the power increase.
- With this, we tried to tweak GRX and GRY injection. The following is after the work. We could increase GTRX to 0.204 when the Xarm is aligned to green. This suggests that GRX injection is not aligned nicely yet. But the beams are also not centered on TMs. We should center them first.
- GTRX and GTRY cameras are adjusted to have nicer images. In GRX path, the second and last lens before the PD and CCD was pulled ~ 1 cm behind its original position and both beams realigned. Then, on GRY path, the beam was re-centered on the first and only lens, the whole assembly pushed forward by ~ 2 cm and the beams re-centered.
- Center the IR beam on TMs (first by our eyeballs; better to use A2L after arm locking is recovered and coils are balanced)
- Tweak GRX and GRY injection (restore GRY PZTs?)
- Install ETMXT camera (if it is easy)
- Lock Xarm and Yarm (C1:LSC-TRX/Y_OUT needs to be fixed for triggering. Can we use other PDs for triggering?)
- MICH locking (REFL and AS PDs might need to be re-aligned; they are not receiving much light)
- RTS model for BHD needs to be updated
After typing up the elog, I decided to try locking the IMC again - now it locks again with the "OLD" gain settings. I tested it ~5 times, the autolocker brings the lock back and the PC drive levels are normal. IMC transmission and MC REFL DC light levels in lock are normal. The PC Drive RMS voltage is <1V. What's more, there is no longer any evidence of 60 Hz line harmonics any more in the PMC diagnostics channels. Compare attachment #1 to this elog.
I undid the changes Koji made to the autolocker gains, and am trying the old settings again. Let' see how stable or otherwise the config is. I must've jiggled some poor cable connection back into a good spot while working on the PSL table?
Anyway, this helps Kruthi and Milind.
Note that I have removed an SR785, an oscilloscope, some SRS instruments from the PSL and PMC last night.
But they (and RF Network Analyzer) were not there when the problem started.
We should record the IMC error (at test point monitor) too? If the IMC locks on Monday too, I'll do it.
Over the last 24 hours, the IMC autolocker was able to keep the MC locked ~60% of the time. This is not particularly good, but is an improvement on ~2 weeks ago when the IMC couldn't be locked.
There are two periods, which I've indicated by vertical cursors, between which the autolocker was doing something strange - usually this kind of trend is caused by one or more of the VME crates being unresponsive and the autolocker gets stuck, but I confirmed that both c1psl and c1iool0 are telnet-able. So I conclude that the stability and reliability of the IMC loop is still not as good as it used to be.
Note also that while the PC drive RMS level mostly hovers around 1 V, there are several excursions above that level. This in itself isn't a new phenomenon. I will do some more characterization by measuring the in-loop error signal spectrum and maybe the OLTF of the IMC locking loop.
Let' see how stable or otherwise the config is. I must've jiggled some poor cable connection back into a good spot while working on the PSL table? Or the NPRO decided to be less noisy on Sunday.
I was a little confused why the In1 Gain had to be as high as +10dB - before the changes to the RF chain, we were using +27dB, and we expect the changes made to have increased the modulation depth by a factor of ~25, so I would have expected the new In1 Gain to be more like 0dB.
While walking by the PSL table, I chanced upon the scope monitoring PMC transmission, and I noticed that the RIN was unusually high (see the scope screenshot below). We don't have the projector on the wall anymore, but it doesn't look like this has shown up in the SLOW monitor channel anyways. Disabling the MC autolocker / closing the PSL shutter had no effect. I walked over to the amplifier setup in 1X2, and noticed that the SMA cable connecting the output of the amplifier to the EOM drive was flaky. By touching the cable a little, I noticed that the trace on the scope appeared normal again. Turning off the 29.5MHz modulation source completely returned the trace to normal.
So I just made a new cable of similar length (with the double heat shrink prescription). The PMC transmission looks normal on the scope now. I also re-aligned the PMC for good measure. So presumably, we were not driving the EOM with the full +27dBm of available power. Now, the In1 Gain on the MC servo board is set to +2dB, and I changed the nominal FSS FAST gain to +18dB. The IMC OLTF now has a UGF of ~165kHz, though the phase margin is only ~27 degrees..
MC Servo Board
29.5 MHz RF Modulation Source
IMC Demodulation Board
I wanted to do a quick check to see if the observed signal levels were in agreement with tests done on the workbench with the Marconi. The mixers used, JMS-1H, have an advertised conversion loss of ~7dB (may be a little higher if we are not driving the LO at +17dBm). The Lissajous ellipse above is consistent with these values. I didn't measure powers with the MC REFL PD plugged into the demod board, but the time series plot above suggest that I should have ~0dBm power in the MC REFL PD signal at 29.5MHz for the strongest flashes (~0.3Vpp IF signal for the strong flashes).
MC Servo Board
Some general remarks
I would think that we want to fix the I/Q orthog inside the demod board by trimming the splitter. Mixing the Q phase signal to the I would otherwise allow coupling of low frequency Q phase junk from HOMs into the MC lock point.
Of course this doesn't matter for the IMC locking as we only use the I phase signal, but
Motivation: see this elog
I was fiddling around for a few days trying to implement the method outlined in this paper to null this offset - I will post a separate elog about my efforts but Valera pointed out that we could try injecting an AF modulation at the IN2 input of the MC Servo Board. Last night, we hooked up an SR function generator (f = 312Hz, A = 0.01Vpp, IN2 gain = -5dB) to the unused BNC IN2 input of the MC Servo board. To avoid any additional offsets from the AO path during this measurement, I disconnected the LEMO cable (it is labelled).
We looked at the spectrum of the MC transmission around 312Hz and also 2*f = 624Hz. As a result of this modulation, we expect in the transmitted power, dP/P, a 2f term with amplitude ~(X_mod/X_0)^2 and a term at f with amplitude ~(X_offset * X_mod / X_0^2) - I may have missed out some numerical factors of order 1. So the latter should vanish if the offset at the error point is truly zero and the lock-point is the center of the resonance. Last night, we found that an offset in the range of -0.25 V to -0.19 V nulled this peak in the DTT spectrum. Today, the number was -0.05V. So the true offset seems to vary from lock to lock. Here are spectra around f=312Hz for a few different values of the offset slider (the center of the resonance seems to be -0.05V on the MEDM slider at this time).
Do these numbers make sense? Some time ago, I had pulled out the MC Servo board to find out what exactly is going on at this offset summing point. The MEDM slider goes from -10V to 10V, and by measuring the voltage at TP5 (see schematic below), I found that there is a 1/40 scaling factor between what is actually applied and the number on the MEDM slider (so for example, the numbers in the legend in the above plot have to be divided by 40). I've modified the MC Servo Board MEDM screen to reflect this. When I had pulled the board out, I noticed that in addition to the offset voltage applied via the backplane connector, there was also a potentiometer (R50 in the schematic below). I had nulled the voltage at TP5 using this potentiometer, but I guess drifts of ~5mV are possible.
Discussion on calibration of offset slider in Hz/V:
I've yet to do a rigorous calibration of this slider into Hz, but looking at the spectrum of the transmitted intensity at 2f, we estimated the coefficient (X_mod/X_0) ~ 3e-3 for an offset of 0.2V. dP/P ~1 when the applied modulation equals the linewidth of the cavity, which is 3.6kHz. So 0.2V of offset slider corresponds to ~ 10Hz frequency offset. In other words, I estimate the slider calibration to be 50Hz/V. So with the full range of +/- 10V, we should be able to scan ~1kHz of frequency offset. What does this imply about the variation of the offset slider value that removes the peak at 1f between locks? As mentioned above, this variation is ~0.2V over a day - with the calibration mentioned above, this corresponds to a change in cavity length of ~10um, which seems reasonable to me...
So how did all of this tie in with WFS SUM offsets? We did the following:
I neglected to screenshot the StripTool from the times we were doing these trials but I have the times, I will pull up some dataviewer plots and upload them here tomorrow...
Now that all the front end models are running, I re-aligned the IMC, locked it manually, and then tweaked the alignment some more. The IMC transmission now is hovering around 15300 counts. I re-enabled the Autolocker and FSS Slow loops on Megatron as well.
Today I got the mx/open-mx networking working for the front ends. This required some tweaking to the network interface configuration for the diskless front ends, and recompiling mx and open-mx for the newer kernel. Again, this will all be documented.
IMC is locking now. There was nothing wrong: just a careful alignment + proper gain adj
=== Primary Alignment ===
- I used WFS error signals as the indicator of the PDH error signals. Checked C1:IOO-WFS1_(I/Q)n_ERR and ended up C1:IOO-WFS1_I4_ERR as it showed the largest PDH error PP.
- Then used MC2 and MC3 to align the IMC by maximizing the PDH error and the MC trans (C1:IOO-MC_TRANS_SUM_ERR)
=== Locking procedure ===
Note that the MC REFL path is still configured for the full power input
- (Only at the beginning) Run scripts/MC/mcdown for initialization / Run scripts/MC/MC2tickleOFF just in case
- Enable IOO-MC-SW1 (MC SERVO switch right after "IN1 Gain (dB)").
- Disable 40:4000 boost
- Increase VCO Gain from -15 to 0
- Jiggle IN1 Gain from low to +31 until the lock is achieved
- As soon as the lock is acquired, enable 40:4000
- Increase VCO Gain to +10
- Turn up "SUPER BOOST" from 0 to 3
=== Lock loss procedure ===
- Disable IOO-MC-SW1
- Disable 40:4000 boost
- Reduce VCO Gain 0
- Turn down "SUPER BOOST" to 0
- Then jiggle IN1 Gain again to lock the IMC
=== MC2 spot ===
- It was obvious that the MC2F spot was not on the center of the optic.
- I tried to move the spot on the camera as much as possible, but this did not make the trans beam to the center of the MC end QPD
- I had the impression that the trans beam started to be clipped when the beam was moved towards the end QPD,
We need to reestablish the reasonable/consistent MC2 spot on the mirror, the MC end optics, and the QPD.
We will need to use MC2 dithering and A2L coupling to determine the center of the mirror
But as long as the transmission is maximized, the transmitted beam thru MC1 and MC3 follows the input beam. So we can continue the vent work
The current maximized transmission was ~1300. MC1 refl CCD view was largely off -> The camera path was adjusted.
=== MC2 alignment note ===
During the alignment, I noticed a sudden change of the MC2 alignment. There might be some hysteresis in the MC2 suspension. If you are locking the IMC and noticed significant misalignment, the first thing to try is to touch MC2 alignment.
See trend. This is NOT symptomatic of some frozen slow machine - if I disable the WFS servo inputs, the lock holds just fine.
Turns out that the beam was almost completely missing the WFS2 QPD. WTF ðŸ˜¤. I re-aligned the beam using the steering mirror immediately before the WFS2 QPD, and re-set the dark offsets for good measure. Now the IMC remains stably locked.
Please - after you work on the interferometer, return it to the state it was in. Locking is hard enough without me having to hunt down randomly misaligned/blocked beams or unplugged cables.
I took this opportunity to do some WFS offset updates.
I think this offset setting thing is not so good. People do this every few years, but putting offsets in servos means that you cannot maintain a stable alignment when there are changes in the laser power, PMC trans, etc. The better thing is to do the centering of the WFS spots with the unlcoked beam after the control offsets have been offloaded to the suspensions.
To test the hypothesis that the IMC lock duty cycle is affected by the PRM alignment. Rana pointed out today that the input faraday has not been tuned to maximize the output->input isolation in a while, so the idea is that perhaps when the PRM is aligned, some of the reflected light comes back towards the PSL through the Faraday and hence, messes with the IMC lock.
I've made a simple script - the pseudocode is the following:
The idea is to keep looping the above over the weekend, so we can expect ~100 datapoints, 50 each for PRM misaligned/aligned. The times at which PRM was aligned/misaligned is also being logged, so we can make some spectrograms of PC drive RMS (for example) with PRM aligned/misaligned. The script lives at /opt/rtcds/caltech/c1/scripts/SUS/FaradayIsolationTest/FaradayIsolCheck.py. Script is being run inside a tmux session on pianosa, hopefully the machine doesn't crash over the weekend and MC1/CDS stays happy.
A more direct measurement of the input Faraday isolation can be made by putting a photodiode in place of the beam dump shown in Attachment #1 (borrowed from this elog). I measured ~100uW of power leaking through this mirror with the PRM misaligned (but IMC locked). I'm not sure what kind of SNR we can expect for a DC measurement, but if we have a chopper handy, we could put a chopper (in the leaked beam just before the PD so as to allow the IMC to be locked) and demodulate at that frequency for a cleaner measurement? This way, we could also measure the contribution from prompt reflections (up to the input side of the Faraday) by simply blocking the beam going into the vacuum. The window itself is wedged so that shouldn't be a big contributor.
I stopped the test earlier today morning around 11:30am. The log file is located at /opt/rtcds/caltech/c1/scripts/SUS/FaradayIsolationTest/PRM_stepping.txt. It contains the times at which the PRM was aligned/misaligned for lookback, and also the number of MC unlocks during every 30 minute period that the PRM alignment was toggled. This was computed by:
I think this method is a pretty reliable proxy, because the MC autolocker certainly takes >3 seconds to re-acquire the lock (it has to run mcdown, wait for the next cavity flash, and run mcup in the meantime).
Preliminary analysis suggests no obvious correlation between MC lock duty cycle and PRM alignment.
I leave further analysis to those who are well versed in the science/art of PRM/IMC statistical correlations.
There seems to be significant phase loss in the TTFSS path, which is limiting the IMC OLTF to <100 kHz.
See Attachment #1 and #2. The former shows the phase loss, while the latter is just to confirm that the optical gain of the error point is roughly the same, since I noticed this after working on and replacing the RF frequency distribution unit. Unfortunately there have been many other changes also (e.g. the work that Rana and Koji did at the IMC rack, swapping of backplane controls etc etc - maybe they have an OLTF measurement from the time they were working?) so I don't know which is to blame. Off the top of my head, I don't see how the RF source can change the phase lag of the IMC servo at 100 kHz. The only part of the IMC RF chain that I touched was the short cable inside the unit that routes the output of the Wenzel source to the front panel SMA feedthrough. I confirmed with a power meter that the power level of the 29.5 MHz signal at that point is the same before and after my work.
The time domain demod monitor point signals appear somewhat noisier in todays measurement compared to some old data I had from 2018, but I think this isn't significant. Once the SR785 becomes available, I will measure the error point spectrum as well to confirm. One thing I noticed was that like many of our 1U/2U chassis units, the feedthrough returns are shorted to the chassis on the RF source box (and hence presumably also to the rack). The design doc for this box makes many statements about the precautions taken to avoid this, but stops short of saying if the desired behavior was realized, and I can't find anything about it in the elog. Can someone confirm that the shields of all the connectors on the box were ever properly isolated? My suspicion is that the shorting is happening where the all-metal N-feedthroughs touch the drilled surfaces on the front panel - while the front and back surfaces of the panel are insulating, the machined surfaces are not.
This is an unacceptable state but no clear ideas of how to troubleshoot quickly (without going piece by piece into the IMC servo chain) occur to me. I still don't understand how the freq source work could have resulted in this problem but I'm probably overlooking something basic. I'm also wondering why the differential receiving at the TTFSS error point did not require a gain adjustment of the IMC servo? Shouldn't the differential-receiving-single-ended-sending have resulted in an overall x0.5 gain?
Update 8 Dec 1200: To test the hypothesis, I bypassed the SR560 based differential receiving and restored the original config. I am then able to run with the original gain settings, and you see in Attachment #4 that the IMC OLTF UGF is back above 100 kHz. It is still a little lower than it was in June 2019, not sure why. There must be some saturation issues somewhere in the signal chain because I cannot preserve the differential receiving and retain 100 kHz UGF, either by raising the "VCO gain" on the MC servo board, setting the SR560 to G=2, or raising the "Common Gain Adjust" on the FSS box by 6 dB. I don't have a good explanation for why this worked for some weeks and failed now - maybe some issue with the SR560? We don't have many working units so I didn't try switching it.
So either there is a whole mess of lines or the frequency noise suppression is limited. Sigh.
It is not clear to me why installing an attenuator to prevent amplifier saturation has necessitated a 10dB increase in the IN1 gain and 3dB increase in the VCO gain. Initially, I was trying to compensate for the gain by increasing the FSS "Common Gain" but in that setting, I found an OLTF measurement impossible. The moment I enabled the excitation input to the CM board, the lock was blown, even with excitation amplitudes as small as -60dBm (from the Agilent network analyzer).
This may also be a good opportunity to test out one of the aLIGO style FET mixer demod boards (recall we have 2 spare from the 4 that were inside the ALS demod box). I'm going to ask Steve to package these into a 1U chassis so that I can try that setup out sometime. From a noise point of view, the aLIGO boards have the advantage of having a x100 preamp stage straight after the mixer+LPF. We may need to replace the lowpass filter though, I'm not sure if the one installed is 1.9MHz or 5MHz.
I've left an SR785 and AG4395 near 1X2 in anticipation of continuing this work tomorrow.
Unrelated to this work - seems like the WFS DC and RF offsets had not been set in a while so I reset these yesterday. The frequent model restarts in recent times may mean that we have to reset these to avoid using dated offset values.
Re-measured the demod board noises after replacing the suspect ERA-5SMs, with LO driven by a marconi at the "nominal" level of 2.5dBm, and RF input terminated. Attachment #1 is the input referred voltage noise spectra. I used the FET low noise pre-amp box for this purpose. I cannot explain the shape of the spectra above 1kHz. I tried doing the measurement on a minicircuits mixer (non-surface mount) and found the shape to be flat throughout the SR785 span. Unclear what else could be going on in the demod board though, all the other components on it are passive (except the ERA-5SMs which were replaced). I considered adopting a PMC style demod setup where we do the demod using some separate Minicircuits Mixer+LowPass filter combo. But the RF flashes for the IMC monitored at the RFmon port are ~0.2Vpp, and so the RF input to the mixer is expected to be ~2Vpp. The minicircuits mixer selection guide recommends choosing a diode mixer with LO level at least 10dBm above the expected RF input signal level, and we don't have any standalone mixers that are >Level 7. I've asked Steve to package the aLIGO demod board in the meantime, but even that might not be a plug and play replacement as the IF preamp stage has ~120degrees phase lag at 1MHz, which is significantly higher than the existing board which just has a SCLF5 low pass filter after the mixer and hence has <45degrees phase lag at 1MHz.
This elog by koji inspired me to consider power supply as a possible issue.
The demod board receives +/-24V DC (which is regulated down to +/-15V DC by 7815/7915), and also +15V DC via the backplane. The ERA-5SM receives DC power from the latter (unregulated) +15V DC. I can't think of why this is the case except perhaps the regulators can't source the current the amp wants? In any case, it doesn't look feasible to change this by cutting any traces on the PCB to me. While I had the board out, I decided to replace the JMS-1H mixers in a last ditch effort to improve the demod board noise. Unfortunately I'm having trouble de-soldering these MCL components from the board. So for now, I'm leaving the demod board out, IMC unlocked. Work will continue tomorrow.
After some persistence, I managed to get the mixers off.
Unfortunately, the coherent noise between the arms persists so the sensing noise injection must be happening elsewhere. IMC seems to lock fine though so I'm leving the autolocker on
I took some transfer functions of the IMC loop and crossover, being careful that the PC drive never exceeding 1V during the measurements.
I then did some algebra to try and back out the individual loop paths, without having to make assumptions/approximations about the loop gain being high enough. This only really works in the region where both the open loop and crossover measurements have coherence.
It seems to me that the PZT path has pretty low phase margin on its own, but maybe this is ok, since its never really meant to run solo. The EOM path shape is harder to understand.
The data I took, and code that made the above plot is attached. This afternoon, I'll post an update comparing the measured OLG and crossover to earlier measurements.
The promised historical comparisons follow. The crossover looks mostly the same as before. There is a new feature in the OLG at 50-60kHz; what could've changed about the EOM path in that time?
We want to measure the IMC round-trip loss using the Isogai et. al. ringdown technique. I spent some time looking at the various bits and pieces needed to make this measurement today, this elog is meant to be a summary of my thoughts.
Does this sound like a sensible plan? Or do I need to do any further checks?
We expect that the MC sus are susceptible to the temperature change and the alignment drifts away with time.
Here is the proper alignment procedure.
0) Assume there is no TEM00 flash or locking, but the IMC is still flashing with higher-order modes.
1) Use the CCD camera and WFS DC spots to bring the beam to the nominal position.
2) Use only MC2 and MC3 to align the cavity to have low-order modes (TEM00,01,02 etc)
3) You should be able to lock the cavity on one of these modes. Minimize the reflection (maximize the transmission) for that mode.
4) This should allow you to jump to a better lower-order mode. Continue alignment optimization only with MC2/3 until you get TEM00.
5) Optimize the TEM00 alignment only with MC2/3
6) Look at the MC end QPD. use one of the scripts in scripts/MC/moveMC2 . Note that the spot moves opposite to the name of the scripts. i.e. MC2_spot_down moves the spot up, MC2_spot_right moved the spot left, etc...
These scripts move MC1/2/3 and try to keep the good MC transmission.
7) moveMC2 scripts are not perfect. As you use them, it makes the MC alignment gradually degraded. Use MC2 and MC3 to recover good transmission.
8) If MC2 spot is satisfactory, you are done.
Step 6-8 can be done with the WFS on. This way, you can skip step 7 as the WFS servo takes care of it. But if the spot move is too fast, the servo can't keep up with the change. If so, you have to wait for the settling of the servo. Once the spot position is satisfactory, MC servo relief should be run so that the servo offset (in actuation) can be offloaded to the bias slider.
I've added two curves to the NB. Both are measured (with FET preamp) at the output of the demod board, with the LO driven at the nominal level by the Wenzel RF source pickoff (as it would be when the IMC is locked) and the RF input connected to the IMC REFL PD. For one curve, I simply closed the PSL shutter, while for the other, I left the PSL shutter open, but macroscopically misaligned MC2 so that there was no IMC cavity. So barring RFAM, there should be no PDH signal on the REFL PD, but I wanted to have light on there. I'm not sure if I understand the difference between these two curves though, need to think on it. Perhaps the IMC REFL PD's optical/electrical response needs to be characterized?
Next curve to go on here is the demod board noise with the PSL shutter closed but the IMC REFL PD connected to the RF input (or maybe even better, have light on the PD, but macroscopically misalign MC2 so there is no 29.5MHz PDH signal), just to make sure there isn't anything funky going on there...
Since ~ 2 hours ago, the IMC autolocker has not been able to keep the IMC locked. I don't see any obvious trends in the wall StripTool that may point to what's going on. For the brief periods in which a TEM00 mode is locked, the PC Drive RMS level is ~5x what the nominal level is, and while the autolocker is trying to lock the IMC, the PC drive RMS level is hovering around 4V DC, which is high. The PMC Error and Control signal spectra show huge 60 Hz (and harmonics) peaks, and indeed this is visible in the time domain signals as well (on ndscope or on the oscilloscope on the PSL table), but this is not a new feature in the last two hours. Usually, this kind of problem signals that either/both the c1psl or c1iool0 slow machines need to be power-cycled, but I confirmed that both machines are online and telnet-able. Possibilities: (i) some card in the c1psl / c1ioo crates have failed or (ii) something in the MC/FSS electronics chain has failed or (iii) there is a huge amount of excess high-frequency noise from the NPRO.
I am leaving the PSL shutter closed.
WFS loops were running for past 2 hours when I made the overall gain slider zero at:
PDT: 2022-10-18 20:42:53.505256 PDT
UTC: 2022-10-19 03:42:53.505256 UTC
The output values are fixed to a good alignment. IMC transmission is about 14100 counts right now. I'll turn on the loop tomorrow morning. Data from tonight can be used for monitoing open loop noise.
Turning WFS loops back on at:
PDT: 2022-10-19 09:48:16.956979 PDT
UTC: 2022-10-19 16:48:16.956979 UTC
Also reply to: 40m/16125
I migrated the code used in 40m/16125 to our scripts git repo and used it to apply offsets to IMC optics and noticing the parabolic change in the transmission values. Fitting the data with parabola and using the calculations mentioned in the previous post, we get following angular actuation calibration at DC from the PIT/YAW alignment output channels (cts) to actual motion in (urad):
*Note that in the previous post, the radius of curvature of MC2 used was wrong and has been corrected in this calculation to 17.87 m taken from Gautam's thesis Table A.1
Due to lack of time, we ran test faster on MC2, hence more uncertainty in it's results. Also, during MC1 YAW test, lock for breifly lost which required me to manually throw away some data points, but it did not affect the quality of fit much. Please see attached the data plots and fit.
For calibration at AC, another test needs to be performed which I did not do right now. 40m/16125 also describes how to do that, so someone can repeat that in future.
It would be good if someone can post here the actuation calibration in radians, so that we can have a physical calibration of the sensing matrix in counts/radian.
We came in this morning and noted the IMC was grossly misaligned, with MC3 still damped but with >= 100 rms motion in all coil monitors (a lot but not enough to trip the WD)... Turning off the WFS didn't do much so it was obviously an issue with the recent f2A output filters, so we turned all off (though only MC3 had this excess motion). After this we aligned IMC, engaged the lock and turned WFS back on.
There was no elog about f2A beyond this test scheduled to run Friday, I guess the filters were meant to stay on long term?
Two days ago I opened the PSL shutter by switching the switch on the shutter driver. That caused the shutter's switch on the medm screen to work in reversed mode: open meant closed and closed meant open.
I fixed that. Now the medm screen switch state is correct.
The IMC has been misbehaving for the last 5 hours. Why? I turned the WFS servos off. afaik, aaron was the last person to work on the IFO, so i'm not taking any further debugging steps so as to not disturb his setup.
That was likely me. I had recentered the beam on the PD I'm using for the armloss measurements, and I probably moved the wrong steering mirror. The transmission from MC2 is sent to a steering mirror that directs it to the MC2 transmission QPD; the transmission from this steering mirror I direct to the armloss MC QPD (the second is what I was trying to adjust).
Note: The MC2 trans QPD goes out to a cable that is labelled MC2 op lev. This confusion should be fixed.
I realigned the MC and recentered the beam on the QPD. Indeed the beam on MC2 QPD was up and left, and the lock was lost pretty quickly, possibly because the beam wasn't centered. Lock was unstable for a while, and I rebooted C1PSL once during this process because the slow machine was unresponsive.
When tweaking the alignment near MC2, take care not to bump the table, as this also chang es the MC2 alignment.
Once the MC was stably locked, I was able to maximize MC transmission at ~15,400 counts. I then centered the spot on the MC2 trans QPD, and transmission dropped to ~14800 counts. After tweaking the alignment again, it was recovered to ~15,000 counts. Gautam then engaged the WFS servo and the beam was centered on MC2 trans QPD, transmission level dropped to ~14,900.
Gautam was doing some DRMI locking, so I replaced the photodiode at the AS port to begin loss measurements again.
I increased the resolution on the scope by selecting Average (512) mode. I was a bit confused by this, since Yuki was correct that I had only 4 digits recorded over ethernet, which made me think this was an i/o setting. However the sample acquisition setting was the only thing I could find on the tektronix scope or in its manual about improving vertical resolution. This didn't change the saved file, but I found the more extensive programming manual for the scope, which confirms that using average mode does increase the resolution... from 9 to 14 bits! I'm not even getting that many.
There's another setting for DATa:WIDth, that is the number of bytes per data point transferred from the scope.
I tried using the *.25 scope instead, no better results. Changing the vertical resolution directly doesn't change this either. I've also tried changing most of the ethernet settings. I don't think it's something on the scripts side, because I'm using the same scripts that apparently generated the most recent of Johannes' and Yuki's files; I did look through for eg tds3014b.py, and didn't see the resolution explicitly set. Indeed, I get 7 bits of resolution as that function specifies, but most of them aren't filled by the scope. This makes me think the problem is on the scope settings.
I ran a BNC from the PD on the AS table along the cable rack to a free ADC channel on the LSC whitening board. I lay the BNC on top of the other cables in the rack, so as not to disturb anything. I also was careful not to touch the other cables on the LSC whitening board when I plugged in my BNC. The PD now reads out to... a mystery channel. The mystery channel goes then to c1lsc ADC0 channels 9-16 (since the BNC goes to input 8, it should be #16). To find the channel, I opened the c1lsc model and found that adc0 channel 15 (0-indexed in the model) goes to a terminator.
Rather than mess with the LSC model, Gautam freed up C1:ALS-BEATY_FINE_I, and I'm reading out the AS signal there.
I misaligned the x-arm then re-installed the AS PO PD, using the scope to center the beam then connecting it to the BNC to (first the mystery channel, then BEATY). I turned off all the lights.
I went to misalign the x-arms, but the some of the control channels are white boxed. The only working screen is on pianosa.
The noise on the AS signal is much larger than that on the MC trans signal, and the DC difference for misaligned vs locked states is much less than the RMS (spectrum attached); the coherence between MC trans and AS is low. However, after estimating that for ~30ppm the locked vs misaligned states should only be ~0.3-0.4% different, and double checking that we are well above ADC and dark noise (blocked the beam, took another spectrum) and not saturating the PD, these observations started to make more sense.
To make the measurement in cds, I also made the following changes to a copy opf Johannes' assess_armloss_refl.py that I placed in /opt/rtcds/caltech/c1/scripts/lossmap_scripts/armloss_cds/ :
I started taking a measurement, but quickly realized that the mode cleaner has been locked to a higher order mode for about an hour, so I spend some time moving the MC. It would repeatedly lock on the 00 mode, but the alignment must be bad because the transmission fluctuates between 300 and 1400, and the lock only lasts about 5 minutes.
Back to loss measurements.
I replaced the PD I've been using for the AS beam.
I misaligned the x arm.
I tried to lock the y arm, but PRC was locked so I could was unable. Gautam reminded me where the config scripts are.
The armloss measurement script needed two additional modifications:
I ran successfully the loss measurement script for the x and y arms. I'm getting losses of ~100ppm from the first estimates.
I made the following changes to the lossmap script:
When the optic aligns itself not at the ideal position, I'm noticing that it often locks on a 01. When the cavity is then misaligned and restored, it can no longer obtain lock. To fix this, I've moved my 'save' commands to just before the loop begins. This means the script may take longer to run, but as long as the cavity is initially locked and well aligned, this should make it more robust against wandering off and never reacquiring lock.
I left the lossmap script running for the x-arm. Next would be to run it for the y arm, but I see that after stepping to a few positions the lock is again lost. It's still trying to run, but if you want to stop it no data already taken will be lost. To stop it, go to the remaining terminal open on rossa and ctrl+c
the analysis needs:
I made additional measurements on the x and y arms, at 5 offset positions for each arm (along with 6 measurements at the "zeroed" position).
While I stopped by the lab this morning to pick up some things, I took the opportunity to continue the recovery.
At some point, we should run the suspension eigenmode routine (kick optics, let them ringdown, measure peak locations and Qs) to confirm that the remaining suspensions are okay, will also help in actuation re-allocation efforts on ETMY. But I didn't do this today.
Leaving the lab at 1150.
The autolocker was struggling to lock the IMC. I disabled the autolocker and locked the IMC manually. It seems happy right now.
With PMC trans at 0.717 counts, the IMC trans sum is ~15230.
The MC autolocker hasn't been so snappy recently, and has been especially fussy today. Previously, the mcup script was triggered immediately once the transmission was above a certain threshold. However, this could waste time if it was just an errant flash. Hence, I've added a 0.5 second delay and a second threshold check before mcup is triggered.
After breaking the lock 5ish times, it does seem to come back quicker.
Recently we wondered at the meeting what the IMC round trip loss was. I had done several ringdowns in the winter of 2017, but because the incident light on the cavity wasn't being extinguished completely (the AOM 0th order beam is used), the full Isogaio et. al. analysis could not be applied (there were FSS induced features in the reflection ringdown signal). Nevertheless, I fitted the transmission ringdowns. They looked like clean exponentials, and judging by the reflection signals (see previous elogs in this thread), the first ~20us of data is a clean exponential, so I figured we may get some rough value of the loss by just fitting the transmission data.
The fitted storage time is .However, this number isn't commensurate with the 40m IMC spec of a critically coupled cavity with 2000ppm transmissivity for the input and output couplers.
Attachment #1: Expected storage time for a lossless cavity, with round-trip length ~27m. MC2 is assumed to be perfectly reflecting. The IMC length is known to better than 100 Hz uncertainty because the marconi RF modulation signal is set accordingly. For the 40m spec, I would expect storage times of ~40 usec, but I measure almost 30% longer, at ~60 usec.
Attachment #2: Fits and residuals from the 10 datasets I had collected. This isn't a super informative plot because there are 10 datasets and fits, but to eye, the fits are good, and the diagonal elements of the covariance matrix output by scipy's curve_fit back this up. The function used to fit the t > 0 portions of these signals (because the light was extinguished at t=0 by actuating on the AOM) is , where A and tau are the fitted parameters. In the residuals, the same artefacts visible in the reflection signal are seen.
Attachment #3: Scatter plot of the data. Width of circles are proportional to fit error on individual measurements (i just scaled the marker size arbitrarily to be able to visually see the difference in uncertainty, the width doesn't exactly indicate the error), while the dahsed lines are the global mean and +/- 1 sigma levels.
Attachment #4: Cavity pole measurement. Using this, I get an estimate of the loss that is a much more believable .