I installed nds2 again, this time successfully with
conda install -c conda-forge python-nds2-client
[Mirko / Kiwamu]
The resonant box has been installed together with a 3 dB attenuator.
The demodulation phase of the MC lock was readjusted and the MC is now happily locked.
We needed more modulation depth on each modulation frequency and so for the reason we installed the resonant box to amplify the signal levels.
Since the resonant box isn't impedance matched well, the box creates some amount of the RF reflections (#5339).
In order to reduce somewhat of the RF reflection we decided to put a 3 dB attenuator in between the generation box and the resonant box.
(what we did)
+ attached the resonant box directly to the EOM input with a short SMA connector.
+ put stacked black plates underneath the resonant box to support the wight of the box and to relief the strain on the cable between the EOM and the box.
+ put a 3 dB attenuator just after the RF power combiner to reduce RF reflections.
+ readjusted the demodulation phase of the MC lock.
(Adjustment of MC demodulation phase)
The demodulation phase was readjusted by adding more cable length in the local oscillator line.
After some iterations an additional cable length of about 30 cm was inserted to maximize the Q-phase signal.
So for the MC lock we are using the Q signal, which is the same as it had been before.
Before the installation of the resonant box, the amplitude of the MC PDH signal was measured in the demodulation board's monitor pins.
The amplitude was about 500 mV in peak-peak (see the attached pictures of the I-Q projection in an oscilloscope). Then after the installation the amplitude decreased to 400 mV in peak-peak.
Therefore the amplitude of the PDH signal decreased by 20 %, which is not as bad as I expected since the previous measurement indicated 40 % reduction (#2586).
Gautam and I were talking about some modulation and demodulation and wondered what is the power combining situation for the triple resonant EOM installed 8 years ago. And we noticed that the current setup has additional ~5dB loss associated with the 3-to-1 power combiner. (Figure a)
N-to-1 broadband power combiners have an intrinsic loss of 10 log10(N). You can think about a reciprocal process (power splitting) (Figure b). The 2W input coming to the 2-port power splitter gives us two 1W outputs. The opposite process is power combining as shown in Figure c. This case, the two identical signals are the constructively added in the combiner, but the output is not 20Vpk but 14Vpk. Considering thge linearity, when one of the port is terminated, the output is going to be a half. So we expect 27dBm output for a 30dBm input (Figure d). This fact is frequently oversight particularly when one combines the signals at multiple frequencies (Figrue e). We can avoid this kind of loss by using a frequency-dependent power combiner like a diplexer or a triplexer.
With all of the shaking (man-made and divine), it was a hard to debug this problem. Summary of fixes:
At least the DC indicators are telling me that the IMC locking is back to a somewhat stable state. I have not yet checked the frequency noise / RIN.
I resume my IMC ringdown activities now that the IMC is aligned again.
To avoid any accidental misalignments Gautam turned off all the inputs to the WFS servo.
I set up a PD and a lens as in attachment 1 (following Gautam's setup).
I connect the REFL, TRANS and INPut PDs to the oscilloscope.
I connect a Siglent function generator to the AOM driver. I try to shut off the light to the IMC using 1V DC waveform and pressing the output button manually. However, it produced heavily distorted step function in the PMC trans PD.
I use a square wave with a frequency of 20mHz instead with an amplitude of 0.5V offset of 0.25V and dutycycle of 1% so there will be minimal wasted time in the off state. I get nice ringdowns (attachment 2) - forgot to take pictures. The autolocker slightly misaligns the M2 every time it is acting, so I manually align it everytime the IMC gets unlocked.
Data analysis will come later.
I remove the PD and lens and reenable the WFS servo inputs. The IMC locks easily. The WFS outputs are very different than 0 now though.
I analyze the IMC ringdown data from last night.
Attachment 1 shows the normalized raw data. Oscillations come in much later than in Gautam's measurement. Probably because the IMC stays locked.
Attachment 2 shows fits of the transmitted PD to unconstrained double exponential and the Zucker model.
Zucker model gives time constant of 21.6us
Unconstrained exponentials give time constants of 23.99us and 46.7us which is nice because it converges close to the Zucker model.
One factor which hampers locking efforts is the apparent drift of the input beam into the IFO. Over timescales of ~1 hour, I have noticed that the spot on the AS camera drifts significantly (~1 spot size) in pitch. The IPPOS QPD bears out this observation, see Attachment #1. The IMC WFS control signals do not show a correlated drift, hence my claim that the TTs are to blame.
I am able to correct this misalignment by moving TT1 in pitch (see Attachment #2, which shows some signals from a ~1 hour PRMI lock, during which time the pointing drifted, and I corrected it by moving TT1 pitch). Assuming the problem is purely TT1 pitch drifting, this corresponds to 3mm / 6m ~500urad of shift in 1 hour - seems very large. The fact that the drift is only present in pitch and doesn't really show up in yaw makes me think the problem is likely mechanical (unless the voltage to the top two coils is drifting relative to the bottom, but no LR drift, which would be very coincidental). At the moment, this is just an annoyance, but it'd be good for this problem to be fixed.
I haven't yet figured out how to make ndscope export these plots to SVG preserving the dark color theme, hence the weird light axes...
I extended the ringdown data analysis to the reflected beam following Isogai et al.
The idea is that measuring the cavity's reflected light one can use known relationships to extract the transmission of the cavity mirrors and not only the finesse.
The finesse calculated from the transmission ringdown shown in the previous elog is 1520 according to the Zucker model, 1680 according to the first exponential and 1728 according to the second exponential.
Attachment 1 shows the measured reflected light during an IMC ringdown in and out of resonance and the values that are read off it to compute the transmission.
The equations for m1 and m3 are the same as in Isogai's paper because they describe a steady-state that doesn't care about the extinction ratio of the light.
The equation for m2, however, is modified due to the finite extinction present in our zeroth-order ringdown.
Modelling the IMC as a critically coupled 2 mirror cavity one can verify that:
Where is the coupled light power
is the power rejected from the cavity (higher-order modes, sidebands)
is the cavity gain.
and are the power reflectivity and transmissivity per mirror, respectively.
is the power attenuation factor. For perfect extinction, this is 0.
Solving the equations (m1 and m3 + modified m2), using Zucker model's finesse, gives the following information:
Loss per mirror = 84.99 ppm
Transmission per mirror = 1980.77 ppm
Coupling efficiency (to TEM00) = 97.94%
I translate the results obtained in the previous elog to the IMC 3 mirror cavity. I assume the loss in each mirror in the IMC is equal and that M2 has a negligible transmission.
I find that to a very good approximation the loss per IMC mirror is 2/3 the loss per mirror in the 2 mirror cavity model. That is the loss per mirror in the IMC is 56 ppm. The transmission per mirror in the IMC is the same as in the 2 mirror model, which is 1980 ppm.
The total transmission is the same as in the 2 mirror model and is given by:
where is the coupling efficiency to the TEM00 mode.
We measure the IMC transfer function using SR785.
We hook up the AOM driver to the SOURCE OUT, Input PD to CHANNEL ONE and the IMC transmission PD to CHANNEL TWO.
We use the frequency response measurement feature in the SR785. A swept sine from 100KHz to 100Hz is excited with an amplitude of 10mV.
Attachment 1 shows the data with a fit to a low pass filter frequency response.
IMC pole frequency is measured to be 3.795KHz, while the ringdowns predict a pole frequency 3.638KHz, a 4% difference.
The closeness of the results discourages me from calibrating the PDs' transfer functions.
I tend to believe the pole frequency measurement a bit more since it coincides with a linewidth measurement done awhile ago Gautam was telling me about.
I think of trying to try another zero-order ringdown but with much smaller excitation than what used before (500mV) and than move on to the first-order beam.
Also, it seems like the reflection signal in zero-order ringdown (Attachment 2, green trace) has only one time constant similar to the full extinction ringdown. The reason is that due to the fact the IMC is critically coupled there is no DC term in the electric field even when the extinction of light is partial. The intensity of light, therefore, has only one time constant.
Fitting this curve (Attachment 3) gives a time constant of 18us, a bit too small (gives a pole of 4.3KHz). I think a smaller extinction ringdown will give a cleaner result.
We did some quick DC balancing of the MC2 coil drivers to reduce the l2a coupling. We updated the gains in the C1:SUS-MC2_UL/UR/LR/LLCOIL to be 1, -0.99, 0.937,-0.933, respectively. The previous values were 1, -1, 1, -1.
The procedures are the following:
Drive UL+LR and change the gain of LR to zero pitch.
Drive UR+LL and change the gain of LL to zero pitch.
Lastly, drive all 4 coils and change UR & LR together to zero yaw.
We used C1:SUS-MC2_LOCKIN1_OSC to create the excitations at 33 Hz w/ 30,000 cts. The angular error signals were derived from IMC WFSs.
While this time we did things by hand, in the future it should be automated as the procedure is sufficiently straightforward.
Today, I did the following tests (and so was touching electronics/cables at/around 1X2):
Results to follow.
After this work, I reverted the EPICS channels to the usual values. The IMC can be locked.
In the style of the KA characterization of the CM board, the AO path gain EPICS slider (IN2) of the IMC servo board was stepped by 1 dB through the full available range of -32 dB to +31 dB. For each value of the requested gain, I measured the TF from the injected signal (to IN2) to TP1A on the IMC servo board. I used the BNC connector for this test, whereas we use the LEMO connector for the AO path. The source was tee-d off at the SR785 side, with one leg going to IN2 of the IMC servo board, and the other going to CH1A of the SR785. TP1A of the IMC board was connected to CH2A of the SR785.
Attachment #1 - Measured gain vs requested gain.
Attachment #2 - Frequency dependent transfer functions
The motivation here is to try and figure out why I cannot engage the AO path smoothly in the CARM handoff part of lock acquisiton. I plan to use this information to do some loop modeling and project laser frequency noise coupling in various stages of the lock acquisition process.
Sometime between 1PM and 6PM on Tuesday, excess laser frequency noise shows up in MCF at around 800 Hz, as shown in Attachment #1. Sigh.
While I show the MCF spectrum here, I confirmed that this noise is not injected by the IMC loop (with the PSL shutter closed, and the IMC servo board disconnected from the feedback path to the NPRO, the PMC error and control points still show the elevated noise, see Attachment #2). I don't think the problem is from the PMC loop - see Attachment #3 which is the ALS beat out-of-loop noise with the PMC unlocked (the PSL beam doesn't see the cavity before it gets to the ALS setup, and we only actuate on the cavity length for that loop, so this wasn't even really necessary).
Was there some work on the PSL table on Tuesday afternoon that can explain this?
I did some preliminary debugging of this, and have localized the problem to the output path (after MC slow) on the IMC Servo card. Basically, I monitored the spectrum of the ALS beat frequency fluctuations under a few different conditions:
Toggling C1:IOO-MC_FASTSW, which supposedly isolates the post-MC slow (a.k.a. MCL) part of the servo, I see no difference. I am also reasonably confident this switch itself works, because I can break the IMC lock by toggling it. So pending a more detailed investigation, I am forced to conclude that the problem originates in the part of the IMC servo board after the MCL pickoff. Some cabling was removed at 1X2 on Tuesday between the times when there was no excess and when it showed up, but it's hard to imagine how this could have created this particular problem.
My old scheme was flawed as I used pitch as the readback. The pitch signal could not distinguish the cross-coupling due to coil imbalance and that due to the natural suspension L2P. A new scheme based on yaw alone has been developed and will be integrated into ifo_test. For now we revert the C1:SUS-MC2_UL/UR/LR/LLCOIL gains back to 1, -1, 1, -1.
I lowered the (FAST) PZT gain on the IMC/FSS servo today.
I noticed that the MC locks looked unstable a lot of the day, and during lock the PCDRIVE channel is above 1 Vrms (which means the loop is oscillating, ttypically at the PZT/EOM crossover frequency).
I changed the default setting from 22 to 20 19 dB in the PSL Settings screen so the mcup script will use this for now. Feel free to revert if this turns out to be a Fluke (which you would think is a terrible name for a company, but...)
since the summary pages are working again, I was clicking through and noticed that there's a wandering peak in the whitened IMC spectrogram that goes from 10-30 Hz over the course of a day.
anyone know what this is ?
There was an abrupt change in the MC_F spectrum between August 4 and August 5, judging by the summary pages - the 1 and 3 Hz resonances are no longer visible in the spectrum. Possibly, this indicates some electronics failure on the MC servo board / whitening board, the CDS settings don't seem to have changed. There is no record of any activity in the elog around those dates that would explain such a change. I'll poke around at 1X2 to see if anything looks different.
Update 1740: I found that the MCL / MCF cables were disconnected. So since August 5, these channels were NOT recording any physical quantity. Because their inputs weren't terminated, I guess this isn't a clean measurement of the whitening + AA noise, but particularly for MC_F, I guess we could use more whitening (see Attachment #1). Probably also means that the wandering ~10-30Hz line in the spectrogram is a electronics feature. The connections have now been restored and things look nominal again.
that's a very curious disconnection
the "Pentek" whitening board that carries the MC channels has jumpers to enable either 1 or 2 stages of 15:150 whitening. Looks lik MC_F has 2 and MC_L has 1.
I guess the MC_F signal is so low because of the high gain on the FSS board. We could lower the FSS common gain and increase the IMC board's VCO gain to make up for this. Maybe 6 dB would be enough. IF that is risky, we could also up the analog gain on the whitening board.
This elog suggests that there is uniformly 1 stage engaged across all channels. I didn't look at the board to see what the jumper situation is, but only 1 stage of whitening is compensated digitally for both _F and _L. The Pomona box attached to the NPRO PZT input is also compensated digitally to convert counts to frequency.
I tried the gain re-allocation between VCO gain and FSS COMM (and also compensated for the cts to Hz conversion in MCF), but it doesn't seem to have the desired effect on the MCF SNR in the 5-50Hz band. Since the IMC stays locked, and I had already made the changes to mcup, I'll keep these gains for now. We can revert to the old settings if the IMC locking duty cycle is affected. Explicitly, the changes made were:
VCO gain: +7dB ---> +13 dB
FSS COMM: +6 ddB ---> +0 dB
The mcdown script wasn't modified, so the lock acquisition gains are the same as they've been.
We wanted to track down the excess noise seen in MC_F and other places (see the previous report by Gautam)
Setup1: The IMC was locked and MC_F signal between 500 and 1500Hz was observed. The DTT template was saved as /users/Templates/MC/MCF_noise_201023.xml
- Suspected mech resonance/jitter coupled with clipping or any other imperfections. Poked the various optics and optomechanics on the table. Basically no change. If we tap the laser chassis and the optics close to the laser source, we occasionally unlocked the IMC
- When we touched (lifted) the Innolight controller box from the shelf, for the first time we saw a significant change in the shape of the noise spectrum. The peak around the 700Hz shited towards lower frequency by a few %. Other peaks have no obvious change in the shapes and the heights.
- While observing the MC_F signal on the laptop, we went to the back of the laser controller. Placing a hand close to the fan clearly changes the peak frequency lower. By temporarily disconnecting the fan from the power supply for a short moment, the 700Hz peak could be eliminated. We also tried to see the noise level with the slow thermal servo and diagnosis DB cable disconnected, but we didn't see any significant change of the noise level.
Setup 2: Using the ALS phase tracker, we can observe the relative freq noise of the PSL laser and the ETMY AUX laser without any servo involved. This way we can freely disconnect any cables from the lasers. The measurement template for DTT was saved as /users/Templates/ALS/Y_ALS_FINE_PHASE_OUT_102320.xml
- Noise spectrum before disconnecting the cable (REF0, RMS REF1)
- The Fast PZT input to the PSL was disconnected => This made all the peaks (including the 700Hz) disappeared (REF2, RMS REF3)
- The Fast PZT input was restored as before, then the chain was disconnected at the input of the HV PZT driver (Thorlabs) => Again, this made the peaks disappeared (REF4, RMS REF5)
- The chain was disconnected at the input of the TTFSS box => Again, this made the peaks disappeared (REF6, RMS REF7)
- Disconnected the demod input and the AO cables from the IMC servo board => This made the peaks came back (REF8)
- Disconnected all the input/peripheral cables from the IMC servo board except for the connection to the TTFSS box => Still the excess noise was observed (REF9)
- In addition to the above, the cable to the FSS box was disconnected but the ground was still touching the MC servo board => This made the peaks disappeared (REF10)
The conclusion is that the noise is injected from the main circuit of the IMC servo board.
Next time we will check if the backplane connection is doing something wrong. Also, we'll test if the presence of the RF signals does something bad to the IMC board via EMI and RFI.
We have reverted the connection and tested if we lock the IMC and Y arm. ==> We saw at least they were locked for a short period. The things are still stabilizing, but left them turned on so they keep trying to lock automatically for the night.
In fact, the problem was the grounding issue (presumably on the IOO racks).
A temporary differential receiver at the TTFSS side was built using an SR560 and a few ponoma cables. This removed the structures ~850Hz.
The MC Servo Output was disconnected from the TTFSS box and monitored with SR785. The 850Hz structure was kept visible no matter what cables, including all the acromag DB cables, were removed. This made me suspicious about the measurement setup. The SR785 was connected to an AC power strip under the SP table and this was too far from the IOO rack.
The SR785 was connected to the AC power strip on 1X2, and now the difference becomes clear. No matter if the acromag cables are connected or not, the connection (particularly ground connection) between the MC servo module and the TTFSS box causes the MC servo output contaminated. (Comparison between Blue and Orange of Attachment #1). During the measurement, the EPICS switch for the fast path was disengaged (=no signal) and the VCO gain (...so called. It's just the MC Servo Gain) was set to be 0dB.
To test if the differential receiving of the MC Servo Output at the PSL helps to reduce this noise, I've built a simple (hacky) differential receiver using an SR560. (Attachment #2)
This kept the noise level same as the disconnected case (Comparison between Green and Orange of Attachment #1, I don't think the difference between them is not significant), while the IMC is locked as before.
Note that we can see that the 36kHz line was significantly reduced. Did we remove this annoying noise?
After talking with Gautam, we decided to leave this configuration while the SE-Diff cable was replaced with a more robust one. (See Attachment #3)
The PSL laser frequency performance was evakluated in the following two ways as we did last week:
1) Use the beat frequency of the free running PSL and the Y-end laser (Attachment #4). The PSL shutter was closed and thus the IMC was not locked.
2) Use the IMC MCF while the IMC was locked. (Attachment #5)
For both cases, the improvement was confirmed.
I also tried to check the reported issue by Gautam on this elog. He used 1Hz BW, but I cheated with 16Hz BW and 10x12.8kHz span PSDs. (Attachment #6)
For the measurement, IN1 GAIN of the IMC Servo was set to be 0dB and the OUT2 was switched to monitor the IN1 noise, while IN1 was terminated by a 50Ohm.
As I mentioned above, the AC power of SR785 was taken from a 1X2 power strip. Is this the reason for the power line forest look less severe compared to the previous case???
Anyway, I tried to use the same differential receiving technique (but with gain of x100) to see if this helps. The differential receiver helped to reduce the structure above 50kHz. The floor noise level was observed to be higher. I didn't pursue this any further, but the forest of the power line looked like a part of the measurement noise. This is indicative that the grounding condition on 1X2 is really not great and we need to review the configuration of the acromag grounding.
Apart from the questionable wiring on the Acromags, one other important difference is in the way the connections were made between the old VME crates to the Eurocrate backplanes, and how we do it now. The thick cables had their sheilds connected to the eurocrate ground (or at least, there was a dedicated ground lug on those cables which we screwed on to the ground terminals on the Eurocrate backplanes). However, in our current configuration, we interface the Acromag ADCs and DACs to the backplane via these adaptor boards. The shields of the DSUB cables are presumably NOT connected to the Eurocrate grounds. This should also be investigated as one potential cause of the grounding issue - while on some of the Eurocrate modules, the P1/P2 connectors may have either the "A" or "C" row of connectors shorted to ground, some may not, and the TTFSS may suffer from such an issue?
Note that we have this problem in all of the slow machines that were upgraded to Acromag (if this turns out to be the issue).
In fact, the problem was the grounding issue (presumably on the IOO racks).
Looking back through the elog, 1mtorr is the pressure at which it is deemed safe to send the full power beam into the IMC. After replacing the HR mirror in the MCREFL path with a 10% reflective BS, I just cranked the power back up. IMC is locked. With the increased exposure on the MC2T camera, lots of new scattered light has become visible.
While proceeding with the interferometer recovery, I noticed that there appeared to be no light on WFS2. I confirmed on the AP table that the beam was indeed hitting the QPD, but the DC quadrants are all returning 0. Looking back, it appears that the failure happened on Monday 26 October at ~6pm local time. For now, I hand-aligned the IMC and centered the beams on the WFS1 and MC2T QPDs - MCT is ~15000 cts and MC REFL DC is ~0.1, all consistent with the best numbers I've been able to obtain in the past. I don't think the servo will work without 1 sensor without some retuning of the output matrix.
It would appear that both the DC and RF outputs of WFS2 are affected - I dithered the MC2 optic in pitch (with the WFS loop disabled) at 3.33 Hz, the transmission and WFS1 sensors see the dither but not WFS2. It could be that I'm just not well centerd on the PD, but by eye, I am, so it would appear that the problem is present in both the DC and RF signal paths. I am not going into the PD head debugging today.
I checked the backplane connection for IMC WFS2 and found that the cables for IMC WFS2 and the IMC demod were swapped during my IMC noise hunting activities. I reverted it just now.
But we need to check if this damaged anything such as the WFS2 head, the WFS2 demod, etc, once the IMC locking is back.
There seems to be significant phase loss in the TTFSS path, which is limiting the IMC OLTF to <100 kHz.
See Attachment #1 and #2. The former shows the phase loss, while the latter is just to confirm that the optical gain of the error point is roughly the same, since I noticed this after working on and replacing the RF frequency distribution unit. Unfortunately there have been many other changes also (e.g. the work that Rana and Koji did at the IMC rack, swapping of backplane controls etc etc - maybe they have an OLTF measurement from the time they were working?) so I don't know which is to blame. Off the top of my head, I don't see how the RF source can change the phase lag of the IMC servo at 100 kHz. The only part of the IMC RF chain that I touched was the short cable inside the unit that routes the output of the Wenzel source to the front panel SMA feedthrough. I confirmed with a power meter that the power level of the 29.5 MHz signal at that point is the same before and after my work.
The time domain demod monitor point signals appear somewhat noisier in todays measurement compared to some old data I had from 2018, but I think this isn't significant. Once the SR785 becomes available, I will measure the error point spectrum as well to confirm. One thing I noticed was that like many of our 1U/2U chassis units, the feedthrough returns are shorted to the chassis on the RF source box (and hence presumably also to the rack). The design doc for this box makes many statements about the precautions taken to avoid this, but stops short of saying if the desired behavior was realized, and I can't find anything about it in the elog. Can someone confirm that the shields of all the connectors on the box were ever properly isolated? My suspicion is that the shorting is happening where the all-metal N-feedthroughs touch the drilled surfaces on the front panel - while the front and back surfaces of the panel are insulating, the machined surfaces are not.
This is an unacceptable state but no clear ideas of how to troubleshoot quickly (without going piece by piece into the IMC servo chain) occur to me. I still don't understand how the freq source work could have resulted in this problem but I'm probably overlooking something basic. I'm also wondering why the differential receiving at the TTFSS error point did not require a gain adjustment of the IMC servo? Shouldn't the differential-receiving-single-ended-sending have resulted in an overall x0.5 gain?
Update 8 Dec 1200: To test the hypothesis, I bypassed the SR560 based differential receiving and restored the original config. I am then able to run with the original gain settings, and you see in Attachment #4 that the IMC OLTF UGF is back above 100 kHz. It is still a little lower than it was in June 2019, not sure why. There must be some saturation issues somewhere in the signal chain because I cannot preserve the differential receiving and retain 100 kHz UGF, either by raising the "VCO gain" on the MC servo board, setting the SR560 to G=2, or raising the "Common Gain Adjust" on the FSS box by 6 dB. I don't have a good explanation for why this worked for some weeks and failed now - maybe some issue with the SR560? We don't have many working units so I didn't try switching it.
So either there is a whole mess of lines or the frequency noise suppression is limited. Sigh.
The servos are almost certainly not optimal - but we have the IFO sort of working, so before we make any changes, let's make a strong case for it. Once the loop TFs and noises (e.g. the sensing noise reinjection you maybe saw) are fully characterized and a new loop is shown to perform better, then we can make the changes, but until then, let's continue using the "nominal" configuration and keep all the WFS loops on . I turned everything back on.
BTW, MC2_ASCPIT_IN1 isn't the correct channel to measure the sensing noise re-injection, you need some other sensor, e.g. is the MC transmission (de)stabilized. 0-20 Hz is where I expect the WFS is actually measuring above the sensing noise.
I spent an hour today evening checking out the remote waveplate operation. Basic remote operation was established 👍 . To run a test on the main beam (or any beam for that matter), we need to lay out some long cabling, and install the controller in a rack. I will work with Jordan in the coming days to do these things. Apart from the hardware, some EPICS channel will need to be added to the c1ioo.db file and a python script will need to be set up as a service to allow remote operation.
Satisfied that the unit works basically as expected, I decided to stop for today. My thinking was that we can have the ESP300 installed in 1X1 or 1X2 (depending on where space is more readily available). I will upload have uploaded a cartoon here so people can comment if they like/dislike my plan.
Once everything is installed, we can run some tests to see if the rotary motion disturbs the PSL in any meaningful way. I will upload some photos to the picasa later. Photos here.
I spent some time today setting up a workable user interface to control the waveplate.
So this system is ready to be installed once Jordan and I find some time to lay out cabling + install the ESP300 controller in a rack.
At the moment, there is no high power and there is minimal risk of damaging anything, but someone should double check my logic to make sure that we aren't gonna burn the precious IFO optics. We should also probably hook up a hardware interlock to this controller.
I went through some aLIGO documentation and believe that they are using a custom made potentiometer based angle sensor rather than the integrated Newport (or similar) sensor+motor. My reading of the situation was that there were several problems to do with hysterisis, the "find home" routine etc. I guess for our purposes, none of these are real problems, as long as we are careful not to randomly rotate the waveplate through a full 180 degrees and go through the full fringe in the process. Need to think of a clever way to guard against careless / accidental MEDM button presses / slider drags.
Unrelated to this work: I haven't been in the lab for ~a week so I took the opportunity today to go through the various configs (POX/POY/PRMI resonant carrier etc). I didn't make a noise budget for each config but at least they can be locked 👍 . I also re-aligned the badly misaligned PMC and offloaded the somewhat large DC WFS offsets (~100 cts, which I estimate to be ~150 nNm of torque, corresponding to ~50 urad of misalignment) to the IMC suspensions' slow bias voltages.
We did the following this afternoon.
The IMC stayed locked throughout our work, and judging by the CDS overview screen, we don't seem to have done any lasting damage, but I will run more tests. Note that the waveplate isn't yet installed in the beam path - I may do this later today evening depending on lab activity, but for now, it is just sitting on the lower shelf inside the PSL enclosure. I will post some photos later.
Update: The waveplate was installed. I gave it a couple of rounds of cleaning by first contact, and visually, it looked good to me. More photos uploaded. I also made some minor improvements to the MEDM screen, and setup the communication script with the ESP300 to run as a systemd service on c1psl. Let's see how stable things are... I think the philosophy at the sites is to calibrate the waveplate rotation angle in terms of power units, but i'm not sure how the unit we have performs in terms of backlash error. We can do a trial by requesting ~100 "random" angles, monitoring the power in s- and p-polatizations, and then quanitfying the error between requested and realized angles, but I haven't done this yet. I also haven't added these channels to the set recorded to frames / to the burt snapshot - do we want to record these channels long term?
MC was unlocked and struggling to recover this morning due to misguided WFS offsets. In order to recover from this kind of issue, we
The MC is now restored and the plan is to let it run for a few hours so the offsets converge; then run the WFS relief script.
We found that megatron is unable to properly run scripts/MC/WFS/mcwfsoff and scripts/MC/WFS/mcwfson scripts. It fails cdsutils commands due to a library conflict. This meant that WFS loops were not turned off when IMC would get unlocked and they would keep integrating noise into offsets. The mcwfsoff script is also supposed to clear up WFS loop offsets, but that wasn't happening either. The mcwfson script was also not bringing back WFS loops on.
Gautam fixed these scripts temprorarily for running on megatron by using ezcawrite and ezcaswitch commands instead of cdsutils commands. Now these scripts are running normally. This could be the reason for wildly fluctuating WFS offsets that we have seen in teh past few months.
gautam: the problem here is that megatron is running Ubuntu18 - I'm not sure if there is any dedicated CDS group packaging for Ubuntu, and so we're using some shared install of the cdsutils (hosted on the shared chiara NFS drive), which is complaining about missing linked lib files. Depending on people's mood, it may be worth biting the bullet and make Megatron run Debian10, for which the CDS group maintains packages.
Came this morning and saw that the IMC is unlocked.
Went into MC Lock screen and see that the watchdog is down and the PSL shutter is closed. I tried to open the shutter but nothing happened - no REFL signal or beam on the MC REFL camera .
Thinking this has something to do with the watchdog I upped the watchdog:
ezcawrite C1:SUS-MC2_LATCH_OFF 1
The watchdog on the MEDM screen became green but the shutter still seemed unresponsive. I went to the PSL table and made sure that the shutter is working. I opened the AS table and saw there no MC REFL beam anywhere.
Thinking that MC1 must be completely misaligned I opened the MC align screen to find that indeed all the alignment values has been zeroed! (attachment).
I burt restore c1iooepics from Mar 4th 00:19. Didn't help.
I try to burt restore c1susepics from Mar 1st 13:19. Still zero.
I try to burt restore c1susaux from Mar 1st 00:19 -> seems like alignment values have been restored.
I open the shutter. Beam is flying! MC Watchdogs tripped! I close the shutter. OK, I need to wait until the MCs are dampped enough. MC2 and MC3 have relaxed so I enable their watchdogs. MC1 is still swinging a bit. I turn on damping for MC1 as well.
MC locked immediately but the REFL is still high like 1.2. Is it normal?
I turn on the WFSs and the REFL went down to 0.3 nice. I run the MC WFS relief script.
Hmm, the bias values were reset at 2022-03-03-20:01UTC which is 2022-03-03-12:01 PST with no apparent disruption of the data acquisition (= no resetting of the RTS). Not sure how this could happen.
Came this morning, opened the PSL and there was not even a beam on the MC REFL.
Looking at the big monitor it seems like the WFS signals went through the roof during the "auto-alignment" night session.
I restored the MC alignment from before the misalignment happen and wait for the SUS to damp. Once the RMS values went below 200 I enabled the watchdog and the coil outputs.
I opened the PSL shutter and the IMC locked immediately. I turned on the WFS servo and the MC REFL DC went down to 0.3. I run the WFS relief script.
The IMC alignment was restored and the IMC is nicely locking.
Once the vacuum level recovered P<1e-4 torr, the PSL shutter was able to be opened.
The IMC was still flashing, so the lock to TEM00 was possible.
Once it was locked, the MC2 alignment was tweaked and the autolocker and the WFS kicked in to help the locking/alignment.
The transmission is ~13k and seems reasonable considering the low PMC transmission of the PMC (0.672)
It seems that early this morning MC got very misaligned. Yuta was able to align the Mode Cleaner again by individually adjusting the MC1 MC2, and MC3. Once transmission reach ~12000, we went ahead and turned on WFS. Oddly enough, the transmission began plummeting and MC fell out of lock. After this, Yuta reset the WFS offsets and realigned the WFS QPDs. We then locked MC and turned on WFS once again, but the same issue happened. After fiddeling around with this, we found the if we set C1:IOO-MC2_TRANS_PIT_OUTPUT and C1:IOO-WFS1_YAW_OUTPUT equal to 0, WFS does not cause this issue. Is there a proper to reset WFS, aside from only zeroing the offsets?
This issue is very weird and still unresolved. Without WFS loops, we'll have to realign IMC often and we might loose IMC alignment completely during weekends or long weekends.
I tried following things today but nothign worked:
We need to investigate this as first priority. Maybe some cable is loose, some PD power supply not working etc. Until we fix this, people should align IMC to > 12000 transmission counts whenever they have a spare 5 min. We need to work in place of WFS for sometime.
I could run the WFS servo (6dofs) for more than 15min by flipping the sign for the MC2 Pit and WFS1 Yaw. (See attachments)
This may mean that the sign of the loops / the input matrix / the output matrix, as well as the sensors and actuators, have the problem.
Isn't it the time to measure the sensing/actuation matrices? Maybe Tomislav already has the data?
I have reverted the changes as you may need more careful investigation.
We found that MC1 local damping loop signs were revereted to the state before our standardization on June 7th (40m/16898), but the WFS output matrix was not reverted.
This caused the sign flip in the feedback to MC1, which caused the IMC WFS issue.
This probably happened when we were restarting the models after RTS modeling (40m/16935). We might have used wrong snap files for burt-restoring.
We went back to the snapshot taken at 09:19 June 21, 2022 and now the IMC WFS is working,
I found the IMC was largely misaligned and was not locking. The WFS feedback signals were saturated and MC2 was still largely misaligned in yaw after resetting the saturation.
It seemed that the MC WFS started to put the large offset at 6:30AM~7:00AM (local).
MC2 was aligned and the lock was recovered then the MC WFS seems working for ~10min now.
MC WFS Demod board needs some attention.
Tomislav has been measuring a very high noise level in the MC WFS demod output (which he promised to elog today!). I thought this was a bogus measurement, but when he, and Paco and I tried to measure the MC WFS sensing matrix, we noticed that there is no response in any WFS, although there are beams on the WFS heads. There is a large response in MC2 TRANS QPD, so we know that there is real motion.
I suspect that the demod board needs to be reset somehow. Maybe the PLL is unlocked or some cable is wonky. Hopefully not both demod boards are fried.
Please leave the WFS loops off until demod board has been assessed.
This is just my intuition but the IMC servo seems not so optimized. I can increase the servo gain by 6~10dB easily. And I couldn't see that the PC drive went mad (red) as I increase the gain (=UGF).
The IMC needs careful OLTF measurements as well as the high freq spectrum observation.
It seems that I have worked on the IMC servo tuning in 2014 July/Aug. Checking these elogs would be helpful.
It looks like Tomislav's measurements of the WFS demod board noise were actually of the cable that goes from the whitening to the ADC. So the huge low frequency excess that he saw is not due to wind, but just the inverse whitening of the digital system?
In any case, today, I looked at the connections from the Whitening to the ADC. It goes through an interface chassis to go from ribbon to SCSI. The D-Sub connectors there have the common problem in many of the LIGO D-sub connectors: namely that the strain relief nuts are too tall and so the D connector doesn't seat firmly - its always about to fall out. JC, can you please take a look at this and order a set of low profile nuts so that we can rework this chassis? Its the one between the WFS whitening and the SCSI cables which go to the ADCs.
After pushing them in, I confirmed that the WFS are working, by moving all 6 DoF of the MC mirrors via bias slider, and looking at the step responses (attached). As you can see, all sensors see all mirrors, even if they are noisy.
Next up: get a breakout for the demod output connector and measure the noise there.
For today, I aligned the IMC by hand, then centred the WFS beams by unlocking the IMC and aligning the bright beam. I noticed that the WFS1 beam was being dumped randomly, so I angled the WFS1 by ~3 deg and dumped the specular reflection on a razor blade dump. To handle the sign change in the MC1 actuation (?), I changed the sign in the MC1 ASC filter banks. MCWFS loops still nto closing, but they respond to mirror alignment.
I wasn't sure how the IMC servo was optimized recently. We used to have the FSS over all gain (C1:PSL-FSS_MGAIN) of +6dB a few years back. It is not 0dB. So I decided to do a couple of measurements.
1) Default setting:
2) Looked at the power spectrum at TEST1A output (error signal)
TEST1A is the signal right after the input gain stage (C1:IOO-MC_REFL_GAIN). Prior to the measurement, I've confirmed that the UGF is ~100Hz even at +0dB (see next section). It was not too bad even with the current default. Just wanted to check if we can increase the gain a bit more.
The input gain was fixed at +4dB and the FSS overall gain C1:PSL-FSS_MGAIN was swept from +0 to +6.
At +5dB and +6dB, the servo bump was very much visible (Attachment 1).
I decided to set the default to be +4dB (Attachment 3).
3) Took OLTF at 0dB and 4dB for the FSS overall gain.
Now the comparison of the opel loop transfer functions (OLTF) for C1:PSL-FSS_MGAIN at 0dB and 4dB. The OLTF were taken by injectiong the network analyzer signal into EXCA and measure the ratio between TEST1A and TEST1B (A/B).
C1:PSL-FSS_MGAIN +0 -> UGF 100kHz / Phase Margin ~50deg
C1:PSL-FSS_MGAIN +4 -> UGF 200kHz / Phase Margin 25~30deg
The phase margin was a bit less but it was acceptable.
4) IMC FSR
Took the opportunity to check the FSR of the IMC. Connected a cable to the RF MON of the IMC REFL demod board. Looked at the peak at 40.56MHz (29.5MHz + 11.066MHz). The peak was not so clear at 11.066195MHz (see 40m ELOG 15845). The peak was anyway minimized and the new modulation frequency was set to be 11.066081MHz (new FSR). The change is 10ppm level and it is within the range of the temp drift.
I wanted to check what's wrong with the WFS.
I played with MC2 misalignment to check the error signals.
MC2 pitch and yaw misalignment optically produce a vertical translation and horizontal rotation of the cavity axis at the waist, respectively. So it is thought to be a more separated excitation of the cavity axis.
Then I noticed that WFS2 error signals in general has high (~100%) pitch-yaw coupling. So it was suspicious.
I went to the rack and found that WFS2 SEG4 RF input (labeled "8") was not completely inserted. (Attachment 1)
It seemed that the LEMO connector or the receptacle didn't latch properly anymore and could be easily pulled.
I gave some elbow grease to fix this but in vain. I ended up to use LEMO-BNC adapters which somehow offered a robust connection.
Desipite the insightful discovery, this was not the intrinsic solution to the issue. I checked the past signal history, but I don't think this loose connection caused the missing signal.
Next, I needed to go a bit deeper. The WFS sensors are supposed to be adjusted to I phase where the PDH signal maximally shows up. And all the segments are supposed to have the same sign in terms of the PDH signal.
I've unlocked the IMC and turned on MC2 tickling. This swept the cavity over the resonances.
WFS1 SEG1I~3I showed about the same waveform, but SEG4 Q shows the PDH signal rather than SEG 4 I.
Then tried the same test for WFS2. The SEG4 I signal has the sign-flipped PDH signal compared to WFS2 SEG1I-SEG3I.
I quickly adjusted the demod phase of WFS1 SEG4 and WFS2 SEG4 to correct them,
WFS1 SEG4 103.9-> -20
WFS1 SEG4 -58 -> 120
This in fact made the pitch and yaw separated but flipped (Pitch signal shows up in WFS1Y and yaw signal shows up in WFS1P. Same for WFS2)
These modifications were reverted upon my leaving.
Now things are much more subtle now. And I need to do a more careful quantitative analysis of the demodulation phases / input matrix / output matrix.
Note: It seems that I had worked on IMCWFS on Dec 21, 2016
Continued to work on the WFS repair
Demod phase adjustment:
- Use the PDH signal to adjust the demodulation phase to have uniform signals between the segments.
- Excited laser frequency at 1234Hz by injecting 10mVpp into IMC Servo Board IN2. The input was enabled on the MC Servo screen and given the input gain of 0dB.
- Looked at the ~real time spectrum in WFS1/2 SEG1/2/3/4 I&Q after the phase rotators. Changed the demod phases 1) to have ~0deg transfer function between C1:IOO-MC_F to C1:IOO-WFSi_Ij 2) to minimize the freq signal in Q phases.
(See Attachment 1)
- Resulting change of the demod phases:
WFS1 SEG1 52.0 -> 38.0deg
WFS1 SEG2 54.0 -> 53.0deg
WFS1 SEG3 16.6 -> 33.2deg
WFS1 SEG4 103.9 ->-37.1deg
WFS2 SEG1 17.0 -> 57.8deg
WFS2 SEG2 26.6 -> 51.5deg
WFS2 SEG3 24.5 -> 44.0deg
WFS2 SEG4 -58.0 ->103.7deg
SEG4 of both WFSs had significant phase rotation. A quick check of the power spectrum indicates that the Q signals have significantly (<x1/10) lower signals (Attachment 2/3/4). So that's good.
Transfer function measurement
Now the ASCPIT/ASCYAW of the MC1/2/3 suspension were excited and the transfer functions to WFS1/2 SEG1/2/3/4 and MC Trans P/Y were measured. The analysis will come later.
Again here the Q signals have significantly lower sensitivity to the mirror motion. So it is consistent with the above observation of the spectra.
However, the quick check of the transfer functions indicated that the conventional input matrices result in the flipped dependence of the combined error signals in pitch and yaw.
This might indicate that some of the cables were not inserted into the demod board properly although the cables at the demod boards show no indication of anomaly. (See the photos in ELOG 17048)
It might be the case that the cable had been inserted with a special unusual arrangement.
In any case, this can be fixed at the input matrix. Native change of the input matrix made WFS1PIT/WFS1YAW/WFS2PIT/WFS2YAW/MC2Trans YAW servos running (after some adjustment of the servo signs).
The MC2TRANS PIT servo didn't seem to settle and run away no matter which sign is used.
It's probably better to look at the sensing matrix and figure out the proper input/output matrix carefully. So at this moment, no WFSs are working.
Note that I left the new demod phases in the system
During the transfer function measurement some filters were turned off to make the shaking smoother:
IMC ASC filters were turned off to make the FResp flat:
- MC1 ASCP/Y FM1/FM5 OFF
- MC2 ASCP/Y FM1/FM5/FM6 OFF
- MC3 ASCP/Y FM1/FM5 OFF
60Hz comb OSEM Input filters were also turned off to make the transfer functions simpler:
- MC1 INPUT FM2 OFF (60Hz comb)
- MC2 INPUT FM2 OFF (60Hz comb)
- MC3 INPUT FM2 OFF (60Hz comb)
cf. Past IMCWFS commissioning http://nodus.ligo.caltech.edu:8080/40m/12684
OK... It seems that all the 6 dof of the IMC WFS servo loops were closed with some condition...
- Measured the transfer functions from ASCPIT/YAW_EXC of each suspensions to WFS segs.
- FInd the proper input matrix for PIT and YAW for WFS1 and WFS2
- Closed loops one by one => This was not so successful because the loop shape was quite conditional.
- Closed WFS1/WFS2 loops one by one only with FM4 (0.8Hz Zero / (100Hz pole)^2). Adjust the gains to have the UGF at a few Hz.
- Found that the separation between WFS1P and WFS2P was not good. This caused instability of these loops when the gains were matched. I ended up lowering the gain of WFS1P by a factor of 10. This made the loop OK to work. FM3 (Integrator below 0.8Hz) worked fine.
- FM9 Rolloff filters (RLP12) makes the loops unstable.
- The MC2 spot loops (MC2_TRANS_PIT/YAW) are supposed to be slow loops. From the time series behavior it looks they are working.
MEDM Snapshots (Attchaments 1~4)