40m QIL Cryo_Lab CTN SUS_Lab TCS_Lab OMC_Lab CRIME_Lab FEA ENG_Labs OptContFac Mariner WBEEShop
  40m Log, Page 344 of 344  Not logged in ELOG logo
ID Dateup Author Type Category Subject
  17305   Wed Nov 23 16:34:33 2022 KojiUpdateCDSNew donatella testing

Let's use this entry to list of the test results of new donatella.

  • Firefox - OK (typing elog now)
  • sitemap - OK
  • Diaggui (DTT) looks working fine for the current and past data
  • Dataviewer looks working fine for the current and past (slow/fast) data
  • ndscope - fine
  • foton - works fine
  • burtgooey - ?
  17306   Wed Nov 23 17:12:34 2022 RadhikaUpdateALSXARM green laser lock debugging

I tested the mixer by feeding it a 300 kHz signal sourced from a Moku:Go. I kept the LO input the same - 231.25 kHz from the signal generator. The mixer output was a ~70 kHz waveform as expected, so demodulation is not the issue in green locking.

Next I'll align the arm cavities with IR and check to see if the green REFL signal looks as expected. If not, we'll have to invesitage the REFL PD. If the signal looks fine, and we now know it's being properly demodulated, the issue must lie further downstream.

  17307   Wed Nov 23 17:21:44 2022 JCUpdateVACInterlocks may have been tripped due to N2 pressure loss

[Paco, JC]

While changing out one of the N2 tanks today, one of the fitting stripped. This caused a major loss of pressure. I replaced one fitting then realized there was a second leak around the area of the gauge. Paco and I changed this and everything should be back up and running. Thhe interlocks may have been tripped  within the last 2 hours.


Attachment 1: Screen_Shot_2022-11-23_at_5.22.57_PM.png
Attachment 2: Screen_Shot_2022-11-23_at_5.23.09_PM.png
  17308   Wed Nov 23 17:28:39 2022 YehonathanUpdateBHDSome more calculations

Fields at the BHD BS. More on this later.

Attachment 1: Fields_at_BHDBS.pdf
  17309   Wed Nov 23 20:58:23 2022 yutaSummaryBHDBHD_DIFF sensitivity to BS dither with MICH Offset with different BH55 demodulation phases

[Anchal, Paco, Yuta]

Attachment #1 is the same plot as 40m/17303 but with MICH sensitivity for ASDC and AS55 also included (in this measurement, BH55 demodulation phase was set to 140.07 deg to minimize I fringe).
Y-axis is now calibrated in to counts/m using BS actuation efficiency 26.54e-9 /f^2 m/counts (40m/17285) at 311.1 Hz.
2nd X-axis is calibrated into MICH offset using the measured AS55_Q value and it's MICH sensitivity, 8.81e8 counts/m (this is somehow ~10% less than our usual value 40m/17294).
ASDC have similar dependence with BHDC_SUM on MICH offset, as expected.
AS55_Q have little dependence with MICH offset on MICH offset, as expected.

This plot tells you that even a small MICH offset at nm level can create MICH sensitivity for BHDC_DIFF, even if we control LO phase to have BH55_Q to be zero, as MICH offset shifts zero crossing of BH55_Q for LO phase.

Notebook: /opt/rtcds/caltech/c1/Git/40m/scripts/CAL/BHD/BH_DIFFSens_pydemod.ipynb

Attachment #2 is the same plot, but BH55 demodulation phase was tuned to 227.569 deg to have no MICH signal in BH55_Q (a.k.a measurement (c)).
In this case, LO phase will be always controlled at 0 deg (90 deg away from optimal), even if we change the MICH offset, as BH55_Q will not be sensitive to MICH.
In this plot, BHD_DIFF have little sensitivity to MICH, irrelevant of MICH offset, as expected.
MICH sensitivity for BH55_I is also constant, which indicate that LO phase is constant over this measurement, as expected.

Notebook: /opt/rtcds/caltech/c1/Git/40m/scripts/CAL/BHD/BH_DIFFSens_pydemod.ipynb

Attachment #3 is the same plot, but BH55 demodulation phase was tuned to 70 deg.
This demodulation phase was tuned within 5 deg to maximize MICH signal in BHD_DIFF with large MICH offset (20).
In this case, LO phase will be always controlled at 90 deg (optimal), even if we change the MICH offset, as BH55_Q will not be sensitve to LO carrier x AS sideband component of the LO phase signal.
In this plot, BHD_DIFF have high sensitivity to MICH, irrelevant of MICH offset (at around zero MICH offset it is hard to see because LO_PHASE lock cannot hold lock, as there will be little LO phase signal in BH55_Q, and measurement error is high for BHD_DIFF and BH55 signals).
MICH sensitivity for BH55_I and BH55_Q is roughly constant, which indicate that LO phase is constant over this measurement, as expected.

These plots indicate that BH55 demodulated at MICH dither frequency can be used to control LO phase robustly at 90 deg, under unknown or zero MICH offset.

Notebook: /opt/rtcds/caltech/c1/Git/40m/scripts/CAL/BHD/BH_DIFFSens_pydemod.ipynb

LO phase delay:
 From these measurements of demodulation phases, I guess we can say that phase delay for 55 MHz in LO path with respect to MICH path (length difference in PR2->LO->BHDBS and PR2->ITMs->AS->BHDBS) is

2*(227.569-70(5)-90)-90 = 45(10) deg

 This means that the length difference is (omegam=5*2*pi*11.066195 MHz)

c * np.deg2rad(45(10)+360) / omegam = 6.1(2) m   (360 deg is added to make it close to the design)

  Is this consistent with our design? (According to Yehonathan, it is 12.02 m - 5.23 m = 6.79 m)

  Attachment #4 illustrates signals in BH55.

 - Lock LO PHASE with BH55 demodulated at MICH dither frequency (RF+audio double demodulation), and repeat the same measurement
 - Finer measurement at small MICH offsets (~1nm) to see how much MICH offset we have
 - Repeat the same measurement with BH55_Q demodulation phase tuned everytime we change the MICH offset to maximize LO phase sensitivity in BH55_Q (a.k.a measurement (b)).
 - What is the best way to tune BH55 demodulation phase?

Attachment 1: BHDIFF_rand_SensvsMICHOffset_pydemod.pdf
Attachment 2: BHDIFF_rand_SensvsMICHOffset_pydemod_NoMICHinBH55Q.pdf
Attachment 3: BHDIFF_rand_SensvsMICHOffset_pydemod_BH55at70deg.pdf
Attachment 4: MICHBHD_BH55.pdf
  17310   Thu Nov 24 11:23:35 2022 AnchalUpdateSUSLow noise state

I've turned off HEPA fan and all lights at
PST: 2022-11-24 11:23:59.509949 PST
UTC: 2022-11-24 19:23:59.509949 UTC
GPS: 1353353057.509949

c1ioo model has been updated to acquire C1:IOO-MC2_TRANS_PIT_OUT  and C1:IOO-MC2_TRANS_YAW_OUT at 512 Hz rate.

I'll update when I turn the HEPA on again. I plan to turn it on for a few hours everyday to keep the PSL enclosure clean.

Turned on HEPA again at:

PST: 2022-11-25 12:14:34.848054 PST
UTC: 2022-11-25 20:14:34.848054 UTC
GPS: 1353442492.848054

However this was probably not a low noise state due to vacuum disruption mentioned here.

  17311   Thu Nov 24 15:37:45 2022 AnchalUpdateASCIMC WFS output matrix diagonalization effort

I tried following the steps and the method I was using converged to same output matrix upto 2 decimal points but there is still left over cross coupling as you can see in Attachment 1. With the new output matrix, WFS loop can be turned on with full overall gain of 1.


  • I switched off +20dB FM2 on C1IOO-WFS1_PIT and increased gain C1:IOO-WFS1_PIT_GAIN from 0.1 to 1 to be uniform with other filters.
  • Output matrix change:
    • Old matrix:
      -2.   4.8 -7.3
       3.6  3.5 -2. 
       2.   1.  -6.8
    • New Matrix:
      3.44  4.22 -7.29
      0.75  0.92 -1.59
      3.41  4.16 -7.21
  • I think the main change that allowed the WFS loop to become stable was the 0,0 element sign change.


  • I made overall gain C1:IOO-WFS_GAIN 0
  • Switched of (0:0.8) FM3 on PIT filter modules (IOO-WFS1_PIT, IOO-WFS2_PIT, IOO-MC2_TRANS_PIT)
  • Changed ramp time to 2 seconds on all these modules
  • Used offset of 10000 for WFS2 and MC2_TRANS, and 30000 for WFS1 (for some reason, response to WFS1 step was much lower than others)
  • Measured the following sensor channels
  • First I took 30s average of these channels, then applied the offsets in the three modules one by one and recorded steps in each sensor.
  • Measured step from reference value taken before, and normalized each step to the DOF that was actually stepped to get a matrix.
  • Inverted this matrix and multiplied with existing output matrix. Made sure column norm1 is same as before and column signs are same as before.
  • Repeated a few times.

Note: The standard deviation on the averages was very high even after averaging for 30s. This data should be averaged after low passing high frequencies but I couldn't find the filter module medm screens for these signals, so I just proceeded with simple averaging of full rate signal using cdsultis avg command.

Fri Nov 25 12:46:31 2022

The WFS loop are unstable again. This could be due to the matrix balancing done while vacuum was disrupted. The above matrix does not work anymore.

Attachment 1: WFS_Step_DCResponses_Offsets_Marked.png
  17312   Fri Nov 25 12:15:46 2022 AnchalUpdateVACVacuum Gate valves restored

I came today to find that PSL shutter was closed. I orginially thought some shimmer obersvations are underway in the quiet state. But that was not the case. When I tried to open the shutter, it closed back again indicating a hard compliance condition making it close. This normally happens when vacuum level is not sufficient, so I opened the vacuum screena dn indeed all gate valves were closed. This most probably happend during this interlock trip. So the main volume was just slowly leaking and reached to milli torr level today.

Lesson for future: Always check vacuum status when interlock trips.

[Paco, Anchal]

Paco came by to help. We went to asia (the Asus laptop at vacuum workstation) but could not open the medm or find the nfs mounted files. The chiara change did something and nfs mounted directories are not available on asia of c1vac. We rebooted asia and the nfs mount was working again. We can't simply restart c1vac because it runs acromag channels for vacuum system and needs to be done more carefully, a task for Monday.

After restarting asia, we opened the the vacuum control medm screen and followed the vaccum pump down instructions (mainly opening of the gate vales as the pumps were already on). Point to keep in mind, rule of thumb, do not open valve between a turbo pump and a volume if the pressure differential is more than 3 orders of magnitude. Saving turbo pumps is the priority.. Now the main volume is pumping down.

Attachment 1: Screenshot_2022-11-25_12-29-23.png
  17313   Fri Nov 25 15:35:23 2022 AnchalUpdateSUSLow noise state

Turned off HEPA at:

PST: 2022-11-25 15:34:55.683645 PST
UTC: 2022-11-25 23:34:55.683645 UTC
GPS: 1353454513.683645

Turned on HEPA back at:

PST: 2022-11-28 11:14:31.310453 PST
UTC: 2022-11-28 19:14:31.310453 UTC
GPS: 1353698089.310453

  17314   Sun Nov 27 15:30:22 2022 ChrisUpdateOptimal ControlIMC alignment controller testing

Five more mode cleaner alignment controllers were tested this morning (remotely). These were designed to run in tandem with the standard controller, instead of supplanting it. Before the test, c1ioo was burt restored back to the settings of the previous test on Oct 28, and in MC TRANS PIT/YAW filter banks the 80 dB gain filters were disengaged and outputs were enabled. Subsequently, all settings were returned to the original values. Each test consisted of five minutes with pitch alignment uncontrolled, five minutes with the standard controller only, and twenty minutes with both controllers enabled. GPS times for each phase of testing are the following:

  • musgo
    • OL start 1353602764
    • CL start 1353603074
    • policy start 1353603410
  • musgo_ghost
    • OL start 1353604697
    • CL start 1353605007
    • policy start 1353605355
  • musgo_stumble
    • OL start 1353606574
    • CL start 1353606884
    • policy start 1353607229
  • musgo_goldfish
    • OL start 1353608446
    • CL start 1353608756
    • policy start 1353609099
  • musgo_late
    • OL start 1353610321
    • CL start 1353610631
    • policy start 1353610971
  17315   Mon Nov 28 11:15:23 2022 AnchalUpdateCDSFront ends DAC Kill (DK) got activated; restored FEs

[Anchal, Paco, Yehonathan, JC]

Last night at 9:15 pm PST (Nov 27th, 2022) some kind of disruption happened to FEs. See attachment 1 to see the changes in FE state words of the IOP models. on c1lsc, c1sus and c1scex, change of 140 happend, that's 2nd, 3rd and 7th bit of the FE word was flipped, which I think is the TIM, ADC and DAC KILL (DK). When we came in morning, IMC suspensions were undamped and not responsive to coil kicks, vertex suspensions the same case, ETMX also same. The c1sus2 modelw as all in red.

To fix this, we restarted all rtcds models on all FEs by sshing into the computers and doing:

rtcds restart --all

Then we burt restored all models to 27th Nov, 3:19 am point doing following on rossa:

~>cd /opt/rtcds/caltech/c1/Git/40m/scripts/cds
cds (main)>./burtRestoreAndResetSUS.sh /opt/rtcds/caltech/c1/burt/autoburt/snapshots/2022/Nov/27/03:19

Note: this issue was previously seen when fb1 was restarted without shutting down the FEs, and once when the martian switch was disrupted while FE models were running.

I'm not sure why this happened this time, what caused it at 9:15 pm yesterday, and why only c1lsc, c1sus and c1iscex models went to DAC KILL state. This disruption should be investigated by cds upgrade team.

Attachment 1: FE_State_Word_Changes.png
  17316   Mon Nov 28 11:21:25 2022 JCUpdatePSLPMC input beam alignment

C1:PSL-PMC_PMCTRANSPD was increased from 0.72 to 0.731

Attachment 1: Screenshot_from_2022-11-28_11-20-31.png
  17317   Mon Nov 28 16:53:22 2022 AnchalSummaryBHDF2A filters on LO1 LO2 AS1 and AS4

[Paco, Anchal]

I changed the script in /opt/rtcds/caltech/c1/Git/40m/scripts/SUS/outMatFilters/createF2Afilters.py to read the measured POS resonant frequencies stored in /opt/rtcds/caltech/c1/Git/40m/scripts/SUS/InMatCalc/resFreqs.yml instead of using the estimate sqrt(g/len). I then added Q = 3 F2A filters into FM1 output filter of LO1, LO2, AS1 and AS4 suspensions in anticipation of BHD locking scheme work.

  17318   Mon Nov 28 16:58:20 2022 PacoUpdateCDSpypi package added

[Paco, Tega]

I added the pypi package "restoreEpics" to the donatella clone under test. This is required by some of Anchal's scripts that turn on F2A filters as well as other recovery stages during some measurements.

  17319   Mon Nov 28 18:21:50 2022 PacoSummaryBHDBH44 prep

I checked the LSC rack to evaluate what we might need to generate 44 MHz rf in the hypothetical case we go from BH55 to BH44 (a.k.a. double RF demod scheme). There is an 11 MHz LO port labeled +16 dBm (measured 9 Vpp ~ 23 dBm actually) on the left hand side. Furthermore, there is an unused 55 MHz port labeled "Spare 55 LO". I checked this output to be 1.67 Vpp ~ +8.4 dBm. Anyways the 55 MHz doesn't look very nice; after checking it on the spectrum analyzer it seems like lower frequency peaks are polluting it so it may be worth checking the BH55 LO (labeled REFL 55) signal to see if it's better. Anyways we seem to have the two minimum LOs needed to synthesize 44 MHz in case we move forward with BH44.

[Paco, Yuta]

We confirmed the noisy 55 MHz is shared between AS55, BH55 and any other 55 MHz LOs. Looking more closely at the spectrum we saw the most prominent peaks at 11.06 MHz and 29.5 MHz (IMC and PMC nominal PM freqs). This 55 MHz LO is coming all the way from the RF distribution box near the IOO rack. According to this diagram, this 55 MHz LO should have gone through a bandpass filter; interestingly, checking the RF generation box spare 55 MHz the output is *cleaner* and displays ~ 17 dBm level... ??? Will continue investigating when we actually need this RF.

  17320   Mon Nov 28 20:14:27 2022 AnchalUpdateASCAS WFS proposed path to IMC WFS heads

In Attachment 1, I give a plan for the proposed path of AS beam into the IMC WFS heads to use them temporarily as AS WFS. Paths shown in orange are the existing MC REFL path, red for the existing AS path, cyan for the proposed AS path, and yellow for the existing IFO refl path.  We plan to overlap AS beam to the same path by installing the following new optics on the table:

  • M1 will be a new mirror mounted on a flipper mount reflecting 100% of AS beam to SW corner of the table.
  • M2 will be a new fixed mirror for steering the new AS beam path to match with MC WFS path.
  • M3 will be the existing beamsplitter used to pick off light for MC refl camera. We'll just mount this on a flipper so that it can be removed from the path. Precaution will be required to protect the CCD from high intensity MC reflection by putting on more ND filters.
  • The AS beam would need to be made approximately 1 mm in beam width. The required lenses for this would be placed between M1 and M2.

I request people to go through this plan and find out if there are any possible issues and give suggestions.

PS: Thanks JC for the photos. I got it from foteee google photos. It would be nice if these are also put into the 40m wiki page for photos of optical tables.

RXA: Looks good. I'm not sure if ND filters can handle the 1 W MC reflection, so perhaps add another flipper there. It would be good if you can measure the power on the WFS with a power meter so we know what to put there. Ideally we would match the existing power levels there or get into the 0.1-10 mW range.

Attachment 1: F5B115E5-885F-463C-9645-BB2EB73B6144_1_201_a.jpeg
  17321   Tue Nov 29 11:38:37 2022 JCHowToLSCLock Single Arm After MICH lock

I tried locking single arm this morning, but I was unable to because the triggers were set to lock strictly to MICH. Anchal explained this to me and helped me out. I figured this information would be useful and should be logged somewhere. In red, this is accessed through the IFO --> Configure. Choosing X Arm and Y Arm will change the triggers on the C1LSC page for the single arm locking (In the Black Box.) 

Attachment 1: Screen_Shot_2022-11-29_at_11.01.56_AM.png
  17322   Tue Nov 29 15:32:32 2022 AnchalUpdateBHDc1hpc model updates to support double audio dither

Many changes have been done to c1hpc to support dual demodulation at audio frequencies. We moved away with ASS style of lockin setup as the number of connections and screens required would become very large. Instead now, the demodulation is done for a selected oscillator, on a selected signal. Similarly, the demodulated signal can be further demodulated for another selected oscillator. Please familarize yourself with new screen and test the new model. The previous version of the model is kept as backup alogn with all it's medm screens, so nothing is lost. Shown as an example in the screenshot, AS1 and BS oscillators can be turned on, and BHDC_DIFF signal can be demodulated first with BS and next with AS oscillator to get the signal.

Attachment 1: Screenshot_2022-11-29_15-36-05.png
  17323   Wed Nov 30 10:48:10 2022 JCConfigurationGeneralLED Instead of Incandescent Lights

All incondescent lightsbulbs have been switched over to LED.


Electrical came by today to see the lights. The issue may be the switches, but they will come by tomorrow to solve the issue. A couple light bulbs were noticed to be out, but they no longer have incandescent lights. . . only LED. I figured this would be preffered because of the reduction on noise. I would prefer to go ahead and ask to change all the incandescent bulbs to LED bulbs. Are there any objections to this?


  17325   Wed Nov 30 14:25:34 2022 TegaUpdateCDSAMD display driver installation

After swapping out the HDD on donatella, I noticed that the display resolution was stuck on 700x400 and could not be changed. To fix this issue, I edited `/etc/apt/sources.list` to include the following:

deb http://ftp.us.debian.org/debian/ testing main non-free contrib
deb-src http://ftp.us.debian.org/debian/ testing main non-free contrib

to make `non-free` packages available in our repository, then I ran:

sudo apt-get update
sudo apt-get install firmware-amd-graphics

After the installation was complete, I did a reboot and the problem was fixed.

  17326   Wed Nov 30 18:27:07 2022 PacoUpdateGeneralMaking the Jenne laser great again


I picked up the 950 nm laser that Koji tested before and restored the Jenne laser (may long it live).

I changed a couple of things; first I took out the old laser from the hose and noted the new laser has a shorter fiber patch cable, so I replaced the hose with a flexible fiber jacket I found around the X arm storage. I also added a 14 pin DIP socket so the laser is now always showing its part no. and is easy to install/uninstall. Last but not least I replaced the black insulating material right at the output of the laser package because it was decaying badly and spread a bunch of dusty residue all over the place. I used some foam instead.  See various attachments for visual support.

I was careful, testing the connections with a cheap LED to validate the setup before I connected the laser-- this made everything smooth and I finished the work by verifying the laser is outputting light at the other end of the fiber 1x2 coupler. The PD testing with Jenne's resucitated laser should proceed normally now.

RXA: Huzzah!!

Attachment 1: PXL_20221201_004623398.jpg
Attachment 2: PXL_20221201_015752792.jpg
  17328   Wed Nov 30 20:01:08 2022 ranaSummaryCalibrationSingle arm cal with 5 lines

I don't think you need to record the excitations. They are just sine waves. The amplitude you can read off from the OSC screen. You just have to have the BEAT channel recorded and you can demod it to get the calibration. If the BEAT channel is calibrated in Hz, and you know the 40m arm length, then you're all done.



Basically, only the DARM line was recorded (DQ channs) so I modified the c1cal to store the SIG_OUT and DEMOD_I_IN1 channels for both BEATX and BEATY cal signals. This means I need to repeat this measurement. In the meantime I am also going to try and rerun calibrate the BEAT HZ transfer function.


  17329   Thu Dec 1 20:43:25 2022 AnchalSummaryCalibrationSingle arm cal with 5 lines

[Anchal, Paco]

We are doing this attempt again in following configuration:

  • PSL shutter is closed. (So IR laser is free running)
  • Beanote frequency between Y arm and Main laser is about 45 MHz.
  • Green laser on Y end is locked. Transmission is above 1.1 (C1:ALS-TRY_OUT)
  • All calibration oscillators are turned on and set to actuate ITMY. See screenshot attached.
  • The calibration model was changed to demodulate the C1:ALS-BEATY_FINE_PHASE_OUT channel insteald. We'll have DQ channels before mixing with oscillator, after mixing, and also after applying a 4th order 30 Hz butterworth filter.

Start time:

PST: 2022-12-01 20:44:23.982114 PST
UTC: 2022-12-02 04:44:23.982114 UTC
GPS: 1353991481.982114

Stop time:

PST: 2022-12-02 14:32:29.547603 PST
UTC: 2022-12-02 22:32:29.547603 UTC
GPS: 1354055567.547603

Attachment 1: SingleArmCalibration.png
  17330   Fri Dec 2 15:59:55 2022 RadhikaUpdateALSXARM green laser lock debugging

I took a transfer function measurement of the XEND PDH servo box, from servo input to piezo output [Attachment 1]. The servo gain knob was set to 100. The swept sine input was 50 mVpp, as to not saturate the servo components. I toggled the local boost on/off for these measurements. With the boost on, coherence was lost from ~100Hz-10kHz, and the saturation light indicators were flashing. I will retake this measurement shortly.

Atachment 2 is from a previous measurement of this PDH servo TF, found here. For this measurement, boost was off and the gain knob was set to 2.0. (If there is a more recent measurement than 2010, please point me to it.)

Attachment 1: XEND_green_pdhservo_TF.pdf
Attachment 2: G1_PDHbox_TF.png
  17331   Sat Dec 3 10:13:56 2022 AnchalUpdateSUSLow noise state

I've turned off HEPA fan and all lights at:

PST: 2022-12-03 10:13:03.184705 PST
UTC: 2022-12-03 18:13:03.184705 UTC
GPS: 1354126401.184705

  17332   Sat Dec 3 17:42:25 2022 AnchalUpdateASCIMC WFS Fixed for now

Today I did a lot of steps to eventually reach to WFS locking stably for long times and improving and keeping the IMC transmission counts to 14400. I think the main culprit in thw WFS loop going unstable was the offset value set on MC_TRANS_PIT filter module  (C1:IOO-MC_TRANS_PIT_OFFSET). This value was roughly correct in magnitude but opposite in sign, which created a big offset in MC_TRANS PIT error signal which would integrate by the loops and misalign the mode cleaner.

WFS offsets tuning

  • I ran C1:IOO-WFS_MASTER > Actiona > Correct WFS DC offsets script while the two WFS heads were blocked.
  • Then I aligned IMC to maximize transmission. I also made PMC transmission better by walking the input beam.
  • Then, while IMC is locked and WFS loops are off, I aligned the beam spot on WFS heads to center it in DC (i.e. zeroing C1:IOO-WFS1_PIT_DC, C1:IOO-WFS1_YAW_DC, C1:IOO-WFS2_PIT_DC, C1:IOO-WFS2_YAW_DC)
  • Then I ran C1:IOO-WFS_MASTER > Actiona > Correct WFS DC offsets script while keeping IMC locked (note the script says to keep it unlocked, but I think that moves away the beam). If we all agree this is ok, I'll edit this script.
  • Then I checked the error signals of all WFS loops and still found that C1:IOO-MC_TRANS_PIT_OUTPUT and C1:IOO-MC_TRANS_YAW_OUTPUT have offsets. I relieved these offsets by averaging the input to these filter moduels for 100s and updating the offset. This is where I noticed that the PIT offset was wrong in sign.

WFS loops UGF tuning

  • Starting with only YAW loops, I measured the open loop transfer functions (OLTFs) for each loop by simultaneously injecting gaussian noise from 0.01 Hz to 0.5 Hz using diaggui at the loop filter module excitation points and taking ration of IN1/IN2 of the filter modules.
  • Then I scaled the YAW output matrix columns to get UGF of 0.1 Hz when YAW loop was along turned on.
  • Then I tried to do this for PIT as well but it failed as even with overall gain of 0.1, the PIT loops actuate a lot of YAW motion causing the IMC to loose lock eventually.
  • So I tried locking PIT loops along with YAW loops but with 0.1 overall gain. This worked for long enough that I could get a rough estimate of the OLTFs. I scaled the columns of PIT output matrix and slowly increased the overall gain while repeating this step to get about 0.1 Hz UGF for all PIT loops too.
  • Note though that the PIT loop shape did not come out as expected with a shallower slope and much worse coherence for same amount of excitation in comparison to YAW loops. See attached plots.
  • Never the less, I was able to reach to an output matric which works at overall gain of 1. I tested this configuration for atleast 15 minutes but the loop was working even with 6 excitations happening simultaneously for OLTF measurement.
  • We will need to revisit PIT loop shapes, matrix diagonalization, and sources of noise.

OLTF measurements were done using this diaggui file. The measurement file got deleted by me by mistake, so I recreated the template. Thankfully, I had saved the pdf of the measurements, but I do not have same measurement results in the git repo.


Attachment 1: IMC_WFS_OLTF.pdf
  17333   Sun Dec 4 10:03:02 2022 ChrisUpdateOptimal ControlIMC alignment controller testing

Another five mode cleaner alignment controllers were tested last night (remotely), running in tandem with the standard controller. As before, c1ioo was burt restored to Oct 28 and the MC TRANS PIT/YAW 80dB gain filters were disabled before the test. Each test consisted of five minutes with pitch alignment uncontrolled, five minutes with the standard controller only, and twenty minutes with both controllers enabled.

The first four tests went smoothly, but the last controller (goldfish_short) repeatedly broke the lock. Eventually I got it running with an output gain of 0.5, strong enough to see the misbehavior without unlocking the mode cleaner.

GPS times for each phase of testing were the following:

  1. ichabod
    • Open loop 1354165556
    • Closed loop 1354165866
    • Policy 1354166241
  1. ichabod_2
    • Open loop 1354167462
    • Closed loop 1354167772
    • Policy 1354168113
  1. ichabod_3
    • Open loop 1354169357
    • Closed loop 1354169667
    • Policy 1354170006
  1. goldfish_long
    • Open loop 1354171297
    • Closed loop 1354171607
    • Policy 1354172022
  1. goldfish_short
    • Open loop 1354173255
    • Closed loop 1354173565
    • Policy 1354173924 (output gain 1.0, immediately unlocked the cavity)
    • Policy 1354175008 (output gain 0.1, 2 min)
    • Policy 1354175189 (output gain 0.3, 2 min)
    • Policy 1354175376 (output gain 0.5, 20 min)
  17334   Sun Dec 4 16:44:04 2022 AnchalUpdateASCIMC WFS Fixed for now

Today, I worked on WFS loop output matrix for PIT DOFs.

  • I began with the matrix that was in place before Nov 15.
  • I followed the same method as last time to fist get all UGFs around 0.06 Hz with overall gain of 0.6 on the WFS loops.
  • This showed me that MC2_TRANS_PIT loop shape matches well with the nice working YAW loops, but the WFS1 and WFS2 loops still looked flat like before.
  • This indicated that output matrix needs to be fixed for cross coupling between WFS1 and WFS2 loops.
  • I ran this script WFSoutMatBalancing.py which injects low frequency (<0.5 Hz) oscillations when the loops are open, and measures sensing matrix using error signals. I used 1000s duration for this test.
  • The direct inverse of this sensing matrix fixed the loop shape for WFS1 indicating WFS1 PIT loop is disentangled from WFS2 now.
  • Note this is a very vague definition of diagonalization, but I am aiming to reach to a workign WFS loop asap with whatever means first. Then we can work on accurate diagonalization later.
  • I simply ran the script WFSoutMatBalancing.py again for another 1000s and this time the sensing matrix mostly looked like an identity.
  • I implemented the new output matrix found by direct inversion and took new OLTF.Again though, the WFS2_PIT loop comes out to be flat. See Attachment 1.
  • Then noting from this elog post, I reduced the gain values on MC2 TRANS loops to 0.1 I think it is better to use this place to reduce loop UGF then the output matrix as this will remind us that MC2 TRANS loops are slower than others by 10 times.
  • I retook OLTF but very unexpected results came. The overall gain of WFS1_YAW and WFS2_YAW seemed to have increased by 6. All other OLTFs remained same as expected. See attachment 2.
  • To fulfill the condition that all UGF should be less than 0.1 Hz, I reduced gains on WFS1_YAW and WFS2_YAW loops but that made the YAW loops unstable. So I reverted back to all gains 1.
  • We probably need to diagonalize Yaw matrix better than it is for letting MC2_TRANS_YAW loop to be at lower UGF.
  • I'm leaving the mode cleaner in this state and would come back in an hour to see if it remains locked at good alignment. See attachment 3 for current state.

Sun Dec 4 17:36:32 2022 AG: IMC lock is holding as strong as before. None of the control signals or error signals seem to be increasing monotonously over the last one hour. I'll continue monitoring the lock.

Mon Dec 5 11:11:08 2022 AG: IMC was locked all night for past 18 hours. See attachment 4 for the minute trend.

Attachment 1: IMC_WFS_OLTF_All_Gains_1.pdf
IMC_WFS_OLTF_All_Gains_1.pdf IMC_WFS_OLTF_All_Gains_1.pdf
Attachment 2: IMC_WFS_OLTF_Nom_Gain.pdf
IMC_WFS_OLTF_Nom_Gain.pdf IMC_WFS_OLTF_Nom_Gain.pdf
Attachment 3: WFS_Loop_Configuration.png
Attachment 4: WFS_Loop_Performance.png
  17335   Mon Dec 5 12:05:29 2022 AnchalUpdateCDSc1sus2 all FE models crashed spontaneously

Just a few minutes ago, all models on FE c1sus2 crashed. I'm attaching some important files that can be helpful in investigating this. CDS upgrade team, please take a look.

I fixed this by running following on c1sus2:

controls@c1sus2:~$ rtcds restart --all
Attachment 1: c1sus2Failed.txt
controls@c1sus2:~$ rtcds status
found host specific environment file '/etc/advligorts/systemd_env_c1sus2'
Kernel Module Status
mbuf                   24576  40 c1bac,c1x07,c1su2,c1hpc,c1su3
gpstime                40960  5 c1bac,c1x07,c1su2,c1hpc,c1su3
dis_kosif             131072  5 dis_ix_ntb,dis_intel_dma,dis_ix_dma,dis_sisci,dis_irm
dis_ix_ntb            307200  1 dis_irm
dis_ix_dma             36864  1 dis_ix_ntb
dis_irm               462848  4 dolphin_proxy_km,dis_sisci
dis_sisci            1093632  0
... 28 more lines ...
Attachment 2: c1sus2Failed.png
  Draft   Mon Dec 5 16:24:45 2022 AnchalUpdateASCIMC WFS servo diagnosis

Also reply to: 40m/17255

I ran the togl

ELOG V3.1.3-