40m QIL Cryo_Lab CTN SUS_Lab TCS_Lab OMC_Lab CRIME_Lab FEA ENG_Labs OptContFac Mariner WBEEShop
 40m Log, Page 128 of 335 Not logged in
ID Date Author Type Category Subject
13168   Sat Aug 5 11:04:07 2017 gautamUpdateSUSMC1 glitches return

See Attachment #1, which is full (2048Hz) data for a 3 minute stretch around when I saw the MC1 glitch. At the time of the glitch, WFS loops were disabled, so the only actuation on MC1 was via the local damping loops. The oscillations in the MC2 channels are the autolocker turning on the MC2 length tickle.

Nikhil and I tried the usual techniques of squishing cables at the satellite box, and also at 1X4/1X5, but the glitching persists. I will try and localize the problem this weekend. This thread details investigations the last time something like this happened. In the past, I was able to fix this kind of glitching by replacing the (high speed) current buffer IC LM6321M. These are present in a two places: Satellite box (for the shadow sensor LED current drive), and on the coil driver boards. I think we can rule out the slow machine ADCs that supply the static PIT and YAW bias voltages to the optic, as that path is low-passed with a 4th order filter @1Hz, while the glitches that show up in the OSEM sensor channels do not appear to be low-passed, as seen in the zoomed in view of the glitch in Attachment #2 (but there is an LM6321 in this path as well).

Attachment 1: MC1_glitch_Aug42017.png
Attachment 2: MC1_glitch_zoomed.png
13178   Wed Aug 9 15:15:47 2017 gautamUpdateSUSMC1 glitches return

Happened again just now, although the characteristics of the glitch are very different from the previous post, its less abrupt. Only actuation on MC1 at this point was local damping.

Attachment 1: MC1_glitch.png
13418   Wed Nov 8 14:28:35 2017 gautamUpdateGeneralMC1 glitches return

There hasn't been a big glitch that misaligns MC1 by so much that the autolocker can't lock for at least 3 months, seems like there was one ~an hour ago.

I disabled autolocker and feedback to the PSL, manually aligned MC1 till the MC_REFL spot looked right on the CCD to me, and then re-engaged the autolocker, all seems to have gone smoothly.

Attachment 1: MC1_glitchy.png
Attachment 2: 6AFDA67D-79B1-469C-A58A-9EC5F8F01D32.jpeg
13284   Fri Sep 1 08:25:08 2017 SteveUpdateSUSMC1 glitching

MC1, MC2 and MC3 damping turned off to see glitching action at 9:57am

 Quote: There was a pretty large glitch in MC1 about an hour ago. The misalignment was so large that the autolocker wasn't able to lock the IMC. I manually re-aligned MC1 using the bias sliders, and now IMC locks fine. Attached is a 90 second plot of 2K data from the OSEMs showing the glitch. Judging from the wall StripTool, the IMC was well behaved for ~4 hours before this glitch - there is no evidence of any sort of misalignment building up, judging from the WFS control signals.

Attachment 1: MC1glitching.png
Attachment 2: MC1kicks.png
13286   Fri Sep 1 16:27:39 2017 gautamUpdateSUSMC1 glitching

I re-enabled the MC SUS damping and IMC locking for some IFO work just now.

 Quote: MC1, MC2 and MC3 damping turned off to see glitching action at 9:57am

13426   Tue Nov 14 08:54:37 2017 SteveUpdateIOOMC1 glitching
Attachment 1: MC1_glitching.png
13253   Fri Aug 25 11:11:26 2017 gautamUpdateGeneralMC1 kicked again

Looks like MC1 got another big kick just under 4 hours ago. None of the other optics show any evidence of a glitch so it seems unlikely that this was some sort of global event. It's been well behaved for ~2weeks now. IMC was unlocked. I manually re-aligned MC1, at which point the autolocker was able to lock the IMC.

Looking at this plot, it seems that LR and UL coils seem to have the largest kicks. UR barely saw it. Not sure what (if anything) to make of this - apparently the optic moved by ~20urad with the UR magnet approximately the pivot.

Attachment 1: MC1_glitch.png
13283   Thu Aug 31 21:40:24 2017 gautamUpdateGeneralMC1 kicked again

There was a pretty large glitch in MC1 about an hour ago. The misalignment was so large that the autolocker wasn't able to lock the IMC. I manually re-aligned MC1 using the bias sliders, and now IMC locks fine. Attached is a 90 second plot of 2K data from the OSEMs showing the glitch. Judging from the wall StripTool, the IMC was well behaved for ~4 hours before this glitch - there is no evidence of any sort of misalignment building up, judging from the WFS control signals.

Attachment 1: MC1_glitch.png
16159   Tue May 25 10:22:16 2021 Anchal, PacoSummarySUSMC1 new input matrix calculated and uploaded

The test was succesful and brought back the IMC to lock point at the end.

We calculated new input matrix using same code in scripts/SUS/InMatCalc/sus_diagonalization.py . Attachment 1 shows the results.

The calculations are present in scripts/SUS/InMatCalc/MC1.

We uploaded the new MC1 input matrix at:

Unix Time = 1621963200

 UTC May 25, 2021 17:20:00 UTC Central May 25, 2021 12:20:00 CDT Pacific May 25, 2021 10:20:00 PDT

GPS Time = 1305998418

This was done by running python scripts/SUS/general/20210525_NewMC1Settings/uploadNewConfigIMC.py on allegra. Old IMC settings (before Paco and I started workin on 40m) can be restored by running python scripts/SUS/general/20210525_NewMC1Settings/restoreOldConfigIMC.py on allegra.

Everything looks as stable as before. We'll look into long term trends in a week to see if this helped at all.

Attachment 1: SUS_Input_Matrix_Diagonalization.pdf
15434   Sun Jun 28 15:30:52 2020 gautamUpdateSUSMC1 sat-box de-lidded

Judging by the summary pages, some 18 hours after this change was made and the board re-installed, the MC1 shadow sensors began to report frequent glitches. I can't think of a plausible causal connection, especially given the 18 hour time lag, but also hard to believe there isn't one? As a result, the IMC is no longer able to stay locked for extended periods of time. I did the usual cable squishing, and also took off the lid to see if that helps the situation.

While the reduced series resistance means there is more current flowing through the slow path

1. There isn't actually an increase in the net current flowing through the satellite box - this change just re-allocates the current from the fast path to the slow path, but by the time it reaches the satellite box, the current is flowing through the same conductor.
2. afaik, the current buffers on the coil driver aren't overdriven - they are rated for 300 mA. No individual coil is drawing more than 30 mA.
3. the resistors themselves should be running sufficiently below their rated power of 3W (I estimate 2.5 V ^2 / 100 ohms ~ 60 mW).
4. The highest current should be through the UL and LR coils according to the voltage outputs from the Acromag. But the UL coil doesn't show significant glitching, and the LL one does despite drawing negligible DC current.

The attached FLIR camera image re-inforces what we already know, that the thermal environment inside the satellite box is horrible. The absolute temperature calibration may be off, but it was difficult to touch the components with a bare finger, so I'd say its definitely > 70 C.

 Quote: I implemented this change today. We only had 100 ohm, 3W resistors in stock (no 200 ohm with adequate power rating). Assuming 10 V is dropped across this resistor, the power dissipation is V^2/R ~ 1 W, so we should have sufficient margin. DCC entry has been updated with new schematic and photo of the component side of the board. Note that the series resistance of the fast actuation path was untouched.
Attachment 1: 20200628T144138.jpg
15435   Sun Jun 28 16:29:58 2020 ranaUpdateSUSMC1 sat-box de-lidded

does the FLIR have an option to export image with a colorbar?

How about just leave the lid open? or more open? I don't know what else can be done in the near term. Maybe swap with the SRM sat box to see if that helps?

15436   Sun Jun 28 17:36:35 2020 gautamUpdateSUSMC1 sat-box de-lidded

Hmm I can't seem to export with the colorbar, might be just my phone though. I tried to add some "cursors" with the temperature at a few spots, but the font color contrast is poor so you have to squint really hard to see the temperatures in the photo I attached.

I'll leave the MC1 box open overnight and see if that improves the situation, and if not, I'll switch in the SRM satellite box tomorrow.

 Quote: does the FLIR have an option to export image with a colorbar? How about just leave the lid open? or more open? I don't know what else can be done in the near term. Maybe swap with the SRM sat box to see if that helps?
15438   Mon Jun 29 11:55:46 2020 gautamUpdateSUSMC1 sat-box de-lidded

There was no improvement to the situation overnight. So, I did the following today:

1. Ramped bias voltages for SRM and MC1 to 0, shutdown watchdogs.
2. Switched SRM and MC1 satellite boxes. The SRM satellite box lid was opened, while the MC1 lid was left open. The boxes have also been re-labelled lest there be some confusion about which box belongs where.
3. Restored watchdogs and bias voltages. Curiously, the MC1 optic now only requires half the bias voltages it did before to have the correct DC alignment for the optic. The Satellite box is just supposed to be a passive conduit for the drive current, so this is indicative of some PCB traces/cabling being damaged inside what was previously the MC1 satellite box?

IMC is now locked again, I will monitor for glitching/stability.

Update 6pm PDT: as shown in Attachment #1, there is a huge difference in the stability of the lock after the sat box swap. Let's hope it stays this way for a while...

 Quote: I'll leave the MC1 box open overnight and see if that improves the situation, and if not, I'll switch in the SRM satellite box tomorrow.
Attachment 1: SatBoxSwap.jpg
15440   Mon Jun 29 20:30:53 2020 KojiUpdateSUSMC1 sat-box de-lidded

Sigh. Do we have a spare sat box?

15712   Mon Dec 7 11:25:31 2020 gautamUpdateSUSMC1 suspension glitchy again

The MC1 suspension has begun to show evidence of glitches again, from Friday/Saturday. You can look at the suspension Vmon tab a few days ago and see that the excess fuzz in the Vmon was not there before. The extra motion is also clearly evident on the MCREFL spot. I noticed this on Saturday evening as I was trying to recover the IMC locking, but I thought it might be Millikan so I didn't look into it further. Usually this is symptomatic of some Satellite box issues. I am not going to attempt to debug this anymore.

16138   Thu May 13 11:55:04 2021 Anchal, PacoUpdateSUSMC1 suspension misbehaving

We came in the morning with the following scene on the zita monitor:

The MC1 watchdog was tripped and seemed like IMC struggled all night with misconfigured WFS offsets. After restoring the MC1 WD, clearing the WFS offsets, and seeing the suspension damp, the MC caught lock. It wasn't long before the MC unlocked, and the MC1 WD tripped again.

We tried few things, not sure what order we tried them in:

• Letting suspension loops damp without the WFS switched on.
• Letting suspension loops damp with PSL shutter closed.
• Restoring old settings of MC suspension.
• Doing burt restore with command:
burtwb -f /opt/rtcds/caltech/c1/burt/autoburt/snapshots/2021/May/12/08:19/c1mcsepics.snap -l /tmp/controls_1210513_083437_0.write.log -o /tmp/controls_1210513_083437_0.nowrite.snap -v <

Nothing worked. We kept seeing that ULPD var on MC1 keeps showing kicks every few minutes which jolts the suspension loops. So we decided to record some data with PSL shutter closed and just suspension loops on. Then we switched off the loops and recorded some data with freely swinging optic. Even when optic was freely swinging, we could see impulses in the MC1 OSEM UL PD var which were completely uncorrelated with any seismic activity. Infact, last night was one fo teh calmer nights seismically speaking. See attachment 2 for the time series of OSEM PD variance. Red region is when the coil outputs were disabled.

### Inference:

• We think something is wrong with the UL OSEM of MC1.
• It seems to show false spikes of motion when there is no such spike present in any other OSEM PD or the seismic data itself.
• Currently, this is still the case. We sometimes get 10-20 min of "Good behavior" when everything works.
• But then the impulses start occuring again and overwhelmes the suspension loops and WFS loops.
• Note, that other optic in IMC behaved perfectly normally throughout this time.
• In the past, it seems like satellite box has been the culprit for such glitches.
• We should look into debugging this as ifo is at standstill because of this issue.
• Earlier, Gautum would post Vmon signals of coil outputs only to show the glitches. We wanted to see if switching off the loops help, so we recorded OSEM PD this time.
• In hindsight, we should probably look at the OSEM sensor outputs directly too rather than looking at the variance data only. I can do this if people are interested in looking at that too.
• We've disabled the coil ouputs in MC1 and PSL shutter is off.

Edit Thu May 13 14:47:25 2021 :

Added OSEM Sensor timeseries data on the plots as well. The UL OSEM sensor data is the only channel which is jumping hapazardly (even during free swinging time) and varying by +/- 30. Other sensors only show some noise around a stable position as should be the case for a freely suspended optic.

Attachment 2: MC1_Glitches_Invest2.pdf
14836   Thu Aug 8 12:01:12 2019 gautamUpdateIOOMC1 suspension oddness

At ~1am PDT today, all the MC1 shadow sensor readbacks (fast CDS channels and Slow Acromag channels, latter not shown here) went to negative values. Of course a negative value makes no sense. After ~3 hours, they came back to positive values again. But since then, the shadow sensor RMS noise has been significantly higher in the >20 Hz band, and there are frequent glitches which kick the suspension. The IMC has been having trouble staying locked. I claim that this has to do with the Satellite box.

No action being taken now while I work on the ALS. In the past the problem has fixed itself.

Attachment 1: MC1_suspension.png
Attachment 2: MC1_suspension.pdf
14842   Mon Aug 12 19:58:23 2019 gautamUpdateIOOMC1 suspension oddness

Repair plan:

1. Get "spare" satellite box working --- Chub
• According to elog14441, this box has flaky connectors which probably need to be remade
2. Re-make the 64-pin IDC crimped connection on the cable from the coil driver board to sat. box, at the Satellite box end --- Chub and gautam

Any other ideas? The problem persists and it's annoying that the IMC cannot be locked.

2011   Mon Sep 28 02:24:05 2009 ranaUpdateLockingMC1/3 Dewhitening found OFF: Turned back ON

While trying to make the OAF work, I found that the XYCOM switches for MC1/3 have been set in the bad way for awhile. This means that the hardware filters were bypassed and that MC1 & MC3 were moving around too much at high frequency and possibly causing trouble with the locking. I have put them back into the default position.

On Friday, Jenne and I were playing around with turning the dewhitening off/on to see if it efffected the OAF stability. At the time, I didn't pay too much attention to what the state was. Looks like it was in the wrong state (hardware bypassed) when we found it. For the OAF work, we generally want it in that bypassed state, but its bad because it makes noise in the interferometer. The bits in question are bits 16-23 on the XYCOM screen.

I have updated the snapshot and set the screen in the appropriate settings. I used a swept sine measurement to verify the filter state. In the attached plot, green corresponds to XYCOM green and red corresponds to red.

Attachment 1: C1SUS_SRM_XYCOM1.png
Attachment 2: Untitled.png
9521   Mon Jan 6 18:32:17 2014 RANAUpdateIOOMC1/3 kicked this morning at 8:30

The trend shows a big jolt to the MC1/3 pointing this morning at 8:30.

### Was anyone working anywhere near there today? There is no elog.

If not, we will have to put a 'no janitor' sign on all of the 40m doors permanently to prevent mops misaligning our interferometer.

Attachment 1: kicked.png
9522   Mon Jan 6 20:52:09 2014 JenneUpdateIOOMC1/3 kicked this morning at 8:30

When I got in this morning at 9-something (9:45 maybe?), Steve was taking dust photos on the AS table, of the MC Refl path.  Other than that, I don't have any information.

Also, Tuesday is our traditional janitor day, so I'm hesitant to put our blame there.  (I think we've kept Tuesdays, even though we're on a less-often schedule....Steve will have to correct me if I'm wrong on this).

5932   Thu Nov 17 22:24:19 2011 DenUpdateAdaptive FilteringMC1_COIL

Analyzing coherence between MC length and signals on MC1, MC2 and MC3 coils we have noticed that MC1 COIL signal is not coherent to MC length at all at interesting frequencies 0.1 - 1 Hz.

We try to explain this phenomena.

Attachment 1: MC1COIL-crop.pdf
Attachment 2: MC2COIL-crop.pdf
Attachment 3: MC3COIL-crop.pdf
5934   Thu Nov 17 23:44:48 2011 DenUpdateIOOMC1_SENSOR

We've found that one of the  MC1_SENSORS does not work properly.

See the figure.

Attachment 1: MCSENSORS.pdf
5935   Thu Nov 17 23:47:43 2011 DenUpdateIOOMC1_SENSOR

The most interesting plot did not uploaded in the previous elog.

Attachment 1: MC1SENSOR-crop.pdf
7453   Mon Oct 1 07:25:48 2012 SteveUpdateSUSMC2 & ETMX sus damping restored
11447   Mon Jul 27 16:47:53 2015 ericqUpdateIOOMC2 -> MCL Actuator TF

Our noise cancellation SURFS will be doing online subtraction on the mode cleaner length, among other things.

I made a measurement of the MC2 actuator transfer function by injecting noise from 1-100Hz into LSC_MC2_EXC for about 15 minutes, then estimating the TF from MC2_OUT to IOO_MC_L with CSD/PSD. The inverse of this TF will be applied to their Wiener target data to give us the direct subtration filter we want.

I figured I would post the results here for posterity. The last time this seems to have been done is in ELOG 5900. There are some differences found here, the effective Q of the 1Hz pendulum resonance seems lower, and the behavior above 20Hz has definitely changed.

IIR fits will be done by one of the SURFs to be used in their Wiener filter calculations.

Data attached!

Attachment 1: mc2_2_mcl.png
Attachment 2: MC2_2_MCL_TF.txt.zip
11539   Fri Aug 28 20:15:49 2015 ranaUpdateIOOMC2 -> MCL Actuator TF
 qQuote: I made a measurement of the MC2 actuator transfer function by injecting noise from 1-100Hz into LSC_MC2_EXC for about 15 minutes, then estimating the TF from MC2_OUT to IOO_MC_L with CSD/PSD. The inverse of this TF will be applied to their Wiener target data to give us the direct subtration filter we want.

I think what happened here is you forgot to undo the MC_F whitening filter which is the Generic Pentek Interface board next to the MC servo board. I suggest you guys measure this on Monday so you can correctly estimate the MC length noise. And then perhaps undo the whitening in the anti-whitening filter of this filter bank so that the signal which is recorded is in units of kHz.

This should allow your online subtraction filter to be more correct: roughly speaking, the phase shift below a pole or zero is going to be 45*(f/fp) deg. Since we expect there to be 2 zeros at 15 Hz, it would be 9 deg phase shift at 1.5 Hz and limit the subtraction to ~80%.

11548   Mon Aug 31 07:49:11 2015 ericqUpdateIOOMC2 -> MCL Actuator TF
 Quote: I think what happened here is you forgot to undo the MC_F whitening filter which is the Generic Pentek Interface board next to the MC servo board. I suggest you guys measure this on Monday so you can correctly estimate the MC length noise. And then perhaps undo the whitening in the anti-whitening filter of this filter bank so that the signal which is recorded is in units of kHz. This should allow your online subtraction filter to be more correct: roughly speaking, the phase shift below a pole or zero is going to be 45*(f/fp) deg. Since we expect there to be 2 zeros at 15 Hz, it would be 9 deg phase shift at 1.5 Hz and limit the subtraction to ~80%.

While it is true that the whitening filter was incorrectly handled, I don't think this should change the subtraction performance since the MC_L data used for the Wiener filter training was also taken without undoing the whitening filter.

11462   Thu Jul 30 02:06:20 2015 IgnacioUpdateIOOMC2 <-> MCL Actuator TF fitted

Eric downloaded MC2 to MCL transfer function data (H) as well as its inverse, MCL to MC2 (Hinv). He also downloaded new MCL and MC2 data.

I used vectfit to fit the MC2 to MCL transfer function,

The ZPK parameters for this fit were,

Zeros              1278.36719876674 + 0.00000000000000i                    -100.753249679343 + 0.00000000000000i                    -18.6014192997845 + 13.0294910760217i                    -18.6014192997845 - 13.0294910760217i

Poles              -1.11035771175328 + 7.03549674098987i                    -1.11035771175328 - 7.03549674098987i                    -18.8655320274072 + 0.00000000000000i                    -690.294337433234 + 0.00000000000000i

Gain               0.00207206036014220

Using the above vectfit model, I filtered the raw MC2 signal to get 'MCL'. The PSD's of the raw MCL data and the filtered MC2 result is shown below,

The lack of accuracy of the transfer function at replicating MCL at frequencies lower than 0.7Hz is expected, the vectfit model I generated fails to follow accurately the raw transfer function data. My question: Does it matter? My guess: Probably not. In order to mitigate seismic noise from the mode cleaner we are mainly concerened with the 1-3 Hz region.

I also used vectfit to fit the transfer function for MCL to MC2,

This one was harder to fit accurately for some reason, I could do it with four pairs of zeros and poles but it took some preweighting.

The ZPK parameters for the above fit were,

Zeros              0.173068278283995 + 0.00000000000000i                    0.995140531040529 + 0.0268079821980457i                    0.995140531040529 - 0.0268079821980457i                    0.894476816129099 + 0.00000000000000i

Poles              -19.9566906920707 + 18.0649464375308i                    -19.9566906920707 - 18.0649464375308i                    -109.275971483008 + 0.00000000000000i                    -1791.88947801703 + 0.00000000000000i

Gain               1237.46417532120

Similarly, using this ZPK model, I filtered the MCL signal to get 'MC2'. I plotted the PSD for the MC2 signal and the filtered MCL to get,

Again, the lack of accuracy of the filtered MC2 at replicating MCL below 0.7 Hz and above 12 Hz is due to the inverse transfer function failing to converge in these ranges.

Attachment 1: TF_BODE.png
Attachment 2: MC2_2_MCL.png
Attachment 3: TF_INV_BODE.png
Attachment 4: MCL_2_MC2.png
12996   Wed May 17 11:10:31 2017 SteveUpdateCamerasMC2 CCD video camera back in place

Olympus camera is removed and our old CCD camera is back to monitor the face of MC2

 Quote: Olympus SP570 UZ - without  IR blocker, set up as Atm.3  Camera distance to MC  face ~85 cm,  IOO-MC_TRANS_SUM 16,300 counts, Lexan cover on not coated viewport. Image mode: RAW + JPG,  M-costum,  manual focus,  Lens: Olympus 4.6 - 92 mm, f2.8 - 4.5,  Apeture: F2.8 - 8,  Image pick up device: 1/2.33" CCD (primary color filter) Atm.1,       212k.jpg of raw 15 MB,  exp 0.025s,   apeture 2.97,  f 4.6,   iso 64,   Atm.2,        Copied through my Cannon S100  (  3.3 MB.jpg of raw from UFraw photo shop )I will look up the original raw file for details.

15048   Tue Nov 26 13:33:33 2019 YehonathanUpdateCamerasMC2 Camera rotated by 90 degrees

MC2 analog camera was rotated by 90 degrees. Orientation correctness was verified by exciting the MC2 Yaw degree of freedom.

Attached before and after photos of the camera setup.

Attachment 1: MC2AnalogCameraAfter.jpg
Attachment 2: MC2AnalogCameraBefore.jpg
15982   Wed Mar 31 22:58:32 2021 Anchal, PacoUpdateSUSMC2 Coil Balancing Test

A cross-coupling test has been set to trigger at 05:00 am on April 1st, 2021. The script is waiting on tmux session 'cB' on pianosa. /scripts/SUS/OutMatCalc/MC2crossCoupleTest.py is being used here. The script will switch on oscillator in LOCKIN1 of MC2 at 13 Hz and 200 counts and would send it along the POS, PIT and YAW vectors on output matrix one by one, each for 2 minutes. It will take data from C1:IOO-MC_F_DQ, C1:IOO-MC_TRANS_PIT_ERR and C1:IOO-MC_TRANS_YAW_ERR and use it to measure 'sensing matrix' S. Sensing matrix S is defined as the cross-coupling between excited and sensed DOF and we ideally want it to be an identity matrix. The code will use the measured S to create a guess matrix A which on being multiplied by ideal coil output matrix would give us a rotated coil output matrix O. This guess O will be applied and the measurement will be repeated. On each iteration, next, A matrix is defined by:

$A_{k+1} = (1 + \beta) A_k - \beta S_k A_k$

This recursive algorithm converges A to the inverse of initial S. The above relation is derived by noticing that in steady state $A S = \mathrm{I} \Rightarrow A = A S A \Rightarrow A = A - \beta(A S A - A)$. I've taken this idea from a mathematics paper I found on some more complex stuff (c.f. https://doi.org/10.31219/osf.io/yrvck).

At each iteration, all three matrices A, O and S will be stored in a text file for analysis later.

The code has the error-catching capability and would restore the optic to the status quo if an error occurs or watchdogs trip due to earthquakes.

15984   Thu Apr 1 13:56:49 2021 Anchal, PacoUpdateSUSMC2 Coil Balancing Test Results

The coil balancing attempt failed. The off-diagonal values in the measured sensing matrices either remained the same or increased.

The attempt in the morning was too slow. By the time we reached, it had reached to iteration 7 only and still nowhere near optimum sensing matrix had reached. We still needed to see if the optimum would eventually reach if more iterations happened.

So we worked a bit on speeding up the data loading process and then ran the code again which now was running much faster. Still within 1 hr or so, we saw it had reached to iteration 7 with no sign of sensing matrix getting any better.

<Paco left for vaccination>

To determine if the method would work in principle, I decided to stop the current run and start with a 0.5 Hz bandwidth run (so about 7 averages with 8s duration data and welch method). This completed 20 iterations before Gautum came. But it was clear now that the method is not converging to a better solution. Need to find a bug in the implementation of the algorithm mentioned in last post or find a better algoritm.

Attachment 1 is the plot of how the sensing matrix's distance from the identity matrix increased over iterations in the last run.

Attachment 2 is the plot for different off-diagonal terms in the sensing matrix. It is clear that POS->PIT,YAW coupling is not being measured properly as it remains constant.

Attachment 3 Gautum told us that there is some naming error in nds and MC_TRANS_PIT/YAW can be read through C1:IOO-MC_TRANS_PIT_ERR and C1:IOO-MC_TRANS_YAW_ERR channels instead. To test if they indeed point to same values, we did a test of exciting YAW degree through LOCKIN1 and seeing if the peaks are visible in the channels. This was also done to give Radhika an opportunity to do something I could confidently mentor about. and to experience using diaggui.

Attachment 1: SDistanceFromIdentity.pdf
Attachment 2: SmatIterations.pdf
Attachment 3: TestingExcitationAlongYAW.pdf
15985   Thu Apr 1 18:01:06 2021 Anchal, PacoUpdateSUSMC2 Coil Balancing Test Results Success??

After fixing a few things we felt were wrong in our implementation of the algorithm, we ran the coil balancing for 12 iterations with just 11s per excitation and still taking CSD with 0.1 Hz bandwidth. This time we saw the distance of sensing matrix from identity going down.

### Performance Analysis

• Attachment 1 shows the trend of distance of Sensing matrix from identity matrix over iterations.
• Attachment 2 shows the trend of off-diagonal terms in sensing matrix over iterations.
• Attachment 3 shows the ASD for the different sensed DOF when excited in different DOFs with the new output matrix. This is the better truth of what happened by the end. The true sensing matrix is proportional to the peak heights in this plot. Rows are different sensed DOFs (POS, PIT, YAW) and columns are excited DOFs (POS, PIT, YAW). The black dotted curves are ASD when no excitation was present.

### Next step

• We want to run it for longer, more iterations and more duration to get better averaging. Hopefully, this will do a better job. We'll try running this new code tomorrow at 5:00am.
• We'll work on using uncertainties of measured data.
• Use awg to excite all DOF together at different frequencies and make the code faster.
Attachment 1: SDistanceFromIdentity.pdf
Attachment 2: SmatIterations.pdf
Attachment 3: MC2CoilCrossCoupling_opt.png
15987   Thu Apr 1 18:48:45 2021 gautamUpdateSUSMC2 Coil Balancing Test Results Success??

In these results, can you also include the new matrix and what the relative imbalances were?

### Proof-of-principle

• We excited PIT and YAW dofs using LOCKIN1 in MC2 on Monday.
• We analyzed this data in a simple analysis explained in Attachment 1 python notebook (also present at /users/anchal/20210323_AnalyszingCoilActuationBalance/)
• Basically, we tried to estimate the cross coupling in 2x2 matrix from actuated DOF to sensed DOF, inverted it, and applied it to output matrix to undo the cross coupling.
• Attachments 2 and 3 show how much we performed in undoing the cross coupling.
• The ratio of 13.5 Hz peaks shows how much coupling is still present.

### Going towards 3x3 Coil balancing:

• In a conversation with Rana yesterday, we understood that we can use MC_F data as POS sensing data out of the loop.
• So today, we repreated the excitation measurements while exciting POS, PIT and YAW dofs from LOCKIN1 on MC2 and measuring C1:IOO-MC_F, C1:SUS-MC2_ASCPIT_IN1 and C1:SUS-MC2_ASCPIT_IN2.
• Data from MC_F is converted into units of um using factor 9.57e-8 um/Hz.
• We changed the excitation amplitude in order to see cross coupling peaks when they were not visible with low excitation.
• The data was measured while new calculated input matrix was loaded which from our calculations diagonalized the sensing matrix of OSEMs.

### Some major changes:

• We actually found that the C1:SUS-MC2_ASCPIT_IN1 showed a broadband increase in noise today (from Monday) by factor of about 100 in range 0-20 Hz.
• We were not sure why this changed from our 22nd March measurement.
• We checked if the gain values in the loops changed in alst 3 days, but they didn't.
• Then we realized that the WFS1_PIT and WFS2_PIT switched that we turned ON on Tuesday were the only changes that were made in the loop.
• We turned back OFF C1:IOO-WFS1_PIT_SW1 and C1:IOO-WFS2_PIT_SW1. This actually brought back the noise level of C1:SUS-MC2_ASCPIT_IN1 down to what it was on Monday.

Attachment 1: CoilActuationBalancing.ipynb.tar.gz
Attachment 2: MC2_CoilBalancePITnorm_excSamePIT.pdf
Attachment 3: MC2_CoilBalanceYAWnorm_excSameYAW.pdf
Attachment 4: 20210325_IMC_CoilBalance.tar.gz

[Paco, Anchal]

• Today we spent the morning testing the scripts under ~/c1/scripts/SUS/OutMatCalc/ that automate the procedure (which we have been doing by hand) and catch any "bad" behavior instances that we have identified. In such instances, the script sets up to restore the IMC state smoothly.
• After some testing and debugging, we managed to get some data for MC2 using ~/c1/scripts/SUS/OutMatCalc/getCrossCouplingData.py

We ran the coil balancing procedure 4 times while iterating through the output matrix optimization.

Attachment 1, pages 1 to 4 show the progression of cross coupling from current output matrix (which is theoretical ideal) to the latest iteration. We plot the sensed DOF ASD which we used to determine the cross coupling when different excitations are fed using the LOCKIN1 feeding 13Hz oscillation of 200 counts amplitude along the vector defined in output matrix. That means, when we change the output matrix, in subsequent tests, we alos change the exciation direction along with it.

Unfortunately, we don't see a very good optimizations over iterations. While we see some peaks going down in sensed PIT and sensed POS (through MC_F), we rather see an increase in cross coupling in the sensed YAW.

### Scripts:

• For running the tests, we used script in scripts/SUS/OutMatCalc/crossCoupleTest.py and wrote commanding scripts in the /users/anchal/20210329_MC2_TestingNewOutMat .
• The optimization code is at in scripts/SUS/OutMatCalc/outMatOptimize.py.
• The code reads sensed DOF data using nds2 and calculated cross spectral density among the sensed DOF at the excitation frequencies.
• This is normalized by the power spectral density of reference data (no excitation) and power spectral density of position data to create a TF estimate.
• The real values of the sensor matrix thus created is used to get the inverse matrix.
• The inverse matrix is first normalized along each row by diagonal elements to get 1 there and then multiplied by previous output matrix to create a new output matrix.
• I guess, reading the code will be a better way of understanding this algorithm.
Attachment 1: MC2OutMatCrossCouple_Old-to-It3.pdf
Attachment 2: 20210329_MC2_CrossCoupleTest.tar.gz

For this technique to work, (i) the WFS loops must be well tuned and (ii) the beam must be well centered on MC2. I am reasonably certain neither is true. For MC2 coil balancing, you can use a HeNe, there is already one on the table (not powered), and I guess you can use the MC2 trans QPD as a sensor, MC won't need to be locked so you can temporarily hijack that QPD (please don't move anything on the table unless you're confident of recovering everything, it should be possible to do all of this with an additional steering mirror you can install and then remove once your test is done). Then you can do any variant of the techniques available once you have an optical lever, e.g. single coil drive, pringle mode drive etc to do the balancing.

I think Hang had some technique he tried recently as well, maybe that is an improvement.

I think there's been some mis-communication. There's no updated Hang procedure, but there is the one that Anchal, Paco and I discussed, which is different from what is in the elog.

We'll discuss again, and try to get it right, but no need to make multiple forks yet.

16063   Wed Apr 21 11:38:27 2021 Anchal, PacoUpdateSUSMC2 Damping Gains Optimized

We did a step response test with MC2 Suspensoin Damping Gains and optimized them to get <5 oscillations in ringdown.

### Procedure:

• We uploaded the diagonalized input matrix.
• We uploaded the coil balancing gains at high frequencies found in 16054.
• We applied Eg2CtQ1 filter module for DC gain balancing foun inf 16055.
• We set TRAMP to 0 in C1:SUS-MC2_SUSPOS_TRAMP, C1:SUS-MC2_SUSPIT_TRAMP, and C1:SUS-MC2_SUSYAW_TRAMP.
• We played with offsets to get a good step height. Finally we used:
• C1:SUS-MC2_SUSPOS_OFFSET: 3000
• C1:SUS-MC2_SUSPIT_OFFSET: 100
• C1:SUS-MC2_SUSYAW_OFFSET: 100
• We looked at channels C1:SUS-MC2_SUSPOS_INMON, C1:SUS-MC2_SUSPIT_INMON, and C1:SUS-MC2_SUSYAW_INMON on a striptool screen to see the step response of the switching on/off of the offsets.
• We tried to decrease/increase gain to get <5 oscillations during ringdown due to the step inputs.
• Restored everything back to old values at the end.

### Results:

• Gain in POS was found to be already good. In PIT and YAW we changed the gains from 10 -> 30.
• Attachment 1 shows the striptool screen when offset was switched ON/Off in POS, PIT and YAW respectively after appling the optimized gains.
• Attachment 2 shows the same test with old gains for comparison.

In the afternoon, we'll complete doing the above steps for MC1 and MC3. Their coil balancing has not been done on DC so, it is bit non-ideal right now. We'll look into scripting this process as well.

Attachment 1: MC2_DampGainStepTestWithNewGains.png
Attachment 2: MC2_DampGainStepTestWithOldGains.png
16086   Mon Apr 26 18:55:39 2021 Anchal, PacoUpdateSUSMC2 F2A Filters Tested

Today we tested the F2A filters created from the DC gain values listed in 16066.

### Filters:

• For a DC gain $G_{DC}$ required for balancing the coil at DC and $f_0$ being the resonance frequency of the mode (POS in this case), we calculate the filter using:
$\frac{1 + i \frac{f_z}{f Q} - \frac{f_z^2}{f^2}}{1 + \frac{f_0}{f} - \frac{f_0^2}{f^2}}$where $f_z = f_0 \sqrt{G_{DC}}$.
• Attachment 1 shows the motivation for choosing the resonant frequency in the formula above. It makes gain at DC as $G_{DC}$ while keeping AC gain as 1.
• Attachment 2 shows the transfer functions of the filters uploaded.
• Filters are named Eg2CtQ3, Eg2CtQ7 and Eg2CtQ10 for Q=3,7,10 filters respectively. (Named for Eigenmode Basis to Cartesian Basis conversion filters, aka F2A filters).

### Testing procedure:

• We uploaded the new input matrix listed in 16066.
• We then uploaded the coil output gains (AC gains) that are also listed in 16066.
• Then we reduced the C1:IOO-WFS_GAIN to 0.05 (by a factor of 20).
• Rana asked us to test the WFS sensors' impulse response to observe a minimum 10s decay to ensure that the UGF of WFS control loops is at or below 0.1 Hz.
• We were unable to have any effect on this decay actually. We tried setting offsets without tramps in multiple places but whenever we were able to excite this loop, it will always damp down in about 5-6s regardless of the value of C1:IOO-WFS_GAIN.
• So we moved on.
• Then, with MC locked we took reference data with no excitation or filters uploaded. (dotted curves)
• We took cross spectral density from C1:IOO-MC_F to C1:IOO-MC_TRANS_PIT_IN1, C1:IOO-MC_TRANS_YAW_IN1, C1:IOO-WFS1_PIT_IN1, C1:IOO-WFS1_PIT_IN1, C1:IOO-WFS2_PIT_IN1, and C1:IOO-WFS2_PIT_IN1.
• We were also looking at the power spectral density of these channels.
• Then using awggui (after the fix we did as in 16085), we added noise in C1:SUS-MC2_LSC_EXC as uniform noise between 0.05 Hz to 3.5 Hz with amplitude of 100 and gain of 100.
• We took a set of data without switching on the filters to have a comparison later. (Dash-dort curves)
• We then took data after switching on the filters. (Solid curves)

### Next:

• Tomorrow we'll repeat this for MC1 and MC3 if we get a favourable grade in our work here.
• Even if not, we'll jsut conclude the suspension optimization work tomorrow morning and get into main interferometer.
Attachment 1: f2a.pdf
Attachment 2: IMC_F2A_Params_MC2.pdf
Attachment 3: MC2_F2A_FilterChar_POS2Ang.pdf
11711   Fri Oct 23 21:58:10 2015 ranaUpdateSUSMC2 F2P mis-tuned

The OSEMs cannot be used for coil balancing above ~10 Hz. The main coupling path from OSEM drive to sensor is not through the mirror motion, but instead direct electrical coupling of the drive wires to the sensing wires.

I put this in the elog every ~1-2 years since people keep trying it, but it keeps coming back like a zombie.

Better to use the MC angular sensors for L2A decoupling. Not perfect, but better than OSEMs. For the TMs we can use the OLs.

11716   Tue Oct 27 03:46:26 2015 ericqUpdateSUSMC2 F2P mis-tuned

D'oh. Good point.

Reverted for now; I'm thinking about doing laser pointer->MC2 QPD...

11702   Tue Oct 20 15:51:44 2015 ericqUpdateSUSMC2 F2P tuned

Using a modified version of Hang's deMod_deCoup scripts, I tuned the MC2 coil output matrix to minimize the appearance of POS drive in the SUSPIT signal at 28Hz. Up until now, there was no F2P compensation. This reduced the force to pitch coupling at 28Hz by 8dB

Old: POS -> 1 x UL, 1 x UR, 1 x LL, 1x LR

New: POS -> 1.1054 x UL, 1.1054 x UR, 0.8946 x LL, 0.8946 LR

I checked the MCL spectrum before and after this change with OAF on, this did not spoil the feedforward length subtraction in any noticible way.

The script lives in userapps/release/isc/c1/scripts/decoup, but I've symlinked it to /opt/rtcds/caltech/c1/scripts/decoup.

The script modification I made had to do with how the I and Q data is collected. Before, it was sporadically probing the I and Q FM output monitor EPICS values; I changed it to use the avg function of cdsutils, which calculates the mean and std from the 16kHz data and have seen it improve results by 1dB or so. I've been in touch with Jenne to propagate this to the sites.

9815   Tue Apr 15 16:21:18 2014 KojiUpdateIOOMC2 LSC offset was set to be -5000

Yesterday, MC2 alignment was slipping all day. Even when the WFS was off (i.e. there wa sno actuation), I had continual misalignment caused by MC2

I was afraid that the MC2 mirror is on a bistable position somehow. So I gave -5000 offset on the MC2 LSC. We'll see how it makes the MC happier.

5708   Thu Oct 20 01:40:33 2011 KojiSummarySUSMC2 Misaligned 2:27PM on Wednesday

There looks some activity at around MC2 on Wednesday afternoon.
It caused the misalignment of MC2. Misalignment was not found in MC1/3.

It seems that the incident beam on the MC was aligned in the evening.
This increased the MC transmission but it is vibible that the spot on MC2 is shifted from the center.

We need an action on this issue tomorrow in the daytime.

Attachment 1: MC2_misalign.png
5716   Thu Oct 20 18:57:35 2011 SureshSummarySUSMC2 Misaligned 2:27PM on Wednesday

 Quote: There looks some activity at around MC2 on Wednesday afternoon. It caused the misalignment of MC2. Misalignment was not found in MC1/3. It seems that the incident beam on the MC was aligned in the evening. This increased the MC transmission but it is vibible that the spot on MC2 is shifted from the center. We need an action on this issue tomorrow in the daytime.

I am working on fixing this.  You might some strange stuff going on in the control room screens.  Pls ignore it till I am done.

5717   Fri Oct 21 02:36:44 2011 SureshSummarySUSMC2 Misaligned 2:27PM on Wednesday : MC Realigned

Quote:

 Quote: There looks some activity at around MC2 on Wednesday afternoon. It caused the misalignment of MC2. Misalignment was not found in MC1/3. It seems that the incident beam on the MC was aligned in the evening. This increased the MC transmission but it is vibible that the spot on MC2 is shifted from the center. We need an action on this issue tomorrow in the daytime.

I am working on fixing this.  You might some strange stuff going on in the control room screens.  Pls ignore it till I am done.

I have realigned the MC by recentering the spots on all the MC optics.  The current spot positions (in mm) are:

MC1P     MC2P     MC3P      MC1Y      MC2Y     MC3Y

0.2245    0.3364   -0.2801   -1.8891    0.1631   -1.744

Initially the lockins 2 and 5 showed very small outputs.  This was traced to the fact that we have recently switched on a 28Hz ELP filter module in the MC2 ASC filter bank which introduces an extra phase of about 75deg..  See this elog.

When the MC ASS lockins were initially setup, the phase was set with this filter module switched off.  Since quite some time has passed since the last calibration of these phases, I readjusted the phases to minimise the  Q_OUTPUT and I also adjusted the GAINs in the SIG filter banks  of all the six lockins so that their I_OUT's drop by the calibration value of -2.65 when an offset of 0.1 is introduced into the MC suspension output matrices.  Two short scripts in the $scripts$/ASS/ directory help in setting and removing these offsets.  They are called MCxoffsetOn and MCxoffsetOff.   They have to be edited appropriately to address each DoF of the MC.

The $scripts$/ASS/mcassUp script., which sets up everything to make the MC spot decentering measurement, has been edited to set these new phases and gains.  The old settings have been commented out.

I then centered the spots on the WFS sensors and the MC_TRANS QPD.  We are now ready to make the MC WFS output matrix transfer coef measurement again, but this time with the WFS loops closed.

5718   Fri Oct 21 02:57:38 2011 SureshSummarySUSMC2 Misaligned 2:27PM on Wednesday : cause traced

Quote:

 Quote: There looks some activity at around MC2 on Wednesday afternoon. It caused the misalignment of MC2. Misalignment was not found in MC1/3. It seems that the incident beam on the MC was aligned in the evening. This increased the MC transmission but it is vibible that the spot on MC2 is shifted from the center. We need an action on this issue tomorrow in the daytime.

I am working on fixing this.  You might some strange stuff going on in the control room screens.  Pls ignore it till I am done.

While chatting with Jenne I learnt that some substantial amount of work had taken place yesterday around the MC2 chamber.  This was associated with the relocating of seismometers.  ref elog

I reiterate what is well known for quite sometime:  MC2 table is not well isolated from the ground.  And we should not approach this chamber unless absolutely necessary. I have blocked off the area around it which we should avoid.  It is a serious waste of time and effort to realign the MC each time the MC2 table decides to settle into a new position.

Steve tells me that the mild-steel frame supporting the chamber+MC2_table sits with two legs on one concrete slab while the other two legs sit on another one.   The frame is also quite weak without sufficient gussets or cross connects.  The next time we have a major shutdown we must replace this frame with a more sturdy one which sits on one slab (preferably the one on which the rest of the MC sits).

Till we improve this mounting, I suggest that we avoid that area as much as possible.

ELOG V3.1.3-