Started working on this. First created a git repo for tracking https://git.ligo.org/40m/susaux.git

I have looked through the folder to see what needs doing and I think I know what needs to be done for the final case by just following the same pattern for the other optics, which I am listing below

- Create database file for the BHD optics, say C1_SUS-AUX_LO1.db by copying another optic database file say C1_SUS-AUX_SRM. Then replace the optic name.

- Insert a new line "C1:SUS-LO1_LATCH_OFF" in the file autoBurt_watchdogs.req

- Populate the file autoBurt.req with the appropriate channels for LO1

- Populate the file C1SUSaux_post.sh with the corresponding commands for LO1

- Add the line dbLoadDatabase("/cvs/cds/caltech/target/c1susaux/C1_SUS-AUX_LO1.db") to the file C1SUSaux.cmd

For the temporary watchdog, we comment everything I have just talked about, and do only what come next.

My question is the following:

I understand that we need to use the OUT16 slow channel as a temporary watchdog since we don't currently have access to the slow channels bcos the Acromag units have not been installed. My guess from Koji's instructions is that we need to update the channels in the last two fields "INPA" and "INPB" below

record(calc,"C1:SUS-LO1_UL_CALC")
{
        field(DESC,"ANDs Enable with Watchdog")
        field(SCAN,"1 second")
        field(PHAS,"1")
        field(PREC,"2")
        field(HOPR,"40")
        field(LOPR,"-40")
        field(CALC,"A&B")
        field(INPA,"C1:SUS-LO1_UL_COMM  PP  NMS")
        field(INPB,"C1:SUS-LO1_LATCH_OFF  PP  MS")
}

Suppose we replace the channel for INPA with C1:SUS-LO1_ULCOIL_OUT16, what about INPB. Is this even the right thing to do as I am just guessing here?

I don't remember the syntax of the db file, but here this calc channel computes A&B. And A&B corresponds to INPA and INPB.

        field(CALC,"A&B")
        field(INPA,"C1:SUS-LO1_UL_COMM  PP  NMS")
        field(INPB,"C1:SUS-LO1_LATCH_OFF  PP  MS")

What is this LATCH doing?

After some work on the reference database file, we now have a template for temporary watchdog implementation for LO1 located here "/cvs/cds/caltech/target/c1susaux/C1_SUS-AUX_LO1.db".

Basically, what I have done is swap the EPICS asyn analog input readout for the COIL and OSEM to accessible medm channels, then write out watchdog enable/disable to coil filter SW2 switch. Everything else in the file remains the same. I am worried about some of the conversions but the only way to know more is to see the output on the medm screen.

To test, I restarted c1su2 but this did not make the LO1 database available, so I am guessing that we also need to restart the c1sus, which can be done tomorrow.

As the new db is made for c1susaux, 1) it needs to be configured to be read by c1susaux 2) it requires restarting c1susaux 3) it needs to be recorded by FB 4) and restartinbg FB.
(^-Maybe not super exact procedure but conceptually like this)

cf.
https://wiki-40m.ligo.caltech.edu/How_To/Add_or_rename_a_daq_channel

Temp software watchdog now operational for LO1 and the remaining optics!

Koji helped me understand how to write to switches and we tried for a while to only turnoff the output switch of the filters instead of the writing a zero that resets everything in the filter.

Eventually, I was able to move this effort foward by realising that I can pass the control trigger along multiple records using the forwarding option 'FLNK'. When I added this field to the trigger block, record(dfanout,"C1:SUS-LO1_PUSH_ALL"), and subsequent calculation blocks, record(calcout,"C1:SUS-LO1_COILSWa") to record(calcout,"C1:SUS-LO1_COILSWd"), everything started working right.

[Ian, Koji]

In preparation for the replacement of the suspension electronics that control the ETMY, I took measurments of the system excluding the CDS System. I took transfer functions from the input to the coil drivers to the output of the OSEMs for each sensor: UL, UR, LL, LR,  and SIDE. These graphs are shown below as well as all data in the compressed file.

We also had to replace the oplev laser power supply down the y-arm. The previous one was not turning on. the leading theory is that it's failure was caused by the power outage. We replaced it with one Koji brought from the fiber display setup.

I also am noting the values for the OSEM DC output

In addition the oplev position was:

(KA ed) We only care about PERROR and YERROR (because P/YOUT are servo output)

Edit: corrected DC Output values

[Koji, Ian]

We looked at a few power supplies and we found one that was marked "CHECK IF THIS WORKS" in yellow. We found that the power supply worked but the indicator light didn't work. I tried a two other lights from other power supplies that did not work but they did not work. Koji ordered a new one.

I moved the network-enabled power strip from above the power supplies on rack 1y4 to below. Nothing was powered through the strip when I unplugged everything and I connected everything to the same port after.

[Ian, Koji] - Activity on 25th (Fri)

We continued working on the ETMY electronics replacement.

- The units were fixed on the rack along with the rack plan.

- Unnecessary Eurocard modules were removed from the crate.

- Unnecessary IDC cables and the sat amp were removed from the wiring chain. The side cross-connects became obsolete and they also were removed.

- A 18V DC power strip was attached to one of the side DIN rails.

Warning:

- Right now the ETMY suspension is free and not damped. We are relying on the EQ stops.

Next things to do:

- Layout the coil driving cables from the vacuum feedthru to the sat amp (2x D2100675-01 30ft ) [40m wiki]

- Layout DB cables between the units

- Layout the DC power cables from the power strip to the units

- Reassign ADC/DAC channels in the iscey model.

- Recover the optic damping

- Measure the change of the PD gains and the actuator gains.

[Paco, Ian]

paco helped me wire the ETMY 1Y4 rack. We wired the following (copied from Koji's email):

1. Use DB9-DB9 to complete the wiring between
   1. 16bit DAC AI Chassis - End DAC Adapter (4 cables)
   2. End DAC Adapter - HAM-A Coil Driver (2 cables)
   3. AA Chassis - End ADC Adapter (2 cables)
2. Koji already brought two special DB9-DB15 cables (in plastic bags) to the end. They connect the HAM-A coil drivers to the satellite amp. At this time, we skip Low Noise HV Bias Driver.
3. Bring two 30ft DB9 (called #1, aka D2100675-01) cables from the office area to the end. I collected one end and left them there.
4. All the new units have +/-18V DC supply in the back. Find the orange cables behind the 40m vacuum duct around Y-end and connect the units and the DC power strip. Use short cables if possible to save the longer ones.

the cables we used:

I attached pictures below.

The replacement key switches and Ne Indicators came in. They were replaced and work fine now.

The power supply units were tested with the X end HeNe display. It turned out that one unit has the supply module for 1350V 4.9mA while the other two do 1700V 4.9mA.
In any case, these two ignited the HeNe Laser (1103P spec 1700V 4.9mA).

The 1350V one is left at the HeNe display and the others were stored in the cabinet together with spare key SWs and Ne lamps.

For the Y-end electronics replacement, we want to remove unused power supplies. In fact, we already removed the +/-5V supplies from the stack. I was checking what supply voltages are used by the Eurocard modules. I found that D990399 QPD whitening board had the possible use of +/-5V.

The 40m Y-end version can be found here D1400415. The +/-5V supply voltages are used at the input stage AD620 and the QPD bias voltage of -5V.

AD620 can work with +/-15V. Also the bias voltage can easily be -15V. So I decided to cut the connector legs and connected +5V line to +15V, and -5V line to -15V.

With this modification, I can say that the eurocards only use the +/-15V voltages and nothing else.

The updated schematics can be found as D1400415-v6

The DC power strip at Y-end was connected to the bottom two Sorensen power supplies. They are configured to provide +/-18V.

[Koji, Ian]

1) We attached the 30 coil driving cables to the vacuum feed through to the sat amp  [40m wiki] they run along the cable tray then up and down into the rack.

2) we checked all DB and power cables. We found that the anti-imaging filter had a short and got very hot when plugged in. the back power indicator lights turned on fine but the front panel stayed off. We removed it and replaced it with the one that was on the test stand marked for the BHD. This means we need to fix the broken one and Koji mentioned getting another one.

3) we reassigned the ADC and DAC channels in the iscey model and the asy model. we committed a version before we made any changes.

4) Finally we tested the setup to make sure the ETM was being damped.

Next step:

1) Measure the change of the PD gains and the actuator gains. See pervious elog

Oplev saga

Summary

- The new coil driver had not enough alignment range to bring the oplev beam back to the QPD center
- The coil driver output R was reduced from 1.2k to 1.2k//100 = 92.3 +/- 0.4 Ohm
- Now the oplev spot could be moved to the center of the QPD

- The damping gains (POS/PIT/YAW) and the oplev gains were reduced by a factor of 1/10.
- The damping and the oplev servos work now. Fine gain tuning is necessary.

To Do:
- DC value / TF measurements
- Adjust damping gains
- RFM issue
- Connection check
- Cable labeling

== Alignment Range ==

- Since c1auxey was removed, we no longer have C1:SUS-ETMY_PIT_COMM and C1:SUS-ETMY_YAW_COMM. At this moment, all the alignment is taken with the offset input from the fast real-time system via C1:SUS-ETMY_PIT_OFFSET and C1:SUS-ETMY_YAW_OFFSET.

- The oplev spot could not be moved on the center of the QPD without exceeding the DAC output range (~+ or -32000) for the coils. (Attachment 1)

- This is because the old system had a slow but large current range (Rout = 100) and a small current range for the fast control. Until we commission the new HV BIAS Driver, we have to deal with the large DC current with the HAM-A coil driver.

== Modification to the output resistances ==

The following units and the channels were modified. Each channel had a differential current driver and two output resistances of 1.2K. 100Ohm (OHMITE 43F100, 3W) wire wound resistors were added to them in parallel, making the resulting output R of ~92Ohm.

- ETMY HAM A Coil Driver 1: S2100622 (Attachments 2/3) CH1/2/3
- ETMY HAM A Coil Driver 2: S2100621 (Attachments 4/5) CH3

- This modification allowed me to align the oplev spot to the center of the QPD. C1:SUS-ETMY_PIT_OFFSET and C1:SUS-ETMY_YAW_OFFSE are +2725 (8%FS) and -2341 (7%FS), respectively.
- The previous alignment slider values were -0.9392 and 0.7615 (out of 10). These are the reasonable numbers, considering the change of the Rout from 100 to 92Ohm, and the sign flip.
(By the way, autoBurt files for c1auxex and c1auxey were not properly configured and the history of C1:SUS-ETM*_*_COMM was not recorded.)

== Damping Servos ==

- Now, the POS/PIT/YAW servos experience ~x10 gains. So temporarily these gains were reduced (POS 20->2, PIT 6->0.6, YAW 4->0.4) and the loops are stable when engaged.
- Also the gains of the OPLEV servos were reduced from -4.5 to -0.45. The loops are stable when engaged.

== Snapshot of the working condition ==

Attachent 6 shows the screenshot for the snapshot of the working condition.

To Do

- The damping servos were tested without proper PD whitening compensation.
  -> It turned out this is not necessary as our modified PD whitening has the pole and zero at the same freqs as before.

- Compare the DC values of the OSEM outputs and compensate for the gain increase by the "cts2um" filter.

- The end RTS suffers from the RFM issue. There is no data transmitted from the vertex to the end. I suspect we need to restart the c1rfm process. But this will likely suspend all the vertex real-time machines. Careful execution is necessary.

- c1iscey has all the necessary analog connections. But they are not tested. When we lock the green/IR cavity, we'll need them.

- The cable labeling is only half done.

Now that the ETMSUS is back up and running I reran my measurements from the beginning of the process. The results below show a change in gain between the before and after measurements. I have given values of the low-frequency section below.

Average Gain difference from the TFs: 18.867  (excluding thee side change)

I also am noting the new values for the OSEM DC output: average gain increase: 9.004

In addition, the oplev position was:

All data and settings have been included in the zip file

From the average gain increase of the TFs which indicates the increase of the whole system and the increase in gain from the OSEM we can calculate the gain from the actuators.

18.867/9.004 = 2.09

thus the increase in gain on the actuator is about 2.09

EDIT: I updated the side TF with one with better SNR. I increased the excitation amplitude.

[Paco, Ian]

• Begin free swinging test for AS1 at 10:05 AM, set for ~ 2.04 hours.
  • Test failed because damping failed to disable.
• Restart free swinging test for AS1 at 15:06, set for 2.04 hours.
  • Success (Attachment #1 shows the DOF input matrix diagonalization effect)

Of slight concern is the side to other degrees of freedom coupling, but this is definitely an improvement from last time.

I updated the gain of the ct2um filter on the OSEMS for ETMY and decreased their gain by a factor of 9 from 0.36 to 0.04.

I added a filter called "gain_offset" to all the coils except for side and added a gain of 0.48.

together these should negate the added gain from the electronics replacement of the ETMY

[Anchal, Paco]

We installed a HV kepco power supply for the green PZT steering the YAUX beam. We did this in anticipation of the YARM alignment to take place this afternoon. We borrowed an unused DC power supply labeled "OMC Power spply" and made an appropriate SMA connection (Attachment #1), Then we set the Vset to 100.0 Volts and the current limit to 10 mA. Once we enabled the output we saw the 5.6 mA of current drawn by the eurocard in accordance with the wiki log (Attachment #2). 

It may not be possible to use the PZT as per this so this work may not have a direct impact on our upcoming alignment task.

We probably bumped the ethernet cable (martian network) on c1iscey, so the FE models stopped showing up on the medm screens. When we connected it back, it seemed like the FE kept running the model and only IPC showed error. We restarted the rtcds models on c1iscey and burtrestored to today morning 5:19 am.

ETMY is properly damping now and the oplev is roughly centered as well but the OPLEV Servos are off. We did not turn them on.
We should be able to carry out our cavity arm alignment today afternoon on both arms now.

I'm not sure why but the PIT and YAW offset values of +2725 and -2341 were not sufficient for the reflected OPLEV beam to reach the OPLEV QPD. I had to change the C1:SUS-ETMY_PIT_OFFSET to 5641 and C1:SUS-ETMY_YAW_OFFSET to -4820 to come back to center of the OPLEV QPD. We aligned the Oplevs to center before venting, so hopefully this is our desired ETMY position.

On another note, the issue of ETMY unable to damp was simple. The alignment offsets were on to begin with with values above 1000. This meant that whenever we enabled coil output, ETMY would necessarily get a kick. All I had to do was keep alignment offsets off before starting the damping and slowly increase the alignment offsets to desired position.

After Ian updated the cts2um filters for OSEM, shouldn't the damping gains be increased back by factor of 10 to previos values? Was the damping gain for SIDE ever changed? we found it at 250.

Can you explain why gain_offset filter was required and why this wasn't done for the side coil?

The point of changing the gains was to return the system to its origional state. ie I wanted the over all gain of the physical components to be the same as when we started. From the CDS side of things nothing else should be changed. The damping filters should remain at their origional values. The cts2um filter was changed to counteract a change in the electronics (replacing them). These changes should cancel eachother out. As for the side control, on 3/4/22 koji reduced the output resistors for the 4 face OSEMs but did not change the the SD one. there fore the SD did not need the same adjustment as the others.

[Ian, Paco]

• We removed the "cheby" filters from AS4, PR2 and PR3 which had been misplaced after copying from the old SUS models. After removing them, the new SOS damped fine. Note that because of the Input matrices, the filters have to be enabled all at once for the MIMO loop to make sense.
• We also disabled the "Cheby" filter on BS and saw it damp better. We don't understand this yet, but perhaps it's just a consequence of the many changes in the BSC that have rendered this filter obsolete.
• we also reduced the damping gains on PR2, PR3 and AS4 to prevent overflow values. After the adjustments the optics were damping fine.

[JC Koji]

To give more alignment ranges for the SUS alignment, we started updating the output resistors of the BHD SUS coil drivers.
As Paco has already started working on LO1 alignment, we urgently updated the output Rs for LO1 coil drivers.
LO1 Coil Driver 1 now has R=100 // 1.2k ~ 92Ohm for CH1/2/3, and LO1 Coil Driver 2 has the same mod only for CH3. JC has taken the photos and will upload/update an elog/DCC.

We are still working on the update for the SR2, LO2, AS1, and AS4 coil drivers. They are spread over the workbench right now. Please leave them as they're for a while.
JC is going to continue to work on them tomorrow, and then we'll bring them back to the rack.

[JC Koji]

Quick report: JC has done all the mods for the coil driver circuit in the morning and we worked on the reinstallation of them in the afternoon.
I'll check the damping loops / sus servo settings. JC is going to make an ELOG entry and DCC updates for more precise record of the mods.

[JC, Yuta]

We did a quick health check of suspesions after the pump down.

Summary:
 - ITMX LRSEN is too bright (~761) and not responding to any optic motions (we knew this before the pump down)
 - ITMY ULCOIL is not working
 - LO1 LLCOIL is not working
 - Damping loops need to be retuned, especially for ETMY (too much damping), SRM, PR3 and AS4 (damping too weak)
 - MC1 sensor outputs are minus instead of plus
 - LO2 OSEMs got stuck during the pump down, but now it is free after some kicks. OSEM sensorr values almost came back (see attached)

What we did:
 1. Kicked optics with C1:SUS-{optic}_{UL,LL,UR,LR,SD}COIL_OFFSET one by one with offsets of +/- 10000 (or 100000), and checked if C1:SUS-{optic}_{UL,LL,UR,LR,SD}SEN_OUT16 move in both directions.

 2. Check if the optic damps nicely.

 3. Attached photo of the note is the result.

[Paco, JC]

Paco and I attempted to calculate the input matrices from May 24, 2022, but the OSEM data was all saturated and not very useful. Therefore, we decided to manually investigate the appropriate coil offsets for all BHD SUS. Before, the default offset kick was 30000 counts, but we found that LO1, AS1, AS4, and PR2 cannot take more than 5000 counts. As for LO2, SR2, and PR3 cannot take more than 2000 counts before saturating. Note that all these kick test were taken by kicking OSEM UL on all BHD Optics.

We started the freeSwing.py script on tmux freeSwing session for tomorrow at 1:00 am for only the 5000 count offset SUS.

[Paco, Yuta]

We investigated the ITMY ULCOIL issue (40m/16873).
ULSEN is sensing the optic motion but ULCOIL cannot move the optic.
We confirmed that the coil input is there upto satellite amplifier output.
We also checked that ULCOIL have 3.3 mH and 16 Ohms, which are consistent with other coils.
Mystery remains...
We need to investigate ITMY ULCOIL in the next vent.

What we did:
 - Checked again that C1:SUS-ITMY_ULCOIL_OFFSET does not kick ITMY using OSEM sensor signals and oplev signals. ULSEN moves when ITMY is kicked by other coils.
 - Checked that kick gives voltage changes at coil driver and satellite amplifier output. We unplugged J1 DB25 cable from the feedthru flange and checked the signals sent to coil with oscilloscope.
 - Measured inductance (using BK PRECISION LCR meter) and resistance (using Fluke) of coils for ITMY. Below is the result. UL coil seems to be consistent with other coils. (It seems like BK PRECISION one wil give wrong resistance if the dial is set to the resistance value which is too low compared with the one you want to measure. If you want to measure 16Ω, set the dial to larger than 20Ω, not 2Ω)

Feedthru connector: ITMY1
Pin 3-15 / R = 16.3Ω / L = 3.32 mH (UL)
Pin 7-19 / R = 16.4Ω / L = 3.30 mH (UR)
Pin11-23 / R = 16.2Ω / L = 3.31 mH (LL)

Feedthru connector: ITMY2
Pin 3-15 / N/A
Pin 7-19 / R = 16.3Ω / L = 3.30 mH (SD)
Pin11-23 / R = 16.4Ω / L = 3.33 mH (LR)

Discussions:
 - UL is the only short OSEM in ITMY OSEMs.
 - ITMY have dumbells for magnets.
 - If UL magnet is off, ULSEN would not work. Something not magnetic is working for shadow sensing for UL? Dumbells?
 - ULSEN just sensing some coupling from other OSEMs?

Anchal, Paco and I agreed to follow the polarity conventions below for suspension damping loops.
Some of the polarity/gains were changed to homogenize all the suspensions to the convention.
All the suspensions are homogenized except for MC1 (which have all - in sensor inputs and all - in coil outputs) and AS1 (SDCOIL_GAIN have the same sign as LL).

*SEN_GAIN to be +1
INMATRIX to be the following, and calibrated so that SUSPOS/SIDE_IN will be um and SUSPIT/YAW_IN1 will be urad (calibration to be done later)
+ + + + *
+ + - - *
+ - - + *
* * * * +
SUS*_GAINS to be +
TO_COIL gains to be the following
+1 +1 +1  *
+1 +1 -1  *
+1 -1 +1  *
+1 -1 -1  *
 *  *  * +4
*COIL_GAIN to be the following or flipped one so that SUS*_GAINS will be +
+
-
+
-
+

To do this, the following changes were made

For BS, ITMX, ITMY, PRM, SRM, ETMX, ETMY, MC1, MC2 and MC3 ("old" suspensions), TO_COIL_5_4_GAIN (for side) is changed from +1 to +4 and SUSSIDE_GAINs are divided by 4 accordingly.
For ETMX, the sign of SUSSIDE_GAIN is flipped to +, and SDCOIL_GAIN to be -1 (it was +1).
For MC1, *SEN_GAINs are - (not following the convension; see 40m/16894). The sign of INMATRIX_4_5 (for side) is flipped to +, SUSPOS/PIT/YAW_GAIN are flipped to +, and *COIL_GAIN are flipped to - (not following the convension). IMC WFS output matrix components for MC1 were also flipped. 

Since UL coil actuation is lost, we modified the output matrix of ITMY to use only UR, LR and LL face coils for POS, PIT and YAW actuation. The output matrix was changed to following:

After this change, the damping was still working as good as before. I took PIT to POS/PIT/YAW and YAW to POS/PIT/YAW coupling measurements by exciting C1:SUS-ITMY_ASCPIT[YAW]_EXC and seeing effect at C1:SUS-ITMY_SUS[POS/PIT/YAW]_IN1 when the damping loops were off. Attached are the results. We were able to reduce PIT to YAW and YAW to PIT coupling by 10 dB by this simple change in output matrix. More coil balancing or off-diagonal termsmight help more and should be attempted if required. The coupling to POS did not change much.

Note that attachment 1 shows transfer functions from excitation point to the DOF sensing inputs while attachment two looks at ratio of C1:SUS-ITMY_SUS[POS/PIT]_IN1 to C1:SUS-ITMY_SUSYAW_IN1 which is the actual quantity of interest. I didn't repeat the PIT measurement due to lack of time.

Also note that all such measurements are being recorded in our new measurements git repo. We'll populate this repo with diaggui template+data files as we do measurements.

To see if the ULCOIL channel of the ITMY coil driver is working or not, I swapped ITMY coil driver and ITMX coil driver by swapping DB15 cable (see Attachment #2).

With this swap, I confirmed that ITMX can be kicked with C1:SUS-ITMY_ULCOIL_OFFSET, but ITMY cannot be kicked with C1:SUS-ITMX_ULCOIL_OFFSET (see Attachment #1).

This means that the issue is not the in-air electronics.
Mystery remains again...
We need to investigate ITMY ULCOIL in the next vent.

I revereted the swap and confirmed that damping loops work fine again.

what was the result of the inductance measurement? should be ~3.3 mH as measured from the flannge or cable that goes to the flange from sat amp.

ITMY ULCOIL was measured to have ~3.3 mH as measured from the flange. RTFE 40m/16896 .
 

The servo gain for MC1 in /opt/rtcds/caltech/c1/Git/40m/scripts/MC/WFS/reliefWFS was flipped to account for COIL_GAIN flip done in 40m/16898.
The reliefWFS script now works fine.

ezcaservo -r 'C1:SUS-MC2_ASCPIT_OUT16' -g ${g} -t ${ts} C1:SUS-MC2_PIT_COMM & 
ezcaservo -r 'C1:SUS-MC2_ASCYAW_OUT16' -g ${g} -t ${ts} C1:SUS-MC2_YAW_COMM & 
ezcaservo -r 'C1:SUS-MC1_ASCPIT_OUT16' -g -${g} -t ${ts} C1:SUS-MC1_PIT_COMM &  
ezcaservo -r 'C1:SUS-MC1_ASCYAW_OUT16' -g -${g} -t ${ts} C1:SUS-MC1_YAW_COMM & 
ezcaservo -r 'C1:SUS-MC3_ASCPIT_OUT16' -g ${g} -t ${ts} C1:SUS-MC3_PIT_COMM & 
ezcaservo -r 'C1:SUS-MC3_ASCYAW_OUT16' -g ${g} -t ${ts} C1:SUS-MC3_YAW_COMM & 
 

Its good that the inductance test passed. This means that the coil is OK. How does the inspection photo look? This is the one you guys took of the ITM OSEM that shows the position of the magnet w.r.t. the coil. Also, how does the free swinging spectra look? Either one of these might indicate a broken magnet, or a sticky EQ stop.

We checked the photos we have, but we didn't have the photos which show ULCOIL situation clearly.

Free swing of ITMY (and others) will be done this weekend to see the OSEM spectra and resonant frequencies.

[Anchal, Yuta]

In the weekend, I ran a free swing test on all optics. During this test, LO2 magnets got stuck to the coil because LO2 PIT alignment was very high, making its lower OSEMs almost fully dark and upper OSEMs almost fully bright. Today we realized that LO2 is actually stuck and is not coming off even when we dither PIT alignment. We tried several ways but could not get this off.

Do we have any other method to get magnets off in vaccum?

It will be pretty bad if we try anything related to BHD with LO beam reflecting off a stuck mirror. Does anyone have any suggestions other than venting and fixing the issue?

[JC, Anchal, Yuta]

We are working on resonant frequency idendification from the free swing test done last weekend.
Table below is the resonant frequencies identified, and attached are the plots of peak identification for some of our new suspensions.
To identify the resonant frequencies, the kicks were done in each degrees of freedom so that we can assume, for example, SUSPOS will be mostly excited when kicked in POS and the heighest peak is at the POS resonant frequency.
For PR3, AS1 and ETMY, the resonant frequency idendification needs to be done in the order of POS, PIT, YAW, SIDE and identified frequencies need to be removed when finding a peak.
Other than that, the identification was done without any prior assumptions on the suspensions.
For ITMY, ETMY, PR2, PR3, AS1, AS4, yaw has lower resonant frequencies than pitch, as opposed to other suspensions.
For LO1, POS and PIT frequencies might be swapped because LLCOIL is not working (40m/16898) and POS/PIT kicks both might be excited SUSPOS/PIT.
LO1 coil output matrix was temporarily modified so that we use only two coils for POS/PIT/YAW excitation (Attachment #7), as we did for ITMY (40m/16899).

The scripts for the free swinging test and analysis live in /Git/40m/scripts/SUS/InMatCalc

     POS    PIT    YAW    SIDE
BS   0.990  0.748  0.794  0.959 
ITMY 0.987  0.739  0.634  0.948 fPIT > fYAW
ETMY 0.979  0.816  0.649  0.954 fPIT > fYAW
ITMX 0.978  0.586  0.758  0.959 
ETMX 0.962  0.725  0.847  1.000 
PRM  0.939  0.541  0.742  0.990 
PR3  1.019  0.885  0.751  0.989 fPIT > fYAW
PR2  0.996  0.816  0.724  0.999 fPIT > fYAW
SRM  0.969  0.533  0.815  0.985 
SR2  0.978  0.720  0.776  0.997 
LO1  0.926  1.011  0.669  0.993 POS AND PIT MIGHT BE SWAPPED
LO2  0.964  0.998  0.995  0.990 WRONG DUE TO STUCK (40m/16913)
AS1  1.028  0.832  0.668  0.988 fPIT > fYAW
AS4  1.015  0.800  0.659  0.991 fPIT > fYAW
MC1  0.967  0.678  0.797  0.995 
MC2  0.968  0.748  0.815  0.990 
MC3  0.978  0.770  0.841  0.969 

I checked the state of the LO2 suspension. I found that the coil driver Enable Mon was all red. Meaning, the actuation signals were not delivered to the coil driver. I wasn't sure if this was intentional or not.

Enabled the coils with "WD Master" Shutdown -> Normal.

Immediately I saw the OSEMS flipped the sign because there was an (non-intentional) alignment offset in pitch. I've adjusted the pitch offset so that all the OSEM PDs have the voltages 4~5V.

That's it.

We tracked the issue of LO1 LLCOIL not actuating LO1, and found that the DB9 cable from the coil driver to the sat amp was loose.
I tightened the screws and now it is working.
Never ever connect cables without screwing the connectors tightly!

What I did:
 - Measured the resistance and the inductance of each coil with BK PRECISION LCR meter, as I did for ITMY (Attachment #1, 40m/16896). The result is the following and it shows that LLCOIL is there.

Feedthru connector: LO1 1
Pin 3-15 / R = 16.0Ω / L = 3.27 mH (UL)
Pin 7-19 / R = 15.8Ω / L = 3.27 mH (UR)
Pin11-23 / R = 15.7Ω / L = 3.27 mH (LL)

Feedthru connector: LO1 2
Pin 3-15 / N/A
Pin 7-19 / R = 15.6Ω / L = 3.22 mH (SD)
Pin11-23 / R = 15.9Ω / L = 3.30 mH (LR)

 - Swapped the DB25 cable which goes to the feedthru LO1 1 and feedthru LO1 2. LLCOIL could be drived from LR coil driver and LRCOIL could not be drived from LL coil driver. SD and UR worked fine with the swap. This means that there is something wrong with the LL driving.
 - Went to see the rack which have coil drivers and sat amp for LO1, and immediately found that the DB9 cable was loose (Attachment #2). Tightened them and the issue was fixed.
 - C1:SUS-LO1_TO_COIL matrix gains were reverted to default values (Attachment #3).

[Anchal, Yuta]

We fixed the issue of ITMY ULCOIL not driving ITMY by replacing one of the 64pin ribbon cable in the satellite amplifier box.
We thought the coil driver and the sat amp box are OK by checking the voltage change at the output of the sat amp box by giving an offset to UL coil driver, but it was not giving a current change, probably due to too much contact resistance in the cables.
It was sneaky because it was not completely disconnected.

All the coils for our suspensions are now working!

What we did:
 - Using breakout boards, the output current of sat amp box was measured using FLUKE multimeter. It turned out that UL is not giving measurable current. We also confirmed that UR coil driver can drive UL by re-directing the current from UR coil driver to UL. This means that the UL magnet was not de-magnetized!
 - Measured the coil resistance from at the coil driver output and found that UL coil seen from there has too high resistance which cannot be measured with the multimeter, whereas UR coil was measured to be ~30 Ohms.
 - Went back to the feedthru and measured the resistance of UL coil. Upto the output of the Satellite Amp Terimator, the resistance was measured to be ~16 Ohms, but not at the input of the Satellite Amp Terimator (Attachment #1,2).
 - It turned out that #16 pin of 64pin ribbon cable in between the Satellite Amp Terimator (LIGO-D990021) and the Satellite Amp board (LIGO-D961289) at the Satellite Amp Terimator side was not good (Attachment #3).
 - Replaced the cable and confirmed that ULCOIL can kick ITMY (Attachment #4).
 - C1:SUS-ITMY_TO_COIL matrix was reverted to default values.

Next:
 - We might have to re-commission Yarm ASS again since pitch-yaw coupling have changed. -> EDIT: Checked that it works (except for ITM PIT L), including offloading offsets (writeASS_offsets.py), 18:30 local.
 - Now that LO1 LLCOIL issue is solved and LO2 stuck is solved, we should do the free swing test again to identify the resonant frequencies.
 - OSEM sensor diagonalization (input matrix), coil balancing (and F2A)

I used the same fre swing data to diagnolize input matrices of following optics:
MC1, MC2, MC3

BS ETMX ETMY

AS1 AS4 SR2 PR2 PR3

For all these optics, the new input matrices worked well. Next step should be to take the local damping open loop transfer functions and standardize the loops to same UGF.

What didn't work:

• The calculated input matrix for ITMX differed from existing matrix too much, including overall sign of rows POS and PIT. Even after correcting those signs, I was not able to get a good damping loop configuration. So I have committed the new matrix to the repo but have not implement it. More close analysis or another test might be required for this optic.
• LO1, LO2, and ITMY were not analysed because their free swing test was not valid. LO1 and ITMY had non-working coils and LO2 was stuck during the test. We'll take another free swing test for these three optics (and possible ITMX) in near time.

All diagonalization results are present in https://git.ligo.org/40m/scripts/-/tree/main/SUS/InMatCalc

For looking at the results at this point, go to this commit: https://git.ligo.org/40m/scripts/-/tree/7ef6a47d1b2051a0732f46477624a9e625737fe8

Over the weekend, I ran freeSwing test with sequential kicks in specific DOFs for LO1, ITMY, ITMX. LO1 results were successfully used to diagonalize LO1 input matrix. There are some issues for ITMY, ITMX still. I could not run LO2 test.

LO1

The free swing test ran successfully, resonant frequencies for different DOFs was extracted, and new input matrix was calculated. The new matrix was only slightly different from before and it worked fine with existing damping loops. The observed resonance frequencies were different from previous values by POS: -6 mHz, PIT : -3 mHz, YAW: -9 mHz, SIDE: -2 mHz. Attached are the diagonalization result.

ITMX

The peculiarity of ITMX remained even after the second free swing test. The calculated input matrix is very different from existing one with sign flips across PIT and POS rows. I found that our LR osem is always bright in ITMX at the current alignment position. I see that LR osem comes in range when C1:SUS-ITMX_PIT_COMM is raised above 0.5. Maybe we should run this test when we know for sure ITMX is in correct position.

ITMY

In ITMY on the other hand, I found that SIDE OSEM was completely bright. This happened during the YAW kick to ITMY. We'll need to reduce kick amplitudes for ITMY and redo this test.

LO2

For LO2, I could not initiate the test. On reducing the alignment offsets for LO2 (so that it doesn't get stuck in the fre swing test), the damping loops were not working. This is a clear evidence also that input matrix is different for different positions of the optic. We need to think about some other strategy to do this test, maybe see if ideal input amtrix works at no offsets and use that to damp during the test.

git repo : https://git.ligo.org/40m/scripts/-/tree/main/SUS/OutMatCalc

local dir:  /opt/rtcds/caltech/c1/Git/40m/scripts/SUS/OutMatCalc

Here is an update on our recent attempt at diagonalization of the SUS output matrices. There are two different parts to this: the first is coil balancing using existing F2P code which stopped working because of an old-style use of the print function and the second which should now focus on the mixing amongst the various degrees of freedoms (dof) without a DC/AC split I believe. The F2P codes are now working and have been consolidated in the git repo.  

TODO:

• The remaining task is to make it so that we only call a single file that combines the characterization code and filter generation code, preferably with the addition of a safety feature that restores any changed values in case of an error or interruption from the user. The safety functionality is already implemented in the output matrix diagonalization stem of the code, so we just need to copy this over.  
• Improve the error minimization algorithm for minimizing the cross-coupling between the various dof by adjusting the elements of the output matrix. 

Previous work 

https://nodus.ligo.caltech.edu:8081/40m/4762

https://nodus.ligo.caltech.edu:8081/40m/4719

https://nodus.ligo.caltech.edu:8081/40m/4688

https://nodus.ligo.caltech.edu:8081/40m/4682

https://nodus.ligo.caltech.edu:8081/40m/4673

https://nodus.ligo.caltech.edu:8081/40m/4327

https://nodus.ligo.caltech.edu:8081/40m/4326

https://nodus.ligo.caltech.edu:8081/40m/4762

[Paco, JC, Yuta]

This morning, while investigating the source of a burning smell, we turned off the c1SUS 1X4 power strip powering the sorensens. After this, we noticed the MC1 refl was not on the camera, and in general other vertex SUS were misaligned even though JC had aligned the IFO in the morning to almost optimum arm cavity flashing. After a c1susaux modbusIOC service restart and burt restore, the problem persisted.

We started to debug the sus rack chain for PRM since the oplev beam was still near its alignment so we could use it as a sensor. The first weird thing we noticed was that no matter how much we "kicked" PRM, we wouldn't see any motion on the oplev. We repeatedly kicked UL coil and looked at the coil driver inputs and outputs, and also verified the eurocard had DC power on which it did. Somehow disconnecting the acromag inputs didn't affect the medm screen values, so that made us suspicious that something was weird with these ADCs.

Because all the slow channels were in a frozen state, we tried restarting c1susaux and the acromag chassis and this fixed the issue.

[Yuta, Tega]

We diagnosed the suspension damping of the IMC/BHD/recycling optics by kicking the various degree of freedom (dof) and then tuning the gain so that we get a residual Q of approx. 5 in the cases where this can be achieved.

MC1: Good
MC2: SIDE-YAW coupling, but OK
MC3: Too much coupling between dofs, NEEDS ATTENTION
LO1: Good
LO2: Good
AS1: POS-PIT coupling, close to oscillation, cnt2um off, NEEDS ATTENTION
AS4: PIT-YAW coupling, cannot increase YAW gain because of coupling, No cnt2um, No Cheby, NEEDS ATTENTION
PR2: No cnt2um, No Cheby
PR3: POS-PIT coupling, cannot increase POS/PIT/YAW gain because of coupling, No cnt2um, No Cheby, NEEDS ATTENTION
SR2: No cnt2um

[Anchal, Tega]

As a first step to characterize all the local damping loops, we ran an open loop transfer function measurement test for all BHD optics, taking transfer function using band-limited (0.3 Hz to 10 Hz) gaussian noise injection at error points in different degrees of freedom. Plots are in the git repo. I'll make them lighter and post here.

We have also saved coherence of excitation at the IN1 test points of different degrees of freedom that may be later used to determine the cross-coupling in the system.

The test ran automatically using measSUSOLTF.py script. The script can run the test parallelly on all suspensions in principle, but not in practice because the cdsutils.getdata apparently has a limitation on how many real-time channels (we think it is 8 maximum) one can read simultaneously. We can get around this by defining these test points at DQ channels but that will probably upset the rtcds model as well. Maybe the thing to do is to separate the c1su2 model into two models handling 3 and 4 suspensions. But we are not sure if the limitation is due to fb or DAQ network (which will persist even if we reduce the number of testpoints on one model) or due to load on a single core of FE machines.

The data is measured and stored here. We can do periodic tests and update data here.

Next steps:

• Run the test for old optics as well.
• Fit the OLTF model with the measured data, and divide by the digital filter transfer function to obtain the plant transfer function for each loop.
• Set maximum noise allowed in the local damping loop for each degree of freedom, and criteria for Q of the loop.
• Adjust gains and or loop shape to reach the requirements on all the suspensions in a quantitative manner.
• (optional) Add a BLRMS calculation stream in SUS models for monitoring loop performance and in-loop noise levels in the suspensions.
• More frequency resolution, please. (KA)

I made measurements of old optics OLTF today. I have reduced the file sizes of the plots and data now. It is interesting that it is allowed to read 9 channels simultaneously from c1mcs or c1sus models, even together. The situation with c1su2 is a bit unclear. I was earlier able to take measurements of 6 channels at once from c1su2 but not I can't read more than 1 channel simultaneously. This suggests that the limit is dictated by how much a single model is loaded, not how much we are reading simultaneously. So if we split c1su2 into two models, we might be able to read more optics simultaneously, saving time and giving us the ability to measure for longer.

Attached are the results for all the core optics. Inferences will be made later in the week.

Note: Some measurements have very low coherence in IN2 channels in most of the damping frequency region, these loops need to be excited harder. (eg PIT, POS, YAW, on ITMs and ETMs).

for damping and OL loops, we typically don't measure the TF like this because it takes forever and we don't need that detailed info for anything. Just use the step responses in the way we discussed at the meeting 2 weeks ago. There's multiple elog entries from me and others illustrating this. The measurement time is then only ~30 sec per optic, and you also get the cross-coupling for free. No need for test-point channels and overloading, just use the existing DQ channels and read back the response from the frames after the excitations are completed.

[Yehonathan, Paco]

This morning, while attempting to align the IFO to continue with noise-budgeting, we noted the XARM lock was not stable and showed glitches in the C1:LSC-TRX_OUT (arm cavity transmission). Inspecting the SUS screens, we found the ULSEN rms ~ 6 times higher than the other coils so we opened an ndscope with the four face OSEM signals and overlay the XARM transmission. We immediately noticed the ULSEN input is noisy, jumping around randomly and where bigger glitches correlated with the arm cavity transmission glitches. This is appreciated in Attachment #1.

Signal chain investigation

We'll do a full signal investigation on ITMX SUS electronics to try and narrow down the issue, but it seems the glitches come and go... Is this from the gold satamp box? ...