40m QIL Cryo_Lab CTN SUS_Lab CAML OMC_Lab CRIME_Lab FEA ENG_Labs OptContFac Mariner WBEEShop
  40m Log, Page 354 of 354  Not logged in ELOG logo
ID Dateup Author Type Category Subject
  17822   Sun Sep 3 08:37:18 2023 HirokiUpdateGeneralPut away beam chopper

*This work was done on Aug. 11th.

I put away the beam chopper used in T&R measurement into the shelf in YARM (Attachment 1).

Attachment 1: IMG_1497.jpg
IMG_1497.jpg
  17823   Sun Sep 3 08:47:43 2023 HirokiUpdateOptical LeversFound a fiber-coupled visible diode laser (iFLEX-1000)

[Koji, Hiroki]

*This work was done on Aug. 11th.

We found a fiber-coupled visible diode laser (iFLEX-1000) in a shelf on YARM (Attachment 1 and 2).
This laser may be used as the replacement of the He-Ne laser for OPLEV.

Attachment 1: IMG_1498.jpg
IMG_1498.jpg
Attachment 2: IMG_1499.jpg
IMG_1499.jpg
  17824   Tue Sep 5 04:48:20 2023 HirokiSummaryGeneralSummary of the late submitted entries

After I came back to Japan, I wrote and revised some Elog entries that I was not able to finish during my stay.
I am sorry for the late submission and revision.

Toward locking PRFPMI

Flow sensor

  • elog #17765
    I additionaly attached the schematic of the wiring to this existing entry.
     
  • elog #17779
    Calibration of the flow sensors.

Others

  • elog #17822
    Location of the beam chopper used in T&R measurement.
     
  • elog #17823
    Location of the diode laser (iFLEX-1000) that might be used as the replacement of the He-Ne laser for OPLEV.
  17825   Tue Sep 5 11:04:33 2023 ranaUpdateASSReducing XARM-ASS Errors

I recommend usinng the DC offset method that Koji and I used for measuring the IMC WFS sensing matrix (not the AC method that Anchal used). With a sensing matrix, you should be able to do some partial inversion.

Without any sensing matrix inversion, we would have to rely on a gain hierarchy for getting the loops to work.

With some approximate matrix inversion, the loops are more indepedent of each other. Also if you look at the spectrum of the error signals, it should be clear that the sensing noise is pretty large, and so that sets a natural upper limit to the UGFs. We only want integrator (1/f) loops, but the LPFs cause some extra phase lag.

Quote:

[Radhika, Murtaza]

XARM ASS

  17827   Tue Sep 5 18:46:08 2023 KojiSummaryGeneralDsub 9 (DB9) cable inventory

I went to Section Y7 to check the stock DB9-MF cables. The custom dsub cables are also there.

  • DB9 1FT / QTY 26
  • DB9 2.5FT / QTY 42
  • DB9 5FT / QTY 49
  • DB9 10FT / QTY 30
  • DB9 15FT / QTY 9

Let's say this is (26, 42, 49, 30, 9)

My estimation was:
1Y0/1 (9, 26, 8, 10, 0)
1Y4 or 1X9 (0,8,2,0,0)
1X3/4/5 (11,30,10,12,0)

And the rest was for acromag.

This means that:
No matter how my estimation was wrong, we have prenty of cables for the 1X3/4/5 electronics swap.

  17828   Wed Sep 6 12:53:11 2023 Paco, Radhika, MurtazaUpdateASSReducing XARM-ASS Errors

[Radhika, Murtaza]

To create the sensing matrix, we tried the DC offset method by giving offsets in the Pitch and Yaw DOFs for ITMX, ETMX and the BS respectively. The signals we looked at were the demodulated ETMX_L, ETMX_T and ITMX_T. We wrote a quick notebook that does the following things for each DOF:

1. Calculate the mean error signal (over 10s)
2. Give an offset of 3 steps in each DOF corresponding to their step size serially with some buffer time (restoring the offset after each DOF)
3. Calculate the new mean error signal (over 10s)
4. Find the difference in error signals and divide by their respective step sizes to get each sensor's sensitivity to the offset.
5. Invert to obtain the sensing matrix.

Sensing Matrix for Pitch:

\begin{bmatrix} ETM P \\ ITMP \\ BSP \end{bmatrix} = \begin{bmatrix} 1.63590936e+00 && -9.67386830e+00 && 2.65620052e+00 \\ 7.58125093e-03 && -2.23977732e-02 && 1.02654009e-01 \\ 2.03270795e-02 && -4.60362844e-02 && 3.66501511e-02 \end{bmatrix} \begin{bmatrix} ETM P L err\\ ETMP T err\\ ITM P Terr \end{bmatrix}

Sensing Matrix for Yaw:

\begin{bmatrix} ETM Y \\ ITM Y \\ BS Y \end{bmatrix} = \begin{bmatrix} -4.69197238 && 19.11902408 && -13.96830153 \\ -0.04886671 && 0.103685 && -0.06974976 \\ 0.03923102 && -0.12202067 && 0.10306225 \\ \end{bmatrix}\begin{bmatrix} ETM YLerr \\ ITM YTerr \\ BS YTerr \end{bmatrix}

*NEED TO TRY THESE OUT*

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

The Output Matrix for the from intuition was set to *Attachment 2* which improved the net average transmission (see Attachment 1), but wasn't really stable after the improvement.

Attachment 1: X_ARM_ASS_FIGHT.png
X_ARM_ASS_FIGHT.png
Attachment 2: X_ARM_ASS_MAT_INTUITIOn.png
X_ARM_ASS_MAT_INTUITIOn.png
  17830   Thu Sep 7 14:09:37 2023 KojiSummaryElectronicsVertex Electronics Transition

The vertex electronics transition work will begin on Monday. We expect the ongoing ASS-X work to be completed by then. But if it needs more time, we must hear a shouting signal from the ASS team.
Is there any other preparation to be done this week to reasonably compensate for changes in gain and TF associated with the transition?


In preparation for the transition, we want to have long custom DSUB25 cables (D2100675) approximately laid out (I mean on the floor, etc) this week. JC takes care of this.

  • The lengths of the cables can be found in the attached wiring diagram.
  • Both ends of the cables need to be labeled.
  • At which side do we want to absorb the slack?

Transition Plan

  • Suspension damping and watchdogs are appropriately taken care of, although we soon stop/remove everything.
  • We first remove any existing units not going to be used in the circuit (except around the Eurocard crate oplev interface P2 of the wiring diagram).
  • The wirings at the side cross-connects are removed. This includes the removal of the thick cables on the cable racks. This would become a heavy work.
  • The DC power strips are attached to the racks, and the DC power wiring should be done at this point. We check the DC supply voltages.
  • Install the new units as per the above rack layout and proceed with the DSUB connections. We have sufficient number of DSUB cables (this ELOG).
  • Turn the units on one by one to detect any unit failure, just in case. If they are all on, we start work on the CDS restoration work.
Attachment 1: diagram_1X345.pdf
diagram_1X345.pdf diagram_1X345.pdf
Attachment 2: rack_plan.pdf
rack_plan.pdf
  17831   Thu Sep 7 16:25:01 2023 MurtazaUpdateSUSETMX Testing

COIL BALANCING FOR ETMX
Summary : I ran coil balancing on ETMX using the CoilStrengthBalancing.ipynb script to get a feel for it, no changes were required from the last time it was run by Paco. I realized I was measuring the wrong signal for the POS coupling (C1:SUS-ETMX_SUSPOS_IN1) while trying to minimize the BUT-POS coupling. This was stupid because the shadow sensors and actuation coils in this case are the OSEMs. The LSC error signal would be more appropriate for measuring the POS coupling.

The convention for the actuation vector used for the coils is [UL, UR, LL, LR]. The frequency, excitation counts are given through the LOCKIN1 channel (SUS->ETMX->LOCKIN1->f(Hz), Amp). The excitation vector is set in (SUS->ETMX->Output Filters->LOCKIN1)
Here, the excitation frequency used = 13Hz.
For "decoupling" the degrees of freedom, the script used is given in /opt/rtcds/caltech/c1/Git/40m/scripts/SUS/coilStrengthBalancing/ETMX/CoilStrengthBalancing.ipynb. In here, small steps are taken to "remove" the DOF contribution such that \\new_gain = old_gain +/- e*(decoupling DOF vector)\\. (e = step size)
The decoupling signals are observed in diaggui in the frequency range of (0 - 20Hz) and a bandwidth of 0.5Hz with exponential averaging (10 averages)

1. Minimizing BUT-POS coupling
Here, the LSC error signal (C1:LSC-POX11_I_IN1) is observed to measure the coupling in POS. For this, the arm is kept locked to obtain a decent error signal.
SANITY CHECK: This was tested by exciting POS [1,1,1,1] at 13Hz and measuring at the LSC error signal in diaggui which indeed showed a peak at 13Hz indeed. Damp filters and OPLEV servos were enabled to prevent the loss of lock.
The initial excitation was given to the Butterfly DOF [1, -1, -1, 1] at 10000 counts and 5 steps were taken in both directions of the POS vector to decouple the POS DOF. The initial peak showing up in the LSC error signal was already at a minimum. The excitation was ramped up to 20000 counts, where the peak was still very small. Thus, no change was made here (Attachment 1).

2. Minimizing POS-PIT coupling
Here, the OPLEV signal for PIT (C1:SUS-ETMX_OL_PIT_IN1) is observed to measure the coupling in PIT. The damping filters and OPLEV servos are disabled.
The initial excitation is given to POS [1, 1, 1, 1] at 5000 counts and 5 steps were taken in both directions of the PIT vector to decouple the PIT DOF. The initial peak showing up in the OPLEV signal was already at a minimum. Thus, no change was made here (Attachment 2).

3. Minimizing POS-PIT coupling
Here, the OPLEV signal for YAW (C1:SUS-ETMX_OL_YAW_IN1) is observed to measure the coupling in YAW. The damping filters and OPLEV servos ared disabled.
The initial excitation is given to POS [1, 1, 1, 1] at 5000 counts and 5 steps were taken in both directions of the YAW vector to decouple the YAW DOF. The initial peak showing up in the OPLEV signal was already at a minimum. Thus, no change was made here (Attachment 3).

4. Minimizing PIT-YAW coupling
This was one rather robust and was not susceptible to the decoupling process. Here, the OPLEV signal for both PIT (C1:SUS-ETMX_OL_PIT_IN1) and YAW (C1:SUS-ETMX_OL_YAW_IN1) are used to measure their relative coupling. Either of the DOF can be excited and while the other DOF can be used for the decoupling vector. Here, PIT was excited and the decoupling DOF vector was YAW. The damping filters and OPLEV servos ared disabled.
The initial excitation was given to the PIT DOF [1, 1, -1, -1] at 5000 counts and 5 steps were taken in both directions of the YAW vector to decouple the YAW DOF. The initial peak showing up in the YAW signal was already at a minimum. The excitation was ramped up to 10000 counts, however the YAW peak barely moved. Thus, no change was made here (Attachment 4).

To next.

Attachment 1: ETMX_BUT_POS_COUPLING.png
ETMX_BUT_POS_COUPLING.png
Attachment 2: ETMX_POS_PIT_COUPLING.png
ETMX_POS_PIT_COUPLING.png
Attachment 3: ETMX_POS_YAW_COUPLING.png
ETMX_POS_YAW_COUPLING.png
Attachment 4: ETMX_PIT_YAW_COUPLING.png
ETMX_PIT_YAW_COUPLING.png
  17832   Thu Sep 7 18:42:02 2023 Paco, Radhika, MurtazaUpdateASSReducing XARM-ASS Errors

[Radhika, Murtaza]

We recalculated the sensing matrix for XARM ASS by collecting each sensor's step response to an offset in each DOF. This produced the following dense output matrix A (see Attachment 1 for rows/cols):

      [[-0.02047695,  0.        , -0.10262752,  0.        , -0.0157128 , 0.        ],
       [ 0.        ,  0.16908344,  0.        , -0.00929291,  0.        ,-0.35916455],
A =    [-0.28050764,  0.        ,  0.26982002,  0.        , -0.55100297, 0.        ],
       [ 0.        ,  0.85501491,  0.        ,  0.0606197 ,  0.        , 0.27568672],
       [-0.95963335,  0.        ,  0.95742611,  0.        ,  0.83435534, 0.        ],
       [ 0.        , -0.49026554,  0.        , -0.99811768,  0.        , 0.89162641]]

Turning the XASS gain up slowly to ~0.15, we observed that several error signals diverged and transmission started to drop. Debugging this matrix proved difficult since there were many nonzero elements to consider. So we reverted to build the matrix from our intuition, considering the centering and input pointing loops, and using the YASS output matrix as a reference.

The YARM ASS servo gains are all +1. The YASS output matrix has the following length (centering) signal mapping:

ITM PIT/YAW L ----> ETM feedback
(ETM PIT/YAW L - ITM PIT/YAW L) ----> ITM feedback

We mirrored this in the XASS output matrix. Note that previously the ITM L error signals were not used for XASS. To simplify the process, we decided to just work out the beam centering first and ignore the input pointing coming from the beam splitter (setting BS PIT/YAW matrix elements to 0). We also set all the XARM ASS servo gains to +1. See the output matrix below:

We cleared the outputs and turned on the XARM GAIN slowly (0.1) and immediately noticed the YAW signals in ETM start to diverge (C1:ASS-XARM_ETM_YAW_T_DEMOD_I_OUT16, C1:ASS-XARM_ETM_YAW_L_DEMOD_I_OUT16). We turned down the XARM gain and flipped the sign for the signal going to ETM YAW. (suspect a difference in sign convention).

To check the stability, we sequentially gave offsets in PIT/YAW for ETM and ITM. We saw the signal (C1:ASS-XARM_ETM_PIT_L_DEMOD_I_OUT16) oscillate wildly at a frequency of ~(1/15 Hz). We suspected the ASS loop was driving these oscillations so we turned down the gain going to ETM PIT to 0.25 which worked really well and the transient oscillation of further checks was gone.

We saw similar wild oscillatory signals in ITM PIT (C1:ASS-XARM_ITM_PIT_T_DEMOD_I_OUT16, C1:ASS-XARM_ITM_PIT_L_DEMOD_I_OUT16) on applying offsets so we reduced the gain going to ITM PIT to 0.3. (0.25 and 0.3 are arbitary relatively smaller weights, can be fine tuned).

We checked the stability of this setup as a whole by giving a few offsets to ITMX and ETMX, with a servo gain of 0.15 it did a great job! (0.25 made it diverge once again). See final state for centering in Attachment 1, and error signal suppression in Attachment 2. (Ignore XAUX transmission in grey - we were toggling the shutter.) Note that the Length error signals were successfully suppressed, but the dark green/brown Transmission error signals were not fed back and thus remain nonzero.

WE SHALL INVESTIGATE THE INPUT POINTING NEXT FEEDING BACK TO THE BS and ITMX. We will give an update shortly about whether restoring XARM ASS is feasible by Monday.

Attachment 1: XASS_sensing_mat_2023-09-07.png
XASS_sensing_mat_2023-09-07.png
Attachment 2: XASS_centering_only.png
XASS_centering_only.png
Attachment 3: X_ASS_L_Loop_only.png
X_ASS_L_Loop_only.png
  17833   Thu Sep 7 21:09:17 2023 PacoUpdate eCARM and eDARM to ALS CARM and ALS DARM

Tonight I managed to lock CARM and DARM under ALS control only

Configuration

  • Arm cavities well aligned, TRY ~ 1.07, TRX ~ 0.98, GTRY ~ 1.2, GTRX ~ 0.77
  • HEPA off, WFS 60s offloaded, PD offsets removed, all lights inside the lab were off
  • BEATX and BEATY ool residual noise shown in Attachment #1.
  • Error points, the A path was the same as what is used for electronic FPMI. For the B path, I describe the tuning below.
    • CARM_A = 0.5 * POX11_I + 0.5 * POY11_I
    • DARM_A = 0.19 * POX11_I - 0.19 * POY11_I
    • CARM_B = -0.7 * ALSX + 0.4 * ALSY  
    • DARM_B = -0.25 * ALSX - 0.17 * ALSY
  • Power normalization of the error signals was 0.5 * TRX + 0.5 * TRY in both paths.
  • LSC filter banks are the same ones we use for electronic FPMI, and the gains were
    • CARM = 0.011  (UGF ~ 200 Hz) using FM1 to FM5, FM6 and FM8
    • DARM = 0.055 (UGF ~ 150 Hz) using FM1 to FM5, FM6 and FM8
  • Control points, I temporarily disabled violin filters around 600 Hz to ease the lock acquisition ... we should really use the VIO TRIG here to avoid having to do this.
    • CARM = -0.734 * MC2
    • DARM = 0.5 * ETMX - 0.5 * ETMY

ALS error signal tuning

To find the error signals for CARM/DARM, I turned on the oscillators (at 307.8 and 313.31 Hz respectively) with 150 counts and enabled FM10 (Notch for sensing matrix) in the CARM and DARM servo banks. I then removed the ALS offsets (C1:LSC-ALSX_OFFSET, C1:LSC-ALSY_OFFSET) and looked at the transfer functions shown in Attachment #2. I optimized the ALS blending until I maximized the CARM and DARM A to B paths and minimized CARM and DARM cross couplings. The signs were chosen to leave a phase of 0.

Handoff

After measuring the OLTFs for eCARM and eDARM (loop closed with the A error point) and tuning the ALS error signals, I gradually blended the A and B paths and checked the OLTFs for CARM and DARM. During this I realized I needed to disable some of the notch violin filters because they sometimes made the DARM loop unstable after >50% blending. In the end the simultaneous CARM_A/DARM_A to CARM_B/DARM_B handoff was successful in 0.5 seconds. Attachment #3 shows the OLTFs under ALS control.

CARM offset

After getting nominally stable ALS control, I tried adding an offset. The LSC CARM offset range was insufficient, so I ended up directly scanning the C1:LSC-ALSX_OFFSET and C1:LSC-ALSY_OFFSET. The first couple of attempts the ramp time was set to 2.0 seconds, and a step of 0.01 was enough to break the lock. I managed to hold the control with as much as C1:LSC_CARM_A_IN1 offset by ~ 500 (rms ~ 200 counts). I roughly estimate this to be ~ 5% of the CARM pole which is 4 kHz in this case so overall 200 Hz which is not that large.

Attachment 1: ALSCARMDARMlock_oolnoise_Screenshot_2023-09-07_21-32-31.png
ALSCARMDARMlock_oolnoise_Screenshot_2023-09-07_21-32-31.png
Attachment 2: ALSerrortuning_Screenshot_2023-09-07_21-33-10.png
ALSerrortuning_Screenshot_2023-09-07_21-33-10.png
Attachment 3: ALSCARMDARM_oltfs_Screenshot_2023-09-07_21-36-34.png
ALSCARMDARM_oltfs_Screenshot_2023-09-07_21-36-34.png
  17834   Fri Sep 8 17:11:53 2023 KojiUpdateCDSFSSSlow restoration

I came to the lab to see the recovery work from the power glitch this morning 8:15AM. All CDS seems up. The suspensions are somewhat aligned. Some of them were not damped. The oplevs were off. Radhika is working on the recovery of the FP arms.


I noticed that FSS Slow servo is not working. I always forget what is the right way to turn it on. Here is the summary:

How to turn on FSSSlow (2023 Sept version)

  • Go to megatron
  • sudo systemctl enable FSSSlow
  • sudo systemctl start FSSSlow
  • sudo systemctl status FSSSlow

    FSSSlow.service - Script to run the PID temperature control servo for the PSL
       Loaded: loaded (/opt/rtcds/caltech/c1/Git/40m/scripts/PSL/FSS/FSSSlow.service; enabled; vendor pre
       Active: active (running) since Fri 2023-09-08 17:10:52 PDT; 1s ago
     Main PID: 2088 (python3)
        Tasks: 6 (limit: 4674)
       CGroup: /system.slice/FSSSlow.service
               └─2088 /usr/bin/python3 /opt/rtcds/caltech/c1/Git/40m/scripts/PSL/FSS/PIDLocker.py PIDConf

    Sep 08 17:10:52 megatron systemd[1]: Started Script to run the PID temperature control servo for the

  17835   Sat Sep 9 16:25:07 2023 RadhikaUpdateASSReducing XARM-ASS Errors

[Radhika, Murtaza]

This post summarizes XARM ASS efforts from Friday 9/8 and Saturday 9/9.

FRIDAY

On Friday, we continued with our previous output matrix that used the length error signals (ITM/ETM PIT/YAW L) to feed back to ITMX and ETMX (see the previous ELOG). In that state we did not use the transmission error signals and had no feedback going to the BS. We then tried to use the transmission error signals ITM PIT/YAW L as a proxy for BS input pointing and feed them back to the BS. For both PIT and YAW, both signs of feedback resulted in diverging T error signals and a decrease in transmission.

SATURDAY

On Saturday, we used the transmission error signals (ITM/ETM PIT/YAW T) in the sensing matrix to build the output matrix. We got it to a state where we could get the controlled error signals to converge by just feeding back to the ITMX and ETMX (Attachments 1,2). Once we had this working, we tried to feed back a combination of (ETM PIT/YAW L and ITM PIT/YAW T) to correct BS pointing. However, any combinations and signs to the BS dropped transmission and led to diverging error signals.
We then attempted to use the latest working XASS output matrix (before the acromags were pulled out) and see the effect of flipping signs in there (one optic+DOF at a time) We then tried to use the sign logic from the previously working ETM/ITM feedback we got partially working; however the error signals did not converge with any combination.

SUMMARY:

- We are able to successfully feed back to ITMX and ETMX, using either length or transmission error signals. It is when we try to add BS feedback that ASS fails. This can be due to the fact that we need to consider the relative servo gains when treating these loops separately, like Koji mentioned.

- The sensing matrix approach might be the only way to simultaneously optimize feedback for all optics, avoiding the need to tune servo gains. We will revisit this approach on Monday.

     - Koji pointed out that we are reading out the low-passed error signals in order to calculate each step response - we will need to consider our sampling rate and duration of averaging accordingly.

     - It will be harder to iteratively flip signs of each matrix element for this dense matrix, and we will have to be clever about which sign combinations we try for actuation.

Attachment 1: XASS_transmission_mat_only.png
XASS_transmission_mat_only.png
Attachment 2: XASS_transmission_err_only.png
XASS_transmission_err_only.png
  17836   Mon Sep 11 19:35:27 2023 RadhikaUpdateASSReducing XARM-ASS Errors

[Radhika, Murtaza, Paco]

Today we decided to take a closer look at the demod phases of the T and L error signals for XARM ASS. By eye we tuned the phases to minimize the signal in Q. Here are the new demod phases:
(THE DEMODULATION PHASE VALUE DO NOT RESTORE BACK TO THE ORIGINAL VALUES WHEN DITHER IS TURNED ON.)

C1:ASS-XARM_ETM_PIT_L_DEMOD_PHASE:  15 -> 35
C1:ASS-XARM_ETM_YAW_L_DEMOD_PHASE: 176 -> 180
C1:ASS-XARM_ITM_PIT_L_DEMOD_PHASE:   0 -> -5
C1:ASS-XARM_ITM_YAW_L_DEMOD_PHASE:  10 -> -10
C1:ASS-XARM_ETM_PIT_T_DEMOD_PHASE:  10 -> -3.5
C1:ASS-XARM_ETM_YAW_T_DEMOD_PHASE: -10 -> -5
C1:ASS-XARM_ITM_PIT_T_DEMOD_PHASE:   0 -> -15
C1:ASS-XARM_ITM_YAW_T_DEMOD_PHASE:  -5 -> 30

We also noticed that MEDM indicator for dithering on (white --> green LO symbol) for ETM_YAW_L_OSC was tied to the wrong excitation gain channel (C1:ASS-XARM_ITM_YAW_OSC_CLKGAIN instead of C1:ASS-XARM_ETM_YAW_OSC_CLKGAIN). We went ahead and changed this in [insert medm file location]. So now the right green LO symbol appears when the appropriate excitation is turned on.

  17837   Tue Sep 12 18:49:51 2023 KojiUpdateGeneralTransformed 3x 18bit AI chassis into 16bit

For the preparation of the electronics upgrade, three 18bit DAC AI chassis were transformed to 16bit version.

The power supply connections were touched, so the units were tested with +/-18V, and they work as expected.

Attachment 1: PXL_20230913_000718873.jpg
PXL_20230913_000718873.jpg
Attachment 2: PXL_20230913_011622783.MP.jpg
PXL_20230913_011622783.MP.jpg
Attachment 3: PXL_20230913_011644441.MP.jpg
PXL_20230913_011644441.MP.jpg
Attachment 4: PXL_20230913_011631099.jpg
PXL_20230913_011631099.jpg
  17838   Tue Sep 12 18:55:55 2023 KojiSummaryElectronicsVertex Electronics Transition

We are ready to do the transition from Wed 1PM.

The items for the upgrade was collected around the vertex area (Attached photo).

- aLIGO-style DC power strip (+/-18V) x3
- DC power cables (orange +/-18V)
- Electronics units for the upgrade.
- DSUB (DB9) cables
- Custom DSUB15-DSUB25 cables
- Custom DSUB25 cables

 

Attachment 1: PXL_20230913_013735728.jpg
PXL_20230913_013735728.jpg
  17839   Tue Sep 12 23:10:06 2023 KojiSummaryElectronicsVertex Electronics Transition

Note on Sorensen:

- Eurocard crate requires +-15V. We can place two 15V Sorensens on 1X4 for Eurocard crate or just leave the current +/-15V supplies.

- The aLIGO units requires +/-18V. We can place two 15V Sorensens on 1X5 or just leave two of the current supplies and set them to +/-18V.

  17840   Wed Sep 13 12:46:03 2023 KojiSummaryElectronicsVertex Electronics Transition ~ final prep
  • [OK] Reflected the sorensen setup (minimal change from the conventional config. (See the attachment)
  • Before destroying the current setup, bring the alignment biases for the vertex 8 sus to zero and record all the OSEM values.
    => Radhika did it (next ELOG)

    -> This will give us the ratio of the OSEM error signals to know the gain ratios between before and after. Also this will make it easier to bring the alignment back.

  • [OK] I suppose the oplevs are still aligned. We don't need to be too nervous about the oplev spot too much.

  • How to compensate the coil force cal? Do we know the ratio from the ETM coil driver swap? (What were the coil output Rs? What are they now? Are the ratios reasonable?)

    • Currently:
      - PRM/BS/ITMX/ITMY DAC output for face coils differential

      - SRM/MC2/MC1/MC3 DAC output for face coils single ended
      - All 8 SD coils single ended
      - Coil Output Rs
        According to D1700218

        PRM unknown to be checked
        BS 100Ohm
        ITMX 100Ohm
        ITMY 100Ohm
        SRM 100Ohm
        MC2 430? unknown to be checked
        MC1 430? unknown to be checked
        MC3 430? unknown to be checked

    • New setup
        DAC output differential
        AI has the gain of 1 / HAM coil driver has a gain of 1.2
        Coil output Rs:

        For all the face coils 1.2k // 100 ~ 92Ohm
        For all the side coils 1.2k

 

Attachment 1: rack_plan_1X345.pdf
rack_plan_1X345.pdf
  17841   Wed Sep 13 13:50:48 2023 RadhikaSummaryElectronicsVertex Electronics Transition ~ final prep

OSEM values for 8 vertex optics ~before~ electronics upgrade (averaged over 60 s):

['C1:SUS-BS_ULSEN_IN1', 'C1:SUS-BS_URSEN_IN1', 'C1:SUS-BS_LRSEN_IN1', 'C1:SUS-BS_LLSEN_IN1', 'C1:SUS-BS_SDSEN_IN1', 'C1:SUS-ITMX_ULSEN_IN1', 'C1:SUS-ITMX_URSEN_IN1', 'C1:SUS-ITMX_LRSEN_IN1', 'C1:SUS-ITMX_LLSEN_IN1', 'C1:SUS-ITMX_SDSEN_IN1', 'C1:SUS-ITMY_ULSEN_IN1', 'C1:SUS-ITMY_URSEN_IN1', 'C1:SUS-ITMY_LRSEN_IN1', 'C1:SUS-ITMY_LLSEN_IN1', 'C1:SUS-ITMY_SDSEN_IN1', 'C1:SUS-PRM_ULSEN_IN1', 'C1:SUS-PRM_URSEN_IN1', 'C1:SUS-PRM_LRSEN_IN1', 'C1:SUS-PRM_LLSEN_IN1', 'C1:SUS-PRM_SDSEN_IN1', 'C1:SUS-SRM_ULSEN_IN1', 'C1:SUS-SRM_URSEN_IN1', 'C1:SUS-SRM_LRSEN_IN1', 'C1:SUS-SRM_LLSEN_IN1', 'C1:SUS-SRM_SDSEN_IN1', 'C1:SUS-MC1_ULSEN_IN1', 'C1:SUS-MC1_URSEN_IN1', 'C1:SUS-MC1_LRSEN_IN1', 'C1:SUS-MC1_LLSEN_IN1', 'C1:SUS-MC1_SDSEN_IN1', 'C1:SUS-MC2_ULSEN_IN1', 'C1:SUS-MC2_URSEN_IN1', 'C1:SUS-MC2_LRSEN_IN1', 'C1:SUS-MC2_LLSEN_IN1', 'C1:SUS-MC2_SDSEN_IN1', 'C1:SUS-MC3_ULSEN_IN1', 'C1:SUS-MC3_URSEN_IN1', 'C1:SUS-MC3_LRSEN_IN1', 'C1:SUS-MC3_LLSEN_IN1', 'C1:SUS-MC3_SDSEN_IN1']

[1943.7368693033854, 1597.8396443684896, 1667.4166076660156, 1709.3347656250003, 1760.058290608724, 1013.2686828613281, 1833.5396423339844, 2096.6071411132816, 786.2030975341798, 1152.7826700846356, 923.4767690022786, 447.51257578531903, 678.4539184570312, 971.2538736979167, 720.261508178711, 2458.092639160156, 2245.09764811198, 710.1112263997396, 258.2382120768228, 1225.202982584636, 3048.1043741861986, 3092.901733398437, 77.94405148824053, 181.08351745605466, 5145.644441731771, -6879.474226888021, -6954.500219726562, -3615.385673014323, -5468.913354492189, -3260.200516764323, 1555.9213073730468, 1625.1883911132813, 348.9968526204428, 1011.3114247639974, 1386.0941060384114, 1014.3825622558594, 1218.7048522949217, 1642.9303202311198, 1621.7448221842449, 720.2911783854166]

  17842   Wed Sep 13 14:02:29 2023 KojiSummaryElectronicsVertex Electronics Transition ~ final prep

- 4x 1064nm NPROs are OFF. The lab hall is laser safe, although the oplev lasers are on.
  The Laser Warning Signs were turned off by the interlock switch at the PSL enclosure (control room side)


- Watch dogs were turned to "disabled"

- Halted c1sus using dolphin fencing. This worked very well.
The previous report of dolphin fencing not working was due to a typo in my instruction (wrong -disable -> correct --disable).

controls@fb1:~ 0$ ./dolphin_ix_port_control.sh --disable 192.168.113.40 1
--disable
Disabling switch_ip 192.168.113.40, port 1
Complete - csr write addr=0x0001C050, val=0x20820090 (with ret=0)

ssh c1sus

controls@c1sus:~$ rtcds stop --all

 

  17843   Wed Sep 13 17:26:26 2023 KojiSummaryElectronicsVertex Electronics Transition ~ DAY 1

[Radhika, Paco, Murtaza, Koji]

- Removed all the units that will not be used in the new setup.
- Removed the sidepanel crossconnects
- Removed most of the sidepanel power lines except for the top eurocrate at the top of 1X4 (requires +/-15 pale orange and blue).
- Removed the acromag connections
- Removed the connectors of the long suspension cables


We'll resume the work at 10AM. (We'll have breaks for lunch and the seminar at 3PM)

=== Next steps ===

- Continue to remove the long suspension cables.
- Attach the DC power strip

- Continue to clean up the power lines on the rack side
- Prepare the power lines (Guralp requires +/-15V, the Eurocard crate +/-15V, new power strips (x3) +/-18V)
- Install the units on the racks.

Attachment 1: rn_image_picker_lib_temp_d1ceae3a-04ba-4bdb-8fdc-1f9d8d3000e6.jpg
rn_image_picker_lib_temp_d1ceae3a-04ba-4bdb-8fdc-1f9d8d3000e6.jpg
Attachment 2: rn_image_picker_lib_temp_7757df3b-1daf-4a12-a93c-14fd54b6355b.jpg
rn_image_picker_lib_temp_7757df3b-1daf-4a12-a93c-14fd54b6355b.jpg
Attachment 3: rn_image_picker_lib_temp_c8e55ebb-eb8e-4eb2-bcb2-e12083f3aadc.jpg
rn_image_picker_lib_temp_c8e55ebb-eb8e-4eb2-bcb2-e12083f3aadc.jpg
  17844   Thu Sep 14 11:46:28 2023 KojiSummaryElectronicsVertex Electronics Transition ~ DAY 2

[Radhika, Paco, Murtaza, Koji]

Morning work:

  • Power Strip assembly
  • Power Strip cable crimping
  • c1lsc fiber routing (the PCIE fiber was in danger) / c1lsc machine was stopped after dolphin fencing
  • We tried to reroute the c1lsc fiber above the racks such that it does not get pitched by the rack doors,
    but it seemed that the fiber was damaged (Attachment 1) and the c1lsc can't talk with the IO chassis anymore. We need to replace the fiber.
    It seems that P/N is PCIEO-4G3-100.0-11 (Samtec)

Afternoon work:

  • Removed the sidepanels of 1X3 and 1X4 for easier work
  • Removed the long DB25 cables from the old sat boxes.
  • DC power strips are installed.
  • Finished cleaning up the side cross-connects
  • Checked the DC supply conditions.
    • +/-15V Eurocard crate + Guralp requires 0.5A / 0.6A
    • +/-18V appered on the DC power strips correctly
  • Cleaned up the floor a bit
  • Murtaza noticed that there was some strange intermittent noise around the 1Y0/1 racks.
    It looks like one of the fans for the c1ioo IO chassis are dying.

Evening work:

  • Made coil driver short plugs (Attachment 2). This enables the coils while the coil modes are set to be Acquire mode.
  • Rack nuts inserted

Tomorrow plan (10:30AM):

  • c1sus IO chassis installation
  • Unit installation
  • Unit powering tests (before connecting them)
  • Cabling between the units
  • Long DB cable installation (sat box removal)
  • c1lsc fiber replacement
  • Some above cable removal (fiber, old custom DB25 for MC1, etc)

 

Attachment 1: PXL_20230915_023755839.jpg
PXL_20230915_023755839.jpg
Attachment 2: PXL_20230915_012131487.jpg
PXL_20230915_012131487.jpg
  17845   Fri Sep 15 12:51:29 2023 KojiSummaryElectronicsVertex Electronics Transition ~ DAY 3

[Radhika, Paco, Murtaza, Koji]

We made great progress today. It's going well so far.

Morning work (10:30AM~):

  • Installed c1sus IO chassis
  • Installed all units
  • Removed long (previous) custom DB25 cables for MC1

Afternoon work

  • Connected all the units to the DC power strip
  • Unit powering test was done before the inter-unit DSUBs were connected.
    • We found one AA chassis don't turn on even though internal +/-15V seems supplied. We pulled the unit out.
    • All the other units were fine.
    • Typical current draw of the units: (unit name, positive supply current, negative supply current)
      • AA 0.5A 0.5A (18W)
      • AI 0.3A 0.3A  (11W)
      • BIO 0A 0A (0W)
      • Trillum I/F 0.1A 0.1A (0.4W)
      • Sat amp 0.3A 0.3A (11W)
      • Coil Driver 0.3A 0.2A (9W)
  • Cabling between the units
    • Done except for the extracted AA unit and for the BIO units (not needed until we have Acromag).
  • Long DB cable installation (sat box removal)
    • Halfway
  • c1lsc fiber tracking/replacement (not yet done)
    • We found the spare box with 3 more cables behind the X-arm tube. The replacement has not been done yet.

Evening work

  • c1sus powering up and CDS check
    • After the people had left, I tried to start up c1sus. I had used dolphin fencing, but it worked like a charm!
  • ADC1 AA chassis repair

To Do on Mon (10AM~)

  • Long DB cable installation (contd)
  • DSUB cable labeling
  • Sus control system recovery
  • c1lsc fiber reinstallation and system recovery
  • Tool / Debris cleaning

Eventual needs:

  • We need good crimping tools.
  • Supply shortage of the crimping connectors.
  • Fibers should not be routed together with electronics cables. Fibers should be distributed through tubes hanging on the cable racks
  • Cable strain relief
  • Move the noisy CDS and DC power supplies to the drill press room.

Rack nut policy

Out rack nut / screws are so much contaminated.
It's a mixture of #10-32 (standard) / M5 (wrong) / M6 (wrong).
Even the labeled bottles are contaminated.
Don't believe the installed rack nuts/screws, even if they seemed to work fine. They may be a metric pair.

Golden standard (Attached photo)

  • rack nuts marked 1032
  • small washers
  • 10-32 tapered-round head screws

If you find other hardware, don't mix them with any stocks. Give them to Rack Nut Police (=Koji). He will hide them to some where secret.

 

Attachment 1: PXL_20230915_194539031~2.jpg
PXL_20230915_194539031~2.jpg
Attachment 2: PXL_20230916_005624158.MP.jpg
PXL_20230916_005624158.MP.jpg
Attachment 3: PXL_20230916_005531262.MP.jpg
PXL_20230916_005531262.MP.jpg
  17846   Fri Sep 15 18:50:34 2023 KojiUpdateCDSDolphin Fencing Investigation / Full CDS crash / nodus reboot / recovered all

Dolphin Fencing technique

I believe that the dolphin emits some glitches to the other hosts during the host machine shutting down and starting up.
However, if the dolphin is disabled, that FE process will not run.

Therefore, we need some technique:

  • When you have a real-time host to be restarted, we can disable the dolphin of that machine.
    e.g. If c1lsc has a problem, run the following command on fb1.
    ./dolphin_ix_port_control.sh --disable 192.168.113.40 2
    This allows us to restart the c1lsc in a safe way.
     
  • Restart c1lsc in the above example. Go to c1lsc and run
    sudo reboot; exit
     
  • This above brings you back to the previous host you were (suppose it is fb1). Run ping on that restarting machine.
    ping c1lsc
     
  • While the c1lsc is shutting down, ping still has the response. Once the restart starts, it makes no response. Then, you can enable dolphin.
    ./dolphin_ix_port_control.sh --enable 192.168.113.40 2
     
  • The process comes back automatically. You'll see DK status during the restart. I should disappear once all the models are up.

 

  17847   Fri Sep 15 22:57:15 2023 KojiSummaryElectronicsc1sus ADC1 AA chassis fix

c1sus ADC1 AA chassis fix.

  • Brought the chassis on the workbench.
  • Opened the chassis and "alas!" The internal power cables were not connected to the PCBs. This makes sense why there was no current draw at all.
  • The cables were connected.
  • The unit was tested with +/-18V power. At least, the diff outputs of the AA boards were all 0V.
  • A missing connector screw for the external power connector was fixed

The unit was installed on the 1X5 rack, and the DB9 cables were connected.
Repair mission completed.

I've turned on the Eurocard crate (+/-15V) and the AA units (+/-18V) to check if the oplev channels are working. (It seems to be running well)
Also, the AA units are not too hot so far.

Attachment 1: PXL_20230916_025737289.jpg
PXL_20230916_025737289.jpg
  17848   Mon Sep 18 10:26:12 2023 KojiSummaryElectronicsVertex Electronics Transition ~ DAY 4

[JC, Paco, Radhika, Koji]

Morning/Afternoon work:

  • Long DB cable installation (contd)
    • 14 DB25 cables went through the cable-rack bridge above the ITMX chamber. It's twice the previous # of cables.
    • These cables were connected to the chamber flanges. ITMY Flange1 had been having the 2nd connector malfunction. So the cables were connected to the connector 1 and 3 (as before).
  • DSUB cable labeling
    • All the (long) cables connected to the units were labeled appropriately.
  • c1lsc fiber reinstallation
    • We found one fiber cable (AlpenIO Inc PCIe 4x10G 100m AIO-PCIe4X-100 (2010)) already routed from 1Y4 to the PSL rack. This is the spare JC told us. We routed the host end to 1Y7.
    • c1lsc is up and running as before. All the models are up an running (burtrestore still needed).
  • c1sus channel assignment
    • We have the swap of ADC0/1/2 so that the oplev ADC will have ADC0 (duo tone at CH31)
    • The channel assignments were modified:
      • SUS numbering "n": (0-PRM / 1-BS / 2-ITMX / 3-ITMY / 4-SRM / 5-MC2 / 6-MC1 / 7-MC3)
      • Face OSEMs ADC1 CH (n x 4 + 0~3, UL/LL/UR/LR)
      • Side OSEM ADC2 CH n
      • Oplev Ch ADC0 CH (n x 4 + 0~3)
      • Face OSEMs DAC0 CH (n x 4 + 0~3)
      • Side OSEM DAC1 CH n

  • Sus damping control recovery
    • We need to lookin to MC3 UL/UR, PRM SD, SRM UL/LL/UR, ITMX all, ITMY face. See next post.
  • Tool / Debris cleaning
Attachment 1: PXL_20230918_233642072.MP.jpg
PXL_20230918_233642072.MP.jpg
Attachment 2: PXL_20230918_233659231.jpg
PXL_20230918_233659231.jpg
  17849   Mon Sep 18 18:38:02 2023 RadhikaSummaryElectronicsVertex Electronics Transition ~ DAY 4

[Koji, Paco, Radhika]

We recorded the 5 OSEM sensor readings for each of the 8 upgraded optics.

1. The correct scale factor seems to be 9x for the sensible OSEM readings. This is consistent with the scale factor calculated here.

2. The expected counts for each sensor is between 10,000-15,000 cts.

3. Several OSEM sensor values have bad readings of ~0 cts, or a few orders of magnitude smaller than expected:

      - ITMX UL/UR/LR/LL
      - ITMY LR/LL/SD
      - SRM UL/UR/LL
      - MC3 UL/UR

4. Several OSEM sensor values have readings < 0 (there's overlap with the previous group):

      - ITMX UR/LR/LL/SD
      - ITMY SD
      - PRM SD
      - SRM UL/UR/LL
      - MC3 UL/UR

5. MC1 has a consistent scale factor of 2.15, quite smaller than expected. Note that its OSEM readings were negative before the upgrade and now positive; hence negative MC1 ratios below.

Here is the full matrix of OSEM ratios after/before upgrade:
 

  UL UR LR LL SD
BS 7.53508553 9.80529834 10.82905875 8.61769586 1.02324445
ITMX 2.18227361e-03 -2.45700834e-03 -2.88710265e-03 -7.09635068e-03 -8.06591717
ITMY 5.11154762 3.57325499e+01 9.24866311e-03 9.05692306e-01 -3.00528448e-02
PRM 7.22051955 7.45579712 4.8889959 84.63438221 -1.4148838
SRM -8.86075335e-04 -8.39000436e-04 1.68963158e+02 -3.14350110e-02 1.12586402
MC1 -2.15229241 -2.14837141  -2.14954979 -2.16571496 -2.13574099
MC2 9.09337655 9.17967725  25.52931668 9.03775091 8.89290556
MC3 -4.60602513e-03 -1.06510781e-03 9.05368452 8.99212312 9.07521747

Next steps
 

We will debug the corresponding circuits tomorrow.

  17852   Mon Sep 18 20:16:03 2023 KojiSummaryElectronicsVertex Electronics Transition ~ DAY 4

- Here is the thought how does the factor of 9 come from:

  • We are driving OSEM LEDs at 35mA rather than at 25mA. (Honeywell LED SME2470 has quite a linear response for  Irradiance vs. Forward Current.)
  • The TIA of the OSEM PD is now 121K instead of the previous 39.2K
  • The OSEM output is received by differential AA.

--> Naive estimation is (35/25) x (121k/39.2k) x 2 = 8.64.

- MC1 sat amp has already been replaced with the aLIGO version by Gautam. I wonder where this factor of 2.15 came from (not 2...?).


- Coil driver response:

Previous setup

--> BS/PRM/ITMX/ITMY 2 VDAC/118 Ohm = 1.7e-2 A/V x VDAC

--> MC1/MC2/MC3/SRM VDAC/118 Ohm = 8.5e-3 A/V x VDAC

--> All the side coils VDAC/118 Ohm = 8.5e-3 A/V x VDAC

New setup

  • The AIs have the gain of 1.
  • The coil driver has a gain of 1.2.
  • The output Rs for the face coils are 1.2k//100Ohm = 92Ohm

--> 2 VDAC * 1.2 / (92+18) Ohm = 2.2e-2 A/V x VDAC

BS/PRM/ITMX/ITMY face coils will have x1.3 more actuation.

MC1/MC2/MC3/SRM face coils will have x2.6 more actuation.

  • The output Rs for the side coils are 1.2k

--> 2 VDAC * 1.2 / 1200 Ohm = 2.0e-3 A/V x VDAC

MC1/MC2/MC3/SRM will have less actuation by a factor of 1/4.25.

 

 

ITMX 400Ohm
ITMY 400Ohm
BS 100
PRM 100
SRM 100
MC2 427.5/410/411/409.4/410
MC1 434.5/428.4/430.6/432.5/434.0
MC3 432.2/409.4/409.3/410.6/413.8
 

  17853   Mon Sep 18 23:09:40 2023 KojiSummaryElectronicsVertex Electronics Transition ~ DAY 4

MC1 is ready for the damping test

Trouble shooting plan

  • Is the LED on? => Check all the LED mon outputs of the sat amp. It should show 5V if the output current is 35mA. If the constant current loop is open (eg no LED / connection failure etc), it rails at the supply voltage.
  • Also the CCD videos should show the status of the LEDs (at least for the TMs and the MC mirrors)
  • Then is the PD out responding? => Check all the PD mons.
  • If the PD mons are normal, but there is no signal it can be the AA problem. Inject test signal to that channel on the AA and see if we can see some number on the CDS.
     
  • Is the coil current flowing? => Check if the coil drv mons are responding.
  • If not, check if the AI output has the DAC output in that channel.
  • If the DAC signal is there, but no current it can be the driver issue, or the coil/cable/flange connection issue.
  17854   Tue Sep 19 17:25:38 2023 RadhikaSummaryElectronicsVertex Electronics Transition ~ DAY 5

MCs OSEM input / coil output gain tuning

Seeing that MC1, MC2, MC3 OSEM readings looked reasonable and consistent, I worked on updating the input OSEM cts2um filter for the 3 suspensions. MC1 OSEM input gains were changed by a factor of 2.15; MC2 and MC3 OSEM input gains were changed by a factor of 8.64 (see previous ELOG for source of these factors).

OLD cts2um gains (units are um/ct):

  UL UR LR LL SD
MC1 0.105 0.078 0.065 0.087 0.09
MC2 0.415 0.361 0.782 0.415 0.36
MC3 0.509 0.424 0.365 0.376 0.36

NEW cts2um gains (units are um/ct):

  UL UR LR LL SD
MC1 0.0488 0.0363 0.0302 0.0405 0.0419
MC2 0.0480 0.0418 0.0905 0.0480 0.0417
MC3 0.0589 0.0491 0.0422 0.0435 0.0417

Next, I moved onto the coil output filters for MC1, MC2, MC3. There was no gain filter already in place for these coil outputs, so I created one called V2A. (This name can be changed. Note: for other optics the coil actuation scaling filters are titled "xN" for scaling N. Eventually we will find an elegant way to set these scalings.)  The coil outputs for MC1, MC2, MC3 were changed by a factor of 1/2.6, or 0.385 (see previous ELOG for source of this factor).

I ran into an issue saving the foton filter coefficients: the filters appear to be saved; however, "Load Coefficients" does not load them onto the medm screen for MC2_URCOIL and all MC3 coils. I've tried toggling the save button and Load Coefficients button, but no luck. I checked and the filters are saved in opt/rtcds/caltech/c1/chans/C1MCS.txt. When changing an existing filter gain, the change is not being applied to the output channel.

MC1 damping

Since all MC1 coil filters saved and loaded successfully but not MC2 or MC3, I was only able to test the damping of MC1. Turning on the damping filters did not supress motion, so I made the following changes:

1. Prior to the upgrade, MC1 OSEM readings were all negative. Now they are positive, so I reversed the signs of all OSEM input gains (-1 -----> +1) [Attachment 1]. This is now consistent with all other optics' OSEM sensor gains.
2. I noticed that all MC1 coil output gains were negative. The [UL, UR, LR, LL, SD] coil output sign convention for each other optic is [+, -, +, -, +], or that but flipped. So I flipped the signs of MC1 UL/LR/SD coil output gains to match the [+, -, +, -, +] pattern.

Attachment 2 shows the damping of MC1 after these changes were made. It looks side the SIDE mode is underdamped and we may want to increase its servo gain. (I flipped the sign of the coil output and confirmed it ringed up, so the sign is correct.)

Next steps

1. Debug why foton is not saving new coil output scaling filters for MC2 UR and all MC3 coils.

2. Assess MC2 and MC3 damping.

 

Attachment 1: MC1_OSEM_input_gains.png
MC1_OSEM_input_gains.png
Attachment 2: MC1_damping_kick.png
MC1_damping_kick.png
  17855   Tue Sep 19 19:21:10 2023 RadhikaSummaryElectronicsVertex Electronics Transition ~ DAY 5

[Koji, Radhika]

Update to the Foton/Load Coefficients issue:

- "Save" in foton writes/updates filters correctly to chans/C1MCS.txt. However, "Load Coefficients" is currently not reading C1MCS.txt and therefore not loading any changes at all.

    - We saved a backup of C1MCS.txt and replaced it with one from this morning, hoping to see that the new V2A filters vanish from MC1 coil outputs. When we loaded coefficients, the V2A filters remained.

- We then checked if this issue was happening with C1SUS by adding a test filter to ETMY_ULCOIL. Indeed "Load Coefficients" loaded the filter. So it seems to just be a C1MCS issue.

- We restarted C1MCS twice and no change.

  17856   Tue Sep 19 19:48:20 2023 KojiSummaryElectronicsVertex Electronics Trouble shooting

[Paco, Murtaza, JC, Koji, Radhika]

MC1/MC2 was working fine.

At this point MC3, SRM, and ITMY are also working fine.


The custom DB25 cables between the sat amps and the flanges are difficult to mate.

  • The finger tight was not enough to make all the contacts. Fastening the screws with a screwdriver made MC3 start working fine on CDS.
  • The custom cable fastening screw on PRM(1st) was stripped at the flange side. It needs a thread dyeing. The SRM(2st) has the hex nut broken on the sat amp. Need to be fixed.

We checked if the LED mon shows the correct values. When it is connected it shows 5V. If the LED is not connected it shows 0.8V. It goes 0.08V in an unknown state.

  • SRM1 all channels were 0V. It turned out that the connection inside the vacuum chamber seemed mirrored. Right now we have temporary mirror ribbon cables to fix this issue. We need two shielded mirror cables for SRM1 and SRM2.
  • SRM2 was random (5V, 0.8V, and 0V)
  • BS1/2/ITMY1 looked fine.
  • ITMY2 was strange.
  • ITMX1/2 were completely silent.

We suspected the cable pinouts/ cable mating issue etc, but it turned out that the SRM2 cable was mislabeled and the ITMY2 cable was connected to the SRM sat amp. That's why it was so random. We corrected the connection and SRM and ITMY started working fine on CDS.

We used the OSEM simulation box to test the sat amps. That suggests that the BS/PRM/ITMX problem may be coming from the SAT amps. We need to look into the sat amps.

  17857   Tue Sep 19 20:49:08 2023 KojiSummaryElectronicsVertex Electronics Trouble shooting

[Murtaza, Koji]

ITMX / BS / PRM sat amps were removed from the rack and checked on the workbench. They all work fine with the OSEM simulation box.


With the correct circuit, the LED mon should be 5V, and the PD readout should be 2.6~3.0V (i.e. the differential output has twice the voltage difference of this number).

ITMX Sat Amp fixed:

  • The internal wiring for CH1-4 was not connected (or disconnected by mechanical impact) (Attachment 1)
  • PD1 channel for CH1 had a metal debris on the transimpedance opamp (Attachment 2)
  • The internal board for CH5-6 was connected in the opposite direction. This was because both connectors on the board had the wrong genders.
    This was replaced with a spare. (There was two spares and I consumed one now) (Attachment 3)
     
  • Put a ventilated lid instead of the solid lid.

BS Sat Amp:

  • All the CHs just worked fine.

PRM Sat Amp:

  • Found the bias selector jumpers had not been installed. Fixed. (Attachment 4/5)
Attachment 1: PXL_20230920_031834833.jpg
PXL_20230920_031834833.jpg
Attachment 2: PXL_20230920_033913077.jpg
PXL_20230920_033913077.jpg
Attachment 3: PXL_20230920_034246461.jpg
PXL_20230920_034246461.jpg
Attachment 4: PXL_20230920_035930889.jpg
PXL_20230920_035930889.jpg
Attachment 5: PXL_20230920_035933871.MP.jpg
PXL_20230920_035933871.MP.jpg
  17858   Tue Sep 19 23:40:33 2023 KojiSummaryElectronicsVertex Electronics Wed Plan

Plan for Sept 20, 2023

For morning people:

  • We don't need to replace the long cables. They seem all fine.
  • Close the lid of the repaired sat amps. Use a lid with ventilation slits (there is an extra with the empty unit on the same desk).
  • Install the sat amps back to the racks. Connect all the cables us. Check if this makes the OSEM values to positive 10~20k counts.
    • If not, check LED mons and PD mons. If they are OK the sat amp is working fine. Track the signal down to the AI chassis to see if the units after the sat amp are working well.
  • The SRM2 and ITM2 cables ( connected to the sat amps at the back of the units) cross (i.e. have twisted) at the rack. Please reroute and nicely coil them up.
  • Fix the custom cable issues: "The custom cable fastening screw on PRM(1st) was stripped at the flange side. It needs a thread dyeing. The SRM(2st) has the hex nut broken on the sat amp. Need to be fixed."
  • Put the proper labels on the long cables at the flanges and the sat amps. The labels should indicate where the connectors are supposed to be connected.
  • Clean up the mess and the tools from the lab.

The

After the weekly meeting, we'll continue to work on the suspension control. The lab will be turned to be LASER HAZARD in the afternoon.

  17859   Wed Sep 20 00:03:22 2023 KojiSummaryElectronicsre: Filter Coefficient Loading Issue

I asked CDS mattermost for help. Chris (Wipf) checked it and reported it is working fine as usual (without fixing anything).

I've reverted the copied C1MCS.txt back in the chans dir (/opt/rtcds/caltech/c1/chans). The filter coefficients were loaded from the GDS screen. The filters were properly updated.

Here is the info from Chris:

Some transient NFS problem, maybe?

One possible clue is that the filter file that the FE actually loads is not chans/<model>.txt,
but chans/tmp/<model>.txt. Before loading, the filter file is copied into the tmp directory,
and a diff of the two files is written to chans/tmp/<model>.diff.
This diff file should normally be an empty file, indicating that the two files match.
But it was not empty at first, so there must have been some issue with the previous load.
After I reloaded, the diff then became empty.

  17860   Wed Sep 20 00:20:09 2023 KojiSummaryElectronicsVertex Electronics ~ change in the actuator calibration

I found the actuator calibration is more complicated. The numbers I reported in the previous elog was not correct.

Here I summarize the numbers of the voltage-to-current conversion.

=== Previous===

Coils DAC
receiver
Coil driver
gain
Coil driver
output R (Ohm)
Coil
R (Ohm)
VDAC Voltage
to Current conversion (mA/V)
PRM Face Diff (2) 1 100 18 17.
PRM Side SE (1) 1 100 18    8.5
BS Face Diff (2) 1 100 18 17.
BS Side SE (1) 1 100 18    8.5
ITMX Face Diff (2) 1 400 18   4.8
ITMX Side SE (1) 1 400 18   2.4
ITMY Face Diff (2) 1 400 18   4.8
ITMY Side SE (1) 1 400 18   2.4
SRM Face SE (1) 1 100 18   8.5
SRM Side SE (1) 1 100 18   8.5
MC2 Face SE (1) 1 420 18   2.3
MC2 Side SE (1) 1 420 18   2.3
MC1 Face SE (1) 1 420 18   2.3
MC1 Side SE (1) 1 420 18   2.3
MC3 Face SE (1) 1 420 18   2.3
MC3 Side SE (1) 1 420 18   2.3

=== New ===

e.g. ITMX face coil electronics are x4.6 stronger than the previous coil electronics.

Coils DAC
receiver
Coil driver
gain
Coil driver
output R (Ohm)
Coil
R (Ohm)
VDAC Voltage
to Current conversion (mA/V)

Ratio
New/Old

PRM Face Diff (2) 1.2 92 18 22. 1.3
PRM Side Diff (2) 1.2 1200 18     2.0 0.235
BS Face Diff (2) 1.2 92 18 22. 1.3
BS Side Diff (2) 1.2 1200 18     2.0 0.235
ITMX Face Diff (2) 1.2 92 18 22. 4.6
ITMX Side Diff (2) 1.2 1200 18     2.0 0.83
ITMY Face Diff (2) 1.2 92 18 22. 4.6
ITMY Side Diff (2) 1.2 1200 18     2.0 0.83
SRM Face Diff (2) 1.2 92 18 22. 2.6
SRM Side Diff (2) 1.2 1200 18     2.0 0.235
MC2 Face Diff (2) 1.2 92 18 22. 9.6
MC2 Side Diff (2) 1.2 1200 18     2.0 0.87
MC1 Face Diff (2) 1.2 92 18 22. 9.6
MC1 Side Diff (2) 1.2 1200 18     2.0 0.87
MC3 Face Diff (2) 1.2 92 18 22. 9.6
MC3 Side Diff (2) 1.2 1200 18     2.0 0.87

 

  17861   Wed Sep 20 14:04:58 2023 RadhikaSummaryElectronicsVertex Electronics ~ change in the actuator calibration

MC1/MC2/MC3 damping restored

I tweaked the coil actuation gains for MC1/MC2/MC3 according to Koji's updated calculations:

1/9.6 for face coils
1/0.87 for side coils

With foton and load coefficients working as expected, these coil output filters were successfully added to MC2 and MC3.

Damping tests

Note: While burt restoring C1MCS to a pre-upgrade state, a "NOT OK" flag popped up and the coil balancing gains for MC1/MC2/MC3 were reset to +-1. Koji showed me how to access the original values in c1mcs.snap (using grep) and I restored the coil gains to their values from 9/12/2023.

- *Recall from past ELOG that MC1 OSEM input gains all switched from -1 ---> +1; and MC1 coil output gains changed signs from [-,-,-,-,-] ----> [+,-,+,-,+]. No changes were made to MC2 or MC3.*
- Turned on damping filters
- Gave an offset of 10000 cts to C1:SUS-MC1/2/3_ULCOIL_OFFSET. OSEM striptools can be found in Attachments 1,2,3.

Next steps

- Get BS/ITMX/ITMY/PRM/SRM online and apply new sensor/actuator scaling factors
- Confirm damping works as expected for above suspensions
- Bring IFO to nominal alignment
- Revisit upgraded suspensions and perform fine tuning (input matrix diagonalization, coil balancing)
Attachment 1: MC1_damping_kick_2023-09-20.png
MC1_damping_kick_2023-09-20.png
Attachment 2: MC2_damping_kick_2023-09-20.png
MC2_damping_kick_2023-09-20.png
Attachment 3: MC3_damping_kick_2023-09-20.png
MC3_damping_kick_2023-09-20.png
  17862   Wed Sep 20 17:02:22 2023 RadhikaSummaryElectronicsVertex Electronics ~ change in the actuator calibration

[Paco, Radhika]

IMC LOCKED

We used the pre-upgrade C1:SUS-MC1/2/3_SUSPIT/YAW/POS_INMON values as a baseline to restore IMC alignment.

Procedure we followed:

1. Use MCR spot position to align MC1.
2. Move MC3 to try to hit OSEMs on MC2F. Note down these MC3 PIT/YAW offset values and navigate to their center to align MC3.
3. Now move MC2 to steer the beam back around the cavity and hit MC2 OSEMs once again. Alignment is very close! Continue to move until flashing is observed.
4. IMC autolocker kicks in; burt restore c1iooepics.snap to restore WFS.
  17863   Wed Sep 20 17:28:17 2023 RadhikaSummaryElectronicsre: Filter Coefficient Loading Issue

I noticed the same issue today with C1RMS.txt, when trying to update the coil actuation gains for SRM. The filter changes were saved to chans/C1RMS.txt, so next I checked chans/tmp/. There is no chans/tmp/C1RMS.txt, or chans/tmp/C1RMS.diff. The updated filters do not load.

Update from Chris:

C1RMS.txt is a remnant from some model that doesn't exist anymore. It can be removed.

I went ahead and deleted chans/C1RMS.txt and chans/tmp/C1RMS.txt.

  17864   Wed Sep 20 18:18:38 2023 KojiSummaryElectronicsVertex Electronics Wed Plan

[Koji, Murtaza, JC]

Regarding PRM/BS:

  • PRM2 cable and BS2 cable were wrongly connected. This was corrected.
     
  • This makes the BS face OSEM values reasonable.
    However, the side signal is still close to zero. We confirmed that the sat amp outputs (LED mon/PD mon/PD diff out) looked reasonable for all five BS OSEMs.
    The side signal issue stays downstream of the Vertex ADC adapter.

     
  • PRM2 has no issue with the sat amp.
  • PRM1: We found that all the LED mon goes down to 0.17V when the vacuum flange is connected.
    It was found that the reference voltage for the LED (TP11 of D080276) went down to low number (like 0.15V) when the in-vac OSEMs were connected.
    I found that this output was not stable. So, I replaced the U4 chip (AD8672), but this didn't help the voltage sagging issue.
  • Murtaza and I started checking the short circuits on the flange. We found that Pin 5 (OSEM PD1 Kathode) and Pin 1 (invac cable shield?) only have 5.1 Ohm. Pin 1 is connected to the vacuum chamber.
    • What does it mean? The PD has the reverse bias voltage of 10V applied on the PD Kathode. This bias voltage is shorted to ground via 5 Ohm. To keep the bias line at 10V, we need 2 A.
    • We don't have many options:
      - We can disconnect the internal wire for pin5 from the cable. (Prepare a ribbon cable). This should make the other OSEM PDs properly biased.
      We may be able to use an independent power supply to provide some amount of reverse bias (10V 2A is too much. Probably 1V 0.2A or 2.5V 0.5A?) so that the UL PD somewhat work.
  17865   Thu Sep 21 12:02:25 2023 KojiUpdateGeneralPower Outage Sept 21, 2023 ~9AM

[JC, Paco, Koji]

We had a power outage on Sept 21, 2023 at ~9AM. This is the third power outage this month as far as I remember.

- JC reported the outage was ~2sec. Some UPS supported machines were affected, while some unsupported machines also survived the incident cf c1psl (what!?)

- Some machines were rebooted by itself (cf the RTS hosts).

- megatron and optimus were powered up. Autolockers (optimus) and FSSSlow (megatron) were restored.

 

c1vac was still on, but the machine didn't come back online.

- The network adapter was reactivated by running the following commands

> cd /sbin
> sudo ifdown eth0
> sudo ifup eth0

However, the acromag seemed freezer, so c1vac was shutdown, the acromag chassis was power cycled, and the c1vac was rebooted. This brought c1vac fully functional again.
Rebooting made TP1 stop (gracefully)


The vacuum pressure of the main volume was high.

- We found that the vacuum pressure was up to 1e-2 torr in the afternoon when we started the recovery. In fact, the main gate valve was close at the power outage last week. See attachments.

- We made sure the valves were properly open/closed and started TP1 again. Once TP1 reached 33.6K RPM we opened the main volume to recover the vacuum pressure.

- The vacuum pressure came back to <1e5 Torr.

Attachment 1: Screenshot_2023-09-21_at_12.19.22.png
Screenshot_2023-09-21_at_12.19.22.png
Attachment 2: Screenshot_2023-09-21_at_12.18.25.png
Screenshot_2023-09-21_at_12.18.25.png
  17866   Thu Sep 21 14:22:02 2023 RadhikaSummaryElectronicsVertex Electronics CDS Update

I recalculated the scale factors between OSEM sensor readings after/before the upgrade. The expected factor is 8.64, although we may want to rethink this if measurements are disagreeing.]

If ITMY can be restored, we can proceed to locking the YARM while PRM/ITMX/BS are worked on.
 

1. BS values seem reasonable

[UL: 7.52500126;  UR: 9.78603403;  LR: 10.80333519;  LL: 8.58031299;  SD: 4.90237845]

The SD reading is positive and nonzero, even though its still smaller than the face sensor readings by a factor of 2.
 

2. ITMX SD still negative

[UL: 12.76223943;  UR: 6.40189513;  LR: 8.52507381;  LL: 21.55229024;  SD: -7.9675249]


3. ITMY SD flipped from positive to negative ~3pm 9/21. LL is too small.

[UL: 5.21918023;  UR: 35.90315905;  LR: 19.1338805;  LL: 0.90731276;  SD: -3.68291338]


4. PRM still not reliable

[UL: -0.00064235;  UR: -0.00061758;  LR: -0.00154483;  LL: -0.00583698;  SD: -0.00283635]


5. SRM positive but scale factors widely inconsistent, order of magnitude greater than expected (~8.64).

[UL: 7.22499189;  UR: 3.36944117;  LR: 116.14296786;  LL: 114.59917629;  SD: 1.91497475]
  17867   Fri Sep 22 16:20:25 2023 RadhikaSummaryElectronicsVertex Electronics CDS Update

Today we tried to debug the unreasonable OSEM readings (see previous ELOG)

PRM process

Starting state: PRM face + side values bogus (~0)

1. Somehow through retightening connections, PRM LR+SD counts looked reasonable (and positive). Yay!

2. Toggled on and off the PRM SATAMP; removed Ch1-4 and Ch5-8 PRM inputs

    - Result: BS SD becomes negative when PRM SATAMP is on and Ch 5-8 cable is connected.

2. We disconnected the PRM SATAMP and plugged the PRM inputs into the SRM SATAMP. The SRM SATAMP output was routed to the PRM input on the SATAMP adapter.

    - Result: PRM UL/UR/LL readings still 0.

                 BS SD still negative when SRM SATAMP is turned on and Ch 5-8 from PRM are connected.

                   ---> SRM SATAMP gives same results as PRM SATAMP; PRM SATAMP likely not faulty.

3. Replaced PRM chamber connections with satellite test box for channels 1-4.

    - Result: reasonable PRM UL/UR/LL readings ---> Pin 5 shorting on chamber side is causing issues with Ch1-4.

ITMX - ITMY process

Starting state: ITMX SD counts negative, ITMY SD counts negative (depends on ITMX connection)
To test ITMX (SD negative since change), ITMY (SD negative dependent on ITMX)

 

 We first switched [ITMX, ITMY]  in the following sequence to get the following results (F = average face values, S = side value)(0 = OFF, 1 = ON)

- [0,0] -> ITMX[F,S] = [300, -5000] ITMY[F,S] = [0, -800]

- [1,0] -> ITMX[F,S] = [~, ~] ITMY[F,S] = [1000, -3000]

- [0,1] -> ITMX[F,S] = [-30, -3000] ITMY[F,S] = [~, ~]

- [1,1] -> ITMX[F,S] = [15000, -10000] ITMY[F,S] = [10000, -3000]

 

A separate test was done to test ITMX-ITMY coupling on ITMY-side

1. ITMX SAT AMP OFF -> ITMY SIDE GOOD 

2. ITMX SAT AMP ON (Ch 1-4, 5-8 DISCONNECTED) -> ITMY SIDE =/2

3. ITMX SAT AMP ON (Ch 1-4 CONNECTED) -> NO CHANGE FROM 2.

4. ITMX SAT AMP ON (Ch 1-4, 5-8 CONNECTED) -> SAME MAGNITUDE AS 2., FLIPS SIGN

 

To check if ITMX was faulty from the chamber end for the SIDE DOF, the satellite test box was used for CH 5-8

ITMY SIDE SIGN STILL NEGATIVE 

 

For the final sanity check for the effect of ITMX on ITMY side sign, we swapped the ITMX and SRM SAT AMPS ({front -> PD OUT 1, 2}, {back -> Ch 1-4, Ch 5-8})

ITMY SIDE SIGN STILL NEGATIVE 

 

Summary

PRM Ch1-4 shorting issue on chamber side (UL/UR/LL)

BS/ITMY/ITMX SD <0 all seem to be caused by SATAMP adapter or downsteam in ADC2

SUSPECTED FAULTY SAT AMP ADAPTER for ITMX-ITMY SIDE COUPLNG

Attachment 1: IMG_9200.jpg
IMG_9200.jpg
Attachment 2: IMG_9201.jpg
IMG_9201.jpg
  17868   Fri Sep 22 18:26:08 2023 RadhikaSummaryElectronicsVertex Electronics CDS Update

[Koji, Radhika, Murtaza]

All upgraded suspensions have reasonable OSEM readings! Ready for damping tests and alignment next week.

We fixed PRM OSEM reading by isolating pin5 of the first DB25 [17871]. This makes the PRM UL unbiased by the PD seems to be receiving some light.

The PRM/BS/ITMX/ITMY SATAMP adapter was removed and the front-end pins were checked for shorting. Indeed, a short was found in the SIDE1-4 ribbon cable inside the sat-amp adapter, from the wires being compressed to one side of the dsub-ribbon adapter at the input joint [Attachment 1]. We reclamped the ribbon and verified there was no shorting and that the pins were properly aligned [Attachment 2]. This means PRM/BS and ITMX/ITMY SIDE signals should no longer be cross coupled.

All OSEM counts looked good after these fixes. Only a few ITMY OSEMS looked low, but Koji checked both PDMON voltages for ITMY, and we confirmed with calibration that the OSEM counts were reasonable.

Attachment 1: PXL_20230923_003545536.jpg
PXL_20230923_003545536.jpg
Attachment 2: PXL_20230923_005555079.jpg
PXL_20230923_005555079.jpg
  17869   Fri Sep 22 19:05:19 2023 KojiUpdateGeneralPower Outage Sept 21, 2023 ~9AM

Pumping configuration changed. Now TP2 is backing TP1 and TP3 is pumping annuli.

 

Attachment 1: Screenshot_2023-09-23_01-52-36.png
Screenshot_2023-09-23_01-52-36.png
  17870   Fri Sep 22 19:31:24 2023 KojiUpdateSUSFixed IMC/IFO alignment screens

Fixed IMC/IFO alignment screens

Attachment 1: Screenshot_2023-09-23_02-28-42.png
Screenshot_2023-09-23_02-28-42.png
Attachment 2: Screenshot_2023-09-23_02-28-52.png
Screenshot_2023-09-23_02-28-52.png
  17871   Fri Sep 22 19:38:06 2023 MurtazaSummaryElectronicsVertex Electronics CDS Update

PRM CHANNEL 1-4 (BS FEEDTHROUGH 1-3)

[Koji, Radhika, Murtaza]

Connector on the BS Chamber that feeds to PRM UL/LL/UR coils (PRM 1 in Attachment 1) has pin 5 shorted to pin 1 inside the chamber angry
- To resolve this, a DB25 connector was recycled from the old coil drivers
- pin 5 was isolated by cutting (green cable on the DB25 connector)
- The connector was attached between the chamber and the cable that runs through to the rack (Attachment 1)
- The connector was labelled (Attachment 2)

The PD outputs were read on the PRM SATAMP (Pins 1-4, Pin 5 (Ground))
Pin 1 ~ 5.2V
Pin 2 ~ 5.3V
Pin 3 ~ 6.7V

Pin 4 ~ 0V (Blank Pin)

No need to apply an external bias to Pin 5!

Can be fixed during the next vent!

Attachment 1: IMG_9203.jpg
IMG_9203.jpg
Attachment 2: IMG_9202.jpg
IMG_9202.jpg
ELOG V3.1.3-