I recalculated the scale factors between OSEM sensor readings after/before the upgrade. The expected factor is 8.64, although we may want to rethink this if measurements are disagreeing.]

If ITMY can be restored, we can proceed to locking the YARM while PRM/ITMX/BS are worked on.
 

1. BS values seem reasonable

The SD reading is positive and nonzero, even though its still smaller than the face sensor readings by a factor of 2.
 

2. ITMX SD still negative

3. ITMY SD flipped from positive to negative ~3pm 9/21. LL is too small.

4. PRM still not reliable

5. SRM positive but scale factors widely inconsistent, order of magnitude greater than expected (~8.64).

Today we tried to debug the unreasonable OSEM readings (see previous ELOG)

PRM process

Starting state: PRM face + side values bogus (~0)

1. Somehow through retightening connections, PRM LR+SD counts looked reasonable (and positive). Yay!

2. Toggled on and off the PRM SATAMP; removed Ch1-4 and Ch5-8 PRM inputs

    - Result: BS SD becomes negative when PRM SATAMP is on and Ch 5-8 cable is connected.

2. We disconnected the PRM SATAMP and plugged the PRM inputs into the SRM SATAMP. The SRM SATAMP output was routed to the PRM input on the SATAMP adapter.

    - Result: PRM UL/UR/LL readings still 0.

                 BS SD still negative when SRM SATAMP is turned on and Ch 5-8 from PRM are connected.

                   ---> SRM SATAMP gives same results as PRM SATAMP; PRM SATAMP likely not faulty.

3. Replaced PRM chamber connections with satellite test box for channels 1-4.

    - Result: reasonable PRM UL/UR/LL readings ---> Pin 5 shorting on chamber side is causing issues with Ch1-4.

ITMX - ITMY process

Starting state: ITMX SD counts negative, ITMY SD counts negative (depends on ITMX connection)
To test ITMX (SD negative since change), ITMY (SD negative dependent on ITMX)

 We first switched [ITMX, ITMY]  in the following sequence to get the following results (F = average face values, S = side value)(0 = OFF, 1 = ON)

- [0,0] -> ITMX[F,S] = [300, -5000] ITMY[F,S] = [0, -800]

- [1,0] -> ITMX[F,S] = [~, ~] ITMY[F,S] = [1000, -3000]

- [0,1] -> ITMX[F,S] = [-30, -3000] ITMY[F,S] = [~, ~]

- [1,1] -> ITMX[F,S] = [15000, -10000] ITMY[F,S] = [10000, -3000]

A separate test was done to test ITMX-ITMY coupling on ITMY-side

1. ITMX SAT AMP OFF -> ITMY SIDE GOOD 

2. ITMX SAT AMP ON (Ch 1-4, 5-8 DISCONNECTED) -> ITMY SIDE =/2

3. ITMX SAT AMP ON (Ch 1-4 CONNECTED) -> NO CHANGE FROM 2.

4. ITMX SAT AMP ON (Ch 1-4, 5-8 CONNECTED) -> SAME MAGNITUDE AS 2., FLIPS SIGN

To check if ITMX was faulty from the chamber end for the SIDE DOF, the satellite test box was used for CH 5-8

ITMY SIDE SIGN STILL NEGATIVE 

For the final sanity check for the effect of ITMX on ITMY side sign, we swapped the ITMX and SRM SAT AMPS ({front -> PD OUT 1, 2}, {back -> Ch 1-4, Ch 5-8})

ITMY SIDE SIGN STILL NEGATIVE 

Summary

PRM Ch1-4 shorting issue on chamber side (UL/UR/LL)

BS/ITMY/ITMX SD <0 all seem to be caused by SATAMP adapter or downsteam in ADC2

SUSPECTED FAULTY SAT AMP ADAPTER for ITMX-ITMY SIDE COUPLNG

[Koji, Radhika, Murtaza]

All upgraded suspensions have reasonable OSEM readings! Ready for damping tests and alignment next week.

We fixed PRM OSEM reading by isolating pin5 of the first DB25 [17871]. This makes the PRM UL unbiased by the PD seems to be receiving some light.

The PRM/BS/ITMX/ITMY SATAMP adapter was removed and the front-end pins were checked for shorting. Indeed, a short was found in the SIDE1-4 ribbon cable inside the sat-amp adapter, from the wires being compressed to one side of the dsub-ribbon adapter at the input joint [Attachment 1]. We reclamped the ribbon and verified there was no shorting and that the pins were properly aligned [Attachment 2]. This means PRM/BS and ITMX/ITMY SIDE signals should no longer be cross coupled.

All OSEM counts looked good after these fixes. Only a few ITMY OSEMS looked low, but Koji checked both PDMON voltages for ITMY, and we confirmed with calibration that the OSEM counts were reasonable.

PRM CHANNEL 1-4 (BS FEEDTHROUGH 1-3)

[Koji, Radhika, Murtaza]

Connector on the BS Chamber that feeds to PRM UL/LL/UR coils (PRM 1 in Attachment 1) has pin 5 shorted to pin 1 inside the chamber 
- To resolve this, a DB25 connector was recycled from the old coil drivers
- pin 5 was isolated by cutting (green cable on the DB25 connector)
- The connector was attached between the chamber and the cable that runs through to the rack (Attachment 1)
- The connector was labelled (Attachment 2)

The PD outputs were read on the PRM SATAMP (Pins 1-4, Pin 5 (Ground))

No need to apply an external bias to Pin 5!

Can be fixed during the next vent!

The vertex electronics transition work will begin on Monday. We expect the ongoing ASS-X work to be completed by then. But if it needs more time, we must hear a shouting signal from the ASS team.
Is there any other preparation to be done this week to reasonably compensate for changes in gain and TF associated with the transition?

In preparation for the transition, we want to have long custom DSUB25 cables (D2100675) approximately laid out (I mean on the floor, etc) this week. JC takes care of this.

• The lengths of the cables can be found in the attached wiring diagram.
• Both ends of the cables need to be labeled.
• At which side do we want to absorb the slack?

Transition Plan

• Suspension damping and watchdogs are appropriately taken care of, although we soon stop/remove everything.
• We first remove any existing units not going to be used in the circuit (except around the Eurocard crate oplev interface P2 of the wiring diagram).
• The wirings at the side cross-connects are removed. This includes the removal of the thick cables on the cable racks. This would become a heavy work.
• The DC power strips are attached to the racks, and the DC power wiring should be done at this point. We check the DC supply voltages.
• Install the new units as per the above rack layout and proceed with the DSUB connections. We have sufficient number of DSUB cables (this ELOG).
• Turn the units on one by one to detect any unit failure, just in case. If they are all on, we start work on the CDS restoration work.

We are ready to do the transition from Wed 1PM.

The items for the upgrade was collected around the vertex area (Attached photo).

- aLIGO-style DC power strip (+/-18V) x3
- DC power cables (orange +/-18V)
- Electronics units for the upgrade.
- DSUB (DB9) cables
- Custom DSUB15-DSUB25 cables
- Custom DSUB25 cables

Note on Sorensen:

- Eurocard crate requires +-15V. We can place two 15V Sorensens on 1X4 for Eurocard crate or just leave the current +/-15V supplies.

- The aLIGO units requires +/-18V. We can place two 15V Sorensens on 1X5 or just leave two of the current supplies and set them to +/-18V.

[Radhika, Paco, Murtaza, Koji]

- Removed all the units that will not be used in the new setup.
- Removed the sidepanel crossconnects
- Removed most of the sidepanel power lines except for the top eurocrate at the top of 1X4 (requires +/-15 pale orange and blue).
- Removed the acromag connections
- Removed the connectors of the long suspension cables

We'll resume the work at 10AM. (We'll have breaks for lunch and the seminar at 3PM)

=== Next steps ===

- Continue to remove the long suspension cables.
- Attach the DC power strip
- Continue to clean up the power lines on the rack side
- Prepare the power lines (Guralp requires +/-15V, the Eurocard crate +/-15V, new power strips (x3) +/-18V)
- Install the units on the racks.

[Radhika, Paco, Murtaza, Koji]

Morning work:

• Power Strip assembly
• Power Strip cable crimping
• c1lsc fiber routing (the PCIE fiber was in danger) / c1lsc machine was stopped after dolphin fencing
• We tried to reroute the c1lsc fiber above the racks such that it does not get pitched by the rack doors,
  but it seemed that the fiber was damaged (Attachment 1) and the c1lsc can't talk with the IO chassis anymore. We need to replace the fiber.
  It seems that P/N is PCIEO-4G3-100.0-11 (Samtec)

Afternoon work:

• Removed the sidepanels of 1X3 and 1X4 for easier work
• Removed the long DB25 cables from the old sat boxes.
• DC power strips are installed.
• Finished cleaning up the side cross-connects
• Checked the DC supply conditions.
  • +/-15V Eurocard crate + Guralp requires 0.5A / 0.6A
  • +/-18V appered on the DC power strips correctly
• Cleaned up the floor a bit
• Murtaza noticed that there was some strange intermittent noise around the 1Y0/1 racks.
  It looks like one of the fans for the c1ioo IO chassis are dying.

Evening work:

• Made coil driver short plugs (Attachment 2). This enables the coils while the coil modes are set to be Acquire mode.
• Rack nuts inserted

Tomorrow plan (10:30AM):

• c1sus IO chassis installation
• Unit installation
• Unit powering tests (before connecting them)
• Cabling between the units
• Long DB cable installation (sat box removal)
• c1lsc fiber replacement
• Some above cable removal (fiber, old custom DB25 for MC1, etc)

[Radhika, Paco, Murtaza, Koji]

We made great progress today. It's going well so far.

Morning work (10:30AM~):

• Installed c1sus IO chassis
• Installed all units
• Removed long (previous) custom DB25 cables for MC1

Afternoon work

• Connected all the units to the DC power strip
• Unit powering test was done before the inter-unit DSUBs were connected.
  • We found one AA chassis don't turn on even though internal +/-15V seems supplied. We pulled the unit out.
  • All the other units were fine.
  • Typical current draw of the units: (unit name, positive supply current, negative supply current)
    • AA 0.5A 0.5A (18W)
    • AI 0.3A 0.3A  (11W)
    • BIO 0A 0A (0W)
    • Trillum I/F 0.1A 0.1A (0.4W)
    • Sat amp 0.3A 0.3A (11W)
    • Coil Driver 0.3A 0.2A (9W)
• Cabling between the units
  • Done except for the extracted AA unit and for the BIO units (not needed until we have Acromag).
• Long DB cable installation (sat box removal)
  • Halfway
• c1lsc fiber tracking/replacement (not yet done)
  • We found the spare box with 3 more cables behind the X-arm tube. The replacement has not been done yet.

Evening work

• c1sus powering up and CDS check
  • After the people had left, I tried to start up c1sus. I had used dolphin fencing, but it worked like a charm!
• ADC1 AA chassis repair

To Do on Mon (10AM~)

• Long DB cable installation (contd)
• DSUB cable labeling
• Sus control system recovery
• c1lsc fiber reinstallation and system recovery
• Tool / Debris cleaning

Eventual needs:

• We need good crimping tools.
• Supply shortage of the crimping connectors.
• Fibers should not be routed together with electronics cables. Fibers should be distributed through tubes hanging on the cable racks
• Cable strain relief
• Move the noisy CDS and DC power supplies to the drill press room.

Rack nut policy

Out rack nut / screws are so much contaminated.
It's a mixture of #10-32 (standard) / M5 (wrong) / M6 (wrong).
Even the labeled bottles are contaminated.
Don't believe the installed rack nuts/screws, even if they seemed to work fine. They may be a metric pair.

Golden standard (Attached photo)

• rack nuts marked 1032
• small washers
• 10-32 tapered-round head screws

If you find other hardware, don't mix them with any stocks. Give them to Rack Nut Police (=Koji). He will hide them to some where secret.

[JC, Paco, Radhika, Koji]

Morning/Afternoon work:

• Long DB cable installation (contd)
  • 14 DB25 cables went through the cable-rack bridge above the ITMX chamber. It's twice the previous # of cables.
  • These cables were connected to the chamber flanges. ITMY Flange1 had been having the 2nd connector malfunction. So the cables were connected to the connector 1 and 3 (as before).
• DSUB cable labeling
  • All the (long) cables connected to the units were labeled appropriately.
• c1lsc fiber reinstallation
  • We found one fiber cable (AlpenIO Inc PCIe 4x10G 100m AIO-PCIe4X-100 (2010)) already routed from 1Y4 to the PSL rack. This is the spare JC told us. We routed the host end to 1Y7.
  • c1lsc is up and running as before. All the models are up an running (burtrestore still needed).
• c1sus channel assignment
  • We have the swap of ADC0/1/2 so that the oplev ADC will have ADC0 (duo tone at CH31)
  • The channel assignments were modified:
    • SUS numbering "n": (0-PRM / 1-BS / 2-ITMX / 3-ITMY / 4-SRM / 5-MC2 / 6-MC1 / 7-MC3)
    • Face OSEMs ADC1 CH (n x 4 + 0~3, UL/LL/UR/LR)
    • Side OSEM ADC2 CH n
    • Oplev Ch ADC0 CH (n x 4 + 0~3)
    • Face OSEMs DAC0 CH (n x 4 + 0~3)
    • Side OSEM DAC1 CH n

• Sus damping control recovery
  • We need to lookin to MC3 UL/UR, PRM SD, SRM UL/LL/UR, ITMX all, ITMY face. See next post.
• Tool / Debris cleaning

[Koji, Paco, Radhika]

We recorded the 5 OSEM sensor readings for each of the 8 upgraded optics.

1. The correct scale factor seems to be 9x for the sensible OSEM readings. This is consistent with the scale factor calculated here.

2. The expected counts for each sensor is between 10,000-15,000 cts.

3. Several OSEM sensor values have bad readings of ~0 cts, or a few orders of magnitude smaller than expected:

4. Several OSEM sensor values have readings < 0 (there's overlap with the previous group):

5. MC1 has a consistent scale factor of 2.15, quite smaller than expected. Note that its OSEM readings were negative before the upgrade and now positive; hence negative MC1 ratios below.

Here is the full matrix of OSEM ratios after/before upgrade:
 

Next steps
 

We will debug the corresponding circuits tomorrow.

- Here is the thought how does the factor of 9 come from:

• We are driving OSEM LEDs at 35mA rather than at 25mA. (Honeywell LED SME2470 has quite a linear response for  Irradiance vs. Forward Current.)
• The TIA of the OSEM PD is now 121K instead of the previous 39.2K
• The OSEM output is received by differential AA.

--> Naive estimation is (35/25) x (121k/39.2k) x 2 = 8.64.

- MC1 sat amp has already been replaced with the aLIGO version by Gautam. I wonder where this factor of 2.15 came from (not 2...?).

- Coil driver response:

Previous setup

• BS/PRM/ITMX/ITMY had the differential receiver (cf https://nodus.ligo.caltech.edu:8081/40m/17840)
• All the removed coil drivers have output Rs of 100 Ohm
• The coil resistance is 16~20Ohm.

--> BS/PRM/ITMX/ITMY 2 V_DAC/118 Ohm = 1.7e-2 A/V x V_DAC

--> MC1/MC2/MC3/SRM V_DAC/118 Ohm = 8.5e-3 A/V x V_DAC

--> All the side coils V_DAC/118 Ohm = 8.5e-3 A/V x V_DAC

New setup

• The AIs have the gain of 1.
• The coil driver has a gain of 1.2.
• The output Rs for the face coils are 1.2k//100Ohm = 92Ohm

--> 2 V_DAC * 1.2 / (92+18) Ohm = 2.2e-2 A/V x V_DAC

BS/PRM/ITMX/ITMY face coils will have x1.3 more actuation.

MC1/MC2/MC3/SRM face coils will have x2.6 more actuation.

• The output Rs for the side coils are 1.2k

--> 2 V_DAC * 1.2 / 1200 Ohm = 2.0e-3 A/V x V_DAC

MC1/MC2/MC3/SRM will have less actuation by a factor of 1/4.25.

ITMX 400Ohm
ITMY 400Ohm
BS 100
PRM 100
SRM 100
MC2 427.5/410/411/409.4/410
MC1 434.5/428.4/430.6/432.5/434.0
MC3 432.2/409.4/409.3/410.6/413.8
 

MC1 is ready for the damping test

Trouble shooting plan

• Is the LED on? => Check all the LED mon outputs of the sat amp. It should show 5V if the output current is 35mA. If the constant current loop is open (eg no LED / connection failure etc), it rails at the supply voltage.
• Also the CCD videos should show the status of the LEDs (at least for the TMs and the MC mirrors)
• Then is the PD out responding? => Check all the PD mons.
• If the PD mons are normal, but there is no signal it can be the AA problem. Inject test signal to that channel on the AA and see if we can see some number on the CDS.
   
• Is the coil current flowing? => Check if the coil drv mons are responding.
• If not, check if the AI output has the DAC output in that channel.
• If the DAC signal is there, but no current it can be the driver issue, or the coil/cable/flange connection issue.

MCs OSEM input / coil output gain tuning

Seeing that MC1, MC2, MC3 OSEM readings looked reasonable and consistent, I worked on updating the input OSEM cts2um filter for the 3 suspensions. MC1 OSEM input gains were changed by a factor of 2.15; MC2 and MC3 OSEM input gains were changed by a factor of 8.64 (see previous ELOG for source of these factors).

OLD cts2um gains (units are um/ct):

NEW cts2um gains (units are um/ct):

Next, I moved onto the coil output filters for MC1, MC2, MC3. There was no gain filter already in place for these coil outputs, so I created one called V2A. (This name can be changed. Note: for other optics the coil actuation scaling filters are titled "xN" for scaling N. Eventually we will find an elegant way to set these scalings.)  The coil outputs for MC1, MC2, MC3 were changed by a factor of 1/2.6, or 0.385 (see previous ELOG for source of this factor).

I ran into an issue saving the foton filter coefficients: the filters appear to be saved; however, "Load Coefficients" does not load them onto the medm screen for MC2_URCOIL and all MC3 coils. I've tried toggling the save button and Load Coefficients button, but no luck. I checked and the filters are saved in opt/rtcds/caltech/c1/chans/C1MCS.txt. When changing an existing filter gain, the change is not being applied to the output channel.

MC1 damping

Since all MC1 coil filters saved and loaded successfully but not MC2 or MC3, I was only able to test the damping of MC1. Turning on the damping filters did not supress motion, so I made the following changes:

Attachment 2 shows the damping of MC1 after these changes were made. It looks side the SIDE mode is underdamped and we may want to increase its servo gain. (I flipped the sign of the coil output and confirmed it ringed up, so the sign is correct.)

Next steps

1. Debug why foton is not saving new coil output scaling filters for MC2 UR and all MC3 coils.

2. Assess MC2 and MC3 damping.

[Koji, Radhika]

Update to the Foton/Load Coefficients issue:

- "Save" in foton writes/updates filters correctly to chans/C1MCS.txt. However, "Load Coefficients" is currently not reading C1MCS.txt and therefore not loading any changes at all.

    - We saved a backup of C1MCS.txt and replaced it with one from this morning, hoping to see that the new V2A filters vanish from MC1 coil outputs. When we loaded coefficients, the V2A filters remained.

- We then checked if this issue was happening with C1SUS by adding a test filter to ETMY_ULCOIL. Indeed "Load Coefficients" loaded the filter. So it seems to just be a C1MCS issue.

- We restarted C1MCS twice and no change.

• [OK] Reflected the sorensen setup (minimal change from the conventional config. (See the attachment)

• Before destroying the current setup, bring the alignment biases for the vertex 8 sus to zero and record all the OSEM values.
  => Radhika did it (next ELOG)
  -> This will give us the ratio of the OSEM error signals to know the gain ratios between before and after. Also this will make it easier to bring the alignment back.

• [OK] I suppose the oplevs are still aligned. We don't need to be too nervous about the oplev spot too much.

• How to compensate the coil force cal? Do we know the ratio from the ETM coil driver swap? (What were the coil output Rs? What are they now? Are the ratios reasonable?)

  • Currently:
    - PRM/BS/ITMX/ITMY DAC output for face coils differential
    - SRM/MC2/MC1/MC3 DAC output for face coils single ended
    - All 8 SD coils single ended
    - Coil Output Rs
      According to D1700218
      PRM unknown to be checked
      BS 100Ohm
      ITMX 100Ohm
      ITMY 100Ohm
      SRM 100Ohm
      MC2 430? unknown to be checked
      MC1 430? unknown to be checked
      MC3 430? unknown to be checked

  • New setup
      DAC output differential
      AI has the gain of 1 / HAM coil driver has a gain of 1.2
      Coil output Rs:
      For all the face coils 1.2k // 100 ~ 92Ohm
      For all the side coils 1.2k

OSEM values for 8 vertex optics ~before~ electronics upgrade (averaged over 60 s):

['C1:SUS-BS_ULSEN_IN1', 'C1:SUS-BS_URSEN_IN1', 'C1:SUS-BS_LRSEN_IN1', 'C1:SUS-BS_LLSEN_IN1', 'C1:SUS-BS_SDSEN_IN1', 'C1:SUS-ITMX_ULSEN_IN1', 'C1:SUS-ITMX_URSEN_IN1', 'C1:SUS-ITMX_LRSEN_IN1', 'C1:SUS-ITMX_LLSEN_IN1', 'C1:SUS-ITMX_SDSEN_IN1', 'C1:SUS-ITMY_ULSEN_IN1', 'C1:SUS-ITMY_URSEN_IN1', 'C1:SUS-ITMY_LRSEN_IN1', 'C1:SUS-ITMY_LLSEN_IN1', 'C1:SUS-ITMY_SDSEN_IN1', 'C1:SUS-PRM_ULSEN_IN1', 'C1:SUS-PRM_URSEN_IN1', 'C1:SUS-PRM_LRSEN_IN1', 'C1:SUS-PRM_LLSEN_IN1', 'C1:SUS-PRM_SDSEN_IN1', 'C1:SUS-SRM_ULSEN_IN1', 'C1:SUS-SRM_URSEN_IN1', 'C1:SUS-SRM_LRSEN_IN1', 'C1:SUS-SRM_LLSEN_IN1', 'C1:SUS-SRM_SDSEN_IN1', 'C1:SUS-MC1_ULSEN_IN1', 'C1:SUS-MC1_URSEN_IN1', 'C1:SUS-MC1_LRSEN_IN1', 'C1:SUS-MC1_LLSEN_IN1', 'C1:SUS-MC1_SDSEN_IN1', 'C1:SUS-MC2_ULSEN_IN1', 'C1:SUS-MC2_URSEN_IN1', 'C1:SUS-MC2_LRSEN_IN1', 'C1:SUS-MC2_LLSEN_IN1', 'C1:SUS-MC2_SDSEN_IN1', 'C1:SUS-MC3_ULSEN_IN1', 'C1:SUS-MC3_URSEN_IN1', 'C1:SUS-MC3_LRSEN_IN1', 'C1:SUS-MC3_LLSEN_IN1', 'C1:SUS-MC3_SDSEN_IN1']

[1943.7368693033854, 1597.8396443684896, 1667.4166076660156, 1709.3347656250003, 1760.058290608724, 1013.2686828613281, 1833.5396423339844, 2096.6071411132816, 786.2030975341798, 1152.7826700846356, 923.4767690022786, 447.51257578531903, 678.4539184570312, 971.2538736979167, 720.261508178711, 2458.092639160156, 2245.09764811198, 710.1112263997396, 258.2382120768228, 1225.202982584636, 3048.1043741861986, 3092.901733398437, 77.94405148824053, 181.08351745605466, 5145.644441731771, -6879.474226888021, -6954.500219726562, -3615.385673014323, -5468.913354492189, -3260.200516764323, 1555.9213073730468, 1625.1883911132813, 348.9968526204428, 1011.3114247639974, 1386.0941060384114, 1014.3825622558594, 1218.7048522949217, 1642.9303202311198, 1621.7448221842449, 720.2911783854166]

- 4x 1064nm NPROs are OFF. The lab hall is laser safe, although the oplev lasers are on.
  The Laser Warning Signs were turned off by the interlock switch at the PSL enclosure (control room side)

- Watch dogs were turned to "disabled"
- Halted c1sus using dolphin fencing. This worked very well.
The previous report of dolphin fencing not working was due to a typo in my instruction (wrong -disable -> correct --disable).

controls@fb1:~ 0$ ./dolphin_ix_port_control.sh --disable 192.168.113.40 1
--disable
Disabling switch_ip 192.168.113.40, port 1
Complete - csr write addr=0x0001C050, val=0x20820090 (with ret=0)

ssh c1sus

controls@c1sus:~$ rtcds stop --all

[Paco, Murtaza, JC, Koji, Radhika]

MC1/MC2 was working fine.

At this point MC3, SRM, and ITMY are also working fine.

The custom DB25 cables between the sat amps and the flanges are difficult to mate.

• The finger tight was not enough to make all the contacts. Fastening the screws with a screwdriver made MC3 start working fine on CDS.
• The custom cable fastening screw on PRM(1st) was stripped at the flange side. It needs a thread dyeing. The SRM(2st) has the hex nut broken on the sat amp. Need to be fixed.

We checked if the LED mon shows the correct values. When it is connected it shows 5V. If the LED is not connected it shows 0.8V. It goes 0.08V in an unknown state.

• SRM1 all channels were 0V. It turned out that the connection inside the vacuum chamber seemed mirrored. Right now we have temporary mirror ribbon cables to fix this issue. We need two shielded mirror cables for SRM1 and SRM2.
• SRM2 was random (5V, 0.8V, and 0V)
• BS1/2/ITMY1 looked fine.
• ITMY2 was strange.
• ITMX1/2 were completely silent.

We suspected the cable pinouts/ cable mating issue etc, but it turned out that the SRM2 cable was mislabeled and the ITMY2 cable was connected to the SRM sat amp. That's why it was so random. We corrected the connection and SRM and ITMY started working fine on CDS.

We used the OSEM simulation box to test the sat amps. That suggests that the BS/PRM/ITMX problem may be coming from the SAT amps. We need to look into the sat amps.

[Murtaza, Koji]

ITMX / BS / PRM sat amps were removed from the rack and checked on the workbench. They all work fine with the OSEM simulation box.

With the correct circuit, the LED mon should be 5V, and the PD readout should be 2.6~3.0V (i.e. the differential output has twice the voltage difference of this number).

ITMX Sat Amp fixed:

• The internal wiring for CH1-4 was not connected (or disconnected by mechanical impact) (Attachment 1)
• PD1 channel for CH1 had a metal debris on the transimpedance opamp (Attachment 2)
• The internal board for CH5-6 was connected in the opposite direction. This was because both connectors on the board had the wrong genders.
  This was replaced with a spare. (There was two spares and I consumed one now) (Attachment 3)
   
• Put a ventilated lid instead of the solid lid.

BS Sat Amp:

• All the CHs just worked fine.

PRM Sat Amp:

• Found the bias selector jumpers had not been installed. Fixed. (Attachment 4/5)

Plan for Sept 20, 2023

For morning people:

• We don't need to replace the long cables. They seem all fine.
• Close the lid of the repaired sat amps. Use a lid with ventilation slits (there is an extra with the empty unit on the same desk).
• Install the sat amps back to the racks. Connect all the cables us. Check if this makes the OSEM values to positive 10~20k counts.
  • If not, check LED mons and PD mons. If they are OK the sat amp is working fine. Track the signal down to the AI chassis to see if the units after the sat amp are working well.
• The SRM2 and ITM2 cables ( connected to the sat amps at the back of the units) cross (i.e. have twisted) at the rack. Please reroute and nicely coil them up.
• Fix the custom cable issues: "The custom cable fastening screw on PRM(1st) was stripped at the flange side. It needs a thread dyeing. The SRM(2st) has the hex nut broken on the sat amp. Need to be fixed."
• Put the proper labels on the long cables at the flanges and the sat amps. The labels should indicate where the connectors are supposed to be connected.
• Clean up the mess and the tools from the lab.

The

After the weekly meeting, we'll continue to work on the suspension control. The lab will be turned to be LASER HAZARD in the afternoon.

[Koji, Murtaza, JC]

Regarding PRM/BS:

• PRM2 cable and BS2 cable were wrongly connected. This was corrected.
   
• This makes the BS face OSEM values reasonable.
  However, the side signal is still close to zero. We confirmed that the sat amp outputs (LED mon/PD mon/PD diff out) looked reasonable for all five BS OSEMs.
  The side signal issue stays downstream of the Vertex ADC adapter.
   
• PRM2 has no issue with the sat amp.
• PRM1: We found that all the LED mon goes down to 0.17V when the vacuum flange is connected.
  It was found that the reference voltage for the LED (TP11 of D080276) went down to low number (like 0.15V) when the in-vac OSEMs were connected.
  I found that this output was not stable. So, I replaced the U4 chip (AD8672), but this didn't help the voltage sagging issue.
• Murtaza and I started checking the short circuits on the flange. We found that Pin 5 (OSEM PD1 Kathode) and Pin 1 (invac cable shield?) only have 5.1 Ohm. Pin 1 is connected to the vacuum chamber.
  • What does it mean? The PD has the reverse bias voltage of 10V applied on the PD Kathode. This bias voltage is shorted to ground via 5 Ohm. To keep the bias line at 10V, we need 2 A.
  • We don't have many options:
    - We can disconnect the internal wire for pin5 from the cable. (Prepare a ribbon cable). This should make the other OSEM PDs properly biased.
    We may be able to use an independent power supply to provide some amount of reverse bias (10V 2A is too much. Probably 1V 0.2A or 2.5V 0.5A?) so that the UL PD somewhat work.

I found the actuator calibration is more complicated. The numbers I reported in the previous elog was not correct.

Here I summarize the numbers of the voltage-to-current conversion.

=== Previous===

=== New ===

e.g. ITMX face coil electronics are x4.6 stronger than the previous coil electronics.

MC1/MC2/MC3 damping restored

I tweaked the coil actuation gains for MC1/MC2/MC3 according to Koji's updated calculations:

With foton and load coefficients working as expected, these coil output filters were successfully added to MC2 and MC3.

Damping tests

Note: While burt restoring C1MCS to a pre-upgrade state, a "NOT OK" flag popped up and the coil balancing gains for MC1/MC2/MC3 were reset to +-1. Koji showed me how to access the original values in c1mcs.snap (using grep) and I restored the coil gains to their values from 9/12/2023.

Next steps

[Paco, Radhika]

IMC LOCKED

We used the pre-upgrade C1:SUS-MC1/2/3_SUSPIT/YAW/POS_INMON values as a baseline to restore IMC alignment.

Procedure we followed:

I was poking around at the LSC rack to try and set up a temporary arrangement whereby I take the signals from the DAC differentially and route them to the D990694 differentially. The situation is complicated by the fact that, afaik, we don't have any break out boards for the DIN96 connectors on the back of all our Eurocrate cards (or indeed for many of the other funky connecters we have like IDE/IDC 10,50 etc etc). I've asked Steve to look into ordering a few of these. So I tried to put together a hacky solution with an expansion card and an IDC64 connector. I must have accidentally shorted a pair of DAC pins or something, because all models on the c1lsc FE crashed. On attempting to restart them (c1lsc was still ssh-able), the usual issue of all vertex FEs crashing happened. It required several iterations of me walking into the lab to hard-reboot FEs, but everything is back green now, and I see the AS beam on the camera so the input pointing of the TTs is roughly back where it was. Y arm TEM00 flashes are also seen. I'm not going to re-align the IFO tonight. Maybe I'll stick to using a function generator for the THD tests, probably routing non AI-ed signals directly is as bad as any timing asynchronicity between funcGen and DAQ system...

Summary:

Around 5pm local time, the three vertex FEs crashed. AFAIK, no one was in the lab or working on anything CDS related, so this is worrying.

Details:

• Reboot script was used to bring all FEs back - only soft reboots were required.
• The IMC and arms can now be locked.
• I think combination of burt + SDF would have reverted all the settings as they should be, but if something appears off, it could be that some EPICS value didn't get reset correctly.

 3 of the 4 remote controlled illuminators at the vertex are installed and can now be turned on via sitemap. There are a total of 15 controls for "Illum", but only the 3 labeled with MC, BS-PRM and ITMY-SRM are functional.

As marked up in the photos.

Attachment 5: The electronics units removed. Cleaning half way down. (KA)

Attachment 6: Moved most of the units to 1X3B rack ELOG 17125 (KA)

The analog electronics units piled up along the wall was moved into 1X3B rack which was basically empty. (Attachments 1/2/4)

We had a couple of unused Sun Machines. I salvaged VMIC cards (RFM and Fast fiber networking? for DAQ???) and gave them to Tega.
Attachment 3 shows the eWastes collected this afternoon.

 I shut down damping to the Vertex optics and removed Binary IO  Adapter chassy BO0 and BO1

About a week ago I discussed the BO0's power indicator lights with Kiwamu. They were  not on or they were blinking on-off.

I put screws into ps connectors in the back, but it did not helped.

[Jamie, Koji]

- We found the reason why some of the LEDs had no light. It was because the LEDs were blown as they were directly connected to the power supply.
The LEDs are presumably designed to be connected to a 5V supply (with internal current-limiting resistor of ~500Ohm). The too much current
with the 15V (~30mA) made the LED blown, or the life-time of them shorter.

- Jamie removed all of the BO modules and I put 800Ohm additional resister such that the resultant current is to be 12mA.
The LEDs were tested and are fine now.

- The four BO boxes for C1SUS were restored on the rack. I personally got confused what should be connected where
even though I had labeled for BO0 and BO1. I just have connected CH1-16 for BO0. The power supplies have been connected only to BO0 and BO1.

- I tested the whitening of PRM UL sensor by exciting PRM UL sensor. The transfer function told us that the pendulum response can be seen
up to 10-15Hz. When the whitening is on, I could see the change of the transfer function in that freq band. This is good.
So the main reason why I could not see theis was that the power supply for the BOs were not turned on.

- I suppose Jamie/Joe will restore all of the BO boxes on the racks tomorrow. I am going to make a test script for checking the PD whitenings.

- I was investigating the SUS whitening issue.

- I could not find any suspension which can handle the input whitening switch correctly.

- I went to 1X5 rack and found that both of the two binary output boxes were turned off.
As far as I know they are pulling up the lines which are switched by the open collector outputs.

- I tried to turn on the switch. Immediately I noticed the power lamps did not work. So I need an isolated setup to investigate the situation.

- The cables are labelled. I will ask steve to remove the boxes from the rack.

4 units of Vertex SUS DAC adapter (https://dcc.ligo.org/LIGO-D2100035) ready.

https://dcc.ligo.org/LIGO-S2101689

https://dcc.ligo.org/LIGO-S2101690

https://dcc.ligo.org/LIGO-S2101691

https://dcc.ligo.org/LIGO-S2101692

The units are completely passive right now and has option to extend to have a dewhitening board added inside.
So the power switch does nothing.

Some of the components for the dewhitening enhancement are attached inside the units.

I've now made a collection of sensing matrix measurements. 

In all of the plots below, the radial scale is logarithmic, each grid line is a factor of 10. The units of the radial direction are calibrated into demod board output Volts per meter. The same radial scale is used on all plots and subplots.

I did two PRMI measurements: with MICH locked and excited with either the ITMS or the BS + PRM compensation. This tells us if our PRM compensation is working; I think it is indeed ok. I though I remembered that we came up with a number for the SRM compensation, but I haven't been able to find it yet. 

The CARM sensing int he PRFPMI measurement has the loop gain at the excitation frequency undone. All excitations were simultaneously notched out of all control filters, via the NotchSensMat filters. 

The angular scale is set to the analog I and Q signals; the dotted lines show the digitial phase rotation angle used at the time of measurement. 

The Vertex crane is smarter and safer now.  This upgrade ensures that the two sections of I-beam (8ft, 4ft) remain firmly latched to form a straight member till the latch is released.

In specific, it ensures that problems such as this one do not occur in the future.

The new safety features are:

When the I-beam sections are latched together, a pneumatic piston ensures that the latch is secure. 

If the latch is not engaged the trolley does not move outward beyond the end of the 8-foot section of the I beam.

If the trolley is out on the 4-foot section of the beam then we cannot disengage the latch.

How does it work?

The state of the Limit Switch 1 changes when the trolly goes past it.    The Limit Switch 2 gets pressed when the two sections are latched together.

The pneumatic piston raises or lowers the latch.  The Pneumatic Latch Switch operates a pneumatic valve controlling the state of the piston.

The new controller now has Pneumatic Latch Switch in addition to the usual Start, Stop, Up, Down, In and Out buttons. 

Each of the Up, Down, In and Out buttons have two operational states:  Half pressed (low speed) and Full pressed (High Speed).  Their functions remain the same as before.

The new Pneumatic Switch:

When this switch is 'Engaged' and the 4 ft section is swung in-line with the 8 ft section, the two sections get latched together.

To unlatch them we have to throw the switch into the 'Disengage' state.  This makes the piston push the latch open and a spring rotates the 4 ft section about its pivot.

Limit Switch 2 is not pressed (I-beams not aligned straight) ==> Limit Switch 1 will prevent the trolley from out going beyond the 8 ft section.

While Limit Switch 2 is pressed we cannot disengage the latch.

Note: 

   The pneumatic piston requires 80psi of pressure to operate.  However we have only 40psi in the lab and the piston seems to operate quite well at this pressure as well.  I believe a request has been made to get an 80psi line laid just for this application.

Jon and I were surveying the CDS situation so that he can prepare a report for discussion with Rolf/Rich about our upcoming BHD upgrade. In our poking around, we must have bumped something somewhere because the c1ioo machine went offline, and consequently, took all the vertex models out. I rebooted everything with the reboot script, everything seems to have come back smoothly. I took this opportunity to install some saturation counters for the arm servos, as we have for the CARM/DARM loops, because I want to use these for a watch script that catches when the ALS loses lock and shuts stuff off before kicking optics around needlessly. See Attachment #1 for my changes.

Horizontal beam centering along XARM

I tried to improve beam centering along XARM from the control room. First I realigned IMC until counts were recovered on MC TRANS QPD. The final IMC alignment state resulted in 960 cts transmission [Attachment 1].

Next I aligned the arms until transmission in both arms reached ~0.08 [Attachment 2]. In that state I recorded ETM/ITM beam spot positions below. Note that although the beam was miscentered by over 5mm on ETMX (horizontal), it was under 1mm on ITMX. Thus it seemed like a pointing issue. I then tried to lock MICH to record the beam spot on BS, but it never locked stably and the measurements were bogus.

Optic    LSCDoF     freq.(Hz)    ampl. (counts)    gpstime    Opt. gain (counts/nm)    Opt. gain_std    v (mm)    v_std      h (mm)    h_std

ETMY    YARM    211.11    500    1385168096    112.24    4.68           -1.25   0.47  -1.48   0.41
ITMY    YARM    211.11    500    1385168173    100.55    3.18            3.39   0.74   1.71   0.40
ETMX    XARM    211.11    500    1385168993    70.39     4.36           -2.92   0.60   8.39   1.56
ITMX    XARM    211.11    500    1385169082    79.36     3.17            2.90   0.60  -1.64   0.85

I proceeded to try to walk the beam a bit along XARM by moving TT2 in yaw and compensating with PR2. I moved both by ~40 steps and XARM transmission improved to just under 0.1 [Attachment 3].  However, the beam spot measurements looked pretty much the same; mode-matching into XARM was improved while pointing stayed constant. Next I locked PRY to measure the final beam spots on PRM/PR2/PR3.

Optic    LSCDoF     freq.(Hz)    ampl. (counts)    gpstime    Opt. gain (counts/nm)    Opt. gain_std    v (mm)    v_std      h (mm)      h_std

ETMY    YARM    211.11    500    1385234260    133.31    6.03            2.28   0.75   -0.74   0.43
ITMY    YARM    211.11    500    1385234343    121.44    3.14            3.91   0.52   -0.82   0.43
ETMX    XARM    211.11    500    1385234433    88.98     3.75           -0.76   0.41    6.16   1.18
ITMX    XARM    211.11    500    1385234546    103.35    4.50            3.60   0.67   -0.90   0.23
PRM     PRY    211.11    1000    1385236336    5.57      0.73           -8.13   16.67  -2.75   2.38
PR2     PRY    211.11    1000    1385236409    8.65      1.88            2.17   0.54   -7.71   3.39
PR3     PRY    211.11    1000    1385236483    6.39      1.74            4.83   1.50    9.17   6.25

With such high transmission in the arm cavities, alignment looked good and we decided it was OK proceed with aligning the LO/POP beams.

Very encouraging results from the test last night.  The new configuration did not crash once overnight, and seemed to write out full, second trend, and minute trend frames without issue.  However, full validity of all the written out frames has not been confirmed.

overview

The configuration under test involves two separate daqd binaries instead of one.  We usually run with what is referred to as a "framebuilder" (fb) configuration:

• fb: a single daqd binary that:
  • collect the data from the front ends
  • coallate full data into frame file format
  • calculates trend data
  • writes frame files to disk.

The current configuration separates the tasks into multiple separate binaries: a "data concentrator" (dc) and a "frame writer" (fw):

• dc:
  • collect data from front ends
  • coallate full data into frame file format
  • broadcasts frame files over local network
• fw:
  • receives frame files from broadcast
  • calculates trend data
  • writes frame files to disk

This configuration is more like what is run at the sites, where all the various components are separate and run on separate hardware.  In our case, I tried just running the two binaries on the same machine, with the broadcast going over the loopback interface.  None of the systems that use separated daqd tasks see the failures that we've been seeing with the all-in-one fb configuration (and other sites like AEI have also seen).

My guess  is that there's some busted semaphore somewhere in daqd that's being shared between the concentrator and writer components.  The writer component probably aquires the lock while it's writing out the frame, which prevents the concentrator for doing what it needs to be doing while the frame is being written out.  That causes the concentrator to lock up and die if the frame writing takes too long (which it seems to almost necessarily do, especially when trend frames are also being written out).

results

The current configuration hasn't been tweaked or optimized at all.  There is of course basically no documentation on the meaning of the various daqdrc directives.  Hopefully I can get Keith Thorne to help me figure out a well optimized configuration.

There is at least one problem whereby the fw component is issuing an excessively large number of re-transmission requests:

2016-06-15_09:46:22 [Wed Jun 15 09:46:22 2016] Ask for retransmission of 6 packets; port 7097
2016-06-15_09:46:22 [Wed Jun 15 09:46:22 2016] Ask for retransmission of 8 packets; port 7097
2016-06-15_09:46:22 [Wed Jun 15 09:46:22 2016] Ask for retransmission of 3 packets; port 7097
2016-06-15_09:46:22 [Wed Jun 15 09:46:22 2016] Ask for retransmission of 5 packets; port 7097
2016-06-15_09:46:22 [Wed Jun 15 09:46:22 2016] Ask for retransmission of 5 packets; port 7097
2016-06-15_09:46:22 [Wed Jun 15 09:46:22 2016] Ask for retransmission of 5 packets; port 7097
2016-06-15_09:46:22 [Wed Jun 15 09:46:22 2016] Ask for retransmission of 5 packets; port 7097
2016-06-15_09:46:22 [Wed Jun 15 09:46:22 2016] Ask for retransmission of 6 packets; port 7097
2016-06-15_09:46:23 [Wed Jun 15 09:46:23 2016] Ask for retransmission of 1 packets; port 7097

It's unclear why.  Presumably the retransmissions requests are being honored, and the fw eventually gets the data it needs.  Otherwise I would hope that there would be the appropriate errors.

The data is being written out as expected:

 full/11500: total 182G
drwxr-xr-x  2 controls controls 132K Jun 15 09:37 .
-rw-r--r--  1 controls controls  69M Jun 15 09:37 C-R-1150043856-16.gwf
-rw-r--r--  1 controls controls  68M Jun 15 09:37 C-R-1150043840-16.gwf
-rw-r--r--  1 controls controls  68M Jun 15 09:37 C-R-1150043824-16.gwf
-rw-r--r--  1 controls controls  69M Jun 15 09:36 C-R-1150043808-16.gwf
-rw-r--r--  1 controls controls  69M Jun 15 09:36 C-R-1150043792-16.gwf
-rw-r--r--  1 controls controls  68M Jun 15 09:36 C-R-1150043776-16.gwf
-rw-r--r--  1 controls controls  68M Jun 15 09:36 C-R-1150043760-16.gwf
-rw-r--r--  1 controls controls  69M Jun 15 09:35 C-R-1150043744-16.gwf

 trend/second/11500: total 11G
drwxr-xr-x  2 controls controls 4.0K Jun 15 09:29 .
-rw-r--r--  1 controls controls 148M Jun 15 09:29 C-T-1150042800-600.gwf
-rw-r--r--  1 controls controls 148M Jun 15 09:19 C-T-1150042200-600.gwf
-rw-r--r--  1 controls controls 148M Jun 15 09:09 C-T-1150041600-600.gwf
-rw-r--r--  1 controls controls 148M Jun 15 08:59 C-T-1150041000-600.gwf
-rw-r--r--  1 controls controls 148M Jun 15 08:49 C-T-1150040400-600.gwf
-rw-r--r--  1 controls controls 148M Jun 15 08:39 C-T-1150039800-600.gwf
-rw-r--r--  1 controls controls 148M Jun 15 08:29 C-T-1150039200-600.gwf
-rw-r--r--  1 controls controls 148M Jun 15 08:19 C-T-1150038600-600.gwf

 trend/minute/11500: total 152M
drwxr-xr-x 2 controls controls 4.0K Jun 15 07:27 .
-rw-r--r-- 1 controls controls  51M Jun 15 07:27 C-M-1150023600-7200.gwf
-rw-r--r-- 1 controls controls  51M Jun 15 04:31 C-M-1150012800-7200.gwf
-rw-r--r-- 1 controls controls  51M Jun 15 01:27 C-M-1150002000-7200.gwf

The frame sizes look more or less as expected, and they seem to be valid as determined with some quick checks with the framecpp command line utilities.

[Larisa, Aidan,Steve,Suresh]

   Today was the first session for implementing the new video cabling plan laid out in the document " CCD_Cable_Upgrade_Plan_Jan11_2011.pdf"  by Joon Ho attached to his elog entry 4139.  We started to check and label all the existing cables according to the new naming scheme. 

So far we have labeled the following cables. Each has been checked by connecting it to a monitor near the Video Mux and a camera at the other end.

C1:IO VIDEO 8ETMYF

C1:IO-VIDEO 6 ITMYF

C1:IO-VIDEO 21 SRMF

C1:IO-VIDEO 25 OMCT

C1:IO-VIDEO 19 REFL

C1:IO-VIDEO 22 AS

C1:IO-VIDEO 18 IMCR

C1:IO-VIDEO 14 PMCT

C1:IO-VIDEO 12 RCT

C1:IO-VIDEO 9 ETMXF

C1:IO-VIDEO 1 MC2T

Next we need to continue and finish the labeling of existing cables.  We then choose a specific set of cables which need to be laid together and proceed to lay them after attaching suitable lables to them.

Steve is summarizing the Video Matrix choices into this Wiki page:

http://lhocds.ligo-wa.caltech.edu:8000/40m/Electronics/VideoMUX

Requirements:

Price: < 5k$

Control: RS-232 and Ethernet

Interface: BNC (Composite Video)

Please check into the page on Monday for a final list of choices and add comments to the wiki page.

 [Kiwamu, Larisa]

The following Video MUX inputs(cameras) and outputs(monitors) have been checked:

MC2F, FI, AS Spare, ITMYF, ITMXF, ETMYF, ETMXF, PSL Spare, ETMXT, MC2T, POP, MC1F/MC3F, SRMF, ETMYT, PRM/BS, CRT1(MON1), ETMY Monitor, CRT2(MON2), CRT4(MON4), MC1 Monitor, CRT3(MON3), PSL1 Monitor, PSL2 Monitor, CRT6(MON6), CRT5(MON5), ETMX Monitor, MC2 Monitor, CRT9, CRT7(MON7), CRT10, and Projector.

Their respective statuses have been updated on the wiki:   (wiki is down at the moment, I will come back and add the link when it's back up)

Now the video matrix is responding correctly and the web interface shows up. (Attachment 1)

Also the video buttons respond as usual. I pushed Locking Template button to bring the setting back to nominal. (Attachment 2)

In this video: https://youtu.be/iphcyNWFD10, the comments focus on the orange crocs, my wrinkled shirt, and the first aid kit.

What mode will you get if lock the cavity PRM - ITMY/ITMX/TEST MIRROR without PR2, PR3 and BS?

Is it possible to skip MC1, MC3 and lock the laser to this test cavity to make sure that this is not actuator/electronics noise?

I think Den accidentally edited and overwrote my entry, rather than replying, so I'm going to recreate it from memory:

I aligned the PRM-flat test cavity (although not as well as Jamie and Koji did later in the evening) and took some videos. Note that these may not be as relevant any more, since Jamie and Koji improved things after I left.

Also, before doing anything with the cavity, I tuned up the PMC since the pitch input alignment wasn't perfect (we were getting ~0.7 transmission), and also tuned up the MC alignment and remeasured the MC spot positions, to maintain a record.