I tried to perform a simple enabling test of coils using c1susaux2 modbus channels but failed. I'm able to do the enabling of coils using the windows GUI of acromag card but I can not do it when the cards are connected to the computer subnetwork. The issue is two-fold:
There's also an issue in reading back the ENABLE_MON channels. Here we suspect that one of the optical isolator box that we have been using might have a short in one of it's output channel. I'll investigate this more tomorrow. Again, the issue is two-fold. The EPICS channel values do not really change. So there is clearly some issue of communicating with the acromag cards.
[Anchal, Yehonathan, Ian]
We installed c1susaux2 acromag chassis in 1Y0 with c1susaux2 computer. We connected PD monitors, Binary inputs, Binary outputs, and Run/Acquire RTS signals for 6 of the 7 suspensions. We ran out of DB9 cables to connect PR3. Of the ones that were connected, LO2, AS1, AS4, SR2, and PR2 are showing no issues in the functionality from the chassis. For LO1, everything is working except for UR EnableMon channel. The enable monitor does not show an ON state for the coil even though the coil driver chassis shows that it is ON via the LED lights. A possible reason could be that a wire got disconnected when we closed the chassis (there are a lot of wires pushing against each other. Another reason could be that the optical isolator ISO10 could have developed a bad channel on channel 2. The circuit was tested before closing the chassis, so not sure what went wrong after closing it.
PR2 is showing a non-acromag chassis related issue. As soon as we close the loop by enabling the coils, the watchdog triggers because the loop is unstable. Not sure what has changed for PR2, but someone should take a look at it.
For the issue with LO1, I suggest we keep a note that the C1:SUS-LO1_UR_ENABLEMon channel is faulty and don't take its value seriously. We should diagnose and fix this issue once we have more reasons to disconnect the chassis and open it.
I routed the XXX_COIL_DW signals from the 7 SOS blocks in c1su2.mdl (located at /cvs/cds/rtcds/userapps/trunk/sus/c1/models/c1su2.mdl) to the binary outputs from the FE model. The routing is done such that when these binary outputs are routed through the binary interface card mounted on 1Y0, they go to the acromag chassis just installed and from there they go to the binary inputs of the coil drivers together with the acromag controlled coil outputs.
I have not restarted the rtcds models yet. This needs more care and need to follow instructions from 40m/16533. Will do that sometime later or Koji can follow up this work.
I have restarted c1su2 model with the connections of Run Acquire switch to analog filters on coil drivers. Following steps were taken:
First ssh to c1sus2 and then:
controls@c1sus2:~ 0$ rtcds make c1su2
### building c1su2...
Parsing the model c1su2...
Building EPICS sequencers...
Building front-end Linux kernel module c1su2...
RCG source code directory:
The following files were used for this build:
Successfully compiled c1su2
Compile Warnings, found in c1su2_warnings.log:
WARNING *********** No connection to subsystem output named SUS_DAC1_12
WARNING *********** No connection to subsystem output named SUS_DAC1_13
WARNING *********** No connection to subsystem output named SUS_DAC1_14
WARNING *********** No connection to subsystem output named SUS_DAC1_15
WARNING *********** No connection to subsystem output named SUS_DAC2_7
WARNING *********** No connection to subsystem output named SUS_DAC2_8
WARNING *********** No connection to subsystem output named SUS_DAC2_9
WARNING *********** No connection to subsystem output named SUS_DAC2_10
WARNING *********** No connection to subsystem output named SUS_DAC2_11
WARNING *********** No connection to subsystem output named SUS_DAC2_12
WARNING *********** No connection to subsystem output named SUS_DAC2_13
WARNING *********** No connection to subsystem output named SUS_DAC2_14
WARNING *********** No connection to subsystem output named SUS_DAC2_15
controls@c1sus2:~ 0$ rtcds install c1su2
### installing c1su2...
Installing system=c1su2 site=caltech ifo=C1,c1
Installing start and stop scripts
Updating testpoint.par config file
/opt/rtcds/rtscore/branches/branch-3.4/src/epics/util/updateTestpointPar.pl -par_file=/opt/rtcds/caltech/c1/target/gds/param/archive/testpoint_220315_135808.par -gds_node=26 -site_letter=C -system=c1su2 -host=c1sus2
Installing GDS node 26 configuration file
Installing auto-generated DAQ configuration file
Installing Epics MEDM screens
Running post-build script
/opt/rtcds/userapps/release/cds/common/scripts/generate_KisselButton.py 4 5 C1:SUS-AS1_INMATRIX > /opt/rtcds/caltech/c1/medm/c1su2/C1SUS_AS1_INMATRIX_KB.adl
/opt/rtcds/userapps/release/cds/common/scripts/generate_KisselButton.py 2 4 C1:SUS-AS1_LOCKIN_INMTRX > /opt/rtcds/caltech/c1/medm/c1su2/C1SUS_AS1_LOCKIN_INMTRX_KB.adl
/opt/rtcds/userapps/release/cds/common/scripts/generate_KisselButton.py 5 6 C1:SUS-AS1_TO_COIL --fi > /opt/rtcds/caltech/c1/medm/c1su2/C1SUS_AS1_TO_COIL_KB.adl
/opt/rtcds/userapps/release/cds/common/scripts/generate_KisselButton.py 4 5 C1:SUS-AS4_INMATRIX > /opt/rtcds/caltech/c1/medm/c1su2/C1SUS_AS4_INMATRIX_KB.adl
/opt/rtcds/userapps/release/cds/common/scripts/generate_KisselButton.py 2 4 C1:SUS-AS4_LOCKIN_INMTRX > /opt/rtcds/caltech/c1/medm/c1su2/C1SUS_AS4_LOCKIN_INMTRX_KB.adl
/opt/rtcds/userapps/release/cds/common/scripts/generate_KisselButton.py 5 6 C1:SUS-AS4_TO_COIL --fi > /opt/rtcds/caltech/c1/medm/c1su2/C1SUS_AS4_TO_COIL_KB.adl
/opt/rtcds/userapps/release/cds/common/scripts/generate_KisselButton.py 4 5 C1:SUS-LO1_INMATRIX > /opt/rtcds/caltech/c1/medm/c1su2/C1SUS_LO1_INMATRIX_KB.adl
/opt/rtcds/userapps/release/cds/common/scripts/generate_KisselButton.py 2 4 C1:SUS-LO1_LOCKIN_INMTRX > /opt/rtcds/caltech/c1/medm/c1su2/C1SUS_LO1_LOCKIN_INMTRX_KB.adl
/opt/rtcds/userapps/release/cds/common/scripts/generate_KisselButton.py 5 6 C1:SUS-LO1_TO_COIL --fi > /opt/rtcds/caltech/c1/medm/c1su2/C1SUS_LO1_TO_COIL_KB.adl
/opt/rtcds/userapps/release/cds/common/scripts/generate_KisselButton.py 4 5 C1:SUS-LO2_INMATRIX > /opt/rtcds/caltech/c1/medm/c1su2/C1SUS_LO2_INMATRIX_KB.adl
/opt/rtcds/userapps/release/cds/common/scripts/generate_KisselButton.py 2 4 C1:SUS-LO2_LOCKIN_INMTRX > /opt/rtcds/caltech/c1/medm/c1su2/C1SUS_LO2_LOCKIN_INMTRX_KB.adl
/opt/rtcds/userapps/release/cds/common/scripts/generate_KisselButton.py 5 6 C1:SUS-LO2_TO_COIL --fi > /opt/rtcds/caltech/c1/medm/c1su2/C1SUS_LO2_TO_COIL_KB.adl
/opt/rtcds/userapps/release/cds/common/scripts/generate_KisselButton.py 4 5 C1:SUS-PR2_INMATRIX > /opt/rtcds/caltech/c1/medm/c1su2/C1SUS_PR2_INMATRIX_KB.adl
/opt/rtcds/userapps/release/cds/common/scripts/generate_KisselButton.py 2 4 C1:SUS-PR2_LOCKIN_INMTRX > /opt/rtcds/caltech/c1/medm/c1su2/C1SUS_PR2_LOCKIN_INMTRX_KB.adl
/opt/rtcds/userapps/release/cds/common/scripts/generate_KisselButton.py 5 6 C1:SUS-PR2_TO_COIL --fi > /opt/rtcds/caltech/c1/medm/c1su2/C1SUS_PR2_TO_COIL_KB.adl
/opt/rtcds/userapps/release/cds/common/scripts/generate_KisselButton.py 4 5 C1:SUS-PR3_INMATRIX > /opt/rtcds/caltech/c1/medm/c1su2/C1SUS_PR3_INMATRIX_KB.adl
/opt/rtcds/userapps/release/cds/common/scripts/generate_KisselButton.py 2 4 C1:SUS-PR3_LOCKIN_INMTRX > /opt/rtcds/caltech/c1/medm/c1su2/C1SUS_PR3_LOCKIN_INMTRX_KB.adl
/opt/rtcds/userapps/release/cds/common/scripts/generate_KisselButton.py 5 6 C1:SUS-PR3_TO_COIL --fi > /opt/rtcds/caltech/c1/medm/c1su2/C1SUS_PR3_TO_COIL_KB.adl
/opt/rtcds/userapps/release/cds/common/scripts/generate_KisselButton.py 4 5 C1:SUS-SR2_INMATRIX > /opt/rtcds/caltech/c1/medm/c1su2/C1SUS_SR2_INMATRIX_KB.adl
/opt/rtcds/userapps/release/cds/common/scripts/generate_KisselButton.py 2 4 C1:SUS-SR2_LOCKIN_INMTRX > /opt/rtcds/caltech/c1/medm/c1su2/C1SUS_SR2_LOCKIN_INMTRX_KB.adl
/opt/rtcds/userapps/release/cds/common/scripts/generate_KisselButton.py 5 6 C1:SUS-SR2_TO_COIL --fi > /opt/rtcds/caltech/c1/medm/c1su2/C1SUS_SR2_TO_COIL_KB.adl
Then on rossa, run activateSUS2DQ.py which creates a file C1SU2.ini.NEW. Remove old backup file C1SU2.ini.bak, rename C1SU2.ini to C1SU2.ini.bak and rename C1SU2.ini.NEW to C1SU2.ini:
Then ssh back to c1sus2 and restart the rtcds model:
Then restart daqd services from rossa and burtrestore to latest snap of c1su2epics.snap:
All suspensions are back online and everything is same as before now. Will test later the Run/Acquire switch functionality.
[Anchal, Yehonathan] (on Y Arm)
We first checked if the PZT mirrors M1 and M2 can be controlled. They indeed show no motion even after being connected with a power supply. So green injection path can not be aligned using cds controls right now.
We also noticed that all ETMY slow controls and monitors are offline. That's because the electronics upgrade did not include acromag chassis. This means that the DC bias adjustment is not accessible for ETMY.
[Paco, Ian] (on X Arm)
We installed c1auxey1 computer and the acromag chassis in 1Y4. The computer has been configured properly for nfs mounts to happen and we have initialized a git repo for /cvs/cds/caltech/target/c1auxey1 directory which stores all files for running modbusIOC service on this computer. We connected 18V power source but have not connected the 24V power yet as we need to make a new connector for it. Going on what Koji recommended, we'll connect the 24V power input to 18 V strip as well as the acromags can run on that voltage too.
I started the modbusIOC service on c1auxey1 and added PD variance channels for UR and SD as well. There are unfortunately two issues here:
I'm not sure why but the PIT and YAW offset values of +2725 and -2341 were not sufficient for the reflected OPLEV beam to reach the OPLEV QPD. I had to change the C1:SUS-ETMY_PIT_OFFSET to 5641 and C1:SUS-ETMY_YAW_OFFSET to -4820 to come back to center of the OPLEV QPD. We aligned the Oplevs to center before venting, so hopefully this is our desired ETMY position.
On another note, the issue of ETMY unable to damp was simple. The alignment offsets were on to begin with with values above 1000. This meant that whenever we enabled coil output, ETMY would necessarily get a kick. All I had to do was keep alignment offsets off before starting the damping and slowly increase the alignment offsets to desired position.
- This modification allowed me to align the oplev spot to the center of the QPD. C1:SUS-ETMY_PIT_OFFSET and C1:SUS-ETMY_YAW_OFFSE are +2725 (8%FS) and -2341 (7%FS), respectively.
C1:SUS-ETMY_YAW_OFFSE are +
We did the following alignment steps on Yarm today:
After this, we came back to the control room. One peculiar thing to note is that the C1:LS-Y_REFL_DC_OUT channel is inverted i.e. it is showing negative values for the reflection DC voltage. On this signal, we see a little bit higher-order mode flashing but it is not bright enough to be seen on the face camera of the suspension. We'll continue aligning the cavity using CDS now to get TEM00 mode.
After trying for a bit, we were able to get flash of about 2000 counts which is about 16% of the max value of 12200 counts. We adjusted the ITMY angular position using the IFO_ALIGN screen but used the OPTICAL_ALIGN offset screen to adjust the angular position of ETMY. The pitch and yaw values for ITMY are 1.0360 and -0.260 respectively whereas the pitch and yaw values for ETMY are 5664.0 and -4937.0 respectively.
Another issue, Green Y Shutter can not be controlled with the EPICS controls right now. This needs to be investigated.
Uneloged: yesterday, Paco and Ian tried locking the green laser on Yend to the Y-arm. Nothing noteworthy happened, no luck.
After Ian updated the cts2um filters for OSEM, shouldn't the damping gains be increased back by factor of 10 to previos values? Was the damping gain for SIDE ever changed? we found it at 250.
Can you explain why gain_offset filter was required and why this wasn't done for the side coil?
I updated the gain of the ct2um filter on the OSEMS for ETMY and decreased their gain by a factor of 9 from 0.36 to 0.04.
I added a filter called "gain_offset" to all the coils except for side and added a gain of 0.48.
together these should negate the added gain from the electronics replacement of the ETMY
[Anchal, Paco, Shruti]
[Anchal, JC, Ian, Paco]
We have now fixed all issues with the PD mons of c1susaux2 chassis. The slow channels are now reading same values as the fast channels and there is no arbitrary offset. The binary channels are all working now except for LO2 UL which keeps showing ENABLE OFF. This was an issue earlier on LO1 UR and it magically disappeared and now is on LO2. I think the optical isolators aren't very robust. But anyways, now our watchdog system is fully functional for all BHD suspended optics.
[Paco, Anchal, Ian, JC]
We attempted the alignment of IR beam into the arm cavities. We used PR2 and PR3 (moved manually as well as using cdsutils) and got the YAW aligned pretty good on both X and Y directions. PIT alignment however turned out to be much harder to align. PR2 PR3 didn't have much range, so we zeroed there offset and tried to use TT1, TT2, MMT1, and MMT2 to align the PIT but it would get clipped before reaching BS table if we were to correct for PIT misalignment happening downstream. We concluded that the issue is that one of the PR2, PR3 mirrors have too much PIT offset in equilibrium position. We have requested Koji to change the output resistors in the coil drivers of PR2 and PR3 so that we can correct for the PIT offset in them directly using the coils and reduce load on upstream optics. We have tweaked TT1, TT2, MMT1, and MMT2 positions today, so we do not have the previous reference anymore.
After Koji reduced the output resistors on PR2/PR3 coil drivers, we got much better actuation range. We aligned TT1 TT2 again to get beam centered on PR2 and PR3. Then we used only PR2 and PR3 to do the input beam alignment to Y arm cavity. Using access in ETMY chamber, we aligned the input beam parallel to cavity axis. Slight changes were required in ETMY alignment offsets to get first roundtrip in same spot on ITMY. remaining alignment is finer and needs to be done with a help of a reflection photodiode and cameras in the control room. Immediate next step is to setup POY path for locking the Yarm with IR.
Side note: Because of the large PIT correction required in PR3, we found that our upper OSEMs were hitting totally bright limit and lower OSEMs were hitting totally dark limit on PR3. This also destablized our damping loops. We pushed the upper OSEMs slighlty and pulled back the lower OSEMs slightly to get the PD signal in half shadow region again. This worked and our damping loops are stable again. However, we think we should repeat free swing test in future to diagonalize the input matrix for new OSEM positions.
[Anchal, Paco, JC, Tega]
Today we have aligned the Yarm cavity for IR, with verified 1.5 roundtrips. We also placed following optics in the BS table.
We also cleared the in-air BS table of all previous optics. JC and Tega setup HeNe laser for Oplevs roughly for now. Tega also transported the POY RFPD from ITMY in-air table to the BS in-air table. We aligned the POY path to the table, but we had to move ITMY after that to get 2nd roundtrip in the arm cavity, which misaligned our POY path again. POY path would need to be modified tomorrow.
Nice. Please put this information on the DCC pages of the coil driver units. You'll find links to all the units in this document tree LIGO-E2100447. For each page, click on "Change Metadata" from the left panel and add the change made to the resistor (including the resistor name on PCB, previous and new value), and add a link to your previous elog post which has more details like photos, to "Notes and Changes", and upload an updated version of the circuit schematic by creating an annotation in the previous circuit schematic pdf. Every unit that has a serial number in the lab has a DCC page (if not, we should create one) where we should track all such hard changes.
If someone gets time, let's put in all the cable posts and clean up our cable routing on the tables.
[Anchal, Tega, JC]
We installed cable posts in ITMY, BS, and ITMX chambers for all the new suspensions. Now, there is no point where the OSEM connections are hanging freely.
In BS chamber, we installed one post for LO2 near the north edge of the table and another post for PR3 on the Western edge with the blue cable running around the table on the floor.
In ITMY chamber, we installed the cable post in between AS1 and AS4 with the blue cables running around the table on the floor. This is to ensure the useful part of the table remains empty for future and none of the OSEM cables are taught in air.
I investigated this issue today. At first, it seemed that only new suspension testpoints are inaccessible. I was able to use diaggui for a measurement on MC2. The DAQ network cable between 1X4 and 1Y1 was tied and is very taught now (we should relieve this as soon as possible, best solution is to lay down a longer cable over the bridge). My hypothesis is that the DAQ network might have broken while tying this cable and it probably did not come back since then.
The simplest solution would have been to restart c1su2 models. As I restarted those models though, I found that c1lsc and c1sus models failed. This is very unusual as c1su2 models are independent and share nothing with the other vertex models. So I had to restart all the FE computers eventually. But this did not solve the issue. Worse, now the DAQ isn't working for the vertex machiens as well.
Next step was to try restarting fb1 itself. We switched off all the FE computers, restarted fb1, stopped daqd_* processes, reloaded gpstime module, restarted open-mx, mx, nds and daqd_* process. But the mx.service failed to load with following error message:
● mx.service - LSB: starts MX driver for Myrinet card
Loaded: loaded (/etc/init.d/mx)
Active: failed (Result: exit-code) since Mon 2022-04-25 17:18:02 PDT; 1s ago
Process: 4261 ExecStart=/etc/init.d/mx start (code=exited, status=1/FAILURE)
Apr 25 17:18:02 fb1 mx: Loading mx driver
Apr 25 17:18:02 fb1 mx: insmod: ERROR: could not insert module /opt/mx/sbin/mx_mcp.ko: File exists
Apr 25 17:18:02 fb1 mx: insmod: ERROR: could not insert module /opt/mx/sbin/mx_driver.ko: File exists
Apr 25 17:18:02 fb1 systemd: mx.service: control process exited, code=exited status=1
Apr 25 17:18:02 fb1 systemd: Failed to start LSB: starts MX driver for Myrinet card.
Apr 25 17:18:02 fb1 systemd: Unit mx.service entered failed state.
(Ignore the timestamp above, I ran the command again to capture the error message.)
However, I was able to start all the FE models without any errors and daqd processes are also all running without showing any errors. Everything is green in CDS screen with no error messages. But the only thing still wrong is mx.service which is not running.
From my limited knowledge and experience, mx.service is a one-time script that mounts mx devices in /dev and loads the mx driver. I tried running the script /opt/mx/sbin/mx_start_stop :
This gave the same error. On searching little bit online, "insmod: ERROR; cound not insert module" error comes up when the kernel version of the driver doesnot match the Linux kernel (whatever that means!). Such deep issues should not appear out of nowhere in a previosuly perfectly runnig system. I'll check around more what changed in fb1, network cables etc.
[Anchal, Paco, JC]
We had to open ITMY, ETMY chamber doors to get the cavity aligned again. Once we did that, we regained cavity flashing and were able to align the input injection and cavity alignment to get transmission flashing to 1.0 (C1:LSC-TRY_OUT_DQ). JC later centered both ITMY and ETMY oplevs. The ITMY oplev had become completely out of range.
We also opened ITMX, ETMX chamber doors to get Xarm alignment. Again, it seems that ITMX had moved a lot due to cable post installation.
To be continued
This would be a daily first task in the morning. We'll need to check the status of arm alignment and optimize it back to maximum every morning for the rest of the day's work.
Today, when I came, on openin gthe PSL shutter, IMC was aligned good, both arms were flashing but YARM maximum transmission count was around 0.7 (as opposed to 1 from yesterday) and XARM maximum transmission count was 0.5 (as opposed to 1 from yesterday). I did not change the input alignment to the interferometer. I only used ITMY-ETMY to regain flashing count of 1 on YARM and used BS and tehn ITMX-ETMX to regain flashing count of 0.9 to 1 in XARM.
Even thought the oplevs were centered yesterday, I found the oplev had drifted from the center and the optimal position also is different for all ooptics except EMTY and BS. It is worth nothign that in optimal position both PIT and YAW of ITMY and ITMX are off by 70-90 uradians and ETMX Pit oplev is off by 55 uradians.
I tried out this stack today and found some change of plans.
tl,dr; Jordan is preparing PLS-T238 and TR-1.5 with venting holes and C&B and they would be ready by tomorrow. I have collected all other parts for assembly, still looking for the mirror but I know other lab members know where it is, so no big issue there.
The assemly of this mirror is complete. A slight change here as well, we were supposed to use the former POYM1 (Y1-2037-0) mirror for POP_SM5 but I could not find it. It was stored on the right most edge of the table (see 40m/16450), but it is not there anymore. I found another undocumented mirror on the flow bench on the left edge marked (2010 July: Y1-LW1-2037-UV-0-AR) which means this mirror has a wedge of 1 degree and an AR coating as well. We do not need or care about the wedge or AR coating, so we can use this mirror for POP_SM5. Please let me know if someone was saving this mirror for some other purpose.
I'll finish assembly of POP_SM4 tomorrow and install them in ITMX chamber and resurrect POP path.
Here is more detail of the POP_SM4 mount assembly.
It's a combination of BA2V + PLS-T238 + BA1V + TR-1.5 + LMR1V + Mirror: CM254-750-E03
Between BA1V and PLS-T238, we have to do a washer action to fix the post (8-32) with a 1/4-20 slot. Maybe we can use a 1" post shim from thorlabs/newport.
Otherwise, we should be able to fasten the other joints with silver-plated screws we already have/ordered.
I think TR-1.5 (and a shim) has not been given to Jordan for C&B. I'll take a look at these.
We investigated the low power issue with POX11 photodiode.
Restored arm algiment to get 0.8 max flashing on YARM and 1 max flashing on XARM. I had to move input alignment using TT2-PR3 pair and realign YARM cavity axis using ITMY-ETMY pair.
I would like to advertise this useful tool that I've been using for moving cavity axis or input beam direction. It's a simple code that makes your terminal kind of videogame area. It moves two optics together (either in same direction or opposite direction) by arrow key strokes (left, right for YAW, up, down for PIT). Since it moves two optics together, you actually control the cavity axis or input beam angle or position depening on the mode.
We found that one of the Y1-1037-45P marked mirror that we used was actually curved. So we removed it and used a different Y1-1037-45P mirror, adjusted the position of the lens and got the beam to land on POX11 RFPD successfully.
Then in control room, we maximized the POX11_I_ERR PDH signal amplitude by changing C1:LSC-POX11_PHASE_R to 42.95 from -67.7. We kept the C1:LSC-POX11_PHASE_D same at 90. We were getting +/- 200 PDH signal on POX_I_ERR.
Then in our attempt to lock the XARM, when we ran the "Restore XARM (POX)" script, YARM locked!
We are not sure why the YARM locked, we might have gotten lucky today. So we ran ASS on YARM and got the transmission (TRY_OUT) stable at 1. The lock is very robust and retrievable.
Coming back to XARM, we realized that the transmission photodiode used for XARM was the low-gain QPD instead of the thorlabs high gain photodiode. The high-gain photodiode was outputing large negative counts for some reason. We went to the Xend to investigate and found that the high gain photodiode was disconnected for some reason. Does anyone know/remember why we disconnected this photodiode?
We connected the photodiode back and it seems to work normally. We changed the photodiode selection back to high gain photodiode for TRX and on 40 dB attenuation, we see flashing between 1.4 to 1.6. However, we were unable to lock the XARM. We tried changing the gain of the loop, played a little bit with the trigger levels etc but couldn't get it to lock. Next shift team, please try to lock XARM.
[Paco, Anchal, Yuta]
We opened the BSC and ITMX chamber in the morning (Friday) to investigate POX11 beam clipping. We immediately found that the POX11 beam was clipping by the recently installed cable posts, so luckily no major realingment had to be done after reinstalling the cable post in a better location.
Because we had the BSC open, we decided to steer the AS1 mirror to align the AS path from ITMY all the way to the vertex chamber. Relatively small AS1 offsets (of ~ 2000 counts each) were added on PIT / YAW to center the beam on ASL (there is slight clipping along PIT, potentially because of the AS2 aperture. We then opened the vertex chamber and located the AS beam with relative ease. We decided to work on this chamber, since major changes propagate heavily downstream (simply changing the IMC pointing).
Anchal removed old optics from the vertex chamber and we installed the steering pair of mirrors for AS path. This changed the balance of the vertex table by a lot. By using the MC REFL camera beam spot we managed to coarsely balance the counterweights and recover the nominal IMC injection pointing. Simply reenabling the IMC autolocker gave us high transmission (~ 970 counts out of the typical 1200 these days).
The final IMC alignment was done by Anchal with delicate PIT motion on the input injection IMC miror to maximize the transmission (to our satisfaction, Anchal's motion was fine enough to keep the IMC locked). The end result was quite satisfying, as we recovered ~ 1200 counts of MC transmission.
Finally, we looked at the arm cavity transmission to see if we were lucky enough to see flashing. After not seeing it, we adjusted TT1 / TT2 to correct for any MMTT1 pitch adjustment needed after the vertex table rebalancing. Suprisingly, we didn't take too long and recovered the nominal arm cavity pointing after a little adjustment. We stopped here, but now the vertex table layout is final, and AS beam still needs to be aligned to the vertex in-air table.
I installed POP_SM4 and POP_SM5 in the ITMX chamber in the nominal positions. This must have affected the ITMX Oplev because I could see that one of the ITMX oplev beam was going through POP_SM5. It needs to be changed in order to follow the original plan. However, since POP_SM5 is a 1064 line mirror, it is transparent to the opleve beam, so maybe we can just use the ITMX oplev in the current fashion.
[Paco, JC, Anchal]
We balanced the IMC table back again to point that got us 50% of nominal transmission from IMC. Then we tweaked the steering mirror for injection to IMC to get up to 90% of nominal transmission. Finally, we used WFS servo loop to get to the 100% nominal transmission from IMC. However, we found that the WFS loop has been compromised now. It eventually misaligns IMC if left running for a few minutes. This needs to be investigated and fixed.
We first aligned the single arm cavity resonance for both arms to get maximum flashing. As we opened the chamber, I found that the POP beam was mostly hitting the POP_SM4 mirror but was clipping about 2 mm on the top edge.
I used TT2-PR3 to lower the injection beam angle and moved pairs of ITMY-ETMY, and ITMX-ETMX to recover as much flashing as I could in the both arms. Then, I moved PR2 in pitch from 49 to 71 to maximize the arm flashing again. After these steps, the POP beam was clearly within the POP_SM4 mirror but still in the upper half of the optic and there was maybe just a mm of clearance from the top edge. I decided to raise POP_SM4 mirror by 0.14" spacer. Now the beam is still in upper half of the mirror but has a good clearance from the edge.
The POP beam is coming outside in the in-air table at as a rising beam in the nominal path near the center of the window. This beam needs to be directed to the POP camera and RFPD on the far-side of the table.
We investigated why WFS loop wasn't working. It seemed like WFS1 PIT error signal has a huge offset which would push the loop to misalign all optics' PIT. So we did the following steps:
As I went to correct the ITMX Oplev mirrors, I found that both mirrors were placed in very different positions than the design position. Part of the reason I think was to preserve outside oplev path, and party because a counterweight was in ITMXOL1 position. I had to do following steps to correct this:
During the above work, i must have kicked the cable between the vacuum flange and the satellite amplifier box for ITMX. This disconnected all the OSEMs and Coils. We tried several things to debug this and finally found that nudging the connections on Sat Amp box brought the OSEMs and coils back online. Note that the connector was not partially out or in a state that obviously showed disconnection of the pins. I'm glad we are putting in new electronics soon for the vertex optics as well.
I successfully steered out the two output beams from BHD BS to ITMY table today. This required significant changes on the table, but I was able to bring back the table to balance coarsely and then recover YARM flashing with fine tuning of ITMY.
Thanks Chris for the fix. We are able to access the testpoints now but we started facing another issue this morning, not sure how it is related to what you did.
The steps we have tried to fix this are:
These above steps did not fix the issue. Since we have the testpoints (C1:SUS-ETMX_TRX_OUT & C1:SUS-ETMY_TRY_OUT) for now to monitor the transmission levels, we are going ahead with our upgrade work without resovling this issue. Please let us know if you have any insights.
[Anchal, Paco, Yuta]
I centered WFS1 PD so that IMC WFS Servo does not go out of range.
[Anchal, Paco, Yuta, JC]
I modified the script freeSwing.py to use damping loop output switches to free the optic instead of watchdog or coil output filters. This ensures that the free swing test is being done at the nominal position of the optic. I started tests for LO1, LO2, As2, As4, PR2, PR3, and SR2 in a tmux session names freeSwing on rossa.
Note: LO2 face OSEMs are hardly sensitive to any motion right now due to excessive pitch offset required for LO beam. We should relieve this offset to LO1 and rerun this test later.
Since UL coil actuation is lost, we modified the output matrix of ITMY to use only UR, LR and LL face coils for POS, PIT and YAW actuation. The output matrix was changed to following:
After this change, the damping was still working as good as before. I took PIT to POS/PIT/YAW and YAW to POS/PIT/YAW coupling measurements by exciting C1:SUS-ITMY_ASCPIT[YAW]_EXC and seeing effect at C1:SUS-ITMY_SUS[POS/PIT/YAW]_IN1 when the damping loops were off. Attached are the results. We were able to reduce PIT to YAW and YAW to PIT coupling by 10 dB by this simple change in output matrix. More coil balancing or off-diagonal termsmight help more and should be attempted if required. The coupling to POS did not change much.
Note that attachment 1 shows transfer functions from excitation point to the DOF sensing inputs while attachment two looks at ratio of C1:SUS-ITMY_SUS[POS/PIT]_IN1 to C1:SUS-ITMY_SUSYAW_IN1 which is the actual quantity of interest. I didn't repeat the PIT measurement due to lack of time.
Also note that all such measurements are being recorded in our new measurements git repo. We'll populate this repo with diaggui template+data files as we do measurements.
In the weekend, I ran a free swing test on all optics. During this test, LO2 magnets got stuck to the coil because LO2 PIT alignment was very high, making its lower OSEMs almost fully dark and upper OSEMs almost fully bright. Today we realized that LO2 is actually stuck and is not coming off even when we dither PIT alignment. We tried several ways but could not get this off.
Do we have any other method to get magnets off in vaccum?
It will be pretty bad if we try anything related to BHD with LO beam reflecting off a stuck mirror. Does anyone have any suggestions other than venting and fixing the issue?
I finally got YARM AS to work today. It is hard to describe what worked, I did a lot of monkey business and some dirty offset measurements to create the ASS output matrix that gave results. Note that I still had to leave out ITMY PIT L error signal, but transmission was maximizing without it. The beam does not center fully on ITMY in Pit direction right now, but we'll mvoe on from this problem for now. Future people are welcome to try to make it work for this last remaining error signal as well.
I used the same fre swing data to diagnolize input matrices of following optics:
MC1, MC2, MC3
BS ETMX ETMY
AS1 AS4 SR2 PR2 PR3
For all these optics, the new input matrices worked well. Next step should be to take the local damping open loop transfer functions and standardize the loops to same UGF.
What didn't work:
All diagonalization results are present in https://git.ligo.org/40m/scripts/-/tree/main/SUS/InMatCalc
For looking at the results at this point, go to this commit: https://git.ligo.org/40m/scripts/-/tree/7ef6a47d1b2051a0732f46477624a9e625737fe8
Over the weekend, I ran freeSwing test with sequential kicks in specific DOFs for LO1, ITMY, ITMX. LO1 results were successfully used to diagonalize LO1 input matrix. There are some issues for ITMY, ITMX still. I could not run LO2 test.
The free swing test ran successfully, resonant frequencies for different DOFs was extracted, and new input matrix was calculated. The new matrix was only slightly different from before and it worked fine with existing damping loops. The observed resonance frequencies were different from previous values by POS: -6 mHz, PIT : -3 mHz, YAW: -9 mHz, SIDE: -2 mHz. Attached are the diagonalization result.
The peculiarity of ITMX remained even after the second free swing test. The calculated input matrix is very different from existing one with sign flips across PIT and POS rows. I found that our LR osem is always bright in ITMX at the current alignment position. I see that LR osem comes in range when C1:SUS-ITMX_PIT_COMM is raised above 0.5. Maybe we should run this test when we know for sure ITMX is in correct position.
In ITMY on the other hand, I found that SIDE OSEM was completely bright. This happened during the YAW kick to ITMY. We'll need to reduce kick amplitudes for ITMY and redo this test.
For LO2, I could not initiate the test. On reducing the alignment offsets for LO2 (so that it doesn't get stuck in the fre swing test), the damping loops were not working. This is a clear evidence also that input matrix is different for different positions of the optic. We need to think about some other strategy to do this test, maybe see if ideal input amtrix works at no offsets and use that to damp during the test.
The 0x2000 error in c1su2 happens whenever we make it and install it as the default data acquisition rates are too much in the suspension model. Earlier we used activateSUS2DQ.py to fix this. I followed the suggestion in 40m/16537 to include COIL_OUT at 16k, damping channels at 256 Hz and OL channels at 1024 Hz. I created new suspension model at /cvs/cds/rtcds/userapps/trunk/sus/c1/models/lib/sus_single_control_new.mdl. The model also contains filter modules names C1SUS_OPT_BIASPOS, C1SUS_OPT_BIASPIT, C1SUS_OPT_BIASYAW which acts on the alignment offsets so that a low pass filter can be added there and alignment offsets always happen slowly. The new suspension model is now used inc1su2 for teh 7 new suspensions, and now the model starts without errors.
Still remaining to fix: IPC communication between c1hpc and c1lsc.
This issue is very weird and still unresolved. Without WFS loops, we'll have to realign IMC often and we might loose IMC alignment completely during weekends or long weekends.
I tried following things today but nothign worked:
We need to investigate this as first priority. Maybe some cable is loose, some PD power supply not working etc. Until we fix this, people should align IMC to > 12000 transmission counts whenever they have a spare 5 min. We need to work in place of WFS for sometime.
I added today demodulation of C1:LSC-BEATX/Y_FINE_I/Q in the c1cal demodulation where different degrees of freedom can be dithered. For McCal (formerly soCal), we'll dither the arm cavity for which we can use any of the DOFs (like DARM) to send the dither to ETMX/ETMY. Then with green laser locked as well, we'll get the calibration signal from the beatnotes in the demodulaed channels. We can also read right after the mixing in c1cal model and try differnt poles for integration .
I've also added medm screens in the sensing matrix part of LSC screen. These let you see demodulation of beatnote frequency signals.