[paco]

Added input filters, input matrix, damping filters, output matrix, coil filters, and copy the state over from LO1 into AS1 screen in anticipation for damping.

Added input filters, input matrix, damping filters, output matrix, coil filters, and copy the state over from LO1 into AS4 screen in anticipation for damping.

[Anchal, Paco]

Yesterday we noticed that one of the ADC channels was overflowing. I checked the signal chain and found that CH3 on PR2 Sat Amp was railing. After a lot of debugging, our conclusion is that possible the PD current input trace is shorted to the positive supply through a finite resistance on the PCB. This would mean this PCB has a manufacturing defect. The reason we come to this conclusion is that even after removing the opamp U3 (AD822ARZ), we still measure 12.5 V at the pins of R25 (100 Ohm input resistance)

Please see the schematic for reference. We also checked the resistance between input of R25 (marked PDA above) and positive voltage rail and it came out as 3 kOhms. While I all other channels, this value was 150 kOhms.

I would like it if someone else also takes a look at this. We probably would need to change the PCB in this chassis or use a spare chassis.

The unit was tested before by Tege. The test included testing the testpoint voltages only. He summarized his work in this doc. The board number is S2100737. Here are the two comments about it:
"This unit presented with an issue on the PD1 circuit of channel 1-4 PCB where the voltage reading on TP6, TP7 and TP8 are -15.1V,  -14.2V, and +14.7V respectively, instead of ~0V.  The unit also has an issue on the PD2 circuit of channel 1-4 PCB because the voltage reading on TP7 and TP8 are  -14.2V, and +14.25V respectively, instead of ~0V."

"Debugging showed that the opamp, AD822ARZ, for PD2 circuit was not working as expected so we replaced with a spare and this fixed the problem. Somehow, the PD1 circuit no longer presents any issues, so everything is now fine with the unit."

Note:  No issues were reported on PD3 circuit is is malfunctioning now.

I used the open light level output of 908 for ITMX side OSEM from 40m/16549 to roughly calibrate cts2um filter module in LO1 OSEM input filters. All values were close to 0.033. As the calibration reduces the signal value by about 30 times, I increased all damping gains by a factor of 30. None of loops went into any unstable oscillations and I witnessed damping of kicks to the optic.

In-loop power spectrum

I also compared in-loop power spectrum of ETMX and LO1 while damping. ETMX was chosen because it is one of the unaffected optics by the upgrade work. ITMX is held by earthquake stops to avoid unnecessary hits to it while doing chamber work.

Attachment 1 and 2 show the power spectrum of in-loop OSEM values (calibrated in um). At high frequencies, we see about 6 times less noise in LO1 OSEM channel noise floor in comparison to ETMX. Some peaks at 660 Hz and 880 Hz are also missing. At low frequencies, the performance of LO1 is mostly similar to EMTX except for a peak (might be loop instability oscillation) at 1.9 Hz and another one at 5.6 Hz. I'll not get into noise hunting or loop optimization at this stage for the suspension. For now, I believe the new electronics are damping the suspensions as good as the old electronics.

We found that there was a small offset (~300 mV) at TP6 and TP8, in PD2 circuit (CH2 of the board). I replaced U3 AD822ARZ but did not see any affect. I disconnected the adaptor board in the back and saw that the offset went away. This might mean that the cable had some flaky short to a power supply pin. However, when I just reinserted the adaptor board back again, there was no offset. We could not find any issue with the board after that to fix, so we left it as it is. If this board gives offset issues in the future, most probably the ribbon cable would be the suspect.

Now all ADC channels are showing no offset or overflows in C1SU2 chassis.

I tested 2 more optics found by Paco and Yehonathan in QIL.

I would like someone to redo the second test. I'm not sure what was happening but I could not find the transmitted beam at all on my card even with all lights out. This is either too good a coating and not useful for us or I did something wrong while measuring it.

V6-704, V6-706 mirror seemed like a good candidate as the paper with it said it would be a 200 ppm mirror. But I measured a lot more transmission than that. Now that I see that paper more carefully, it is a 45 degree s-pol mirror, probably that's why it had so much transmission for p-pol at near-normal incidence.

LO1 is set to go through a free swinging test at 1 am tonight. We have used this script (scripts/SUS/InMatCalc/freeSwing.py) reliably in the past so we expect no issues, it has a error catching block to restore all changes at the end of the test or if something goes wrong.

To access the test, on rossa, type:

Then you can kill the script if required by Ctrl-C, it will restore all changes while exiting.

The frree swinging test was successful. I ran the input matrix diagonalization code (scripts/SUS/InMAtCalc/sus_diagonalization.py) on the LO1 free swinging data collected last night. The logfile and results are stroed in scripts/SUS/InMatCalc/LO1 directory. Attachment 1 shows the power spectral density of the DOF bassis data (POS, PIT, YAW, SIDE) before and after the diagonalization. Attachment 2 shows the fitted peaks.

0.12

0.137

0.338

0.321

0.004

1.282

1.087

-0.57

-0.375

-0.843

1.07

-0.921

-1.081

0.91

0.098

-0.042

0.383

0.326

-0.099

0.857

The new matrix was loaded on LO1 input matrix and this resulted in no control loop oscillations at least. I'll compare the performance of the loops in future soon.

I did this simple calculation where I assumed 1W power from laser and 10% transmission past IMC. We would go ahead with V6-704/V6-705 ATFilms 3/8" optic. It would bring down the PRC gain to ~30 but will provide plenty of light for LO beam and alignment.

[Anchal, Paco, Yehonathan]

Connected in-cir cable to new flange on ITMY Chamber

Connected OSEM one-by-one. Starting from top right  to left (PIn1)

1st connector: LL -> UR -> UL

2nd connector: LR -> SD

Loosening all OSEMs and taking them out and noting full bright readings:

• SD: 29564 -> 14787
• LR: 30902 -> 15451
• UR: 29280 -> 14640
• LL: 27690 -> 13845
• UL: 27668 -> 13834

:( We had to stop here as we were unable to actuate on the side coils. We checked the signal chain and found that the monitor output of AS1 LL/SD coil driver is responding to offset changes in the coil output filter module of AS1 side. However, when we connected the output of the coil driver through a breakout board to the AS1 Sat Amp, we saw no signal. We tried switching the coil driver bo with another one one the rack but we found the exact same issue. This led us to believe that something must be wrong with the AS1 Sat Amp. We checked the output of the AS1 LL/SD coil driver without connecting it to the sat amp and found that the output was responding to our CDS changes. Then we checked the second "Coil Input" port of the AS1 Sat Amp, and found that pins 2-7 and pins 3-8 are shorted. This means channel 5 and 8 on this box would be shorted. This is the reason why we were unable to actuate on the coils. I'll work on debugging this box, my first guess is that the ribbon cable is bad.

AS1 Sat Amp (S2100741) has a critical PCB issue on it's Ch5-8 board S2100548. This board is supposed to just feed through the coil driver signal from the front DB9 connector to the back DB25 connector but it has a short between pins 2 and 7 at the "Coil Input" end (connector J1). The short persists even after I disconnect the sat amp to the flange connector on the back of this board, which definitely means the short is present in the passive channeling through the PCB or at the soldering points of the two DB connectors. I just flipped the board and found that the soldering connections are clean and separate. I think we'll have to use one of the spare sat amp boxes for AS1 for now, while we either declare this one manufacture defected or fix the issue.

I actually found the short on the PCB trace by just looking carefully at it. Attachment 1 shows the photo of it. Maybe we can fix this by simply cutting the tumor between the two traces (why are these traces so close together in such a large board anyways!!!), but I'm not sure if that is a reliable way of fixing this issue. I'll wait for Koji's comments on what to do with this. We'll recommence OSEM tuning for AS1 tomorrow with fixed electronics.

I fixed the issue in AS1 Sat Amp (S2100741) by using a razor blade. I cut the short between the two places, cleaned up the area and covered it with electrical tape. However, later feedback from Rana was to not use electrical tape as it dries up and becomes brittle and lfaky in long run. So after the AS1 OSEM tuning is over, I'll open this box again and use something else to insulate the exposed area. See attached pictures for current status.

[Anchal (vacuum work), Paco (outside)]

After the AS1 Sat Amp fix (40m/16579), we today were able to tune all OSEMs to the mid-bright level. But when we were about to call it, we were told that the new PEEK earthquake stop screw and bolts need to go on the thin suspended optics. Against better judgment, we decided to install the new back earthquake stop in-situ since we had tuned all OSEMs already. I installed the new stop but after that found that in the process I have broken off the side magnet and LR magnet from the optic adaptor and they are inside the OSEM coils now. This means we'll have to redo the AS1 suspension almost from scratch again . We have transported AS1 to the cleanroom where the work on re-suspension has begun.

After the debacle with AS1 (40m/16580), we decided the put the PEEK earthquake stop by first removing the lower OSEM plate and then doing it. So I unfastened AS4 from its position with the earthquake stops in place and moved the suspension to the center of the table. Then I carefully removed the bottom OSEM plate. But I found out that the LR magnet is broken and lying on the floor of the suspension . Given my past on the same day, it could be me breaking it again during the moving of the suspension of taking off the OSEM plate or there is a small chance that this break happened before. Regardless of fault, this meant we have to resuspend AS4 again as well. So we transported AS4 back to the clean room and the work on it's re-suspension has begun.

I corrected the calculation by adding losses by the arm cavity ends times the arm cavity finesse and also taking into account the folding of the cavity mirror. I used exact formula for finesse calculation and divided it by pi to get the PRC gain from there. Attachment 3 is the notebook for referring to the calculations I made.

Note that using V6-704 would provide 35 mW of LO power when PRFPMI is locked and 113 uW for alignment, but will bring down the PRC Gain to 17.5.

pre-2010 ITM (if it is still an option) would provide 12 mW of LO power when PRFPMI is locked and 28 uW for alignment, but will keep the PRC Gain to 24.6.

I still have to do a curvature check on the V6-704 optic.

Yeah, I counted the loss from arm cavities as the transmission from ETMs on each bounce. I assumed Michelson to be perfectly aligned to get no light at the dark port.  Should I use some other number for the round-trip loss in the arm cavity?

I labeled all the newly installed flanges and connected the in-air cables (40m/16530) to appropriate ports. These cables are connected to the CDS system on 1Y1/1Y0 racks through the satellite amplifiers. So all new optics now can be damped as soon as they are placed. We need to make more DB9 plugs for setting "Acquire" mode on the HAM-A coil drivers since our Binary input system is not ready yet. Right now, we only have 2 such plugs which means only one optic and be damped at a time.

I updated the arm cavity roundtrip losses due to scattering. Yehonathan told me that arm cavity looses 50ppm every roundtrip other than the transmission losses. With the updated arm cavity loss:

AS4 was succesfully suspended and trasported to ITMY chamber (40m/16589). We placed it near the door to make it easy to tune the OSEMs. We connected the OSEMs and found that the LL OSEM is not responding. Even though the the OSEMs are completely out right now, there was no signal on this OSEM. This could be an issue in either at the LED driver circuit or the PD circuit in AS4 Sat Amp box, or it could be the OSEM that is bad. We'll investigate further next day. For now, we recorded the full brightness reading for the OSEMs:

• UL: 32767  -> 16383
• UR: 29420 -> 14710
• LR: 30100 -> 15050
• SD: 29222 -> 14611

Another thing to note is that UL value above is not changing at all. I checked the CDS screen and the the ADC input is overflowing in complete bright position of the OSEM.

I tested the monitor ports on the SR2 Sat Amp Box but found that all LED Mon and PD Mon are giving expected values. I disconnected the cable to OSEM and checked the PD monitors and found no offsets in case of no PD current. I realised that PD transimpedance offset should be checked with PD inputs shorted instead. So I created a male DB 25 connector with pinds 2-3, 50-6, 8-9 and 11-12 shorted. This on connecting to the OSEM cable at the back of sat amp boxes should short the PD inputs. On using this plug, I found no offsets in any of the Sat Amp PD output channels.

Discussion with Koji

It is possible that the issue is there because the magnet is missing the LED-PD path way because it is offset sideways. In fact, in my limited memory, I do not recall seeing the UL OSEM signal ever going to complete darkness either. Tomorrow, we should take a photo of the OSEMs from the back and see if any sideways adjustment of the top OSEM plate is required. If any adjustment is required, we must take the OSEMs out and then do the adjustment. Do not attempt to adjust OSEM plate with OSEMs inserted in-situ. That will most probably knock off the magnets.

It was indeed the issue of the top OSEM plate not being in the right place horizontally. But the issue was more non-trivial. I believe because of the wedge in thick optics, there is a YAW offset in the optic in the free hanging position. I had to readjust the OSEM plate 4 times to be able to get full dark to bright range in both upper OSEMs. After doing that, I tuned the four OSEMs somewhat near the halfway point and once I was sure I'm inside the sensitive region in all face OSEMs, I switched on POS, PIT, and YAW damping. Then I was able to finely tune the positions of both upper OSEMs.

However, on reaching to lower right OSEM, I found again the same issue. I had to stop to go to the 40m meeting, I'll continue this work in the afternoon. But OSEM plate adjustment in the horizontal direction, particularly for thick optics is required to be done before transporting them. I achieved the best position by turning the OSEM 90 degrees and using the OSEM LED/PD plates to determine the position. This was the final successful trial I did in adjusting the plate position horizontally.

I have further updated my calculation. Please find the results in the attached pdf.

Following is the description of calculations done:

Arm cavity reflection:

Reflection fro arm cavity is calculated as simple FP cavity reflection formula while absorbing all round trip cavity scattering losses (between 50 ppm to 200 ppm) into the ETM transmission loss.

So effective reflection of ETM is calculated as

The magnitude and phase of this reflection is plotted in page 1 with respect to different round trip loss and deviation of cavity length from resonance. Note that the arm round trip loss does not affect the sign of the reflection from cavity, at least in the range of values taken here.

PRC Gain

The Michelson in PRFPMI is assumed to be perfectly aligned so that one end of PRC cavity is taken as the arm cavity reflection calculated above at resonance. The other end of the cavity is calculated as a single mirror of effective transmission that of PRM, 2 times PR2 and 2 times PR3. Then effective reflectivity of PRM is calculated as:

Note, that field transmission of PRM is calculated with original PRM power transmission value, so that the PR2, PR3 transmission losses do not increase field transmission of PRM in our calculations. Then the field gain is calculated inside the PRC using the following:

From this, the power recycling cavity gain is calculated as:

The variation of PRC Gain is showed on page 2 wrt arm cavity round trip losses and PR2 transmission. Note that gain value of 40 is calculated for any PR2 transmission below 1000 ppm. The black verticle lines show the optics whose transmission was measured. If V6-704 is used, PRC Gain would vary between 15 and 10 depending on the arm cavity losses. With pre-2010 ITM, PRC Gain would vary between 30 and 15.

LO Power

LO power when PRFPMI is locked is calculated by assuming 1 W of input power to IMC. IMC is assumed to let pass 10% of the power (). This power is then multiplied by PRC Gain and transmitted through the PR2 to calculate the LO power.

Page 3 shows the result of this calculation. Note for V6-704, LO power would be between 35mW and 15 mW, for pre-2010 ITM, it would be between 15 mW and 5 mW depending on the arm cavity losses.

The power available during alignment is simply given by:

If we remove PRM from the input path, we would have sufficient light to work with for both relevant optics.

I have attached the notebook used to do these calculations. Please let me know if you find any mistake in this calculation.

The main issue with SR2 OSEMs, now that I think of it, was that the BS table was very inclined due to the multiple things we removed (including counterweights). Today the first I did was level the BS table by placing some counterweights in the correct positions. I placed the level in two directions right next to SR2 (clamped in its planned place), and made the bubble center.

While doing do, at one point, I was trying to reach the far South-West end of the table with the 3x heavy 6" cylindrical counterweight in my hand. The counterweight slipped off my hand and fell from the table. See the photo in attachment 1. It went to the bottommost place and is resting on its curved surface.

This counterweight needs to be removed but one can not reach it from over the table. So to remove it, we'll have to open one of the blank flanges on the South-west end of BS chamber and remove the counterweight from there. We'll ask Chub to help us on this. I'm sorry for the mistake, I'll be more careful with counterweights in the future.

Moving on, I tuned all the SR2 OSEMs. It was fairly simple today since the table was leveled. I closed the chamber with the optic free to move and damped in all degrees of freedom.

Photos: https://photos.app.goo.gl/CQ6VouSB1HX2DPqW6

SUSPENSION STATUS UPDATED HERE

AS4 is set to go through a free swinging test at 10 pm tonight. We have used this script (Git/40m/scripts/SUS/InMatCalc/freeSwing.py) reliably in the past so we expect no issues, it has a error catching block to restore all changes at the end of the test or if something goes wrong.

To access the test, on allegra, type:

Then you can kill the script if required by Ctrl-C, it will restore all changes while exiting.

SR2 is set to go through a free swinging test at 3 am tonight. We have used this script (Git/40m/scripts/SUS/InMatCalc/freeSwing.py) reliably in the past so we expect no issues, it has a error catching block to restore all changes at the end of the test or if something goes wrong.

To access the test, on allegra, type:

Then you can kill the script if required by Ctrl-C, it will restore all changes while exiting.

The free swinging test was successful. I ran the input matrix diagonalization code (/opt/rtcds/caltech/c1/Git/40m/scripts/SUS/InMAtCalc/sus_diagonalization.py) on theSR2 free-swinging data collected last night. The logfile and results are stored in /opt/rtcds/caltech/c1/Git/40m/scripts/SUS/InMAtCalc/SR2 directory. Attachment 1 shows the power spectral density of the DOF basis data (POS, PIT, YAW, SIDE) before and after the diagonalization. Attachment 2 shows the fitted peaks.

1.09

0.914

0.622

0.798

-0.977

1.249

0.143

-1.465

-0.359

0.378

0.552

-1.314

-0.605

1.261

0.118

0.72

0.403

0.217

0.534

3.871

The new matrix was loaded on SR2 input matrix and this resulted in no control loop oscillations at least. I'll compare the performance of the loops in future soon.

The free swinging test was successful. I ran the input matrix diagonalization code (/opt/rtcds/caltech/c1/Git/40m/scripts/SUS/InMAtCalc/sus_diagonalization.py) on the AS4 free-swinging data collected last night. The logfile and results are stored in /opt/rtcds/caltech/c1/Git/40m/scripts/SUS/InMAtCalc/AS4 directory. Attachment 1 shows the power spectral density of the DOF basis data (POS, PIT, YAW, SIDE) before and after the diagonalization. Attachment 2 shows the fitted peaks.

0.844

0.707

0.115

0.253

-1.646

0.122

0.262

-1.319

-1.459

0.214

1.087

-0.901

-0.874

1.114

0.016

0.622

0.934

0.357

0.045

3.822

The new matrix was loaded on AS4 input matrix and this resulted in no control loop oscillations at least. I'll compare the performance of the loops in future soon.

I placed LO2 in its planned position in BS chamber, inserted the OSEMs, and tuned their position to halfway brightness. At the end of the work, I was able to damp the optic successfully. The full open (full brightness) OSEM ADC counts are:

UL 25743.  -> 12876
UR 27384. -> 13692
LL 25550. -> 12775
LR 27395 -> 13697
SD 28947 -> 14473

Today's OSEM tuning was relatively unhappening. I have only following two remarks:

• BS table was 3 SOS near the East end and PRM is parked in the center, thus the table is very unevenly balanced. I had to use all available counter weights to make it flat near the LO2 suspension.
• The side OSEM for LO2 is not exactly centered (probably due to table imbalance). I was able to balance the table to a point though that the side OSEM was responsive to full range and I was able to damp the optic.

Free swinging test set to trigger

LO2 is set to go through a free swinging test at 10 pm tonight. We have used this script (Git/40m/scripts/SUS/InMatCalc/freeSwing.py) reliably in the past so we expect no issues, it has a error catching block to restore all changes at the end of the test or if something goes wrong.

To access the test, on allegra, type:

Then you can kill the script if required by Ctrl-C, it will restore all changes while exiting.

Photos: https://photos.app.goo.gl/Ff3yGBprj9xgPbnLA

SUSPENSION STATUS UPDATED HERE

I agree. That's an interesting idea. But does that mean that there is an always working inverse matrix solution or that any solution will be vulnerable to the alignment biases.

I think we can also calculate the matrix rotation required as a function of dc biases and do that rotation in the simulimk model.

The free swinging test was successful. I ran the input matrix diagonalization code (/opt/rtcds/caltech/c1/Git/40m/scripts/SUS/InMAtCalc/sus_diagonalization.py) on the LO2 free-swinging data collected last night. The logfile and results are stored in /opt/rtcds/caltech/c1/Git/40m/scripts/SUS/InMAtCalc/LO2 directory. Attachment 1 shows the power spectral density of the DOF basis data (POS, PIT, YAW, SIDE) before and after the diagonalization. Attachment 2 shows the fitted peaks.

0.46

1.237

1.094

0.318

0.98

1.091

0.252

-1.512

-0.672

-0.088

0.722

-1.014

-0.217

1.519

0.314

-0.747

1.523

1.737

-0.534

3.134

The new matrix was loaded on LO2 input matrix and this resulted in no control loop oscillations at least. I'll compare the performance of the loops in future soon.

For some reasonf the free swing test showed only one resonance peak (see attachment 1). This probably happened because one of the earthquake stops is touching the optic. Maybe after the table balancing, the table moved a little over its long relazation time and by the time the free swing test was performed at 3 am, one of the earthquake stops was touching the optic. We need to check this when we open the chamber next.

I noticed that our current suspension damping loops for the new SOS were railing the DAC outputs. The reason being that cts2um module has not been updated for most optics and thus teh OSEM signal (with the new Sat Amps) is about 30 times stronger. That means our usual intuition of damping gains is too high without applying correct conversion cts2um filter module. I reduced all these gains today and nothing is overflowing the c1su2 chassis now. I also added two options in the "!" (command running drop down menu) in the sus_single medm screens for opening ndscope for monitoring coil outputs or OSEM inputs of the optic whose sus screen is used.

[Anchal, Paco]

We removed the old PR3 housed in a tip-tilt style suspension and put it on the North flow bench in the cleanroom. I put PR3 in an accessible position near the North West edge of the BS chamber and balanced the table again with many weights. The OSEM tuning was very uneventful and easy. Following are the full brightness ADC counts for the OSEMs:
UL 25693. -> 12846
UR 24905. -> 12452
LL 23298. -> 11649
LR 24991. -> 12495
SD 26357. -> 13178

I was able to damp the optic easily after the OSEM installation with no issues.

Photos: https://photos.app.goo.gl/jcAqwFJoboeUuR7F9

AS1, PR2 and PR3 are set to o go through a free swinging test at 3 am. We have used this script (Git/40m/scripts/SUS/InMatCalc/freeSwing.py) reliably in the past so we expect no issues, it has a error catching block to restore all changes at the end of the test or if something goes wrong.

To access the test, on allegra, type:

Then you can kill the script if required by Ctrl-C, it will restore all changes while exiting.

The free swinging test was successful. I ran the input matrix diagonalization code (/opt/rtcds/caltech/c1/Git/40m/scripts/SUS/InMAtCalc/sus_diagonalization.py) on the PR2 free-swinging data collected last night. The logfile and results are stored in /opt/rtcds/caltech/c1/Git/40m/scripts/SUS/InMAtCalc/PR2 directory. Attachment 1 shows the power spectral density of the DOF basis data (POS, PIT, YAW, SIDE) before and after the diagonalization. Attachment 2 shows the fitted peaks. I chose to not type out the matrix values now. One can find them in teh repo links above.

The free swinging test was successful. I ran the input matrix diagonalization code (/opt/rtcds/caltech/c1/Git/40m/scripts/SUS/InMAtCalc/sus_diagonalization.py) on the PR3 free-swinging data collected last night. The logfile and results are stored in /opt/rtcds/caltech/c1/Git/40m/scripts/SUS/InMAtCalc/PR3 directory. Attachment 1 shows the power spectral density of the DOF basis data (POS, PIT, YAW, SIDE) before and after the diagonalization. Attachment 2 shows the fitted peaks. I chose to not type out the matrix values now. One can find them in teh repo links above.

We still did not get a good AS1 free swinging spectrum. It seems the Side OSEM is reporting coupling to too many DOFs and some extra resonances than we expect. So we did not upload the new input matrix calculated from this test. I'm attaching teh peak fitting (attachment 1) and the diagonalization attempt (attachment 2) to give some idea of what happened. Note how different this fre swinging spectrum looks from the other optics. Given that Yehonathan also felt that somthing might be off with this optic, we need to reevaluate if the suspension has wire not aligned in grove, or it is imbalanced or something else if touching it still.

Please edit this same entry throughout the day for the shutdown elogging.

I took a screenshot of C0VAC_MONITOR.adl to ensure that all pnematic valves are in closed positions:

The status message says "All pnematic valves closed" and the latest error message is about "V7 closed, N2 < 6.50e+01".

I found out that there was no autoburt happening for c1vac channels. I created an autoBurt.req file for the vac.db file and saved one snapshot. I also added the path of this file in autoburt/.requestfilelist . Let's see if autoburting starts by that for this file as well.

With this, I think we can safely shutdown acromag chassis. Hopefully, the relays are configured such that the valves are nominally closed in absence of a control signal. After the chassis is shut down, wwe can shutdown C1VAC by:

[Chub, Jordan]

At the 1x8 rack, the following were switched off on their respective front panels:

PTP2 & PTP3 Controller
MKS Gauge controller
PRP Gauge Controller
G2P316a & b Controllers
Sorenson
Serial Device Server
Both UPS's

Powered off from back of unit:

TP1 Controller
Acromag chassis

TP2 and 3 controllers were unplugged from respective power strips (labeled C2 and C3)

C1vac and the laptop at the workstation were shut down

Manual Gate valve was closed

[Anchal, Paco]

Bringing back CDS took a lot of work yesterday. I'm gonna try to summarize the main points here.

mx_start_stop

For some reason, fb1 was not able to mount mx devices automatically on system boot. This was an issue I earlier faced in fb1(clone) too. The fix to this problem is to run the script:

To make this persistent, I've configured a daemon (/etc/systemd/system/mx_start_stop.service) in fb1 to run once on system boot and mount the mx devices as mentioned above. We did not see this issue of later reboots yesterday.

gpstime

Next was the issue of gpstime module out of date on fb1. This issue is also known in the past and requires us to do the following:

Again, to make this persistent, I've configured a daemon (/etc/systemd/system/re-add-gpstime.service) in fb1 to run the above commands once on system boot. This corrected gpstime automatically and we did not face these problems again.

time synchornization

Later we found that fb1-FE computers, ntp time synchronization was not working and the main reason was that fb1 was unable to access internet. As a rule of thumb, it is always a good idea to try pinging www.google.com on fb1 to ensure that it is connected to internet. The issue had to do with fb1 not being able to find any namespace server. We fixed this issue by reloading bind9 service on chiara a couple of times. We're not really sure why it wasn't working.

After the above, we saw that fb1 ntp server is working fine. You see following output on fb1 when that is the case:

On the FE models, timedatectl should show that NTP synchronized feild is yes. That wasn't happening even after us restarting the systemd-timesyncd service. After this, I just tried restarting all FE computers and it started working.

CDS

We had removed all db9 enabling plugs on the new SOSs beforehand to keep coils off just in case CDS does not come back online properly.

Everything in CDS loaded properly except the c1oaf model which kepy showing 0x2bad status. This meant that some IPC flags are red on c1sus, c1mcs and c1lsc as well. But everything else is green. See attachment 1. I then burtrestroed everything in the /opt/rtcds/caltech/c1/burt/autoburt/snapshots/2022/Feb/4/12:19 directory. This includes the snapshot of c1vac as well that I added on autoburt that day. All burt restore statuses were green OK. I think we are in good state now to start watchdogs on the new SOSs and put back the db9 enabling plugs.

Future work:

When somebody gets time, we should make cutom service files in fb1:/etc/systemd/system/ symbolic links to a repo directory and version control these important services. We should also make sure that their dependencies and startup order is correctly configured. I might have done a half-assed job there since I recently learned how to make unit files. We should do the same on nodus and chiara too. Our hope is that on one glorious day, the lab can be restarted without spending more than 20 min on booting up the computers and network.

I found out that the ESP300 service needs to be run in root mode for it to be able to connect to the USB port of HWP motor controller. While doing this change, I noticed that the channels hosted by c1psl might have a duplication conflict with some other channel hosting computer, because a lot of them show the Warning: "Identical process variable names on multiple servers" which is not good. Someone should look into this conflict.

I added instructions on the power control MEDM screen as it was very non-trivial to use. I have set the power such that the C1:IOO-MC_RFPD_DCMON is 5.6 and this happened at C1:IOO-HWP_POS_SET 2.29.

Something is wrong with the Video MUX. The system did not turn back on with full functionality. Even though we see the screens as they were before the power shutdown, we have lost control on switching any of the videos. I went to check the wiki page about Video MUX which told be we should be able to see the configuration screen on this link, but the page wasn't opening. I went and removed the power cable and put it back in. That brought back the configuration page. Still, I could not change any of the video feeds however this time, I could see the EPICS channel values (like C1:VID-QUAD1_4) change. I tired to go to the configuration page and change the matrix values from the control tab there. I found out that the matrix was mislabeled and while making the changes, I started seeing blue screen on QUAD1_3 (where MC2T was set before). I set the QUAD1_3 (output 23) to MC2T (input 16), but no change. The EPICS values are also set properly, so I don't understand the reason behind blue screen. The same happened when I tried to use:

Weirdly, this caused the QUAD1_4 screen to go blue. Running following had no effect:

So, I'm not sure what to do. This really needs to be fixed! I wanted to see teh MC2F camera so that I can align IMC, that was the whole reason for this rabit hole. Help needed.

Yeah, this is a known issue actually. We go to ASC screen and manually swich off all the outputs after every reboot. We haven't been able to find a way to set default so that when the model comes online, these outputs remain switched off. We should find a way for this.

I found that two computers are not powering up in the control room, Ottavia and Allegra. Allegra was important for us as it had the current version of LIGO CDS workstation installed on, providing us with options to use latest packages written by LIGO CDS team. I think the power issue should be resolvable if someone opens it and knows what thye are doing. Do we have any way of getting fuse repairs on such computers? Both these computers are Dell XPS 420.

We increased the input power to IMC by replacing the 98% transmission BS by a 10% transmission BS on the detection table (reverse of what mentioned in 40m/16408 see attachment 8-9). We then realigned the BS so that MC RFPD is centered. Then we realigned two steering mirrors to get the beam centered on the WFS1 and WFS2 QPD. Then we increased the power of the input beam to get 5.307 reading on the C1:IOO-MC_RFPD_DCMON channel. We did this so that we can align the IMC. Once we have it aligned, we'll go back to low poer for doing chamber work.

Beware, there is about 1W beam on the detection table right now.

The autoBurt file for FE already has the C1:ASC-ETMX_PIT_SW2 (and other channels for ETMY, ITMX, ITMY, BS and for YAW) present, and I checked the last snapshot file from Feb 7th, 2022, which has 0 for these channels. So I'm not sure why when FE boots up, it does not follow the switch configuration. Fr safety, I changed all the gains of these filter modules, named like C1:ASC-XXXX_YYY_GAIN (where XXXX is ETMX, ETMY, ITMX, ITMY, or BS , and YYY is PIT or YAW) to 0.0. Now, even if the FE loads with switches in ON configuration, nothing should happen. In future, if we use this model for anything, we can change the gain values which won't be hard to track as the reason why no signal moves forward. Note, the BS connections from this model to BS suspension model do not work.

I reconfigured the MC reflection path for low power. This meant the following changes:

• Replaced the 10% reflection BS by 98% reflection beam splitter
• Realigned the BS angle to get maximum on C1:IOO-MC_RFPD_DCMON when cavity is unlocked.
• Then realigned the steering mirrors for WFS1 and WFS2.
• I tried to align the light for MC reflection CCD but then I realized that the pickoff for the camera is too low for it to be able to see anything.

Note, even the pick-off for WFS1 and WFS2 is too low I think. The IOO WFS alignment does not work properly for such low levels of light. I tried running the WFS loop for IMC and it just took the cavity out of the lock. So for low power scenario, we would keep the WFS loops OFF.

As discussed in the meeting, I removed the extra beam splitter that dumps most of the beam going towards WFS photodiodes. This beam splitter needs to be placed back in position before increasing the input power to IMC at nominal level. This is to get sufficient light on the WFS photodiodes so that we can keep IMC locked for more than 3 days. Currently IMC is unlocked and misaligned. I have marked the position of this beam splitter on the table, so putting it back in should be easy. Right now, I'm trying to align the mode cleaner back and start the WFS loops once we get it locked.

I found a peculiar issue today. The C1:IOO-MC_RFPD_DCMON remains constantly 0. I wonder if the RFPF output is being read properly. I opened the table and used an oscilloscope to confirm that the DC output from the MC REFL photodiode is coming consistently but our EPICs channel is not reading it. I tried restarting the modbusIOC service but that did not affect anything. I power cycled the acromag chassis while keeping the modbusIOC service off, and then restarted teh modbusIOC service. After this, I saw more channels got stuck and became unresponsive, including the PMC channels. So then I rebooted c1psl without doing anything to the acromaf chasis, and finally things came back online. Everything looks normal to me now but I'm not sure if one of the many channels is not in the right state. Anyways, problem is solved now.

I think I have aligned the cavity, including MC1 such that we are seeing flashing of fundamental mode and significant transmission sum value as well.However, I'm unable to catch lock following Koji's method in 40m/16673. Autolocker could not catch lock either. Maybe I am doing something wrong, I'll pickup again tomorrow, hopefully the cavity won't drift too much in this time.

c1teststand has been restructured. There is no port computer called 'c1teststand' anymore. When you ssh into the c1teststand network using ssh c1teststand from inside martian or from outside network using the method mentioned in this wiki page , you would land into chiara (clone) computer and you can navigate into any teststand network computer from there.

I'll be repurposing 1U c1teststand computer into the new c1susaux2 slow machine now. All files from home directory and from /etc directory of former c1teststand have been zipped and stored in /home/controls of chiara (clone). Just a aside, the network configuration of teststand can be done from inside the teststand network, by going to a browser on either fb1 (clone) or chair (clone) and going to address 10.0.1.1. The login and password are same as our usual workstation username and password.