I want to monitor the PMC TRANS and REFL levels on the PSL table - previously there were some cables going to the oscilloscope on the shelf but someone had removed these. I re-installed them just now. While there, I disconnected the drive to the AOM - there must've been some DC signal going to it because when I removed the cable, the PMC and IMC transmission were recovered to their nominal levels.

Summary:

On Monday, I hooked up an AG4395 to the PMC error point (using the active probe). The idea was to take a spectrum of the PMC error point every time the FSS PC drive RMS channel indicated an excursion from the nominal value. An initial look at the results don't suggest that this technique is particularly informative. I'll have to think more about a workaround, but please share your ideas/thoughts if you have some.

Also, the feature in the spectrum at ~110 kHz makes me suspect some kind of loop instability. I'll measure the IMC loop OLG at the next opportunity.

Details:

• The PMC servo bandwidth is ~2 kHz, so above this, the PMC error point should be a faithful monitor of the PSL frequency noise, provided the sensing noise is low enough.
• The PMC error point sensing noise is ~100nV/rtHz (I'm monitoring this straight after the Minicircuits mixer+bandpass filter that we are using as a demodulator). This corresponds to ~2 Hz/rtHz, using the ~10 MHz/V PDH discriminant calibration from January. Seems consistent with this elog.
• I was hoping to see if there was a particular frequency band in which the noise gets elevated, and if the crossover frequency is a few kHz and the IMC servo BW is ~110 kHz, I would have expected this to be in the 10-100 kHz region. Possibly my frequency resolution isn't good enough? But with the Agilent, doing a finer grid would mean a longer measurement time, in which case the IMC might lose lock before the measurement is done.
• But, as shown in Attachment #1, there isn't any clear evidence, from the ~20 excursions that were recorded last night. The color of the line is meant to be indicative of the average value of the PC drive RMS channel in the measurement time.
• A significant bottleneck in this whole process is that it takes ~1 minute to initiate the GPIB measurement, and download the data. The pseudo-code I used is:
  • While the IMC is locked, watch PCdrive RMS EPICS channel's "ALARM" state, which becomes non-zero when the PCdrive RMS exceeds 1 V (this is how it is defined in the EPICS db record right now).
  • Make sure this isn't a transient feature - I do this by waiting 5 seconds and checking that the ALARM flag is still flagged.
  • Initiate a AG4395 measurement over GPIB - I use the measurement span of 1 kHz - 1 MHz with a BW/span ratio of 0.1%, 5 averages.
  • Check that the IMC is still locked (if it got unlocked while the measurement was made, presumably the measurement is garbage).
• Is there a better monitor of the laser frequency noise? I can imagine using POX/POY which I think have a lower electronics noise floor but I'm not sure if that's true at 100 kHz and having the arms locked in addition to the IMC seems more complicated...
• Since we are planning a laser upgrade, is this worth spending more time on? I may leave the measurement running on pianosa in a tmux session while I'm not in the lab...

I had set up the 4395 to do this automatically a few years ago, but it looked at the FSS/IMC instead. When the PCDRIVE goes high there is this excess around ~500 kHz in a broad hump.

But the IMC loop gain changes sometimes with alignment, so I don't know if its a loop instability or if its laser noise. However, I think we have observed PCDRIVE to go up without IMC power dropping so my guess is that it was true laser noise.

This works since the IMC is much more sensitive than PMC. Perhaps one way to diagnose would be to lock IMC at a low UGF without any boosts. Then the UGF would be far away from that noise making frequency. However, the PCDRIVE also wouldn't have much activity.

The PMC has been unlocked since September 11 sometime (summary pages are flaky again). I re-locked it just now. I didn't mess with the HEPA settings for now as I'm not using the IFO at the moment, so everything should be running in the configuration reported here. The particulate count numbers (both 0.3um and 0.5um) reported is ~x5-8 of what was reported on Thursday, September 10, after the HEPA filters were turned on. We don't have logging of this information in any automated way so it's hard to correlate things with the conditions in Pasadena. We also don't have a working gauge of the pressure of the vacuum envelope.

The RGA scanning was NOT enabled for whatever reason after the vacuum work. I re-enabled it, and opened VM1 to expose the RGA to the main volume. The unit may still be warming up but this initial scan doesn't look completely crazy compared to the reference trace which is supposedly from a normal time based on my elog scanning (the timestamp is inherited from the c0rga machine whose clock is a bit off).

Update 1500: I checked the particle count on the PSL table and it barely registers on the unit (between 0-20 between multiple trials) so I don't know if we need a better particle coutner or if there is negligible danger of damage to optics due to particulate matter.

The HEPA filters on the PSL enclosure are no longer running. I tried cycling the power switch on the NW corner of the enclosure, and also turned the dial on the Variac down to zero and back up to maximum, no effect. Judging by the indicator LED on it, the power is making it to the Variac - bypassing the Variac and directly connecting the HEPA to the AC power seems to have no effect either.

I can't be sure, but I'm almost certain I heard them running at max speed an hour ago while I was walking around inside the VEA. Probably any damage that can happen has already been done, but I dialled down the Innolight injection current, and closed its shutter till the issue can be resolved.

The PMC servo railed and so I re-locked it at ~half range. I've been noticing that the diurnal drift of the PZT control voltage has been larger than usual - not sure if it's entirely correlated with temperature on the PSL table. Anyway the cavity is locked again so all is good.

PMC lost lock between 21:00 and 22:00 UTC on April 11th as seen in the summary pages:

https://nodus.ligo.caltech.edu:30889/detcharsummary/day/20210411/psl/#gallery-4

That's between 2pm and 3pm on Sunday for us. We're not sure what caused it. We will attempt to lock it back.

Mon Apr 12 08:45:53 2021: we used milind's python script in scripts/PSL/PMC/pmc_autolocker.py. It locked the PMC in about a minute and then IMC catched lock succefully.

However, the PMC transmission PD shows voltage level of about 0.7V. On medm, it is set to turn red below 0.7 and yellow above. In Summary pages in the past, it seems like this value has typically been around 0.74V. Simil;arly, the reflection RFPD DC voltage is around 0.063 V right now while it is supposed to be around 0.04 nominally So the lock is not so healthy.

We tried running this script and the bashscript version too (scripts/PSL/PMC/PMCAutolocker) a couple of times but it was unable to acquire lock.

Then we manually tried to acquire lock by varying the C1:PSL-PMC_RAMP (with gain set to -10 dB) and resetting PZT position by toggling Blank. After a few attempts, we were able to find the lock with transmission PD value around 0.73V and reflection RFPD value around 0.043. PZT control voltage was 30V and shown in red in medm to begin with. So we adjusted the output ramp again to let it come to above 50V (or maybe it just drifted to that value by itself as we could se some slow drift too). At Mon Apr 12 09:50:12 2021 , the PZT voltage was around 58V and shown in green.

We assume this is a good enough point for PMC lock and move on.

PMC lost lock again at around 16:00 April 12. I was able to lock it again but the transmission is only 0.6 now and REFL is 0.14.

Rana came in and realigned the PMC stirring mirrors. Now the transmission is 0.757V, and the REFL is 0.03V.

I noticed that the PZT was around 250V. Given that the PMC got unlocked at 16:00, which is around the peak temperature time in the lab (lagging behind the outside weather), due to the PZT voltage going down to 0V, I figured that the PZT voltage would go up during the night when the lab gets cold and therefore will likely go out of range again.

I found a different working point at 150V and relocked the PMC.

We (rana, yehonathan and i) briefly talked about having high power going into the IFO. I worked on some calcs a couple of years ago, that are summarized here. There is some discussion in the linked page about how much power we even need. In summary, if we can have

• T_PMC ~85% which is what I measured it to be back in 2019
• T_IMC * T_inputFaraday ~60% which is what I estimate it to be now
• 98% mode matching into the IMC
• power recycling gain of 40-45 once we improve the folding mirror situation in the recycling cavities
• and a gain of 270-280 in the arm cavities (20-30ppm round trip loss)

then we can have an overall gain of ~2400 from laser to each arm cavity (since the BS divides the power equally between the two arms). The easiest place to get some improvement is to improve T_IMC * T_inputFaraday. If we can get that up to ~90%, then we can have an overall gain of ~4000, which is I think the limit of what is possible with what we have.

We also talked about the EOM. At the same time, I had also looked into the damage threshold as well as clipping losses associated with the finite aperture of our EOM, which is a NewFocus 4064 (KTP is the Pockel medium). The results are summarized in Attachments #1 and #2 respectively. Rana thinks the EOM can handle factor of ~3 greater power than the rated damage threshold of 20W/mm^2.

{Yehonathan, Rana}

We unpacked the content of the amplifier crate in front of the water fountain (see attachments). Inside we found:

1. Amplifier head. (attachment 1)
2. Amplifier electronics and pump diodes (attachment 2).
3. Optical fiber (attachment 3).
4. 2 Long water hoses (~2m) and 2 short ones.
5. Network cable.
6. A wooden plate.
7. Cable sleeve (attachment 2)?
8. Some manuals will be uploaded to the wiki soon.

Please don't move/touch any of that stuff

Things that we need to consider/obtain:
1. A suitable power cable (attachment 4) with suitable power ratings (800W according to the amplifier specs). The connector head is C19 I believe.
2. A chiller. Rana says Aidan knows where to find one. Should we chill the amplifier head as well?
3. A mounting plate for the amplifier head with good thermal conductivity.
4. The pump wavelength is 808nm, we need to get suitable safety goggles.
5. Where to put the big electronics box. Preferably on 1X1 or 1X2.
6. How do we arrange the different components on the table? We also need to mode match the beam into the amplifier.

A couple of years ago, I got some info about the amplifier setup at the sites from Terra - sharing here in case there is some useful info in there (our setup will be rather different, but it looked to me like our Amp is a 2017 vintage and it may be that the performance is not the same as reported in the 2019 paper).

collection of docs (table layout in 'Proposed....setup') : https://dcc.ligo.org/LIGO-T1700046

LVC 70W presentation: https://dcc.ligo.org/LIGO-G1800538 

I guess we should double check that the beam size everywhere (in vacuum and on the PSL table) is such that we don't exceed any damage thresholds for the mirrors used. 

Some more relevant documents provided by Matt:

Phase III:70W amplifier integration at LIGO

70W amplifier External Shutter

aLIGO PSL high power attenuator

I surveyed a bit the 1X1/2 area to plan for the installation of the laser amplifier.

There is a vacancy at the bottom of 1X2 (attachment 1). I measured the dimensions of the diode box (DB) and it should fit. The optical fiber bundle is 75m long and should reach the amplifier head on the table easily.

According to the specs, the maximum power consumption of the DB is 800W (typically 600W), it should probably have its own circuit breaker. It can easily draw more than a few amps. The rack power strips are connected to this 4 socket box (attachment 2), is this just another power strip? It is connected to a circuit breaker with a 30A rating. How do we proceed from here?

In any case, we will need at least 2 meters of power cable.

I also tried to find a suitable place for a water chiller. A few suggestions are in the attachments. Basically either between the electronics shelves and the small rack next to 1X2 or next to the small rack close to the optical table. Maybe put it where the ladder sits and find another place for the ladder. Other options?

We would also need a windows machine running the Beckhoff software. The idea is that all the different laser components (DB, chillers, interlocks, switches) are connected to the EtherCat (over the ethernet infrastructure) so that the Beckhoff code can recognize a failure and switch off everything.

The things that are monitored:

1. Is the NPRO on?

2. Is the flow rate from the chillers enough?

3. Is the temperature of the diodes in the normal range?

4. Is one of the interlocks open?

5. Was one of the emergency buttons pushed?

6. Was the key switch on the DB turned to OFF?

The DB is EtherCat ready but the rest of the signals need to be interfaced somehow. Do we have to buy these EtherCAT terminals?

• Ideally, we put the chiller outside of the interferometer area. The PSL chiller used to be in the control room near the door by IMC REFL. We could also put it in the drill press room.
• Once we figure out a couple of places where the Diode Box can go, we can ask facilities to make the appropriate power connections. They will have to eval the situation to figure out if the main power to the lab needs to be shut down.
• Can we put the laser diode box in the drill press room too? Then the hoses can be short. Perhaps less EMI getting into our sensitive places.

Shutdown Procedure:
PSL Shutter closed / MC Autolocker disabled / PSL mechanical shutter closed / Laser injection current turned to zero / Laser turn off (red button) / Laser key turned off

The laser stat before the shutdown:
- LD Temp A: Set 22.07 (Untouched)
- LD Temp B: Set 21.03(Untouched)
- Laser Injection Current: Dial 9.53, Actual 2.100 -> Dial was moved to zero upon shutting down
- Laser Crystal Temp: Dial 3.34 (untouched)  Set 30.57 Actual 30.60 (Untouched)

PSL Table covering

- Because of the so many cables going up and down, sealing the PSL table with the metalized sheet was not easy. Therefore, the sheets have been just softly laid above the optics. (Attachment 1)
- The largest sheet which covers the east half of the table was taped to the table at the bottom, so that the air from the chimneys (see below) does not come up to the table

- The large dust could come from the opening of the enclosure during the filter replacement. So it was considered to be easier to seal the openings. (Attachment 2)
- Of course, the HEPAs are going to be tested after the maintenance work. It means that vent paths were needed so that the seals do not explode with the pressure (together with dust).
- Thus, the tubes of the sheets are attached to the seals to form "chimneys" for guiding the airflow beneath the table. (Attachment 2/3/4)
- This configuration was not meant to be sufficiently strong for a continuous run of the fans. Long running of the HEPAs may cause the failure of the seal tapes.
  Therefore the HEPA test should be done with a low flow rate and/or a short period of high flow.

- Once the work has been done, all the sheets should be carefully removed without scattering the fallouts onto the optics.

I also located the (possible) HEPA filters in the lab. (Attachments 1~3)

Oh! This is NO-NO! We can't place anything in front of the mains breakers. (Attachment 2)
I relocated the objects (Attachment 3)

I went to the TCS lab to take a look at the chillers lying around. I spotted two chillers:

1. Thermoflex1400 (attachment 1,2). Spec sheet.

2. Polyscience Recirculator 6000 series (attachment 3,4). Manual.

The Thermoflex has various communication ports. The Recirculator doesn't have any communication ports, but it is connected to a flow meter with what seems to be an electronic readout (attachment 5). Manual.

Both chillers have similar capacity ~ 4 gallons/minute. Thermoflex has 2 times more reservoir capacity than the Recirculator.

None of them seem to be Bechkoff-ready.

I guess we can have interlock code handling mixed signals Beckhoff+Non beckhoffs?

[Anchal, Koji]

Removed the top sheet

• Opened first from the door side so that any dust would spill outside.
• Then rolled the sheet inward to meet in the middle.
• Repeated this twice for the 2 HEPA filters.

Removed the sheets on the table

• Lifted sheet up making sure the top side face outside always.
• Rolled it sideways halfway through.
• Cut down the sheet vertically.
• Slided the doors to the other side and rolled the remaining half.
• On the door side, the sheets above the ALS optics were simply lifted off.

Restarting PSL

• Turned on the HEPAs at the max speed
• Switched on laser to jsut above the threshold
  • Before the 1st eom, power was 20mW 
  • After the EOM/AOM, 18mW. So about 90% transmission through all polarizing optics.
  • We saw the resonances of the PMC but could not lock it even with highest gain available (30 dB).
• Increased the input power to PMC to 100mW
  • Locked the PMC at 30dB gain
  • The transmitted power was ~50-60 mW. (Had to use power meter suspended by hand only.
  • The right before the IMC (after the 2nd EOM) 48mW. So none of the alignment was lost.
• Opened the PSL shutter.
• We were able to see IMC reflection signal.
• We were also able to see IMC catching lock as the servo was left ON earlier.
• Switched off the servo.
• Decided to increase the power while watching PMC Trans/Refl and IMC REFL
• Injection diode current to innolight was increased slowly to 2.10A. Saw a mod hopping region aroun 1.8A.
• We recovered the PMC Trans >0.7 V.
• PZT was near the edge, so moved by one FSR.
• The PMC refelction signal is still shown in red at around 48 mV.

Back to control room

• IMC was locked almost immediately by manually finding the lock while keeping IMC WFS off to preserve the offsets from yesterday.
• Then switch on IMC WFS. Working good.
• Then unlocked the servo and switched on IMC Autolocker. Lock was caught immediately.

Decided to start locking the arms

• The arm transmissions were flashing but at 0.2~0.3 level.
• Decided to adjust TT1 and TT2 Pitch and Yaw to align the light going into the arms.
• This made TRY ~0.6 / TRX ~0.8 at the peak of the flashing
• Locked the arms. (By switching on C1:LSC-MODE_SELECT which engages all servos).
• Used ASS to align Yarm then align Xarm. Procedure:
  • Sitemap > ASC > c1ass
  • Open striptool to look at progress. ! Scripts YARM > striptool.
  • Switch on ASS. ! More Scripts > ON
  • Wait for the TRY to reach to around 0.97.
  • Freeze the outputs. ! Scripts > Freeze Outputs.
  • Offload the offsets to preserve the output. ! More Scripts > OFFLOAD OFFSETS.
  • Switch off ASS. ! More Scripts > OFF
  • Repeted this for XARM.
• At the end, both XARM and YARM were locked with TRX ~ 0.97 and TRY ~ 0.96.

The new HEPA speed controllers are attached at the middle of the HEPA unit (not at the edge of the unit)... (Attachment 1)
You still need a step./stool to touch the knob and need a ladder for a more precise setting.

We still don't know the optimal speed of the nominal IFO operation. For now, the HEPAs are running at the max speed (Attachment 2).
Once we know the optimal setting, we mark the knobs so that we can see them only with the step.

When adjusting the blower speed, give the blower at least 30 seconds to speed up or slow down to the set speed.  The flywheel effect of the big motor armature and blower mass requires time to follow the control current.  Note the taller Flanders HEPA filters.  These and the new intake filters should keep the PSL air clean for a long time!

I was a bit surprised by these numbers suggesting the PMC transmission is only 50-60%. I went to the table today and confirmed that it is more like 85% (1.3 W in, 1.1 W transmitted, both numbers from with the FieldMate power meter), as I claimed in 2019. Even being conservative with the power meter errors, I think we can be confident T_PMC will be >80% (modulo any thermal effects with higher power degrading the MM). There isn't any reliable record of what the specs of the PMC mirrors are, but assuming the IO couplers have T=4000ppm and the end mirror has T=500ppm as per Alan's plot, this is consistent with a loss of something like 300ppm loss per mirror - seems very high given the small beam spots, but maybe these mirrors just aren't as high quality as the test masses?

It's kind of unfortunate that we will lose ~20% of the amplifier output through the first filter, but I don't see an easy way to clean these mirrors. It's also not clear to me if there is anything to be gained by attempting a cleaning - isn't the inside of the cavity supposed to be completely isolated from the outside? Maybe some epoxy vaporization events degraded the loss?

Koji mentioned that the mode of the laser is different for lower diode currents. So that might be the reason why we got less transmission at the low input power but more afterward.

According to the aLIGO 70W amplifier interlock concept the flow rate of the chiller should be between 5 and 40 l/min. The chillers I found in the TCS lab both have around 15 l/min flow rate so we should be fine in that regard.

Assuming that the power consumption of the diode box is ~800W and that the optical output power of the diode is ~ 300W of optical power, the chillers need to be able to remove the remaining power. At room temperature, they both have enough cooling capacity according to their specs.

As for the idea to put the chiller and diode box in the drill room: There are not a lot of options here. The only viable place is the SW corner (attachment 1). I was told this place is used sometimes for liquid nitrogen dewar. Alternatively, if possible, we can move the fire extinguishers to the SW corner and use the NW corner. In that way, we don't have to clear all the junk from the SW corner, as long as the extinguishers are still accessible.

I made a sketch (attachment 2) showing a possible setup for a diode box + chiller rack. The fiber and network cable can go through the hole in the wall that already exists for the N2. It will have to get bigger though (attachment 3). The rack would also need to host some Acromag unit to convert the communication channel of the chiller/flow meter to Ethernet. The Acromag on 1X7 has no spare channels.

The only power socket in the room, to which the drill is connected, is circuit #36 which is connected to panel L in the lab. The breaker's ampacity seems to be 20A if I'm reading the number on the breaker correctly.

I will upload some plots later - but in summary, I set the HEPA speed to ~40%. I used (i)IMC transmission RIN, (ii) Arm cavity transmission RIN and (iii) ALS beat noise as 3 diagnostics, to see how noise in various frequency bands for these signals change as a function of the HEPA speed. The MC2T RIN shows elevated noise between 1-10Hz at even the lowest speed I tried, ~20% of the max on each blower. The elevated noise extended to ~50-70 Hz for HEPA speeds >40% of the maximum, and the arm cavity RIN and ALS signals also start to become noisy for speeds >60% of the maximum. So I think 40% is a fine speed to run at - for squeezing measurement we may have to turn off the HEPA for 10mins but for the usual single arm / PRMI / DRMI locking, this should be just fine. For the elevated ALS noise - I'm not sure if the coupling is happening over the top of the enclosure where the fiber bringing light from EX comes close to the HEPA filters, or if it is happening inside the PSL enclosure itself, near the beat mouth - but anyways, at the 40% speed, I don't see any effect on the ALS noise.

I checked with a particle counter at the SW corner of the PSL table (which is the furthest away we can be on the table from the HEPA blowers) after leaving the blowers on for ~30mins and it registered 0 for both 0.3um and 0.5um sized particles (if the blowers are off, the respective numbers are 43 and 9 but I forgot what the units were, and I believe they have to be multiplied by 10). 

I have not yet marked the speed control units yet in case there is some other HEPA science that needs to be done before deciding what is the correct setting. But I think I can get the PRFPMI lock without much issue with this lower speed, which is what I will try later today evening.

I believe that there is an internal setting for the minimum flow, so the flow is not linear ("0%" is not zero), but we should mark this flow speed once you find this is sufficiently low for the locking too.

The PSL was too hot, so I turned on the south HEPA on the PSL. The north one was on and the south one was off (or so slow as to be inaudible and no vibration, unlike the north one). Lets watch the trend over the weekend and see if the temperature comes down and if the PMC / WFS variations get less. Fri May 14 17:46:26 2021

The NPRO has been off since ~1AM this morning it looks like. Is this intentional? Can I turn it back on (or at least try to)? The interlock signal we are recording doesn't report getting tripped but I think this has been the case in the past too.

After getting the go ahead from Koji, I turned the NPRO back on, following the usual procedure of diode current ramping. PMC and IMC locked. Let's see if this was a one-off or something chronic.

Looks like the fan lowered the temperature as expected. Need to get a few more days of data to see if its stabilized, or if that's just a fluke.

The vertical line at 00:00 UTC May 18 is about when I turned the fans up/on.

Fluke. Temp fluctuations are as usual, but the overall temperature is still lower. We ought to put some temperature sensors at the X & Y ends to see what's happening there too.

PMC has been unlocked since ~ 2:30 AM. Seems like the PZT got saturated. I moved the DC output adjuster and the PMC locked immidiatly although with a low transmission of 0.62V (>0.7V is the usual case) and high REFL.

IMC locked immidiately but IFO seems to be completely misaligned. The beams on the AS monitor are moving quite alot syncronously. BS watchdog tripped. I enabled the coil outputs. Waiting for the RMS motion to relax...

Its not relaxing. RMS motion is still high. I disabled the coils again and reenabled them. This seems to have worked. Arms were locked quite easily but the ETMs oplevs were way off and the ASS couldn't get the TRX and TRY more than 0.7. I align the ETMs to center the oplev. I realign everything else and lock the arms. Maximium TR is still < 0.8.

{Yehonathan, Anchal}

I went to get a sandwich around 10:20 AM and when I came back BS was moving like crazy. We shutdown the watchdog.

We look at the spectra of the OSEMs (attachment 1). Clearly, the UR sensing is bad.

We took the BS sattelite box out. Anchal opened the box and nothing seemed wrong visually. We returned the box and connected it to the fake OSEM box. The sensor spectra seemed normal.

We connected the box to the vacuum chamber and the spectra is still normal (attachment 2).

We turn on the coils and the motion got damped very quickly (RMS <0.5mV).

Either the problem was solved by disconnecting and connecting the cables or it will come back to haunt us.

[Paco, Yuta]

We have increased the output power from the PSL table to 951 mW (it was 96.7 mW).
IMC was recovered including WFS, and both arms are flashing nicely in IR.
We tweaked the alignment of GRX and GRY injection to align them with IR, but it was hard.
Right now IR beams are not centered on TMs. We should center them first.

What we did:
Power increase and IMC recovery
 - Replaced a beam splitter which splits the beam into IMC REFL RF PD path and WFS path from R=98% to R=10% one. Reflection goes to RF PD.
 - Put a R=98% beam splitter back into WFS path.
 - We also tried to put a window in front of IMC REFL camera to recover the arrangement in 40m wiki, but the beam reflected from the window was too weak for us to align. So, we decided not to place a window in front of the camera.
 - Attached photos are the IMC REFL path before and after the work.
 - Measured the PSL output power as Koji did in elog 40m/16672. It was measured to be 96.7+/- 0.5 mW.
 - Rotated the HWP using the Universal Motion Controller (it was not possible for us to do it from the MEDM screen). The position was changed from 73.99 deg to 36.99 deg. Output power was measured to be 951 +/- 1 mW
 - IMC locked without any other changes.
 - Changed C1:IOO-WFS_TRIGGER_THRESH_ON to 5000 (was 500). IMC WFS also worked.
 - After running MC WFS relief script, WFS DC offsets and RF offsets are adjusted following the steps in elog 40m/16835. Below are the results.

C1:IOO-WFS1_SEG1_DC.AOFF => -0.0008882080010759334
C1:IOO-WFS1_SEG2_DC.AOFF => -0.0006527877490346629
C1:IOO-WFS1_SEG3_DC.AOFF => -0.0005847311617496113
C1:IOO-WFS1_SEG4_DC.AOFF => -0.0010395992663688955
C1:IOO-WFS2_SEG1_DC.AOFF => -0.0025944841559976334
C1:IOO-WFS2_SEG2_DC.AOFF => -0.003191715502180159
C1:IOO-WFS2_SEG3_DC.AOFF => -0.0036688060499727726
C1:IOO-WFS2_SEG4_DC.AOFF => -0.004011172490815322

IOO-WFS1_I1         :  +1977.7 ->    +2250 (Significant change)
IOO-WFS1_I2         :  +3785.8 ->  +3973.2
IOO-WFS1_I3         :  +2014.2 ->  +2277.7 (Significant change)
IOO-WFS1_I4         :  -208.83 ->  +430.96 (Significant change)
IOO-WFS1_Q1         :  +2379.5 ->  +1517.4 (Significant change)
IOO-WFS1_Q2         :  +2260.4 ->  +2172.6
IOO-WFS1_Q3         :  +588.86 ->  +978.98 (Significant change)
IOO-WFS1_Q4         :  +1654.8 ->  +195.38 (Significant change)
IOO-WFS2_I1         :  -1619.9 ->  -534.25 (Significant change)
IOO-WFS2_I2         :  +1610.4 ->  +1619.8
IOO-WFS2_I3         :  +1919.6 ->  +2179.8 (Significant change)
IOO-WFS2_I4         :    +1557 ->  +1426.6
IOO-WFS2_Q1         :   -62.58 ->  +345.56 (Significant change)
IOO-WFS2_Q2         :  +777.01 ->  +805.41
IOO-WFS2_Q3         :  -6183.6 ->  -5365.8 (Significant change)
IOO-WFS2_Q4         :  +4457.2 ->  +4397.

IFO Alignment
 - Aligned both arms using IR. Both arm flashes at the following, which is consistent with the power increase.
 C1:SUS-ETMX_TRX_OUT_DQ ~1.1
 C1:SUS-ETMY_TRY_OUT_DQ ~0.6
 - With this, we tried to tweak GRX and GRY injection. The following is after the work. We could increase GTRX to 0.204 when the Xarm is aligned to green. This suggests that GRX injection is not aligned nicely yet. But the beams are also not centered on TMs. We should center them first.
 C1:ALS-TRX_OUT_DQ ~0.13
 C1:ALS-TRY_OUT_DQ ~0.07
 - GTRX and GTRY cameras are adjusted to have nicer images. In GRX path, the second and last lens before the PD and CCD was pulled ~ 1 cm behind its original position and both beams realigned. Then, on GRY path, the beam was re-centered on the first and only lens, the whole assembly pushed forward by ~ 2 cm and the beams re-centered.

Next:
 - Center the IR beam on TMs (first by our eyeballs; better to use A2L after arm locking is recovered and coils are balanced)
 - Tweak GRX and GRY injection (restore GRY PZTs?)
 - Install ETMXT camera (if it is easy)
 - Lock Xarm and Yarm (C1:LSC-TRX/Y_OUT needs to be fixed for triggering. Can we use other PDs for triggering?)
 - MICH locking (REFL and AS PDs might need to be re-aligned; they are not receiving much light)
 - RTS model for BHD needs to be updated

The commissioners complained about the PMC alignment. The PMC input beam was aligned. It made the transmission improved from 0.72 to 0.74.

FYI: Which steering mirrors do we use for the PMC beam alignment?
The mounts are indicated with the red arrows in Attachment 2.
You have to move these two in common and differential for each pitch and yaw.
The first steering (the right one in the picture) has the beam going through the immediate back of the mount.
So touching the yaw knob of this mount needs some care so that you don't block the PMC refl beam.

[Paco, Tomislav, Yuta]

Somehow, when we were trying to measure WFS open loop transfer functions, PMC unlocked many times for the past two hours and PMC transmission got low.
PMC iput beam was aligned again, and IMC WFS DC offsets and RF offsets were adjusted.
PMC transmission is now C1:PSL-PMC_PMCTRANSPD~0.75, and IMC transmission is C1:IOO-MC_TRANS_SUM~1.4e4.
Actually, IMC transmission once reached 1.5e4 at 06-16-2022 20:01 UTC with PMC transmission of 0.75 (see Attached). There might be a better alignment.

For the PSL HEPA, we wanted it to remain at full speed during the vent, when anyone is working on the PSL, or when there is a lot of dust due to outside conditions or cleaning in the lab.

For NORMAL conditions, the policy is to turn it to 30% for some flow, but low noise.

I think we ought to lock one of the arms on IR PDH and change the HEPA flow settings and plot the arm error signal, and transmitted power for each flow speed to see what's important. Record the times of each setting so that we can make a specgram later

MCAutolocker/FSSSlow are not properly documented and not properly working.
Tidy up the script and documentation, or bring it back to megatron

I was aware that the FSSSlow was misbehaving since the shutdown upon the July power outage.
- FSS Slow servo did nothing even though the apparent settings in C1:PSL_SLOW screen looked fine and heart beat blinking
- Wanted to restart FSSSlow at megatron. Despite the login message showing how to do it, the system service does not exist anymore, because it was moved somewhere.
- Searched 40m wiki but found no info how to kill and restart it
- Found an elog. It was moved to docker on optimus ELOG 16480 . The restart procedure can only be found here. Please fix all the documentation inconsistency >> Anchal

According to this elog, the following commands need to be run for starting up MCAutolocker and FSSSlow on optimus:

cd /opt/rtcds/caltech/c1/Git/40m/scripts/MC
sudo docker-compose up -d

- Problem continues. Now FSSSlow is running but only when the IMC is locked. It does not stop even when the IMC lock is lost. How can we debug docker thing?
- This is minor but the MCAutolocker log (/opt/rtcds/caltech/c1/scripts/MC/logs/AutoLocker.log) is no longer updated even though MCAutolocker is running. Was it moved somewhere?

The FSSSlow script was not properly documented and it was not working, so I had to use one that I knew worked from CTN. This scripts lives in

/opt/rtcds/caltech/c1/Git/40m/scripts/PSL/FSS/PIDLocker.py

and uses a configuration file

/opt/rtcds/caltech/c1/Git/40m/scripts/PSL/FSS/PIDConfigFSS.yml

The script runs all the time inside docker container which keeps it running. The heart-beat blinker shows if the script is active or not, but it only starts working when C1:IOO-MC_LOCK is 1 and C1:PSL-FSS_SLOWLOOP is 1. The second channel is a button on C1PSL_SLOW screen to engage autolocker. It has to be turned on for FSS to work.

docker instructions:

The following message is displayed on login in optimus:

For checking log files of autolocker script, on optimus do:

For checking log files of FSS PID loop, on optimus do:

In the above commands, add < | tail -15> to just see the most recent 15 lines in the log file. change 15 to whatever number of lines you want to see from the end.

At any time, if you want to know how docker is running stuff, check out the /opt/rtcds/caltech/c1/Git/40m/scriptsdocker-compose.yml file for self-explanatory script usage.

I'll add some documentation on the wiki soon. That is indeed required and should have been done already.

Debugging scripts:

All scripts could be debugged by running them on rossa by directly using python command. You can stop the docker container on optimus using:

and then run the file on rossa to check it's behavior. After debugging and fixing any issues, please commit the file to gitlab repo and go back to optimus and restart the docker container:

I'll add this procedure to a wiki page as well.

Reverting back to systemd on megatron

The setup on megatron was not removed at all. All service files exist in same place and old scripts can be started in the old manner by doing following on megatron:

For FSSSlow:

For MC autolocker:

For diabling these services again, do:

Note that one should stop docker containers on optimus before starting these systemd services to avoid conflicting scripts running together.

I have added above instructions on megatron motd. So on loging into megatron, these updated instructions would come.

If someone wants to fix the old scripts and use systemd for managing those scripts, please do so but I won't be able to help in debugging those old scripts. The shell scripts are very complicated and beyond my knowledge and python scripts are lacking documentation.

I'm happy to help debug or extend the functionality of the new scripts that live in the git directory.

> it only starts working when C1:IOO-MC_LOCK is 1 and C1:PSL-FSS_SLOWLOOP is 1.

- OK. Your new MCAutolocker does not reflect the lock status to C1:IOO-MC_LOCK. This causes FSS Slow to go crazy when the IMC is not locked. Can you fix that?

- So C1PSL_SLOW.adl screen, which spawns by the "SLOW Servo" button on the FSS screen, has no effect on the FSS SLOW servo anymore. It is obsolete. At least the screen (or the link to the screen) should be removed. (Work on it once you are back.)

- Also, please make a wiki page and copy the description on the previous page.

Added C1:IOO-MC_LOCK to ALConfigMC.yml which solved the isse with FSS Slow. We should tune the FSS Slow Servo PID coefficients for a better performance.

the C1PSL_SLOW.adl screen is not obsolete. It can be used to change the PID coefficients, engage/disengage the PID loop, monitor the PID script blinker, and monitor FAST actuator value C1:PSL-FSS_FAST. the functionality of this screen has not changed from before.

I've also added a wiki page for scripts documentation.

can't we just go back to the old python script that was working for many years, and tested? I imagine that as soon as someone besides you tries to debug the docker setup, this is what will happen.

I measured the sideband frequencies for PMC and IMC lock (to use it for Mariner PMC and IMC design).

PMC: 35.498912(2) MHz
IMC: 29.485038(2) MHz

Details:
 - Mini-Circuits UFC-6000 was used. The spec sheet says the frequency accuracy in 1-40 MHz is +/- 2 Hz.
 - "29.5 MHz OUT" port of 40m Frequency Generation Unit (LIGO-T1000461) was connected to UFC-6000 to measure IMC sideband frequency.
 - "LO TO SERVO" port of Crystal Frequency Ref (LIGO-D980353) was connected to UFC-6000 to measure PMC sideband frequency.
 - It seems like IMC sideband frequency changed from 29.485 MHz to 29.491 MHz back in 2011 (40m/4621). We are back to 29.485 MHz. I'm not sure what happened after this.

[Yuta, JC, Radhika]

PMC input beam was aligned again, bringing transmission from 0.70 to ~0.75. To avoid blocking the PMC refl beam, I found success handling the yaw knob of the first steering mirror from below.

[JC, Paco]

I attempted to align PMC Beam from a transmission of 0.72. I failed to do so on my own, but Paco arrived and helped me out. Transmission has gone from from 0.72 to ~0.73.

[Paco, Anchal, JC]

C1:PSL-PMC_PMCTRANSPD ~ 0.715 this morning, this was increased to ~0.730. There also seems to be an earthquake going on and the MC is flashing.

C1:PSL-PMC_PMCTRANSPD was increased from 0.72 to 0.731

[JC, Paco, Yuta]

It seems like PMC transmission (C1:PSL-PMC_PMCTRANSPD) dropped to ~0.68 from ~0.74 on Dec 27.
We tried to tweak PZT offset for PMC loop and input alignment to PMC, but PMC transmission didn't increased.
PSL laser temperature was also sweeped in the range 30.3 - 31.6 degC, but didn't help. The PSL temperature was reverted to original 30.61(1) degC.
Power measured at PSL output now is 893 mW (measured at our standard place shown in 40m/16672), which used to be 951 mW in June 2022 (40m/16886).
Power measured at PMC input (see attached photo) now is 1.18 W.

Next:
 - What was the previous PMC input power we had?
 - Sweep PSL laser temperature for larger range

PMC aligned (Attachment #1).
Over the past month, PMC transmission is actually slowly growing from 0.68 to 0.70 (Attachment #2), since it suddenly dropped from 0.72 on Dec 27 (40m/17390).

PMC hit a drop around mid-day on Saturday and has been going down since. As of now,

C1:PSL-PMC_PMCTRANSPD - 0.664

I will go in and adjust now.

This is super! And now is the time to replace the internal fan!

We removed the PSL controller internal (broken) fans after it tripped due to overheat.

Background

[Mayank, Radhika]

While aligning Xarm I noticed sudden loss of beam. On Radhikas suggestion we cheked the PSL and found out that the PSL controller was in off state (No lights on front and back panel). We restored the situation by unplugging and replugging the power cord. The PSL worked fine for a few minutes (~ 30 ) and then tripped again.This time the front panel  OFF light was on . See attached image (Attachment #1).

Fix

[Paco, Mayank, Koji-remote]

We disconnected the PSL controller and took this opportunity to investigate the controller's internal cooling mechanism. After disassembling the top panel of the chassis, we saw there are two SUNON - KD1205PHB2 fans meant to run at 12 VDC (1.7 W) connected to the bottom pcb inside the controller. After disconnecting them from this board, we tested them with an externally supplied dc voltage and confirmed they no longer worked. We noted the cooling mechanism is based on a long aluminum heat sink to which most ICs are attached, and the fans are meant to provide airflow towards the rear aperture on the chassis. We followed Koji's suggestion and for now, removed the damaged components (detailed pictures of this operation have been posted in a google photos album elsewhere) to allow heat to flow out more easily.  We reassembled the controller chassis and reinstalled it with the external fan providing the necessary airflow to prevent the unit from tripping again due to overheating. Then we turned on PSL and recovered PMC and IMC locks.

Aftermath

We took a C1:IOO-MC_F_DQ trace after this work to confirm our earlier findings; the trace is attached in Attachment #2. The noise bumps are present as expected. This is still not a desirable configuration so next step would be replacing the external fan, or even better, find the appropriate spare of the internal units and berid the external one.