40m QIL Cryo_Lab CTN SUS_Lab TCS_Lab OMC_Lab CRIME_Lab FEA ENG_Labs OptContFac Mariner WBEEShop
  40m Log  Not logged in ELOG logo
Entry  Mon Jan 13 23:30:53 2020, Yehonathan, Summary, PSL, Changes made since Gautam left 
    Reply  Tue Jan 14 20:17:09 2020, gautam, Summary, General, IFO recovery DCerrors.png
       Reply  Wed Jan 15 10:04:19 2020, gautam, Summary, General, POX / POY locking restored singleArms.png
Message ID: 15121     Entry time: Tue Jan 14 20:17:09 2020     In reply to: 15119     Reply to this: 15123
Author: gautam 
Type: Summary 
Category: General 
Subject: IFO recovery 

Summary:

There was no light entering the IFO. I worked on a few things to bring the interferometer to a somewhat usable state. The goal is to get back to PRFPMI locking ASAP.

Details:

Problem: All fast models report a "0x4000" DC error. See Attachment #1.

Solution: I think this is a "known" issue that happened last new year too. The fix was to add a hard-coded 1 second offset to the daqd config files. However, incrementing/decreasing this offset by +/- 1 second did not fix the errors for me today. I'll reach out to JH for more troubleshooting tips.

Update 15 Jan 2020 830am: The problem is now fixed. See here.

Problem: c1susaux and c1auxey were unresponsive.

Solution: Keyed c1auxey. Rebooted c1susaux and as usual, manually started the eth0/eth1 subnets. The Acromag crate did not have to be power-cycled. ITMY got stuck in this process - I released it using the usual bias jiggling. Why did c1susaux fail? When did it fail? Was there some un-elogged cable jiggling in that part of the lab?

Problem: IMC autolocker and FSS slow processes aren't running on megatron after the upgrade.

Solution: Since no one bothered to do this, I setup systemd infrastructure for doing this on megatron. To run these, you do:

sudo systemctl start MCautolocker.service
sudo systemctl start FSSSlow.service

and to check their status, use:

sudo systemctl status MCautolocker.service
sudo systemctl status FSSSlow.service

The systemd setup is currently done in a naive way (using the bash executable to run a series of commands rather than using the systemd infrastructure itself to setup variables etc) but it works. I confirmed that the autolocker can re-acquire IMC lock, and that the FSS loop only runs when the IMC is locked. I also removed the obsolete messages printed to megatron's console (by editing /etc/motd) on ssh-login, advising the usage of initctl - the updated message reflects the above instructions.

In order to do the IMC locking, I changed the DC voltage to the AOM to +1V DC (it was +0.8 V DC). In this setting, the IMC refl level is ~3.6 V DC. When using the undiffracted AOM beam, we had more like +5.6 V DC (so now we have ~65% of the nominal level) from the IMC REFL PD when the IMC was unlocked. IIRC, the diffraction efficiency of the AOM should be somewhat better, at ~85%. Needs investigation, or better yet, let's just go back to the old configuration of using the undiffracted beam.

There was also an UN-ELOGGED angry change of the nominal value of the PMC servo gain to 12.8, and no transfer function measurement. There needs to be a proper characterization of this loop done to decide what the new nominal value should be.

I'm going to leave the PSL shutter open and let the IMC stay locked for stability investigations. Tomorrow, I'll check the single-arm locking and the ALS system.

Attachment 1: DCerrors.png  23 kB  | Hide | Hide all
DCerrors.png
ELOG V3.1.3-