40m QIL Cryo_Lab CTN SUS_Lab TCS_Lab OMC_Lab CRIME_Lab FEA ENG_Labs OptContFac Mariner WBEEShop
  40m Log  Not logged in ELOG logo
Entry  Tue Feb 2 23:29:35 2021, Koji, Update, CDS, CDS crash and CDS/IFO recovery 
    Reply  Wed Feb 3 15:24:52 2021, gautam, Update, CDS, CDS crash and CDS/IFO recovery 
       Reply  Wed Feb 3 18:53:31 2021, Koji, Update, CDS, CDS crash and CDS/IFO recovery 
          Reply  Wed Feb 3 21:28:02 2021, gautam, Update, CDS, CDS crash and CDS/IFO recovery 
Message ID: 15791     Entry time: Tue Feb 2 23:29:35 2021     Reply to this: 15792
Author: Koji 
Type: Update 
Category: CDS 
Subject: CDS crash and CDS/IFO recovery 

I worked around the racks and the feedthru flanges this afternoon and evening. This inevitably crashed c1lsc real-time process.
Rebooting c1lsc caused multiple crashes (as usual) and I had to hard reboot c1lsc/c1sus/c1ioo
This made the "DC" indicator of the IOPs for these hosts **RED**.

This looked like the usual timing issue. It looked like "ntpdate" is not available in the new system. (When was it updated?)

The hardware clock (RTC) of these hosts are set to be PST while the functional end host showed UTC. So I copied the time of the UTC time from the end to the vertex machines.
For the time adjustment, the standard "date" command was used

> sudo date -s "2021-02-03 07:11:30"

This made the trick. Once IOP was restarted, the "DC" indicators returned to **Green**, restarting the other processes were straight forward and now the CDS indicators are all green.

controls@c1iscex:~ 0$ timedatectl
      Local time: Wed 2021-02-03 07:35:12 UTC
  Universal time: Wed 2021-02-03 07:35:12 UTC
        RTC time: Wed 2021-02-03 07:35:26
       Time zone: Etc/UTC (UTC, +0000)
     NTP enabled: yes
NTP synchronized: no
 RTC in local TZ: no
      DST active: n/a

NTP synchronization is not active. Is this OK?


With the recovered CDS, the IMC was immediately locked and the autolocker started to function after a few pokes (like manually running of the "mcup" script). However, I didn't see any light on the AS/REF cameras as well as the test mass faces. I'm sure the IMC alignment is OK. This means the TTs are not well aligned.

So, burtrestored c1assepics with 12:19 snapshot. This immediately brought the spots on the REFL/AS.

Then the arm were aligned, locked, and ASSed. I tried to lock the FP arms. The transmissions were at the level of 0.1~0.3. So some manual alignment of ITMY and BS were necessary. After having the TRs of ~0.8, I still could not lock the arms. The signs of the servo gains were flipped to -0.143 for X arm and -0.012 for Y arm, and the arms were locked. ASS worked well and the ASS offsets were offloaded to the SUSs.

 

ELOG V3.1.3-