40m QIL Cryo_Lab CTN SUS_Lab TCS_Lab OMC_Lab CRIME_Lab FEA ENG_Labs OptContFac Mariner WBEEShop
  40m Log  Not logged in ELOG logo
Entry  Tue Feb 2 23:29:35 2021, Koji, Update, CDS, CDS crash and CDS/IFO recovery 
    Reply  Wed Feb 3 15:24:52 2021, gautam, Update, CDS, CDS crash and CDS/IFO recovery 
       Reply  Wed Feb 3 18:53:31 2021, Koji, Update, CDS, CDS crash and CDS/IFO recovery 
          Reply  Wed Feb 3 21:28:02 2021, gautam, Update, CDS, CDS crash and CDS/IFO recovery 
Message ID: 15792     Entry time: Wed Feb 3 15:24:52 2021     In reply to: 15791     Reply to this: 15794
Author: gautam 
Type: Update 
Category: CDS 
Subject: CDS crash and CDS/IFO recovery 

Didn't get a chance to comment during the meeting - This was almost certainly a coincidence. I have never had to do this - I assert, based on the ~10 labwide reboots I have had to do in the last two years, that whether the timing errors persist on reboot or not is not deterministic. But this is beyond my level of CDS knowledge and so I'm happy for Rolf / Jamie to comment. I use the reboot script - if that doesn't work, I use it again until the systems come back without any errors.

Quote:

This looked like the usual timing issue. It looked like "ntpdate" is not available in the new system. (When was it updated?)

The hardware clock (RTC) of these hosts are set to be PST while the functional end host showed UTC. So I copied the time of the UTC time from the end to the vertex machines.
For the time adjustment, the standard "date" command was used

> sudo date -s "2021-02-03 07:11:30"

This made the trick. Once IOP was restarted, the "DC" indicators returned to **Green**, restarting the other processes were straight forward and now the CDS indicators are all green.

I don't think this is a problem, the NTP synchronization is handled by timesyncd now.

Quote:

NTP synchronization is not active. Is this OK?

I defer restoring the LSC settings etc since I guess there is not expected to be any interferometer activity for a while.

ELOG V3.1.3-