40m QIL Cryo_Lab CTN SUS_Lab TCS_Lab OMC_Lab CRIME_Lab FEA ENG_Labs OptContFac Mariner WBEEShop
  40m Log  Not logged in ELOG logo
Entry  Thu Mar 11 14:27:56 2021, gautam, Update, CDS, timesync issue? timesync.pngc1lscMods.png
    Reply  Thu Mar 11 18:46:06 2021, gautam, Update, CDS, cds reboot CDSoverview.png
       Reply  Fri Mar 12 03:28:51 2021, Koji, Update, CDS, cds reboot 
          Reply  Fri Mar 12 11:02:38 2021, gautam, Update, CDS, cds reboot timeDrift.png
Message ID: 15911     Entry time: Fri Mar 12 11:02:38 2021     In reply to: 15910
Author: gautam 
Type: Update 
Category: CDS 
Subject: cds reboot 

I looked into this a bit today morning. I forgot exactly what time we restarted the machines, but looking at the timesyncd logs, it appears that the NTP synchronization is in fact working (log below is for c1sus, similar on other FEs):

-- Logs begin at Fri 2021-03-12 02:01:34 UTC, end at Fri 2021-03-12 19:01:55 UTC. --
Mar 12 02:01:36 c1sus systemd[1]: Starting Network Time Synchronization...
Mar 12 02:01:37 c1sus systemd-timesyncd[243]: Using NTP server (ntpserver).
Mar 12 02:01:37 c1sus systemd[1]: Started Network Time Synchronization.
Mar 12 02:02:09 c1sus systemd-timesyncd[243]: Using NTP server (ntpserver).

So, the service is doing what it is supposed to (using the FB,, as the ntpserver). You can see that the timesync was done a couple of seconds after the machine booted up (validated against "uptime"). Then, the service is periodically correcting drifts. idk what it means that the time wasn't in sync when we check the time using timedatectl or similar. Anyway, like I said, I have successfully rebooted all the FEs without having to do this weird time adjustment >10 times

I guess what I am saying is, I don't know what action is necessary for "implementing NTP synchronization properly", since the diagnostic logfile seems to indicate that it is doing what it is supposed to.

More worryingly, the time has already drifted in < 24 hours.


I want to emphasize the followings:

  • FB's RTC (real-time clock/motherboard clock) is synchronized to NTP time.
  • The other RT machines are not synchronized to NTP time.
  • My speculation is that the RT machine timestamp is produced from the local RT machine time because there is no IRIG-B signal provided. -> If the RTC is more than 1 sec off from the FB time, the timestamp inconsistency occurs. Then it induces "DC" error.
  • For now, we have to use "date" command to match the local RTC time to the FB's time.
  • So: If NTP synchronization is properly implemented for the RT machines, we will be free from this silly manual time adjustment.
Attachment 1: timeDrift.png  505 kB  | Hide | Hide all
ELOG V3.1.3-