40m QIL Cryo_Lab CTN SUS_Lab TCS_Lab OMC_Lab CRIME_Lab FEA ENG_Labs OptContFac Mariner WBEEShop
  40m Log  Not logged in ELOG logo
Entry  Wed Jan 22 18:17:46 2014, Jenne, Update, CDS, fb timing was off 
    Reply  Thu Jan 30 11:59:03 2014, manasa, Update, CDS, fb timing was off 
    Reply  Wed Feb 26 23:14:07 2014, Jenne, Update, CDS, fb timing was off 
    Reply  Mon Mar 3 10:42:53 2014, Jenne, Update, CDS, fb timing was off 
       Reply  Mon Mar 3 11:55:39 2014, Koji, Update, CDS, fb timing was off 
       Reply  Mon Mar 10 11:42:36 2014, Jenne, Update, CDS, fb timing was off 
          Reply  Mon Mar 17 12:31:58 2014, manasa, Update, CDS, fb timing was off 
Message ID: 9567     Entry time: Wed Jan 22 18:17:46 2014     Reply to this: 9587   9679   9683
Author: Jenne 
Type: Update 
Category: CDS 
Subject: fb timing was off 

Since this morning, the fb's timing has been off.  Steve pointed it out to me earlier today, but I didn't have a chance to look at it until now. 

This was different from the more common problem of the mx stream needing to be restarted - that causes 3 red blocks per core, on all cores on a computer, but it doesn't have to be every computer.  This was only one red block per core in the CDS FE status screen, but it was on every core on every computer. 

The error message, when you click into the details of a single core, was 0x4000.  I elog searched for that, and found elog 6920, which says that this is a timing issue with the frame builder.  Since Jamie had already set things on nodus' config correctly, all I did was reconnect the fb to the ntp: 

fb$ sudo /etc/init.d/ntp-client restart

As in elog 6920, the daqd stopped, then restarted itself, and cleared the error message. It looks like everything is good again.

I suspect (without proof) that this may have to do with the campus network being down this morning, so the computers couldn't sync up with the outside world.

ELOG V3.1.3-