40m QIL Cryo_Lab CTN SUS_Lab TCS_Lab OMC_Lab CRIME_Lab FEA ENG_Labs OptContFac Mariner WBEEShop
  40m Log  Not logged in ELOG logo
 List  |  Find  |  Login  |  Help 
Entry  Sat Jan 19 15:05:37 2013, Jenne, Update, Computers, All front ends but c1lsc are down upside_down_cat-t2.jpg
    Reply  Sat Jan 19 18:23:31 2013, rana, Update, Computers, All front ends but c1lsc are down FE.png
       Reply  Wed Jan 30 13:50:27 2013, Jenne, Update, Computers, c1iscex still down 
          Reply  Thu Jan 31 10:23:39 2013, Jamie, Update, Computers, c1iscex still down 
             Reply  Fri Feb 15 15:21:07 2013, Jamie, Update, Computers, c1iscex IO-chassis dead 
                Reply  Tue Feb 19 15:10:02 2013, Jamie, Update, CDS, c1iscex alive again 
                   Reply  Thu Feb 21 12:56:38 2013, Jenne, Update, CDS, c1iscex dead again 
                      Reply  Thu Feb 21 14:32:02 2013, Jamie, Update, CDS, c1iscex models restarted 
Message ID: 8128     Entry time: Thu Feb 21 14:32:02 2013     In reply to: 8126
Author: Jamie 
Type: Update 
Category: CDS 
Subject: c1iscex models restarted 

Quote:

c1iscex is dead again.  Red lights, no "breathing" on the FE status screen.

The c1iscex machine itself wasn't dead, the models were just not running.  Here are the last messages in dmesg:

[130432.926002] c1spx: ADC TIMEOUT 0 7060 20 7124
[130432.926002] c1scx: ADC TIMEOUT 0 7060 20 7124
[130433.941008] c1x01: timeout 0 1000000 
[130433.941008] c1x01: exiting from fe_code()

I'm guessing maybe the timing signal was lost, so the ADC stopped clocking.   Since the ADC clock is the everything clock, all the "fe" code (ie. models) aborted. Not sure what would have caused it.

I restarted all the models ("rtcds restart all") and everything came up fine. Obviously we should keep our eyes on things, and note if anything strange was happening if this happens again.

ELOG V3.1.3-