40m QIL Cryo_Lab CTN SUS_Lab TCS_Lab OMC_Lab CRIME_Lab FEA ENG_Labs OptContFac Mariner WBEEShop
  40m Log  Not logged in ELOG logo
Entry  Mon Aug 12 21:27:03 2013, manasa, Update, CDS, c1iscex needs help 
    Reply  Tue Aug 13 07:40:53 2013, Steve, Update, CDS, c1iscex needs help c1iscexSick.png
       Reply  Tue Aug 13 17:20:54 2013, Koji, Update, CDS, [Fixed] c1iscex needs help 
          Reply  Mon Nov 25 10:57:14 2013, Koji, Update, CDS, woes on the X-end hosts Screenshot.png
             Reply  Mon Nov 25 12:57:54 2013, Jamie, Update, CDS, timing problem at c1iscex IO chassis 
                Reply  Mon Nov 25 17:28:33 2013, Jenne, Update, CDS, timing problem at c1iscex IO chassis 
                Reply  Wed Nov 27 14:45:49 2013, Jenne, Update, CDS, timing problem at c1iscex IO chassis 
                   Reply  Wed Nov 27 16:29:21 2013, Jenne, Update, CDS, Accidentally turned off SUS IO chassis 
                Reply  Mon Dec 2 16:04:47 2013, Jamie, Update, CDS, c1iscex timing problem mysteriously disappears??? (thanksgiving miracle???) festatus.png
                   Reply  Mon Dec 2 17:05:13 2013, Jenne, Update, CDS, c1iscex timing problem mysteriously disappears??? (thanksgiving miracle???) 
                      Reply  Tue Dec 3 07:42:23 2013, Steve, Update, CDS, c1iscex timing problem mysteriously disappears??? (thanksgiving miracle???) 
             Reply  Wed Dec 4 13:37:34 2013, Jenne, Update, CDS, c1auxex down again 
Message ID: 9433     Entry time: Mon Dec 2 16:04:47 2013     In reply to: 9426     Reply to this: 9434
Author: Jamie 
Type: Update 
Category: CDS 
Subject: c1iscex timing problem mysteriously disappears??? (thanksgiving miracle???) 

Quote:

There is definitely a timing distribution malfunction at the c1iscex IO chassis.  There is no timing link between the "Master Timer Sequencer D050239" at the 1X6 and the c1iscex IO chassis.  Link lights at both ends are dead.  No timing, no running models.

It does not appear to be a problem with the Master Timer Sequencer.  I moved the c1iscey link to the J15 port on the sequencer and it worked fine.  This means its either a problem with the fiber or the timing card in the IO chassis.  The IO timing card is powered and does have what appear to be normal status lights on (except for the fiber link lights).  It's getting what I think is the nominal 4V power.  The connection to the IO chassis backplane board look ok.  So maybe it's just a dead fiber issue?

I do not know what could have been the problem with c1auxex, or if it's related to the fast timing issue.

I just got over here from Downs, where I managed to convince Todd to let me borrow one of their three remaining timing slave boards for c1iscex.  I walked down to the X end to replace the board only to discover that the link light on the existing timing board was back!  c1iscex was not responding, so I hard rebooted the machine, and everything came up rosy (all green!):

festatus.png

To repeat, I DID NOTHING.  The thing was working when I got here.  I have no idea when it came back, or how, but it's at least working for the moment.  I re-enabled the watchdog for ETMX SUS and it's now damped normally.

I'm going to hold on to the timing card for a couple of days, in case the failure comes back, but we'll need to return it to Downs soon, and probably think about getting some spare backups from Columbia.

ELOG V3.1.3-