40m QIL Cryo_Lab CTN SUS_Lab TCS_Lab OMC_Lab CRIME_Lab FEA ENG_Labs OptContFac Mariner WBEEShop
  40m Log  Not logged in ELOG logo
Entry  Fri Jun 1 02:33:40 2012, Jenne, Update, Computers, c1sus and c1iscex - bad fb connections 
    Reply  Fri Jun 1 08:01:46 2012, steve, Update, Computers, c1sus and c1iscex are down compdown.png
    Reply  Fri Jun 1 09:50:50 2012, Jamie, Update, Computers, c1sus and c1iscex - bad fb connections 
       Reply  Fri Jun 1 14:40:24 2012, Jamie, Update, Computers, c1sus and c1iscex - bad fb connections 
          Reply  Mon Jul 7 13:44:21 2014, Jenne, Update, CDS, c1sus - bad fb connection 
Message ID: 10135     Entry time: Mon Jul 7 13:44:21 2014     In reply to: 6742
Author: Jenne 
Type: Update 
Category: CDS 
Subject: c1sus - bad fb connection 

Quote:

 

I managed to recover c1sus.  It required stopping all the models, and the restarting them one-by-one:

$ rtcds stop all     # <-- this does the right to stop all the models with the IOP stopped last, so they will all unload properly.

$ rtcds start iop

$ rtcds start c1sus c1mcs c1rfm

I have no idea why the c1sus models got wedged, or why restarting them in this way fixed the issue.

 In addition to needing obnoxiously regular mxstream restarts, this afternoon the sus machine was doing something slightly differently.  Only 1 fb block per core was red (the mxstream symptom is 3 fb-related blocks are red per core), and restarting the mxstream didn't help.  Anyhow, I was searching through the elog, and this entry to which I'm replying had similar symptoms.  However, by the time I went back to the CDS FE screen, c1sus had regular mxstream symptoms, and an mxstream restart fixed things right up. 

So, I don't know what the issue is or was, nor do I know why it is fixed, but it's fine for now, but I wanted to make a note for the future.

ELOG V3.1.3-