40m QIL Cryo_Lab CTN SUS_Lab TCS_Lab OMC_Lab CRIME_Lab FEA ENG_Labs OptContFac Mariner WBEEShop
  40m Log  Not logged in ELOG logo
Entry  Mon Feb 24 13:40:13 2014, Jenne, Update, CDS, Computer weirdness with c1lsc machine Screenshot-Untitled_Window-1.png
    Reply  Mon Feb 24 15:25:29 2014, Jenne, Update, CDS, Computer weirdness with c1lsc machine 
       Reply  Mon Feb 24 16:26:14 2014, Jenne, Update, CDS, NTP fell out of sync on front end machines - fixed 
Message ID: 9662     Entry time: Mon Feb 24 13:40:13 2014     Reply to this: 9663
Author: Jenne 
Type: Update 
Category: CDS 
Subject: Computer weirdness with c1lsc machine 

I noticed that the fb lights on all of the models on the c1lsc machine are red, and that even though the MC was locked, there was no light flashing in the IFO. Also, all of the EPICS values on the LSC screen were frozen.

Screenshot-Untitled_Window-1.png

I tried restarting the ntp server on the frame builder, as in elog 9567, but that didn't fix things.  (I realized later that the symptom there was a red light on every machine, while I'm just seeing problems with c1lsc. 

I did an mxstream restart, as a harmless thing that had some small hope of helping (it didn't). 

I logged on to c1lsc, and restarted all of the models (rtcds restart all), which stops all of the models (IOP last), and then restarts them (IOP first).  This did not change the status of the lights on the status screen, but it did change the positioning of some optics (I suspect the tip tilts) significantly, and I was again seeing flashes in the arms.  The LSC master enable switch was off, so I don't think that it was trying to send any signals out to the suspensions.  The ASS model, which sends signals out to the input pointing tip tilts runs on c1lsc, and it was about when the ass model was restarted that the beam came back.  Also, there are no jumps in any of the SOS OSEM sensors in the last few hours, except me misaligning and restoring the optics.  I we don't have sensors on the tip tilts, so I can't show a jump in their positioning, but I suspect them.

I called Jamie, and he suggested restarting the machine, which I did.  (Once again, the beam went somewhere, and I saw it scattering big-time off of something in the BS chamber, as viewed on the PRM-face camera).  This made the oaf and cal models run (I think they were running before I did the restart all, but they didn't come back after that.  Now, they're running again).  Anyhow, that did not fix the problem.  For kicks, I re-ran mxstream restart, and diag reset, to no avail.  I also tried running the sudo /etc/init.d/ntp-client restart command on just the lsc machine, but it doesn't know the command 'ntp-client'. 

Jamie suggested looking at the timing card in the chassis, to ensure all of the link lights are on, etc.  I will do this next.

ELOG V3.1.3-