40m QIL Cryo_Lab CTN SUS_Lab TCS_Lab OMC_Lab CRIME_Lab FEA ENG_Labs OptContFac Mariner WBEEShop
  40m Log  Not logged in ELOG logo
Entry  Fri Jun 2 12:32:16 2017, gautam, Update, General, Power glitch 
    Reply  Fri Jun 2 16:02:34 2017, gautam, Update, General, Power glitch IMG_7399.JPG
       Reply  Fri Jun 2 22:01:52 2017, gautam, Update, General, Power glitch - recovery power_glitch_recovery.pngIMG_7406.JPGIMG_7407.JPGIMG_7400.JPG
          Reply  Sun Jun 4 15:59:50 2017, gautam, Update, General, Power glitch - recovery powerGlitchRecovery.png
Message ID: 13038     Entry time: Sun Jun 4 15:59:50 2017     In reply to: 13036
Author: gautam 
Type: Update 
Category: General 
Subject: Power glitch - recovery 

I think the CDS status is back to normal.

  • Bit 2 of the C1RFM status word was red, indicating something was wrong with "GE FANUC RFM Card 0".
  • You would think the RFM errors occur in pairs, in C1RFM and in some other model - but in this case, the only red light was on c1rfm.
  • While trying to re-align the IFO, I noticed that the TRY time series flatlined at 0 even though I could see flashes on the TRANSMON camera.
  • Quick trip to the Y-End with an oscilloscope confirmed that there was nothing wrong with the PD.
  • I crawled through some elogs, but didn't really find any instructions on how to fix this problem - the couple of references I did find to similar problems reported red indicator lights occurring in pairs on two or more models, and the problem was then fixed by restarting said models.
  • So on a hunch, I restarted all models on c1iscey (no hard or soft reboot of the FE was required)
  • This fixed the problem
  • I also had to start the monit process manually on some of the FEs like c1sus. 

Now IFO work like fixing ASS can continue...

Attachment 1: powerGlitchRecovery.png  23 kB  | Hide | Hide all
powerGlitchRecovery.png
ELOG V3.1.3-