40m QIL Cryo_Lab CTN SUS_Lab TCS_Lab OMC_Lab CRIME_Lab FEA ENG_Labs OptContFac Mariner WBEEShop
  40m Log  Not logged in ELOG logo
Message ID: 13643     Entry time: Tue Feb 20 21:14:59 2018
Author: gautam 
Type: Update 
Category: CDS 
Subject: RFM network errors 

I wanted to lock the single arm POX/POY config to do some tests on the BeatMouth. But I was unable to.

  • I tracked the problem down to the fact that the TRX and TRY triggers weren't getting piped correctly to the LSC model
  • In fact, all RFM channels from the end machines were showing error rates of 16384/sec (i.e. every sample).
  • After watchdogging ETMX, I tried restarting just the c1scx model - this promptly took down the whole c1iscex machine.
  • Then I tried the same with c1iscey - this time the models restarted successfully without the c1iscey machine crashing, but the RFM errors persisted for the c1scy channels.
  • I walked down to EX and hard rebooted c1iscex.
  • c1iscex came back online, and I ssh-ed in and did rtcds start --all.
  • This brought all the models back online, and the RFM errors on both c1iscex and c1iscey channels vanished.

Not sure what to make of all this, but I can lock the arms now.

ELOG V3.1.3-