40m QIL Cryo_Lab CTN SUS_Lab TCS_Lab OMC_Lab CRIME_Lab FEA ENG_Labs OptContFac Mariner WBEEShop
  40m Log  Not logged in ELOG logo
Entry  Tue Oct 17 23:07:52 2017, gautam, Update, CDS, FEs unresponsive 
    Reply  Wed Oct 18 01:41:32 2017, jamie, Update, CDS, FEs unresponsive 
       Reply  Wed Oct 18 02:09:32 2017, gautam, Update, CDS, FEs unresponsive 
          Reply  Wed Oct 18 09:21:22 2017, jamie, Update, CDS, FEs unresponsive 
             Reply  Wed Oct 18 23:11:53 2017, gautam, Update, CDS, FEs unresponsive 
Message ID: 13387     Entry time: Wed Oct 18 02:09:32 2017     In reply to: 13386     Reply to this: 13388
Author: gautam 
Type: Update 
Category: CDS 
Subject: FEs unresponsive 

I was looking at the ASDC channel on dataviewer, and toggling various settings like whitening gain. At some point, the signal just froze. So I quit dataviewer and tried restarting it, at which point it complained about not being able to connect to FB. This is when I brought up the CDS_OVERVIEW medm screen, and noticed the frozen 1pps indicator lights. There was certainly something going on with the end FEs, because I was able to ping the machine, but not ssh into it. Once the 1pps lights came back, I was able to ssh into c1iscex and c1iscey, no problems.

Could it be that some of the mx processes stalled, but the systemctl routine automatically restarted them after some time? 

Quote:

So this wasn't just an EPICS freeze?  I don't see how this had anything to do with any of the work I did earlier today.  I didn't modify any of the running front ends, didn't touch either of the end station machines or the DAQ, and didn't modify the network in any way.  I didn't leave anything running.

If you couldn't access test points then it sounds like it was more than just EPICS.  It sounds like maybe the end machines somehow fell of the network momentarily.  Was there anything else going on at the time?

 

ELOG V3.1.3-