40m QIL Cryo_Lab CTN SUS_Lab TCS_Lab OMC_Lab CRIME_Lab FEA ENG_Labs OptContFac Mariner WBEEShop
  40m Log  Not logged in ELOG logo
Entry  Tue May 8 11:41:16 2018, gautam, Update, General, IFO maintenance ITMX_stuck.png
    Reply  Wed May 9 17:30:04 2018, gautam, Update, General, Input beam misaligned InputBeamGone.pngTTpointing.png
       Reply  Wed May 9 19:51:07 2018, gautam, Update, General, Input beam misaligned DACweirdness.pngDACerror.png
          Reply  Sun May 13 15:15:18 2018, gautam, Update, General, CDS crash vertexFEs_crashed.png
             Reply  Sun May 13 17:31:51 2018, gautam, Update, General, CDS crash CDS_overview_20180513.pngAS_1210293643.jpeg
                Reply  Sun May 13 20:48:38 2018, johannes, Update, General, CDS crash 
                Reply  Thu May 17 11:56:37 2018, gautam, Update, General, EPICS process died on c1ioo 
                   Reply  Thu May 24 10:16:29 2018, gautam, Update, General, All models on c1lsc frontend crashed c1lsc_crashed.png
    Reply  Thu May 10 08:45:16 2018, Steve, Update, General, 4.5M eq. Cabazon, CA Cabazon4.5m79m.png4.5Meq.png
       Reply  Thu May 10 11:38:19 2018, gautam, Update, General, ITMY UL ITMY_UL.pdf
       Reply  Wed Aug 22 08:44:09 2018, Steve, Update, General, earth quake yesterday_EQs.png
          Reply  Fri Aug 24 08:04:37 2018, Steve, Update, General, small earth quake small_EQ.png
             Reply  Mon Aug 27 09:14:45 2018, Steve, Update, PEM, small earth quakes  small_EQs_vs_SUSs.png
                Reply  Tue Nov 27 10:50:20 2018, Steve, Update, PEM, earth quake Mexico 5.5M.Mexico.png
                   Reply  Thu Nov 29 08:13:33 2018, Steve, Update, PEM, EQ 3.9m So CA 3.9mSoCA.pngVac_as_today.pngas_is.png
          Reply  Wed Aug 29 09:20:27 2018, Steve, Update, SUS, local 4.4M earth quake 4.4_La_Verne.png3.4_&_4.4M_EQ.png
             Reply  Wed Aug 29 11:46:27 2018, Jon, Update, SUS, local 4.4M earth quake 
             Reply  Thu Sep 20 08:17:14 2018, Steve, Update, SUS, local 3.4M earth quake local_3.4M.png
Message ID: 13838     Entry time: Sun May 13 17:31:51 2018     In reply to: 13837     Reply to this: 13839   13852
Author: gautam 
Type: Update 
Category: General 
Subject: CDS crash 

As suspected, this was indeed a path problem. Johannes will elog about it later, but in short, it is related to some path variables being changed in order to try and streamline the EPICS processes on the new c1auxex machine (Acromag Era). It is confusing that futzing around with the slow computing system messes with the realtime system as well - aren't these supposed to be decoupled? Once the paths were restored by Johannes, everything compiled and restarted fine. We even have a beam on the AS camera, which was what triggered this whole thingyes.

Anyways, Attachment #1 shows the current status. I am puzzled by the red TIMING indicators on the c1x04 and c1x02 processes, it is absent from any other processes. How can this be debugged further?

Quote:
 

I suspect this is some kind of path problem - the EPICS_BASE bash variable is set to /cvs/cds/rtapps/epics-3.14.12.2_long/base on the FEs, while /cvs isn't even mounted on the FEs (nor do I think it should be). I think the correct path should be /opt/rtapps/epics-3.14.12.2_long/base. Why should this have changed?

Attachment 1: CDS_overview_20180513.png  2.140 MB  Uploaded Sun May 13 18:34:32 2018  | Hide | Hide all
CDS_overview_20180513.png
Attachment 2: AS_1210293643.jpeg  7 kB  Uploaded Sun May 13 18:42:36 2018  | Hide | Hide all
AS_1210293643.jpeg
ELOG V3.1.3-