40m
QIL
Cryo_Lab
CTN
SUS_Lab
TCS_Lab
OMC_Lab
CRIME_Lab
FEA
ENG_Labs
OptContFac
Mariner
WBEEShop
|
40m Log |
Not logged in |
 |
|
Wed Jul 19 08:37:21 2017, Jamie, Update, CDS, Update on front-end/DAQ rebuild
|
Wed Jul 19 14:26:50 2017, Jamie, Update, CDS, Update on front-end/DAQ rebuild
|
Fri Jul 21 18:03:17 2017, Jamie, Update, CDS, Update on front-end/DAQ rebuild
|
Sun Jul 23 22:16:55 2017, Jamie, gautam, Update, CDS, front-end now running with new OS, RCG 
|
Mon Jul 24 10:45:23 2017, gautam, Update, CDS, c1iscex models died
|
Mon Jul 24 10:59:08 2017, Jamie, Update, CDS, c1iscex models died
|
Mon Jul 24 19:28:55 2017, Jamie, Update, CDS, front end MX stream network working, glitches in c1ioo fixed
|
Mon Jul 24 19:57:54 2017, gautam, Update, CDS, IMC locked, Autolocker re-enabled
|
Wed Jul 26 19:13:07 2017, Jamie, Update, CDS, daqd showing same instability as before
|
Fri Jul 28 20:22:41 2017, Jamie, Update, CDS, possible stable daqd configuration with separate DC and FW
|
Mon Jul 31 15:13:24 2017, gautam, Update, CDS, FB ---> FB1
|
Mon Jul 31 18:44:40 2017, Jamie, Update, CDS, CDS system essentially fully recovered
|
Thu Aug 3 19:46:27 2017, Jamie, Update, CDS, new daqd restart procedure
|
Fri Aug 4 09:07:28 2017, rana, Update, CDS, CDS system essentially NOT fully recovered
|
Thu Aug 10 14:25:52 2017, gautam, Update, CDS, Slow EPICS channels -> Frames re-enabled
|
Fri Aug 11 00:10:03 2017, gautam, Update, CDS, Slow EPICS channels -> Frames re-enabled
|
Fri Aug 11 11:14:24 2017, gautam, Update, CDS, Slow EPICS channels -> Frames re-enabled
|
Fri Aug 11 18:53:35 2017, gautam, Update, CDS, Slow EPICS channels -> Frames re-enabled
|
Fri Aug 11 19:34:49 2017, Jamie, Update, CDS, CDS final bits status update
|
|
Message ID: 13149
Entry time: Fri Jul 28 20:22:41 2017
In reply to: 13145
Reply to this: 13152
13153
|
Author: |
Jamie |
Type: |
Update |
Category: |
CDS |
Subject: |
possible stable daqd configuration with separate DC and FW |
|
|
This week Jonathan Hanks and I have been trying to diagnose why the daqd has been unstable in the configuration used by the 40m, with data concentrator (dc) and frame writer (fw) in the same process (referred to generically as 'fb'). Jonathan has been digging into the core dumps and source to try to figure out what's going on, but he hasn't come up with anything concrete yet.
As an alternative, we've started experimenting with a daqd configuration with the dc and fw components running in separate processes, with communication over the local loopback interface. The separate dc/fw process model more closely matches the configuration at the sites, although the sites put dc and fwprocesses on different physical machines. Our experimentation thus far seems to indicate that this configuration is stable, although we haven't yet tested it with the full configuration, which is what I'm attempting to do now.
Unfortunately I'm having trouble with the mx_stream communication between the front ends and the dc process. The dc does not appear to be receiving the streams from the front ends and is producing a '0xbad' status message for each. I'm investigating. |