Den tried adding a SINGLE acquire channel to the c1ioo, which for some reason hung c1ioo and took down the entire DAQ network (at least all communication between the front ends and the fb). We recovered by restarting c1ioo and restarting mx_stream on all the rest of the front-ends
After "recovering", though, c1ioo is failing to load models, or at least it's IOP. Here is the tail of dmesg when trying to start the IOP:
[ 1751.140283] c1x03: Initializing space for daqLib buffers
[ 1751.140284] c1x03: Initializing Network
[ 1751.140285] c1x03: Found 1 frameBuilders on network
[ 1751.250658] CPU 2 is now offline
[ 1751.250657] c1x03: Sync source = 4
[ 1751.250657] c1x03: Waiting for EPICS BURT Restore = 1
[ 1751.310008] c1x03: Waiting for EPICS BURT 0
[ 1751.310008] c1x03: BURT Restore Complete
[ 1751.310008] c1x03: Initialized servo control parameters.
[ 1751.311699] c1x03: DAQ Ex Min/Max = 1 3
[ 1751.311699] c1x03: DAQ XEx Min/Max = 3 53
[ 1751.311733] c1x03: DAQ Tp Min/Max = 10001 10007
[ 1751.311733] c1x03: DAQ XTp Min/Max = 10007 10507
[ 1751.311737] c1x03: DIRECT MEMORY MODE of size 64
[ 1751.311737] c1x03: daqLib DCU_ID = 33
[ 1751.311737] c1x03: Invalid num daq chans = 0
[ 1751.311737] c1x03: DAQ init failed -- exiting
The chan file for this model (/opt/rtcds/caltech/c1/chans/daq/C1X03.ini) looks totally fine, has two un-acquired channels uncommented, and has otherwise not been touched. The C1:FEC-33_MSGDAQ is also reading: "ERROR reading DAQ file!"
I'm at a loss for what is going on. I've tried restarting every CDS process on the machine, restarting the model multiple times, restarting fb, and even restarting the entire c1ioo machine, all to no affect.