Came in, found all front-ends down.
Keyed a bunch of crates, no luck:
Requesting coeff update at 0x40f220 w/size of 0x1e44
No response from EPICS
Powered off/restarted c1dcuepics. Still no luck.
Powered off megatron. Success! Ok, maybe it wasn't megatron. I also did c1susvme1 and c1susvme2 at this time.
BURT restored to Nov 26, 8:00am
But everything is still red on the C0_DAQ_RFMNETWORK.adl screen, even though the front-ends are running and synced with the LSC. I think this means the framebuilder or the DAQ controller is the one in trouble--I keyed the crates with DAQCTRL and DAQAWG a couple of times, with no luck, so it's probably fb40m. I'm leaving it this way--we can deal with it tomorrow.
I found the red sea when I came in this morning.
I tried several things.
I'm now going to restart the single front -ends and burtgooey them if necessary.
Everything is back on.
Restarted all the front ends. As usual c1susvme2 was stubborn but eventually it came up.
I burt-restored all the front-ends to Nov 26 at 8am.
The mode cleaner is locked.