I tried to restart c1ioo becuase I can't live without him.
I couldn't ssh or ping c1ioo, so I did hardware reboot.
c1ioo came back, but now ADC/DAC stats are all red.
c1ioo was OK until 3am when I left the control room last night. I don't know what happened, but StripTool from zita tells me that MC lock went off at around 4pm.
c1ioo was still all red on the CDS status screen, so I tried a couple of things.
mxstreamrestart (which aliases on the front ends to sudo /etc/init.d/mx_stream restart) didn't help
sudo /etc/init.d/mx_stream restart
sudo shutdown -r now didn't change anything either....c1ioo came back with red everywhere and 0x2bad on the IOP
sudo shutdown -r now
eventually doing as Jamie did for c1sus in elog 6742, rtcds stop all, then rtcds start all fixed everything. Interestingly, when I tried rtcds start iop, I got the error
Cannot start/stop model 'iop' on host c1ioo, so I just tried rtcds start all, and that worked fine....started with c1x03, then c1ioo, then c1gcv.
rtcds stop all
rtcds start all
rtcds start iop
Cannot start/stop model 'iop' on host c1ioo
There seems to be a problem with how models are coming up on boot since the upgrade. I think the IOP isn't coming up correctly for some reason, which is then preventing the rest of the models from starting since they depend on the IOP.
The simple way to fix this is to run the following:
rtcds restart all
The "restart" command does the smart thing, by stopping all the models in the correct order (IOP last) and then restarting them also in the correct order (IOP first).