Frame builder is down. PRM has tripped its watch dogs. I have reset the watch dog on PRM and turned on the OPLEV. It has damped down. Unable to check what happened since FB is not responding.
There was an minor earthquake yesterday morning which people could feel a few blocks away. It could have caused the the PRM to unlock.
Jamie,Rolf, is it okay or us to restart the FB?
If it's down it's alway ok to restart it. If it doesn't respond or immediately crashes again after restart then it might require some investigation, but it should always be ok to restart it.
I tried restarting the fb in two different ways. Neither of them re-established the connection to dtt or epics.
1) I restarted the fb from the control room console with the 'shutdown' command. No change.
2) I halted the machine with 'shutdown -h now' and restarted it with the hardware reset button on its front-panel. No change.
The console connected to the fb showed that the network file systems did not load. Could this have resulted in failure to start several services since it could not find the files which are stored on the network file system?
The fb is otherwise healthy since I am able to ssh into it and browse the directory structure.