|
40m
QIL
Cryo_Lab
CTN
SUS_Lab
TCS_Lab
OMC_Lab
CRIME_Lab
FEA
ENG_Labs
OptContFac
Mariner
WBEEShop
|
| 40m Log |
Not logged in |
 |
|
|
 Fri Jun 8 14:40:55 2018, c1lsc, Update, CDS, i am dead 
|
 Fri Jun 8 20:15:08 2018, gautam, Update, CDS, Reboot script
| |
Mon Jun 11 18:49:06 2018, gautam, Update, CDS, c1lsc dead again
| |
Wed Dec 4 09:11:42 2019, Yehonathan, Update, CDS, Reboot script
| |
Wed Dec 4 12:13:10 2019, gautam, Update, CDS, Reboot script
|
|
|
Message ID: 13935
Entry time: Fri Jun 8 20:15:08 2018
In reply to: 13934
Reply to this: 13942
15071
|
| Author: |
gautam |
| Type: |
Update |
| Category: |
CDS |
| Subject: |
Reboot script |
|
|
Unfortunately, this has happened (and seems like it will happen) enough times that I set up a script for rebooting the machine in a controlled way, hopefully it will negate the need to repeatedly go into the VEA and hard-reboot the machines. Script lives at /opt/rtcds/caltech/c1/scripts/cds/rebootC1LSC.sh. SVN committed. It worked well for me today. All applicable CDS indicator lights are now green again. Be aware that c1oaf will probably need to be restarted manually in order to make the DC light green. Also, this script won't help you if you try to unload a model on c1lsc and the FE crashes. It relies on c1lsc being ssh-able. The basic logic is:
- Ask for confirmation.
- Shutdown all vertex optic watchdogs, PSL shutter.
- ssh into c1sus and c1ioo, shutdown all models on these machines, soft reboot them.
- ssh into c1lsc, soft reboot the machine. No attempt is made to unload the models.
- Wait 2 minutes for all machines to come back online.
- Restart models on all 3 vertex FEs (IOPs first, then rest).
- Prompt user for confirmation to re-enable watchdog status and open PSL shutter.
|
|
|