40m QIL Cryo_Lab CTN SUS_Lab TCS_Lab OMC_Lab CRIME_Lab FEA ENG_Labs OptContFac Mariner WBEEShop
  40m Log  Not logged in ELOG logo
Entry  Mon Mar 24 19:42:12 2014, Charles, Frogs, VAC, Power Failure 
    Reply  Mon Mar 24 21:36:28 2014, Koji, Update, General, Power Failure 
       Reply  Mon Mar 24 22:13:37 2014, steve, Update, General, Power Failure poweroutage.png
          Reply  Tue Mar 25 14:52:57 2014, steve, Update, VAC, vacuum is recovered 17hrsNoPumping.png
             Reply  Fri Mar 28 16:26:20 2014, steve, Update, VAC, RGA scan after power failure afterPowerFailure.png
          Reply  Tue Mar 25 16:11:24 2014, Koji, Update, General, Power Failure 
             Reply  Wed Mar 26 11:30:07 2014, Koji, Update, General, Power Failure 
                Reply  Wed Mar 26 22:22:43 2014, Manasa, Update, General, Recovery 
                   Reply  Thu Mar 27 10:34:39 2014, ericq, Update, General, Recovery 
Message ID: 9750     Entry time: Tue Mar 25 16:11:24 2014     In reply to: 9748     Reply to this: 9752
Author: Koji 
Type: Update 
Category: General 
Subject: Power Failure 

As far as I know the system is running as usual. I had the IMC locked and one of the arm flashing.
But the other arm had no flash and none of the arms were locked before kunch time.

This morning Steve and I went around the lab to turn on the realtime machines.

Also we took the advantage of this opportunity to shutdown linux1 and nodus
to replace the extension cables for their AC power.

I also installed a 3TB hard disk on linux1. This was to provide a local daily copy of our
working are. But I could not make the disk recognized by the OS.
It seems that there is a "2TB" barrier that the disk bigger than 2.2TB can't be recognized
by the older machines. I'll wait for the upgrade of the machine.

Rebooting the realtime machines did not help FB to talk with them. I fixed them.
Basically what I did was:

- Stop all of the realtime codes by running rtcds kill all on c1lsc, c1ioo, c1sus, c1iscex, c1iscey

- run sudo ntpdate -b -s -u pool.ntp.org on c1lsc, c1ioo, c1sus, c1iscex, c1iscey, and fb

- restart realtime codes one by one. I checked which code makes FB unhappy. But in reality
  FB was happy with all of them running.

Then slow machines except for c1vac1 and c1vac2 were burtrestored.


Zach reported that svn was down. I went to the 40m wiki and searched "apache".
There is an instruction how to restart apache.

ELOG V3.1.3-