40m QIL Cryo_Lab CTN SUS_Lab TCS_Lab OMC_Lab CRIME_Lab FEA ENG_Labs OptContFac Mariner WBEEShop
  40m Log  Not logged in ELOG logo
Entry  Thu Sep 23 02:45:30 2010, Koji, Summary, Computers, nodus gracefully rebooted 
    Reply  Thu Sep 23 10:34:20 2010, rana, Frogs, Computers, nodus gracefully rebooted 
       Reply  Thu Sep 23 11:15:20 2010, Koji, Frogs, Computers, nodus gracefully rebooted 
       Reply  Thu Sep 23 13:16:57 2010, Koji, Frogs, Computers, nodus gracefully rebooted 
Message ID: 3597     Entry time: Thu Sep 23 02:45:30 2010     Reply to this: 3598
Author: Koji 
Type: Summary 
Category: Computers 
Subject: nodus gracefully rebooted 

Zach> Nodus seemed to be working fine again, and I was browsing the elog with no
Zach> problem. I tried making an entry, but when I started uploading a file it
Zach> became unresponsive. Tried SSHing, but I get no prompt after the welcome
Zach> blurb. ^C gives me some kind of tcsh prompt (">"), which only really
Zach> responds to ^D (logout). Don't know what else to do, but I assume someone
Zach> knows what's going on.

By gracefully rebooting nodus, the problem was solved.


It (">") actually was the tcsh prompt, but any commands with the shared or dynamic link libraries looked unfunctional.

I could use
    cd /.../...
and
    echo *
to browse the directory tree. The main mounted file systems like /, /usr, /var, /cvs/cds/caltech looked fine.
I was afraid that the important library files were damaged.

I tried
    umountall
and
    mountall
in order to flush the file systems.
These should run even without the libraries as mount must properly work even before /usr is mounted.

They indeed did something to the system. Once I re-launch a new login shell, the prompt was still ">"
but now I could use most of the commands.

I have rebooted by usual sudo-ing and now the services on nodus are back to the functional state again.

# nodus was working in the evening at around 9pm. I even made an e-log entry about that.
# So I like to assume this is not directly related to the linux1 incident. Something else could have happened.

ELOG V3.1.3-