40m QIL Cryo_Lab CTN SUS_Lab TCS_Lab OMC_Lab CRIME_Lab FEA ENG_Labs OptContFac Mariner WBEEShop
  40m Log  Not logged in ELOG logo
Entry  Mon Jun 16 21:56:53 2014, Jenne, Update, Computer Scripts / Programs, fstab updates, MC autolocker running, FSS slow loop alive 
    Reply  Tue Jun 17 12:04:40 2014, manasa, Update, Computer Scripts / Programs, MC autolocker NOT running, FB needs some attention FE_red.png
       Reply  Tue Jun 17 16:52:40 2014, ericq, Update, Computer Scripts / Programs, autolocker confusion 
          Reply  Tue Jun 17 19:39:29 2014, rana, Update, Computer Scripts / Programs, autolocker confusion 
             Reply  Wed Jun 18 11:28:19 2014, Manasa, Update, Computer Scripts / Programs, autolocker confusion 
                Reply  Wed Jun 18 13:29:48 2014, rana, Update, Computer Scripts / Programs, autolocker confusion 
                   Reply  Wed Jun 18 17:38:14 2014, Jenne, Update, Computer Scripts / Programs, autolocker running again 
                      Reply  Thu Jun 19 16:49:15 2014, rana, Update, Computer Scripts / Programs, autolocker running again 
                         Reply  Thu Jun 19 22:04:23 2014, ericq, Update, Computer Scripts / Programs, caget/caput now return in reasonable time 
                            Reply  Fri Jun 20 09:46:49 2014, manasa, Update, IOO, MC WFS servo turned ON 
                            Reply  Fri Jun 20 19:07:44 2014, rana, Update, Computer Scripts / Programs, op340m DNS 
                               Reply  Tue Jun 24 22:46:15 2014, ericq, Update, Computer Scripts / Programs, op340m crons 
       Reply  Tue Jun 17 17:04:26 2014, ericq, Update, Computer Scripts / Programs, FB troubles 
Message ID: 10050     Entry time: Tue Jun 17 17:04:26 2014     In reply to: 10048
Author: ericq 
Type: Update 
Category: Computer Scripts / Programs 
Subject: FB troubles 

Quote:

Also, the CDS FE status screen had red lights blinking as if it required an 'mxstream restart'. I did the same and it did not fix the problem. So I tried to restart fb using the usual 'telnet fb 8087'; but could not restart fb that way.

 FB is acting strange. When ssh-ing in, certain commands cause an inescapable hang, which can't be ctrl-c'd out of. Telling it to reboot does nothing. This kind of situation was seen by me before, when we were getting all the front ends back, I eventually hard rebooted it, hoping it was a one time thing. Guess it's not. 

Looking at the dmesg output, daqd seems to be segfault-ing all over the place. This may be related... Here are some examples:

451314.730502] daqd[17339]: segfault at 7ff589ae3b30 ip 00007ff589ae3b30 sp 00007ff49931dfb8 error 15 in libmyriexpress.so[7ff589ae3000+1000]

[530516.313238] daqd[18442] general protection ip:7f3f2ce73a6c sp:7f3e29949d50 error:0

[530516.313250] daqd[18420] general protection ip:7f3f2ce73a6c sp:7f3e2a19fd50 error:0 in libc-2.10.1.so[7f3f2ce3f000+14c000]

[530516.313262]  in libc-2.10.1.so[7f3f2ce3f000+14c000]

[530516.327083] daqd[18412]: segfault at 3b04c9cd0 ip 00007f3f2ce73a6c sp 00007f3e2a4a7d50 error 4 in libc-2.10.1.so[7f3f2ce3f000+14c000]

[537695.364481] daqd[18489]: segfault at 12dbbcae0 ip 00007fa35a3b8a0a sp 00007fa298381af0 error 6 in libmyriexpress.so[7fa35a399000+28000]

[577316.821618] daqd[18758]: segfault at 7f5c4d3e9b30 ip 00007f5c4d3e9b30 sp 00007f5b5cc23fb8 error 15 in libmyriexpress.so[7f5c4d3e9000+1000]

I'm not inclined to go reboot it right now, but not sure how to address these problems...

 

 

ELOG V3.1.3-