40m QIL Cryo_Lab CTN SUS_Lab TCS_Lab OMC_Lab CRIME_Lab FEA ENG_Labs OptContFac Mariner WBEEShop
  40m Log, Page 284 of 335  Not logged in ELOG logo
ID Date Author Type Category Subjectup
  3753   Thu Oct 21 13:05:19 2010 KojiSummaryComputerselog change and rossa tex

This is cool though the projector is flashing the blue screen alternately.

I gave the dual head video card (ATI RADEON something) to Yuta a month ago.
It is on the top of Zita. This would make the things more fun.

Quote:

Yuta has also successfully set up zita to run the projector, so we should clear our some of the boxes and bookshelves in that area so that the projection can be larger.

 

  3436   Wed Aug 18 19:14:59 2010 JenneUpdateelogelog dead again

The elog was so dead this time that the restart script didn't work.  I followed the restart-by-hand instructions instead, with success.

  3439   Thu Aug 19 01:49:14 2010 JenneUpdateelogelog dead again

Quote:

The elog was so dead this time that the restart script didn't work.  I followed the restart-by-hand instructions instead, with success.

 Just for added interest, I tried a different method when the restart script broke.  The "start-elog-nodus" script has a line "kill elogd".  This seems not to be actually killing anything anymore, which means the elog can't restart.  So this time I went for "kill <pid number>", and then ran the startup script.  This worked.  So it's the "kill elogd" which isn't working reliably.

  3440   Thu Aug 19 09:51:43 2010 josephbUpdateelogelog dead again

I found the elog dead again this morning, and the script didn't kill again. I modified the script to use the following line instead of "pkill elogd":

kill `ps -ef | grep elogd | grep -v grep | grep -v start-elog-nodus | awk '{print $2}'`

Hopefully the script will be a bit more robust in the future.

Quote:

Quote:

The elog was so dead this time that the restart script didn't work.  I followed the restart-by-hand instructions instead, with success.

 Just for added interest, I tried a different method when the restart script broke.  The "start-elog-nodus" script has a line "kill elogd".  This seems not to be actually killing anything anymore, which means the elog can't restart.  So this time I went for "kill <pid number>", and then ran the startup script.  This worked.  So it's the "kill elogd" which isn't working reliably.

 

  5232   Sun Aug 14 19:06:50 2011 JenneUpdateelogelog dead. Brought back to life

like the subject says...

  2979   Tue May 25 07:58:23 2010 kiwamuUpdateelogelog down

I found the elog got down around 7:30 am in this morning.

So I restarted it by running the script: "start-elog-nodus" as instructed on the wiki.

http://lhocds.ligo-wa.caltech.edu:8000/40m/How_To/Restart_the_elog

  3433   Wed Aug 18 12:02:29 2010 AlastairConfigurationComputerselog had crashed again...

...I restarted it.

  2702   Tue Mar 23 15:38:26 2010 AlbertoUpdateelogelog just restarted

I found the elog down and I restarted it.

Then, after few seconds it was down again. Maybe someone else was messing with it. I restarted an other 5 times and eventually it came back up.

  1947   Tue Aug 25 23:16:09 2009 Alberto, ranaConfigurationComputerselog moved in to the cvs path

In nodus, I moved the elog from /export to /cvs/cds/caltech. So now it is in the cvs path instead of a local directory on nodus.

For a while, I'll leave a copy of the old directory containing the logbook subdirectory where it was. If everything works fine, I'll delete that.

I also updated the reboot instructions in the wiki. some of it also is now in the SVN.

  1686   Fri Jun 19 13:38:42 2009 AlbertoConfigurationComputerselog rebooted

Today I found the elog down, so I rebooted it following the instructions in the wiki.

  1688   Fri Jun 19 14:30:47 2009 AlbertoConfigurationComputerselog rebooted

Quote:

Today I found the elog down, so I rebooted it following the instructions in the wiki.

 I have the impression that Nodus has been rebooted since last night, hasn't it?

  1689   Sun Jun 21 00:08:26 2009 ranaConfigurationComputerselog rebooted

 

nodus:log>dmesg

Sun Jun 21 00:06:43 PDT 2009
Mar  6 15:46:32 nodus sshd[26490]: [ID 800047 auth.crit] fatal: Timeout before authentication for 131.215.114.93
Mar 10 11:11:32 nodus sshd[22775]: [ID 800047 auth.crit] fatal: Timeout before authentication for 131.215.114.93
Mar 11 13:27:37 nodus sshd[7768]: [ID 800047 auth.error] error: connect_to 131.215.115.52 port 7000: Connection refused
Mar 11 13:27:37 nodus sshd[7768]: [ID 800047 auth.error] error: connect_to nodus port 7000: failed.
Mar 11 13:31:40 nodus sshd[7768]: [ID 800047 auth.error] error: connect_to 131.215.115.52 port 7000: Connection refused
Mar 11 13:31:40 nodus sshd[7768]: [ID 800047 auth.error] error: connect_to nodus port 7000: failed.
Mar 11 13:31:45 nodus sshd[7768]: [ID 800047 auth.error] error: connect_to 131.215.115.52 port 7000: Connection refused
Mar 11 13:31:45 nodus sshd[7768]: [ID 800047 auth.error] error: connect_to nodus port 7000: failed.
Mar 11 13:34:58 nodus sshd[7768]: [ID 800047 auth.error] error: connect_to 131.215.115.52 port 7000: Connection refused
Mar 11 13:34:58 nodus sshd[7768]: [ID 800047 auth.error] error: connect_to nodus port 7000: failed.
Mar 12 16:09:23 nodus sshd[22785]: [ID 800047 auth.crit] fatal: Timeout before authentication for 131.215.114.93
Mar 14 20:14:42 nodus sshd[13563]: [ID 800047 auth.crit] fatal: Timeout before authentication for 131.215.114.93
Mar 25 19:47:19 nodus sudo: [ID 702911 local2.alert] controls : 3 incorrect password attempts ; TTY=pts/2 ; PWD=/cvs/cds ; USER=root ; COMMAND=/usr/bin/rm -rf kamioka/
Mar 25 19:48:46 nodus su: [ID 810491 auth.crit] 'su root' failed for controls on /dev/pts/2
Mar 25 19:49:17 nodus last message repeated 2 times
Mar 25 19:51:14 nodus sudo: [ID 702911 local2.alert] controls : 1 incorrect password attempt ; TTY=pts/2 ; PWD=/cvs/cds ; USER=root ; COMMAND=/usr/bin/rm -rf kamioka/
Mar 25 19:51:22 nodus su: [ID 810491 auth.crit] 'su root' failed for controls on /dev/pts/2
Jun  8 16:12:17 nodus su: [ID 810491 auth.crit] 'su root' failed for controls on /dev/pts/4

nodus:log>uptime
 12:06am  up 150 day(s), 11:52,  1 user,  load average: 0.05, 0.07, 0.07

  2179   Thu Nov 5 12:34:26 2009 kiwamuUpdateComputerselog rebooted

I found elog got crashed. I rebooted the elog daemon just 10minutes before.

  1623   Sun May 24 11:24:08 2009 robUpdateComputerselog restarted

I just restarted the elog.  It was crashed for unknown reasons.  The restarting instructions are in the wiki.

  1785   Fri Jul 24 11:04:11 2009 AlbertoConfigurationComputerselog restarted

This morning I found the elog down. I restarted it using the procedure in the wiki.

  1809   Wed Jul 29 19:31:17 2009 ranaConfigurationComputerselog restarted

Just now found it dead. Restarted it. Is our elog backed up in the daily backups?

  1893   Wed Aug 12 15:02:33 2009 AlbertoConfigurationComputerselog restarted

In the last hour or so the elog crashed. I  have restarted it.

  1944   Tue Aug 25 21:26:12 2009 AlbertoUpdateComputerselog restarted

I just found the elog down and I restarted it.

  2106   Fri Oct 16 16:44:39 2009 Alberto, SanjitUpdateComputerselog restarted

This afternoon the elog crashed. We just restarted it.

  2567   Wed Feb 3 10:46:12 2010 AlbertoUpdateelogelog restarted

 Again, this morning Zach told me that the elog had crashed while he was trying to post an entry.

I just restarted it.

  2569   Thu Feb 4 00:59:52 2010 ranaUpdateelogelog restarted

 I restarted the ELOG on NODUS just now. Our attempt to set up error logging worked - it turns out ELOG was choking on the .ps file attachment.

So for the near future: NO MORE .PS files! Use PDF - move into the 20th century at least.

matlab can directly make either PNG or PDF files for you, you can also use various other conversion tools on the web.

Of course, it would be nice if nodus could handle .ps, but its a Solaris machine and I don't feel like debugging this. Eventually, we'll give him away and make the new nodus a Linux box, but that day is not today.

  2669   Fri Mar 12 13:52:18 2010 ZachUpdateelogelog restarted

 The elog was down and I ran the restart script.

  2758   Fri Apr 2 08:52:21 2010 AlbertoUpdateelogelog restarted

i just restarted the elog for the third time in the past 12 hours.

I checked the elog.log file to debug the problem. It doesn't contain eveidence of any particular cause, except for png/jpg file uploads happened last night.

I'm not sure we can blame Image Magic again because the last crash seems to be occurred just after an entry with e jpg picture was included in the body of the message. I think Image Magic is used only for previews of attachments like pdfs or ps.

Maybe we should totally disable image magic.

  2792   Mon Apr 12 17:48:32 2010 AidanUpdateComputer Scripts / Programselog restarted

 The elog crashed when I was uploading a photo just now. I logged into nodus and restarted it.

  2817   Tue Apr 20 13:00:52 2010 ZachUpdateelogelog restarted

 I restarted the elog with the restart script as it was down.

  2853   Wed Apr 28 08:55:19 2010 ZachUpdateelogelog restarted

 Restarted the elog with the script as it was down.

  2854   Wed Apr 28 09:21:16 2010 ZachUpdateelogelog restarted

And again.

Quote:

 Restarted the elog with the script as it was down.

 

  3390   Tue Aug 10 01:05:17 2010 ZachUpdateelogelog restarted

 Elog was down. Restarted with the script.

  3396   Wed Aug 11 02:44:37 2010 ZachUpdateelogelog restarted

You'll never guess what happened: the elog crashed!

I restarted it with the script. Yay.

  3401   Wed Aug 11 16:13:59 2010 JennaUpdateelogelog restarted

The elog crashed, so we restarted it again.

  3406   Thu Aug 12 11:39:27 2010 ZachUpdateelogelog restarted

with script

  3533   Tue Sep 7 14:42:02 2010 DmassConfigurationelogelog restarted

elog crashed on an upload. restarted and it worked fine with the same file.

  3537   Tue Sep 7 22:21:17 2010 AlbertoConfigurationComputerselog restarted
  3538   Tue Sep 7 22:21:47 2010 DmassConfigurationelogelog restarted

Quote:

elog crashed on an upload. restarted and it worked fine with the same file.

 Again. Resubmitted an old entry with just text changes. Elog hung for 5 min +.

  3567   Mon Sep 13 12:09:42 2010 kiwamuSummaryGeneralelog restarted

 Elog didn't respond, so I restarted it with the script.

Before killing the process somehow two elog daemons were running at the same time. I killed those two manually. 

  3613   Mon Sep 27 21:57:59 2010 ZachUpdateelogelog restarted

 took two runs of the script as usual

  4017   Mon Dec 6 22:43:19 2010 FrankConfigurationelogelog restarted

I restarted the elog because i changed the configuration for the cryo-elog.

Used the "start-elog.csh" script in /cvs/cds/caltech/elog/

  5348   Tue Sep 6 21:00:48 2011 ZachUpdateelogelog restarted

I restarted the elog with the script as it was not up when I tried to make a post. It was again unresponsive when I went to submit, but this time the script couldn't restart it. The log said it couldn't bind to 8080, which usually happens if the daemon is still running. I pkilled it, then reran the script, and it appears to be working.

  10817   Fri Dec 19 14:25:48 2014 diegoUpdateComputer Scripts / Programselog restarted

elog was not responding for unknown reasons, since the elogd process on nodus was alive; anyway, I restarted it. 

  3993   Tue Nov 30 11:44:36 2010 ZachUpdateelogelog restarted -- SCRIPT adapted for 2.8.0

 I have created an updated version of the "start-elog-nodus" script and put it in .../elog/elog-2.8.0. It seems to work fine.

  3382   Sat Aug 7 10:47:38 2010 KojiSummaryelogelog restarted / source of the trouble eliminated

Nancy notified me that the elog crashed. It was fixed.


I restarted elog, but it kept crashing. Some of the entries on Aug 6th caused the trouble.

I tried to refresh the pictures in entries 3376, 3377, 3378. Still it kept crashing.
I started to dig into the elog file itself. (/cvs/cds/caltech/elog/elog-2.7.5/logbooks/40m/100806a.log)

FInally I found that there was some invalid reply links in the entry 3379.

Date: Fri, 06 Aug 2010 19:29:59 -0700
Reply to: 3379
In reply to: 3379

The entry is refering this entry itself. That is weird. So I deleted the reply-to and in-reply-to lines.
Then elogd got happy.

In fact, 3379 was a dupulication of 3380, so I deleted this entry.

  4438   Thu Mar 24 13:56:05 2011 josephbUpdateelogelog restarted at 1:55pm

Restarted elog.

  16041   Fri Apr 16 11:31:00 2021 ranaUpdateelogelog stuck ~10 AM today

found it unresponsive. Restarted fine using procedure documented in wiki

  5750   Fri Oct 28 02:41:18 2011 SureshMetaphysicselogelog unresponsive: restarted

Elog did not respond despite running the /cvs/cds/caltech/elog/start-elog.csh  script two times.  

It worked the after the third restart. 
 

  7136   Thu Aug 9 12:55:15 2012 ZachUpdateelogelog was being a pain in the ass; I restarted it

The elog was not responding. I attempted to restart it by running .../start-elog.csh, but this didn't work (even after the usual ~2 times it needs).

Somehow pkill did not kill the daemon, so I kill -9'd it and that did the trick. I ran the start script once more and it came back online.

  3429   Tue Aug 17 09:06:08 2010 AlbertoUpdateelogelog was down

I just restarted the elog after I found it down a few minutes ago.

  7757   Wed Nov 28 17:40:28 2012 jamieOmnistructureComputerselog working again on firefox 17

Koji and I figured out what the problem is.  Apparently firefox 17.0 (specifically it's user-agent string) breaks fckeditor, which is the javascript toolbox the elog uses for the wysiwyg text editor.  See https://support.mozilla.org/en-US/questions/942438.

The suspect line was in elog/fckeditor/editor/js/fckeditorcode_gecko.js.  I hacked it up so that it stopped whatever crappy conditional user agent crap it was doing.  It seems to be working now.

Edit by Koji: In order to make this change work, I needed to clear the cache of firefox from Tool/Clear Recent History menu.

  510   Sun Jun 1 19:39:35 2008 tobinConfigurationComputerselog, etc
Phil Ehrens gave me a DVD of the 40m elog, apache, and (Jamie's) SVN archive.
I copied it to nodus:/home/controls/dvd-from-ehrens.  Once we get the elog
running on nodus, we can copy the datafile over again from dziban (so that
we don't lose any elog entries) and switch over.
  3475   Fri Aug 27 07:21:13 2010 AlastairUpdateGeneralelog...

 was down.  I restarted the version in the 2.7.5 folder.  It went down again almost immediately but stayed up after the second restart.

  12049   Sat Mar 26 18:28:24 2016 KojiUpdateelogelogd flakiness

Elogd have been restarted several times today because it died everytime I submit something.
Here is the copy of the log.


GET /OMC_Lab/255?cmd=loc&value=Submit HTTP/1.1
Returned 6 bytes
GET /40m/elog.rdf HTTP/1.1
Returned 17109 bytes
TCP connection #1 on socket 5 closed
POST /OMC_Lab/ HTTP/1.1
Returned 20 bytes
GET /OMC_Lab/255 HTTP/1.1
Returned 53721 bytes
GET /ckeditor/skins/moono/images/arrow.png HTTP/1.1
Returned 489 bytes
POST /OMC_Lab/ HTTP/1.1
*** buffer overflow detected ***: /export/home/elog/elog/elogd terminated
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(__fortify_fail+0x37)[0x7f1435639e57]
/lib/x86_64-linux-gnu/libc.so.6(+0x108d50)[0x7f1435638d50]
/lib/x86_64-linux-gnu/libc.so.6(+0x1081b9)[0x7f14356381b9]
/lib/x86_64-linux-gnu/libc.so.6(_IO_default_xsputn+0xdd)[0x7f14355ab0cd]
/lib/x86_64-linux-gnu/libc.so.6(_IO_vfprintf+0x25a8)[0x7f143557ac18]
/lib/x86_64-linux-gnu/libc.so.6(__vsprintf_chk+0x94)[0x7f1435638254]
/lib/x86_64-linux-gnu/libc.so.6(__sprintf_chk+0x7d)[0x7f143563819d]
/export/home/elog/elog/elogd[0x426405]
/export/home/elog/elog/elogd[0x473b7f]
/export/home/elog/elog/elogd[0x4abfb2]
/export/home/elog/elog/elogd[0x4ad7fb]
/export/home/elog/elog/elogd[0x4b0af5]
/export/home/elog/elog/elogd[0x4b1eb9]
/export/home/elog/elog/elogd[0x403568]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed)[0x7f143555176d]
/export/home/elog/elog/elogd[0x404299]
======= Memory map: ========
00400000-004e6000 r-xp 00000000 fc:00 10361276                           /export/home/elog/elog-3.0.d/elogd
006e5000-006e6000 r--p 000e5000 fc:00 10361276                           /export/home/elog/elog-3.0.d/elogd
006e6000-007c6000 rw-p 000e6000 fc:00 10361276                           /export/home/elog/elog-3.0.d/elogd
007c6000-0173d000 rw-p 00000000 00:00 0
0214d000-02656000 rw-p 00000000 00:00 0                                  [heap]
7f14342f8000-7f143430d000 r-xp 00000000 fc:00 2883628                    /lib/x86_64-linux-gnu/libgcc_s.so.1
7f143430d000-7f143450c000 ---p 00015000 fc:00 2883628                    /lib/x86_64-linux-gnu/libgcc_s.so.1
7f143450c000-7f143450d000 r--p 00014000 fc:00 2883628                    /lib/x86_64-linux-gnu/libgcc_s.so.1
7f143450d000-7f143450e000 rw-p 00015000 fc:00 2883628                    /lib/x86_64-linux-gnu/libgcc_s.so.1
7f143450e000-7f14348cd000 rw-p 00000000 00:00 0
7f1434a34000-7f1434d39000 r--p 00000000 fc:00 530477                     /usr/lib/locale/locale-archive
7f1434d39000-7f1434d4f000 r-xp 00000000 fc:00 655527                     /usr/local/lib/libz.so.1.2.8
7f1434d4f000-7f1434f4e000 ---p 00016000 fc:00 655527                     /usr/local/lib/libz.so.1.2.8
7f1434f4e000-7f1434f4f000 r--p 00015000 fc:00 655527                     /usr/local/lib/libz.so.1.2.8
7f1434f4f000-7f1434f50000 rw-p 00016000 fc:00 655527                     /usr/local/lib/libz.so.1.2.8
7f1434f50000-7f1434f52000 r-xp 00000000 fc:00 2883655                    /lib/x86_64-linux-gnu/libdl-2.15.so
7f1434f52000-7f1435152000 ---p 00002000 fc:00 2883655                    /lib/x86_64-linux-gnu/libdl-2.15.so
7f1435152000-7f1435153000 r--p 00002000 fc:00 2883655                    /lib/x86_64-linux-gnu/libdl-2.15.so
7f1435153000-7f1435154000 rw-p 00003000 fc:00 2883655                    /lib/x86_64-linux-gnu/libdl-2.15.so
7f1435154000-7f1435307000 r-xp 00000000 fc:00 2883609                    /lib/x86_64-linux-gnu/libcrypto.so.1.0.0
7f1435307000-7f1435506000 ---p 001b3000 fc:00 2883609                    /lib/x86_64-linux-gnu/libcrypto.so.1.0.0
7f1435506000-7f1435521000 r--p 001b2000 fc:00 2883609                    /lib/x86_64-linux-gnu/libcrypto.so.1.0.0
7f1435521000-7f143552c000 rw-p 001cd000 fc:00 2883609                    /lib/x86_64-linux-gnu/libcrypto.so.1.0.0
7f143552c000-7f1435530000 rw-p 00000000 00:00 0
7f1435530000-7f14356e4000 r-xp 00000000 fc:00 2884139                    /lib/x86_64-linux-gnu/libc-2.15.so
7f14356e4000-7f14358e3000 ---p 001b4000 fc:00 2884139                    /lib/x86_64-linux-gnu/libc-2.15.so
7f14358e3000-7f14358e7000 r--p 001b3000 fc:00 2884139                    /lib/x86_64-linux-gnu/libc-2.15.so
7f14358e7000-7f14358e9000 rw-p 001b7000 fc:00 2884139                    /lib/x86_64-linux-gnu/libc-2.15.so
7f14358e9000-7f14358ee000 rw-p 00000000 00:00 0
7f14358ee000-7f1435943000 r-xp 00000000 fc:00 2884155                    /lib/x86_64-linux-gnu/libssl.so.1.0.0
7f1435943000-7f1435b42000 ---p 00055000 fc:00 2884155                    /lib/x86_64-linux-gnu/libssl.so.1.0.0
7f1435b42000-7f1435b45000 r--p 00054000 fc:00 2884155                    /lib/x86_64-linux-gnu/libssl.so.1.0.0
7f1435b45000-7f1435b4c000 rw-p 00057000 fc:00 2884155                    /lib/x86_64-linux-gnu/libssl.so.1.0.0
7f1435b4c000-7f1435b6e000 r-xp 00000000 fc:00 2884145                    /lib/x86_64-linux-gnu/ld-2.15.so
7f1435d57000-7f1435d5c000 rw-p 00000000 00:00 0
7f1435d6a000-7f1435d6e000 rw-p 00000000 00:00 0
7f1435d6e000-7f1435d6f000 r--p 00022000 fc:00 2884145                    /lib/x86_64-linux-gnu/ld-2.15.so
7f1435d6f000-7f1435d71000 rw-p 00023000 fc:00 2884145                    /lib/x86_64-linux-gnu/ld-2.15.so
7ffd85795000-7ffd85997000 rw-p 00000000 00:00 0                          [stack]
7ffd859b2000-7ffd859b4000 r-xp 00000000 00:00 0                          [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]
er_id"
Received unknown cookie "ajs_group_id"
Received unknown cookie "ajs_anonymous_id"
Received unknown cookie "__utma"
Received unknown cookie "_ga"
Received unknown cookie "__unam"
Received unknown cookie "__utma"
Received unknown cookie "tk_ni"
Received unknown cookie "ajs_user_id"
Received unknown cookie "ajs_group_id"
Received unknown cookie "ajs_anonymous_id"
Received unknown cookie "cache_disable"
Received unknown cookie "NO_CACHE"
Received unknown cookie "__utma"
Received unknown cookie "_ga"
Received unknown cookie "__unam"
Received unknown cookie "__utma"
Received unknown cookie "tk_ni"
Received unknown cookie "ajs_user_id"
Received unknown cookie "ajs_group_id"
Received unknown cookie "ajs_anonymous_id"
Received unknown cookie "__utma"
Received unknown cookie "_ga"
Received unknown cookie "__unam"
Received unknown cookie "__utma"
Received unknown cookie "tk_ni"
Received unknown cookie "ajs_user_id"
Received unknown cookie "ajs_group_id"
Received unknown cookie "ajs_anonymous_id"
Received unknown cookie "__utma"
Received unknown cookie "_ga"
Received unknown cookie "__unam"
Received unknown cookie "__utma"
Received unknown cookie "tk_ni"
Received unknown cookie "ajs_user_id"
Received unknown cookie "ajs_group_id"
Received unknown cookie "ajs_anonymous_id"
Received unknown cookie "__utma"
Received unknown cookie "_ga"
Received unknown cookie "__unam"
Received unknown cookie "__utma"
Received unknown cookie "tk_ni"
Received unknown cookie "ajs_user_id"
Received unknown cookie "ajs_group_id"
Received unknown cookie "ajs_anonymous_id"
ELOG V3.1.3-