40m QIL Cryo_Lab CTN SUS_Lab TCS_Lab OMC_Lab CRIME_Lab FEA ENG_Labs OptContFac Mariner WBEEShop
  40m Log  Not logged in ELOG logo
Entry  Tue Jan 7 22:44:45 2014, Jenne, Update, CDS, daqd on fb is segfaulting every ~30 seconds 
    Reply  Tue Jan 7 23:08:01 2014, jamie, Update, CDS, /frames is full, causing daqd to die 
       Reply  Tue Jan 7 23:13:47 2014, jamie, Update, CDS, /frames is full, causing daqd to die 
          Reply  Tue Jan 7 23:50:27 2014, jamie, Update, CDS, /frames space cleared up, daqd stabilized 
             Reply  Tue May 13 17:45:21 2014, rana, Update, CDS, /frames space cleared up, daqd stabilized 
                Reply  Thu May 15 01:42:07 2014, rana, Update, CDS, /frames space cleared up, daqd stabilized 
Message ID: 9533     Entry time: Tue Jan 7 23:13:47 2014     In reply to: 9531     Reply to this: 9535
Author: jamie 
Type: Update 
Category: CDS 
Subject: /frames is full, causing daqd to die 

Quote:

So why is /frames full?  Apparently the wiper script is either not running, or is failing to do it's job.  My guess is that this is a side effect of the linux1 raid failure we had over xmas.

It actually looks like the wiper script has been running fine.  There is a log from Tuesday morning:

controls@fb ~ 0$ cat /opt/rtcds/caltech/c1/target/fb/wiper.log

Tue Jan  7 06:00:02 PST 2014

Directory disk usage:
/frames/trend/minute_raw 385289132k
/frames/trend/second 100891124k
/frames/full 12269554048k
/frames/trend/minute 1906772k
Combined 12757641076k or 12458633m or 12166Gb

/frames size 13460088620k at 94.78%
/frames is below keep value of 95.00%
Will not delete any files
df reported usage 97.72%
controls@fb ~ 0$

So now I'm wondering if something else has been filling up the frames today.  Has anything changed today that might cause more data than usual to be written to frames?

I'm manually running the wiper script now to clear up some /frames.  Hopefully that will solve the problem temporarily.

ELOG V3.1.3-