40m QIL Cryo_Lab CTN SUS_Lab TCS_Lab OMC_Lab CRIME_Lab FEA ENG_Labs OptContFac Mariner WBEEShop
  40m Log  Not logged in ELOG logo
Entry  Thu Aug 17 22:54:49 2017, ericq, Update, Computers, Trying to access JetStor RAID files 
    Reply  Tue Aug 22 15:17:19 2017, ericq, Update, Computers, Old frames accessible again 
       Reply  Tue Aug 22 15:40:06 2017, gautam, Update, Computers, Old frames accessible again  
          Reply  Tue Aug 22 17:11:15 2017, gautam, Update, Computers, c1iscex model restarts 
             Reply  Tue Aug 22 18:36:46 2017, gautam, Update, Computers, All FE models compiled against RCG3.4 CDS_Aug22.png
                Reply  Fri Sep 1 16:55:27 2017, gautam, Update, Computers, Testpoints now accessible again 
Message ID: 13240     Entry time: Tue Aug 22 15:40:06 2017     In reply to: 13239     Reply to this: 13242
Author: gautam 
Type: Update 
Category: Computers 
Subject: Old frames accessible again  

[jamie, gautam]

I had some trouble getting the daqd processes up and running again using Jamie's instructions.

With Jamie's help however, they are back up and running now. The problem was that the mx infrastructure didn't come back up on its own. So prior to running sudo systemctl restart daqd_*, Jamie ran sudo systemctl start mx. This seems to have done the trick.

c1iscey was still showing red fields on the CDS overview screen so Jamie did a soft reboot. The machine came back up cleanly, so I restarted all the models. But the indicator lights were still red. Apparently the mx processes weren't running on c1iscey. The way to fix this is to run sudo systemctl start mx_stream. Now everything is green.

Now we are going to work on trying the fix Rolf suggested on c1iscex.


It turns out the problem was just a bent pin on the SCSI cable, likely from having to stretch things a bit to reach optimus from the RAID unit.frown

I hooked it up to megatron, and it was automatically recognized and mounted. yes

I had to turn off the new FB machine and remove it from the rack to be able to access megatron though, since it was just sitting on top. FB needs a rail to sit on!

At a cursory glance, the filesystem appears intact. I have copied over the achived DRFPMI frame files to my user directory for now, and Gautam is going to look into getting those permanently stored on the LDAS copy of 40m frames, so that we can have some redundancy.

Also, during this time, one of the HDDs in the RAID unit failed its SMART tests, so the RAID unit wanted it replaced. There were some spare drives in a little box directly under the unit, so I've installed one and am currently incorporating it back into the RAID.

There are two more backup drives in the box. We're running a RAID 5 configuration, so we can only lose one drive at a time before data is lost.


ELOG V3.1.3-