40m QIL Cryo_Lab CTN SUS_Lab TCS_Lab OMC_Lab CRIME_Lab FEA ENG_Labs OptContFac Mariner WBEEShop
  40m Log, Page 279 of 344  Not logged in ELOG logo
ID Date Author Type Category Subjectup Text Attachments
  14912   Mon Sep 30 11:20:43 2019 gautamUpdateCDSc1iscaux testing - CM board code updatedDATED, SEE   CMsoftTest.png 
  15423   Mon Jun 22 17:51:50 2020 gautamUpdateCDSc1iscaux was downThe machine needed a hard reboot as it
was un-ssh-able. 

The exact time that the machine
  
  4570   Tue Apr 26 22:56:01 2011 kiwamuUpdateLSCc1iscaux2 and c1iscaux restratedWhile checking whitening filters on the
LSC rack, I found some epics controls for
the whitening looked not working.
  
  5992   Wed Nov 23 22:58:33 2011 KojiUpdateGeneralc1iscaux2 is back (re: recovery from the power shutdown)Keying again did not help to solve the
issue. I turned off the power at the back
of the crate, and turn it on again. 
  
  6598   Thu May 3 17:15:38 2012 KojiUpdateLSCc1iscaux2 rebooted/burtrestored[Jenne/Den/Koji]
We saw some white boxes on the LSC
screens.
  
  5979   Tue Nov 22 18:15:39 2011 jamieUpdateCDSc1iscex ADC found dead. Replaced, c1iscex working againc1iscex has not been running for a couple
of days (since the power shutdown at least).
I was assuming that the problem was recurrence
  
  8088   Fri Feb 15 15:21:07 2013 JamieUpdateComputersc1iscex IO-chassis deadI appears that the c1iscex IO-chassis is
either dead or in a very bad  state. 
The PCIe interface card in the IO-chassis
  
  8109   Tue Feb 19 15:10:02 2013 JamieUpdateCDSc1iscex alive againc1iscex is back up.  It is communicating
with it's IO chassis, and all of it's models
(c1x01, c1scx, c1spx) are running again.
  
  4179   Thu Jan 20 18:20:55 2011 josephbUpdateCDSc1iscex computer and c1sus computer swappedSince
the 1U sized computers don't have enough
slots to hold the host interface board, RFM
  
  4478   Thu Mar 31 19:58:11 2011 kiwamuUpdateCDSc1iscex crashedAfter
I did several things to add new DAQ channels
on c1iscex it suddenly became out of network.
  
  8126   Thu Feb 21 12:56:38 2013 JenneUpdateCDSc1iscex dead againc1iscex is dead again.  Red lights,
no "breathing" on the FE status
screen.
  
  6194   Thu Jan 12 23:19:56 2012 KojiUpdateSUSc1iscex is fine nowc1iscex
is working as before and the optic is damped.

What I checked
  
  3965   Mon Nov 22 17:48:11 2010 josephbUpdateCDSc1iscex is not seeing its Binary Output cardProblem:
c1iscex does not even see its 32
channel Binary output card.  This means
  
  3926   Mon Nov 15 16:26:46 2010 josephbUpdateCDSc1iscex is now running and the network hasn't diedProblem:
c1iscex was spamming the network
with error messages.
  
  13242   Tue Aug 22 17:11:15 2017 gautamUpdateComputersc1iscex model restarts[jamie, gautam]

We tried to implement the fix
that Rolf suggested in order to solve (perhaps
  
  13135   Mon Jul 24 10:45:23 2017 gautamUpdateCDSc1iscex models diedThis morning, all the c1iscex
models were dead. Attachment #1
 c1iscexFailure.png 
  13136   Mon Jul 24 10:59:08 2017 JamieUpdateCDSc1iscex models died


  
  8128   Thu Feb 21 14:32:02 2013 JamieUpdateCDSc1iscex models restarted

    

       
            
  
  5060   Fri Jul 29 12:39:26 2011 jamieUpdateCDSc1iscex mysteriously crashedc1iscex was behaving very strangely this
morning.  Steve earlier reported that
he was having trouble pulling up some channels
  
  9000   Mon Aug 12 21:27:03 2013 manasaUpdateCDSc1iscex needs helpI started to modify the c1asx model to
reduce the RFM model from hitting its max
time.
  
  9002   Tue Aug 13 07:40:53 2013 SteveUpdateCDSc1iscex needs help 
 Sorrensen ps ouput of +15V
at rack 1X9 was current limited to 10.3V
 c1iscexSick.png 
  1126   Mon Nov 10 11:32:49 2008 robUpdateComputersc1iscex rebooted
it was running a few cycles late
  
  10642   Mon Oct 27 22:19:17 2014 JenneUpdateCDSc1iscex restartedI'm not sure why, but c1iscex did not want
to do an mxstream restart.  It would
complain at me that  "* ERROR: 
  
  4020   Tue Dec 7 16:09:53 2010 josephbUpdateCDSc1iscex statusI swapped out the IO chassis which could
only handle 3 PCIe cards with the another
chassis which has space for 17, but which
  
  7963   Wed Jan 30 13:50:27 2013 JenneUpdateComputersc1iscex still down[Koji, Jenne]
We noticed that the iscex computer
is still down, but the IOP is (was) running. 
  
  7970   Thu Jan 31 10:23:39 2013 JamieUpdateComputersc1iscex still down

    

       
            
  
  9433   Mon Dec 2 16:04:47 2013 JamieUpdateCDSc1iscex timing problem mysteriously disappears??? (thanksgiving miracle???)

    

       
            
 festatus.png 
  9434   Mon Dec 2 17:05:13 2013 JenneUpdateCDSc1iscex timing problem mysteriously disappears??? (thanksgiving miracle???)Steve was trying to do something to it
this morning, but I'm not exactly clear on
what it was.  Maybe that helped? 
  
  9435   Tue Dec 3 07:42:23 2013 SteveUpdateCDSc1iscex timing problem mysteriously disappears??? (thanksgiving miracle???)

    

       
            
  
  13079   Sun Jun 25 22:30:57 2017 gautamUpdateGeneralc1iscex timing troublesI saw that the CDS overview screen indicated
problems with c1iscex (also ETMX was erratic).
I took a closer look and thought it might
 c1iscex_status.png 
  13081   Mon Jun 26 22:01:08 2017 KojiUpdateGeneralc1iscex timing troublesI tried a couple of things, but
no fundamental improvement of the missing
LED light on the timing board.
  
  13085   Wed Jun 28 20:15:46 2017 gautamUpdateGeneralc1iscex timing troubles[Koji, gautam]

Here
is a summary of what we did today to fix
 CDS_status_28Jun2017.png 
  11778   Wed Nov 18 10:10:53 2015 ericqUpdateCDSc1iscey IO chassis missing bracketsSteve and I inadvertently discovered
that the c1iscey IO chassis doesn't
have brackets to secure the cards where the
 IMG_1505.jpgIMG_1506.jpg 
  11789   Thu Nov 19 15:16:24 2015 ericqUpdateCDSc1iscey IO chassis now has brackets[Steve, ericq]

Brackets for the c1iscey IO
chassis cards have been installed. Now, I
  
  4773   Tue May 31 15:45:37 2011 JamieUpdateCDSc1iscey IOchassis powered off for some reason. repowered.We found that both of the c1iscey models
(c1x05 and c1scy) were unresponsive, and
weren't coming back up even after reboot. 
  
  5791   Wed Nov 2 21:49:59 2011 KatrinUpdateCDSc1iscey computer died againwhile I was not doing anything on the machine.   
  6002   Thu Nov 24 15:27:15 2011 kiwamuUpdateCDSc1iscey hardware rebootedThe
c1iscey machine crashed around 1:00 AM last
night and I did a hard-ware reboot
  
  2551   Thu Jan 28 09:14:51 2010 AlbertoConfigurationComputersc1iscey, c1iscex, c1lsc, c1asc rebootedThis morning the LSC scripts wheren't running
properly. I had to reboot c1iscey, c1iscex,
c1lsc, c1asc .
  
  7550   Mon Oct 15 20:45:58 2012 jamieUpdateIOOc1lsc DAC0 now connected to tip-tilt SOS DW boardsThe tip-tile SOS dewhite/AI boards are
now connected to the digital system.
 20121015_190340.png 
  7553   Tue Oct 16 00:08:26 2012 DenUpdateIOOc1lsc DAC0 now connected to tip-tilt SOS DW boards

    

       
            
 DSC_4748.JPG 
  7561   Tue Oct 16 20:40:06 2012 DenUpdateIOOc1lsc DAC0 now connected to tip-tilt SOS DW boards  Full cable path from coil driver
to osem input is now ready. I've tested Ch1-4
of the left AI and left coil driver. 15 pin
 DSC_4749.JPGDSC_4755.JPG 
  12309   Mon Jul 18 18:44:52 2016 varunUpdateCDSc1lsc FE recoveredc1lsc FE is up and running.

Details:

2) The machine was manually rebooted.
 CDS_status160718.png 
  12303   Thu Jul 14 23:38:59 2016 varunUpdateCDSc1lsc FE unresponsiveToday, at around 10:30, c1lsc machine froze
and stopped responding to ping and ssh after
I compiled and restarted c1daf. I think it
  
  12304   Fri Jul 15 12:21:28 2016 varunUpdateCDSc1lsc FE unresponsivec1lsc is up and running, Eric did a manual
reboot today.

  
  5610   Tue Oct 4 07:51:36 2011 steveUpdateCDSc1lsc and c1sus are still down 
 c1lsc and c1sus are still down.
Only ETMX and ETMY are damped
  
  5607   Mon Oct 3 20:47:51 2011 kiwamuUpdateCDSc1lsc and c1sus didn't run[Mirko / Jenne / Kiwamu]
Just a quick update. All the realtime
processes on the c1lsc and c1sus
  
  5608   Mon Oct 3 21:20:30 2011 JenneUpdateCDSc1lsc and c1sus didn't run[Jenne, Mirko, Kiwamu, Koji, and Jamie
by phone]
We just got off the phone with Jamie. 
  
  5611   Tue Oct 4 10:35:08 2011 MirkoUpdateCDSc1lsc and c1sus running again[Alex, Mirko]
Alex fixed the computers this morning.
It was in fact a dolphin problem:
  
  4706   Thu May 12 23:12:40 2011 kiwamuUpdateCDSc1lsc crashedThis
is my third time to crash a real-time machine.
This time I crashed c1lsc.
  
  13942   Mon Jun 11 18:49:06 2018 gautamUpdateCDSc1lsc dead againWhy is this happening so frequently now?
Last few lines of error log:

 36.png 
ELOG V3.1.3-