40m QIL Cryo_Lab CTN SUS_Lab CAML OMC_Lab CRIME_Lab FEA ENG_Labs OptContFac Mariner WBEEShop
  40m Log, Page 54 of 357  Not logged in ELOG logo
ID Date Author Type Categoryup Subject Text Attachments
  11053   Fri Feb 20 12:08:10 2015 ericqUpdateCDSBad CDS behaviorI've been able to get all models running.
Most optics are damped, but I'm having
trouble with the ITMs, BS and PRM. 
  
  11054   Fri Feb 20 12:29:01 2015 ericqUpdateCDSAll optics dampedI noticed some diagnostic bits in the c1sus
IOP complaining about user application timing
and FIFO under/overflow (The second and fourth
  
  11055   Fri Feb 20 14:44:47 2015 ericqUpdateCDSBad CDS behaviorIn an effort to ease the IO load on the
framebuilder, I've cleaned up the DQ
channels being written to disk. The biggest
  
  11063   Wed Feb 25 04:21:58 2015 ericqUpdateCDSSome model updatesI changed the suspension library block
to acquire the SUS_[optic]_LSC_OUT channels
at 16k for sensing matrix investigations.
  
  11103   Thu Mar 5 19:13:17 2015 ranaUpdateCDSmxStream all restart at ~7:10 pm    
  11130   Tue Mar 10 20:08:23 2015 ericqUpdateCDScdsRampMuxMatrix Backported to 40m RCGI have successfully backported the cdsRampMuxMatrix
part for use in our RCG v2.5 system.
This involved grabbing new files, merging
  
  11245   Fri Apr 24 21:31:20 2015 KojiSummaryCDSautomatic mxstream resettingWe were too much annoyed by frequent stall
of mxstream. We'll update the RCG when
time comes (not too much future).
  
  11247   Sat Apr 25 00:20:16 2015 ranaUpdateCDSmegatron python autoMC cronUpgraded python on megatron. Added lines
to the crontab to run autoMX.py. Edited crontab
to have a PYTHONPATH so that it can run .py
  
  11250   Sat Apr 25 22:17:49 2015 ranaUpdateCDSMXstream restart script working (beta)Since python from crontab seemed intractable, I replaced autoMX.py   
  11280   Mon May 11 13:21:25 2015 manasaUpdateCDSc1lsp and c1sup not runningI found the c1lsp and
c1sup models not running anymore on c1lsc
  
  11282   Mon May 11 14:08:19 2015 manasaUpdateCDSc1lsp and c1sup removed?I just found out that c1lsp and c1sup models
no more exist on the FE status medm screens.
I am assuming some changes were done to the
  
  11285   Tue May 12 08:51:08 2015 ericqUpdateCDSc1lsp and c1sup removed?


  
  11293   Sat May 16 20:37:09 2015 ranaHowToCDSBypassing the CDSUTILS prefix issueThe CDSUTILS package
has a feature where it substitutes in a C1
  
  11302   Mon May 18 16:56:12 2015 ericqHowToCDSBypassing the CDSUTILS prefix issue


  
  11304   Mon May 18 17:44:30 2015 ranaHowToCDSBypassing the CDSUTILS prefix issueToo weird. I undid me changes. We'll
have to make the C1: stuff work inside each
python script.
  
  11312   Tue May 19 17:03:34 2015 KojiUpdateCDSMXstream restart script working (beta)AutoMX is resetting mx_stream every 5 minutes.
Basically everytime AutoMX is called,
it resets mx_stream. Is the mx_stream
  
  11314   Tue May 19 18:38:33 2015 ranaUpdateCDSMXstream restart script working (beta)Good catch; that was some seriously bad
programming on my part. I had some undeclared
variable garbage going on. I fixed it and
  
  11340   Mon Jun 1 10:26:53 2015 ericqUpdateCDSRCG Upgrade ImminentWe
are planning on upgrading the 40 CDS system
to the latest version of the LIGO RCG software,
  
  11342   Mon Jun 1 20:05:36 2015 ranaUpdateCDSRCG Upgrade Imminent

  
  11347   Mon Jun 8 15:51:31 2015 ericqUpdateCDSRCG DiagnosticsI've started making some model changes
for RCG diagnostic tests. 

I put some blocks down in C1TST
 newTST.png 
  11358   Mon Jun 15 11:54:44 2015 ericqUpdateCDSParts not in SVNI ran the following command to find which
models/parts are not under version control,
or have modifications not commited:
  
  11365   Fri Jun 19 03:00:56 2015 ericqUpdateCDSLibrary Model Parts examinedAll simulink diagrams being used at the
40m are now under version control. I have
compiled, installed, and restarted all current
  
  11380   Fri Jun 26 23:18:52 2015 Eve ChaseUpdateCDSSummary Page UpdatesMotivation:

My SURF project largely focuses
on updating and improving the 40m summary
  
  11384   Tue Jun 30 11:33:00 2015 JamieSummaryCDSprepping for CDS upgradeThis is going to be a big one.  We're
at version 2.5 and we're going to go
to 2.9.3.
  
  11390   Wed Jul 1 19:16:21 2015 JamieSummaryCDSCDS upgrade in progressThe CDS upgrade is now underway

Here's
what's happened so far:


Installed and linked in
  
  11393   Tue Jul 7 18:27:54 2015 JamieSummaryCDSCDS upgrade: progress!After a couple of days of struggle, I made
some progress on the CDS upgrade today:

 2.9-RTS-OK.pngC1X04_GDS_TP.png 
  11394   Tue Jul 7 23:26:19 2015 KojiUpdateCDSAttempt to list CDS issuesAs Jamie succeded to realize somewhat workable
condition of the 40m CDS, I tried to list
the obvious CDS issues so that we can attack
 burt.log.zip 
  11396   Wed Jul 8 20:37:02 2015 JamieSummaryCDSCDS upgrade: one step forward, two steps backAfter determining yesterday that all the
daqd issues were coming from the frame writing,
I started to dig into it more today. 
  
  11397   Wed Jul 8 21:02:02 2015 JamieSummaryCDSCDS upgrade: another step forward, so we're back to where we started (plus a bit?)Koji did a bit of googling to determine
that 'Wrong Network' status message
could be explained by the fb myrinet 
  
  11398   Thu Jul 9 13:26:47 2015 JamieSummaryCDSCDS upgrade: new mx 1.2.16 installedI rebuilt/installed mx 1.2.16 to use "ether-mode",
instead of the default MX-10G:


controls@fb
  
  11400   Thu Jul 9 16:50:13 2015 JamieSummaryCDSCDS upgrade: if all else fails try throwing metal at the problemI roped Rolf into coming over and adding
his eyes to the problem.  After much
discussion we couldn't come up with any
  
  11402   Mon Jul 13 01:11:14 2015 JamieSummaryCDSCDS upgrade: current assessmentdaqd is still behaving unstably. 
It's still unclear what the issue is.

The current failures look like
  
  11404   Mon Jul 13 18:12:50 2015 JamieSummaryCDSCDS upgrade: left running in semi-stable configurationI have been watching daqd all day and I
don't feel particularly closer to understanding
what the issues are.  However, things
  
  11406   Tue Jul 14 09:08:37 2015 JamieSummaryCDSCDS upgrade: left running in semi-stable configurationOvernight daqd restarted itself only about
twice an hour, which is an improvement:


controls@fb /opt/rtcds/caltech/c1/target/fb
  
  11408   Tue Jul 14 10:28:02 2015 ericqSummaryCDSCDS upgrade: left running in semi-stable configurationThere remains a pattern to some of the
restarts, the following times are all
reported as restart times. (There are others
  
  11409   Tue Jul 14 11:57:27 2015 jamieSummaryCDSCDS upgrade: left running in semi-stable configuration


  
  11410   Tue Jul 14 13:55:28 2015 jamieUpdateCDSrunning test on daqd, please leave undisturbedI'm running a test with daqd right
now, so please do not disturb for the moment.

I'm
temporarily writing frames into a tempfs,
  
  11412   Tue Jul 14 16:51:01 2015 JamieSummaryCDSCDS upgrade: problem is not disk accessI think I have now determined once and
for all that the daqd problems are NOT due
to disk IO contention.
  
  11413   Tue Jul 14 17:06:00 2015 jamieUpdateCDSrunning test on daqd, please leave undisturbedI have reverted daqd to the previous configuration,
so that it's writing frames to disk. 
It's still showing instability.
  
  11415   Wed Jul 15 13:19:14 2015 JamieSummaryCDSCDS upgrade: reducing mx end-points as last ditch effortI tried one last thing, suggested by Keith
and Gerrit.  I tried reducing the number
of mx end-points on fb to zero, which should
  
  11417   Wed Jul 15 18:19:12 2015 JamieSummaryCDSCDS upgrade: tentative stabilty?Keith Thorne provided his eyes on the situation
today and had some suggestions that might
have helped things
  
  11427   Sat Jul 18 15:37:19 2015 JamieSummaryCDSCDS upgrade: current statusSo it appears we have found a semi-stable
configuration for the DAQ system post upgrade:

 cds-good.pngsus-damped.png 
  11428   Sat Jul 18 16:03:00 2015 jamieUpdateCDSEPICS freezes persistI notice that the periodic EPICS freezes
persist.  They last for 5-10 seconds. 
MEDM completely freezes up, but then it comes
  
  11429   Sat Jul 18 16:59:01 2015 jamieUpdateCDSunloaded, turned off loading of, symmetricom kernel module on fbfb has been loading a 'symmetricom'
kernel module, presumably because it was
once being used to help with timing. 
  
  11435   Wed Jul 22 14:10:04 2015 ericqUpdateCDSRCG DiagnosticsNow that we've seemed to landed on
a working configuration, I've re-ran
the tests first described in ELOG
 filterShapes.pngCDS_diag_codes.zip 
  11461   Wed Jul 29 21:40:39 2015 KojiSummaryCDSStatus of the frame data syncingThe trend data hasn't been
synced with LDAS since Jul 27 5AM local.

40m:
  
  11465   Thu Jul 30 11:47:54 2015 KojiSummaryCDSStatus of the frame data syncingToday it was synced at 5AM but
that was all.

40m:
  
  11479   Wed Aug 5 10:56:07 2015 ericqUpdateCDSMany models crashed Last night around 1AM, many of the the
frontend models crashed due to an ADC timeout.
(But none of the IOPs, and all the c1lsc
 crashes.png 
  11551   Tue Sep 1 02:44:44 2015 KojiSummaryCDSc1oaf, c1mcs modified for the IMC angular FF[Koji, Ignacio]

In order to allow us to work on
the IMC angular FF, we made the signal paths
 c1oaf_screenshot1.pngc1oaf_screenshot2.pngc1mcs_screenshot.pngOAF_MEDM1.pngOAF_MEDM2.png 
  11564   Thu Sep 3 02:12:08 2015 ranaUpdateCDSSimulink Webview updatedBack in 2011, JoeB wrote
some entries on how to automatically
update the Simulink webview stuff.
  
ELOG V3.1.3-