40m QIL Cryo_Lab CTN SUS_Lab TCS_Lab OMC_Lab CRIME_Lab FEA ENG_Labs OptContFac Mariner WBEEShop
  40m Log, Page 177 of 344  Not logged in ELOG logo
ID Date Author Typeup Category Subject
  7074   Wed Aug 1 22:37:21 2012 JenneUpdateASSFilters installed in the ASS

As part of trying to figure out what is going on with the ASS, I wanted to figure out what filters are installed on which lockins.

Each "DoF"(1-6) has a zpk(0.1,0.0001,1)gain(1000), which is a lowpass with 60dB of gain at DC, and unity gain at high frequencies.

For the lockins, since there are so many, I made a spreadsheet to keep track of them (attached). 

So, what's the point?  The point is, I think that all of the LOCKIN_I filter modules should be the same, with a single low pass filter.  The Q filter banks don't matter, since we don't use those signals, and the signals are grounded inside the model.  The phase of each lockin was / should be tuned such that all of the interesting signal goes to I, and nothing goes to Q.  The SIG filter modules seem okay, in that they're all the same, except for their band pass frequency.  I just need to check to see what frequency the ASS scripts are trying to actuate at, to make sure we're bandpassing the correct things.

 

Attachment 1: ASS_lockin_filters_1Aug2012.pdf
ASS_lockin_filters_1Aug2012.pdf ASS_lockin_filters_1Aug2012.pdf
  7075   Thu Aug 2 00:43:55 2012 JenneUpdateASSMajor cleanup of the ASS model

[Jamie, Jenne]

We are trying to figure out what the story is with the ASS, and in order to make it more human parse-able, we cleaned up the c1ass.mdl. 

So far, we have made no changes to how the signals are routed.  The local oscillators from each lockin still get summed, and go directly to the individual optics, and the demodulated signals from each lockin go through the sensing matrix, the DoF filters, then the control output matrix, and then on to the individual optics.  So far, so good.

Much of the cleanup work involved making a big library part, which is then called once for PIT and once for YAW in the ass top level, rather than have 2 code-copies, which give Jamie conniptions.  Inside the library part GoTo and From tags were used, rather than having all the lines cross over one another in a big spaghetti mess.

One of the big actual changes to the ass was some name changes. Rather than having mysterious "ASS-LOCKIN(1-30)", they are now named something like "ASS-PIT_LOCKIN_ETMY_TRY", indicating that this is in the pitch set, actuating on ETMY, and looking at TRY for the demodulated signal.  The "DOF" channels are similar to what they were, although we would like to change them in the future.....more on this potential change later.  Previously they were "ASS-DOF(1-10)", but now they are "ASS-PIT_DOF(1-5)" and "ASS-YAW_DOF(1-5)". This channel naming, while it makes things make more sense, and is easier to understand, means that all of the ASS scripts need to be fixed.  However, they all needed updating / upgrading anyway, so this isn't the end of the world.

This channel name fixing also included updating names of IPC (shmem/dolphin/rfm things) blocks, which required matching changes in the SUS, RFM and LSC models.  All 4 models (ASS, SUS, RFM, LSC) were recompiled and installed.  They all seem fine, except there appears to be a dolphin naming mismatch between OAF and SUS that showed up when the SUS was recompiled, which presumably it hadn't been in a while.  We need to figure this out, but maybe not tonight.  Den, if you have time, it would be cool if you could take a look at the OAF and SUS models to make sure the names match when sending signals back and forth.

______________________

We also had a long chat about the deeper meaning of the ASS. 

What should we be actuating on, and what should we be sensing?  A potential thought is to rename our DOF channels to actual DoF names: input axis translation, input axis angle, cavity axis translation, cavity axis angle.  Then actuate the dither lines on a cavity degree of freedom, sense the influence on TRX, TRY and an LSC PD (as is currently done), then actuate on the cavity degree of freedom.

Right now, it looks like the actuation is for individual optics, the sensing is the influence on TRX, TRY and an LSC PD, then actuate on a cavity degree of freedom.  So the only change with the new idea is that we actuate in the DoF basis, not the optics basis.  So the Lockin local oscillators would go through the control output matrix.  This makes more sense in my head, but Jamie and I wanted to involve more people in the conversation before we commit. 

The next question would be: how do we populate the control output matricies?  Valera (or someone) put something in there, but I couldn't find anything on the elog about how those values were measured/calculated/came-to-be.  Any ideas?  If we want to dither and then push on isolated degrees of freedom, we need to know how much moving of which optics affects each DoF.  Is this something we should do using only geometry, and our knowledge of optic placements and relative distances, or is this measurable?

  7076   Thu Aug 2 03:06:57 2012 SashaUpdateSimulationsLS Plant (LSP) is officially ONLINE

My ls plant compiled!! The RCG code can now be found in /opt/rtcds/rtscore/tags/advLigoRTS-2.5. I uploaded a copy of c1lsp.mdl onto the svn.

The weird "failed to connect" error was due to the fact that I named my inputs the same thing as my goto/from tags, so the RCG got confused. Once I renamed my inputs, it worked! I'm not sure what happened to the original "not enough parts" error; it didn't appear a single time during the rebuilding process. Anyway, I made the PDH block much neater, though the lines between PDH and ADC are looking wonky (this is purely an aesthetic problem, not a "oh god my simulation will DIE right now if I don't fix it" problem). I'll fix it in the morning; screenshot attached!

The original c1lsp was kind of sad. I updated it extensively and brought it into the modern era with color. The original c1lsp.mdl should also be on the svn. Tommorow, I'll get started on figuring out how to get LIGO specific noises from white noise.

Attachment 1: Screenshot-1.png
Screenshot-1.png
  7077   Thu Aug 2 04:58:00 2012 MashaUpdatePEM70 Meter Long Guralp 1 Cable

The parts Jenne and I ordered arrived today, so we made a long cable for Guralp 1 using a 24 + 1 wire 70 meter long cable, a female 37-pin DSub, and a 26-pin milspec. The pin map is the same as the one I specified in my previous E-log. I soldered both the milspec attachment and the DSub attachment, and used a Multimeter to check the connectivity of the cables. 20 of 20 connections worked (beeped), so I plugged  the cable into the Gurlap 1 seismometer and the Guralp box.

The time series comparison for the two cables

Old cable:

BeforeCable.png

New cable: (I had to move GUR 1, so it's still stabilizing in the X and Y time series)

 

 AfterCable.pngNew

The current signal spectrum

 

 NewCableFreq.png

The BLRMS on the seismic strip also look similar using the two cables - it's more visible on the wall, but I will include a StripTool picture:

New Cable BLRMS (similar to old cable BLRMS)

 NewCableStrip.png

  7079   Thu Aug 2 21:40:55 2012 JamieUpdateSimulationsETMX simplant model revived

I revived the ETMX simplant model, c1spx.  It's running on cpu4 on c1iscex, and interfaces with C1scx via SHMEM.

The channel names for the simplant suspensions will be SUP, so the channel from this model will C1:SUP-ETMX_.

Next I'll try to get the ITMX and LSC ("LSP") simplant models running so we can run a "full" cavity simulation.

Sasha has been working on LSP, so we should be ready to do something with that soon.  In the mean time she's going to fix up the SPX MEDM screens, since some channel names have been changed since it was last run.

  7081   Thu Aug 2 23:31:04 2012 JenneUpdateASSMajor cleanup of the ASS model

Jamie re-redid the ASS model a few hours ago.

I have just compiled it, and restarted c1ass.  (The model from last night is currently called c1ass3.mdl)  I had to delete and re-put inthe goto and from tags for the LSC signal coming in from the shmem.  For some reason, it kept claiming that the inputs using the from tags were not connected, even when I redid the connections.  Finally deleting and dragging in new goto and from tags made the model happy enough to compile.  Whatever.  I'm going to let Jamie do the svn-ing, since he's the one who made the changes.  Before I had figured out that it was the tags, I was concerned that the shmem was unhappy, so there was no signal connecting to the input of the goto tag, and that was somehow bad....anyhow, I recompiled the LSC model to re-create the shmem sender, but that had no effect, since that wasn't the problem.

The change from last night is that now the library parts are by DoF.  There is only one matrix in each library part, before the servo filters. Now we can DC-actuate on a single mode (ETM or ITM, pitch or yaw), and see how it affects all 4 sensors (the demodulated signals from the lockins). We need to measure the sensing matrix to go from the several sensors to the servo input.

  7082   Fri Aug 3 03:24:13 2012 JenneUpdateASSNew ASS screens

I wanted to try out the ASS tonight, but I wanted some kinds of screens thrown together so I would know what I was doing.  Turns out screens take longer than I thought.  Am I surprised? Not really. 

They're probably at the ~85% mark now, but I should be able to try out the ASS tomorrow I think.

  7083   Fri Aug 3 13:05:28 2012 DenUpdatePEMshims

As we do not have legs for Trillium, I was advised to use shims to adjust the levels. However, they produce extra resonance at ~30 Hz + harmonics. Coherence is lost at these frequencies.

shims.png

  7084   Fri Aug 3 14:52:11 2012 JenneUpdatePEMshims

Quote:

As we do not have legs for Trillium, I was advised to use shims to adjust the levels. However, they produce extra resonance at ~30 Hz + harmonics. Coherence is lost at these frequencies.

 Brian Lantz / Dan Clark are looking around their lab to see if they forgot to ship the feet with the T-240.  They had taken the feet off to put it in a pod. 

  7085   Sat Aug 4 17:32:31 2012 DenUpdatedigital noisefilter checker

The script estimates digital noise produces by online filters. First version of Matlab files and complied c files are in scripts/digital_noise directory.

Algorithm for 1 filter bank (max number of filters = 10):

  1.        extract sos - representation from Foton file for each filter (Matlab)
  2.        download data from corresponding DQ channel using NDS (Matlab)
  3.        find filters that are switched on (Matlab)
  4.        filter signal using Df2 and BQF with single and double precision (C)
  5.        estimate digital filter noise (Matlab)
  6.        calculate power spectral density and plot the result (Matlab)

More details on (2)

Often DQ channels have reduced sampling rate. In this case the script will upsample data adding zeros.

AI filter is not applied. But in the end only the frequency range (0, DQ RATE / 2) is analyzed.

More details on (3):

This is done by reading C1:MODEL-BANK_NAME_SW1R and C1:MODEL-BANK_NAME_SW2R channels.
 
_SW1R channel value is the sum of the following numbers:
  • input switch ON / OFF => 4 / 0
  • filters 1 - 6 ON /OFF
    • 1 => 48 / 0
    • 2 => 192 / 0
    • 3 => 768 / 0
    • 4 => 3072 / 0
    • 5 => 12288 / 0
    • 6 => 49152 / 0
    • If a switch is ON but there is no corresponding filter (one green and one red line under the switch) then the switch value is divided by 3

_SW2R channel value is the sum of the following numbers:

  • decimation switch ON / OFF => 512 / 0
  • output switch ON / OFF => 1024 / 0
  • filters 7 - 10 ON /OFF
    • 7 => 3 / 0
    • 8 => 12 / 0
    • 9 => 48 / 0
    • 10 => 192 / 0
    • If a switch is ON but there is no corresponding filter (one green and one red line under the switch) then the switch value is divided by 3

Note: as for now Matlab script assumes that input, output and decimation filters are switched ON and there are no turned ON filter switches that do not correspond to any filters

More details on (5)

Digital noise using double precision is estimated by extrapolation of digital noise with single precision. The last is calculated by subtracting outputs of the filters with single and double precision. Then this noise is multiplied by 3 * 10-7.

This extrapolation number was achieved by printf tests of the number 0.123456789012345678 with single and double precision on C. Using type 'float' variables 10 significant numbers show up, using type 'double' - 17.

I also did 'calibration tests' to achieve extrapolation number - signal was filters with an aggresive low-pass filter. At high frequencies filter output spectrum is flat => digital noise amplitude must be the same. The plot shows GUR1_X channel filtered with low-pass chebyshev type 1 filter.

gur1x.png

However, extrapolation number is not the same for all cases. In the following example of analyzing BS_SUSPOS filter bank using extrapolation 3 * 10-7 we get noise that is slightly overestimated. In some other examples we need to take a larger number. But in average, I think, this is a good approximation.

C1SUS_BS_SUSPOS.png

To avoid extrapolation problem we can use long double precision (~19 digits). I was able to do this with gcc compiler. However, in mex compiler using long double in filter calculations, I do not get any better precision then using double precision. I'll think more about it.

  7086   Sun Aug 5 13:48:40 2012 DenUpdateCDSMove to RCG 2.5 tag release

Quote:

I moved the RCG to the advLigoRTS-2.5 tag

 After that RFM -> OAF communication through PCIE became bad again. Inside CommData2.c cache flushing is not allowed

// If PCIE comms show errors, may want to add this cache flushing
            #if 0
            if(ipcInfo[ii].netType == IPCIE)
                clflush_cache_range (ipcInfo[ii].pIpcData->dBlock[sendBlock][ipcIndex].data, 16);
            #endif

As a result, a significant part of MC_F and other signals is lost during RFM -> OAF transmission (270 - 330 out of 2048 per second)

erros.png   overview.png    oaf.png

 


Last time when I replaced 0 for 1, it suspended SUS machine because of the code bug. Alex modified a couple of files in the old version and it started to work. Do you know if this bug is fixed in the new version?

  7087   Mon Aug 6 07:59:33 2012 steveUpdateSUSPRM oplevs servo is still bad

Quote:

Quote:

Quote:

Yuta claims he fixed the PRM oplev by centering it the other day, but no one has left it on and watched it for a long while, to make sure it's okay.  We watched it now for ~2 min, and it was good, but we're leaving the oplevs off anyway for the night.  Tomorrow we should restore PRM (it's currently restored), turn on the oplevs, and let it sit to make sure it doesn't go crazy.

 

 PRM oplev servo was turned on with PITgain 0.5  and YAWgain  -0.7

Note: gain settings were PIT  1.0  and  YAW --0.5   on Jun 1, 2012 that I measured Feb 23, 2012

 It is still oscillating. Gains turned down to zero.

 Earthquake test our suspensions            PRM damping restored.             Oplev servo gains turned to zero.

Attachment 1: PRMeq.png
PRMeq.png
  7088   Mon Aug 6 09:46:31 2012 steveUpdateIOOpoweroutage turns laser off

. Power outage turned off the PSL Innolight laser on Sunday afternoon.  It  was turned back on and  locked happily right on. The green lasers were not effected.

 

CALIFORNIA INSTITUTE OF TECHNOLOGY

                 FACILITIES MANAGEMENT

            UTILITY & SERVICE INTERRUPTION

 

**PLEASE POST**

 

Building:         CAMPUS WIDE     

 

Date:             SUNDAY, AUGUST 6, 2012          

 

Time:             3:41 PM          

 

Interruption:     ELECTRICAL POWER DISTRIBUTION

  

Contact:          MIKE ANCHONDO, X-4999, OR TOM BRENNAN, X-4984      

 

* THIS PAST SUNDAY AFTERNOON ABOUT 3:40 PM, PASADENA WATER AND POWER

 EXPERIENCED A FAULT ON THEIR POWER DISTRIBUTION SYSTEM.  THIS CAUSED

  A SEVERE VOLTAGE SAG WHICH AFFECTED THE CALTECH CAMPUS. THE FAULT WAS

  NOT ON A CALTECH CIRCUIT.

 

(If there is a problem with this Interruption, please notify

the Service Center X-4717 or the above Contact as soon as possible.

If no response is received we will proceed with the interruption.)

        

                        Jerry Thompson,

                        Interim Director of Campus Operations & Maintenance

 

 

  7089   Mon Aug 6 10:53:32 2012 steveUpdateVACstatus: vauum normal

 

 The power outage did not have any effect on the vacuum system.  I had to open VM1 to the RGA because of flaky CC1 cold cathode gauge fluctuations.

We are running on the vertically positioned CC1 again.

Note: rga scans with closed VM1 are back ground scans.

Attachment 1: CCgauges.png
CCgauges.png
  7090   Mon Aug 6 11:07:06 2012 ManasaUpdate40m UpgradingOptical layout updated

ACAD files of the 40m optical layout have been updated as per the vent in Aug 2011.

Files are available at the 40m svn docs-->Upgrade12-->Opt_Layout2011.

 

  7091   Mon Aug 6 16:48:43 2012 steveUpdateSUSSRM oplev is clipping

The SRM oplev beam is clipping.

  7092   Mon Aug 6 17:15:15 2012 JenneUpdateASSFilter banks not working

I was trying to load some filters into the ASS so that I can try it out, but for some reason the filter banks aren't working - clicking the on/off buttons doesn't do anything, filters (which exist in the .txt file generated by foton) don't load.

I've emailed cds-announce to see if anyone has any ideas.

  7093   Mon Aug 6 19:37:50 2012 JamieUpdateCDSdaqd and CDS network problems today

For some reason this afternoon we've been experiencing a lot of problems with the framebuilder, and with the CDS network in general.  The framebuilder has been very unresponsive, although the daqd logs seem to indicate that things are ok.  All models will loose contact with fb for very long stretches.  Attempts to kill/restart daqd don't seem to fix the problem.

These problems seem to be associated with the general CDS network issues as well.  The network seems to become very slow, and the workstations all become very slow.  The later I assume is because of the network and that so much of the work we do is on network mounted filesystems (/opt/rtcds, /ligo, etc.).

My current speculation is that daqd on fb is doing something stupid, like trying to read or write a bunch of stuff from /frames, which is also network mounted, and that clogs up the entire network.  Some serious network debugging is going to be needed to figure out what's going on, though.

I'm afraid daqd is caught in some bad state now, though.  It's not responding to anything, and every attempt to kill it seems to bring it back into the bad state.  Hopefully I can get Alex to help me figure out what's going on tomorrow.   Maybe it will clear up on it's own tonight...

  7094   Mon Aug 6 19:54:53 2012 JamieUpdateCDSdaqd and CDS network problems today

It looks like daqd is indeed caught in some bad state.  It seems to die at some point after making GPS corrections to minute trender:

...
[Mon Aug  6 19:45:13 2012] Minute trender made GPS time correction; gps=1028342727; gps%60=27
tail: `fb/logs/daqd.log' has been replaced;  following end of new file
263596
MX endpoint opened
startup file interpreter thread tid=140334118615312
calling yyparse(5, 6)
[Mon Aug  6 19:50:08 2012] ->5: #set avoid_reconnect
[Mon Aug  6 19:50:08 2012] ->5: set thread_stack_size=102400
[Mon Aug  6 19:50:08 2012] new threads will be created with the stack of size 102400K
[Mon Aug  6 19:50:08 2012] ->5: set allow_tpman_connect_fail
[Mon Aug  6 19:50:08 2012] ->5: #set dcu_status_check=5
[Mon Aug  6 19:50:08 2012] ->5: #set symm_gps_offset=-1
[Mon Aug  6 19:50:08 2012] ->5: #set symm_gps_offset=31535998
[Mon Aug  6 19:50:08 2012] ->5: ##set symm_gps_offset=347155213
[Mon Aug  6 19:50:08 2012] ->5: #set symm_gps_offset=378691215
[Mon Aug  6 19:50:08 2012] ->5: #set symm_gps_offset=378691212
[Mon Aug  6 19:50:08 2012] ->5: #set symm_gps_offset=315964799
[Mon Aug  6 19:50:08 2012] ->5: set symm_gps_offset=315964801
[Mon Aug  6 19:50:08 2012] ->5: set debug=0
[Mon Aug  6 19:50:08 2012] ->5: set log=2
[Mon Aug  6 19:50:08 2012] ->5: set zero_bad_data=0
[Mon Aug  6 19:50:08 2012] ->5: set dcu_status_check=9
[Mon Aug  6 19:50:08 2012] ->5: set controller_dcu=33
[Mon Aug  6 19:50:08 2012] ->5: set master_config="/opt/rtcds/caltech/c1/target/fb/master"
[Mon Aug  6 19:50:10 2012] finished configuring data channels
[Mon Aug  6 19:50:10 2012] ->5: configure channels begin end
Unable to find GDS node 90 system c1x00 in INI files
Unable to find GDS node 92 system c1tst2 in INI files
Unable to find GDS node 95 system c1x10 in INI files
[Mon Aug  6 19:50:10 2012] ->5: tpconfig "/opt/rtcds/caltech/c1/target/gds/param/testpoint.par"
[Mon Aug  6 19:50:10 2012] ->5: set gps_leaps = 820108813
[Mon Aug  6 19:50:10 2012] ->5: set detector_name="CIT"
[Mon Aug  6 19:50:10 2012] ->5: set detector_prefix="C1"
[Mon Aug  6 19:50:10 2012] ->5: set detector_longitude=-90.7742403889
[Mon Aug  6 19:50:10 2012] ->5: set detector_latitude=30.5628943337
[Mon Aug  6 19:50:10 2012] ->5: set detector_elevation=.0
[Mon Aug  6 19:50:10 2012] ->5: set detector_azimuths=1.1,4.7123889804
[Mon Aug  6 19:50:10 2012] ->5: set detector_altitudes=1.0,2.0
[Mon Aug  6 19:50:10 2012] ->5: set detector_midpoints=2000.0, 2000.0
[Mon Aug  6 19:50:10 2012] ->5: set num_dirs = 10
[Mon Aug  6 19:50:10 2012] ->5: set frames_per_dir=225
[Mon Aug  6 19:50:10 2012] ->5: set full_frames_per_file=1
[Mon Aug  6 19:50:10 2012] ->5: set full_frames_blocks_per_frame=16
[Mon Aug  6 19:50:10 2012] ->5: set frame_dir="/frames/full", "C-R-", ".gwf"
[Mon Aug  6 19:50:10 2012] ->5: set trend_num_dirs=10
[Mon Aug  6 19:50:10 2012] ->5: set trend_frames_per_dir=1440
[Mon Aug  6 19:50:10 2012] ->5: set trend_frame_dir= "/frames/trend/second", "C-T-", ".gwf"
[Mon Aug  6 19:50:10 2012] ->5: set raw-minute-trend-dir="/frames/trend/minute_raw"
[Mon Aug  6 19:50:10 2012] ->5: set nds-jobs-dir="/opt/rtcds/caltech/c1/target/fb"
[Mon Aug  6 19:50:10 2012] ->5: set minute-trend-num-dirs=10
[Mon Aug  6 19:50:10 2012] ->5: set minute-trend-frames-per-dir=24
[Mon Aug  6 19:50:10 2012] ->5: set minute-trend-frame-dir="/frames/trend/minute", "C-M-", ".gwf"
[Mon Aug  6 19:50:10 2012] ->5: start main 10
[Mon Aug  6 19:50:12 2012] main started
[Mon Aug  6 19:50:12 2012] ->5: start profiler
[Mon Aug  6 19:50:12 2012] ->5: # comment out this block to stop saving data
[Mon Aug  6 19:50:12 2012] frame saver started
[Mon Aug  6 19:50:12 2012] ->5: start frame-saver
[Mon Aug  6 19:50:13 2012] ->5: sync frame-saver
[Mon Aug  6 19:50:13 2012] ->5: start trender
[Mon Aug  6 19:50:13 2012] trender started
[Mon Aug  6 19:50:13 2012] trend frame saver started
[Mon Aug  6 19:50:13 2012] ->5: start trend-frame-saver
[Mon Aug  6 19:50:14 2012] ->5: sync trend-frame-saver
[Mon Aug  6 19:50:14 2012] minute trend frame saver started
[Mon Aug  6 19:50:14 2012] ->5: start minute-trend-frame-saver
[Mon Aug  6 19:50:14 2012] Done creating ADC structures
[Mon Aug  6 19:50:15 2012] ->5: sync minute-trend-frame-saver
[Mon Aug  6 19:50:15 2012] raw minute trend frame saver started
[Mon Aug  6 19:50:15 2012] ->5: start raw_minute_trend_saver
[Mon Aug  6 19:50:15 2012] ->5: #frame-writer "225.225.225.1" broadcast="131.215.113.0" all
[Mon Aug  6 19:50:15 2012] ->5: #sleep 5
[Mon Aug  6 19:50:15 2012] producer started
[Mon Aug  6 19:50:15 2012] ->5: start producer
[Mon Aug  6 19:50:15 2012] ->5: start epics dcu
[Mon Aug  6 19:50:15 2012] MX receiver thread started
[Mon Aug  6 19:50:15 2012] edcu started
[Mon Aug  6 19:50:15 2012] ->5: start epics server "C0:DAQ-DC0_" "C1:DAQ-DC0_"
[Mon Aug  6 19:50:15 2012] epics server started
[Mon Aug  6 19:50:15 2012] ->5: start listener 8087
[Mon Aug  6 19:50:15 2012] ->5: start listener 8088 1
[Mon Aug  6 19:50:15 2012] ->5: sleep 60
[Mon Aug  6 19:50:15 2012] Epics server started
[Mon Aug  6 19:50:15 2012] EDCU has 2553 channels configured; first=0

[Mon Aug  6 19:50:18 2012] Minute trender made GPS time correction; gps=1028343032; gps%60=32
...

The "tail:..." line indicates that the log was moved and replaced, which indicates a daqd restart.  As far as I know this was not manually triggered.

After the restart the same thing happens again.  About once every five minutes.

  7095   Mon Aug 6 20:08:45 2012 JamieUpdateCDSdaqd and CDS network problems today

When daqd is caught in this state it can not be killed.  It's in "uninterruptable sleep" ('D' state in the top output below).  This usually indicates that it's waiting for the kernel, usually due to some missing or hung IO.

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                                                      
28038 controls  20   0 4430m 2.0g  19m D    0 27.1   0:15.00 daqd                                                                                         

The memory footprint also seems to be getting big.  It's clearly trying to do something stupid that it can't handle.

  7096   Mon Aug 6 20:22:50 2012 JamieUpdateCDSdaqd segfaulting after five minutes

I tried running daqd manually, and sure enough it segfaults after about five minutes (see log below).  I've uncommented it from /etc/inittab on fb and I'm leaving it off for now until we can figure out what's going on.

controls@fb /opt/rtcds/caltech/c1/target/fb 0$ /opt/rtcds/caltech/c1/target/fb/daqd -c /opt/rtcds/caltech/c1/target/fb/daqdrc
263596
MX endpoint opened
startup file interpreter thread tid=139790943115536
calling yyparse(5, 6)
[Mon Aug  6 20:15:27 2012] ->5: #set avoid_reconnect
[Mon Aug  6 20:15:27 2012] ->5: set thread_stack_size=102400
[Mon Aug  6 20:15:27 2012] new threads will be created with the stack of size 102400K
[Mon Aug  6 20:15:27 2012] ->5: set allow_tpman_connect_fail
[Mon Aug  6 20:15:27 2012] ->5: #set dcu_status_check=5
[Mon Aug  6 20:15:27 2012] ->5: #set symm_gps_offset=-1
[Mon Aug  6 20:15:27 2012] ->5: #set symm_gps_offset=31535998
[Mon Aug  6 20:15:27 2012] ->5: ##set symm_gps_offset=347155213
[Mon Aug  6 20:15:27 2012] ->5: #set symm_gps_offset=378691215
[Mon Aug  6 20:15:27 2012] ->5: #set symm_gps_offset=378691212
[Mon Aug  6 20:15:27 2012] ->5: #set symm_gps_offset=315964799
[Mon Aug  6 20:15:27 2012] ->5: set symm_gps_offset=315964801
[Mon Aug  6 20:15:27 2012] ->5: set debug=0
[Mon Aug  6 20:15:27 2012] ->5: set log=2
[Mon Aug  6 20:15:27 2012] ->5: set zero_bad_data=0
[Mon Aug  6 20:15:27 2012] ->5: set dcu_status_check=9
[Mon Aug  6 20:15:27 2012] ->5: set controller_dcu=33
[Mon Aug  6 20:15:27 2012] ->5: set master_config="/opt/rtcds/caltech/c1/target/fb/master"
[Mon Aug  6 20:15:30 2012] finished configuring data channels
[Mon Aug  6 20:15:30 2012] ->5: configure channels begin end
GDS server NODE=19 HOST=c1iscex DCUID=19
GDS server NODE=20 HOST=c1sus DCUID=20
GDS server NODE=21 HOST=c1sus DCUID=21
GDS server NODE=22 HOST=c1lsc DCUID=22
GDS server NODE=25 HOST=c1iscex DCUID=61
GDS server NODE=28 HOST=c1ioo DCUID=28
GDS server NODE=33 HOST=c1ioo DCUID=33
GDS server NODE=34 HOST=c1ioo DCUID=34
GDS server NODE=36 HOST=c1sus DCUID=36
GDS server NODE=38 HOST=c1sus DCUID=38
GDS server NODE=39 HOST=c1sus DCUID=39
GDS server NODE=40 HOST=c1lsc DCUID=40
GDS server NODE=42 HOST=c1lsc DCUID=42
GDS server NODE=45 HOST=c1iscex DCUID=45
GDS server NODE=46 HOST=c1iscey DCUID=46
GDS server NODE=47 HOST=c1iscey DCUID=47
GDS server NODE=48 HOST=c1lsc DCUID=48
GDS server NODE=50 HOST=c1lsc DCUID=50
GDS server NODE=51 HOST=c1ioo DCUID=51
GDS server NODE=60 HOST=c1lsc DCUID=60
GDS server NODE=61 HOST=c1iscex DCUID=61
GDS server NODE=62 HOST=c1sus DCUID=62
Unable to find GDS node 90 system c1x00 in INI files
GDS server NODE=91 HOST=c1lsc DCUID=60
Unable to find GDS node 92 system c1tst2 in INI files
Unable to find GDS node 95 system c1x10 in INI files
TP: node = 19, host = c1iscex, dup = 0, prog = 0x31002013, vers = 1
Initialized TP interface node=19, host=c1iscex
TP: node = 20, host = c1sus, dup = 0, prog = 0x31002014, vers = 1
Initialized TP interface node=20, host=c1sus
TP: node = 21, host = c1sus, dup = 0, prog = 0x31002015, vers = 1
Initialized TP interface node=21, host=c1sus
TP: node = 22, host = c1lsc, dup = 0, prog = 0x31002016, vers = 1
Initialized TP interface node=22, host=c1lsc
TP: node = 25, host = c1iscex, dup = 0, prog = 0x31002019, vers = 1
Initialized TP interface node=25, host=c1iscex
TP: node = 28, host = c1ioo, dup = 0, prog = 0x3100201c, vers = 1
Initialized TP interface node=28, host=c1ioo
TP: node = 33, host = c1ioo, dup = 0, prog = 0x31002021, vers = 1
Initialized TP interface node=33, host=c1ioo
TP: node = 34, host = c1ioo, dup = 0, prog = 0x31002022, vers = 1
Initialized TP interface node=34, host=c1ioo
TP: node = 36, host = c1sus, dup = 0, prog = 0x31002024, vers = 1
Initialized TP interface node=36, host=c1sus
TP: node = 38, host = c1sus, dup = 0, prog = 0x31002026, vers = 1
Initialized TP interface node=38, host=c1sus
TP: node = 39, host = c1sus, dup = 0, prog = 0x31002027, vers = 1
Initialized TP interface node=39, host=c1sus
TP: node = 40, host = c1lsc, dup = 0, prog = 0x31002028, vers = 1
Initialized TP interface node=40, host=c1lsc
TP: node = 42, host = c1lsc, dup = 0, prog = 0x3100202a, vers = 1
Initialized TP interface node=42, host=c1lsc
TP: node = 45, host = c1iscex, dup = 0, prog = 0x3100202d, vers = 1
Initialized TP interface node=45, host=c1iscex
TP: node = 46, host = c1iscey, dup = 0, prog = 0x3100202e, vers = 1
Initialized TP interface node=46, host=c1iscey
TP: node = 47, host = c1iscey, dup = 0, prog = 0x3100202f, vers = 1
Initialized TP interface node=47, host=c1iscey
TP: node = 48, host = c1lsc, dup = 0, prog = 0x31002030, vers = 1
Initialized TP interface node=48, host=c1lsc
TP: node = 50, host = c1lsc, dup = 0, prog = 0x31002032, vers = 1
Initialized TP interface node=50, host=c1lsc
TP: node = 51, host = c1ioo, dup = 0, prog = 0x31002033, vers = 1
Initialized TP interface node=51, host=c1ioo
TP: node = 60, host = c1lsc, dup = 0, prog = 0x3100203c, vers = 1
Initialized TP interface node=60, host=c1lsc
TP: node = 61, host = c1iscex, dup = 0, prog = 0x3100203d, vers = 1
Initialized TP interface node=61, host=c1iscex
TP: node = 62, host = c1sus, dup = 0, prog = 0x3100203e, vers = 1
Initialized TP interface node=62, host=c1sus
TP: node = 91, host = c1lsc, dup = 0, prog = 0x3100205b, vers = 1
Initialized TP interface node=91, host=c1lsc
[Mon Aug  6 20:15:30 2012] ->5: tpconfig "/opt/rtcds/caltech/c1/target/gds/param/testpoint.par"
[Mon Aug  6 20:15:30 2012] ->5: set gps_leaps = 820108813
[Mon Aug  6 20:15:30 2012] ->5: set detector_name="CIT"
[Mon Aug  6 20:15:30 2012] ->5: set detector_prefix="C1"
[Mon Aug  6 20:15:30 2012] ->5: set detector_longitude=-90.7742403889
[Mon Aug  6 20:15:30 2012] ->5: set detector_latitude=30.5628943337
[Mon Aug  6 20:15:30 2012] ->5: set detector_elevation=.0
[Mon Aug  6 20:15:30 2012] ->5: set detector_azimuths=1.1,4.7123889804
[Mon Aug  6 20:15:30 2012] ->5: set detector_altitudes=1.0,2.0
[Mon Aug  6 20:15:30 2012] ->5: set detector_midpoints=2000.0, 2000.0
[Mon Aug  6 20:15:30 2012] ->5: set num_dirs = 10
[Mon Aug  6 20:15:30 2012] ->5: set frames_per_dir=225
[Mon Aug  6 20:15:30 2012] ->5: set full_frames_per_file=1
[Mon Aug  6 20:15:30 2012] ->5: set full_frames_blocks_per_frame=16
[Mon Aug  6 20:15:30 2012] ->5: set frame_dir="/frames/full", "C-R-", ".gwf"
[Mon Aug  6 20:15:30 2012] ->5: set trend_num_dirs=10
[Mon Aug  6 20:15:30 2012] ->5: set trend_frames_per_dir=1440
[Mon Aug  6 20:15:30 2012] ->5: set trend_frame_dir= "/frames/trend/second", "C-T-", ".gwf"
[Mon Aug  6 20:15:30 2012] ->5: set raw-minute-trend-dir="/frames/trend/minute_raw"
[Mon Aug  6 20:15:30 2012] ->5: set nds-jobs-dir="/opt/rtcds/caltech/c1/target/fb"
[Mon Aug  6 20:15:30 2012] ->5: set minute-trend-num-dirs=10
[Mon Aug  6 20:15:30 2012] ->5: set minute-trend-frames-per-dir=24
[Mon Aug  6 20:15:30 2012] ->5: set minute-trend-frame-dir="/frames/trend/minute", "C-M-", ".gwf"
[Mon Aug  6 20:15:30 2012] ->5: start main 10
Allocated move buffer size 11616356 bytes
[Mon Aug  6 20:15:32 2012] main started
[Mon Aug  6 20:15:32 2012] ->5: start profiler
[Mon Aug  6 20:15:32 2012] ->5: # comment out this block to stop saving data
[Mon Aug  6 20:15:32 2012] frame saver started
[Mon Aug  6 20:15:32 2012] ->5: start frame-saver
[Mon Aug  6 20:15:33 2012] ->5: sync frame-saver
[Mon Aug  6 20:15:33 2012] ->5: start trender
[Mon Aug  6 20:15:33 2012] trender started
[Mon Aug  6 20:15:33 2012] trend frame saver started
[Mon Aug  6 20:15:33 2012] ->5: start trend-frame-saver
[Mon Aug  6 20:15:34 2012] ->5: sync trend-frame-saver
[Mon Aug  6 20:15:34 2012] minute trend frame saver started
[Mon Aug  6 20:15:34 2012] ->5: start minute-trend-frame-saver
[Mon Aug  6 20:15:34 2012] Done creating ADC structures
[Mon Aug  6 20:15:35 2012] ->5: sync minute-trend-frame-saver
[Mon Aug  6 20:15:35 2012] raw minute trend frame saver started
[Mon Aug  6 20:15:35 2012] ->5: start raw_minute_trend_saver
[Mon Aug  6 20:15:35 2012] ->5: #frame-writer "225.225.225.1" broadcast="131.215.113.0" all
[Mon Aug  6 20:15:35 2012] ->5: #sleep 5
[Mon Aug  6 20:15:35 2012] producer started
[Mon Aug  6 20:15:35 2012] ->5: start producer
[Mon Aug  6 20:15:35 2012] ->5: start epics dcu
[Mon Aug  6 20:15:35 2012] MX receiver thread started
[Mon Aug  6 20:15:35 2012] edcu started
[Mon Aug  6 20:15:35 2012] ->5: start epics server "C0:DAQ-DC0_" "C1:DAQ-DC0_"
[Mon Aug  6 20:15:35 2012] epics server started
[Mon Aug  6 20:15:35 2012] ->5: start listener 8087
[Mon Aug  6 20:15:35 2012] ->5: start listener 8088 1
[Mon Aug  6 20:15:35 2012] ->5: sleep 60
Creating C1:DAQ-DC0_PEM_SLOW_STATUS
Creating C1:DAQ-DC0_PEM_SLOW_CRC_CPS
Creating C1:DAQ-DC0_PEM_SLOW_CRC_SUM
Creating C1:DAQ-DC0_C1X01_STATUS
Creating C1:DAQ-DC0_C1X01_CRC_CPS
Creating C1:DAQ-DC0_C1X01_CRC_SUM
Creating C1:DAQ-DC0_C1X02_STATUS
Creating C1:DAQ-DC0_C1X02_CRC_CPS
Creating C1:DAQ-DC0_C1X02_CRC_SUM
Creating C1:DAQ-DC0_C1SUS_STATUS
Creating C1:DAQ-DC0_C1SUS_CRC_CPS
Creating C1:DAQ-DC0_C1SUS_CRC_SUM
Creating C1:DAQ-DC0_C1OAF_STATUS
Creating C1:DAQ-DC0_C1OAF_CRC_CPS
Creating C1:DAQ-DC0_C1OAF_CRC_SUM
Creating C1:DAQ-DC0_C1ALS_STATUS
Creating C1:DAQ-DC0_C1ALS_CRC_CPS
Creating C1:DAQ-DC0_C1ALS_CRC_SUM
Creating C1:DAQ-DC0_C1X03_STATUS
Creating C1:DAQ-DC0_C1X03_CRC_CPS
Creating C1:DAQ-DC0_C1X03_CRC_SUM
Creating C1:DAQ-DC0_C1IOO_STATUS
Creating C1:DAQ-DC0_C1IOO_CRC_CPS
Creating C1:DAQ-DC0_C1IOO_CRC_SUM
Creating C1:DAQ-DC0_C1MCS_STATUS
Creating C1:DAQ-DC0_C1MCS_CRC_CPS
Creating C1:DAQ-DC0_C1MCS_CRC_SUM
Creating C1:DAQ-DC0_C1RFM_STATUS
Creating C1:DAQ-DC0_C1RFM_CRC_CPS
Creating C1:DAQ-DC0_C1RFM_CRC_SUM
Creating C1:DAQ-DC0_C1PEM_STATUS
Creating C1:DAQ-DC0_C1PEM_CRC_CPS
Creating C1:DAQ-DC0_C1PEM_CRC_SUM
Creating C1:DAQ-DC0_C1X04_STATUS
Creating C1:DAQ-DC0_C1X04_CRC_CPS
Creating C1:DAQ-DC0_C1X04_CRC_SUM
Creating C1:DAQ-DC0_C1LSC_STATUS
Creating C1:DAQ-DC0_C1LSC_CRC_CPS
Creating C1:DAQ-DC0_C1LSC_CRC_SUM
Creating C1:DAQ-DC0_C1SCX_STATUS
Creating C1:DAQ-DC0_C1SCX_CRC_CPS
Creating C1:DAQ-DC0_C1SCX_CRC_SUM
Creating C1:DAQ-DC0_C1X05_STATUS
Creating C1:DAQ-DC0_C1X05_CRC_CPS
Creating C1:DAQ-DC0_C1X05_CRC_SUM
Creating C1:DAQ-DC0_C1SCY_STATUS
Creating C1:DAQ-DC0_C1SCY_CRC_CPS
Creating C1:DAQ-DC0_C1SCY_CRC_SUM
Creating C1:DAQ-DC0_C1ASS_STATUS
Creating C1:DAQ-DC0_C1ASS_CRC_CPS
Creating C1:DAQ-DC0_C1ASS_CRC_SUM
Creating C1:DAQ-DC0_C1CAL_STATUS
Creating C1:DAQ-DC0_C1CAL_CRC_CPS
Creating C1:DAQ-DC0_C1CAL_CRC_SUM
Creating C1:DAQ-DC0_C1MCC_STATUS
Creating C1:DAQ-DC0_C1MCC_CRC_CPS
Creating C1:DAQ-DC0_C1MCC_CRC_SUM
Creating C1:DAQ-DC0_C1MCP_STATUS
Creating C1:DAQ-DC0_C1MCP_CRC_CPS
Creating C1:DAQ-DC0_C1MCP_CRC_SUM
Creating C1:DAQ-DC0_C1LSP_STATUS
Creating C1:DAQ-DC0_C1LSP_CRC_CPS
Creating C1:DAQ-DC0_C1LSP_CRC_SUM
Creating C1:DAQ-DC0_C1SPX_STATUS
Creating C1:DAQ-DC0_C1SPX_CRC_CPS
Creating C1:DAQ-DC0_C1SPX_CRC_SUM
Creating C1:DAQ-DC0_C1SUP_STATUS
Creating C1:DAQ-DC0_C1SUP_CRC_CPS
Creating C1:DAQ-DC0_C1SUP_CRC_SUM
[Mon Aug  6 20:15:35 2012] Epics server started
[Mon Aug  6 20:15:35 2012] EDCU has 2553 channels configured; first=0

Symmetricom status: LOCKED
Starting at gps 1028344552 prev_gps 1028344552 frac 312500000 f 314094022
[Mon Aug  6 20:15:38 2012] Minute trender made GPS time correction; gps=1028344552; gps%60=52
Segmentation fault (core dumped)

  7097   Mon Aug 6 20:27:59 2012 JamieUpdateSimulationsMore work on getting simplant models running: c1lsp and c1sup

I'm trying to get more of the simplant models running so that we (me and Sasha Surf) can get a full real-time cavity simplant running.  As I reported last week, c1spx is running again on c1iscex.

The two new simplant models are c1sup, which holds the simplant for ITMX, and c1lsp, which holds the IFO simplant, specifically the one we're working on for XARM.

Here's the relevant info:

model  host     dcuid  cpu
c1spx  c1iscex  61     4
c1sup  c1sus    62     6
c1lsp  c1lsc    60     6

c1spx and c1sup will be running the sus_single_plant parts for ETMX and ITMX simplant.  All the simplant suspension channels will be names "SUP" (as opposed to "SUS" for control).

c1lsp is now running, but c1sup won't run for unknown reasons.  The c1sup model is not very complicated, and in fact is more-or-less identical to c1spx.  It compiles and installs and even loads, but it completely unresponsive after loading.  Unfortunately I've had enough CDS bullshit for today, so I'll try to figure out what's going on tomorrow.

  7098   Mon Aug 6 23:05:22 2012 JenneUpdateSUSPRM oplevs servo is fine

The PRM was pointing totally the wrong way, so there was no light on the oplev PD.  I restored the PRM, turned the gains back to (0.15, -0.3) as per Yuta's elog 6952, and it seems just fine to me.

I want to check the data from last night / the weekend to see when the mispointing happened, but dataviewer can't connect to the fb, since Jamie is still working his magic.  I'm pretty sure I restored all of the optics after Eric finished playing with MICH Friday night, but it's possible that I forgot one, I suppose.  If it wasn't me, then I'm curious when it happened.

  7099   Tue Aug 7 00:22:10 2012 JenneUpdateASSFilter banks working

Quote:

I was trying to load some filters into the ASS so that I can try it out, but for some reason the filter banks aren't working - clicking the on/off buttons doesn't do anything, filters (which exist in the .txt file generated by foton) don't load.

I've emailed cds-announce to see if anyone has any ideas.

 

When the network / fb went bad this afternoon, I had been working on the ASS model, shortening the names of the filter banks to fix the problem from elog 7092.  I wanted to finish working on that, so the ASS model is now rebuilt with slightly shorter names in the filterbanks (which fixes the problem of the filter banks not working).

------------------------------------------------------------

I mentioned this to Jamie the other day, but here's the error that you get when the GoTo/From tags aren't working:

>>rtcds make c1ass
### building c1ass...
Cleaning c1ass...
Done
Parsing the model c1ass...
IPC 9 8 is C1:LSC-ASS_LSC
IPC 9 8 is ISHME
IPC 10 9 is C1:RFM-LSC_TRX
IPC 10 9 is IPCIE
IPC 11 10 is C1:RFM-LSC_TRY
IPC 11 10 is IPCIE

INPUT XARM_LSC_in is NOT connected

INPUT YARM_LSC_in is NOT connected

***ERROR: Found total of ** 2 ** INPUT parts not connected

make[1]: *** [c1ass] Error 255
make: *** [c1ass] Error 1

I don't know why these tags weren't working, but there was a GoTo tag on the output of the LSC shmem block, and then Froms on each of the XARM_LSC_in and YARM_LSC_in.  The other day I played around with a bunch of different things (grounding inputs, terminating outputs, whatever), but finally replacing the tags with identical ones freshly taken from CDS_PARTS made it happy. 

  7100   Tue Aug 7 03:20:56 2012 JenneUpdateASSASS setup, on, off scripts written

I wrote new setup, on and off scripts for the arm ass.  They take the arm as an argument, so it's the same script for both arms.  Scripts are in ...../scripts/ASS/ , and have been checked in to the 40m svn.

So far the on script doesn't really do anything, since I haven't chosen values for the CLKGAINs of the lockins.  The old values were 30 for lockins 12, 14, 27, 29 and 250 for lockins 7, 9, 22, 24.  Unfortunately, I have no memory of which lockin means what in the old numbered system.  I'll have to look that up somehow.  Or, just dither the optics using some value and look at the spectrum to see the resulting SNR and just pick something that gives me reasonable SNR.

I modified the ASS model slightly:

* Added an overall gain to the ASS_DOF2 library part, between the matrix and the servo inputs so we can do soft startups.  Self - remember that the main ASS screen needs to be modified to reflect this!

* Rearranged the order that the demodulated signals go into the matrix.  I hadn't paid attention, and the old ordering had the transmission (TRX/TRY) demod signals interleaved with the LSC demod signals.  I've changed it to be all the TR signals, then all the LSC signals.  I think this makes more sense, since we will use these inputs separately, so now they're on different halves of the matrix.

  7101   Tue Aug 7 11:46:24 2012 JamieUpdateCDSAlex working on daqd

Alex is apparently working on daqd (remotely).  I'll report back when I find out more.

  7102   Tue Aug 7 14:17:07 2012 JamieUpdateCDSdaqd running again; related to c1sup issue

So daqd's problem was apparently the bad/non-running c1sup model.  The c1sup model, which I reported on attempting to get running in 7097, was not running because there were no available CPUs on the c1sus FE machine.  This was due to my stupid undercounting of the number of CPUs.  Anyway, for reasons I don't understand, this was causing daqd to segfault.  Removing c1sup from c1sus "fixed" the problem.

Alex agreed that daqd should definitely not be segfaulting in this circumstance.  It's still unclear exactly what daqd was looking at that was causing it to crash.

I'm going to move c1sup to c1iscex, which has a lot of spare CPUs.

  7103   Tue Aug 7 14:34:01 2012 JamieUpdateCDSjk. daqd still segfaulting

Quote:

So daqd's problem was apparently the bad/non-running c1sup model.  The c1sup model, which I reported on attempting to get running in 7097, was not running because there were no available CPUs on the c1sus FE machine.  This was due to my stupid undercounting of the number of CPUs.  Anyway, for reasons I don't understand, this was causing daqd to segfault.  Removing c1sup from c1sus "fixed" the problem.

Alex agreed that daqd should definitely not be segfaulting in this circumstance.  It's still unclear exactly what daqd was looking at that was causing it to crash.

I'm going to move c1sup to c1iscex, which has a lot of spare CPUs.

I spoke too soon.  It's still segfaulting, but at a different place. Alex and I are looking into it.

But another mystery solved is the cause of all the network slowness: the daqd core dump.  When daqd segfaults it dumps it's core, which can typically be >4G, to /opt/rtcds/caltech/c1/target/fb/core.  This is of course an NFS mount from linux1, so it's dumping 4G on the network, which not surprisingly clogs the network.

  7104   Tue Aug 7 15:01:38 2012 JenneUpdateComputer Scripts / Programsmedmrun now allows args to pass to scripts

Previously, medmrun didn't accept arguments to pass along to the script it was going to run.  Jamie has graciously taken a moment from fixing the computer disaster to help me update the medmrun script.

Now the ASS scripts are call-able from the screen.

  7105   Tue Aug 7 15:04:23 2012 JamieUpdateCDSdaqd problem was root-owned files and directories

Apparently the last problem was because of root-owned frame directories that daqd was trying to write to.  During debugging Alex had run daqd as root, but it's supposed to run as controls.  All the /frame directories are supposed to be owned by controls.  When daqd was run as root, it created new frame directories owned by root, which controls couldn't write to when I restarted daqd the proper way.  Once we chown'd the directories daqd started running again.

Alex also put in a "fix" for the core dump problem.  He touched an empty core file owned by root:

-rw-r--r-- 1 root root 0 Aug  7 14:38 /opt/rtcds/caltech/c1/target/fb/core

This will prevent any dying daqd process owned by controls from dumping it's core at that location.  Personally I think this is a horribly hacky "solution" that doesn't actually fix any of the issues that were causing the segfaults to begin with, but it might prevent some of the network slow down we see when the core does dump.  It's mostly just masking the problem, though, so I'm tempted to remove it so we all feel the pain when daqd starts shitting all over the network again.

  7106   Tue Aug 7 16:11:23 2012 steveUpdateSUSoptical table box wall proposal

 Sanwiched wall as shown: 1" clear acrylic, 2 layers of 0.004" thick "window tint", 1 layer of  0.007" thermashield and  0.125" yellow acrylic

Visibility: 70 %,    Transmission of 1064 nm  2-3 % at 0-50 degrees incident  power density 0.7 W/mm2

Max power 100 mW

        

Attachment 1: IMG_1479.JPG
IMG_1479.JPG
Attachment 2: IMG_1478.JPG
IMG_1478.JPG
  7107   Tue Aug 7 16:21:16 2012 ranaUpdateSUSoptical table box wall proposal

Quote:

 Sanwiched wall as shown: 1" clear acrylic, 2 layers of 0.004" thick "window tint", 1 layer of  0.007" thermashield and  0.125" yellow acrylic

Visibility: 70 %,    Transmission of 1064 nm  2-3 % at 0-50 degrees incident  power density 0.7 W/mm2

Max power 100 mW        

 Good. The power density and max power are not important (especially since you don't define a quantitative way to spec them). We ONLY care about the transmission.

  7108   Tue Aug 7 18:38:50 2012 LizUpdateComputer Scripts / ProgramsDaily Summary Pages are in their final form!

Please check the summary pages out at the link below and let me know if there are any modifications I should make!  All existing pages are up to date and contain all of the pages I have.

Questions, comments, and suggestions will be appreciated! Contact me at endavison@umail.ucsb.edu

https://nodus.ligo.caltech.edu:30889/40m-summary/

  7109   Tue Aug 7 21:34:50 2012 YaakovUpdateSTACISMore noise data

Yesterday I plugged the geophone and accelerometer output into the ADC, rather than the SR785, so I could collect for longer and take more data at once.

As per Rana's suggestion, I am also now taking the geophone output after the first op-amp in the circuitry following the geophone (a low-noise op-amp, OPA227). It acts as a buffer so I'm not just measuring other local noise sources (which explains why the geophone noise curve sort of matched the SR785 noise curve in my old plots).

With these changes, I remeasured the accelerometer and geophone noises as well as collected an ASD of a geophone sitting on the STACIS in open loop operation. I also looked up the noise specs for the various op-amps in the geophone pre-amp and high voltage board; everything I found, I added in quadrature to come up with an approximate op-amp noise value for the STACIS. All of this is plotted below:

budget.bmpbudget.fig

I left the y-axis in V/rtHz instead of converting it to m/s/rtHz so that the op-amp noise could be compared to the other noises. All sensor data was taken with the sensors horizontal (noise data taken in granite and foam).

The accelerometer and geophone noise still appear to be similar, and the op-amp noise, at least according to specs, is low compared to the other noises. This implies there's not much to gain from switching the geophones with accelerometers nor with swapping out the op-amps for lower-noise components (unless the ones I couldn't find specs for were high-noise, though it seems like mainly low-noise components were used). 

  7110   Tue Aug 7 23:30:58 2012 ranaUpdateEnvironmentNearby EQ

 Just felt an EQ. Impulse moved some vertical blinds by several mm.

Tue Aug 07 23:26:06 2012 

  7111   Tue Aug 7 23:33:34 2012 DenUpdateEnvironmentNearby EQ

Quote:

 Just felt an EQ. Impulse moved some vertical blinds by several mm.

Tue Aug 07 23:26:06 2012 

 All optics except MC2 and ETMX are crazy

watchdogs.png      mc1.png    gur.png

  7112   Tue Aug 7 23:33:44 2012 ranaUpdateSTACISMore noise data

Looks like you're just measuring the ADC noise. You should add ADC noise to your plot. To compare the geophones with the accelerometers, you have to correct for the preamp gain and plot them both in the same units.

To get above the ADC noise you can use an SR560 preamp. (AC Coupled, G = 100)

  7113   Wed Aug 8 09:46:29 2012 MashaUpdateEnvironmentETMX EQ

[Sasha, Masha, Liz, Eric]

 

A bunch of surfs in the lab just noticed that ETMX is going crazy (laser is shifting everywhere) due to a 4.5 EQ that just hit LA. The optic is already shut down according to the watchdogs.

  7114   Wed Aug 8 10:15:13 2012 jamieUpdateEnvironmentAnother earthquake, optics damped

There were another couple of earthquakes at about 9:30am and 9:50am local.

earthquake.png

All but MC2 were off the watchdogs.  I damped and realigned everything and everything looks ok now.

Screenshot-Untitled_Window.png

  7115   Wed Aug 8 10:38:43 2012 LizUpdateComputer Scripts / ProgramsWeek 8/Summary Pages update

Over the past week, I have been working on my progress report and finalizing the summary pages.  I have a few more things to address in the pages (such as starting at 6 AM, including spectrograms where necessary and generating plots for the days more than ~a week ago) but they are mostly finalized.  I added all of the existing acoustic and seismic channels so the PEM page is up to date.  The microphone plots include information about the transfer factor that I found on their information sheet (http://www.primomic.com/).  If there are any plots that are missing or need editing, please let me know!

I also modified the c1_summary_page.sh script to run either the daily plots or current updating plots by taking in an argument in the command line.  It can be run ./c1_summary_page.sh 2012/07/27
 or ./c1_summary_page.sh now to generate the current day's pages.  (Essentially, I combined the two scripts I had been running separately.)  I have been commenting my code so it is more easily understandable and have been working on writing a file that explains how to run the code and the main alterations I made.  The most exciting thing that has taken place this week is that  the script went from taking ~6 hours to run to taking less than 5 minutes.  This was done by using minute trends for all of the channels and limiting the spectrum plot data.

The summary pages for each day now contain only the most essential plots that give a good overview of the state of the interferometer and its environment instead of every plot that is created for that day.

I am waiting for Duncan to send me some spectrogram updates he has made that downsample the timeseries data before plotting the spectrogram.  This will make it run much more quickly and introduce a more viable spectrogram option.

 

Today's Summary Pages can be accessed by the link on the wiki page or at:

https://nodus.ligo.caltech.edu:30889/40m-summary/archive_daily/20120808/

  7120   Wed Aug 8 13:37:46 2012 KojiUpdateComputer Scripts / ProgramsWeek 8/Summary Pages update

Hey, the pages got significantly nicer than before. I will continue to give you comments if I find anything.

So far: There are many 10^-100 in logarithmic plots. Once they are removed, we should be able to see the seismic excitation during these recent earth quakes?

Incidentally, where the script is located? "./" isn't the absolute path description.

Quote:

Over the past week, I have been working on my progress report and finalizing the summary pages.  I have a few more things to address in the pages (such as starting at 6 AM, including spectrograms where necessary and generating plots for the days more than ~a week ago) but they are mostly finalized.  I added all of the existing acoustic and seismic channels so the PEM page is up to date.  The microphone plots include information about the transfer factor that I found on their information sheet (http://www.primomic.com/).  If there are any plots that are missing or need editing, please let me know!

I also modified the c1_summary_page.sh script to run either the daily plots or current updating plots by taking in an argument in the command line.  It can be run ./c1_summary_page.sh 2012/07/27
 or ./c1_summary_page.sh now to generate the current day's pages.  (Essentially, I combined the two scripts I had been running separately.)  I have been commenting my code so it is more easily understandable and have been working on writing a file that explains how to run the code and the main alterations I made.  The most exciting thing that has taken place this week is that  the script went from taking ~6 hours to run to taking less than 5 minutes.  This was done by using minute trends for all of the channels and limiting the spectrum plot data.

The summary pages for each day now contain only the most essential plots that give a good overview of the state of the interferometer and its environment instead of every plot that is created for that day.

I am waiting for Duncan to send me some spectrogram updates he has made that downsample the timeseries data before plotting the spectrogram.  This will make it run much more quickly and introduce a more viable spectrogram option.

 

Today's Summary Pages can be accessed by the link on the wiki page or at:

https://nodus.ligo.caltech.edu:30889/40m-summary/archive_daily/20120808/

 

  7121   Wed Aug 8 18:01:58 2012 JenneUpdateIOOMC autolocker threshold changed

Jan and Manasa are going to elog about their work later, but it involved putting a BS/window/some kind of pick off in front of the MC Trans QPD, so the total light on the MC Trans QPD is now ~16000 rather than ~26000 counts.  I changed the threshold in the MC autolocker to 5000, so now the MC Trans PD must see at least 5000 counts before the autolocker will engage the boosts, WFS, etc.  Actually, this threshold I believe should have been some several thousand value, but when I went in there, it was set to 500 counts, for low power MC mode during a vent.  It had never gotten put back after the vent to some higher, nominal value.

  7123   Wed Aug 8 20:34:29 2012 JenneUpdateASSTrouble measuring sensing matrix

I turned on the ASS, without closing the loops, to try to measure the sensing matrix. 

The Yarm was locked (Eric did a nice job earlier - he'll ELOG ABOUT IT before he goes home!), and I used an LO CLKGAIN of 300 on all of the TRY Lockins.  Then I put on and took away a 10% offset in pitch, but it's almost impossible to see the difference. 

The attached is a truly awful screenshot, but you can kind of see what's going on.  The big steps are me increasing the LO gain, but around "0" on the x-axis I changed the pitch offset from 10% away to nominal.  Since there are such big oscillations, the change is basically non-existent.  Grrrr. I'll look at it again tomorrow, since I have an exiting bike ride home ahead of me....

 

  7124   Wed Aug 8 20:50:39 2012 KojiUpdateASSTrouble measuring sensing matrix

From the log, I couldn't understand what has been done.

The procedure we should perform is

  1. Dither total 4 dofs of the two mirrors with different frequencies. Some fluctuation of TRY is even visible in dataviewer.
  2. The cavity is aligned at the beginning. These 4 peaks in TRY in DTT is small or invisible. Some 2nd harmonics are visible.
  3. Misalign one of the dofs. Some peaks get bigger.
  4. Correspoding lockin output becomes bigger.

Then you can start measuring the sensing matrix. At which part did the attempt fail?

Quote:

I turned on the ASS, without closing the loops, to try to measure the sensing matrix. 

The Yarm was locked (Eric did a nice job earlier - he'll ELOG ABOUT IT before he goes home!), and I used an LO CLKGAIN of 300 on all of the TRY Lockins.  Then I put on and took away a 10% offset in pitch, but it's almost impossible to see the difference. 

The attached is a truly awful screenshot, but you can kind of see what's going on.  The big steps are me increasing the LO gain, but around "0" on the x-axis I changed the pitch offset from 10% away to nominal.  Since there are such big oscillations, the change is basically non-existent.  Grrrr. I'll look at it again tomorrow, since I have an exiting bike ride home ahead of me....

 

  7125   Wed Aug 8 20:51:56 2012 ManasaUpdate40m UpgradingOptical layout updated

Quote:

ACAD files of the 40m optical layout have been updated as per the vent in Aug 2011.

Files are available at the 40m svn docs-->Upgrade12-->Opt_Layout2011.

 

 To ease the pain of hunting files, optical layout ACAD files have been moved to a new directory 40M_Optical Layout in the repository. Relevant files from directories Upgrade12 and upgrade 08 will be moved to "40M_Optical Layout" very soon and eventually these old directories will be removed. 

  7129   Thu Aug 9 00:23:11 2012 JenneUpdateASSTrouble measuring sensing matrix

Quote:

From the log, I couldn't understand what has been done.

The procedure we should perform is

  1. Dither total 4 dofs of the two mirrors with different frequencies. Some fluctuation of TRY is even visible in dataviewer.
  2. The cavity is aligned at the beginning. These 4 peaks in TRY in DTT is small or invisible. Some 2nd harmonics are visible.
  3. Misalign one of the dofs. Some peaks get bigger.
  4. Correspoding lockin output becomes bigger.

Then you can start measuring the sensing matrix. At which part did the attempt fail?

Quote:
 

 Cavity started out aligned pretty well, but not 100%.  Transmission was ~0.8 . Perhaps this was part of the problem.

I realize now that you mention it, it was totally amateur hour of me to only look at the lockin outputs on StripTool (plus POY and TRY on Dataviewer), and not look at TRY on DTT...or any spectra at all.  Not so intelligent.  I could see some fluctuation of TRY on Dataviewer that corresponded to me turning on the oscillators, as well as the spot wiggling on the camera view of ETMYT a teeny bit.

When applying a 10% misalignment to ETMY Pit (by adding 0.1 to the Pit components of the output matrix, as is done in the MC spot position calibration), I could see that there was a small jump in the StripTool trace, but it was much smaller than the ambient fluctuations of the output. 

I just looked back and realized that I must have forgotten to add my screenshot, but it's saved on a desktop on Rossa.  It would be better if I had attached the data, but from that you see that the average of the lockin output signal didn't change very much in the last several minutes of the measurement, but the fluctuations (no misalignment offsets) are pretty big, maybe ~10% or 15% the size of the signal.  Then when I added the misalignment to one mirror (ETMY PIT), there is a very small jump in the lockin signal, but it is much, much smaller than the size of the ambient fluctuations.  Perhaps a long average would result in a "real" value, but by looking by eye, I can't see a discernible difference in the average value of the lockin outputs.

My plan is to do as you say, dithering all 4 optics, and misaligning a single optic's single DoF (Pit or Yaw), and seeing how that misalignment affected each of the sensors (the lockin outputs).  Then put that DoF back to nominal, and misalign a different DoF, rinse and repeat.

Okay, so this is a little stream-of-consciousness-y, and you're going to think I'm really dumb, but I just realized that I haven't set the phase of the lockin demodulators yet.  So I think I need to dither the optics, and go through each of the sensors, and adjust the phase until the peak in TRY in DTT is maximized for the I phase, and minimized for the Q phase (since we use the I-output).  Bah.  Bad Jenne.

  7131   Thu Aug 9 01:26:03 2012 KojiUpdateASSTrouble measuring sensing matrix

That's a good point, but I suspect that you end up with the in-phase (0deg) as the response of the IFO is immediate compared with the dithering frequency
as long as the whitening/dewhitening are properly compensated in the digital realm.

Quote:

 Okay, so this is a little stream-of-consciousness-y, and you're going to think I'm really dumb, but I just realized that I haven't set the phase of the lockin demodulators yet.  So I think I need to dither the optics, and go through each of the sensors, and adjust the phase until the peak in TRY in DTT is maximized for the I phase, and minimized for the Q phase (since we use the I-output).  Bah.  Bad Jenne.

 

  7132   Thu Aug 9 04:26:51 2012 SashaUpdateSimulationsAll c1spx screens working

As the subject states, all screens are working (including the noise screens), so we can keep track of everything in our model! :D I figured out that I was just getting nonsense (i.e. white noise) out of the sim plant cause the filter matrix (TM_RESP) that controlled the response of the optics to a force (i.e. outputted the position of the optic DOF given a force on that DOF and a force on the suspension point) was empty, so it was just passing on whatever values it got based on the coefficients of the matrix without DOING anything to them. In effect, all we had was a feedback loop without any mechanics.

I've been working on getting the mechanics of the suspensions into a filter/transfer function form; I added something resembling that into foton and turned the resulting filter on using the shiny new MEDM screens. However, the transfer functions are a tad wonky (particularly the one for pitch), so I shall continue working on them. It had a dramatic effect on the power spectrum (i.e. it looks a lot more like it should), but it still looks weird.

Still haven't found the e-log Jamie and Rana referred me to, concerning the injection of seismic noise into the simulation. I'm not terribly worried though, and will continue looking in the morning. Worst case scenario, I'll use the filters Masha made at the beginning of the summer.

Masha and I ate some of Jamie's popcorn. It was good.

  7133   Thu Aug 9 07:24:58 2012 SashaUpdateSimulationsAll c1spx screens working

Quote:

As the subject states, all screens are working (including the noise screens), so we can keep track of everything in our model! :D I figured out that I was just getting nonsense (i.e. white noise) out of the sim plant cause the filter matrix (TM_RESP) that controlled the response of the optics to a force (i.e. outputted the position of the optic DOF given a force on that DOF and a force on the suspension point) was empty, so it was just passing on whatever values it got based on the coefficients of the matrix without DOING anything to them. In effect, all we had was a feedback loop without any mechanics.

I've been working on getting the mechanics of the suspensions into a filter/transfer function form; I added something resembling that into foton and turned the resulting filter on using the shiny new MEDM screens. However, the transfer functions are a tad wonky (particularly the one for pitch), so I shall continue working on them. It had a dramatic effect on the power spectrum (i.e. it looks a lot more like it should), but it still looks weird.

Still haven't found the e-log Jamie and Rana referred me to, concerning the injection of seismic noise into the simulation. I'm not terribly worried though, and will continue looking in the morning. Worst case scenario, I'll use the filters Masha made at the beginning of the summer.

Masha and I ate some of Jamie's popcorn. It was good.

 Okay! Attached are two power spectra. The first is a power spectrum of reality, the second is a power spectrum of the simPlant. Its looking much better (as in, no longer obviously white noise!), but there seems to be a gain problem somewhere (and it doesn't have seismic noise). I'll see if I can fix the first problem then move on to trying to find the seismic noise filters.

Attachment 1: Screenshot.png
Screenshot.png
Attachment 2: Screenshot-1.png
Screenshot-1.png
  7135   Thu Aug 9 12:31:36 2012 JenneUpdateASSASS rebuilt again
I was (in between Eric's measurements) driving the YARM ASS dithers, and noticed that instead of driving ITMY PIT, I was driving ITMX PIT. I looked in the model, and when I re-did the model after an svn revert a few days ago, it looks like I got the shmem parts for the ITM PIT signals backwards. I have fixed that, rebuilt, installed and restarted the ass model.
ELOG V3.1.3-