40m QIL Cryo_Lab CTN SUS_Lab TCS_Lab OMC_Lab CRIME_Lab FEA ENG_Labs OptContFac Mariner WBEEShop
  40m Log, Page 119 of 344  Not logged in ELOG logo
ID Date Author Type Category Subject
  11361   Mon Jun 15 22:36:40 2015 rana, kojiUpdateGreen LockingBeatBox Assay: not looking good

Because the ALS beatbox schematic is out-of-date and misleading, we pulled the box to photograph the current implementation and figure out how to proceed. The box is out on the EE bench right now. Schematic Doc added to 40m Document tree: https://dcc.ligo.org/LIGO-D1102241. Some notes:

  1. The soldering on this board is pretty messy and there are a lot of flying wire and flying component hacks. I wouldn't trust all of the connections.
  2. The GV-81 RF amps in the front end are both stuffed. The 1 dB compression point is 19 dBm, so we want to use them below 10 dBm output. They have a gain of +10.5 dB, so that means they should not be used with and input to the beatbox of more than -10 dBm. Otherwise there will be nonlinear noise generation.
  3. Not stuffed: U1-Comparator, A1-attenuator, U2-splitter.
  4. Why is the filter after the mixer only 2nd order?? That's not a valid filter choice in any RF world. How much do we want to cut off the 2f mixer output before sending into our low noise, audio frequency (and prone to downconversion) amplifier? The Mini-Circuits amplifiers would have given us >60 dB attenuation in the stop band. This one is only going to give us 20-30 dB when the beat frequency is low. Get rid of diplexer. The schematic claims that its just one pole?? Seems like a 2nd order LP filter to me.
  5. The modified schematic (see Koji elog 8855) shows that an OP27 is used for the whitening stage. The current noise of the OP27 with the 3k resistor makes the OP27 current noise dominate below 1 Hz. And what is going on with that filter capacitor choice? We never want to use these tiny things for sensitive filter applications. (cf. Sigg doc on resistor and capacitor choice, the noise reduction book by Ott, H&H, etc.). That's why we have the larger metal-poly, paper, mylar, etc. caps sitting around.

Probably we ought to install a little daughter board to avoid having to keep hacking this dead horse. Koji has some of Haixing'g maglev filter boards. Meanwhile Koji is going to make us a new beatbox circuit in Altium and we can start fresh later this summer.

Interesting link on new SMD cap technology.

Photos of circuit as found

  11360   Mon Jun 15 20:36:48 2015 ranaUpdateIOOIOO QPDs centred

after re-aligning the beam into the PMC, I touched up the steering mirros into the IOO QPDs so that the beams are now centered again. Please don't adjust these references without prior authorization and training.

This plot shows the 10-minute trends for these QPDs over the last 400 days.

Attachment 1: Untitled.png
Untitled.png
  11359   Mon Jun 15 16:55:39 2015 ericqUpdatePEMAccelerometers installed

The accelerometers have been installed at MC1 and MC2. MC2 data is live, I haven't yet run the cables from the MC1 set to the preamp yet, though. 

Attachment 1: MC2.jpg
MC2.jpg
Attachment 2: MC1.jpg
MC1.jpg
Attachment 3: mc2accspectra.png
mc2accspectra.png
  11358   Mon Jun 15 11:54:44 2015 ericqUpdateCDSParts not in SVN

I ran the following command to find which models/parts are not under version control, or have modifications not commited:

grep "mdl" $(cat models.txt) | awk '{print $NF}' | sort | uniq | xargs svn status

models.txt includes lines like "/opt/rtcds/caltech/c1/rtbuild/c1ass.log" for each running model. These are the build logs that detail every file being sourced. 

The resultant list is:

?       /opt/rtcds/userapps/release/cds/common/models/BLRMS_HIGHFREQ.mdl
C       /opt/rtcds/userapps/release/cds/common/models/rtdemod.mdl
M       /opt/rtcds/userapps/release/cds/common/models/SCHMITTTRIGGER.mdl
?       /opt/rtcds/userapps/release/isc/c1/models/blrms.mdl
?       /opt/rtcds/userapps/release/isc/c1/models/IQLOCK.mdl
?       /opt/rtcds/userapps/release/isc/c1/models/PHASEROT.mdl
?       /opt/rtcds/userapps/release/sus/c1/models/QPD_WHITE_CTRL_MUX.mdl

I will commit the uncommited c1 parts, and think about what to do about the rtdemod and SCHMITTRIGGER parts. 

Once I check out the latest revision of the userapps repo (in a seperate location), I will do something similar to look for models that have changed since the svn revision that is checked out in our running system. 

  11357   Sat Jun 13 23:52:14 2015 ericqUpdateLSCBeatbox needs whitening gain

Nice find!

We ought to use our noise model of the ALS signal chain to determine what the right gains are, rather than hunt and peck. More likely we'll start from the right gains.​

Once the gains and/or whitening filters make sense, maybe we'll see some effect from fixing the green PDH loops.

  11356   Sat Jun 13 18:37:46 2015 ericqUpdateLSCBeatbox needs whitening gain

Here are the promised details!

I was worried about the lack of whitening gain, as I saw that the DC phase tracker Q output (which is the magnitude of the signal in the beatbox's I-Q plane) was no more than 200 ADC counts for X (~120mV), and 800 for Y (~500mV). I.e. this is the largest DC value that either the I or Q ADC channels can see, and the RMS fluctuations are on the order of mV, meaning we're wasting our entire ADC rangeno

However, I also had doubts about this since, even in the nominal state, the ASD of the ADC signals before dewhitening was higher than the expected ADC noise level. However, because of the non-linearity of the conversion of the BEAT_I and Q signals into the phase tracker output, evaluating the contribution of the beatbox output and ADC input voltage noises takes a few more steps. 

So, I hooked up a Marconi as the signal source for the beatbox's X channel , with no modulation (presumably the phase noise of the Marconi output is significantly lower than the sensitivity of the beatbox). For all of these measurements, the beat frequency was kept around 50MHz, with an amplitude of -30dBm on the control room analyzer, which is a typical X ALS operating point. 

At this point, the beatbox output noise was below the ADC noise (as measured by an SR785). Nevertheless, I found that the beat spectrum driven by the Marconi lined up to be very close to the ALS beat spectrum across a wide band, explaining much of the noise. 

At this point, I inserted SR560s in between the beatbox I and Q outputs, and the AA chassis leading to the ADC. A gain of 10 reduced the resultant phase tracker noise by that same factor at nearly all frequencies. A further increase in gain did not lead to the noise changing appreciably, probably because the real beatbox noise was now contributing, as is suggested by some common peaks in the direct beatbox output phase tracker spectra.

Going back to the real green beat signal with the SR560s still at G=10, I obtained the result shown in ELOG 11355. I will soon repeat this process with the Y ALS. 

As I mentioned in the previous ELOG, we may be further helped by more whitening gain than can be provided by the SR560s (and we obviously need a robust long term circuit for this gain). If it then turns out we are limited by beatbox noise to a degree we are not happy with, we could perhaps look into reintroducing some RF gain into the X beat. As Koji mentions in ELOG 8855, the beatbox operates best at an RF input of around -4dBm.

Attachment 1: ADCdiagnosis.png
ADCdiagnosis.png
  11355   Fri Jun 12 17:09:58 2015 ericqUpdateLSCBeatbox needs whitening gain

Short entry just to preview a new development; more detail about this investigation will soon follow. 

The beatbox I and Q signals are too small at the ADC! I was able to reduce the RMS out of loop ALS sensitivity (arm locked on POX) by 300Hz with G=10 SR560s between the beatbox output and the ADC whitening chassis input. Increasing the beat note amplitude via RF amplifier had no positive effect. 

There is still a reasonable gap between this and the beatbox's noise levels, as measured with a marconi. There may be additional headroom for whitening gain; the SR560 maximum output range is smaller than the ADC input range. 

The high frequency noise has >0.5 coherence with the PDH error signal above a kHz or so, but not much below that. 

We should probably either modify the output whitening of the beatbox, or introduce some (variable?) whitening gain in a seperate circuit. 

Attachment 1: beatGain.png
beatGain.png
  11354   Fri Jun 12 08:40:17 2015 SteveUpdateVACVacuum comp. rebooted

Koji and Steve,

One computer expert and one vacuum expert required.

Quote:

The serial connections to the vacuum gauges were recovered by rebooting c1vac1 and c1vac2.

Steve claimed that the vacuum screen had showed "NO COMM" at the vacuum pressure values.
The epics connection to c1vac was fine. We could logged in to c1vac1 with telnet too although c1vac2 had no response.

After some inspection, we decided to reboot the slow machines. Steve manually XXXed YYY valves (to be described)
to prepare for any possible unwanted switching. Initially Koji thought only c1vac2 can be rebooted. But it was wrong.
If the reset button is pushed, all of the modules on the same crate is reset. So everything was reset. After ~3min we still
don't have the connection to c1vac1 restored. We decided to another reboot. This time I pushed c1vac1 reset button.
After waiting about two minutes, the ADCs started to show green lights and the switch box started scanning.
We recovered the telnet connection to c1vac1 and epics functions. c1vac2 is still note responding to telnet, and
the values associated with c1vac2 are still blank.

Steve restored the valves and everything was back to normal.

Atm 1,  problem condition: gauges are not reading for a week, error message "NO COMM" and all computer LEDs are green

Atm 2, prepare to safe reboot:

            a, close V1, disconnect it's power cable and turn off Maglev, wait till rotation stops

            b, close PSL shutter ( take adrenaline if needed )

            c, close V4, V5, VA6 valves and disconnect their cables. "Moving" error message indicating this condition.

               V1 is not showing "Moving" because its power cable disconnected only! It will show it if its position indicator cable is disconnected too. There is no need for that.

               These valves closed and disabled will not allow accidental venting of main volume.

            d, push reset, reseting c1vac2 will reset c1vac1 also, wait ~ 6 minutes

"Vacuum Normal" valve configuration was restored after succesful reboot as follows:

             a, reconnect cable and open V4 and V5 at P2 & P3 <1e-1 Torr

             b, observe that P2  < 1e-3 Torr and retsart Maglev

             c, wait till Maglev reaches full speed of 560 Hz and reconnect-open V1

             d, reconnect-open VA6 at P3 <1e-3 Torr

NOTE: VM1 valve was locked in open position and it was not responding before and after reboot

          Error message on Atm2 is indicating this locked condition: "opening VM1 will vent IFO"

          This is a fauls message. The valve is frozen in open position. We need a softwear expert help.

 

 

Attachment 1: vacMonNoGauges.png
vacMonNoGauges.png
Attachment 2: prepReboot.png
prepReboot.png
  11353   Thu Jun 11 19:40:59 2015 KojiUpdateVACVacuum comp. rebooted

The serial connections to the vacuum gauges were recovered by rebooting c1vac1 and c1vac2.

Steve claimed that the vacuum screen had showed "NO COMM" at the vacuum pressure values.
The epics connection to c1vac was fine. We could logged in to c1vac1 with telnet too although c1vac2 had no response.

After some inspection, we decided to reboot the slow machines. Steve manually XXXed YYY valves (to be described)
to prepare for any possible unwanted switching. Initially Koji thought only c1vac2 can be rebooted. But it was wrong.
If the reset button is pushed, all of the modules on the same crate is reset. So everything was reset. After ~3min we still
don't have the connection to c1vac1 restored. We decided to another reboot. This time I pushed c1vac1 reset button.
After waiting about two minutes, the ADCs started to show green lights and the switch box started scanning.
We recovered the telnet connection to c1vac1 and epics functions. c1vac2 is still note responding to telnet, and
the values associated with c1vac2 are still blank.

Steve restored the valves and everything was back to normal.

  11352   Wed Jun 10 15:54:14 2015 SteveUpdateVACVacuum comp. rebooted

Koji and Steve succeded rebooting C1vac1, C1vac2 and pressure reading is working now

More tomorrow .........

 

Attachment 1: afterReb061015.png
afterReb061015.png
  11351   Wed Jun 10 03:19:58 2015 ericqUpdateLSCAUX PDH error measured in CDS

Looking over the old noise budget in the green locking paper, it seems the main technical noise sources were the AUX PDH error and DFD noise. I'm working on quantifying the current state of these noises. 

Rather than lugging out the analyzers every time I wanted to make a measurement of the AUX PDH error signals, I set out to make the existing digitized channels (ALS-[X/Y]_ERR_MON) usable for easier, and continuous, monitoring. Sadly, up until now the signals were poorly conditioned, and drowning in ADC noise. (When locked, the Y error signal was only +-10 counts!)

Of course, given the bandwidth of the green servos (10kHz), this won't tell us the full story of the what the green PDH lock is doing, but does indicate how much residual frequency noise exists in the ALS control band. 

I'm currently using SR560s at G=20 at each end to amplify the ERR MON outputs of the uPDH boards before sending them to the ADCs; now that I've found a gain that works, I'll modify the error point monitor buffer opamps inside the uPDH boards themselves during the daytime. 

The AUX Y error signal was going into an AA board with some funky filtering going on that I didn't want to mess with. Instead, I've moved the signal to the pentek generic board whose first four channels are used for the oplev segments, and the second quartet are unused, save for the TST channel I hooked up yesterday. 

On the pentek board, I changed the 4th order 800Hz lowpass to a 4th order 8kHz lowpass on the last three channels through some resistor swapping. (At first it was just the last two, but I found I was getting weird signals in the 32nd channel; and if I recall correctly from my cymac work, the 32 ADC channel is used for some timing signal or something...). I also turned off the 1:10 whitening filters on the last four channels via PCB jumper. 

I then unhooked the PZT drive and let the PDH error signals swing around, to calibrate the ADC counts into HZ. Now, the ALS-[X/Y]_ERR_MON_OUT_DQ are calibrated in green Hz! Here are the spectra.

As we've seen in the past (ELOG 10464), the X green is limited by the dark noise of the PD from 10-100Hz. This isn't so great. The RMS noise from 300Hz downwards (which is maybe the band where the ALS control would inject noise into the mirror motion) is about Y:10Hz X:40Hz.

During this time, the test masses were not under any longnitudinal control, so I'm not sure why there is such a difference in the height of the suspension resonance peaks, unless there's some differene in the low frequency PDH TFs that I've forgotten about. 

Now, with these references, we can easily check if the PDH loops change qualitatively over longer time periods.

I'll be including the effect of these noises in the upcoming revised ALS noise budget.

Attachment 1: AUXspectra.png
AUXspectra.png
  11350   Wed Jun 10 02:50:39 2015 ericqUpdatePEMWilcoxon Accelerometer Huddle Test

Here are some results for the 3-corner hat subtraction for the six accelerometers, from ~1 hour of data that didn't look to have any sharp features/glitches from human activity in the lab. 

I used the python uncertainties package to try and get an estimate of the uncertainty in the subtracted noise floor, by taking into account every possible possible combination of 3 sensors and the fluctuations in the spectrograms of the subtracted signals. I've attached the python code if anyone is interested in the implementation. 

I pulled out the accelerometer data sheets to read their slightly varying V/g calibration (which differs on the order of a few percent from unit to unit). This improved the subtraction factor from ~20 to over 100 at some frequencies. I've edited the filter modules such that the OUT_DQ channels are already calibrated into m/s^2.

Attachment 1: hats2Acc.png
hats2Acc.png
Attachment 2: 3hatcode.zip
  11349   Tue Jun 9 10:57:12 2015 SteveUpdateSUS BS oplev laser tuned

Lenses removed from oplev beam path at elog entry 11246

 

Attachment 1: 100dBStrend0425.png
100dBStrend0425.png
Attachment 2: 18dBSdrift.png
18dBSdrift.png
  11347   Mon Jun 8 15:51:31 2015 ericqUpdateCDSRCG Diagnostics

I've started making some model changes for RCG diagnostic tests. 

I put some blocks down in C1TST and C1RFM to test the delays of all-digital loops and one loop with a direct DAC->ADC (which currently uses a janky 1-pin lemo -> BNC -> 2-pin lemo situation (which will be improved)). 

Here's what C1TST looks like now. 

I've taken TFs of all three loops. The all digital loops are flat on the order of microdBs. 

The delay in loop A (single loop, one model) is consistent with one 16k cycle, plus or minus 0.25 nsec. 

The delay in loop C (single loop, two models connected via RFM) is consistent with two 16k cycles, plus or minus 0.5 nsec. 

I haven't yet grabbed the whitening and AA/AI shapes for loopB, to calibrate the real delay. 

All of these files currently live in /users/ericq/2015-06-CDSdiag, but I'll make somewhere outside of the user directory to collect all of these tests soon. 

Attachment 1: newTST.png
newTST.png
  11346   Fri Jun 5 11:59:59 2015 ericqBureaucracyGeneralMaintenance Tasks, IFO upgrades

At wednesday's meeting, Rana, Koji, Steve, and I started making a list of maintenence tasks that should be done/checked on a regular basis. The actual scheduling of these has not yet been considered. They include:

  • N2 Tank pressure / cylinder replacement
  • Headlamp, walkie talkie battery recharge
  • Workstation software updates
  • Coffee bean and filters
  • Multimeter battery levels
  • Sorensen DC power supply voltage settings and current draws
  • UPS' status (Vacuum, NFS host, workstations)
  • SR560s, battery powered scopes plugged in
  • Rack Fuses intact
  • Take pictures of electronics racks, optical tables
  • Replace PSL HEPA filter

Next, we brainstormed work that can be done to improve the interferometer performance, and what order/precedence they should take. 

In the end, it was decided that the plan for the next few weeks was to focus on improving the ALS noise levels, and, more importantly, seeking to make the performance more consistent. We need to know what is limiting us, and what we extent we can expect to improve things. To this end, I am working on reviving a ALS noise budget; using the noise budget from the green locking paper to inform a simulinkNB kind of thing.

Here are all of the items we listed during this brainstorming session.

Some near-term/priority tasks are:

  • Installing the accelerometers near MC1 and 2
  • Installing green steering PZT mirrors at the Y end table, commission dither alignment
  • Improving the X end green mode matching
  • ALS noise budgeting
  • Upgrade the realtime system to RCG v2.9

More down the line, other things we thought about were:

  • Cleanup of bench power supplies (FSS box has one, where else?)
  • Fixing the ETMX suspension issues 
  • Upgrade SOS suspension code to the appropriate aLIGO block
  • Upgrade to the green PDH electronics
  • Understanding / tuning the FSS servo laser PZT vs. PC crossover
  • Undestanding / tuning the 11MHz vs 55MHz modulation phase
  • Replacing the slow vxworks machines with the Acromag setup Aiden has set up for the CTN lab
  • QPD upgrade
  • New/better green beatbox
  • Finalize the manifestation of the IR beat control (Freq counter vs. fast DFD)
  • Explore the idea of using an analog output of the ALS beatbox as fast input to the CM board
  • Replace triple resonant EOM driving circuit with double resonant one
  • X end table layout/enclosure upgrade
  11345   Wed Jun 3 17:04:08 2015 ericqUpdatePEMWilcoxon Accelerometer Huddle Test

I've set up the Wilcoxon accelerometers on the SP table for a huddle test, to estimate their noise levels. 

They're clamped down to the table along the same axis, and their housings are in good contact, in hopes to make them as correlated as possible. 

Steve helped me move the DAQ cables from under the BS/PRM oplev table, to dropping from the cable tray above the POX table, so I could get them plugged in at the SP table. This also helps reduce the rats nest by the access connector, and is a fine location for when the accelerometers are attached at MC1 & MC2. 

A quick glance at DTT shows coherence of >0.9 from about 2-100Hz for all six. A three-corner-hat deal will provide an easy estimate of the noise floor of each one. If we feel like being fancy / accounting for possible gain differences, we could try some MISO wiener action, but that is probably overkill. 

Attachment 1: AccHuddle.jpg
AccHuddle.jpg
  11344   Wed Jun 3 11:55:52 2015 SteveUpdateOptical LeversPRM-BS oplev

I'm getting ready change the Newport Ultima U100-AC to SS-Polaris-K1  LOW DRIFT MIRROR MOUNTS

Note: there is only one lens in the PRM lunching path (  only realized later ) , so the spots are large ~ 3 mm at PRM qpd and ~4.5 mm at BS qpd

The spots are well centered.

Atm3,  the spots were well centered yesterday ( the PRM is misaligned in pitch and retsore does not work today )


 

Attachment 1: PRM-BS_oplEr_noLenses.png
PRM-BS_oplEr_noLenses.png
Attachment 2: PRM-BSopl_3jun2015.jpg
PRM-BSopl_3jun2015.jpg
Attachment 3: opServos.png
opServos.png
Attachment 4: 10d_trend_opl.png
10d_trend_opl.png
Attachment 5: opl_trend_40d.png
opl_trend_40d.png
  11343   Tue Jun 2 21:22:07 2015 rana, kojiConfigurationIOOAOM inserted in beam and aligned

We spent an hour today to put the AOM back in the beam before the PMC and verified that the diffraction is working.

  1. The fuse holder was missing from the rack. We inserted a 5A fuse. We expect that the quiesscent draw is < 0.5 A. The power is from the +24V Sorensen supply.
  2. The alignment was tricky, but we optimized it as well as we could in translation and the RZ direction. Its a fixed mount still.
  3. We noticed that according to the datasheet, the polarization is wrong! It wants S-Pol light and we're giving it P-Pol. How come no one noticed this? We expect that the efficiency is reduced because of this. We (Steve) need to brainstorm what kind of mount we can use there to mount it at 90 deg to the plane of the table.
  4. The lens after the AOM has f = +400 mm. The distance from the AOM to the lens is ~800-900 mm so its not so terrible. However, if someone were to put the AOM halfway between the turning mirrors there, the beam diffraction would be canceled.
  5. The AOM input impedance seems to be 50 Ohm as advertised. The previous Koji entry claim of 25 Ohm is mysterious. We checked the Ohmage by sending a signal into the AM input of the AOM using the DS345 which as a 50 Ohm output. 1 Vpp from the DS345 made 1 Vpp on the input of the AM input as measured by Oscope connected by T with high impedance setting.
  6. With 0.5 V offset and a 1 Vpp signal, we get ~20-25% modulation of the power.sad
  7. We have left it running with a 4444.4 Hz modulation and a small amplitude. This is to see if we can use this to measure the cavity poles of the MC and the arms.
  8. We noticed some hash on the Teed input monitor. It was backstreaming of the RF drive. Whoever uses this thing in an ISS feedback ought to make sure to put an RF choke between the servo and this AOM driver.

We also removed a 50/50 pickoff mirror which was used to take one of the NPRO -> EOM polarizer reject beams and send it across the table into a floppy dump. Its now hitting a closer floppy dump. Let's stop using these crappy anodized aluminum flappers anywhere, Steve.

We also noticed that the PMC REFL path uses a W1 from CVI to send the PMC reflection to the REFL RFPD. The dim beam from the AR coated surface is being used rather than the bright beam from the uncoated surface. Ooops. Steve, can you please order another W1 for 1064 from CVI, but get it with a 2-3 deg wedge angle? This one has a wedge which is too small.

  11342   Mon Jun 1 20:05:36 2015 ranaUpdateCDSRCG Upgrade Imminent

yes

Quote:

 

Additionally, we want to have some set of tests and diagnostics, to make sure we have not introduced unwanted behavior. 

To this end I will create some test models and DTT templates, where a series of measurements can be run, like

  • OLTF/delay measurement of a single all-digital loop within one model
  • OLTF/delay measurements of a few all-digital loops split across two models, using IPC communcation, RFM, dolphin
  • Hook up DAC -> resistor/amplifier/??? -> ADC, to check things like DAC output, ADC noise levels, IOP delays.

I'll run these test before touching anything, and make sure I understand all of the results, so that an apples-to-apples comparison can be made after the upgrade is complete. 

I got goosebumps just imagining this.

  11341   Mon Jun 1 15:22:19 2015 SteveUpdatePEMGuralps X- short cable is bad

 

Quote:

Koji and Steve,

The result: bad Guralp x-arm cable.

I will swap the short cables tomorrow at the base.

 

Short 46" long cables at the base plates were swapped. Their solderings looked horrible.

This cable actually worked at 5-5-2015

Bad cable at ETMY station now.  The new cable should be a little bit longer ~52"

Attachment 1: seismGur1-2.png
seismGur1-2.png
  11340   Mon Jun 1 10:26:53 2015 ericqUpdateCDSRCG Upgrade Imminent

We are planning on upgrading the 40 CDS system to the latest version of the LIGO RCG software, in three weeks when Jamie is back in town. 

Jamie and I spoke last week, to hash out a general plan for upgrade and preperations I can make in the meantime. 


Preparation of our models for the upgrade includes;

  • Check if any of the default RCG parts (filter modules, etc.) have substantially different default behavior / ports
  • Cleaning up unterminated/hanging connections in the diagrams (Jamie tells me RCG is more strict about this now)
  • Going through all of the build logs for our models to find what custom blocks are being pulled in from the userapps svn
  • Confirm all of our running blocks and models are committed to svn 
  • In a new, isolated, folder, checkout the latest version of the userapps repo
  • See what blocks have changed, and what model changes might be neccesary. 

Once we think we know what needs to be changed in our models, we can check out the latest version of the RCG source, without linking it as the active version, and create a new build directory, without touching the old one, and create new copies of the 40m models with the neccesary modifications. This way, we can work on getting all of the 40m models compiled, without touching any of the live, running, systems

Once our models are compiling successfully, we can work on building daqd, nds, mxstream, etc. 


Additionally, we want to have some set of tests and diagnostics, to make sure we have not introduced unwanted behavior. 

To this end I will create some test models and DTT templates, where a series of measurements can be run, like

  • OLTF/delay measurement of a single all-digital loop within one model
  • OLTF/delay measurements of a few all-digital loops split across two models, using IPC communcation, RFM, dolphin
  • Hook up DAC -> resistor/amplifier/??? -> ADC, to check things like DAC output, ADC noise levels, IOP delays.

I'll run these test before touching anything, and make sure I understand all of the results, so that an apples-to-apples comparison can be made after the upgrade is complete. 


Updates will be posted as I hash things out. I'm sure we have not yet thought of everything to think about and test, so ideas and feedback are very welcome. 

  11339   Mon Jun 1 08:32:14 2015 SteveUpdateSUSPRM damping restored

Local earthquake 3.8 Mag tripped only PRM

Vac monitor is not communicating.

PSL HEPA turned on
 

Attachment 1: indio3.8Meq.png
indio3.8Meq.png
  11338   Fri May 29 15:12:39 2015 KojiUpdateComputer Scripts / ProgramsChiara Backup Hiccup

Actual data

Attachment 1: backup_hours.pdf
backup_hours.pdf
  11337   Fri May 29 12:49:53 2015 KojiUpdateComputer Scripts / ProgramsChiara Backup Hiccup

In fact, the file access is supposed to be WAY faster now than in the RAID case.

As noted in ELOG 9511, it was SCSI-2(or 3?) that had ~6MB/s thruput. Previously the backup took ~2hours.
This was improved to 30min by SATA HDD on llinux1.

I am looking at /opt/rtcds/caltech/c1/scripts/backup/rsync.backup.cumlog

In fact, this "30-min backup" was true until the end of March. After that the backup is taking 1h~1.5h.

This could be related to the recent NFS issue?

  11336   Fri May 29 11:28:42 2015 ericqUpdateComputer Scripts / ProgramsChiara Backup Hiccup

I've changed the chiara local backup script to read a folder exclusion file, and excluded /users/public_html/detcharsummary, and things are working again. 

This was neccesary because the summary pages are being updated every half hour, which is faster than the time it takes for the backup script to run, so the file index that it builds at the start becomes invalid later on in the process. 


Thinking about chiara's disk, it strikes me that when we went from the linux1 RAID to a single HDD on chiara, we may have tightened a bottleneck on our NFS latency, i.e. we are limited to that single hard drive's IO rates. This of course isn't the culprit for the more recent dramatic slowdowns, but in addition to fixing whatever has happened more recently, we may want to consider some kind of setup with higher IO capability for the NFS filesystem. 

  11335   Fri May 29 02:05:08 2015 ericqUpdateLSCEnd Laser temperatures set

Both green beatnotes have been found with nominal amplitudes. (X: -30dBm Y: -20dBm), at temperatures which don't seem to be prone to mode hopping. 

X = 42.64, Y ~40.15

Both arms can lock on ALS, but as Koji mentioned in ELOG 11334, the ALS noise seems anomalously high. 


Details

The temperatures I posted in the previous log ended up not being so useful. To find the right end laser temperatures, I looked at the IR beatnotes on the control room analyzer out to 1GHz, and swept around the SLOW_SERVO2_OFFSET channels to change the laser temperature. During this time, the end green shutters were closed, the PSL shutter was closed, the FSS slow servo was off, and the FSS_SLOWDC was set to 0.0. (The green PSL shutter had to be open, because the IR beat fiber is coupled after it.)

For each arm, I found three temperatures where an IR beat could be observed; as Koji mentioned on Wednesday, we should use the middle mode. For each of the arms, scanning around from the middle to move the beat by +-1GHz did not cause a mode hop - the beat stayed visible on the scope. Once I found a real IR beat for the X arm, I took at look at the RF output of the X BBPD, and found my alignment from the other night was actually pretty good; I made a minor touch up to maximize the green beat. 

For the AUX X innolight, I was able to find the actual temperature of the laser crystal when at the correct point, remove the digital offset, and return to the same temperature with the controller front panel dial. This temperature is 42.64 degrees Celcius. There is no diode temperature control on the unit, as far as I could tell, but the maximum green transmitted power through the X arm is about the same as it's ever been, around GTRX=0.5. 

My motivation for doing this was that I always have problems remembering the historical typical values for the digital temperature offset. It seems much cleaner to me to set things up such that a beat is visible "at the origin" (i.e. FSS_SLOWDC=0 and SLOW_SERVO2_OFFSET=0) I suppose that this will also depend on what mode of the PMC we're locked to. During my time working today, its been locked near the middle of its range, currently sitting at 145V. 

However, for the AUX Y lightwave, I was a little perplexed to find that moving the the laser crystal setpoint around does not apear to change the real laser temperature at all. Thus I could not offload the digital offset in the same way. Aditionally, the lightwave controller does not have the same temperature measuring accuracy as the innolight, even with the back panel voltage readout that is hooked up to the multimeter that lives under the optical table. The best I can tell is around 40.1-40.2 degrees C. Y_SLOW_SERVO2_OFFSET of -10690 counts gives a beat <100MHz at FSS_SLOWDC=0. This is actually very close to the previous operating point, so the same mode seems to still work. 

The arms now easily lock on ALS, albeit with higher noise. With arms locked on ALS, POX and POY show >1kHz rms noise. frown 


I gave the PRFPMI locking script a few tries, but it's having problems keeping the PRMI locked. The ALS is a few times noisier than usual, and I haven't revisited the validity of the PRC angular feedforward, so I'm not so surprised. 

  11334   Thu May 28 21:10:46 2015 KojiUpdateGreen LockingALS-X noise hunting

I have been looking at the X-end ALS setup.
I was playing with the control bandwidth to see the effect to the phase tracker output (i.e. ALS err).
For this test the arm was locked with the IR and the green beat note was used as the monitor.

From the shape of the error signal, the UGF of the green PDH was ~10kHz. When I increased the gain
to make the servo peaky, actually the floor level of the ALS err became WORSE. I did not see any improvement
anywhere. So, high residual error RMS cause some broadband noise in the ALS??? This should be checked.
Then when the UGF was lowered to 3kHz, I could see some bump at 3kHz showed up in the ALS error.
I didn't see the change of the PSD below 1kHz. So, more supression of the green PDH does not help
to improve the ALS error?

Then, I started to play with the phase tracker. It seems that someone already added the LF booster
to the phase tracker servo. I checked the phase tracker error  and confirmed it is well supressed.
Further integrator does not help to reduce the phase tracker error.

For the next thing I started to change the offset of the phase tracker. This actually changes
the ALS error level!
The attached plot shows the dependence of the ALS error PSD on the phase tracker
output. At the time of this measurement, the offset of -10 exhibited the best noise level.
This was, indeed, factor of 3~5 improvement compared to the zero offset case below 100Hz.

I'm afraid that this offset changes the beat frequency as I had the best noise level at the offset of -5
with a different lock streatch. We should look at this more carefully. If the beat freq changes the offset,
this give us another reason to fix the beat frequency (i.e. we need the frequency control loop.

= Today's ALSX error would have not been the usual low noise state.
We should recover the nominal state of the ALS and make the same test =

Attachment 1: 150528_ALS.pdf
150528_ALS.pdf
  11333   Thu May 28 17:12:32 2015 ericqUpdateIOOFSS SLOW not engaged: is this intentional?

Yes, I had turned it off while looking for the PSL/X AUX beat, and forgot to turn it back on.

I will post an elog with more detail this evening, but I found a temperature which restored the X green beatnote at its nominal amplitude (-30dBm) with no mode hops within +-1 IR beat GHz, and offloaded the slow offset slider to the X-end laser crystal dial. I will look for the Y beatnote after dinner. 

Currently the control room analyzer is hooked up to recieve the Y IR and green beats; no X signals. 

  11332   Thu May 28 17:00:04 2015 KojiUpdateIOOFSS SLOW not engaged: is this intentional?

I found that FSS SLOW servo is not engaged. Is this intentional test to keep the NPRO temp constant?
This is making the FSS Fast unhappy (~ -7.5V right now).

  11331   Thu May 28 16:43:52 2015 SteveUpdatePEMGuralps swapped

Koji and Steve,

The result: bad Guralp x-arm cable.

I will swap the short cables tomorrow at the base.

 

Attachment 1: GursSwapped.png
GursSwapped.png
  11330   Thu May 28 15:10:44 2015 ranaUpdatePEMSeismic Confusion

You are plotting the STS channels, not Guralp. These are for the Trillium 240 seismometer.

Also, you cannot tell if the seismometer is working by plotting the MEAN trend. That just gives the average and we need the fluctuations. Better off looking at the spectrum like I did last time.

And....its not good enough to just do the bubble. You have to do the mass centering procedure that you and I did last time with the breakout paddle.

  11329   Thu May 28 10:42:19 2015 SteveUpdatePEMbad Guralp X-cable
Quote:

Tried swapping cables at the Guralp interface box side. It seems that all of our seismic signal problems have to do with the GUR2 cable being flaky (not surprising since it looks like it was patched with Orange Electrical tape!! rather than proper mechanical strain relief).

After swapping the cables today, the GUR2 DAQ channels all look fine: i.e. GUR1 (the one at the Y end) is fine, as is its cable and the GUR2 analog channels inside the interface box.

OTOH, the GUR1 DAQ channels (which have GUR2 (EX) connected into it) are too small by a factor of ~1000. Seems like that end of the cable will need to be remade. Luckily Jenne is still around this week and can point us to the pinout / instructions. Looks like there could be some shorting inside the backshell, so I've left it disconnected rather than risk damaging the seismometer. We should get a GUR1 style backshell to remake this cable. It might also be possible that the end at the seismometer is bad - Steve was supposed to swap the screws on the granite-aluminum plate on Thursday; I'll double check.

The Guralps were swapped.

What I did:   turned DC power off at 1X1,  hand carried them,  centered  each leveling bubbles, gently locked the jack screws and turned power back on.

ETMY at the east end now has CMG-T40-0008, sn T4157 as channel C1:PEM-SEIS_STS_1_Y_OUT_DQ.........

ETMX at south end now has CMG-T40-0053, sn T4Q17 as channel C1:PEM-SEIS_STS_2_Y_OUT_DQ.........

Conclusion:  Guralps are fine.  X  cable is bad. It was bad 6 months ago when it was made.

We can still swap the 3ft short cables at the granite base if Rana has not done it.

 

Attachment 1: Gurs180dXbad.png
Gurs180dXbad.png
Attachment 2: swappedGurs.png
swappedGurs.png
  11328   Wed May 27 17:14:08 2015 ericqUpdateLSCX Aux Laser crystal temperature changed

Rana suspects that the lack of X beatnote is related to the PSL laser temperature change (ELOG 11294). 

I used the information on the wiki and old elogs (wiki-40mELOG 6732), to deduce that the new end laser temperatures should be:

  • X end-> 38.98 C
  • Y end-> 35.80 C

I went out to the X end and found the laser crystal temperature set to 40.87, which is not what the measurements I linked to suggest would be the ideal temperature for the previous NPRO laser temperature of 30.89, which would be 37.02. I could not find any elog describing the choice of this setpoint.  

I've changed the X end laser crystal temperature to the value above. I've hooked up the X IR and green beatnotes to go the control room analzyer, and have been looking for the beatnote as I adjust the digital temperature offset, but haven't found it yet...

If this proves totally fruitless, we can just put the lasers back to their original temperatures, since it's unclear if it helped the PC drive noise levels. 

  11327   Wed May 27 15:20:54 2015 ericqUpdateComputer Scripts / ProgramsChiara Backup Hiccup

The local chiara backups are still failing due to vanished source files. I've emailed Max about the summary page jobs, since I think they're running remotely. 

  11326   Wed May 27 02:53:57 2015 ericqUpdateGeneralifo recovery log

Given my suspicion of fault with the X Green BBPD, Koji generously provided me with another one that he had confirmed to be working. 

However, I turns out I was mistaken. With the existing BBPD, I did indeed witness a beat in the RF output, but it is somehow something like 20dBm smaller than it should be. This is why I missed it the other night. Here's a video of a RF output on a scope, wherein the beat is only barely visible because I've set the trigger level very low. I could not make the beatnote any larger through any alignment motions; I had gotten to this point by doing near/far field overlap on the PSL table. 

I'm not sure what could have caused this. Mode mismatch? By eye, the beam spots looked about the same in the near and far fields, and we haven't had to touch the mode matching in quite some time... I've given up on trying to solve this for tonight. 


Just for kicks, I hooked up the fiber PD IR beatnotes as inputs to the ALS DFD. The X beat is too small to even really see above the control room analyzer's noise floor, but the Y beat looked big enough. With the arms locked on IR, the phase tracker output RMS was a few kHz, so not even worth thinking about any further. Not so surprising. 

Finally, I put back / hooked up everything in its nominal state. 

  11325   Tue May 26 19:57:11 2015 ranaUpdateComputer Scripts / ProgramsifoCoupling

Looks like a very handy code, especially with the real statistical tests.

I would make sure to use much smaller excitation amplitudes. Since the coupling is nonlinear, we expect that its only a good noise budget estimator when the excitation amplitude is less than a factor of 3 above the quiesscent excitation.

  11324   Tue May 26 11:05:10 2015 SteveUpdatePEMworked around ETMY seism.

The cable tray holder cross beam is removed and cut short for good access to seismometer.

Attachment 1: ETMYseismic.png
ETMYseismic.png
Attachment 2: cut.jpg
cut.jpg
  11323   Sun May 24 14:45:02 2015 KojiHowToGeneralHow to launch StripTools at specified locations

LLO Operator Tips:

koji.arai@cr9:/opt/rtcds/userapps/trunk/asc/l1/scripts/initial_alignment$ cat autostart_strips.sh 
 

#!/bin/bash

# Baffle window setup 1500x480

StripTool -xrm 'StripTool.StripGraph.geometry:1500x470+0+24' /opt/rtcds/userapps/trunk/asc/l1/scripts/initial_alignment/baffle_pd.stp &
sleep 1

# DC signals setup

StripTool -xrm 'StripTool.StripGraph.geometry:1500x470+0+470' /opt/rtcds/userapps/trunk/asc/l1/scripts/initial_alignment/dc_signals.stp &
sleep 1

# WFS prx mich sry setup

StripTool -xrm 'StripTool.StripGraph.geometry:1500x470+0-24' /opt/rtcds/userapps/trunk/asc/l1/scripts/initial_alignment/wfs_prx_mich_sry.stp &
sleep 1


exit

  11322   Sat May 23 22:43:10 2015 ericqUpdateGeneralifo recovery log

Running train-of-thought elog:


East N2 cylinder found empty, replaced. West is >2kpsi


Removed Yuta-specific code from damprestore. A grep for 'yuta' in the python files within /scripts/ shows some other occurances, but nothing that is really in use at this time. New feature of damprestore.py: remembers oplev status.


Ran LSCoffsets. 

WFS offsets relieved (all <20).

Adjusted FSS offset to minimize MC_FAST_MON

ASS ran (but the arm alignment has been astoundingly stable lately. I haven't touched it all this week)

ITMX is the only optic that got a correction over 20 counts. 

BS and *TM oplev spots look well centered, except for ITMX. 

I undid the gain reduction rana introduced because X ASS seemed to be really slow. It is currently fine in its older state. What's going on here?


Some network latency stuff is going on, even freezing up terminals when trying to write text files. This may (or may not) be correlated with the summary page rysnc jobs on nodus. It occurs to me that we have a DAQ ethernet network seperate from the martian network, but the frame transfers need to go through the martian network, since nodus is the only way out to the outside world. If we had a machine/gateway directly from the DAQ network to the caltech network, the martian network wouldn't get bogged down when frames are being uploaded


GTRY = 0.55, ok. Aligned GTRX to 0.52, also ok

Y beatnote was found easily. Have spent >30 minutes looking for X green beatnote. Typical FSS slow and X temperature ranges don't seem to be giving much. Will check the beat alignment with a scope, but if the beat is too high to begin with, it may not work...


I suspect a problem with the X Green BBPD

I could see the IR beatnote between the PSL and AUX X lasers at the input to the frequency counter. (I believe it is a real beatnote because it reacted as expected to the end temperature moving, and stabilizing the end laser to the arm). However, when placing the IR beatnote at a frequency which should've made the green beatnote visible on an analyzer and/or scope, no beatnote was found. I played with the beat alignment to no avail; the DC output of the BBPD behaved as expected, but I never saw anything in the RF output or on the control room analyzer. sad I also checked the beatnote signal chain by hooking up a 1mV 26MHz signal into the cable that hooks up to the RF out of the BBPD; the signal showed up clearly on the control room analyzer. 

I'm not sure what may have happened. ELOG 9996 may be related. 

I'm calling it a night. 

  11321   Fri May 22 18:09:58 2015 ericqUpdateComputer Scripts / ProgramsifoCoupling

I've started working on a general routine to measure noise couplings in our interferometers. Often this is done with swept sine measurements, but this misses the nonlinear part of the coupling, especially if the linear part is alreay reduced through some compensation or feedforward scheme. Rana suggested using a series of narrow band-limited noise injections. 

The structure I'm working on is a python script that uses the AWG interface written by Chris W. to create the excitations. Afterwards, I calculate a series of PSD estimates from the data (i.e. a spectrogram), and apply a two-sample, unequal variance, t-test to test for statisically significant increases in the noise spectra to try and evaluate the nonlinear contriubutions to the noise. I've started a git repository at github.com/e-q/ifoCoupling with the code. 

So far, I've tested one such injection of noise coupling from the ETMX oplev error point to the single arm length error signal. It's completely missing the user interface and structure to do a general series of measurements, but this is just organizational; I'm trying to get the math/science down first. 

Here's a result from today:

Median, instead of the usual mean, PSDs are used throughout, to reject outliers/glitches.

The linear part of the coupling can be estimated using the coherence / spectrum height in the excitation band, but I'm not sure what the best what to present/paramerize the nonlinear parts of each individaul excitation band's result is.

Also, I anticipate being able to write an excitation auto-leveling routine, gradually increasing the exctiation level until the excited spectrum is some amount  noisier than the baseline spectrum, up to some maximum amount configurable by the user. 

The excitation shaping could probably be improved, too. It's currently and elliptic + butterworth bandpass for a sharp edge and rolloff. 

I'm open to any thoughts and/or suggestions anyone may have!

Attachment 1: ETMX_PIT_L_coupling.png
ETMX_PIT_L_coupling.png
  11320   Fri May 22 12:09:57 2015 ranaUpdateSUSDampRestore script problem

I will move it back. We need to fix our scripts to not use any users/ libraries ever again.

Quote:

PRM watchdog tripped, but the damprestore.py script wouldn't run. 

It turns out the script tries to import some ezca stuff from /users/yuta (angry), which had been moved to /users/OLD/yuta (crying). 

I've moved the yuta directory back to /users/ until I fix the damprestore script. 

 

  11319   Fri May 22 11:59:54 2015 ericqUpdateSUSDampRestore script problem

PRM watchdog tripped, but the damprestore.py script wouldn't run. 

It turns out the script tries to import some ezca stuff from /users/yuta (angry), which had been moved to /users/OLD/yuta (crying). 

I've moved the yuta directory back to /users/ until I fix the damprestore script. 

  11318   Wed May 20 11:41:59 2015 ericqUpdateGeneralsome status

West cyclinder is empty, east is at 2000 psi; regulated N2 pressure is 64psi. I'll replace the west one after the meeting.

  11317   Wed May 20 03:08:27 2015 ranaUpdateGeneralsome status

I think that the real clue was that the dropouts are in some channels and not in others:

https://nodus.ligo.caltech.edu:30889/detcharsummary/day/20150519/psl/

As it turns out, the channel with no dropouts is the RAW PSL RMTEMP channel. All the others are the minute trends. So something is up with the trend making or the trend reading in the cluster.

Quote:

There's a few hours so far after today's c1cal shut off that the summary page shows no dropouts. I'm not yet sure that this is related, but it seems like a clue.

 

  11316   Tue May 19 19:24:30 2015 ranaUpdatePEMSeismic BLRMS filters

I was wondering about the design of the BLRMS fitlers for the seismic channels since the STS ones seem to have so little gain compared to the Guralps.

Here are some plots of the Bode magnitude and impulse responses of the bandpass filters (before the low passing). There's a bunch of entries from Masha on this from her SURF summer. Can anyone comment on why they are all so different?

One of the old Masha entries speaks of designing the lowpass filter in an intelligent way: by adjusting the filter order until the power in the stopband is less than 1% of the power in the passband. Seems like we could do that for bandpass too. For now I have made the names reasonable and changed all of the BP filters to 4th order Butterworth.

Also, it turns out that the Vel2Vel (gain ~0.02) filters were mistakenly on in the STS BP filter banks. The GUR inputs have a gain to scale the counts to velocity, but the STS seem to already be in microns/sec (where is this gain?) so I turned off and deleted the Vel2Vel filters; in any case the gain should not be done seperately in each BP bank, but altogether before the BP filtering.

Attachment 1: BLRMS_BP.pdf
BLRMS_BP.pdf
Attachment 2: BLRMS_imp.pdf
BLRMS_imp.pdf
  11315   Tue May 19 18:55:12 2015 rana, ericQUpdateGeneralsome status

After one day the pressures are east/west = 2200/450 PSI

Quote:

 

  1. New east pressure reading is 2500 PSI. Regulated N2 pressure is 68 PSI.

 

  11314   Tue May 19 18:38:33 2015 ranaUpdateCDSMXstream restart script working (beta)

Good catch; that was some seriously bad programming on my part. I had some undeclared variable garbage going on. I fixed it and re-implemented the script in CRON on megatron. The log file shows that it has detected no problems for the last several checks. I'll check it again tomorrow to see if its doing good or bad.

  11313   Tue May 19 17:53:38 2015 ranaUpdateModern ControlBrushing up on Wiener Filtering

Good to see that misofw.m is still alive and well. Todo:

  1. Apply weighting to the filter via time domain SOS prefiltering of the signals (see Jenne's code from the LLO Global FF paper)
  2. Consider using some of the MC SUSPOS signals. Its not strictly legal, but I wonder if its responsible for out noise below 1 Hz.
  3. Steve was in the midst of hooking up the Wilcoxones to get better subtraction at 5-15 Hz, but couldn't find cables. We need to hunt them down in W Bridge.
  4. Attach one of the medium or big Lings to the MC2 chamber platform and see if we can do this in hardware without driving the suspension. WE want to use a DAC channel (from where?) and pipe it to the Ling using a medium high current drive. Might start with the 50 Ohm output of the SR560 and later use a BUF634 ala Crackle coil driver.
  11312   Tue May 19 17:03:34 2015 KojiUpdateCDSMXstream restart script working (beta)

AutoMX is resetting mx_stream every 5 minutes. Basically everytime AutoMX is called,
it resets mx_stream. Is the mx_stream stalling such often? Or the script is detecting false alerms?


> tail -200 /opt/rtcds/caltech/c1/scripts/cds/autoMX.log

Tue May 19 16:43:01 PDT 2015
LSC - FB bad. Runnning restart:
 * Stopping mx_stream ...                                                 [ ok ]
 * Starting mx_stream ...                                                 [ ok ]
Connection to c1sus closed.
 * Stopping mx_stream ...                                                 [ ok ]
 * Starting mx_stream ...                                                 [ ok ]
Connection to c1lsc closed.
 * Stopping mx_stream ...                                                 [ ok ]
 * Starting mx_stream ...                                                 [ ok ]
Connection to c1ioo closed.
 * Stopping mx_stream ...                                                 [ ok ]
 * Starting mx_stream ...                                                 [ ok ]
Connection to c1iscex closed.
 * Stopping mx_stream ...                                                 [ ok ]
 * Starting mx_stream ...                                                 [ ok ]
Connection to c1iscey closed.
0
Tue May 19 16:48:02 PDT 2015
LSC - FB bad. Runnning restart:
 * Stopping mx_stream ...                                                 [ ok ]
 * Starting mx_stream ...                                                 [ ok ]
Connection to c1sus closed.
 * Stopping mx_stream ...                                                 [ ok ]
 * Starting mx_stream ...                                                 [ ok ]
Connection to c1lsc closed.
ssh_exchange_identification: read: Connection reset by peer
 * Stopping mx_stream ...                                                 [ ok ]
 * Starting mx_stream ...                                                 [ ok ]
Connection to c1iscex closed.
 * Stopping mx_stream ...                                                 [ ok ]
 * Starting mx_stream ...                                                 [ ok ]
Connection to c1iscey closed.
0
Tue May 19 16:53:01 PDT 2015
LSC - FB bad. Runnning restart:
 * Stopping mx_stream ...                                                 [ ok ]
 * Starting mx_stream ...                                                 [ ok ]
Connection to c1sus closed.
 * Stopping mx_stream ...                                                 [ ok ]
 * Starting mx_stream ...                                                 [ ok ]
Connection to c1lsc closed.
 * Stopping mx_stream ...                                                 [ ok ]
 * Starting mx_stream ...                                                 [ ok ]
Connection to c1ioo closed.
 * Stopping mx_stream ...                                                 [ ok ]
 * Starting mx_stream ...                                                 [ ok ]
Connection to c1iscex closed.
 * Stopping mx_stream ...                                                 [ ok ]
 * Starting mx_stream ...                                                 [ ok ]
Connection to c1iscey closed.

  11311   Tue May 19 16:18:57 2015 ericqUpdateGeneralcrons fixed

I wrapped rampdown.py in rampdown.sh, which is just these lines:

#!/bin/bash
source /ligo/cdscfg/workstationrc.sh
/opt/rtcds/caltech/c1/scripts/SUS/rampdown.py > /dev/null 2>&1

This is now what megatron's cron runs. It appears to be working.

I also added the workstationrc line to the n2 and chiara HDD checking scripts that run on nodus, which should resolve the issue from ELOG 11249

ELOG V3.1.3-