40m QIL Cryo_Lab CTN SUS_Lab TCS_Lab OMC_Lab CRIME_Lab FEA ENG_Labs OptContFac Mariner WBEEShop
  40m Log, Page 213 of 344  Not logged in ELOG logo
ID Date Author Type Category Subject
  6617   Mon May 7 21:19:00 2012 JenneUpdateSUSETMX oplev

I'm still not super happy with the low power level of the ETMX oplev, so I went to investigate.

OpLevETMX_1100days.png

This is a 3-year plot.  The first ~year in the plot, the oplev sum is ~15000 cts, and in the most recent year, it's ~1000 cts.  A new HeNe was installed in May 2011 (elog 4686), with an output of 2.6mW, after the old one had died.  When the new one was installed, Steve said that it was giving ~1400 counts, so maybe 1000 isn't too, too embarrassing.  There is, however, a lens on the injection side, which is AR coated for 1064.  The power before this lens (measured with the Ophir, set to 632nm) was ~2.6mW.  The power after this lens was ~1.5mW.  Now THAT is embarrassing.  I'm adding replacing that lens to the to-do list (elog 6595), although I don't want to do it until such a time (maybe in an hour, maybe in a few days?) when I've got the Xarm locked / aligned, so I can nicely re-center the oplev.  UPDATE:  The lens is a KBC 037 (-100) lens, and the sticker on the lens mount says coated for 1064.  We don't have any KBC037's in the visible lens kits, so we need to get one before I can do this replacement (PURCHASED 10pm).

There is an elog (elog 5004) from July 2011, which mentions that these channels have not been saved for a long time, so that's why there's the year-long gap. 

 

  6616   Mon May 7 21:05:38 2012 DenUpdateCDSbiquad filter form

I wanted to switch the implementation of IIR_FILTER from DIRECT FORM II to BIQUAD form in C1IOO and C1SUS models. I modified RCG file /opt/rtcds/rtscore/release/src/fe/controller.c by adding #define CORE_BIQUAD line:

#ifdef OVERSAMPLE
#define CORE_BIQUAD      
#if defined(CORE_BIQUAD)

C1IOO model compiled, installed and is running now. C1SUS model compiled, but during installation I've got an error:

controls@c1sus ~ 0$ rtcds install c1sus


Installing system=c1sus site=caltech ifo=C1,c1
Installing /opt/rtcds/caltech/c1/chans/C1SUS.txt
Installing /opt/rtcds/caltech/c1/target/c1sus/c1susepics
Installing /opt/rtcds/caltech/c1/target/c1sus
Installing start and stop scripts
/opt/rtcds/caltech/c1/scripts/killc1sus
Performing install-daq
Updating testpoint.par config file
/opt/rtcds/caltech/c1/target/gds/param/testpoint.par
/opt/rtcds/rtscore/branches/branch-2.5/src/epics/util/updateTestpointPar.pl -par_file=/opt/rtcds/caltech/c1/target/gds/param/archive/testpoint_120507_205359.par -gds_node=21 -site_letter=C -system=c1sus -host=c1sus
Installing GDS node 21 configuration file
/opt/rtcds/caltech/c1/target/gds/param/tpchn_c1sus.par
Installing auto-generated DAQ configuration file
/opt/rtcds/caltech/c1/chans/daq/C1SUS.ini
Installing EDCU ini file
/opt/rtcds/caltech/c1/chans/daq/C1EDCU_SUS.ini
Installing Epics MEDM screens
Running post-build script

ERROR: Could not find file: test.py
Searched path: :/opt/rtcds/userapps/release/cds/c1/scripts:/opt/rtcds/userapps/release/cds/common/scripts:/opt/rtcds/userapps/release/isc/c1/scripts:/opt/rtcds/userapps/release/isc/common/scripts:/opt/rtcds/userapps/release/sus/c1/scripts:/opt/rtcds/userapps/release/sus/common/scripts:/opt/rtcds/userapps/release/psl/c1/scripts:/opt/rtcds/userapps/release/psl/common/scripts
Exiting
make: *** [install-c1sus] Error 1

Jamie, what is this test.py?

  6615   Mon May 7 20:15:37 2012 DenUpdateIOOWFS noise in MC

Quote:

OK. Then we should make this number bigger such that the coherence is still completely maintained.
Is this set in the auto locker? Or manually set?

Quote:

  This effect is due to C1:IOO-WFS1_PIT_LIMIT=2000. When I turned if off, coherence between C1:IOO-WFS_PIT input and output signals restored

 

 

LIMIT is set manually, auto locker does not change it. I've put C1:IOO-WFS1_PIT_LIMIT=4000, it seems to be fine for now.

  6614   Mon May 7 19:39:52 2012 KojiUpdateIOOWFS noise in MC

OK. Then we should make this number bigger such that the coherence is still completely maintained.
Is this set in the auto locker? Or manually set?

Quote:

  This effect is due to C1:IOO-WFS1_PIT_LIMIT=2000. When I turned if off, coherence between C1:IOO-WFS_PIT input and output signals restored

 

  6613   Mon May 7 17:28:29 2012 DenUpdateIOOWFS noise in MC

Quote:

One filter bank adds noise to WFS signals and because of that we loose coherence between MC_F and seismic motion.

 This effect is due to C1:IOO-WFS1_PIT_LIMIT=2000. When I turned if off, coherence between C1:IOO-WFS_PIT input and output signals restored

wfs_limit.png

The other thing is that WFS actuate on the angular motion, but this couples to the position motion. We need to diagonalize the actuator.

  6612   Mon May 7 12:57:34 2012 DenUpdateIOOWFS noise in MC

I've measured coherence between seismometer signals and OSEMS of MC 1,2,3

gur_suspos_coh.png

GUR1 was rotated on the angle pi / 4 relative to x arm to match suspos axis of MC1 and MC3. Coherence between MC2 and GUR2_X is high, between MC3 and GUR1_X is ~0.2 but is compensated by GUR1_Y, between MC1 and GUR1_Y below 1Hz is ~0.5 and is not compensated by GUR1_X.

Then I measured coherence between GUR1_Y and MC3 OSEMS in 3 regimes :

  • FEEDBACK OFF, WFS OFF
  • FEEDBACK ON, WFS OFF
  • FEEDBACK ON, WFS ON

mc1_gur.png

Once WFS are ON, signal becomes noisy, FEEDBACK is OK. Then I measured coherence between MC_F and GUR2_X with WFS ON and OFF

mcl_wfs.png

Coherence between MC_F and GUR2_X is less when WFS are ON in the range 0.2 - 1 Hz and 4 - 10 Hz. Moreover PSD of MC_F seems to be higher at 10 - 100 Hz. But this may be caused by other reasons.

Then I measured the coherence between IN and OUT signals in WFS_SERVO

wfs_servo.png

One filter bank adds noise to WFS signals and because of that we loose coherence between MC_F and seismic motion.

  6611   Mon May 7 01:07:58 2012 DenUpdateIOOseismic in mcf

I tried to figure out where additional (to seismic) noise enters to MC_F, so the coherence below 1 Hz is low (~0.2-0.5). I've examined noise in the path

PD -> DEMOD -> MC BOARD -> FSS -> PZT Controller -> LASER

  • I terminated ADC and measured its noise
  • connected MC BOARD OUT to ADC, terminated INPUT1 of the MC BOARD and measured noise
  • connected DEMOD Q OUT to MC BOARD INPUT1, terminated PD INPUT on the DEMODULATOR and measured noise
  • connected PD to DEMOD, blocked the beam incident to the PD and measured noise

pd-mcf.jpeg

 MC BOARD noise shows up only below 0.1 Hz and at 10 Hz where the whitening filter starts to work. SNR is ~2 at 4 Hz, so we might want to slightly improve whitening filter. But other then that path PD->DEMOD->MC BOARD is not responsible for additional noises below 1 Hz.

Next I connected SR785 to the laser and measured the closed loop feedback signal while MC was locked.

fb.png     coh_err_fb.png

Coherence between MC BOARD OUT1 and feedback signal is high enough to assume that FSS and PZT controller are also not responsible for additional noise.

From the other hand MC2 SUSPOS measured by OSEMS shows good coherence with GUR1_X

mc2_suspos.jpg

That means that MC2 is indeed driven by seismic motion. In order to figure out if this is the case for MC1 and MC3, I rotated GUR2 by 45 degrees. When it will calm down, I'll measure coherence between OSEMs and seismic motion.

  6610   Sun May 6 01:41:55 2012 DenUpdatePEMseismic in x,y arms

I locked x,y arms and measured coherence between POS{X,Y}11_I_ERR, MC_F and seismometer signals.

coh_xyf.png

Surprisingly, coherence between POSY and GUR1Y is low, but with GUR1X is relatively high. I wonder if this is due to MC that brings this x-axis noise to the arms.

  6609   Sun May 6 00:11:00 2012 DenUpdateCDSmx_stream

c1sus and c1iscex computers could not connect to framebuilder, I restarted it, did not help. Then I restarted mx_stream daemon on each of the computers and this fixed the problem.

sudo /etc/init.d/mx_stream restart

  6608   Sat May 5 20:42:59 2012 DenUpdatePSLPMC

[Koji, Den]

Koji was right that I misaligned everything during the alignment work. I assumed that PMC should autolock and when I saw that it did not, I thought the laser is misaligned.

What we did:

1. Aligned mirrors to get the beam on the PD PMC REFL and PMCR camera. The PSL-PMC_RFPDDC was ~800 mV.

2. We disabled PMC servo, switching it to test position and changed "DC output adjust" by 0.01 in a loop

while true
do
    ezcawrite "C1:PSL-PMC_RAMP" -4.50
    ezcastep "C1:PSL-PMC_RAMP" "+0.01,450" -s "0.1"
    ezcawrite "C1:PSL-PMC_RAMP" 0.0
    ezcastep -s "0.1" -- "C1:PSL-PMC_RAMP" "-0.01,450"
done

3. While the script was running we adjusted the position of the beam on the far PMC mirror looking at an IR viewer. The goal is to align two steering mirrors to catch some resonances. We monitored them on the oscilloscope and on the PMCT camera.

4. We locked PMC and aligned steering mirrors.

  6607   Sat May 5 12:23:38 2012 KojiUpdatePSLPMC

No matter how you connect/disconnect, touching the laser may cause the PMC unlocked.

At least, I don't see the PMC reflection on the PD.
This means that the beam towards the PMC is largely misaligned.

If you are not sure what is misaligned, stop touching the table.
Close the shutter of the laser on the laser housing and leave the optics as they are.

  6606   Sat May 5 10:20:21 2012 DenUpdatePSLPMC

Quote:

I suspect that it was just unlocked when you had disconnected the cable.

There is not reflection now. It seems that it is now misaligned after the alignment work.

So what you need is "align while scanning PZT -> lock -> align".

Quote:

Unfortunately, after the measurement when I unplugged BNS connector from the laser, I misaligned PMC. For several hours I adjusted the mirrors but could not significantly improve transmitted signal. I'll return to this issue tomorrow.

 

 No, no, it was unlocked after I connected the cable back. The beam was even not on the PMC. I'll try PZT -> lock -> align.

  6605   Sat May 5 09:13:02 2012 KojiUpdatePSLPMC

I suspect that it was just unlocked when you had disconnected the cable.

There is not reflection now. It seems that it is now misaligned after the alignment work.

So what you need is "align while scanning PZT -> lock -> align".

Quote:

Unfortunately, after the measurement when I unplugged BNS connector from the laser, I misaligned PMC. For several hours I adjusted the mirrors but could not significantly improve transmitted signal. I'll return to this issue tomorrow.

 

  6604   Sat May 5 01:24:07 2012 DenUpdatePSLPMC

I was interested what whitening filter do we have between MC servo and ADC. The shape is in the figure below, SR provided 1 V white noise. Before the whitening filter MC_F is measured in Volts with SR and ADC (for ADC the shape is calculated using the whitening filter form):

whitening.jpg     mcf_v.png

I also wondered if FSS or PZT servo can add noise to the mode cleaner length signal and what is their gain. It should be big, as the laser's calibration is ~1 MHz/V => to account for seismic noise of 10^-6 m at 1 Hz, the voltage given to the laser should be ~ 1 V. And it is indeed the case. The gain is ~1000. I measured the coherence between MC_F and the laser fast input. It is 1 in the range measured (0.05 - 100 Hz). FSS and PZT do not add significant noise.

Unfortunately, after the measurement when I unplugged BNS connector from the laser, I misaligned PMC. For several hours I adjusted the mirrors but could not significantly improve transmitted signal. I'll return to this issue tomorrow.

  6603   Fri May 4 17:46:46 2012 JenneUpdateGreen LockingPSL doubling oven back on

While walking past the PSL, I noticed that the PSL's doubling oven's heater was still disabled from the power outage.  As with the ETMY heater, I hit the Enable button, and it started warming up (according to the front panel at least).

  6602   Fri May 4 17:44:42 2012 JenneUpdateGeneralOpLev 90-day trend

Quote:

After fixing the PRM tonight, all of the oplevs look okay. 

.....except ITMX, which looks like it's power dropped significantly after the CDS upgrade.  To be investigated tomorrow.

 I had a look-see at ITMX's oplev.  I can't see any clipping, so maybe the power is just low?  One thing that was funny though is the beam coming directly from the laser.  There is the main, regular beam, and then there is a thin horizontal line of red light also coming straight out of the laser.  I don't know what to do about that, except perhaps put an iris right after the HeNe, to block the horizontal part?  I'm not sure that it's doing anything bad to the optic though, since the horizontal part gets clipped by other optics before the beam enters the vacuum, so there is no real irregularity on the beam incident on the QPD.

I realigned the oplev on the QPD, using last night's ITMX alignment + whatever drift it picked up over night, so it may need re-recentering after Xarm is nicely aligned.

  6601   Thu May 3 22:37:44 2012 JenneUpdateGeneralOpLev 90-day trend

After fixing the PRM tonight, all of the oplevs look okay. 

.....except ITMX, which looks like it's power dropped significantly after the CDS upgrade.  To be investigated tomorrow.

Attachment 1: OpLevTrends_90days_Ending3May2012
  6600   Thu May 3 21:13:48 2012 KojiSummarySUSITMX/PRM/BS OPLEV aligned

[Jenne/Den/Koji]

We locked Xarm/Yarm and manually alignmed ITMX/ITMY/BS/ETMX/ETMY/PZT1/PZT2.

ITMY OPLEV was largely misaligned ==> The beam was centered on the QPD.

----

Then we aligned PRM using SB locking PRMI.

We noticed that OPLEV servo does not work. It made the PRM just noiser.

We went into the PRM table and found that the OPLEV beam was clipped in the vacuum chamber.
We tried to maximize the reflected beam from the window by touching the steering mirrors at the injection side.

Then the reflected beam was introduced to the center of the QPD.

After the alignment, the OPLEV QPD SUM increased to 4000ish from 200ish.
According to the OPLEV trend data, this is a nominal value of the QPD SUM.

Now the OPLEV servo does not go crazy.

 --

BS OPLEV beam was centered on the QPD.

  6599   Thu May 3 19:52:43 2012 JenneUpdateCDSOutput errors from dither alignment (Xarm) script

epicsThreadOnceOsd epicsMutexLock failed.
Segmentation fault
Number found where operator expected at -e line 1, near "0 0"
        (Missing operator before  0?)
syntax error at -e line 1, near "0 0"
Execution of -e aborted due to compilation errors.
Number found where operator expected at (eval 1) line 1, near "*  * 50"
        (Missing operator before  50?)
epicsThreadOnceOsd epicsMutexLock failed.
Segmentation fault
Number found where operator expected at -e line 1, near "0 0"
        (Missing operator before  0?)
syntax error at -e line 1, near "0 0"
Execution of -e aborted due to compilation errors.
syntax error at -e line 1, at EOF
Execution of -e aborted due to compilation errors.
Number found where operator expected at (eval 1) line 1, near "*  * 50"
        (Missing operator before  50?)
epicsThreadOnceOsd epicsMutexLock failed.
status : 0
I am going to execute the following commands
ezcastep -s 0.6 C1:SUS-ETMX_PIT_COMM +-0,50
ezcastep -s 0.6 C1:SUS-ITMX_PIT_COMM +,50
ezcastep -s 0.6 C1:SUS-ETMX_YAW_COMM +,50
ezcastep -s 0.6 C1:SUS-ITMX_YAW_COMM +-0,50
ezcastep -s 0.6 C1:SUS-BS_PIT_COMM +0,50
ezcastep -s 0.6 C1:SUS-BS_YAW_COMM +0,50
hit a key to execute the commands above

  6598   Thu May 3 17:15:38 2012 KojiUpdateLSCc1iscaux2 rebooted/burtrestored

[Jenne/Den/Koji]

We saw some white boxes on the LSC screens.
We found c1iscaux2 is not running.

Once the target was power-cycled, these epics channels are back.
Then c1iscaux2 were burtrestored using the snapshot at 5:07 on 4/16, a day before the power glitch.

  6597   Thu May 3 13:35:56 2012 JenneBureaucracyGeneral40m Meeting Action Items

Action Items:

IFO:

         Arm cavity sweeps, mode scan - JENNE

        Align AS OSA (others?) - JENNE       

        Investigate PRMI glitches, instability (take PRM oplev spectra locked, unlocked, to see if PRM is really moving) - KOJI, JENNE, DEN

         Connect up beatbox for Xarm use - KOJI with JENNE       

 

TT:

Prepare electronics for TTs (coil drivers) - JAMIE

In-air TT testing to confirm we can control / move TTs before we vent (starting in ~2 weeks) - SURESH

Connect TTs to digital system and controls, lay cables if needed - JAMIE with SURESH

OAF:

OAF comparison plot, both online and offline, comparing static, adaptive and static+adaptive - DEN

Static-only OAF noise budget (Adaptive noise budget as next step) - DEN

Build amplifiers for new small microphones - DEN 

SLC:

Black glass: to clean&bake - KOJI

Scattered light measurement at the end stations: design / confirmation of the mechanical parts/optics/cameras - JAN

MM:

IPPOS beam measurement - SURESH with JENNE

AS beam measurement (if beam is bright enough) - SURESH and JENNE

Mode matching calculations, sensitivity to MC waist measurement errors, PRM position - JENNE

Think up diagnostic measurement to determine mode matching to PRC while chambers are open, while we tweak MMT - JAMIE, JENNE, KOJI, SURESH

Computers:

       Upgrade Rossa, Allegra to Ubuntu, make sure Dataviewer and DTT work - JAMIE

 

  6596   Thu May 3 13:19:17 2012 JenneUpdateLockingMore success last night

I locked each arm, and the Michelson last night, no problems after I increased the Yarm gain from 0.1 to 0.2 .  I checked the green beam alignment just before going home, and both of the green beams are locking on ~03 or 04 modes, so aligning them is on the list for today.

  6595   Thu May 3 13:07:05 2012 KojiBureaucracyGeneralInterferometer check-up session

From 2PM, Jenne/Den/Koji. Finished at 10:30PM

  • General
    • More LCD screens! Wider screens!
      • Allegra - only can handle single head right now, but should prepare for the eventual upgrade of that videocard/whole computer and get 2 monitors.
      • Every workstation should have 2 24" monitors.  We currently have 2 good ones (one on Rossa, one on Pianosa), so we need 6.
    • Remove Control room shelf currently holding OSA 'scopes
    • Adjust both TV shelves so that they're at the same height, ~6" higher than the current left shelf.
      This gives space for stacking the 24" monitors for each computer.
    • Audio system for the signals!!!! Even a crappy one!
           /cvs/cds/ligo/centos5.5/cds/project/gds-2.14.2/Monitors/rockIFO
           /cvs/cds/gds/gds/Monitors/rockIFO
           (really /cvs/cds, not /opt)

  • IOO
    • PZT or Picomotor mounts for PSL/ALS beams
    • ALS on the both arm simultaneously / common/diff ALS scripts
  • SUS
    • PRM OPLEV servos are sometimes not stable. (fixed, JCD/DM/KA, 5/2/2012)
    • ETMX oplev - replace 1064nm lens with 632nm lens (KBC037, -100)
  • IFO
    • It seems that there is an occasional common-mode power transient in the arm transmissions.
      -> Track down the cause and fix.
    • Fix arm ASS even on CentOS
    • Drift of the green incident axis -> Assess the amount of the drift / replace the mount
    • Calibration of POP22 / AS110
    • PMC/IMC/ARM characterization (loss, finesse, reflectivity, etc)
  • CDS
    • Capture OSA signals in CDS (the 'scope TDS1001B has a USB port in the back for connecting to the computer)
    • Transmon (arms) for high and low power
    • POX11 whitening is not toggling the analog whitening???
    • CentOS machines cannot open simulink models properly (get "Bad Link"s everywhere, so you can't do anything useful).
    • Dataviewer and DTT can't connect to framebuilder from CentIOS machines
  • Electronics
    • Actuator noise level characterization (coil driver response in V/m & coil driver noise level V/rtHz)
    • Beat box
    • I/Q DFD
    • Improvement of POP22/110/AS110 RF circuits.
  • MEDM
    • OL alert in the watch dog screen (this is awesome!) should have small text saying "OL" (done, JCD, 5May2012)
    • Complete 40m overview screen - everything should be clickable with pseudo 3D icons
    • BETTER SUSPENSION SCREEN!!!
    • Script to generate a MEDM tree
    • Resurrect MEDM snapshots
       
    • The LSC screen has some "white" boxes -> investigate and fix them (done, KA, 5/2/2012)
    • Make the LSC control screen (compact) in the vertical arrangement (done, KA, 5/2/2012)
    • The signals of the watch dog screen should go red if any of the WDs are not enabled.
      =>Copy the logic of the signals in LSC screen.
      (done, KA, 5/2/2012)
    • Remove c1gfd from the cds status screen (done, KA, 5/2/2012)
    • Add "BURT" status indicators on the cds status screen (done, KA, 5/2/2012)
  • SCRIPT
    • Locking scripts with integrator triggers / disabling when unlocked
    • Daily diagnosis of the MC spot positions (there must be something already...)
    • Daily/occasional adjustment of the incident axis on the MC
    • Suspension "misalign/restore" script is too wild => make a new script (Done, JCD, 7May2012)
    • IFO_CONFIGURE scripts still do a burt restore, so step the optics.  Need to remove optic alignment from config .snap files, use reg restore script instead.
    • OPLEV/OSEM trending script before the IFO work for diagnosis.
    • Auto-locker for PMC/Arm/etc
  • Video
    • If each video screen has a caption, that would be great
    • GUI interface of "videoswitch" Mike!
    • Mouse Jiggler for Zita (called Swinput?)
  • Ubuntu
    • burttoday is not aliased in ubuntu. burttoday:      aliased to cd /opt/rtcds/caltech/c1/burt/autoburt/today/
    • ASS doesn't run on Ubuntu!
      ezcawrite: error while loading shared libraries: libca.so: cannot open shared object file: No such file or directory
      ezcaswitch: error while loading shared libraries: libca.so: cannot open shared object file: No such file or directory
       
  6594   Wed May 2 21:04:06 2012 DenUpdatePEMEM 172 coherence

I measured coherence between 2 EM 172 microphones in a "quiet room" with SR785

em_coh.png

High-frequency noise (>2k) is SR785 noise - I'm not using any amplifier now, the signal from microphone is sent directly to SR785 and is weak at high frequencies.

  6593   Wed May 2 19:47:20 2012 DenUpdatePEMEM 172 nonlinearities

I've checked new small EM172 microphones for nonlinearities using Koji's Mac Book speakers. EM172 + Mac nonlinearities are presented at the figure

500Hz_noise.png

The Mac's sound frequency was specified to be 500 Hz. Harmonics are seen at 1k, 1.5k, but their amplitude is ~30 times less.

  6592   Tue May 1 17:42:15 2012 Mike J.UpdateComputersSensoray

I've upgraded the Sensoray GUI so it can now switch the video channel it receives, thanks to the videoswitch script.

V4L2_Capture_Demo_r01.png

  6591   Tue May 1 08:18:50 2012 JamieUpdateCDSFrame Builder is down

Quote:

 

I tried restarting the fb in two different ways.  Neither of them re-established the connection to dtt or epics.

 Please be conscious of what components are doing what.  The problem you were experiencing was not "frame builder down".  It was "dtt not able to connect to frame builder".  Those are potentially completely different things.  If the front end status screens show that the frame builder is fine, then it's probably not the frame builder.

Also "epics" has nothing whatsoever to do with any of this.  That's a completely different set of stuff, unrelated to DTT or the frame builder.

  6590   Mon Apr 30 22:58:57 2012 JenneUpdateElectronics11MHz Marconi set to default after power outage

After a power outage, a Marconi comes back to it's defaults.  It needed to be reset to the values in elog 5530.  I'm putting a label on the Marconi so we don't have to look it up next time.

Before fixing the Marconi, POY11, AS11 and AS55 all looked like noise, no real signals, even though the arm is flashing.  Now they all look PDH-y, so things are better.

  6589   Mon Apr 30 22:41:34 2012 JenneUpdateSUSETMY oplev power supply dead

Quote:

And has the temp controller of the green been recovered too? (Push enable button)

 

 Just now, yes.

  6588   Mon Apr 30 21:24:43 2012 KojiUpdateSUSETMY oplev power supply dead

And has the temp controller of the green been recovered too? (Push enable button)

 

  6587   Mon Apr 30 21:05:32 2012 JenneUpdateSUSETMY oplev power supply dead

[Jenne, MikeJ]

The ETMY oplev quad sum was ~0, so we went to investigate.  The power supply was dead.  Keying it on and off didn't fix things, and it was certainly getting power since the front indicator light was flickering.  We replaced it with a different power supply, and the oplev is back.

Also, the Y-green AUX laser was off, presumably from the power outage a while back.  I turned it back on.

 

EDIT JD:  The first power supply we used didn't work....the oplev was on for a few minutes, then went off again.  We switched to one of the new (still JDSU) power supplies from Edmund Optics, and it's been happily working for at least an hour now.

  6586   Mon Apr 30 20:43:33 2012 SureshUpdateCDSFrame Builder is down

Quote:

Quote:

Quote:

Frame builder is down.  PRM has tripped its watch dogs.  I have reset the watch dog on PRM and turned on the OPLEV. It has damped down.  Unable to check what happened since FB is not responding.

There was an minor earthquake yesterday morning which people could feel a few blocks away.  It could have caused the the PRM to unlock.

Jamie,Rolf,  is it okay or us to restart the FB?  

 If it's down it's alway ok to restart it.  If it doesn't respond or immediately crashes again after restart then it might require some investigation, but it should always be ok to restart it.

I tried restarting the fb in two different ways.  Neither of them re-established the connection to dtt or epics.

1) I restarted the fb from the control room console with the 'shutdown' command.  No change.

2) I halted the machine with 'shutdown -h now' and restarted it with the hardware reset button on its front-panel. No change.

The console connected to the fb showed that the network file systems did not load.  Could this have resulted in failure to start several services since it could not find the files which are stored on the network file system?

The fb is otherwise healthy since I am able to ssh into it and browse the directory structure.

[Mike, Rana]

The fb is okay.  Rana found that it works on Pianosa, but not on Allegra or Rossa.  It also works on Rosalba, on which Jamie recently installed Ubuntu.

The white fields on the medm  'Status' screen for fb are an unrelated problem.

 

 

  6585   Mon Apr 30 18:46:34 2012 ranaUpdateComputersmegatron

Last week I found that megatron was still off after the power outage. Apparently, the power recovery checklist had not been followed during the recovery.

Please remember to use the checklist and post the checklist results after each power outage. Megatron is now on and functioning.

  6584   Mon Apr 30 16:56:05 2012 SureshUpdateCDSFrame Builder is down

Quote:

Quote:

Frame builder is down.  PRM has tripped its watch dogs.  I have reset the watch dog on PRM and turned on the OPLEV. It has damped down.  Unable to check what happened since FB is not responding.

There was an minor earthquake yesterday morning which people could feel a few blocks away.  It could have caused the the PRM to unlock.

Jamie,Rolf,  is it okay or us to restart the FB?  

 If it's down it's alway ok to restart it.  If it doesn't respond or immediately crashes again after restart then it might require some investigation, but it should always be ok to restart it.

I tried restarting the fb in two different ways.  Neither of them re-established the connection to dtt or epics.

1) I restarted the fb from the control room console with the 'shutdown' command.  No change.

2) I halted the machine with 'shutdown -h now' and restarted it with the hardware reset button on its front-panel. No change.

The console connected to the fb showed that the network file systems did not load.  Could this have resulted in failure to start several services since it could not find the files which are stored on the network file system?

The fb is otherwise healthy since I am able to ssh into it and browse the directory structure.

  6583   Mon Apr 30 13:58:25 2012 JamieUpdateCDSFrame Builder is down

Quote:

Frame builder is down.  PRM has tripped its watch dogs.  I have reset the watch dog on PRM and turned on the OPLEV. It has damped down.  Unable to check what happened since FB is not responding.

There was an minor earthquake yesterday morning which people could feel a few blocks away.  It could have caused the the PRM to unlock.

Jamie,Rolf,  is it okay or us to restart the FB?  

 If it's down it's alway ok to restart it.  If it doesn't respond or immediately crashes again after restart then it might require some investigation, but it should always be ok to restart it.

  6582   Mon Apr 30 13:00:50 2012 SureshUpdateCDSFrame Builder is down

Frame builder is down.  PRM has tripped its watch dogs.  I have reset the watch dog on PRM and turned on the OPLEV. It has damped down.  Unable to check what happened since FB is not responding.

There was an minor earthquake yesterday morning which people could feel a few blocks away.  It could have caused the the PRM to unlock.

Jamie,Rolf,  is it okay or us to restart the FB?  

  6581   Fri Apr 27 13:32:06 2012 DenUpdatePEMseism channels

A few weeks ago I found that GUR2_X signal is biased from 0 to 800 counts in average. I decided that the corresponding channel in the readout box is bad - adds DC voltage to the signal. I stopped using GUR2_XYZ channels of the seism readout box. Now the same thing happened with the GUR1_XYZ channels. I checked the signals coming out from the seism box with the oscilloscope and they were fine. So the problem is not in the readout box. Then I applied 1 V sine wave to the input of AA board to the GUR1_X and ACC_MC1_Z channels. GUR1_X channel still shows noise. Something is wrong with these channels inside the AA board or in the ADC.

pem.png

 

Edited by Den: GUR1_XYZ_IN1 signals are empty though GUR1_XYZ are fine. So the problem is just that GUR1_XYZ_IN1 are not acquired for now though some of the ACC_IN1 channels contain the signal. I need to correct .ini files.

  6580   Fri Apr 27 12:12:14 2012 DenUpdateCDSc1ioo is back

Rolf came to the 40m today and managed to figure out what the problem is. Reading just dmesg was not enough to solve the problem. Useful log was in

>> cat /opt/rtcds/caltech/c1/target/c1x03/c1x03epics/iocC1.log

Starting iocInit
The CA server's beacon address list was empty after initialization?
iocRun: All initialization complete
sh: iniChk.pl: command not found
Failed to load DAQ configuration file

iniChk.pl checks the .ini file of the model.

>> cat /opt/rtcds/rtscore/release/src/drv/param.c


int loadDaqConfigFile(DAQ_INFO_BLOCK *info, char *site, char *ifo, char *sys)
{

  strcpy(perlCommand, "iniChk.pl ");
  .........
  strcat(perlCommand, fname); // fname - name of the .ini file
  ..........
}

So the problem was not in the C1X03.ini. The code could not find the perl script though it was in the /opt/rtcds/caltech/c1/scripts directory. Some environment variables are not set. Rolf added /opt/rtcds/caltech/c1/scripts/ to $PATH variable and c1ioo models (x03, ioo, gcv) started successfully. He is not sure whether this is a right way or not, because other machines also do not have "scripts" directory in their PATH variable.

>> cat /opt/rtcds/caltech/c1/target/c1x03/c1x03epics/iocC1.log

Starting iocInit
The CA server's beacon address list was empty after initialization?
iocRun: All initialization complete

Total count of 'acquire=0' is 2
Total count of 'acquire=1' is 0
Total count of 'acquire=0' and 'acquire=1' is 2

Counted 0 entries of datarate=256     for a total of 0
Counted 0 entries of datarate=512     for a total of 0
Counted 0 entries of datarate=1024     for a total of 0
Counted 0 entries of datarate=2048     for a total of 0
Counted 0 entries of datarate=4096     for a total of 0
Counted 0 entries of datarate=8192     for a total of 0
Counted 0 entries of datarate=16384     for a total of 0
Counted 0 entries of datarate=32768     for a total of 0
Counted 2 entries of datarate=65536     for a total of 131072

Total data rate is 524288 bytes - OK

Total error count is 0

Rolf mentioned about automatic set up of variables - kiis of smth like that - probably that script is not working correctly. Rolf will add this problem to his list.

  6579   Fri Apr 27 09:27:48 2012 DenUpdateCDSsus watchdogs?

Quote:

Why are all the suspension watchdogs tripped?  None of the suspension models are running on c1ioo, so they should be completely unaffected.  Steve, did you find them tripped, or did you shut them off?

In either event they should be safetly turned back on.

 I've turned off the coils. Though non of them are on the c1ioo, who knows what can happen when we'll try to run the models again.

  6578   Fri Apr 27 09:00:15 2012 JamieUpdateCDSsus watchdogs?

Why are all the suspension watchdogs tripped?  None of the suspension models are running on c1ioo, so they should be completely unaffected.  Steve, did you find them tripped, or did you shut them off?

In either event they should be safetly turned back on.

  6577   Fri Apr 27 08:11:19 2012 steveUpdateCDSc1ioo condition

 

 

Attachment 1: c1ioo.png
c1ioo.png
  6576   Thu Apr 26 20:44:23 2012 JamieUpdateCDSdaq network failure, c1ioo failing to start models

Den tried adding a SINGLE acquire channel to the c1ioo, which for some reason hung c1ioo and took down the entire DAQ network (at least all communication between the front ends and the fb).  We recovered by restarting c1ioo and restarting mx_stream on all the rest of the front-ends

After "recovering", though, c1ioo is failing to load models, or at least it's IOP. Here is the tail of dmesg when trying to start the IOP:

[ 1751.140283] c1x03: Initializing space for daqLib buffers
[ 1751.140284] c1x03: Initializing Network
[ 1751.140285] c1x03: Found 1 frameBuilders on network
[ 1751.250658] CPU 2 is now offline
[ 1751.250657] c1x03: Sync source = 4
[ 1751.250657] c1x03: Waiting for EPICS BURT Restore = 1
[ 1751.310008] c1x03: Waiting for EPICS BURT 0
[ 1751.310008] c1x03: BURT Restore Complete
[ 1751.310008] c1x03: Initialized servo control parameters.
[ 1751.311699] c1x03: DAQ Ex Min/Max = 1 3
[ 1751.311699] c1x03: DAQ XEx Min/Max = 3 53
[ 1751.311733] c1x03: DAQ Tp Min/Max = 10001 10007
[ 1751.311733] c1x03: DAQ XTp Min/Max = 10007 10507
[ 1751.311737] c1x03: DIRECT MEMORY MODE of size 64
[ 1751.311737] c1x03: daqLib DCU_ID = 33
[ 1751.311737] c1x03: Invalid num daq chans = 0
[ 1751.311737] c1x03: DAQ init failed -- exiting

The chan file for this model (/opt/rtcds/caltech/c1/chans/daq/C1X03.ini) looks totally fine, has two un-acquired channels uncommented, and has otherwise not been touched. The C1:FEC-33_MSGDAQ is also reading: "ERROR reading DAQ file!"

I'm at a loss for what is going on. I've tried restarting every CDS process on the machine, restarting the model multiple times, restarting fb, and even restarting the entire c1ioo machine, all to no affect.

  6575   Thu Apr 26 18:17:56 2012 SureshUpdateIOOMC WFS: Tweaked the WFS offsets

[Jamie, Suresh]

Yesterday Den and Koji reported that the WFS loops were causing the MC to become unlocked.  They had aligned the PMC.   The input beam into the MC seems to be well aligned.  MCREFL DC close to minimum it gets while MC is locked (~0.45 V).

I checked and saw that the WFS heads and the MC2_TRANS_QPD had picked up DC offsets.  To reset them I turned off the MC_autolocker and closed the PSL shutter.

The ADC offsets were set using this script /cvs/cds/rtcds/caltech/c1/scripts/MC/WFS/WFS_QPD_offsets.  (Jamie fixed the paths to ezcaservo to get this script to work)

The WFS sensor head offsets were manually set to adjust the Q and I signals from the sensor head to zero.  (This operation is supposed to be done by a script which is available, but I will check it out before I direct people to it).

Then we noticed that the ASC outputs were turned off.  (Presumably Koji turned them off yesterday, when the MC was repeatedly unlocking due to the WFS loops).

We turned on the ASC outputs and the MC stayed locked with reasonable outputs on the WFS output channels.  (+/-100)

However, engaging the WFS servo increases the MCREFL DC signal to 0.7 V from the 0.45 V value when the servo is not engaged.  This could be because of DC offsets in the WFS servo filters.   I will adjust these offsets to maintain good MC transmission when the WFS servo is engaged.

 

 

 

  6574   Thu Apr 26 18:15:59 2012 JamieUpdateCDSpossible issue with mx_stream on front ends

I'm noticing what appears to be occasional failures of mx_stream on the front end machines.  It doesn't happen that frequently, but I've noticed it a couple of times already since the upgrade.

The symptom is that the DC Status goes to "0xbad" (red) and the "FE NET" goes red for all models on a given front end.

The solution seems to be restarting mx_stream on the given front end:    sudo  /etc/init.d/mx_stream restart"

There is nothing in the mx_stream log:

 controls@c1sus ~ 0$ cat /opt/rtcds/caltech/c1/target/fb/mx_stream_logs/c1sus.log 
 c1x02
 c1sus
 c1mcs
 c1rfm
 c1pem
 mmapped address is 0x7f43740ec000
 mapped at 0x7f43740ec000
 mmapped address is 0x7f43700ec000
 mapped at 0x7f43700ec000
 mmapped address is 0x7f436c0ec000
 mapped at 0x7f436c0ec000
 mmapped address is 0x7f43680ec000
 mapped at 0x7f43680ec000
 mmapped address is 0x7f43640ec000
 mapped at 0x7f43640ec000
 send len = 263596
 Connection Made

but I do see some funny messages in the front end dmesg:

 [200341.317912] DXH Adapter 0 : Heartbeat alive-check for node=12 failed (cnt=8387 state=0x1 deb=0 val=0).
 [200341.318670] DXH Adapter 0 : Session for node 12 is disabled - Status = 0x5
 [200341.319062] Session callback reason=1 status=5 target_node=12
 [200341.319069] Session callback reason=3 status=0 target_node=12
 [200341.359534] (map_table_check_access:752):my id 1 ->  remote id 2 : entry was valid - is now tentatively valid
 [200341.859584] DXH Adapter 0 : Probe failure for node=12 - disabling session probeStatus=0x40000f02
 [200341.860335] DXH Adapter 0 : Session for node 12 is disabled - Status = 0x3
 [200341.860728] Session callback reason=1 status=3 target_node=12
 [200374.006111] DXH Adapter 0 : Set reachable remote node list.
 [200409.020670] DXH Adapter 0 : Set reachable remote node list.
 [200409.021076] DXH Adapter 0 : Session for node 12 is deleted - Status = 0x0
 [200409.021468] Session callback reason=5 status=0 target_node=12
 [200412.362824] (map_table_insert:648):** successfully inserted **(valid unicast) inst 0 node 1->0 fwd 0 fwd_tp 4 egress 0
 [200418.025994] (map_table_check_access:752):my id 1 ->  remote id 0 : entry was valid - is now invalid
 [200418.025998] (map_table_insert:648):** successfully inserted **(valid unicast) inst 0 node 1->2 fwd 0 fwd_tp 4 egress 0
 [200421.743916] Session callback reason=0 status=0 target_node=12
 [200422.073776] DXH Adapter 0 : Set reachable remote node list.
 [200422.342446] Session callback reason=7 status=0 target_node=12
 [200422.342454] DXH Adapter 0 : Session for node 12 is ok.

I'm awaiting feedback from experts.

 

  6573   Thu Apr 26 16:35:34 2012 JamieSummaryCDSrosalba now running Ubuntu 10.04

This morning I installed Ubuntu 10.04 on rosalba.  This is the same version that is running on pianosa.  The two machines should be identically configured, although rosalba may be missing some apt-getable packages.

  6572   Thu Apr 26 11:56:10 2012 DenUpdatePEMBlue Bird Pre-Amplifier

Quote:

Quote:

Usually R is for resistors and D is for diodes. Do you think from the schematic that we should put diodes into the R slots?

 That guys are resistors.

 You are right, they just looked like they were too small to be resistors when I glanced at them. 

  6571   Thu Apr 26 09:22:17 2012 steveUpdate under the east end optical table

Quote:

I added an U channel based bottom shelf at the south end today.

 I'm working on similar shelf at ETMY.  Precondition: NPRO in bypass mode, heater for doubling in bypass........since power outage?  Optical level servo turned off.......

U-channel based shelf in place.  Oplev servo is back on at 11:15am    The table may moved.  The oplev return is missing the quad by a few milimeter.

  6570   Wed Apr 25 21:24:10 2012 DenUpdateComputer Scripts / Programsc1oaf

C1OAF model, codes and medm screens are updated. All proper files are commited to svn and updated at the new model path.

  6569   Wed Apr 25 19:36:19 2012 DenUpdatePSLPMC aligned

[Koji, Den]

We have aligned PMC,  the WFS are not working yet.

  6568   Wed Apr 25 19:32:57 2012 DenUpdatePEMBlue Bird Pre-Amplifier

Quote:

Usually R is for resistors and D is for diodes. Do you think from the schematic that we should put diodes into the R slots?

 That guys are resistors.

ELOG V3.1.3-