Kiwamu and Steve,

This is one of the better oplev set up we have. Single bounce from TM, no mirror on stack. One lens and one  mirror on ISCT. Old Uniphase 1103P laser on heafty 3" od mounts.

Somewhat  big ~5-6 mm spot on qpd in  1,300 counts.

The intensity noise effect can be seen at 1 Hz and 3-20 Hz

Oplev servo was OFF

The following optics were kicked:
MC1 MC2 MC3 ETMX ETMY ITMX ITMY PRM SRM BS
Sun Oct  2 02:13:40 PDT 2011
1001582036

They will automatically get back after 5 hours.

The input matrices of ETMX, ITMX and PRM have been newly inverted.

Those were the ones having some troubles (see #5444, #5547).

After a coarse adjustment of the damping Q factors, they look damping happily.

optic	spectra	inverted matrix	BADNESS
ETMX		pit     yaw     pos     side    butt UL    0.848   0.108   1.578   0.431   1.025  UR    0.182  -1.892   1.841  -0.128  -1.172  LR   -1.818  -1.948   0.422  -0.122   0.939  LL   -1.152   0.052   0.159   0.438  -0.864  SD    1.756  -3.794  -0.787   1.000  -0.132	5.37028
ITMX		pit     yaw     pos     side    butt UL    0.510   0.584   1.228   0.458   0.203  UR    0.783  -1.350   0.348  -0.050   0.555  LR   -1.217   0.065   0.772   0.264   0.312  LL   -1.490   2.000   1.652   0.772  -2.930  SD   -0.635   0.853  -1.799   1.000  -1.597	7.5125
PRM		pit     yaw     pos     side    butt UL    0.695   1.422   1.774  -0.333   0.954  UR    1.291  -0.578   0.674  -0.068  -0.939  LR   -0.709  -1.022   0.226   0.014   0.867  LL   -1.305   0.978   1.326  -0.251  -1.240  SD    0.392  -0.437  -0.500   1.000   0.420	5.09674

(some notes)

 Before running the peakFit scripts, I woke up the nds2 sever process on Mafalda because it hasn't been recovered from the power outage.

To start the nds2 process I followed the instruction by Jamie (#5094).

Then I started requesting the data of the last night's free swinging test (#5594)

However the NDS2_GetData command failed everytime when data with long duration were requested.

It maybe because some of the data are missing in sometimes, but I haven't seriously checked the data stored in fb.

So for the reason, I had to use a short duration of 1200 sec (default is 3600 sec). That's why spectra look noisier than usual.

 [Suresh, Jenne]

Suresh pointed out the oddity that all of the EX, EY slow channels were showing white boxes on the medm screens on all of the workstations except Rosalba.  I don't know why Rosalba seemed to be working, but whatever.  I'm not 100% sure that Rosalba was even working properly....I shutdown ETMX and ETMY's watchdogs before we went to boot computers, but when I came back to the control room the 2 optics were rung up anyway.  Turning back on the watchdogs, the optics immediately began to damp happily.

Since Kiwamu had trouble with the slow channels for EX, we decided to key some crates. 

We keyed the c1auxey, and c1auxex crates, waited a few seconds, and then things looked okay in medm-land.  I looked at the "View Backup" for ETMX, and there were no values for the DC sliders, so since the arms are both flashing right now, I did a "save", and then confirmed that I can misalign and restore the optic.  However, since I didn't fully align/lock the cavity, the saved value for right now shouldn't be fully trusted.  We might have to manually align the cavity using the BS.

I assume it's because the burt restore didn't work for the SUS summary screen, but all of the values for the ifo suspensions (not the MCs...they're okay) are red. 

I am trying to run Rana's setSensors.py script, but am failing.  Any inspiration would be appreciated:

rosalba:SUS_SUMMARY>./setSensors.py 1001708529 500 .1 .25
['./setSensors.py', '1001708529', '500', '.1', '.25']
/cvs/cds/caltech/apps/linux64/python/lib64/python2.4/site-packages/nds/__init__.py:28: RuntimeWarning: No protocol specified, attempting protocol nds_v2
  super(daq, self).__init__(host, port)
Connecting NDS2 .... authenticate done
Traceback (most recent call last):
  File "./setSensors.py", line 81, in ?
    mean = acquire(x)
  File "./setSensors.py", line 73, in acquire
    daq.request_channel(chans[x])
Boost.Python.ArgumentError: Python argument types in
    daq.request_channel(daq, str)
did not match C++ signature:
    request_channel(_daq_t {lvalue}, daq_channel_t*)

I'm not exactly sure what the problem is.  Line 73, looks like it should have 2 arguments in the daq.request_channel, but even if I put in the "daq" variable (which is set a few lines above), I get the exact same error.  So...something else is wrong.  Ideas from someone who "speaks" python??

 My guess is that this has something to do with the NDS client version you're using.  Try running the script on a machine where pynds and nds-client are known to be compatible, like pianosa.

The He-Ne laser which has been used for the PRM and BS oplevs were found to be dead.

According to the trend data shown below, it became dead during the dolphin issue.

(During the dolphin issue the output from the oplev QPDs are digitally zero)

JDSU 1103P died after 4 years of service. It was replaced with new identical head of 2.9 mW output. The power supply was also changed.

The return spots of 0.04 mW  2.5 mm diameter on qpds are BS  3,700 counts and PRM 4,250 counts.

Kiwamu and Steve,

The He/Ne oplev shows no coherece so relative intensity noise is not limiting factor for the oplev servo

The SRM bounce peak 18.33 Hz. Suresh helped me to retune filter through Foton, but we failed to remove it.

ITMX_OPLEV_PERROR shows high coherence with oplev laser. This is our lousiest set up. I will work on it next week.

Generally speaking we can say that JDSU-Uniphase 1103P and 1125P laser intensity noise do not limit oplev servo performance.

However, the overall well being of filters, gain settings, beam sizes on qpds are poor.

Yesterday Koji was claiming that the rms monitor of ULSEN on ETMY didn't well go down.

Indeed something bad is happening on ULSEN of ETMY.

I guess it could be a loose connection.

(The unit of Y axis in the plot is not true. Don't believe it !)

The SRM oplevs were found to be oscillating because of a small phase margin.

I reduced the gains to the half of them. The peak which Steve found today maybe due to this oscillation.

 I lost UL osem voltage this morning when I was checking the actual connection at rack ETMY

This after noon I disconnected  the 64 pins IDE connector from satelite amp at the rack, and the two 25 pins Dsubs at this juction board.

UL OSEM recovered after reconnecting these three connectors.

Atm3, bad connection.........noisy UL

 Atm2 is before optical path adjustment. The idea was to remove possible clipping in vacuum.

Coherense significantly reduced below 4 Hz

Today I will replace the He/Ne laser 1125P with 1103P

 I replaced the JDSU-Uniphase 1125P by 1103P He/Ne laser. This new laser had 2.8 mW output yesterday. It degraded to 0.5 mW by this morning.

The beam size on the QPD is ~3 mm  This should give us  better sensitivity. These are not the perfect lenses at all, but we have them here.

On the other hand, there are still some coherence below 1 Hz, so the laser intensity noise or clipping dominating  this  part of the spectrum.

I offered to help Kiwamu with some of the 40m work. The first task was to figure out why the ETMX side OSEM monitor was so low, since we know that the depth is about right. It was showing ~0.13 V to the others' ~0.7 V.

TL,WR: There was a wire disconnected from the breakout panel on the side of the rack

I started by pulling the board out and checking to make sure that it was working properly. I injected a sine wave to the SIDE IN and found that it showed up in the signal coming out of the back (into the crate) just fine (see below). One strange thing I noticed while testing the board is that both inputs for each used channel of the MAX333 switches on the board are shorted to their respective outputs. That is, the switches seem to be open to BOTH 0 and 1 logic states. This seems counterintuitive, but perhaps there's something about how these work that I don't know.

Then I went about tracing the signal from the back of the crate to the breakout panel on the side of the rack. I opened it up, verified that the ribbon cable was transmitting correctly, and as I went to plug it back in I noticed that one of the wires---the correct one---had come completely undone.

The screw clamp appeared to be a bit finicky, as I had to loosen and tighten it a few times before it finally seemed to grab hold of the wire. It probably just wasn't tight in the first place and the wire was pulled out. Anyway, things are working now:

 Good job ! Thank you so much

 ETMX oplev had 6 mm diameter beam on the qpd.  I relayed the beam path with 2 lenses  to get back  3 mm beam on the qpd

BRC 037  -100 Bi _concave lens and PCX 25  200 VIS do the job. Unfortunately the concave lens has the AR 1064.

The following optics were kicked:
SRM
Wed Oct 19 04:22:53 PDT 2011
1003058589
 

I made some efforts to fix the situation of SRM but it is still bad.

The POS motion wasn't well damped. Something is wrong either (maybe both) sensing part or actuation part.

I am going to check the sensing matrix with the new free swinging spectra (#5690)

(Symptom)

 When I was trying to lock SRMI I found that the fringes observed at the AS camera didn't show spatial higher order modes, which is good.

So I thought the SRM suspension became quiet, but it actually wasn't. Because the RMS monitor of the SRM OSEMs already went to about 30 counts.

At the same time the opelev error signals were well suppressed. It means some DOFs which were insensitive to the oplev were ringing up, namely POS and SIDE.

According to the LSC error signal and the ASDC signal, I believe that the POS was going wild (although I didn't check the OSEM spectra).
 

(Some efforts)

 + Readjusted the f2a filters (see the attachment).

 + Tried to eliminate a coupling between the POS and SIDE drives by tweaking the output matrix.

    => In order to eliminate the coupling from the POS drive to SIDE sensor, I had to put a comparable factor into an element.

     So it might be possible that the POS sensor was showing the SIDE signal and vice versa.

      In order to check it I left SRM free swinging (#5690).

Sorry Kiwamu, I realized too late that you were freeswigging. Hopefully 4 hrs was enough.

 Coherence at 1 and 2-3 Hz only. He/Ne laser intensity noise is not an issue.

 Kiwamu removed the 18.3 Hz ocsillation by turning down the oplev servo gain.

The SRM input matrix has been readjusted.

However still there is the unwanted coupling from the POS drive to SIDE signal and from the SIDE drive to POS signal.

			BADNESS
SRM		pit     yaw     pos     side    butt UL    0.871   0.975   1.115  -0.295   1.096   UR    1.015  -1.025   1.128  -0.140  -1.053   LR   -0.985  -0.984   0.885  -0.088   0.831   LL   -1.129   1.016   0.872  -0.243  -1.020   SD    0.084   0.061   3.534   1.000  -0.018	4.24965

There looks some activity at around MC2 on Wednesday afternoon.
It caused the misalignment of MC2. Misalignment was not found in MC1/3.

It seems that the incident beam on the MC was aligned in the evening.
This increased the MC transmission but it is vibible that the spot on MC2 is shifted from the center.

We need an action on this issue tomorrow in the daytime.

I am working on fixing this.  You might some strange stuff going on in the control room screens.  Pls ignore it till I am done.

  I have realigned the MC by recentering the spots on all the MC optics.  The current spot positions (in mm) are:

MC1P     MC2P     MC3P      MC1Y      MC2Y     MC3Y

0.2245    0.3364   -0.2801   -1.8891    0.1631   -1.744

Initially the lockins 2 and 5 showed very small outputs.  This was traced to the fact that we have recently switched on a 28Hz ELP filter module in the MC2 ASC filter bank which introduces an extra phase of about 75deg..  See this elog.

When the MC ASS lockins were initially setup, the phase was set with this filter module switched off.  Since quite some time has passed since the last calibration of these phases, I readjusted the phases to minimise the  Q_OUTPUT and I also adjusted the GAINs in the SIG filter banks  of all the six lockins so that their I_OUT's drop by the calibration value of -2.65 when an offset of 0.1 is introduced into the MC suspension output matrices.  Two short scripts in the $scripts$/ASS/ directory help in setting and removing these offsets.  They are called MCxoffsetOn and MCxoffsetOff.   They have to be edited appropriately to address each DoF of the MC.

The $scripts$/ASS/mcassUp script., which sets up everything to make the MC spot decentering measurement, has been edited to set these new phases and gains.  The old settings have been commented out.

I then centered the spots on the WFS sensors and the MC_TRANS QPD.  We are now ready to make the MC WFS output matrix transfer coef measurement again, but this time with the WFS loops closed.

While chatting with Jenne I learnt that some substantial amount of work had taken place yesterday around the MC2 chamber.  This was associated with the relocating of seismometers.  ref elog. 

I reiterate what is well known for quite sometime:  MC2 table is not well isolated from the ground.  And we should not approach this chamber unless absolutely necessary. I have blocked off the area around it which we should avoid.  It is a serious waste of time and effort to realign the MC each time the MC2 table decides to settle into a new position.

Steve tells me that the mild-steel frame supporting the chamber+MC2_table sits with two legs on one concrete slab while the other two legs sit on another one.   The frame is also quite weak without sufficient gussets or cross connects.  The next time we have a major shutdown we must replace this frame with a more sturdy one which sits on one slab (preferably the one on which the rest of the MC sits).

Till we improve this mounting, I suggest that we avoid that area as much as possible.

 Thinks to do before the NEXT realignment:

B,  tie 4 ancher bolts on table legs to the floor

C,  tie 4 dog clamps between table and chamber

D,  check the locked position of the 4 x 4 positioning screws

E,  check bellow protecting 4 tubes are not shorting

A,  here is the concrete  slab cut

It reminds me to check the  IFO vacuum envelope dog clamps on the chambers to floor  with torque wrench.

I found that the sum of the ITMX oplev signals had gone down to zero yesterday.

I checked the ITMX table and found two iris on the He-Ne laser path were blocking the beam on their apertures.

I guess this is because we were working around there for installation of POP/POX and may have touched some of the oplev optics.

Then I fully opened the apertures of those two iris and the sum went back to nominal of 600 counts.

I finally got around to wrapping up the SUS input matrix diagonalizer. The files I have added to ...scripts/SUS/peakFit are:

• kickAll: Restores all SUS angle biases using /cvs/cds/rtcds/caltech/c1/medm/c1ifo/cmd/C1IFO_OPTICrestore.cmd XXXX &, then runs 'freeswing all'. Finally, writes an elog entry with the time when the optics were kicked and saves the gps time to 'kickAll.time'.
• diagAllSUS.m: An M-file that calls all the other individual M-files needed for the matrix generation. What it does:
  • Reads kick time from kickAll.time
  • Runs getSensors.m to get time series from all optics' sensors
  • Runs makeSUSSpectra.m to generate spectra from time series
  • Runs findPeaks.m to fit spectra for peak frequencies
  • Loops through all optics and runs, in sequence:
    • inmat = findMatrix(XXXX) to generate the matrix for each optic
    • writeSUSinmat(XXXX,inmat) to write that matrix to the frontend
• diagAllSUS: A wrapper for diagAllSUS.m. It also writes an elog entry and attaches the pre and post spectra demonstrating the diagonalization. The entry following this one is an example.

The following lines should eventually be added to the controls@nodus crontab:

0 8 * * 0 /cvs/cds/rtcds/caltech/c1/scripts/SUS/peakFit/kickAll

0 18 * * 0 /cvs/cds/rtcds/caltech/c1/scripts/SUS/peakFit/diagAllSUS

(i.e., do the kick at 8am on Sundays and run the diagonalization 10 hrs later at 6pm). They can be done on a different machine but then my elog commands will need to be modified.

Before we add them, we should check that they do, in fact, work. We can do this sometime while I'm at the 40m this week.
 

I actually tried to do this last night, but I was getting a terminal error from nodus. Jamie told me to just manually redefine the TERM variable and it would work. So, I have added kickAll and diagAllSUS to controls@nodus's crontab:

nodus:~>crontab -l

0 5 * * * /opt/rtcds/caltech/c1/scripts/backup/rsync.backup

0 7 * * * /opt/rtcds/caltech/c1/scripts/backup/check_backup.sh 

0 17 * * 0 /cvs/cds/caltech/users/jenne/LIGOX/LIGOX_Pizza_Reminders.sh

0 8 * * 0 /cvs/cds/rtcds/caltech/c1/scripts/SUS/peakFit/kickAll

0 18 * * 0 /cvs/cds/rtcds/caltech/c1/scripts/SUS/peakFit/diagAllSUS

Shade 12 black glass-green lines: 1/8" thick,  6" wide  x 7 " high  mounted on angle bracket below.  Aperture 1" diameter. Let me know what do you think.

It turns out that nodus doesn't know how to NDS2, so I can't run diagAllSUS as a cron job on nodus. To further complicate things, no other machines can run the elog utility, so I am going to have to do something shifty...

 Actually in the last 40m meeting we discussed the SUS diagnosis and decided not to post the results on the elog.

Alternatively we will have a summary web page which will contain all the information (sensitivity, UGF monitors, etc.,) and will be updated everyday like GEO.

This will be a place where we should post the SUS results every week.

So we don't need to worry about the cron-elog job, and for running the scripts you can simply use one of the lab machines as a cron host.

Once you get the scripts running on a machine as a cron job, it will be the point where we should quit developing the SUS diagonalizer and pass it to the web summary people.

EDIT: Ran Jamie's NDS2 reset method and it's still not serving up MC data.

I am debugging the SUS input matrix diagonalization code (which I have affectionately termed "Dr. SUS") offline by running everything except the actual writing of the matrices (%writeSUSinmat()) in a sandbox directory. Some notes:

• nodus doesn't know NDS2, but this isn't a problem given Kiwamu's reply, since we don't want to push results to the elog. I will still have the script push "optics kicked---don't touch" alerts to the elog, but I realized this can be done easily from a remote computer via in-script SSH commands. My next choice was pianosa, but even though she knows NDS2, she spits out MEX errors when I try to get data. Third stop was allegra, and she plays nice 
• NDS2 is unable to read the MC mirror sensor data. Data from all other optics are retrieved without a problem. I verified that raw data was being written by plotting playback data in dataviewer, and I tried various times and the problem persisted. I suspect the problem is on the NDS2 server side:

The most recent entry in .../users/jzweizig/nds2-mafalda/channel_history shows nothing for MC1 2 or 3

Now that NDS2 is working, I was able to run all parts of diagAllSUS (except for the matrix writing) without errors. Barring issues with standard commands (freeswing, turnOnWatchDogs.csh, caput, etc.), Dr. SUS should be ready to go.

As requested, I have suppressed the generation of an elog post containing the results of the diagonalization.

Dr. SUS will run on allegra as a cron job at 8:00am on Sundays:

allegra:peakFit>crontab -l

PATH=/bin/:/usr/bin/:/cvs/cds/caltech/apps/linux64/matlab/bin/:/cvs/cds/caltech/apps/linux/gds/bin/:/cvs/cds/caltech/apps/linux/tds/bin/

PWD=/cvs/cds/caltech/apps/linux/gds/lib/

LD_LIBRARY_PATH=/cvs/cds/caltech/apps/linux/gds/lib

01 05 * * 0 cat /cvs/cds/caltech/users/kiwamu/work/20091026_OMC-LSC-diag/src/OMC_LSC_timing.m | matlab -nosplash -nodesktop > /dev/null

0 8 * * 0 /cvs/cds/rtcds/caltech/c1/scripts/SUS/peakFit/runDrSUS

Also: should these variable definitions really be here?

 I locked the PMC and MC resonated in TM00 right on. Than I started checking anchoring screws with the torque wrench.

B,  3/8 foor bolts were tight, They were set  to 50 ft/lbs

C,  dog clamp bolts were tight at 80 ft/lbs

D,  horizontal position locks were lose. They were locked by finger.

E,  below covers were reset not to short

The MC is locking in TM01 mode now 

Cron starts with only a very minimal PATH.  However, this shouldn't be an issue if you specify the full path to the commands.

The rest of the env vars should probably not be there.

Also, Zach:  using /cvs/cds is now forbidden.  We need to put this script in the right place.

• I have replaced all instances of /cvs/cds/rtcds with /opt/rtcds in the Dr. SUS code. 

• We discussed placing a human in charge of whether or not the input matrices get written to the frontend. I am not sure if we reached a definitive conclusion. Currently, the code is set to write the matrices automatically. What to do?
  
  • If we decide that oversight is necessary, I suggest that we leave the publishing of the results to the elog intact. This way, it will be someone's job to read the weekly Dr. SUS diagnosis on Sunday night (or Monday morning), and run a simple script that writes the matrices. This seems like the most reasonable solution.

I await responses...

My inclination is to not do the writing of the matrices automatically nor to do the weekly kicks. Its nice to have long locks of the MC, etc.

I suggest just making the kick on Sundays when someone intentionally asks for it (e.g. by pressing a button on Friday). The free-swinging ringdown ought to be a opt-in kind of feature, not opt-out.

Ok, here's the deal:

• For the time being, I have written a "doirun" bit into runDrSUS (i.e. it runs if doirun is 1 and doesn't if it's 0). This is a bad way of doing this, so in the end I think we should put a switch on the IFO MEDM and have the script read the value when the cron job is run. If you want it to be an opt-in rather than a toggle, we can have the script write it back to 0 every time. I don't know how to do this yet because I am an MEDM n00b, but I will do it soon.
• Since we have decided to keep a human in the loop on the writing to the frontend, I have kept the elog results push.
• I have also edited diagAllSUS.m so that it archives all computed matrices (hierarchy: .../scripts/SUS/peakFit/inMats/(gps_time_of_kick)/inMat(optic_name).mat). There is a 'writematrices' bit in the M-file, currently set to 0.
• I have written the script 'writeAllInMats' and the accompanying M-file 'writeAllInMats.m'. This allows the user to write whichever set of input matrices he or she desires (syntax: writeAllInMats (gps_time_of_kick)). If no argument is given, then it reads the most recent kick time from 'kickAll.time' and writes the corresponding matrices.

So, here is an example of how this works:

1. Someone decides to do a diagonalization on a particular weekend, (eventually) clicking a switch in MEDM
2. Cron runs runDrSUS at 8am that Sunday. This:
   1. Kicks all the optics, lets them swing for 5 hours, then reengages the watchdogs. The kick time is saved in kickAll.time, and an alert is posted to the elog
   2. Runs diagAllSUS, which computes and saves all matrix data. A report of the results is posted to the elog.
3. On Monday morning---or whenever---someone looks at the entry and decides whether or not to write the files
   1. If the results are good, he or she runs writeAllInMats and the latest matrices are written
   2. If the results are bad, he or she does nothing. The matrices are still archived and can be written at any time in the future.

The code is set to run tomorrow morning. Everything but the writing will be done.

 Dr. SUS failed while trying to get the sensor data. Specifically, it couldn't get ETMY data. This is odd, because in my tribulations last week I ended up having to add the ETMY_SENSOR channels manually into the NDS2 channel files. After doing this, I was able to get ETMY data just fine (though I admitted that we would have problems again as soon as we wanted to update the channel files). I even ran the diagAllSUS code in a sandbox and it pulled data---and generated input matrices---just fine.

The error persists even if I try to get the data manually:

>> d = NDS2_GetData({'C1:SUS-ETMY_SENSOR_UL'},t0,10,'mafalda.martian:31200');

Connecting.... authenticate done

Warning: daq_recv_next failed

??? Error using ==> NDS2_GetData

Fatal Error getting channel data.

I think Jamie is still waiting for J.Z.'s help with this, but it is probably pointless to keep trying to run this code before NDS2 is working again. Another option is to just use NDS, but I think certain people are opposed to this. 

The ETMX oplev returning beam is well centered on the qpd. We lost this signal 2 days ago. I will check on the qpd.

ETMX sus damping restored.       c1iscex computer is down

 As reported a couple days ago, the ETMX IO chassis has no timing signal.  This is why we there is no QPD signal and why we turned off the ETMX watchdog.  In fact, I believe that there is probably nothing coming out of the ETMX suspension controller at all.

I'm working on getting the timing board replaced.  Hopefully today.

Mirko and  Den are measuring MC_F, which they will report about later, but I noticed that MC2 is totally crazy right now.  It shouldn't matter that they are doing things (like unplugging the feedback to the PSL's PZT), because we actuate on the laser, not on the MC.  I disabled the MC autolocker before they started working. 

Anyhow, somehow MC2 got kicked up (whatever, that happens), but it won't re-damp.  I think it's the WFS.  The yaw output from the WFS is truely crazy. 

I have disabled the WFS output / ASC input on the MC SUS screens, and MC2 was then able to damp.  My disabling only the MC2 WFS input at first kicked up MC1 and 3, so I disabled all of the WFS stuff, and all 3 MC mirrors are again happy. 

SURESH: FIX ME!  (signed, The WFS)