I'm not currently sure how to apply my template to seismic.strip shown on the wall (I saved it as seismic.strip on Pianossa and copied the old file to seismic.stripOld). I understand the job is being run on Megatron. I'll play around with this later tomorrow. (In other words, the display currently on the wall, while it does not have the Nan spikes like yesterday and this morning does not currently display the template I made).

Hi everybody,

Sorry for flooding the ELOG about the PEM channels. Today I

- Changed all of the GUR1 and GUR2 filters to elliptic, and lowered the orders of their low-pass filters.

- Lowered the order of the low-pass filters on the STS channels

- Changed the parameters in seismic.strip, which I saved as MashaTemplate2.

Attached is the most recent status of the channels as seen with StripTools:

 FIGURED IT OUT - THERE WAS A PROBLEM WITH THE LOW PASS FILTERS (TOO HIGH ORDER). FIXING IT NOW, SHOULD BE GOOD IN AN HOUR. 

UPDATE: I changed all of the GUR1Z channels to order-5 elliptic filters. I approximated the attenuation for each one by setting the integral from _CutoffFrequency to 10^3 Hz of 10^(-Percent(f)/20) df to 0.01, where Percent(f) is a linear approximation of the relationship between the log of the frequency and the dB level (with the attenuation defining one of the points). Right now the Nan problem continues to persist, even after I loaded the coefficients. In Dataviewer, the channels look relatively normal for the past 10 minutes, as does the data when viewed with tdsdata.

Hi everybody,

Last night I (with the help of Jenne and Jenne's advice - not to implicate her in this or anything) changed the filters for GUR1, GUR2, and STS in C1:PEM-RMS, adding a butterworth bandpass filter at each corresponding frequency band as well as a gain to convert from counts to micros/sec, and then adding a low pass filter in case of aliasing upon squaring.

Currently the seismic signals are going crazy, and producing "Nan" output on the strip graph (which leads to the instantaneously sharp spikes - which leads to the entire signal being filled on the visualizer on the wall). I checked the DataViewer output and the tdsdata output using both grep and wc, and it seems that both every single signal point is present and is a real number (also not a small real number, thereby debunking floating-point error). I'm currently not sure why seismic-strip reads out 'Nan' - perhaps because it's taking the log of 0, taking a negative log, taking the root of a negative number, or dividing by zero.

Does anyone know if the seismic-strip Nan issue is a program bug? If it's not (and therefore a filter bug), please let me know as well.

I'll be in lab for the rest of the night changing the butterworth filters to odd-order elliptic filters (at Rana's suggestion), as well as changing the cut-off frequency for the low-pass filters.

I'll E-log about it when I'm done.

Just to be sure that my numbers are correct - The STS, GUR1, and GUR2 channels all have gain 10, right? (I parsed through the e-log, and these seem to be the most recent numbers

Thanks for your help,

Masha

From calculation, phase fluctuation of reflected beam from length stabilized arm is not disturbing MI lock.

Easy calculation:
  The phase PD at AS port sense is

phi = phi_x - phi_y = 2*l_MICH*omega/c + (phi_X - phi_Y)

  where l_MICH is the Michelson differential length change, omega is laser frequency, phi_X and phi_Y are phase of arm reflected beam. From very complicated calculation,

phi_X ~ F/2 * Phi_X

  at near resonance. Where F is arm finesse, Phi_X is the round trip phase change in X arm. So,

phi = 2*l_MICH*omega/c + F/2 * 2*L_DARM*omega/c

  Our ALS stabilizes arm length in ~ 70 pm(see elogs #6835,  #6858). Finesse for IR is ~450. Considering l_MICH is ~ 1 um, MICH signal at AS port should be larger than stabilized DARM signal by an order of magnitude.

Length sensing matrix of FPMI:
  Calculated length sensing matrix of 40m FPMI is below. Here, I'm just considering 11 MHz modulation. I assumed input power to be 1 W, modulation index 0.1i, Schnupp asymmetry 26.6 mm. PRM/SRM transmissivity is not taken into account.

[W/m]     DARM      CARM      MICH
REFL_I    0         1.69e8    0
REFL_Q    7.09e1    0        -3.61e3
AS_I      0         0         0
AS_Q      1.04e6    0         3.61e3

  Maybe we should use REFL_Q as MICH signal, but since IQ separation is not perfect, we see too much CARM. I tried to lock MI with REFL11_Q yesterday, but failed.

 I can not understand what really happened here with CC1 gauge or there was really a pressure glitch.

attachment 1,  pump down day 3 with 4th of July fireworks in the lab

attachment 2,  before and after vent in 9 days

 I'm looking for some movement indicators of the vent-pump down events.

There is a package that provides the mex source, but it doesn't actually provide the mex binaries.  The problem is that the binary depends on the matlab version, so you can't possibly provide binaries for every version.

The solution is to just build the binaries from the source package.  We should put together a nice script that builds the binaries from the source, and installs them in the directory of your choosing.  If we get something nice working, we can probably get them to include it with the package, to make it easier in the future.

Here's what's included in the source package:

controls@pianosa:~ 0$ sudo apt-get install nds2-client-matlab
...
controls@pianosa:~ 0$ dpkg -L nds2-client-matlab | sort
/.
/usr
/usr/share
/usr/share/doc
/usr/share/doc/nds2-client-matlab
/usr/share/doc/nds2-client-matlab/changelog.Debian.gz
/usr/share/doc/nds2-client-matlab/changelog.gz
/usr/share/doc/nds2-client-matlab/copyright
/usr/share/matlab
/usr/share/matlab/NDS2_GetChannels.m
/usr/share/matlab/NDS2_GetData.m
/usr/share/matlab/NDS_GetChannels.m
/usr/share/matlab/NDS_GetData.m
/usr/share/matlab/NDS_GetMinuteTrend.m
/usr/share/matlab/NDS_GetSecondTrend.m
/usr/share/matlab/src
/usr/share/matlab/src/NDS2_GetChannels.c
/usr/share/matlab/src/NDS2_GetData.c
/usr/share/matlab/src/NDS_GetChannels.c
/usr/share/matlab/src/NDS_GetData.c
/usr/share/matlab/src/nds_mex_utils.c
/usr/share/matlab/src/nds_mex_utils.h
controls@pianosa:~ 0$ 

Why do we have these timing blanks?

Handing off the servo from ALS to LSC for one arm is quite easy because servo filters are pretty much same for ALS and LSC. I demonstrated it Y arm during MI is locked.
We need DARM/CARM-kind of handing off in the near future.

What I did:
  1. Brought both arms to IR resonance.
  2. Brought X arm to off resonance.
  3. Locked MI in bright fringe(why can't I lock in dark fringe, when one arm is on resonance?) using AS55_Q and BS.
  4. Ran /opt/rtcds/caltech/c1/scripts/ALS/handofftoLSC.py Yarm to handoff. It decreases ALS gain and increases LSC gain in 30 sec ramp time. It also turns on some filters for LSC. Make sure you turn off filter triggers for LSC.

 Below is the plot of what I did. You can see LSC feedback signal gradually increasing and TRY getting more stable.
 I was dissapointed with ALS not having any DQ channels for feedback signal. I will make them DQ channels tomorrow.

MI with X arm length stabilized off resonance and Y arm length stabilized at resonance using ALS succeed, but I couldn't bring X arm to IR resonance. This maybe because of too much phase fluctuation. I will calculate it later.

What I did:
  1. Brought X arm to IR resonance.
  2. Brought Y arm to IR resonance.
  3. Brought X arm to off-resonance.
  4. Brought Y arm to off-resonance. (1-4 are to play with arms)
  5. Locked MI in dark fringe using AS55_Q as error signal and BS as actuator.
  6. Brought Y arm to IR resonance. This flips sign, so MI lock will be bright fringe.
  7. Brought X arm to IR resonance. This destroys MI lock.

  Below is the plot showing what I did


  I also tried to lock MI after both arms are stabilized at resonance, but it failed, too.
  MI + X arm ALS fails. I think this is from too much BS motion to compensate phase fluctuation of arm reflected beam.
  MI + Y arm ALS fails when I want to lock in dark fringe. Only bright fringe works.


New compact MEDM screen for ALS:
  It has (almost) everything you need for ALS. It lives in /opt/rtcds/caltech/c1/medm/c1gcv/master/C1ALS_COMPACT.adl.
  Features;

  - Button for turning on/off ALS. It even does "clear history"!
      (light brown button "ON plus", "ON minus", "OFF"; runs /opt/rtcds/caltech/c1/scripts/ALS/easyALS.py; Currently, you have to guess the sign of gain. Ctrl-C if the sign was wrong. It will be nice if script can handle this. Use lockin to detemrine the sign?)

  - Button for finding IR resonance.
      (pink button "IRres"; runs /opt/rtcds/caltech/c1/scripts/ALS/findIRresonance.py)

  - Button for bringing arm length to off-resonance.
      (pink button "-10", "+10"; steps +/- 10 deg offset)

  - Button for toggling green shutters.
      (green button "shutter"; runs /opt/rtcds/caltech/c1/medm/c1gcv/cmd/toggle(X|Y)Shutter.py)

  - Button for switching monitors.
      (grey button "Video (X|Y)arm"; runs /opt/rtcds/caltech/c1/scripts/general/Video_(X|Y)arm.csh)

  - Slider for changing temperature of end lasers. You can also open temperature servo screens from orange "(X|Y)SLOW" button.

 The reason I started looking at BLRMS and c1sus today was that the BLRMS striptool was totally wacky.  I finally figured out that the pemepics hadn't been burt restored, so none of the channels were being filtered.  It's all better now, and will be even better soon when Masha finishes updating the filters (she'll make her own elog later)

I was trying to use a new BLRMs c-code block that the seismic people developed, instead of Mirko's more clunky version, but putting this in crashed c1sus.

I reverted to a known good c1pem.mdl, and Jamie and I did a reboot, but c1sus is still funny - none of the models are actually running. 

rtcds restart all - all the models are happy again, c1sus is fine.

But, we still need to figure out what was wrong with the c-code block.

Also, the BLRMS channels are listed in a Daq Channels block inside of the (new) library part, so they're all saved with the new CDS system which became effective as of the upgrade.  (I made the Mirko copy-paste BLRMS into a library part, including a DAQ channels block before trying the c-code.  This is the known-working version to which I reverted, and we are currently running.)

Cavity g-factor for X arm is 0.3737 +/- 0.002, Y arm is 0.3765 +/- 0.003.
If ITMs are flat and arm length L = 39 +/- 1 m, this means RoC of ETMX and ETMY is 62 +/- 2 m and 63 +/- 2 m respectively.

Calculation:
  Transverse mode spacing is expressed by

nu_TMS / nu_FSR = arccos(sqrt(g1*g2)) / pi

  where g1 and g2 is g-factor

gi = 1 - L/Ri

 of ITM/ETM.

  For mode-scan, we swept laser frequency nu. Let's assume this sweep was linear and we can replace laser frequency with time. From the mode-scan result, TMS can be derived by

  t_TMS = sum((n_i-n)*(t_i-t)) / sum((n_i-n)^2)

  where n_i is the order of transverse mode, n is average of n_i's, t_i is the time i-th order mode appeared and t is average of t_i's.
  Since I could only recognize up to 3rd order mode, this can be rewritten as

  t_TMS = 1.5/5 * t_0 + 0.5/5 * t_1 - 0.5/5 * t_2 - 1.5/5 * t_3

  FSR is time between TEM00s. So, g1*g2 can be calculated by

g1*g2 = (cos(pi*t_TMS/t_FSR))^2


X arm result:
  From the 8FSR mode-scan data (see elog #6859), X arm HOM positions in sec are;

HOM 0    242.00     214.76     187.22     159.27     131.33    102.96     74.61     46.00     17.51
HOM 1    234.29     206.78     179.20     150.96     122.90     94.58     66.27     38.10
HOM 2    226.36     198.91     170.80     142.92     114.62     86.51     58.05     29.65
HOM 3    218.14     190.65     162.71     134.78     106.68     78.27     49.95     21.25

  Calculated FSR and TMS in sec are;

FSR    27.24     27.54     27.95     27.94     28.37     28.35     28.61     28.49
TMS     7.951     8.020     8.193     8.151     8.223     8.214     8.220     8.270

  Calculated cavity g-factor are;

g1*g2    0.3699     0.3720     0.3662     0.3704     0.3761     0.3765     0.3839     0.3748

  By taking average,

g1*g2 = 0.3737 +/- 0.002  (error in 1 sigma)


Y arm result:
  From 8FSR mode-scan data (see elog #6832), Y arm HOM positions in sec are;

HOM 0    246.70     218.15     190.06     161.87     133.26    104.75     76.01     47.19     18.60
HOM 1    238.83     210.55     181.88     153.47     124.93     96.08     67.51     39.01
HOM 2    230.48     202.21     173.64     144.80     116.43     86.17     59.84     31.43
HOM 3    222.15     193.47     165.33     137.13     108.60     80.04     51.17     22.25

  Calculated FSR and TMS in sec are;

FSR    28.55     28.09     28.19     28.61     28.51     28.74     28.82     28.59
TMS     8.200     8.238     8.243     8.289     8.248     8.404     8.219     8.240

  Calculated cavity g-factor are;

g1*g2    0.3841     0.3657     0.3683     0.3765     0.3778     0.3683     0.3904     0.3811

  By taking average,

g1*g2 = 0.3765 +/- 0.003  (error in 1 sigma)


Conclusion:
  If ITMs are flat and arm length L = 39 +/- 1 m, this means RoC of ETMX and ETMY is 62 +/- 2 m and 63 +/- 2 m respectively. Designed RoC is 57.35 m.
  Error of RoC is dominated by arm length error. So, we need more precise measurement of the length. This can be done when scan is calibrated and we can measure FSR in frequency.
  Also, we need evaluation of linearity of the sweep. This also can be done by calibration.

From my conversations with JZ and Leo, it seemed there was no package that generated the appropriate mex files. It was clear that the right ones weren't there from the absence of a /cvs/cds/caltech/apps/linux64/lib/matlab2010b directory. I'm sorry if I screwed anything up with pynds, but I have repeatedly asked for help with NDS2+matlab and no one has done anything.

It would be nice to do it via apt if there indeed is a versioned package that can make the mexs. Sorry again if I jumped the gun, but I didn't think anyone was going to do anything.

I guess the only 32-bit machine I can think of is mafalda.

About tconvert, I think the best solution is to make a new wrapper M-file. gps was just a convenient remnant of mDV, but all that we need is some matlab function that can output a GPS time given a date/time string. We can use whatever command-line utility you want.

This was indeed another leap second timing issue.  I'm guessing nodus resync'd from whatever server is posting the wrong time, and it brought everything out of sync again.  It really looks like the caltech server is off.  When I manually sync form there the time is off by a second, and then when I manually sync from the global pool it is correct.

I went ahead and updated nodus's config (/etc/inet/ntp.conf) to point to the global pool (pool.ntp.org).  I then restarted the ntp daemon:

  nodus$ sudo /etc/init.d/xntpd stop
  nodus$ sudo /etc/init.d/xntpd start

That brought nodus's time in sync.

At that point all I had to do was resync the time on fb:

  fb$ sudo /etc/init.d/ntp-client restart

When I did that daqd died, but it immediately restarted and everything was in sync.

No no, this is totally unnecessary.  NDS2 was already installed on every machine from the official packaged releases (apt-get install nds2-client), and it's known to work fine. We use it with pynds all the time. If the matlab component is not working we should figure out the right way to fix it with the existing packages.

In general, please only manually install software as a very last resort.  Manually installed software doesn't get maintained, where as the officially packaged stuff is being actively maintained by the collaboration. If there is a problem with the distributed packaging we should report it and get it fixed (and hint I was the one who built the original Debian packaging for nds2, so I know how to fix all the issues).  I'm trying to bring the 40m out of the dark days of complete chaos, where random software was installed in random locations.

That's because this wasn't the problem!

This sounds like it's more likely the issues. You did the right thing by going to apt to fix the authentication packages.  It's curious to me that you did that here, whereas you went totally out of band for the nds2 client stuff.  Why?

The matlab mex files are the other problem.  But there is also a nds2-client-matlab Debian/Ubuntu package for that as well.  The problem is that the package just distributes the source, and it needs to be compiled.  I'll help figure out a good way to do that.

Again, this is handled by the packaging!  Just use apt and the right architecture is installed automatically.

But what 32 bit machines are you referring to?  I think basically everything is 64 bit nowadays.

We are now using lalapps_tconvert for tconvert.  We're not using that ligotools crap anymore.  I've aliased that to tconvert on the command line, but maybe matlab isn't getting the message.  I'll try to think of a more robust solution (e.g. make a wrapper script).

 I was bad and didn't read the elog before touching things, so I did a daqd restart, and mxstream restart on all the front ends, but neither of those things helped.  Then I saw the elog that Jamie's working on figuring it out.

All the front-ends are showing 0x4000 status and have lost communication with the frame builder.  It looks like the timing skew is back again.  The fb is ahead of real time by one second, and strangely nodus is ahead of real time by something like 5 seconds!  I'm looking into it now.

I tried to lock FPMI using ALS, but I could not take care of ALS for both arms + MI. So, I decided to try one arm + MI.
I don't know why, but I couldn't make it. We need investigation.

Procedure I took:
  1. Align FPMI.

  2. Misalign ETMY.

  3. Press CLEAR HISTORY for C1:ALS-BEATY_FINE_PHASE filter module.
    Are there any command to do this?

  4. Stabilize X arm length.
    I made a script for turning on ALS servo nicely. It currently lives in /users/yuta/scripts/easyALS.py. You have to specify the arm(X or Y) and sign of the gain. It needs to be refined.

  5. Sweep the offset and stabilize X arm lenth to IR resonance.
   (Ran /opt/rtcds/caltech/c1/scripts/ALS/findIRresonance.py Xarm)

  6. Tried to lock MI. I tried to do this by feeding back the signal to BS or ITMs. Both worked fine when ALS holds X arm to IR off-resonance, but I couldn't lock MI when ALS holds X arm to IR resonance. This may come from too much phase fluctuation from X arm reflection. We should investigate this.

Handing off the servo from ALS to LSC:
  I made a script to do this. It just decreases ALS gain and increases LSC gain with 30 sec ramp time. It needs to be refined, so it currently lives in /users/yuta/scripts/handofftoLSC.py. It worked fine without loosing IR transmission.

ALS stability:
  Current stabiliy of the ALS servo is not enough. It doesn't stay for more than ~ 10min. I suspect this is from frequency servo of end lasers losing lock, or beat signals being too small for the beat box because of intensity fluctuation of green transmission. We definitely need to align end greens, but it is painful.

ALS looks OK. I tried to lock FPMI using ALS, but I feel like I need 6 hands to do it with current ALS stability. Now I have all computers being so slow.

It was fine for 7 hours after Jamie the Great fixed this, but fb went down couple times and DAQ status for all models now shows 0x4000. I tried restarting mx_stream and restarting fb, but they didn't help.

I aligned FPMI and greens. There's no recognizable difference between before and after the vent.

What I did:
  1. Aligned Y arm to maximize Y green transmission.
  2. Used PZT1/2 to maximize TRY. But since PZT1 doesn't work so much, I had to align Y arm, too (mostly ETMY). This decreases green transmission, but I will leave it.
  3. Aligned BS and X arm to maximize TRX
  4. Fine tune them to minimize ASDC during FPMI lock, without decreasing TRX
  5. Aligned X end green to get TEM00 transmission.

Now, TRY and TRX are both  ~0.89.
Green transmission from Y and X arm are ~123 uW and ~275 uW respectively. Their max we got so far was ~200 uW and ~255 uW.
I still see clipped beam at AS, which I think is from the Faraday edge, as we found in elog #6897.
Below is the Sensoray capture of some ports, and MEDM screen shots to compare with before vent(see #6893).
There are two AS captures, one is without MI lock and the other is with MI lock. Note that PRM/SRM is misalined.




Next:
 - I will check ALS
 - I keep Y end green optics untouched since elog #6776, to use it as a reference. We need to realign them if tip-tilts are installed in vacuum, or PZTs are installed in both ends.

Issues in PRC:
  1. Power recycling gain is too low (~ 15 instead of 40, according to Kiwamu).
  2. Mode matching to both arms are ~90%(see #6859), but PRC has terrible mode.
       Clipping/flipping in PRC?
  3. From cameras, beam spot moves so much when PRMI is locked.
       Alignment? Mirrors(especially PR2/3) moves too much?
  4. Error signals are glitchy when PRMI is locked.
       Servo design? Mirrors moves too much?

What we have learned from the vent:
  1. PRM, PR2, PR3 was not flipped.
  2. Their suspensions looked OK, too.
  3. We noticed clipping at BS and Faraday. They must be avoided when tip-tilts are installed on next vent.

  4. Took useful photos for next vent. Positions of green optics on optical layout CAD must be updated.
  5. It is not so difficult to recover the IFO state after cycling the vacuum if we use attenuator setup using PBS (see elog #6892).  But, of course, we need plans before cycling.

Commissioning Plan:
  - measure PRMI power recycling gain from POP
  - FPMI using ALS
  - measure PRFPMI power recycling gain from TRY/X
  - correlation between beam spot motion at POP camera and glitch
  - correlation between PR2/PR3 motion and glitch (how can we measure PR2/3 motion? set up oplevs?)
  - mode scan for PRC, using AS AUX laser
  - beam profile measurement at REFL,POP
  - refine servo design of MICH and PRCL

After plenty of work, NDS2 can now be used to get site data within MATLAB using the following machines:

• allegra
• megatron
• ottavia
• pianosa
• rosalba
• rossa

What I did

NDS2 was not working on any of the machines, so the first thing I did was simply to install the newest version. I downloaded the latest tarball (0.9.1) from the LDAS Wiki, unzipped and installed it

/users/zach $ tar -xvf nds2-client-0.9.1.tar

/users/zach $ cd nds2-client-0.9.1

/users/zach $ sudo ./configure --prefix=/cvs/cds/caltech/apps/linux64 --with-matlab=/cvs/cds/caltech/apps/linux64/matlab/bin/matlab

/users/zach $ sudo make

/users/zach $ sudo make install

Even with the new version, it still didn't work.

Solution: The main problem was that the cyrus-sasl-gssapi authentication protocol was not installed on these machines, so that even with a kerberos ticket the datalink could not be established. Using information from the LDAS Wiki, I used aptitude to install it as:

$ sudo aptitude install lscsoft-auth

This group installs both the SASL protocol and the package python-kerberos

I also needed to update the kerberos config file for each machine, which is located at /etc/krb5.conf. I found that ottavia had a nice one with many realms, so I copied that one over to the other machines. In any case where there was an old config file overwritten, it is now /etc/krb5.conf.old.

Finally, the matlab path for NDS2 was still set to the old 2010a directory (/cvs/cds/caltech/apps/linux64/lib/matlab2010a) that was created by the NDS2 install when Rana originally did it. The new install I made above created the appropriate 2010b mexa64 files, so I changed the matlab path within matlab to this one:

>> rmpath /cvs/cds/caltech/apps/linux64/lib/matlab2010a

>> addpath /cvs/cds/caltech/apps/linux64/lib/matlab2010b

>> savepath

Now everything works fine on all these machines. As in Rana's original post, you get data in the following way:

$ kinit albert.einstein %then enter password

$ matlab -nosplash -nodesktop

>> d = NDS2_GetData({'H1:LSC-NPTRX_OUT16.mean'},963968415,6000,'nds.ligo.caltech.edu:31200')

d = 

            name: 'H1:LSC-NPTRX_OUT16.mean' 

            chan_type: 'm-trend'             

            rate: 0.0167       

            data_type: 'real_8'     

            signal_gain: 1   

            signal_offset: 0     

            signal_slope: 1     

            signal_units: ''   

            start_gps_sec: 963968415     

            duration_sec: 6000             

            data: [100x1 double]           

            exists: 1

>> quit % since you've seen that the data is really here

$ kdestroy % so that no one uses your credentials

Some thoughts

• I would like to extend this to the 32-bit machines, but I have to figure out the best way to install the proper NDS2 client without interfering with the 64-bit version. I think it is just a matter of specifying the matlabroot in the .../linux/ instead of .../linux64/
• It would be nice to find a way that the nice tool gps('MM/DD/YYYY XX:XX:XX UTC'), which calls the ligotool executable tconvert, can be automatically usable when calling NDS2 functions. Right now, there seems to be an issue preventing that: even though tconvert can be run in the terminal, gps() returns an error and even directly running unix('tconvert now') or !tconvert returns the same error. I have emailed Peter Shawhan to see if he has any advice.

I got a call from Koji and Yuta that something was wrong with the CDS system.  I somehow had an immediate suspicion that it had something to do with the recent leap second.

It took a while for nodus to respond, and once he finally let me in I found a bunch of the following in his dmesg, repeated and filling the buffer:

Jul  3 22:41:34 nodus xntpd[306]: [ID 774427 daemon.notice] time reset (step) 0.998366 s
Jul  3 22:46:20 nodus xntpd[306]: [ID 774427 daemon.notice] time reset (step) -1.000847 s

Looking at date on all the front end systems, including fb, I could tell that they all looked a second fast, which is what you would expect if they had missed the leap second.  Everything syncs against nodus, so given nodus's problems above, that might explain everything.

I stopped daqd and nds on fb, and unloaded the mx drivers, which seemed to be showing problems.  I also stopped nodus's xntp:

  sudo /etc/init.d/xntpd stop

His ntp config file is in /etc/inet/ntp.conf, which is definitely the WRONG PLACE, given that the ntp server is not, as far as I can tell, being controlled by inetd.  (nodus is WAY out of date and desperately needs an overhaul.  it's nearly impossible to figure out what the hell is going on in there).  I found an old elog of Rana's that mentioned updating his config to point him to the caltech NTP server, which is now listed in the config, so I tried manually resyncing against that:

  sudo ntpdate -s -b -u 131.215.239.14

Unfortunately that didn't seem to have any effect.  This was making me wonder if the caltech server is off?  Anyway, I tried resyncing against the global NTP pool:

  sudo ntpdate -s -b -u pool.ntp.org

This seemed to work: the clock came back in sync with others that are known good.  Once nodus time was good I reloaded the mx drivers on fb and restarted daqd and nds.  They seemed come up fine.  At this point front ends started coming back on their own.  I went and restarted all the models on the machines that didn't (c1iscey and c1ioo).  Currently everything is looking ok.

I'm worried that there is still a problem with one of the NTP servers that nodus is sync'ing against, and that the problem might come back.  I'll check in again later tonight.

MC came back to the state as it was before the vent.

What I did:
  1. Removed beam attenuating setup on PSL table(see elog #6892).

  2. Removed 100% reflection mirror before the MC reflection PD and put 10% BS back, so that we can have MC WFS. Also, I changed C1:IOO-MC_RFPD_DCMON.HOPR to 5.

  3. Removed autolockMCmain40m_low_power from crontab on op340m, and put autolockMCmain40m again.

  4. Aligned MC and ran /opt/rtcds/caltech/c1/scripts/MC/WFS/WFS_FilterBank_offsets to adjust WFS offsets.

  5. Measured beam spot positions. They looked same as before the vent.

# filename    MC1pit    MC2pit    MC3pit    MC1yaw    MC2yaw    MC3yaw    (spot positions in mm)
./dataMCdecenter/MCdecenter201206290135.dat    2.914584    4.240889    2.149244    -7.117336    -1.494540    4.955329    before vent
./dataMCdecenter/MCdecenter201207011253.dat    3.294659    3.416584    2.620511    -6.691800    -3.164084    4.806517    after vent
./dataMCdecenter/MCdecenter201207032009.dat    3.737099    3.994597    3.087857    -6.442053    -0.992543    4.714607    after pumping (now)

  6. I also turned on high voltage power supplies for input and output PZTs

  7. Below is captured Sensoray images of the current state.



Next:
  I will go on to check if IFO works as it was before or not, but I think we should center MC beam spot positions and see if we can avoid clipping in the near future.

I put them in the "visible optics" drawer of the newish, metal optics cabinet with the thin drawers down the Y arm.

I'm still having issues with the STACIS oscillating uncontrollably with the slightest extra vibration, but with some more added weight both x and z direction are stable if you don't disturb the setup.

I took more transfer functions of the STACIS. In the last data I took Jenne pointed out that the geophone signals were not correlated well with the driving signal, so I increased the amplitude of the driving signal and am looking in x and y too instead of just z. 

Details of the driving signal: 25 mV, swept sine from 0.1 to 100 Hz from the SR785. 

NOTE: The data below was all transferred from the SR785 using netGPIB, which works fine, if anyone was interested in using it.

Open loop in the y direction, taken with the y geophone (magnitude on top, phase on bottom):

Open loop in the x direction, taken with the x geophone (with some extra weight to try to make the closed loop more stable):

Open loop in the x direction, taken with accelerometer instead of geophone:

We have received the dichroic optics from Laseroptik.  The coatings are:

HR:

• 532nm: T(s+p) > 97%
• 1064nm:  R(p) > 99.9%

AR:

• 532nm: R(s+p) < 1%
• 1064nm: R(p) < 2%

We got two sets with these coatings:

• 6x: 50 x 9.5mm, 2 degree wedge
• 8x: 25 x 6.35mm, 2 degree wedge
• 1x: 25 x 3mm, witness

Vacuum Normal State is reached in 9 hours.  CC1 =  2e-5 Torr

Aux dry pump #3 is still running.  The RGA is not pumped yet.

This past Friday I swapped out a burnt resistor on the spare STACIS unit I'm working with and powered it up. Here's the setup:

And here's what happened:

X an Y directions: When I switched from open to closed loop (making the internal geophones provide feedback), the STACIS started making a loud noise- it seemed like it was oscillating uncontrollably.

Z direction: The same thing happened in z until I added some weight to the top of the STACIS- then it quieted down, and seemed to work okay. The geophone signal dropped considerably compared to the open loop signal, which is expected if the feedback is working.

Then I tried driving the PZTs with a signal from the SR785 network analyzer. With an amplitude of tens of mV and frequencies from around .1 to 200 Hz, I could see the accelerometers I mounted on top of the STACIS definitely register motion, which means I was successfully driving the PZTs.

Below are transfer functions of the STACIS as I drove the PZTs from .1 to 100 Hz at 10 mV. The top graph is open loop, the second is closed loop. These were measured with the internal geophones.

In the bottom graph, "A" is closed loop and "B" is open loop, where the transfer functions were taken with the accelerometers instead of the geophones.

Many photos were taken by many different people....most of the fuzzy ones are by yours truely (doing a reach-around to get to hard-to-reach places), so sorry about that.

I put all the photos from yesterday and today into 6 new albums on Picasa:  https://picasaweb.google.com/foteee

The album titles are generally descriptive, and I threw in a few comments where it seemed prudent.

Big note:  The tip tilt on the ITMX table does, in fact, have the arrow pointing in the correct direction.  Photo is in the TT album from today.

[Koji, Steve, Jamie, Jenne, Yuta]

We opened BS and ITMX chambers, took lots of photos, and closed them with heavy doors.
I turned off high voltage power supplies for PZTs and blocked PSL beam. We are ready for the pumping tomorrow.

Important photos we took:
  - positions of green optics at BS chamber, which was moved on the vent on Aug 2011
  - positions of PZT mirrors and cable connectors at BS chamber, which will be replaced with tip-tilts on the next vent
  - arrow on PR2 pointing HR (it was correct)
  - tried to take photos of clipping IR beam at BS OSEM holder from ITMX chamber
 
 We also took bunch of other photos.


Beam dump needed at BS chamber:
  We also checked some un-dumped beams at BS chamber. We need dumps;
  - behind MMT1, for unwanted transmitted beam
  - behind IPPOSSM3, for unwanted transmitted beam (IPPOSSM3 is the last mirror in BS chamber for IPPOS)

 Yuta just told Jamie and I that when he and Koji were looking at things yesterday, they saw that the beam spot was roughly at the center of the PRM, but was clipping on the lower OSEM holder plate on the BS.  This indicates that the beam spot on the BS is much too low.  The easiest way I can see this happening is poor pitch pointing with the tip tilts, which we unfortunately don't have active control over.

Recall elog 3425, where I mentioned some pretty bad pitch pointing after a TT was moved from the cleanroom, to the chamber, back to the cleanroom.  I think that we may need to check the pitch pointing at the chamber before installing tip tilts in the future.

 Start pumping on Monday before Steve goes home.

[Koji, Yuta]

We aligned PRMI and inspected BS chamber. Last inspection by Jamie and I (see elog #6897) was done when nothing is aligned, so I wanted to see the difference.
Aligning PRMI at low power was difficult for me, because I see no fringe at ASDC PD nor REFLDC PD. I just aligned them by looking at AS/REFL camera. The beam shape at AS looked as bad as when the usual power.

No significant change was found inside the vacuum. We still see clipping at the Faraday, and also, we saw clipping by BS coil holder. Using PZT1, we could make it better, but this might be causing PRC problem -- BS is inside the PRC, too.

We also took some pictures of PR3 and PRM(attached). The arrow pointing HR is correctly pointing inside the PRC. Seeing is believing.

Yuta's plan:
  We might have to avoid clipping at BS (and Faraday) by aligning input optics inside the vacuum. If we are going to align them, I think we should start from centering MC beam spot positions and the whole alignment could take more than a week. I don't want to spend too much time on the alignment. Also, we are going to install tip-tilts on the next big vent, so we have to redo the alignment anyway.
  So, my plan is as follows;

1. Take lots of photos and close the door on Monday(June 2).

2. Pump on Tuesday(June 3).

3. Restart working on ALS. For example, demonstration of FPMI using ALS.

4. We also can do some characterization of PRC, like measuring power recycling gain for PRMI/PRFPMI, mode scan for PRC using AUX laser from AS port, and so on. We need some calculation for clipping tolerance, too.

  Any objections?

I modified autolocker for MC in low power mode (/opt/rtcds/caltech/c1/scripts/MC/autolockMCmain40m_low_power) to make it work with the current directory structure.
autolockMCmain40m_low_power currently runs on op340m and it is in crontab.

34 * * * *  /opt/rtcds/caltech/c1/scripts/general/scripto_cron /opt/rtcds/caltech/c1/scripts/MC/autolockMCmain40m_low_power >/cvs/cds/caltech/logs/scripts/mclock.cronlog 2>&1


MC intra-cavity power:
  Currently, incident beam to the MC measured at PSL table is ~15 mW. Reflected power from MC (C1:IOO-MC_RFPD_DCMON) is 0.94 when MC unlocked, and is 0.088 when locked.
  That means, considering MC1/3 power transmission is 2000ppm (calculated finnesse=1570), intra-cavity power in MC is ~7 W.

  15 mW * (0.94-0.088)/0.94 / 2000ppm = 7 W

  We can increase the power by factor of ~2, if needed.


MC beam spot positions:
  I aligned MC to maximize transmission (C1:IOO-MC_TRANS_SUM_ERR), and measured the MC beam spot posisions in atm, low power.

# filename    MC1pit    MC2pit    MC3pit    MC1yaw    MC2yaw    MC3yaw    (spot positions in mm)
./dataMCdecenter/MCdecenter201206290135.dat    2.914584    4.240889    2.149244    -7.117336    -1.494540    4.955329    before vent
./dataMCdecenter/MCdecenter201207011253.dat    3.294659    3.416584    2.620511    -6.691800    -3.164084    4.806517    after vent

  They look the same within the error of the measurement, except for the spot positions on MC2, which we don't care.


Autolocker should be refined:
  To make autolockMCmain40m_low_power, I copied autolockMCmain40m and just changed

- lockthresh from 500 to 100
- use mcdown_low_power instead of mcdown
- use mcup_low_power instead of mcup

  The difference between mcdown_low_power and mcdown should be only

- ezcawrite C1:IOO-MC_REFL_GAIN 31 for lowpower, 9 for usual
- ezcawrite C1:IOO-MC_VCO_GAIN 10 for lowpower, -5 for usual

  The difference between mcup_low_power and mcup should be only

- ezcawrite C1:IOO-MC_REFL_GAIN 31 for lowpower, 12 for usual
- ezcawrite C1:IOO-MC_VCO_GAIN 31 for lowpower, 25 for usual

  Currently, they are not like that. Somebody good at shell scripts should combine them and make it into one code with an option something like usual/low-power.

  We could set up a simple pick  off after the Faraday  and bring it  out the north window of IOO chamber. No monitor needed, just take the cover off when you want to see it.

Most people have no idea how to get the MC through the F

[Yuta, Koji, Jamie]

We went into ITMX chamber to inspect the situation there.  We looked for clipping and flipping at PR2, and found none, although we noticed that the beam at PR2 looks a little clipped.

We then went back into the BS chamber and took a closer look at the beam incident on PRM, and the situation with PR3.  The PRM incident beam looked a little clipped, which we expected from the PR2 observation.  But the beam looks well centered on PRM and PR3.  As best I could tell the beam is reflecting off the front surface of PR3, as expected.

Looking at the beam around MMT1 and PJ2 (the second PZT on BS), we could tell that the beam incident and reflected off of MMT1 looked round, where as the beam incident on PJ2 looked clipped.  Using my tallness super power I was able to reach into the IMC chamber and confirm that the beam going from MMT1 to MMT2 clips fairly badly on the edge of the Faraday.   Koji speculates that this is the result of a misalignment of the PSL output beam into the MC.  In any event, it's not clear how this would be the cause of our PRC woes.

We decided to close up for the night, and let Yuta work on aligning PRMI.  We need to figure out what the heck to do now.

We've been watching the input power reduce, from 18 mW initially when we first went in, to about 5mW now.  It seems to be leveling out now.  It's unclear what would have been causing it.  Drift of the input polarization?

[Koji, Steve, Jamie, Yuta]

So, PRM was NOT flipped......

We opened the BS chamber and quickly checked the arrow on the PRM pointing HR. It turned out to be correct, the arrow was pointing towards the arm cavity. We opened the ITMX chamber, too, to check PR2 later.
BS chamber and ITMX chamber is now closed with the light door.

But it was a one step forward anyway, because we could prove PRM was innocent.

What to do next:
  We know that the mode-matching of the incident beam and both arms are pretty good. So, dirty modes come from PRC.
  We will check beam clipping, mirrors, suspensions in PRC.
  I expect the chambers to be closed on Monday(July 2) afternoon and start pumping on Tuesday(July 3) morning.

 The 4 hrs vent plot at 3 torr/min rate.

Nitrogen was used from 1e-6 torr to 35 torr  at intake pressure 14 PSI

The rest was filled with 5 cylinders of Instrument Grade Air at intake pressure 14 PSI

We can start opening chamber at 3 pm today

 Steve, Yuta and Jamie

Jam nuts were checked and oplev servos were turned off. Sus summery is below with strain gauge values. Are the strain gauge values have any meaning when the PZT contorrels are off??????????????????

1. Turned off high voltage power supplies for PZT1/2 (input PZTs) and OMC stage 1/2. They live in 1Y3 rack and AUX_OMC_NORTH rack.

2. Restored all IFO optics alignment to the position where I aligned this afternoon (for SRM, I didn't aligned it; it restored at the saved value on May 26).

3. Centered all the oplevs. They can be used for a reference for alignment change before and after the vent.

I will leave PSL mechanical shutter and green shutters closed just in case.

Some MEDM screenshots below.

[Koji, Jamie, Yuta]

We attenuated the incident beam (1.2 W -> 11 mW) to the vacuum chamber to be ready for the vent.
The beam spot on the MC mirrors didn't changed significantly, which means the incident beam was not shifted so much.

What we did:
 1. Installed HWP, PBS(*) and another HWP between the steering mirrors on PSL table for attenuating the beam. We didn't touched steering mirrors(**), so the incident beam to the IFO should be recovered easily, by just taking HWPs and PBS away. The power to the MC was reduced from 1.2 W to 11 mW.

(*) We stole PBSO from the AS AUX laser setup.
(**) Actually, we accidentally touched one of the steering mirrors, but we recovered them. We did the recovery tweaking the touched nob and minimizing the MC reflection. We confirmed the incident beam was recovered by measuring MC beamspot positions(below).

 2. Aligned PBS by minimizing MC reflection, adjusted first HWP so that the incident beam will be ~10 mW, and adjusted last HWP to minimize MC reflection (make the incident beam to the MC be p-polarization).

 3. To do the alignment and adjusting, we put 100% reflection mirror (instead of 10% BS) for the MC reflection PD to increase the power to the PD. That means, we don't have MC WFS right now.

 4. Tweaked MC servo gains to that we can lock MC in low power mode. It is quite stable right now. We didn't lose lock during beam spot measurement.

 5. Measured beam spot positions on the MC mirrors and convinced that the incident beam was not shifted so much (below). They look like they moved ~0.2 mm, but it is with in the error of the MC beam spot measurement.

# filename      MC1pit  MC2pit  MC3pit  MC1yaw  MC2yaw  MC3yaw  (spot positions in mm)
./dataMCdecenter/MCdecenter201206281154.dat     3.193965        4.247243        2.386126        -6.639432       -0.574460       4.815078    this noon
./dataMCdecenter/MCdecenter201206282245.dat     3.090762        4.140716        2.459465        -6.792872       -0.651146       4.868740    after recovered steering mirrors
./dataMCdecenter/MCdecenter201206290135.dat     2.914584        4.240889        2.149244        -7.117336       -1.494540       4.955329    after beam attenuation

 6. Rewrote matlab code sensemcass.m to python script sensemcass.py. This script is to calculate beam spot positions from the measurement data(see elog #6727). I think we should make senseMCdecenter script better, too, since it takes so much time and can't stop and resume the measurement if MC is unlocked.

We see some ghost beam spots at POP. This may come from the back of PR2 and PR3. Also, they may change mode matching because of thermal lensing, mirror deformation, and other unexpected reasons. I thought we should check every mirrors in PRC, if PRM is not flipped.

We are going to check PRM just because we spent so much time for the PRC problem, and still don't have the solution or evidence.
PRM flipping is kind of the only idea for the root of all evil -- terrible beam shape, low PR gain, unstable PRMI lock.
So, I want to check with my eye during the stay.

I don't think we have to redo magnet gluing. It's okay to leave them on HR side.

 What do you mean by PR2, PR3 flipping?  They are (supposed to be) flat mirrors, so obviously they should be installed correctly, but they won't change the mode matching in a huge way if they're backwards, right?

For the PRM, I recommend checking (a) the arrow inscribed on the thinner side of the optic and (b) that the arrow *actually* points to the HR side.  I'm pretty sure I installed all the optics with the arrow pointing away from the OSEMs, but I never did a thorough check that the arrow always actually pointed to the HR coated side.  I don't remember any optics where I said "hmmm, that's funny, the arrow is pointing backwards", but nor did I write down that I had checked.

Also, hopefully the PRM is correct.  If however it's not, that means that all of the magnets are glued onto the HR side, and we'll have to redo all of the magnet gluing.  The guiderods should be fine, but all 6 magnets would need redoing.  If we were very, very careful and didn't break any of the magnets off of the dumbbells, it's a 24 hour turnaround due to drying time.  Since inevitably we break magnets away from dumbbells, conservatively we should think about a 48 hour turnaround. 

Koji, Jamie and I talked together and I decided to VENT TOMORROW MORNING. Main purpose of this vent is to see if PRM is flipped or not.

Vent schedule:
June 28 (Thu)
  Prepare for the vent tonight

June 29 (Fri)
  Start vent in the morning
  Look into PRC in the evening. If PRM was flipped, we will correct them. We'll use REFL to align the PRM. If PRM was not flipped, look into PR2,PR3 and other related optics.

June 30 (Sat)
July 1 (Sun)
  Thinking time. I can work if needed.

July 2 (Mon)
  If we need something else to do, do it.
  If not, start pumping.
  July 4th is the Independence Day. So, I need IFO working before July 4th.

Check List:
 We will just open the BS chamber.
  - PRM flipping
  - PR2, PR3 flipping
  - PRC suspensions
  - Cipping check in PRC

 Cycling the vacuum is easy. Why not vent starting Thursday evening and pop the doors on Friday morning? Inspect on Friday and pump on Monday morning.

To be fair, this is Kiwamu's idea. And nothing is reasonable before it is confirmed quantitatively.