[Manasa, Yuta]

We set up POP camera and POPDC PD, and centered REFL PDs.
We also tried to center AS55 PD, but AS55 seems to be broken.

What we did:
 1. POP path alignment:
   Shot green laser pointer from ITMX table at where POPDC PD was sitting and centered green beam at optics in the POP path. Steered POPM1/M2 mirrors in the ITMX chamber to make green laser overlap with the PRM-PR2 beam as far as I can reach from ITMX chamber. We removed some ND filters and a BS for attenuating POP beam because POP power was somehow so low. Currently, POP is pick-off of the beam which goes from PRM to PR2.

 2. POP camera and PD:
   We first used camera to find the beam at where POPDC PD was sitting because it is much easier to find focused beam. Put an iris in front of the camera, and put POP DC behind it. Steered a mirror in front of PD to maximize DC output.

 3. REFL PDs:
   Steered mirrors in the REFL path to center the beam and maximized DC outputs, as usual.

 4. AS55:
   AS55 was not responding very much to the flashlight nor AS beam. C1:LSC-ASDC_OUT looked funny. By swapping the ribbon cables of AS55, REFL55, and REFL165, I confirmed that AS55 PD itself is broken. Not the ribbon cable nor PD circuit at LSC rack. I don't know what happened. AS55 was working on Feb 8 (elog #8030).

Result:
  We aligned PRMI coarsely. POP(right above) looks much better than before. REFL (left below) still looks elliptic, but ellipticity differs with the position on the camera. Some astigmatism is happening somewhere. AS (right below) looks pretty nice with MI aligned.


Next:
  1. Fix AS55? Or replace it with POP55 PD, which is currently unused.
  2. Confirm we are getting the right error signals or not, and lock PRMI.

We will start preparing for pumping down. Main goal for this is to demonstrate PRFPMI using ALS.
Here are to-dos before we pump down.

Feb 18 eveing
- check input beam and Y arm alignment again
- IPPOS/IPANG alignment
- check all oplevs

Feb 19 morning
- open ETMX chamber heavy door
- align BS to X end
- adjust OSEM values (added by YM)
- center beam on all AS optics
- make sure AS/REFL is clear
- take picture of flipped PR2 (added by YM)
- make sure green is not clipped by new PRM oplev mirrors  (added by YM)
- center all oplevs

Feb 19 afternoon - Feb 20 morning
- close PSL shutter
- close all heavy doors and put the access connector back
- start pumping down

Feb 20 evening
- start aligning IFO

I undertook the investigation of the AS55 PD. I found the PD is not broken.

I tested the PD on the PD test bench and it works just fine.

I attatched the characterization result as there has been no detailed investigation of this PD as far as I remember.

The transimpedance gain at 55MHz is 420Ohm, and the shotnoise intercept current is 4.3mA.

[Yuta, Sendhil]

We aligned IPPOS, IPANG and all OPLEVs (except for ETMX and SRM).

1. First aligned the IPPOS by tweeking the steering mirrors inside the BS chamber.

2. Aligned the IPANG by tweeking the steering mirrors inside the BS chamber and ETMY chamber.

3. Aligned the OPLEVS for the BS and PRM was done by tweeking the steering mirrors inside the BS chamber and checked that OPLEV beams were not clipped. 

4. Centred the OPLEV beams for the ITMY and ETMY.

5. For the OPLEV of ITMX the alignment was done by tweeking the steering mirrors inside the ITMX chamber.

[Yuta, Sendhil, Jamie, Jenne, Rana]

1. After the MC centering, we tried to align the IFO using IPPOS and IPANG as reference. This did not recover the alignment perfectly. We were clipping at the BS aperture. Using TTs, we centered the beam at BS and PRM.
2. Using TTs, the beam was centered at ITMY and ETMY.
3. IPPOS and IPANG mirrors in-vacuum were aligned and were centered at the out-of-vacuum optics.
4. We checked the centering of the beam on optics in the BS and ITMY chamber. (Yuta will make an elog with the layout)
5. We retro-reflected ITMY at the BS and aligned ETMY such that we saw a couple of bounces in the arm cavity.
6. Using BS, the beam was steered to go through the center of ITMX and ETMX.
7. At this point we were able to see the MI fringes at the AS port.
8. We made fine alignments to the ITMX such that we saw MI reflected at the Farday.
9. We retro-reflected ITMX and aligned ETMX to see the beam bounce at the ITMX.
10. We aligned PRM such that PRC flashes. But we were not happy with the flashes (they were in higher order modes). We suspect that minor tuning of the input pointing might be necessary.
11. We closed for the day

I attached clipping/centering checklist for the alignment.
Blue ones are the ones we checked today. Red ones should be checked tomorrow. Circles indicate centering on the optics, rectangles indicate clipping check, and arrows indicate retro-reflecting or bounces.
We found mis-centering on MMT1, PR2 and SR3 tonight (by ~0.5 beam diameter). They are also indicated.

I think we don't want to touch MMT1 and PR2 anymore, because they change input beam pointing.
I'm a little bit concerned about high beam on SR3, because we had PRC flashing in vertical higher order modes. We also see ETMX slider values high in pitch (~ 5.4).
Also, the diameter of ETMX reflected beam on ITMX looked larger and dimmer than ITMX transmitted beam, which doesn't seem reasonable.


Wednesday, Feb 20:
 - tweak TT1/TT2 and PRM so PRC flashes
 - re-check Yarm/Xarm bounces
 - center beam on all AS optics, starting from SR2
 - make sure REFL and AS is clear
 - check if TRY/TRX are coming out from the ends
 - check beam centering on mirrors in IMC/OMC chamber as far as you can reach
 - inject green from both ends
 - make sure green beams are not clipped by mirrors on BS chamber, IMC/OMC chamber
 - re-center all oplevs, with no clipping
 - check all OSEM values
 - take pictures of flipped PR2 and input TTs (and everything)
 - close all heavy doors and put the access connector back

Thursday, Feb 21:
 - make sure we can lock PRMI
 - start pumping down when Steve arrives

After in-vac alignment work last night, PRC is flashing brighter than PRMI alignment last week.
My hypothesis is that "we aligned PRM to junk MI fringe last week". Possibly, we used MI fringe caused by AR reflection of ITMs, or MI fringe reflected from SRM.

Videos:
  PRC flashing last week (youtube, elog #8085, elog #8091)

  PRC flashing this time (Lens in-front of AS camera was taken out)



My hypothesis can explain:
 - why we had dimmer POP last week (flash in half-PRC was way brighter even when we had more attenuators (youtube))
 - why I thought AS55 is broken (AS was too dim)

Conclusion:
  Be careful of junk beams.

I have updated the vacuum checklist for in vacuum alignment.  Please take a look (https://wiki-40m.ligo.caltech.edu/vent/checklist) and see if I missed anything.  My goal was to make it incredibly step-by-step so there can be no mistakes.

[Yuta, Manasa, Sendhil, Jenne, Steve, Jamie, Koji, Evan]

The interferometer is well-aligned, and ready for pump-down.  The access connector is in place, as are the ETM heavy doors.  We will do ITM and BS doors tomorrow, then begin pumping.

Before we redid the ITM pointing, I confirmed that I could see both POX and POY on their respective tables, on a camera, unclipped.  I should check again quickly now that the ITM pointing has been finalized.

We went back to the arms, to perfect the ITM pointing.  Input beam was already centered at ETMY.  ETMY was pointing so that beam reflected to ITMY.  ITMY was adjusted a few (less than 4?) steps of 1e-3 size, to make reflected beam hit center of ETMY. 

BS was already pointing so beam hit center of ETMX.  ETMX was pointing to hit center of ITMX.  ITMX was adjusted a few (less than 4 again?) steps of 1e-3 size to make reflected beam hit center of ETMX.

Checked centering on AS path.  AS beam comes out of the vacuum a little low, but this wasn't discovered until after the access connector was in place.  We could adjust PZT3 (last AS mirror on BS table that sends beam over to OMC table), but we don't want to do this since we won't be able to re-confirm centering on the 3 mirrors on the OMC table.

Green beams (first Y, then X) were aligned using out-of-vac steering mirrors until beams were flashing in their respective arm cavities.  Green Y is a little close to the edge of the bottom periscope mirror, on the "up" periscope.  Since there is no steering between the arm and this periscope, fixing it would require moving the periscope.  We leave this to the next vent, when we finally install the BS table extension.  We were flashing a higher order yaw mode (5ish nodes) for the Y arm, and the very edge of the higher order mode on one side was a little bit clipped after reflecting off the steering mirror on the OMC table.  This is happening because that mirror is in the mount backwards (so we have access to the knobs).  We are confident that the straight-through beam is well centered on that mirror, so once we get it aligned to TEM00, there will be no clipping. We then did the X arm green, which was flashing a pitch higher order mode (again 5ish nodes).  The very edge of the higher order mode is clipping a little bit on the top mirror of the "down" periscope on the IMC table, but again the straight through beam is okay, and we're confident that the TEM00 mode will make it unclipped.  We could have touched some steering mirrors on the BS table, but since this was once upon a time well aligned, we don't want to futz with it.

Corner oplevs are all centered on their QPDs.  (The ETM oplevs were centered a few days ago).

Access connector and ETM doors are on.

The last 3 vertex doors will go on tomorrow when Steve gets in, and then we'll start pumping. 

There are no in-vac PZTs that need to be turned on (we've been using the output steering PZTs as non-energized fixed mirrors for some time), so we can lock at our leisure tomorrow afternoon.

[Manasa, Evan]

Manasa and I are trying to get the AS beam onto the AS camera with a focusing lens. Currently, the mirror immediately preceding the camera has been removed and the camera and lens are sitting directly behind the BS.

[Sendhil, Yuta]

After aligning IFO and putting the access connector on, we also centered IPANG/IPPOS and all oplevs (except SRM).
To avoid clipping of PRM/BS oplevs, we re-arranged oplev steering mirrors on BS table.

What we did:
  1. Checked IPANG comes out unclipped after putting on the access connector.
  2. Centered IPANG on its QPD.
  3. Checked oplevs beams for ITMX/ITMY centered on in-vac mirrors, and centered them on their QPDs.
  4. Checked IPPOS beam is centered on the mirrors inside BS chamber, and centered IPPOS on its QPD.
  5. Tweaked oplev mirrors on BS chamber to make PRM/BS oplev beam unclipped and centered on mirrors, and centered them on their QPDs. To avoid clipping of oplev beams in BS table, we re-arranged oplev steering mirrors on BS table (outside the vaccum).


Current status:
  QPD values, IFO_ALIGN/MC_ALIGN screens, OSEM values attached.

  - IR incident beam and IFO aligned
  - X/Y end green coming out to PSL table (in higher order modes)
  - IPANG/IPPOS available
  - All oplevs available
  - AS/REFL/POP cameras ready
  - access connector, ETMX/ETMY heavy doors on
  - ITMX/ITMX/BS heavy doors are not on
  - AS/REFL/POP PDs not centered
  - POX/POY/TRX/TRY not aligned
  - AS beam coming out of the OMC chamber low by ~ 1 beam diameter (my bad)


Tomorrow:
  - Align AS/REFL/POP PD and lock PRMI
  - Take pictures of ITMX/ITMY/BS stacks
  - Put heavy doors on ITMX/ITMY/BS chambers
  - Start pumping down

Please confirm the SRM OL beam is not too bad and also find where the mis-aligned SRM puts its beam. WE want to be sure that there is not too much unwanted scattering from SRM into the PRFPMI.

Currently, SRM is misaligned in pitch so that SRM reflected beam hits on the top edge of SR3 (not on the mirror, but on the cage holding the mirror).
We also confirmed that SRM oplev beam is coming out from the chamber unclipped, and centered on QPD when SRM is "aligned".

Blue ones are the ones we checked yesterday.
Green ones are the ones we checked today.
Red ones are the ones we couldn't check.

We noticed mis-centering on green optics and partial clipping of higher order modes, but we did not touch any green optics in-vac. This is because green beam from Y end and X end has different spot positions on the green optics after periscopes. We confirmed that direct green beam from ends are not clipped.

I believe we have checked everything important. Any other concerns?

 Is the beam going towards the OMC going to cause backscatter because of uncontrolled OMC or can we park that beam somewhere dark?

I'm not sure about the OMC situation at 40m. I think there are no direct beam reflected back into IFO from OMC path. There must be some backscatter, but we have to open OMC chamber again to put a beam dump.
I don't think we want to put one in OMC path for this pump-down, but we can put a beam dump to dump reflected beam from mis-aligned SRM tomorrow, if available.

[Koji, Yuta]

We aligned Y arm to Y green and tweaked TT1/TT2 to get IR locked in Y arm.

Alignment procedure:
  1. Align ETMY/ITMY to maximize TEM00 green transmission to PSL table. We reached ~240 uW.

  2. Aligned PRM and TT2 so that PRM reflected beam go through FI and get ITMY-PRM cavity flashing. This is to coarsely align input pointing to Y arm. After this alignment, we got tiny Y arm flash. Input pointing to IPANG QPD was lost.

  3. Aligned TT1/TT2 to maximize TRY in TEM00. We reached ~0.92.

Failed procedure:
  I was struggling with finding Y arm flash. I was using IPPOS/IPANG as input pointing reference, and slider values (C1:SUS-(ITMY|ETMY)_(PIT|YAW)_COMM) or OSEM values (C1:SUS-(ITMY|ETMY)_SUS(PIT|YAW)_IN1) before pumping for Y arm alignment reference. But it was a lot more easier if Y arm is aligned to green and having Yarm cavity axis assured.

Next:
  - X arm flash in IR
  - Steer X end green
  - If X arm or AS looks bad, adjust IR input pointing and Y arm alignment. We have to steer Y end green afterwards.

 That's good news. I was ready to give up and say we should vent and remove the baffles. It will be interesting if you can find out how much the sensors and OL and IPANG are off from their pre-pump values. We should think about how to have better references.

Also, what is the story with the large drift we are seeing in IPANG?

[Jenne, Evan, Yuta]

After Y alignment, X arm is aligned to IR and we got both arms locked in IR.
There's some dift (input pointing?) and this made aligning both arms tough. I will elog about it later.
Attached is ETMYF. ETMXF, ITMYF, ITMXF when both arms are locked by IR.

Alignment Procedure:
  1. Algined BS/ITMX to get MI fringe in AS. We got X arm flashing at this point.
  2. Use BS/ITMX/ETMX to get TRX maximized, without losing good MI fringe in AS. We reached 0.75.
  3. There was clipping of TRX beam at Xend optics. Since whole IFO alignment is started from Y green, this clipping is because of poor Y green pointing. But we needed clear TRX for aligning Xarm, so we re-arranged Xend TRX path to avoid clipping.

X arm issues:
  - Beam motion at TRX is larger than TRY. Turning off clean table air didn't help. Maybe we need BS oplev on or ETMX coil gain balancing.
  - X green scatters into TRX PD and ETMXT camera. Fix them.

[Jenne,Yuta]

Both arms are aligned starting from Y green.
We have all beams unclipped except for IPANG. I think we should ignore IPANG and go on to PRMI locking and FPMI locking using ALS.
IPANG/IPPOS and oplev steering mirrors are kept un-touched after pumping until now.

Current alignment situation:
 - Yarm aligned to green (Y green transmission ~240 uW)
 - TT1/TT2 aligned to Yarm (TRY ~0.86)
 - BS and Xarm alined to each other (TRX ~ with MI fringe in AS)
 - X green is not aligned yet
 - PRMI aligned

Current output beam situation:
 IPPOS - Coming out clear but off in yaw. Not on QPD.
 IPANG - Coming out but too high in pitch and clipped half of the beam. Not on QPD.
 TRY   - On PD/camera.
 POY   - On PD.
 TRX   - On PD/camera.
 POX   - On PD.
 REFL  - Coming out clear, on camera (centered without touching steering mirrors).
 AS    - Coming out clear, on camera (centered without touching steering mirrors).
 POP   - Coming out clear, on camera (upper left on camera).

Oplev values:
Optic    Pre-pump(pit/yaw)    PRFPMI aligned(pit/yaw)
ITMX    -0.26 /  0.60         0.25 /  0.95
ITMY    -0.12 /  0.08         0.50 /  0.39
ETMX    -0.03 / -0.02        -0.47 /  0.19
ETMY     0.37 / -0.62        -0.08 /  0.80
BS      -0.01 / -0.18        -1    /  1 (almost off)
PRM     -0.34 /  0.03        -1    /  1 (almost off)

 All values +/- ~0.01. So, oplevs are not useful for alignment reference.

OSEM values:
Optic    Pre-pump(pit/yaw)    PRFPMI aligned(pit/yaw)
ITMX    -1660 / -1680        -1650 / -1680
ITMY    -1110 /   490        -1070 /   440
ETMX     -330 / -5380         -380 / -5420
ETMY    -1890 /   490        -1850 /   430
BS        370 /   840          360 /   800
PRM      -220 /  -110         -310 /  -110

  All values +/- ~10.
  We checked that if there's ~1200 difference, we still see flash in Watec TR camera. So, OSEM values are quite good reference for optic alignment.

IPANG drift:
  On Saturday, when Rana, Manasa, and I are trying to get Y arm flash, we noticed IPANG was drifting quite a lot in pitch. No calibration is done yet, but it went off the IPANG QPD within ~1 hour (attached).
  When I was aligning Yarm and Xarm at the same time, TRY drifted within ~1 hour. I had to tweak TT1/TT2 mainly in yaw to keep TRY. I also had to keep Yarm alignment to Y green. I'm not sure what is drifting so much. Suspects are TT2, PR2/PR3, Y arm and Y green.

  I made a simple script(/opt/rtcds/caltech/c1/scripts/Alignment/ipkeeper) for keeping input pointing by centering the beam on IPPOS/IPANG using TT1/TT2. I used this for keeping input pointing while scanning Y arm alignment to search for Y arm flash this weekend (/opt/rtcds/caltech/c1/scripts/Alignment/scanArmAlignment.py). But now we have clipped IPANG.


So, what's useful for alignment after pumping?:
  Optic alignment can be close by restoring OSEM values. For input pointing, IPPOS/IPANG are not so useful. Centering the beam on REFL/AS (POP) camera is a good start. But green works better.

We want to measure the g-factor of the PRC using the beat note of the main laser with an auxiliary NPRO laser.

We are going to phase lock the NPRO to the main laser (taking it from POY) and then we will inject the NPRO  through the AS edge of the ITMY.

Today Sendhil and I installed the auxiliary laser on the ITMY table moving it from the AS table.

We also installed the beam steering optics, except the BS which will launch the beam through the AR edge of the ITMY.

To do: install the BS, take the POY beam and mix it with the auxiliary laser on a photodiode to phase lock the two lasers, do better calculations for the mode matching optics to be used for the auxiliary laser beam.

[Jenne, Manasa, Yuta]

We temporarily centered the beam on IPANG to see input pointing drift. From eyeball, drift was ~ 0.1 mrad/h in pitch.

What we did:
  1. Aligned TT1/TT2 and aligned input pointing to Yarm.

  2. Tweaked TT2 in pitch to center the beam on the first steering mirror of IPANG path. We still saw Yarm flash in higher order modes at this point. Before tweaking, the beam was hitting at the top edge.

  3. Centered the beam on IPANG QPD.

  4. Moved IPPOS first steering mirror because IPPOS beam was not on the mirror (off in yaw, on mirror edge). Also, IPPOS beam was coming out clipped in yaw.

  5. Centered the beam on IPPOS QPD. We put lens in the path to focus the beam on the QPD.

  6. Left input pointing untouched for 4 hours.

  7. Restored TT2 again. We tried to align Y arm with IPANG available, but it was not possible without touching TRY path and AS was also clipped.

Result:
  Below is the trend of IPANG sum, X, and Y. IPANG Y (IBQPD_Y) drifted by ~0.8 counts in 4 hours. IPANG is not calibrated yet, but Jenne used her eyeball to measure beam position shift on IPANG steering mirror. It shifted by ~2 mm. This means, input pointing drifts ~0.1 mrad/h in pitch.


Discussion:
  Compared with yaw, pitch drift is quite large considering beam size at ETMY(~5 mm). We can monitor input pointing drift in weekends get longer trend.

Note:
  - IPANG and IPPOS are both changed from the state before pumping.

I'm working on getting the input beam centered on the Yarm optics.  To do this, I measured the spot positions, move the tip tilts, realign the cavity, then measure the new spot positions.  While doing this, I am also moving the BS and Xarm optics to keep the Xarm aligned, so that I don't have to do hard beam-finding later.

Here is the plot of spot measurements today.  The last measurement was taken with no moving, or realigning, just several hours later after speaking with our Indian visitors.  I'm closer than I was, but there is more work to do.

Checking the drift in input pointing (TT2 is the main suspect)

I have centered IPPOS and the 2/3 part of IPANG that comes out of vacuum to the QPDs to see the drift in input pointing over the weekend or atleast overnight.

If anybody would be working with the IFO alignment over the weekend, do so only after recording the drift in IPANG and IPPOS or if you will be working later tonight, center them ion the QPDs before leaving.

I centered ipang and ippos on the QPDs (using only the steering mirrors) and wanted to see the drift over the weekend.

Observation
1. IPANG has drifted (QPD sum changed from -6 to -2.5); but it is still on the QPD.
2. IPPOS does not show any drift.
3. In the plot: The jump in IPANG on the left occured when I centered the beam to the QPD and that on the right is from the 4.7 earthquake and its aftershocks this morning.

Follow-up questions
1. Do we need to worry about this drift?
2. Which of the two TTs is resposible for the drift?
3. Do the TTs tend to drift in the same direction everytime?

P.S. The TTs were not touched to center on IPANG and IPPOS. The last time they were touched was nearly 6 hours before the centering. So the question of any immediate hysteresis is ruled out.

[Jenne, Manasa]

Spot centering on Y arm - DONE!

Alignment procedure
1. I went back to the IFO alignment slider positions from Friday. The Y arm was flashing in HOM because the earthquake this morning tripped all suspensions and the slider values were not real. X arm did not have any flashes.

2. Y arm aligned using TT1 and TT2. Spot centering measured using Jenne's A2L_Yarm script.

Spot positions:
           ITMY    ETMY
Pitch    6.48    4.39
Yaw     -7.42    -3.135

3. I started centering in pitch. I used the same in-vac alignment method (down on TT1 and up on TT2 in pitch) and measured spot positions.

4. When the spot positions were centered in pitch, I started with yaw alignment.

5. I had to use TT1 to center on ITMY and move TT2 and ITMY to center on ETMY.

6. Spot positions after centering:

                          ITMY    ETMY
Pitch    -1.22    -1.277
Yaw       0.42    -0.731

7. I wanted to go back and tweak the pitch cenetering; but framebuilder failed and dataviewer kept loosing connection to fb

Notes
AS seems clipped. Although it could be because of the misaligned BS.

IPANG was centered on the QPD, but it is so clipped, that I'm not sure we can trust it.  Max sum right now is -4, rather than the usual -8 or -9.

Tomorrow:
Once fb is fixed, we should align the X-arm which will be followed by green alignment.

Mystery
Over the last few weeks, it has been observed that there is some strong seismic activity that starts at around 9PM everyday and goes on for a couple of hours. It seems unlikely that it is our geologist neighbour (Jenne met with the grad student who works on the noisy experiment).
 

Steve just told those of us in the control room that the custodian who goes into the IFO room regularly steps on the blue support beams to reach the top of the chambers to clean them.  Since we have seen in the past that stepping on the blue tubes can give the tables a bit of a kick, this could help explain some of the drift, particularly if it was mostly coming from TT2.  The custodian has promised Steve that he won't step on the blue beams anymore.

This doesn't explain any of the ~1 hour timescale drift that we see in the afternoons/evenings, so that's still mysterious.

[Tega, JC]

Tega and I went in to adjust the POP being in the ITMX Table. The beam entered the table high, so we adjusted the this by adding mirrors (The highlighted in Turqoise are mirrors which adjust the pitch of the beam). All the mirrors are set and we are now in the process of adjusting the PD.

[Paco]

Got POP beam centered on camera and nominally on the two PDs. Attachment #1 shows "carrier" camera.

[Yehonathan, JC]

Yehonathan and I attempted to align the LO2 beam today through the BS chamber and ITMX Chamber. We found the LO2 beam was blocked by the POKM1 Mirror. During this attempt, I tapped TT2 with the Laser Card. This caused the mirror to shake and dampen into a new postion. Afterwards, when putting the door back on ITMX, one of the older cables were pulled and the insulation was torn. This caused some major issues and we have been able to regain either of the arms to their original standings.

[Yuta, Anchal, Paco]

As described briefly by JC, there were multiple failure modes going during this work segment.

ITMX SatAmp SAGA

Indeed, the 64 pin crimp cable from the gold sat amp box broke when work around ITMX chamber was ongoing. We found the right 64 pin head replacement around and moved on to fix the connector in-situ. After a first attempt, we suddenly lost all damping on vertex SUS (driven by these old sat amp electronics) because our c1susaux acromag chassis stopped working. After looking around the 1x5 rack electronics we noted that one of the +- 20 VDC Sorensens were at 11.6 VDC, drawing 6.7 A of current (nominally this supply draws over 5 Amps!) so we realized we had not connected the ITMX sat amp correctly, and the DC rail voltage drop busted the acromag power as well, tripping all the other watchdogs ...

We fixed this by first, unplugging the shorted cable from the rack (at which point the supply went back to 20 VDC, 4.7 A) and then carefully redoing the crimp connector. The second attempt was successful and we restored the c1susaux modbusIOC service (i.e. slow controls).

ITMY SatAmp SAGA

As we restored the slow controls, and damped most vertex suspensions, we noticed ITMY UL and SD osems were reading 0 counts both on the slow and fast ADCs. We suspected we had pulled some wires around when busy with the ITMX sat amp saga. We found that Side OSEM cLEMO cable was very loose on the whitening board. In fact, we have had no side osem signal on ITMY for some time. We fixed this. Nevertheless the UL channel remained silent... We then did the following tests:

• Test PD mon outputs on the whitening card. We realized the whitening cards were mislabeled, with ITMX and ITMY flipped . We have labeled them appropriately.
• Tested input DB15 cable with breakout board.
• Went to the ITMY sat amp box and used the satellite box TESTER 2 on J1. It seemed correct.
• We opened the chamber, tested the in-vacuum segments, they all were ok.
• We flipped UR-UL OSEMs and found that the UL OSEM is healthy and works fine on UR channel.
• We tested the in-air cable between satellite box and vacuum flange and it was ok too.
• We suspected that the satellite box tester  is lying, so we replaced the satellite box with the spare old MC1 satellite box, and indeed that solved the issue.

DO NOT TRUST THE SATELLITE BOX TESTER 2.

Current state:

• IMC locking normally.
• All suspensions are damping properly.
• Oplevs are not centered.
• No flashing on either of the arms. We had no luck in ~20 min of attempt with just input injection changed.
• On kicking PR3, we do see some flashing on XARM, which means XARM cavity atleast is somewhat aligned.
• All remaining tasks before pumpdown are still remaining. We just lost the whole day.

There was some uncertainty as to which channels were being input into the Adaptive Filtering screen, so I checked it out to confirm.  As expected, the rows on the ASS_TOP_PEM screen directly correspond to the BNC inputs on the PEM_ADCU board in the 1Y6 (I think it's 6...) rack.  So C1:ASS-TOP_PEM_1_INMON corresponds to the first BNC (#1) on the ADCU, etc. 

After checking this out, I put text tags next to all the inputs on the ASS_TOP_PEM screen for all of the seismometers (which had not been there previously).  Now it's nice and easy to select which witness channels you want to use for the adaptation.

When Sanjit and I were looking at the adaptive filtering system on Monday and Friday, we noticed that turning on the Accelerometers (which had been used in the past) seemed to do good things, but that turning on the seismometers (which I just put into the system last week) made the OAF output integrate up.  Rana pointed out that this is an indication of a missing high pass filter.  And indeed, when I put the seismometers in, I neglected to copy the high pass filter at low frequencies, and the low pass at 64Hz from the accelerometer path to the seismometer path.  The accelerometers had a HP at 1Hz, which is okay since they don't really do useful things down to the mHz level.  I gave all of the seismometers HP at 1mHz.  These are now in the filter banks in the ASS_TOP_PEM screen.  The accelerometers are on channels 15, 16, 17, 18, 19, 20 and the seismometers are on channels 2, 3, 4, 10, 11, 12, 24.

I now need to modify the upass script to turn these filters on before doing adaptive filtering.

[Jenne, Sanjit]

It seems now that we are able to get the OAF system to do a pretty good job of approximating the MC_L signal, but we can't get it to actually do any subtracting.  I think that we're not correctly setting the phase delay between the witness and the MC_L channels or something (I'm not sure though why we get a good filter match if the delay is set incorrectly, but we do get a good filter match for very different delay settings: 1, 5, 100, 1000 all seem to do equally well at adjusting the filter to match MC_L). 

The Matt Evans document in elog 395 suggests measuring the phase at the Nyquist frequency, and calculating the appropriate delay from that.  The sticking point with this is that we can't get test points for any channel which starts with C1:ASS.  I've emailed Alex to see what he can do about this.  Elog 1982 has a few words about how we're perhaps using a different awgtpman on the ass machine than we used to, which may be part of the problem. 

The golden plan, which in my head will work perfectly, is as follows: Alex will fix the testpoint problem, then Sanjit and I will be able to measure the phase between our OAF signal and the incoming MC_L signal, we will be able to match them as prescribed in the Matt Evans document, and then suddenly the Adaptive Filtering system will do some actual subtracting!

The plot below shows the Reference MC_L without any OAF system (black), the output of the OAF (green), and the 'reduced' MC_L (red).  As you can see, the green trace is doing a pretty good job of matching the black one, but the red trace isn't getting reduced at all.

[Rana, Jenne]

The OAF system did something useful today!  Attached is a plot.  Black is the reference (13 averages) with the OAF off.  Blue is the output of the OAF, and red is the reduced MC_L signal (13 averages).  If you turn tau and mu both to 0, it "pauses" the filter, but keeps the feedforward system working, so that you can take a long average to get a better idea of how well things are working. If you ramp down the output of the CORR filter bank, that lets you take a long average with the OAF "off", but doesn't mess up your nicely adapted filter.  The cyan and gold traces in the upper plot are 2 of the Guralp channels, so you can see the real seismic motion.

In the lower plot, you can see that the cyan and light green seismic channels have good coherence with IOO-MC_L (the names don't really mean anything right now...these 2 seismometer channels are the 2 Guralps' channels, one per end of the MC, which are aligned with the MC.)  The dark blue trace is the coherence between IOO-MC_L and the output of the OAF.

500 taps, delay=5, 2 Guralp channels (the ones aligned with the MC), tau~0.00001 (probably), and mu~0.01 or 0.005

[Sanjit, Jenne]

The up and down scripts accessible from the OAF (still C1:ASS-TOP) screen are now totally functional and awesome.  They are under the blue ! button.  The up script can either be for the Seismometers, or the Accelerometers at this time.  The only difference between these 2 is which burt restore file they look at:  the seismometer version puts all 7 seismometer channels in the PEM selecting matrix, while the accelerometer version puts the 6 accelerometer channels in that matrix.  Both scripts also turn on HP_1mHz filters in the ERR_EMPH filter bank and all of the witness filter banks, and the AA32 and AI32 filters in ERR_EMPH, CORR and PEM filter banks.  This makes all of the starting filters the same between the witness paths and the error path.

If you want to use a different combination of sensors, run one of the up scripts, then change the PEM matrix by hand. 

The down script disables the output to the optics, and resets the adapted filter coefficients.  DO NOT use this script if you're trying to "pause" the filter to take some nice long averages.

[Jenne, Sanjit]

As per Matt's instructions in his OAF document (elog 395) in the Tuning section, Sanjit and I took a transfer function measurement from the output of the OAF system, to the input.  i.e. we're trying to measure what happens out in the real world when we push on MC1, and how that is fed back to the input of our filter as MC_L.  The game plan is to measure this transfer function, and read off the phase at the nyquist frequency, and use this value to calculate the appropriate sample-and-hold delay to be used in the OAF.  The downsample rate for the OAF is 32, so that we're running at 64Hz instead of the 2048Hz of the front-end.  Thus, our Nyquist frequency is 32Hz.

                            DownSampleRate

Phase@Nyquist * ------------------------

                                    180

In the attached figure we do a swept sine from CORR_EXC to ERR_EMPH_OUT to determine the transfer function.  Here, we turn off all of the filters in both the CORR and EXC banks, because those are already matched/taken into account in the PEM filter banks. 

Using the cursor on DTT, we find that the phase at 29.85Hz is -228.8deg, and at 37.06Hz is -246.0deg.  Extrapolating, this means that at 32Hz, we expect about -234deg phase.  Using our handy-dandy formula, this means that we should try a delay of 41 or 42 (41.6 is between these two...) 

We'll give this a shot!

 As Rana pointed out to me last night, I was using continuous phase, which is not good.  When using regular phase, I find: (29.85Hz, 131.216deg), (37.06Hz, 113.963deg), so extrapolating gives (32Hz, 126.07deg).  Plugging this in to our handy-dandy formula, we get a delay of 22.4, so we should try both 22 and 23.

 Here's a plot of the spectra of the seismometers and MCL. The coherence shows which axes are aligned right now: MC1_X is coherent with GUR_NS which means that its mis-oriented.

I've now swapped the "MC1" cables: so the old "NS" now goes into EW and the old EW now goes into NS. VERT is unchanged.

Also fixed the channel names - the Guralp previously named MC1 is now GUR1 and the other one is GUR2. Also no more EW, NS, & VERT. Its all XYZ.

DAQD restarted with the new channel names.

I remeasured the OAF time delay using the OAF-TF template from the Templates/ directory.

Troublingly, I found the MC1 dewhitening switches set OFF - please make sure that the MC1 dewhitening is back ON after each OAF tuning so that the interferometer locking is not hosed.

The OAF-TF template had the excitation amplitude set ~20x too high. I reduced it and the coherence was still > 0.95. The phase at 32 Hz was still ~126 deg as Jenne had measured, but since the phase at DC is 180 deg, the overall phase lag is just 180-126 = 54 deg. So the delay should be 54/180 * 32 = 9.7 => 10. Luckily, Jenne is working on an instructional manual for OAF that will make all of this crystal clear.

 I spiffed up the order of the cables / sensors plugged into the PEM ADCU.  Now all of the seismometers are labeled as Rana left them, and the 2 Guralp's have their sets of 3 channels next to eachother in channel-number-land.  None of the accelerometer names/cabling have changed recently.  In the table, Cable-label refers to the physical tag tied to the end of the cables plugged into the ADCU...they are meant to be descriptive of what seismometer channels they are hooked up to, and then the names change to something useful for us when they come into the DAQ system.  Also, the labels of input channels on the ASS_TOP_PEM screen have been updated accordingly.

[Rana, Jenne]

There is some craziness going on with the delay in the PEM path for the OAF.  We plot the difference between the C1:PEM-SEIS_GUR1_X and C1:ASS-TOP_PEM_10.  These are physically the same channel, plugged into the PEM ADCU, and then the signal is used as a regular PEM channel, and is also sent to the ASS computer and used there for the OAF system.  As you can see in the blue trace on the bottom plot, there is a huge amount of delay, and it's very noisy.  We also plot the _GUR2_X / ASS-TOP_PEM_2 pair (red), and it has a similar amount of delay, but it is not nearly as fuzzy and noisy.  For comparison, we plot the SUS-MC2_MCL (which is identical to IOO-MC_L) and ASS-TOP_ERR_MCL pair (green), and they don't have any big overall delay problems, so it's not totally a problem with the signals getting to the ASS computer.

This problem was present during/after all of the following attempts to fix it:

* The sample rate on the ASS computer is 2048.  The PEM channels were being acquired the ADCU at 512.  We changed the ADCU sampling rate to 2048 to match.

* We soft rebooted the ASS computer, in case it was a timing problem.

* Doing a "sudo shutdown -r now" while logged in as controls.

We might also try resetting/power cycling c0dcu in the morning.  Alex has been emailed to help us try to figure this out.

In other news, the time delay that we measure from the plot gives us 180degrees in ~210Hz.  This corresponds to a little more than 2msec of delay, with the C1:ASS version lagging behind the C1:PEM version.  (2 samples at 840Hz) Converting to the 2048 sampling rate, we have a delay of 4.8, so 5 front-end cycles.  Since Rana measured this morning that the delay indicated by the transfer function is 10 cycles, and this delay shows that the ASS lags the actual seismometer signal by 5 cycles, we should subtract this 5 from the 10 from the transfer function, giving us a final sample-and-hold delay of 5.  Coincidentally(?), 5 is the delay that was found in the C1:ASS-TOP screen, after it's one year of dormancy.  The point of the delay feature in the code is to help match the delay in the two signal paths: the PEM path and the output path of the filter.  Since the output has a lag of 10, and the PEM path has a lag of 5, to make them match, we artificially put in a delay of 5.

The c1ass computer, which is now used for the OAF system, has many remnants from the days when it was actually used as an ASS.  These PIT and YAW filter banks and other things were taking up a lot of unnecessary space, so I deleted them in the ass.mdl file.  These files are all backed up, so we can always revert back to an older version when we want some Alignment Stabilization again someday.  I then did a make ass, following the instructions on the 40m Wiki -> Computers and Scripts -> Simulink to Front-End Code page.  Rana moved some things around, most notably all of the things (like the ASS screens) which were only in ...../users/alex/.... are now in ....../caltech/cds/advLigo/..... .  This required a few restarts of the c1ass machine (after a couple different versions of the simulink diagram....one to make sure we knew how to do it, and then again actually deleting the unused portions).

The big lesson of the night was that there are 2 signal paths for the PEM channels.  As is shown in Figure 3 in the mevans document, the PEM channels get the matching filters when they go to the adaptation algorithm, but when they go to the FIR filter, they do not get the matching filters. This is implemented by taking the output of the giant PEM matrix, and having a duplicate of each of the channels "selected for adaptation", one which gets filtered through the PEM_N_ADPT banks, and one which goes straight (in code-land) to the FIR filter.  So, it seems like all the filters which we had been including in the input side of the matrix for matching purposes need to be put in the output side.  One of the AA32 filters needs to stay in the input side, for actual anti imaging of the PEM channels, then we put the AA32 and AI32 which are for matching the ERR_EMPH and CORR filter banks up in the PEM_N_ADAPT banks.  Rana and I made these filters, and they are now turned on appropriately with the OAF down script (so that all the filters are ready and waiting for the OAF to be turned on).

A little success with getting the 3Hz peak reduced, but not a lot beyond that.  Tomorrow I'll put the accelerometers back where they used to be to see if they help out at all.

 Alex came in a week ago Friday to help figure this timing problem out, and some progress was made, although there's more to be done. 

Here are the (meager) notes that I took while he was working:

we can rename the tpchn_C1_new back to tpchn_C1, but the _new one works right now, so why change it?

need to find dcuDma.c source code...this is (?) what sends the PEM channels over to ASS.  Found:  source code is dcu.c, th
en the binary is dcuDma.o  Trying to recompile/remake dcuDma to make everything (maybe) good again.

Possibility: maybe having so many channels written to the RFM takes too long? shouldn't be  a problem, but maybe it is.  I
n the startup.cmd (or similar?) change the number of ISC modules to 1, instead of 2, since we only have one physical board
 to plug BNCs into, even though we have 2 isc boards.  c0dcu1 rebooted fine with the one isc board.  now can't get ass tes
tpoints to try the DTT timing measurement again.  rebooting fb40m to see if that helps.  fb40m is back, but we still don't
 have ASS testpoints.  Alex had to leave suddenly, so maybe more later.

Also, next possibility is that c0dcu and c1ass are not synched together properly....we should look at the timing of the AS
S machine.
 

After these adventures, the noisy trace in the timing delay (in the plot in elog 2066) has become quiet, as shown below (The blue trace, which was noisy in 2066 is now hiding behind the red trace).  However, the overall timing delay problem still exists, and we don't quite understand it.  Alex and I are meeting tomorrow morning at the 40m to try and suss this out.  Our first plan of attack is to look at the ASS code, to see if it puts any weird delays in.

Rana pointed out that the delay may be caused by the 110B DAQ, as it integrates over 2ms (5 clock cycles at 2048Hz on the fe computer), to make low noise measurement. However, the C0DCU knows about this delay and corrects it by fudging the time stamp, before sending it to the frame builder, so that the time stamps match the actual measurement time. But, the ASS computer is not aware of such an integration time, so it does not adjust the time. We verified that it is indeed the case. This is what we did (as suggested by Rana):

We split the signal from the MODE cleaner board "OUT" port using a T-splitter to the original PENTEK board (C1:SUS-MC2-MCL-IN) and the PEM ADCU channel #2. Then measured the mutual delays between the signals that are processed by C0DCU and the ASS computer for both the MC_L signal and the corresponding output through the PEM channel. We clearly see the same delay (compare red and brown in the bottom panel) between the signals that are going through 110B and the PENTEK DAQ. This delay is a bit noisy, possibly because the PENTEK is not as low noise as the 110B is.

There is some delay (pink curve in the bottom panel) between the PENTEK DAQ and the frame builder corrected 110B output, much smaller than 2ms, could be ~200-400 u sec. Which should correspond to the 1 or 1/2 cycle delay caused by the PENTEK DAQ.

So, once we have the planned advLIGO DAQ system, there should not be any long delay. Perhaps, to solve the problem and make OAF functional soon, we will upgrade the PEM DAQ asap, rather than waiting for the rest of the upgrades...

An email from Rolf about the delay in the 110Bs:

"...we do take the ~2msec pipeline delay into account when we send the data to DAQ. If I remember correctly, the delay is about 39 samples. On startup, the first 39 samples are 'thrown away', such that, from then on, data lines up with the correct time (just read 2msec later then Penteks)."

[Alex, Jenne, Sanjit]

Alex came to the 40m today, and did several awesome things in OAF-land.

We discovered that there is, in fact, an ADC board connected to the ASS machine.  The tricky bit is that it only has a ribbon cable connector, so before we can use this ADC, we need to figure out how to make a breakout board/cable/something to connect the seismometer/accelerometer/microphone BNCs to this little board.  This is the same little board that connects the timing slave to the ASS machine.  For good or for ill, the timing slave is connected to this board via clip-doodles.  Potentially we can connect an ADC tester board to this board, and go from seismometer BNCs to clipdoodles to the tester board, but I'm not in love with the idea of utilizing clipdoodles as a semi-permanent solution until the upgrade.  I emailed Ben to see if he has a better idea, or (better yet) some spare hardware now that's the same as we'll use after the upgrade.  If we can use this ADC, it may solve our timing problem which is caused by the 110B ADC used by the PEM computer. Alex showed Sanjit and I how to connect the ASS's ADC card to the simulink diagram, when we're ready for that.

We also poked around in the code, and it seems that we can now save and restore OAF coefficients at will.  I added buttons to the OAF (ASS) screen, and Alex made it so the OAF coefficients are saved in RFM shared memory whenever you click the "save coeffs" button, and are restored when you click the "restore coeffs" button.  The buttons are the same as the 'Reset' button which has been there for a long time, so they seem to maybe have a similar problem in that you have to hold the button for a while in order for the code to realize that the button has been depressed.  We couldn't fix this easily, because it looks like our SimuLink cds stuff is a little out of date.  Some day (before/when Joe and Peter make new screens for the new 40m), we need to update these things.  Alex was concerned that it might take a while to do this, if the update broke some of the blocks that we're currently using.  Also, Sanjit and I now need to check that the coefficient-saving is going as planned.  When I have DTT open, and the OAF running, I see a certain shape to the signal which is sent to MC1 to correct for the seismic motion.  This shape includes at least several peaks at resonant frequencies that exist in our stacks/suspensions.  I can then save the coefficients, reset the active filter, and then restore the coefficients.  When I do this while watching DTT, it seems as though the general shape of the filter is restored, but none of the detailed features are.  The reason for this is still under investigation. 

The code-modifications involved a few iterations of 'remaking the ass'.

[Sanjit,Jenne]

Sanjit has been working today on trying to get the OAF coefficients to save properly.  Alex got us most of the way, but right now it's looking like the filter that is being saved is totally constant (all the values are the same).  We're poking around trying to figure out why this is. 

Also, we're starting again (as we should have been for the last week or so since Alex came in to help us) to check in the TOP_XFCODE whenever we make changes to it, and when we recompile the front end code. 

We are manually restarting assepics, but the terminal logs us out after sometime and ass may crash. I set autologout=0 in the terminal for the time being. Once the testing process is over, assepics will start automatically when the computer is turned on, so we wont have to worry about this.

(if ass crashes tonight, it is not unexpected!)

I made some changes in the code (all commented in the installed and SVN version) to print the filter coefficients. I got crazy output. Sometimes memory bugs lead to such crazy behavior. So far I could not find any bugs, but will have to spend more time on it.