We will start preparing for pumping down. Main goal for this is to demonstrate PRFPMI using ALS.
Here are to-dos before we pump down.

Feb 18 eveing
- check input beam and Y arm alignment again
- IPPOS/IPANG alignment
- check all oplevs

Feb 19 morning
- open ETMX chamber heavy door
- align BS to X end
- adjust OSEM values (added by YM)
- center beam on all AS optics
- make sure AS/REFL is clear
- take picture of flipped PR2 (added by YM)
- make sure green is not clipped by new PRM oplev mirrors  (added by YM)
- center all oplevs

Feb 19 afternoon - Feb 20 morning
- close PSL shutter
- close all heavy doors and put the access connector back
- start pumping down

Feb 20 evening
- start aligning IFO

I undertook the investigation of the AS55 PD. I found the PD is not broken.

I tested the PD on the PD test bench and it works just fine.

I attatched the characterization result as there has been no detailed investigation of this PD as far as I remember.

The transimpedance gain at 55MHz is 420Ohm, and the shotnoise intercept current is 4.3mA.

[Yuta, Sendhil]

We aligned IPPOS, IPANG and all OPLEVs (except for ETMX and SRM).

1. First aligned the IPPOS by tweeking the steering mirrors inside the BS chamber.

2. Aligned the IPANG by tweeking the steering mirrors inside the BS chamber and ETMY chamber.

3. Aligned the OPLEVS for the BS and PRM was done by tweeking the steering mirrors inside the BS chamber and checked that OPLEV beams were not clipped. 

4. Centred the OPLEV beams for the ITMY and ETMY.

5. For the OPLEV of ITMX the alignment was done by tweeking the steering mirrors inside the ITMX chamber.

Hmm......
I thought AS55 is broken because it was not responding to the AS beam nor flashlight in DC. What's the DC gain difference between AS55 and POP55 (or REFL55)?

These are things need to be done for demonstrating PRFPMI using ALS.
All of these should be done before March 8!

CDS:
    - Fix c1iscex -JAMIE (done Feb 19: elog #8109)
    - Is ASS and A2L working? -JENNE
    - Are all whitening filters for PDs toggling correctly? -JENNE, JAMIE

PRMI locking:
    - Adjust I/Q rotation angles for error signals -JENNE, YUTA
    - Adjust filters -JENNE, YUTA
    - Coil balancing for BS (and ITMs/ETMs) -YUTA

PRC characterization in PRMI:
    - Measure PR2 loss from flipping -MANASA
    - Measure mode matching ratio -JENNE, YUTA
    - Measure finesse, PR gain -JENNE, YUTA
    - Calibrate PRM and/or ITM oplevs -MANASA, YUTA
    - Measure g-factor by tilting PRM or ITMs -JAMIE, YUTA
    - Calculate expected mode matching ratio and g-factor -JAMIE
    - Calculate expected finesse, PR gain -JENNE
    - Align aux laser into AS port? -ANNALISA?

ALS:
    - What's the end green situation? Optical layout changed? Laser temperature in CDS? -MANASA
    - What's the PSL green situation? Green trans cameras/PD? -JENNE, MANASA
    - Make ALS handing off to DARM/CARM LSC script -JENNE, YUTA
    - Demonstrate FPMI using ALS -JENNE, YUTA
    - Phase tracker characterization -YUTA, KOJI

PRFPMI:
    - Measure mode matching between PRC and arms -JENNE, YUTA
    - Measure PR gain -JENNE, YUTA
    - Calculate expected finesse, PR gain -JENNE

Others:
    - Update optical layout CAD after PR2 flipping -JAMIE, MANASA
    - AS55 situation? -YUTA
    - Look into PMC drift -JENNE, MANASA
    - Measure RFAM contribution to error signals -YUTA

Please fix, add or update if you notice anything.

10010 Ohm for POP55 vs 50 Ohm for AS55 (cf. http://nodus.ligo.caltech.edu:8080/40m/4763)

I wonder if you used an LED flash light, which emits no IR.

I didn't use LED flash light. We learned from the past (elog #7355). I checked that POP55 and REFL55/165/33/11 are clearly responding to flash flight, but I didn't expect that much difference in DC gain.
I wonder why we could align AS beam to AS55 in Feb 8 (elog #8030), but not in Feb 15 (elog #8091). I will check during the pump down.

[Steve, Yuta, Koji]

The ETMX heavy door was removed.

    See TT DB25 pin swapping   elog#7869 

In order to address the issue of low MC1 OSEM voltages, Yuta and I looked at the IMC table levelling.  Looking with the bubble level, Yuta confirmed that the table was indeed out of level in the direction that would cause MC1 to move closer to it's cage, and therefore lower it's OSEM voltages.  Looking at the trends, it looks like the table was not well levelled after TT1 installation.  We should have been more careful, and we should have looked at the MC1/3 voltages after levelling.

Yuta moved weights around on the table to recover level with the bubble level.  Unfortunately this did not bring us back to good MC1 voltages.  We speculate that the table was maybe not perfectly level to begin with.  We decided to try to recover the MC1 OSEM voltages, rather than go solely with the bubble level, since we believe that the MC suspensions should be a good reference.  Yuta then moved weights around until we got the MC1/3 voltages back into an acceptable range.  The voltages are still not perfect, but I believe that they're acceptable.

The result is that, according to the bubble level, the IMC table is low towards MC2.  We are measuring spot positions now.  If the spot positions look ok, then I think we can live with this amount of skew.  Otherwise, we'll have to physically adjust the MC1 OSEMS.

c1iscex is back up.  It is communicating with it's IO chassis, and all of it's models (c1x01, c1scx, c1spx) are running again.

The problem was that the IO chassis had no connection to the computer.  The One Stop card in the IO chassis, which is the PCIe bridge from the front-end machine and the IO chassis, was showing four red lights instead of the dozen or so green lights that it usually shows.  Upon closer inspection, the card appeared to be complaining that it had no connection to the host card in the front-end machine.  Un-illuminated lights on the host card seemed to be pointing to the same thing.

There are two connector slots on the expansion card, presumably for a daisy chain situation.  Looking at other IO chassis in the lab I determined that the cable from the front-end machine was plugged into the wrong slot in the One Stop card.  wtf.

Did someone unplug the cable connecting c1iscex to it's IO chassis, and then replug it in in the wrong slot?  A human must have done this.

After spending a good deal of time learning how to use the SR785, I was able to characterize my prototype circuit. The transfer function from a swept sine measurement looks very similar to the theoretically calculated transfer function (both of which are attached). The frequency response of the circuit was considered over the range 10 Hz - 10 kHz, which contains the eventual working range of the ISS (at least to my knowledge).

Note that OP27 op-amps were used instead of the high-speed AD829 op-amps that will be implemented in the actual design. This was done as a result of the limitations and inherent noise characteristics of the breadboard on which the prototype was built.

Unfortunately, I saved the wrong dataset (i.e. phase of the transfer function, not magnitude) and thus the presented function here is image generated by the SR785.

RXA: One must learn to use the python-GPIB interface to not lose data in the future.

I finished working out the circuit and figured out where the broken connections were. This is diagrammed in my notebook (will draw up more nicely and include in future elog post). Within the QPD circuitry, it seems like there are already opamps which regulate the circuit. I need to discuss the final diagram further with Eric.

I rewired the circuit inside the QPD box, which took awhile because it was difficult to solder the wires to such small locations without having multiple wires touch. This is completed, and on Friday I will begin to make the circuit to add/subtract signals to give pitch and yaw.

[Jenne, Manasa, Yuta]

Since we levelled IMC stack, we had to center beam spots on MC mirrors again.
We did this by steering PSL mirror in yaw (about same amount but opposite direction to what we did in elog #8077)
Residual beam tilt compared with a line through MC1 and 3 actuator nodes is ~ 15 mrad, mainly in yaw.

[Yuta, Sendhil, Jamie, Jenne, Rana]

1. After the MC centering, we tried to align the IFO using IPPOS and IPANG as reference. This did not recover the alignment perfectly. We were clipping at the BS aperture. Using TTs, we centered the beam at BS and PRM.
2. Using TTs, the beam was centered at ITMY and ETMY.
3. IPPOS and IPANG mirrors in-vacuum were aligned and were centered at the out-of-vacuum optics.
4. We checked the centering of the beam on optics in the BS and ITMY chamber. (Yuta will make an elog with the layout)
5. We retro-reflected ITMY at the BS and aligned ETMY such that we saw a couple of bounces in the arm cavity.
6. Using BS, the beam was steered to go through the center of ITMX and ETMX.
7. At this point we were able to see the MI fringes at the AS port.
8. We made fine alignments to the ITMX such that we saw MI reflected at the Farday.
9. We retro-reflected ITMX and aligned ETMX to see the beam bounce at the ITMX.
10. We aligned PRM such that PRC flashes. But we were not happy with the flashes (they were in higher order modes). We suspect that minor tuning of the input pointing might be necessary.
11. We closed for the day

I attached clipping/centering checklist for the alignment.
Blue ones are the ones we checked today. Red ones should be checked tomorrow. Circles indicate centering on the optics, rectangles indicate clipping check, and arrows indicate retro-reflecting or bounces.
We found mis-centering on MMT1, PR2 and SR3 tonight (by ~0.5 beam diameter). They are also indicated.

I think we don't want to touch MMT1 and PR2 anymore, because they change input beam pointing.
I'm a little bit concerned about high beam on SR3, because we had PRC flashing in vertical higher order modes. We also see ETMX slider values high in pitch (~ 5.4).
Also, the diameter of ETMX reflected beam on ITMX looked larger and dimmer than ITMX transmitted beam, which doesn't seem reasonable.


Wednesday, Feb 20:
 - tweak TT1/TT2 and PRM so PRC flashes
 - re-check Yarm/Xarm bounces
 - center beam on all AS optics, starting from SR2
 - make sure REFL and AS is clear
 - check if TRY/TRX are coming out from the ends
 - check beam centering on mirrors in IMC/OMC chamber as far as you can reach
 - inject green from both ends
 - make sure green beams are not clipped by mirrors on BS chamber, IMC/OMC chamber
 - re-center all oplevs, with no clipping
 - check all OSEM values
 - take pictures of flipped PR2 and input TTs (and everything)
 - close all heavy doors and put the access connector back

Thursday, Feb 21:
 - make sure we can lock PRMI
 - start pumping down when Steve arrives

After in-vac alignment work last night, PRC is flashing brighter than PRMI alignment last week.
My hypothesis is that "we aligned PRM to junk MI fringe last week". Possibly, we used MI fringe caused by AR reflection of ITMs, or MI fringe reflected from SRM.

Videos:
  PRC flashing last week (youtube, elog #8085, elog #8091)

  PRC flashing this time (Lens in-front of AS camera was taken out)



My hypothesis can explain:
 - why we had dimmer POP last week (flash in half-PRC was way brighter even when we had more attenuators (youtube))
 - why I thought AS55 is broken (AS was too dim)

Conclusion:
  Be careful of junk beams.

I have updated the vacuum checklist for in vacuum alignment.  Please take a look (https://wiki-40m.ligo.caltech.edu/vent/checklist) and see if I missed anything.  My goal was to make it incredibly step-by-step so there can be no mistakes.

[Yuta, Manasa, Sendhil, Jenne, Steve, Jamie, Koji, Evan]

The interferometer is well-aligned, and ready for pump-down.  The access connector is in place, as are the ETM heavy doors.  We will do ITM and BS doors tomorrow, then begin pumping.

Before we redid the ITM pointing, I confirmed that I could see both POX and POY on their respective tables, on a camera, unclipped.  I should check again quickly now that the ITM pointing has been finalized.

We went back to the arms, to perfect the ITM pointing.  Input beam was already centered at ETMY.  ETMY was pointing so that beam reflected to ITMY.  ITMY was adjusted a few (less than 4?) steps of 1e-3 size, to make reflected beam hit center of ETMY. 

BS was already pointing so beam hit center of ETMX.  ETMX was pointing to hit center of ITMX.  ITMX was adjusted a few (less than 4 again?) steps of 1e-3 size to make reflected beam hit center of ETMX.

Checked centering on AS path.  AS beam comes out of the vacuum a little low, but this wasn't discovered until after the access connector was in place.  We could adjust PZT3 (last AS mirror on BS table that sends beam over to OMC table), but we don't want to do this since we won't be able to re-confirm centering on the 3 mirrors on the OMC table.

Green beams (first Y, then X) were aligned using out-of-vac steering mirrors until beams were flashing in their respective arm cavities.  Green Y is a little close to the edge of the bottom periscope mirror, on the "up" periscope.  Since there is no steering between the arm and this periscope, fixing it would require moving the periscope.  We leave this to the next vent, when we finally install the BS table extension.  We were flashing a higher order yaw mode (5ish nodes) for the Y arm, and the very edge of the higher order mode on one side was a little bit clipped after reflecting off the steering mirror on the OMC table.  This is happening because that mirror is in the mount backwards (so we have access to the knobs).  We are confident that the straight-through beam is well centered on that mirror, so once we get it aligned to TEM00, there will be no clipping. We then did the X arm green, which was flashing a pitch higher order mode (again 5ish nodes).  The very edge of the higher order mode is clipping a little bit on the top mirror of the "down" periscope on the IMC table, but again the straight through beam is okay, and we're confident that the TEM00 mode will make it unclipped.  We could have touched some steering mirrors on the BS table, but since this was once upon a time well aligned, we don't want to futz with it.

Corner oplevs are all centered on their QPDs.  (The ETM oplevs were centered a few days ago).

Access connector and ETM doors are on.

The last 3 vertex doors will go on tomorrow when Steve gets in, and then we'll start pumping. 

There are no in-vac PZTs that need to be turned on (we've been using the output steering PZTs as non-energized fixed mirrors for some time), so we can lock at our leisure tomorrow afternoon.

[Manasa, Evan]

Manasa and I are trying to get the AS beam onto the AS camera with a focusing lens. Currently, the mirror immediately preceding the camera has been removed and the camera and lens are sitting directly behind the BS.

[Sendhil, Yuta]

After aligning IFO and putting the access connector on, we also centered IPANG/IPPOS and all oplevs (except SRM).
To avoid clipping of PRM/BS oplevs, we re-arranged oplev steering mirrors on BS table.

What we did:
  1. Checked IPANG comes out unclipped after putting on the access connector.
  2. Centered IPANG on its QPD.
  3. Checked oplevs beams for ITMX/ITMY centered on in-vac mirrors, and centered them on their QPDs.
  4. Checked IPPOS beam is centered on the mirrors inside BS chamber, and centered IPPOS on its QPD.
  5. Tweaked oplev mirrors on BS chamber to make PRM/BS oplev beam unclipped and centered on mirrors, and centered them on their QPDs. To avoid clipping of oplev beams in BS table, we re-arranged oplev steering mirrors on BS table (outside the vaccum).


Current status:
  QPD values, IFO_ALIGN/MC_ALIGN screens, OSEM values attached.

  - IR incident beam and IFO aligned
  - X/Y end green coming out to PSL table (in higher order modes)
  - IPANG/IPPOS available
  - All oplevs available
  - AS/REFL/POP cameras ready
  - access connector, ETMX/ETMY heavy doors on
  - ITMX/ITMX/BS heavy doors are not on
  - AS/REFL/POP PDs not centered
  - POX/POY/TRX/TRY not aligned
  - AS beam coming out of the OMC chamber low by ~ 1 beam diameter (my bad)


Tomorrow:
  - Align AS/REFL/POP PD and lock PRMI
  - Take pictures of ITMX/ITMY/BS stacks
  - Put heavy doors on ITMX/ITMY/BS chambers
  - Start pumping down

Please confirm the SRM OL beam is not too bad and also find where the mis-aligned SRM puts its beam. WE want to be sure that there is not too much unwanted scattering from SRM into the PRFPMI.

Currently, SRM is misaligned in pitch so that SRM reflected beam hits on the top edge of SR3 (not on the mirror, but on the cage holding the mirror).
We also confirmed that SRM oplev beam is coming out from the chamber unclipped, and centered on QPD when SRM is "aligned".

Blue ones are the ones we checked yesterday.
Green ones are the ones we checked today.
Red ones are the ones we couldn't check.

We noticed mis-centering on green optics and partial clipping of higher order modes, but we did not touch any green optics in-vac. This is because green beam from Y end and X end has different spot positions on the green optics after periscopes. We confirmed that direct green beam from ends are not clipped.

I believe we have checked everything important. Any other concerns?

 Is the beam going towards the OMC going to cause backscatter because of uncontrolled OMC or can we park that beam somewhere dark?

I'm not sure about the OMC situation at 40m. I think there are no direct beam reflected back into IFO from OMC path. There must be some backscatter, but we have to open OMC chamber again to put a beam dump.
I don't think we want to put one in OMC path for this pump-down, but we can put a beam dump to dump reflected beam from mis-aligned SRM tomorrow, if available.

I have found two great FET input chips that rival the storied, discontinued AD743. In some ways, they are even better. These parts are the OPA140 and the OPA827.

Below is a plot of the input-referred voltage noise of the two op amps with R_source = 0, along with several others for comparison. The smooth traces are LISO models. The LT1128 and AD797 are BJT-input parts, so their voltage noise is naturally better. However, the performance you see here for the FET parts is the same you would expect for very large source impedances, due to their extremely low current noise by comparison. I have included the BJTs so that you can see what their performance is like in an absolute sense. I have also included a "measured" trace of the LT1128, since in practice their low-frequency noise can be quite higher than the spec (see, for example, Rana's evaluation of the Busby Box). The ADA4627 is another part I was looking into before, the LT1012 is a less-than-great FET chip, and the AD797 a less-than-great BJT.

As you can see, the OPA140 actually outperforms the AD743 at low frequencies, though it is ~2x worse at high frequencies. The OPA827 comes close to the AD743 at high frequencies, but is a bit worse at low ones. Both the OPA140 and OPA827 have the same low-frequency RMS spec, so I was hoping it would be a better all-around part, but, unfortunately, it seems not to be.

The TI chips also have a few more things on the AD743:

• Input current noise @ 1kHz
  • AD743: 6.9 fA/rtHz
  • OPA827: 2.2 fA/rtHz
  • OPA140: 0.8 fA/rtHz (!)
• Input bias (offset) current, typ
  • AD743: 30 pA (40 pA) --- only for V_supply = ±5 V
  • OPA827: ±3 pA (±3 pA) --- up to ±18V
  • OPA140: ±0.5 pA (±0.5 pA) (!) --- up to ±18V
• Supply
  • Both OPA140 and OPA827 can be fed single supplies up to 36V absolute maximum
  • The OPA140 is a rail-to-rail op amp

These characteristics make both parts exceptionally well suited for very-high source impedance applications, such as very-low-frequency AC-coupling preamplifiers or ultra-low-noise current sources.

(Apologies---the SR785 I was using had some annoying non-stationary peaks coming in. I verified that they did not affect the broadband floor).

R.I.P., AD743

c1iscex is dead again.  Red lights, no "breathing" on the FE status screen.

 Yuta retrieved the IR card that had fallen to the bottom of the IOO chamber, just before we put on the access connector yesterday.  The clean "pickle picker" long grabber tool worked wonders.

The c1iscex machine itself wasn't dead, the models were just not running.  Here are the last messages in dmesg:

[130432.926002] c1spx: ADC TIMEOUT 0 7060 20 7124
[130432.926002] c1scx: ADC TIMEOUT 0 7060 20 7124
[130433.941008] c1x01: timeout 0 1000000 
[130433.941008] c1x01: exiting from fe_code()

I'm guessing maybe the timing signal was lost, so the ADC stopped clocking.   Since the ADC clock is the everything clock, all the "fe" code (ie. models) aborted. Not sure what would have caused it.

I restarted all the models ("rtcds restart all") and everything came up fine. Obviously we should keep our eyes on things, and note if anything strange was happening if this happens again.

[Steve, Manasa, Jenne, Sendhil, Evan, Yuta]

We put heavy doors on ITMX/ITMY/BS chamber and started pumping down from annulus.

What we did:
  1. Replaced POP55 with AS55 back, because it was not broken.
  2. Centered on AS55, REFL55, REFL11, POPDC PD.
  3. Tried to lock PRMI, but I couldn't lock even MI stably for more than 1 min. I believe this is because it was noisy this morning. But I checked again that REFL/POP/AS beams are coming out without clipping and we have some error signals.
  4. Noticed AS beam has less range in left (on AS camera), so we tweaked OM4 a little to make more room.
  6. Took pictures inside ITMX and BS chambers
  7. Put heavy doors on ITMX/ITMY/BS chambers.
  8. Started pumping down annulus.
  9. Recentered IPANG/IPPOS and oplevs on their QPDs.

POP, REFL, AS:

[Steve, Jenne, Yuta, Manasa]

We have kept the laser ON at low power through the pump down process. As we pumped down, at around 400torr, we found that the PSL mech shutter closed. Steve explained  that it was due to an interlock with a pressure gauge. To keep the IFO running, we switched the shutter from N.C (normally close) to N.O (normally open). This should be undone after the pumpdown.

In the process of figuring out, we reset the shutter and switched it ON and OFF a couple of times.

 We are pumping down with RP1 & RP3 oily roughing pumps at 3 Torr/min speed. The butterfly valve was just removed.

The PSL shutter is open to vacuum at ~100  mW 1064 nm. The inter lock should close it at ~5 mTorr.

Rana will shut down pumping tonight.

He will close V3 from screen, close RV1 valve with torque wheel, turn off roughing pumps and disconnect hose between RV1 and roughing pumps.

I will restart pumping early morning tomorrow.

This plot shows the trend of the OL during the past several hours of roughing pumping.

The big steps at the start of the pump down is NOT due to the pumping, but is instead the "recentering" that Yuta did. Looks like he was unable to find zero on the ETMY.

Some of the rest of the drift is probably just the usual diurnal variation, but there does seem to be some relation to the pumping trend. I guess that the shift of ~0.3 in the ITMX and ITMY pitch is real and pressure related.

We need to figure out how to put the OL calibration factor into the SUS-OL screens.

[Yuta, Manasa]

We have aligned the Y-arm to lock in green. The green beams at the PSL table were clipping at the attenuation optics we installed for the vent (HWP-PBS-HWP). We had to move the polarization changing wave plate to get the green beam on the steering mirror. We installed the GRNT camera on the PSL table and aligned the arms to get TEM00 flashing. Green TRX PD was then installed and the trans power was brought to a maximum of 210uW.
We will use this to align the IR to the arm when we are back in full power tomorrow.

[Rana, Yuta, Koji]

21:05 at the pressure of 10torr
V3 closed. RV1 manually closed. RPs were turned off. And the bellows between RV1 and the rughing pumps are disconnected.

 Pump down is restarted after 10.5 hrs stop overnight.

 IFO   P1 pressure is 1 mTorr.  It is ready for high power light.

The Maglev took over the pumping at 500 mTorr

[Yuta, Manasa]

We turned IFO power back to 1.25W by removing the attenuator and forgot that the Y1 mirror before the REFL PD must be replaced with BS 10% before getting to full power. The PD is dead  and now we are in the process of fixing it. Forgive us for all our sins!

On the other note, we have changed the mech shutter mode from N.O back to N.C. So the shutter now works as usual from the medm screen.

 [Yuta, Manasa]

We replaced the dead photodiode on MC REFL PD with a new one (GAP 2000). We measured the frequency response of the PD and tuned the resonant frequency using inductor L5 (in the circuit diagram) to be 29.575MHz - over an average of 10 measurements.

Riju is measuring the characteristics of the PD and will be posting an elog in detail.

Today I tested the circuitry within the QPD to make sure I had put it back together correctly. The output was able to detect when a laser pointer was shone on different quadrants of the QPD when hooked up to the oscilloscope, so fixing the QPD worked! 

Following this, I spent awhile understanding what was going on with the circuit reading from the QPD. I concluded that the opamp on the QPD circuit acted as a low pass filter, with corner frequency of about 17 kHz, which serves our purposes (we are only interested in frequencies below 1 kHz). I diagrammed the circuit that I plan to build to give pitch and yaw (attached). It will be necessary to make small modifications to the circuitry already built (removing some resistors), which I have started on. 

Next time, I will construct the circuit that adds/subtracts signals on a breadboard and test it with the QPD circuit that is already built. 

At around 2:30pm today something brought down most of the martian network.  All control room workstations, nodus, etc. were unresponsive.  After poking around for a bit I finally figured it had to be linux1, which serves the NFS filesystem for all the important CDS stuff.  linux1 was indeed completely unresponsive.

Looking closer I noticed that the Fibrenetix FX-606-U4 SCSI hardware RAID device connected to linux1 (see #1901), which holds cds network filesystem, was showing "IDE Channel #4 Error Reading" on it's little LCD display.  I assumed this was the cause of the linux1 crash.

I hard shutdown linux1, and powered off the Fibrenetix device.  I pulled the disk from slot 4 and replaced it with one of the spares we had in the control room cabinets.  I powered the device back up and it beeped for a while.  Unfortunately the device was requiring a password to access it from the front panel, and I could find no manual for the device in the lab, nor does the manufacturer offer the manual on it's web site.

Eventually I was able to get linux1 fully rebooted (after some fscks) and it seemed to mount the hardware RAID (as /dev/sdc1) fine.  The brought the NFS back.  I had to reboot nodus to get it recovered, but all the control room and front-end linux machines seemed to recover on their own (although the front-ends did need an mxstream restart).

The remaining problem is that the linux1 hardware RAID device is still currently unaccessible, and it's not clear to me that it's actually synced the new disk that I put in it.  In other words I have very little confidence that we actually have an operational RAID for /opt/rtcds.  I've contacted the LDAS guys (ie. Dan Kozak) who are managing the 40m backup to confirm that the backup is legit.  In the mean time I'm going to spec out some replacement disks onto which to copy /opt/rtcds, and also so that we can get rid of this old SCSI RAID thing.

I accidentally overwrote crontab in op340m with an empty file.
By checking /var/cron in op340m, I think I restored it.
But somehow, autolockMCmain40m does not work in cron job, so it is currently running by nohup.

What I did:
  1. I ssh-ed op340m to edit crontab to change MC autolocker to usual power mode. I used "crontab -e", but it did not show anything. I exited emacs and op340m.
  2. Rana found that the file size of crontab went 0 when I did "crontab -e".
  3. I found my elog #6899 and added one line to crontab

55 * * * *  /opt/rtcds/caltech/c1/scripts/general/scripto_cron /opt/rtcds/caltech/c1/scripts/MC/autolockMCmain40m >/cvs/cds/caltech/logs/scripts/mclock.cronlog 2>&1

  4. It didn't run correctly, so Rana used his hidden power "nohup" to run autolockMCmain40m in background.
  5. Koji's hidden magic "/var/cron/log" gave me inspiration about what was in crontab. So, I made a new crontab in op340m like this;

34 * * * *  /opt/rtcds/caltech/c1/scripts/general/scripto_cron /opt/rtcds/caltech/c1/scripts/MC/autolockMCmain40m >/cvs/cds/caltech/logs/scripts/mclock.cronlog 2>&1
55 * * * * /opt/rtcds/caltech/c1/scripts/general/scripto_cron /opt/rtcds/caltech/c1/scripts/PSL/FSS/RCthermalPID.pl >/cvs/cds/caltech/logs/scripts/RCthermalPID.cronlog 2>&1
07 * * * * /opt/rtcds/caltech/c1/scripts/general/scripto_cron /opt/rtcds/caltech/c1/scripts/PSL/FSS/FSSSlowServo >/cvs/cds/caltech/logs/scripts/FSSslow.cronlog 2>&1
00 * * * * /opt/rtcds/caltech/c1/burt/autoburt/burt.cron >> /opt/rtcds/caltech/c1/burt/burtcron.log
13 * * * * /cvs/cds/caltech/conlog/bin/check_conlogger_and_restart_if_dead
14,44 * * * * /opt/rtcds/caltech/c1/scripts/SUS/rampdown.pl > /dev/null 2>&1

  6. It looks like some of them started running, but I haven't checked if they are working or not. We need to look into them.

Moral of the story:
  crontab needs backup.

[Yuta, Manasa, Sendhil, Rana]

With MC REFL PD fixed, we aligned MC in high power enabling a fully functional MC autolocker.
We then unlocked MC and aligned the PD and WFS QPDs. Also we checked the MC demodulator and found a ~20% leakage between the Q-phase and I-phase. This must be fixed later by changing the cable length.

We adjusted MC offsets using /opt/rtcds/caltech/c1/scripts/MC/WFS/WFS_FilterBank_offsets.
We then measured the MC spot positions using  /opt/rtcds/caltech/c1/scripts/ASS/MC/mcassMCdecenter
Spot positions seem to have shifted by 2mm in yaw.

We will proceed with aligning the arms now.

I tried to align and lock Y arm for the first time after pumping.
But I couldn't lock Y arm for more than ~1 sec. Why?


What I did:
  1. Centered IPANG/IPPOS using input TT1/TT2.

  2. Restored ITMY/ETMY slider values when it was aligned before pumping. I saw tiny flashes in TRY PD at this point.

  3. Replaced Ygreen REFL camera with ETMYT camera to see transmitted beam mode.

  4. Used TT1/TT2 and ITMY/ETMY to get ~ 0.4 peak in normalized TRY PD output (C1:LSC_TRY_OUT).

  5. Centered POY beam on POY11 PD.

  6. Changed I/Q mixing angle (C1:LSC-POY11_PHASE_R) from -61 deg to -16 deg to get good PDH signal in I_ERR (attached).

  7. Ran LSCoffsets script (now on LSC_OVERVIEW screen) to adjust PD offsets.

  8. Tried to lock Yarm with different gains, but failed. When lock is acquired, TRY fluctuates ~50 % and unlocks suddenly.


What I found:
  1. There was some OFFSETs left turned on in suspension screens. Don't leave them on!! They change alignment of the optics. I will leave it on until we complete Yarm alignment.

  2. C1:SUS-(ITMY|ETMY)_ASC(PIT|YAW) was kept oscillating the optic since Dec 17, 2013. I think this is from interrupted ASS script. Your script should restore everything when interrupted!


Next:
  - Beamspot on ITMY looks off-centered. Maybe A2L is causing unstable lock?
  - Maybe F2A is causing unstable lock?
  - More alignment?
  - FSS related? crontab related?

apache has been restarted.
How to: search "apache" on the 40m wiki

The camera that Yuta means in his elog from last night/this morning is the scattering camera at the Yend.  The reason (I think) that he had to do this is that Manasa and Jan took the cable for the ETMYT camera, and were using it for their scattering camera.  They mention in elog 8072 that they installed a camera, but they didn't say anything about having taken the ETMYT cable.  This is the kind of thing that is useful to elog!

Anyhow, I have removed the Watec that belongs with the scattering setup, that Yuta borrowed, and put it back on the scattering table-on-a-pedestal. I then realigned the usual ETMYT camera (that Yuta moved out of the way to install the borrowed Watec), and put the ETMYT cable back to its usual place, connected to the Sony camera's box on the floor.

tl;dr: ETMYT camera is back to original state.

EDIT later:  I put the Watec back, since it is more sensitive to IR, so now we have a Watec in the regular ETMYT place.

I found some daily cron jobs for op340m I missed last night. Also, I edited timings of hourly jobs to maintain consistency with the past. Some of them looks old, but I will leave as it is for now.
At least, burt, FSSSlowServo and autolockMCmain40m seems like they are working now.
If you notice something is missing, please add it to crontab.

07 * * * * /opt/rtcds/caltech/c1/burt/autoburt/burt.cron >> /opt/rtcds/caltech/c1/burt/burtcron.log
13 * * * * /opt/rtcds/caltech/c1/scripts/general/scripto_cron /opt/rtcds/caltech/c1/scripts/PSL/FSS/FSSSlowServo >/cvs/cds/caltech/logs/scripts/FSSslow.cronlog 2>&1
14,44 * * * * /cvs/cds/caltech/conlog/bin/check_conlogger_and_restart_if_dead
15,45 * * * * /opt/rtcds/caltech/c1/scripts/SUS/rampdown.pl > /dev/null 2>&1
55 * * * *  /opt/rtcds/caltech/c1/scripts/general/scripto_cron /opt/rtcds/caltech/c1/scripts/MC/autolockMCmain40m >/cvs/cds/caltech/logs/scripts/mclock.cronlog 2>&1
59 * * * * /opt/rtcds/caltech/c1/scripts/general/scripto_cron /opt/rtcds/caltech/c1/scripts/PSL/FSS/RCthermalPID.pl >/cvs/cds/caltech/logs/scripts/RCthermalPID.cronlog 2>&1

00 0 * * * /var/scripts/ntp.sh > /dev/null 2>&1
00 4 * * * /opt/rtcds/caltech/c1/scripts/RGA/RGAlogger.cron >> /cvs/cds/caltech/users/rward/RGA/RGAcron.out 2>&1
00 6 * * * /cvs/cds/scripts/backupScripts.pl
00 7 * * * /opt/rtcds/caltech/c1/scripts/AutoUpdate/update_conlog.cron

According to Google, you can add a line in the crontab to backup the crontab by having the cronback.py script be in the scripts/ directory. It needs to save multiple copies, or else when someone makes the file size zero it will just write a zero size file onto the old backup.

Calendars:  The calendar issue discussed previously (http://nodus.ligo.caltech.edu:8080/40m/8098) where the numbers are squished together is very difficult for me to find.  I am not going to worry about it for the time being.

Multiprocessing:   Reviewed the implementation of Multiprocessing in python (using Multiprocessing package).  Wrote a simple test function and ran it on megatron, to verify that multiprocessing could successfully take advantage of megatron’s multiple cores – it could.  Now, I will work on implementing multiprocessing in the program.  I began testing at a section in the program where a for loop calls process_data() (which has a long runtime) multiple times.  The megatron terminals I had open began to run very slowly.  Why?  I believe that the process_data() function loads data into global variables to accomplish its task.  The global variables in the original implementation were cleared before the subsequent calls to process_data().  But in the multiprocessing version, the data is not cleared, meaning the memory fills quickly, which drastically reduces performance.  In the short term, I could try generating fewer processes at a time, wait for them to finish, then clearing the data, then generating more processes, etc.  This will probably generate a nominal performance boost.  In the long-term, restructuring of the way the program handles data may help (but not for sure).  In the coming week I will experiment with these techniques and try to decrease the run time of the program.