[lydia, ericq, gautam]

We set about following the instructions linked in the previous elog. A few notes/remarks:

1. It is important to run the ntpdate commands before restarting the models. Sometimes, multiple restarts of the models were required to turn all the indicator blocks on the MEDM screen green.
2. There was also an issue of multiple ntpd processes running on the same machine, which obviously caused all sorts of timing havoc. EricQ helped us diagnose and fix these. At the moment, all the lights are green on the CDS status MEDM screen
   
3. On the hardware side, apart from the usual suspects of frontends/megatron/optimus/fb needing to be rebooted, I noticed that the ETMX OSEM lights were off on the control room monitors. Investigation pointed to the 2 20V sorensens at the X end outputting 0V, 0A after the power glitch. We turned down both dials, and then gradually ramped them up again. Both Sorensens now read +/-20V, 0.3A, which is in agreement with the label stuck onto them.
4. Restarted MC autolocker and FSS Slow scripts on megatron. I have not yet looked at the status of the nds2 server on megatron.
5. 11 MHz Marconi has yet to be restarted - but I am unable to get even the IMC locked at the moment. For some reason, the RMS of the MC1 and MC3 coils are way higher than what I am used to seeing (~5mV rms as compared to the <1mV rms I am used to seeing for a damped optic). I will investigate further. Leaving MC autolocker disabled for now.

Here is a link to an elog with the steps I had to follow the last time there was a similar power glitch.

The RAID array restart was also done not too long ago, we should also do a data consistency check as detailed here, if not already..

If someone hasn't found the time to do this, I can take care of it tomorrow afternoon after I am back.

Jamie started the fm40m Raid rebuilding. It has been beeping since the power outage.

Summary pages have no reading since power glitch.

                UTILITY & SERVICE INTERRUPTION

**PLEASE POST**

Building:               Central Engineering Services (C.E.S.)

LIGO Gravitational Physics building adjacent to C.E.S. 40M- Lab


Safety Storage adjacent to CES

Steele House 

Keck Lab

Date:                   Saturday, October 1, 2011

Time:                   8:00 a.m. To 9:00 a.m.            

Interruption:   Electricity

Contact:                Mike Anchondo ext. 4999  Tom Brennan 4984

*This interruption is required for maintenance of high voltage switchgear in Campus Sub Station.

(If there is a problem with this Interruption, please notify

 the Service Center X-4717 or the above Contact as soon as possible.

 If no response is received we will proceed with the interruption.)

                                Reza Ohadi,

                                Director, Campus Operations & Maintenance

                                                                                                                                   received this note: at 4:11pm Tuesday, Feb 7, 2017

**PLEASE POST**

Building:         Campus

Date:             Wednesday, February 8, 2017

Time:             7:30 AM – 8:30 AM  

Contact:          Rick Rodriguez x-2576

Pasadena Water and Power (PWP) will be performing a switching operation of the

Caltech Electrical Distribution System that is expected to be transparent to Caltech,

but could result in a minor power anomaly that might affect very sensitive equipment.

IMPACT: Negligible impact......?

There may be temporary  power interruption tomorrow!

PS:we did not see any effect   

[Valera Yuta Kiwamu Koji]

Kiwamu burtrestored c1psl. We measured the power levels around the PMC.

With 2.1A current at the NPRO:

Pincident = 1.56W
Ptrans_main = 1.27W
Ptrans_green_path = .104W

==> Efficiency =88%

----

We limited  the MC incident power to ~50mW. This corresponds to the PMC trans of 0.65V.
(The PMC trans is 1.88V at the full power with the actual power of 132mW)

Now a power normalization is doable for the LSC error signals.

It is working fine, but at some point we may want to have some kind of a saturation filter or limiter to avoid dividing a signal by a small number.

 (How to set the normalization)

•   Click a small matrix panel on the LSC OVERVIEW window (shown in the attached screen shot below).
  •     This will give you a pop-up-window, which shows a matrix to route the normalization signals

•   Choose a numerator channel, which you want to divide, and choose denominator channels, which you want to use as a power normalization factor.
•   Put some number in the corresponding matrix elements.
•   Once you put a non-zero element in the matrix, the corresponding numerator channel will be divided by the specified denominator channels.
  •     Otherwise the static normalization factors (e.g. C1:LSC-AS55_POW_NORM, etc.,) will be used for the denominator.

It turned out that the power normalization need a modification.

I will work on it tomorrow and it will take approximately 2 hours to finish the modification.

     Concept of Power Normalization         

1.  Static power normalization, which should be placed before the input matrix.
2.  Dynamic power normalization, which should be placed after the input matrix.

The dynamic power normalization system has been modified such that the normalization happen after the LSC input matrix.

Looks like there was a power outage. The control room workstations were all off (except for op440m). Rosalba and the projector's computer came back, but rossa and allegra are not lighting up their monitors.

linux1 and nodus and fb all appear to be on and answering their pings.

I'm going to leave it like this for the morning crew. If it

The monitors for allegra and rossa's seemed to be in a weird state after the power outage.  I turned allegra and rossa on, but didn't see anything.  However, I was after awhile able to ssh in.  Power cycling the monitors did apparently got them talking with the computers again and displaying.

I had to power cycle the c1sus and c1iscex machines (they probably booted faster than linux1 and the fb machines, and thus didn't see their root and /cvs/cds directories).  All the front ends seem to be working normally and we have damped optics.

The slow crates look to be working, such as c1psl, c1iool0, c1auxex and so forth.

Kiwamu turned the main laser back on.

I checked the vacuum system and judged there is no apparent issue.

The chambers and annulus had been vented before the power failure.
So the matters are only on the TMPs.

TP1 showed the "Low Input Voltage" failure. I reset the error and the turbine was lift up and left not rotating.
TP2 and TP3 seem rotating at 50KRPM and the each lines show low pressur (~1e-7)
although I did not find the actual TP2/TP3 themselves.

There had to be a power outage. Laser and air condition turned back on. The vacuum is OK

Sorensen DC power supplies were tripped, so they were reset: at AUX OMC South 18V and 28V for RF PS and at 1X1 24V

Power Outage confirmed:

There was a power outage.

The IFO pressure is 12.8 mTorr-it and it is not pumped. V1 is still closed. TP1 is not running. The Rga is not powered.

The PSL output shutter is still closed. 2W Innolight turned on and manual beam block placed in its beampath.

3 AC units turned on at room temp 84F

I've been looking into recovering the seismic BLRMs for the BS Trillium seismometer. It looks like the problem is probably in the anti-aliasing board. There's some heavy stuff sitting on top of it in the rack, so I'll take a look at it later when someone can give me a hand getting it out.

In detail, after verifying that there are signals coming directly out of the seismometer, I tried to inject a signal into the AA board and see it appear in one of the seismometer channels.

1. I looked specifically at C1:PEM-SEIS_BS_Z_IN1 (Ch9), C1:PEM-SEIS_BS_X_IN1 (Ch7), and C1:PEM-ACC_MC2_Y_IN1 (Ch27). All of these channels have between 2000--3000 cts.
2. I tried injecting a 200 mVpp signal at 1.7862 Hz into each of these channels, but the the output did not change.
3. All channels have 0 cts when the power to the AA board is off.
4. I then tried to inject the same signal into the AA board and see it at the output. The setup is shown in the first attachment. The second BNC coming out of the function generator is going to one of the AA board inputs; the 32 pin cable is coming directly from the output. All channels give 4.6 V when when the board is powered on regardless of wheter any signal is being injected.
5. To verify that the AA board is likely the culprit, I also injected the same signals directly into the ADC. The setup is shown in the second attachment. The 32 pin cable is going directly to the ADC. When injecting the same signals into the appropriate channels the above channels show between 200--300 cts, and 0 cts when no signal is injected.

1. I had to manually reboot c1lsc, c1sus and c1ioo.
2. I edited the line in /etc/rt.sh (specifically, on FB /diskless/root.jessie/etc/rt.sh) that lists models running on a given frontend, to exclude c1dnn and c1oaf, as these are the models that have been giving us most trouble on startup. After this, I was able to bring back all models on these three machines using rtcds restart --all. The original line in this file has just been commented out, and can be restored whenever we wish to do so.
3. mx_stream processes are showing failed status on all the frontends. As a result, the daqd processes are still not working. Usual debugging methods didn't work.
4. Restored all sus dampings.
5. Slow computers all seem to be responsive, so no action was required there.
6. Burtrestored c1psl to solve the "sticky slider" problem, relocked PMC. I didn't do anything further on the PSL table w.r.t. the manual beam block Steve has placed there till the vacuum situation returns to normal.

@Steve: I noticed that we are down to our final bottle of N2, not sure if it will last till 2 Jan which is presumably when the next delivery will come in. Since V1 is closed and the PSL beam is blocked, perhaps this doesn't matter.

from Steve: there are spare full N2 bottles at the south end outside and inside. I replaced the N2 on Sunday night. So the system should be Ok as is.

I also hard-rebooted megatron and optimus as these were unresponsive to ping.

*Seems like the mx_stream errors were due to the mx process not being started on FB. I could fix this by running sudo systemctl start mx on FB. After which I ran sudo systemctl restart daqd_*. But the DC errors persist - not sure how to fix this. Elogging suggests that "0x4000" errors are connected to timing problems on FB, but restarting the ntp service on FB (which is the suggested fix in said elogs) didn't fix it. Also unsure if mx process is supposed to automatically start on FB at startup.

Mostly back to nominal operating conditions now.

1. EX TransMon QPD is not giving any sensible output. Seems like only one quadrant is problematic, see Attachment #1. I blame team EX_Acromag for bumping some cabling somewhere. In any case, I've disabled output of the QPD, and forced the LSC servo to always use the Thorlabs "High Gain" PD for now. Dither alignment servo for X arm does not work so well with this configuration - to be investigated.
2. BS Seismometer (Trillium) is still not giving any sensible output.
   • I looked under the can, the little spirit level on the seismometer is well centered.
   • I jiggled all the cabling to rule out any obvious loose connections - found none at the seismometer, or at the interface unit (labelled D1002694 on the front panel) in 1X5/1X6.
   • All 3 axes are giving outputs with DC values of a few hundred - I guess there could've been some big earthquake in early December which screwed the internal alignment of the sensing mass in the seismometer. I don't know how to fix this.
   • Attachment #2 = spectra for the 3 channels. Can't say they look very seismicy . I've assumed the units are in um/sec.
   • This is mainly bothering me in the short term because I can't use the angular feedforward on PRC alignment, which is usually quite helpful in DRMI locking.
   • But I think the PRM Oplev loop is actually poorly tuned, in which case perhaps the feedforward won't really be necessary once I touch that up.

What I did today (may have missed some minor stuff but I think this is all of it):

1. At EX:
   • Toggled power to Thorlabs trans monitoring PD, checked that it was actually powered, squished some cables in the e- rack.
   • Removed PDA55 in the green path (put there for EX laser AM/PM measurement). So green beam can now enter the X arm cavity.
   • Re-connected ALS cabling.
   • Turned on HV supply for EX Green PZT steering mirrors (this has to be done every time there is a power failure).
2. At ITMY table:
   • Removed temporary HeNe RIN/ Oplev sensing noise measurement setup. HeNe + 1" vis-coated steering mirror moved to SP table.
   • Turned on ITMY/SRM Oplev HeNe.
   • Undid changes on ITMY Oplev QPD and returned it to its original position.
   • Centered ITMY reflected beam on this QPD.
3. At vertex area
   • Looked under Trillium seismometer can - I've left the clamps undone for now while we debug this problem.
   • Power-cycled Trillium interface box.
   • Touched up PMC alignment.
4. Control room
   • Recover IFO alignment using combination of IR and Green beams.
   • Single arm locking recovered, dither alignment servos run to maximize arm transmission. Single arm locks holding for hours, that's good.
   • The X arm dither alignment isn't working so well, the transmission never quite hits 1 and it undergoes some low frequency (T~30secs) oscillations once the transmission reaches its peak value.
   • Had to do the usual ipcrm thing to get dataviewer to run on pianosa.

Next order of business:

1. Recover ALS:
   • aim is to replace the vertex area ALS signals derived from 532nm with their 1064nm counterparts.
   • Need to touch up end PDH servos, alignment/MM into arms, and into Fibers at ends etc.
   • Control the arms (with RMs misaligned) in the CARM/DARM basis using the revised ALS setup.
   • Make a noise budget - specifically, we are interested in how much actuation range is required to maintain DARM control in this config.
2. Recover DRMI locking
   • Continue NBing.
   • Do a statistical study of actuation range required for acquiring and maintaining DRMI locking.

As mentioned in my previous elog, the CDS overview screen "DC" indicators are all RED (everything else is green). Opening up the displays for individual CPUs, the error message shown is "0x4000", which is indicative of some sort of timing error. Indeed, it seems to me that on the FB machine, the gpstime command shows a gps time that is ~1 second ahead of the times on other FE machines.

Running gpstime on other FE machines throws up an error, saying that it cannot connect to the network to update leap second data. Not sure what this is about...

I double checked the GPS timing module, we had some issues with this in the recent past. But judging by its front panel display, everything seems to be in order...

Rolf came here in the morning, but not sure what he did or if Jamie remotely did something. But the screen is green.

Just putting the relevant line from email from Rolf which at least identifies the problem here:

There still seems to be something funky with the X arm transmission PDs - I can't seem to get the triggering to switch between the QPD and the Thorlabs PD, and the QPD signal seems to be wildly fluctuating by several orders of magnitude from 0.01-100. The c1iscex FE was pulled out, and it seemed to me like someone was doing some cable re-arrangement at the X end.

I will look into this tomorrow. 

I measured the RC transmitted light signals here at the 40m. I made all connections through the PSL patch panel.

Other than two steering mirrors in front of the periscope, and the steering mirror for the RFPD which were used to steer

the beam into the cavity and the RFPD respectively, no optics are adjusted.

We re-aligned the beam into the cavity (the DC level increased from 2 V to 3.83V) (Fig2) (We could not recover the power back to what it was 90 days ago)

and the reflected beam to the center of the RFPD.

I measured the spectral density of the signal of the transmitted beam behind RefCav in both time and frequency domain.

This will be compared with the result from PSL lab later, so I can see how stable the signal should be.

I did not convert Vrms/rtHz to Hz/rtHz because I only look at the relative intensity of the transmitted beam which will be compared to the setup at PSL lab. 

 We care about this power fluctuation because we plan to measure

 photo refractive noise on the cavity's mirros

(this is the noise caused by dn/dT in the coatings and the substrate,

the absorption from fluctuating power on the coating/mirror changes

the temperature which eventually changes the effective length of the cavity as seen by the laser.)

      The plan is to modulate the power of the beam going into the cavity,

the absorption from ac part will induce frequency noise which we want to see.

Since the transmitted power of the cavity is proportional to the power inside the cavity.

 Fluctuations  from other factors, for example, gain setting,  will limit our measurement. 

That's why we are concerned about the stability of the transmitted beam and made this measurement.

How do you calibrate this to Hz/rtHz?

Koji, Steve

 It was a bad experience again with our vacuum system.  The valves went crazy as we rebooted the computer. This was required for the swap in of a good 24V power supply.

The IFO was vented to 27 Torr through the annuloses, VA6, V7, Maglev,VM2 and VM1 (VC2 was open too)

I just opened the PSL shutter after a 4 hours pumpdown.

Condition: annuloses are not pumped, the IFO and the RGA are pumped as Atm2 shows

I will be here tomorrow morning to switch over to vacuum normal. 

More details later

I'm leaving the 40m now. IFO is aligned. Everything look good.

- The main volume P1=5e-4, CC1=1.4e-5 is still pumped by TP1 and TP2

- RGA P4<0e-4, CC4 2.1e-7, is pumped by TP3

- The annuluses are isolated.

- RP1/2/3 are off.

 Events of the power supply swap:

1, Tested 24V DC ps from Todd

2, Closed V1, VM1 and all annulos valves to create safety net for the reboot. Turbo pumps left on running.

3, Turned computer off

4, Swap power supplies and turned it on

5, Turning the power on of c1vac2 created caos switching of valves. This resulted in a air vent as shown below.

6, VM1 was jammed and it was unable to close. The IOO beam shutter closed and the IFO was venting with air for a few minutes. Maglev did an emergency shut down. TP2's V4 and TP3' V5 closed. RP1 and RP3                           roughing         pumps turned on, their hose was not connected as usual. The RGA shut down to protect itself.

7, Closed annulos valves, stoped the vent at P1 27 torr as the vacuum control was  manually recovered

8, The Maglev and the annuloses were roughed out 500 mtorr . The Maglev was restarted.

9, The IFO pump down followed std procedure from 27 torr. VM1 was moving again as the pressure differential was removed from it.

 Remember: next time at atm .....rough down the cryo volume from 27 torr !

The recovery- pumpdown reached valve configuration  vacuum normal at 20 hours, cc1 7.7e-6 Torr

Lesson learned: turn all pumps off, close all valves before you reboot ! like you would prepare for AC power shut down.

I was notified by CIT Utilities that there was a power surge or short power outage this after noon.

Lab conditions are normal:  c1ioo is down.  The south arm AC was off......I turned it back on.

 Today I wanted to check that AS and REFL beams are real and contain proper information about interferometer. For this I locked YARM using AS55_I and REFL11_I. Then I compared spectrum with POY11_I locking. Everything is the same. I've also adjusted phase rotations of AS55 (0.2 ->24) and REFL11 (-34.150 -> -43).

Then I've locked MICH and aligned EMTs such that ASDC was close to zero. Then I locked PRCL and aligned PRM. Power buildup was 50. 

I compared PCRL and XARM angular motions by misaligning the cavities and measuring power RIN. Divergence angles for both cavities I calculated to be 100 urad.

XARM pointing noise sums from input steering TTs, PR2 and PR3 TTs, BS, ITMX, ETMY.

PRCL noise - from input TT, PRM, PR2 and PR3 TT, BS, ITMX, ITMY.

I would expect these noises to be the same as angular motion of different optics measured by oplves is simular. We do not have oplves on TT but they are present in both passes.

I measured RIN and converted to angle. Sharp 1 Hz resonance at XARM pointing spectrum is due to EMTX, it is not seen by PRCL. Other then that XARM is much quiter, especially at 3 - 30 Hz.

As PRM  is the main difference in two passes, I checked its spectrum. When PRCL was locked I excited PRM in pitch and yaw. I could see this excitation at RIN only when the peak was 100 times higher then background seismic noise measured by oplev.

 How is the cavity g-factor accounted for in this calculation?

 I assume that pointing noise and dc misalignment couples 00 to 01 by a factor theta / theta_cavity

Inside the cavity 01 is suppressed by 2/pi*F*sin(arccos(sqrt(g_cav))).

For the XARM this number is 116 taking g-factor to be 0.32. So all pointing noise couples to power RIN.

Suppression factor inside PRC is 6.5 for g-factor 0.97. This means that 85% of jitter couples to RIN, I accounted for this factor while converting RIN to angle.

I did not consider translational motion of the beam. But still PRC RIN can not be explained by oples readings as we can see exciting optics in pitch and yaw. I suspect this RIN is due to PR3, as it can create stronger motion in yaw than in pitch due to incident angle and translational motion of the mirror. I do not have a number yet.

 Maybe its equivalent, but I would have assumed that the input beam is fixed and then calculate the cavity axis rotation and translation. If its small, then the modal expansion is OK. Otherwise, the overlap integral can be used.

For the ETM motion, its a purely translation effect, whereas its tilt for the ITM. For the PRM, it is also a mostly translation effect as calculated at the PRC waist position (ITM face).

I made another estimation assuming that PRCL RIN is caused by translation of the cavity axis:

• calibrated RIN to translation, beam waist = 4mm
• measured PRM yaw motion using oplev
• estimated PR3 TT yaw motion: measured BS yaw spectrum with oplev OFF, divided it by pendulum TF with f0=0.9 Hz, Q=100 (BS TF), multiplied it by pendulum TF with f0 = 1.5 Hz, Q = 2 (TT TF with eddy current damping), accounted for BS local damping that reduces Q down to 10.

PRM and TT angular motion to cavity axis translation I estimated as 0.11 mm/urad and 0.22 mm/urad assuming that TTs are flat. We can make a more detailed analysis to account for curvature.

I think beam motion is caused by PR3 and PR2 TT angular motion. I guess yaw motion is larger because horizontal g-factor is closer to unity then vertical.

 In order to get translation to RIN, we need to know the offset of the input beam from the cavity axis...

This should be possible to calibrate by putting a pitch and yaw excitation lines into the PRM and measuring the RIN.

See secret document from Koji.

I want to redo this estimate of where RIN comes from, since Den did this measurement before I put the lens in front of the POP PD. 

While thinking about his method of estimating the PR3 effect, I realized that we have measured numbers for the pendulum frequencies of the recycling cavity tip tilt suspensions. 

I have been secreting this data away for years.  My bad.  The relevant numbers for Tip Tilts #2 and #3 were posted in elog 3425, and for #4 in elog 3303.  However, the data for #s 1 and 5 were apparently never posted.  In elog 3447, I didn't put in numbers, but rather said that the data was taken.

Anyhow, attached is the data that was taken back in 2010.  Look to elog 7601 for which TT is installed where. 

Conclusion for the estimate of TT motion to RIN - the POS pendulum frequency is ~1.75Hz for the tip tilts, with a Q of ~2.

The Central Plant building will be undergoing seismic upgrades in the near future.  The adjoining north wall along the Y arm will be the first to have this work done, from inside the Central Plant.  Project manager Eugene Kim has explained the work to me and also noted our concerns.  He assured me that the seismic noise from the construction will be minimized and we will always be contacted when the heaviest construction is to be done.

Tomorrow at 11am, I will bring Mr. Kim and a few others from the construction team to look at the wall from inside the lab.  If you have any questions or concerns that you want to have addressed, please email them to me or contact Mr. Kim directly at x4860 or through email at eugene.kim@caltech.edu . 

 [Koji Jenne]

The lasers were shutdown

The racks were turned off

We could not figure out how to turn off JETSTOR

The control room machines were turned off

FInally we will turn off nodus and linux1 (with this order).

Hope everything comes back with no trouble

(Finger crossing)

I've been preparing for testing Gabriele's deep neural network MICH/PRCL reconstruction.  No changes to the front end have been made yet, this is all just prep/testing work.

Background:

We have been unable to get Gabriele's nn.c code running in kernel space for reasons unknown (see tests described in previous post).  However, Rolf recently added functionality to the RCG that allows front end models to be run in user space, without needing to be loaded into the kernel.  Surprisingly, this seems to work very well, and is much more stable for the overall system (starting/stopping the user space models will not ever crash the front end machine).  The nn.c code has been running fine on a test machine in this configuration.  The RCG version that supports user space models is not that much newer than what the 40m is running now, so we should be able to run user space models on the existing system without upgrading anything at the 40m.  Again, I've tested this on a test machine and it seems to work fine.

The new RCG with user space support compiles and installs both kernel and user-space versions of the model.

Work done:

• Create 'c1dnn' model for the nn.c code.  This will run on the c1lsc front end machine (on core 6 which is currently empty), and will communicate with the c1lsc model via SHMEM IPC.  It lives at:
  • /opt/rtcds/userapps/release/isc/c1/models/c1dnn.mdl
• Got latest copy of nn.c code from Gabriele's git, and put it at:
  • /opt/rtcds/userapps/release/isc/c1/src/nn/
• Checked out the latest version of the RCG (currently SVN trunk r4532):
  • /opt/rtcds/rtscore/test/nn-test
• Set up the appropriate build area:
  • /opt/rtcds/caltech/c1/rtbuild/test/nn-test
• Built the model in the new nn-test build directory ("make c1dnn")
• Installed the model from the nn-test build dir ("make install-c1dnn")

Test:

I tried a manual test of the new user space model.  Since this is a user space process running it should have no affect on the rest of the front end system (which it didn't):

• Manually started the c1dnn EPICS IOC:

  • $ (cd /opt/rtcds/caltech/c1/target/c1dnn/c1dnnepics && ./startupC1)

• Tried running the model user-space process directly:

  • $ taskset -c 6 /opt/rtcds/caltech/c1/target/c1dnn/bin/c1dnn -m  c1dnn

Unfortunately, the process died with an "ADC TIMEOUT" error.  I'm investigating why.

Once we confirm the model runs, we'll add the appropriate SHMEM IPC connections to connect it to the c1lsc model.

I tried moving the model to c1ioo, where there are plenty of free cores sitting idle, and the model seems runs fine.  I think the problem was just CPU contention on the c1lsc machine, where there were only two free cores and the kernel was using both for all the rest of the normal user space processes.

So there are two options:

• Use cpuset on c1lsc to tell the kernel to remove all other processes from CPU6 and save it just for the c1dnn model.  This should not have any impact on the running of c1lsc, since that's exactly what would be happening if we were running the model in kernel space (e.g. isolating the core for the front end model).  The auxilliary support user space processes (epics seq/ioc, awgtpman) should all run fine on CPU0, since that's what usually happens.  Linux is only using the additional core since it's there.  We don't have much experience with cpuset yet, though, so more offline testing will be required first.
• Run the model on c1ioo and ship the needed signals to/from c1lsc via PCIe dolphin.  This is potentially slightly more invasive of a change, and would put more work on the dolphin network, but it should be able to handle it.

I'm going to start testing cpuset offline to figure out exactly what would need to be done.

[Koji, Jamie, Yuta]

We attenuated the incident beam (1.2 W -> 11 mW) to the vacuum chamber to be ready for the vent.
The beam spot on the MC mirrors didn't changed significantly, which means the incident beam was not shifted so much.

What we did:
 1. Installed HWP, PBS(*) and another HWP between the steering mirrors on PSL table for attenuating the beam. We didn't touched steering mirrors(**), so the incident beam to the IFO should be recovered easily, by just taking HWPs and PBS away. The power to the MC was reduced from 1.2 W to 11 mW.

(*) We stole PBSO from the AS AUX laser setup.
(**) Actually, we accidentally touched one of the steering mirrors, but we recovered them. We did the recovery tweaking the touched nob and minimizing the MC reflection. We confirmed the incident beam was recovered by measuring MC beamspot positions(below).

 2. Aligned PBS by minimizing MC reflection, adjusted first HWP so that the incident beam will be ~10 mW, and adjusted last HWP to minimize MC reflection (make the incident beam to the MC be p-polarization).

 3. To do the alignment and adjusting, we put 100% reflection mirror (instead of 10% BS) for the MC reflection PD to increase the power to the PD. That means, we don't have MC WFS right now.

 4. Tweaked MC servo gains to that we can lock MC in low power mode. It is quite stable right now. We didn't lose lock during beam spot measurement.

 5. Measured beam spot positions on the MC mirrors and convinced that the incident beam was not shifted so much (below). They look like they moved ~0.2 mm, but it is with in the error of the MC beam spot measurement.

# filename      MC1pit  MC2pit  MC3pit  MC1yaw  MC2yaw  MC3yaw  (spot positions in mm)
./dataMCdecenter/MCdecenter201206281154.dat     3.193965        4.247243        2.386126        -6.639432       -0.574460       4.815078    this noon
./dataMCdecenter/MCdecenter201206282245.dat     3.090762        4.140716        2.459465        -6.792872       -0.651146       4.868740    after recovered steering mirrors
./dataMCdecenter/MCdecenter201206290135.dat     2.914584        4.240889        2.149244        -7.117336       -1.494540       4.955329    after beam attenuation

 6. Rewrote matlab code sensemcass.m to python script sensemcass.py. This script is to calculate beam spot positions from the measurement data(see elog #6727). I think we should make senseMCdecenter script better, too, since it takes so much time and can't stop and resume the measurement if MC is unlocked.

1. Turned off high voltage power supplies for PZT1/2 (input PZTs) and OMC stage 1/2. They live in 1Y3 rack and AUX_OMC_NORTH rack.

2. Restored all IFO optics alignment to the position where I aligned this afternoon (for SRM, I didn't aligned it; it restored at the saved value on May 26).

3. Centered all the oplevs. They can be used for a reference for alignment change before and after the vent.

I will leave PSL mechanical shutter and green shutters closed just in case.

Some MEDM screenshots below.

I checked out the elog from the vent in October 2016 when the OMC was removed from the path. In the vent in a couple weeks, we'd like to get the beam going through the OMC again. I wasn't really there for this last vent and don't have a great sense for how things go at the 40m, but this is how I think the procedure for this work should approximately go. The main points are that we'll need to slightly translate and rotate OM5, rotate OM6, replace one mirror that was removed last time, and add some beam dumps. Please let me know what I've got wrong or am missing.

[side note, I want to make some markup on the optics layouts that I see as pdfs elsewhere in the log and wiki, but haven't done it and didn't much want to dig around random drawing software, if there's a canonical way this is done please let me know.]

Steps to return the OMC to the IFO output:

1. Complete non-Steve portions of the pre-vent checklist (https://wiki-40m.ligo.caltech.edu/vent/checklist)
2. Steve needs to complete his portions of the checklist (as in https://nodus.ligo.caltech.edu:8081/40m/12557)
3. Need to lock some things before making changes I think—but I’m not really sure about these, just going from what I can glean from the elogs around the last vent
   1. ​Lock the IMC at low power
   2. Align the arms to green
   3. Lock the arms
   4. Center op lev spots on QPDs
   5. Is there a separate checklist for these things? Seems this locking process happens every time there is a realignment or we start any work, which makes sense, so I expect it is standardized.
4. Turn/add optics in the reverse order that Gautam did
   1. ​Check table leveling first?
   2. Rotate OM5 to send the beam to the partially transmissive mirror that goes to the OMC; currently OM5 is sent directly to OM6. OM5 also likely needs to be translated forward slightly; Gautam tried to maintain 45 deg AOI on OM5/6.
   3. A razor beam dump was also removed, which should be replaced (see attachment 1 on https://nodus.ligo.caltech.edu:8081/40m/12568)
   4. May need to rotate OM6 to extract AS beam again, since it was rotated last time
   5. Replace the mirror just prior to the window on the AP table, mentioned here in attachment 3: https://nodus.ligo.caltech.edu:8081/40m/12566
      1. ​There is currently a rectangular weight on the table where the mirror was, for leveling
5. Since Gautam had initially made this change to avoid some backscattered beams and get a little extra power, we may need to add some beam dumps to kill ghosts
   1. This is also mentioned in 12566 linked above, the dumps are for back-reflection off the windows of the OMC
6. Center beam in new path
7. Check OMC table leveling
8. AS beam should be round on the camera, with no evidence of clipping on any optics in the path (especially check downstream of any changes)

POX11 (see this entry) is now listed as REFL11 (on the very top row).

We will rename POY11 to POP11 for DRMI locking.

The files are on https://nodus.ligo.caltech.edu:30889/svn/trunk/suresh/40m_RF_upgrade/.

There is a planned power outage tomorrow, Saturday from 7am till midnight.

I vented all annulies and switched to ALL OFF configuration. The small region of the RGA is still under vacuum.

The vac-rack: gauges, c1vac1 and UPS turned off.

Gautam and Steve,

We have calibrated the load  cells. The support beams height monitoring is almost ready.

The danger of this measurment that  the beams height changes can put shear and torsional forces on this formed (thin walled) bellow

They are designed for mainly axial motion.

The plan is to limit height change to 0.020" max

0, center oplev at X arm locked

1, check that  jack screws are carrying full loads and set height indicator dials to zero ( meaning: Stacis is bypassed )

2, raise beam height with aux leveling wedge  by 0.010"  on all 3 support point and than raise it an other 0.005"

3, replace levelling wedge with load cell that is centered and shimmed.     Dennis   Coyne pointed out that the Stacis foot has to be loaded at the center of the foot and formed bellow can shear at their limits.

4, lower the support beam by 0.005" ......now full load on the cells

Note: jack screw heights will not be adjusted or  touched.......so the present condition will be recovered

[ Dennis Coyne'  precision answer ]

Differential Height between Isolators

According to a note on the bellows drawing (D990577-x0/A), the design life of the bellows at ± 20 minutes rotational stroke is 10,000 cycles. A 20 minute angular (torsional) rotation of the bellows corresponds to 0.186" differential height change across the 32" span between the chamber support beams (see isolator bracket, D000187-x0/B).

Another consideration regarding the bellows is the lateral shear stress introduced by the vertical translation. The notes on the bellows drawing do not give lateral shear limits. According to MDC's web page for formed bellows in this size range the lateral deflection limit is approximately 10% of the "live length" (aka "active length", or length of the convoluted section). According to the bellows drawing the active length is 3.5", so the maximum allowable lateral deflection should be ~0.35".

Of course when imposing a differential height change both torsional and lateral shear is introduced at the same time. Considering both limits together, the maximum differential height change should be < 0.12".

One final consideration is the initial stress to which the bellows are currently subjected due to a non-centered support beam from tolerances in the assembly and initial installation. Although we do not know this de-centering, we can guess that it may be of the order of ~ 0.04". So the final allowable differential height adjustment from the perspective of bellows stress is < 0.08".   Steve:  accumulated initial stress is unknown.  We used to adjust the original jack screws for IFO aligment in the early days of ~1999. This kind of adjustment was stopped when we realized how dangereous it can be. The fact is that there must be unknown amount of accumulated initial stress. This is my main worry but I'm confident that 0.020" change is safe.

So, with regard to bellows stress alone, your procedure to limit the differential height change to <0.020" is safe and prudent.

However, a more stringent consideration is the coplanarity requirement (TMC Stacis 2000 User's Manual, Doc. No. SERV 04-98-1, May 6, 1991, Rev. 1), section 2, "Installation",which stipulates < 0.010"/ft, or < 0.027" differential height across the 32" span between the chamber support beams. Again, your procedure to limit the differential height change to < 0.02" is safe.

Centered Load on the STACIS Isolators

According to the TMC Stacis 2000 User's Manual (Document No. SERV 04-98-1, May 6, 1991, Rev. 1), section 2, "Installation", typical installations (Figure 2-3) are with one payload interface plate which spans the entire set of 3 or 4 STACIS actuators. Our payload interface is unique.

Section 2.3.1, "Installation Steps": "5. Verify that the top of each isolator is fully under the payload/interface plate; this is essential to ensure proper support and leveling. The payload or interface plate should cover the entire top surface of the Isolator or the entire contact area of the optional jack."

section 2.3.2, "Payload/STACIS Interface": "... or if the supporting points do not completely cover the top surface of each Isolator, an interface plate will be needed."

The sketch in Figure 2-2 indicates an optional leveling jack which appears to have a larger contact surface area than the jacks currently installed in the 40m Lab. Of course this is just a non-dimensioned sketch. Are the jacks used by the 40m Lab provided by TMC, or did we (LIGO) choose them? I beleive Larry Jones purchased them.

A load centering requirement is not explicitly stated, but I think the stipulation to cover the entire top surface of each actuator is not so much to reduce the contact stress but to entire a centered load so that the PZT stack does not have a reaction moment.

From one of the photos in the 40m elog entry (specifically jack_screw.jpg), it appears that at least some isolators have the load off center. You should use this measurement of the load as an opportunity to re-center the loads on the Isolators.

In section 2.3.3, "Earthquake Restraints" restraints are suggested to prevent damage from earth tremors. Does the 40m Lab have EQ restraints? Yes, it has

Screw Jack Location

I could not tell where all of the screw jacks will be placed from the sketch included in the 40m elog entry which outlines the proposed procedure.

Load Cell Locations

The sketch indicates that the load cells will be placed on the center of the tops of the Isolators. This is good. However while discussing the procedure with Gautam he said that he was under the impression that the load cell woudl be placed next to the leveling jack, off-center. This condition may damage the PZT stack. I suggest that the leveling jack be removed and replaced (temporarily) with the load cell, plus any spacer required to make up the height difference. Yes

If you have any further question, just let me know.

    Dennis

Dennis Coyne
Chief Engineer, LIGO Laboratory
California Institute of Technology
MC 100-36, 1200 E. California Blvd.

follow up email from Dennis 5-13-2018. The last line agrees with the numbers in elog13821.

The vent will take place on Wednesday.

Plan for Tuesday :

  (Morning) Preparation of necessary items for the low power MC (Steve / Jamie)

  (Daytime) Measurement of the MC spot positions (Suresh)

  (Daytime) Arm length measurement (Jenne)

  (Nighttime) Locking of the low power MC (Kiwamu / Volunteers)

Plan for Wednesday :

  (Early morning) Final checks on the beam axis, all alignments and green light (Steve / Kiwamu / Volunteers )

  (Morning) Start the vent (Steve)

  (daytime-nighttime) Taking care of the Air/Nitrogen cylinders (Everybody !!)

Status of the vent preparation :

  (not yet) Low power MC

  (ongoing) Measurement of the arm lengths

  (ongoing) Measurement of the MC spot positions

  (80% done) Estimation of the tolerance of the arm length (#5076)

  (done) Alignment of the Y green beam (#5084)

  (done) Preparation of beam dumps (#5047)

  (done) Health check of shadow sensors and the OSEM damping gain adjustment (#5061)

  (done) Alignment of the incident beam axis (#5073)

  (done) Loss measurement of the arm cavities (#5077)

Status update for the vent preparation:

The punchline is : We can not open the chamber on Monday !

##### Task List for the vent preparation #####

  (not yet) Low power MC

  (not yet) Measurement of the arm lengths

  (not yet) Alignment of the Y green beam (#5066)

  (not yet) Measurement of the MC spot positions

  (80% done) Estimation of the tolerance of the arm length (#5076)

  (done) Preparation of beam dumps (#5047)

  (done) Health check of shadow sensors and the OSEM damping gain adjustment (#5061)

  (done) Alignment of the incident beam axis (#5073)

  (done) Loss measurement of the arm cavities (#5077)