If I catch anyone putting small booties into the large bootie bin, I will make said person eat small booties.

Gautam and Steve,

The medm monitor & vac control screens were totally blank since ~ May 24, 2017    Experienced vacuum knowledge is required for this job.

IDENTIFY valve configuration:

                        How to confirm valve configuration when all vac mons are blank?  Each valve has a manual-mechanical position indicator. Look at pressure readings and turbo pump controllers. VAC NORMAL configuration was confirmed based on these information.

Preparation: disconnect valves ( disconnect meaning: valve closes and stays paralized ) in this sequence VC2, VC1 power, VA6, V5, V4 & V1 power,      at ifo pressure 7.3E-6 Torr-it  ( it  = InstruTech cold cathode gauge )

                            This gauge is independent from all other rack  mounted   instrumentation and it is still not logged.

                            Switching to this valve configuration with disconnected valves will insure NOT  venting of the vacuum envelope by accidental glitching voltage drop or computer malfunction.

RESET  v1Vac1 .........in 2-3 minutes........ ( v1Vac1 - 2 )  the vac control screen started reading pressures & position

                    Connected cables to valves (meaning: valve will open if it was open before it was disconnected and it will be control able from computer ) in the following order: V4, V1 power, V5, VA6, VC2 & VC1 power,      at ifo 2E-5 Torr-it.....

                     ....vac configuration is reading VAC NORMAL,

                     ifo 7.4E-6 Torr-it

We have to hook up the it-cold cathode gauge to be monitored - logged !  this should be the substitute for the out of order CC1 pressure gauge.

We corrected the MICH locking snap file C1configure_MI.req and saved an updated C1configure_MI.snap. Now the 'Restore MICH' script in IFO_CONFIGURE>!MICH>Restore MICH works. The corrections included adding the correct rows of PD_DOF matrices to be at the right settings (use AS55 as error signal). The MICH_A_GAIN and MICH_B_GAIN needed to be saved as well.

We also were able to get to PRMI SB resonance. PRM was misalgined earlier from optimal position and after some manual aligning, we were able to get it to lock just by hitting IFO_CONFIGURE>!PRMI>Restore PRMI SB (3f).

Yesterday, I had the great pleasure to solder a tiny 4 x 4 mm op amp with 16 legs (AD8336).

I figured out that the best and fastest way to do it is

1. to put solder with the soldering iron on every contact of the electronic board (top side)
2. heat the bottom side of the electronic board with a heat gun
3. use a needle to test if the solder is melted
4. if it is melted place the op amp on the electronic board
5. apply some vertical force on the op amp for proper contact and heat for 1 to 2 more minutes
6. done

This morning the pencil soldering iron of our Weller WD2000M Soldering Station suddenly stopped working and got cold after I turned the station on. The unit's display is showing a message that says "TIP". i checked out the manual, but it doesn't say anything about that. I don't know what it means. Perhaps burned tip?

Before asking Steve to buy a new one, I emailed Weller about the problem.

 There should be a supply of extra tips in the Blue Spinny Cabiney (I can never remember it's French name....)  The drawer is something like the top row of one of the bottom sets of drawers.  You can pick the shape of tip you want, and stick it in.

Albeto and Koji

We took the tip replacement from the blue tower.

I am looking at http://www.cooperhandtools.com/brands/weller/ for ordering the tips.

The burnt one seems to be "0054460699: RT6 Round Sloped Tip Cartridge for WMRP Pencil" We will buy one.

The replaced one is "0054460299: RT2 Fine Point Cartridge for WMRP Pencil" We will buy two.

I like to try this: "0054460999: RT9 Chisel Tip Cartridge for WMRP Pencil" We will buy one.

Q and Ignacio were taking a second look at the Pentek interface board which we're using to acquire the POP QPD, ALS trans, and MCF/MCL channels. It has a differential intput, two jumper able whitening stages inside and some low pass filtering.

I noticed that each channel has a 1.5 kHz pole associate with each 150:15 whitening stage. It also has 2 2nd order Butterworth low pass at 800 Hz. Also there's a RF filter on the front end. We don't need all that low passing, so I started modifying the filters. Tonight I moved the 800 Hz poles to 8000 Hz. Tomorrow we'll move the others if Steve can find us enough (> 16) 1 nF SMD caps (1206 NPO).

After this those signals ought to have less phase lag and more signal above 1 kHz. Since the ADC is running at 64 kHz, we don't need any analog filtering below 8 kHz.

Solid works 2010 was installed to m3, an windows machine in the control room.

Have fun !

Each bottle has matched seals. They are not interchangeable.

It is critical that the solvent do not reach the rubber bulb. Practice with the pipet.

In case of solvent touching the suction bulb: do not let the solvent go back into the bottle! Remove bulb, let it dry out and rinse pipet.

It is essential that the solvent bottle must be rinsed and refilled if it's content met with the rubber bulb.

Use glass syringe with SS needle in critical application: Hamilton ~0.1 ml

I ran the ASS/ADS for the arms because the X-arm was way out. There was also some problem with its locking due to bad ramps in FM2. I copied over the filters from YARM and then adjusted some of the ramps and thresh trigs in the filter file until the transients in POX got smaller. Basically, you should not really be ramping on Integrators. Secondly, we should do some testing when adjusting the filter parameters.

I hooked up the 4395 to the MC servo board OUT2 so that we can monitor the error point when the PCDRIVE goes nuts.

Somehow some DAQ channels for C1SUS have disappeared from the DAQ channel list.

Indeed there are only a few DAQ channels listed in the C1SUS.ini file.

I ran the activateDQ.py and restarted daqd.

Everything looks okay.  C1SUS and C1PEM were restarted because they became frozen.

I found again the ini files had been refreshed.

I ran the activateDQ.py script (link to the script wiki page) and restarted the daqd process on fb.

The activateDQ.py script should be included into the recompile or rebuild scripts so that we don't have to run the script everytime by hands.

I am going to add this topic on the CDS todo list (wiki page).

• with WFS off, I use MC_ALIGN screen to get erasonable power and better cenetering on mc2 QPD
• With good alignment, I unlocked MC and aligned DC beams on WFS
• Re-lock IMC.
• Turn on WFS w OLD matrix.
• Let's see how it goes Mon Oct 9 15:18:20 2023

It still works after a couple of hours. Around 6 PM I did some small alignment changes to bring the control signals closer to zero.

So my conclusion is that it wasn't a matrix issue, but just that the alignment was so off that we were out of the linear range or maybe just off the MC2 QPD. Hopefully it behaves well tonight.

Since the MC_TRANS_PIT and MC_TRANS_YAW control signals were still small even though the sum was ok, I found the C1:IOO-MC_TRANS_PIT/YAW FM5 error point filter which was a +80 dB compensation of the -80 dB C1:IOO-MC2_TRANS_PIT/YAW servo filter was off. This is probably a result of poorly updated snap files so burt missed it after recent model restarts. I will keep an eye on the WFS in the short-term. Attachments #1-3 show the relevant filter locations.

Found some LSC scripts didn't run on pianosa. Particularly all the scripts on the C1:IFO_CONFIGURE screen don't run.

They need to be fixed.

When I came to the 40m, I found most of the FB signals are dead.

The suspensions were not dumped but not too much excited. Use watchdog switches to cut off the coil actuators.

Restarted mxstream from the CDS_FE_STATUS screen. The c1lsc processes got fine. But the FB indicators for c1sus, c1ioo, c1iscex/y are still red.

Sshed into c1sus/ioo, run rtcds restart all . This made them came back under control.

Same treatment for c2iscex and c1iscey. This made c1sus stall again. Also c1iscey did not come back.

At this point I decided to kill all of the rt processes on c1sus/c1ioo/c1iscex/c1iscey to avoid interference between them.
And started to restart from the end machines.

c1iscex did not come back by rtcds restart all.
Run lsmod on c1iscey and found c1x05 persisted stay on the kernel. rmmod did not remove the c1x05 module.
Run software reboot of c1iscey. => c1iscey came back online.

c1iscey did not come back by rtcds restart all.
Run software reboot of c1iscex. => c1iscex came back online.

c1ioo just came back by rtcds restart all.

c1sus did not come back by rtcds restart all.
Run software reboot of c1sus => c1sus came back online.

This series of restarting made the fb connections of some of the c1lsc processes screwed up.
Run the following restarting commands => all of the process are running with FB connection.
rtcds restart c1sup
rtcds restart c1ass
rtcds restart c1lsc

Enable damping loops by reverting the watchdog switches.

All of the FE status are green except for the c1rfm bit 2 (GE FANUC RFM CARD 0).

First, things that were done:

• I was troubled by the odd-looking noise in the EOM temperature signal, so I investigated the circuit with a probe and found that there was quite a bit of line pickup, which I traced to the wires going to and from the RTD (if I placed a dummy resistor directly on the board, it went down markedly).
  • I put a 3-Hz RC LPF between the AD620 and the driver input buffer, which reduced the line noise significantly
  • The error signal looks much cleaner and there are no longer strong peaks in the error spectrum at ~1+ Hz and harmonics
• I had tried earlier to increase the gain of the servo at the driver input stage. It seemed to stay stable. Since I knew the error signal  with the loop closed was at the level of the ADC noise, I decided to push my luck with increasing the servo gain and juice up the AD620 gain from 100 to 990.
  • The servo stayed stable and the error signal level is now manageable.

Things that I noticed:

• With the latest increase in gain, I measured that the error signal was suppressed with the loop closed (the suppression is below ~0.1 Hz, and the reason that the high-frequency level is different is because it has been amplified above the ADC noise by the time of the second trace).

• Despite the above, the Stochmon signals remained unchanged no matter what I did. I noticed that the Stochmon signals, too, were fluctuating basically at bit-level. I terminated the 11-MHz signal and compared it to the normal level---it is not exactly the same, but only a factor of 2-3 lower, which is not great. Of course, the RMS detector is logarithmic, but I think we still want the dark noise to be at least an order of magnitude lower here.
  • I tried to amplify the signal with an SR560, but since the DC level is supposed to be ~1-2 V, I could only get about 2x gain---not enough.

Conclusion

I think there are two things that could be happening here, given the above information:

1. We are limited by the noise of the temperature sensing circuit, which would explain why the in-loop error signal is suppressed while the RAM levels appear not to be. This should be easy enough to test (though there's not enough time right now) with an out-of-loop sensor.
2. The RAM is not dominated by temperature noise here. With the loop open, one would expect to see coherence between the RAM signal and the temperature sensor, if indeed one was the cause of the other. Instead, we see that---while the 11- and 55-MHz signals ARE pretty coherent with each other---there is no appreciable coherence between the temperature and the Stochmon signal.

About the analog CARM control with ALS:

We're looking at using a Sigg designed remotely switchable delay line box on the currently undelayed side of the ALS DFD beat. For a beat frequency of 50MHz, one cycle is 20ns, this thing has 24ns total delay capability, so we should be able to get pretty close to a zero crossing of the analog I or Q outputs of the demod board. This can be used as IN2 for the common mode board. 

Gautam is testing the functionality of the delay and switching, and should post a link to the DCC page of the schematic. Rana and Koji have been discussing the implementation of the remote switching (RCG vs. VME). 

I spent some time this afternoon trying to lock the X arm in this way, but instead of at IR resonance, just wherever the I output of the DFD had a zero crossing. However, I didn't give enough thought to the loop shapes; Koji helped me think it through. Tomorrow, I'll make a little pomona box to go before the CM IN2 that will give the ALS loop shape a pole where we expect the CARM coupled cavity pole to be (~120Hz), so that the REFL11 and ALS signals have a similar shape when we're trying to transition. 

The common mode board does have a filter for this kind of thing for single arm tests, but puts in a zero as well, as it expects the single arm pole, which isn't present in the ALS sensing, so maybe I'll whip up something appropriate for this, too. 

Something odd is happening with the CM board. Measuring from either input to OUT1 (the "slow output") shows a nice flat response up until many 10s of kHz. 

However, when I connect my idependently confirmed 120Hz LPF to either input, the pole frequency gets moved up to ~360Hz and the DC gain falls some 10dB. This happens regardless if the input is used or not, I saw this shape at a tee on the output of the LPF when the other leg of the tee was connected to a CM board input. 

This has sabotaged my high bandwidth ALS efforts. I will investigate the board's input situation tomorrow.

1. Unresponsive c1psl, c1iool0, c1auxey and c1iscaux VME crates were keyed.
2. c1psl channels were burt-restored, did a burtrestore, and re-locked the PMC. Tweaked the pointing into the PMC on the PSL table to increase the PMC transmission from ~0.69 to ~0.71.
3. Re-locked IMC. Ran WFS offset script to relieve the ~100 DAC counts (~10 urad) DC offset from the WFS servos to the IMC suspensions (a serious calibration of this into physical units should be made part of the planned 40m WFS activity). Now that I think about it, since we change the IMC alignment to match the input beam alignment, some post-IMC clipping could modulate the power incident on the ITMs, which is a source of error for the arm cavity loss determination using DC reflection. We need a better normalizing data stream than the IMC transmission.
4. The IFO_OVEREVIEW medm screen was modified such that the threshold for the PMC transmitted beam to be visible was lowered from 0.7 to 0.6, so that now there is a continuous beam line from the NPRO to the PRM when the IMC is locked even when the PMC transmission degrades by 5% due to thermally driven pointing drifts on the PSL table.
5. The wmctrl utility on pianosa wasn't working so well, I wasn't able to use my usual locking MEDM autoconfig scripts. Turned out to be due to a zombie MEDM window which I killed with xkill, now it is working okay again.
6. The misaligned XARM was re-aligned and the loss measuring PDA520 at the AS port was removed from the beam path (mainly to avoid ADC saturations the fringing Michelson will cause).
7. I noticed that the ETMX Oplev HeNe SUM level has degraded to ~50% of its power level from 200 days ago [Attachment #1], may need a new HeNe here soon. @Chub, do we have spare HeNes in stock?

I want to collect some data with the arms locked to investigate the possibility/usefullness of having seismic feedforward implemented for the arms (it is already known to help the IMC length and PRC angular stability at low frequencies). To facilitate diagnostics I modified the file /users/Templates/Seismic/Seismic_vs_TRXTRYandMC.xml to have the correct channel names in light of Lydia's channel name changes in 2016. Looking at the coherence data, the alignment of the cartesian coordinate system of the Seismometers at the ends and the global interferometer coordinate system can be improved.

I don't know if for the MISO filter design if there is any difference in using TRX/TRY as the target, or the arm length control signal.

Data collection started at 1249018179. I've setup a script running in a tmux shell to turn off the LSC enable in 2 hours.

 [Koji and Kiwamu]

 We did some more vacuum works today. It is getting ready to pump down.

 (what we did) 

 - alignment of the POY mirrors. Now the beam is coming out from the ITMY chamber successfully

  - leveling of the tables (except for the IOO and OMC chamber)

  - realigned the beam axis down to the Y arm because the leveling of the BS table changed the alignments.

 - installed IP_POS mirrors

 - aligned the green beam and made it overlap with IR beam path.

 - repositioned green steering mirrors since one of them are too close to the dark beam path

[Rana, Suresh, Kiwanu]

 We did the following things:

   *  taking the VCO stability data from the error signal instead of the feedback

   *  tried calibrating the signal but confused

   *  increased the modulation depth of the green end PDH.

--

 We found that a cable coming out from the VCO box was quite touchy. This cable was used for taking the feedback signal.

When we touched the cable it made a big noise in the feedback. So we decided to remove the cable and take the signal from the error point (i.e. just after the mixer and the LPF.)

In order to correct that signal to the one in terms of the feedback signal, we put a digital filter which is exactly the same as that of the PLL (pole at 1.5 Hz, zero at 40 Hz, G=1) .

However for some reasons the signal shown in the digital side looked completely mis-calibrated by ~ 100. We have no idea what is going on.

Anyway we are taking the data over tonight because we can correct the signal later. The 2nd round data started from AM1:40

What is the point to use the error instead of the feedback? It does not make sense to me.

If the cable is flaky why we don't solder it on the circuit? Why we don't put a buffer just after the test point?

It does not make sense to obtain the error signal in order to estimate the freeruning noise without the precise loop characterization.
(i.e. THE FEEDBACK LOOP TRINITY: Spectrum, Openloop, Calibration)

RA: I agree that feedback would be better because we could use it without much calibration. But the only difference between the "error signal" and the "feedback signal" in this case is a 1.6:40 pole:zero stage with DC gain of 0 dB. So we can't actually use either one without calibration and the gain between these two places is almost the same so they are both equally bad for the SNR of the measurement. I think that Suresh and Kiwamu are diligently reading about PLLs and will have a more quantitative result on Monday afternoon.

[Koji / Kiwamu]

 We did several tests to figure out what could be a source of the computer issue.

The Dolphin switch box looks suspicious, but not 100% sure.

(what we did)

 + Removed the pciRfm sentence from the c1x04 model to disable the Dolphin connection in the software.

 + Found no difference in the Makefile, which is supposed to comment out the Dolphin connection sentences.

   ==> So we had to edit the Makefile by ourselves

 + Did a hand-comilpe by editing the Makefile and running a make command.

 + Restarted the c1x04 process and it ran wihtout problems.

   ==> the Dolphin connection was somehow preventing the c1x04 process from runnning.

 +  Unplugged the Dolphin cables on the back side of the Dolphin box and re-plug them to other ports.

   ==> didn't improve the issue.

 + During those tests, c1lsc frequently became frozen. We disabled the automatic-start of c1lsc, c1ass, c1oaf by editting rtsystab.

  ==> after the test we reverted it.

 + We reverted all the things to the previous configuration.

For the EY, instead of balancing the table, I just moved the weight approximately so that the ETMY OSEMS were at half light, but didn't check the level since ETMY is the only optic.

Some notes on OMC/AS work (Aaron/Gautam can amend/correct):

- Beam is now well centered in OMC MMT. Hits input coupling mirror and cleanly exits the vacuum to the AS table.

- Didn't see much on OMC trans, but PDs are good based on flashlight test.

- just before closing, re-aligned beam in yaw so that it gets close to the east screw on the input coupler. Aaron and I think we maybe saw a flash there with the OMC length PZT being driven at full range by a triangle wave.

- with OMC Undulators (aka tip/tilt PZT mirrors) energized, the beam was low on PZT1 mirror. We pitched ITMY by ~150 micro-rad and that centered the beam on PZT1 mirror. ITMY-OL is probably not better than 100 urad as a DC reference?

- We checked the range of Undulator 1 and we were getting ~5 mrad of yaw of the beam for the full range, and perhaps half of that in pitch. Rob Ward emailed us from Oz to say that the range is robably 2.7 mrad, so that checks out.

Even if the ITMY has to be in the wrong position to get the beam to the OMC, we can still do the heater tests in one position and then do the OMC checkout stuff in the other position.

Gautam suspects that there is a possible hysterical behaviour in the Undulators which is related to the MC3 glitching and the slow machine hangups and also possibly the illuminati.

[aaron]

-We noticed a ghost beam that from MC REFL (MMT2) that should be dumped during the next vent--it travels parallel to the OMC's long axis and nearly hits one of the steering mirrors for OMC refl.

-We measured the level of the table and found it ~3 divisions off from level, with the south end tilted up

-Gautam rotated and slightly translated OM5 to realign the optic, as expected. No additional optics were added.

-Gautam and I tested the TT piezo driver. We found that 3.6V on the driver's input gave 75V (of 150V) at the output, at least for yaw on piezo 1. However, as Gautam mentioned, during testing it seemed that the other outputs may have different (nonzero) offset voltages, or some hysterisis.

Here I show two photos of the latest ABSL (ABSolute Length measurement) setup.

Figure.1 : A picture of the ABSL setup on the AP table.

  The setup has been a little bit modified from the before (#4923).

 As I said on the entry #4923, the way of sampling the ABSL laser wasn't so good because the beam, which didn't go through the faraday, was sampled.

In this latest configuration the laser is sampled after the faraday with a 90% beam splitter.

The transmitted light from the 90% BS (written in pink) is sent to the PSL table through the access tube which connects the AP and PSL table .

FIgure.2: A picture of the ABSL setup on the PSL table.

 The 10% sampled beam ( pink beam in the picture) eventually comes to the PSL table via the access tube (the hole on the left hand side of the picture).

Then the ABSL beam goes through a mode matching telescope, which consists of a combination of a concave and a convex lens.

The PSL laser (red line in the picture) is sampled from a point after the doubling crystal.

The beam is combined at a 50 % BS, which has been setup for several purposes( see for example #3759 and #4339 ) .

A fast response PD (~1 GHz) is used for the beat-note detection.

• When plotting the subtraction performance, we mainly care about the 0.5 - 10 Hz band, so we care about the RMS in this band. Don't integrate over the whole band.
• When calculating the Wiener filter, you must use the pre-weighting so as to not let the Wiener residual be dominated by the out of band signals. We don't want the filter to try to do anything outside of the 0.5 - 10 Hz band.
• Somehow, we want to assign a penalty for the filter to have high frequency gain. We do NOT want to slap on an ad-hoc low pass filter. The point of the Wiener filtering is to make the optimum.
• What is the reason for the poor filter performance from 0.5 - 2 Hz ? If we use the frequency domain (Dmass) subtraction technique, we can do better, so there's some inefficiency in this process.
• we're getting too much of the 3 Hz stack mode coupling into MCL. I think this means that our damping filters should be using RG around the suspension eigenmodes rather than just simple velocity damping. We had this years ago, but it caused some weird interaction with the angular loops...to be puzzled out.

Good progress in IFO locking tonight, with the arm powers reaching about half the full resonant maximum. 

Still to do is check out some weirdness with the OMC DAC, fix the wireless network, and look at c1susvme2 timing.

 {Joe, Kiwamu}

Today we did the following works in order to get ready for the new CDS test.

 - solved the DAC issue.

 - checked all the channel assignments of the ADC and the DAC.

 - preparation for modification of the AA filter chassis. 

 - checked DAC cable length.

 - connected the power cables of the BO boards to Sorensens.

Although we performed those works, we still couldn't do the actual damping tests.

To do the damping tests, we have to modify the AA chassis to let the SCSIs go in it. Now Joe and Steve are working for this issue.

 Also we found that we should make three more 37pin Dsub - 40pin IDC cables.  

But this is not a critical issue because the cables and the connectors are already in our hands. So we can make them any time later.

(Notes)

 (DAC issue)

  Now all the DAC channels are working correctly.  

There had been a combination of some issues.

  When I posted the elog entry last time, the DAC was not working at all (see here).

 But in the last week Joe found that the IO process didn't correctly run. He modified the IOP file named 'c1x02.mdl' and ran it after compiling and installing it.

 This made the situation better because we then were able to see the most of the signals coming out from the DACs.

     However we never saw any signals associated with SIDE_COILs.

 We checked the DAC cards, their slots and their timing signals. But they all were fine.

At that time we were bit confused and spent a couple of days because the DAC signals appeared at a different slot some time after we rebooted the computer. Actually this issue still remains unsolved...

Finally we found that SIDE_COILs had an input matrix which didn't show up in the medm screen.

We put 1 in the matrix and we successfully got the signal coming out from the DAC.

(channel assignments)

We checked all the channel assignments of the DACs and the ADCs.

All the channels are assigned correctly (i.e. pin number and channel name).

(AA chassis)

 We have been planning to put the SCSI cables into the AA chassis to get the ADC signals.

As Joe said in the past entry (see here) , we need a modification on the AA chassis to let the SCSIs go in it.

Joe and Steve will put an extension component so that we can make the chassis wider and eventually SCSI can go in.

(DAC cable length)

 In the default plan we are going to reuse some DAC cables which are connected to the existing systems.

To reuse them we had to make sure that the length of those cables are long enough for the new CDS.

After stopping the watchdogs, we disconnected those DAC cables and confirmed they were long enough.

Now those cables are connected to the original place where they have been before.

The same test will be performed for the binary outputs.

(power cables to Sorensens)

 Since the binary output boards need +/- 15V power, we hooked up the power cables to Sorensens sitting on the new 1X5 rack.

After cabling them, we turned on the power and successfully saw the green LEDs shining on the back panel of the boards.

[Manasa, Yuta]

Since we have setup POP22 PD now(elog #8192), we could confirm that sideband power builds up when PRMI is sideband locked.

Plot:
  Here's some plot of PRC intra-cavity powers and MICH,PRCL error signals. As you can see from POP22, we locked at the peak of 11MHz sideband. There was oscillation at ~500 Hz, but we couldn't optimize the gain yet.



Movie:
  Here's 30 sec movie of AS, POP, REFL when acquiring (and losing) PRMI sideband lock. It was pretty hard to take a movie because it locks pretty seldom (~1 lock / 10 min).



Locking details:
  For MICH lock, we used ITMs instead of BS for reducing coupling between PRCL.
  Also, AS55 phase rotation angle was coarsely optimized by minimizing MICH signal in I.
  For PRCL lock, we used REFL55_I_ERR instead of REFL33_I_ERR. It had better PDH signal and we coarsely optimized phase rotation angle by minimizing PRCL PDH signal in Q.

 == PRMI sideband ==
  MICH: AS55_Q_ERR, AS55_PHASE_R = -12 deg,  MICH_GAIN = -0.1, feedback to ITMX(-1),ITMY(+1)
  PRCL: REFL55_I_ERR, REFL55_PHASE_R = 70 deg, PRCL_GAIN = -15, feedback to PRM

  We set POP22_PHASE_R = -170 deg by minimizing Q.

Issues:
 - We tried to use REFL55_Q_ERR to lock MICH, but couldn't. It looks like REFL error signals are bad.
 - We tried to use POP22_I_ERR to trigger PRCL lock, but it didn't work.

Yesterday (Sep 25) evening: I had to reboot c1psl, c1iool0, and c1aux to recover nominal IMC locking

Today megatron had no response and I had to reboot it with the reset button. MCautolocker and FSSSlow were recovered and the IMC is locking as usual.

Some of the sub-suspension screens need labels to describe what those row and column are.

1) Checked the N2 pressures: the unregulated cylinder pressures are both around 1500 PSI. How long until they get to 1000?

2) The IMC has been flaky for a day or so; don't know why. I moved the gains in the autolocker so now the input gain slider to the MC board is 10 dB higher and the output slider is 10 dB lower. This is updated in the mcdown and mcup scripts and both committed to SVN. The trend shows that the MC was wandering away after ~15 minutes of lock, so I suspected the WFS offsets. I ran the offsets script (after flipping the z servo signs and adding 'C1:' prefix). So far powers are good and stable.

3) pianosa was unresponsive and I couldn't ssh to it. I powered it off and then it came back.

4) Noticed that DAQD is restarting once per hour on the hour. Why?

5) Many (but not all) EPICS readbacks are whiting out every several minutes. I remote booted c1susaux since it was one of the victims, but it didn't change any behavior.

6) The ETMX and ITMX have very different bounce mode response: should add to our Vent Todo List. Double checked that the bounce/roll bandstop is on and at the right frequency for the bounce mode. Increased the stopband from 40 to 50 dB to see if that helps.

7) op340 is still running ! The only reason to keep it alive is its crontab:

op340m:SUS>crontab -l

07 * * * * /opt/rtcds/caltech/c1/burt/autoburt/burt.cron >> /opt/rtcds/caltech/c1/burt/burtcron.log
#46 * * * * /opt/rtcds/caltech/c1/scripts/general/scripto_cron /opt/rtcds/caltech/c1/scripts/PSL/FSS/FSSSlowServo > /cvs/cds/caltech/logs/scripts/FSSslow.cronlog 2>&1
#14,44 * * * * /cvs/cds/caltech/conlog/bin/check_conlogger_and_restart_if_dead
15,45 * * * * /opt/rtcds/caltech/c1/scripts/SUS/rampdown.pl > /dev/null 2>&1
#10 * * * *  /opt/rtcds/caltech/c1/scripts/general/scripto_cron /opt/rtcds/caltech/c1/scripts/MC/autolockMCmain40m >/cvs/cds/caltech/logs/scripts/mclock.cronlog 2>&1
#27 * * * * /opt/rtcds/caltech/c1/scripts/general/scripto_cron /opt/rtcds/caltech/c1/scripts/PSL/FSS/RCthermalPID.pl >/cvs/cds/caltech/logs/scripts/RCthermalPID.cronlog 2>&1

00 0 * * * /var/scripts/ntp.sh > /dev/null 2>&1
#00 4 * * * /opt/rtcds/caltech/c1/scripts/RGA/RGAlogger.cron >> /cvs/cds/caltech/users/rward/RGA/RGAcron.out 2>&1
#00 6 * * * /cvs/cds/scripts/backupScripts.pl
00 7 * * * /opt/rtcds/caltech/c1/scripts/AutoUpdate/update_conlog.cron
00 8 * * * /opt/rtcds/caltech/c1/scripts/crontab/backupCrontab

added a new script (scripts/SUS/rampdown.py) which decrements every 30 minutes if needed. Added this to the megatron crontab and commented out the op340m crontab line. IF this works for awhile we can retire our last Solaris machine.

8) To see if we could get rid of the wandering PCDRIVE noise, I looked into the NPRO temperatures: was - T_crystal = 30.89 C, T_diode1 = 21 C, T_diode2 = 22 C. I moved up the crystal temp to 33.0 C, to see if it could make the noise more stable. Then I used the trimpots on the front of the controller to maximimize the laseroutput at these temperatures; it was basically maximized already. Lets see if there's any qualitative difference after a week. I'm attaching the pinout for the DSUB25 diagnostics connector on the back of the box. Aidan is going to help us record this stuff with AcroMag tech so that we can see if there's any correlation with PCDRIVE. The shifts in FSS_SLOW coincident with PCDRIVE noise corresponds to ~100 MHz, so it seems like it could be NPRO related.

For some reason, my email address is the one that megatron complains to when cron commands fail; since 11:15PM last night, I've been getting emails that the rampdown.py line is failing, with the super-helpful message: expr: syntax error

Yes - my rampdown.py script correctly ramps down the watchdog thresholds. This replaces the old rampdown.pl Perl script that Rob and Dave Barker wrote.

Unfortunately, cron doesn't correctly inherit the bashrc environment variables so its having trouble running.

On a positive note, I've resurrected the MEDM Screenshot taking cron job, so now this webpage is alive (mostly) and you can check screens from remote:

https://nodus.ligo.caltech.edu:30889/medm/screenshot.html

It looks like daqd isn't being restarted, but in fact crashing every hour.

Going into the logs in target/fb/logs/old, it looks like at 10 seconds past the hour, every hour, daqd starts spitting out:

An ELOG search on this kind of phrase will get you a lot of talk about FB transfer problems. 

I noticed the framebuilder had 100% usage on its internal, non-RAID, non /frames/, HDD, which hosts the root filesystem (OS files, home directory, diskless boot files, etc), largely due to a ~110GB directory of frames from our first RF lock that had been copied over to the home directory. The HDD only has 135GB capacity. I thought that maybe this was somehow a bottleneck for files moving around, but after deleting the huge directory, daqd still died at 4PM. 

The offsite LDAS rsync happens at ten minutes past the hour, so is unlikely to be the culprit. I don't have any other clues at this point. 

Today at 5 PM we replaced the east N2 cylinder. The east pressure was 500 and the west cylinder pressure was 1000. Since Steve's elogs say that the consumption can be as high as 800 per day we wanted to be safe.

1. We closed the black valve before the regulator and closed the valve on the cylinder.
2. We unscrewed the brass fill line to the cylinder.
3. We unchained the cylinder and put in the dolly (and attached the chains on there).
4. We rolled in a fresh cylinder from outside using the red dolly (it should have chains).
5. We put it in place, hooked up the chains, and screwed on the brass nozzle with the large adjustable wrench (need to put a non-adjustable here).
6. Opened up the cylinder valve.
7. Opened up the black valve.
8. New east pressure reading is 2500 PSI. Regulated N2 pressure is 68 PSI.

The c1cal model was maxing out its CPU meter so I logged onto c1lsc and did 'rtcds c1cal stop'. Let's see if this changes any of our FB / DAQD problems.

There's a few hours so far after today's c1cal shut off that the summary page shows no dropouts. I'm not yet sure that this is related, but it seems like a clue.

After one day the pressures are east/west = 2200/450 PSI

I think that the real clue was that the dropouts are in some channels and not in others:

https://nodus.ligo.caltech.edu:30889/detcharsummary/day/20150519/psl/

As it turns out, the channel with no dropouts is the RAW PSL RMTEMP channel. All the others are the minute trends. So something is up with the trend making or the trend reading in the cluster.

West cyclinder is empty, east is at 2000 psi; regulated N2 pressure is 64psi. I'll replace the west one after the meeting.

$TARGET_DIR = /cvs/cds/caltech/target

• $TARGET_DIR/c1psl and $TARGET_DIR/c1iool0 moved to $TARGET_DIR/preAcromag_oldVME/
• $TARGET_DIR/c1psl1 moved to $TARGET_DIR/c1psl 
• $TARGET_DIR/c1psl/*.service and $TARGET_DIR/C1_PSL.cmd modified - i executed :%s/c1psl1/c1psl/g in vim.
• $TARGET_DIR/preAcromag_oldVME/c1psl/autoBurt.req and $TARGET_DIR/preAcromag_oldVME/c1iool0/autoBurt.req catenated into $TARGET_DIR/c1psl/autoBurt.req. The first snapshot at 16:19 has been verified.

It remains to (Jon is taking care of these)

• add a line to modbusIOC.service on the new c1psl machine that restores the latest burt snapshot on startup (this necessitated installation of a debian jessie libXp6 package on our debian buster machine because our shared EPICS is soooooooooooooo oooooooold)
• change the hostname from c1psl1 to c1psl
• update martian.hosts

 *  Temporary strain relief for the heliax cables on 1X2 (Steve)

 *  RF diagrams and check lists (Suresh)

      => In the lunch meeting we will discuss the details about what we will do for the RF installation.

 *  Electronics design and plan for Green locking (Aidan / Kiwamu)

      => In the lunch meeting we will discuss the details.

 *  LSC model (Koji)

 *  Video cable session (team)

 * LPF for the laser temperature control (Larisa)