It's may be the janitor's doing.
I noticed that the HEPA filers were off. They are turned on at 20%
Steve should be able to get another copy of the EY doubler mount made up if we really don't have another one sitting in the Manasa end table box which Koji mentioned.
I located the second doubler mount, it was sitting inside a cabinet along the Y-arm. So this will not have to be machined. The doubling oven mount is black in colour.
So as things stand now, the only thing that needs to be machined is a non-green mount for the IR faraday (IO-5-1064-HP) - is it possible to just coat the existing mount with a different color? I've got a drawing for this part ready, but it seems unnecessary to machine the whole thing from scratch when only the color is an issue. Steve was talking about dipping this in some sort of solution and taking the green off. But if this isn't possible, I'll send Steve the drawings tomorrow so that he can place the order with the machine shop...
I will work on the mode-matching calculations over the next couple of days to make sure we have all the mirrors and lenses we need.
Found it locked on TEM01 mode.
Sweets in the fridge for non-PhD holders, courtesy of the highest levels of Caltech.
Johannes found dripping water at the vac rack. It is safe. It is not catching anything. Actual precipitation was only 0.62"
Its not a good idea to use green mounts with green lasers. Steve should be able to get another copy of the EY doubler mount made up if we really don't have another one sitting in the Manasa end table box which Koji mentioned.
I did a quick sweep of the lab to find out what hardware has already been acquired for the X-end table upgrade. The attached PDF is an inventory check in the spirit of this elog.
Some things we have to decide:
I have not gotten around to planning the layout or doing drawings. I will try and first work through a mode-matching solution to make sure we have all the required lenses. It may be that we need some 1" or 2" mirrors as well. The beam from the lightwave NPRO is quite elliptical, but we have a number of cylindrical lenses in hand already if we decide we want to use these, so I guess we don't have to worry about this...
This is quite a preliminary list, and I will add/update over the coming days as I do more detailed planning, but have I missed out anything obvious?
We worked on getting the DRFPMI back up and running, hoping the ALS performance was good enough.
We did succeed in bringing in enough of the AO path to stabilize arm powers > 100, but failed at the full RF DARM handoff.
REFL165 angle was adjusted to -86 to minimize PRCL in the Q signal.
The AS110 signals are mysteriously huger than they used to be. Whitening gain reduced to 15dB from 27dB. Old trigger thresholds are still fine.
The new AUX X laser has a different sign for the temperature-> frequency coupling, so our usual convention of "beatnote goes up when temp slider goes up" meant the ALSX input matrix elements had to change sign.
We think the POPDC PD (which I think is the POP2F PD) may be miscentered, since in PRMI configuration, its maximum does not coincide with the REFLDC minimum, and leaves a sizeable TEM10 lobe on the REFL camera. This was a pain.
Why is the transmission of X green so low? Perhaps you can phase lock the IR and then scan the X frequency, using the X arm as the analyzer. i.e. put a slow ramp into MC2 to pull the PSL frquency and thus the green frequency. You can record a movie of the scan using the framegrabber and record the green transmission peaks to see how big the mode match is exactly (which modes are so big)
Next steps in recovering ALS and trying to lock again
Since I could not determine how many volts at the LO input of the pomona box input corresponds to how many volts at the laser PZT, I measured the transfer function between these points using the Agilent network analyzer. The measured TF suggests that for a function generator output of 2Vpp, we get approximately 75mrad of phase modulation, which compares reasonably well with the value of 120mrad reported here. I did not attempt to further increase the LO output signal to push this number closer to 120mrad, as with 2Vpp from the function generator we get +7dBm at the mixer, which is what it wants - so I wanted to avoid any attenuators etc...
Attachments #2 and #3
After ensuring that we have appreciable phase modulation, I set out to measure the PDH OLTFs and adjust the gain on the uPDH boxes accordingly. The X end gain is at 6.0, and the Y end gain is at 4.0. Before measuring the Y-end OLTF, I adjusted the steering mirrors to increase GTRY to ~0.45. GTRX remains a paltry 0.05... But the UGFs seem satisfactory..
Finally, I took the ALS noise spectrum for the green beats. The beat note amplitudes on the network analyzer in the control room are still puny compared to what we had, -40dBm for Y and -45dBm for X. But the phase tracker Q values are ~1000 and ~3000 for X and Y respectively, which are pretty close to what these were if memory serves me right. There may still be some room for optimization of the PDH loop gains etc, and we could perhaps look at lowering the gain of the REFL PD at the X end? I also have yet to do the sweep for the 3 temperatures at which we can find a beatnote and park at the middle one...
These spectra suggest we could even possibly try locking? We are approximately a factor of 3 above the reference for X and on par with the reference for Y....
Unrelated to this work: I also realinged the PMC, PMC transmission is now 0.730V up from ~0.65V.
We went and looked at the monitor plugged into FB. All kinds of messages were being spammed to the screen (maybe RAM errors), and nothing could be done to interrupt. Sadly, a hard reboot of FB was neccesary.
Video of error messages: https://youtu.be/7rea_kokhPY
After the reboot, it just took a couple of model restarts to get the CDS screen happy.
I came in to check the status of the nitrogen and noticed that the striptool panels in the control room were all blank.
I am leaving things in this state for now. It is unclear why this should have happened, it doesn't seem like there was a power glitch?
I've been a little behind on my elogs so here is an update of the end laser situation.
IR beat for X-end recovered
AM/PM characterization of newly installed Lightwave
Sorry, that was me; taking some photos of the PSL and EX mirrors.
PSL Table doors were open, and the laser shutter was closed.
Doors have been closed, laser has been opened.
All 3 cranes inspected by professional and load tested with 450 lbs at max reach.
The good news: both green beatnotes have now been found. The problem was alignment on the green beat PD on the PSL table which I fixed. They are about -40dBm in amplitude (compare to -25dBm we used to see). But looking at the phase tracker Q output seems to suggest that there is adequate signal...
The bad news: the ALS noise still looks bad (see attachment)- I think the IR beat for the Y was perhaps marginally better. The beat amplitude for the X beat was optimized on the PSL table with the help of the oscilloscope. There may be some headroom for improvement with the Y beat.
I also did the AM/PM measurement for the replaced lightwave, chose an LO frequency based on this, and took the loop OLTF, plots to follow...
Bob cleaned the safety glasses. They were sonicated in warm 2% Liquinox water for 10 minutes. Steve checked them by transmission measurement of 1064 nm at 150 mW
Summary of work done tonight:
[Koji, Johannes, gautam]
With Koji's and Johannes' help, I managed to resolve the coupling the pick-off IR beam into the fiber at the X end. I will put up a more detailed elog about how this was done - but in summary, we have about 31% coupling efficiency into the fiber, which isn't stellar, but I felt this was adequate to find a beatnote. Koji also pointed out that the collimation telescope attached to the fiber at the X-end is poorly mounted - this is something to fix when we swap endtables, but this was not addressed right now because if we were to adjust this, we would also have to adjust the mode matching into the fiber.
I then attempted to tune the temperature to find the IR beatnote. While doing so, I noticed some strange features of the controller - there are essentially two display modes relevant to laser crystal temperature, one which allows us to change the setpoint and one which is an actual readback of the temperature (this one can't be adjusted). While tuning the temperature, I noticed that the latter display ("LT") did not change in value. On a hunch, I disconnected the "SLOW" control BNC on the front panel, and voila, I was able to tune the setpoint and observe the measured temperature shift accordingly. I was thus able to find a reasonably strong IR beatnote (-9dBm) at T ~ 44.6 deg C (the beat PD was set to 0dB attenuation, i.e. high gain mode). However, the moment I reconnected the SLOW control BNC, the beatnote vanished (it gradually shifted out of range of the HP network analyzer), and the same thing happens if I terminate the SLOW control BNC connector! I don't understand this behaviour, as the manual says that the range of voltages accepted to this input is +/-10V, so I would assume 0V means do nothing, but clearly this isn't the case, as the beatnote is being shifted in frequency by > 1GHz, and the tuning coefficient is listed as 5GHz/V in the manual. This situation needs further investigation.
Since I had a reasonable IR beatnote setup, I returned the HP analyzer to the control room and tried to see if a green beatnote was present as well - I first ran ASS, then maximized the green transmission using the PZT mirrors, but no beatnote is evident. The contrast isn't great, the ratio of AUX power to PSL power on the green beat PD is something like 5:1, so this probably requires some tuning as well. I will update this elog after today evening's activities...
The crane inspection is scheduled for this coming Friday from 8-12
Amstron batteries replaced after 11 months with SP-12-5.5HR, 2 years warranty from replaceUPSbattery.com
Batteries replaced after 3.5 years with Amstron AP-1250F2, 8x 12V 6Ah
APC Smart -UPS 2200 model: SUA2200RM2U batteries were replaced by compatible RBC43, 8x 12V5A
Note: the replace battery LED did not go out ( well pasted 24 hrs ) till the self test bottom was hold down for 2-3 sec
I continued the hunt for a green beatnote today - I decided to take the output from the RF amplifiers sitting on the PSL table and directly connect it to the analyzer in the control room while I swept the temperature of the end laser 10,000 counts on either side of a temperature at which I had taken this measurement - so I expect the beatnote should be found somewhere in this neighbourhood. But I did not see any peaks throughout the sweep. I re-checked that the mode overlap onto the BBPD is reasonable. We have considerably less transmitted green power from the arm now than we did before the laser swap (by a factor of ~3) but I still expected to see some sort of beat signal.
It would be handy to have the IR beat set up as well for this process, but as mentioned in a previous elog, I was getting only ~0.1 mW of IR power incident on the coupler at the end table last week. As I had suspected, tweaking the alignment of the steering optics for the pick-off IR beam after the doubler improved the situation somewhat, and I am now getting about 1mW of IR power incident on the coupler at the end table. But I've not been able to adjust the alignment into the fiber at the end such that I get any IR light at the PSL table.
I spent some more time today trying to optimize the modulation frequency and amplitude for the X end PDH, and the alignment/mode-matching of the green to the arm. Some notes:
10:15 power glitch today. ETMX Lightwave and air conditions turned back on
The CDS situation was not as catastrophic as the last time, it was sufficient for me to ssh into all the frontends and restart all the models. I also checked that monit was running on all the FEs and that there was no date/time issues like we saw last week. Everything looks to be back to normal now, except that the ntpd process being monitored on c1iscex says "execution failed". I tried restarting the process a couple of times, but each time it returns the same status after a few minutes.
Chiara reports an uptime of >195 days, so its UPS is working fine
FB, megatron, optimus booted via front panel button.
Jetstor RAID array (where the frames live) was beeping, since its UPS failed as well. The beep was silenced by clicking on "View Events/Mute Beeper" at 192.168.113.119 in a browser on a martian computer. I've started a data consistency check via the web interface, as well. According to the log, this was last done in July 2015, and took ~19 hrs.
Frontends powered up; models don't start automatically at boot anymore, so I ran rtcds start all on each of them.
rtcds start all
All frontends except c1ioo had a very wrong datetime, so I ran sudo ntpdate -b -s -u pool.ntp.org on all of them, and restarted the models (just updating the time isn't enough). There is an /etc/ntp.conf in the frontend filesystem that points to nodus, which is set up as an NTP server, but I guess this isn't working.
sudo ntpdate -b -s -u pool.ntp.org
PMC locking was hindered by sticky sliders. I burtrestored the c1psl.snap from Friday, and the PMC locked up fine. (One may be fooled by the unchanged HV mon when moving the offset slider into thinking the HV KEPCO power supplies need to be brought down and up again, but it's just the sliders)
Mode cleaner manually locked and somewhat aligned. Based on my memory of PMC camera/transmission, the pointing changed; the WFS need a round of MC alignment and WFS offset setting, but the current state is fine for operation without all that.
After the discussion at the meeting today, I decided to try and lock the green by sweeping through PZT dither frequencies in the vicinity of 200kHz without worrying about the AM/PM ratio for now. I was able to lock the PDH loop relatively quickly, at an empirically determined PZT dither frequency of 213.873kHz, 2Vpp (the amplitude was copied from the value at the Y-end). For today's efforts, I borrowed the sum+HPF pomona box from the Y-end, I will make a replica given that we are using Lightwave lasers at both ends now. After adjusting the PZT sliders and lenses on the translational stages at the endtable to maximize the green transmission as best as I could, I was able to get GTRX up to about 0.07 - this is far off from the value of ~0.25-0.3 I seem to remember us having with the old setup, even though we have more green light into the arm cavity. I will take a measurement of the loop transfer function to see what sort of bandwidth we have...
Safety audit went smothly.
Crane inspection is scheduled for March 4
Safety glasses will be measured before April 1
what is next?
Atm 3, Ron Drever could not celebrate with us because of health issues.
Given that we were seeing green flashes in the arms, I tried to see if I could get the green locked to the arm in a nice mode. For a start, I tried hooking up the PDH box and LO using the same settings as was being used previously. However, this did not work. I suppose we will have to do the whole AM/PM measurement for the Lightwave as well before we can determine what would be a suitable frequency for the LO. The AM measurement was relatively straightforward, I just repeated the same steps as detailed here. The two attachments show the AM response (one from 10kHz to 5MHz, the other for a narrower range of 100kHz to 1MHz, both with an excitation amplitude of 0dBm). To see if I could guess some sweetspot for operation, I tried setting the LO frequency to the two marked notch frequencies but was unsuccessful in getting the PDH lock going. At the moment, the alignment for the optics that picks off the IR after the doubler and routes it to the fiber are ccompletely misaligned, I will align these and do the PM measurement tomorrow and then we should conclusively be able to say what the appropriate frequency is to actuate on the PZT.
Unrelated to this work: the KEPCO high voltage power supply that drives the green steering mirror PZTs was switched off - I suppose this has been the case since the power outage last week. I turned it back on and reset it to the nominal settings: Vout = 100V, and Imax_out = 10mA, the driver board is currently drawing ~7mA which I judged to be consistent with the values labelled on the unit.
Looks like another EQ 4.8 took out all the watchdogs, I've restored them, everything looks alright and doesn't look like any magnets got stuck this time...
Good job Q! Give us more details please or play Rachmaninoff Piano Concernto #2
Atm2, EQ 3.5 and 4.1
confirming ITMX is really free
Taking inspiration from J. Lewis et. al, ITMX has been freed.
I tried aligning the green beam, elliptical as it is, to the arm by using the various steering mirrors after the doubling oven. The following was done:
kicks ITMX-UL magnet into stuck position.
Hopefully it is only sticking.
I measured the guralp raw outputs and the TFs using the handheld unit and an FFT analyzer.
The handheld unit was connected to each guralp with the same cable which is confirmed t be functional with the Yend Guralp.
The signal for Z, N, and E directions are obtained from the banana connectors on the handheld unit. Each direction has mass, low gain velocity, and high gain velocity output. The PSDs of the signals were measured with an FFT analyzer. The transfer function from the mass signal to the low/high gain signals were also measured for each direction.
The adjustment screw for the E output of the Xend does not work. I had to tilt the Xend Guralp using the leg screws to bring the E signal to zero.
Attachment 1: Raw voltage PSD for all outputs
Attachment 2: Comparison of the low gain vel outputs
- All of the mass output show similar PSDs.
- Low gain velocity outputs shows somewhat similar levels. I still need to check if the output is really the ground velocity or not.
- High gain velocity outputs are either not high gain, broken, or not implemented.
- We need to calibrate the low gain output using signal injection, huddle test, or something else.
Attachment 3: TFs between each mass output and the low or high gain outputs
- TFs between the mass signal and the low vel signals show the similar transfer functions between the channels.
- The high gain outputs show low or no transfer function with regard to the mass signals.
Eric and I spent some time yesterday night trying to recover the green in the arm after the laser swap. The problem essentially was that though I was getting ~800uW of green out of the doubling oven, the mode wasn't clean, and hence, the beam profile looked really messed up just before entering the arm cavity.We got to a point where we thought we were getting a good mode out of the doubling oven (as judged by propagating this beam onto the wall with the help of a mirror). But we were only getting ~400uW of green power. I tried tweaking the alignment of the oven on the 4 axis stage for a while, but was not able to improve the situation much. So I decided to start from scratch:
I am beginning to wonder if this ellipticity is inherent from the IR beam from the laser? My beamscan results suggest that the beam is more divergent in the "P direction" as compared to the "S direction", which is borne out by these photographs. And if this is indeed the case, do we need to add cylindrical lenses to correct this?
Unrelated to this work: The ITMX Oplev seems to have wandered off so the X arm won't lock. I am not realigning the Oplev for now, but am turning the ITMX Oplev servo off for the night.
Perhaps related to my work on the endtable: The ETMX oplev MEDM readings seemed to be frozen, though there was red light on the QPD on the endtable. Checking the CDS overview screen, I saw that all models on c1iscex had crashed. I sshed into c1iscex and restarted all the models, but the IOP block remained red. I checked the datetime, and found that this was wrong - so I followed the instructions here, but the "Diag Word" block remains red. I am shutting down the watchdog for ETMX and leaving this as is for now... This seems to have happened before...
I manually aligned the IMC. Spot positions are all < 1.5mm. PMC trans of ~0.74, MC2 Trans of ~15400, MC Refl ~0.4, which is better than its been for some time now.
Somehow the WFS DC offsets were off, which made it look like it was impossible to center the beam on WFS2. The script for setting these wasn't working so I fixed it, ran it. WFS and MC2 trans offsets were set, WFS are back on and have been holding MC REFL nice and low for ~3 hours.
Arms were dither aligned, wrote the offsets to SDF files. Oplevs need centering. No further daqd crashes.
daqd has indeed continued to be unstable. I found system times had drifted apart again... I think something weird happened in the booting of the frontends. The monit processes were not running on any of the frontends. I ntpdate'd again, and manually started monit on each fronted via sudo /etc/init.d/monit start.
sudo /etc/init.d/monit start
TP3 drypump replaced after 10,344 hrs at 750 mTorr foreline pressure.
The foreline pressure is 13 mTorr after 8 hrs of running, TP3: 50K rpm, 0.14 Amp with all annuloses pumped.
The annulos pressures are 0.3 - 5 mtorr
Frame builder just crashed again
I was able to realign the arms, lock them, and have run the dither align to maximize IR transmission - looks like things are back to normal now. For the Y-end, I used the green beam initially to do some coarse alignment of the ITM and ETM, till I was able to see IR flashes in the control room monitors. I then tweaked the alignment of the tip-tilts till I saw TEM00 flashes, and then enabled LSC. Once the arm was locked, I ran the dither align. I then tweaked ITMX alignment till I saw IR flashes in the X arm as well, and was able to lock it with minimal tweaking of ETMX. The LSC actuation was set to ETMX when the models were restarted - I changed this to ITMX actuation, and now both arms are locked with nominal IR transmissions. I will center all the Oplev spots tomorrow before I start work on getting the X green back - I've left the ETM Oplev servos on for now.
While I was working, I noticed that frame builder was periodically crashing. I had to run mxstream restart a few times in order to get CDS back to the nominal state. I wonder if this is a persistent effect of the date/time issues we were seeing earlier today?
Pasadena reported the Sunday night event as a power transient caused by the trip of a power transmission line. This affected the entire city. Once the loss was detected, the system automatically switches to an alternate source. It was about one second for the system to recover.
2W Innolight, both Lightwaves at the ends, PSL HEPA, Ref Cavity and 3 AC units turned on.
The 40m vacuum did not trip. It is vacuum normal.
The Jetstore computer is indicating power failer.
Looks like that's the case. LIGO GC also sent an e-mail that there was a popwer glitch.
I came into the 40m a few minutes ago, and noticed the following (approximately in this order):
To investigate further, I checked today's summary pages, and whatever caused this, happened around 730am today morning (approx 5 hours ago). I also saw that all the watchdogs were tripped, except MC3, BS and SRM.
I then tracked down the beeping - I believe that it is coming from Megatron.(in fact, it is coming from the Jetstor..)
I also found that the PSL is OFF, and the Marconi, though ON, has the display parameters set to values that I normally see when it is first turned ON (i.e. the carrier frequency is 1200MHz, the output is -140dBm etc - this is what led me to suspect that somehow the power connection was interrupted? As far as the workstation computers are concerned, I don't think ROSSA was affected, but pianosa is frozen and donatella is at the login screen. The CDS overview MEDM screen refuses to load correctly (though some of the other MEDM screens are working fine). I'm not entirely sure how to go about fixing all of this, so for now, I'm leaving the PSL off and I've shutdown the remaining watchdogs.
It just occurred to me to check the status of the vacuum - the MEDM screen seems to suggest everything is fine (see Attachment #1). I went down to the X end to do a quick check on the status of the turbo pumps and everything looks normal there...
800e-6 / 0.225^2 = 0.016
I thought Kiwamu had roughtly 2%/W.
After carefully tweaking the mode-matching of the IR into the crystal and the four-axis translation stage on which the doubling oven is mounted, I managed to recover 800uW of green power going into the green Faraday. Considering we have ~225mW of IR power coming out of the IR faraday (and roughly that amount going into the SHG crystal), I'd say this is pretty consistent (if not slightly better) with a recent power budget I had made for the X end. The amount of green power we get out of the doubling crystal is very sensitive to the alignment of the crystal to the beam axis. I suspect we could improve the situation slightly if the mode-matching lenses were mounted on translational stages so we could tweak their position, but the current situation on the X endtable does not provide space for this. In any case, I'd say we are at least as good as we were before, and so this should be an adequate fix until the new end-table is installed (though I don't know why we aren't seeing the predicted SHG conversion efficiency of 3-4% as predicted by Kiwamu's calculations, we are getting more like .36% conversion efficiency)...
Because the alignment of the beam before the doubling oven had changed, I had to adjust the steering mirrors to make the green beam go into the green faraday. I had placed a couple of irides for the green beam as a reference of the old path into the arm, and I used these to adjust some of the green mirrors to center the green beam on these. However, I did not observe any flashes in the arm. I will check if we are still mode-matched to the arm, and if the lenses downstream of the doubling oven need to be moved....
It shipped out for repair evaluation.
Arrived to Hayward,CA 2016Feb16
After the discussion at the meeting, I decided to go ahead and open the top of the oven so that I could get a visual on where the crystal was located - this helped in the alignment, and I was able to get some green light out of the oven. I had to tweak the position of the Doubling oven a little (with the top open) in order to align the crystal to the beam axis. However - I was only able to get ~140uW of green light going into the Faraday. I had measured the power at various points along the beam path recently with the old setup. We used to have ~860uW of green going into the Faraday there. To see if I could improve the situation a little, I checked that the beam was reasonably centered on both apertures of the IR Faraday, and then removed the irides upstream of the doubling oven. These were preventing me from placing the lenses exactly as per the a la mode solution. Once the irides were removed, I moved the lenses to their optimal positions as best as I could with a tape measure to mark out distances. I then further tweaked the position of the doubling oven using the 4 axis stage, monitoring the green power while doing so. The best I could get was ~200uW. Perhaps the positions of the lenses need to be optimized further. I also checked the IR power before and after the IR Faraday - these numbers are ~260mW and ~230mW respectively (I maximized the transmitted power through the Faraday by rotating the HWP, the QWP that was in the beam path has now been removed as the Lightwave outputs linearly polarized light), and compare favourably to the numbers in the old setup. Doing a naive scaling accounting for the fact that we have less power going into the doubling crystal, I would expect ~700uW of green light coming out, so it looks like the mode matching into the doubling crystal is indeed sub-optimal. However, now that things are roughly aligned, I hope the optimization will go faster...
Some updates on the laser swap situation:
2. Implementing the new solution:
As I check the manual of the Innolight (pg17) and the datasheet of the Lightwave, I wonder if the Quarter Wave Plate that was placed immediately after the Innolight laser head is even necessary now - I assume the purpose of the combination of QWP+HWP was to turn the elliptically polarized light from the Innolight into linearly polarized light before the Faraday. But the Lightwave already produces linearly polarized light. I will check out what is the configuration on the Y-end table...