ETMX suspension damping restored.
We gave DRFPMI locking a shot, with the ALS out-of-loop noises as attached. I figured the ALSX noise might be tolerable.
After the usual alignment pains, we got to DRMI holding while buzzing around resonance. Recall that we have not locked since Koji's repair of the LO levels in the IMC loop, so the proper AO gains are a little up in the air right now. There were hopeful indications of arm powers stabilizing, but we were not able to make it stick yet. This is perhaps consistent with the ALSX noise making things harder, but not neccesarily impossible; we assuredly still want to fix the current situation but perhaps we can still lock.
On a brighter note, I've only noticed one brief EPICS freeze all night. In addition, the wall StripTools seem totally contiuous since ~4pm, whereas I'm used to seeing some blocky shapes particularly in the seismic rainbow. Could this possibly mean that the old WiFi router was somehow involved in all this?
While carrying out my end-table power investigations, I decided to take a quick look at the out-of-loop ALSX noise - see the attached plot. The feature at ~1kHz seems less prominent (factor of 2?) now, though its still present, and the overall noise above a few tens of Hz is still much higher than the reference. The green transmission was maximized to ~0.19 before this spectrum was taken.
We managed to access the trends for the green reflected and transmitted powers from a couple of months back when things were in their nominal state - see Attachment #2 for the situation then. For the X arm, the green reflected power has gone down from ~1300 counts (November 2015) to ~600 counts (january 2016) when locked to the arm and alignment is optimized. The corresponding numbers for the green transmitted powers (PSL + End Laser) are 0.47 (November 2015) and ~0.18 (January 2016). This seems to be a pretty dramatic change over just two months. For the Y-arm, the numbers are: ~3500 counts (Green REFL, Nov 2015), ~3500 counts (Green REFL, Jan 2016) ~1.3 (Green Trans, Nov 2015), ~1 (Green Trans, Jan 2016). So it definitely looks like something has changed dramatically with the X-end setup, while the Y-end seems consistent with what we had a couple of months ago...
Last week, Eric and I noticed that the green transmission levels at the PSL table seem much lower now than they did a month or two ago. To investigate this, I attempted to reproduce a power budget for the X endtable setup - see the attached figure (IR powers measured with calorimeter, green powers measured with Ophir power meter). A summary of my observations:
It is worth noting that two years ago, the IR power from the AUX-Y laser was ~280 mW, so we should still be getting "enough" green power for ALS?
Just an other local earthquake 3.6 Mag Ludlow, Ca
No obvious sign of damage
Just not just pedagogical ! Freq domain MISO coherence based subtraction estimation is much faster than calculating MISO WF. And since each bin is independent of each other, this gives us an estimate of how low the noise can go, whereas the Wiener filter is limited by Kramers-Kronig. We should be able to use this on the L1 DARM channel to do the noise hunting as well as estimating the subtraction efficacy of the pseudo channels that you and Rory come up with.
If you can code up a noise hunter example using DARM + a bunch of aux channels, we could implement it in the summary pages code.
The anticlimatic resolution to my subtraction confusion: Spectral leakage around 1Hz. Increasing the FFT length to 256 sec now shows that the FIR WF pretty much achieves the ideal subtraction.
If nothing else, it's good to have worked out how MISO coherence works.
No obvious sign of damage.
We checked the UGF of the AUX X PDH servo, found a ~6kHz UGF with ~45 degree phase margin, with the gain dial maxed out at 10.0. Laser current is at 1.90, direct IR output is ~300mW.
We recovered ALS readout of IR-locked arms. While the GTRX seemed low, after touching up the beam alignment, the DFD was reporting a healthy amount of signal. ALSY was perfectly nominal.
ALSX was a good deal higher than usual. Furthermore, there's a weird shape around ~1kHz that I can't explain at this point. It's present in both the IR and green beats. I don't suspect the DFD electronics, because the Y beat came through fine. The peak has moderate coherence with the AUX X PDH error signal (0.5 or so), but the shape of the PDH error signal is mostly smooth in the band in which the phase tracker output is wonky, but a hint of the bump is present.
Turning the PDH loop gain down increases the power spectrum of the error signal, obviously, but also smoothens out the phase tracker output. The PDH error signal spectrum in the G=10 case via DTT is drowning in ADC noise a bit, so we grabbed it's spectrum with the SR785 (attachment #2, ASD in V/rtHz), to show the smoothness thereof.
Finally, we took the X PDH box to the Y end to see how ALSY would perform, to see if the box was to blame. Right off the bat, when examining the spectrum of error signal with the X box, we see many large peaks in the tens of kHz, which are not present at the same gain with the Y PDH box. Some opamp oscillation shenanigans may be afoot... BUUUUUT: when swapping the Y PDH box into the X PDH setup, the ~1kHz bump is identical. ugh
In preparation for tonight's work, I did the following:
On the PSL table:
At the IOO Rack area:
At the X-end:
At the Y-end:
Having done all this, I checked the green transmission levels for both arms (PSL green shutter closed, after running ASS to maximize IR transmission). GTRY is close to what I remember (~0.40) while the best I could get GTRX to is ~0.12 (I seem to remember it being almost double this value - maybe the alignment onto the beat PD has to be improved?). Also, the amplitudes of the beatnotes on the network analyzer are ~-50dBm, and I seem to remember it being more like -25dBm, so maybe the alignment on the PD is the issue? I will investigate further in the evening. It remains to measure the OLTF of the X-end PDH as well.
I took several measurements today using the revised PLL scheme of using the Marconi just as an LO, and actuating on the Laser PZT to keep the PLL locked (I will put up a sketch soon). On the evidence of the attached plots (spectra of PLL control signal), I guess we can conclude the following:
Attachment #2: Measured OLG of PLL for the PSL+X and PSL+Y combinations. The UGF in both cases looks to be above 100 kHz, so I didn't do any calibration for the spectra attached. The gain on the SR560 was set to 200 for all measurements.
Attachment #3: Measured spectra of PLL control signal for various diode currents, with one reading from the PSL+Y combination plotted for comparison. When we took some data last night, Eric noted that there was a factor of ~6 increase in the overall frequency spectrum level at higher currents, I will update the plots with last night's data as well shortly. I found it hardest to keep the PLL locked at a diode current of 2.00 A across all measurements.
Attachment #4: Measured spectra of PLL control signal at two different crystal temperatures. There does not seem to be any significant dependance on temperature, although I did only do the measurement at two temperatures.
Attachment #4 Attachment #1: All the data used to make these plots (plus some that have yet to be added to the plots, I will update them).
Unrelated to this work:
When I came in this afternoon, I noticed that the PMC was unlocked. The usual procedure of turning the servo gain to -10dB and playing around with the DC output adjust slider on the MEDM screen did not work. Eric toggled a few buttons on the MEDM screen after which we were able to relock the PMC using the DC output adjust slider.
Air condition maintenance is happening. It should be done by 10am
PMC locked and ETMX suspension damping restored.
Gautam will soon follow up with detailed analysis, but here is a brief summary of some of our activities and findings.
Please note that there is a long BNC cable still laid out from the IOO rack area to the X end table; watch your step!
EDIT 01/12/2016 6PM: I've updated the plots of the in-loop spectra such that they are calibrated throughout the entire domain now. I did so by inferring the closed-loop transfer function (G/(1-G)) from the measured open-loop transfer function (G), and then fitting the inferred TF using vectfit4 (2 poles). The spectra were calibrated by multiplying the measured spectra by the magnitude of the fitted analytic TF at the frequency of interest.
EricQ brought back one of the Marconis that was borrowed by the Cryo lab to the 40m today (it is a 2023B - the Marconi used for all previous measurements in this thread was 2023A). Koji had suggested investigating the frequency noise injected into the PLL by the Marconi, and I spent some time investigating this today. We tried to mimic the measurement setup used for the earlier measurements as closely as possible. One Marconi was used as a signal source, the other as the LO for the PLL loop. All measurements were done with the carrier on the signal Marconi set to 310MHz (since all our previous measurements were done around this value). We synced the two Marconis by means of the "Frequency Standard" BNC connector on the rear panel (having selected the appropriate In/Out configurations digitally first). Two combinations were investigated - with either Marconi as LO and signal source. For each combination, I adjusted the FM gain on the Marconi (D in the plot legends) and the overall control gain on the SR560 (G in the plot legends) such that their product remained approximately constant. I measured the PLL OLG at each pair to make sure the loop shape was the same throughout all trials. Here are the descriptions of the attached plots:
Attachment #1: 2023A as LO, 2023B as source, measured OLGs
Measured OLG for the various combinations of FM gain and SR560 gain tested. The UGF is approximately 30kHz for all combinations - the exceptions being D 1.6MHz, G=1e4 and D=3.2MHz, G=1e4. I took the latter two measurements just because these end up being the limiting values of D for different carrier frequencies on the Marconi.
Attachment #2: 2023A as LO, 2023B as source, measured spectra of control signal (uncalibrated above 30kHz)
I took the spectra down to 2Hz, in two ranges, and these are the stitched versions.
Attachment #3: 2023B as LO, 2023A as source, measured OLGs
Attachment #4: 2023B as LO, 2023A as source, measured spectra of control signal (uncalibrated above 30kHz)
So it appears that there is some difference between the two Marconis? Also, if the frequency noise ASD-frequency product is 10^4 for a healthy NPRO, these plots suggest that we should perhaps operate at a lower value of D than the 3.2MHz/V we have been using thus far?
As a quick trial, I also took quick spectra of the PLL control signals for the PSL+Aux X and PSL+Aux Y beat signals, with the 2023B as the LO (Attachment #5). The other difference is that I have plotted the spectrum down to 1 Hz (they are uncalibrated above 30Hz). The PSL+Y combination actually looks like what I would expect for an NPRO (for example, see page 2 of the datasheet of the Innolight Mephisto) particularly at lower frequencies - not sure what to make of the PSL+X combination. Also, I noticed that the amplitude of the PSL+Y beatnote was going through some large-amplitude (beat-note fluctuates between -8dBm and -20dBm) but low frequency (period ~10mins) oscillations. This has been observed before, not sure why its happening though.
More investigations to be done later tonight.
With the Y end laser, I was able to lock the PLL with a lower actuation range (1.6MHz/V), and with the PSL in both the free-running and MCL locked configurations.
I took spectra (attached) with the same actuation range (3.2 MHz/V) for the AUX X+PSL and AUX Y+PSL combinations (PSL shutter closed) just to keep things consistent. It looks like there is hardly any difference between the two combinations - could the apparent factor of 3 worse performance of the X end laser have been due to different actuation ranges on the Marconi?
I've not managed to take a spectrum for the proposed replacement Lightwave laser on the PSL table, though with Eric's help, I've managed to find the beatnote (at a temperature of 53.0195 degrees). I had to do some minor alignment tweaking for this purpose on the PSL table - the only optics I touched were the ones in the pink beam path in attachments 1 and 2 in this elog (the setup used to make the measurement is also qualitatively similar to attachment 3 in the same elog, except for the fact that we are feeding back to the Marconi and not the laser - a detailed sketch with specific components used will be put up later). I'll try and measure the frequency noise of this laser as well over the weekend and put up some spectra.
With regards to possibly switching out the Lightwave on the PSL table for the (faulty?) Innolight at the X end, I've verified the following:
It remains to characterize the beam coming out from the Lightwave laser and do a mode matching calculation to see if we can use the same optics currently in place (with slight rearrangement) to realize a satisfactory mode-matching solution - I've obtained a beam profiler to do this from Liyuan and have the software setup, but have yet to do the beam scan - the plan is to do this on the SP table, but we've put off moving the Lightwave laser off the PSL table until we (i) establish conclusively that the X end laser is malfunctioning and (ii) check the frequency nosie of the Lightwave relative to the Aux lasers currently at the ends.
The area around the Marconi is in a little disarray at the moment with a bunch of cables, SR560s, analyzers etc - I didn't want to disconnect the measurement setup till we're done with it. I have however turned both IR beat PDs on the PSL table off, and have reconnected the Marconi output to the Frequency Generation Unit and have set the carrier back to 11.066209MHz, +13dBm.
I configured a new wifi bridge for a GPIB Instruments.
The some facts are described on https://wiki-40m.ligo.caltech.edu/Network
The setting up wasn't so straight forward. I added more details there as a linked page.
One thing I had to do with the martian wifi router was that I had to separate the name of SSIDs for 2GHz and 5GHz networks.
Now the data download from Agilent is super fast!
The first establishing the connection takes the most of the time, and the data transfer takes literary nothing.
controls@pianosa|netgpibdata > time ./netgpibdata -i 192.168.113.167 -d AG4395A -a 10 -f meas01
Connecting to host 192.168.113.167, GPIB 10...
Data will be written into meas01.dat.
Parameters will be written into meas01.par.
Writing measurement data to file...
Writing to the parameter file.
Unless this is the limit from the way you guys set up the PLL, it seems like there's no difference between the two lasers that's of any import. So then the locking problem has been something else all along - perhaps its noise in the X-PDF lock somehow? PDH box oscillations?
Here are some results from measuring the PSL / AUX Y beat.
With the Y end laser, I was able to lock the PLL with a lower actuation range (1.6MHz/V), and with the PSL in both the free-running and MCL locked configurations. (In the latter, I had to do a bit of human-turning-knob servo to keep the control signal from running away). I also took a spectrum with the marconi detuned from the beat frequency, to estimate the noise from the PD+mixer+SR560.
It looks like the AUX X laser is about 3 times noisier than the Y, though the Y laser looks more like a 10^5 noise-frequency product, whereas I thought we needed 10^4.
Gautam is investigating the PSL / AUX PSL beat with Koji's setup now.
The next step is to compare this data with the same measurement with the PSL and the AUX laser on the PSL table (or the end Y laser). If these show a lot lower noise level, we can say 1) the x-end laser is malfunctioning and 2) the y-end and AUX laser on the PSL are well low noise.
Here is some of the promised data. As mentioned, changing diode current and crystal temperature didn't have much effect on the frequency noise spectrum; but the spectrum itself does seem too high for our needs.
At each temperature, we started measuring the spectrum at 1.8A, and stepped the current up, hoping to reach 2.0 A.
At 47.5 C, we were able to scan the current from 1.8 to 2.0 A without much problem. At 49.0C, the laser mode would hop away above 1.95A. At 50.4C it would hop away above 1.85A. The spectra were not seen to change when physically disconnecting the PZT actuation BNC from the rear of the laser.
The flattening out at the upper end is likely due to the SR560 output noise. I foolishly neglected to record the output spectrum of it, but with the marconi external modulation set to 3.2MHz/V, the few Hz/rtHz above 20k translates to a signal on the order of uV/rtHz, which seems reasonable.
Data and code attached.
The puzzle continues...
I found some reference for computing "multicoherence," which should properly estimate the potential MISO subtraction potential in situations where the witness channels themselves have nontrivial coherence. Specifically, I followed the derivations in LIGO-P990002. The underlying math is related to principal component analysis (PCA) or gram-schmidt orthogonalization.
This produced the following results, wherein the Wiener subtraction is still below what the coherences predict.
I've attached the data and code that produced this plot.
Brief summary of tonights work:
Our "requirement" for the end laser is as follows: We expect to (and have in the past) achieved ALS sensitivity of 1Hz/rtHz at 100 Hz. If the end PDH loop is 1/f from 100Hz-10kHz, then we have 40dB of supression at 100Hz, meaning the free running AUX laser noise should be no more than 100Hz/rtHz at 100Hz.
So, if we expect both the PSL and AUX lasers to have this performance when free running, we would get the green curve below. We do not.
I'll post more details about the exact currents, temperatures and include calibrated plots for the >30kHz range later. Here's the OLG for kicks.
The new wifi router, a Netgear R6400, has been installed, next to the old one which is disconnected (but not yet removed).
Same SSID, and I've added only the wireless MAC addresses of viviana, paoloa and asia, the three thinkpads inside.
Qualitatively, dataviewer at the X end seems pretty snappy. I'll do some more quantitative comparison of the two routers at some point soon. I will update the wiki, too.
Projector light bulb ordered
[ Manasa, Ericq and Steve ]
Vivitek D952HD with 186 hours installed.
IFO restored after 4.5 Mag Banning, Ca earthquake.
In the modern times, people use glue traps to catch rats instead of springs. They are less hazardous to people and don't spread rat fluid on the floor.
Turning on the MCL path (in addition to the MCL FF we always have on) let me lock the PLL for multiple seconds, but low frequency excursions still break it in the end. I was able to briefly observe a level of ~50Hz/rtHz at 1kHz, which may or may not be real. Tomorrow we'll send the PLL control signal to MC2, which should lock it up just fine and give us time to twiddle laser diode current, measure the PLL loop shape, etc.
Please look around when working close to these five locations. Use flashlights or leave lights on.
These mechanial traps are HAZARDOUS !
No visitors or tours till Monday, Jan 11 2016
The problem here is that the MC displacement noise is leading to large frequency excursions of the PSL beam. Options
Our janitor confirmed that Q was not hallucinating about this animal. The dropping size indicating a good size one in the IFO room.
One of the mechanical traps moved from the control room to the east arm, close to the " machine shop " door.
I'm going to get more traps.
Two mechanical and two sticdky traps were set to catch univited visitor.
Absolutely no food or food remains into inside garbage cans!!!!!!!!!!!!!!!!!!!!!!!!!
A small rat / large mouse just ran through the control room. Ugh.
We set out to lock a marconi to the IR fiber beat of PSL + AUX X to measure some frequency noise, and failed.
In short, the Marconi's 1.6MHz max external FM isn't enough oomph to stabilize the PLL error signal. It's actually evident on the Agilent that the beat moves around a few times more than that, which I should've noticed sooner... We could briefly "lock" the PLL for a few tenths of a second, but weren't able to get a spectrum from this.
We also tried using the digital phase tracker temperature servo for some help at ~DC; this worked to the extent that we didn't have to twiddle the Marconi carrier frequency to stay on top of the fringes as the beat wandered, but it didn't otherwise stabilize the beat enough to make a difference in locking the PLL.
I suppose one more thing to try is to lock the PSL laser itself to each AUX laser in turn via PLL, and look for different / excess noise.
The Green and IR beat electronics are a in a little bit of disarray at the moment, but it's not like anyone else is going to be using them for the time being...
I've re-measured the noise breakdown for the Y-end AUX PDH system. Spectra are attached. I've also measured the OLTF of the PDH loop, from which the UGF appears to be ~8.5kHz.
As Eric and Koji pointed out, the spectra uploaded here were clearly wrong as there were breaks in the spectra between decades of frequency. I redid the measurements, this time being extra careful about impedance mismatch effects. All measurements were made from the monitor points on the PDH box, which according to the schematic found here, have an output impedance of 49.9 ohms. So for all measurements made using the SR785 which has an input impedance of 1Mohm, or those which had an SR560 in the measurement chain (also high input impedance), I terminated the input with a 50ohm terminator so as to be able to directly match up spectra measured using the two different analyzers. I'm also using my more recent measurement of the actuator gain of the AUX laser to convert the control signal from V/rtHz to Hz/rtHz in the plotted spectra.
As a further check, I locked the IR to the Y-arm by actuating on MC2, and took the spectrum of the Y-arm mirror motion using the C1CAL model. We expect this to match up well with the in-loop control signal at low frequencies. However, though the shapes seem consistent in Attachment #2 (light orange and brown curves), I seem to be off by a factor of 5- not sure why. In converting the Y-arm mirror motion spectrum from m/rtHz to Hz/rtHz, I multiplied the measured spectrum by , which I think is the correct conversion factor (FSR/(0.5*wavelength))?
I redid this measurement and have now determined the actuator gain to be 4.61 +/- 0.10 MHz/V. This is now pretty consistent with the expected value of ~5MHz/V as reported here.
I made the following changes to the old methodology:
I also took spectra of the phase tracker output and error signal to make sure I was choosing my excitation frequencies in regions where there were no peaks already present (Attachment #1).
The scatter of measured actuator gains at various excitation frequencies is shown in Attachment #2.
We changed the password for controls on nodus this afternoon. We also zeroed out the authorized_keys file and then added back in the couple that we want in there for automatic backups / detchar.
Also did the recommended Ubuntu updates on there. Everything seems to be going OK so far. We think nothing on the interferometer side cares about the nodus password.
We also decided to dis-allow personal laptops on the new Martian router (to be installed soon).
Suspensions recovered after 4.4 Mag EQ
Suspensions are recovered after 4.2 Mag earth quake. No obvoius sign of damage.
PMC locked manually and PRM sus damping restored.
Fire alarm went off several minutes ago. Talked to security and they said there was no fire. It beeped twice again just now. No one has been working on the IFO today.
The EOM upstream of the PMC is used as the phase corrector for the FSS/IMC servo. It is also used to apply the 35.5 MHz PDH RF sidebands for the PMC locking. There is a Pomona box which is used to merge the two signals onto a single cable for the EOM.
Does this circuit make sense to anyone?
This LHO log indicates that EPICS slow down could be due to NFS activity. Could we make some trend of NFS activity on Chiara and then see if it correlates with EPICS flatlines?
I wonder if our EPICS issues frequency is correlated to the Chiara install.
However, the one that was recovered from the ATF lab is no longer locking to the Rubidium reference frequency, although it was locked at the time we disconnected it from the ATF lab. I emailed the support staff at SRS, who seem to think that either the internal oscillator has drifted too far, or the Rb lamp is dead. Either ways, it needs to be repaired. They suggested that I run a check by issuing some serial commands to the unit to determine which of these is actually the problem, but I've been having some trouble setting up the serial link - I will try this again tomorrow.
The Rubidium standard we had sent in for repair and recalibration has come back. I checked the following:
However, I am still having trouble setting up a serial communications link with the FS725 with a USB-serial adaptor - I've tried with a Raspberry Pi and my Mac (using screen to try and connect), and also using one of the old Windows laptops lying around on which I was able to install the native software supplied by SRS (still using the USB-serial adaptor to establish connection though). Could it be that the unit is incompatible with the USB-serial adaptor? I had specifically indicated in the repair request that this was also a problem. In any case, this doesn't seem to be crucial, though it would have been nice for diagnostics purposes in the future...
I've stored the repaired FS725 inside the electronics cabinet (marked "Eletronics Modules") for now (the other unit was returned to Antonio in W. Bridge some weeks ago).
I think there should be a scientifically based aveluation of the ETMY enclosure so we can make the ETMX better.
Meanwhile I'm counting pieces to move on with the south end table cover.
When I came in this afternoon, I saw that the PZT voltage to the PMC had railed. Following the usual procedure of turning the servo gain to zero and adjusting the DC offset, I got the PMC to relock, but the PMCR level was high and the alignment looked poor on the control room monitor. So I tweaked the input alignment on the PSL till I felt it was more reasonable. The view on the control room monitor now looks more like the usual state, and the "REFL (V)" field on the PMC MEDM screen now reads 0.02-0.03 which is the range I remember it being in nominally.
I'll finish up the beat / frequency noise parts of the diagnosis tomorrow later, but I've done some investigation of the AUX X laser RIN.
I placed a PDA255 at one of the rejected beams from the PBS on the downstream side of the IR faraday, making sure the power didn't saturate the PD. I measured the RIN on a SR785, and simultaneously looked at the signal on a 100MHz scope.
The RIN has a very strong dependence on the laser diode current, and no noticable dependence on the crystal temperature or the presence of the PDH modulation / temperature control cables. Here are some traces, note that "nominal" current up until recently was 2.0A.
When adjusting the diode current, a peak beings to appear in the tens of kHz, eventually noticible in the DC power trace on the scope. The point at which this occurs is not fixed.
At all times, I saw a strong intensity fluctuation at around 380-400kHz on the scope whose amplitude fluctuated a fair amount (at least 75mVrms over Vdc=6.5V, but would often be 2 or 3 times that).
I didn't look at the frequency noise while doing this, because the WiFi at the X end was too slow, I'll do more tomorrow in the daytime.
Here's how we should diagnose the EX laser:
I've not updated the MEDM screens to reflect the two new paths yet, but will do so soon. It also remains to install appropriate filters for the servo path that takes the frequency readout as the input.
A few more related changes:
I've set up two IPC channels that take the output from the digital frequency counters and send them to the end front-ends (via the RFM model). A summary of the steps I followed:
I've set things up such that we can select either the "PZT IN" or the frequency counter as the input to the slow servo, via means of a EPICS variable called "FC_SWITCH" (so C1:ALS-X_FC_SWITCH or C1:ALS-Y_FC_SWITCH). If this is 0, we use the default "PZT IN" signal, while setting it to 1 will change the input to the slow servo to be the frequency readout from the digital frequency counter. I've not updated the MEDM screens to reflect the two new paths yet, but will do so soon. It also remains to install appropriate filters for the servo path that takes the frequency readout as the input.
Tangentially related to this work: I've modified the FC library block so that it outputs frequency in MHz as opposed to Hz, just for convenience..