Back on Feb 20th (elog 11056) Q replaced all of our oplev parts with the aLIGO version.
Unfortunately, after this it has seemed like there was something not quite right with the optical lever servos.
Since, when the models were changed which gave us an extra underscore in the oplev names, Q did a find-and-replace in the foton text files, I was worried that this might have broken things. I'm not entirely sure how it would have broken them (I didn't see any difference in a diff), but I've heard enough horror stories about the delicacy of the foton text files.
Anyhow, I opened the last archived foton files from just before Q made the change, and copy-and-pasted the design strings from the old filter banks to the new ones. Hopefully this fixes things.
Figuring out the problem with frequency counter readouts:
RF amplifier box:
The frequency counter was setup to read the beat frequency after amplification of the RFPD signals. The output as seen on the frequency counter screen as well as the epics values was intermittent. Also, locking the arms and changing the end laser frequencies did not change the frequency counter outputs. It looks like the RF amplifier is oscillating (Oscillating frequencies were ~107 MHz and 218 MHz) and the input circuit needs to be modified and the connections need better insulation.
As of now, I have connected the RFPD outputs to the frequency counter sans any amplification and we are able to get a readout of the beat frequency at > 40MHz (Frequency counter requires much higher amplitude signals for frequencies lower than that).
The frequency counter has 4 ranges of operation:
Range 1 : 1 - 40MHz
Range 2: 40 - 190MHz
Range 3: 190 - 1400MHz
Range 4: 1400 - 6000MHz
I set up the beat frequency readout in various configurations to troubleshoot and have recorded my observation (attachments). The amplifiers in all cases is just a single ZFL-500LN.
There seems to be a problem with the RF amplifier and the frequency counter when they are set together. I am not able to figure it out and stuck right here
Attachment 1: schematic for readout and corresponding observations
Attachment 2: Oscilloscope screen snapshot for schematic 3
Attachment 3: Spectrum analyzer screen snapshot for schematic 3
More info: RFPD used is Thorlabs FPD310 and frequency counter is Mini Circuits UFC-6000. The RF amplifier has decoupling capacitors soldered across the power supply.
Here is a longer stretch of data, from the first RF-only lock on Saturday night. Unfortunately daqd had died about 400 seconds before the lockloss, so I can't show the RF signals coming on.
ALS was on the _A channels for CARM and DARM, so when those go to zero (about -300 seconds for CARM, and about -200 seconds for DARM), we're using RF signals only for the error signals.
CARM noise definitely improves, but holy smokes does DARM start to look good! Although, right at the end it starts to look like REFL11I is getting bigger. Not sure why, but we'll have to watch out for this.
Here's the equivalent plot for the second lock stretch. This is the one that was handled by the carm_up script. It looks like I had about 150 seconds of RF-only lock here.
DARM error is getting bigger with time jnear the end, even though I wasn't working on alignment here. Zooming in, DARM is oscillating at 16.4 Hz, which is the bounce frequency. I thought I had my bounce/roll filters on, but somehow it still got a little rung up. It just rings up to a steady state though, it's not getting huge, so I don't know that it was the cause of the lockloss.
According to the official rules, we only need 8 seconds to declare it "locked".
I wonder if the double cavity pole compensation filter for CARM was on for all the attempts yesterday? IF it looks like it will not saturate, it would be more stable to have the whitening on for REFL11 / AS55. Since on Friday, I set the REFL165 demod phase just by minimizing the MICH control signal with the arms on resonance, we ought to check out the PRMI degeneracy with the ETMs misaligned.
Speaking of signal mixing: Although we weren't able to get the carrier term cancelled in the 3*f1 signals by the relative mod phase method, I wonder if we can do it by mixing the 3*f1 and 3*f2 signals in the input matrix. Might help to keep the PRMI more stable, if that's an issue.
P.S. I have done some scripts directory / SVN cleanup. Adding some directories that were not in (like lockloss) and then removing stuff from the repo using 'svn rm --keep-local filenames' for the image and data files.
As I (very excitedly) reported in elog 11116, I was able to follow the error signal blending procedure from last night, and get CARM and DARM onto digital non-normalized RF signals. The lock held for about 3 minutes after this transition (elog 11118 has plot of this).
I was then able to script what I did (in the carm_up script), and repeat the transition. Q joined me in the control room, but we have not been able to complete the transition a third time.
Here's the sequence that worked the two times:
After those two attempts, we ran the LSC offset script, since that hadn't been done since early yesterday. We did a quick CARM sweep, and REFL11 seemed to be centered around 0 counts, so we removed the 0.025 count offset from the CARM_B filter bank.
For later attempts, we keep seeing oscillations in the lockloss plots around 50 Hz, as if we're seeing gain peaking at the low side of the phase bubble. We have tried turning off various filters at various levels of RF gain, but none of the combinations seems to be excellent. Turning off the FM6 bounce/roll filter in CARM was particularly bad (immediately lost arm transmitted power), but others weren't good either (eg turning off FM3 boost lost arm powers within a second or so). When we lose arm powers, the RF signals aren't valid, so if you don't turn them off fast enough (and ALS is still on with enough gain), you'll lose the full IFO lock. If you're fast though, you can turn off the CARM and DARM _B outputs and not have to start from scratch.
There seemed to be a very fine line to walk between not enough gain (~50Hz oscillations), and too much gain (200-300Hz oscillations). It has been pretty frustrating later in the evening. We seem to only have about 3dB of gain margin on the low side, when all the boosts are on. Not excellent.
When the RF signals had a moderate amount of gain, but ALS was still holding CARM and DARM, Q checked the phases of REFL11 and AS55 with excitation lines. He rotated AS55 from -55 deg to -30 deg (+25 deg) and REFL11 from 144 deg to 164 deg (+20 deg).
Prior to the all-digital attempts, I tried several times to turn on the AO path, without success. I think that the best that I got was 0dB on the CM board input 1 gain, +14dB on the CM board's AO gain, and -30dB on the MC board's AO gain before the mode cleaner lost lock.
I was hoping that I could get CARM entirely to RF signals, and that would make things more stable and less complicated, and I could try again to turn on the AO path, but we haven't been able to do this tonight.
A few times in the later attempts we tried turning on the UGF servos for CARM or DARM. I'm not sure if the lines kicked things out of lock, or if the UGF servos went a little crazy, or what, but we never survived for more than a few seconds after turning on the excitations.
There is a problem with the optical lever servos. I had thought I'd been seeing it ever since Q re-did the models, and now I'm pretty sure that's what's up. Q is hot on the trail of figuring out what may have changed that shouldn't have. We may want to revert to an old Foton file, and re-copy the old filters into the new filter banks just in case. The watchdog damprestore scripts have been tweaked to clear the oplev filter bank histories before turning on the oplevs, and this seems to solve the symptom of kicking the optic when oplevs are engaged.
Although we haven't been able to make the transition to RF-only a third time, I think we're getting there. Progress has certainly been made in the last 2 days!
This elog will be about work that happened yesterday. I will write a reply to this with work from this evening's success.
Work started with the plan of trying ALS fool, using the new triggering scheme (elog 11114).
The PRMI was having a bit of trouble holding lock with REFL165, so we checked its demod phase. On Monday (elog 11095) we rotated the REFL165 phase from -91 deg to -48 deg while in PRFPMI configuration (I think the -91 was from PRMI-only phase setting). However, Friday night we saw that MICH was super noisy, especially when the CARM and DARM offsets were near zero. Rana rotated REFL165's phase until the MICH noise seemed to get lower (by at least an order of magnitude in the control signal), while we were at zero offset everywhere. We were not driving and looking at any lines/peaks, just the overall spectra. The final REFL165 demod phase is -80.
We tried engaging the fool path with no success.
First, Rana moved the low frequency boost in the MC filter bank from 20:1 to 0.3:0.03. This gave the whole loop at least 20 or 30 degrees of phase at all frequencies below the design UGF (a few hundred Hz? Don't quite remember). To check this, we put in a "plant" filter, and turned on the locking filter (3:3000^2) and the low freq boost and the plant, and the phase never touched 180 at any low freq. This is so that we can ramp on this filter bank's gain without having an unstable unity gain crossing anywhere. Also, I added two +10dB filters to the first two filter modules, so that we could ramp on the gain at the input rather than the output.
Last night we were actuating CARM on MC2 and DARM on the ETMs, and the MC filter bank was set to actuate on MC2. Even with super duper low gain in the MC filter bank, so that the control signal was much less than one (1) count, it would make CARM unhappy. The CARM filter bank's output was doing +/- a hundred or more counts, so why a few tenths of a count mattered, we couldn't figure out. We were using the power trigger for the MC filter bank, but not the zero-crossing trigger. Since the fool tuning was checked while actuating on the ETMs, we wonder if maybe the tuning isn't valid for MC2 actuation? Maybe there's enough of a difference between them that the fool needs to be re-tuned for MC2 actuation? Fool had the complex pair of poles at 1Hz, the "comp1" filter to give phase lag, and a gain of 22.
I think that at some point we even turned off the fool path, but left the MC path on with a little bit of gain, and the audible noise over the speakers didn't seem to change in character at all. Weird.
We ended up leaving the fool path for another time, and started working on error signal blending at the CARM filter bank input. This is pretty similar to Kiwamu's self-locking principle.
Our goal was to ramp up the gain of the RF error signal at low frequency, while letting ALS keep hold of things at higher frequencies.
CARM and DARM sweeps from earlier seemed to indicate that the RF signals are valid without normalization above transmitted powers of 50 or so, so we thought we'd give those a whirl for this error signal blending.
From doing a CARM sweep through resonance, we guessed roughly that the REFL11 (non-normalized) slope was about a factor of 10,000 larger than the ALS slope. We put a 1e-4 into the input matrix element REFL11I -> CARM_B. For some reason, REFL11 seemed to be centered around -250 counts, so we put an offset of +0.025 ( = 250*1e-4) into the CARM_B filter module to compensate for this.
Since we thought that a gain of 1 in the CARM_B filter bank would make it equal to ALS, we tried some lower gains to start with. 0.3 kicked it out of lock, so we ended up liking and using 0.15. With this low gain on, we tried turning on a low frequency boost, 20:1, but that didn't do very much. We turned that off, and instead turned on an integrator, 20:0, which totally made things better. The transmitted arm power was staying higher more of the time.
From a DARM sweep, we thought that AS55Q (non-normalized) should also have an input matrix element of 1e-4 for DARM. We gave DARM_B a gain of 0.1, which seemed good and not too high. Again, trying the gentle boost didn't do much, so we went with the integrator.
At this point, since both RF signals were being used as error signals with integrators, we declared that at least at DC we were on RF signals. Hooray!!
After this, we started increasing the CARM_B gain a little, and decreasing the CARM_A gain. When Rana finally set the CARM_A element to zero, we lost lock. We realized that this is because we didn't include a zero to compensate for the arm cavity pole, which the IR signal will see, but the ALS won't.
We decided that the plan of attack would be to get back to where we were (DC error signals on RF), and try to start engaging the AO path.
I have in my notebook that at 9:49pm CARM was no longer using ALS as an error signal, and at 9:50pm, DARM was no longer using ALS as an error signal. It looks like I was locked for 3+ minutes after getting to RF-only signals.
The increase in power near the end of the lock stretch was me trying to improve the dark port contrast by touching the ETMX alignment. DARM was definitely oscillating as I improved the dark port contrast, so I was trying to hand-lower the gain as I worked on the alignment.
Exciting! How long was it?
[Jenne, with Matt and Fujimi as witnesses]
It might be about time to throw that champagne in the fridge. Nice. Not quite close enough to talk about popping it open, but we'll want it chilled just in case...
I still haven't logged yesterday's work, and I'm still working now, so no details, but I just handed both CARM and DARM over to non-normalized RF signals, and had the arms stable at powers of about 105. I was workinig on the ETM alignment, and the power was increasing, so I think that's where the extra power will come from. I was lowering the DARM gain as I improved the alignment, because the optical gain was increasing so much. I probably just didn't do that fast enough for the last aligning, which is why I lost lock.
Anyhow, here's a plot, because I'm excited:
Last report on model change / screen work from yesterday.
The ALSwatch script has always been just looking at the EPICS output of the CARM and DARM filter banks, and if it saw a single saturation, it would run the down script. This was non-ideal because (a) the EPICS channels aren't the real signals, and (b) sometimes we'll hit the rails briefly and that's okay - we want to shut things down only when we're constantly saturating.
It turns out that there was a pre-existing saturation monitor part in CDS_PARTS, which I have used. There is one each looking at the output of the CARM and DARM filter banks. The threshold for what saturation means is set as an epics input. The part outputs a running count (number of saturations since the last time it was not saturated, resets each time it goes non-saturating) and a total number since the last reset (also an epics input).
(To be continued... still writing)
More work from yesterday.
Rana and I had discussed on Thursday night that we probably want to be able to use the zero crossing of an error signal to trigger a servo on, but not to un-trigger it. So, now the zero crossing trigger is latched, using the power trigger to reset the latch.
Also, the input to the zero crossing trigger is the input to the MC servo, before the triggered switch. This allows us to look at the normalized error signals rather than just the non-normalized ones, if that's what we're trying to lock on. This signal is taken before the triggered switch, so that it's looking at whatever is coming out of the input matrix (including normalization).
So. If the absolute value of the MC error signal goes below the threshold, it outputs a 1, no matter what the arm power is. If the arm power is high, the power trigger also outputs a 1. These are AND-ed together, so only if both are 1 do you actually trigger the MC filter bank. If the zero crossing trigger has been set to 1, it will stay at 1 until the arm power goes low enough to untrigger the power trigger. So, even if you have a little bit of noise on the error signal and it pops above the threshold momentarily, this won't cause the servo to un-trigger.
This is implemented using a "set-reset latch". The output of the latch is the zero crossing trigger, which is AND-ed with the power trigger. This final AND-ing, in addition to doing what we want, solves the ambiguity that is inherent in SR-latches for one combination of inputs.
The trigger screen has been modified to reflect these model changes.
Here's a screenshot of the model, which includes some notes for anyone who opens the model since it's a bit confusing:
This is work that I did yesterday but didn't have time to elog. Since it seems non-trivial to give ourselves ramping matrices, but we only really needed the ramping in the DoF selector matrices, I've replaced the separate _A and _B parts with full filterbanks. Recall from elog 10910 that I had given each degree of freedom's _A and _B input options an offset, an epics monitor and a test point. Now those are removed, and handled inside of the filter banks. The outputs of the filter banks sum together.
This required some screen modifications, but everything should work the same way that it did before this change. I've also changed the DAQ channels from the _A_ERR and _B_ERRs that I had hand-created to now be the _A_OUT and _B_OUT test points from the filter banks (acquired at 2048Hz).
I have not yet modified the burt snapshots for the ifo configure screen. The arms will work the same as always, since they didn't have any selector matrix stuff ever, but the rest still need tweaking.
MC Refl alignment follow up: the alignment from last night seems still good today. We should keep an on the MC WFS DC spots and not let them get beyond 0.5.
The fit FWHM is 10.444kHz +-55Hz.
If we take the FSR from ELOG 9804, this implies an Xarm fineese of 380 +- 2.
Assuming an ITMX transmission of 1.4%, this means an Xarm loss of 240 +- 90ppm.
This is substatially lower than the ~500ppm I had measured via the unlocked/locked ASDC power method, but still pretty high.
Since we were able to get continuous frequency counter values into the digital system, I decided to give it a quick spin with a calibrated single arm ALS scan. This should be repeated when amplifiers are in place, because the Y IR beatnote is wandering around in a way I don't trust and I'm not sure if the frequency counters have good absolute calibration...
Neverthess, I did a 5 minute scan through the Xarm, and fit it nicely to a lorentzian peak.
Since Domenica was not picking up an IP address and hence not responding to pings or ssh even after power cycling, I pulled it out from the IOO rack and connected it to a monitor. After a bunch of hit and trials, we figured out that the problem was related to the power adapter of the Rpi discussed here : http://www.raspberrypi.org/forums/viewtopic.php?f=28&t=39055
The power adapter has been swapped and this issue has been resolved. Domenica has been remounted on the IOO rack but left with the top lid off for the timebeing.
RF amplifier box:
The frequency counter was setup to read the beat frequency after amplification of the RFPD signals. The output as seen on the frequency counter screen as well as the epics values was intermittent. Also, locking the arms and changing the end laser frequencies did not change the frequency counter outputs. It looks like the RF amplifier is oscillating (Oscillating frequencies were ~107 MHz and 218 MHz) and the input circuit needs to be modified and the connections need better insulation.
I was asked by Koji to point out where a schematic of the triple resonant circuit is.
It seems that I had posted a schematic of what currently is installed (see elog 4562 from almost 4 yrs ago!).
(Some transfomer story)
Then I immediately noticed that it did not show two components which were wideband RF transformers. In order to get an effective turns ratio of 1:9.8 (as indicated in the schematic) from a CoilCrfat's transformer kit in the electronics table, I had put two more transformers in series to a PWB1040L which is shown in the schematic. If I am not mistaken, this PWB1040L must be followed by a PWB1015L and PWB-16-AL in the order from the input side to the EOM side. This gives an impedance ratio of 96 or an effective turns ratio of sqrt(96) = 9.8.
(An upgrade plan document)
Also, if one wants to review and/or upgrade the circuit, this document may be helpful:
This is a document that I wrote some time ago describing how I wanted to make the circuit better. Apparently I did not get a chance to do it.
We played around tonight with different possible ways of transitioning DARM to normalized AS55Q. Before each try, we would use ezcaservo (or just eyeball it) to make sure that the normalized RF signals had a mean of zero, so that we knew we were pretty close to zero offset in both CARM and DARM.
We tried something that is similar in flavor to Kiwamu's self-locking technique - we summed in some normalized AS55Q to the DARM error point (using the DoF selector matrix that I created a few weeks ago), and then tried to engage a little low frequency boost. We tried several times, but we never successfully made the transition.
In the end, we just did a direct transition over to normalized AS55Q, and lost lock after several seconds. The buzzing that we hear didn't change noticeably after the transition, which indicates that most of the noise is due to CARM (which makes since, since it has a much smaller linewidth). The problem with holding DARM is that occassionally we will have a CARM fluctuation that lets the arm power dip too low, and DARM's error signal isn't valid at low arm powers. So, we need to work on getting CARM stabilized before we will have a hope of holding on to DARM.
Here's the lockloss plot from that last lock:
Also this evening, I scanned back and forth over the CARM zero crossing while locked on ALS, to see what the RF error signals looked like. Normalized REFL55 seems to have much more high frequency noise near the edges of the linear range than does REFL11. Also, the REFL 11 signal is much larger. So, what I think I want to try to do is use ALS fool to lower the CARM noise by a bit, then make the DARM transition. Then, we can come back to CARM and ramp up the gain.
With these CARM sweeps, I think that I know the relative gain and sign between ALScomm and the normalized REFL signals, and the REFL signals versus the normalized versions. I think that 100*REFL11I/(TRX+TRY) gives the same slope at the zero crossing as just plain REFL11I. Same factor of 100 is true for REFL55I. The REFL11 slope is 20,000 times larger than the ALS slope, while the REFL55 slope is -500 times the ALS slope (note that REFL55 has a minus sign). We can probably trigger the Fool on when the arm powers are above 50, and trigger off when they're below 20. For the zero crossings, the REFL55 threshold should be about 20, and the REFL11 threshold should be about 500.
I also need to re-think the triggering logic for ALSfool. We probably don't want the zero crossing logic to be able to un-trigger the lock, just in case we get an extra noise blip. So, we want to trigger on with an AND, but only trigger off if the arm powers go too low. Also, the zero crossing logic should look at the normalized error signals, not the plain signals.
We need to modify the ALSwatch logic so that it doesn't look at EPICS values for the thresholding. There may be an updated filter module that includes a saturation monitor, but otherwise we can use the saturation monitor part that is in the OSC section of CDS_PARTS. We'll set the threshold on this to match the limiter in the filter bank. Then, if the filterbank output is constantly hitting the limiter, we should run the down scripts.
This has been done before:
Arm length measurements and g-factor estimates in 2012, but only with an accuracy of ~30 cm. However, Yuta was able to get many FSRs somehow.
In the attached plot you can see that the MC REFL fluctuations started getting larger on Feb 24 just after midnight. Its been bad ever since. What happened that night or the afternoon of Feb 23?
The WFS DC spot positions were far off (~0.9), so I unlocked the IMC and aligned the spots on there using the nearby steering mirrors - lets see if this helps.
Also, these mounts should be improved. Steve, can you please prepare 5 mounts with the Thorlabs BA2 or BA3 base, the 3/4" diameter steel posts, and the Polanski steel mirror mounts? We should replace the mirror mounts for the 1" diameter mirrors during the daytime next week to reduce drift.
I think we've seen this in simulations, but it's a little disheartening to see in real life. AS55Q looks like it flattens out pretty significantly right around the DARM=0 point.
Right now I have the arms held on ALS (CARM=-1*MC2, DARM=2*ETMY, as Q used last night), and the PRFPMI is on REFL165I&Q. I have set CARM to be as close to zero offset as I can (so I get all the usual buzzing), and then I'm sweeping the DARM offset between +3 and -3 counts (roughly +/-3nm) with a 3 second ramp and looking at normalized AS55Q. The channel called "DARM_B_ERR" is 0.006*AS55Q/(TRX + TRY). The arm transmissions, as well as the ASDC are plotted as well - ASDC is scaled to fit on the same axes as the transmissions.
Anyhow, here's the time series of the DARM sweeps. AS55 demod phase of -55 degrees seems to give the cleanest signal (within 5deg steps); this is the same phase that we've been using all week.
I just realized that the "damprestore" script that can be called from the watchdog screen did not have the new oplev names. I have updated it, and added it to the svn.
Working around the PSL table
I have put the FOL fiber box on the PSL table. The fibers carrying AUX and PSL light are connected and the RFPDs have been powered up. I can see the beat frequency on the frequency counters; but for some reason Domenica (that brings the frequency counter values on the medm screens) is not visible on the network even after hard reboot of the raspberry pi. I am neither able to ping nor ssh into the machine. I have to pull the module out and add a monitor cable to troubleshoot (my bad I should have left the monitor cable with the raspberry pi in the first place). Anyways, I have handed over the IFO to Q and will play with things again sometime later.
The fiber box on the PSL table is only left there temporariliy till I have things working. It will be stowed on the rack properly later.
In case someone wants the fiber box out of the PSL table, please make sure to power down the RFPDs using the black rocker switches on the side of the box and then disconnect the cables and fibers.
I pulled out the RF amplifier box from the IOO rack and swapped the amplifiers for FOL beat frequency amplification. The earlier gain of 62dB (ZFL500LN + ZFL500LN) was reduced to 40dB gain (ZFL500LN+ZFL2AD).
I also swapped one of the broken sma cables that was connecting the two amplifiers with a good one. Front ports of the module were relabeled and the box was put back on the rack.
During the course of this work, I had to turn OFF the green BBPDs on the PSL table to protect them and they have been powered up after putting the module back.
As Koji found one of the spare channels of the ALS/FOL RF amplifier box nonfunctional yesterday, I pulled it out to fix it. I found that one of the sma cables did not conduct.
It was replaced with a new cable from Koji. Also, I rearranged the ports to be consistent across the box, and re-labeled with the gains I observed.
It has been reinstalled, and the Y frequency counter that is using one of the channels shows a steady beat freq.
I cannot test the amplitude of the green X beat at this time, as Koji is on the PSL table with the PSL shutter closed, and is using the control room spectrum analyzer. However, the dataviewer trace for the fine_phase_out_Hz looks like free swinging cavity motion, so its probably ok.
The BS oplev servo was kicking up the BS. It was turned off
Brief elog of my activities tonight:
I was able to transition the digitial CARM control to REFL11 through the common mode board a total of one time, lock broke after a few seconds.
My suspicion was that when we did this on Monday, we unintentionally had a reasonable DARM offset, which reduced the finesse enough to let us take linear transfer functions and hop over. So, tonight, I intentionally looked at transitioning to CM_SLOW at some DARM offset. Using DARM offset of a few times 0.1 really calms the "buzzing" down, and makes it fairly straightforward to measure linear CARM sensing TFs. However, the CARM optical plant seems to change a fair amount depending on the DARM offset, in such a way that I was not able to compensate well enough to repeatedly transition.
Before I did anything else tonight, I measured the ALS noise down to 0.1 Hz, as a benchmark of how things are behaving.
With the arms locked on POX/POY, the HZ calibrated ALS channels reported
Then, with the arms CARM/DARM locked on ALS, the PDH signals reported (using a line and the HZ channels for conversion)
Not bad! I roughly estimate this to mean ~90pm RMS CARM/DARM motion. (If X was as good as Y, it would be ~50pm...)
Some things I feel are worth noting:
Tomorrow, I'll post some transfer functions of the difference between the ALS and CARM plants that I measured.
As discussed at today's meeting, we would like to (re)measure the Arm cavity lengths to ~mm precision, and their g-factors. Any arm length mismatch affects the reflection phase of the sidebands in the PRMI, which might be one source of our woes. Also, as I mentioned in a previous elog, the g-factors influence whether our 2f sidebands are getting pulled into the interferometer or not.
These both can be done by scanning the arm on ALS and measuring the green beat frequency at each IR resonance. (Misaligning the input beam will enhance the TM10 Mode content, and let us measure its guoy phase shift)
I started working on this today, but I have measurements to do, since at the time of today's measurements, I was fooled by the limits of the ALS offset sliders that I could only scan through two FSRs. Looking back at Manasa's previous measurment (ELOG 9804), I see now that more FSRs are possible.
Ways I will try to improve the measurement:
Just for kicks, here are scans from today.
Much of tonight was spent fighting with ETMX. This time, ASC was definitely off, there was nothing coming out of the ASC filter banks except the static output of the ASS. I tried turning off the 1000 count POS offset, but I think that made it a little worse. I ended up putting the offset back.
It's a little confusing, since it sometimes moves when there is no LSC actuation. However, it definitely moves when there is some LSC actuation. I did a test where every time I enabled the IR arm locking and caught lock, I saw a step in the SUSPIT and SUSYAW error signals. Once lock was aquired, it would settle and stay somewhere. If I unlocked the cavity, there was no "undo" step - it just stayed where it was. I wasn't letting it sit long enough to see if it spontaneously moved during this test.
Here's a plot of this test. The only button I'm touching is the LSC enable button. ASC is off, ASS is frozen (DC values exist, but no dither, no feedback). This was done when the 1000 count POS offset was off. The steps are less bad when the offset is on.
In between fighting with the ETM, I was able to do several trials with the PRFPMI.
I was playing with CARM and ALS fool.
First, I used REFL55 normalized by the sum of the transmissions as the error signal for the MC filter bank and saw that REFL11 (as an out of loop signal) got much more smooth, and centered around zero. However, I wasn't able to get the same thing with REFL11. No matter the sign I used for the MC filter bank, the IFO would squeak (some high freq gain peaking I think), and then I'd lose lock. This was true whether I used REFL11 through the common mode board or just directly into the ADC.
Just now, I did one trial of switching DARM over to AS55Q, just to grab a spectra of the MICH noise that Q and I saw yesterday.
I'm a little confused by some delay that seems to exist between the "A" and "B" error signals (right after the LSC input matrix) and the _IN1 point of the servo filters. I didn't save the measurement (bad Jenne), but there's a ~40 degree difference between DARM_A_ERR/DARM_IN2 and DARM_IN1/DARM_IN2. I don't think there should be anything there. Anyhow, it makes the DARM loop measurements look funny. If you just look at, say, DARM_B_ERR/DARM_IN2, you'll think that there's no way that the loop will be stable. However, it will actually be fine.
For tomorrow, we should take the DARM loop measurement with much less actuation. As with last night, I blew the lock by trying to measure the DARM loop.
Just in case there was some confusion, the champagne on my desk is not to be opened before I get back, no matter how many signals are transitioned to RF.
A slightly more coherent elog for last night's work.
All night, we've been using REFL165 to hold the PRMI. It's working very nicely. To help it catch lock, I've set the gain in the PRCL filter bank high, and then the *0.6 filter triggers on. The carm_cm_up script now will lock the PRMI on REFL165.
We had to reset the REFL165 phase after we acquired lock - it was -91, but now is -48. I'm not sure why it changed so significantly from the PRMI-only config to the PRFPMI config.
We measured the ALS fool cancellation with the arms held off resonance, at arm powers of a few. Although, they were moving around a lot, but the measurement stayed nice and smooth. Anyhow, we get almost as good of cancellation as we saw with the single arm (after we made sure that both phase trackers had the same UGF):
We were able to partly engage it one time, but we lost lock at some point. Since the frame builder / daqd decided that that would be just the *perfect* time to crash and restart, we don't have any frame data for this time. We can see up to a few seconds before the lockloss, while we were ramping up the RF PD loop gain though, and MICH was hitting the rails. I'm not sure if that's what caused the lockloss, but it probably didn't help.
The ALS fool gain was 22, and we were using FMs 4, 6 (the pendulum and Rana's "comp1"), the same filters that were used for the single arm case. The LSC-MC filter bank gain lost lock when we got to about 5.6 (we were taking +3dB steps).
We were using REFL55I/(TRX + TRY) as our CARM RF error signal. We were using REFL55 rather than REFL11 because we were worried that REFL11 didn't look good - maybe it was saturating or something. To be looked into.
Here's the striptool that was running at the time, since we don't have frame data:
At this point, since we weren't sure what the final gain should be for the RF CARM signal, and we could sit at nice high arm powers (arm powers of 30ish correspond to CARM offsets of about 50pm), we decided to try just a straight jump over to the RF signals.
The first time around, we jumped CARM to (-0.2)*REFL55/(TRX+TRY), but we only stayed lock for 1 or 2 seconds. That was around 1:55am.
We decided that perhaps it would be good to get DARM moved over first, since it has a much wider linewidth, so the rest of the trials for the night were transitioning DARM over to (0.0006)*AS55Q/(TRX + TRY). AS55 was saturating, so we reduced its analog gain from 18dB to 9dB and re-ran the LSC offsets. The MICH noise was pretty high when we were at low CARM offset, although we noticed it more when DARM was on AS55. In particular, there is some peak just below 40Hz that is causing a whole comb of harmonics, and dominating the MICH, PRC and DARM spectra. I will try to get a snapshot of that tonight - I don't think we saved any spectra from last night. Turning off DARM's FM3 boost helped lower the MICH noise, so we think that the problem is significant coupling between the two degrees of freedom.
After the first one or two tries of getting DARM to AS55, we started engaging the arm ASC loops - they helped the dark port contrast considerably. The POP spot still moves around, but the dark port gets much darker, and is more symmetric with the ASC on.
Jenne and I were musing the other night that the PC drive RMS may have a "favorite" laser temperature, as controlled by the FSS Slow servo; maybe around 0.2.
I downloaded the past 30 days of mean minute trend data for MC Trans, FSS Slow and PC Drive, and took the subset of data points where transmission was more than 15k, and the FSS slow output was within 1 count of zero. (This was to exclude some outliers when it ran away to 3 for some days). This was about 76% of the data. I then made some 2D histograms, to try and suss out any correlations.
Indeed, the FSS slow servo does like to hang out around 0.2, but this does not seem to correlate with better MC transmission nor lower PC drive.
In the following grid of plots, the diagonal plots are the 1D histograms of each variable in the selected time period. The off diagnoal elements are the 2D histograms. They're all pretty blob-y, with no clear correlation.
Safety glasses were measured and they are all good. I'd like to measure your personal glass if it is not on this picture.
Safety audit went soothly. We thank all participients.
1, Bathroom water heater cable to be stress releived and connector replaced by twister lock type.
2, Floor cable bridge at the vacuum rack to be replaced. It is cracked.
3, Sprinkler head to be moved eastward 2 ft in room 101
4, Annual crane inspection is scheduled for 8am Marc 3, 2015
5, Annual safety glasses cleaning and transmission measurement will get done tomorrow morning.
Konecranes' Fred inspected and load tested all tree cranes at with 450 lbs
The temperature of the east and south ends are normal, they are about the same.
After some searching, including help from 4 security guys (I think they don't have a lot to do at 4:30am :), we found that Ottavia is super warm, and smelled burn-y. She has been powered down and unplugged. Security guys may call Steve's desk to follow up later today.
Better elog tomorrow - notes for now:
REFL165 for PRMI has been "a champ" (quote from Q). We're able to sit on ALS at average arm powers of 30ish. Nice.
Some ALSfool work - measured cancellation almost as good as single arm.
One time transitioned CARM -> normalized REFL55I
Many times did DARM -> normalized AS55Q, see lots of noise at 39ish Hz - may be coupling from MICH??
Arm ASC loops helped improve dark port contrast.
Note to selfs: Need to make sure DTT templates have correct freq ordering - must be small freqs to large freqs.
It's super cold in the control room and EE bench area tonight. I'm wondering if, similar to what happened on Dec 29th (http://nodus.ligo.caltech.edu:8080/40m/10846) the campus steam is off? Or just our heater is broken? The thermostat is cranked up to 80 over by the bathrooms (this is usually ~74F), but we're still cold.
It's 69F in the control room right now (usually mid-high 70s).
EDIT, JCD, 4am: It's 64.3F in the desk area, 67.8F in the control room. It also smells in the control room like some heater has been off for a while, and is turning back on - that burned dust smell that happens after you haven't turned on the heater all summer.
EDIT again: The burn-y smell is getting stronger I think. Security is sending someone over to come check it out.
This has been edited several times over the last several hours, as I try to change different parameters, to see if they affect the movement of ETMX. So far, I don't know what is causing the motion. If it is there, it is only present when the LSC is engaged, so I don't think it's wobbling constantly on a twisted wire.
FINAL EDIT, 9:10pm: The arm ASC was turning itself on when the arms were locked. Whelp, that was only 3 hours of confusion. Blargh.
For his penance for leaving the arm ASC engaged, Q has made a set of warning lights on the LSC screen, right next to the ASS warning lights.
ETMX might be having one of those days today, which is lame.
So far tonight, I have run the LSC offset script, set the FSS slow value to +0.2, and run the arm ASS scripts. Nothing too crazy I think.
Sometimes when I lock the single arms, the ETMs move around like crazy. Other times, not. What is going on here??? The ETMs don't move at all when they are not being actuated on with the LSC.
In this screenshot you can see the end of a POX/POY lock stretch where everything was nice and good. Then, the arms were unlocked, and they have a bit of a DC offset. After settling from that step, they continue sitting nice and still. Then, I relock the cavities on POX and POY a little before -4 minutes. ETMY takes a moment to pull itself together, but then it's steady. ETMX just wobbles around for several minutes, until I turn off the LSC enable switch (happened after the end of this plot).
I'm not going to be able to lock like this. Eeek!
This is somehow related to light being in the Xarm. This next plot was taken while the arms were held with ALS in CARM/DARM mode.
I closed and re-opened all 3 green shutters. Now (at least the last 8 arm locks in the last 6 mintues) ETMX has never gone wobbly, except for a little bit right after acquisition, to deal with whatever the DC offset it. Why is this changing?
The arms were fine for one long ~30 minute lock while I stepped out for dinner. At some point after returned, the MC lost lock. When the arms came back, ETMX was being fussy again. Then, it decided that it was done.
In this plot, at -1 minute I started the ASS. Other than that, I did not touch any buttons at all, just observed. I have no idea why at about -3 minutes the bad stuff seems to go away.
I was curious if it had to do with the DC pointing of the optics, so I unlocked the arms, put ETMX about where it was during the long good lock stretch, then reaquired lock. I had to undo a little of that so that it would lock on TEM00, but at the beginning of the lock stretch (starting at about -3) the pitch is about the same spot. But, the oscillations persist. This time it was clear that the oscillations were around 80 mHz, and they started getting bigger until they settled to an amplitude they seemed to like.
Seems pretty independent from FSS temp. There are 3 lock stretches in the next plot (easier to see by looking at the Yarm transmission, green trace). The first one, the FSS slow was at 0.35. the middle one, it was around 0.05. The last one, it was around -0.4. Other than the different DC pointings (which I don't know if they are related), I don't see anything qualitatively different in the movement of ETMX.
Using PRX, I remeasured the relative actuation strengths of the BS and PRM to see if the PRM correction coefficient we're using is good.
My result is that we should be using MICH -> -0.2655 x PRM + 0.5*BS.
This is very close to our current value of -0.2625 x PRM, so I don't think it will really change anything.
The reason that the BS needs to be compensated is that it really just changes the PRM->ITMX distance, lx, while leaving the PRM-ITMY distance, ly, alone. I confirmed this by locking PRY and seeing no effect on the error signal, no matter how hard I drove the BS.
I then locked PRX, and drove an 804Hz oscillation on the BS and PRM in turn, and averaged the resultant peak heights. I then tried to cancel the signal by sending the excitation with opposite signs to each mirror, according to their relative meaured strength.
In this way, I was able to get 23dB of cancellation by driving 1.0 x PRM - 0.9416 * BS.
Now, in the PRMI case, we don't want to fully decouple like this, because this kind of cancellation just leaves lx invarient, when really, we want MICH to move (lx-ly) and PRCL to move (lx+ly). So, we use half of the PRM cancellation to cancel half of the lx motion, and introduce that half motion to ly, making a good MICH signal. Thus, the right ratio is 0.5*(1.0/0.9416) = 0.531. Then, since we use BS x 0.5, we divide by two again to get 0.2655. Et voila.
Here are a bunch of PDFs of time series from last night's CARM sweeps. The y-axes are all calibrated (except for the TRX/TRY, which are just normalized to single arm power, as usual) to real units - meters for the error signals, and Newtons for the control signals. The y-axes for each plot are the same on all PDFs (ex, the control signal plot in the lower left has the same range for all cases) so that it is easy to compare directly.
The most striking thing is that while the PRMI is held on REFL33, the MICH control signal saturates as we go through arm resonance. If the PRMI is held on REFL165, there is no such problem. I think we're going to have a lot more luck keeping the PRMI on REFL 165.
Plots while held on REFL 33:
Plots while held on REFL 165:
** along the way, I noticed that the reason this notebook hasn't been working since last night is that someone sadly installed a new anaconda python distro today without telling anyone by ELOG. This new distro didn't have all the packages of the previous one. I've updated it with astropy and uncertainties packages.
My bad, sorry!
Yesterday, I was trying to install a package with anaconda's package manager, conda, but it was crashing in some weird way. I wasn't able to fix it, which led me to create a fresh installation.
MUX input 7 to ITMXF camera cable was replaced by temporary cable labeled as 888
The problem remains to be the same black stripe at the bottom of the image The single picture is OK.
One of the things that I looked at tonight was whether or not I could hold the PRMI on REFL165 at CARM offset of 0, and it turns out that I can. Hooray. The next step was having a look to see if it is actually less noisy than the REFL33 lock.
I calibrated REFL33 and REFL165 to meters (I have the data to do the same for 11 and 55, but haven't done so yet). This way, we can directly compare the signals from each PD.
I scanned between +3 and -3 CARM digital offset (which we think is about 1nm/count while held on ALS), with a ramp time of 10 seconds. I did this several times while the PRMI was locked on both REFL33 and REFL165. Here are the gps times for 8 examples where the PRMI did not lose lock during the sweep:
Here are screen shots from the first REFL33 sweep, and the first REFL165 sweep. DTT can't print 3 plots together, so I'll have to make this nicer later. The top plot is the error signals, calibrated to meters. The middle plot is the control signals, that need to be calibrated to Newtons. The bottom plot is the arm powers, so you can see roughly where we were in the sweep.
We'd like to see a MIST simulation, or perhaps e2e, to see what the predicted disturbance is for each of the error signals during the CARM resonance. We want to make sure that the loops are engaged for all of the degrees of freedom for the simulation.
Recipes for tonight:
REFL165 sometimes has a tough time catching lock by itself, but if you add either REFL33 or REFL55 error signals to the REFL165 signals, it'll catch, and then you can just remove the extra error signals. Also, it doesn't stay locked very robustly unless you include the PRCL FM1 boost.
I have adapted one of Evan's python scripts into an ipython notebook for calculating our PRMI sensing matrix - the work is ~half done.
The script gets the data from the various PD channels (like REFL33_I) and demdoulates it at the modulation frequencies. At the moment its using just the sensing channels, but with the recent addition of the SUS-LSC_OUT_DQ channels, we can demod the actuation channels as well and not have to hand code the exc amplitudes and the basolute phase. Please ignore the phase for the moment.
The attached PDF shows the demod (including lowpass) outputs for a 2 minute stretch of PRMI locked on f2. Next step is to average these numbers and make the radar plots with the error bars. The script is scripts/LSC/SensingMatrix/PRMIsensMat.ipynb and is in the SVN now.
I've fixed the Radar plot making part, so that's now included too. The radial direction is linear, so you can see from the smearing of the blobs that the uncertainty is represented in the graphics due to each measurement being a small semi-transparent dot. Next, we'll put the output of the statistics on the plot: mean, std, and kurtosis.
I have clarified my elog from last night to indicate that the sensing matrix in the "33MHz cancellation" configuration was measured with the PRMI held on REFL55 I&Q.
Also, I just re-read my control room notes from yesterday, and I typed the wrong demod phases into the table last night. The elog has been edited. Most significantly, the REFL33 demod phase did not change by 70 degrees. It did change by 10 degrees, but that is likely from the fact that it was formerly tuned for PRCL in PRFPMI, and last night I tuned it for PRCL in PRMI-only.
I shutdown the +/-15V and the +/-24V sorensons on the IOO rack to connect the FOL RF PDs to the rack power supply.
They were turned back ON after the work. PMC and MC were relocked.
We ran power cables and sma cables for the FOL fiber module from the PSL table to the IOO rack.
No more illegal power supply at the LSC rack
The amplifiers are now being powered by the rack power supply through fuse blocks.
To make new connections, I shutdown the +/-15 V low noise power supplies. They were turned back ON after the work.
If so, or if not but you care about the signal that passes through these amplifiers, I suggest you remove this temporary power supply and wire the power from the rack power supplies through the fuse blocks and possibly use a voltage regulator.
In 24 hours, that power supply will be disconnected and the wires snipped if they are still there.
Over the past few days, I've occasionally been peeking at the framebuilder IO load to see If I could correlate anything with it, but it's usually been low when I looked. I.e. with daqd and all models running, the %wa time was in the few percents at most.
Just now, I was seeing some EPICS sluggishness, and sure enough, the %wa was in the 50-60 range. I used iostat -xmh 5 on the framebuilder to see that /dev/sda, the /frames drive, was at 100% utilization, which means it was reading and writing as fast as it possibliy could.
iostat -xmh 5
I ssh'd over to nodus, and with iotop found that an rsync job was running (rsync -am --exclude .*.gwf full 22.214.171.124::40m/full), and its IO rates corresponded very closely to the data read rates on the framebuilder from /frames.
rsync -am --exclude .*.gwf full 126.96.36.199::40m/full
I killed the rsync process on nodus, and the %wa time on the framebuilder dropped to near zero. The ASS striptools, where I had noticed the sluggishness, immediately started updating faster.
While rsync is supposed to play nice with a system's IO demands, maybe it only knows about nodus's IO usage, not fb which is the underlying NFS server where the frames live. I think it would be good to throttle the bandwidth of these jobs to a specific bandwidth. 50MB/s seemed like too much, so maybe 10MB/s is ok?