Looks like something went nuts in late April. We have yet to try a hard reboot.
The MOPA is taking the long weekend off.
Steve went out to wipe off the condensation inside the MOPA and found beads of water inside the NPRO box, perilously close to the PCB board. He then measured the water temperature at the chiller head, which is 6C. We decided to "reboot" the MOPA/chiller combo, on the off chance that would get things synced up. Upon turning off the MOPA, the neslab chiller display immediately started displaying the correct temperature--about 6C. The 22C number must come from the MOPA controller. We thus tentatively narrowed down the possible space of problems to: broken MOPA controller and/or clog in the cooling line going to the power amplifier. We decided to leave the MOPA off for the weekend, and start plumbing on Tuesday. It is of course possible that the controller is the problem, but we think leaving the laser off over the weekend is the best course of action.
we did a hard reboot of c1susvme1, c1susvme2, c1sosvme, and c1susaux. We are hoping this will fix some of the weird suspension issues we've been having (MC3 side coil, ITMX alignment).
I just restarted the elog. It was crashed for unknown reasons. The restarting instructions are in the wiki.
Guralp Vert1b and Guralp EW1b are plugged back in to PEM ADCU #10 and #12 respectively. Guralp NS1b remains plugged in. So, PEM-SEIS_MC1_X,Y,Z should now corrsp to seismometer as before.
steve, rob, alberto
Steve installed two rotary flow meters into the MOPA chiller system--one at the chiller flow output and one in the NPRO cooling line. After some hijinks, we discovered that the long, insulated chiller lines have the same labels at each end. This means that if you match up the labels at the chiller end, at the MOPA end you need switch labels: out goes to in and vice-versa. This means that, indubitably, we have at some point had the flow going backwards through the MOPA, though I'm not sure if that would make much of a difference.
Steve also installed a new needle valve in the NPRO cooling line, which works as expected as confirmed by the flow meter.
We also re-discovered that the 40m procedures manual contains an error. To turn on the chiller in the MOPA start-up process, you have to press ON, then RS-232, then ENTER. The proc man says ON, RS-232, RUN/STOP.
The laser power is at 1.5W and climbing.
The chiller HT alarm started blinking, as the water temperature had reached 40 degrees C, and was still rising. We turned off the MOPA and the chiller. Maybe we need to open the needle valve a bit more? Or maybe the flow needs to be reversed? The labels on the MOPA are backwards?
The chiller appears to be broken. We currently have it on, with both the SENSOR and RS-232 unplugged. It's running, circulating water, and the COOL led is illuminated. But the temperature is not going down. The exhaust out the back is not particularly warm. We think this means the refrigeration unit has broken, or the chiller computer is not communicating with the refrigerator/heat exchanger. Regardless, we may need a new chiller and a new laser.
steve, alberto, rob
After some futzing around with the chiller, we have come to the tentative conclusion that the refrigeration unit is not working. Steve called facilities to try to get them to recharge the refrigerant (R-404a) tomorrow, and we're also calling around for a spare chiller somewhere in the project (without luck so far).
The repair man thinks it's a bad start capacitor, which is 240uF at 120V. Steve has ordered a new one which should be here tomorrow, and with luck we'll have lasing by tomorrow afternoon.
I drained the water and removed side covers from the Neslab RTE 140 refrigerated water cooler unit this morning. The hoses to the laser were disconnected.
This abled you to see the little window of refregerant R404A was free of bubles, meaning: no recharge was needed.
The circulator bath was refilled with 7 liters of Arrowhead distilled water and the unit was turned on.
The water temp was kept 20.00+- .05C without any load. Finally the AC-repair man Paul showed up.
He measured the R404A level to be as specified: 23-24 PSI on the suction side and 310 PSI on the discharge side.
The unit was working fine. Paul found an intermittently functioning starting capacitor on the compressor that was removed.
The 240 micro Farad 120VAC cap will arrive tomorrow
Steve, Rob and Alberto
Starting capacitor 216 miroFarad was installed on the compressor. Water lines were connected to the MOPA as corrected, so the flow meter readings are logical.
Now IN means flowing water in the direction of black arrow on the hose.
We struggled with the Neslab presetting: temp, bauds rate and other unknowns till Rob found the M6000 manual on Peter king's website.
Alberto realized that the chiller temp had to be reset to 20C on water chiller.
I put 1mg of Chloramin T into the water to restrict the growth of algae in the bath.
The NPRO heat sink was around ~20C without flow meter wheel rotation and the PA body ~25C by touch of a finger
I just opened up the needle valve a litle bit so the flow meter wheel would started rotating slowly.
That small glitch at the end of this 3 hrs plot shows this adjustment.
I locked to PSL loops, then tweaked the alignment of the MC to get it to lock.
I first steering MC1 until all the McWFS quads were saturated. This got the MC locking in a 01 mode. So I steered MC1 a little more till it was 00. Then I steered MC2 to increase the power a little bit. After that, I just enabled the MC autolocker.
c1susvme2 has been running just a bit late for about a week. I rebooted it.
The plot shows SRM_FE_SYNC, which is the number of times in the last second that c1susvme2 was late for the 16k cycle. Similarly for ETMX.
The reboot appears to have worked.
The laser power seems to have become more stable after fixing the laser chiller. The power is lower than it used to be (MOPA amplitude 2.5 versus 2.7) but, as shown in the attchement, it became more steady.
We worked on tuning the DD handoff tonight. We checked the DD PD alignments and they looked fine. First I tuned the 3 demod phases to minimize offsets. Then I noticed that the post-handoff MICH xfer function needed an increase in gain to look like the pre-handoff xfer function (which has a UGF of about 25 Hz). I increased the MICH PD9_Q gain from 2 to 7 in the input matrix. But, the handoff to PRC still failed, so tomorrow we will try to find out why.
In the plot, ref0 is before MICH handoff, and ref1 is after MICH handoff. There is also a PRC trace (before PRC handooff).
Tonight I centered the oplevs for ITMX/Y, SRM, PRM, BS.
After doing that I noticed that the BS drifted a little from where I had set it.
Rana, Alberto, Pete
We have the DD handoff nominally working. Sometimes, increasing the SRC gain at the end makes MICH get unstable. This could be due to a non-diagonal term in the matrix, or possibly because the DRM locks in a funky mode sometimes.
To get the DD handoff working, first we tuned demod phases in order to zero the offsets in the PD signals handed-off-to. Based on transer function measurements, I set the PRC PD6_I element to 0.1, and set the PD8_I signal to 0, since it didn't seem to be contributing much. We also commented out the MICH gain increase at the end of the DD_handoff script.
It could still be more stable, but it seems to work most of the time.
The Neslab chiller is working well. It's temp display shows 20.0 C rock solid. Flow meter rotating at 13.5Hz at the out put of the chiller.
The MOPA temp was measured with a hand held thermocouple . The PA was 34 C and 29 C at NPRO heat sink.
The NPRO flow meter was not rotating at this time. There was just trickeling water flow though the meter.
I closed the needle valve this point. It needed 8 turns clockwise. This drives head temp to 19.9 C
Than I opened the needle valve 9 turns and the flow meter wheel was rotaing at ~ 1 Hz
We gained a little power. Can you explain this?
I played with the DD handoff during the day. The DRM dark port was flickering like a candle flame in Dracula's castle. The demod offsets for the handoff signals looked fine. After MICH handoff, the MICH_CTRL started to get unstable at some low frequency, maybe 3 Hz (I didn't measure). So I increased the MICH gain from 0.1 to 0.17 and it settled down. PRC and SRC went fine. Then the DD_handoff script raised the MICH gain to 0.7, and an instability started to grow in MICH_CTRL (at some higher frequency). I decreased the MICH gain from 0.7 to 0.5, and it settled down and stayed stable.
Looks like yesterday was particularly noisy. It's unclear to me why diurnal variation much more visible in MC1_Y, and why the floor wanders.
The first plot shows 5 days. The second plot shows 20 days.
We were stymied early in the evening by a surreptitiously placed, verbo-visually obfuscated command in the drstep script.
tdsavg 5 C1:LSC-PD4_DC_IN1
was causing grievous woe in the cm_step script. It turned out to fail intermittently at the command line, as did other LSC channels. (But non-LSC channels seem to be OK.) So we power cycled c1lsc (we couldn't ssh).
Then we noticed that computers were out of sync again (several timing fields said 16383 in the C0DAQ_RFMNETWORK screen). We restarted c1iscey, c1iscex, c1lsc, c1susvme1, and c1susvme2. The timing fields went back to 0. But the tdsavg command still intermittently said "ERROR: LDAQ - SendRequest - bad NDS status: 13".
The channel C1:LSC-SRM_OUT16 seems to work with tdsavg every time.
Let us know if you know how to fix this.
Did you try restarting the framebuilder?
What you type is in bold:
op440m> telnet fb40m 8087
After fixing the tp problem, I tried locking again. Grabbing and DD handoff, no problem. Died earlier than last night, handing off CARM to REFL_DC, around arm power of 4 or so. Seems to happen after turning off the moving zero, Rob says it might be touchy in daytime.
Lock acquisition is proceeding smoothly for the most part, but there is a very consistent failure point near the end of the cm_step script.
Near the end of the procedure, while in RF common mode, the sensing for the MCL path of the common mode servo is transitioned from a REFL 166I signal which comes into the LSC whitening board from the demodulator, to another copy of the signal which has passed through the common mode board, and is coming out of the Length output of the common mode board. We do this because the signal which comes through the CM board sees the switchable low-frequency boost filter, and so both paths of the CM servo (AO and MCL) can get that filter switched on at the same time.
The problem is occurring after this transition, which works reliably. However, when the script tries to remove the final CARM offset, and bring the offset to zero, lock is abruptly lost. DARM, CM, and the crossover all look stable, and no excess noise appears while looking at the DARM, CARM, MCF spectra. But lock is always lost right about the same offset.
I've seen this before. At that time, the problem was gone spontaneously the next day.
You could stop just before the offset reaches zero and then try to slowly reduce the offset manually to see where is the threshold.
I measured the ETMY oplev beam size at a couple different distances away from the HeNe by taking out the steering mirror and letting the light propagate a ways. I put the steering mirror back, aligned the oplev, and was able to relock the Yarm, so I think it's all back as it has been the last couple of weeks.
Now I need t o do some geometry and ray-tracing matrices to decide what focal length lens to buy, then we'll have a shiny new ETMY oplev.
Well, it hasn't gone away yet. It happened Sat, Mon, and Tues afternoon, as well as Friday. The threshold varies slightly, but is always around ~200-300 cnts. I've tried reducing the offset with the signal coming from the CM board and the signal not going through the CM board, I've also tried jumping the signal to zero (rather than a gradual reduction).
Tonight we'll measure the MC length and set the modulation frequencies, and maybe try some MZ tweaking to do RFAMMon minimization.
Pete, Rob, Alberto,
yesterday we thought that some of the problems we were having in locking the IFO might be related to a change of the length of the mode cleaner. So today we decided to measure it again.
We followed the Sigg-Frolov technique (see 40m Wiki, Waldman, Fricke). For the record, the MC_AO input corresponds to IN2 on the MC Servo board.
We obtained: L = 27.092 +/- 0.001 m
From the new measurement we reset the frequencies of the Marconis to the following values:
I moved the mobile HEPA filter from ITMX's north door to ITMX-ISCT and covered it up with a merostate tent to accommodate the aluminum foil particle measurement on June 5
It lowered the 40m baseline counts by about a factor of 3 of 0.5 micron and a factor of 2 of 1.0 micron.
The HEPA filter is sweeping the floor and blowing the particles upwards. The MET ONE counter is on the top of the IOOC looking south at ~75 degrees upward.
The PMC alarm was on this morning. It was relocked at lower HV
The FSS_RMTEMP jumped 0.5 C so The PZT was compensating for it.
tonight we worked on the tuning of the double demod phases for the handoff of the short DOFs control signals.
Only MICH can now undergo the handoff. PRC can't make it.
Basically, we tuned the PD6 demod phase and reduced the offset in PD6_I. Then we tuned the relative gain of PD6_I and PD2_I so that the two open loop transfer function of the control loops would match. We tried that in several ways and several times but without success.
I guess we're missing to do/check something.
After poking around for a few minutes several facts became clear:
1) At least one GPIB interface has a hard ethernet connection (and does not currently go through the wireless).
2) The wireless on the laptop works fine, since it can connect to the router.
3) The rest of the martian network cannot talk to the router.
This led to me replugging the ethernet cord back into the wireless router, which at some point in the past had been unplugged. The computers now seem to be happy and can talk to each other.
This afternoon I tuned the handoff script for the SRC, after that Rob eralier during the day had already adjusted that for PRC. To do that, I followed the procedure in the Wiki.
After that the SRC could get locked with the double demod signals. the open loop transfer function emasurement on the PRC loop showed that it was nearly unstable. Rob reduced a little its gain to improve the stability.
The DD handoff is now working and we can get back to locking the interferometer.
I started two scripts, senseDRM and loadDRMImatrixData.m, which Peter will bang on until they're correct. They're in the $SCRIPTS/LSC directory. The first is a perl script which uses TDS tools to drive the DRM optics and measure the response at the double demod photo-detectors, and write these results to a series of files loadable by matlab. The second loads the output from the first script, inverts the resulting sensing matrix to get an input matrix, and spits out a tdswrite command which can be copied and pasted into a terminal to load the new input matrix values.
What's left is mainly in figuring out how to do the matrix inversion properly. Right now the script does not account for the output matrix, the gains in the feedback filters at the measurement frequency, or the fact that we'll likely want the UGF of our loops to be less than the measurement frequency. Peter's going to hash out these details.
I had trouble getting the SRC handoff from SD to DD to work. I tried different gains, flipping the PD7 & 8 demod phases by 180 degrees, and messing with the output matrix to reduce cross-couplings in the state with MICH & PRC on DD and SRC on SD. Eventually I decided to try to make the DRM matrix diagonalization work.
It does, mostly. The handoff is now stable, and the loops all have UGFs around 100Hz. So, tonight anyways, it's possible to run senseDRM and then loadDRMImatrixData.m and run the resulting tdswrite command, and have a working handoff. I had to eliminate a few PDs (PD5 & PD10) to get it to work properly.
It would be nice if this script would measure all the PDs and allow individual setting of loop UGFs and measurement frequencies.
Lock is still being lost, right at the end of the process when trying to reduce the CARM offset to zero.
The IFO is locked, at the operating point (zero CARM offset). The problem with reducing the residual CARM offset in the last stage appears to have been because the common mode gain was getting too high, and so the loop was going unstable at high frequencies.
The cm_step script is currently a confusing mess, with all the debugging statements. I'll clean it up this afternoon and check that it still works.
Last night Rob ran senseDRM and loadDRMImatrixData and came up with the following for the input matrix:
tdswrite C1:LSC-ITMTRX_b2 0.065778 \
C1:LSC-ITMTRX_d2 2.2709 \
C1:LSC-ITMTRX_f2 2.9361 \
C1:LSC-ITMTRX_122 0.42826 \
C1:LSC-ITMTRX_b3 -0.064839 \
C1:LSC-ITMTRX_d3 -0.016913 \
C1:LSC-ITMTRX_f3 -0.021576 \
C1:LSC-ITMTRX_123 -0.0025243 \
C1:LSC-ITMTRX_b5 0.3719 \
C1:LSC-ITMTRX_d5 1.3109 \
C1:LSC-ITMTRX_f5 -0.16412 \
C1:LSC-ITMTRX_125 0.39574 \
C1:LSC-ITMTRX_33 0 \
C1:LSC-ITMTRX_42 0 \
Today, I reran these and got the following, and DD_handoff remained happy:
tdswrite C1:LSC-ITMTRX_b2 -0.10329 \
C1:LSC-ITMTRX_d2 2.0344 \
C1:LSC-ITMTRX_f2 3.2804 \
C1:LSC-ITMTRX_122 0.22516 \
C1:LSC-ITMTRX_b3 -0.076292 \
C1:LSC-ITMTRX_d3 -0.014603 \
C1:LSC-ITMTRX_f3 -0.12101 \
C1:LSC-ITMTRX_123 0.0054128 \
C1:LSC-ITMTRX_b5 0.33521 \
C1:LSC-ITMTRX_d5 1.1425 \
C1:LSC-ITMTRX_f5 -0.32759 \
C1:LSC-ITMTRX_125 0.25877 \
C1:LSC-ITMTRX_33 0 \
C1:LSC-ITMTRX_42 0 \
I wanted to remeasure with the canonical output matrix (-0.7 from MICH to PRM and 0.7 from MICH to SRM), but the DRM freaked out when MICH to PRM went below -0.3.
With the common mode servo bandwidth above 30kHz and the BOOST on (1), I was able to switch on the test mass dewhitening. Finally.
For the 40m Upgrade, we plan to eliminate the Mach-Zehnder and replace it with a single EOM driven by all three modulation frequencies that we'll need: f1=11MHz, f2=5*f1=55MHz, fmc=29.5MHz.
A frequency generator will produce the three frequencies and with some other electronics we'll properly combine and feed them to the EOM.
The frequency generator will have two crystals to produce the f1 and fmc signals. The f2 modulation will be obtained by a frequency multiplier (5x) from the f1.
The frequency multiplier, for the way it works, will inevitably introduce some unwanted harmonics into the signals. These will show up as extra modulation frequencies in the EOM.
In order to quantify the effects of such unwanted harmonics on the interferometer and thus to let us set some limits on their amplitude, I ran some simulations with Optickle. The way the EOM is represented is by three RF modulators in series. In order to introduce the unwanted harmonics, I just added an RF modulator in series for each of them. I also made sure not to leave any space in between the modulators, so not to introduce phase shifts.
To check the effect at DC I looked at the sensing matrix and at the error signals. I considered the 3f error signals that we plan to use for the short DOFs and looked at how they depend on the CARM offset. I repeated the simulations for several possible amplitude of the unwanted harmonics. Some results are shown in the plots attached to this entry. 'ga' is the amplitude ratio of the unwanted harmonics relative to the amplitude of the 11 & 55 MHz modulations.
Comparing to the case where there are no unwanted harmonics (ga = 0), one can see that not considerable effect on the error signals for amplitudes 40dB smaller than that of the main sidebands. Above that value, the REFL31I signals, that we're going to use to control PRCL, will start to be distorted: gain and linearity range change.
So 40 dB of attenuation in the unwanted harmonics is probably the minimum requirement on the frequency multiplier, although 60dB would provide a safer margin.
I'm still thinking how to evaluate any AC effect on the IFO.
** TODO: Plot DC sweeps with a wider range (+/- 20 pm). Also plot swept sines to look for changes in TFs out to ~10 kHz.