ETMY damping restored.
Cryo interlock closed VC1 ~2 days ago. P1 is 6.3 mTorr. Cryo temp 12K stable, reset photoswitch and opened VC1
The 40m frame builder is currently being patched to be able utilize the full 14 TB of the new raid array (as opposed to being limited to 2 TB). This process is expected to take several hours, during which the frame builder will be unavailable.
After looking at some oplev noise spectra in DTT, we discovered that the ETMY quad (serial number 115) was noisy. Particularly, in the XX_OUT and XX_IN1 channels, quadrants 2 (by a bit more than an order of magnitude over the ETMX ref) and 4 (by a bit less than an order of mag). We went out and looked at the signals coming out of the oplev interface board; again, channels 2 and 4 were noise compared to 1 and 3 by about these same amounts. I popped in the ETMX quad and everything looked fine. I put the ETMX quad back at ETMX, and popped in Steve's scatterometer quad (serial number 121 or possibly 151, it's not terribly legible), and it looks fine. We zeroed via the offsets in the control room, and I went out and centered both the ETMX and ETMY quads.
Attached is a plot. The reference curves are with the faulty quad (115). The others are with the 121.
I adjusted the ETMY quad gains up by a factor of 10 so that the SUM is similar to what it was before.
At the request of people down at LLO I've been trying to work on the reliability and speed of the GigE camera code. In my testing, after several hours, the code would tend to lock up on the camera end. It was also reported at LLO after several minutes the camera display would slow down, but I haven't been able to replicate that problem.
I've recently added some additional error checking and have updated to a more recent SDK which seems to help. Attached are two plots of the frames per second of the code. In this case, the frames per second are measured as the time between calls to the C camera code for a new frame for gstreamer to encode and transmit. The data points in the first graph are actually the averaged time for sets of 1000 frames. The camera was sending 640x480 pixel frames, with an exposure time of 0.01 seconds. Since the FPS was mostly between 45 and 55, it is taking the code roughly 0.01 second to process, encode, and transmit a frame.
During the test, the memory usage by the server code was roughly 1% (or 40 megabytes out of 4 gigabytes) and 50% of a CPU (out a total of CPUs).
It seems that the MC3 problem is intermittent (one-day trend attached). I tried to take advantage of a "clean MC3" night, but the watch script would usually fail at the transition to DC CARM and DARM. It got past this twice and then failed later, during powering up. I need to check the handoff.
I checked the four rear coils on ETMX by exciting XXCOIL_EXC channel in DTT with amplitude 1000@ 500 Hz and observing the oplev PERROR and YERROR channels. Each coil showed a clear signal in PERROR, about 2e-6 cts. Anyway, the coils passed this test.
At Rob's request I've added the following features to the camera code.
The camera server, which can be started on Ottavia by just typing pserv1 (for camera 1) or pserv2 (for camera 2), now has the ability to save individual jpeg snap shots, as well as taking a jpeg image every X seconds, as defined by the user.
The first text box is for the file name (i.e. ./default.jpg will save the file to the local directory and call it default.jpg). If the camera is running (i.e. you've pressed start), prsessing "Take Snapshot to" will take an image immediately and save it. If the camera is not running, it will take an image as soon as you do start it.
If you press "Start image capture every X seconds", it will do exactly that. The file name is the same as for the first button, but it appends a time stamp to the end of the file.
There is also a viedo recording client now. This is access by typing "pcam1-mov" or "pcam2-mov". The text box is for setting the file name. It is currently using the open source Theora encoder and Ogg format (.ogm). Totem is capable of reading this format (and I also believe vlc). This can be run on any of the Linux machines.
The viewing client is still accessed by "pcam1" or "pcam2".
I'll try rolling out these updates to the sites on Monday.
The configuration files for camera 1 and camera 2 can be found by typing in camera (which is aliased to cd /cvs/cds/caltech/apps/linux64/python/pcamera) and are called pcam1.ini, pcam2.ini, etc.
This is the two arms locked, for an hour. No integrator in either loop, but from this it looks like ETMY may have a bigger length2angle problem than ETMX. I'll put some true integrators in the loops and do this again.
There appear to be at least two independent problems: the coil balancing for ETMY is bad, and something about ITMX is broken (maybe a coil driver).
The Y-arm becomes significantly misaligned during long locks, causing the arm power to drop. This misalignment tracks directly with the DC drive on ETMY. Power returns to the maximum after breaking and re-establishing lock.
ITMX alignment wanders around sporadically, as indicated by the oplevs and the X-arm transmitted power. Power returns to previous value (not max) after breaking and re-establishing lock.
Both loops have integrators.
Even more plots for the Wiener filtering!
We have a set of spectrograms, which show (in color) the amplitude spectrum, at various times during a one month stretch of time, during S5. Each vertical data-'stripe' is 10min long.
We also have a set of band-limited plots, which take the spectra at each time, and integrate under it, for different frequency bands.
Each set of plots has the following 3 plots: The raw DARM spectrum, a ratio of residual/raw, and the residuals, normalized to the first one (on which the wiener filter was trained).
The residuals are the DARM spectrum, after subtracting the Wiener-filtered seismometer witness data.
From the ratio plots, it looks like the wiener filter is pretty much equally effective at the time on which the filter was trained, as one month later. Static filters may be okey-dokey for a long period of time with for the seismic stuff.
Earthquake mag 4.0 at Lennox, Ca trips MC2 watchdogs http://quake.usgs.gov/recenteqs/Quakes/ci10411545.html
See 40m accelerometers as they see it.
I found some neat signal analysis software for my mac (http://www.faberacoustical.com/products/), and took a spectrum of the ambient noise coming from the cryopump. The two main noise peaks from that bad boy were nowhere near 3.7 kHz.
Recently the watch script was having difficulty grabbing a lock for more than a few seconds. Rob discovered that the violin notch filters which were activated in the script were causing the instability. We're not sure why yet. The script seems significantly more stable with that step commented out.
All oplevs servos turned off to protect our suspentions from vibration due to drilling and pounding in CES high bay area.
This activity will be done from 10 am till 3 pm today.
Meanwhile our IFO-air conditions are turned off for maintenance.
Their performance of 6 months is shown on plot.
I also made xfer fctns of the 4 piston coils on ETMY and ETMX with OL_PIT. (I looked at all 4 even though the attached plot only shows three.) So it looks ike the coils are OK.
Looks like something went nuts in late April. We have yet to try a hard reboot.
The MOPA is taking the long weekend off.
Steve went out to wipe off the condensation inside the MOPA and found beads of water inside the NPRO box, perilously close to the PCB board. He then measured the water temperature at the chiller head, which is 6C. We decided to "reboot" the MOPA/chiller combo, on the off chance that would get things synced up. Upon turning off the MOPA, the neslab chiller display immediately started displaying the correct temperature--about 6C. The 22C number must come from the MOPA controller. We thus tentatively narrowed down the possible space of problems to: broken MOPA controller and/or clog in the cooling line going to the power amplifier. We decided to leave the MOPA off for the weekend, and start plumbing on Tuesday. It is of course possible that the controller is the problem, but we think leaving the laser off over the weekend is the best course of action.
we did a hard reboot of c1susvme1, c1susvme2, c1sosvme, and c1susaux. We are hoping this will fix some of the weird suspension issues we've been having (MC3 side coil, ITMX alignment).
I just restarted the elog. It was crashed for unknown reasons. The restarting instructions are in the wiki.
Guralp Vert1b and Guralp EW1b are plugged back in to PEM ADCU #10 and #12 respectively. Guralp NS1b remains plugged in. So, PEM-SEIS_MC1_X,Y,Z should now corrsp to seismometer as before.
steve, rob, alberto
Steve installed two rotary flow meters into the MOPA chiller system--one at the chiller flow output and one in the NPRO cooling line. After some hijinks, we discovered that the long, insulated chiller lines have the same labels at each end. This means that if you match up the labels at the chiller end, at the MOPA end you need switch labels: out goes to in and vice-versa. This means that, indubitably, we have at some point had the flow going backwards through the MOPA, though I'm not sure if that would make much of a difference.
Steve also installed a new needle valve in the NPRO cooling line, which works as expected as confirmed by the flow meter.
We also re-discovered that the 40m procedures manual contains an error. To turn on the chiller in the MOPA start-up process, you have to press ON, then RS-232, then ENTER. The proc man says ON, RS-232, RUN/STOP.
The laser power is at 1.5W and climbing.
The chiller HT alarm started blinking, as the water temperature had reached 40 degrees C, and was still rising. We turned off the MOPA and the chiller. Maybe we need to open the needle valve a bit more? Or maybe the flow needs to be reversed? The labels on the MOPA are backwards?
The chiller appears to be broken. We currently have it on, with both the SENSOR and RS-232 unplugged. It's running, circulating water, and the COOL led is illuminated. But the temperature is not going down. The exhaust out the back is not particularly warm. We think this means the refrigeration unit has broken, or the chiller computer is not communicating with the refrigerator/heat exchanger. Regardless, we may need a new chiller and a new laser.
steve, alberto, rob
After some futzing around with the chiller, we have come to the tentative conclusion that the refrigeration unit is not working. Steve called facilities to try to get them to recharge the refrigerant (R-404a) tomorrow, and we're also calling around for a spare chiller somewhere in the project (without luck so far).
The repair man thinks it's a bad start capacitor, which is 240uF at 120V. Steve has ordered a new one which should be here tomorrow, and with luck we'll have lasing by tomorrow afternoon.
I drained the water and removed side covers from the Neslab RTE 140 refrigerated water cooler unit this morning. The hoses to the laser were disconnected.
This abled you to see the little window of refregerant R404A was free of bubles, meaning: no recharge was needed.
The circulator bath was refilled with 7 liters of Arrowhead distilled water and the unit was turned on.
The water temp was kept 20.00+- .05C without any load. Finally the AC-repair man Paul showed up.
He measured the R404A level to be as specified: 23-24 PSI on the suction side and 310 PSI on the discharge side.
The unit was working fine. Paul found an intermittently functioning starting capacitor on the compressor that was removed.
The 240 micro Farad 120VAC cap will arrive tomorrow
Steve, Rob and Alberto
Starting capacitor 216 miroFarad was installed on the compressor. Water lines were connected to the MOPA as corrected, so the flow meter readings are logical.
Now IN means flowing water in the direction of black arrow on the hose.
We struggled with the Neslab presetting: temp, bauds rate and other unknowns till Rob found the M6000 manual on Peter king's website.
Alberto realized that the chiller temp had to be reset to 20C on water chiller.
I put 1mg of Chloramin T into the water to restrict the growth of algae in the bath.
The NPRO heat sink was around ~20C without flow meter wheel rotation and the PA body ~25C by touch of a finger
I just opened up the needle valve a litle bit so the flow meter wheel would started rotating slowly.
That small glitch at the end of this 3 hrs plot shows this adjustment.
I locked to PSL loops, then tweaked the alignment of the MC to get it to lock.
I first steering MC1 until all the McWFS quads were saturated. This got the MC locking in a 01 mode. So I steered MC1 a little more till it was 00. Then I steered MC2 to increase the power a little bit. After that, I just enabled the MC autolocker.
c1susvme2 has been running just a bit late for about a week. I rebooted it.
The plot shows SRM_FE_SYNC, which is the number of times in the last second that c1susvme2 was late for the 16k cycle. Similarly for ETMX.
The reboot appears to have worked.
The laser power seems to have become more stable after fixing the laser chiller. The power is lower than it used to be (MOPA amplitude 2.5 versus 2.7) but, as shown in the attchement, it became more steady.
We worked on tuning the DD handoff tonight. We checked the DD PD alignments and they looked fine. First I tuned the 3 demod phases to minimize offsets. Then I noticed that the post-handoff MICH xfer function needed an increase in gain to look like the pre-handoff xfer function (which has a UGF of about 25 Hz). I increased the MICH PD9_Q gain from 2 to 7 in the input matrix. But, the handoff to PRC still failed, so tomorrow we will try to find out why.
In the plot, ref0 is before MICH handoff, and ref1 is after MICH handoff. There is also a PRC trace (before PRC handooff).
Tonight I centered the oplevs for ITMX/Y, SRM, PRM, BS.
After doing that I noticed that the BS drifted a little from where I had set it.
Rana, Alberto, Pete
We have the DD handoff nominally working. Sometimes, increasing the SRC gain at the end makes MICH get unstable. This could be due to a non-diagonal term in the matrix, or possibly because the DRM locks in a funky mode sometimes.
To get the DD handoff working, first we tuned demod phases in order to zero the offsets in the PD signals handed-off-to. Based on transer function measurements, I set the PRC PD6_I element to 0.1, and set the PD8_I signal to 0, since it didn't seem to be contributing much. We also commented out the MICH gain increase at the end of the DD_handoff script.
It could still be more stable, but it seems to work most of the time.