Recovery from the power shutdown
- Turned on the raid disk of linux1.
- Woke linux1 up. No fsck this time.
- Woke up all the lab machines.
- Turned on all the electronics racks' AC powers
- Woke up fb and then front end machine (the raid for fb had been already up as I turned on the AC powers)
- Turned on all the electronics racks' DC powers (Sorensens, Kepcos, and etc.)
- Turned on the Marcnois which is driving the RF generation box.
- Woke up all the lasers (PSL and End lasers)
- Some burtrestoring (c1ioo, c1sus, c1susaux, c1msc, c1psl, c1iool0, c1auxey, c1auxex, c1oaf, c1pem)
- Ran autolockMC scripts on op340m => After relocking of PMC a lock of MC was acquired immediately.
- Turned on the PZT HV drivers.
- One of the Sorensens in 1X8 rack is showing the current limit sign. This is exactly the same situation as we saw before (#5592).
Currently it's off. It needs an investigation to find who is drawing such a large amount of current.
- C1SCX is not properly running. Rebooting the machine didn't help. This needs to be fixed.
The symptom is : (1) all the values are frozen in the screens. (2) the c1scx status screens shows NO SYNC sign. (3) however the timing board looks blinking happily.
- One of the VME rack on 1X3 is not showing the +/-15V green LED lights.
This is the one on very upper side of the rack, which contains the old c1lsc machine and c1iscaux2. If we are still using c1iscaux2, it needs to be fixed.
DO NOT CHANGE THE IFO ALIGNMENT UNTIL TOMORROW MORNING OR FURTHER NOTICE.
Plus, MC has to be kept locked with the WFS.
An RFAM measurement is ongoing
This is a trend for a day long showing the REFL11/55 demod signals, REFLDC (corresponding to the MC transmitted power) and the PSL booth temperatire.
There are sudden jumps in the REFL55_I and REFL11_Q signals around 5:00 AM this morning, also at the same time the temperature suddenly went up.
But the quality of the signal turned out to be not so good because the fluctuation is still within 1 bit of the ADCs,
we have to try it again with a bigger gain in the analog whitening circuit.
Right now the fluctuation of the green beat-note seems mostly covered by unknown noise which is relatively white.
The 2nd trial of the Y arm ALS noise budgeting :
(Removal of broad band noise)
I will add the dark noise of the broad-band beat-note PD and the MFD read out noise on the budget.
I found that the slow machine c1auxey, which controls and monitoring the ETMY suspension things, were not responding.
The machine responded to ping but I wasn't able to telnet to it.
I went down there and power-cycled it by keying the power of the VME rack, and then it came back and seems working properly.
I have no idea why it ran into such condition.
I have changed the MC2_YAW DC bias because the PZT1_YAW was railing.
I also realigned the steering mirrors in zig-zag path since the mode cleaner tended to resonate with higher order modes after I have changed the MC2 bias.
C1:SUS-MC2_YAW_COMM = -1.1548 => -1.1208
(The broad-band noise vs. gain of the Y end green PDH)
Last night I was trying to identify the broad band noise which is white and dominant above 20 Hz (#5970).
I found that the level of the noise depended on the servo gain of the Y end green PDH loop.
Decreasing the servo gain lowers the noise level by a factor of 2 or so. This was quite repeatable.
(I changed the gain knob of the PDH box from the minimum to a point where the servo starts oscillating)
(Malfunction in the comparator)
However I had to give up further investigations because the comparator signal suddenly became funny: sometimes it outputs signals and sometimes not.
It seems the comparator circuit became broken for some reason. I will fix it.
[Rana / Kiwamu]
As a part of the ALS noise budgeting we took a look at the Y end PDH setup to see if we are limited by an effect from the Amplitude Modulation (AM).
(AM transfer function)
(Y table setup needs more improvements)
[Rana / Mirko / Kiwamu]
The watchdogs on the MC suspensions are not working.
Switching off the watchdogs doesn't stop feeding signals to the suspensions.
For tonight, we will leave the controller of the MC suspensions switched off so that the computer won't smash the optics accidentally.
The 1X8 Sorensen's issue has been solved somehow.
As a part of the ALS noise budgeting we took a look at the Y end PDH setup to see if we are limited by an effect from the RF Amplitude Modulation (AM).
The AM transfer function of the Y end laser has been measured again, but using the frequency-doubled laser this time.
Here is the latest plot of the AM transfer function. The Y-axis is calibrated to RIN (Relative Intensity Noise) / V.
IFBW (which corresponds to a frequency resolution) was set to 100 Hz and the data was averaged about 40 times in a frequency range of 100 kHz - 400 kHz.
Also the zipped data is attached.
It is obvious that out current modulation frequency of 179 kHz (178850 Hz) is not at any of the notches.
It could potentially introduce some amount of the offset to the PDH signal, which allows the audio frequency AM noise to couple into the PDH signal.
Currently I am measuring how much offset we have had because of the mismatched modulation frequency and how much the offset can be reduced by tuning the modulation frequency.
I have restarted the c1sus machine around 9:00 PM yesterday and then shut it down around 4:00 AM this morning after a little bit of taking care of the interferomter.
c1sus has been shutdown so that the optics dont bang around. This is because the watch dogs are not working.
However I couldn't close the ALS loop somehow.
Locking activity last night:
It became able to close the ALS loop (beat-note signal was fed back to ETMY).
The UGF was about 60 Hz, but somehow I couldn't bring the UGF higher than that.
Every time when I increased the UGF more than 60 Hz, the Y end PDH was unlocked (or maybe ETMY became crazy at first).
Perhaps it could be a too much noise injection above 60 Hz, since I was using the coarse frequency discriminator.
Anyway I will try a cavity sweep and the successive noise budgeting while holding the arm length by the beat-note signal.
Another thing : I need a temperature feedback in the Y end green PDH loop, so that the PZT voltage will be offloaded to the laser temperature.
I have restarted the c1sus machine and burt-restored c1sus and c1mcs to the day before Thank giving, namely 23rd of November.
I have restarted the c1sus machine around 9:00 PM yesterday and then shut it down around 4:00 AM this morning after a little bit of taking care of the interferometer.
I reset the modulation frequency to 11065910 Hz (#5530). It had been at 11065399 Hz probably since the power shut down.
Here is a 48 hours trend of the RFAM monitor (a.k.a StochMon):
The upper plot is the DC output from the StochMon PD and the lower plot shows the calibrated RIN (Relative Intensity) at each modulation frequency.
I have downloaded minutes trends of StochMon for 48 hours staring from 6:00AM of Nov/24.
I followed Koji's calibration formula (#6009) to get the actual peak value (half of the peak-peak value) of the RF outputs and then divided them by the DC output to make them RIN.
It looks the RINs are hovering at ~ 4 x 10-4 and fluctuate from 1x10-4 to 1x10-3. Those numbers agree with what we saw before (#5616)
So it seems the StochMon is working fine.
New RFAM mon calibration
I have shut down the c1sus machine at 3:30 AM.
[Zach / Kiwamu]
Woke up the c1sus machine in order to lock PSL to MC so that we can observe the effect of not having the EOM heater.
I left the EOM stabilization running overnight, so we can finally see how the EOM temperature stabilization does over long periods of time.
The controller was turned on at ~8:40 UTC, and you can see that the Stochmon signals quiet down a lot right at that time.
Indeed the fluctuation of the RFAM became quieter with the temperature control ON.
However the absolute value of the RFAMs stayed at relatively high value.
I guess we should be able to set the right temperature setpoint such that the absolute value of the RFAM is smaller.
Here is the calibrated RFAM data (for 5 hours around the time when Zach activated the temperature control last night):
Okay I have turned ON the temperature control at 2:40 AM and will leave it ON for a while.
I was hesitant to claim that this is definitely true without the control data we were taking after the heater was turned off today. This is because before I replaced the malfunctioning op amp last night, the heater was actually ON and injecting temperature noise into the system that would not be there with it off. I think the best idea is to compare the data from today (heater on vs. heater off, but with functioning circuit).
The watchdogs' issue has been solved and they are now working fine.
It was just because one of the Sorensens had been off.
Tonight we noticed that, in fact, the watchdogs don't work for any of the corner optics (I confirmed that they do work for the ETMs).
I am going to try handing off the ALS servo to the IR PDH servo on the Y arm and measure the noise.
- first I need to investigate why the Y end PDH servo becomes unstable when the ALS is engaged with a high UGF.
So far I still kept failing to increase the UGF of the ALS servo for some reason (see #6024).
Every time when I increased the UGF more then 50 Hz, the Y arm PDH lock became unlocked. It needs an explanation and a solution.
Another thing: During several trials in this evening I found the ETMY_SUSPOS_GAIN had been set to 1, so I reset it to 20, which gives us the damping Q of about 5.
(Temperature feedback activated)
As planed in #6024 I have activated the temperature feedback, so that the PZT control signal is offloaded to the temperature. And it seems working fine.
Currently the gain is set to 0.03, which gives us a time constant of ~30 sec for offloading the control signal.
The signal observed by the coarse frequency discriminator was actually dominated by the ADC noise above 30 Hz.
It means that once increasing the UGF more than 30 Hz the servo will feed the ADC noise to the test mass and shake it unnecessarily.
I guess this could be one of the reasons of the unstable behavior in the Y end PDH lock (#6071).
(But still it doesn't fully explain the instability).
To improve the situation I am going to do the following actions:
(1) Installation of a whitening filter (probably use of SR560s)
(2) Redesign of the servo filter
Here is a brief noise budget of the coarse sensor.
Gray curve: free running noise when no servo is applied
Green curve : in-loop noise when the ALS loop is closed with the coarse frequency-discriminator. The UGF was at 30 Hz.
Red curve : ADC noise of the coarse discriminator
Eventually the instability in the Y end PDH servo turned out to be some kind of an alignment issue.
After carefully realigning the green beam to the Y arm, the UGF of the ALS loop became able to be at more than 50 Hz.
With this UGF it became able to suppress the arm motion to the ADC noise level (few 100 pm in rms).
Now I am scanning the arm length to look for a TEM00 resonance.
I have noticed that the spatial fringe pattern of the reflected green light was very sensitive to the pitch motion of ETMY when the green light was locked to the Y arm.
So I realigned the last two launching mirrors to minimize the reflected light. Indeed the misalignment was mainly in the pitch direction.
I basically translated the beam upward by a couple of mm or so.
The amount of the DC reflection is about 2.4 V when it is unlocked and it is now 0.77 mV when the green light is locked.
I guess this could be one of the reasons of the unstable behavior in the Y end PDH lock (#6071). (But still it doesn't fully explain the instability).
I succeeded in handing off the servo from that of the ALS to IR-PDH.
However the handing off was done by the coarse sensor instead of the fine sensor because I somehow kept failing to hand off the sensor from the coarse to the fine one.
The resultant rms in the IR-PDH signal was about a few 100 pm, which was fully dominated by the ADC noise of the coarse sensor.
Tomorrow I will try :
(1) Using the fine sensor.
(2) Noise budgeting with the fine sensor.
Here is the actual time series of the handing off.
Next step: Kiwamu needs to find his happy mode cleaner place, and we'll realign the PSL beam to the MC. The PSL-MC axes were mismatched pretty badly according to Suresh anyway, so this had to be done no matter what.
No real progress.
Probably I spent a bit too much time realigning the beat-note optical path.
(what I did)
Since the c1lsc machine became frozen I restarted the c1lsc machined and daqd.
Then I burtrestored c1lsc, c1ass and c1oaf to this evening. They seem running okay.
Status update of the Y arm green lock:
+ Recent goal : automation of the single arm green lock
(Things to be done)
+ Recent goal : automation of the single arm green lock
I have restarted the daqd process at 1:01 PM since I have added some new ALS's daq channels.
About Noise Budget
How to improve it ?
The 60 Hz line noise has gone away.
To test it, we are shaking all of the suspension biases +/-1.0 with a script.
The hysteresis test has been aborted.
All of the suspensions have accumulated unexpectedly big DC biases of about 5 from their nominal points.
Koji has modified the script for the hysteresis measurement.
A new test started from 16:05 PT, Dec 18th and takes a couple of hours to finish the measurement.
Do not touch the suspensions until further notice.
Here are the latest plots that I have obtained from the Friday night:
The residual motion in the arm displacements reached 70 pm in rms.
The measurement finished at ~ 21:50 PT.
I have recentered the oplev beams, including BS, ITMs and ETMs.
Another hysteresis test has begun at 1:50 PT, Dec/19.
It will finish after 3 or 4 hours. During the measurement the PSL mechanical shutter will be kept closed.
Scripting of the single arm automated lock script is 80% done.
The remaining 20 % is not something immediately needed and I start decreasing the priority on the Y arm ALS.
I made the first trial of locking a Power-recycled single arm.
This is NOT a work in the main stream,
but it gives us some prospects towards the full lock and perhaps some useful thoughts.
Lock Acquisition Steps
Actual Time Series
Below is a plot of the actual lock acquisition sequence in time series.
Here is the Gantt chart we discussed in the 40m meeting today.
Based on the discussions we had, I applied a little bit of corrections on the chart but the main stream remains the same.
Assumptions on the parameter estimations
After I did a fine alignment of the X green beam path on the PSL table, the X arm beat-note was also obtained.
Here is a picture of the latest setup. The blue lines represent S-polarizing green beams.
During I was working on the PSL table HEPA was at 80 %, and after the work I brought it to 20 %.
I have edited c1scx.ini by hand in order to acquire some green locking related channels.
Somehow c1sus.ini, c1mcs.ini, c1scx.ini and c1scy.ini are not accessible via the daqconfig script.
As far as I remember it had been accessible via daqconfig a week ago when I edited c1scy.ini.
Anyway I had to edit it by hand. They need to be fixed at some point
The picture below is a screen shot of the LSC real time model, zoomed in the new LOCKIN part.
The LOCKIN module consists of three big components:
How to set the thresholds
The below is a screenshot of the trigger matrix screen. The thresholds column is pointed by a big white arrow.
Of course, DO NOT set the upper threshold value to be smaller than that of the lower threshold. Otherwise it won't correctly work.
Also if you want to have the usual trigger rather than the Schmitt trigger, simply put the upper and lower thresholds at the same values.
Some new screens have been made for the new multiple-LOCKIN system running on the LSC realtime controller.
The medm screens are not so pretty because I didn't spend so long time for it, but it is fine for doing some actual measurements with those new screens.
So the basic works for installing the multiple-LOCKIN are done.
The attached figure is a screen shot of the LOCKIN overview window.
As usual most of the components shown in the screen are clickable and one can go to deeper levels by clicking them.