We have stabilized the ALS for Y arm and concluded that although the PDH servo could be stabilized, it drifts and loses stability over a span of few hours. (See masayuki's elog today)
We wanted to follow the same systematic procedure like in the previous elog to look at the condition of the X arm as well.
In order to stabilize the green PDH servo, we held the arm using the IR PDH and aligned the end-green to the X arm.
We see 2 TEM00-like modes and one oblong TEM00+TEM01 mode that can lock to the cavity. It is not clear to me as yet as to how to differentiate between these 2 TEM-00 like modes and how we should decide between them.
One of the TEM00-like mode is strongly matched to the arm cavity. Normalized GTRX measures 0.6 counts. The other TEM00-like mode is weakly matched to the cavity. Normalized GTRX measures 0.12 counts. This might be the reason why Jenne and Masayuki were seeing a lower beat amplitude. Camera images are shown below.
On another note, we found that an oblong mode (looks like a TEM00+TEM10 mode) also locks to the cavity. The mode looks weird in that, only one half of the mode is seen moving due to seismic noise and the other part does not. I am not sure how I can describe this...so here is a 10 second video of how this mode looks like.
Looking into control signals and error signals of the Y arm green PDH servo,
1. The saturation of feedback signal (PZT_OUT) at +/-4000 counts (less than 5V) comes from only the readout saturating. The signals looked fine on the oscilloscope.
2. We did a sine sweep at the PZT_OUT and optimized the LO frequency. The LO frequency did not need any change.
3. The error signal has some offset to it. We are not sure where this comes from.
We have been seeing that whenever green loses lock, the spot position moves down in pitch on the ETMYF camera and the GTRY camera. This led us to think about if the lock loss originated from the PDH or from the cavity.
We looked into dataviewer channels of green, IR, oplev and suspension for the following cases:
1. Green and IR PDH locked
2. Green locked and arms flashing for IR
3. Green shutter closed and IR PDH locked
4. Green shutter closed and arms flashing for IR
5. Arms flashing for IR and ETMY oplev servo turned off.
Dataviewer snapshots of glitch in all the above cases are saved in masayuki's folder users/masayuki/ALS/kicked_mirror/
In all the above cases, we could still see the glitch. We could conclude that the problem lied with the ETMY SUS.
Shown below is the dataviewer snapshot of ETMY sus and shadow sensor channels. The glitch exists even when the oplev servo is turned off pointing to problems associated with the ETMY suspension.
The MC had been unlocked for the last 4 hours and was crying out to me so I gave it some attention. Its happier now.
From the trend (AtM #1), I saw that the MC2 suspension has moved by ~10 microradians. Since the MC cavity divergence angle is lambda/(pi*w0) ~ 200 microradians, this isn't so much, but enough to cause it to lock on bad modes sometimes. Attackmint too shows that there's not much in monotonic drift over the last 40 nights.
I moved back MC2 to its old alignment with these commands:
ezcaservo -r C1:SUS-MC2_SUSPIT_INMON -s -1017 -g 0.0009 C1:SUS-MC2_PIT_COMM -t 300
ezcaservo -r C1:SUS-MC2_SUSYAW_INMON -s 490 -g 0.0009 C1:SUS-MC2_YAW_COMM -t 332
Then I went out to the table and aligned the beam into MC using the last two steering mirrors good enough so that the WFS coming on doesn't make the visibility any better. In this nominal state, I unlocked the MC and then aligned the reflected beam onto the center of the LSC PD as well as the WFS. The beam on the first WFS is a little small - next time someone wants to improve our Gouy phase telescope, we might try to make it bigger there. On the LSC PD, the beam was off-center by a few hundred microns.
Masayuki was running LAN cables near the MC2 chamber. This caused the MC2 suspension to move and unlocked the MC. I looked at the long term (2 days) and short term (2 hours) trend of the MC suspensions. I restored the old alignment as described above using ezcaservo.
C1:SUS-MC2_SUSPIT_INMON was restored to 1020 and C1:SUS-MC2_SUSYAW_INMON was restored to 490.
Attachment: Dataviewer trend (2 hours)
MC has not been very happy since last night.
What I did to fix this:
1. Disabled autolocker and WFS and aligned the MC to bring MC REFL down to <0.50
2. When I re-enabled autolocker, MC was losing lock everytime WFS turned ON.
3. I relocked MC, measured the spot positions and moved MC spot positions by running /opt/rtcds/caltech/c1/scripts/ASS/MC/mcassMCdecenter
4. I reset the WFS offsets by running /opt/rtcds/caltech/c1/scripts/MC/WFS/WFS_FilterBank_offsets
5. MC is locked and looks happy right now with REFL DCMON at ~0.5
8 day minute trend of some of the IMC alignment signals.
That step ~2 days ago in the WFS2 yaw control signal shows that I didn't do such a good job on yaw.
Nic is going to come over some time and give us a new Gouy telescope that let's us have bigger beams on the WFS. At LLO, Hartmut demonstrated recently how bigger beams can reduce offsets somehow...mechanism TBD.
Also, we must angle the WFS and figure out how to dump the reflections at the same time that we rework the table for the telescope.
Steve, can you please put 2 mounted razor dumps near the WFS for this purpose??
Tuesday: Razor dumps are waiting for you.
The MC has not been able to hold lock for over a couple of hours since yesterday. I aligned the MC yesterday (elog 9320) and it lost lock in a couple of hours. I realigned the MC again around noon today only to see it drifting and lose lock again.
I have attached the MC trend for the 2 hours when the MC drifted slowly from its happy to sad state.
Information acknowledged from Steve:
The last steering mirror mount for IR on the PSL was swapped for a more robust one. Prior to swapping the ibeam positions on the PSL IOO QPDS in ang and pos were recorded.
What I did henceforth:
1. Once the last steering mirror was installed, I walked the beam to restore input pointing using the last 2 steering mirrors. It needed a lot of work in yaw as expected.
2. When the input pointing was recovered, MC locked right away in TEM00. I measured the MC spot positions and compared it with Jenne's measurement made prior to the swap. The spot positions were pretty close.
3. The input pointing was adjusted in pitch and yaw (on the last steering mirror) in small steps. MC alignment was recovered and spot positions were measured each time. After several iterations, the MC spot positions were pretty much centered. I recentered the WFS and reset the WFS offsets. MC is now locked with WFS enabled at ~16900 counts.
4. Since the arms were aligned this morning, I used the Y arm as reference and corrected for the input pointing using tip-tilts.
5. Arms locked right away. Note: ASS doesn't seem to be doing it's job. I had to manually align the arms for maximum on TRX and TRY.
6. MICH and PRMI lock were also recovered.
7. I started to check the status with ALS as well. But for reasons unknown, I don't see any ADC counts corresponding to the beat note. Looking at the beatbox there aren't any signs of disconnected cables. I am saving this as a morning job to fix it.
The power supply to the ADC box on the IOO rack (that reads the beat I & Q signals) was pulled out because it did not run through any fuse and was connected directly to the power supply.
There were already connections running from the +/-5 V power supply. They were powering the mode cleaner demod board rack. In order to remove the ADC power connectors from the power supply, I notified Jenne in the control room because turning off the power supply would affect the MC. I switched off the +/-5V power supplies at the same time. The ADC power connectors were removed. The +/-5V power supplies were then turned ON again at the same time. Jenne relocked the MC after this.
I have still not connected the ADC to the fuse rack power supply because this requires the +/-5V power supplies to be turned OFF again in order to pull out new connections from the fuse rack and I need to make a new ADC power connector with thicker wires.
I switched OFF the +/-5V power supplies on the IOO rack to hook up the ADC power connectors through 250mA fuses to +/-5V. Since these power supplies were powering the MC demod boards, MC remained unlocked during the process. I turned the power supplies back ON and MC relocked itself after this.
This problem is now worse - the sliders on IFO_ALIGN for ETMY are white. I can't telnet to the machine either, although auxex works okay. Rather, it looks like maybe I'm getting to auxey, but then I'm immediately booted. I can ping both c1auxex and c1auxey with no problem.
Heeeeelllp please. Is this just a "shut off, then turn back on" problem? I'm wary of hard rebooting things, with all the warnings and threats in the elog lately. I've sent an email to Jamie to ping him.
There are some vague instructions in the wiki, but they begin at doing the burt restores, not actually restarting the computers: wiki Back in July, elog 8858 was written, from which the wiki instructions seem to be based. But in the elog it says "...went to the /cvs/cds/caltech/target/ area and started to (one by one) inspect all of the targets to see if they were alive.", but I don't know what "inspected" means in this case. I probably should, since I've been here for something like a millennia, but I don't.
This is what was done (as I recollect) when we said "inspected":Tenet into the computer, ping them and look at the status. Since c1auxey is not responding, here is how c1auxex responds.
controls@rossa:/cvs/cds/caltech/target 0$ telnet c1auxex
Connected to c1auxex.martian.
Escape character is '^]'.
c1auxex > h
value = 0 = 0x0
c1auxex > i
NAME ENTRY TID PRI STATUS PC SP ERRNO DELAY
---------- ------------ -------- --- ---------- -------- -------- ------- -----
tExcTask _excTask fde244 0 PEND 87094 fde1ac 3006b 0
tLogTask _logTask fdb944 0 PEND 87094 fdb8a8 0 0
tShell _shell ddad00 1 READY 6d974 dda9c8 3d0001 0
tRlogind _rlogind fbc11c 2 PEND 2b604 fbbdf4 0 0
tTelnetd _telnetd fba278 2 PEND 2b604 fba1a8 0 0
tTelnetOutT_telnetOutTa db7578 2 READY 2b604 db72e0 0 0
tTelnetInTa_telnetInTas db6060 2 READY 2b5dc db5d68 0 0
callback _callbackTas f7941c 40 PEND 2b604 f793d4 0 0
scanEvent ee7ca8 ecacb4 41 PEND 2b604 ecac6c 0 0
tNetTask _netTask fd75b8 50 READY 6be6c fd7550 0 0
scanPeriod ee78f8 ecd554 53 READY 6d192 ecd508 0 0
scanPeriod ee78f8 f23e48 54 DELAY 6d192 f23dfc 0 6
tFtpdTask _ftpdTask fb7848 55 PEND 2b604 fb778c 0 0
scanPeriod ee78f8 f266e8 55 READY 6d192 f2669c 0 0
scanPeriod ee78f8 f38678 56 READY 6d192 f3862c 0 0
callback _callbackTas f7bcbc 57 PEND 2b604 f7bc74 0 0
scanPeriod ee78f8 f906d8 57 DELAY 6d192 f9068c 0 59
scanPeriod ee78f8 f995ac 58 DELAY 6d192 f99560 0 238
scanPeriod ee78f8 f9c908 59 DELAY 6d192 f9c8bc 0 538
callback _callbackTas fa4c1c 65 PEND 2b604 fa4bd4 0 0
scanOnce ee7764 f9f96c 65 PEND 2b604 f9f92c 0 0
epicsPrint f0501c e88fa0 70 PEND 2b604 e88f64 c0002 0
ts_Casync ee5bae f76b7c 70 DELAY 6d192 f76880 3d0004 178
tPortmapd _portmapd fb8d60 100 PEND 2b604 fb8c2c 16 0
EgRam ea00e4 fa14ac 100 PEND 2b604 fa1458 0 0
CA client _camsgtask d85878 180 PEND 2b604 d85774 3d0004 0
CA client _camsgtask df91e8 180 PEND 2b604 df90e4 0 0
CA client _camsgtask d98bf4 180 PEND 2b604 d98af0 0 0
CA client _camsgtask e03cd0 180 PEND 2b604 e03bcc 0 0
CA client _camsgtask ddf2b8 180 PEND 2b604 ddf1b4 0 0
CA client _camsgtask faaec8 180 PEND 2b604 faadc4 0 0
CA client _camsgtask d79f3c 180 PEND 2b604 d79e38 0 0
CA TCP _req_server f305dc 181 PEND 2b604 f30540 0 0
CA repeaterf109e2 f215a8 181 PEND 2b604 f21474 0 0
CA event _event_task d7fe58 181 PEND 2b604 d7fe10 0 0
CA event _event_task d6ce5c 181 PEND 2b604 d6ce14 0 0
CA event _event_task dab7e0 181 PEND 2b604 dab798 0 0
CA event _event_task d76efc 181 PEND 2b604 d76eb4 0 0
CA event _event_task d9bddc 181 PEND 2b604 d9bd94 0 0
CA event _event_task d9a864 181 PEND 2b604 d9a81c 0 0
CA event _event_task da8d8c 181 PEND 2b604 da8d44 0 0
CA UDP _cast_server f2f064 182 READY efcabe f2efe4 0 0
CA online _rsrv_online f2d84c 183 DELAY 6d192 f2d7bc 0 265
EV save_res_event_task de88dc 189 PEND 2b604 de8894 3006b 0
save_restor_save_restor df61cc 190 PEND 2b604 df5c44 3d0002 0
RD save_res_cac_recv_ta fb47d8 191 READY 2b604 fb46a4 3d0004 0
logRestart f05d42 e861c0 200 PEND+T 2b604 e86174 33 1714
taskwd ef4d46 e85030 200 DELAY 6d192 e84f7c 0 224
value = 0 = 0x0
I have just turned on the PSL Innolight laser. The laser shut down with unknown reason a day ago.
PSL NPRO shut down again today for reasons unknown. I was working near the IOO rack and noticed there was no light at both the refl and trans PMC cameras. Jenne and I checked the PSL and found the 'OFF' red switch on the laser driver lit up. Switching ON the green button brought the laser back. PMC and MC autolocked after this.
After I aligned the IR interferometer (no ASS - we still need to figure out what's going on with that), I am trying to find the green beatnotes for each arm.
First, I locked the green lasers to each arm.
I then went out to the PSL table and aligned the Green Yarm path by overlapping the near-field and far-field of the yarm transmission and the PSL green pickoff. I then turned on the power for the Beat PDs, since it was off (I confirmed that the outputs were plugged into the beatbox, so they are seeing 50 ohms). I assume that the beat PDs were off since Manasa pulled the Beatbox last week, but there is no elog reference!! Anyhow, after seeing a real signal, I maximized the DC power on the beat PD for the Yarm. I then maximized the light on the DC transmission PD for the Yarm.
I looked at the Xarm, and the near-field alignment looks okay, but I haven't checked the far-field.
I started looking for the beatnotes from the control room:
I am changing the SLOW_SERVO2_OFFSETs by 30 counts, and then unlocking and relocking the arms, and checking to see if I see a peak on the RF spectrum analyser.
The Y offset started at -10320, and I found a beatnote at -11230 (beatnote is about 26MHz). The X offset started at 4500. Going larger seemed to get me to a less bright TEM00 mode, so I switched and have been searching by going down in offset, but haven't yet found the beatnote. I suspect that I actually need to align the X path on the PSL table. The Y beatnote is very small, about -30dBm, so I also need to tweak the alignment by maximizing the peak value.
I found the beatnotes for both the X and Y arm ALS this morning. The beat amplitudes measured -5dBm and -18dBm respectively and occurred at SLOW SERVO2 OFFSET 4550 and -10340. I had to only tweak the Y green PSL alignment to increase the beat amplitude.
I locked both the arms using ALS and they were stably locked until MC unlocked for a moment (nearly 16 minutes).
The only thing missing in the list of things you looked into is the status of the PSL slow actuator adjust. Check if this is near zero.
Since we have never tried to lock PRMI on carrier after the folding mirrors were flipped, I tried to lock PRCL on carrier.
I thought this might give us some idea about the PRC stability for resonance or some clue as to what happens to the PRM suspensions and PRMI stability when we have carrier resonating in the cavity.
I changed the sign of the PRCL gain and also tried increasing the gain. But this did not work and I was not able to carrier lock PRMI. May be I am missing to change some parameter that is very trivial?
Since we have never tried to lock PRMI on carrier after the folding mirrors were flipped, I tried to lock PRCL on carrier.
PRMI could not be locked on carrier using 3f. The configuration from the last time when PRMI was carrier locked (elog) were used and PRMI locked on carrier with these settings.
== PRMI carrier ==
MICH: AS55_Q_ERR, AS55_PHASE_R = -12 deg, MICH_GAIN = -0.2, feedback to ITMX(-1),ITMY(+1)
PRCL: REFL55_I_ERR, REFL55_PHASE_R = 70 deg, PRCL_GAIN = 1.0, feedback to PRM
Below is the video capture showing the PRM front and back face when carrier flashes with few second locks.
EDIT by JCD:
The demod phase numbers that Manasa is quoting above were correct back in March, when the elog she's quoting from was written. They are not true now, since we've adjusted things in the last 8 months. Also, I'm using a gain of -1.5 for MICH, and +1.5 for PRCL. MICH has no FMs triggered, PRCL has FM 2,3,6 triggered. Since we won't be using this configuration for full locking, but just for some tests, I'm currently using AS55 Q for MICH, and REFL 55I for PRCL, and using the ITMs to actuate on MICH for today.
The trend shows a big jolt to the MC1/3 pointing this morning at 8:30.
If not, we will have to put a 'no janitor' sign on all of the 40m doors permanently to prevent mops misaligning our interferometer.
The MC trend for the last 2 days shows that the MC suspensions were kicked again earlier today. Looking back at the suspension channel INMONs along with the MC trans sum shows that the suspensions get kicked everytime MC locks and unlocks. (Attch:1)
So I checked the effect of WFS on the suspensions by disabling and enabling the WFS feedback servo (Attch:2).
Since the IMC is not at it best pointing, whenever the MC autolocker runs and enables the WFS, the suspensions look like they are getting kicked. But really, it's just the WFS doing their job.
Edit, JCD: What this really means is that our DC MC pointing is bad, and we need to move the MC suspensions to offload the WFS. (All of the WFS output numbers for MC1 and 3 were around 100, which is pretty big for those numbers). We should resurrect the WFS offloading scripts so that we can do this more regularly, and not have to do it by hand.
Aligned MC to offload the WFS
1. Turned OFF the WFS feedback servo.
2. Aligned the MC suspensions by moving the pit and yaw sliders. MC trans sum brought from ~11000 counts to ~15000 counts. MC RFPD DCMON reads 0.45 counts.
3. Turned ON the WFS servo. The WFS output now reads in the order of 0 to +/-15.
4. Measured the MC spot positions. The spot positions look like they moved for the better compared to what they were yesterday.
We turned off the WFS servos, and looked at the MC REFL DC, and saw that it was still good, so we said that since the MC spots are pretty good, that we'll keep this alignment for now.
Rana put the beam back on the center of the IOO QPDs on the PSL table.
We switched a steering mirror in the WFS path that was the wrong handed-ness to be the correct handed-ness, then put the beam on the centers of the WFS. We turned on the WFS, and everything seems good.
There were no major drifts in the WFS error signals while we were gone for dinner, so the MC seems okay for now.
The last 4 hour trend for WFS error signals show some amount of drift. We should still look at the long term trend to solve the issue.
For the IPANG telescope design, we are in the 'beyond the Rayleigh range' regime. So using a single lens to make the beam small is not a great idea. I have put down a solution where we use a pair of lenses; one of which will be mounted in-vacuum in the ETMY chamber and the other on the endtable.
This way we will also allow have some freedom to configure the layout out-of vacuum in case the need arises. The layout will look something like in the cartoon:
I also made a choice of using longer focal length lenses (CVI 2" lenses f =1 m). Below is the beam path summary for IPANG telescope. I have used the waist diameter at the ITM for propagation. The endtable is roughly at 41.2m. The QPD will be placed in front of the waist (w0=47um).
Since this morning, the fb's timing has been off. Steve pointed it out to me earlier today, but I didn't have a chance to look at it until now.
This was different from the more common problem of the mx stream needing to be restarted - that causes 3 red blocks per core, on all cores on a computer, but it doesn't have to be every computer. This was only one red block per core in the CDS FE status screen, but it was on every core on every computer.
The error message, when you click into the details of a single core, was 0x4000. I elog searched for that, and found elog 6920, which says that this is a timing issue with the frame builder. Since Jamie had already set things on nodus' config correctly, all I did was reconnect the fb to the ntp:
fb$ sudo /etc/init.d/ntp-client restart
As in elog 6920, the daqd stopped, then restarted itself, and cleared the error message. It looks like everything is good again.
I suspect (without proof) that this may have to do with the campus network being down this morning, so the computers couldn't sync up with the outside world.
The above timing problem has been repeating (a couple of times this week so far). It does not seem to be related to the campus network.
The same solution was applied.
[EricQ, Gabriele, Manasa]
We found we had lost the Y arm pointing from yesterday. We tried to recover the pointing for a couple of hours and finally decided to take the ETMY heavy door off.
The input beam was aligned to the Y arm. We also got AS and REFL out of vacuum and on the cameras.
We put back the light doors and tried to lock the arms, but did not succeed as yet.
Things to do:
1. Lock arms for IR
2. Realign POP path
3. Recenter all oplevs
4. Try to check the state of PRC after the length change
5. Take in-vacuum pictures
We are close to the end of the vent except for a couple of issues.
* POP is not visible on the IR card. But we see POP flashes unclipped on the camera and also spikes in POP DC. So we are assuming that the POP path hasn't gone far off. If anybody has suggestions for a better method to check this, we could give it a try.
* PRM suspension has not been behaving well. PRM is being kicked around every 5-10 seconds when the PRC is aligned (as seen on REFL camera). We are not sure where this is coming from. The first time we saw this happening was when we were trying to lock PRC at low power even before we took the heavy doors off. So we are pretty sure this is not caused by the foil cover on the OSEMs. We tried turning ON/OFF the oplev servo, turning ON/OFF the damping loops and also checked the connections in the feedthrough and satellite box for the PRM. The OSEM sensor values for the suspension also seem to match the ones on the wiki.
GET CAMERA IMAGES OF EVERYTHING
Check all OpLevs centered, in and out of vacuum
Close PSL shutter & green shutters at the ends
This is solved.
The ASC for PRC for left turned ON. Turning it OFF solved the problem.
If there is no feedback regarding the POP alignment or anything to check with modified PRC length, we will close tomorrow morning.
This sounds great! The only suggestion that I have is for checking POP. If you have the beam on the camera, you can hold a card in front of each mirror, and find out where the edge of the beam is. Introduce the card from the side, and watch for the point where you just start to see the beam on the camera be obstructed. Repeat for the other side, and you have an idea of the centering of the beam.
I think this is most important for the in-vac mirrors, since the beam is large-ish, and we have to hit both steering mirrors at ~45 degrees.
This check was done and we had to move one of the steering mirrors in pitch. Else, everything was just fine.
In-vacuum pictures of PR2 and PR3 new positions were taken. MC spot positions measured to be < 1mm and oplevs were centered.
I checked the alignment one last time. The arms locked, PRM aligned, oplevs centered.
We went ahead and put the heavy doors ON. Steve is pumping down now!
Y arm green: Nothing much was disturbed. I touched the steering mirrors and brought GTRY from 0.2 to 0.9.
X arm green: The PDH lock was not very stable mostly because of the low power in green. I changed the oven temperature for the doubler to 36.4 corresponding to maximum green power. GTRX increased from 0.1 to 0.9
Both the X and Y arm green alignment were tuned on the PSL table to their respective beat PDs.
The PSL green shutter was not responding to the medm buttons. I found the PSL green shutter set to 'local' and 'N.O' (these are switches in the shutter controller). I do not see any elog and not sure as to why the controller was even touched in the first place. I set the shutter controls to 'remote' and 'N.C'.
The X and Y arms were locked successfully using ALS and the arms could be scanned and held to support IR resonance.
The same procedure as in elog 9219 was followed. In-loop noise was measured to be between 200-300 Hz rms for the lock.
ALS settings for the lock
X arm : FM 2, 3, 5, 6, 7, 8, 10 Gain = 11.0
Y arm : FM 2, 3, 5, 6, 7, 8, 10 Gain = 10.0
The MC has been funny since yesterday. I checked the suspensions INMON channels and they seemed ok. So I went ahead and tweaked the alignment with WFS disabled (yesterday). Although the WFS PDs were cenetered at this point, the WFS servo was throwing the MC in a not-so-happy state. We worked with the WFS servo OFF all of yesterday.
* I fine tuned the MC alignment from yesterday (TRANS_SUM > 17800 counts)
* measured the spot positions
* recentered the spots on the WFS PDs (was already quite centered)
*reset the WFS filterbank offsets.
The MC has been locked happily since then with autolocker and WFS servo enabled.
Although the morning MC tuning looked stable, Koji pointed out that the MC_REFL_OFFSET was changed from its nominal value.
The offset was reset and this caused drift in the MC_TRANS_SUM.
To fix this:
- disabled the WFS servo
- aligned MC using MC1 and MC3
- centered beam on the MC_REFL
- reset WFS offsets
- locked MC
MC looks happy now.
ALS is in a very different state from a couple of days ago when we could successfully lock the arms and scan.
The green alignment to the arms had drifted.
PSL green alignment on the PSL table was off. The PSL green was not even on the steering mirror. Did anyone work around the PSL table in the last couple of days?
After aligning and finding the beat note, I found the ALS servo very noisy. The error signal had 10 times more rms noise than what was achieved earlier this week and there were some new 60Hz peaks as well.
Overall, we could not do any PRMI+ALS arms today
We need to change several scripts for use with the new ALS-in-the-LSC paradigm:
* Watch arms (to turn off ALS if we lose the beatnote, before pushing optics too hard)
* Find IR resonance
* Offset from resonance
None of these should be difficult, just changing the filter bank names to match the new ones (ex. LSC-XARM rather than ALS-XARM, and LSC-ALSX rather than ALS-OFFSETTER1).
So far, I have changed the "find resonance" script (ALSfindIRresonance.py). I believe, in principle, to first order, that my modifications should work, however I have not yet tested the script. So. If you use it, watch the output of the script and ensure it's doing what it ought. I'll check it after the lunch meeting and update this log entry. (I changed the name of the "OFSFILT" variable, line 26, and also modified line 114. Both of those lines have comments on how to revert the changes).
I have also changed the "offset from resonance" script (ALSchangeOffset.py). Again, since I'm not locking right now, I have not tested this script either. So, pay attention if you need to use it, before I check it. (I changed the name of the OFSFILT variable, and the check which arm logic around line 37. Again, both of those lines have comments on how to revert the changes.)
Watch arms script (ALSdown.py) has been modified and now watches the LSC-$ARM filter module instead of the ALS-$ARM filter module. Threshold has been kept the same +/-5000 counts to the ETM suspensions. The script has been tested and works just fine. It exists in the same place scripts/ALS/.
Jenne's modified versions of ALSfindResonance.py and ALSchangeOffset.py were tested and work just fine.
The IMC has not been behaving well since this morning and totally not happy when Q was finishing his measurements. The WFS servo had large offsets in pitch. Looking back at the trend and using ezcaservo to restore the suspensions did not help.
I realigned the IMC and brought TRANS SUM to ~18000 and MCREFL to < 0.5. The spot positions are not very good; nearly 2 mm off in pitch on MC1 and MC3. But after the alignment of MC, the WFS servo offsets were below +/-20.
The MC has been locked stably with WFS servo ON for the last few hours.
P.S. I did not touch the WFS pointing or reset the WFS offsets.
Step by step description of transition from 2arm ALS to Common/Differential LSC for FPMI
- Step 0: Place the frequencies of the arm green beams at the opposite side of the carrier green.
- Step 1: Activate stablization loops for ALSX and ALSY simultaneously.
(Use LSC filter modules for the control. This still requires correct handling of the servo and filter module triggers)
- Step 2: Activate stablization loops for ALS Common and Differential by actuating ETMX and ETMY
I locked the arms using ALS error signals and the LSC filter modules. But when I try to acquire CARM and DARM using ALS, the arms lose lock when the matrix elements ALSX to Yarm and ALSY to X arm reach -/+0.9
What I did:
1. ALS locking of arms
(i) Found arm beat notes
(ii) Input matrix POX and POY elements set to '0'
(iii) Aux matrix elements ALSX to Xarm and ALSY to Y arm set to '1'
(iv) Power normalization matrix elements for TRX and TRY set to '0'
(v) Triggers for arm lock over ridden and the FM triggers were set to 'manual'
(vi) Arm servo gains set to '0'
(vii) All but FM5 were disabled
(viii) Phase tracker history reset and servo actuation set to ETMs
(ix) Servo gain increased in steps (+/-10 for the arms)
(x) FM1, FM6, FM7 enabled (see note 1 below)
(xi) FM9 enabled
Arms were locked with ~2000Hz rms
2. CARM and DARM locking
(i) Scanned the arms for IR resonance
(ii) Moved off-resonance (Stepped arm servo offsets by 30 counts)
(iiI) Stepped matrix elements ALSY to X arm and ALSX to Y arm ezcastep C1:LSC-PD_DOF_MTRX_6_29 +-0.1 C1:LSC-PD_DOF_MTRX_7_28 +0.1
Whenever the matrix elements reached -/+0.9, the arms were kicked out of lock. I don't see anything obvious as to why this is happening even after nearly 10+times of redoing.
1. I found the filters for the arm servos different for X and Y. FM1 and FM8 were missing in one of the filter modules. Jenne remembered Rana modifying and removing the unnecessary filters in one arm. We put back FM1 (low pass filter) which might not be necessary for PDH lock but is necessary for ALS. FM8 is now added to FM7.
2. To self : Check ALS Y arm power outlets (60Hz frequency comb seen in the error signal)
MC remained locked with WFS enabled all through last night and this morning. Koji dropped by and looked at the MC. The MC WFS servo, though stable, was at the edge of becoming unstable. This was because I did not touch the WFS pointing on the QPDs yesterday after realigning. So I recentered the WFS, reset the WFS filterbank offsets and reenabled the servo.
I measured the spot positions on MC mirrors for reference.
Spot positions in mm (MC1,2,3 pit MC1,2,3 yaw): [1.405767579680834, 0.79369009503571208, 1.3220430681427462, -1.2937873599406551, -1.1704264340968924, -1.2518046122798692]
ALS common locked by actuating on MC2 and ALS Differential locked by actuating on ETMX and ETMY (Stable lock acquired for over an hour).
Common and Differential offsets were swept to obtain IR resonance in both the arms (arms stayed on resonance for over 15 minutes).
1. Configured LSC settings to allow locking using ALS error signals.
2. Locked common and differential using ALS error signals
XARM 1 -1
YARM 1 1
X arm servo settings:
FIlters: FM1, FM5, FM6, FM7, FM9
Gain = -8.0
Y arm servo settings:
Filters: FM1, FM5, FM6, FM7, FM9
Gain = +8.0
ETMX 1 0
ETMY 0 1
3. Transitioned CARM control output to actuate on MC2 instead of ETMX
SUS-MC2_LSC servo gain = 1.0
The transition was done in very small steps : actuating on MC2 in -0.01 steps at the outmatrix upto -1.0 while reducing the ETMX actuation to 0 simultaneously.
DARM still stayed locked only with actuation on ETMY.
4. Transitioned DARM control to ETMX and ETMY.
Used ezcastep to step up DARM control (Y arm output) actuation on ETMX and step down the actuation on ETMY.
Final output matrix
ETMX 0 -0.5
ETMY 0 0.5
MC2 -1.0 0
Noise plot in attachment.
5. Finding arm resonance
Used ezcastep to gradually build up offsets in CARM (LSC-XARM_OFS) to find IR resoance in one arm (Y arm).
Introducing a small (order of 0.5) DARM offset (LSC-YARM_OFS) shifted the Y arm off-resonance.
Used CARM offset to get back the Y arm to resonance.
Changing CARM and DARM offsets alternately while tracking the Y arm resonance got us to a point where we had both the arms resonating for IR.
6. At this point the MC decided to give up and we lost lock.
1. We found that the WFS2 YAW output filterbank had the output switched OFF (probably accidentally by one of us). This was reenabled. Please be careful while manually turning ON and OFF the MC WFS servos.
Nic, Jenne, EricQ, and Koji should describe the demonstartion of CESAR achieved tonight.
Q and I have started to use the ALS slow servo for the end aux lasers while locking the arms using ALS. The servo prevents us from hitting the limits of the PZT range for the end lasers and a better PDH locking.
But keeping the servo ON causes the slow output to drift away making it hard to find the beat note everytime the arm loses lock. The extensive beat note search following the unlock can be avoided by clearing history of the slow servo.
I have modified the IFOconfigure scripts and the corresponding .req files for the X arm and Y arm in burt. I have also added configure scripts to save and restore LSC settings for locking the arms using ALS error signals.
The settings are yet to be saved and the scripts should also be checked if they are working as required.
Off again. Restarted ntp on fb.
I tried to repeat Koji's PRMI lock using REFL165I/Q. I was not able to lock PRMI stably. All I could get was momentary PRMI sb locks (few seconds) using AS55Q for MICH and REFL165Q for PRMI. I tried to transition MICH locks from AS55Q to REFL165I/Q and this did not work well; I lost even the momentary locks.
The demod phases for both AS55 and REFL165 were also very different.
AS55 WHTN: 21dB demod phase -78.7deg
REFL165 WHTN: 45dB demod phase -80.7deg
AS55Q x1.00 MICH
MICH POP110I 100up/10down / FM Trig FM2/3/6/7/9 35up 2down 5sec delay
PRCL POP110I 100up/10down / FM Trig FM2/3/6/9 35up 2down 0.5sec delay
MICH OFS 0.0 / Gain -10 / Limiter ON
PRCL OFS 0 / Gain -0.023 / Limiter ON
MICH ITMX -1.0 / ITMY +1.0
PRCL PRM 1.0
I've also turned on the MC2 TRANS path to gather some data over the weekend on how well or bad it works. Please turn it off on Monday.
MC2_TRANS path in WFS servo turned OFF.
While I'm looking at the PRM ASC servo model, I tried to use the current servo filters for the ASC
as Manasa aligned the POP PDs and QPD yesterday. (BTW, I don't find any elog about it)
The POP PD was showing only ~200 counts which was very low compared to what we recollect from earlier PRMI locks (~400 counts). Also, the POP ASC QPD was also not well-aligned.
While holding PRMI lock on REFL55, I aligned POP path to its PD (maximize POP DC counts) and QPD (centered in pitch and yaw).
X and Y green
The X green totally lost its pointing because of the misaligned PZTs from last week's power failure. This was recovered.
Y arm green alignment was also recovered.
I have included Yarm CESAR to the LSC model. It was just a copy paste of the Xarm CESAR. Since we are now meditating about implementing CCESAR and DCESAR, I did not run or install the model as yet.
Since we don't have an arm length precision measurement (i.e. better than centimeters), why not just do as Koji suggests and use the ALS to get the frequency spacing between a few red FSR and then we have the measurement solid ?
Arm lengths measured using ALS. Both the arms were estimated to have the same length (to the order of a centimeter) 37.51 m
I used ALS error signal to lock the arms and scanned the arm to find 4 consecutive IR resonances. From the beat note frequencies measured using the spectrum analyser during IR resonance, the FSR, and hence the length of the arms were calculated.
The estimated lengths are not very precise down to a mm given the resolution of the spectrum analyser. We have brought out the rubidium clock to use as reference for the spectrum analyser to challenge the measurements.
The MC had not been stable lately with WFS drifting constantly. I checked the servo and found that the MC_TRANS path was still running. It turned out that the autolocker script enables the TRANS path in the locking process. I have turned the MC_TRANS path servo inputs OFF and now it is no more a part of the WFS servo.
P.S. Jenne fixed the PMC alignment mostly in pitch to bring it up to 0.81 from 0.77. But the temperature fluctuations have still not got us to the sweet spot for optimum PMC trans.
Arm lengths were measured using ALS
X arm length = 37.79 +/- 0.05 m
Y arm length = 37.81 +/- 0.01 m
Whats and whys
We want to measure the arm length with an accuracy of say a mm.
This would mean a measurement precision of 1e-3/40=25ppm. (1mm in 40m)
So the required measurement resolution on the spectrum analyser is 25ppm*4MHz=100Hz (assuming the cavity FSR is roughly 4MHz).
Although the spectrum analyser does not limit the measurement precision, we are limited by the noise in ALS at 1000Hz rms. So we can use ALS only to measure arm length precise to the order of a few mm.
RXA: Not that we really need to right now, but even with an ALS noise of 1000 Hz, we can can do better just by averaging at each resonance point. And fitting a line as you have already done gets even better:
The Spectrum analyser was reference locked to the rubidium clock @10MHz for these measurements.
The FSRs of the arms
X arm = 3.9671e+06 +/- 4.8535e+03 Hz
Y arm = 3.9648e+06 +/- 1.1064e+03 Hz
1&2. Plots representing the arm scans showing the beat frequency for which IR resonates in the arm vs the ALS offset (position of the ETM).
3. Data and code (zip file)
P.S. We had trouble scanning the arms using ALS. This was because the slow servo was not enabled. Hence ALS was losing its PDH lock everytime we scanned past a couple of FSRs.
Last night, as well as tonight, the ALS seems not quite as robust as it was earlier in the week.
I have just taken noise spectra, and ALS is definitely more noisy than usual.
These plots are with the arms held in CARM and DARM mode, with servo gains of 8. I was seeing the beginnings of gain peaking at a gain of 10, so I turned it back to 8. Our ALS in-loop RMS is usually something like a few hundred Hz, but I'm seeing over 1kHz, so I have a factor of 4 or 5 too much noise. Why?!?!?
I have noticed that ALS noise has been at 1KHz rms since LSC arm lock servos have been used to lock arms using ALS error signals. May be this has not been given much attention.
But looking more closely at the ALS noise (better dtt resolution for noise power spectrum) , there seems to be too much noise suppression at <1Hz and not much happening at around 10Hz.
Attachment 1 (data files at /users/manasa/data/140421/)
So I made a bunch of transfer function measurements for ALS and phase tracker servo. Koji will be using these and redesigning the servo filters so that we can get more suppression at 10Hz.
Other than this I also found that the Y arm showed more high frequency noise as compared to the X arm. (Edit by manasa: Thinking back now, this could be related to the onset of 60Hz noise at the Y end elog 9838. But still has to be looked at after fixing TRY)
Tip: Once the arms are ALS locked, enabling the SLOW_SERVO helps hold the lock stably.
P.S. I realigned the Y green to the arm and brought GTRY to 0.93
Find out what makes Y arm in-loop noise at high frequency higher than X arm.
This evening, I was not able to successfully transition CARM from ALS to 1/sqrt(trans) signals. The TRY time series looked odd, so I took a spectra, and we have huge 60Hz noise in TRY.
Manasa, can you please take a look, and see if you can figure out what is going on? We need TRY so that we can transition to 1/sqrt(trans) signals for CARM. Thanks!!
I went to the Y end to look at the TRY 60Hz noise situation this morning. While looking at TRY noise on dtt, I found that just lifting the cable away from the cable bunch that runs out of the table suppressed the noise drastically.
I removed the unwanted bnc connector in the path of the already long TRY cable running from the PD to the 1Y4 rack and isolated it from the bunch. TRY became less noisy.
But the noise was back again earlier in the evening and it looks like the noise is very much related to the TRY cable. TRY cable might have moved from its sweet spot while I was around checking cable connections yesterday.
I couldn't find a spare to replace it right away today (We need a BNC to 4 pin lemo).
To find noise source
1. Swapped the power cable of the PD and checked that it is connected to the right power source.
2. Changed the aluminium base of the post holding the diode so that the diode is floating
3. Grounded the table and the rack
4. Routed the cable on the other side of the beam tube to isolate it from other cables.
After all the above, we still found that shaking the cable was making TRY noisy.
I pulled out the PD whitening board to replace the 4 pin lemo connector with a bnc connector so that we can swap the cable with a new one. So there is no TRY right now.
The MON outputs of the Y end QPD whitening board were hot earlier today while pulling it out of the crate. After swapping the 4 pin lemo connector with an isolated panel mount bnc connector, I stuck the board back into the crate and this immediately kicked the ETMY suspension. Jenne and I went to the Y end to look at what was going on. We removed the board from the crate after smelling something burning. The MON output ports of the whitening board were super hot this time. There is no sign of any components melting on the board (comparing the board with its pictures that were taken earlier) and a tester board stuck into the crate lights up just fine.
So the back panel is still ok. We need to troubleshoot or replace the whitening board.
Edit, JCD: The attached photos are from right after I replaced the "Rgain" resistor, elog 9823. What they show is that it looks like some of the melting / burning may have already been happening before I pulled the board, and I just never noticed :( In particular, look at the resistors on the main board above the blue "G" sticker. There isn't a difference that I can tell between this photo from last week, and today's situation.
We turned on the MC2_TRANS paths for both PIT/YAW tonight.
I turned off the BLP200 and turned on the RLP7 (RLP always are better than BLP). G_PIT = -0.111, G_YAW = 0.111. On Monday, let's let Steve look at the trends and determine if this centering servo is bad or good.
The MC was showing slow but periodic alignment drifts and eventually unlocked around noon. I looked up the alignment trend (Attach: 2 day trend)
MC_TRANS_PIT_ERR and MC_TRANS_Y_ERR show that the MC_TRANS servo slowly drifted the IMC alignment causing it to lose lock from time to time (mostly in yaw).
To confirm that the drift was NOT due to off-centering in the MC2_TRANS QPD, I turned off the WFS servo, moved MC2 to recenter the trans beam on the QPD, and re-enabled WFS servo.
MC_TRANS path in WFS is still left enabled.
The Y arm locks stably for IR PDH now.
The reason for ETMY getting kicked during lock acquisition was finally found to be related to the limiter value set in the Y arm servo. We reduced the limiter value unintentionally and found that the lock acquisition stayed smooth. The limiter value was stepped in 1000s from 7000 and eventually found that the ETMY suspension was kicked when we try to acquire lock with the limiter value was set at 11000.
The limiter for X arm at 11000 is not causing any trouble at the moment.
In the process, we did a bunch of things through the evening to troubleshoot IR locking of the Y arm.
Earlier today running the IFO configure script did not restore the arms to lock and both the ETMs needed to be aligned to lock the arms. The arms stayed locked for 15 minutes and the Y arm lost lock eventually leaving the ETMY in a misaligned state.
The state of the Y arm was similar to what Jenne has explained in ELOG where the ETMY was kicked during lock acquisition and would move to a misaligned state.
To trouble shoot, there were several things that were done. A few of them might not have any direct correlation to the locking issue but could just be a coincidence.
1. The trigger time for the filters in the arm filter modules were set such that they switch on after the SUS violin filters. Arm FM trigger time = 3 s (previously set at 0.1s) and SUS violin trigger time = 1s. This reduced the number of lock loss events.
2. There was some drop in transmission when the bounce filter of Y arm (FM6) turned ON. This was fixed by changing its ramp time (initially set at 1s). The filter has been modified to turn on immediately upon arm lock acquisition before the other triggered filters in the filter module turn on.
3. The QPD and SUS signal cables running to the rack were checked to be intact. Koji found some of them to be loose. But this had no evident correlation with the arm locking problem.
4.The oplev and PD alignment was checked at the Y end. The high gain trans PD for Y arm was checked for good alignment by looking at TRY. It was found that the EXIT light at the Y end is injecting some noise to the transmission PD.
5. The ETMY was given offsets in PIT, YAW and POS and the OSEM sensor values were checked to see if the suspension is behaving well. It was behaving well.
c1sus and c1isey were not talking to fb. The usual mxstream restart did not help.
>>telnet fb 8087
All lights on the FE status screen are green now.
Note that Steve did an mxstreamrestart earlier today because the same machines c1sus and c1isey were not talking to fb.
I. The Y arm stayed stable through last night and I have saved the arm lock settings to IFOconfigure.
II. ALS X arm noise measurements
I looked at the before and after noise of ALSX.
Phase tracker gain = 85
Xarm servo gain = -17
The rms in loop noise has dropped from 3KHz to 500 Hz.
Attachment 1: Phase tracker OLTF
Attachment 2: Free running noise and in loop noise
Attachment 3: Out of loop noise (measured with arms locked using PDH for IR)
Attachment 4: ALS arm servo OLTF
xml data files can be found in /users/manasa/data/140430/