We attempted to move the ETMY suspension near the access port in preparation for the cleaning process. The plan was to move in the face restraints first to the point of almost making contact, then the ones underneath so the optic is sitting on them, followed by the top one facing down, and then bringing in the stops on the faces.
While moving in the stoppers I noticed that the far lower stopper on the HR side was barely touching the face of the optic in its resting position and was basically pushing it sideways when moved forward. It was just on the edge, so I tried to compensate minimally by moving the underneath stops a little further on the near side, trying to let it 'slide' over a little so the screw would have better contact. I must have been too generous with the adjustment, because while proceeding I noticed at some point that the stick magnets on one side of the optic were not attached anymore but laying inside the OSEMs. The side magnet was also missing, it is now sitting on the suspension jig base plate. The dumbbells all seem intact, but we'll test them before we reglue the magnets to the optic. This is extremely unfortunate, but hopefully won't take too long to fix. At the very least, as Koji put it, the cleaning will be easier with the optic out of the suspension. Still, what a bummer.
We moved ITMY from its original position to a place near the access point. We took the OSEMs off first, and noticed that the short flat head screw driver was still a little too long to properly reach the set screws for the lower OSEMs. We were able to gradually loosen them, though and thus remove the lower OSEMs as well. We had to move a cable tower out of the way, but used clamps to mark its position. After making sure the optic is held by its earthquake stops, we moved it to its cleaning location. All magnets are still attached.
With Koji's help, I've hacked together an arrangement that will allow us to monitor the output of the coil driver to the UL coil.
The arrangement consists of a short custom ribbon cable with female DB25 connectors on both ends - the particular wire sending the signal to the UL coil has a 100 ohm resistor wired in series, because the coil has resistance ~20ohm, and the output of the coil driver board has a series 200(?) ohm resistor, so by directly monitoring the voltage at this point, we may not see a glitch as it may register too small. Tangentially related: the schematic of the coil driver board suggests that the buffered output monitor has a gain of 0.5.
To monitor the voltage, I use the board to which the 4 Oplev signals are currently hooked up. Channel 7 on this particular board (corresponding to ADC channel 30 on c1scx) was conveniently wired up for some prior test, so I used this channel. Then, I modified the C1SCX model to add a testpoint to monitor the output of this ADC. Then, I turned OFF the input on the coil output filter for the UL Coil (i.e. C1:SUS-ETMX_ULCOIL_SW1) so that we can send a known, controlled signal to the UL Coil by means of awggui. Next, I added an excitation at 5 Hz, amplitude 20 counts (as the signal to the coil under normal conditions was approximately of this amplitude) to the excitation channel of the same filter module, which is the state I am leaving the setup in for the night. I have confirmed that I see this 5Hz oscillation on the monitor channel I set up. Oddly, the 0 crossings of the oscillations happen at approximately -1000 counts and not at 0 counts. I wonder where this offset is coming from? The two points I am monitoring the voltage across is shown in the attached photograph - the black clip is connected to the lead carrying the return signal from the coil.
I also wanted to set up a math block in the model itself that monitors, in addition to the raw ADC channel, a copy from which the known applied signal has been cancelled, as presumably a glitch would be more obvious in such a record. However, I was unable to access the excitation channel to the ULCOIL filter from within the SCX model. So I am just recording the raw output for tonight...
I've made a few changes to the monitoring setup in the hope we catch a glitch in the DAC output/ sus coil driver electronics. Summary of important changes:
It remains to see if we will actually be able to see the glitch in long stretches of data - it is unclear to me how big a glitch will be in terms of ADC counts.
The relevant channels are : C1:SCX-UL_DIFF_MON and C1:SCX-UL_DIFF_MON_EPICS (pardon the naming conventions as the setup is only temporary after all). Both these should be hovering around 0 in the absence of any glitching. The noise in the measured signal seems to be around 2 ADC counts. I am leaving this as is overnight, hopefully the ETMX coil drive signal chain obliges and gives us some conclusive evidence...
I have not committed any of the model changes to the SVN.
It may be advantageous to look at the coil output data from when the OSEM damping is on, to try and reproduce the real output signal amplitude that gets sent to the coils.
The amplitude of the applied signal (20) was indeed chosen to roughly match what goes to the coils normally when the OSEM damping is on.
There appears to be no evidence of a detectable glitch in the last 10 hours or so (see attachment #1 - of course this is a 16Hz channel and the full data is yet to be looked at)... I guess the verdict on this is still inconclusive.
Yesterday, I expanded the extent of the ETMX suspension coil driver investigation. I set up identical monitors for two more coils (so now we are monitoring the voltage sent to UL, UR and LL - I didn't set one up for LR because it is on a second DB25 connector). Furthermore, I increased the excitation amplitude from ~20 to ~2000 (each coil had an independent oscillator at slightly different frequency between 5Hz and 8.5 Hz), the logic being that during LSC actuation we send signals of approximately this amplitude to the coils and we wanted to see if a larger amplitude signal somehow makes the system more prone to glitches.
Over ~10 hours of observation, there is no clear evidence of any glitch. About 2 hours ago (~930am PDT Fri Jul 8), the watchdog tripped - but this was because even though I had increased the trip threshold to ~800 for the course of this investigation, megatron runs this script every 20 minutes or so that automatically reduces this threshold by 17 counts - so at some point, the threshold went lower than the coil voltage, causing the watchdog to trip. So this was not a glitch. The other break around 2am PDT earlier today was an FB crash.
Do we now go ahead and pull the suspension out, and proceed with the swap?
Move the suspension on the south clean bench and make more close inspection. We need to remove the OSEMs.
Then unmount the mirror. Bring it to the clean room and work on the bond removal.
Meanwhile, set up all suspension components inclusing the alignment test setup.
While ETMX is out, I'm leaving the larger amplitude excitations to the coils on over the weekend, in case any electronic glitch decides to rear its head over the weekend. The watchdog should be in no danger of tripping now that we have removed the ETM.
Unrelated to this work: while removing the ETMX suspension from the chamber, I also removed the large mirror that was placed inside to aid photo taking, so that there is no danger of an earthquake knocking it over and flooding the chamber with dust.
[Koji, Manasa, Sujan]
Tomorrow we'll make final checks of the optics inside the chamber.
Then we will pump down the chamber.
[q, Jenne, Manasa]
I figured out that didn't change the initial guess for the fit routine in Gabriele's code. I also changed the fminsearch criteria to least squares fitting, instead of minimax. The consistency checks now look just as good as the previous time we did these kind of measurements, no disagreements bigger than 1.6mm.
Thus, the current estimate of the SRC length after yesterday's motions is 5402mm, where we desire 5399mm. So, we will try to move SRM 3mm closer to SR2, after confirming that we are not clipping the POY beam. After all that, we will level the table.
Rough summary of today's progress:
I didn't really see anything out of the ordinary on the ETMX suspension. Earthquake stops had clearance, OSEMS were secure, no visible glue degredation on face magnets. Inspection with green LED flashlight didn't reveal any obscene dirtieness on either face, just a few particles here and there. The top of the opic barrel unsurprisingly has a good amount of particulate. The wire grooves are way too small to resolve anything at this point, other than that they exist.
The suspension footprint is already marked, tomorrow we can move the suspension closer to the door to get an even closer look at it, before removing it from the chamber.
I'd suggest clamping and moving it to the flow bench so you can inspect with a bright light. Then remove the wire and inspect the standoff, but hurry up with getting it in the soak bath so you can start on the cleaning of the other ones.
I wonder if we're really sure that its a mechanical problem with ETMX.
Gautam tells me that the local damping was always ON when looking for the jumps. This means that the coil driver was still hooked up and we can't rule out glitches in the DAC or the coil driver.
The UL OSEM shows the biggest movement (10 microns). The LR shows the second most (6-7 microns). The others are 2x less. So its consistent with a voltage change on UL,
Is this consistent with a slip in one of the wire standoffs? I think no.
One glitch was seen to occur without a change in the output voltage monitors in ELOG 11744
I performed a visual inspection of ITMY in its natural habitat today. I did not get any great pictures from the HR side because it's located very towards the edge of the table towards the arm. Before that I checked the levelness of the table. East-west direction was fine, north-south was slightly off but still within the marks for 'level'.
The AR side had several speckles, a few of them located somewhat near the geometrical center of ITMY. The top of the barrel was worse of, as expected. The HR side was a little better, but there were a few pieces of dust? near near the center. Sample pictures are attached, I uploaded all the good ones to Picasa.
Clamps that mark the position of ITMY were already in place. I did not move the optic just yet, and we will have to move a cable block out of the way to bring ITMY near the opening for us to work on it. We will markt the position of that to preserve the weight distribution. Then we can probably take some better before/after pictures. Tomorrow I will be looking at ETMY.
It took a little time, but I relocked the IMC and realigned to the point where the PRC is flashing, visible on REFL and AS, and tiny flashes are visible in TRY.
We are getting ready to vent.
After everyone's work today (good teamwork everybody!!), we are a GO for the vent.
Steve, please check the jam nuts, and begin the vent when you get in. Thanks.
I have just centered IPPOS, as well as PSL POS and PSL ANG (also called IOO POS and IOO ANG on the screens). Annalisa is working on placing mirrors to get the IPANG beam to its QPD, so that one will be centered later.
Green steering mirrors have been swapped with PZT mirrors at the X end table. We aligned the green to the X arm.
X arm green transmission +PSL green ~ 0.95
That's better than before the swap...woohooo
Centering of the oplev beams: done
Recording the OSEM values: done
There seems to be an unexplained oscillation in X arm cavity transmission for IR when the cavity is locked using the POX error signal.
Their origin is not related to the oplevs because the oscillation does not exist when LSC is OFF and the arms are controlled only by the oplevs and OSEMs.
Low power MC locking
- Rotated HWP right after the laser
- Put a knife edge beam dump at the output of the PBS after the HWP.
- Replaced the PO mirror for the MC refl by an HR mirror.
Input offset from 0 to 0.29
Servo Gain from 10 to 30
=> Transmission 0.84 (1.2W at the MC input) to 0.069 (100mW)
VCO Gain from 25 to 31
MC REFL: Unlocked 3.6 Locked 0.38-0.40
[Annalisa, Manasa, Jenne, Koji]
We are working on the vent preparation.
First of all, there was no light in the interferometer.
Obviously there were lots of IFO activity in the weekend. Some were elogged, some were not.
Annalisa took her responsibility to restore the alignment and the arms recovered their flashes.
The odd thing was that the ASS got instable after we turned down the TRY PD gain from +20dB to +10dB (0dB original).
We increased the TRY gain by factor of 10 (that's the "10dB" of this PDA520. See the spec sheet) to compensate this change.
This made the ASS instable. Anyway we reduced the gain of TRY PD to 0dB. This restored the ASS.
Jenne took some more data for the QPD spectrum calibration.
Link to the vent plan
As Q mentioned in elog 10527, (prompted by Koji's email this afternoon) we are prepping the IFO for vent. Here is a copy of the pre-vent checklist from the wiki, updated as we work:
1 & 2: Locked arms on IR, ran ASS. Unlocked IFO, aligned PRM for good POP flashes, aligned SRM for symmetric AS flashes. Aligned all oplevs. Used PZTs to align Xgreen to arm. Used knobs to align Ygreen to arm. With PS:L green shutter closed, Xgreen = 0.520, Ygreen = 0.680.
3: Moved MC servo output cable that goes to ADC from OUT2 (which we had been using for monitoring AO path signals) back to its usual OUT1 (which is MC_L). This is used in the spot position measurement script. Spots at: [2.32, -0.50, 1.97, -1.11, 0.26, -1.86] mm.
4: Done -Q
5: Removed a PD that was monitoring the light coming backwards through the Faraday that sits just after the laser, just in case (confirmed that beam dump behind PD was catching beam). Other port of PBS just had regular black hole dump. Adjusted half wave plate until we had ~90mW just before injection into the vacuum.
6: Completed. Locked MC manually at transmission of ~1150, but low power autolocker isn't working. This isn't a critical thing, and can be fixed at any point during the vent. -Q
7: Shutters closed. Ready for Steve to check nuts and begin venting! -Q
I've begun prepping the IFO for the vent, and completed most of the IFO related items on the checklist. The power into the MC has been cut, but the low-power autolocker has not been checked. I will finish up tomorrow and post the go ahead. PSL shutter is closed for tonight.
Following the checklist, I did these:
@Steve & Chub, we are ready to vent tomorrow (Monday Nov 19).
I believe this completes the non-Chub portions of the pre-vent checklist, we will start letting air into the main volume ASAP tomorrow morning after crossing off the remaining items.
Main goal of this vent is to investigate the oddness of the YARM suspensions. I leave the PSL NPRO on overnight in the interest of data gathering, it's been running ~10 hrs now - I suspect it'll turn itself off before we are ready to vent in the AM.
After lengthy discussion, I have determined that we should vent now, with the primary goal of this vent to replace the input steering PZTs with the new active tip-tilts. Since we still don't understand exactly what's going on with the PRC, it's unclear what we would do to attack the problem. We need to do more modeling and measurements first. We should limit the goal of this vent to replacing the PZTs, and then close up and do more measurements with better modeling and improved input point in hand.
There is limited time this week before everyone leaves for two-week holidays on Thursday or Friday. The reason to not vent would be that we don't want to leave the IFO at air during the two week holiday. People seem to think that this is not a problem, so we don't gain anything by waiting. Therefore we vent now and do what we can before people take off.
Now: Jenne and Manasa are doing vent prep. Manasa is lowering the input power and preparing the mode cleaner. Jenne centered IPPOS and IPANG. This will allow us to check how the input alignment changed during the vent.
12/18: Steve is going to start the vent as soon as he gets back from the dentist, at around 10am. He will regulate the vent such that we are ready to lock the mode cleaner by 4pm. At that time we will lock the MC and recheck the input alignment with IPPOS/ANG, with the idea being to see if anything moves during the vent.
12/19: First thing in the morning we take off the access connector ONLY. The access connector is all we'll need to replace PZT1. Put light door on the OMC chamber side immediately, since we won't need to get in there at all. We won't need the light access connector.
For the rest of Wednesday we'll remove PZT1 and install the new active TT. We'll be using the current out-of-vac cable for PZT1 for the new TT. We should only have to modify the rack end of the cable to accommodate the coil driver. This should be a small modification. Given that we have no wiring diagrams we'll have to pin it out in situ.
12/20: Hopefully finish up TT installation.
Jenne leaves 12/20, Manasa and Jamie leave 12/21. We will either leave light doors on access connector holes, or possibly Rana, Koji, and Steve will replace access connector on Friday so that we can pump down to 1 Torr or something so that we leave it there over the holiday.
PZT2/TT2 installation. This will be less straight forward since the new TT has a bigger foot print than PZT2 and will block the PRM optical levers. We'll need one additional steering mirror to redirect the oplev around the TT. See elog 7815.
Once the new TTs are installed, we'll reevaluate where we're at. If PRC modeling has progressed and we have an idea of something to work on with the PRC, we can. Otherwise, we'll just button up, pump down, and get on with some better PRC measurements.
I aligned the PSL angle and position QPDs...then attenuated the input laser power using HWP-PBS-HWP combo from 1.28W to 100mW following instructions from elog 6892 and elog 7299.
To enable MC locking, I replaced the 10% BS before the MC_REFL PD with Y1 mirror. I tweaked the steering mirrors at the PSL table a teeny tiny bit to enable MC locking. MC is now locked at low power with 1.0 transmission.
The access connector is all we'll need to replace PZT1.
Really? Can you be sure of the input pointing this way?
We only anticipate opening up the IOO chamber and the EY chamber.
Vent preparation: see here.
@SV, we are ready to vent tomorrow. Aaron is supposed to show up ~830am to assist.
Thanks for the good preparation. IFO pressure P1=20 Torr
Jam nuts checked. Oplev servos turned off. Particle count checked. Vertex crane functionality checked.
ANTS ALERT please watch out for ants. We have them in the control room.
We are now ready to take the doors off. I've already done the basic prep work (loosened bolts, cleaned chamber, carts for tools, fresh ameristat on portable HEPAs etc).
If everything else looks good, I'll start letting the dry N2 into the main volume after lunch.
- The vacuum chambers have been vented.
- The north heavy door of the BS chamber has been opened by Genie (not by the crane).
It was replaced by the light door. The door is currently closed.
- The MC has been locked with 20mW incident and aligned. MC REFL was left unchanged but lock was able to be achieved.
- The optics before MCT CCD and MCT QPD have been adjusted for the low power operation.
- The first HWP for the variable optical attanuator (HWP/PBS/HWP pair) was set to be 86deg from the maximum transmission at 126deg.
The incident power of 19mW has been measured.
- The PSL mechanical shutter has been manually opened. Two other beam blocks has been removed.
- I found the MC was totaly misaligned with no resonance.- I tried to align it based on the previous OSEM values but in vain.
• How to align the MC mirrors from the scratch
- MC1 has been aligned so as to maximize REFL PD and DC signal of WFS QPDs.
- MC3 has been aligned by looking at the scattered light on the MC2 frames. The spot is centered on the MC2 approximately.
- MC2 has been aligned so that any resonance is seen in MC_F.
• Modification of the MCT optics
- The ND filter before the MCT was removed.
- The Y1-45S mirror before the MCT CCD, which is also used to steer the beam to the MCT QPD path, was replaced to BS1-50-45P.
The reason I used 45P is to obtain higher reflectivity. Because S has higher reflectivity than P in the each layer, I expected to have higher reflectivity for S than 50%.
- The MC REFL path has not been untouched.
• Modification of the servo
- The lock was attempted after alignment of the mirrors. Here how to lock the MC is described below.
1. Run script/MC/mcloopson
2. Open the MC Servo screen in MEDM
3. Change the input gain from 6dB to 22dB.
Change offset from 0.78 to -0.464 (such that the length output has no offset).
Change VCO Gain from 3dB to 21dB
Change MC Length path Gain from 0.3 to 1.6
The IMC isn't resonant for a TEM00 mode at the time of writing - we are waiting for the stack to relax, at which point if the IMC isn't resonant for a TEM00 mode, we will tweak the input pointing into the IMC (we want to use the suspended cavity as the reference, since it is presumably more reliable than the table from which we removed ~50 kgs of weight and shifted the balance.
So all the primary vent objectives have been achieved 🙌 . The light doors are on the chamber right now. I'm measuring the free-swinging spectra of ETMY overnight. Barring any catastrophic failures and provided all required personnel are available, we will do the final pre-close-up checks, put the heavy doors back on, and pump down starting 10 am Monday, 9 Nov 2020. Some photos here.
Basic IFO alignment checks were done.
Tomorrow, we should do some visual checks of the chambers / EQ stops on ETMY etc but I don't see any major problems at the moment...
Barring any catastrophic failures and provided all required personnel are available, we will do the final pre-close-up checks, put the heavy doors back on, and pump down starting 10 am Monday, 9 Nov 2020.
> I didn't bother to align the green beams to the arm cavities or re-center the Oplevs - is this necessary? It is a step in the pre-close up checklist, so maybe we should do it... The green transmission does reach the PSL table...
I don't think so. The beam is reaching the PSL, so we have no motivation to change the green alignment. Regarding the oplev, the green refl should come back to the PDH PD and this gives us additional beam reference. As soon as we find the green resonance after the pumping, we can tweak the green axis so that the spots on the mirrors become reasonable (as well as the green trans CCD on the PSL table).
[Chub, Jordan, Anchal, Koji]
- Checked the main volume is isolated.
- TP1 and TP2 were made isolated from other volumes. Stopped TP1. Closed V4 to isolate TP1 from TP2.
- TP3 was made isolated. TP3 was stopped.
- We wanted to vent annuli, but it was not allowed as VA6 was open. We closed VA6 and vented the annuli with VAVEE.
- We wanted to vent the volume between VA6, V5, VM3, V7 together with TP1. So V7 was opened. This did not change the TP1 pressure (P2 = 1.7mmTorr) .
- We wanted to connect the TP1 volume with the main volume. But this was not allowed as TP1 was not rotating. We will vent TP1 through TP2 once the vent of the main volume is done.
- Satrted venting the main volume@Oct 18, 2021 9:45AM PDT
- We started from 10mTorr/min, and increased the vent speed to 200mTorr/min, 700mTorr/min, and now it is 1Torr/min @ 20Torr
- 280Torr @11:50AM
- 1atm @~2PM
We wanted to vent TP1. We rerun the TP2 and tried to slowly introduce the air via TP2. But the interlock prevents the action.
Right now the magenta volume in the attachment is still ~1mTorr. Do we want to open the gate valves manually? Or stop the interlock process so that we can bypass it?
[Chub, Jordan, Yehonathan, Anchal, Koji]
North door of the BS chamber opened