Today morning, I suspended ETMY and made the same checks dscribed below. The clamping went smoothly, 5 in. lb. of torque seems sufficient, in the limited observation time, there has been no evidence of wire sag. Today afternoon, we will go about putting the OSEM coils in, setting their equilibrium points etc. This may need to be re-done once the optic is in the chamber and the first contact has come off, but at least we can coarsely place them in the relative convenience of the cleanroom.
GV EDIT 9.15pm 22 Aug: Eric had a look at both towers and pointed out that I had neglected to use washers on the wire stops. After consultation with Steve, I decided that it is not worth it to remove the clamp and re-suspend the optic - it is likely that the current suspension process will have caused new grooves in the suspension block, which will have to be removed, and the sanding process did not work so well last time. In any case, the net effect of this will be that the actual torque with which the clamp is tightened will be slightly different from 5 in. lb., but since there is no evidence that the clamp isn't tight enough / is too tight, I think it is okay to push ahead.
We worked on trying to insert the OSEMs in the optimal positions such that the coupling of the bounce mode into the OSEM sensor signals was minimised.
First, I gave the barrel of the optic a wipe with some optical tissue + acetone in order to remove what looked like some thin fibres of dried first contact. It may be that while I was applying the F.C., the HEPA air flow deposited these on the barrel. In any case, they came off easily enough. There is still a few specks of dust on various parts of the barrel, but it is likely that these can just be removed with the ionized air jet, which we can do after putting the optic in the chamber.
We then did the usual OSEM insertion till the magnets neutral position was such that the sensor output was ~50% of the fully open value (turned the HEPA off for the remainder of this work). I tweaked the bottom OSEM plate a little in order to center the magnets relative to the coil as best as possible. Once this was done, we attempted to look at spectra of the sensor outputs, with 0.05 Hz bandwidth - however, we were unable to identify any peak at 16.4 Hz, which is what a Jan 2015 measured value wiki page claims the bounce mode frequency is (although this was an in vacuum measurement). There were a couple of peaks at ~15.7 Hz and ~16.7 Hz, but I can't think of any reason why the bounce mode resonance should have changed so much - after all, this is ETMY for which no standoff regluing was done. The only difference is that there is some first contact + peek mesh on the HR face now, but I doubt this can modify the bounce resonance frequency so much (this is just my guess, I will have to back this up with a calculation).
Anyways we decided to take this up again tomorrow. Things are progressing fairly well now, I hope to be able to put in ETMY back into the chamber at some point tomorrow and commence re-alignment of the interferometer. I've left the OSEMs in for today, with the EQ stops not engaged but close by. HEPA has been turned back on.
Summary: Today we moved the suspended ETMY optic back into the chamber from the cleanroom. Once in the chamber, we positioned the optic using the stops that marked the previous position of the optic. We then shortened the arm length by 19mm (in order to match the X and Y arm lengths. The F.C. coat on the HR face was removed prior to the final placement of the optic. We then adjusted the OSEM positions in their holders to get the sensor outputs to half their maximum value.
We did not get to check where the input beam hits the optic or see if the pitch balance of the optic is such that the reflected beam makes it back to the ITM. The plan for tomorrow is to do this.
Part 1: Cleanroom work
Part 2: Transportation of optic
Part 3: Chamber work
Plan for tomorrow:
Attachment #1: Wire is in groove in side without OSEM
Attachment #2: Wire is in groove in side with OSEM (picture taken with OSEM coil removed)
Attachment #3: UL magent relative to OSEM coil
Attachment #4: LL magent relative to OSEM coil
Attachment #5: LR magnet relative to OSEM coil
Attachment #6: UR magnet relative to OSEM coil
Attachment #7: Side magnet relative to OSEM coil
Attachment #8: ETMY HR face with F.C. film removed. Non-covered part isn't super clean, but the covered part itself does not have any large specks of dust visible.
Attachment #9: Scheme adopted to shorten Y arm length by 19mm.
Attachment #10: Current situation inside EY chamber. Counterweight that was moved to balance the table is indicated.
We've seen for some time now that one of the PRM OSEM signals has been gone, and all of the SRM signals seem dark. We had tried squishing various cables to no avail.
Today I played some "musical satellite boxes," in an attempt to see if the problems are in the chambers or in the signal chains. That is, I swapped the OSEM cables from the vacuum feedthroughs between the satellite boxes, and observed what happened.
It seems clear that something is up with SRM inside the chamber. For PRM, it's not so clear...
Somehow, issues with the LR channel follow both the PRM OSEMs and the PRM satellite box.
PRM LR first went dark on Jul 2nd, after the IFO was vented, but before we took any doors off (which happened on the 5th). I'm not sure what may have caused this.
SRM OSEMS first went dark on the evening of Jul 18, the day before ELOG 12310, when ITMY was moved in the same chamber. Maybe this ELOG was written about work the day before, but the sensors show disturbances over the course of hours. I think we need to double check the connections in chamber.
[lydia, johannes, gautam]
While struggling to minimize the bounce mode coupling into the sensor signals, we briefly poked into the ITMY chamber, and think that we understand the origin of the problem, at least for the SRM.
Essentially, we believe that moving the ITM from its nominal position to the edge of the table has shifted the table leveling such that the optic (SRM) is tilted backwards (hence the magnets are completely occluding the LEDs) and that perhaps the optic is in contact with one or more of the bottom EQ stops (hence the signal is stationary, no oscillations visible. The timing of the signals going dark as Eric mentioned supports this hypothesis. The reason why we believe this to be the case is that when I was trying to loosen the screw on the clamp holding the ITMY cage to the table, we saw ~1Hz signals from all 5 SRM OSEM sensors, though they were well away from the nominal equilibrium values. The arrangement of towers in the chamber right now did not permit me to get a good look at the SRM magnets, but I believe they are all still attached to the optic, and that they are NOT stuck to the OSEM coils. If this is indeed the case, putting ITMY back in will solve the issue completely.
It is not clear what has happened to the LR coil on the PRM - could it be that during the venting process, somehow the LR magnet got stuck to the OSEM? If so, can we free it by the usual bias jiggling?
There was some confusion as to the order in which we should go about trying to recover the Y arm. But here are the steps we decided on in the end.
Yesterday, Eric, Johannes and I tried to do step 1, but after some hours of beam walking, we were unsuccessful. Today morning, Koji suggested that the ITM wedge could be playing a part - essentially, over 40m, the wedge would shift the beam horizontally by ~30cm, which is kind of what we were seeing yesterday. That is, with 0 biases to the tip tilts, we could find the beam in the ETM chamber, towards the end of the table, ~30cm away from where it should be (since the input pointing is adjusted taking this effect into account, but we were doing all of our alignment attempts without the ITM in).
So, we shifted strategy today. The idea was to trust that the green beam was well aligned to the cavity axis (we had maximized the green transmission before the vent), and set the pitch bias voltage to ETMY by making the reflected beam overlap with itself. This was done successfully, and we needed to apply a pitch bias of ~-2.70 (value on the MEDM screen slider), which agrees well with what I was seeing in the cleanroom. We then adjusted the OSEMs to bring the sensor outputs to half their nominal maximum value. Next, we went into the ITMX chamber, and were able to find the green beam, at the right height, and approximately where we expect the center of the ITM to be (this supports the hypothesis that the green input pointing was pretty good). I am however concerned if this is truly the right value of the bias for making a cavity with the ITM, because the pre-vent value of the pitch bias slider for ETMY was at -3.7, which is a 30% difference from the current value (and I can't think of a reason why this should have changed, the standoffs weren't touched for ETMY). If we go ahead and fine tune the OSEMs rotationally assuming this is the right bias to have, we may end up with sub-optimal bounce mode coupling into the sensor signals if we have to apply a significantly larger/smaller offset to realise a cavity? The alternative is to put in the ITM, and set the pitch balance using the IR beam, and then go about rotating OSEMs. The obvious downside is that we have to peel the F.C. off, risking dirtying the ITMs.
For much of the rest of the day, we were trying to play with the rotation of the OSEM coils in order to minimize the bounce mode coupling into the sensor signals. We weren't able to come up with a good scheme to do this measurement, and I couldn't find any elog which details how this was done in the past. The problem is we have no target as to how good is good enough, and it is extremely difficult to gauge whether our rotation has improved the situation or not. For instance, with no rotation of the OSEMs, by observing the bounce mode peak height over a period of 20-30 minutes, we saw the peak height change by a factor of at least 3. This is not really surprising I guess, because the impulses that are exciting the bounce mode are stochastic (or at least they should be), and so it is very hard to make an apples to apples comparison as to whether a rotation has improved the situation on.
After some thought, the best I can come up with is the following. If anyone has better ideas or if my idea is flawed, or if this is a huge waste of time, please correct me!
Of course, this method assumes that the excitation into the bounce mode is a constant over time. I'm also attaching the spectrum of the OSEM sensor signals right now - the optic is in the chamber, free swinging (no damping) with the door on (so it is fairly quiet). The LR signal seems to be the best (indeed seems to match the levels in this plot), but it is not clear whether the others can be improved or not.
There was also some concern as to whether we will be able to see the beam in the ETMX chamber once the ITM has been re-installed. Assuming we get 100mW out of the IMC, PRM transmission of 5.5%, and ITM transmission of 1.4%, we get ~35uW incident on the ETM, which while isn't a lot, should be sufficient to see using an IR card.
I've been noticing that the ETMY UL sensor output has been erratic over the last few days. It seems to be jumping around a lot, even though there is no discernable change in any of the other sensor signals. Damping is OFF, which means the sensor signals should just be a reflection of actual test mass motion. But the fact that only one sensor output is erratic leads me to believe that the problem is in the electronics. I've also double checked that we aren't touching any EQ stops. Also, we had centered all the sensor outputs to half their maximum value pretty carefully. But looking at the Striptool traces, I now find that the UL sensor output has settled at some other value. Simply removing the OSEM connector and plugging it in again leads to the sensor output going back to the carefully centered value. Could it be that the photodiode has gone bad? If so, do we have spare OSEMs to use? I will also re-squish the satellite box cables to see if that fixes the problem.
Attachment #1: Sensor output spectra around the bounce mode peak. Nothing was touched inside the chamber between the time this spectrum was taken and the spectrum I put up last night (in fact the chamber was closed)
Attachment #2: UL sensor output is erratic, while the others show no glitching. This supports the hypothesis that the problem is electronic. The glitch itself happened while the chamber was closed.
Attachment #3: The only difference between this trace and Attachment #2 is that the UL connector was removed and plugged in (OSEM wasn't touched)
This problem has existed well before the vent
We do indeed have a box of clean spare OSEMs, it should be out with all of the other boxes of clean stuff we had for the suspension building. You could also try swapping in a different satellite box, to see if the circuit powering the OSEM PD is to blame.
We worked on reducing the bounce mode coupling into the sensor signals today. After some trial and error, essentially following the procedure I had put up in my previous elog, we think we were successful in reducing the coupling. We have now left the optic free swinging, so that we can collect some data and look at a spectrum with finer bandwidth. But as per the methodology we followed, we saw that the peak height corresponding to the bounce mode increased when we rotated the OSEM either side of its current position (except for the side OSEM, which we felt was in a good enough position to warrant not touching it and messing it up - of course only the spectrum will tell us if we are right or not. I also took some pictures with the camera with the IR filter removed, but we couldn't get any real information from these photos. I also checked with Jenne and Jamie who both suggested that they didn't have any metric with which they judged if the rotation of the OSEM was good enough or not. So we will wait to have a look at the spectrum from later tonight, and if it looks reasonable enough, I vote we move on. As Eric suggested, perhaps we can repalce the UL OSEM coil and see if that solves the apparent UL coil problem. Then we should move on to putting the arm cavity together.
Addendum 11pm 26 Aug 2016: I've uploaded the spectra - looks like our tweaking has gained us a factor of ~2 on LL, LR and SD, and no significant improvement on UL and UR compared to yesterdays spectrum.
I wanted to observe the UL coil for any excursions over the weekend. Looking at the 2 day trend, something is definitely wrong. These glitches/excursions are much more pronounced than what is seen in the pre-vent plots Steve had put up.
In order to try and narrow down whether the problem is with the Satellite box or the LED/PD themselves, I switched the Satellite box at the Y end with the Satellite box for ITMY (at ~930pm tonight). Hopefully over a 12 hour observation period, we see something that will allow us to make some conclusion.
It looks like the problem is indeed in the Satellite box. Attachment #1 shows the second trend for the last 12 hours (~930pm 28 Aug 2016 - 930am 29 Aug 2016) for the ITMY and ETMY sensor signals. The satellite boxes for the two were switched during this time (the switch is seen at the leftmost edge of the plots). After the switch, ETMY UL has been well behaved, though ITMY UL shows evidence of excursions similar to what we have been seeing. All the ITMY coils are pulled out of the suspension cage currently, and are just sitting on the optical table, so they should just be reading out a constant value. I think this is conclusive evidence that the problem is with the Satellite box and not the OSEM itself. I will pull the Satellite box out and have a look at its innards to see if I can find the origin of the problem...
I opened up the ETMY satellite box to investigate the glitches seen in the UL sensor output.
Attachments #1 & 2: The connection to J4 from the satellite amplifier goes through a "satellite amplifier termination board", whose function, according to the schematic, is to prevent oscillations of the output amplifiers for the PD outputs. This seems to have been attached to the inside cover of the Satellite box by means of some sort of sponge/adhesive arrangement. The box itself gets rather hot however, and the sponge/adhesive was a gooey mess. I believe it is possible that some pins on the termination board were getting shorted - so if the 100 ohm resistor for the Ul channel that is meant to prevent the output amplifier oscillating was getting shorted, this could explain the problem.
For now, I cleaned off the old sponge/adhesive as best as I could, and used 4 pads of thick double sided tape (with measured resistance > 60Mohm) to affix the termination board to the inside of the box lid. In the ~3 hours since I have plugged the satellite box back in, there has been no evidence of any glitching.
Of course, it could be that the problem has nothing to do with the termination board, and perhaps an OpAmp in the UL signal chain is damaged, but I stopped short of replacing these for now. I plan to push on with putting the IFO back together, and will keep an eye on this problem to see if more action is needed.
Also, if the inside of the ETMY satellite box had this problem of the sponge/adhesive giving way, it may be that something similar is going on in the other boxes as well. This remains to be investigated.
[gautam, johannes, lydia]
We decided to try some different approaches on minimizing the ETMY bounce coupling today, since the peak height in the previously attched spectrum was higher than the previously recorded levels in 2011 for all but the LR OSEM.
How to minimize particles entering the vacuum envelope.
Just the way it was in August 2011 vent and before.
The portable HEPAs were set up at ETMY and ITMY with CP STAT 100 curtains.
The 40m particles on the floor at ITMY 3000-5000 counts of 0.5 micron cf / min and 0.3 micron size particles are 55,000 - 65,000 counts cf / min
At this condition the MET One Counter #3 on the floor inside the tent goes to zero count of 0.5 micron and 20-40 counts cf / min for 0.3 micron when the tent is slightly overpressured.
100 Sapphire prisms ordered. Delivery date 9-30-2016
~60 deg. prisms,
Size A=B=C=2 mm, length 5 mm
Surface quality: 5 micron RMS
Tolerances: +- 0.1 mm
This prism will be used as a mechanical component
No crystal orientation required
There are some issues with 5 mm sapphire prism Atm5. It will cause interference between one of the prisms and the Side OSEM.
Here are some drawings to see the issues with larger wire standoff.
The 2 mm prism will work.with a 1 mm longer dumbell.
Quotes requested from http://photomachining.com/laser-micromachining-photomachining-contact.html and http://www.optocity.com/
Today we installed ITMY into position in the chamber.
We did some quick checks with the green beam and the IR beam. With the help of the custom Iris for the suspension towers, we gauged that both beams are pretty close to the center of the test mass. So we are in a not unreasonable place to start trying to align the beam. Of course we didn't check if the beam makes it to the ETM today.
The SRM OSEM sensor problem seems to have been resolved by moving the ITM back to its place as we suspected. The values are converging, but not to their pre-vent values (attachment #2). We can adjust these if necessary I guess... Or perhaps this fixes itself once the table returns to its neutral position. This remains to be monitored.
In the never-ending B-R mode reduction saga - we found what we think is an acceptable configuration now. Spectrum attached (Attachment #3). The top two OSEMs are now nearly 90 degrees rotated, while the bottom two are nearly horizontal. Anyways I guess we just have to trust the spectra. I should also point out that the spectra change rather significantly from measurement to measurement. But I think this is good enough to push ahead, unless anyone thinks otherwise?
[Lydia, Gautam, Koji, Johannes]
Summary of things done today:
Unless we get lucky and get the green light to flash in the cavity by playing with the mirror alignment, we will open the ETMY chamber tomorrow. On one hand we can look for the reflected green light in the chamber, or alternatively the IR beam transmitted by ITMY. This way we can obtain estimates for the OSEM biasing and perform the final centering of the OSEMs. We will then also address the bounce mode minimization in ITMY and check if the previous orientations still hold.
Koji tweaked the alignment sliders till we were able to get the Y arm locked to green in a 00 mode, GTRY ~ 0.5 which is the prevent number I have in my head. The green input pointing looks slightly off in yaw, as the spot on the ITM looks a little misaligned - I will fix this tomorrow. But it is encouraging that we can lock to the green, suggests we are not crazily off in alignment.
[Ed by KA: slider values: ETMY (P, Y) = (-3.5459, 0.7050), ITMY (P, Y) = (0.3013, -0.2127)]
While we were locked to the green, ITMY UL coil acted up quite a bit - with a large number of clearly visible excursions. Since the damping was on, this translated to somewhat violent jerking of ITMY (though the green impressively remained locked). We need to fix this. In the interest of diagnosis, I have switched in the SRM satellite box for the ITM one, for overnight observation. It would be good to narrow this down to the electronics. Since SRM is EQ-stopped, I did not plug in any satellite box for SRM. The problem is a difficult one to diagnose, as we can't be sure if the problem is with the LED current driver stage or the PD amplifier stage (or for that matter, the LED/PD themselves), and because the glitches are so intermittent. I will see if any further information can be gleaned in this regard before embarking on some extreme measure like switching out all the 1125 OpAmps or something...
Does anyone know if we have a spare satellite box handy?
Some more numbers we found while working in/around the chamber today:
These numbers were measured using our particle counter, which has a pump rate of 0.1 cfm, so the numbers above are 10x the numbers shown on the instrument after a measurement to account for this.
Essentially, the chamber is pretty dirty. Peeling the F.C with hard to reach optics like the ITM installed in place is not really feasible, and after peeling the F.C, we are looking at a best case of an additional 1-2 weeks in air to align the IFO, during which the optic is apparently exposed to quite a lot of particulates. In fact, with the high intensity flashlight left on, I actually saw some flecks of dust occassionally floating around inside the chamber while I was working on the optic. But this is just something we have to accept I guess.
So if the SRM satellite box is good, than the ITMY sensor UL or vacuum cabeling from sersor to sat amp is bad.
Is the spare sat amp is bad ?
This is an option to isolate the vacuum chamber from the dusty 40m lab: 4x8 HEPA unit or Air Curtain
It does not limit crane operations. Here is some science based approach to air filters
Lets put horizontal and vertical witness plates next to our arm cavity TMs just before pumpdown.
On the bounce roll balancing:
Recall that back in 2006, the main issue was not with the bounce mode coupling into the OSEMs but instead with too much cross-coupling between the damping loops themselves:
Old elogs from Osamu (reader / readonly). Osamu will be here in a couple weeks and can try to explain what he was doing back then.
The problem was that without a good input matrix, the low frequency motion of the suspension point was dominated by the damping noise rather than the seismic noise. The bounce mode is a nice indicator of whether the OSEM is oriented up/down but its not the most important thing. More important is that the magnet is in the actual LED beam, not just the apparent center of the OSEM.
Then we should be able to fix things by running the diagonalization script and correcting the input matrix (which depends somewhat on the DC alignment).
I balanced the ITMX and ETMX tables into level position today, for which I had to move quite a few of the on-table weights. I'm recording their original positions for future use here.
This table was only off in 'pitch', I moved the middle weight to a new location as shown in the pictures. I added secondf disk weight on top of the one I moved, this one has to come out again when we install ETMX.
I moved some weights around as shown in the image, but didn't have to add any. We simply have to move them back to their original location when the time comes.
While in the chambers, I also took some pictures of the ETMX window and PR2, motivated by the dirty state of SR2. We might want to consider cleaning both, specifically PR2 is relatively easily accessible and can be cleaned when we open the ITMX chamber to remove its FC and move it back into position.
[Gautam, Lydia, Johannes]
The next step is the tip tilt fine alignment of the IR into the arm, using TRY, from which we removed the ND filter for the time being.
[johannes, lydia, gautam]
GV EDIT Sep 5: These numbers do make sense if the ND filter that was on the Transmon QPD had ND = 0.6 (there are two at the end, one labelled ND 0.6 and the other labelled ND10 though the latter label looks like some custom label so I don't really trust that value), even though only one was on, unfortunately I don't remember which. So, for 10% of input power with a factor of 8 increase because the ND filter is removed and also that the 50% BS has been replaced with a HR mirror, we expect a transmission level of ~0.6 (compared to the normalized value under normal IFO operation) which is close to what we see...
In any case, I think we can work on putting in the X arm now and work on recovering that.
To do for the Y-arm (now that the F.C. is off, we should try and do this in as few chamber openings as possible):
Then we need to do all of this for the X arm as well. The PRM LR coil is still giving no output - I will try moving the bias sliders around to see if this is a stuck magnet situation, but perhaps it is not. Since Eric's 3-satellite-box-monte did not yield any positive results, we have to consider the possibility that the LED or PD themselves are damaged. If so, I don't see any workaround without opening up the BS-PRM chamber, but if we can avoid this, we should. Perhaps when ITMX is open we can use the camera with the IR filter removed to see if all the OSEM LEDs are functional through the beam tube.
We are also piping POY11 error to the DAFI model and can hear it in the control room.
Rana suggested reviving the MC autolocker - I've made some changes to the low power MC autolocker scripts and they've been working the few times I tried today evening, but let's see how it does over the weekend. I've also changed the Y axis of the StripTool on the wall to better reflect the low-power range..
The ITMX table had relaxed overnight into a slightly misaligned state overnight - since the ITMX table holds PR2 and hence can affect the input pointing, we decided to fix this before commencing alignment work today. The misalignment was not as bad as what Johannes observed prior to his first re-leveling attempt, but was ~1 division on the spirit level. So I decided to move one set of weights to level the table again. It is entirely possible that over the next couple of days, the table will shift slightly again, but the hope is that we are closer to the 'ideal' orientation of the table now... Pictures to follow...
Hi 40m people,
As Rana is saying, the bounce mode does not matter, or we cannot do anything. Generally speaking, the bounce mode cannot be damped by the setting of 40m SUS. Some tweak techniques may damp a bounce mode by res-gain or something, but it is not a proper way, I think.
This is also that Rana is already saying that the important thing is to find a good direction of OSEM to hit the LED beam to the magnet. Even if the magnet is not located at the center of OSEM hole, still you can find the optimal orientation of OSEM to hit the LED beam to the center of magnet by rotating the OSEM.
I know only an old document of T040054 that Shihori summarized how to adjust the matrix at the 40m. Too bad input/output matrix may introduce some troubles, but even roughly adjusted matrix should be still fine.
I will be at Caltech on 12-14 of September. If I can help something, I am willing to work with you!
I looked at the PRM free swing spectra. The modes look like they're at the right frequencies, so pointing more and more towards a LED or satellite box issue.
Some of the frequencies have changed between the 2011 in-vac measurement and our 2016 in-air measurement, but that seems within usual parameters.
In the morning, Steve will start opening the north BS door so that we can enter to inspect the PRM LR OSEM.
For the ITMY, I squished together the cables which are in the 'Cable Interface Board' which lives in the rack. This thing takes the 64 pin IDC from the satellite module and converts it into 2 D-sub connectors to go to the PD whitening board and the coil driver board. Lets see if the ITMY OSEM glitches change character overnight.
[Teng, Johannes, Lydia, gautam]
The modes look like they're at the right frequencies, so pointing more and more towards a LED or satellite box issue.
We peeked into the BS-PRM chamber via the ITMX chamber to see if we could shed any light on this situation. It's hard to get a picture that is in focus, but it looks quite clear that the LR LED (in the lower left when viewed from the HR side) isn't anywhere near as bright as the rest (see Attachment #1). Various hypothesis include failed LED / piece of Al foil blocking the LED / teflon aperture slipped over the LED. But looks like we can't solve this without opening up the BS-PRM chamber. The plan tomorrow is to open up the chamber, pull out the problematic coil. Once we have a better idea of what is going wrong, we can decide what the appropriate course of action is - replace the OSEM or something else.
As part of the diagnosis, I switched the PRM and SRM satellite boxes earlier today evening around 6pm. They remain in this switched state for now.
Steve, we plan to take the BS-PRM heavy door off tomorrow morning.
Edit 7.30pm: I have managed to recover Y-arm in air locking, and the transmission is up at ~0.6 again which is what we were seeing prior to touching anything on the BS-PRM table, so it looks like the tip-tilt has not gone badly astray... I have also restored the Satellite boxes so that both PRM and SRM have their designated boxes
Detailed elog to follow but summary of todays activities:
[steve, teng, johannes, lydia, gautam]
Depending on how the X arm situation is, we will finish putting back all the heavy doors on Monday and start the pumpdown
GV Edit 11.30pm:
Looks like on Monday, we will look to put the heavy doors on ITMY, ITMX and ETMX chambers, and begin the pumpdown
We ran the scripts to diagonalize the damping matrices using the free swinging data from staurday night/sunday morning. The actual entries used for damping have not been changed. However, we did generate updated matrices for all the main optics (not including the mode cleaner optics, which were not free swinging over the weekend).
Last night from 8:30 pm to 8:30 am PDT, ETMY UL signal was glitchy again. As of now it seems to have quieted back down, but we pushed on the cables on the board at the Y end to hopefully prevent it from coming back. After doing so it still seems to be behaving well.
We continued to work on the diagonalization scripts today and devised a way of choosing starting parameters that seems to work much better, and is easier to use, than tuning up to 15 parameters by hand per optic.
We still noticed phase problems with ITMY, which appear to be preventing good diagonalization (See Attachment 1). Almost every degree of freedom has a significant imaginary part in the sensing matrix. We looked at the phases of the cross spectra in DDT and saw that indeed, the OSEM signals do not have the appropriate relative phases at the peak frequencies, especially in PIT and YAW (see Attachment 2: the phase at the peak is about 30 degrees when it should be 180). These phases are different for data takes ~24 hours apart, but are still wrong. We also looked at this information for ETMY and saw the correct behavior. We temporarily moved the pitch and yaw sliders for ITMY and looked at the OSEM response on a striptool, and the signals moved in the expected way. Can anyone suggest a reason why this would be happening? Is there another stretch of data (besides this past weekend) which would be good to compare to?
Today the main optics were free swinging for several hours, so I attempted diagonalization in vacuum.
We built matrices for ITMY and ETMY by driving one degree of freedom at a time with awggui, while the damping was on. These have been applied to the damping loops.
Pitch:1158085097 Yaw: 1158086537 Pos: 1158089237 Side: 1158087977
Pitch: 1158095897 Yaw: 1158097577 Pos: 1158099377 Side: 1158100817
All is not lost. I've stuck and unstuck optics around a half dozen times. Can you please post the zoomed in time series (not trend) from around the time it got stuck? Sometimes the bias sliders have to be toggles to make the bias correct. From the OSEM trend it seems like it got a large Yaw bias. May also try to reseat the satellite box cables and the cable from the coil driver to the cable breakout board in the back of the rack.
Here's the timeseries plots. I've zoomed in to right after the problem- did you want before? We pretty much know what happened: c1susaux was restarted from the crate but the damping was on, so as soon as the machine came back online the damping loops sent a huge signal to the coils. (Also, it seems to be down again. Now we know what to do first before keying the crate.) It seems like both right side magnets are stuck, and this could probably be fixed by moving the yaw slider. Steve advised that we wait for an experienced hand to do so.
susaux is responsible for turning on/off the inputs to the coil driver, but not the actual damping loops. So rebooting susaux only does the same as turning the watchdogs on/off so it shouldn't be a big issue.
Both before and after would be good. We want to see how much bias and how much voltage from the front ends were applied. l1susaux could have put in a huge bias, but NOT a huge force from the damping loops. But I've never seen it put in a huge bias and there's no way to prevent this anyway without disconnecting cables.
I think its much more likely that its a little stuck due to static charge on the rubber EQ stop tips and that we can shake it lose with the damping loops.
ITMX is free, OSEM signals all rougly centered.
This was accomplished by rocking the static alignment (i.e. slow controls) pitch and yaw offsets until the optic broke free. This took a few volts back and forth. At this point, I tried to find a point where the optic seemed to freely swing, and hopefully have signals in all 5 OSEMS. It seemed to be free sometimes but mostly settling into two different stationary states. I realized that it was becoming torqued enough in pitch to be leaning on the top-front or top-back EQ stops. So, I slowly adjusted the pitch from one of these states until it seemed to be swinging a bit on the camera, and three OSEM signals were showing real motion. Then, I slowly adjusted the pitch and yaw alignments to get all OSEMS signals roughly centered at half of their max voltage.
I had hoped to do some ALS work, but I realized too late that we loaned our HP analyzer to Andrew. I decided instead to do some ETMX testing.
I have a script running that'll misalign both ETMs and back by about 0.5mrad with half hour rests in between. It'll be done around 6AM.
Seems like the angular position was fairly stable, though there is some change in the ETMX pitch that could be hysterisis or normal drift. I didn't mention it explicity in the previous log, but the misalignment was purely in pitch. I'll give it another shot with a bigger misalginment, and maybe a mix of pitch and yaw.
This afternoon around 2:45, ITMX started ringing up at ~.9Hz for about a minute and then got stuck again. When I noticed this evening, I tried to free it with the alignment sliders but was unable to see any signal on UL or UR. It also looks like the damping for ITMY was turned off at the same time ITMX got stuck (not at the start of its ring up). SRM also has a spike in its motion at this time, and another one minute later that ended up with the LR OSEM at a much higher level, though the mirror does not appear to be stuck. We didn't see any strange behavior from any of the other optics.
Teng and I were working on diagnosing a problem with the ITMY UL whitening, but by the time we disconnected any applicable cables, the damping for ITMY was already off. Later we unplugged the ITMX PD whitening cables after verifying that the ITMX damping was also already off. This problem may have occured earlier, while Teng, Eric, and I were examining and pushing in the cables at 1X5 without unplugging anything.
We found that the reason for the bad phase on the ITMY free swing data is because the whitening filter for UL is not being properly turned on. We are in the process of investigating the source of this problem. Right now all the cables to the PD whitening boxes in 1X5 are switched between ITMY and ITMX.
The earth quake shook ITMX free for a short while.
When we plugged in the back cables yesterday on the whitening boxes after switching them, two of the ITMX PDMon channels (UR and LR) got stuck at 0. This caused me to believe ITMX was still stuck even when it was freed. However, it was left in a stuck state overnight and freed again today after this issue was discovered. The alignment sliders have been set to 0 as a safety net to keep ITMX from getting stuck again if c1susaux is restarted again. We switched the cables back and the problem was still there.
The ITMY UL whitening filter problem, which the cables were originally switched to diagnose, was also still there. Ericq suggested we turn off all the whitening filters in order to get diagonalization data that would not show a phase difference between coils. We ran the diagonalization again with all the dewhitening filters off and got much cleaner results, with no visible cross-coupling peaks remaining between the degrees of freedom (see attachemnt 1). We did not apply this matrix to the damping, however, because there are elements which have the wrong sign compared to the ideal matrix. Significant adjustments to the output matrix will probably need to be made if this result is to be used. We also verified that the phase problem had been solved in DTT, where we saw the same sign discrepancies as in the matrix below.
Damping can be turned back on, using the old, non-diagonalized matrix currently in effect. There is enough free swing data to diagonalize ITMY now, so feel free to mess with it.
Matrix (wrong signs red, suspiciously small elements orange):
pit yaw pos side butt
UL 1.633 0.138 1.224 0.136 0.984
UR -0.202 -1.768 1.179 0.132 -1.028
LR -2.000 0.094 0.776 0.107 1.001
LL -0.165 2.000 0.821 0.111 -0.987
SD 0.900 1.131 -1.708 1.000 -0.107