We had a redo of elog entry 975 tonight. The noisy OSEM was fixed by jiggling the rack end of the long cable. Don't know exactly where--I also poked around the OSEM PD interface board.
In the attached PDF the reference trace is the noisy one.
I've noticed that the glitchy behaviour in ITMY UL shadow sensor readback is back - as mentioned above, I looked at the Sat. Box and could not find anything wrong with it, perhaps I'll plug the tester box in over the Thanksgiving weekend and see if the glitches persist...
I left the tester box plugged in from Thursday night to Sunday afternoon, and in this period, the glitches still appeared in (and only in) the UL channel.
So yesterday evening, I pulled the Sat. Box. out and checked the DC voltages at various points in the circuit using a DMM, including the output of the high current buffer that supplies the drive current to the shadow sensor LEDs. When we had similar behaviour in the PRM box, this kind of analysis immediately identified the faulty component as the high current buffer IC (LM6321M) in the bad channel, but everything seems in order for the ITMY box.
I then checked the Satellite Amplifier Termination Board, which basically just adds 100ohm series resistors to the output of the PD readout, and all the resistors seem fine, the piece of insulating material affixed to the bottom of this board is also intact. I then used the SR785 in AC coupled mode to look at the high frequency spectrum at the same points I checked the DC voltages with the DMM (namely the drive voltage to the LEDs, and the PD readout voltages on the PCB as well as on the pins of the connector on the outside of the box after the termination board (leading to the DAQ), and nothing sticks out here in the UL channel either. Of course it could be that the glitches are intermittent, and during my tests they just weren't there...
I am hesitant to start pulling out ICs and replacing them without any obvious signs of failure from them, but I am out of debugging ideas...
One possibility is that the problem lies upstream of the Sat. Box - perhaps the UL channel in the Suspension PD Whitening and Interface Board is faulty. To test, I have now hooked up ITMY Sat. Box. + tester box to the signal chain of ETMY. If I can get the other tester box back from Ben, I will plug in the ETMY sat. box. + tester to the ITMY signal chain. This should tell us something...
400 days plot. Satelite amp ITMY has been swapped with ETMY
Unlabeled sat.amps are labeled. This plot only makes sense if you know the Cuh-Razy sat amp locations.
We fixed the issue of ITMY ULCOIL not driving ITMY by replacing one of the 64pin ribbon cable in the satellite amplifier box.
We thought the coil driver and the sat amp box are OK by checking the voltage change at the output of the sat amp box by giving an offset to UL coil driver, but it was not giving a current change, probably due to too much contact resistance in the cables.
It was sneaky because it was not completely disconnected.
All the coils for our suspensions are now working!
What we did:
- Using breakout boards, the output current of sat amp box was measured using FLUKE multimeter. It turned out that UL is not giving measurable current. We also confirmed that UR coil driver can drive UL by re-directing the current from UR coil driver to UL. This means that the UL magnet was not de-magnetized!
- Measured the coil resistance from at the coil driver output and found that UL coil seen from there has too high resistance which cannot be measured with the multimeter, whereas UR coil was measured to be ~30 Ohms.
- Went back to the feedthru and measured the resistance of UL coil. Upto the output of the Satellite Amp Terimator, the resistance was measured to be ~16 Ohms, but not at the input of the Satellite Amp Terimator (Attachment #1,2).
- It turned out that #16 pin of 64pin ribbon cable in between the Satellite Amp Terimator (LIGO-D990021) and the Satellite Amp board (LIGO-D961289) at the Satellite Amp Terimator side was not good (Attachment #3).
- Replaced the cable and confirmed that ULCOIL can kick ITMY (Attachment #4).
- C1:SUS-ITMY_TO_COIL matrix was reverted to default values.
- We might have to re-commission Yarm ASS again since pitch-yaw coupling have changed. -> EDIT: Checked that it works (except for ITM PIT L), including offloading offsets (writeASS_offsets.py), 18:30 local.
- Now that LO1 LLCOIL issue is solved and LO2 stuck is solved, we should do the free swing test again to identify the resonant frequencies.
- OSEM sensor diagonalization (input matrix), coil balancing (and F2A)
We investigated the ITMY ULCOIL issue (40m/16873).
ULSEN is sensing the optic motion but ULCOIL cannot move the optic.
We confirmed that the coil input is there upto satellite amplifier output.
We also checked that ULCOIL have 3.3 mH and 16 Ohms, which are consistent with other coils.
We need to investigate ITMY ULCOIL in the next vent.
What we did:
- Checked again that C1:SUS-ITMY_ULCOIL_OFFSET does not kick ITMY using OSEM sensor signals and oplev signals. ULSEN moves when ITMY is kicked by other coils.
- Checked that kick gives voltage changes at coil driver and satellite amplifier output. We unplugged J1 DB25 cable from the feedthru flange and checked the signals sent to coil with oscilloscope.
- Measured inductance (using BK PRECISION LCR meter) and resistance (using Fluke) of coils for ITMY. Below is the result. UL coil seems to be consistent with other coils. (It seems like BK PRECISION one wil give wrong resistance if the dial is set to the resistance value which is too low compared with the one you want to measure. If you want to measure 16Ω, set the dial to larger than 20Ω, not 2Ω)
Feedthru connector: ITMY1
Pin 3-15 / R = 16.3Ω / L = 3.32 mH (UL)
Pin 7-19 / R = 16.4Ω / L = 3.30 mH (UR)
Pin11-23 / R = 16.2Ω / L = 3.31 mH (LL)
Feedthru connector: ITMY2
Pin 3-15 / N/A
Pin 7-19 / R = 16.3Ω / L = 3.30 mH (SD)
Pin11-23 / R = 16.4Ω / L = 3.33 mH (LR)
- UL is the only short OSEM in ITMY OSEMs.
- ITMY have dumbells for magnets.
- If UL magnet is off, ULSEN would not work. Something not magnetic is working for shadow sensing for UL? Dumbells?
- ULSEN just sensing some coupling from other OSEMs?
To see if the ULCOIL channel of the ITMY coil driver is working or not, I swapped ITMY coil driver and ITMX coil driver by swapping DB15 cable (see Attachment #2).
With this swap, I confirmed that ITMX can be kicked with C1:SUS-ITMY_ULCOIL_OFFSET, but ITMY cannot be kicked with C1:SUS-ITMX_ULCOIL_OFFSET (see Attachment #1).
This means that the issue is not the in-air electronics.
Mystery remains again...
We need to investigate ITMY ULCOIL in the next vent.
I revereted the swap and confirmed that damping loops work fine again.
what was the result of the inductance measurement? should be ~3.3 mH as measured from the flannge or cable that goes to the flange from sat amp.
ITMY ULCOIL was measured to have ~3.3 mH as measured from the flange. RTFE 40m/16896 .
Its good that the inductance test passed. This means that the coil is OK. How does the inspection photo look? This is the one you guys took of the ITM OSEM that shows the position of the magnet w.r.t. the coil. Also, how does the free swinging spectra look? Either one of these might indicate a broken magnet, or a sticky EQ stop.
We checked the photos we have, but we didn't have the photos which show ULCOIL situation clearly.
Free swing of ITMY (and others) will be done this weekend to see the OSEM spectra and resonant frequencies.
For good measure:
While doing initial measurements for the new global damping infrastructure I discovered that the ETMY loop between the OSEM actuation and the OSEM sensors has a gain that is 2.5 times greater than the ITMY. The result is that to get the same damping on both, the damping gain on the ETMY must be 2.5 times less than the ITMY. I do not know where this is coming from, but I could not find any obvious differences between the MEDM matrices and gains.
I uploaded a screenshot of measured transfer functions of the damped ITMY and ETMY sus's. Notice that the ETMY measurement is 2.5 times higher than the ITMY. The peak also has a lower Q, despite having the same damping filters running because of this mysterious gain difference. Lowering the damping gain of the ETMY loop by this 2.5 factor results in similar Q's.
Just to find out where we are currently, I plotted the ITMY and SRM oplev spectra along with the ETMY oplev spectra. ETMY seems to be very good, so comparing with this seemed useful, so we know how much we have to improve by. The SRM power spectrum appears to be around 2 orders of magnitude higher than ETMY over pretty much the whole measurement band. The ITMY power spectrum is not so bad as the SRM above about 60Hz. Next thing to do is to check the dark noise level for the ITMY and SRM QPDs.
The title of this post should of course have been " ... - comparison with ETMY" not " ... - comparison with ITMY"
Today I worked on getting the ITMY and SRM oplevs back in working order. I aligned the SRM path back onto the QPD. I put excitations on the ITMY and SRM in pitch and yaw and observed the beam at the QPDs to check for clipping. They looked clean from clipping.
I divided the open loop transfer functions by the filter response and the sensor responses (previously measured calibration factors) to leave just the actuator responses. I've attached the actuator responses plotted in radians/count and phase over frequency.
Next step: fit the actuator response with poles and zeros.
EDIT: I divided by the wrong filter function earlier - the plots there now are divided by the correct filter function
Here are the results of the complex fitting. The residuals are bigger this time, but still probably small enough to be ok(?), with the possible exception of ITMY PITCH (due again I think to the data points straddling the resonance).
ITMY YAW actuator response complex fit
-- Fit completed after 282 iterations--
I used an fminsearch function to fit the SRM and ITMY actuator response magnitudes. The testfunction was just that for a single second order pole, but it gave what I consider to be good fits for the following reasons:
*for 3 of the 4 fits the residuals were less than 0.5% of the summed input data points. The worst one (ITMY pitch) was about 2.7%, which I think is due to the resonance happening to be right in the middle of two data points.
*the tolerance of 1 part in 10^9 was reached quickly from not very finely tuned starting points.
The test function was: G=abs(Gp./(1+1i.*f./fp./Qp-(f./fp).^2)), where G(f) is the actuator response magnitude, Gp is the pole gain, fp is the pole frequency, and Qp is the pole Q factor.
In the end I just fitted the response magnitude. I was initially fitting the complex response function, but ran into problems which I think were cased by overall phase offsets between the data and test function. Can I canvass for opinion if fitting the magnitude is OK, or should I try again fitting the phase too?
Anyway, here are the results of the fits, and I've attached plots of each too (each one in linear and log y axis because each on its own might be misleading for fits):
EDIT - I added more points to the otherwise sparse looking fitted curves
ITMY PITCH actuator response fit
-- Fit completed after 190 iterations--
Started with: Gain = 3e-06,
Q factor = 5,
Pole frequency = 1,
Fit results: Gain = 1.32047e-06,
Q factor = 4.34542,
Pole frequency = 0.676676
Residual (normalised against the sum of input datapoints) = 0.0268321
ITMY YAW actuator response fit
-- Fit completed after 156 iterations--
Fit results: Gain = 1.14456e-06,
Q factor = 8.49875,
Pole frequency = 0.730028
Residual (normalised against the sum of input datapoints) = 0.00468077
SRM PITCH actuator response fit
-- Fit completed after 192 iterations--
Fit results: Gain = 7.94675e-06,
Q factor = 7.16458,
Pole frequency = 0.57313
Residual (normalised against the sum of input datapoints) = 0.00301265
SRM YAW actuator response fit
-- Fit completed after 156 iterations--
Fit results: Gain = 3.34179e-06,
Q factor = 9.57601,
Pole frequency = 0.855322
Residual (normalised against the sum of input datapoints) = 0.000840468
Here are the open loop transfer functions for ITMY and SRM. The various settings for the OLTFs were as follows:
Oplev filter used for all OLTFs: 300^2:0
Gains for oplev servos (for each OLTF only the 1 servo for the measured TF was on. They are all set back to 0 now):
SRM yaw gain = 1
SRM pitch gain = -1
ITMY yaw gain = -1
ITMY pitch gain = 1
measurement band = 0.2Hz to 200Hz
points = 33
swept sine magnitude envelope: amp = 2 for f > 60Hz, amp = 0.1 for f < 60Hz
Measurement points were from e.g. C1-SUS-ITMY-OLPIT-IN2 to C1-SUS-ITMY-OLPIT-IN1 to give a TF of -(loop gain).
Next step is to divide this through by the sensor reponse (i.e. the calibration factor measured earlier) and the filter response to get just the actuator response.
I replaced the lenses that were there with a -150mm lens followed by a +250mm lens. This gave a significantly reduced beam size at the QPDs. With the beam analyzer up and running it should be possible to optimize this later this afternoon. Next I will remove the SRM QPD from the path and make measurements of the beam spot position movement and corresponding OSEM values for different DC mirror offsets. I will then repeat the process for ITMY.
The measured calibration factors for the oplevs are as follows:
Kiwamu noticed that the 1/L in the counts per radian should have just been L, which accounts for most of the discrepancy. We checked the input filters on the OSEMs, and they have 10dB of gain at DC. Accounting for this, estimates on the order of 20urad/count, which is much more reasonable!
I found that some of the Optical Lever Servos were ON today and injecting nonsense into the interferometer optics. I have set all of the gains = 0 to save us more headaches.
I had previously set the gains to zero, see the first line of my entry on Monday 5468. I should have the servo and noise characterisation done today for these oplevs today, so we can review it soon.
ITMY gets new Tamron M118FM50 that has improved close focusing. It is a small fixed focal length camera so the video tube cover can be put on.
The Watec LCL-902K 1/2" ccd camera was losing it power supply voltage because of bad connection. It was replaced.
We did the following things in the ITMY chamber today:
1) We tried to get the ITMY stuck again by adjusting the coil gains so that it goes into the orientation where it used to get stuck. We (reassuringly) failed to get it stuck again. This, as we came to know later, is because kiwamu had rotated the side OSEM such that the optic does not get stuck . However the OSEM beam is at about 30 deg to the vertical and the SD is sensitive to POS motion now resulting in the poorer separation of modes as noted by Jenne earlier (5439)
2) We checked the earthquake stops and repositioned two at the bottom (towards the AR side of the optic) which we had backed out earlier.
3) We took pics of all the OSEMS.
4) Checked to see if there are any stray beams with an IR card. There were none.
5) I obtained the max values of the OSEMS by misaligning the optic with the coil offsets. These values are in good agreement with those on the wiki
OSEM UL UR LR LL SD
Max 1.80 1.53 1.68 1.96 2.10
Current 0.97 0.79 0.83 0.97 1.02
We can close the heavy doors tomorrow morning.
[Paco, Anchal, Yuta]
Today, in short we:
C1:SUS-BS_ULSEN_OUT16 = 599
C1:SUS-BS_LLSEN_OUT16 = 575
C1:SUS-BS_URSEN_OUT16 = 602
C1:SUS-BS_LRSEN_OUT16 = 636
C1:SUS-BS_SDSEN_OUT16 = 669
C1:SUS-ITMX_ULSEN_OUT16 = 403
C1:SUS-ITMX_LLSEN_OUT16 = 609
Since UL coil actuation is lost, we modified the output matrix of ITMY to use only UR, LR and LL face coils for POS, PIT and YAW actuation. The output matrix was changed to following:
After this change, the damping was still working as good as before. I took PIT to POS/PIT/YAW and YAW to POS/PIT/YAW coupling measurements by exciting C1:SUS-ITMY_ASCPIT[YAW]_EXC and seeing effect at C1:SUS-ITMY_SUS[POS/PIT/YAW]_IN1 when the damping loops were off. Attached are the results. We were able to reduce PIT to YAW and YAW to PIT coupling by 10 dB by this simple change in output matrix. More coil balancing or off-diagonal termsmight help more and should be attempted if required. The coupling to POS did not change much.
Note that attachment 1 shows transfer functions from excitation point to the DOF sensing inputs while attachment two looks at ratio of C1:SUS-ITMY_SUS[POS/PIT]_IN1 to C1:SUS-ITMY_SUSYAW_IN1 which is the actual quantity of interest. I didn't repeat the PIT measurement due to lack of time.
Also note that all such measurements are being recorded in our new measurements git repo. We'll populate this repo with diaggui template+data files as we do measurements.
ITMY suspention damping restored
The ITMY 10" flange with 10 DSUB-25 feedthroughs has been installed with the cables connected at the in-vac side. This is the first of two flanges, and includes 5 cables ordered vertically in stacks of 3 & 2 for [[OMC-DCPDs, OMC-QPDs, OMC-PZTs/Pico]] and [[SRM1, SRM2]] respectively from right to left. During installation, two 12-point silver plated bolts were stripped, so Chub had to replace them.
The ITMY 10" flange with 4 DSUB-25 feedthroughs has been installed with the cables connected at the in-vac side. This is the second of two flanges, and includes 4 cables ordered vertically in stacks of 2 & 2 for [[AS1-1, AS1-2, AS4-1, AS4-2]] respectively. No major incidents during this one, except maybe a note that all the bolts were extremely dirty and covered with gunk, so we gave a quick swipe with wet cloths before reinstalling them.
[Lydia, Gautam, Koji, Johannes]
Summary of things done today:
Unless we get lucky and get the green light to flash in the cavity by playing with the mirror alignment, we will open the ETMY chamber tomorrow. On one hand we can look for the reflected green light in the chamber, or alternatively the IR beam transmitted by ITMY. This way we can obtain estimates for the OSEM biasing and perform the final centering of the OSEMs. We will then also address the bounce mode minimization in ITMY and check if the previous orientations still hold.
While Anjali is working on the 1um MZ setup, the pesky ITMY was liberated from the OSEMs. The "algorithm" :
While doing this work, I noticed several errors corresponding to EPICS channel conflicts. Turns out the c1susaux2 EPICS server was left running, and the MEDM screens (and possibly several scripts) were confused. There has to be some other way of testing the new crate, on an isolated network or something - please do not leave the modbus service running as it potentially interferes with normal IFO operation. For good measure, I stopped the process and shut down the machine since I saw nothing in the elog about any running tests.
ITMY became stuck during this process
I renamed all channels on c1susaux2 from "C1:SUS-..." to "C1:SUS2-..." to avoid contention. When the new system is ready to install, those channel names can be reverted with a quick search-and-replace edit.
As it turns out, now ITMY has a tendency to get stuck. I found it MUCH more difficult to release the optic using the bias jiggling technique, it took me ~ 2 hours. Best to avoid c1susaux reboots, and if it has to be done, take precautions that were listed for ITMX - better yet, let's swap out the new Acromag chassis ASAP. I will do the arm locking tests tomorrow.
With Chub providing illumination via the camera viewport, I was able to take photos of ITMY this morning. All the magnets look well clear of the OSEMs, with the possible exception of UR. I will adjust the position of this OSEM slightly. To test if this fix is effective, I will then cycle the bias voltage to the ITM between 0 and the maximum allowed, and check if the optic gets stuck.
[gautam, johannes, lydia]
Today we installed ITMY into position in the chamber.
We did some quick checks with the green beam and the IR beam. With the help of the custom Iris for the suspension towers, we gauged that both beams are pretty close to the center of the test mass. So we are in a not unreasonable place to start trying to align the beam. Of course we didn't check if the beam makes it to the ETM today.
The SRM OSEM sensor problem seems to have been resolved by moving the ITM back to its place as we suspected. The values are converging, but not to their pre-vent values (attachment #2). We can adjust these if necessary I guess... Or perhaps this fixes itself once the table returns to its neutral position. This remains to be monitored.
In the never-ending B-R mode reduction saga - we found what we think is an acceptable configuration now. Spectrum attached (Attachment #3). The top two OSEMs are now nearly 90 degrees rotated, while the bottom two are nearly horizontal. Anyways I guess we just have to trust the spectra. I should also point out that the spectra change rather significantly from measurement to measurement. But I think this is good enough to push ahead, unless anyone thinks otherwise?
ITMY is not like the others. Real or just OSEM madness?
I've got the bench set up for the measurement of the beam spot change with DC SRM alignment offsets. The ITMY oplev is aligned and fine to use, but the SRM one isn't until further notice (probably a couple of hours).
The ITMY mirror was released. The OSEM readouts became healthy.
To see what is going on, I changed the PIT DC bias slider on ITMY from 0.8 to -1 or so, and then the optic started showing a free swinging behavior.
If there were no responses to the DC bias, I was going to let people to open the chamber to look at it closer, but fortunately it released the optic.
Then I brought the slider back to 0.8, and it looked still free swinging. Possibly the optic had been stacked on some of the OSEMS as Jamie expected.
ITMY, which is supposed to be fully free-swinging at the moment, is displaying the tell-tale signs of being stuck to one of it's OSEMs.
Do we have a procedure for remotely getting it unstuck? If not, we need to open up ITMYC and unstick it before we pump.
This work happened on Friday, after Nodus and the elog went down....
The guiderod and standoff for ITMY were epoxied, and left drying over the weekend on the flow bench under a foil tent. The flow bench was off for the weekend, so we made tents which hopefully didn't have any place for dust to get in and settle on the mirrors.
There is a small chance that there will be a problem with glue on the arm of the fixture holding the guiderod to the optic. Kiwamu and I examined it, and hopefully it won't stick. We'll check it out this afternoon when we start getting ready for gluing magnets onto optics this afternoon.
ITMY, which is supposed to be fully free-swinging at the moment, is displaying the tell-tale signs of being stuck to one of it's OSEMs. This is indicated by the PDMon values, one of which is zero while the others are max:
ITMY sus damping restored.
First of all I moved the lenses on the ITMY/SRM oplev path to get a smaller spot size on the QPDs. I couldn't get the beam analyzer to work though, so I don't know quite how successful this was. The software brought up the error "unable to connect to framegrabber" or something similar. I don't think the signal from the head was being read by the software. I will try to get the beam analyzer working soon so that we can characterize the other oplev lasers and get decent spot sizes on the QPDs. I searched the elog for posts about the analyzer, and found that it has been used recently, so maybe I'm just doing something wrong in using it.
After this I measured the transfer function for the ITMY oplev yaw. I did a swept sine excitation of the ITMY in yaw with an amplitude of 500, and recorded the OSEM yaw values and the oplev yaw values. This should show a flat response, as both the QPD and the OSEMS should have flat frequency response in the measurement band. This measurement should therefore just yield a calibration from OSEM yaw to oplev yaw. If the OSEM yaw values were already calibrated for radians, we would then immediately have a calibration from oplev yaw values to radians. However, as far as I'm aware, there is not a calibration factor available from OSEM yaw values to radians. Anyway, the TF I measured did not appear to be very flat (see attached plot). Kiwamu suggested I should check the correlation between the OSEM measurements and the oplev QPD measurements - if the correlation is less than 1 the TF is not reliable. Indeed the coherence was poor for this measurement. This was probably because at frequencies above the pendulum frequency, the excitation amplitude of 500 was not enough to cause a measurable change in the optic angle. So, the plot attached is not very useful yet, but I learned something while making it.
Why ITMY UL can not see this earth quake? SRM and PRM are misaligned. ETMX is still not well.
We have to remember to check OSEM - magnet alignment when vented.
Everybody is happy, except ITMY_UL or satalite box.
Gautam shows perfect form in the OMC chamber.
So if the SRM satellite box is good, than the ITMY sensor UL or vacuum cabeling from sersor to sat amp is bad.
Koji tweaked the alignment sliders till we were able to get the Y arm locked to green in a 00 mode, GTRY ~ 0.5 which is the prevent number I have in my head. The green input pointing looks slightly off in yaw, as the spot on the ITM looks a little misaligned - I will fix this tomorrow. But it is encouraging that we can lock to the green, suggests we are not crazily off in alignment.
[Ed by KA: slider values: ETMY (P, Y) = (-3.5459, 0.7050), ITMY (P, Y) = (0.3013, -0.2127)]
While we were locked to the green, ITMY UL coil acted up quite a bit - with a large number of clearly visible excursions. Since the damping was on, this translated to somewhat violent jerking of ITMY (though the green impressively remained locked). We need to fix this. In the interest of diagnosis, I have switched in the SRM satellite box for the ITM one, for overnight observation. It would be good to narrow this down to the electronics. Since SRM is EQ-stopped, I did not plug in any satellite box for SRM. The problem is a difficult one to diagnose, as we can't be sure if the problem is with the LED current driver stage or the PD amplifier stage (or for that matter, the LED/PD themselves), and because the glitches are so intermittent. I will see if any further information can be gleaned in this regard before embarking on some extreme measure like switching out all the 1125 OpAmps or something...
Does anyone know if we have a spare satellite box handy?
Is the spare sat amp is bad ?