Kiwamu mentioned that REFL165 is not responding and its DC out seems saturated at 9V. Koji and I checked to see if changing the power supply to the PD changed its behaviour. It did not.
I then look a close look at the PD and found that the front window of the PD was not clear and transparent. There was a liquid condensation inside the window, indicating an over heating of the PD at some point. It could have arisen due to excessive incident power. The pic below shows this condensation:
I also checked the current flowing through the reverse bias voltage line. There was a voltage drop of 3V across R22 (DCC D980454-01-C) indicating a 150mA of current through the PD. This is way too much above the operating current of about 20mA. The diode must have over heated.
I pulled out the old PD out and installed a new one from stock. The pic below shows the clear window of a new PD.
After changing the PD I checked the DC output voltage while shining a torch light on to the PD. It showed an output of about 30 to 40 mV. This seemed okay because the larger 2mm photodiodes showed ~100mA DC output with the same torch.Below is the current state of the ckt board.
I will tune the PD to 165 MHz tomorrow and measure its transimpedance.
I think I have told a lie in the last meeting -- the measured sensing matrix doesn't look similar to what Optickle predicts.
Smells like something is very wrong.
Those Radar plots are awesome. Even more awesome would be if they were in units of W/m (so that it can be directly compared with Optickle) and so that the numbers are useful even 1 year from now. Otherwise, we will lose the RF transimpedance information and thereby lose everything.
Also, please post the provenance of the counts->V calibration.
Initial pointing or IP-ANG is a pointing monitor of the MC. This beam is launched after the second pzt steering mirror.
IP-ANG is missing the pick up mirror by a few inches at ETMYchamber
1000 days plot show last appearance in Feb 2010
So why don't you use AS55I and Q for the control of PRMI???
Sometimes ago I reported that there have been a kind of upconversion noise when PRM was excited (#6211).
This time I found another one, which showed up when BS was excited.
Assuming this is related to some kind of scattering process and also assuming this is from the same scattering body as that for the PRM driven case,
we may be able to localize and perhaps identify the scattering body.
I found that the DC monitor of the REFL165 was showing 9 V regardless of how much laser power goes to the diode.
I am worried about whether the RF output is also broken.
It needs to be checked and I will leave this to Suresh as one of his morning tasks.
I updated the table which I posted some time ago (#6231). The latest table is shown below.
It seems that the glitches show up only when multiple DOFs are locked.
Interesting thing is that when the low finesse PRMI is locked with a big MICH offset (corresponding to a very low finesse) it doesn't show the glitches.
Qualitatively speaking, the glitch rate becomes higher as the finesse increases.
I will try SRMI tomorrow as this is the last one which I haven't checked the presence of the glitches.
(POX11 --> ETMX)
(AS55 --> ITMs)
(REFL11 --> PRM)
(REFL33 --> PRM)
low finesse PRMI
(ASDC --> ITMs)
(AS55 --> ITMs)
I think I have told a lie in the last meeting -- the measured sensing matrix doesn't look similar to what Optickle predicts.
Smells like something is very wrong.
Measured sensing matrix
Some obvious things:
Expected sensing matrix from a simulation
I added an U channel based bottom shelf at the south end today.
I have measured the sensing matrix of PRMI.
It seems that the MICH signal in the 3f ports (REFL33 and REFL165) were quite tiny, and because of that it is very tough to use them for the actual MICH control.
The data is coming soon.
Schott, Athermal green welding glass, shade #14 reflectivity was measured in 1.2W, ~1 mm diameter beam of MC reflected.
The P polarization measurement was done with the help of half wave plate and PBC
We extracted the fiber that Suresh and Sonali laid over the summer, for the IR beat for the Ygreen laser, and Frank took it back to Bridge to be used in the new fiber distributed reference laser setup.
ITMX, PRM and BS watchdogs are tripped. They were restored.
Stable MC was disabled so I can use MC_REFL 1 W beam to measure green glass .
I reported the procedure to add slow channels to the FB. I guess you already have done Step.1
Tragically, this is more tricksy than I would have thought. The channels we need are "cdsEpicsOutput"s in the model. They don't show up in Dataviewer (fast or slow channels) or the regular fast channel .ini file. Jamie and I don't remember where these channels live and how to get them saved to frames. I'm on top of it though.
So, none of our PEM BLRMS channels are recorded as of right now. All we have for long-term record is the StripTool on the wall. The 0.1-0.3Hz and 0.3-1 Hz traces both show these weirdo things, but the 1Hz and up BLRMS don't have any unusual noise.
Seems like a problem to solve on Monday so that we don't end up without trends like this again.
I did notice however, that the striptool for seismic trends is showing the wrong channels for 3-10 and 10-30 Hz. The other 3 channels are correctly the output after the sqrt is taken, but those two (orange and red on striptool) are before the sqrt, but after the bandpass and low pass. I'll fix that now...
[ Koji / Kiwamu ]
The frequent unlock of the MC are most likely unrelated to ground motion.
Although the reason why MC became unstable is still unclear.
There are two facts which suggest that the ground motion and the MC unlock are unrelated :
(1) It turned out that the seismometer signals (C1:PEM-SEIS-STS_AAA ) have a big cross talk with the MC locking signals.
For example, when we intentionally unlocked the MC, the seismometer simultaneously showed a step-shaped signals, which looked quite similar to what we have observed.
I guess there could be some kind of electrical cross talk happening between some MC locking signals and the seismometer channels.
So we should not trust the signals from the STS seismometers. This needs a further investigation.
(2) We looked at the OSEM and oplev signals of some other suspended optics, and didn't find any corresponding fluctuations.
The suspensions we checked are ETMX, ETMY, ITMX and MC1.
None of them showed an obvious sign of the active ground motions in the past 24 hours or so.
It seems that somehow the seismic noise became louder from about 1:00 AM.
The air cond was off for 2 hrs. I just switched it back on at 15:51
Kiwamu and Steve maybe don't know about how to trend seismic noise. If you just take the mean of the time series, you don't prove that the seismic noise got any higher. The STS has a nominally zero DC output, so the long period level shifts that you see tell you just that there was a DC offset.
This is NOT an increase in seismic noise. To see a seismic trend you should plot the trend of the BLRMS channels that we made especially for this purpose.
Finally I found a company who can do Koji's improved -hard to make- specification on ruby or sapphire wire standoff.
NOT POLISHED excimer laser cut, wire groove radius R 0.0005" + - 0.0002"
$250 ea at 50 pieces of order
The shaking has stopped at 9:32am The AC was turned back on at 11:30am We still do not have any explanation
I gave up tonight's locking activity because the MC can't stay locked.
I walked around the outside of the 40-m building to see what's going on, but no one was jumping or partying.
I am leaving the MC autolocker disabled so that the laser won't be driven crazy and the WFS won't kick the MC suspensions.
The attachment is a 3-hour trend of the seismometer outputs and the MC trans.
Something has started shaking last night. Everybody is claiming to be innocent next door.
I turned off the 40m AC at 11:06
I worked on the OSEM box a little more today, with the hopes of reducing the measured output current noise. I succeeded, at least modestly. It turns out that most of the noise was indeed caused by the crappy resistors.
Below is the circuit for one of the 5 LEDs. The output of the op-amp structure directly after the LT1031 reference is split between 5 stages identical to the structure on the right. I have shown just one (UR) for clarity. The various measurement points are explained below.
I started from the beginning of the circuit, directly after the LT1031, to make sure that the excess noise seen the other day wasn't just from a noisy reference. Below is the measured output voltage noise along with the LISO estimate. Clearly, the LT1031 is performing to spec (as it should, since it's a new part that I just put in). Note that the apparent better-than-spec performance at low frequencies is just from the AC coupling, which I needed due to the high DC level.
Since the reference was in order, the next step was to switch out some of the crappy old resistors for nicer thin-film ones. In case anyone is interested, Frank has done some detailed investigation of excess 1/f current noise in resistors. I measured the voltage noise level at the point labeled "inter-stage measurement" above, first without any modifications and then after swapping the old 10k resistors (R1 & R2) out for nice Vishay thin-film ones. There is clearly a big improvement, and the modified circuit essentially agrees with LISO now down to 1 Hz. Below this, it looks like there could still be an issue.
I wanted to see what the improvement was in the overall output current noise of the system, so I went about measuring the current noise as I had the other day (by measuring the voltage noise across R55 and dividing by the resistance). The performance was already better than the old measurement, but not at the LISO level. So, I replaced the current-setting resistors (R54 & R55)---which were actually 3 parallel resistors on a single pad in each case---by nice Vishay ones, as well. I didn't have any that were close to the original resistance of ~287 ohms, so I put three 1k ones in parallel. This of course shifts the resistance up to 333 ohms, but that only causes a ~16% change in current. I was sure to convert voltage noise into current noise with this new resistance, though.
With this change, the total output current noise is now very close to the LISO estimate as well down to ~1 Hz.
I gave up tonight's locking activity because the MC can't stay locked.
WE currently use long cables to give us the dispersion that we want for the MFD. A cable gives a long delay - both the phase delay and the group delay.
But we only need the dispersion (group delay). We can get this by just using a very sharp low pass filter and having the corner be above the frequency that we have the beat signal.
For example, the MiniCircuits SLP-200+ has got a corner frequency of 200 MHz and a group delay of ~10 ns (like a 3 m vacuum delay). So we would have to use 10 of these to get the delay we now get. The passband attenuation is only 0.5 dB, so we would lose 5 dB. The cost is $35 ea. We have a few on the shelf.
OTOH, if we tune the beat frequency down to 30 MHz, we can use the SLP-30 which has a group delay of 30 ns around 30 MHz. That's like 9m at light speed. We could easily get a nice result by just using 4 or 5 SLP in series.
So why is Kiwamu using cables?? And how should we really choose the beat note frequency??
These observations of the OSEMs were taken while taking transfer functions of oplev YAW at excitation amplitude 0.1
Atm1, C1:SUS-ETMX_SENSOR_SIDE cross coupling
Atm2, C1:SUS-ITMX_SENSOR_LL not excitable
Atm3-4, BS and PRM insensitive
Good OSEM list: ITMY, ETMY and SRM
SUS- BS, ITMX, ITMY, PRM, SRM, ETMX & ETMY_OLPIT transfer funtion with sine wave excitation 0.1 amplitude:
OL_YAW transfer functions are here.
I had two PHDs helping me to overlap the EXML files in DTT. We failed. This job requires professorial help.
There has been no lock of input MC for more than 5days. WTF???
I have fixed a loose mirror of the PMC input and the alignment of the MC2 Yaw.
- The PSL mech-shutter was closed. It has been opened.
- Then, I checked the MC suspensions. Mainly MC2 Yaw has kept drifting. (Fig.1)
In fact, there was no WFS actions during this drift.
Anyway, now MC2 Yaw was aligned and the lock was restored.
- It was very unsatisfactory for me that the PMC alignment kept drifting.
The trend of the PMC REFL and PMC TRANS for a year suggests that:
I went into the PMC setup and tapped several optics in order to find any loose optic.
Immediately I found that the mirror before the AOM was loose. Basically any mild tapping was enough
to misalign the mirror such that the caivty loose the TEM00 mode.
I tightened the retainer set screw of the optic and aligned the PMC again. It looks OK now as I can not
misalign this optic by the tapping anymore. But if it still remains drifting, we need to replace the mount.
Frank pointed out to me that I had dumbly forgotten to include the voltage reference's noise. The LT1031 has an output noise level of ~125 nV/rHz above 10 Hz or so, and this at least makes the estimate much closer. I had also not included an extra LT1125 stage between the reference and the other stages. I guess I was tired.
The estimate is now within a factor of a few of the measured level, and it has roughly the right shape. Around 1 Hz, it looks like the measured data begin to roll up away from the model, though it's tough to say due to the effect of the AC coupling on the analyzer less than a decade below. If there is indeed extra noise here, Frank thinks it could be due to resistor current noise.
I'll switch one or two out for nicer ones and see if things change.
I'm driving C1:SUS-ITMX_OLYAW and PIT_EXC with amplitude 0,1-0.3 while taking transfer funtions of oplev.
The transfer functions are normal. However I noticed that the LL osem is not responding to this excitations
Healthy sensor respons should be like Atm3
I did some more investigation on the OSEM box today.
After removing some capacitors and still finding that the +15V rail was at over +20V, I decided to see if the TO-3 7815 that I removed behaved properly all by itself. It did. After some more poking around, I discovered that whoever assembled the board isolated the case of the regulator from the board. It is through the case that this package gets its grounding, so I removed the mica insulator, remounted the regulator, and all worked fine.
Since I had gotten a spare from Downs, I also replaced the LT1031 (precision 10-V reference), for fear that it had been damaged by the floating voltage regulator.
With the above out of the way, I was finally able to take some measurements. The first thing I did was to look at the LED drivers. I fixed the one stage that I mentioned in my last post by adding two 820-ohm resistors in parallel with the 1k, such that it was very close to all the others (which are 806 || 806 || 1k). With that, using a red LED, I measured a current of 34.5 mA (+/- 0.1) out of each of the 5 stages (UL, UR, LL, LR, S).
I then measured the current noise of each one by monitoring the voltage across the 287-ohm resistor in series with the LED. The driver works by putting the LED in the feedback path of an inverting amp. There is a 10-V input from the LT1031, and the values of the input and feedback resistors determine the current drawn through the LED. There is a buffer (LM6321) in the path to provide the necessary current.
The LISO model I made according to that description seems to make sense. I simply modeled the LED as a small resistor and asked LISO for the current through it. The transfer function shows the proper DC response of -49.15 dB(A/V) --> 34.8 mA @ 10 V, but, the estimated current noise doesn't add up with the measured levels:
I have to get to the bottom of this. Two possibilities are: 1) The buffer adds noise, and/or 2) I am modeling this invalidly.
I also began measuring the PD amplifier noise levels, though I only measured two of them for lack of time. I find it odd that there is a 100-ohm input series resistor on what I thought would be just a transimpedance amplifier. For that reason, I want to look into how the OSEMs are connected to this guy.
In any case, I measured the output noise of two of the PD amps by shorting the input side of the 100-ohm resistors to ground, and then I divided by their TF to get the input noise level. Here it is compared with the LISO estimate. I have plotted them in units of voltage noise at the input side of the resistors for lack of a way to infer the equivalent photocurrent noise level.
Above 2 Hz or so, the measured level agrees with the prediction. Below this, the measured noise level increases as 1/f, while it should go as the standard 1/sqrt(f) (the manufacturer-quoted 1/f corner is at 2 Hz). Another thing to get to the bottom of.
Kiwamu showed me how to do transferfunction of oplev pitch
The attached trend shows a problem with the QPD sums.
Steve - please check on Monday the laser powers and the ETM/ITM reflectivity for HeNe lasers. Maybe we have to increase the transimpedance gain in the heads.
ETMX and ETMY have 0.2 mW returning to their QPDs........so the gain must lower at ETMX
ITMX laser 1103P has only 0.67 mW output and 0.025 mW returning to the QPD.
ITMX and ETMX oplev lanching paths have lenses without AR coating. This is my fault. I will buy them.
ETMX 0.2 mW 900 counts
ITMX 0.025 1300
SRM 0.04 2600
BS 0.05 3500
PRM 0.06 4000
ETMY 0.2 9000
ITMY 0.3 14500
I took one of the spare OSEM satellite amps (schematic) from the cabinet down the Y arm this afternoon to begin testing. I spent most of the day amassing the melange of adapters and connectors I needed to talk to the relic. The most elusive was the über-rare 64-pin IDE connector, for which neither the 40m nor Downs or Bridge had a breakout (despite there being several Phoenix boxes on each electronics rack at the 40m---hmm...). The solution I came up with was to make a breakout cable myself, only there was no 64-pin ribbon. So, I carefully fed a 50-pin and most of a 16-pin ribbon side by side into one push-down connector, and that was that:
I also finally found a 25-pin D-sub breakout just after figuring out the proper pinout for a 25-to-9 adapter, which I thought I was going to have to use. OH WELL.
The first thing I figured I'd do is measure the LED drivers' current noise and see how it compared with LISO. I powered the box up and found that the TO-3 7815 regulator was putting out +20V---bad. I assumed it was broken, so I got another one from Downs and replaced it. Powered it up again and the output was still at +20V (WTF?). My suspicion is that one of the shielding capacitors has failed in some bizarre way, but I didn't have time to check this before I was beckoned to another task. This is where I'll start again next.
Another thing Frank and I noticed as we were figuring out how the driver worked was that the current-specifying resistor of one of the driver stages had not been properly modified along with the others, so it was forcing the feedback loop to rail. This mod was done precariously by adding two perpendicular sandwiched "Radd" resistors on top of the main one, so it's also possible that the ones for this stage had just been knocked off somehow (perhaps by the massive gender-switching ribbon chain hanging down on it). Steve and I noticed that there was a label on the box complaining that some part of the amp for one of the OSEMs wasn't working, but we peeled it off and threw it away because he figured it was outdated.
Anyway, in short, the plan going forward is as follows:
Fri Feb 03 19:57:20 2012
Fri Feb 03 20:25:19 2012 : Aligned all SUS to center their OL beams
Fri Feb 03 20:29:21 2012: Aligned all SUS to make OL_PIT = 0.5
As usual, I noticed several bad things within 30 seconds of sitting in front of the workstation. Today its that there are OFF or missing filters on the MC TRANS.
is this the normal state? Screenshot attached.
Schott, green welding glass, shade 14, 3 mm thick was measured in the beam path of 1.2W, S polarization of 1064nm at ~1 mm diameter size as MC reflected path.
Absorption 95%, R 5% at incident angle 25-50 degrees. It looks like the perfect material for beam trap.
The reason I've killed the c1lsc kernel was the following - when the code starts to run, it initializes some parameters and this takes ~0.2 msec per dof. Now, the old code did nothing with a DOF if C1:OAF-ADAPT_???_ONOFF == OFF. My code still initialized the parameters but then does nothing because no witness channels are given. But it spends 8*0.2 = 1.6 msec for initializing all 8 dof. As the code is called with frequency 2k, this was the reason for crashing. Now I've corrected my code, it compiles, runs and does not kill c1lsc. However, the old code would also kill the kernel if all DOF are filtered. So, when we'll use all 8 DOF, we'll have to split variable initialization.
But this is not the biggest problem. C1OAF model must be corrected, because, as for now, all 8 DOF call the same ADAPT_XFCODE function. As this function uses static variables, they will be all messed up by different DOF signals.
Reminder / Moral: Everything cannot be considered to be "working fine" if the MC isn't locking. See if you can figure out why, and especially if it's something that you screwed up, either fix it, or better yet, ask for help and learn how to fix what you broke.
When I left this morning, Steve was still working with the MC and it was unlocked anyway, I could not check it. By "fine" I meant only watchdogs. The thing is that before starting to work with c1lsc I turned off all the coils. Crazyness that Steve saw was after I turned them on back after reboot. This is a confusing thing - restarting models on c1lsc and burt restoring them is not enough. After I did it, everything at the STATUS MEDM screen was green, but the C1:SUS-???_??PD_VAR values went up after turning on coils. So sus and lsc communicated in a bad manner after the reboot. After restarting x02 model, the watchdogs were fine again.
Cold LED lights replaced hot halogen ones. Flat LED MYAL 6S, model #112560002 24VAC
This is a LATE ENTRY. They were purchased in Jan 2010 and installed 6 of them around May 2010
We moved the MC approximately back to where the sensors for each optic used to be (mostly touching MC2, but a little bit of MC1 to help the refl get back to its max value). MC is now locked, and with the help of the WFS it's back to nominal. I forgot to disable the WFS, so I think we aren't perfectly aligned, but we're close enough for the WFS to get us the rest of the way. We're heading over to JClub right now, so we're going to leave it as-is.
Rough draft of updated interlock drawing by Ben is here.
The mode cleaner is a little misaligned in pitch, and is very misaligned in yaw. The lowest order mode that is flashing is TEM11.
I had a look-see at the SUS sensors, to see if there were any big jumps. There were moderately sized jumps on all 3 mode cleaner optics.
The MC's lockloss was at ~8:22am this morning, and went along with a giganto kick to the optics. Steve tells me that Den might have been kicking up optics while doing computer things this morning, before Steve reminded him to shut off the watchdogs. However, Steve was also taking phots/measuring things near MC Refl, so maybe he's not totally absolved of blame. But this really looks like the optics settled to different places after big kicks.
I'm going to try to align the MC mirrors to get back to the sensor numbers from early this morning before chaos began.
This morning I killed again c1lsc kernel with the new realization of fxlms algorithm. It works fine with gcc compiler during the tests. However, smth forbidden for the kernel is going on. I'll spend some more time on investigatin it. Interesting thing is that I did not even pressed "On" at the OAF MEDM screen to make the code running. c1lsc suspended even before. May be there is some function-name mismatch.
After c1lsc suspention I recomiled back non-working code and rebooted c1lsc. c1sus is also bad after c1lsc reboot as they communicate. I killed x04, lsc, ass, oaf models on the c1lsc computer and sus, mcs, rfm, pem on the c1sus computer. Then I restarted x02 model and restored its burt snapshot from 08:07. After I started all models back and restored their burt snapshots from 08:07. Then I diag reset all started models.
Before starting new fxlms code I've shutted down all the optics so that possible c1lsc suspention would not make them crazy. After reboot I turned the coils back. Everything seems to work fine.
The 2W PSL laser is turned off. The danger laser lights are not illuminated at the entry doors because of malfunctioning electronic circuit!!!
Laser safety glasses are still required! Other lasers are in operation!
BEN fixed the interlock. The laser is turned ON. Thanks for all, Rich and Sam who came over to help. Atm1
All emergency shut- off switches, lights and door indicators are working at this moment. More about this tomorrow.
Atm2, PSL enclosure interlock jungle without REAL schematic drawing.....at this point.... We all agreed it is easier to redo the hole thing than find the problem
Atm3, Emergency shut off switches and illuminated signs from entry doors to AC on-off box ( Use this switches in emergency ONLY, otherwise leave alone , even it is labeled obsolete !)
Summery: I still do not really know what was wrong.