More BHD SUS screens added to sitemap (Attachment 1)
I spent most of today fighting various CDS errors.
Let's see how stable this configuration is. Onto some locking now...
Stability was short-lived it seems. When I came in this morning, all models on c1lsc were dead already, and now c1sus is also dead (Attachment #1). Moreover, MC1 shadow sensors failed for a brief period again this afternoon (Attachment #2). I'm going to wait for some CDS experts to take a look at this since any fix I effect seems to be short-lived. For the MC1 shadow sensors, I wonder if the Trillium box (and associated Sorensen) failure somehow damaged the MC1 shadow sensor/coil driver electronics.
I've left the c1lsc frontend shutdown for now, to see if c1sus and c1ioo can survive without any problems overnight. In parallel, we are going to try and debug the MC1 OSEM Sensor problem - the idea will be to disable the bias voltage to the OSEM LEDs, and see if the readback channels still go below zero, this would be a clear indication that the problem is in the readback transimpedance stage and not the LED. Per the schematic, this can be done by simply disconnecting the two D-sub connectors going to the vacuum flange (this is the configuration in which we usually use the sat box tester kit for example). Attachment #1 shows the current setup at the PD readout board end. The dark DC count (i.e. with the OSEM LEDs off) is ~150 cts, while the nominal level is ~1000 cts, so perhaps this is already indicative of something being broken but let's observe overnight.
Overnight, all models on c1sus and c1ioo seem to have had no stability issues, supporting the hypothesis that timing issues stem from c1lsc. Moreover, the MC1 shadow sensor readouts showed no negative values over a ~12hour period. I think we should just observe this for another day, in any case I don't think there is any urgent IFO related activity scheduled.
I am starting the c1x04 model (IOP) on c1lsc to see how it behaves overnight.
Well, there was apparently an immediate reaction - all the models on c1sus and c1ioo reported an ADC timeout and crashed. I'm going to reboot them and still have c1x04 IOP running, to see what happens.
[97544.431561] c1pem: ADC TIMEOUT 3 8703 63 8767
[97544.431574] c1mcs: ADC TIMEOUT 1 8703 63 8767
[97544.431576] c1sus: ADC TIMEOUT 1 8703 63 8767
[97544.454746] c1rfm: ADC TIMEOUT 0 9033 9 8841
2 more earthquakes in Chile: a M6.4 and a M7.8.
We got them about 15 minutes ago (according to the BLRMS on the wall), but when I go tin, the MC was already locked, and engaging the LSC immediately got me PRMI lock (since that's the alignment state that the IFO was left in).
Prep for this work:
I was trying to get some pics of the optics as a zeroth-level reference for the pre-vent loss with the single arms locked, but since our SL7 upgrade, the sensoray won't work anymore . I'll try fixing this during the daytime.
In the end I decided to access the available spare DAC channels via the C1ASS model - for this purpose, I added a namespace block "TEST" in the C1ASS simulink model, which is a SISO block. Inside is just a single CDS filter module. My idea is to use the EXC of this filter module to inject excitations for measuring various couplings. Rather than have a simple testpoint, we also have the option of adding in some filter shapes in the filter module which could possibly allow a more direct read-off of some coupling TF. Recompiling the model went smooth - there was a crash earlier in the day which required me to hard-reboot c1lsc (and also restart all models on c1sus and c1ioo but no reboots necessary for those machines).
Note that to get the newly added channels to show up in the channel lists in DTT/AWGGUI etc, you need to ssh into fb1 and restart the daqd processes via sudo systemctl restart daqd_*. If I remember right, it used to be enough to do telnet fb 8088 followed by shutdown. This is no longer sufficient.
It took me a while to get the DRMI locking going again. The model restarts earlier in the evening had changed a bunch of EPICS channel settings (and out config scripts don't catch all of these settings). In particular, I forgot to re-enable the x3 digital gain for the ITMs, BS and SRM (necessitated by removing an analog x3 gain on the de-whitening boards). I was hesitant to spend time re-adjusting all damping / oplev loop gains because if we change the series resistor on the coil driver board, we will have to do this again. I also didn't want this arbitrary FM to be enabled in the SDF safe.snap. But maybe it's worth doing it anyways - if nothing it'll be good practise.
Once I hunted down all the setting diffs and tweaked alignment, the DRMI locks were pretty robust.
I had hoped to make some of these TF measurements tonight. But I realized I needed to look up a bunch of stuff in manuals/datasheets, and think about these measurements a little. I wasn't sure if the DW/AI board could drive a signal over 40m of BNC cabling so I added an SR560 (DC coupled, gain=1, low noise mode, 50ohm output used) to buffer the output. The Marconi's external modulation input is high impedance (100k) but for the AOM driver we want 50ohm. For the Marconi, the external input accepts 1Vrms max, while for the AOM driver, we want to drive a signal between 0V and 1V at most.
The general measurement setup is schematically shown in Fig 1. Questions to address:
Am I missing something?
We want to know that we can lock the interferometer with the ALS beat note being generated by beating IR pickoffs (rather than the vertex green transmission). The hope is also to make the ALS system good enough that we can transition the CARM offset directly to 0 after the DRMI is locked with arms held off resonance.
Attachment #1: Shows the layout. The realized MM is ~36 %. c.f. the 85% predicted by a la mode. It is difficult to optimize much more given the tight layout, and the fact that these fast lenses require the beam to be well centered on them. They are reasonably well aligned, but I don't want to futz around with the pointing into the doubling crystal. Consequently, I don't have much control over the pointing.
Attachment #2: Shows pictures of the fiber tips at both ends before/after cleaning. The tips are now much cleaner.
The BeatMouth NF1611 DC monitor reports ~580 mV with only the EY light incident on it. This corresponds to ~60 uW of light making it to the photodiode, which is only 25% of what we send in. This is commensurate with the BS loss + mating sleeve losses.
To find the beat between PSL and EY beams, I had to change the temperature control MEDM slider for the EY laser to -8355 cts (it was 225 cts). Need to check where this lies in the mode-hop scan by actually looking at the X-tal temperature on the front panel of the EY NPRO controller - we want to be at ~39.3 C on the EY X-tal, given the PSL X-tal temp of ~30.61 C. Just checked it, front panel reports 39.2C, so I think we're good.
EY enclosure was closed up and ETMY Oplev was re-enabled after my work. Some cleanup/stray beam dumping remains to be done, I will enlist Chub's help on Monday.
As shown in the Attachments, it seems like IMC DAC and coil driver noise is the dominant noise source above 30Hz. If we assume the region around the bounce peak is real motion of the stack (to be confirmed with accelerometer data soon), this NB is starting to add up. Much checking to be done, and I'd also like to get a cleaner measurement of coil driver and DAC noise for all 3 optics, as there seems to be a factor of ~5 disagreement between the MC3 coil driver noise measurement and my previous foray into this subject. The measurement needs to be refined a little, but I think the conclusion holds.
Since I sunk some time into it already, the motivation behind this work is just to try and make the IMC noise budget add up. It is not directly related to lowering the IR ALS noise, but if it is true that we are dominated by coil driver noise, we may want to consider modifying the MC coil driver electronics along with the ITM and ETMs.
Today, I repeated the coil driver noise measurement. Now, the coil driver noise curve in the noise budget plot (Attachment #1) is the actual measurement of all 12 coils (made with G=100 SR560). I am also attaching the raw voltage noise measurement (input terminated in 50ohms, Attachment #2). Note that POX11 spectrum has now been re-measured with analog de-whitening engaged on all 3 optics such that the DAC noise contribution should be negligible compared to coil driver noise in this configuration. The rows in Attachment #2 correspond to 800 Hz span (top) and full span (bottom) on the FFT analyzer.
The main difference between this measurement, and the one I did middle of last year (which agreed with the expectation from LISO modeling quite well) is that this measurement was done in-situ inside the eurocrate box while last year, I did everything on the electronics benches. So I claim that the whole mess of harmonics seen in the raw measurements are because of some electronics pickup near 1X6. But even disregarding the peaky features, the floor of ~30nV/rtHz is ~6x than what one would expect from LISO modeling (~5nV/rtHz). I confirmed by looking that the series resistance for all 3 MC optics is 430ohms. I also did the measurement with the nominal bias voltages applied to the four channels (these come in via the slow ADCs). But these paths are low-passed by an 8th order low pass with corner @ 1Hz, so at 100 Hz, even 1uV/rtHz should be totally insignificant. I suppose I could measure (with EPICS sine waves) this low-pass filtering, but it's hard to imagine this being the problem. At the very least, I think we should get rid of the x3 gain on the MC2 coil driver de-whitening board (and also on MC1 and MC3 if they also have the x3 factor).
Using the training data that was collected during the MISO MCL FF. I decided to look at more MCL subtractions but this time using the accelerometers as Rana suggested.
I first plotted the coherence between MCL and all six accelerometers and the T240-Z seismometer.
For 1 - 5 Hz, based on coherence, I decided to do SISO Wiener filtering with ACC2X and MISO Wiener filtering with ACC2X and ACC1Y. The offline subtractions were as follows (RMS plotted from 0.1 to 10 Hz):
The subtractions above look very much like what you would get offline when using the T240(X,Y) seismometeres during MISO Wiener filtering. But this data was taken with the MISO filters on. This sort of shows the performance deterioration when one does the online subtractions. This is not surprising since the online subtraction performance for the MISO filters, was not too great at 3 Hz. I showed this in some other ELOG but I show it again here for reference:
Anyways, foor 10 - 20 Hz, again based on coherence, I decided to do SISO Wiener filtering with ACC2Z and MISO Wiener filtering with ACC2Z and ACC1Z (RMS plotted from 10 to 20 Hz):
I will try out these subtractions online by today. I'm still debating wether the MISO subtractions shown here are worth the Vectfit shananigans. The SISO subtractions look good enough.
We did some more measurements on the PRM L2P TF.
We tried to compare the parameter estimation uncertainties of white vs. optimal excitation. We drove C1:SUS-PRM_LSC_EXC with "Normal" excitation and digital gain of 700.
For the white noise exciation, we simply put a butter("LowPass",4,10) filter to select out the <10 Hz band.
For the optimal exciation, we use butter("BandPass",6,0.3,1.6) gain(3) notch(1,20,8) to approximate the spectral shape reported in elog:16384. We tried to use awg.ArbitraryLoop yet this function seems to have some bugs and didn't run correctly; an issue has been submitted to the gitlab repo with more details. We also noticed that in elog:16384, the pitch motion should be read out from C1:SUS-PRM_OL_PIT_IN1 instead of the OUT channel, as there are some extra filters between IN1 and OUT. Consequently, the exact optimal exciation should be revisited, yet we think the main result should not be altered significantly.
While a more detail analysis will be done later offline, we post in the attached plot a comparison between the white (blue) vs optimal (red) excitation. Note in this case, we kept the total force applied to the PRM the same (as the RMS level matches).
Under this simple case, the optimal excitation appears reasonable in two folds.
First, the optimization tries to concentrate the power around the resonance. We would naturally expect that near the resonance, we would get more Fisher information, as the phase changes the fastest there (i.e., large derivatives in the TF).
Second, while we move the power in the >2 Hz band to the 0.3-2 Hz band, from the coherence plot we see that we don't lose any information in the > 2 Hz region. Indeed, even with the original white excitation, the coherence is low and the > 2 Hz region would not be informative. Therefore, it seems reasonable to give up this band so that we can gain more information from locations where we have meaningful coherence.
For the oplev, there are DQ channels you can use so that its possible to look back in the past for long measurements. They have names like PERROR
We report here the analysis results for the measurements done in elog:16388.
Figs. 1 & 2 are respectively measurements of the white noise excitation and the optimized excitation. The shaded region corresponds to the 1-sigma uncertainty at each frequency bin. By eyes, one can already see that the constraints on the phase in the 0.6-1 Hz band are much tighter in the optimized case than in the white noise case.
We found the transfer function was best described by two real poles + one pair of complex poles (i.e., resonance) + one pair of complex zeros in the right-half plane (non-minimum phase delay). The measurement in fact suggested a right-hand pole somewhere between 0.05-0.1 Hz which cannot be right. For now, I just manually flipped the sign of this lowest frequency pole to the left-hand side. However, this introduced some systematic deviation in the phase in the 0.3-0.5 Hz band where our coherence was still good. Therefore, a caveat is that our model with 7 free parameters (4 poles + 2 zeros + 1 gain as one would expect for an ideal signal-stage L2P TF) might not sufficiently capture the entire physics.
In Fig. 3 we showed the comparison of the two sets of measurements together with the predictions based on the Fisher matrix. Here the color gray is for the white-noise excitation and olive is for the optimized excitation. The solid and dotted contours are respectively the 1-sigma and 3-sigma regions from the Fisher calculation, and crosses are maximum likelihood estimations of each measurement (though the scipy optimizer might not find the true maximum).
Note that the mean values don't match in the two sets of measurements, suggesting potential bias or other systematics exists in the current measurement. Moreover, there could be multiple local maxima in the likelihood in this high-D parameter space (not surprising). For example, one could reduce the resonant Q but enhance the overall gain to keep the shoulder of a resonance having the same amplitude. However, this correlation is not explicit in the Fisher matrix (first-order derivatives of the TF, i.e., local gradients) as it does not show up in the error ellipse.
In Fig. 4 we show the further optimized excitation for the next round of measurements. Here the cyan and olive traces are obtained assuming different values of the "true" physical parameter, yet the overall shapes of the two are quite similar, and are close to the optimized excitation spectrum we already used in elog:16388.
There are still many mysteries remaining - e.g. the MICH-->PRCL contribution still can't be nulled. But for now, I have the settings that keep the PRMI locked fairly robustly with REFL55I/Q or REFL165I/Q (I quadrature for PRCL, Q for MICH in both cases), see Attachment #1 and Attachment #2 respectively. For the 1f locking, the REFL55 digital demod phase was fine-tuned to minimize the frequency noise (generated by driving MC2) coupling to the Michelson readout (as the Michelson is supposed to be immune) - the coupling was measured to be ~60dB larger at the PRCL error point than MICH. There was still nearly unity coherence between my MC2 drive and the MICH error point demodulated at the drive frequency, but I was not able to null it any better than this. With the PRMI (ETMs misaligned) locked on the 1f signals, I measured Attachment #1 and used it to determine the demod phase that would best enable REFL165_I to be a PRCL sensor. Rana thinks that if there is some subtle effect in the marginally stable PRC, we would not see it unless we do a mode scan (time consuming to set up and execute). So I'm just going to push on with the PRFPMI locking - let's see if the clean arm mode forces a clean TEM00 mode to be resonant in the PRC, and if that can sort out the lack of orthogonality between MICH/PRCL in the 1f sensors (after all, we only care about the 3f signals in as much as they allow us to lock the interferometer). I'll try the PRMI with arms on ALS tomorrow eve.
I have no idea what to make of how the single frequency lines I am driving in MICH and PRCL show up in REFL11 and REFL33 - the signals are apparently completely degenerate (in optical quadrature). How this is possible even though the PRMI remains stably locked, POP22/POP110/AS110 report stable sideband buildup is not clear to me.
How can you lock the PRMI without the REFL beams? c.f. this entry by Kiwamu
Which signals are you using for the locking?
I think the first priority is to find the fringes of the arms and lock them with POX/POY.
As for the POP, make sure the beam is not clipped because the in-vac steering mirrors
have been supposed to be too narrow to accommodate these two beams.
I was using AS55I for PRCL, and AS55Q for MICH. I snuck that into the last line of an unrelated elog, since I did both things at the same time: see elog 7551. Kiwamu's measurements (elog 6283) of the PRMI sensing matrix show that the PRCL and MICH signals are almost orthogonal in AS55 (although the optickle simulation doesn't agree with that...) He was able to lock PRMI with AS55 I&Q (elog 6293), so I thought we should be able to as well. Locking the PRMI was supposed to help tune the alignment of the PRM, not be the end goal of the night. Also, we only tried locking PRCL in the "middle right" configuration, not the "lower left" configuration, but we were seeing what looked like recycling flashes only in the "lower left" configuration.
I agree in principle that we should be working on the arms. However, since we can't use the old steer-the-beam-onto-the-cage trick to find the beam, I was hoping that we could steer the beam around and see some light leaking out of the ETM, onto the end table. However, with the 1% transmission of the ITMs and ~10ppm transmission of the ETMs, there's not a lot of light back there. I was hoping to align the PRMI so that I get flashes with a gain of 10 if I'm lucky, rather than just the 5% transmission of the PRM. With the PRMI aligned, I was expecting:
(1W through Faraday) * (10 PR gain) * (0.5 BS transmission) * (0.01 ITM transmission) * (10ppm ETM transmission) = 0.5uW on the ETM tables during PRCL flashes.
I was hoping that things would be well enough aligned that I could just go to the end table, and see the light with a viewer, although as I type this, I realize that if the beam was not on the end table (or even if it was...) any time I move the PZTs, I'd have to completely realign the PRMI in order to see the flashes. This seems untenable, unless there are no other options.
We then got sidetracked by trying to see the POP beam, and once we saw the POP beam we wanted to put something down so we could find it again. POP is also small, but not as small as expected at the end:
(1W through Faraday) * (10 PR gain) * (20ppm PR2 transmission) = 0.2mW on POP during PRCL flashes.
POP was very difficult to see, and we were only able to see it by putting the foil in the beam path, and using a viewer. I think that we once were able to see it by looking at a card with the viewer, but it's much easier with the foil. I'd like to find an iris that is shiny (the regular black iris wasn't helpful), to facilitate this alignment. Since we were just looking at the reflection off of the foil, I have no comment yet about clipping vs. not clipping. I do think however that the forward-going beam may have been easier to find....when the PRMI alignment drifted, we lost the beam, but I could still see the forward-going beam. Probably I should switch to that one, since that's the one that was lined up with the in-vac optics.
Ideas are welcome, for how to align the beam to the Yarm (and later to the Xarm), since our old techniques won't work. Aligning the PRMI was a distraction, although in hopes of getting flashes so we could see some light at the end tables. I'm going to go see if I can look through a viewport and see the edges of the black glass aperture, which will potentially be a replacement for the steering-on-the-cage technique, but if that doesn't work, I'm running out of ideas.
Here we trended also the PMC and the MZ. The drop in the PMC happens at the same rate as the MOPA's.
That let us think that the FSS transmitteed power has gone down because of the reference cavity progressive misalignment to the laser beam.
We need to adjust that alignment sometime.
The drop in the NPRO output power (upper row, 3rd plot: Ch10 C1:PSL_126MOPA_126MON) accompained an increase of "fuzziness" in PMCTRANSPD and both coincided in time with the day we tempoarirly removed the flap from the laser chiller's chiller (July 14 2009).
Even more plots for the Wiener filtering!
We have a set of spectrograms, which show (in color) the amplitude spectrum, at various times during a one month stretch of time, during S5. Each vertical data-'stripe' is 10min long.
We also have a set of band-limited plots, which take the spectra at each time, and integrate under it, for different frequency bands.
Each set of plots has the following 3 plots: The raw DARM spectrum, a ratio of residual/raw, and the residuals, normalized to the first one (on which the wiener filter was trained).
The residuals are the DARM spectrum, after subtracting the Wiener-filtered seismometer witness data.
From the ratio plots, it looks like the wiener filter is pretty much equally effective at the time on which the filter was trained, as one month later. Static filters may be okey-dokey for a long period of time with for the seismic stuff.
Kiwamu, Keiko, Anamaria
I started today with a different input beam, so I had to realign the REFL path again. Then we measured the RF signal out of the 4 REFL PDs and found them to be too low. We increased the power to around 10mA for each diode, and we can see the right modulation frequency on each diode, though REFL165 is way too weak so we might need an RF amplifier on it. We will measure demod board noise tomorrow.
We had an issue with REFL165 not giving the right DC level, low by a factor of 10, even though it was receiving the same optical power as the others. We fifteen-checked clipping and alignment, then pulled it out and measured it on the test stand - found it to be ok. So I uplugged its power cable at the rack and connected it to the AS165 slot. Problem sloved. Not sure what was wrong with the other power slot.
Then we found REFL55 to be clipping on its black glass, we fixed that. But the REFL55 DC power still changes a lot with seemingly not huge motions of the PRM. We'll investigate more tomorrow.
We added a lens in the path to REFL165 because unlike the others it is a 1mm diode. All diodes have about half a turn to a full turn flatness of maximum (on tiny steering mirror).
We set the whitening gain on all four diodes to 21 db.
Not sure if we should set the power to be different on these diodes since their sensitivity is different to RF, and now REFL11 sees huge signal.
We continued the DRMI locking attempt and brought in the SRC, using AS55I to control it. It kind of works/stays locked. We did manage to get MICH and PRC better controlled than last night, but with SRC in the mix, something is wrong. We have to redo f2a filters on SRM and hopefully things will be better after Jenne's suspension work tomorrow. Oplevs not optimized yet either.
We intend to realign POY beam path so we can monitor power in cavities.
This morning (about 10am to 11am), I have collected additional transfer function measurements for the T.T. suspension. I have finished taking my measurements. The SR785 has been returned to its place next the the seismometer racks.
The data has been backed up onto the cit40m computer
We characterized Tip Tilts numbers 2 & 3 today. Recall #4 is the one which Koji and I measured some time ago, and #s 1 & 5 have yet to be suspended (that's on the to-do list for tomorrow).
When we began looking at #3 (the one which had been in the BS chamber for a few days, but was removed for characterization) we found that the pitch pointing was way off. The beam was way too low after reflection. So we adjusted that (and got it right on the first try....a miracle!). This does however make me pretty concerned about our in-chamber pointing. Are we destroying our pointing during travel between the cleanroom and the chambers? Is there anything we can do about it? Pointing doesn't seem to be lost when we move them around on the tables in the cleanroom, ie we can pick up a TT, move it, leave it for a while, move it back to the oplev, and the pointing still seems okay. But the TT which was sent to the chambers came back with bad pointing. I'm sure this is related to the hysteresis we see in the pointing if we poke the top of the mirror holder versus the bottom when exciting pitch motion.
For both #2 and #3, we measured the frequency and Q of Pitch, Yaw, Pos, Side, Vert motion. For the Vert motion, we measured both without and with EQ stops as dampers. For the other modes, all were measured with the ECD backplane in place. Pitch and Yaw were measured with reflection off of the mirror surface onto the PD, while Pos, Side and Vert were measured using the wire clamp on the mirror holder to obscure the beam as a shadow sensor.
Pitch: Overdamped, no freq measured, Q < 1/2
Yaw: freq ~1.8Hz, Q between 2-7
Pos: freq ~1.75Hz, Q too low to measure, but above critically damped
Side: freq ~ 1.8Hz, Q~5
Vert no dampers: freq ~20Hz, Q~36
Vert with dampers on outer screws: freq~24Hz, Q~8,
Pitch: no freq measured. Q~1/2? Upon being excited in Pitch, the beam started down way below the photodiode, came up a little past its DC place, and went back down a tiny bit. So not quite overdamped.
Yaw: freq ~1.96Hz, Q very low
Pos: freq ~1.72Hz, Q~3
Side: freq ~1.85Hz, Q~6
Vert no dampers: freq ~20Hz, Q~75
Vert with dampers on outer screws: freq ~20Hz, Q~34 (Frequency stayed constant....we did several measurements both with and without the dampers...but the half life time changed significantly)
Things we noticed: Koji and I had been concerned the last time we were looking at TT#2 because the frequency got lower and the Q seemed to get higher when we added the damping to the vertical blades. Yoichi and I did not find that to be true today. We did notice, however, that the EQ stop screws with the viton had been placed in the holes closer to the clamping point, whereas with TT #4 the screws had been placed in the holes farther from the clamping point. We moved the screws on TT #2 to the outer holes, and noticed that the frequency increased slightly, and the Q significantly decreased. We then followed this outer-hole philosophy when installing screws in TT #3.
To Do List: We need to suspend the ECDs and the Optics for the remaining two Tip Tilts, and to characterize them. We also (probably farther-future) need to take transfer functions using a shaker / shake table with our spare Tip Tilt. After all the TTs are suspended and have their modes measured, we will be ready for installation into the chamber during the next vent.
The inner and outer screw holes of the EQ stop Jenne is talking about are shown in the photo below.
More progress on the AUX-laser cavity scans.
Both data sets are attached.
Procedure tomorrow [comments / suggestions welcome]:
All photos have been uploaded to google photos.
With Chub's help, most of the problems have been resolved. Summary: I judge that we are good to go ahead with an install tomorrow.
Since we don't immediately need the CM board, I say we push ahead with the install - at least that will restore the ability to lock PRMI / DRMI. Then we can debug these issues in situ - I'm certain the issue is related to the EPICS/Modbus setup and not the hardware because I verified the physical channel map using the Acromag windows utility.
I bench tested the functionality of all the c1iscaux Acromag crate channels. Summary: we are not ready for a Monday install, much debugging remains.
> Looking through the manual, I found a recommendation (pg10) that the "IN-" terminal of the Acromag ADC units be tied to the "RTN" pins on the same units. I don't know if this preserves the differential receiving capability of the Acromag ADCs
I suppose, we loose the differential capability of an input if the -IN is connected to whatever defined potential. We should check if the channels are still working as a true differential or not.
2. If the multi bit operation is too complicated to solve, we can use EPICS Calc channels to breakout a value to bits and send the individual bits as same as the other individual binary channels.
As with the last two posts, I eliminated more unnecessary passing through c1rfm for IPC connections between c1mcs and c1ioo.
All models were rebuilt/installed/restarted and svn committed. Everything is working and we have eliminated almost all IPC errors and significantly simplified things.
Jordan and I removed another 10 kg of cabling from 1X2. The c1iool0 crate now has all cabling to it disconnected - but it remains in the rack because I can't think of a good way to remove it without disturbing a bunch of cabling to the fast c1iool0 machine. We can remove it the next time the vertex FEs crash. Cross connects have NOT been removed - we will identify which cross connects are not connected to the fast system and trash those.
Do we want to preserve the ability to use the PZT driver in 1X2?
After discussing with Koji, I removed the PZT driver and associated AI card from the Eurocrate at 1X2. The corresponding backplane connectors were also removed from the cross connects. An additional cable going from the DAC to IDC adaptor on 1X2 was removed. Finally, some cables going to the backplane P1 and P2 connectors for slots in which there were no cards were removed.
Finally, there is the IMC WFS whitening boards. These were reconfigured in ~2016 by Koji to have (i) forever whitening, and (ii) fixed gain. So the signals from the P1 connector no longer have any influence on the operation of this board. So I removed these backplane cables as well.
Some pics attached. The only cross connect cabling remaining on the south side of 1X2 is going to the fast BIO adaptor box - I suspect these are the triggered fast whitening switching for the aforementioned WFS whitening board. If so, we could potentially remove those as well, and remove all the cross connects from 1X1 and 1X2.
Update 1720: indeed, as Attachment #2 shows, the RTCDS BIO channels were for the WFS whitening switching so I removed those cables as well. This means all the xconnects can be removed. Also, the DAC and BIO cards in c1ioo are unused.
We are going to replace the old Sun c1ioo with a modernized supermicro. At the opportunity, remove the DAC and BIO cards to use them with the new machines. BTW I also have ~4 32ch BIO cards in my office.
Power supply bypassing [updated 10pm]:
As mentioned earlier in this thread, I prepared a box with two 10uF, 1kV rated capacitors to bypass the high-voltage rails (see inset in the plot), to see if that improves the performance. However, in measuring the voltage ripple directly with the SR785 (no load connected), I don't see any significant difference whether the decoupling caps are connected or not, see Attachment #1. For this, and all other HV measurements made, I used this box to protect the SR785. One hypothesis is that this box itself is somehow introducting the excess noise, maybe because of leakage currents of the diode pair going into the 1Mohm SR785 input impedance, but I can't find any spec for this, and anyway, these diodes should be at ground potential once the transient has settled and the DC blocking capacitor has charged to its final value.
Note that the 10uF caps have an ESR of 7.2 mOhms. The HP6209 has a source impedance "<20mOhm" when operated as a CV source, per the datasheet. So perhaps this isn't so surprising? The same datasheet suggests the source impedance is 500 mOhms from 1kHz to 100 kHz, so we should see some improvement there, but I only measured out to 2 kHz, and I didn't take much effort to reduce these crazy peaks so maybe they are polluting the measurement out there. There must also be some continuous change of impedance, it cannot be <20 mOhm until 1 kHz and then suddenly increase to 500 mOhms. Anyways, for this particular circuit, the nosie DC-1kHz is what is important so I don't see a need to beat this horse more.
Simplified circuit testing:
I decided to see if I can recover the spec'd voltage noise curve from the PA95 datasheet. For this, I configured the PA95 as a simple G=31 non-inverting amplifier (by not stuffing the 15 uF capacitor in the feedback path). Then, with the input grounded, I measured the output voltage noise on the circuit side of the 25kohm resistor (see inset in Attachment #2). To be consistent, I used the DC blocking box for this measurement as well, even though the output of the PA95 under these test conditions is 0V. Once again, there is considerable excess around ~100 Hz relative to a SPICE model. On the basis of this test, I think it is fair to say that the problem is with the PA95 itself. As far as I can tell, I am doing everything by the book, in terms of having gain > 10, using a sufficiently large compensaiton cap, HV rail decoupling etc etc. Note that the PA95 is a FET input opamp, so the effects of input current noise should be negligible. The datasheet doesn't provide the frequency dependence, but if this is just shot noise of the 1200 pA input bias current (for 300 V rails, per the spec), this is totally negligible, as confirmed by LTspice.
In the spirit of going step-by-step, I then added the feedback capacitor, and still, measured noise in excess of what I would expect from my model + SR785 measurement noise.
Integrated circuit testing:
After the above simplified test, I stuffed a full channel as designed, and tested the noise for various drive currents. To best simulate the operating conditions, an Acromag XT1541 was used to set the DC voltage that determines the drive current through the 25 kohm resistor. The measurements were made on the circuit side of this resistor (I connected a 20ohm resistor to ground to simulate the OSEM). As shown in Attachment #3, the noise with these HP6209 supplies is significantly better than what I saw with the KEPCO supplies, lending further credence to the hypothesis that insufficient PSRR is the root of the problem here. I've added subplots in a few different units - to be honest, I think that reaching this level of measured displacement noise at the 40m at 100 Hz would already be pretty impressive.
So what's next?
The main design change is that a passive R-C-R (4k-3uF-20k) replaces the single 25kohm resistor at the output of the PA95.
Let's see if this fixes the issue. Not that I've also added a pair of input protection diodes to the input of the PA95 in the new design. The idea is that this would protect the (expensive) PA95 IC from, for example, the unit being powered with the +/- 18V rail but not the +/- 300 V rail. As I type this, however, I wonder if the leakage current noise of these diodes would be a problem. Once again, the datasheet doesn't provide any frequency dependence, but if it's just the shot noise of the 1nA expected when the diodes are not reverse biased (which is the case when the PA95 is operating normally since both inputs are at nearly the same potential), the level is ~20 fA/rtHz, comparable to the input current noise of the PA95, so not expected to be an issue. In the worst case, the PCB layout allows for this component to just be omitted.
I generated the following plots from the two sets of huddle test data we have for the accelerometers.
Old data: 6 accelerometers, no cables clamped, no box, 55 mins worth of data.
New data: 3 accelerometers, cables clamped, foam box put on placed and completely sealed, 20 mins worth of data.
I made sure to use the same Impuse response time (6 sec) and sampling frequency (256 Hz), as well as every other parameter for the calculations.
Top left: The resultant self noise curve using the new data, there is definitely and improvement in the 0.5-2 Hz band.
Top right: Resultant self noise using the old data, for the first set of three accelerometers.
Bottom left: Old data result for the remaining three accelerometers.
Bottom right: Old data result, using all six accelerometers as witnesses instead.
There were many locklosses from the point where the arm powers were somewhat stabilized. Attachments #1 and #2 show two individual locklosses. I think what is happening here is that the BS seismometer X channel is glitching, and creating a transient in the angular feedforward filter that blows the lock. The POP QPD based feedback loop cannot suppress this transient, apparently. For now, I get around this problem by boosting the POP QPD feedback loop a bit, and then turning the feedforward filters off. The fact that the other seismometer channels don't report any transient makes me think the problem is either with the seismometer itself, or the readout electronics. The seismometer masses were recently recentered, so I'm leaning towards the latter.
I didn't explicitly check the data, but I am reasonably certain the same effect is responsible for many PRMI locklosses even with the arms held off resonance (though the tolerance to excursions there is higher). Pity really, the feedforward filters were a big help in the lock acquisition...
To study the evilution of the AO path TFs a bit more, I've hooked up POY11_Q Mon to IN1 of the CM board. I will revert the usual setup later in the evening.
Update 1730: I've returned the cabling at 1Y2 to the nominal config, and also reverted all EPICS settings that I modified for this test. Y-arm POY locking works. Attachment #1 shows the summary of the results of this test - note that the AO gain was kept fixed at +5dB throughout the test. I have arbitrarily trimmed the length of the frequency vector for some of these traces so that the noisy measurement doesn't impede visual interpretation of the plots so much. At first glance, the performance is as advertised. I basically followed the settings I had here to get started, and then ramped up various gains to check if the measured OLTF evolved in the way that I expected it to. The phase lead due to the AO path is clearly visible.
Some important differences between this test and the REFL11 blending is (i) in the latter case, there will also be a parallel loop, CARM_A, which is effecting some control, and (ii) the optical gain of CARM-->REFL11_I is much higher than L_Y-->POY. So the initial gain settings will have to be different. But I hope to get some insight into what the correct settings should be from this test. I think IMC servo IN2 gain and AO gain slider on the CM board are degenerate in the effect they have, modulo subtle effects like saturation.
One possibility is that the gain allocation I used yesterday was wrong for the dynamic range of the CARM error signal. In some initial trials today, when I set the CM board IN1 gain to -32dB (as in the case of attempting the CARM RF handoff) and compensated for the reduced POY PDH fringe amplitude by increasing the digital gain for the CM_Slow path, I found that there was no phase advance visible even when I ramped up the IMC IN2 gain to +10dB. So, for the CARM handoff too, I might have to start with a higher CM board IN_1 gain, compensate by reducing the CM_Slow digital gain even more, and then try upping the IMC IN2 gain.
P.S. When the excitation input to the CM board was enabled in order to make TF measurements, I saw significant increase in the RMS of the error signal. Probably some kind of ground loop issue.
This measurement tells you how the gain balance between the SLOW_CM and AO paths should be. Basically, what you need is to adjust the overall gain before the branch of the paths.
Except for the presence of the additional pole-zero in the optical gain because of the power recycling.
You have compensated this with a filter (z=120Hz, p=5kHz) for the CM path. However, AO path still don't know about it. Does this change the behavior of the cross over?
If the servo is not unconditionally stable when the AO gain is set low, can we just turn on the AO path at the nominal gain? This causes some glitch but if the servo is stable, you have a chance to recover the CARM control before everything explodes, maybe?
Summary of discussion between Koji and gautam on 14 Oct:
I tried all of these last night / overnight, here are my findings.
Analog locking of the homodyne phase:
See Attachment #1.
Relative stability of two IFR2023s synchronized to the same FS725 Rb standard:
The electrical LO signal for demodulation of the 44 MHz photocurrent is provided by an IFR2023 signal generator. To maintain a fixed phase relation between this signal, and the phase modulation sidebands imprinted on the interferometer light via a separate IFO2023 signal generator, I synchronize both to the same Rb timing standard (a 10 MHz signal from the FS725 is sent to the rear panel frequency standard input on the IFR). We don't have a direct 44 MHz electrical signal available from the main IFO Marconi at the LSC rack (or anywhere else for that matter). So I decided to do this test at 55 MHz.
A look at the time domain signal:
With the Michelson locked on the dark fringe, the RF44 I and Q signals in the time domain are shown in Attachment #3 for a 1 minute stretch.
I analyzed the data taken yesterday.
The AS11 data in PRMI configuration is very bad, while the AS55 seems good enough:
The phase differences are
AS11 = 21 +- 18 degrees (almost useless due to the large error)
AS55 = 11.0 +- 0.4 degrees
The AS55 phase difference is not the same measured in the last trial, but about half of it. The new length estimates are:
AS11 = 3.2 +- 2.8 cm
AS55 = 0.47 +- 0.01 cm
We can probably forget about the AS11 measurement, but the AS55 result is different from the previous estimate... Maybe this is due to the fact that Eric adjusted the PRCL offset, but then we're going in the wrong direction....
I managed to achieve a few transitions of control of the XARM length using the ALS error signal. The lock is sort of stable, but there are frequent "glitches" in the TRX level. Needs more noise hunting, but if the YARM ALS is also "good enough", I think we'd be well placed to try PRMI/DRMI locking with the arms held off resonance (while variable finesse remains an alternative).
Attachment #1: One example of a lock stretch.
Attachment #2: ASD of the frequency noise witnessed by POX with the arm controlled by ALS. The observed RMS of ~30pm is ~3-4 times higher than the best performance I have seen, which makes me question if the calibration is off. To be checked...
I ran some sensing measurements at various CARM offsets to check if the PRCL-->REFL33 and MICH-->REFL165 signals were being rotated out of the sensing quadrature as I lowered the CARM offset - there was no evidence of this happening. See Attachment #2. Other possibilities:
The IMC went into some crazy state so I'm calling it for the night, need to think about what could be happening and take a closer look at more signals during the CARM offset reduction period for some clues...
I looked at some signals for a 10 second period when the PRMI was locked with at some CARM offset, just before the PRMI lost lock, to see if there are any clues. I don't see any obvious signatures in this set of signals - if anything, the PRM is picking up some pitch offset, this is seen both at the Oplev error point and also in the POP QPD spot position. But why should this be happening as I reduce the CARM offset? The arm transmission is only ~5, so it's hard to imagine that the radiation pressure is somehow torquing the PRM. There are no angular feedback loops actuating on the PRM in this state except the local damping and Oplev loops.
The 1f signals are also changing their mean DC offset values, which may be a signature of a changing offset in the 3f MICH and PRCL error points? The MICH error signal is pretty noisy (maybe I can turn on some LPF to clean this up a bit), but I don't see any DC drift in the PRCL control signal.
I set up a photodiode (PDA10CF) in the IFO REFL beampath and the Agilent NA is sitting on the east side of the PSL enclosure. This was meant to be just a first look, maybe the PDA10CF isn't suitable for this measurement. The measurement condition was - PRM aligned so we have a REFL beam (DC level = 8.4V measured with High-Z). Both ITMs and ETMs were macroscopically misaligned so that there isn't any cavity effects to consider. I collected noise around 11 and 55 MHz, and also a dark measurement, plots to follow. The optics were re-aligned to the nominal config but I left the NA on the east side of the PSL enclosure for now, in anticipation of us maybe wanting to tune something while minimizing a peak.
Attachment #1: Results of a coarse sweep from 5 MHz to 100 MHz. The broadband RIN level is not resolvable above the dark noise of the photodiode, but the peaks at the modulation frequencies (11 MHz, 55 MHz and 29.5 MHz) are clearly visible. Not sure what is the peak at ~44 MHz or 66 MHz. Come to think of it, why is the 29.5 MHz peak so prominent? The IMC cavity pole is ~4kHz so shouldn't the 29.5 MHz be attenuated by 80dB in transmission through the cavity?
Attachment #2: Zoomed in spectra with finer IF bandwidth around the RF modualtion frequencies. From this first measurement, it seems like the RIN/rad level is ~10^5, which I vaguely remember from discussions being the level which is best achieved in practise in the 40m in the past.
Check the RAM due to the EOM? Perhaps the pointing / polarization control into the EOM got degraded.
Tried a bunch of things tonight.
One possibility is that the arm buildup is exerting some torque on the ITMs, which can also change the PRC cavity axis - as the buildup increases, the dominant component of the circulating field in the PRC comes from the leakage from the overcoupled arm cavity. We used to DC couple the ITM Oplev servos when locking the PRMI. The TRX level of 1 corresponds to ~5W of circulating power in the arm cavity, and the static radiation pressure force due to this circulating power is ~30 nN, rising up to 300nN as the TRX level hits 10. So for 1mm offset of the spot position on the ITM, we'd still only exert 300 pN m of torque. I don't see any transient in the Oplev error signals when locking the arm cavity as usual with POX/POY, but on timescales of several seconds, the Oplev error point shows ~3-5 urad of variation.
Some short notes, more details tomorrow.
Attachment #1 shows time series of some signals, from the time I ramp of ALS CARM control to a lockloss. With this limited set of signals, I don't see any clear indication of the cause of lockloss, but I was never able to keep the lock going for > a couple of mins.
Attachment #2 shows the CARM OLTF. Compared to last week, I was able to get the UGF a little higher. This particular measurement doesn't show it, but I was also able to engage the regular boost. I did a zeroth order test looking at the CM_SLOW input to make sure that I wasn't increasing the gain so much that the ADC was getting saturated. However, I did notice that the pk-to-pk error signal in this locked, 5kHz UGF state was still ~1000 cts, which seems large?
Attachment #3 shows the DTT measurement of the relative gains of DARM A and B paths. This measurement was taken when the DARM_A gain was 1, and DARM_B gain was 0.015. On the basis of this measurement, DARM_B (=AS55) sees the excitation injected 16dB above the ALS signal, and so the gain of the DARM_B path should be ~0.16 for the same UGF. But I was never able to get the DARM_B gain above 0.02 without breaking the lock (admittedly the lockloss may have been due to something else).
Attachment #4 shows a zoomed in version of Attachment #1 around the time when the lock was lost. Maybe POP_YAW experienced too large an excursion?
Some other misc points:
Yesterday I plugged the geophone and accelerometer output into the ADC, rather than the SR785, so I could collect for longer and take more data at once.
As per Rana's suggestion, I am also now taking the geophone output after the first op-amp in the circuitry following the geophone (a low-noise op-amp, OPA227). It acts as a buffer so I'm not just measuring other local noise sources (which explains why the geophone noise curve sort of matched the SR785 noise curve in my old plots).
With these changes, I remeasured the accelerometer and geophone noises as well as collected an ASD of a geophone sitting on the STACIS in open loop operation. I also looked up the noise specs for the various op-amps in the geophone pre-amp and high voltage board; everything I found, I added in quadrature to come up with an approximate op-amp noise value for the STACIS. All of this is plotted below:
I left the y-axis in V/rtHz instead of converting it to m/s/rtHz so that the op-amp noise could be compared to the other noises. All sensor data was taken with the sensors horizontal (noise data taken in granite and foam).
The accelerometer and geophone noise still appear to be similar, and the op-amp noise, at least according to specs, is low compared to the other noises. This implies there's not much to gain from switching the geophones with accelerometers nor with swapping out the op-amps for lower-noise components (unless the ones I couldn't find specs for were high-noise, though it seems like mainly low-noise components were used).
Looks like you're just measuring the ADC noise. You should add ADC noise to your plot. To compare the geophones with the accelerometers, you have to correct for the preamp gain and plot them both in the same units.
To get above the ADC noise you can use an SR560 preamp. (AC Coupled, G = 100)