I made a voltage divider using a 20.47kohm and 1.07kohm (both values measured with a DMM). The whole thing is packaged inside a Pomona box I found lying around on the Electronics bench. I have hooked it up to the ALSY_I channel and will leave it so overnight. The INMON of this channel isn't DQed, but for this test, the 16Hz EPICS data will suffice. I've locked the EX laser to the arm, enabled slow temperature servo to allow overnight lock (hopefully) and disabled LSC mode (as locking the arm to the MC tends to break the green lock)
To convert the INMON counts to RF power, I will use (based on my earlier calibration of this monitor channel, see DCC document for the demod chassis).
1AM update: Attachment #1 shows that the RF amplitude has been relatively stable (less than 10% of nominal value variation) over the course of the last hour or so. Even though there is some low frequency drift over timescales of ~20mins, no evidence of the wild ~20dB amplitude changes I saw last week. The signs are encouraging...
overnight update: See Attachment #2 - looking at the past 11 hours of second trend data during which the arm stayed locked, there actually seems to have been more significant variation in the beatnote amplitude. Swings of up to 6dBm are seen on a ~20min timescale, while there is also some longer term drift over 12 hours by a couple of dBm. There is probably a systematic error in the Y-axis, as I measured the RF power at the input of the power splitter at the LSC rack to be ~3dBm, so I expect something closer to 0dBm to be the LO input power which is what I am monitoring. So further debugging is required - I think I'll start by aligning the X fiber coupled beam to one of the fiber's special axes.
To make this setup more permanent, I modified the c1lsc model to pipe the LO power monitor signals from the Demod chassis to unused channels ADC_0_25 (X channel LO) and ADC_0_26 (Y channel LO) in the c1lsc model. I also added a couple of CDS filter blocks inside the "ALS" namespace block in c1lsc so as to allow for calibration from counts to dBm. I didn't add any DQ channels for now as I think the slow EPICS records will be sufficient for diagnostics. It is kind of overkill to use the fast channels for DC voltage monitoring, but until we have acromag channels readily accessible at 1Y2, this will do.
Modified model compiled and installed successfully, though I have yet to restart it given that I'll likely have to do a major reboot of all vertex FEs
I thought a little bit about the design of the preamp we want for the demodulated ALS signals today. The requirements are:
Attachment #3 shows a design I think will work (for now it's a whiteboard sketch, I''ll make this a computer graphic tomorrow). I have basically retained the differential sending and receiving capabilities of the existing Audio I/F amplifier, but have incorporated some whitening gain with a pole at ~150Hz and zero at ~15Hz. I've preserved the DC gain of 10, which seems to have worked well in my tests in the last week or so. Attachments #1 and #2 show the liso modelled characteristics. Liso does not support input-referred noise measurements for differential voltage inputs, so I had to calculate that curve manually - I suspect there is some subtlety I am missing, as if I plot the input referred noise out to higher frequencies, it blows up quite dramatically.
Next step is to actually make a prototype of this. I am wondering if we need a second stage of whitening, as in the current config, we only get 20dB gain at 150Hz relative to DC. Yesterday's beat spectrum measurement shows that we can expect the frequency noise of the ALS signal at ~100Hz to be at the level of ~1uV/rtHz, but this is is around the ADC noise level? If so, 20dB of whitening gain may be sufficient?
Still have to make preamp prototype daughter board with the right whitening shape... This test suggests to me that I should also make the output differential sending...
*Side note: I was wondering why we need the differential receiving stage, followed by a difference amplifier, and then a differential sending stage. After discussing with Koji, we think this is to suppress any common-mode noise from the mixer outputs.
Using one of the prototype PCB boards given to me by Johannes, I put together v1 of this board and tested it.
Attachment #1 - Schematic with stages grouped by function and labelled.
Attachment #2 - Measured vs modelled Transfer function.
Attachment #3 - Measured vs modelled noise. Measurement shown only between positive output and ground, the other port is basically the same. I will update this attachment to reflect the expected signal level in comparison to the noise, but suffice it to say that given the measured input referred noise, we will have plenty of SNR between 0.1Hz and 10kHz. The single stage of whitening should also be sufficient to amplify the signal above ADC noise in the same frequency band
Attachment #4 - Positive output as viewed on a fast (300 MHz) scope using a Tektronix x1 voltage probe.
Attachment #5 - Daughter board noise with measured ALS noise overlaid (the gain of x10 on the existing audio pre-amp has been divided out).
Given the overall good agreement between model and measurement, I am going to test this with the actual RF beat. For this test, we will need a differential receiving AA board to interface the output of the daughter board with the ADC input.
Next step is to actually make a prototype of this.
I thought a little bit about the next steps in testing the daughter board. The idea is to install this into the existing 1U chassis and tap the differential output from the FET Mixers as inputs to the daughter board. Looking at the D0902745 schematic, I think the best way to do this is to simply remove L3, L4, C10, C11, C15 and C16. I will then use the pads for L3 and L4 to pipe the differential output of the FET mixer to the differential input of the daughter board.
The daughter board takes care of whitening the ALS signal.
Then we need to pipe the differential output of the daughter board into the differential input of a differential receiving AA board. Koji and Johannes surveyed the available stockpile from the WB workshop. The best option seems to be to use the available v5 of D070081 and install 4 of them into a 1U chassis unit (also available from WB EE shop). The v5s can be upgraded to v6 by replacing the set of input and output buffer OpAmps with AD8622, as per the revision history notes. Koji ordered 100pcs of these today.
The input to the proposed 1U chassis housing these 8 AA boards (each with 8 channels) is a DB9 connector. The aLIGO demod board chassis that we use to demodulate the ALS signals has a nice DB25 output connector that supplies all the differential I and Q demodulated signals. But since we will install a daughter board, we will hae to hack together some connector solution anyways. I propose using a DB9 connector to pipe the outputs of the daughter board to the inputs of the AA board. Space is tight in the LSC rack, but I think we have space for a 1U chassis (see Attachment #3).
Finally - how to interface the AA board with the ADC? Koji and I discussed options, and seems like the least painful way will be to install a new ADC in the c1lsc expansion chassis in 1Y3. I checked the computer hardware cabinet and there seems to be 1 spare general standards 16bit ADC in there (see Attachment #1). Its health/providence is unknown. But Koji and I will test it after the meeting tomorrow. I also have another ADC card that Jamie and I removed from c1ioo sometime ago. I have labelled it as "GPIO0 LED RED", though I don't remember exactly what the problem was and can't find any elog about it. Incidentally, there are also 2 spare DAC cards available in the cabinet, although their health/rpovidence too is unknown. There are sufficient free slots in the c1lsc expansion chassis (see Attachment #2 though we will need a LIGO ADC adaptor card). Then we can just change the input ADC channels for the ALS signals in the c1lsc model.
In the short term, while the hardware for this plan is being put together, I can test the uncalibrated noise performance of the demod + daughter board combo (uncalibrated because I will make a measurement of voltage noise with an SR785 as opposed to frequency noise). A second daughter board will also need to be assembled - I'm just going to do it on another prototyping board as figuring out how to use Altium will probably take me longer. There is also the matter of fine tuning the polarization axes alignment of the input to the EX fiber coupler.
we did a bunch of tests to figure out the feasibility of the plan I outlined last night. Bottom line is: we appear to have a working 64 channel ADC (but with differential receiving that means 32 channels). But we need an aLIGO ADC adaptor card (I'm not sure of the DCC number but I think it is D0902006). See attached screenshot where we managed to add an ADC block to the IOP model on c1lsc, and it recognizes the additional ADC. The firmware on the (newly installed) working card is much newer than that on the existing card inside the expansion chassis (see Attachment #1).
Note that we have left the working ADC card inside the c1lsc expansion chassis. Plan is to give Rolf the faulty ADC card and at the same time ask him for a working adapter board.
Unrelated to this work: we have also scavenged 4 pcs of v2 of the differential receiving AA board from WB EE shop, along with a 1U chassis for the same. These are under my desk at the 40m for the moment. We will need to re-stuff these with appropriate OpAmps (and also maybe change some Rs and Cs) to make this board the same as v6, which is the version currently in use.
Since we decided to use the Acromag for readback of the temperature sensor for Kira's seismometer temperature control, I enabled logging of the channel Johannes had reserved for this purpose last week. Kira has made the physical connection of a temperature sensor to the BNC input for this channel - it reads back -2.92 V right now, which is around what I remember it being when Kira was doing her benchtop tests. I edited C0EDCU.ini to enable logging of this channel at 16 Hz. Presumably, a study of the ADC noise of the Acromag at low frequencies has to be made to ensure appropriate whitening (if any) can be added. Channel name is C1:PEM-SEIS_EX_TEMP_MON. Similarly, there is C1:PEM_SEIS_EX_TEMP_CTRL which is meant to be the control channel for the servoing. Calibration of the temperature sensor readback into temperature units remains. It also remains to be verified if we can have these slow EPICS channels integrated with a fast control model, or if the PID temperature control will be purely custom-script based as we have for the FSS slow loop.
I removed the fast channels I had setup temporarily in c1als. Recompilation and restart of the model went smoothly.
I spent today making another daughter board (so that we can use the new scheme for I and Q for one arm), testing it (i.e. measuring noise and TF and comparing to LISO model), and arranging all of this inside the 1U demod chassis. To accommodate everything inside, I decided to remove the 2 unused demod units from inside the box. I then drilled a few holes, installed the daughter boards on some standoffs, removed the capacitors and inductors as I outlined yesterday, and routed input and output signals to/from the daughter board. The outputs are routed to a D-sub on the rear panel. More details + better photo + results of testing the combined demod+daughter board signal chain tomorrow...
In Steve's absence, I've tried to keep an eye on the health of the vacuum system. From Attachment #1, the pressure of the main volume seems stable, no red flags there. I also don't here any anomalously loud sounds near the vacuum pumps. I've changed the N2 cylinders that keep V1 open twice, on Wednesday and Sunday of last week. So in summary, the vacuum system looks fine based on all the metrics I know of.
For testing the new IR ALS noise, we had decided that we would like to use the differential output of the demodulated ALS beat signal, as opposed to a single-ended output, as measurements suggested the former to be a lower noise configuration than the latter. For this purpose, Koji and I acquired a couple of old AA boards from the WB electronics shop. These are however, rev2 of the board, whereas the latest version is v6. The main difference between v2 and v6 is that (i) the THS4131 instrumentation amplifier has the Vocm pin grounded in v6 but is floating in v2 and (ii) the buffer opamps are AD8622 in v6 but are AD8672 in v2. But in fact, the boards we have are stuffed with AD8682.
I talked to Rich on Friday, and he seemed to think the AD8672 didn't have any issues noise-wise, the main reason they changed it was because its power consumption was high, and was causing overheating when several of these 1U chassis were packed closely together in an electronics rack. But the AD8682, which is what we have, has comparable power consumption to the AD8622. It is however a JFET opamp, and the voltage noise is a bit higher than the AD8622.
I am sure there is a way to LISO model a differential output opamp like the THS4131, but I thought I'd simulate the noise in LTSPICE instead. But I couldn't get that to work. So instead, I just measured the transfer function and noise of a single channel, for which Koji had expertly hacked together a custom shorting of the THS4131 Vocm pin to ground. Attachments #1 and #2 show the measurement. All looks good. Note that the phase is 180 at DC because I had hooked up the input signal opposite to what it should have been. The voltage noise of the differential outputs (each measured w.r.t. ground, with both inputs shorted to ground by a short patch cable) at 10 Hz is <100nV/rtHz, and the ADC noise is expected to be ~1uV/rtHz, so I think this is fine.
Conclusion: I think for the ALS test, we can just use the AA board in this config without worrying too much about replacing the buffer stage opamps, even though we've ordered 100pcs of AD8622.
Addendum 7 Mar 2018 11am: As per this document, the output noise of the AA board should be <75nV/rtHz from 10 Hz-50 kHz. So maybe the AD8682 noise is a little high after all. I've gotten the LTSpice model working now, will post the comparison of modelled output noise for various combinations here shortly.
I did a quick test of the noise of the new ALS electronics with the X arm ALS. Attachment #1 shows the results - but something looks off in the measurement, especially the "LO driven, RF terminated" trace. I will have to defer further testing to tomorrow. Of course the real test is to digitize these signals and look at the spectrum of the phase tracker output, but I wanted a voltage noise comparison first. Also, note that I have NOT undone the whitening TFs of (z,p) = (15,150) on these traces. I wonder if these noisy signals (particularly the 10Hz multiple harmonics) are an artefact of measurement, or if something is wonky in the daughter board circuits themselves. I am measuring these with the help of a DB9 breakout board and some pomona minigrabbers. Reagrdless, the sort of ripple seen in the olive green trace for the I channel wasn't present when I did the same test with RF signal generators out on the electronics workbench, so I am inclined to think that this isn't a problem with the circuit. I'm measuring with the SR785 with the "A" input setting, but with the ground set to "Float". I need to look into what the difference is between this mode, and the "A-B" mode. At first glance, both seem to be equivalent differential measurements, but I wonder if there is some subtlety w.r.t. pickup noise.
Perhaps I can repeat the test at the output of the AA board. I looked into whether there is a spare +/- 24V DC power supply available at the LSC rack, to power the 1U AA chassis, but didn't see anything there.
Here are the plots. Comments:
I like LTspice for such modeling - the GUI is nice to have (though I personally think that typing out a nodal file a la LISO is faster), and compared to LISO, I think that the LTspice infrastructure is a bit more versatile in terms of effects that can be modeled. We can also easily download SPICE models for OpAmps from manufacturers and simply add them to the library, rather than manually type out parameters in opamp.lib for LISO. But the version available for Mac is somewhat pared down in terms of the UI, so I had to struggle a bit to find the correct syntax for the various simulation commands. The format of the exported data is also not as amenable to python plotting as LISO output files, but i'm nitpicking...
I've gotten the LTSpice model working now, will post the comparison of modelled output noise for various combinations here shortly.
I am almost ready for a digital test of the new ALS electronics. Today, Koji and I spent some time tapping new +/-24VDC DIN terminal blocks at the LSC rack to facilitate the installation of the 1U differential receiving AA chassis (separate elog entry). The missing piece of the puzzle now is the timing adapter card. I opted against trying a test tonight as I am having some trouble bringing c1lsc back online.
Incidentally, a repeat of the voltage noise measurement of the X arm ALS beat looked much cleaner today, see Attachment #1 - I don't have a good hypothesis as to why sometimes the signal has several harmonics at 10Hz multiples, and sometimes it looks just as expected. The problem may be more systematically debuggable once the signals are being digitally acquired.
This required multiple hard reboots, but seems like all the RT models are back for now. The only indicator I can't explain is the red DC field on c1oaf. Also, the SUS model seems to be overclocking more frequently than usual, though I can't be sure. The "timing" field of this model's state word is RED, while the other models all seem fine. Not sure what could be going on.
Will debug further tomorrow, when I probably will have to do all this again as I'll need to recompile c1lsc for the ALS electronics test with the new ADC card from the differential AA board.
As I had found before, restarting the c1oaf model fixed the DC error. There is however still a pesky red indicator light on the "ADC0" in c1oaf. Trying to open up the ADC MEDM screen to investigate this further leads to the blank screen on the bottom right of Attachment #1. Probably has something to do with the fact that the model has an ADC block (because every model needs one?) but no signals are actually being piped to the model directly from the ADC.
Another observation, though I don't have any hypothesis as to why this was happening: on the c1sus machine, the c1sus model would frequently overclock, and then eventually, crash. I observed this behaviour at least 3 times between last night and now. The other models seemed fine though, in fact, IMC stayed locked. Why should this have been the case? It remains to be seen if this was somehow connected to the red DC indicator on c1oaf, though why should this be the case? Isn't the DC just concerned with writing data to frames? Any sort of IPC should be independent? Attachment #2 shows that there's been a definite increase in the maximum time on c1sus clock-cycle since yesterday (it's a 10 day minute trend plot of the model clock cycle timing and also the maximum time). Why? Koji and I did switch off all the Sorensens at the LSC rack for about 30mins, but why should this affect anything at 1X6? There are no red lights in either the c1lsc or c1sus expansion chassis. Curiously, the PRM also seems to be glitchy - as I'm sitting in the control room, I see a spot flashing across vertically on the REFL CRT monitor sporadically. Note that nominally, with PRM misaligned, the REFL CRT should be dark. dmesg on c1sus doesn't shed any light on the issue.
Seems like some high level voodoo .
Yesterday, we installed some new DIN rail connectors at the LSC rack to provide 3 new outputs each for +24V DC and -24V DC. The main motivation was to facilitate the installation and powering of the differential receiving AA board. The regulators used inside the 1U chassis actually claims a dropout voltage of 0.5V and outputs 14V nominally, so a +/-15V DC supply would've perhaps been sufficient, but we decided to leave a bit more margin, and unfortunately, there are no +/-18V DC KEPCO linear power supplies to the LSC rack. Procedure:
The c1lsc frontend models crashed for some reason during this procedure. Now the c1sus frontend model is also behaving weirdly. It is unclear to me if/how this work would have led to these problems, but the temporal correlation (but not causation?) is undeniable.
I was forced into a simultaneous power-cycle rebooting of the three vertex FEs just now. I took the opportunity to completely disconnect the c1sus expansion chassis from all power and then restart it.
Everything is back up right now, and the weird timing issues I noticed in the sus model seem to be gone now (I'll need a longer baseline to be sure and I'll post a trend of the CPU timing tomorrow). It's disconcerting that apparently the only way to get everything back up and running is the nuclear option of power-cycling all FE related electronics. I was considering borrowing an ADC adapter card from the Y end and measuring the calibrated IR ALS noise with the digital system, but if I'm going to have to go through this whole dance each time I do a model recompile on c1lsc (which I'm going to have to in order to get the extra ADC recognized), I'm wondering if it's just better to wait till we get the new adapter cards we ordered. I think I'm going to work on tuning the input coupling into the fiber at EX in the next couple of days instead.
I made a LISO fit of the measured TF of the daughter board, so that I can digitally invert the daughter board whitening. Results attached. (Inverse) Filters have been uploaded to the ALS X Foton filter banks.
Then I looked at the spectrum, see Attachment #1. Disappointingly, it looks like the arm PDH servo is dominating the noise, and NOT unsuppressed EX laser frequency noise,. Not sure why this is so, and I'm feeling too tired to debug this tonight. But encouragingly, the performance of the new ALS signal chain looks very promising. Once I tune up the X arm loop, I'm confident that the ALS noise will be at least as good as the reference trace.
I am leaving c1iscey shutdown until this is fixed. So ETMY is not available for the moment.
Random factoid: Trying to print a DTT trace with LaTeX in the label text on pianosa causes the DTT window to completely crash - so if you dont save the .xml file, you lose your measurement.
I was going to head out but then it occurred to me that I could do another simple test, which is to try and lock the X arm on ALS error signal (i.e. actuate on MC length to keep the beat between EX laser and PSL fixed, while the EX frequency is following the Xarm length). Comparing the in loop (i.e. ALS) error signal with the out-of-loop sensor (i.e. POX), it seems like POX is noisy. The curves were lined up by eye, by scaling the blue curve to match the red at the ~16Hz peaks. This supports my hypothesis in the previous elog. On the downside, could be anything. Electronics in the POX chain? The demod unit itself? Will look into it more tomorrow..
As an aside, controlling the arm with ALS error signal worked quite well, and the lock was maintained for ~1 hour.
Bulb went out ~10am today. Looks like the lifetime of this bulb was <100 days.
Steve: bulb is arriving next week
Bulb is replaced.
we tested my noisy POX hypothesis tonight. By locking the single arm with POX, the arm length is forced to follow PSL frequency, which is itself slaved to IMC length. From Attachment #1, there is no coherence between the arm control signal and MC_F. This suggests to me that the excess noise I am seeing in the arm control signal above 30 Hz is not originating from the PSL. It also seems unlikely that at >30Hz, anything mechanical is to blame. So I am sticking with the hypothesis that something is wonky with POX. For reference, a known "normal" arm control signal spectrum looks like the red curve in this elog.
Kevin suggested I shouldn't be so lazy and test the POY spectrum as well. So we moved the timing card back to c1iscey, went through the usual dance of vertex machine reboots, and then got both single arm locks going. Attached spectrum shows that both POX and POY are noisy. I'm not sure what has changed that could cause this effect. The fact that both POX and POY appear uniformly bad, but that there is no coherence with MC_F, suggests to me that perhaps this has something to do with the work I did with Koji w.r.t. the power situation at the LSC rack. But we just checked that
Another observation we made: note the huge bump around 70Hz in both arm control signals. We don't know what the cause of this is. But we occassionally noticed harmonics of this (i.e. 140, 210 Hz etc) appear in the control signal spectra, and they would grow with time - eventually, the X arm would lose lock (though the Y arm stayed locked).
I'm short on ideas for now so we will continue debugging tomorrow.
Unrelated to this work: Kevin reminded me that the high-pitched whine from the CRT TVs in the control room (which is apparently due to the flyback transformer) is DEAFENING. It's curious that the "chirp" to the eventual 15kHz whine is in opposite directions for the QUAD CRTs and the single display ones. Should be a Ph6 experiment maybe.
Update 2:30pm Mar 13: The furthest back I seem to be able to go in time with Frames is ~Jan 20 2018. Looking for a time when the arms were locked from back then, it seems like whatever is responsible for a noisy POX and POY was already a problem back in January. See Attachment #2. So it appears that the recent work at 1Y2 is not to blame...
The working hypothesis, since the excess noise in single arm locks is coherent between both arms, the excess sensing noise is frequency noise in the IMC locking loop (sensing because it doesn't show up in MC_F). I've started investigating the IMC sensing chain, starting with the power levels of the RF modulation source. Recall that we had changed the way the 29.5MHz signal was sent to the EOM and demod electronics in 2017. With the handheld RF power meter, I measured 13.2dBm coming out of the RF distribution box (this is routed straight from the Wenzel oscillator). This is amplified to 26dBm by an RF amplifier (ZHL-2-S) and sent to the EOM, with a coupled 16dBm part sent to a splitter that supplies the LO signal to the demod board and also the WFS boards. Lydia made a summary of expected RF power levels here, and I too seem to have labelled the "nominal" LO level to the MC_REFL demod board as +5dBm. But I measured 2.7dBm with the RF power meter. But looking closely at the schematic of the splitting circuitry, I think for a (measured) 16.7dBm input to it, we should in fact expect around 3dBm of output signal. So I don't know why I labelled the "nominal" signal level as 5dBm.
Bottom line: we are driving a level 17 mixer with more like +14dBm (a number inferred from this marked up schematic) of LO, which while isn't great, is unlikely to explain the excess noise I think (the conversion loss just degrades by ~1dB). So I will proceed to check further downstream in the signal chain.
It is not clear to me why installing an attenuator to prevent amplifier saturation has necessitated a 10dB increase in the IN1 gain and 3dB increase in the VCO gain. Initially, I was trying to compensate for the gain by increasing the FSS "Common Gain" but in that setting, I found an OLTF measurement impossible. The moment I enabled the excitation input to the CM board, the lock was blown, even with excitation amplitudes as small as -60dBm (from the Agilent network analyzer).
This may also be a good opportunity to test out one of the aLIGO style FET mixer demod boards (recall we have 2 spare from the 4 that were inside the ALS demod box). I'm going to ask Steve to package these into a 1U chassis so that I can try that setup out sometime. From a noise point of view, the aLIGO boards have the advantage of having a x100 preamp stage straight after the mixer+LPF. We may need to replace the lowpass filter though, I'm not sure if the one installed is 1.9MHz or 5MHz.
I've left an SR785 and AG4395 near 1X2 in anticipation of continuing this work tomorrow.
Unrelated to this work - seems like the WFS DC and RF offsets had not been set in a while so I reset these yesterday. The frequent model restarts in recent times may mean that we have to reset these to avoid using dated offset values.
Re-measured the demod board noises after replacing the suspect ERA-5SMs, with LO driven by a marconi at the "nominal" level of 2.5dBm, and RF input terminated. Attachment #1 is the input referred voltage noise spectra. I used the FET low noise pre-amp box for this purpose. I cannot explain the shape of the spectra above 1kHz. I tried doing the measurement on a minicircuits mixer (non-surface mount) and found the shape to be flat throughout the SR785 span. Unclear what else could be going on in the demod board though, all the other components on it are passive (except the ERA-5SMs which were replaced). I considered adopting a PMC style demod setup where we do the demod using some separate Minicircuits Mixer+LowPass filter combo. But the RF flashes for the IMC monitored at the RFmon port are ~0.2Vpp, and so the RF input to the mixer is expected to be ~2Vpp. The minicircuits mixer selection guide recommends choosing a diode mixer with LO level at least 10dBm above the expected RF input signal level, and we don't have any standalone mixers that are >Level 7. I've asked Steve to package the aLIGO demod board in the meantime, but even that might not be a plug and play replacement as the IF preamp stage has ~120degrees phase lag at 1MHz, which is significantly higher than the existing board which just has a SCLF5 low pass filter after the mixer and hence has <45degrees phase lag at 1MHz.
according to the temp sensor readout, which was ~-3.35V which corresponds to ~335K, the temperature of the can is now 60 deg C. This is a bit warm for my liking so i'm turning the heater current down to 0 now by writing 0 to C1:PEM-SEIS_EX_TEMP_CTRL
This elog by koji inspired me to consider power supply as a possible issue.
The demod board receives +/-24V DC (which is regulated down to +/-15V DC by 7815/7915), and also +15V DC via the backplane. The ERA-5SM receives DC power from the latter (unregulated) +15V DC. I can't think of why this is the case except perhaps the regulators can't source the current the amp wants? In any case, it doesn't look feasible to change this by cutting any traces on the PCB to me. While I had the board out, I decided to replace the JMS-1H mixers in a last ditch effort to improve the demod board noise. Unfortunately I'm having trouble de-soldering these MCL components from the board. So for now, I'm leaving the demod board out, IMC unlocked. Work will continue tomorrow.
After some persistence, I managed to get the mixers off.
Unfortunately, the coherent noise between the arms persists so the sensing noise injection must be happening elsewhere. IMC seems to lock fine though so I'm leving the autolocker on
As discussed at the meeting, I decided to calibrate the MC error point into physical units of Hz/rtHz (a.k.a. the PDH discriminant). This is to facilitate the debugging of the hypothesized excess IMC sensing noise. I did this as follows.
Using this, I can now make up a noise budget of sorts for the IMC sensing.
gautam 20180327 4.30pm: I re-checked the PDH error signal calibration using the oscilloscope method. Attachment #3 shows the PDH I and Q error signals and also the output of the RF monitor port, during a TEM00 flash. This attachment should be compared to Attachment #2 of elog 12822, and the answer lines up quite well. From my Finesse model of the IMC, I calculated that the x-axis of the PDH horn-to-horn is ~12.3kHz. Comparing to the top row of Attachment #3, I get a PDH error signal calibration of ~12.4kHz/Vrms, which lines up well with the number quoted above. So I trust my calibration, and hence, the y-axis of my noise budgets in reply to this elog.
I did a preliminary noise budget of the transmitted frequency noise of the IMC. Attachment #1 shows the NB. I'm going to use this opportunity to revisit my IMC modeling. Some notes:
Conclusion: From this study, assuming my PDH discriminant calibration was correct, looks like IMC demod / POX11 demod electronics noises are not to blame (this surprises me since there were apparently so many things wrong on the demod board, and yet that wasn't the worst thing in the IMC chain it would seem ). The POX11 photodiode "dark" noise is also not the problem I think, given the grey curve. Next curve to go on here is the demod board noise with the PSL shutter closed but the IMC REFL PD connected to the RF input (or maybe even better, have light on the PD, but macroscopically misalign MC2 so there is no 29.5MHz PDH signal), just to make sure there isn't anything funky going on there...
I've added two curves to the NB. Both are measured (with FET preamp) at the output of the demod board, with the LO driven at the nominal level by the Wenzel RF source pickoff (as it would be when the IMC is locked) and the RF input connected to the IMC REFL PD. For one curve, I simply closed the PSL shutter, while for the other, I left the PSL shutter open, but macroscopically misaligned MC2 so that there was no IMC cavity. So barring RFAM, there should be no PDH signal on the REFL PD, but I wanted to have light on there. I'm not sure if I understand the difference between these two curves though, need to think on it. Perhaps the IMC REFL PD's optical/electrical response needs to be characterized?
Next curve to go on here is the demod board noise with the PSL shutter closed but the IMC REFL PD connected to the RF input (or maybe even better, have light on the PD, but macroscopically misalign MC2 so there is no 29.5MHz PDH signal), just to make sure there isn't anything funky going on there...
we measured the RIN of the MC2 transmission using the PDA255 I had put on the MC2 trans table sometime ago for ringdowns. Attached are (i) spectra for the RIN, (ii) spectra for the classical rad. pressure noise assuming 500W circulating power and (iii) a tarball of data and code used to generate these plots.
We took a full span measurement (to make sure there aren't any funky high-freq features) and a measurement from DC-800 Hz (where we are looking for excess noise). The DC level of light on the photodiode was 2.76V (measured using o'scope)
I'll add this to the noise budget later. But the measured RIN seems consistent with a 2013 measurement at 100Hz (though the 2013 measurement is using DTT and so doesn't have high frequency information).
Todd informed me that the ADC Timing adaptor boards we had ordered arrived today. I had to solder on the components and connectors as per the schematic, though the main labor was in soldering the high density connectors. I then proceeded to shut down all models on c1lsc (and then the FE itself). Then classic problem of all vertex machines crashing when unloading models on c1lsc happened (actually Koji noticed that this was happening even on c1ioo). Anyways this was nothing new so I decided to push ahead.
I had to get a cable from Downs that connects the actual GS ADC card to this adaptor board. I powered off the expansion chassis, installed the adaptor board, connected it to the ADC card and restarted the expansion chassis and also the FE. I also reconnected the SCSI cable from the AA board to the adaptor card. It was a bit of a struggle to get all the models back up and running again, but everything eventually came back(after a few rounds of hard rebooting). I then edited the c1x04 and c1lsc simulink models to reflect the new path for the X arm ALS error signals. Seems to work alright.
At some point in the afternoon, I noticed a burning smell concentrated near the PSL table. Koji traced the smell down to the c1lsc expansion chassis. We immediately powered the chassis off. But Steve later informed me that he had already noticed an odd burning smell in the morning, before I had done any work at the LSC rack. Looking at the newly installed adaptor card, there wasn't any visual evidence of burning. So I decided to push ahead and try and reboot all models. Everything came back up normally eventually, see Attachment #1. Particle count in the lab seems a little higher than usual (actually, according to my midnight measurement, they are ~factor of 10 lower than Steve's 8am measurements), but Steve didn't seem to think we should read too much into this. Let's monitor the situation over the coming days, Steve should comment on the large variance seen in the particle counter output which seems to span 2 orders of magnitude depending on the time of the day the measurement is made... Also note that there is a BIO card in the C1LSC expansion chassis that is powered by a lab power supply unit. It draws 0 current, even though the label on it says otherwise. I a not sure if the observed current draw is in line with expectations.
The spare (unstuffed) adaptor cards we ordered, along with the necessary hardware to stuff them, are in the Digital FE hardware cabinet along the east arm.
Steve: particle count in the 40m is following outside count, wind direction, weather condition .....etc. The lab particle count is NOT logged ! This is bad practice.
While Kevin and Arijit were doing their MC_REFL PD noise measurements (which they will elog about separately shortly), I noticed a feature around 600kHz that reminded me of the NPRO noise eater feature. This is supposed to suppress the relaxation oscillation induced peak in the RIN of the PSL. Surprisingly, the noise eater switch on the NPRO front panel was set to "OFF". Is this the normal operating state? I thought we want the noise eater to be "ON"? Have to measure the RIN on the PSL table itself with one of the many available pick off PDs. In any case, as Attachment #1 showed, turning the noise eater back on did not improve the excess IMC frequency noise.
I've removed the MC REFL PD unit from the AS table for investigation. So MC won't lock.
PSL shutter was closed and location of PD was marked with sharpie (placing guides to indicate position wasn't convenient). I also kapton taped the PD to minimize dust settling on the PD while I have it out in the electronics area. Johannes has the camera, and my cellphone image probably isn't really high-res enough for diagnostics but I'm posting it here anyways for what it's worth. More importantly - the board is a D980454 revision B judging by the board, but there is no schematic for this revision on the DCC. The closest I can find is a D980454 Rev D. But I can already see several differences in the component layout (though not all of them may be important). Making a marked up schematic is going to be a pain . I'm also not sure what the specific make of the PD installed is.
The lid of the RF cage wasn't on.
More to follow tomorrow, PD is on the electronics workmench for now...
gautam 28 March 2018: Schematic has been found from secret Dale stash (which exists in addition to the secret Jay stash). It has also been added to the 40m electronics tree.
I re-installed the MC REFL photodiode. Centered beam on the PD by adjusting steering mirror to maximize the DC signal level (on o'scope) at the DC monitoring port. Curiously, the DC level on the scope (high-Z) was ~2.66V DC, whereas the MEDM screen reports ~twice that value, at ~5.44 "V". We may want to fix this "calibration" (or better yet, use physical units like mW). Noise-eater On/Off comparison of MC error signals to follow.
It's been a while - but today, all slow machines (with the exception of c1auxex) were un-telnetable. c1psl, c1iool0, c1susaux, c1iscaux1, c1iscaux2, c1aux and c1auxey were rebooted. Usual satellite box unplugging was done to avoid ITMX getting stuck.
I'm probably doing something stupid - but I've not been able to figure this out. In the MC1 and MC3 coil driver filter banks, we have a digital "28HzELP" filter module in FM9. Attachment #1 shows the MC1 filterbanks. In the shown configuration, I would expect the only difference between the "IN1" and "OUT" testpoints to be the transfer function of said ELP filter, after all, it is just a bunch of multiplications by filter coefficients. But yesterday looking at some DTT traces, their shapes looked suspicious. So today, I did the analysis entirely offline (motivation being to rule out DTT weirdness) using scipy's welch. Attachment #2 shows the ASDs of the IN1 and OUT testpoint data (collected for 30s, fft length is set to 2 seconds, and hanning window from scipy is used). I've also plotted the "expected" spectral shape, by loading the sos coefficients from the filter file and using scipy to compute the transfer function.
Clearly, there is a discrepancy for f>20Hz. Why?
Code used to generate this plot (and also a datafile to facilitate offline plotting) is attached in the tarball Attachment #3. Note that I am using a function from my Noise Budget repo to read in the Foton filter file...
*ChrisW suggested ruling out spectral leakage. I re-ran the script with (i) 180 seconds of data (ii) fft length of 15 seconds and (iii) blackman-harris window instead of Hanning. Attachment #4 shows similar discrepancy between expectation and measurement...
Seems like there was a 5.3 magnitude EQ ~10km from us (though I didn't feel it). All watchdogs were tripped so our mirrors definitely felt it. ITMX is stuck (but all the other optics are damping fine). I tried the usual jiggling of DC bias voltage but ITMX still seems stuck. Probably a good sign that the magnet hasn't come off, but not ideal that I can't shake it free..
edit: after a bit more vigorous shaking, ITMX was freed. I had to move the bias slider by +/-10,000 cts, whereas initially I was trying +/-2000 cts. There is a tendency for the optic to get stuck again once it has been freed (while the optic's free swinging motion damps out), so I had to keep an eye out and as soon as the optic was freed, I re-engaged the damping servos to damp out the optic motion quickly.
I found all the EPICS channels for the model c1ioo on the FE c1ioo to be blank just now. The realtime model itself seemed to be running fine, judging by the IMC alignment (as the WFS loops seemed to still be running okay). I couldn't find any useful info in demsg but I don't know what I'm looking for. So my guess is that somehow the EPICS process for that model died. Unclear why.
Kira informed me that she was having trouble accessing past data for her PID tuning tests. Looking at the last day of data, it looks like there are frequent EPICS data dropouts, each up to a few hours. Observations (see Attachment #1 for evidence):
It is difficult to diagnose how long this has been going on for, as once you start pulling longer stretches of data on dataviewer, any "data freezes" are washed out in the extent of data plotted.
While Kevin is working on the MC2 electronics chain - we disconnected the output to the optic (DB15 connector on coil driver board). I decided to look at the 'free' freeswinging MC2 OSEM shadow sensor data. Attachment #1 suggests that the suspension eigenmodes are showing up in the shadow sensors, but the 0.8Hz peak seems rather small, especially compared to the result presented in this elog.
Maybe I'll kick all 3 MC optics tonight and let them ringdown overnight, may not be a bad idea to checkup on the health of the MC suspensions/satellite boxes... [MC suspensions were kicked @1207113574]. PSL shutter will remain closed overnight...
Why is MC2 LR so different from the others???
I am working on IMC electronics. IMC is misaligned until further notice.
activities done today - details/plots/data/evidence tomorrow.
Today, I decided to check the power coupled into the PSL fiber for the BeatMouth. Surprisingly, it was only 200uW, while I had ~3.15mW going into it in January. Presumably some alignment drifting happened. So I re-aligned the beam into the fiber using the steering mirror immediately before the fiber coupler. I managed to get ~2.9mW in without much effort, and I figured this is sufficient for a first pass, so I didn't try too much more. I then tried making an ALS beat spectrum measurement (arm locked to IMC length using POX, green following the arm using end PDH servo). Surprisingly, the noise performannce was almost as good as the reference! See Attachment #1, in which the red curve is an IR beat (while all others are green beats). The Y arm green beat performance isn't stellar, but one problem at a time. Moreover, the kind of coherence structure between the arm error signal and the ALS beat signal that I reported here was totally absent today.
Upon further investigation, I found that the noise level was actually breathing quite significantly on timescales of minutes. While I was able to successfully keep the TEM00 mode of the PSL beam resonant inside the arm cavity by using the ALS beat frequency as an error signal and MC2 as a frequency actuator, the DC stability was very poor and TRX was wandering around by 50%. So my new hypothesis is that the excess ALS noise is because of one or more of
While I did some work in trying to align the PSL IR pickoff into the fiber along the fast (P-pol) axis, I haven't done anything for the X end pickoff beam. So perhaps the fluctuations in the EX IR power is causing beatnote amplitude fluctuations. In the delay line + phase tracker frequency discriminator, I think RF beatnote amplitude fluctuations can couple into phase noise linearly. For such an apparently important noise source, I can't remeber ever including it in any of the ALS noise budgets.
Before Ph237 today, I decided to use my polarization monitoring setup and check the "RIN" of power in the two polarizations coming out of the fiber on the PSL table. For this purpose, I decided to hijack the Acromag channels used for the PSL diagnostics connector Attachment #2 shows that there is fluctuations at the level of ~10% in the p-polarization. This is the "desired" polarization in that I aligned the PSL beam into the fiber to maximize the power in this polarization. So assuming the power fluctuations in the PSL beam are negligible, this translates to sqrt(10) ~3% fluctuation in the RF beat amplitude. This is at best a conservative estimate, as in reality, there is probably more AM because of the non PM fibers inside the beatmouth.
All of this still doesn't explain the coherence between the measured ALS noise and the arm error signal at 100s of Hz (which presumably can only happen via frequency noise in the PSL).
Another "mystery" - yesterday, while I was working on recovering the Y arm green beat signal on the PSL table, I eventually got a beat signal that was ~20mVpp into 50ohms, which is approximately the same as what I measured when the Y arm ALS performance was "nominal", more than a year ago. But while viewing the Y arm beats (green and IR) simultaneously on an o'scope, I wasn't able to keep both signals synchronised while triggering on one (even though the IR beat frequency was half the green beat frequency). This means there is a huge amount of relative phase noise between the green and IR beats. What (if anything) does this mean? The differential noise between these two beats should be (i) phase noise at the fiber coupler/ inside the fiber itself, and (ii) scatter noise in the green light transmitted through the cavity. Is it "expected" that the relative phase noise between these two signals is so large that we can't view both of them on a common trigger signal on an o'scope? Also - the green mode-matching into the Y arm is abysmal.
Anyways - I'm going to try and tweak the PER and mode-matching into the X end fiber a little and monitor the polarization stability (nothing too invasive for now, eventually, I want to install the new fiber couplers I acquired but for now I'll only change alignment into and rotation of the fiber coupler on the EX table). It would also be interesting to compare my "optimized" PSL drift to the unoptimized EX power drift. So the PSL diagnostic channels will not show any actual PSL diagnostic information until I plug it back in. But I suspect that the EPICS record names and physical channel wiring are wrong anyways - I hooked up my two photodiode signals into what I would believe is the "Diode 1 Power" and "Laser crystal temperature" monitors (as per the schematic), but the signals actually show up for me in "Diode 2 Power" (p-pol) and "Didoe 1 Temperature" (s-pol).
Annoyingly, there is no wiring diagram - on my todo list i guess...
@Steve - could you please take a photo of the EX table and update the wiki? I think the photo we have is a bit dated, the fiber coupler and transmon PDs aren't in it...
Attachment #1 shows the drift of the polarization content of the light from EX entering the BeatMouth. Seems rather large (~10%). I'm going to tweak the X end fiber coupling setup a bit to see if this can be improved. This performance is also a good benchmark to compare the PSL IR light polarization drift. I am going to ask Steve to order Thorlabs K6XS, which has a locking screw for the rotational DoF. With this feature, and by installing some HWPs at the input coupling point, we can ensure that we are coupling light into one of the special axes in a much more deterministic way.
THIS CALCULATION IS WRONG FOR THE OVERCOUPLED CAV.
Mode-matching efficiency of EX green light into the arm cavity is ~70*%, as measured using the visibility.
I wanted to get an estimate for the mode-matching of the EX green beam into the arm cavity. I did the following:
This amount of mode-matching is rather disappointing - using a la mode, the calculated mode-matching efficiency is nearly 100%, but 70% is a far cry from this. The fact that I can't improve this number by either tweaking the steering or by moving the MM lenses around suggests that the estimate of the target arm mode is probably incorrect (the non-gaussianity of the input beam itself is not quantified yet, but I don't believe this input beam can account for 30% mismatch). For the Y-arm, the green REFL DC level is actually higher when locked than when ITMY is misaligned. WTF?? Only explanation I can think of is that the PD is saturated when green is unlocked - but why does the ADC saturate at ~3000cts and not 32000?
This data is almost certainly bogus as the AA box at 1X9 is powered by +/-5VDC and not +/-15VDC. I didn't check but I believe the situation is the same at the Y-end.
3000 cts is ~1V into the ADC. I am going to change the supply voltage to this box (which also reads in ETMX OSEMS) to +/-15V so that we can use the full range of the ADC.
gautam Apr 26 630pm: I re-did the measurement by directly monitoring the REFLDC on a scope, and the situation is not much better. I calculate a MM of 70% into the arm. This is sensitive to the lens positions - while I was working on the EX fiber coupling, I had bumped the lens mounted on a translational stage on the EX table lightly, and I had to move that lens around today in order to recover the GTRX level of 0.5 that I am used to seeing (with arm aligned to maximize IR transmission). So there is definitely room for optimization here.