40m QIL Cryo_Lab CTN SUS_Lab TCS_Lab OMC_Lab CRIME_Lab FEA ENG_Labs OptContFac Mariner WBEEShop
  40m Log, Page 231 of 339  Not logged in ELOG logo
ID Date Authorup Type Category Subject
  15730   Thu Dec 10 22:45:42 2020 gautamUpdateSUSMore spare OSEMs

I acquired several spare OSEMs (in unknown condition) from Paco. They are stored alongside the shipment from UF.

  15731   Thu Dec 10 22:46:57 2020 gautamUpdateASCWFS head assembled

The assembly of the head is nearly complete, I thought I'd do some characterization before packaging everything up too nicely. I noticed that the tapped holes in the base are odd-sized. According to the official aLIGO drawing, these are supposed to be 4-40 tapped, but I find that something in between 2-56 and 4-40 is required - so it's a metric hole? Maybe we used some other DCC document to manufacture these parts - does anyone know the exact drawings used? In the meantime, the circuit is placed inside the enclosure with the back panel left open to allow some tuning of the trim caps. The front panel piece for mounting the SMA feedthroughs hasn't been delivered yet so hardware-wise, that's the last missing piece (apart from the aforementioned screws).

Attachment #1 - the circuit as stuffed for the RF frequencies of relevance to the 40m.

Attachment #2 - measured TF from the "Test Input" to Quadrant #1 "RF Hi" output.

  • There is reasonable agreement, but not sure what to make of the gain mismatch at most frequencies.
  • The photodiode itself hasn't been installed yet, so there will be some additional tuning required to account for the interaction with the photodiode's junction capacitance.
  • I noticed that the Qs of the resonances in between the notches is pretty high in this config, but the SPICE model also predicts this, so I'm hopeful that they will be tamed once the photodiode is installed.
  • One thing that is worrying is the feature at ~170 MHz. Could be some oscillation of the LM opamp. All the aLIGO WFS test procedure documentation shows measurements only out to 100 MHz. Should we consider increasing the gain of the preamp from x10 to x20 by swapping the feedback resistor from 453 ohms to 1 kohm? Is this a known issue at the sites
  • Any other comments?

Update 11 Dec: For whatever reason, whoever made this box decided to tap 4-40 holes from the bottom (i.e. on the side of the base plate), and didn't thread the holes all the way through, which is why I was unable to get a 4-40 screw in there. To be fair the drawing doesn't specify the depth of the 4-40 holes to be tapped. All the taps we have in the lab have a maximum thread length of 9/16" whereas we need something with at least 0.8" thread length. I'll ask Joe Benson at the physics workshop if he has something I can use, and if not, I'll just drill a counterbore on the bottom side and use the taps we have to go all the way through and hopefully that does the job.

The front panel I designed for the SMA feedthroughs arrived today. Unfortunately, it is impossible for the D-sub shaped holes in this box to accommodate 8 insulated SMA feedthroughs (2 per quadrant for RF low and RF high) - while the actual SMA connector doesn't occupy so much space, the plastic mold around the connector and the nut to hold it are much too bulky. For the AS WFS application, we will only need 4 so that will work, but if someone wants all 8 outputs (plus an optional 9th for the "Test Input"), a custom molded feedthrough will have to be designed. 

As for the 170 MHz feature - my open loop modeling in Spice doesn't suggest a lack of phase margin at that frequency so I'm not sure what the cause is there. If this is true, just increasing the gain won't solve the issue (since there is no instability at least by the phase margin metric). Could be a problem with the "Test Input" path I guess. I confirmed it is present in all 4 quadrants.

Attachment 1: aLIGO_wfs_v5_40m.pdf
aLIGO_wfs_v5_40m.pdf
Attachment 2: TF_meas.pdf
TF_meas.pdf
  15735   Tue Dec 15 12:38:41 2020 gautamUpdateElectronicsDC power strip

I installed a DC power strip (24 V variant, 12 outlets available) on the NW upright of the 1X1 rack. This is for the AS WFS. Seems to work, all outlets get +/- 24 V DC.

The FSS_RMTEMP channel is very noisy after this work. I'll look into it, but probably some Acromag grounding issue.

In the afternoon, Jordan and I also laid out 4x SMA LMR240 cables and 1x DB15 M/F cable from 1X2 to the NE corner of the AP table via the overhead cable trays.

  15736   Thu Dec 17 15:23:56 2020 gautamUpdateASCWFS head characterization

Summary:

I think the WFS head performs satisfactorily.

  • The (input-referred) dark noise level at the operating frequency of 55 MHz is ~40pA/rtHz (modelled) and ~60 pA/rtHz (measured, converted to input-referred). See Attachment #1. Attachment #5 has the input referred current noise spectral densities, and a few representative shot noise levels.
  • The RF transimpedance gain at the operating frequency is ~500 ohms when driving a 50 ohm load (in good agreement with LTspice model). See Attachment #2 and Attachment #3.
  • The resonant gain to notch ratios are all > 30 dB, which is in line with numbers I can find for the WFS installed at the sites (and in good agreement with the LTspice model as well).
  • There are a few lines visible in the noise measurement. But these are small enough not to be a show-stopper I think.

Details and remarks:

  1. Attachment #4 shows a photo of the setup. 
    • The QPD used was S/N #84.
    • The heat sinks have a bunch of washers because the screw holes were not tappe at time of manufacture.
    • There isn't space to have 8 SMA feedthroughs in the D-shaped cutouts, so we can only have the 4 "RF HI" outputs without some major metalwork.
    • C9 has been remvoed in all channels (to isolate the "TEST INPUT").
  2. I found that some quadrants displayed a ~35 MHz sine-wave of a few mV pk-pk when I had the back of the enclosure off (for tuning the notches). The hypothesis is that this was due to some kind of stray capacitance effect. Anyways, once I closed everything up, for the noise measurement, this peak was no longer visible. With an HP8447A preamp, I measured an RMS voltage of ~2mV rms on an oscilloscope. After undoing the 20 dB gain of the amplifier, each quadrant has an output voltage noise of ~200 uVrms (as returned by the "measure" utility on the scope, I don't know the specifics of how it computes this). Point is, there wasn't any clear sine-wave oscillations like I saw on two channels when the lid was off. 
  3. Some of the lines are present during some measurement times but not others (e.g. Q4 blue vs red curve in Attachment #1). I was doing this work in the elec-bench area of the lab, right next to the network switches etc so not exactly the quietest environment. But anyway, I don't see anything in these measurements that suggest something is seriously wrong.
  4. In the transfer function measurements, above 150 MHz, there are all sorts of features. But I think this is a measurement artefact (stray cable capacitance etc) and not anything real in the RF signal path. Koji saw similar effects I believe, and I didn't delve further into it.
  5. The dark noise of the circuit is such that to be shot noise limited, each quadrant needs 10 mA of DC photocurrent. The light bulb we have has a max current rating of 0.25A, with which I could only get 3 mA DC per quadrant. So the 55 MHz sideband power needed to be shot noise limited is ~50 mW - we will never have such high power. But I think to have better performance will need a major re-work of the circuit design (finite Qs of inductors, capacitors etc).
  6. Regarding the transimpedance gains - in my earlier plots, I omitted the 50ohm input impedance of the AG4395A network analyzer. The numbers I report here are ~half of those earlier in this thread for this reason. In any case, I think this number is what is important, since the ADT-1-1 on the demod board RF input has an input impedance of 50ohm. 
  7. Regarding grounding - the RF ground on the head is actually isolated from the case pretty well. Two locations of concern are (i) the heat sinks for the voltage regulator ICs and (ii) the DB15 connector shield. I've placed electrically insulating (but thermally conducting) pads from TO220 mounting kits between both sets of objects and the case. However, for the Dsub connector, the shape of the pad doesn't quite fit all the way round the connector. So if I over-tighten the 4-40 mounting bolts, at some point, the case gets shorted to the RF ground, presumably because the connector deforms slightly and touches the case in a spot where I don't have the isolating pad installed. I think I've realized a tightness that is mechanically satisfying but electrically isolating.
  8. I will do the fitting at my leisure but the eye-fit is already suggesting that things are okay I think.

If the RF experts see some red flags / think there are more tests that need to be performed, please let me know. Big thanks to Chub for patiently supporting this build effort, I'm pleasantly surprised it worked.

Attachment 1: oNoise.pdf
oNoise.pdf
Attachment 2: Z_Hi.pdf
Z_Hi.pdf
Attachment 3: Z_Low.pdf
Z_Low.pdf
Attachment 4: IMG_9030.jpg
IMG_9030.jpg
Attachment 5: iNoise.pdf
iNoise.pdf
  15737   Fri Dec 18 10:52:17 2020 gautamUpdateCDSRFM errors

As I was working on the IFO re-alignment just now, the rfm errors popped up again. I don't see any useful diagnostics on the web interface.

Do we want to take this opportunity to configure jumpers and set up the rogue master as Rolf suggested? Of course there's no guarantee that will fix anything, and may possibly make it impossible to recover the current state...

Attachment 1: RFMdiag.png
RFMdiag.png
  15741   Sat Dec 19 20:24:25 2020 gautamUpdateElectronicsWFS hardware install

I installed 4 chassis in the rack 1X2 (characterization on the E-bench was deemed satisfactory, I will upload the analysis later). I ran out of hardware to make power cables so only 2 of them are powered right now (1 32ch AA chassis and 1 WFS head interface). The current limit on the +24V Sorensens was raised to allow for similar margin to the limit with the increased current draw.

Remaining work:

  1. Make 2 more power cables for ISC whitening chassis and quad demod chassis.
  2. Make a 2x 4pin LEMO-->DB9 cable to digitize the FSS and PMC diagnostic channels with the new AA chassis. If RnD cables has a very short turnaround time, might be worth it to give this to them as well.
  3. Connect ADC1 on c1ioo machine to new AA chassis (transfer SCSI cable from existing AA unit to the new one). This will necessarily involve some model changes as well.
  4. Make a short cable to connect 55 MHz output from RFsource box to the LO input on the quad demod chassis.
  5. Install the WFS head on the AS table at a suitable location. Probably will need a focusing lens as well. 
  6. Connect WFS head to the signal processing electronics (the cables were already laid out by Jordan and I).
  7. Make the necessary CDS model changes (WFS filters, matrices, servos etc). I personally don't see the need for a new model but if anyone feels strongly about separating the IMC WFS and AS WFS we can set up another model.
  8. Commission the system.

While I definitely bumped various cables, I don't seem to have done any lasting damage to the CDS system (the RFM errors remain of course).

  15743   Mon Dec 21 18:18:03 2020 gautamUpdateCDSMany model changes

The CDS model change required to get the AS WFS signals into the RTCDS system are rather invasive.

  • We use VCS for these models. Linus Torvald may question my taste but I also made local backups of the models, just in case...
  • Particularly, the ADC1 card on c1ioo is completely reconfigured.
  • I also think it's more natural to do all the ASC computations on this machine rather than the c1lsc machine (it is currently done in the c1ass model). So there are many IPC changes as well.
  • I have documented everything carefully, and the compile/install went smoothly.
  • Taking down all the FE servers at 1830 local time
    1. To propagagte the model changes
    2. To make a hardware change in the c1rfm card in the c1ioo machine to configure it as "ROGUE MASTER 0"
    3. To clear the RFM errors we are currently suffering from will require a model reboot anyways.
  • Recovery was completed by 1930 - the RFM errors are also cleared, and now we have a "ROGUE MASTER 👾" on the network. Pretty smooth, no major issues with the CDS part of the procedure to report.
  • The main issue is that in the AA chassis I built, Ch14 (with the first channel as Ch1) has the output saturated to 28V (differential). I'm not sure what kind of overvoltage protection the ADC has - we frequently have the inputs exceed the spec'd +/-20 V (e.g. when the whitening filters are engaged and the cavity is fringing), but pending further investigation, I am removing the SCSI connection from the rear of the AA chassis.

In terms of computational load, the c1ioo model seems to be able to handle the extra load no issues - ~35us/60us per cycle. The RFM model shows no extra computational time

After this work, the IMC locking and POX/POY locking, and dither alignment servos are working okay. So I have some confidence that my invasive work hasn't completely destroyed everything. There is some hardware around the rear of 1X2 that I will clear tomorrow.

Attachment 1: CDSoverview.png
CDSoverview.png
  15744   Tue Dec 22 22:11:37 2020 gautamUpdateCDSAA filt repaired and reinstalled

Koji fixed the problematic channel - the issue was a bad solder joint on the input resistors to the THS4131. The board was re-installed. I also made a custom 2x4-pin LEMO-->DB9 cable, so we are now recording the PMC and FSS ERR/CTRL channel diagnostics again (spectra tomorrow). Note that Ch32 is recording some sort of DuoTone signal and so is not usable. This is due to a misconfiguration - ADC0 CH31 is the one which is supposed to be reserved for this timing signal, and not ADC1 as we currently have. When we swap the c1ioo hosts, we should fix this issue.

I also did most of the work to make the MEDM screens for the revised ASC topology, tried to mirror the site screens where possible. The overview screen remains to be done. I also loaded the anti-whitening filters (z:p 150:15) at the demodulated WFS input signal entry points. We don't have remote whitening switching capability at this time, so I'll test the switching manually at some point.

Quote:

The main issue is that in the AA chassis I built, Ch14 (with the first channel as Ch1) has the output saturated to 28V (differential). I'm not sure what kind of overvoltage protection the ADC has - we frequently have the inputs exceed the spec'd +/-20 V (e.g. when the whitening filters are engaged and the cavity is fringing), but pending further investigation, I am removing the SCSI connection from the rear of the AA chassis.

  15745   Wed Dec 23 10:13:08 2020 gautamUpdateCDSNear term upgrades

Summary:

  1. There appears to be insufficient number of PCIe slots in the new Supermicro servers that were bought for the BHD upgrade.
  2. Modulo a "riser card", we have all the parts in hand to put one of the end machines on the Dolphin network. If the Rogue Master doesn't improve the situation, we should consider installing a Dolphin card in the c1iscex server and connecting it to the Dolphin network at the next opportunity. 

Details:

Last night, I briefly spoke with Koji about the CDS upgrade plan. This is a follow up.

Each server needs a minimum of two peripheral devices added to the PCIe bus:

  • A PCIe interface card that connects the server to the Expansion Chassis (copper or optical fiber depending on distance between the two).
  • A Dolphin or RFM card that makes the IPC interconnects. 
  • I'm pretty certain the expansion chasiss cannot support the Dolphin / RFM card (it's only meant to be for ADCs/DACs/BIO cards). At least, all the existing servers in the 40m have at least 2 PCIe cards installed, and I think we have enough to worry about without trying to engineer a hack.
  • I attach some photos of new and old Supermicro servers to illustrate my point, see Attachment #1

As for the second issue, the main question is if we had an open PCIe slot on the c1iscex machine to install a Dolphin card. Looks like the 2 standard slots are taken (see Attachment #1), but a "low profile" slot is available. I can't find what the exact models of the Supermicro servers installed back in 2010 are, but maybe it's this one? It's a good match visually anyways. The manual says a "riser card" is required. I don't know if such a riser is already installed. 

Questions I have, Rolf is probably the best person to answer:

  1. Can we even use the specified host adaptor, HIB25-X4, which is "PCIe Gen2", with the "PCIe Gen3" slots on the new Supermicro servers we bought? Anyway, the fact that the new servers have only 1 PCIe slot probably makes this a moot point.
  2. Can we overcome this slot limitation by installing the Dolphin / RFM card in the expansion chassis?
  3. In the short run (i.e. if it is much faster than the full CDS shipment we are going to receive), can we get (from the CDS test stand in Downs or the site) 
    • A riser card so that we may install the Gen 1 Dolphin card (which we have in hand) in the c1iscex server?
    • A compatible (not sure what PCIe generation we need) PCIe host to ECA kit so we can test out the replacement for the Sun Microsystems c1ioo server?
    • A spare RFM card (VMIC 5565, also for the above purpose). 
  4. What sort of a test should we run to test the new Dolphin connection? Make a "null channel" differencing the same signal (e.g. TRX) sent via RFM and Dolphin? Or is there some better checksum diagnostic?
Attachment 1: IMG_0020.pdf
IMG_0020.pdf
  15746   Wed Dec 23 23:06:45 2020 gautamConfigurationCDSUpdated CDS upgrade plan
  1. The diagram should clearly show the host machines and the expansion chassis and the interconnects between them.
  2. We no longer have any Gentoo bootserver or diskless FEs.
  3. The "c1lsc" host is in 1X4 not 1Y3.
  4. The connection between c1lsc and Dolphin switch is copper not fiber. I don't know how many Gbps it is. But if the switch is 10 Gbps, are they really selling interface cables that have lower speed? The datasheet says 10 Gbps.
  5. The control room workstations - Debian10 (rossa) is the way forward I believe. it is true pianosa remains SL7 (and we should continue to keep it so until all other machines have been upgraded and tested on Debian 10).
  6. There is no "IOO/OAF". The host is called "c1ioo".
  7. The interconnect between Dolphin switch and c1ioo host is via fiber not copper.
  8. It'd be good to have an accurate diagram of the current situation as well (with the RFM network).
  9. I'm not sure if the 1Y1 rack can accommodate 2 FEs and 2 expansion chassis. Maybe if we clear everything else there out...
  10. There are 2 "2GB/s" Copper traces. I think the legend should make clear what's going on - i.e. which cables are ethernet (Cat 6? Cat 5? What's the speed limitation? The cable? Or the switch?), which are PCIe cables etc etc. 

I don't have omnigraffle - what about uploading the source doc in a format that the excellent (and free) draw.io can handle? I think we can do a much better job of making this diagram reflect reality. There should also be a corresponding diagram for the Acromag system (but that doesn't have to be tied to this task). Megatron (scripts machine) and nodus should be added to that diagram as well.

Please send me any omissions or corrections to the layout.

  15748   Wed Jan 6 15:28:04 2021 gautamUpdateVACVac rack UPS batteries replaced

[chub, gautam]

the replacement was done this afternoon. The red "Replace Battery" indicator is no longer on.

  15749   Wed Jan 6 16:18:38 2021 gautamUpdateOptical LeversBS Oplev glitchy

As part of the hunt why the X arm IR transmission RIN is anomalously high, I noticed that the BS Oplev Servo periodically kicks the optic around - the summary pages are the best illustration of this happening. Looking back in time, these seem to have started ~Nov 23 2020. The HeNe power output has been degrading, see Attachment #1, but this is not yet at the point where the head usually needs replacing. The RIN spectrum doesn't look anomalous to me, see Attachment #2 (the whitening situation for the quadrants is different for the BS and the TMs, which explains the HF noise). I also measured the loop UGFs (using swept sine) - seems funky, I can't get the same coherence now (live traces) between 10-30 Hz that I could before (reference traces) with the same drive amplitude, and the TF that I do measure has a weird flattening out at higher frequencies that I can't explain, see Attachment #3.

The excess RIN is almost exactly in the band that we expect our Oplevs to stabilize the angular motion of the optics in, so maybe needs more investigation - I will upload the loop suppression of the error point later. So far, I don't see any clean evidence of the BS Oplev HeNe being the culprit, so I'm a bit hesitant to just swap out the head...

Attachment 1: missingData.png
missingData.png
Attachment 2: OLRIN.pdf
OLRIN.pdf
Attachment 3: BS_OL_P.pdf
BS_OL_P.pdf
Attachment 4: BS_OL_suppression.pdf
BS_OL_suppression.pdf
  15750   Wed Jan 6 19:00:04 2021 gautamUpdateLSCPhase loss in POX/POY loops

I've noticed that there is some phase loss in the POX/POY locking loops - see Attachment #1, live traces are from a recent measurement while the references are from Nov 4 2018. Hard to imagine a true delay being responsible to cause so much phase loss at 100 Hz. Attachment #2 shows my best effort loop modeling, I think I've got all the pieces, but maybe I missed something (I assume the analog whitening / digital anti-whitening are perfectly balanced, anyway this wasn't messed with anytime recently)? The fitter wants to add 560 us (!) of delay, which is almost 10 clock cycles on the RTS, and even so, the fit is poor (I constrain the fitter to a maximum of 600 us delay so maybe this isn't the best diagnostic). Anyway, how can this change be explained? The recent works I can think of that could have affected the LSC sensing were (i) RF source box re-working, and (ii) vent. But I can't imagine how either of these would introduce phase loss in the LSC sensing. Note that the digital demod phase has been tuned to put all the PDH signal in the "I" quadrature, which is the condition in which the measurement was taken.

Probably this isn't gonna affect locking efforts (unless it's symptomatic of some other larger problem).

Attachment 1: POXloop.pdf
POXloop.pdf
Attachment 2: loopFit.pdf
loopFit.pdf
  15751   Wed Jan 6 22:47:41 2021 gautamUpdateALSNoisy ALS

Summary:

I want to get back to locking the interferometer so I can test out the newly installed AS WFS. However, the ALS noise is far too high, at least the transition of arm length control from IR to ALS fails reliably with the same settings that worked so reliably previously. I worked on investigating it a bit today.

Timeline

In the latter half of last year, I was focused on the air-BHD setup, so I wasn't checking in on the ALS noise as regularly. 

  1. On Aug 17, the noise was fine.
  2. But on Oct 29, the noise is bad (and it continues to remain so, to the point where I cannot lock the interferometer). 
  3. Koji and Anchal confirmed nothing was touched while they were investigating the ALS system, also on Oct 29. The spectra attached in #15650 don't make any sense to me, the noise at 100 Hz cannot be <100mHz/rtHz. So, inconclusive.

Excess noise:

All tests are done with the arm cavity length locked to the PSL frequency using POX. Then, the EX laser is locked to the arm cavity length using the AUX PDH servo. The fluctuations in the beatnote between the two lasers is what is monitored as a diagnostic. See Attachment #1. The reference traces in the top pane are from a known good time. The large excess noise between ~80-200 Hz is what I'm concerned about.

A separate issue that can improve the noise is to track down the noise in the 20-80 Hz band - probably some IMC frequency noise issue.

Noise budget:

See Attachment #2

  • I am pretty confident the electronics after the beat mouth are not to blame - I injected a 50 MHz signal from a Marconi and adjusted the signal amplitude to mimic what we get from the beat mouth. The trace labelled "DFD electronics noise" is the noise in this config.
  • The unsuppressed AUX frequency noise was measured with an SR785 (converted to freqnecy noise units knowing the PDH horn-to-horn voltage and the cavity linewidth). I didn't confirm the sensing noise level (dark noise of the AUX PDH loop), but I figure that at 100 Hz (voltage noise of ~100 uV/rtHz on the SR785), we are above the sensing noise level, and so are truly measuring the in-loop frequency noise of the stabilized AUX laser. I also confirmed that the loop UGF was ~10 kHz and phase margin was ~60 degrees, which are nominal numbers.
  • The fact that the excess noise is only in the X arm channel means the PSL frequency is not to blame.

So what could it be? The only things I can think of are (i) the beat mouth photodiode (NF1611) or (ii) excess noise in the fiber carrying the light from EX to the PSL table (but only on this fiber, and not on the EY fiber). Both seem remote to me - I'll test the former by switching the EX and EY fiber inputs to the beat mouth, but apart from this, I'm out of ideas... 

To avoid this kind of issue, we should really have scripted locks of all the basic IFO configs and record the data to summary pages or something - maybe something to do once Guardian is installed, it'd be pretty hacky to do cleanly with shell scripts.

Attachment 1: ALSX_excess.png
ALSX_excess.png
Attachment 2: budget.pdf
budget.pdf
  15752   Thu Jan 7 19:16:11 2021 gautamUpdateALSNoisy ALS

I'm also wondering why the error monitors for the X and Y loops report such wildly different spectra for the suppressed frequency noise of the AUX laser relative to the cavity length, see Attachment #1. The y-axis should be approximately Hz/rtHz. In both cases, the servo's error point monitor is connected to the DAQ system via a G=10 SR560. With the SR785, I measure for EX a nice bucket-shaped spectrum, bottoming out at ~10 uV/rtHz around 40 Hz, see Attachment #2. The SR560 should have an input-referred noise much less than this at the G=10 setting. The ADC noise level is only ~5 uV/rtHz, and indeed, the EY spectrum shows the expected shape. So what's up with the EX error mon? Tried swapping out the SR560 for a different unit, no change. And both the SR560 noise, and the ADC noise, check out when everything is checked individually. So some kind of interaction once everything is connected together, but it's only present at EX...

Today, I tried switching the EX and EY fibers going into the beat mouth, but I preserved the channel mapping after the beat mouth by switching the electrical outputs as well (the goal was to make sure that the beat photodiodes weren't the issue here, I think the electronics are already exonerated since driving the channel with a Marconi doesn't produce these noisy features). The EX spectrum remains noisy. I've switched everything back to the nominal configuration now to avoid further confusion. So it would appeat that this is real frequency noise that gets added in the EX fiber somehow. What can I do to fix this? The source of coupling isn't at the PSL table, else the EY channel would also show similar features. Visually, nothing seems wrong to me at EX either. So the problem is somehow in the cable tray along which the 40m of fiber is routed? This is already inside some nice foam/tubing setup, what can be done to improve it? Still doesn't explain why it suddenly became noisy...

Attachment 1: ALS_ERR_MON.pdf
ALS_ERR_MON.pdf
Attachment 2: AUXnoise.pdf
AUXnoise.pdf
  15754   Thu Jan 7 21:16:22 2021 gautamUpdateALSNoisy ALS

I thought about it, but wouldn't that show up at the AUX PDH error point? Or because the loop gain is so high there we wouldn't see a small excess? I suppose there could be some clipping on the Faraday or something like that. But the GTRX level and the green REFL DC level when locked are nominal.

Quote:

How about resurrecting the PSL table green beat for the X arm to see if the non-fiber setup shows the same level of the freq noise (e.g. the PDH locking became super noisy due to misalignment etc).

  15756   Fri Jan 8 20:01:11 2021 gautamUpdateALSNoisy ALS

I did this test today. The excess noise around 100 Hz doesn't show up in the green beat.

See Attachment #1. The setup was as usual:

  • X-Arm cavity length stabilized to PSL frequency using the POX locking loop.
  • EX laser frequency locked to the X-Arm cavity length using the AUX PDH loop.
  • The "BEATX" channel records frequency fluctuations in the beat sensed on the IR beat photodiode, while the "BEATY" channel records frequency fluctuations in the beat sensed on the Green beat photodiode.
  • Since the green beat frequency fluctuations are twice that of the IR beat frequency fluctuations, I scaled the former ASD by a factor of 0.5 so as to compare apples to apples.
  • At low frequencies, the green beat is noisier, but that channel doesn't show the excess noise at mid frequencies you see in the IR beat. So the AUX PDH sensing noise is not to blame I think.

So, this excess appears to truly be excess phase noise on the fiber (though I have no idea what the actual mechanicsm could be or what changed between Aug and Oct 2020 that could explain it. Maybe the HEPA?

For this work, I had to spend some time aligning the two green beams onto the beat photodiode. During this time, I shuttered the PSL, disabled feedback via the FSS servo, turned the HEPA high, and kept the EX green locked to the arm so as to have a somewhat stable beat signal I could maximize. Everything has been returned to nominal settings now (obviously, since I locked the arms to get the data).


You may ask, why do we care. In terms of RMS frequency noise, it would appear that this excess shouldn't matter. But in all my trials so far, I've been unable to transition control of the arm cavity lengths from POX/POY to ALS. I suppose we could try using the green beat, but that has excess low frequency noise (which was the whole point of the fiber coupled setup). 

Quote:

How about resurrecting the PSL table green beat for the X arm to see if the non-fiber setup shows the same level of the freq noise (e.g. the PDH locking became super noisy due to misalignment etc).

Attachment 1: ALSX_IR_green.pdf
ALSX_IR_green.pdf
  15761   Tue Jan 12 11:42:38 2021 gautamHowToCDSAcromag wiring investigation

Thanks for the systematic effort.

  1. Can you please post some time domain plots (ndscope perferably or StripTool) to clearly show the different failure modes?
  2. The majority of our AI channels are "Referenced Single Ended Source" in your terminology. At least on the c1psl Acromag crate, there are no channels that are truly differential drive (case #3 in your terminology). I think the point is that we saw noisy inputs when the IN- wasn't connected to RTN. e.g. the thorlabs photodiode has a BNC output with the shield connected to GND and the central conductor carrying a signal, and that channel was noisy when the RTN was unconnected. Is that consistent with your findings?
  3. What is the prescription when we have multiple power supplies (mixture of Sorensens in multiple racks, Thorlabs photodiodes and other devices powered by an AC/DC converter) involved?
  4. I'm still not entirely convinced of what the solution is, or that this is the whole picture. On 8 Jan, I disconnected (and then re-connected) the FSS RMTEMP sensor from the Acromag box, to check if the sensor output was noisy or if it was the Acromag. The problem surfaced on Dec 15, when I installed some new electronics in the rack (though none of them were connected to the Acromag directly, the only common point was the Sorensen DCPS. And between 8 Jan and today, the noise RMS has decreased back to the nominal level, without me doing anything to the grounding. How to reconcile this?
  15763   Thu Jan 14 11:46:20 2021 gautamUpdateCDSExpansion chassis from LHO

I picked the boxes up this morning. The inventory per Fil's email looks accurate. Some comments:

  1. They shipped the chassis and mounting parts (we should still get rails to mount these on, they're pretty heavy to just be supported on 4 rack nuts on the front). idk if we still need the two empty chassis that were requested from Rich.
  2. Regarding the fibers - one of the fibers is pre-2012. These are known to fail (according to Rolf). One of the two that LHO shipped is from 2012 (judging by S/N, I can't find an online lookup for the serial number), the other is 2011. IIRC, Rolf offered us some fibers so we may want to take him up on that. We may also be able to use copper cables if the distances b/w server and expansion chassis are short.
  3. The units are fitted with a +24V DC input power connector and not the AC power supplies that we have on all the rest of the chassis. This is probably just gonna be a matter of convenience, whether we want to stick to this scheme or revert to the AC input adaptor we have on all the other units. idk what the current draw will be from the Sorensen - I tested that the boards get power, and with noi ADCs/DACs/BIOs, the chassis draws ~1A (read off from DCPS display, not measured with a DMM). ~Half of this is for the cooling fans It seems like KT @ LLO has offered to ship AC power supplies so maybe we want to take them up on that offer.
  4. Without the host side OSSI PCIe card, timing interface board, and supermicro servers that actually have enough PCIe slots, we still can't actually run any meaningful test. I ran just a basic diagnostic that the chassis can be powered on, and the indicator LEDs and cooling fans run.
  5. Some photos of the contents are here. The units are stored along the east arm pending installation.

    >     Koji,
    >
    >     Barebones on this order.
    >
    >        1. Main PCIe board
    >        2. Backplane (Interface board)
    >        3. Power Board
    >        4. Fiber (One Stop) Interface Card, chassis side only
    >        5. Two One Stop Fibers
    >        6. No Timing Interface
    >        7. No Binary Cards.
    >        8. No ADC or DAC cards
    >
    >     Fil Clara
    >       
  15765   Thu Jan 14 12:32:28 2021 gautamUpdateCDSRogue master may be doing something good?

I think the "Rogue Master" setting on the RFM network may be doing some good. 5 mins, ago, all the CDS indicators were green, but I noticed an amber light on the c1rfm screen just now (amber = warning). Seems like at GPS time 1294691182, there was some kind of error on the RFM network. But the network hasn't gone down. I can clear the amber flag by running the global diag reset. Nevertheless, the upgrade of all RT systems to Dolphin should not be de-prioritized I think.

Attachment 1: Screenshot_2021-01-14_12-35-52.png
Screenshot_2021-01-14_12-35-52.png
  15767   Fri Jan 15 16:54:57 2021 gautamUpdateCDSExpansion chassis from LHO

Can you please provide a link to this "list of boards"? The only document I can find is T1800302. In that, under "Basic Requirements" (before considering specific motherboards), it is specified that the processor should be clocked @ >3GHz. The 3 new supermicros we have are clocked at 1.7 GHz. X10SRi-F boards are used according to that doc, but the processor is clocked at 3.6 or 3.2 GHz.

Please also confirm that there are no conflicts w.r.t. the generation of PCIe slots, and the interfaces (Dolphin, OSSI) we are planning to use - the new machines we have are "PCIe 2.0" (though i have no idea if this is the same as Gen 2). 

Quote:

The motherboard actually has six PCIe slots and is on the CDS list of boards known to be compatible.

As for the CX4 cable - I still think it's good to have these on hand. Not good to be in a situation later where FE and expansion chassis have to be in different racks, and the copper cable can't be used.

Attachment 1: Screenshot_2021-01-15_17-00-06.png
Screenshot_2021-01-15_17-00-06.png
  15768   Fri Jan 15 17:04:45 2021 gautamUpdateLSCMessed up LSC sensing

I want to lock the PRFPMI again (to commission AS WFS). Have had some success - but in doing characterization, I find that the REFL port sensing is completely messed up compared to what I had before. Specifically, MICH and PRCL DoFs have no separation in either the 1f or 3f photodiodes. 

  • A sensing line driven in PRCL doesn't show up in the AS55 photodiode signal - this is good and as expected.
  • For MICH - I set the MICH--->PRM actuation matrix element so as to minimize the height of the peak at the MICH drive frequency that shows up at the PRCL error point. My memory is that I used to be able to pretty much null this signal in the past, but I can't find a DTT spectrum in the elog as evidence. Anyways, the best effort nulling I can achieve now still results in a large peak at the PRCL error point. Since the sensing matrix doesn't actually make any sense, idk if it is meaningful to even try and calibrate the above qualitative statement into quantitative numbers of cross coupling in meters.
  • With the PRMI locked on 1f error signals (ETMs misaligned, PRCL sensed with REFL11_I, MICH sensed with AS55_Q) - I tried tweaking the digital demod phase of the REFL33 and REFL165 signals. But I find that the MICH and PRCL peaks move in unison as I tweak the demod phase. This suggests to me that both signals are arriving optically in phase at the photodiode, which is weird.
  • The phenomenon is seen also in the REFL11 signal.

I did make considerable changes to the RF source box, and so now the relative phase between the 11 MHz and 55 MHz signals is changed compared to what it was before. But do we really expect any effect even in the 1f signal? I am not able to reproduce this effect in simulation (Finesse), though I'm using a simplified model. I attach two sensing matrices to illustrate what i mean:

  1. Attachment #1 is in the PRFPMI state, with the IFO on RF control (CARM on REFL11, PRCL on REFL165_I, MICH on REFL165_Q, DARM on AS55_Q). 
  2. Attachment #2 is between the transition to RF control (CARM and DARM on ALS, PRCL on REFL165_I, MICH on REFL165_Q). The CARM offset is ~4nm (c.f. the linewidth of ~20pm), so the circulating power in the arm cavities is low.
Attachment 1: PRFPMI_Jan12sensMat.pdf
PRFPMI_Jan12sensMat.pdf
Attachment 2: PRMI3f_ALS_Jan11_largeOffsetsensMat.pdf
PRMI3f_ALS_Jan11_largeOffsetsensMat.pdf
  15769   Sat Jan 16 18:59:44 2021 gautamUpdateLSCModulation depth measurement

I decided to analyze the data I took in December more carefully to see if there are any clues about the weird LSC sensing.

Attachment #1 shows the measurement setup.

  • The PSL shutter was closed. All feedback to both lasers was disconnected during the measurement. I also disabled the input switch to the FSS Box - so the two laser beams being interfered shouldn't have any modulations on them other than the free running NPRO noise and the main IFO modulations.
  • Everything is done in fiber as I had the beams already coupled into collimators and this avoided having to optimize any mode matching on the beat photodiode.
  • The pickoff of the PSL is from the collimator placed after the triply resonant EOM that was installed for the air BHD experiment.
  • The other beam is the EX laser beam, arriving at the PSL table via the 40m long fiber from the end (this is the usual beam used for ALS).
  • I didn't characterize precisely the PLL loop shape. But basically, I wasn't able to increase the SR560 gain any more without breaking the PLL lock. Past experience suggests that the UGF is ~20 kHz, and I was able to get several averages on the AG4395 without the lock being disturbed.

Attachment #2 shows the measured spectrum with the PSL and EX laser frequency offset locked via PLL.

  • The various peaks are identified.
  • There are several peaks which I cannot explain - any hypothesis for what these might be? Some kind of Sorensen pollution? They aren't any multiples of any of the standard RF sources. They are also rather prominent (and stationary during the measurement time, which I think rules out the cause being some leakage light from the EY beam, which I had also left connected to the BeatMouth during the measurement).
  • In the previous such characterization done by Koji, such spurious extra peaks aren't seen.
  • Also, I can't really explain why some multiples of the main modulation are missing (could also be that my peak finding missed the tiny peaks)?
  • The measuremet setup is very similar to what he had - important differences are 
    • Much of the optical path was fiber coupled.
    • Beat photodiode is NF1611, which is higher BW than the PDA10CF.
    • The second laser source was the Innolight EX NPRO as opposed to the Lightwave that was used.
    • The RF source has been modified, so relative phasing between 11 MHz and 55 MHz is different.

Fitting the measured sideband powers (up to n=7, taking the average of the measured upper and lower sideband powers to compute a least squares fit if both are measured, else just that of the one sideband measured) agains those expected from a model, I get the following best fit parameters:

\begin{align*} \Gamma_1 &= 0.193 \pm 0.004 \\ \Gamma_2 &= 0.246 \pm 0.008 \\ \phi &= 75.5^{\circ +17.5^{\circ}}_{\, -40.3^{\circ}} \end{align*}

To be explicit, the residual at each datapoint was calculated as

\Delta = \bigg| \frac{\rm{model}-\rm{measurement}}{\rm{model}}\bigg|^2.

The numbers compare favourably with what Koji reported I think - the modulation depths are slightly increased, consistent with the RF power out of the RF box being slightly increased after I removed various attenuators etc. Note the large uncertainty on the relative phase between the two modulations - I think this is because there are relatively few sidebands (one example is n=3) which has a functional dependence that informs on phi - most of the others do not directly give us any information about this parameter (since we are just measuring powers, not the actual phase of the electric field). 

Attachment #3 shows a plot of the measured modulation profile, along with the expected heights plugging the best fit parameters into the model. The size of the datapoint markers is illustrative only - the dependence on the model parameters is complicated and the full covariance would need to be taken into account to put error bars on those markers, which I didn't do. 

Attachment #4 shows a time domain measurement of the relative phasing between the 11 MHz and 55 MHz signals at the EOM drive outputs on the RF source box. I fit a model there and get a value for the relative phase that is totally inconsistent from what I get with this fit. 

Attachment 1: PLL.pdf
PLL.pdf
Attachment 2: modDepth.pdf
modDepth.pdf
Attachment 3: modProfile.pdf
modProfile.pdf
Attachment 4: EOMpath_postMod.pdf
EOMpath_postMod.pdf
  15772   Tue Jan 19 15:43:24 2021 gautamConfigurationCDSUpdated CDS upgrade plan

Not sure if 1Y1 can accommodate both c1sus2 and c1bhd as well as the various electronics chassis that will have to be installed. There may need to be some distribution between 1Y1 and 1Y3. Does Koji's new wiring also specify which racks hold which chassis?

Some minor improvements to the diagram:

  1. The GPS receiver in 1X7 should be added. All the timing in the lab is synced to the 1pps from this.
  2. We should add hyperlinks to the various parts datasheets (e.g. Dolphin switch, RFM switch, etc etc) so that the diagram will be truly informative and self-contained.
  3. Megatron and nodus, but especially chiara (NFS server), should be added to the diagram. 
  15773   Wed Jan 20 10:13:06 2021 gautamUpdateElectronicsHV Power supply bypassing

Summary:

Installing 10uF bypass capacitors on the High Voltage power supply line for the HV coil driver circuit doesn't improve the noise. The excess bump around a few hundred Hz is still present. How do we want to proceed? 

Details

  • The setup is schematically shown in Attachment #1.
  • Physically, the capacitors were packaged into a box, see Attachment #2.
  • This box is installed between the HVPS and the 2U chassis in which the circuit is housed, see Attachment #3.
  • I measured the noise, (using the same setup as shown here to avoid exposing the SR785 input to any high voltage), for a variety of drive currents. To make a direct comparison, I took two sets of measurements today, one with the decoupling box installed and one without.
  • The results are shown in Attachment #4. You can see there is barely any difference between the two cases. I've also plotted the expected noise per a model, and the measured Johnson noise of one of the 25kohm resistors being used (Ohmite, wirewound). I just stuck the two legs of the resistor into the SR785 and measured the differential voltage noise. There is a slight excess in the measured Johnson noise compared to what we would expect from the Fluctuation Dissipation theorem, not sure if this is something to be worried about or if it's just some measurement artefact.

Discussion:

So what do we do about this circuit? For the production version, I can make room on the PCB to install two 10uF film capacitors on the board itself, though that's unlikely to help. I think we've established that 

  1. The excess noise is not due to the Acromag or the input Acromag noise filtering stage of the circuit, since the excess is present even when the input to the HV stage is isolated and shorted to ground.
  2. There was some evidence of coherence between the supply rails and the output of the HV stage (with input isolated and shorted to ground). The coherence had the "right shape" to explain the excess noise, but the maximum value was only ~0.1 (could have been because I was not measuring directly at the PA95's supply rail pins due to space constraints).
  3. The impedance of 10uF at 100Hz is ~150 ohms. idk what the impedance of the supply pins of the PA95 are at this frequency (this will determine the coupling of ripples in the HVPS output to the PA95 itself.

Do we have any better bipolar HV supply that I can use to see if that makes any difference? I don't want to use the WFS supplies as it's not very convenient for testing.


Not really related directly to this work but since we have been talking about current requirements, I attach the output of the current determining script as Attachment #5. For the most part, having 220ohm resistances on the new HAM-A coil driver boards will lead to ~half the DAC range being eaten up for the slow alignment bias. For things like MC1/MC3, this is fine. But for PRM/SRM/BS, we may need to use 100ohms. Chub has ordered all manner of resistances so we should have plenty of choices to pick from.

Attachment 1: bypassCaps.pdf
bypassCaps.pdf
Attachment 2: IMG_9079.jpg
IMG_9079.jpg
Attachment 3: IMG_9078.jpg
IMG_9078.jpg
Attachment 4: HVampNoise_driven_chassis.pdf
HVampNoise_driven_chassis.pdf
Attachment 5: printCoilCurrents.pdf
printCoilCurrents.pdf
  15775   Wed Jan 20 19:12:16 2021 gautamUpdateCDSSLwebview fixed

After fixing multiple issues, the model webviews are updating, should be done by tomorrow. It should be obvious from the timestamps on the index page which are the new ones. These new screens are better than the old ones and offer more info/details. Please look at them, and let me know if there are broken links etc. Once we are happy with this new webview, we can archive the old files and clean up the top directory a bit. I don't think this adds anything to the channel accounting effort but it's a nice thing to have up-to-date webviews, I found the LLO ones really useful in setting up the AS WFS model.

BTW, the crontab on megatron is set to run every day at 0844. The process of updating the models is pretty heavy because of whatever MATLAB overhead. Do we really need to have this run every day? I modified the crontab to run every other Saturday, and we can manually run the update when we modify a model. Considering this hasn't been working for ~3 years, I think this is fine, but if anyone has strong preference you can edit the crontab.

If someboy can volunteer to fix the MEDM screenshot that would be useful.

  15777   Tue Jan 26 10:58:30 2021 gautamUpdateSUSMC2 tickler stuck on

For whatever reason, the autolocker didn't turn the tickle off for several hours. Seems to work okay now. The linked plot suggests that the coil balancing on MC2 is pretty lousy.

  15782   Thu Jan 28 21:44:45 2021 gautamSummaryBHDHAM-A Coil Driver measurements After modifications

Looks fine to me visually but the verdict can only be made once the z:p locations are quantitatively confirmed, and the noise tests pass. It would be interesting to see what kind of time-domain transient (in N of force) switching on the de-whitening introduces, i guess best done interferometrically.

Quote:

I'll wait for comments until tomorrow to proceed with changes in the other board as well. I'll do noise measurements tomorrow.

  15783   Thu Jan 28 22:34:21 2021 gautamUpdateSUSDe-whitening

Summary:

  1. We will need de-whitening filters for the BHD relay optics in order to meet the displacement noise requirements set out in the DRD. I think these need not be remotely switchable (depends on specifics of LO phase control scheme). SR2, PR2 and PR3 can also have the same config, and probably MC1, MC3 as well.
  2. We will need de-whitening filters for the non test mass core IFO optics (PRM, SRM, BS, and probably MC2).
  3. I am pretty sure we will not be able to have sufficient DAC range for the latter class of optics if we have to:
    1. Supply the DC bias.
    2. Do the LSC and ASC actuation in the presence of reasonable sensing noise levels.
    3. Engage de-whitening to low-pass-filter the DAC noise at ~200 Hz.

Details:

Attachment #1 shows the DAC noise models for the General Standards 16-bit and 18-bit DACs we are expecting to have.

  • The 16-bit model has been validated by me at the 40m a few years ago.
  • We have never used the 18-bit flavor at the 40m, and there are all manner of quirks apparently related to zero crossings and such. So the noise may be up to x2 higher (we won't have as much freedom necessarily as the sites to bias the DAC on one side of the zero crossing if we also need to use the same DAC channel to supply the DC bias current for alignment.

Attachment #2 shows the expected actuation range for DC optic alignment, assuming we use the entire DAC range for this purpose.

  • Clearly, we need to do other things with the same DAC channels as well, so this is very much an upper bound of what will be possible.
  • Let's assume we will not go lower than 100ohms.
  • For all new optics we are suspending, we should aim to get the pitch balancing to within 500urad. With a 2x2m=4m optical lever arm, this corresponds to a 2mm spot shift. Should be doable.
  • This could turn out to be a serious problem for PRM, BS and SRM if we hope to measure squeezing - the <AUX DOF>-->DARM coupling could be at the level of -40dB, and at 200 Hz, the DAC noise would result in PRCL/MICH/SRCL noise at the level of ~10^-15m/rtHz, which would be 10^-17m/rtHz in DARM. I don't think we can get 20dB of feedforward cancellation at these frequencies. For demonstrating locking using a BHD error signal, maybe this is not a big deal.

Attachment #3 shows the current and proposed (by me, just a rough first pass, not optimized in any way yet) de-whitening filter shapes. These shapes can be tweaked for sure.

  • The existing de-whitening filter is way too aggressive. FWIW, the DRD "models" a "4th order Chebyshev low pass filter" which doesn't exist anywhere as far as I know.
  • Since the DAC noise is below 1 uV/rtHz at all frequencies of interest, we never need to have >60dB de-whitening anywhere as the input referred noise of any circuit we build will exceed 1 nV/rtHz.
  • I propose 3 poles, 3 zeros. In the plot, these poles are located at 30Hz, 50Hz, 2kHz, and the zeros are at 300 Hz, 300 Hz, 800 Hz. 
  • The de-whitening is less agressive below 100 Hz, where we still need significant LSC actuation ability. Considering the sensing noise levels at the 40m, I don't know if we can have reasonable LSC and ASC loop shapes and still have the de-whitening.
  • Once again, PRM, SRM and BS will be the most challenging.
  • For the BHD relay optics, once we have the de-whitening, we won't have the option of turning on a high-frequency (~kHz) dither line because of insufficient DAC range. 

Attachment #4 puts everything into displacement noise units. The electronics noise of the coil driver / de-whitening circuit have not been included so at high frequencies, the projection is better than what will actually be realizable, but still well below the BHD requirement of 3e-17 m/rtHz.

Attachment 1: DACnoiseModels.pdf
DACnoiseModels.pdf
Attachment 2: actuationRange.pdf
actuationRange.pdf
Attachment 3: deWhiteTFs.pdf
deWhiteTFs.pdf
Attachment 4: dispNoiseModels.pdf
dispNoiseModels.pdf
  15786   Mon Feb 1 12:30:21 2021 gautamUpdateElectronicsMore careful characterization

Summary:

  1. Swapping out the KEPCO HV supplies (linear) I was using for a pair of HP6209s I borrowed from Rich has improved the noise performance somewhat.
  2. However, there is still an excess relative to the model. I confirmed that this excess originates from the PA95 part of the circuit (see details).
  3. The bypass capacitors don't seem to have any effect on the measured ripple from these HP6209s. Maybe they're internally fitted with some 10uF or similar bypass caps?
  4. The production version of this board, with several improvements (after discussions with Koji and Rich), are on the DCC. They're being fabbed right now and will arrive in ~1 week for more bench testing. 

Power supply bypassing [updated 10pm]:

As mentioned earlier in this thread, I prepared a box with two 10uF, 1kV rated capacitors to bypass the high-voltage rails (see inset in the plot), to see if that improves the performance. However, in measuring the voltage ripple directly with the SR785 (no load connected), I don't see any significant difference whether the decoupling caps are connected or not, see Attachment #1. For this, and all other HV measurements made, I used this box to protect the SR785. One hypothesis is that this box itself is somehow introducting the excess noise, maybe because of leakage currents of the diode pair going into the 1Mohm SR785 input impedance, but I can't find any spec for this, and anyway, these diodes should be at ground potential once the transient has settled and the DC blocking capacitor has charged to its final value.

Note that the 10uF caps have an ESR of 7.2 mOhms. The HP6209 has a source impedance "<20mOhm" when operated as a CV source, per the datasheet. So perhaps this isn't so surprising? The same datasheet suggests the source impedance is 500 mOhms from 1kHz to 100 kHz, so we should see some improvement there, but I only measured out to 2 kHz, and I didn't take much effort to reduce these crazy peaks so maybe they are polluting the measurement out there. There must also be some continuous change of impedance, it cannot be <20 mOhm until 1 kHz and then suddenly increase to 500 mOhms. Anyways, for this particular circuit, the nosie DC-1kHz is what is important so I don't see a need to beat this horse more. 

Simplified circuit testing:

I decided to see if I can recover the spec'd voltage noise curve from the PA95 datasheet. For this, I configured the PA95 as a simple G=31 non-inverting amplifier (by not stuffing the 15 uF capacitor in the feedback path). Then, with the input grounded, I measured the output voltage noise on the circuit side of the 25kohm resistor (see inset in Attachment #2). To be consistent, I used the DC blocking box for this measurement as well, even though the output of the PA95 under these test conditions is 0V. Once again, there is considerable excess around ~100 Hz relative to a SPICE model. On the basis of this test, I think it is fair to say that the problem is with the PA95 itself. As far as I can tell, I am doing everything by the book, in terms of having gain > 10, using a sufficiently large compensaiton cap, HV rail decoupling etc etc. Note that the PA95 is a FET input opamp, so the effects of input current noise should be negligible. The datasheet doesn't provide the frequency dependence, but if this is just shot noise of the 1200 pA input bias current (for 300 V rails, per the spec), this is totally negligible, as confirmed by LTspice.

In the spirit of going step-by-step, I then added the feedback capacitor, and still, measured noise in excess of what I would expect from my model + SR785 measurement noise.

Integrated circuit testing:

After the above simplified test, I stuffed a full channel as designed, and tested the noise for various drive currents. To best simulate the operating conditions, an Acromag XT1541 was used to set the DC voltage that determines the drive current through the 25 kohm resistor. The measurements were made on the circuit side of this resistor (I connected a 20ohm resistor to ground to simulate the OSEM). As shown in Attachment #3, the noise with these HP6209 supplies is significantly better than what I saw with the KEPCO supplies, lending further credence to the hypothesis that insufficient PSRR is the root of the problem here. I've added subplots in a few different units - to be honest, I think that reaching this level of measured displacement noise at the 40m at 100 Hz would already be pretty impressive.

So what's next?

The main design change is that a passive R-C-R (4k-3uF-20k) replaces the single 25kohm resistor at the output of the PA95. 

  • This allows similar current drive range.
  • But adds an LPF to filter out the observerd excess noise at 100 Hz. 

Let's see if this fixes the issue. Not that I've also added a pair of input protection diodes to the input of the PA95 in the new design. The idea is that this would protect the (expensive) PA95 IC from, for example, the unit being powered with the +/- 18V rail but not the +/- 300 V rail. As I type this, however, I wonder if the leakage current noise of these diodes would be a problem. Once again, the datasheet doesn't provide any frequency dependence, but if it's just the shot noise of the 1nA expected when the diodes are not reverse biased (which is the case when the PA95 is operating normally since both inputs are at nearly the same potential), the level is ~20 fA/rtHz, comparable to the input current noise of the PA95, so not expected to be an issue. In the worst case, the PCB layout allows for this component to just be omitted. 

Attachment 1: HVPS.pdf
HVPS.pdf
Attachment 2: HV_testckt.pdf
HV_testckt.pdf
Attachment 3: totalNoise.pdf
totalNoise.pdf
  15792   Wed Feb 3 15:24:52 2021 gautamUpdateCDSCDS crash and CDS/IFO recovery

Didn't get a chance to comment during the meeting - This was almost certainly a coincidence. I have never had to do this - I assert, based on the ~10 labwide reboots I have had to do in the last two years, that whether the timing errors persist on reboot or not is not deterministic. But this is beyond my level of CDS knowledge and so I'm happy for Rolf / Jamie to comment. I use the reboot script - if that doesn't work, I use it again until the systems come back without any errors.

Quote:

This looked like the usual timing issue. It looked like "ntpdate" is not available in the new system. (When was it updated?)

The hardware clock (RTC) of these hosts are set to be PST while the functional end host showed UTC. So I copied the time of the UTC time from the end to the vertex machines.
For the time adjustment, the standard "date" command was used

> sudo date -s "2021-02-03 07:11:30"

This made the trick. Once IOP was restarted, the "DC" indicators returned to **Green**, restarting the other processes were straight forward and now the CDS indicators are all green.

I don't think this is a problem, the NTP synchronization is handled by timesyncd now.

Quote:

NTP synchronization is not active. Is this OK?

I defer restoring the LSC settings etc since I guess there is not expected to be any interferometer activity for a while.

  15795   Wed Feb 3 21:28:02 2021 gautamUpdateCDSCDS crash and CDS/IFO recovery

I am just reporting my experience - this may be a new failure mode but I don't think so. In the new RTCDS, the ntp server for the FEs are the FB, to which they are synced by timesyncd. The FB machine itself has the ntpd service installed, and so is capable of synching to an NTP server over the internet, but also serving as an NTP server for the FEs. The timesyncd daemon may not have started correctly, or the NTP serving from FB got interrupted (for example), but that's all just speculation.

  15798   Wed Feb 10 14:14:58 2021 gautamUpdateElectronicsCustom cables received

We received the custom cables to test the new suspension electronics. They are under my desk. So we are ready.

This batch was a small one - the company says that they can make molded cables if we have a minimum order, something to consider I gues.s.


Update 1900 11 Feb: I verified that the pin outs of the cables are as we intended (for one set of each type of cable). Because this was a small order, the connectors have metal shells, and so for cable #2 (sat box to flange), the two shells are shorted to each other. I can't verify if the shield is isolated from the shell on J5 without cutting open the cable. One thing that occurred to me is that we should give pins 5,8,11 on J4 and 16,20,24 on J5 (respectively) unique identifiers. They should only be shorted to GND on the circuit board itself. To be fixed for the next iteration. I uploaded some photos here.

I was unable to measure the capacitance of the cable using the LCR meter, and didn't opt to try any other method.

Attachment 1: satWiring.pdf
satWiring.pdf
  15800   Wed Feb 10 15:25:45 2021 gautamSummaryBHDSatellite Amplifier Output Offset measurements

Why not just do this test with the dummy suspension box and CDS system? I think Rich's claim was that the intrinsic LED RIN was dominant over any drive current noise but we can at least measure the quadrature sum of the two (which is after all the relevant quantity in terms of what performance we can realize) and compare to a model.

  15802   Wed Feb 10 21:14:03 2021 gautamUpdateElectronicsProduction version of the HV coil driver tested

Summary:

I did what I consider to be a comprehensive set of tests on the production version of the high voltage coil driver circuit. I think the performance is now satisfactory and the circuit is ready for the production build. Barring objections from anyone, I will ask Chub to place an order for components to stuff the 4 necessary units + 1 spare on Friday, 12 Feb (so that people have a full day to comment). A big thanks to Chub and the folks at JLCPCB for dealing with my incessant order requests and patiently supporting this build and letting me turn this around in 10 days - hopefully this is the end of this particular saga.

Schematic is here. All references to component designations are for v4 of the schematic.

Important design changes:

  1. All I/O to this board will be via D9 connectors. This will allow bypassing the high voltage stage in future suspensions while retaining the same cable config in the suspension drive, if that is desired. Some re-arrangement of the grouping of coils was also done for consistency with the planned upgrade.
  2. Differential receiving for the input signal from the Acromag. The nominal quad opamp is LT1125 but if we find noise issues (which I didn't), the OP497 has compatible PCB footprint.
  3. Added input protection dual diode D6 to protect the PA95 as recommended in the datasheet. This should protect the IC if (for example) the HV line isn't plugged in but the Acromag input is non-zero.
  4. Increased the feedback resistance from 30kohms to 12kohms. R16 through R21 are now 20k, while the old design had 5k. The purpose is to reduce the current demand in the feedback path, hopefully this opens up the number of DCPS we can use. To keep the overall gain of 31, the resistor R15 was upped from 1kohms to 4kohms.
  5. Feedback capacitance reduced from 15 uF to 3 uF. This was largely for space saving / ease of layout on the PCB. The resulting corner frequency is increased slightly from 0.35 Hz to 0.45 Hz but this doesn't have any imapct on the performance of the circuit at frequencies of interest (1/2/pi/R/C had R=30k, C=15uF, now R=120k, C=3uF).
  6. Added an R-C-R network at the output of the PA95, before the fast actuation signal is summed and sent to the OSEM coil.
    • This is probably the most important change, noise-performance wise.
    • The purpose of the network is to passively filter out the excess noise we saw at ~100 Hz (the corner from the 4kohm resistor + 3uF cap is at ~13 Hz, so factor of 10 filtering at 100 Hz, which was deemed sufficient, see earlier elogs in the thread). 
    • The Johnson noise contribution of the 20 kohm resistor is slightly higher than the original design which had a 25 kohm series resistor (but no R-C-R passive filter at the output of the PA95). But once again, this was deemed to have negligible effect on the performance at frequencies of interest to us.
    • The total current driving capability of the circuit is almost unchanged since the 20kohm + 4kohm nearly equals the old 25kohm resistance.
  7. Made the Vmon paths for monitoring the HV output of the PA95 differential sending, seems like a good practise to follow for all new designs.
  8. Added on-board bypass capacitors (2 x 10uF WIMA film caps) for cleaning up the HV supply noise.

Tests:

A series of tests were done. Note that only 1 channel was stuffed (I am out of PA95s), and the HP power supplies borrowed from Rich were used for the HV rails. For the +/-18V, a regular bench-top unit was used.

  1. Low voltage stage tests
    • TF of the differential receiving stage was measured and the overall unity gain and corner at 24kHz were verified, see Attachment #1.
    • With the input of the circuit grounded, I measured the noise of the circuit at various points (see legends on Attachment #2). I didn't bother to do a detailed verification against a SPICE model as the levels seemed roughly what is expected.
  2. Overall noise performance with HV stage enabled
    • For a range of drive currents, generated by applying the appropriate voltage using an Acromag XT1541 DAC module to the J1 connector, I measured the voltage on the circuit side of the 20 kohm resistor (by clipping onto the resistor leg. Note that the path to ground for the current was provided by connecting a 20 ohm resistor between pins 1 and 6 on J3a, and then grounding pin 6.
    • Results are shown in Attachment #3
    • For the drive currents at the edge of the range of operation, there is a small excess relative to lower drive currents. My best hypothesis for why this is happening is that the HV rail is too close to the requested output voltage (the HP units are rated for 320V, which is borderline if we want 300V at the output of the PA95). In any case, the R-C-R passive filter means that above ~60 Hz, there is excellent agreement between model and measurement.
  3. Time domain tests
    • Used a function generator. to drive a 50 mHz, 3Vpp sine wave to the "Bias Input" (=J1), and monitored (i) pickoff of drive signal, (ii) High voltage output at the circuit side of the 20kohm resistor, and (iii) the Vmon output (=pins 1/6 on J4), all on a 100 MHz Tektronix scope.
    • Results shown in Attachment #4. Once again, I see no red flags.
    • While I had the unit hooked up to the scope, I also checked the time domain signal with the scope set to 100 ns/div time base. I saw no evidence of any oscillatory features, either when no input signal was applied, or when a DC signal was provided (in which case the scope was set to AC coupling). 👍 
  4. Checked that the protection diodes at various locations in the circuit work.
  5. Checked the pin-mapping on all 6 D9 connectors is consistent with the top level diagram in the schematic.

PCB remarks:

As I was stuffing the board, I noticed a few improvements that can be made. Just noting these here for documentation purposes - these changes are mostly aesthetic and I personally see no need to order another set of PCBs.

  1. In some places, the silkscreen designators don't have the correct "orientation" relative to the component they are designating. I didn't find any serious ambiguity in terms of being misled to stuff the wrong component onto the wrong pads, but in the spirit of doing a professional job...
  2. The current limiting resistors on the +/-430V LEDs (R37/R38) have footprints for leaded components rather than SMT (which is what the +/-15V LEDs have).
  3. R45 and R46, the current limiting resistors for the rear panel power indicator LEDs, have 0805 footprint rather than 1206.
  4. When I drew up the PCB, R23, the 4kohm resistor in the R-C-R network, was set up as a 1W resistor. Let's say there can be 15 mA flowing through this resistor - the power dissipated is 15e-3 ^2 * 4e3 is 0.9W, which is uncomfortably close to the limit. For all the tests above, I used a 3W resistor, and didn't find any serious noise issues. The drilled holes are a little tight for this higher power rated resistor, but I don't think this is a showstopper.

Communications with Apex:

I've been talking to support at Apex, and pointed out that I couldn't match the SPICE model performance even for a simple non-inverting amplifier with the PA95. The feedback I got from them was that 

  1. They don't optimize the SPICE models for input noise and so it was a nice coincidence that model and measurement are somewhat close (but not exactly).
  2. They recommend the PA194, which is actually advertised as "low-noise". The PA95 is apparently not a "low-noise" part, with its 2uVrms input noise. 

Whiel the PA194 is compatible with our voltage and current requirements for this application, it is ~3x the cost, and seems like the R-C-R output filter allows us to realize the goal of 1pA/rtHz, so I'm inclined to stick with the PA95.

Production assembly:

I'd prefer to get as much of the board stuffed by Screaming Circuits as possible. It took me ~3 hours to stuff 1 channel + the power supply parts, standoffs etc. So I estimate it'll take me ~6 hours to stuff the entire board. So not the end of the world if we have to do it in-house.

Attachment 1: inputDiffRecTF.pdf
inputDiffRecTF.pdf
Attachment 2: LVnoises.pdf
LVnoises.pdf
Attachment 3: totalNoise.pdf
totalNoise.pdf
Attachment 4: timeDomainTests.pdf
timeDomainTests.pdf
  15805   Thu Feb 11 18:21:39 2021 gautamUpdateSUSMC suspension glitches

MC1 suspension is glitching again, so this is a good chance to install the new sat box and test it in the field.

  15809   Tue Feb 16 14:56:44 2021 gautamUpdateSUSaLIGO Sat Amp installed, powered

[jordan, gautam]

  • Ran 60ft long cables from 1X4 to MC1/MC3 chamber flange, via overhead cable tray, and top of PSL enclosure for the last ~20ft. Note that it may be that the overhead cable trays cannot support the weight of the cables for 15 SOSs (total 30 shielded cables with 37 wires as twisted pairs) when we eventually add the optics for the BHD upgrade.
  • Installed aLIGO satellite amplifer in 1X4.
  • Tapped +/-20 V (which is the available voltage closest to the required +/-18V). For this, the Sorensens were powered down, and the actual taps were made from the fusable blocks powering the Trillium interface box. We made sure to leave an extra slot so that this kind of additional headache is not required for the next person doing such work.
  • Once installed, I plugged in the dummy suspension box and verified that the unit performs as expected. 
  • Some photos of the installation are here.

After this work, the IMC locked fine, the AS camera has the Michelson fringing, the fast CDS indicators are all green, and the seismometer BLRMS all look good - therefore, I claim no lasting damage was done as a direct result of today's work at 1X4. I will connect up the actual suspension at my leisure later today. Note that the MC1 glitches seem to have gone away, without me doing anything about it. Nevertheless, I think it's about time that we start testing the new hardware. 


Unrelated to this work: while I was testing some characteristics of the MC1 suspension (before we did any work in the VEA, you can see the timestamp in the ndscope), I noticed that the MC1 UL coil channel cannot actually be used to actuate on the optic. The coil driver Vmon channel demonstrates the appropriate response, which means that the problem is either with the Satellite box (it is just a feedthrough, so PCB trace damaged?) or with the OSEM itself (more likely IMO, will know more once I connect the new Satellite Amplifier up). I only show comparison for UL vs UR, but I checked that the other coils seem to be able to actuate the optic. This means we have been running for an indeterminate amount of time with only 3 face actuators on MC1, probably related to me having to do this work


Also unrelated to this work - while poking around at 1X5 rear, I noticed that the power connections to the existing Satellite Boxes are (understatedly) flaky, see connections to T1-T4 in Attachment #2..

Attachment 1: MC1_deadUL.png
MC1_deadUL.png
Attachment 2: IMG_9100.jpg
IMG_9100.jpg
  15812   Wed Feb 17 13:59:35 2021 gautamUpdateDetCharSummary pages

The summary pages had failed because of a conda env change. We are dependent on detchar's conda environment setup to run the scripts on the cluster. However, for some reason, when they upgraded to python3.9, they removed the python3.7 env, which was the cause of the original failure of the summary pages a couple of weeks ago. Here is a list of thoughts on how the pipeline can be made better.

  1. The status checking is pretty hacky at the moment.
    • I recommend not using shell via python to check if any condor jobs are "held".
    • Better is to use the dedicated python bindings. I have used this to plot the job durations, and it has worked well.
    • One caveat is that sometimes, there is a long delay between making a request via a python command, and condor actually returning the status. So you may have to experiment with execution times and build some try/except logic to catch "failures" that are just the condor command timing out and not an actual failure of the summare jobs.
  2. The status check should also add a mailer which emails the 40m list when the job is held. 
    • With htcondor and python, I think it's easy to also get the "hold reason" for the job and add that to the mailer.
  3. The job execution time command is not executing correctly anymore - for whatever reason, the condor_history command can't seem to apply the constraint of finding only jobs run by "40m", although running it without the constraint reveals that these certainly exist. Probably has to do with some recent upgrade of condor version or something. This should be fixed.
  4. We should clear the archive files regularly. 
    • The 40m home directory on the cluster was getting full. 
    • The summary page jobs generate a .h5 archive of all the data used to generate the plots. Over ~1 year, this amounts to ~1TB.
    • I added the cleanArchive job to the crontab, but it should be checked.
    • Do we even need these archives beyond 1 day? I think they make the plotting faster by saving data already downloaded locally, but maybe we should have the cron delete all archive 
  5. Can we make our own copy of the conda env and not be dependent on detchar conda env? The downside is that if something dramatic changes in gwsumm, we are responsible for debugging ourselves.

Remember that all the files are to be edited on nodus and not on the cluster.

  15814   Wed Feb 17 16:11:53 2021 gautamUpdateSUSaLIGO Sat Amp installed, powered

There is some non-trivial sign flipping in the sensors/coils in this new setup because it is a hybrid one with the old interfacing electronics (D000210, D010001) and the new Satellite Amplifier (D080276). So I haven't yet gotten the damping working. I am leaving the PSL shutter closed and will keep working on this today/tomorrow. I have made various changes to the c1mcs realtime model and the c1susaux database record where MC1 is concerned. I have backups of the old ones so we can always go back to that if we so desire.

In the meantime, the PSL shutter is closed and there is no light to the IFO.


Update 1700: I've implemented some basic damping and now the IMC is now locked. The WFS loop runs away when I enable it, probably some kind of weird interaction with the (as of now untuned) MC1 local damping loops. I will write up a more detailed report later.


Update 2300: Did the following:

  1. Re-calibrated the cts2um filter in all SENSOR filter banks to account for the increased transimpedance and LED drive current. I judged the overall scaling to be x0.25 but this can be calibrated against the bounce peak height for example (it lines up pretty well).
  2. Re-measured the input matrix - it was very different from what was loaded. I am measuring this again overnight for some consistency.
  3. Re-tuned the damping gains. Now the optic damps well, and the loops seem file to me, both via broadband noise injection TF and by step response metrics.
  4. Yet, the WFS servo cannot be enabled. The WFS signal is summed in before the output matrix so I don't know why this would have a different behavior compared to the local damping, or indeed, why this has to be changed. Will need some (WFS) sensing/actuation matrix measurements to know more.

Dropping this for tonight, I'll continue tomorrow. Meanwhile, the OSEM input matrix measurement is being repeated overnight. PSL shutter is closed.

  15817   Thu Feb 18 15:33:21 2021 gautamUpdateSUSaLIGO Sat Amp installed, powered and commissioned

The WFS servo was recommissioned. The matrix can be tuned a bit more, but for now, I've recovered the old performance and the alignment doesn't seem to be running away, so I defer further tuning for later. The old Satellite box was handed over to Yehonathan for his characterization of the "spare" OSEMs.

This finishes the recovery of the MC1 suspension, I am now satisfied that the local damping loops are performing satisfactorily, that the WFS servo is also stable, and that POX/POY locking is recovered. On MC1, we even have 4 actuatable face OSEMs and the PIT(YAW) bias adjust slider even moves the optic in PIT(YAW), what a luxury. 

I've SDFed all the changes, and have backup of the old realtime model and C1SUSAUX_MC1 database files if we want to go back for whatever reason. The changes required to make this suspension work are different from what will eventually be required for the BHD suspensions (because of the hybrid iLIGO/aLIGO electronics situation), so I will not burden the readers with the tedious details.

  15818   Thu Feb 18 18:05:04 2021 gautamUpdateSUSaLIGO Sat Amp characterization

Before installation, I performed a bunch of tests on the aLIGO sat amp. All the measurements were made with the dummy suspension box substituting for an actual suspension. Here are the results.

Attachment #1: Transimpedance amplifier noises.

  • Measurement setup: J7 of the Satellite Amp goes to J9 on D1900068 front end (even though the connector is actually labelled "J3" on the box we have - maybe a versioning problem?). The outputs then go to a G=100 SR560 in AC coupled mode (the main purpose here was to block the large DC from the SR785, but I tacked on G=100 while I was at it).  
  • Top panel shows the raw measured voltages.
  • The bottom panel does a bunch of transformations:
    • Undoes the z:p = 3:30 Hz whitening on board the sat amp.
    • Undoes the G=100 gain of the SR560, and the AC coupling poles/zeros of SR560 and SR785.
    • Converts from voltage to current by dividing by the transimpedance gain, 242 kohms. 
  • Some model curves are shown for comparing to the measured spectra. It may be possible that we don't need to modify the nominal z:p = 0.4:10 Hz - I don't think the nominal seismic level will saturate the output even with the 0.4:10 Hz whitening, and it gives us even more clearance to the ADC noise (although we don't need it, we are gain limited at those frequencies, this is mostly a suggestion to reduce the workload).
  • The neon green curve is measured with the actual MC1 suspension plugged in, local damping enabled. It doesn't line up with the nosie floor of the bench tests, probably because the cts/um conversion factor could be off by some factor? Around 1 kHz, you can also see some broad peaks that are reminiscent of those seen in the MC_F spectrum after the c1psl Acromag upgrade. I hypothesize this is due to some poor grounding. Hopefully, once we get rid of the single-ended sending/receiving components in the suspension electronics chain, these will no longer be an issue.

Attachment #2: LED drive current source noises. I mainly wanted to check a claim by Rich in a meeting some time ago that the LED intensity fluctuations are dominated by inherent LED RIN, and not by RIN on the drive current. 

  • Measurement setup: a pair of pomona mini-grabbers was used to clip onto TP3. I found the voltage noise to be sufficiently high that no preamplification was required, and the DC level was <1V, so I just used the SR785 in AC coupled mode. 
  • The dummy suspension box was being driven while the measurement was being made (so the current source is loaded).
  • One channel (CH6) shows anomalously high nosie. I confirmed this was present even after the box was plugged in for ~1 day, so can't be due to any thermal / equilibriating transients.
  • I didn't check for consistency at the monitor testpoint, but that is exposed even with the MC1 suspension plugged in, so we can readily check. Anyways, from the corresponding photodiode curve in Attachment #1, it would seem that this excess RIN in the drive current has no measurable effect on the intensity fluctuations of the LED (the DC value of the paired PD is consistent with the others, ~6V DC). I must say I am surprised by this conclusion. I also checked for coherence between TP3 and the PD output using the SR785, and found none. 🤔 
  • Nevertheless, for the remaining channels, it is clear that the drive current is not shot noise limited for <1kHz. This isn't great. One possible reason is that the collector voltage to Q1 is unregulated (my modeling suggests only ~10dB rejection of collector voltage fluctuations at the output). I believe the current source designed by Luis for A+ makes some of these improvements and so maybe Rich was referring to that design, and not the aLIGO Satellite Amplifier flavor we are using. Anyways, this is just academic I think, the performance is the unit is fine for our purposes.

I will update with the MC1 suspension characterization (loop TFs, step responses etc) later.

Attachment 1: OSEMnoise.pdf
OSEMnoise.pdf
Attachment 2: LEDdriveNoise.pdf
LEDdriveNoise.pdf
  15822   Fri Feb 19 13:38:26 2021 gautamUpdateLSCPRFPMI

I forgot that I had already done some investigation into recovering the PRFPMI lock after my work on the RF source. I don't really have any ideas on how to explain (or more importantly, resolve) the poor seperation of MICH and PRCL sensed in our 3f (but also 1f) photodiodes, see full thread here. Anyone have any ideas? I don't think my analysis (=code) of the sensing matrix can be blamed - in DTT, just looking the spectra of the _ERR_DQ channels for the various photodiodes while a ssingle frequency line is driving the PRM/BS suspension, there is no digital demod phase that decouples the MICH/PRCL peak in any of the REFL port photodiode spectra.

  15825   Fri Feb 19 16:14:16 2021 gautamUpdateSUSCoM Range on 3"->2" Adapter Ring for SOS

I briefly talked with Jordan about this. This suspension will have OSEMs right? With 400ohm series resistance for the coil drivers, we will have ~+/-20mrad actuation range. Of course we'd like to use as much of this for interferometry and not static pitch alignment correction (possibly even increase the series resistance to relax the dewhitening requirements). But what is the target adjustability range in mrad with the dumbell/screw config? My target in the linked elog is 500urad (not any systematic optimum, but will allow us to use most of the DAC range for interferometry). Are these numbers in inches commensurate with this 500urad?

On a related note - are there grooves for the wires to sit in on the side of the sleeve? We looked at the solidworks drawing, and noticed that the groove doesn't extend all the way to the top of the clamp. Also, the material of both the clamping piece and the piece onto which the wire is pressed onto is SS. Don't we want them to be Aluminium (or something softer than the wire) so that the wire makes a groove when the clamp is tightened?

Quote:

We want to move the CoM with the adjustment range so that the residual deviation is adjusted by the bottom dumbbell. 0.0003" is well within the range and good enough.

  15828   Sat Feb 20 10:01:48 2021 gautamSummaryElectronicsA bunch of electronics received

Will we also be receiving the additional 34 Satellite Amplifier PCBs?

  15829   Sat Feb 20 16:20:33 2021 gautamUpdateGeneralHousekeeping + PRMI char

In prep to try some of these debugging steps, I did the following.

  1. ndscope updated from 0.7.9 to 0.11.3 on rossa. I've been testing/assisting the development for a few months now and am happy with it, and like the new features (e.g. PDF export). v0.7.9 is still available on the system so we can revert whenever we want.
  2. Arms locked on POX/POY, dither aligned to maximize TRX/TRY, normalization reset.
  3. PRMI locked, dither aligned to maximize POPDC.
  4. All vertex oplevs re-centered on their QPDs.

While working, I noticed that the annoying tip-tilt drift seems to be worse than it has been in the last few months. The IPPOS QPD is a good diagnostic to monitor stability of TT1/TT2. While trying to trend the data, I noticed that from ~31 Jan (Saturday night/Sunday morning local time), the IP-POS QPD segment data streams seem "frozen", see Attachment #1. This definitely predates the CDS crash on Feb 2. I confirmed that the beam was in fact incident on the IPPOS QPD, and at 1Y2/1Y3 that I was getting voltages going into the c1iscaux Acromag crate. All manner of soft reboots (eth1 network interface, modbusIOC service) didn't fix the problem, so I power cycled the acromag interface crate. This did the trick. I will take this opportunity to raise again the issue that we do not have a useful, reliable diagnsotic for the state of our Acromag systems. The problem seems to not have been with all the ADC cards inside the crate, as other slow ADC channels were reporting sensible numbers.

Anyways, now that the QPD is working again, you can see the drift in Attachment #2. I ran the dither alignment ~4 hours ago, and in the intervening time, the spot, which was previously centered on the AS camera CRT display, has almost drifted completely off (my rough calibration is that the spot has moved 5mm on the AS CCD camera). I was thinking we could try installing the two HAM-A coil drivers to control the TTs, this would allow us to rule out flaky electronics as the culprit, but I realize some custom cabling would be required, so maybe not worth the effort. The phenomenology of the drift make me suspect the electronics - hard for me to imagine that a mechanical creep would stop creeping after 3-4 hours? How would we explain the start of such a mechanical drift? On the other hand, the fact that the drift is almost solely in pitch lends support to the cause being mechanical. This would really hamper the locking efforts, the drift is on short enough timescales that I'd need to repeatedly go back and run the dither alignment between lock attempts - not the end of the world but costs ~5mins per lock attempt.


On to the actual tests: before testing the hardware, I locked the PRMI (no ETMs). In this configuration, I'm surprised to see that there is nearly perfect coherence between the MICH and PRCL error signals between 100Hz-1kHz 🤔 . When the AS55 demodulated signals are whitened prior to digitization (and then de-whitened digitally), the coherence structure changes. The electronics noise (measured with the PSL shutter closed) itself is uncorrelated (as it should be), and below the level of the two aforementioned spectra, so it is some actual signal I'm measuring there with the PRMI locked, and the coherence is on the light fields on the photodiode. So it would seem that I am just injecting a ton of AS55 sensing noise into the PRCL loop via the MICH->PRM LSC output matrix element. Weird. The light level on the AS55 photodiode has increased by ~2x after the September 2020 vent when we removed all the unused output optics and copper OMC. Nevertheless, the level isn't anywhere close to being high enough to saturate the ADC (confirmed by time domain signals in ndscope).

To get some insight into whether the whole RF system is messed up, I first locked the arm cavities with POX and POY as the error signals. Attachment #3 shows the spectra and coherence betweeen these two DoFs (and the dark noise levels for comparison). This is the kind of coherence profile I would expect - at frequencies where the loop gain isn't so high as to squish the cavity length noise (relative to laser frequency fluctuations), the coherence is high. Below 10 Hz, the coherence is lower than between 10-100 Hz because the OLG is high, and presumably, we are close to the sensing noise level. And above ~100 Hz, POX and POY photodiodes aren't sensing any actual relative frequency fluctuations between the arm length and laser frequency, so it's all just electronics noise, which should be incoherent.

The analogous plot for the PRMI lock is shown in Attachment #4. I guess this is telling me that the MICH sensing noise is getting injected into the PRCL error point between 100Hz-1kHz, where the REFL11 photodiode (=PRCL sensor) isn't dark noise limited, and so there is high coherence? I tuned the MICH-->PRM LSC output matrix element to minimize the height of a single frequency line driving the BS+PRM combo at ~313Hz in the PRCL error point. 

All the spectra are in-loop, the loop gain has not been undone to refer this to free-running noise. The OLGs themselves looked fine to me from the usual DTT swept sine measurements, with ~100 Hz UGF.

Attachment 1: IPPOSdeat.pdf
IPPOSdeat.pdf
Attachment 2: TTdrift.pdf
TTdrift.pdf
Attachment 3: POXnPOY.pdf
POXnPOY.pdf
Attachment 4: PRMI.pdf
PRMI.pdf
  15833   Mon Feb 22 14:53:47 2021 gautamUpdateCDSdtt-ezca-tools installed on rossa

The defaults cds-crtools didn't come with some of the older ezcautils (like ezcaread, ezcawrite etc). This is now packaged for debian, so I installled them with sudo apt update && sudo apt install dtt-ezca-tools on rossa. Now, we don't have to needlessly substitute the commands in our old shell scripts with the more modern z read, z write etc.

I am wondering if there is a relative implicit minus sign between the z servo and ezcaservo commands...

  15834   Tue Feb 23 00:10:05 2021 gautamUpdateGeneralDemod char part 1

I measured the conversion efficiencies for all the RFPD demod boards except the POP port ones. An RF source was used to drive the PD input on the demod board, one at a time, and the I/F outputs were monitored on a 300 MHz oscilloscope. The efficiency is measured as the percentage ratio V_IF / V_RF. 

I will upload the full report later, but basically, the numbers I measured today are within 10% of what I measured in 2017 when I previously did such a characterization. The orthogonality also seems fine. 

I believe I restored all the connections at 1Y2 correctly, and I can lock POX/POY and PRMI on 1f signals after my work. I will do the noise characterization tomorrow - but I think this test already rules out any funkiness with the demod setup (e.g. non orthogonality of the digitized "I" and "Q" signals). The whitening part of the analog chain remains untested.

Quote:

But I would still bet on demod chain funniness


Update 2/23 1215: I've broken up the results into the demod boards that do not (Attachment #1) and do (Attachment #2) have a D040179 preamp installed. Actually, the REFL11 AO path also has the preamp installed, but I forgot to capture the time domain data for those channels. The conversion efficiency inferred from the scope was ~5.23 V/V, which is in good agreement with what I measured a few years ago.

  • The scope traces were downloaded.
  • The resulting X/Y traces are fitted with ellipses to judge the gain imbalance and orthogonality.
  • The parameter phi is the rotation of the "bounding box" for the fitted ellipses - if the I and Q channels are exactly orthogonal, this should be either 0 or 90 degrees. There is significant deviation from these numbers for some of the demodulators, do we want to do something about this? Anyways, the REFL11 and AS55 boards, which are used for PRMI locking, report reasonable values. But REFL165 shows an ellipse with significant rotation. This is probably how the CDS phase rotator should be tuned, by fitting an ellipse to the digitized I/Q data and then making the bounding box rotation angle 0 by adjusting the "Measured Diff" parameter.
  • The gain imbalance seems okay across the board, better than 1dB.
  • The POX and POY traces are a bit weird, looks like there is some non-trivial amount of distortion from the expected pure sinusoid.
  • I measured the LO input levels going into each demod board - they all lie in the range 2-3dBm (measured with RF power meter), which is what is to be expected per the design doc. The exception the the 165 MHz LO line, which was 0.4 dBm. So this board probably needs some work. 
  • As I mentioned earlier, the conversion efficiencies are consistent with what I measured in 2017. I didn't break out the Eurocards using an extender and directly probe the LO levels at various points, but the fact that the conversion efficiencies have not degraded and the values are consistent with the insertion loss of various components in the chain make me believe the problem lies elsewhere. 

For completeness, I will measure the input terminated I/F output noise levels later today. Note also that my characterization of the optical modulation profile did not reveal anything obviously wrong (to me at least). 

Attachment 1: noPreamp.pdf
noPreamp.pdf
Attachment 2: withPreamp.pdf
withPreamp.pdf
  15839   Wed Feb 24 11:53:24 2021 gautamUpdateGeneralDemod char part 2

I measured the noise of the I/F outputs of all the LSC demodulators. I made the measurement in two conditions, one with the RF input to the demodulators terminated with 50 ohms to ground, and the other with the RFPD plugged in, but the PSL shutter closed (so the PD dark noise was the input to the demodulator). The LO input was driven at the nominal level for all measurements (2-3 dBm going in to the LO input, measured with the RF power meter, but I don't know what the level reaching the mixer is, because there is a complicated chain of ERA amplifiers and attenuators that determine what the level is). 

As in the previous elog, I have grouped the results into boards that do not (Attachment #1) and do (Attachment #2) have the low noise preamp installed. The top row is for the "Input terminated" measurements, while the bottom is with the RFPD plugged in, but dark. I think not a single board shows the "expected" noise performance for both I and Q channels. In the case where the preamp isn't installed and assuming the mixer is being driven with >17dBm LO, we expect the mixer to demodulate the Johnson noise of 50 ohms, which would be ~1nV/rtHz, and so with the SR785, we shouldn't measure anything in exceess of the instrument noise floor. With the low noise preamp installed, the expected output noise level is ~10nV/rtHz, which should just about be measurable (I didn't use any additional Low Noise front end preamp for these measurements). The AS55_I channel shows noise consistent with what was measured in 2017 after it was repaired, but the Q channel shows ~twice the noise. It seemed odd to me that the Q channels show consistently higher noise levels in general, but I confirmed that the SR785 channel 2 did not show elevated instrument noise at least when terminated with 50 ohms, so seems like a real thing.

While this is clearly not an ideal state of operation, I don't see how this can explain the odd PRMI sensing.

Quote:

For completeness, I will measure the input terminated I/F output noise levels later today. Note also that my characterization of the optical modulation profile did not reveal anything obviously wrong (to me at least). 

Attachment 1: noises_noPreamp.pdf
noises_noPreamp.pdf
Attachment 2: noises_withPreamp.pdf
noises_withPreamp.pdf
  15840   Wed Feb 24 12:11:08 2021 gautamUpdateGeneralDemod char part 3

I did the characterization discussed at the meeting today.

  1. RF signal at 100 Hz offset from the LO frequency was injected into the PD input on the demod boards.
  2. The digitized CDS channels were monitored. I chose to look at the C1:LSC-{PD}_I_OUT and C1:LSC-{PD}_Q_OUT channels. This undoes the effect of the analog whitening, but is before the digital phase rotation.
  3. Attachments #1 and Attachments #2 are for the case where the analog whitening is not engaged, white Attachments #3 and Attachments #4 are for when the whitening is engaged, and they look the same (as they should), which rules out any crazy mismatch between the analog filter and the digital dewhitening filter.
  4. I have absorbed the flat whitening gain applied to the various PDs in the cts/V calibration indicated on these plots. So the size of the ellipse is proportional to the conversion gain.

I think this test doesn't suggest anything funky in the analog demod/whitening/AA/digitization chain. We can repeat this process after the demod boards are repaired and use the angle of rotation of the ellipse to set the "D" parameter in the CDS phase rotator part, I didn't do it today.

Attachment 1: noPreamp.pdf
noPreamp.pdf
Attachment 2: withPreamp.pdf
withPreamp.pdf
Attachment 3: noPreamp_whitened.pdf
noPreamp_whitened.pdf
Attachment 4: withPreamp_whitened.pdf
withPreamp_whitened.pdf
  15841   Wed Feb 24 12:29:18 2021 gautamUpdateGeneralInput pointing recovered

While working at the LSC rack, I lost the input pointing into the IFO (the TT wiring situation is apparently very fragile, and this observation supports the hypothesis that the drifting TTs are symptomatic of some electronics issue). After careful beam walking, it was recovered and the dither alignment system was used to maximize TRX/TRY once again. No lasting damage done. If I can figure out what the pin-mapping is for the TT coils in vacuum, I'm inclined to try installing the two HAM-A coil drivers to control the TTs. Does anyone know where I can find said pin-out? The wiki page links seem broken and there isn't a schematic available there...

Ok it should be possible to back it out from the BOSEM pin out, and the mapping of the in-vacuum quadrupus cable, though careful accounting of mirroring will have to be done... The HAM-A coil driver actually already has a 15 pin output like the iLIGO coil drivers that are currently in use, but the pin mapping is different so we can't just replace the unit. On the bright side, this will clear up 6U of rack space in 1Y2. In fact, we can also consider hooking up the shadow sensor part of the BOSEMs if we plan to install 2 HAM-A coil drivers + 1 Dual satellite amplifier combo (I'm not sure if this number of spares is available in what we ordered from Todd).

ELOG V3.1.3-