There were many locklosses from the point where the arm powers were somewhat stabilized. Attachments #1 and #2 show two individual locklosses. I think what is happening here is that the BS seismometer X channel is glitching, and creating a transient in the angular feedforward filter that blows the lock. The POP QPD based feedback loop cannot suppress this transient, apparently. For now, I get around this problem by boosting the POP QPD feedback loop a bit, and then turning the feedforward filters off. The fact that the other seismometer channels don't report any transient makes me think the problem is either with the seismometer itself, or the readout electronics. The seismometer masses were recently recentered, so I'm leaning towards the latter.
I didn't explicitly check the data, but I am reasonably certain the same effect is responsible for many PRMI locklosses even with the arms held off resonance (though the tolerance to excursions there is higher). Pity really, the feedforward filters were a big help in the lock acquisition...
While the vacuum system was knocked out, I measured the RF transimpedance (using the AM laser setup, didn't do the shot noise intercept current measurement for now) of all the RFPDs (except PMC REFL). At the very least, the following photodiodes are suspect:
For the remaining photodiodes, I measure a transimpedance that is within ~20% of what is on the wiki page. The notches may benefit from some retuning. While I have the data, I will fit this and post a more complete report on the wiki.
Update July 6 1145am: WFS response plots now have legends mapping quadrants, and I've also added the response of a spare PDA10CF (which is now the new POP22/POP110 photodiode).
A more comprehensive report has been uploaded here. I'll zip the data files and add them there too. In summary:
I'll upload the data and analysis notebook + liso fit files to the wiki as well shortly. The data, a Jupyter notebook making the plots, and the LISO fit files have been uploaded here.
I didn't do it this time but it'd be nice to also do the noise measurement and get an estimate for the shot-noise intercept current.
While I have the data, I will fit this and post a more complete report on the wiki.
This problem reared its ugly head again. I am inclined to believe the problem is electronic and not on the light, since the POY channels seem immune to this issue (see Attachment #1). I will investigate in the daytime tomorrow. Note that while the POX photodiode head has ~twice the transimpedance than POY (per measurement), the POY signal gets amplified by a ZHL-500-HLN amplifier before heading to the demod electronics (nominal gain is 19dB = x9). There is also some imbalance in the light level at the photodiodes I guess, because overall, the PDH fringe is ~twice as large for the Y arm as the X arm. Basically, the y-axes of the attached plot cannot be directly compared between POX and POY.
Mostly this is an annoyance - right now, the POX signal is only used for locking and dither aligning the X arm cavity, and so once that is done, the locking can proceed (as long as the other channels, e.g. REFL11, aren't glitching as well...)
in the lab, checkin on the WFS
Sun Jul 5 18:25:50 2020
I redid Gautam's measurements to get a baseline before changing the head, and my results are very different: To me it looks like the WFS2 quadrants are all OK.
I've left the setup as is in case either me or Gautam want to double check. If we're agreed on this response, I'll remove the notches and disable the RF attenuators.
Sun Jul 5 21:42:45 2020
After some hunting, I found this old SURF report with the WFS head measurements. The y-axes don't make much sense to me, and I can't find the actual data anywhere (her wiki page doesn't actually exist). So I think it's still unknown if these heads ever had the advertised transimpedance gain, or if the measured transimpedance of ~1kohm was what it always was.
Koji and I had a discussion last Friday about the suspension electronics. I think there are still a few open questions - see Attachment #1. We should probably make a decision on these soon.
Other useful links:
To facilitate this investigation, I've DQed the 4 face coil outputs for the two ETMs. EX is currently running with 5 times the series resistance of EY, so it'll be a nice consistency check. Compilation, installation etc went smooth. But when restarting the c1scx model, there was a weird issue - the foton file, C1SCX.txt, got completely wiped (all filter coefficients were empty, even though the filter module names themselves existed). I just copied the chiara backup version, restarted the model, and all was well again.
This corresponds to 8 additional channels, recorded at 16k as float 32 numbers, so in the worst case (neglecting any clever compression algorithms), we are using disk space at a rate of ~4 MB/s more. Seems okay, but anyway, I will remove these DQ channels in a few days, once we're happy we have enough info to inform the coil driver design.
spoke too soon - there was an RFM error for the TRX channel, and restarting that model on c1sus took down all the vertex FEs. Anyways, now, things are back to normal I think. The remaining red light in c1lsc is from the DNN model not running - I forgot to remove those channels, this would've been a good chance! Anyways, given that there is an MLTI in construction, I'm removing these channels from the c1lsc model, so the next time we restart, the changes will be propagated.
For whatever reason, my usual locking scripts aren't able to get me to the PRFPMI locked state - some EPICS channel value must not have been set correctly after the model reboot 😞. I'll debug in the coming days.
Fun times lie ahead for getting the new BHD FEs installed I guess 🤡 ....
Looking at signals to the ETMs from the current lock acquisition sequence, the RMS current to a single coil is approximately _____ (to be filled in later).
So we may need a version of the fast coil driver that supports a low noise mode (with large series resistance) and a high-range mode (with lower series resistance for lock acquisition).
Looking at the signals to the test mass coils, it seems borderline to me that we will be able to acquire lock and run in a low noise configuration with the same series resistor in the coil driver circuit. The way I see it, options are:
I only looked at the ETMs for this study. The assumption is that we will have no length actuation on the ITMs, only local damping and Oplev loops (and maybe some ASC actuation?), which can be sufficiently low-pass filtered such that even with coil de-whitening, we won't have any range issues.
Attachment #1 shows the time-domain traces of the coil driver signals as we transition from POX/POY lock to the ALS lock. There are some transients, but I think we will be able to hold the lock even with a 5 kohm resistor (~twice what is on ETMX right now). From just these numbers, it would seem we can even go up to 10 kohms right away and still be able to acquire lock, especially if we re-design the digital feedback loop to have better low-pass filtering of the high-frequency ALS noise, see the next attachment.
Attachment #2 shows the f-domain picture, once the arm lengths are fully under ALS control (~25 seconds onwards in Attachment #1). The RMS is dominated by high frequency ALS length loop noise, which we can possibly improve with better design of the digital control loop.
Finally, Attachment #3 shows the situation once DARM control has been transitioned over to AS55_Q. Note that the vertex DoFs are still under 3f control, so there is the possibility that we can make this even lower noise. However, one thing that is not factored in here is that we will have to de-whiten these signals to low-pass filter the DAC noise (unless there is some demonstrated clever technique with noise-mons or something to subtract the DAC noise digitally). Nevertheless, it seems like we can run safely with 5 kohms on each ETM coil and still only use ~2000 cts RMS, which is ~1/10th the DAC range (to allow for dealing with spurious transients etc).
See Attachments #1 and #2. We don't have any Q3000 QPDs in hand, at least not in the photodiode box stored in the clean optics cabinet at the south end. I also checked a cabinet along the east arm where we store some photodiodes - but didn't find any there either. The only QPDs we have in hand are the YAG-444-4AH, which I believe is what is used in the iLIGO WFS heads.
So how many do we want to get?
The "source" output of the SR785 has a DC offset of -6.66 V. I couldn't make this up.
Upshot is, this SR785 is basically not usable for TF measurements. I was using the unit to characterize the newly stuffed ISC whitening board. The initial set of measurements were sensible, and at some point, I started getting garbage data. Unclear what the cause of this is. AFAIK, we don't have any knob to tune the offset - adjusting the "offset" in the source menu, I can change the level of the offset, but only by ~1 V even if I apply an offset of 10 V. I also tried connecting the ground connection on the rear of the SR785 to the bench power supply ground, no change.
Do we have to send this in for repair?
Grrr. Let's repair the unit. Let's get a help from Chub & Jordan.
Do you have a second unit in the lab to survive for a while?
When I tested Q3000 for aLIGO, the failure rate was pretty high. Let's get 10pcs.
A single channel of this board was stuffed (and other channels partially populated). The basic tests passed, and nothing exploded! Even though this is a laughably simple circuit, it's nice that it works.
HV power supplies:
A pair of unused KEPCO BHK300-130 switching power supplies that I found in the lab were used for this test. I pulled the programmable cards out at the rear, and shorted the positive output of one unit to the negative of the other (with both shorted to the supply grounds as well), thereby creating a bipolar supply from these unipolar models. For the purposes of this test, I set the voltage and current limits to 100V DC, 10mA respectively. I didn't ramp up the supply voltage to the rated 300 V maximum. The setup is shown in Attachment #1.
Need to think more about how to better characterize this noise. An estimate of the required actuation can be found here.
A more careful analysis has revealed some stability problems. I see oscillations at frequencies ranging from ~600kHz to ~1.5 MHz, depending on the voltage output requested, of ~2 V pp at the high-voltage output in a variety of different conditions (see details). My best guess for why this is happening is insufficient phase margin in the open-loop gain of the PA95 high voltage amplification stage, which causes oscillations to show up in the closed loop. I think we can fix the problem by using a larger compensation capacitor, but if anyone has a better suggestion, I'm happy to consider it.
The changes I wanted to make to the measurement posted earlier in this thread were: (i) to measure the noise with a load resistor of 20 ohms (~OSEM coil resistance) connected, instead of the unloaded config previously used, and (ii) measure the voltage noise on the circuit side (= TP5 on the schematic) with some high voltage output being requested. The point was to simulate conditions closer to what this board will eventually be used in, when it has to meet the requirement of <1pA/rtHz current noise at 100 Hz. The voltage divider formed by the 25 kohm series resistor and the 20 ohm OSEM coil simulated resistance makes it hopeless to measure this level of voltage noise using the SR785. On the other hand, the high voltage would destroy the SR785 (rated for 30 V max input). So I made a little Pomona box to alllow me to do this measurement, see Attachment #1. Its transfer function was measured, and I confirmed that the DC high voltage was indeed blocked (using a Fluke DMM) and that the output of this box never exceeded ~1V, as dictated by the pair of diodes - all seemed okay .
Next, I wanted to measure the voltage noise with ~10mA current flowing through the output path - I don't expect to require more than this amount of current for our test masses. However, I noticed some strange features in the spectrum (viewed continuously on the SR785 using exponential averaging setting). Closer investigation using an oscilloscope revealed:
Some literature review suggested that the capacitor in the feedback path, C4 on the schematic, could be causing problems. Specifically, I think that having that capacitor in the feeddback path necessitates the use of a larger compensation capacitor than the nominal 33pF value (which itself is higher than the 4.7pF recommended on the datasheet, based on experience of the ESD driver circuit which this is based on, oscillations were seen there too but the topology is a bit different). As a first test of this idea, I removed the feedback capacitor, C4 - this seemed to do the trick, the oscillations vanished and I was able to drive the output between the high voltage supply rails. However, we cannot operate in this configuration because we need to roll off the noise gain for the input voltage noise of the PA95 (~6 nV/rtHz at 100 Hz will become ~200 nV/rtHz, which I confirmed using the SR785). Using a passive RC filter at the output of the PA95 (a.k.a. a "snubber" network) is not an option because we need to sum in the fast actuation path voltage at the output of the 25 kohm resistor.
Some modeling confirms this hypothesis, see Attachment #2. The quantity plotted is the open-loop gain of the PA95 portion of the circuit. If the phase is 0 degrees, then the system goes unstable.
So my plan is to get some 470pF capacitors and test this idea out, unless anyone has better suggestions? I guess usually the OpAmps are compensated to be unconditionally stable, but in this case maybe the power op-amp is more volatile?
Attachment #1 is a summary of the current to each coil on the suspensions. The situation is actually a little worse than I remembered - several coils are currently drawing in excess of 10mA. However, most of this is due to a YAW correction, which can be fixed somewhat more easily than a PIT correction. So I think the circuit with a gain of 31 for an input range of +/-10 V, which gives us the ability to drive ~12mA per coil through a 25kohm series resistor, will still provide sufficient actuation range. As far as the HV supplies go, we will want something that can do +/- 350 V. Then the current to the coils will at most be ~50 mA per optic. The feedback path will require roughly the same current. The quiescent draw of each PA95 is ~10mA. So per SOS suspension, we will need ~150mA.
If it turns out that we need to get more current through the 25kohm series resistance, we may have to raise the voltage gain of the circuit. Reducing the series resistance isn't a good option as the whole point of the circuit is to be limited by the Johnson noise of the series resistance. Looking at these numbers, the only suspension on which we would be able to plug in a HV coil driver as is (without a vent to correct for YAW misalignment) is ITMY.
Update 2 Sep 2020 2100: I confirmed today that the number reported in the EPICS channel, and the voltage across the series resistor, do indeed match up. The test was done on the MC3 coil driver as it was exposed and I didn't need to disable any suspensions. I used a Fluke DMM to measure the voltage across the resistor. So there is no sneaky factor of 2 as far as the Acromag DACs are concerned (unlike the General Standards DAC).
I unboxed the Trek amplifier today, and performed some basic tests of the functionality. It seems to work as advertised. However, we may have not specified the correct specifications - the model seems to be configured to drive a bipolar output of +/- 125 V DC, whereas for PZT driving applications, we would typically want a unipolar drive signal. From reading the manual, it appears to me that we cannot configure the unit to output 0-250V DC, which is what we'd want for general PZT driving applications. I will contact them to find out more.
The tests were done using the handheld precision voltage source for now. I drove the input between 0 to +5 V and saw an output voltage (at DC) of 0-250 V. This is consistent with the voltage gain being 50V/V as is stated in the manual, but how am I able to get 250 V DC output even though the bipolar configuration is supposed to be +/- 125 V? On the negative side, I am able to see 50V/V gain from 0 to -1 V DC. At which point making the input voltage more negative does nothing to the output. The unit is supposed to accept a bipolar input of +/- 10 V DC or AC, so I'm pretty sure I'm not doing anything crazy here...
Okay based on the markings on the rear panel, the unit is in fact configured for unipolar output. What this means is we will have to map the +/- 10 V DC output from the DAC to 0-5 V DC. Probably, I will stick to 0-2.5 V DC for a start, to not exceed 125 V DC to the PI PZT. I'm not sure what the damage spec is for that. The Noliac PZT I think can do 250 V DC no problem. Good thing I have the inverting summing amplifier coming in tomorrow...
I set up to do the WFS head modifications today, but I was shot down in flames due to a missing AC/DC adapter.
The Prologix GPIB-ethernet dongle needs +8-13 V to run. Some riff raff has removed the adapter and I was thunderstruck to see that it had not been returned.
I did the usual hunt around the lab looking for something with the right specs and connector. I found one that could do +9V and had the right connector, but it didn't light up the adapter so I put it back in black SP table.
I'll order a couple of these (5 ordered for delivery on Wednesday) in case there's a hot demand for the jack / plug combo that this one has. The setup is in the walkway, but I returned the AS table to the usual state and made sure the IMC is locking well.
Clearly this "riff raff" is referring to me. It won't help today I guess but there is one each on the carts holding the SR785 (currently both in the office/electronics bench area), and the only other unit available in the lab is connected to a Prologix box on the Marconi inside the PSL enclosure.
Teledyne AP1053 etc were transported from Rich's office to the 40m. The box is placed on the shelf at the entrance.
My record tells that there are 7 AP1053 in the box. I did not check the number this time.
Increasing the compensation capacitance (470 pF now instead of 33 pF) seems to have fixed the oscillation issues associated with this circuit. However, the measured noise is in excess of the model at almost any frequency of relevance. I believe the problem is due to the way the measurement is done, and that we should re-do the measurement once the unit is packaged in a shielded environment.
Attachment #1 shows (schematically) the measurement setup. Main differences from the way I did the last round of testing are:
Attachment #2 shows the measurement results:
I didn't capture the data, but viewing the high voltage output on an Oscilloscope threw up no red flags - the oscillations which were previously so evident were nowhere to be seen, so I think the capacitor switch did the trick as far as stability is concerned.
There is a large excess between measurement and model out to a few kHz, if this is really what ends up going to the suspension then this circuit is useless. However, I suspect at least part of the problem is due to close proximity to switching power supplies, judging by the comb of ~10 Hz spaced peaks. This is a frequent problem in coil driver noise measurements - previously, the culprit was a switching power supply to the Prologix GPIB box, but now a Linear AC-DC converter is used (besides, disconnecting it had no visible effect). The bench supplies providing power to the board, however, is a switching supply, maybe that is to blame? I think the KEPCO supplies providing +/-250 V are linear. I tried the usual voodoo of twisting the wires used to receive the signal, moving the SR785 away from the circuit board etc, but these measures had no visible effect either.
The real requirement of this circuit is that the current noise above 100 Hz be <1pA/rtHz. This measurement suggests a level that is 5x too high. But the problem is likely measurement related. I think we can only make a more informed conclusion after shielding the circuit better and conducting the test in a more electromagnetically quiet environment.
The unit was repaired and returned to the 40m. Now, with a DMM, I measure a DC offset value that is ~1% of the AC signal amplitude. I measured the TF of a simple 1/20 voltage divider and it looks fine. In FFT mode, the high frequency noise floor levels out around 5-7nV/rtHz when the input is terminated in 50 ohms.
I will upload the repair documents to the wiki.
These were delivered to the 40m today and are on Rana's desk
I'll order a couple of these (5 ordered for delivery on Wednesday) in case there's a hot demand for the jack / plug combo that this one has.
We received 20pcs of stuffed demodulator boards from Screaming Circuits today. Some caveats:
I removed 1 from the group to stuff some components that weren't sent to Screaming Circuits and test the functionality on the benchtop, the remaining have been stored in a plastic box for now as shown in Attachment #1. The box has been delivered to Chub who will stuff the remaining 19 boards once I've tested the one piece.
I'll bring a file binder "40m wiring diagram" to home at the next chance.
There is another one on the shelf in the control room.
(I thought I put it in my bag, but it looks like that I left it somewhere around the fax area)
I packaged the HV coil driver into a 2U chassis, hoping for better shielding from pickup. There is still considerable excess noise in measurement vs model around 100 Hz, see Attachment #1. The projected displacement noise from this noise contribution is shown in Attachment #2 - I've also plotted the contribution from the 4.5kohm (planned value for fast path series resistance) for comparison. Attachment #3 has some photos of the measurement setup so if someone sees some red flags, please let me know.
I've run out of ideas to try and make the measurement cleaner - the presence of the rather prominent power line harmonics suggests that this is still not perfect, but what more shielding can we implement? I have to make the measurement on the circuit side of the 25 kohm series resistor, so I am using some Pomona minigrabbers to clip onto the leg of the wirewound resistor (see photos in Attachment #3), so that's not great maybe, but what's the alternative?
So if this is truly the noise of the circuit, then while it's an improvement on the current situaiton, it's unsatisfying that such a simple circuit can't match the design expectations. But how do we want to proceed?
what is the noise level before the HV stage? i.e. how well is the acromag noise being filtered?
It's not so easy to directly measure this I think, because the filtering is rather aggressive. Attachment #1 shows the measured transfer function (dots) vs the model and Attachment #2 shows the noise. I think this checks out - but I can't definitively rule out some excess noise at 100 Hz from this stage. Because the gain of the HV stage is x31, we'd need a preamp with better than 1nV/rtHz to directly measure the noise I guess. The Acromag noise model in Attachment #2 is based on a measurement I describe here.
Andrew made a battery-powered 0.7 nVrtHz input-referred noise pre-amplifier for gain of 200. That might help you.
we'd need a preamp with better than 1nV/rtHz to directly measure the noise I guess.
RXA: 0.7 nV is OK if you're not interested in low noise measurements. Otherwise, we have the transformer coupled pre-amp from SRS which does 0.15 nV/rHz and the Rai Weiss FET amp which has 0.35 nV for high impedance sources.
I had to go through five SR560s in the lab yesterday evening to find one that had the expected 4 nV/rtHz input noise and worked on battery power. To confirm that the batteries were charged, I left 4 of them plugged in overnight. Today, I confirmed that the little indicator light on the back is in "Maintain" and not "Charge". However, when I unplug the power cord, they immediately turn off.
One of the units has a large DC output offset voltage even when the input is terminated (though it is not present with the input itself set to "GND" rather than DC/AC). Do we want to send this in for repair? Can we replace the batteries ourselves?
I now think the excess noise in this circuit could be coming from the KEPCO switching power supply (in fact, the supplies are linear, and specd for a voltage ripple at the level of <0.002% of the output - this is pretty good I think, hard to find much better).
All component references are w.r.t. the schematic. For this test, I decided to stuff a fresh channel on the board, with new components, just to rule out some funky behavior of the channel I had already stuffed. I decoupled the HV amplifier stage and the Acromag DAC noise filtering stages by leaving R3 open. Then, I shorted the non-inverting input of the PA95 (i.e. TP3) to GND, with a jumper cable. Then I measured the noise at TP5, using the AC coupling pomona box (although in principle, there is no need for this as the DC voltage should be zero, but I opted to use it just in case). The characteristic bump in the spectra at ~100Hz-1kHz was still evident, see the bottom row of Attachment #1. The expected voltage noise in this configuration, according to my SPICE model, is ~10 nV/rtHz, see the analysis note.
As a second test, I decided to measure the voltage noise of the power supply - there isn't a convenient monitor point on the circuit to directly probe the +/- HV supply rails (I didn't want any exposed HV conductors on the PCB) - so I measured the voltage noise at the 3-pin connector supplying power to the 2U chassis (i.e. the circuit itself was disconnected for this measurement, I'm measuring the noise of the supply itself). The output is supposedly differential - so I used the SR785 input "Float" mode, and used the Pomona AC coupling box once again to block the large DC voltage and avoid damage to the SR785. The results are summarized in the top row of Attachment #1.
The shape of the spectra suggests to me that the power supply noise is polluting the output noise - Koji suggested measuring the coherence between the channels, I'll try and do this in a safe way but I'm hesitant to use hacky clips for the High Voltage. The PA95 datasheet says nothing about its PSRR, and seems like the Spice model doesn't include it either. It would seem that a PSRR of <60dB at 100 Hz would explain the excess noise seen in the output. Typically, for other Op-Amps, the PSRR falls off as 1/f. The CMRR (which is distinct from the PSRR) is spec'd at 98 dB at DC, and for other OpAmps, I've seen that the CMRR is typically higher than the PSRR. I'm trying to make a case here that it's not unreasonable if the PA95 has a PSRR <= 60dB @100 Hz.
So what are the possible coupling mechanisms and how can we mitigate it?
What do the analog electronics experts think? I may be completely off the rails and imagining things here.
Update 2130: I measured the coherence between the positive supply rail and the output, under the same conditions (i.e. HV stage isolated, input shorted to ground). See Attachment #2 - the coherence does mirror the "bump" seen in the output voltage noise - but the coherence is. only 0.1, even with 100 averages, suggesting the coupling is not directly linear - anyways, I think it's worth it to try adding some extra decoupling, I'm sourcing the HV 10uF capacitors now.
Yes. The datasheet has a recommendation circuit with 10uF caps. Companies are careful to show reproducible, reliably functional circuit examples on datasheets. So, if the caps are there you should try to replicate the design.
Additional bypass capacitors? I use 0.1 uF, 700V DC ceramic capacitors as bypass capacitors close to the leads of the PA95, as is recommended in the datasheet. Can adding a 10uF capacitor in parallel provide better filtering? I'm not sure if one with compatible footprint and voltage rating is readily available, I'll look around.
true. also try to choose a cap with a goow high frequency response. In the Electronics Noise book by Ott there's some graph about this. I bet you good do a Bing search and also find something more modern. Basically we want to make sure that the self resonance is not happening at low frequencies. Might be tought to find one with a good HF response, a high voltage rating, and > 1uF.
yes, both problems can be fixed. Usually we just order some spare lead-acid batteries from SRS (Steve may have some spare ones somewhere). The DC offset often comes from a busted FET input. I bought 50 of those at one point - they're obsolete. Its also possible to replace the input stage with any old FET pair.
I'll handle the one with the offset if you leave it on my desk.
Since we will have several new 1U / 2U aLIGO style electronics chassis installed in the racks, it is desirable to have a more compact power distribution solution than the fusable terminal blocks we use currently.
I did a quick walkaround of the lab and the electronics rack today. I estimate that we will need 5 units of the 24 V and 5 units of the 18 V power strips. Each end will need 1 each of 18 V and 24 V strips. The 1Y1/1Y2/1Y3 (LSC/OMC/BHD sus) area will be served by 1 each 18 V and 24 V. The 1X1/1X2 (IOO) area will be served by 1 each 18 V and 24 V. The 1X5/1X6 (SUS Shadow sensor / Coil driver) area will be served by 1 each of 18 V and 24 V. So I think we should get 7 pcs of each to have 2 spares.
Most of the chassis which will be installed in large numbers (AA, AI, whitening) supports 24V DC input. A few units, like the WFS interface head, OMC driver, OMC QPD interface, require 18V. It is not so clear what the input voltage for the Satellite box and Coil Drivers should be. For the former, an unregulated tap-off of the supply voltage is used to power the LT1021 reference and a transistor that is used to generate the LED drive current for the OSEMs. For the latter, the OPA544 high current opamp used to drive the coil current has its supply rails powered by again, an unregulated tap-off of the supply voltage. Doesn't seem like a great idea to drive any ICs with the unregulated switching supply voltage from a noise point of view, particularly given the recent experience with the HV coil driver testing and the PSRR, but I think it's a bit late in the game to do anything about this. The datasheet specs ~50 dB of PSRR on the negative rail, but we have a couple of decoupling caps close to the IC and this IC is itself in a feedback loop with the low noise AD8671 IC so maybe this won't be much of an issue.
For the purposes of this discussion, I think both Satellite Amp and Coil Driver chassis can be driven with +/- 24 V DC.
On a side note - after the upgrade will the "Satellite Amplifiers" be in the racks, and not close to the flange as they currently are? Or are we gonna have some mini racks next to the chambers? Not sure what the config is at the sites, and if the circuits are designed to drive long cables.
looks good to me.
The thing I usually look for is how much the downstream system (mixers, etc) can perturb the main oscillator. i.e. we don't want mixer in one chain to reflect back and disturb the EOM chain. But since our demods have amplifiers on the LO side we're pretty immune to that.
I have the setup built for the AA/AI board testing around the PD testing area. Please let me leave it like that for a week or so.
12/4 TF Tested 5 PCBs
12/6 TF Tested 19 PCBs (12min/PCB) - found 1 failure (S2001479 CH1) -> Fixed 12/11
12/8 TF Tested 16 PCBs (12min/PCB)
PSD Tested 4 PCBs (11min/PCB)
12/11 TF Tested 10 PCBs + 1 fixed channel (All channels checked)
PSD Tested 10 PCBs (11min/PCB)
12/14 PSD Tested 4 PCBs (6.5min/PCB) fixed noise issue of 2 ch, TF issue of 1 ch
12/15 PSD Tested 32 PCBs (6.5min/PCB) fixed noise issue of 1ch
Temp dependence measurement
I installed a DC power strip (24 V variant, 12 outlets available) on the NW upright of the 1X1 rack. This is for the AS WFS. Seems to work, all outlets get +/- 24 V DC.
The FSS_RMTEMP channel is very noisy after this work. I'll look into it, but probably some Acromag grounding issue.
In the afternoon, Jordan and I also laid out 4x SMA LMR240 cables and 1x DB15 M/F cable from 1X2 to the NE corner of the AP table via the overhead cable trays.
I installed 4 chassis in the rack 1X2 (characterization on the E-bench was deemed satisfactory, I will upload the analysis later). I ran out of hardware to make power cables so only 2 of them are powered right now (1 32ch AA chassis and 1 WFS head interface). The current limit on the +24V Sorensens was raised to allow for similar margin to the limit with the increased current draw.
While I definitely bumped various cables, I don't seem to have done any lasting damage to the CDS system (the RFM errors remain of course).
Installing 10uF bypass capacitors on the High Voltage power supply line for the HV coil driver circuit doesn't improve the noise. The excess bump around a few hundred Hz is still present. How do we want to proceed?
So what do we do about this circuit? For the production version, I can make room on the PCB to install two 10uF film capacitors on the board itself, though that's unlikely to help. I think we've established that
Do we have any better bipolar HV supply that I can use to see if that makes any difference? I don't want to use the WFS supplies as it's not very convenient for testing.
Not really related directly to this work but since we have been talking about current requirements, I attach the output of the current determining script as Attachment #5. For the most part, having 220ohm resistances on the new HAM-A coil driver boards will lead to ~half the DAC range being eaten up for the slow alignment bias. For things like MC1/MC3, this is fine. But for PRM/SRM/BS, we may need to use 100ohms. Chub has ordered all manner of resistances so we should have plenty of choices to pick from.
Power supply bypassing [updated 10pm]:
As mentioned earlier in this thread, I prepared a box with two 10uF, 1kV rated capacitors to bypass the high-voltage rails (see inset in the plot), to see if that improves the performance. However, in measuring the voltage ripple directly with the SR785 (no load connected), I don't see any significant difference whether the decoupling caps are connected or not, see Attachment #1. For this, and all other HV measurements made, I used this box to protect the SR785. One hypothesis is that this box itself is somehow introducting the excess noise, maybe because of leakage currents of the diode pair going into the 1Mohm SR785 input impedance, but I can't find any spec for this, and anyway, these diodes should be at ground potential once the transient has settled and the DC blocking capacitor has charged to its final value.
Note that the 10uF caps have an ESR of 7.2 mOhms. The HP6209 has a source impedance "<20mOhm" when operated as a CV source, per the datasheet. So perhaps this isn't so surprising? The same datasheet suggests the source impedance is 500 mOhms from 1kHz to 100 kHz, so we should see some improvement there, but I only measured out to 2 kHz, and I didn't take much effort to reduce these crazy peaks so maybe they are polluting the measurement out there. There must also be some continuous change of impedance, it cannot be <20 mOhm until 1 kHz and then suddenly increase to 500 mOhms. Anyways, for this particular circuit, the nosie DC-1kHz is what is important so I don't see a need to beat this horse more.
Simplified circuit testing:
I decided to see if I can recover the spec'd voltage noise curve from the PA95 datasheet. For this, I configured the PA95 as a simple G=31 non-inverting amplifier (by not stuffing the 15 uF capacitor in the feedback path). Then, with the input grounded, I measured the output voltage noise on the circuit side of the 25kohm resistor (see inset in Attachment #2). To be consistent, I used the DC blocking box for this measurement as well, even though the output of the PA95 under these test conditions is 0V. Once again, there is considerable excess around ~100 Hz relative to a SPICE model. On the basis of this test, I think it is fair to say that the problem is with the PA95 itself. As far as I can tell, I am doing everything by the book, in terms of having gain > 10, using a sufficiently large compensaiton cap, HV rail decoupling etc etc. Note that the PA95 is a FET input opamp, so the effects of input current noise should be negligible. The datasheet doesn't provide the frequency dependence, but if this is just shot noise of the 1200 pA input bias current (for 300 V rails, per the spec), this is totally negligible, as confirmed by LTspice.
In the spirit of going step-by-step, I then added the feedback capacitor, and still, measured noise in excess of what I would expect from my model + SR785 measurement noise.
Integrated circuit testing:
After the above simplified test, I stuffed a full channel as designed, and tested the noise for various drive currents. To best simulate the operating conditions, an Acromag XT1541 was used to set the DC voltage that determines the drive current through the 25 kohm resistor. The measurements were made on the circuit side of this resistor (I connected a 20ohm resistor to ground to simulate the OSEM). As shown in Attachment #3, the noise with these HP6209 supplies is significantly better than what I saw with the KEPCO supplies, lending further credence to the hypothesis that insufficient PSRR is the root of the problem here. I've added subplots in a few different units - to be honest, I think that reaching this level of measured displacement noise at the 40m at 100 Hz would already be pretty impressive.
So what's next?
The main design change is that a passive R-C-R (4k-3uF-20k) replaces the single 25kohm resistor at the output of the PA95.
Let's see if this fixes the issue. Not that I've also added a pair of input protection diodes to the input of the PA95 in the new design. The idea is that this would protect the (expensive) PA95 IC from, for example, the unit being powered with the +/- 18V rail but not the +/- 300 V rail. As I type this, however, I wonder if the leakage current noise of these diodes would be a problem. Once again, the datasheet doesn't provide any frequency dependence, but if it's just the shot noise of the 1nA expected when the diodes are not reverse biased (which is the case when the PA95 is operating normally since both inputs are at nearly the same potential), the level is ~20 fA/rtHz, comparable to the input current noise of the PA95, so not expected to be an issue. In the worst case, the PCB layout allows for this component to just be omitted.
We received the custom cables to test the new suspension electronics. They are under my desk. So we are ready.
This batch was a small one - the company says that they can make molded cables if we have a minimum order, something to consider I gues.s.
Update 1900 11 Feb: I verified that the pin outs of the cables are as we intended (for one set of each type of cable). Because this was a small order, the connectors have metal shells, and so for cable #2 (sat box to flange), the two shells are shorted to each other. I can't verify if the shield is isolated from the shell on J5 without cutting open the cable. One thing that occurred to me is that we should give pins 5,8,11 on J4 and 16,20,24 on J5 (respectively) unique identifiers. They should only be shorted to GND on the circuit board itself. To be fixed for the next iteration. I uploaded some photos here.
I was unable to measure the capacitance of the cable using the LCR meter, and didn't opt to try any other method.
I did what I consider to be a comprehensive set of tests on the production version of the high voltage coil driver circuit. I think the performance is now satisfactory and the circuit is ready for the production build. Barring objections from anyone, I will ask Chub to place an order for components to stuff the 4 necessary units + 1 spare on Friday, 12 Feb (so that people have a full day to comment). A big thanks to Chub and the folks at JLCPCB for dealing with my incessant order requests and patiently supporting this build and letting me turn this around in 10 days - hopefully this is the end of this particular saga.
Schematic is here. All references to component designations are for v4 of the schematic.
Important design changes:
A series of tests were done. Note that only 1 channel was stuffed (I am out of PA95s), and the HP power supplies borrowed from Rich were used for the HV rails. For the +/-18V, a regular bench-top unit was used.
As I was stuffing the board, I noticed a few improvements that can be made. Just noting these here for documentation purposes - these changes are mostly aesthetic and I personally see no need to order another set of PCBs.
Communications with Apex:
I've been talking to support at Apex, and pointed out that I couldn't match the SPICE model performance even for a simple non-inverting amplifier with the PA95. The feedback I got from them was that
Whiel the PA194 is compatible with our voltage and current requirements for this application, it is ~3x the cost, and seems like the R-C-R output filter allows us to realize the goal of 1pA/rtHz, so I'm inclined to stick with the PA95.
I'd prefer to get as much of the board stuffed by Screaming Circuits as possible. It took me ~3 hours to stuff 1 channel + the power supply parts, standoffs etc. So I estimate it'll take me ~6 hours to stuff the entire board. So not the end of the world if we have to do it in-house.
For your planning:
Todd provided us a bunch of electronics. I went to Downs to pick them up this afternoon and checked the contents in the box. Basically, the boxes are pretty comprehensive to produce the following chassis
Some panels are missing (we cannibalized them for the WFS electronics). Otherwise, it seems that we will be able to assemble these chassis listed.
They have placed inside the lab as seen in the attached photo.
HAM-A COIL DRIVER (Req Qty 28+8)
- 8 Chassis
- 8 Front Panels
- 8 Rear Panels
- 8 HAM-A Driver PCBs
- 8 D1000217 DC Power board
- 8 D1000217 DC Power board
16bit AA (Req Qty 7)
- 7 CHASSIS
- 6 7 Front Panels (1 missing -> [Ed 2/22/2021] Asked Chub to order -> Received on 3/5/2021)
- 7 Rear Panels
- 28 AA/AI board S2001472-486, 499-511
- 7 D070100 ADC AA I/F
- 7 D1000217 DC Power board
18bit AI (Req Qty 4)
- 4 CHASSIS
- 4 Front Panels
- 4 Rear Panels
- 8 AA/AI board S2001463-67, 90-92
- 4 D1000551 18bit DAC AI I/F
- 4 D1000217 DC Power board
- bunch of excess components
16bit AI (Req Qty 5)
- 5 CHASSIS
- 4 5 Front Panels (D1101522) (1 missing -> [Ed 2/22/2021] Asked Chub to order -> Received on 3/5/2021)
- 3 5 Rear Panels (D0902784) (2 missing -> [Ed 2/22/2021] Asked Chub to order -> Received on 3/5/2021)
- 10 AA/AI board S2001468-71, 93-98
- 5 D1000217 DC Power board
- 5 D070101 DAC AI I/F
Internal Wiring Kit
Asked Chub to order:
- Qty 12 1U Hamilton Chassis
- Qty 5 x Front/Rear Panels/Internal PCBs for D1002593 BIO I/F (The parts and connectors to be ordered separately)
-> Front/Rear Panels received (3/5/2021)
-> PCBs (unpopulated) received (3/5/2021)
-> Components ordered by KA (3/7/2021)
Will we also be receiving the additional 34 Satellite Amplifier PCBs?
We received currently available sets. We are supposed to receive more coil drivers and sat amps, etc. But they are not ready yet.
Koji asked me to test the production version of the coil driver with the KEPCO HV supplies. See Attachment #1 for the results. For comparison, I've added a single trace from the measurements made with the HP supplies. I continue to see excess noise with the KEPCO supplies. Note that in the production version of the board that was tested, there are a pair of 10uF bypass capacitors on the board for the HV supply lines. It is possible that one or both KEPCO supplies are damaged - one was from the ASY setup and one I found in the little rack next to 1X2. The test conditions were identical to that with the HP supplies (as best as I could make it so).
This is very disappointing. Even with KEPCO linear supply with the improved HV driver circuit, the noise level is significantly higher than the 20kOhm R thermal noise.
What is special with the HP supplies? Can you replace KEPCOs with the HP supply, one by one to specify which one is making the noise bad?
I will try the test of switching out KEPCOs one at a time for the HP. Given that the passive RC filter doesn't filter out the excess, I am wondering if the KEPCO is somehow polluting the circuit ground? The measurement was made between the circuit side of R24 (see schematic) and a ground testpoint, so the passive R23/C15 pole should filter the noise above ~15 Hz.