I have made the modifications on the other board D1100687 S2100028 as well. The measurements were taken as mentioned in 40m/15784. All conclusions remain the same as 40m/15784. The attached zip file contains all measurement data, before and after the modifications.
Edit Wed Feb 3 16:44:51 2021 :
Added zero modeled noise in the noise spectrum curves. The acquisition mode curves are in agreement with the model. The noise in Run mode is weirdly lower than predicted by zero.
Power supply bypassing [updated 10pm]:
As mentioned earlier in this thread, I prepared a box with two 10uF, 1kV rated capacitors to bypass the high-voltage rails (see inset in the plot), to see if that improves the performance. However, in measuring the voltage ripple directly with the SR785 (no load connected), I don't see any significant difference whether the decoupling caps are connected or not, see Attachment #1. For this, and all other HV measurements made, I used this box to protect the SR785. One hypothesis is that this box itself is somehow introducting the excess noise, maybe because of leakage currents of the diode pair going into the 1Mohm SR785 input impedance, but I can't find any spec for this, and anyway, these diodes should be at ground potential once the transient has settled and the DC blocking capacitor has charged to its final value.
Note that the 10uF caps have an ESR of 7.2 mOhms. The HP6209 has a source impedance "<20mOhm" when operated as a CV source, per the datasheet. So perhaps this isn't so surprising? The same datasheet suggests the source impedance is 500 mOhms from 1kHz to 100 kHz, so we should see some improvement there, but I only measured out to 2 kHz, and I didn't take much effort to reduce these crazy peaks so maybe they are polluting the measurement out there. There must also be some continuous change of impedance, it cannot be <20 mOhm until 1 kHz and then suddenly increase to 500 mOhms. Anyways, for this particular circuit, the nosie DC-1kHz is what is important so I don't see a need to beat this horse more.
Simplified circuit testing:
I decided to see if I can recover the spec'd voltage noise curve from the PA95 datasheet. For this, I configured the PA95 as a simple G=31 non-inverting amplifier (by not stuffing the 15 uF capacitor in the feedback path). Then, with the input grounded, I measured the output voltage noise on the circuit side of the 25kohm resistor (see inset in Attachment #2). To be consistent, I used the DC blocking box for this measurement as well, even though the output of the PA95 under these test conditions is 0V. Once again, there is considerable excess around ~100 Hz relative to a SPICE model. On the basis of this test, I think it is fair to say that the problem is with the PA95 itself. As far as I can tell, I am doing everything by the book, in terms of having gain > 10, using a sufficiently large compensaiton cap, HV rail decoupling etc etc. Note that the PA95 is a FET input opamp, so the effects of input current noise should be negligible. The datasheet doesn't provide the frequency dependence, but if this is just shot noise of the 1200 pA input bias current (for 300 V rails, per the spec), this is totally negligible, as confirmed by LTspice.
In the spirit of going step-by-step, I then added the feedback capacitor, and still, measured noise in excess of what I would expect from my model + SR785 measurement noise.
Integrated circuit testing:
After the above simplified test, I stuffed a full channel as designed, and tested the noise for various drive currents. To best simulate the operating conditions, an Acromag XT1541 was used to set the DC voltage that determines the drive current through the 25 kohm resistor. The measurements were made on the circuit side of this resistor (I connected a 20ohm resistor to ground to simulate the OSEM). As shown in Attachment #3, the noise with these HP6209 supplies is significantly better than what I saw with the KEPCO supplies, lending further credence to the hypothesis that insufficient PSRR is the root of the problem here. I've added subplots in a few different units - to be honest, I think that reaching this level of measured displacement noise at the 40m at 100 Hz would already be pretty impressive.
So what's next?
The main design change is that a passive R-C-R (4k-3uF-20k) replaces the single 25kohm resistor at the output of the PA95.
Let's see if this fixes the issue. Not that I've also added a pair of input protection diodes to the input of the PA95 in the new design. The idea is that this would protect the (expensive) PA95 IC from, for example, the unit being powered with the +/- 18V rail but not the +/- 300 V rail. As I type this, however, I wonder if the leakage current noise of these diodes would be a problem. Once again, the datasheet doesn't provide any frequency dependence, but if it's just the shot noise of the 1nA expected when the diodes are not reverse biased (which is the case when the PA95 is operating normally since both inputs are at nearly the same potential), the level is ~20 fA/rtHz, comparable to the input current noise of the PA95, so not expected to be an issue. In the worst case, the PCB layout allows for this component to just be omitted.
I found a white paper from Acromag which discusses how to read differential signal using Acromag units. The document categorically says that differential signals are always supposed to be transmitted in three wires. I provides the two options of either using the RTN to connect to the signal ground (as done in Attachment 3) or locally place 10k-100k resistors between return and IN+ and IN- both (Attachment 2).
I have provided possible scenarios for these.
Using an acromag card without making any connection with RTN is basically not allowed as per this document.
I fitted zeros and poles in the measured transfer function of D1100687 S2100027 and got zeros at 130 Hz and 234 Hz and poles at 10Hz and 2845 Hz. These values are different from the aimed values in this doc, particularly the 234Hz zero which was aimed at 530 Hz in the doc.
I also took the noise measurement using the same method as described in 40m/15780. The noise in Acquisition mode seems to have gone up in 10 Hz - 500 Hz region compared to the measurement in 40m/15780 before the modifications.
All channels are consistent with each other.
Edit Mon Feb 1 12:24:14 2021:
Added zero model prediction after the changes. The measurements match with the predictions.
Edit Wed Feb 3 16:46:59 2021:
Attachment #1 shows the DAC noise models for the General Standards 16-bit and 18-bit DACs we are expecting to have.
Attachment #2 shows the expected actuation range for DC optic alignment, assuming we use the entire DAC range for this purpose.
Attachment #3 shows the current and proposed (by me, just a rough first pass, not optimized in any way yet) de-whitening filter shapes. These shapes can be tweaked for sure.
Attachment #4 puts everything into displacement noise units. The electronics noise of the coil driver / de-whitening circuit have not been included so at high frequencies, the projection is better than what will actually be realizable, but still well below the BHD requirement of 3e-17 m/rtHz.
Looks fine to me visually but the verdict can only be made once the z:p locations are quantitatively confirmed, and the noise tests pass. It would be interesting to see what kind of time-domain transient (in N of force) switching on the de-whitening introduces, i guess best done interferometrically.
I'll wait for comments until tomorrow to proceed with changes in the other board as well. I'll do noise measurements tomorrow.
I did the recommended modifications on of the boards with serial number S2100028. These included:
I took transfer function measurements with same method as in 40m/15774 and I'm presenting it here to ensure the modifications are correct and if I should proceed to the next board as well. I didn't have the data used to make plots in here but I think the poles and zeros have landed in the right spot. I'll wait for comments until tomorrow to proceed with changes in the other board as well. I'll do noise measurements tomorrow.
I took some steps to reduce the coupling of 60 Hz harmonics in noise measurement. The box was transferred to the floor instead of on top of another instrument. Measurement was immediately converted into single-ended using SR560 in battery mode with a gain of 10. All of the setups was covered in aluminum foil to increase isolation.
Here I present few wiring diagrams when using Acromag to avoid noisy behavior and ground loops.
Edit Wed Jan 27 13:38:19 2021 :
This solution is not acceptable as well. Even if it is successfull in reading the value, connecting resistor between IN- and RTN will not break the ground loops and the issue of ground loops will persist. Further, IN- connection to RTN breaks the symmetry between IN- and IN+, and hence reduces the common mode rejection which is the intended purpose of differential signal anyways. I'll work more on this to find a way to read differential signals without connecitng IN- and RTN. My first guess is that it would need the GND on the source end to be connected to EarthGND and RTN on acromag end to be connected to EarthGND as well.
Taking inspiration from SR785 on how it reads differential signal, I figured that acromag too always need a way to return current through RTN ports always. That must be the reason why everything goes haywire when RTN is not connected to IN-. Now for single ended signals, we can always short RTN to IN- and keep same GND but then we need to be careful in avoiding ground loops. I'm gonna post a wiring diagram in next post to show how if two signal sources connect to each other separately, a GND loop can be formed if we tie each IN- port to RTN on an acromag.
Coming to the issue of reading a differential signal, what SR785 does is that it connects 50 Ohm resistance between Earth GND and differential signal shields (which are supposed to signal GND). In a floating GND setting, SR785 connects a 1 MOhm resistor between input shield and Earth GND. This can be used to read a differential signal through a single BNC cable since the shiled can take arbitrary voltages thanks ti the 1 MOhm resistor.
We can do the same in acromag. Instead of shorting RTN to IN- ports, we can connect them through a large resistor which would let IN- float but will give a path for current to return through RTN ports. Attached here are few scenarios where I connected IN- to RTN throguh wire, 820 Ohms, 10kOhms and 1MOhms in two sub cases where RTN was left open or was shorted to Earth GND. In all cases, the signal was produced by a 9V battery outputing roughly 8.16V. It seems that 10kOhm resistor between RTN and IN- with RTN connected to Earth GND is the best scenario noise wise. I'll post more results and a wiring diagram soon.
For whatever reason, the autolocker didn't turn the tickle off for several hours. Seems to work okay now. The linked plot suggests that the coil balancing on MC2 is pretty lousy.
I took transfer function and noise measurement of satellite amplifier box's photodiode transimpedance circuit. For the measurement, I created a makeshift connector to convert backside DB25 into DB9 with the 4 channels for PDA input. The output was taken in differential form at the front PD Output port. To feed current to the circuit, I put in 12 kOhm resistors in series at the inputs, so the V/V transfer function measured was multiplied by 12 kOhm to get the transimpedance of the circuit.
Edit Wed Feb 10 15:14:13 2021 :
THE NOISE MEASUREMENT WAS WRONG HERE. SEE 40m/15799.
After fixing multiple issues, the model webviews are updating, should be done by tomorrow. It should be obvious from the timestamps on the index page which are the new ones. These new screens are better than the old ones and offer more info/details. Please look at them, and let me know if there are broken links etc. Once we are happy with this new webview, we can archive the old files and clean up the top directory a bit. I don't think this adds anything to the channel accounting effort but it's a nice thing to have up-to-date webviews, I found the LLO ones really useful in setting up the AS WFS model.
BTW, the crontab on megatron is set to run every day at 0844. The process of updating the models is pretty heavy because of whatever MATLAB overhead. Do we really need to have this run every day? I modified the crontab to run every other Saturday, and we can manually run the update when we modify a model. Considering this hasn't been working for ~3 years, I think this is fine, but if anyone has strong preference you can edit the crontab.
If someboy can volunteer to fix the MEDM screenshot that would be useful.
I have taken transfer functions and noise measurements of the two HAM-A coil driver boxes D1100687 #S2100027 and #S2100028. All transfer functions look as expected. I'm not sure about the noise measurements. If anyone sees flaw in my measurement method, please let me know. I'm not sure why in some channels I got 10Hz harmoni peaks in the noise. That was very strange. Also let me know if my current noise estimate is wrong.
Installing 10uF bypass capacitors on the High Voltage power supply line for the HV coil driver circuit doesn't improve the noise. The excess bump around a few hundred Hz is still present. How do we want to proceed?
So what do we do about this circuit? For the production version, I can make room on the PCB to install two 10uF film capacitors on the board itself, though that's unlikely to help. I think we've established that
Do we have any better bipolar HV supply that I can use to see if that makes any difference? I don't want to use the WFS supplies as it's not very convenient for testing.
Not really related directly to this work but since we have been talking about current requirements, I attach the output of the current determining script as Attachment #5. For the most part, having 220ohm resistances on the new HAM-A coil driver boards will lead to ~half the DAC range being eaten up for the slow alignment bias. For things like MC1/MC3, this is fine. But for PRM/SRM/BS, we may need to use 100ohms. Chub has ordered all manner of resistances so we should have plenty of choices to pick from.
Not sure if 1Y1 can accommodate both c1sus2 and c1bhd as well as the various electronics chassis that will have to be installed. There may need to be some distribution between 1Y1 and 1Y3. Does Koji's new wiring also specify which racks hold which chassis?
Some minor improvements to the diagram:
I've produced updated diagrams of the CDS layout, taking the comments in 15476 into account. I've also converted the 40m's diagrams from Omnigraffle ($150/license) to the free, cloud-based platform draw.io. I had never heard of draw.io, but I found that it has most all the same functionality. It also integrates nicely with Google Drive.
Attachment 1: The planned CDS upgrade (2 new FEs, fully replace RFM network with Gen 1 Dolphin IPC)
Attachment 2: The current 40m CDS topology
The most up-to-date diagrams are hosted at the following links:
Please send me any further corrections or omissions. Anyone logged in with LIGO.ORG credentials can also directly edit the diagrams.
Indeed T1800302 is the document I was alluding to, but I completely missed the statement about >3 GHz speed. There is an option for 3.4 GHz processors on the X10SRi-F board, but back in 2019 I chose against it because it would double the cost of the systems. At the time I thought I had saved us $5k. Hopefully we can get the LLO machines in the near term---but if not, I wonder if it's worth testing one of these to see whether the performance is tolerable.
Can you please provide a link to this "list of boards"? The only document I can find is T1800302....
I confirm that PCIe 2.0 motherboards are backwards compatible with PCIe 1.x cards, so there's no hardware issue. My main concern is whether the obsolete Dolphin drivers (requiring linux kernel <=3.x) will work on a new system, albeit one running Debian 8. The OSS PCIe card is automatically configured by the BIOS, so no external drivers are required for that one.
Please also confirm that there are no conflicts w.r.t. the generation of PCIe slots, and the interfaces (Dolphin, OSSI) we are planning to use - the new machines we have are "PCIe 2.0" (though i have no idea if this is the same as Gen 2).
I decided to analyze the data I took in December more carefully to see if there are any clues about the weird LSC sensing.
Attachment #1 shows the measurement setup.
Attachment #2 shows the measured spectrum with the PSL and EX laser frequency offset locked via PLL.
Fitting the measured sideband powers (up to n=7, taking the average of the measured upper and lower sideband powers to compute a least squares fit if both are measured, else just that of the one sideband measured) agains those expected from a model, I get the following best fit parameters:
To be explicit, the residual at each datapoint was calculated as
The numbers compare favourably with what Koji reported I think - the modulation depths are slightly increased, consistent with the RF power out of the RF box being slightly increased after I removed various attenuators etc. Note the large uncertainty on the relative phase between the two modulations - I think this is because there are relatively few sidebands (one example is n=3) which has a functional dependence that informs on phi - most of the others do not directly give us any information about this parameter (since we are just measuring powers, not the actual phase of the electric field).
Attachment #3 shows a plot of the measured modulation profile, along with the expected heights plugging the best fit parameters into the model. The size of the datapoint markers is illustrative only - the dependence on the model parameters is complicated and the full covariance would need to be taken into account to put error bars on those markers, which I didn't do.
Attachment #4 shows a time domain measurement of the relative phasing between the 11 MHz and 55 MHz signals at the EOM drive outputs on the RF source box. I fit a model there and get a value for the relative phase that is totally inconsistent from what I get with this fit.
I want to lock the PRFPMI again (to commission AS WFS). Have had some success - but in doing characterization, I find that the REFL port sensing is completely messed up compared to what I had before. Specifically, MICH and PRCL DoFs have no separation in either the 1f or 3f photodiodes.
I did make considerable changes to the RF source box, and so now the relative phase between the 11 MHz and 55 MHz signals is changed compared to what it was before. But do we really expect any effect even in the 1f signal? I am not able to reproduce this effect in simulation (Finesse), though I'm using a simplified model. I attach two sensing matrices to illustrate what i mean:
Can you please provide a link to this "list of boards"? The only document I can find is T1800302. In that, under "Basic Requirements" (before considering specific motherboards), it is specified that the processor should be clocked @ >3GHz. The 3 new supermicros we have are clocked at 1.7 GHz. X10SRi-F boards are used according to that doc, but the processor is clocked at 3.6 or 3.2 GHz.
The motherboard actually has six PCIe slots and is on the CDS list of boards known to be compatible.
As for the CX4 cable - I still think it's good to have these on hand. Not good to be in a situation later where FE and expansion chassis have to be in different racks, and the copper cable can't be used.
Koji asked me assemble a detailed breakdown of the parts received from LHO, which I do based on the high-res photos that Gautam posted of the shipment.
Also, I looked into the mix-up regarding the number of PCIe slots in the new Supermicro servers. The motherboard actually has six PCIe slots and is on the CDS list of boards known to be compatible. The mistake (mine) was in selecting a low-profile (1U) chassis that only exposes one of these slots. But at least it's not a fundamental limitation.
One option is to install an external PCIe expansion chassis that would be rack-mounted right above the FE. It is automatically configured by the system BIOS, so doesn't require any special drivers. It also supports hot-swapping of PCIe cards. There are also cheap ribbon-cable riser cards that would allow more cards to be connected for testing, although this is not as great for permanent mounting.
It may still be better to use the machines offered by Keith Thorne from LLO, as they're more powerful anyway. But if there is going to be an extended delay before those can be received, we should be able to use the machines we already have in conjunction with one of these PCIe expansion options.
I think the "Rogue Master" setting on the RFM network may be doing some good. 5 mins, ago, all the CDS indicators were green, but I noticed an amber light on the c1rfm screen just now (amber = warning). Seems like at GPS time 1294691182, there was some kind of error on the RFM network. But the network hasn't gone down. I can clear the amber flag by running the global diag reset. Nevertheless, the upgrade of all RT systems to Dolphin should not be de-prioritized I think.
That's fine, we didn't actually request those. We bought and already have in hand new PCIe x4 cables for the chassis-host connection. They're 3 m copper cables, which was based on the assumption of the time that host and chassis would be installed in the same rack.
I picked the boxes up this morning. The inventory per Fil's email looks accurate. Some comments:
> Barebones on this order.
> 1. Main PCIe board
> 2. Backplane (Interface board)
> 3. Power Board
> 4. Fiber (One Stop) Interface Card, chassis side only
> 5. Two One Stop Fibers
> 6. No Timing Interface
> 7. No Binary Cards.
> 8. No ADC or DAC cards
> Fil Clara
I'm working on a better wiring diagram that takes into account multiple power supplies, how their GND is passed forward to the circuits or sensors using those power supplies and what possible wiring configurations on Acromag would give low noise. I think I have two configurations in mind which I will test and update here with data and better diagrams.
I took some striptool images earlier yesterday. So I'm dumping them here for further comments or inferences.
Thanks for the systematic effort.
I used an Acromag XT1221 in CTN to play around with different wiring and see what works. Following are my findings:
Floating Single Ended Source (Attachment 2):
Differential Source (Attachment 3):
Comments and suggestions are welcome.
Related elog posts:
Edit Tue Jan 26 12:44:19 2021 :
Note that the third wiring diagram mentioned actually does not work. It is an error in judgement. See 40m/15762 for seeing what happens during this.
I dived into the alog to make the OLTFs in the MC_controls example more realistic. I was mainly inspired by these entries:
and Evan's and Dennis's Theses.
Attachment 1 shows the new OLTFs. I tried to make them go like 1/f around the UGF and fall as quickly as possible at higher frequencies. I didn't do more advanced stability checks.
I also noticed that imbalances and detunings in the MC simulation can change the plants significantly. Especially DARM, CARM, and sometimes PRCL. I added the option to fix some OLTFs throughout the simulation. At every iteration, the simulation computes the required control filter to fix the selected OLTFs such that it will match the OLTFs in the undetuned and balanced IFO.
I did this test today. The excess noise around 100 Hz doesn't show up in the green beat.
See Attachment #1. The setup was as usual:
So, this excess appears to truly be excess phase noise on the fiber (though I have no idea what the actual mechanicsm could be or what changed between Aug and Oct 2020 that could explain it. Maybe the HEPA?
For this work, I had to spend some time aligning the two green beams onto the beat photodiode. During this time, I shuttered the PSL, disabled feedback via the FSS servo, turned the HEPA high, and kept the EX green locked to the arm so as to have a somewhat stable beat signal I could maximize. Everything has been returned to nominal settings now (obviously, since I locked the arms to get the data).
You may ask, why do we care. In terms of RMS frequency noise, it would appear that this excess shouldn't matter. But in all my trials so far, I've been unable to transition control of the arm cavity lengths from POX/POY to ALS. I suppose we could try using the green beat, but that has excess low frequency noise (which was the whole point of the fiber coupled setup).
How about resurrecting the PSL table green beat for the X arm to see if the non-fiber setup shows the same level of the freq noise (e.g. the PDH locking became super noisy due to misalignment etc).
If the sensing noise level of the end PDH degraded for some reason, it'd make the out of loop stability worse without making the end pdh error level degraded.
It's just speculation.
I thought about it, but wouldn't that show up at the AUX PDH error point? Or because the loop gain is so high there we wouldn't see a small excess? I suppose there could be some clipping on the Faraday or something like that. But the GTRX level and the green REFL DC level when locked are nominal.
I'm also wondering why the error monitors for the X and Y loops report such wildly different spectra for the suppressed frequency noise of the AUX laser relative to the cavity length, see Attachment #1. The y-axis should be approximately Hz/rtHz. In both cases, the servo's error point monitor is connected to the DAQ system via a G=10 SR560. With the SR785, I measure for EX a nice bucket-shaped spectrum, bottoming out at ~10 uV/rtHz around 40 Hz, see Attachment #2. The SR560 should have an input-referred noise much less than this at the G=10 setting. The ADC noise level is only ~5 uV/rtHz, and indeed, the EY spectrum shows the expected shape. So what's up with the EX error mon? Tried swapping out the SR560 for a different unit, no change. And both the SR560 noise, and the ADC noise, check out when everything is checked individually. So some kind of interaction once everything is connected together, but it's only present at EX...
Today, I tried switching the EX and EY fibers going into the beat mouth, but I preserved the channel mapping after the beat mouth by switching the electrical outputs as well (the goal was to make sure that the beat photodiodes weren't the issue here, I think the electronics are already exonerated since driving the channel with a Marconi doesn't produce these noisy features). The EX spectrum remains noisy. I've switched everything back to the nominal configuration now to avoid further confusion. So it would appeat that this is real frequency noise that gets added in the EX fiber somehow. What can I do to fix this? The source of coupling isn't at the PSL table, else the EY channel would also show similar features. Visually, nothing seems wrong to me at EX either. So the problem is somehow in the cable tray along which the 40m of fiber is routed? This is already inside some nice foam/tubing setup, what can be done to improve it? Still doesn't explain why it suddenly became noisy...
I want to get back to locking the interferometer so I can test out the newly installed AS WFS. However, the ALS noise is far too high, at least the transition of arm length control from IR to ALS fails reliably with the same settings that worked so reliably previously. I worked on investigating it a bit today.
In the latter half of last year, I was focused on the air-BHD setup, so I wasn't checking in on the ALS noise as regularly.
All tests are done with the arm cavity length locked to the PSL frequency using POX. Then, the EX laser is locked to the arm cavity length using the AUX PDH servo. The fluctuations in the beatnote between the two lasers is what is monitored as a diagnostic. See Attachment #1. The reference traces in the top pane are from a known good time. The large excess noise between ~80-200 Hz is what I'm concerned about.
A separate issue that can improve the noise is to track down the noise in the 20-80 Hz band - probably some IMC frequency noise issue.
See Attachment #2.
So what could it be? The only things I can think of are (i) the beat mouth photodiode (NF1611) or (ii) excess noise in the fiber carrying the light from EX to the PSL table (but only on this fiber, and not on the EY fiber). Both seem remote to me - I'll test the former by switching the EX and EY fiber inputs to the beat mouth, but apart from this, I'm out of ideas...
To avoid this kind of issue, we should really have scripted locks of all the basic IFO configs and record the data to summary pages or something - maybe something to do once Guardian is installed, it'd be pretty hacky to do cleanly with shell scripts.
I've noticed that there is some phase loss in the POX/POY locking loops - see Attachment #1, live traces are from a recent measurement while the references are from Nov 4 2018. Hard to imagine a true delay being responsible to cause so much phase loss at 100 Hz. Attachment #2 shows my best effort loop modeling, I think I've got all the pieces, but maybe I missed something (I assume the analog whitening / digital anti-whitening are perfectly balanced, anyway this wasn't messed with anytime recently)? The fitter wants to add 560 us (!) of delay, which is almost 10 clock cycles on the RTS, and even so, the fit is poor (I constrain the fitter to a maximum of 600 us delay so maybe this isn't the best diagnostic). Anyway, how can this change be explained? The recent works I can think of that could have affected the LSC sensing were (i) RF source box re-working, and (ii) vent. But I can't imagine how either of these would introduce phase loss in the LSC sensing. Note that the digital demod phase has been tuned to put all the PDH signal in the "I" quadrature, which is the condition in which the measurement was taken.
Probably this isn't gonna affect locking efforts (unless it's symptomatic of some other larger problem).
As part of the hunt why the X arm IR transmission RIN is anomalously high, I noticed that the BS Oplev Servo periodically kicks the optic around - the summary pages are the best illustration of this happening. Looking back in time, these seem to have started ~Nov 23 2020. The HeNe power output has been degrading, see Attachment #1, but this is not yet at the point where the head usually needs replacing. The RIN spectrum doesn't look anomalous to me, see Attachment #2 (the whitening situation for the quadrants is different for the BS and the TMs, which explains the HF noise). I also measured the loop UGFs (using swept sine) - seems funky, I can't get the same coherence now (live traces) between 10-30 Hz that I could before (reference traces) with the same drive amplitude, and the TF that I do measure has a weird flattening out at higher frequencies that I can't explain, see Attachment #3.
The excess RIN is almost exactly in the band that we expect our Oplevs to stabilize the angular motion of the optics in, so maybe needs more investigation - I will upload the loop suppression of the error point later. So far, I don't see any clean evidence of the BS Oplev HeNe being the culprit, so I'm a bit hesitant to just swap out the head...
the replacement was done this afternoon. The red "Replace Battery" indicator is no longer on.
I wanted to check the functionality of the IMC WFS. I just turned on the WFS servo loops as they were. For the past two hours, they didn't run away. The servo has been left turned on. I don't think there is no reason to keep it turned off.
I don't have omnigraffle - what about uploading the source doc in a format that the excellent (and free) draw.io can handle? I think we can do a much better job of making this diagram reflect reality. There should also be a corresponding diagram for the Acromag system (but that doesn't have to be tied to this task). Megatron (scripts machine) and nodus should be added to that diagram as well.
Please send me any omissions or corrections to the layout.
Last night, I briefly spoke with Koji about the CDS upgrade plan. This is a follow up.
Each server needs a minimum of two peripheral devices added to the PCIe bus:
As for the second issue, the main question is if we had an open PCIe slot on the c1iscex machine to install a Dolphin card. Looks like the 2 standard slots are taken (see Attachment #1), but a "low profile" slot is available. I can't find what the exact models of the Supermicro servers installed back in 2010 are, but maybe it's this one? It's a good match visually anyways. The manual says a "riser card" is required. I don't know if such a riser is already installed.
Questions I have, Rolf is probably the best person to answer:
Koji fixed the problematic channel - the issue was a bad solder joint on the input resistors to the THS4131. The board was re-installed. I also made a custom 2x4-pin LEMO-->DB9 cable, so we are now recording the PMC and FSS ERR/CTRL channel diagnostics again (spectra tomorrow). Note that Ch32 is recording some sort of DuoTone signal and so is not usable. This is due to a misconfiguration - ADC0 CH31 is the one which is supposed to be reserved for this timing signal, and not ADC1 as we currently have. When we swap the c1ioo hosts, we should fix this issue.
I also did most of the work to make the MEDM screens for the revised ASC topology, tried to mirror the site screens where possible. The overview screen remains to be done. I also loaded the anti-whitening filters (z:p 150:15) at the demodulated WFS input signal entry points. We don't have remote whitening switching capability at this time, so I'll test the switching manually at some point.
The main issue is that in the AA chassis I built, Ch14 (with the first channel as Ch1) has the output saturated to 28V (differential). I'm not sure what kind of overvoltage protection the ADC has - we frequently have the inputs exceed the spec'd +/-20 V (e.g. when the whitening filters are engaged and the cavity is fringing), but pending further investigation, I am removing the SCSI connection from the rear of the AA chassis.
The CDS model change required to get the AS WFS signals into the RTCDS system are rather invasive.
In terms of computational load, the c1ioo model seems to be able to handle the extra load no issues - ~35us/60us per cycle. The RFM model shows no extra computational time.
After this work, the IMC locking and POX/POY locking, and dither alignment servos are working okay. So I have some confidence that my invasive work hasn't completely destroyed everything. There is some hardware around the rear of 1X2 that I will clear tomorrow.
Attached is the layout for the "intermediate" CDS upgrade option, as was discussed on Wednesday. Under this plan:
Existing FEs stay where they are (they are not moved to a single rack)
Dolphin IPC remains PCIe Gen 1
RFM network is entirely replaced with Dolphin IPC
I just want to point out that if you move all the FEs to the same rack they can all be connected to the Dolphin switch via copper, and you would only have to string a single fiber to every IO rack, rather than the multiple now (for network, dolphin, timing, etc.).
I installed 4 chassis in the rack 1X2 (characterization on the E-bench was deemed satisfactory, I will upload the analysis later). I ran out of hardware to make power cables so only 2 of them are powered right now (1 32ch AA chassis and 1 WFS head interface). The current limit on the +24V Sorensens was raised to allow for similar margin to the limit with the increased current draw.
While I definitely bumped various cables, I don't seem to have done any lasting damage to the CDS system (the RFM errors remain of course).
Our favorite (flexible) RF cable is Belden's 1671J (Jacketed solder-soaked coax cable). It is compatible RG405. I'm not sure if there is off-the-shelf SMA cables with 1671J.
I re-ordered the below cables, this time going with flexible, double-shielded RG316-DS. Jordan will pick up and return the RG-405 cables after the holidays.
As requested, I placed an order for an assortment of new RF cables: SMA male-male, RG405.
As I was working on the IFO re-alignment just now, the rfm errors popped up again. I don't see any useful diagnostics on the web interface.
Do we want to take this opportunity to configure jumpers and set up the rogue master as Rolf suggested? Of course there's no guarantee that will fix anything, and may possibly make it impossible to recover the current state...