Gabriele left the DataRay beam profiler + peripherals (see Attachment #1) in his office. I picked them up just now and brought them over to the 40m.
A technician came to the lab today at ~4pm. He entered the VEA (with booties and googles), and also the clean and bake lab. The whole procedure lasted ~10 minutes. I did not follow him around, but was available in the control room throughout the process. I think the whole episode went without incident.
BTW, this guy didn't ring the doorbell, I just happened to be here when he came by. I don't know if this is usual practise - are we happy with the technicians entering the VEA and/or clean and bake labs without supervision? AFAIK, this wasn't scheduled.
I finally managed to install a differential-receiving whitening board in 1Y2 - 4 channels are available at the moment. As I claimed, one stage of 15:150 Hz z:p whitening does improve the ALS noise a little, see Attachment #1. While the RMS (from 1kHz-0.5 Hz) does go down by ~10 Hz, this isn't really going to make any dramatic improvement to the 40m lock acquisiton. Now we're really sitting on the unsuppressed EX laser noise above ~30 Hz. This measurement was taken with the arm cavities locked with POX/POY, and end lasers locked to the arm cavities with uPDH boxes as usual. This was just a test to confirm my suspicion, the whitening board is to be used for the air BHD channels, but when we get a few more stuffed, we can install it for the ALS channels too.
With the chosen transimpedance of 300 ohms, in order to be able to see the shot noise of 10 mW of light in the digitized data streams, we'd need all 3 stages of whitening. If we want to be shot noise limited with 1 mW of LO light, we'd need to increase said transimpedance I think.
The measurements were taken with
Of course, it's unlikely we're going to be shot noise limited for any configuration in the short run. But this was also a test of
All 3 tests passed.
No, there should be no unscheduled visits from any inspector, marshal, tech, or vendor. They all have to be escorted or they don't get in. If they have a problem with that, please give them my cell #.
For the ALS, in addition to the beat note spectrum, I think we need to know the loop gain use to feedback to the ETM to determine the true cavity length fluctuation. w/o ALS, the noise would be only due to the seismic noise, OSEM damping noise, and the IR-PDH residual. Those are all suppressed by the ALS loop, but then the ALS loop puts its sensing noise onto the cavity. So, if I'm thinking about this right, the ALS beat noise > 200 Hz doesn't matter so much to the CARM RMS. So the whitening seems to be doing good in the right spot, but we would like to have another boost in the green PDH to up the gain below ~300 Hz?
A single channel of this board was stuffed (and other channels partially populated). The basic tests passed, and nothing exploded! Even though this is a laughably simple circuit, it's nice that it works.
HV power supplies:
A pair of unused KEPCO BHK300-130 switching power supplies that I found in the lab were used for this test. I pulled the programmable cards out at the rear, and shorted the positive output of one unit to the negative of the other (with both shorted to the supply grounds as well), thereby creating a bipolar supply from these unipolar models. For the purposes of this test, I set the voltage and current limits to 100V DC, 10mA respectively. I didn't ramp up the supply voltage to the rated 300 V maximum. The setup is shown in Attachment #1.
Need to think more about how to better characterize this noise. An estimate of the required actuation can be found here.
The mode-matching between the LO and AS beams is now ~50%. This isn't probably my most average mode-matching in the lab, but I think it's sufficient to start doing some other characterization and we can try squeezing out hopefully another 20-30% by putting the lenses on translation stages, tweaking alignment etc.
The main change was to increase the optical path length of the IFO AS path, see Attachment #1. This gave me some more room to put a lens and translate it.
Try the PRMI experiments again, now that I have some confidence that the beams are actually interfering.
See Attachment #3 for the updated spectra - the configuration is PRMI locked with carrier resonant and the homodyne phase is uncontrolled. There is now much better clearance between the electronics noise and the MICH signal as measured in the DCPDs. The "LO only" trace is measured with the PSL shutter closed, so the laser frequency isn't slaved to the IMC length. I wonder why the RIN (seen in the SUM channel) is different whether the laser is locked to the IMC or not? The LO pickoff is before the IMC.
A more careful analysis has revealed some stability problems. I see oscillations at frequencies ranging from ~600kHz to ~1.5 MHz, depending on the voltage output requested, of ~2 V pp at the high-voltage output in a variety of different conditions (see details). My best guess for why this is happening is insufficient phase margin in the open-loop gain of the PA95 high voltage amplification stage, which causes oscillations to show up in the closed loop. I think we can fix the problem by using a larger compensation capacitor, but if anyone has a better suggestion, I'm happy to consider it.
The changes I wanted to make to the measurement posted earlier in this thread were: (i) to measure the noise with a load resistor of 20 ohms (~OSEM coil resistance) connected, instead of the unloaded config previously used, and (ii) measure the voltage noise on the circuit side (= TP5 on the schematic) with some high voltage output being requested. The point was to simulate conditions closer to what this board will eventually be used in, when it has to meet the requirement of <1pA/rtHz current noise at 100 Hz. The voltage divider formed by the 25 kohm series resistor and the 20 ohm OSEM coil simulated resistance makes it hopeless to measure this level of voltage noise using the SR785. On the other hand, the high voltage would destroy the SR785 (rated for 30 V max input). So I made a little Pomona box to alllow me to do this measurement, see Attachment #1. Its transfer function was measured, and I confirmed that the DC high voltage was indeed blocked (using a Fluke DMM) and that the output of this box never exceeded ~1V, as dictated by the pair of diodes - all seemed okay .
Next, I wanted to measure the voltage noise with ~10mA current flowing through the output path - I don't expect to require more than this amount of current for our test masses. However, I noticed some strange features in the spectrum (viewed continuously on the SR785 using exponential averaging setting). Closer investigation using an oscilloscope revealed:
Some literature review suggested that the capacitor in the feedback path, C4 on the schematic, could be causing problems. Specifically, I think that having that capacitor in the feeddback path necessitates the use of a larger compensation capacitor than the nominal 33pF value (which itself is higher than the 4.7pF recommended on the datasheet, based on experience of the ESD driver circuit which this is based on, oscillations were seen there too but the topology is a bit different). As a first test of this idea, I removed the feedback capacitor, C4 - this seemed to do the trick, the oscillations vanished and I was able to drive the output between the high voltage supply rails. However, we cannot operate in this configuration because we need to roll off the noise gain for the input voltage noise of the PA95 (~6 nV/rtHz at 100 Hz will become ~200 nV/rtHz, which I confirmed using the SR785). Using a passive RC filter at the output of the PA95 (a.k.a. a "snubber" network) is not an option because we need to sum in the fast actuation path voltage at the output of the 25 kohm resistor.
Some modeling confirms this hypothesis, see Attachment #2. The quantity plotted is the open-loop gain of the PA95 portion of the circuit. If the phase is 0 degrees, then the system goes unstable.
So my plan is to get some 470pF capacitors and test this idea out, unless anyone has better suggestions? I guess usually the OpAmps are compensated to be unconditionally stable, but in this case maybe the power op-amp is more volatile?
I'm in the lab this morning to interface the two new UPS units with the digital controls system. Will be out by lunchtime. The disruptions to the vac system should be very brief this time.
I'm leaving the lab shortly. We're not ready to switch over the vac equipment to the new UPS units yet.
The 120V UPS is now running and interfaced to c1vac via a USB cable. The unofficial tripplite python package is able to detect and connect to the unit, but then read queries fail with "OS Error: No data received." The firmware has a different version number from what the developers say is known to be supported.
The 230V UPS is actually not correctly installed. For input power, it has a general type C14 connector which is currently plugged into a 120V power strip. However this unit has to be powered from a 230V outlet. We'll have to identify and buy the correct adapter cable.
With the 120V unit now connected, I can continue to work on interfacing it with python remotely. The next implementation I'm going to try is item #2 of this plan [ELOG 15446].
I re-plotted the MCMC results as semi-transparent lines so that probable lines stick out.
This also reveals what is behind the extreme sparsity in the aLIGO simulation results (In the previous post).
There seem to be some bi-stability/phase transition/whatever in the aLIGO simulation. The aLIGO transfer functions are very sensitive to one or more of the DOFs. Not sure which yet.
Listing some talking points from the last week of activity here.
I found that the control MEDM screen was left open on the c1vac workstation. This should be closed every time you leave the workstation, to avoid accidental button pressing and such.
The network outage meant that the EPICS data from the pressure gauges wasn't recorded until I reset everything ~noon. So there isn't really a plot of the outgassing/leak rate. But the pressure rose to ~2e-4 torr, over ~4 hours. The pumpdown back to nominal pressure (9e-6 torr) took ~30 minutes.
Attachment #1 is a summary of the current to each coil on the suspensions. The situation is actually a little worse than I remembered - several coils are currently drawing in excess of 10mA. However, most of this is due to a YAW correction, which can be fixed somewhat more easily than a PIT correction. So I think the circuit with a gain of 31 for an input range of +/-10 V, which gives us the ability to drive ~12mA per coil through a 25kohm series resistor, will still provide sufficient actuation range. As far as the HV supplies go, we will want something that can do +/- 350 V. Then the current to the coils will at most be ~50 mA per optic. The feedback path will require roughly the same current. The quiescent draw of each PA95 is ~10mA. So per SOS suspension, we will need ~150mA.
If it turns out that we need to get more current through the 25kohm series resistance, we may have to raise the voltage gain of the circuit. Reducing the series resistance isn't a good option as the whole point of the circuit is to be limited by the Johnson noise of the series resistance. Looking at these numbers, the only suspension on which we would be able to plug in a HV coil driver as is (without a vent to correct for YAW misalignment) is ITMY.
Update 2 Sep 2020 2100: I confirmed today that the number reported in the EPICS channel, and the voltage across the series resistor, do indeed match up. The test was done on the MC3 coil driver as it was exposed and I didn't need to disable any suspensions. I used a Fluke DMM to measure the voltage across the resistor. So there is no sneaky factor of 2 as far as the Acromag DACs are concerned (unlike the General Standards DAC).
I unboxed the Trek amplifier today, and performed some basic tests of the functionality. It seems to work as advertised. However, we may have not specified the correct specifications - the model seems to be configured to drive a bipolar output of +/- 125 V DC, whereas for PZT driving applications, we would typically want a unipolar drive signal. From reading the manual, it appears to me that we cannot configure the unit to output 0-250V DC, which is what we'd want for general PZT driving applications. I will contact them to find out more.
The tests were done using the handheld precision voltage source for now. I drove the input between 0 to +5 V and saw an output voltage (at DC) of 0-250 V. This is consistent with the voltage gain being 50V/V as is stated in the manual, but how am I able to get 250 V DC output even though the bipolar configuration is supposed to be +/- 125 V? On the negative side, I am able to see 50V/V gain from 0 to -1 V DC. At which point making the input voltage more negative does nothing to the output. The unit is supposed to accept a bipolar input of +/- 10 V DC or AC, so I'm pretty sure I'm not doing anything crazy here...
Okay based on the markings on the rear panel, the unit is in fact configured for unipolar output. What this means is we will have to map the +/- 10 V DC output from the DAC to 0-5 V DC. Probably, I will stick to 0-2.5 V DC for a start, to not exceed 125 V DC to the PI PZT. I'm not sure what the damage spec is for that. The Noliac PZT I think can do 250 V DC no problem. Good thing I have the inverting summing amplifier coming in tomorrow...
Mr Fred Goodbar of Konacrane was in the lab 830am-1130am today. All three cranes in the VEA were inspected, loaded with 450lb test weights, and declared in good working condition and safe to use.
The interferometer subsystems appear normal after the inspection.
Just a quick set of notes detailing changes so that there are no surprises, more details to follow.
I briefly tried some LO PZT mirror dithering tonight, but didn't see the signal. Needs more troubleshooting.
I lowered the (FAST) PZT gain on the IMC/FSS servo today.
I noticed that the MC locks looked unstable a lot of the day, and during lock the PCDRIVE channel is above 1 Vrms (which means the loop is oscillating, ttypically at the PZT/EOM crossover frequency).
I changed the default setting from 22 to 20 19 dB in the PSL Settings screen so the mcup script will use this for now. Feel free to revert if this turns out to be a Fluke (which you would think is a terrible name for a company, but...)
I set up to do the WFS head modifications today, but I was shot down in flames due to a missing AC/DC adapter.
The Prologix GPIB-ethernet dongle needs +8-13 V to run. Some riff raff has removed the adapter and I was thunderstruck to see that it had not been returned.
I did the usual hunt around the lab looking for something with the right specs and connector. I found one that could do +9V and had the right connector, but it didn't light up the adapter so I put it back in black SP table.
I'll order a couple of these (5 ordered for delivery on Wednesday) in case there's a hot demand for the jack / plug combo that this one has. The setup is in the walkway, but I returned the AS table to the usual state and made sure the IMC is locking well.
Clearly this "riff raff" is referring to me. It won't help today I guess but there is one each on the carts holding the SR785 (currently both in the office/electronics bench area), and the only other unit available in the lab is connected to a Prologix box on the Marconi inside the PSL enclosure.
The electronics chain used to drive the three elements of the PI PZT on which a mirror is mounted with the intention of controlling the LO phase has been changed, to now use the Trek Mode603 power amplifier instead of the OMC high voltage driver. Attachment #1 shows the new configuration.
The text of Attachment #1 contains most of the details. The main requirement was to map the DAC output voltage range, to something appropriate for the Trek amplifier. The latter applies a 50V/V gain to the signal received on its input pin, and also provides a voltage monitor output which I hooked up to an ADC channel in c1ioo. The gain of the interfacing electronics was chosen to map the full output range of the DAC (-5 to +5 V for a single-ended receiving config in which one pin is always grounded) to 0-2.5 V at the input of the Trek amplifier, so that the effective high voltage drive range is 0-125 V. I don't know what the damage threshold is for the PI PZT, maybe we can go higher. The only recommendation given in the Trek manual is to not exceed +/-12 V on the input jack, so I have configured D2000396 to have a supply voltage of 11.5 V, so that in the event of electronics failure, we still don't exceed this number.
On the electronics bench, I tested the drive chain, and also measured the transfer function, see Attachment #2. Seems reasonable (the Trek amplifier was driving a 3uF capacitive load used to protect the SR785 measurement device from any high voltage, hence the roll-off). The gain of D2000396 was changed from 1/8 to 1/4 after I realized that the DAC full range is only +/- 5 V when the receiving device is single-ended at both input and output. Maybe the next iteration of this curcuit should have differential sending, to preserve the range.
To test the chain, I used the single bounce beam from the ITM, and interfered it with the LO. Clear fringing due to the seismic motion of the ITM (and also LO phase noise) is visible. In this configuration, I drove the PZT mirror in the LO path at a higher frequency, hoping to see the phase modulation in the DCPD output. However, I saw no signal, even when driving the PZT with 50% of the full DAC range. The voltage monitor ADC channel is reporting that the voltage is faithfully being sent to the PZTs, and I measured the capacitance of the PZTs (looked okay), so not sure what is going on here. Needs more investigation.
Update Aug 30 5pm: Turns out the problem here was a flaky elbow connector I used to pipe the high voltage to the PI PZT, it had some kind of flaky contact in it which meant the HV wasn't actually making it to the PZT. I rectified this and was immediately able to see the signal. Played around with the dark fringe Michelson for a while, trying to lock the homodyne phase by generating a dither line, but had no success with a simple loop shape. Probably needs more tuning of the servo shape (some boosts, notches etc) and also the dither/demod settings themselves (frequency, amplitude, post mixer LPF etc). At least the setup can now be worked on interferometrically.
My power at home winked out for a second this morning, but it looks like either nothing happened in the 40m lab or else it rode it out.
MC is locked - lost lock around 11:25 AM and then relocked.
Teledyne AP1053 etc were transported from Rich's office to the 40m. The box is placed on the shelf at the entrance.
My record tells that there are 7 AP1053 in the box. I did not check the number this time.
Increasing the compensation capacitance (470 pF now instead of 33 pF) seems to have fixed the oscillation issues associated with this circuit. However, the measured noise is in excess of the model at almost any frequency of relevance. I believe the problem is due to the way the measurement is done, and that we should re-do the measurement once the unit is packaged in a shielded environment.
Attachment #1 shows (schematically) the measurement setup. Main differences from the way I did the last round of testing are:
Attachment #2 shows the measurement results:
I didn't capture the data, but viewing the high voltage output on an Oscilloscope threw up no red flags - the oscillations which were previously so evident were nowhere to be seen, so I think the capacitor switch did the trick as far as stability is concerned.
There is a large excess between measurement and model out to a few kHz, if this is really what ends up going to the suspension then this circuit is useless. However, I suspect at least part of the problem is due to close proximity to switching power supplies, judging by the comb of ~10 Hz spaced peaks. This is a frequent problem in coil driver noise measurements - previously, the culprit was a switching power supply to the Prologix GPIB box, but now a Linear AC-DC converter is used (besides, disconnecting it had no visible effect). The bench supplies providing power to the board, however, is a switching supply, maybe that is to blame? I think the KEPCO supplies providing +/-250 V are linear. I tried the usual voodoo of twisting the wires used to receive the signal, moving the SR785 away from the circuit board etc, but these measures had no visible effect either.
The real requirement of this circuit is that the current noise above 100 Hz be <1pA/rtHz. This measurement suggests a level that is 5x too high. But the problem is likely measurement related. I think we can only make a more informed conclusion after shielding the circuit better and conducting the test in a more electromagnetically quiet environment.
Using a heterodyne measurement setup to track both quadratures, I estimated the relative phase fluctuation between the LO field and the interferometer output field. It may be that a single PZT to control the homodyne phase provides insufficient actuation range. I'll also need to think about a good sensing scheme for controlling the homodyne phase, given that it goes through ~3 fringes/sec - I didn't have any success with the double demodulation scheme in my (admittedly limited) trials.
For everything in this elog, the system under study was a simple Michelson (PRM, SRM and ETMs misaligned) locked on the dark fringe.
This work was mainly motivated by my observation of rapid fringing on the BHD photodiodes with MICH locked on the dark fringe. The seismic-y appearance of these fringes reminded me that there are two tip-tilt suspensions (SR2, SR3), one SOS (SRM) + various steering optics on seismic stacks (6+ steering mirrors) between the dark port of the beamsplitter and the AS table, where the BHD readout resides. These suspensions modulate the phase of the output field of course. So even though the Michelson phase is tightly controlled by our LSC feedback loop, the field seen by the homodyne readout has additional phase noise due to these optics (this will be a problem for in vacuum BHD too, the question is whether we have sufficient actuator range to compensate).
To get a feel for how much relative phase noise there is between the LO field and the interferometer output field (this is the metric of interest), I decided to set up a heterodyne readout so that I can simultaneously monitor two orthogonal quadratures.
Attachment #1 shows the detailed measurement setup. I hijacked the ADC channels normally used by the DCPDs (along with the front-end whitening) to record these time-series.
Attachments #2, #3 shows the results in the time domain. The demodulated signal isn't very strong despite my pre-amplification of the PDA10CF output by a ZFL-500-HLN, but I think for the purposes of this measurement, there is sufficient SNR.
This would suggest that there are pretty huge (~200um) relative phase excursions between the LO and IFO fields. I suppose, over minutes, it is reasonable that the fiber length changes by 100um or so? If true, we'd need some actuator that has much more range to control the homodyne phasethan the single PZT we have available right now. Maybe some kind of thermal actuator on the fiber length? If there is some pre-packaged product available, that'd be best, making one from scratch may be a whole project in itself. Attachment #3 is just a zoomed-in version of the time series, showing the fringing more clearly.
Attachment #4 has the same information as Attachment #2, except it is in the frequency domain. The FFT length was 30 seconds. The features between ~1-3 Hz support my hypothesis that the SR2/SR3 suspensions are a dominant source of relative phase noise between LO and IFO fields at those frequencies. I guess we could infer something about the acoustic pickup in the fibers from the other peaks.
To be continued tomorrow. I think it's a good idea to let the newly installed fiber relax into some sort of stable configuration overnight.
After replacement of the fiber delivering the LO beam to the airBHD setup (some photos here), I repeated the measurement outlined here. There may be some improvement, but overall, conclusions don't change much.
The main addition I made was to implement a digital phase tracker servo (a la ALS), to make sure my arctan2 usage wasn't completely bonkers (the CDS block can be deleted later, or maybe it's useful to keep it, we will see). I didn't measure it today, but the UGF of said servo should be >100 Hz so the attached spectrum should be valid below that (loop has not been done, so above the UGF, the control signal isn't a valid representative of the free running noise). Attachment #1 shows the result. The 1 Hz and 3 Hz suspension resonances are well resolved. Anyways, what this means is that the earlier result was not crazy. I don't know what to make of the high frequency lines, but my guess is that they are electronic pickup from the Sorensens - I'm using clip-mini-grabbers to digitize these signals, and other electronics in that rack (e.g. ALS signals) also show these lines.
It is pretty easy to keep the simple Michelson locked for several minutes. Attachment #2 shows the phase-tracker servo output over several minutes. The y-axis units are degrees. If this is to be believed, the relative phase between the two fields is drifting by 12um ove an hour. This is significantly lower than my previous measurement, while the noise in the ~0.5-10 Hz band is similar, so maybe the shorter fiber patch cable did some good?
I think there is also correlation between the PSL table temperature, but of course, the evidence is weak, and there are certainly other effects at play. At first, I thought the abrupt jumps are artefacts, but they don't actually represent jumps >360 degrees over successive samples, so maybe they are indicative of some real jump in the relative phase? Either fiber slippage or TT suspension jumps? I'll double check with the offline data to make sure it's not some artefact of the phase tracker servo. If you disagree with these conclusions and think there is some meaurement/analysis/interpretation error, I'd love to hear about it.
I have left the heterodyne electronics setup at the LSC rack, but it is not powered (because there are some exposed wires). Please leave it as is.
The vac controls are going down now to pull and test software changes. Will advise when the work is completed.
The vac work is completed. All of the vacuum equipment is now running on the new 120V UPS, except for TP1. The 230V TP1 is still running off wall power, as it always has. After talking with Tripp Lite support today, I believe there is a problem with the 230V UPS. I will post a more detailed note in the morning.
Yesterday's UPS switchover was mostly a success. The new Tripp Lite 120V UPS is fully installed and is communicating with the slow controls system. The interlocks are configured to trigger a controlled shutdown upon an extended power outage (> ~30 s), and they have been tested. All of the 120V pumpspool equipment (the full c1vac/LAN/Acromag system, pressure gauges, valves, and the two small turbo pumps) has been moved to the new UPS. The only piece of equipment which is not 120V is TP1, which is intended to be powered by a separate 230V UPS. However that unit is still not working, and after more investigation and a call to Tripp Lite, I suspect it may be defective. A detailed account of the changes to the system follow below.
Unfortunately, I think I damaged the Hornet (the only working cathode ionization gauge in the main volume) by inadvertently unplugging it while switching over equipment to the new UPS. The electronics are run from multiple daisy-chained power strips in the bottom of the rack and it is difficult to trace where everything goes. After the switchover, the Hornet repeatedly failed to activate (either remotely or manually) with the error "HV fail." Its compatriot, the Pirani SuperBee, also failed about a year ago under similar circumstances (or at least its remote interface did, making it useless for digital monitoring and control). I think we should replace them both, ideally with ones with some built-in protection against power failures.
Four new soft channels per UPS have been created, although the interlocks are currently predicated on only C1:Vac-UPS120V_status.
These new readbacks are visible in the MEDM vacuum control/monitor screens, as circled in Attachment 1:
Yesterday I brought with me a custom power cable for the 230V UPS. It adapts from a 208/120V three-phase outlet (L21-20R) to a standard outlet receptacle (5-15P) which can mate with the UPS's C14 power cable. I installed the cable and confirmed that, at the UPS end, 208V AC was present split-phase (i.e., two hot wires separated 120 deg in phase, each at 120V relative to ground). This failed to power on the unit. Then Jordan showed up and suggested to try powering it instead from a single-phase 240V outlet (L6-20R). However we found that the voltage present at this outlet was exactly the same as what the adapter cable provides: 208V split-phase.
This UPS nominally requires 230V single-phase. I don't understand well enough how the line-noise-isolation electronics work internally, so I can think of three possible explanations:
I called Tripp Lite technical support. They thought the unit should work as powered in the configuration I described, so this leads me to suspect #3.
@Chub and Jordan: Can you please look into somehow replacing this unit, potentially with a U.S.-specific model? Let's stick with the Tripp Lite brand though, as I already have developed the code to interface those.
Unlike our older equipment, which communicates serially with the host via RS232/485, the new UPS units can be connected with a USB 3.0 cable. I found a great open-source package for communicating directly with the UPS from within Python, Network UPS Tools (NUT), which eliminates the dependency on Tripp Lite's proprietary GUI. The package is well documented, supports hundreds of power-management devices, and is available in the Debian package manager from Jessie (Debian 8) up. It consists of a large set of low-level, device-specific drivers which communicate with a "server" running as a systemd service. The NUT server can then be queried using a uniform set of programming commands across a huge number of devices.
I document the full set-up procedure below, as we may want to use this with more USB devices in the future.
First, install the NUT package and its Python binding:
$ sudo apt install nut python-nut
This automatically creates (and starts) a set of systemd processes which expectedly fail, since we have not yet set up the config. files defining our USB devices. Stop these services, delete their default definitions, and replace them with the modified definitions from the vacuum git repo:
$ sudo systemctl stop nut-*.service
$ sudo rm /lib/systemd/system/nut-*.service
$ sudo cp /opt/target/services/nut-*.service /etc/systemd/system
$ sudo systemctl daemon-reload
Next copy the NUT config. files from the vacuum git repo to the appropriate system location (this will overwrite the existing default ones). Note that the file ups.conf defines the UPS device(s) connected to the system, so for setups other than c1vac it will need to be edited accordingly.
$ sudo cp /opt/target/services/nut/* /etc/nut
Now we are ready to start the NUT server, and then enable it to automatically start after reboots:
$ sudo systemctl start nut-server.service
$ sudo systemctl enable nut-server.service
If it succeeds, the start command will return without printing any output to the terminal. We can test the server by querying all the available UPS parameters with
$ upsc 120v
which will print to the terminal screen something like
device.mfr: Tripp Lite
device.model: Tripp Lite UPS
driver.version.data: TrippLite HID 0.81
ups.mfr: Tripp Lite
ups.model: Tripp Lite UPS
Here 120v is the name assigned to the 120V UPS device in the ups.conf file, so it will vary for setups on other systems.
If all succeeds to this point, what we have set up so far is a set of command-line tools for querying (and possibly controlling) the UPS units. To access this functionality from within Python scripts, a set of official Python bindings are provided by the python-nut package. However, at the time of writing, these bindings only exist for Python 2.7. For Python 3 applications (like the vacuum system), I have created a Python 3 translation which is included in the vacuum git repo. Refer to the UPS readout script for an illustration of its usage.
2PM: Arrived at the 40m. Started the work for the coupling of the RF modulated LO beam into a fiber. -> I left the lab at 10:30 PM.
The fiber coupling setup for the phase-modulated beam was made right next to the PSL injection path. (See attachment 1)
Now that the old APC Smart-UPS 2200 is no longer in use by the vacuum system, I looked into whether it can be repurposed for the framebuilder machine. Yes, it can. The max power consumption of the framebuilder (a SunFire X4600) is 1.137kW. With fresh batteries, I estimate this UPS can power the framebuilder for >10 min. and possibly as long as 30 min., depending on the exact load.
@Chub/Jordan, this UPS is ready to be moved to rack 1X6/1X7. It just has to be disconnected from the wall outlet. All of the equipment it was previously powering has been moved to the new UPS. I have ordered a replacement battery (APC #RBC43) which is scheduled to arrive 9/09-11.
On Friday, I grabbed the Zurich Instruments HF2LI lock-in amplifier and brought it home. As time permits, I will work towards developing a similar readout script as we have for the SR785.
Over the last couple of days, I've been working towards getting the infrastructure ready to test out the scheme of sensing (and eventually, controlling) the homodyne phase using the so-called RF44 scheme. More details will be populated, just quick notes for now before I forget.
- Loose fiber coupler: Sorry about that. I could not detect something was loose there, although some of the locks were not tightened.
- S incident instead of P: Sorry about that too. I completely missed that the IMC takes S-pol.
I edited /diskless/root.jessie/home/controls/.bashrc so that I don't have to keep doing this every time I do a model recompile.
Where is this variable set and how can I add the new paths to it?
Attachment #1 shows the optical setup currently being used to send the LO field with RF sidebands on it to the air BHD setup.
Attachment #2 shows spectra of the relative phase drift between LO and IFO output field (from the Dark Michelson).
Attachment #3 shows the signal magntiude of the signals used to make the spectra in Attachment #2, during the observation time (10 minutes) with which the spectra were computed. The dashed vertical lines denote the 1%, 50% and 99% quantiles.
Attempts to close a feeddback loop to control the homodyne phase:
since the summary pages are working again, I was clicking through and noticed that there's a wandering peak in the whitened IMC spectrogram that goes from 10-30 Hz over the course of a day.
anyone know what this is ?
As promised some time ago, I've obtained input noise spectra from the sites calibrated to physical units. They are located in a new subdirectory of the BHD repo: A+/input_noises. I've heavily annotated the notebook that generates them (input_noises.ipynb) with aLOG references, to make it transparent what filters, calibrations, etc. were applied and when the data were taken. Each noise term is stored as a separate HDF5 file, which are all tracked via git LFS.
So far there are measurements of the following sources:
These can be used, for example, to make more realistic Hang's bilinear noise modeling [ELOG 15503] and Yehonathan's Monte Carlo simulations [ELOG 15539]. Let me know if there are other specific noises of interest and I will try to acquire them. It's a bit time-consuming to search out individual channel calibrations, so I will have to add them on a case-by-case basis.
- PSL HEPA was running at 33% and is now at 100%
- South End HEPA was not on and is now running
- Yarm Portable HEPA was not running and is now running at max speed: the power was taken beneath the ITMY table. It is better to unplug it when one uses the IFO.
- Yend Portable HEPA was not running and is now running (presumably) at max speed
Particle Levels: (Not sure about the unit. The convention here is to multiply x10 of the reading)
Before running the HEPAs at their maximum
9/10/2020 15:30 / 0.3um 292180 / 0.5um 14420
(cf 9/5/2020 / 0.3um 94990 / 0.5um 6210)
After running the HEPAs at their maximum
The number gradually went down and now became constant at about half of the initial values
9/10/2020 19:30 / 0.3um 124400 / 0.5um 7410
Turns out what was causing the instability in the aLIGO plots were the lock commands which I forgot to remove before running the simulation. Removing these also made the simulation much faster.
Other than that I improved other stuff in the simulations:
Still need to do:
Feel free to add to the todo list.
The PMC has been unlocked since September 11 sometime (summary pages are flaky again). I re-locked it just now. I didn't mess with the HEPA settings for now as I'm not using the IFO at the moment, so everything should be running in the configuration reported here. The particulate count numbers (both 0.3um and 0.5um) reported is ~x5-8 of what was reported on Thursday, September 10, after the HEPA filters were turned on. We don't have logging of this information in any automated way so it's hard to correlate things with the conditions in Pasadena. We also don't have a working gauge of the pressure of the vacuum envelope.
The RGA scanning was NOT enabled for whatever reason after the vacuum work. I re-enabled it, and opened VM1 to expose the RGA to the main volume. The unit may still be warming up but this initial scan doesn't look completely crazy compared to the reference trace which is supposedly from a normal time based on my elog scanning (the timestamp is inherited from the c0rga machine whose clock is a bit off).
Update 1500: I checked the particle count on the PSL table and it barely registers on the unit (between 0-20 between multiple trials) so I don't know if we need a better particle coutner or if there is negligible danger of damage to optics due to particulate matter.
The unit was repaired and returned to the 40m. Now, with a DMM, I measure a DC offset value that is ~1% of the AC signal amplitude. I measured the TF of a simple 1/20 voltage divider and it looks fine. In FFT mode, the high frequency noise floor levels out around 5-7nV/rtHz when the input is terminated in 50 ohms.
I will upload the repair documents to the wiki.
The "source" output of the SR785 has a DC offset of -6.66 V. I couldn't make this up.
These were delivered to the 40m today and are on Rana's desk
I'll order a couple of these (5 ordered for delivery on Wednesday) in case there's a hot demand for the jack / plug combo that this one has.
There was an abrupt change in the MC_F spectrum between August 4 and August 5, judging by the summary pages - the 1 and 3 Hz resonances are no longer visible in the spectrum. Possibly, this indicates some electronics failure on the MC servo board / whitening board, the CDS settings don't seem to have changed. There is no record of any activity in the elog around those dates that would explain such a change. I'll poke around at 1X2 to see if anything looks different.
Update 1740: I found that the MCL / MCF cables were disconnected. So since August 5, these channels were NOT recording any physical quantity. Because their inputs weren't terminated, I guess this isn't a clean measurement of the whitening + AA noise, but particularly for MC_F, I guess we could use more whitening (see Attachment #1). Probably also means that the wandering ~10-30Hz line in the spectrogram is a electronics feature. The connections have now been restored and things look nominal again.
that's a very curious disconnection
the "Pentek" whitening board that carries the MC channels has jumpers to enable either 1 or 2 stages of 15:150 whitening. Looks lik MC_F has 2 and MC_L has 1.
I guess the MC_F signal is so low because of the high gain on the FSS board. We could lower the FSS common gain and increase the IMC board's VCO gain to make up for this. Maybe 6 dB would be enough. IF that is risky, we could also up the analog gain on the whitening board.
After more trials, I think the phase tracker part used to provide the error signal for this scheme needs some modification for this servo to work.
Attachment #1 shows a block diagram of the control scheme.
I was using the "standard" phase tracker part used in our ALS model - but unlike the ALS case, the magnitude of the RF signal is squished to (nearly) zero by the servo. But the phase tracker, which is responsible for keeping the error signal in one (demodulated) quadrature (since our servo is a SISO system) has a UGF that is dependent on the magnitude of the RF signal. So, I think what is happening here is that the "plant" we are trying to control is substantially different in the acquisition phase (where the RF signal magnitude is large) and once the lock is enabled (where the RF signal magnitude becomes comparitively tiny).
I believe this can be fixed by dynamically normalizing the gain of the digital phase tracking loop by the magnitude of the signal = sqrt(I^2 + Q^2). I have made a modified CDS block that I think will do the job but am opting against a model reboot tonight - I will try this in the daytime tomorrow.
I'm also wondering how to confirm that the loop is doing something good - any ideas for an out-of-loop monitor? I suppose I could use the DCPD - once the homodyne phase loop is successfully engaged, I should be able to drive a line in MICH and check for drift by comparing line heights in the DCPD signal and RF signal. This will requrie some modification of the wiring arrangement at 1Y2 but shouldn't be too difficult...
The HEPAs, on the PSL table and near ITMY, were dialled down / turned off respectively, at ~8pm at the start of this work. They will be returned to their previous states before I leave the lab tonight.
This elog suggests that there is uniformly 1 stage engaged across all channels. I didn't look at the board to see what the jumper situation is, but only 1 stage of whitening is compensated digitally for both _F and _L. The Pomona box attached to the NPRO PZT input is also compensated digitally to convert counts to frequency.
I tried the gain re-allocation between VCO gain and FSS COMM (and also compensated for the cts to Hz conversion in MCF), but it doesn't seem to have the desired effect on the MCF SNR in the 5-50Hz band. Since the IMC stays locked, and I had already made the changes to mcup, I'll keep these gains for now. We can revert to the old settings if the IMC locking duty cycle is affected. Explicitly, the changes made were:
VCO gain: +7dB ---> +13 dB
FSS COMM: +6 ddB ---> +0 dB
The mcdown script wasn't modified, so the lock acquisition gains are the same as they've been.
Assembled is the list of dead pressure gauges. Their locations are also circled in Attachment 1.
For replacements, I recommend we consider the Agilent FRG-700 Pirani Inverted Magnetron Gauge. It uses dual sensing techniques to cover a broad pressure range from 3e-9 torr to atmosphere in a single unit. Although these are more expensive, I think we would net save money by not having to purchase two separate gauges (Pirani + hot/cold cathode) for each location. It would also simplify the digital controls and interlocking to have a streamlined set of pressure readbacks.
For controllers, there are two options with either serial RS232/485 or Ethernet outputs. We probably want the Agilent XGS-600, as it can handle all the gauges in our system (up to 12) in a single controller and no new software development is needed to interface it with the slow controls.
I had to make a CDS change to the c1lsc model in an effort to get a few more signals into the models. Rather than risk requiring hard reboots (typcially my experience if I try to restart a model), I opted for the more deterministic scripted reboot, at the expense of spending ~20mins to get everything back up and running.
Update 2230: this was more complicated than expected - a nuclear reboot was necessary but now everything is back online and functioning as expected. While all the CDS indicators were green when I wrote this up at ~1800, the c1sus model was having frequent CPU overflows (execution time > 60 us). Not sure why this happened, or why a hard power reboot of everything fixed it, but I'm not delving into this.
The point of all this was that I can now simultaneously digitize 4 channels - 2 DCPDs, and 2 demodulated quadratures of an RF signal.