This is strange - I was definitely able to launch medm when I was working on this machine remotely on Friday. But now, there does seem to be a problem with this shared library being missing.
First of all, I installed mlocate to find where the shared library files are installed. Then I made the symlink, and now sitemap seems to work again.
Weirdly, my changes to /etc/resolv.conf got overwritten somehow. Was this machine rebooted? Uptime suggests it's only been running for ~6 hours at the time of writing of this elog.
sudo apt install mlocate
sudo ln -s /usr/lib/x86_64-linux-gnu/libreadline.so.7 /usr/lib/x86_64-linux-gnu/libreadline.so.6
when I try 'sitemap' on rossa I get:
medm: error while loading shared libraries: libreadline.so.6: cannot open shared object file: No such file or directory
Indeed, this is now fixed by following instructions from here. I rebooted rossa at ~1250 PDT and confirmed that resolv.conf didn't get overwritten. The resolv.conf file also now has the following useful lines at the head:
# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
# DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
yes, I rebooted yesterday to fix the 'steaking white lines' problem in the video/display
maybe we're supposed to edit something besides resolv.conf since that gets over-written on boot for some linux OS
Last Tuesday evening, while attempting the PRFPMI locking, I noticed a strange feature in the LSC signals, which is shown in Attachment #1 (the PDF exported by dataviewer is 14MB so I upload the jpeg instead). As best as I can tell, the REFL33 and POP22 channels show an abrupt jump in the signal levels, while the other channels do not. POP110 shows a slight jump at around the same time, and the large excursion in AS110_Q actually occurs a few seconds later, and is probably some angular excursion of the PRC/BS. I'm struggling to interpret how this can be explained by some interferometric mechanism, but haven't come up with anything yet. The LO for the 3f error signals is the 2f field, but then why doesn't the POP110 channel show a similar jump if there is some abrupt change in the resonant condition? Is such a change even feasible from a cavity length change point of view? Or did the sideband frequency somehow abruptly jump? But if so, why is the jump much more clearly visible in one sideband than the other?
Does anyone have any ideas as to what could be going on here? This may give some clue as to what's up with the weird sensing matrices, but may also be something boring like broken electronics...
I want to be able to run the dither alignment servo with the PRFPMI locked - I've been thinking about what the scheme should be, and I list here some questions I had while thinking about this.
I wanted to try using rossa as my locking workstation today. However, a few problems became quickly evident. Basically, any of our scripts that rely on the cdsutils package (there are MANY) will not work on rossa, because of some library error. This machine is running Debian 10, while the cdsutils package is being loaded from a pre-compiled install on the shared drive, so perhaps this isn't surprising?
Digging a little more, I found that actually, a version of cdsutils that actually works with python3 is actually shipped with the standard cds-workstation meta-package. This is great news, and we should try and use this where possible I guess. Deferring further debugging for daytime work.
Anyway, I added a symlink: sudo ln -s /usr/lib/x86_64-linux-gnu/libncurses.so.6 /usr/lib/x86_64-linux-gnu/libncurses.so.5, and installed wmctrl using sudo apt install wmctrl.
I noticed these streaky lines again today (but they were not a problem last night). It is annoying if we have to reboot this machine all the time. I wonder if this has something to do with missing drivers. When I ran sudo apt update && sudo apt upgrade, I got several lines like (this isn't the whole stack trace)
W: Possible missing firmware /lib/firmware/nvidia/gp108/acr/ucode_unload.bin for module nouveau
W: Possible missing firmware /lib/firmware/nvidia/gp108/acr/ucode_load.bin for module nouveau
W: Possible missing firmware /lib/firmware/nvidia/gp108/acr/unload_bl.bin for module nouveau
W: Possible missing firmware /lib/firmware/nvidia/gp108/acr/bl.bin for module nouveau
Is this indicative of the graphics drivers being installed incorrectly? I am hesitant to mess with this because I think in the past, it was always trying to update some graphics driver that crashed the whole machine into some weird state where we have to wipe the drive and do a fresh re-install of the OS.
Should we just follow these instructions? The graphics card is apparently Quadro P400, which is one of the supported ones according to the list of supported devices.
Or just swap donatella and rossa monitors and defer the problem for later?
We can probably learn something about the interferometer / top level BHD plan with an in-air BHD setup, even if the noise is bad. Here are some thoughts about how we would do it.
For this first attempt, we don't really care about the PRC filtering. So possible places to pick off an LO beam are:
In all cases, I think the easiest option to actually route whatever beam we choose into a fiber, and then bring it over to whatever cavity we choose to use for an OMC. I'm assuming whatever phase control technique we end up using can cancel the fiber phase noise at relevant frequencies.
LO phase control
There is a question about the range, but I think these are the only two realistic options we can implement on a reasonable time scale.
Again, there are a few options. Here are some pros and cons that come to my mind.
If we can do a vent (we'd just need a single chamber open), I'd go for the option of getting the copper OMC out and using that. Attachment #1 shows the approximate sizes of the various components (OMMT, OMC cavity, DCPDs), while Attachment #2 shows a rough sketch of where things would go on the AP table, with the rectangles approximately to scale.
I'd made a c1omc model sometime ago. Basically, I think we have sufficient ADC/DAC channels in the c1ioo machine for any of the options listed above - but using the copper OMC and associated peripherals would allow the easiest interfacing.
More tomorrow, but I tried the following tonight:
I was looking at some signals from last night, see Attachment #1.
Attachment #2 shows some ASC metrics. My conclusion here is that running the PRCL and MICH dither alignment servos (former demodulating REFLDC and latter demodulating ASDC to get an error signal) that running the dither alignment servo and hand tuning the arm ASC loop offsets improves the mode matching to the IFO, because:
The REFLDC behavior needs a bit more interpretation I think, because if the IFO is overcoupled (as I claim it is), then better alignment would at some point actually result in REFLDC increasing.
All the DC signals recorded by the fast system come from the backplane P2 connector of the PD interface boards. According to the schematic, these signals have a voltage gain of 2. The LSC photodiodes themselves have a nominal DC gain of 50 ohms. So, the conversion from power to digital counts is: 0.8 A/W * 50 V/A * 2 * 3276.8 cts/V * whtGain. Inverting, I get 3.8 uW/ct for a whitening gain of 1. This is power measured at the photodiode - optical losses upstream of the photodiode will have to be accounted for separately.
Assuming a modulation depth of 0.2, the 55 MHz sideband power should be ~20 mW. The Schnupp asymmetry is supposed to give us O(1) transmission of this field to the AS port. Then, the SRM will attenuate the field by a factor of 10, so we expect ~2 mW at the AS port. Let's assume 80 % throughput of this field to the AP table, and then there is a 50/50 beamsplitter dividing the light between the AS55 and AS110 photodiodes. So, we expect there to be ~700 uW of power in the TEM00 mode 55 MHz sideband field. This corresponds to 1600 cts according to the above calibration (the ASDC whitening gain is set to 18 dB). The fact that much smaller numbers were seen for ASDC indicates that (i) the schnupp asymmetry is not so perfectly tuned and the actual transmission of the sideband field to the dark port is smaller, or (ii) one or more optical splitting fractions assumed above is wrong. If the former is true, we can still probably infer the contrast defect if we can somehow get an accurate measurement of the sideband transmission to the dark port.
After some consultation with Erik von Reis at LHO, this workstation is progressing towards being usable for most commissioning tasks. DTT, awggui, foton, and MEDM are all now working well. The main limitation now comes from the fact that many of our python scripts are written for python2, and rossa doesn't have many dependencies installed for python2. I see no reason to build these dependencies on rossa for python2, we should not have to work with an unsupported language. But at the same time, I don't want to completely wipe all our python2 scripts, and make them python3, because this would involve a lot of tedious testing that I'm not prepared to undertake at the moment (the problem is compounded by the fact that pianosa does not have many dependencies installed for python3).
So what I have done in the interim is make python3 versions of the most important scripts I need to get the PRFPMI locking working - they are in the scripts directory and have the same names as their python2 counterparts, but have a 3 appended to their names. So when working on rossa, these are the scripts that are called. Eventually, after a lot more testing, we can depracate the old scripts. Currently, where applicable, the MEDM screens allow for either the python2 or python3 version of the script to be called.
Please, for the time being, do not try and install any new packages on rossa unless you are prepared to debug any problems caused and return the machine to a workable state. If you find some issue with a missing package on rossa, (i) make a note of it on the elog, and (ii) if possible, set up your own conda environment for testing and install dependencies to that environment only.
Main goals tonight were:
After some hunting, I found this old SURF report with the WFS head measurements. The y-axes don't make much sense to me, and I can't find the actual data anywhere (her wiki page doesn't actually exist). So I think it's still unknown if these heads ever had the advertised transimpedance gain, or if the measured transimpedance of ~1kohm was what it always was.
In fact, all these utilities are now available in python3. There may be some bugs (e.g. this), but I've checked basic functionality and things look usable enough for development to proceed. While we can have a python2 env on rossa, I think it's unnecessary.
I too, would prefer py3 for everything, but aren't all the cdsutils / gaurdian things still python2?
I tried using the POX_I error signal for the DC CARM_B path today a couple of times. Got to a point where the AO path could be engaged and the arm powers stabilized somewhat, but I couldn't turn the CARM_A path off without blowing the lock. Now the IMC has entered a temperemental state, so I'm abandoning efforts for tonight, but things to try tomorrow are:
I have some data from a couple of days ago when the PRFPMI was locked as usual (CARM_B on REFL for both DC and AO paths), and the sensing lines were on, so I can measure the relative strength of the sensing lines in POX/REFL and get an estimate of what the correct digital gain should be.
The motivation here is to see if the sensing matrix looks any different with a modified locking scheme.
For the first pass, it's probably easiest to use the existing DCPD amplifier. Looking at the gain and noise performance in Attachment #1, seems totally fine, the electronics noise will not be limiting if we have ~10mW of LO power. I assumed a transimpedance resistor of 1 kohm, and all other numbers as on the schematic (though who knows if the schematic is accurate). The noise should be measured to confirm that the box is performing as expected...
Koji and I had a discussion last Friday about the suspension electronics. I think there are still a few open questions - see Attachment #1. We should probably make a decision on these soon.
Other useful links:
From Attachment #1, looks like the phasing and gain for CARM on POX11 is nearly the same as CARM of REFL11, which is probably why I was able to execute a partial transition last night. The response in POY11 is ~10 times greater than POX11, as expected - though the two photodiodes have similar RF transimpedance, there is a ZFL-500-HLN at the POY11 output. The actual numerical values are 2.5e10 cts/m for CARM-->REFL11_I, 2.6e10 cts/m for CARM-->POX11_I, and 3.2e11 cts/m for CARM-->POY11_I.
So I think I'll just have to fiddle around with the transition settings a little more tonight.
One possible concern is that the POX and POY signals are digitized without preamplificatio, maybe this explains the larger uncertainty ellipse for the POX and POY photodiodes relative to the REFL11 photodiode? Maybe the high frequency noise is worse and is injecting junk in the AO path? I think it's valid to directly compare the POX and REFL spectra in Attachment #2, without correcting for any loops, because this signal is digitized from the LSC demodulator board output (not the preamplified one, which is what goes to the CM board, and hence, is suppressed by the CARM loop). Hard to be sure though, because while the heads are supposed to have similar transimpedance, and the POX photodiode has +12dB more whitening gain than REFL11, and I don't know what the relative light levels on these photodiodes are in lock.
I have some data from a couple of days ago when the PRFPMI was locked as usual (CARM_B on REFL for both DC and AO paths), and the sensing lines were on, so I can measure the relative strength of the sensing lines in POX/REFL and get an estimate of what the correct digital gain should be
I forgot about the pointing - probably we will need another actuator to control the pointing of the AS beam onto the DCPDs. I found a few old PI PZTs (model number is S-320, which is a retired part), one is labelled broken but the others don't indicate a-priori that they are broken. I'll post a more detailed hardware survey later.
You can activate all 3axis
The emergency lamps above the exit sign on the NW entrance to the control room are on. I tried opening and closing the door, but it remains on. Probably nothing to worry about, but noting here anyway.
True - it is now not on anymore.
It happened before too. Doesn't it say it has occasional self-testing or something?
To facilitate this investigation, I've DQed the 4 face coil outputs for the two ETMs. EX is currently running with 5 times the series resistance of EY, so it'll be a nice consistency check. Compilation, installation etc went smooth. But when restarting the c1scx model, there was a weird issue - the foton file, C1SCX.txt, got completely wiped (all filter coefficients were empty, even though the filter module names themselves existed). I just copied the chiara backup version, restarted the model, and all was well again.
This corresponds to 8 additional channels, recorded at 16k as float 32 numbers, so in the worst case (neglecting any clever compression algorithms), we are using disk space at a rate of ~4 MB/s more. Seems okay, but anyway, I will remove these DQ channels in a few days, once we're happy we have enough info to inform the coil driver design.
spoke too soon - there was an RFM error for the TRX channel, and restarting that model on c1sus took down all the vertex FEs. Anyways, now, things are back to normal I think. The remaining red light in c1lsc is from the DNN model not running - I forgot to remove those channels, this would've been a good chance! Anyways, given that there is an MLTI in construction, I'm removing these channels from the c1lsc model, so the next time we restart, the changes will be propagated.
For whatever reason, my usual locking scripts aren't able to get me to the PRFPMI locked state - some EPICS channel value must not have been set correctly after the model reboot 😞. I'll debug in the coming days.
Fun times lie ahead for getting the new BHD FEs installed I guess 🤡 ....
Looking at signals to the ETMs from the current lock acquisition sequence, the RMS current to a single coil is approximately _____ (to be filled in later).
So we may need a version of the fast coil driver that supports a low noise mode (with large series resistance) and a high-range mode (with lower series resistance for lock acquisition).
Attachment #1 - The 80mW pickoff was getting clipped on a BNC cable, and not making it to the doubling oven. 😢 .
Attachment #2 - PSL green shutter removed. Alignment into the doubling oven is extremely tedious, and so I opted to preserve the capability of recovering the green beam by simply removing a single mirror.
Attachment #3 - The beam path for coupling the LO beam into a fiber.
Attachment #4 shows the BHD photodiodes taken from QIL.
The (masked) tech accessed all areas in the lab (office area, control room, VEA) between ~230pm-3pm. The laser safety goggles he used have been kept aside for appropriate sanitaiton.
The DQ channels of the ETM coils were active tonight, so I'll make the coil driver actuation budget over the next couple of days.
Attachment #1 shows the proposed wiring and CDS topology for the in air BHD setup. The PDF document has hyperlinks you can follow to the DCC entries. Main points:
Please comment if I've overlooked something.
Looking at the signals to the test mass coils, it seems borderline to me that we will be able to acquire lock and run in a low noise configuration with the same series resistor in the coil driver circuit. The way I see it, options are:
I only looked at the ETMs for this study. The assumption is that we will have no length actuation on the ITMs, only local damping and Oplev loops (and maybe some ASC actuation?), which can be sufficiently low-pass filtered such that even with coil de-whitening, we won't have any range issues.
Attachment #1 shows the time-domain traces of the coil driver signals as we transition from POX/POY lock to the ALS lock. There are some transients, but I think we will be able to hold the lock even with a 5 kohm resistor (~twice what is on ETMX right now). From just these numbers, it would seem we can even go up to 10 kohms right away and still be able to acquire lock, especially if we re-design the digital feedback loop to have better low-pass filtering of the high-frequency ALS noise, see the next attachment.
Attachment #2 shows the f-domain picture, once the arm lengths are fully under ALS control (~25 seconds onwards in Attachment #1). The RMS is dominated by high frequency ALS length loop noise, which we can possibly improve with better design of the digital control loop.
Finally, Attachment #3 shows the situation once DARM control has been transitioned over to AS55_Q. Note that the vertex DoFs are still under 3f control, so there is the possibility that we can make this even lower noise. However, one thing that is not factored in here is that we will have to de-whiten these signals to low-pass filter the DAC noise (unless there is some demonstrated clever technique with noise-mons or something to subtract the DAC noise digitally). Nevertheless, it seems like we can run safely with 5 kohms on each ETM coil and still only use ~2000 cts RMS, which is ~1/10th the DAC range (to allow for dealing with spurious transients etc).
The LO pickoff has been coupled into a fiber with ~90% MM (8 mW / 9 mW input). While I wait for the DCPD electronics to be found in the Cryo lab, I want to monitor the stability of the pointing, polarization etc, so I'd like to clear some space on the AP table that was occupied for the mode spectroscopy project. If there are no objections before 2pm tomorrow July 21 2020, I will commence this work.
Attachment #1 shows the RIN of the local oscillator beam delivered to the AP table via fiber. I used a PDA520 to make this measurement, while the electronics for the DCPDs are pending. I don't really have an explanation for the difference between the locked IFO trace vs the not locked trace - we don't have an ISS running (but this first test suggests we should) and the beam is picked off before any cavities etc, so this is a reflection of the state of the FSS servo at the times of measurement?
Tried locking CARM using the hybrid REFL (for AO path) and POX 11 (for MCL path) scheme a bunch of times today, but I had no luck. When the CARM offset is zeroed, the PRMI lock is lost almost immediately. Maybe this is indicative of some excess noise in the POX data stream relative to the REFL signal? The one thing I haven't tried is to take the IFO all the way to the locked state, and then transition the MCL actuation from CM_SLOW to POX11_I.
An SR785 is sitting on the North side of the AP table in the walkway - I will clear it tomorrow.
I decided to use the old EY auxiliary optics table, which is now stored along the east arm about 10 m from the end, as a workspace for assembling the little PMCs. I wiped everything down with isopropanol for general cleanliness, removed the metal plate on the south edge of the table enclosure to allow access, covered the table with some clean Aluminium foil, and then moved the plastic box with PMC parts to the table - see Attachment #1. I haven't actually done any assembly just yet, waiting for more info (if available) on the procedure and implements available...
This earthquake and friends had tripped all watchdogs. I used the scripted watchdog re-enabler, and released the stuck ITMX (this operation is still requires a human and hasn't been scripted yet). IMC is locked again and all Oplevs report healthy optic alignment.
Be aware that there is now a KEPCO HV supply that is energized, sitting on the floor immediately adjacent to the OMC rack, east of the AP table. It is currently set to 100 V DC, and a PI PZT installed on the AP table has its 3 PZTs energized by said supply (via an OMC piezo driver). I will post pictures etc of the work from the last 10 days over the weekend.
All the details are in E2000436, and documents linked from there, I think an elog would be much too verbose. In summary, a workable setup consisting of
Last night, I locked the PRMI with the carrier resonant, and convinced myself that the DCPD null stream was sensing the MICH degree of freedom (while it was locked on AS55_Q) with good SNR below ~60 Hz. Above ~60 Hz, in this configuration, the ADC noise was dominating, but by next week, I'll have a whitening board installed that will solve this particular issue. With the optical gain of MICH in this configuration, the ADC noise level was equivalent to ~500 nrad/rtHz of phase noise above ~60 Hz (plots later).
Now, I can think about how to commission this setup interferometrically.
Some tests done today:
All of these tests were done with the PRMI locked with carrier resonant in the recycling cavity (i.e. sidebands rejected to REFL port). I then actuated the BS length DOF with a sine wave at 311.1 Hz, 40 cts amplitude (corresponding to ~8 pm of peak-to-peak displacement).
While it would seem from these graphs that the RIN of the LO beam at these frequencies is rather high, it is because of the ADC noise. More whitening (to be installed in the coming days) will allow us to get a better estimate, should be ~1e-6 I think.
I was just playing today, still need to setup some more screens, DTT templates etc to do more tests in a convenient way.
See Attachment #1. J8 was connected to a "LASTI timing slave" sitting in the rack that Chiara lives in - we don't use this for anything and I confirmed that there was no effect on the RTCDS when I pulled that fiber out. The LASTI timing slave also had a blinky that was blinking when the fiber was plugged in - which I take to believe that the slot works.
Can we get away with just using these two available slots, J8 and J13? Do we really need three new expansion chassis?
See Attachments #1 and #2. We don't have any Q3000 QPDs in hand, at least not in the photodiode box stored in the clean optics cabinet at the south end. I also checked a cabinet along the east arm where we store some photodiodes - but didn't find any there either. The only QPDs we have in hand are the YAG-444-4AH, which I believe is what is used in the iLIGO WFS heads.
So how many do we want to get?
The "source" output of the SR785 has a DC offset of -6.66 V. I couldn't make this up.
Upshot is, this SR785 is basically not usable for TF measurements. I was using the unit to characterize the newly stuffed ISC whitening board. The initial set of measurements were sensible, and at some point, I started getting garbage data. Unclear what the cause of this is. AFAIK, we don't have any knob to tune the offset - adjusting the "offset" in the source menu, I can change the level of the offset, but only by ~1 V even if I apply an offset of 10 V. I also tried connecting the ground connection on the rear of the SR785 to the bench power supply ground, no change.
Do we have to send this in for repair?
That's great. I wonder if we can also get away with not adding new Dolphin infrastructure. I'd really like to avoid changing any IPC drivers.
I believe we will use two new chassis at most. We'll replace c1ioo from Sun to Supermicro, but we recycle the existing timing system.
There was a power outage ~30 mins ago that knocked out CDS, PSL etc. The lights in the office area also flickered briefly. Working on recovery now. The elog was also down (since nodus presumably rebooted), I restarted the service just now. Vacuum status seems okay, even though the status string reads "Unrecognized".
The recovery was complete at 1830 local time. Curiously, the EX NPRO and the doubling oven temp controllers stayed on, usually they are taken out as well. Also, all the slow machines and associated Acromag crates survived. I guess the interruption was so fleeting that some devices survived.
The control room workstation, zita, which is responsible for the IFO status StripTool display on the large TV screen, has some display driver issues I think - it crashed twice when I tried to change the default display arrangement (large TV + small monitor). It also wants to update to Ubuntu 18.04 LTS, but I decided not to for the time being (it is running Ubuntu 16.04 LTS). Anyways, after a couple of power cycles, the wall StripTools are up once again.
I added the EPCIS channels for the c1omc model (gains, matrix elements etc) to the autoburt such that we have a record of these, since we expect these models to be running somewhat regularly now, and I also expect many CDS crashes.
Gabriele left the DataRay beam profiler + peripherals (see Attachment #1) in his office. I picked them up just now and brought them over to the 40m.
A technician came to the lab today at ~4pm. He entered the VEA (with booties and googles), and also the clean and bake lab. The whole procedure lasted ~10 minutes. I did not follow him around, but was available in the control room throughout the process. I think the whole episode went without incident.
BTW, this guy didn't ring the doorbell, I just happened to be here when he came by. I don't know if this is usual practise - are we happy with the technicians entering the VEA and/or clean and bake labs without supervision? AFAIK, this wasn't scheduled.
I finally managed to install a differential-receiving whitening board in 1Y2 - 4 channels are available at the moment. As I claimed, one stage of 15:150 Hz z:p whitening does improve the ALS noise a little, see Attachment #1. While the RMS (from 1kHz-0.5 Hz) does go down by ~10 Hz, this isn't really going to make any dramatic improvement to the 40m lock acquisiton. Now we're really sitting on the unsuppressed EX laser noise above ~30 Hz. This measurement was taken with the arm cavities locked with POX/POY, and end lasers locked to the arm cavities with uPDH boxes as usual. This was just a test to confirm my suspicion, the whitening board is to be used for the air BHD channels, but when we get a few more stuffed, we can install it for the ALS channels too.
With the chosen transimpedance of 300 ohms, in order to be able to see the shot noise of 10 mW of light in the digitized data streams, we'd need all 3 stages of whitening. If we want to be shot noise limited with 1 mW of LO light, we'd need to increase said transimpedance I think.
The measurements were taken with
Of course, it's unlikely we're going to be shot noise limited for any configuration in the short run. But this was also a test of
All 3 tests passed.
A single channel of this board was stuffed (and other channels partially populated). The basic tests passed, and nothing exploded! Even though this is a laughably simple circuit, it's nice that it works.
HV power supplies:
A pair of unused KEPCO BHK300-130 switching power supplies that I found in the lab were used for this test. I pulled the programmable cards out at the rear, and shorted the positive output of one unit to the negative of the other (with both shorted to the supply grounds as well), thereby creating a bipolar supply from these unipolar models. For the purposes of this test, I set the voltage and current limits to 100V DC, 10mA respectively. I didn't ramp up the supply voltage to the rated 300 V maximum. The setup is shown in Attachment #1.
Need to think more about how to better characterize this noise. An estimate of the required actuation can be found here.
The mode-matching between the LO and AS beams is now ~50%. This isn't probably my most average mode-matching in the lab, but I think it's sufficient to start doing some other characterization and we can try squeezing out hopefully another 20-30% by putting the lenses on translation stages, tweaking alignment etc.
The main change was to increase the optical path length of the IFO AS path, see Attachment #1. This gave me some more room to put a lens and translate it.
Try the PRMI experiments again, now that I have some confidence that the beams are actually interfering.
See Attachment #3 for the updated spectra - the configuration is PRMI locked with carrier resonant and the homodyne phase is uncontrolled. There is now much better clearance between the electronics noise and the MICH signal as measured in the DCPDs. The "LO only" trace is measured with the PSL shutter closed, so the laser frequency isn't slaved to the IMC length. I wonder why the RIN (seen in the SUM channel) is different whether the laser is locked to the IMC or not? The LO pickoff is before the IMC.
A more careful analysis has revealed some stability problems. I see oscillations at frequencies ranging from ~600kHz to ~1.5 MHz, depending on the voltage output requested, of ~2 V pp at the high-voltage output in a variety of different conditions (see details). My best guess for why this is happening is insufficient phase margin in the open-loop gain of the PA95 high voltage amplification stage, which causes oscillations to show up in the closed loop. I think we can fix the problem by using a larger compensation capacitor, but if anyone has a better suggestion, I'm happy to consider it.
The changes I wanted to make to the measurement posted earlier in this thread were: (i) to measure the noise with a load resistor of 20 ohms (~OSEM coil resistance) connected, instead of the unloaded config previously used, and (ii) measure the voltage noise on the circuit side (= TP5 on the schematic) with some high voltage output being requested. The point was to simulate conditions closer to what this board will eventually be used in, when it has to meet the requirement of <1pA/rtHz current noise at 100 Hz. The voltage divider formed by the 25 kohm series resistor and the 20 ohm OSEM coil simulated resistance makes it hopeless to measure this level of voltage noise using the SR785. On the other hand, the high voltage would destroy the SR785 (rated for 30 V max input). So I made a little Pomona box to alllow me to do this measurement, see Attachment #1. Its transfer function was measured, and I confirmed that the DC high voltage was indeed blocked (using a Fluke DMM) and that the output of this box never exceeded ~1V, as dictated by the pair of diodes - all seemed okay .
Next, I wanted to measure the voltage noise with ~10mA current flowing through the output path - I don't expect to require more than this amount of current for our test masses. However, I noticed some strange features in the spectrum (viewed continuously on the SR785 using exponential averaging setting). Closer investigation using an oscilloscope revealed:
Some literature review suggested that the capacitor in the feedback path, C4 on the schematic, could be causing problems. Specifically, I think that having that capacitor in the feeddback path necessitates the use of a larger compensation capacitor than the nominal 33pF value (which itself is higher than the 4.7pF recommended on the datasheet, based on experience of the ESD driver circuit which this is based on, oscillations were seen there too but the topology is a bit different). As a first test of this idea, I removed the feedback capacitor, C4 - this seemed to do the trick, the oscillations vanished and I was able to drive the output between the high voltage supply rails. However, we cannot operate in this configuration because we need to roll off the noise gain for the input voltage noise of the PA95 (~6 nV/rtHz at 100 Hz will become ~200 nV/rtHz, which I confirmed using the SR785). Using a passive RC filter at the output of the PA95 (a.k.a. a "snubber" network) is not an option because we need to sum in the fast actuation path voltage at the output of the 25 kohm resistor.
Some modeling confirms this hypothesis, see Attachment #2. The quantity plotted is the open-loop gain of the PA95 portion of the circuit. If the phase is 0 degrees, then the system goes unstable.
So my plan is to get some 470pF capacitors and test this idea out, unless anyone has better suggestions? I guess usually the OpAmps are compensated to be unconditionally stable, but in this case maybe the power op-amp is more volatile?
Listing some talking points from the last week of activity here.
I found that the control MEDM screen was left open on the c1vac workstation. This should be closed every time you leave the workstation, to avoid accidental button pressing and such.
The network outage meant that the EPICS data from the pressure gauges wasn't recorded until I reset everything ~noon. So there isn't really a plot of the outgassing/leak rate. But the pressure rose to ~2e-4 torr, over ~4 hours. The pumpdown back to nominal pressure (9e-6 torr) took ~30 minutes.
Attachment #1 is a summary of the current to each coil on the suspensions. The situation is actually a little worse than I remembered - several coils are currently drawing in excess of 10mA. However, most of this is due to a YAW correction, which can be fixed somewhat more easily than a PIT correction. So I think the circuit with a gain of 31 for an input range of +/-10 V, which gives us the ability to drive ~12mA per coil through a 25kohm series resistor, will still provide sufficient actuation range. As far as the HV supplies go, we will want something that can do +/- 350 V. Then the current to the coils will at most be ~50 mA per optic. The feedback path will require roughly the same current. The quiescent draw of each PA95 is ~10mA. So per SOS suspension, we will need ~150mA.
If it turns out that we need to get more current through the 25kohm series resistance, we may have to raise the voltage gain of the circuit. Reducing the series resistance isn't a good option as the whole point of the circuit is to be limited by the Johnson noise of the series resistance. Looking at these numbers, the only suspension on which we would be able to plug in a HV coil driver as is (without a vent to correct for YAW misalignment) is ITMY.
Update 2 Sep 2020 2100: I confirmed today that the number reported in the EPICS channel, and the voltage across the series resistor, do indeed match up. The test was done on the MC3 coil driver as it was exposed and I didn't need to disable any suspensions. I used a Fluke DMM to measure the voltage across the resistor. So there is no sneaky factor of 2 as far as the Acromag DACs are concerned (unlike the General Standards DAC).