Summary:

The mixer + LPF combo used to demodulate the PMC PDH error signal seems to work as advertised.

Details:

Measurement setup --- Attachment #1. The IF signal was monitored using the scope in High-Z mode.

Results --- Attachment #2.

So the next step is to characterize the RF transimpedance of the PMC RFPD.

[jordan, gautam]

Summary:

The AD602 chip which implements the overall servo gain for the PMC seems to be damaged. We should switch this out at the next opportunity.

Details:

1. According to the PSL cross connect wiring diagram, the VME DAC that provides the control voltage to the VGA stage goes to pins 7/8 of cross connect J16.
   • Jordan and I verified that the voltage at this point [Vout], is related to the PMC_GAIN EPICS slider [dB] value according to the following relationship: .
2. On the PMC servo board, this voltage is scaled by a factor of -1/10. 
   • This was confirmed by peeking at this voltage using a DMM (I clipped onto R31) while the gain slider was varied.
   • This corresponds to +/- 1000 mV reaching the AD602.
   • However, the AD602 is rated to work with a control voltage varying between +/- 625 mV.
   • What this means is that the EPICS slider value is not the gain of the AD602 stage. The latter is given by the relation .
   • @team PSL upgrade: this should be fixed in the database file for the new c1psl machine.
3. Using TP1 and TP2 connected to the SR785, I measured the transfer function of the AD602 for various values of the EPICS slider.
   • Result is shown in Attachment #1.
   • I did this measurement with the PMC locked, so I'm using the in-loop error signal to infer the gain of the VGA stage.
   • As expected, the absolute value of the gain does not match that of the EPICS slider (note that the AD602 has an input impedance of 100 ohms. So the 499 ohm series resistor between TP1 and the input of AD602 makes a 1/5 voltage divider, so the gain seen between TP1 and TP2 has this factor folded in).
   • Moreover, the relative scaling of the gain for various slider values also doesn't appear to be liner.
   • For the highest gain setting of +15 dB, the servo began oscillating, so I think the apparent non-flatness of the gain as a function of frequency is an artefact of the measurement.
   • Nevertheless, my conclusion is that the IC should be changed.

I will pull the board and effect the change later today.

I pulled the board out at 345pm after dialling down all the HV supplies in 1X1. I will reinstall it after running some tests.

doesn't seem so anomolous to me; we're getting ~25 dB of gain range and the ideal range would be 40 dB. My guess is that even thought this is not perfect, the real problem is elsewhere.

Summary:

The RF transimpedance of the PMC PDH RFPD was measured, and found to be 1.03 kV/A. 

Details:

With the new fiber coupled PDFR system, it was very easy to measure the response of this PD in-situ 🎉 . The usual transfer function measurement scheme was used, with the AG4395 RF out modulating the pump current of the diode laser, and the measured transfer function being the ratio of the response of the test PD to the reference PD.

I assume that the amount of light incident on the reference NF1611 photodiode and the test photodiode were equal - I don't know what the DC transimpedance of the PMC REFL photodiode is (can't find a schematic), but the DC voltage at the DC monitor point was 16.4 mV (c.f. -2.04 V for the NF1611). The assumption shouldn't be too crazy because assuming the reference PD has an RF transimpedance of 700 V/A (flat in the frequency range scanned), we get a reasonable shape for the PMC REFL photodiode's transimpedance.

The fitted parameters are overlaid in Attachment #1. The 2f notch is slightly mistuned it would appear, the ratio of transimpedance at f1/2*f1 is only ~10. The source files have been uploaded to the wiki.

Knowing this, the measured PDH discriminant of 0.064 GV/m is quite reasonable:

• expected optical gain from modulation depth assuming a critically coupled cavity is 0.089 GW/m.
• Assume 0.7 A/W responsivity for InGaAs.
• Account for the fact that only 0.8 % of the reflected light reaches the PMC photodiode because of the pickoff window.
• Account for a conversion loss of 4.5 dB in the mixer.
• Account for the voltage division by a factor of 2 at the output of the BLP-1.9 filter due to the parallel 50 ohm termination.
• Then, the expected PDH discriminant is 0.089e9 W/m * 0.7 A/W * 0.8e-2 * 1.03kV/A * 10^(-4.5/20) * 0.5 ~ 0.15 GV/m. This is now within a factor of ~2 of the measured value, and I assume the total errors in all the above assumed parameters (plus the cable transmission loss from the photodiode to the 1X1 rack) can easily add up to this. 

So why is this value so different from what Koji measured in 2015? Because the monitor point is different. I am monitoring the discriminant immediately after the mixer, whereas Koji was using the front panel monitor. The latter already amplifies the signal by a factor of x101 (see U2 in schematic). 

Conclusion:

I still haven't found anything that is obviously wrong in this system (apart from the slight nonlinearity in the VGA stage gain steps), which would explain why the PMC servo gain has to be lower now than 2018 in order to realize the same loop UGF.

While I have the board out, I'll try and do a thorough investigation of TFs and noise of the various stages. There is no light into the IFO until this is done.

To facilitate wiring the c1psl chassis and scripting loopback tests, I've compiled a distilled spreadsheet with the Acromag-to-breakout board wiring, broken down by connector. This information is extractable from the master spreadsheet, but not easily. There were also a few apparent typos which are fixed here.

The wiring assignments at the time of writing are attached below. Here is the link to the latest spreadsheet.

The PMC servo was re-installed at ~345pm. HV supplies were re-energized to their nominal values. I will update the results of the investigation shortly. The new nominal PMC servo gain is +9dB.

Zero order beam PMC ringdown

On Wednesday I installed 3 PDs (see attached photos) measuring: 

1. The input light to the PMC. Flip-mirror was installed (sorry Shruti) on the beam path to the fiber coupler.

2. Reflected light from the PMC.

3. PMC transmitted light.

I connected the three PDs to the oscilloscope and the AOM driver to a function generator. I drive the AOM with a square wave going from 1V to 0V.

I slowly increased the square wave frequency. The PMC servo doesn't seem to care. I reach 100KHz - it seems excessive but still works. In any case, I get the same results doing a single shut-down from a DC level.

I download the traces. I normalize the traces but I don't rescale them (Attachment 4) so that the small extinction can be investigated.

I notice now that the PDs show the same extinction. It probably means I should have taken dark currents data for the PDs.

Also, I forgot to take the reflected data when the PMC is out of resonance with the laser which could have helped us determine the PMC transmission.

Again, the shutdown is not as sharp as I want. There is a noticeable smoothening in the transition around t = 0 which makes the fit to an exponential difficult. I suspect that the function generator is the limiting device now. I hooked up the function generator to the oscilloscope which showed similar distortion (didn't save the trace)

I try to fit the transmission PD trace to a double exponential and to Zucker model (Attachment 5).

The two exponentials model, being much less restrictive, gives a better fit but the best fit gives two identical time constants of 92ns.

The Zucker model gives a time constant of 88ns. Both of these results are consistent with more or less with the linewidth measurement but this measurement is still ridden with systematics which hopefully will become minimized IMC ringdowns.

Summary:

1. I investigated the stage-by-stage transfer functions of the PMC servo up till the HV stage. See Attachment #1. There were no unexpected features.
2. I replaced the AD602 used to implement the VGA capability. After the replacement, the gain of the VGA stage had the desired performance, see Attachment #2, Attachment #3.
3. The servo board was re-installed and the OLTF of the PMC loop was measured. See Attachment #4.

​To avoid driving the PA85 without the HV rails connected, I removed R23. This was re-installed after my characterization.

Input stage:

Since we do the demodulation of the PMC PDH signal off this servo board, the I/F mixer output is connected to the "FP1test" front panel LEMO input.

• A DG190 is used to enable/disable this path.
• Initially I tried checking the enable/disable functionality by measuring the resistance across the IC's I/O pins. However, this method does not work - the resistance read off from a DMM varied from ~23 ohms in the "ON" state to ~123 ohms in the "OFF" state. While the former value is consistent with the spec, the latter is confusing.
• But I confirmed that the switch does indeed isolate the input in the "OFF" state by injecting a signal with a function generator (100 Hz sine wave, 100mVpp) and monitoring the output on an oscilloscope.

Electronic TFs:

Using some Pomona mini-grabbers, I measured the electronic TFs between various points on the circuit. There were no unexpected features, the TFs all have the expected shape as per the annotations on the DCC schematic. I did not measure down to 0.1 Hz to confirm the low frequency pole implemented by U6, and I also didn't measure the RF low pass filter at the input stage (expected corner frequency is 1 MHz). 

VGA characterization:

After replacing the IC, I measured the transfer function between TP1 and TP2 for various values of the control voltage applied to pin 4A on the P1 connector, varying between +/- 5 V DC. 

• Pin 9A on the P1 connector has to be grounded for the signal to be allowed to pass through the VGA. 
• Note that there is an overall gain of -1/10 applied to the control voltage between pin 4A and pin #1 of the AD602, which is what actually sets the gain.
• Furthermore, the input impedance of the AD602 is spec-ed to be 100 ohms. Because of the series resistance of 500 ohms from TP1 to the input of the AD602 (so that the upstream OP27 isn't overdrawn for current), the relation between the control voltage applied to Pin 4A and gain (measured between TP1 and TP2) is modified to G [dB] = 32*(-0.1 * V_pin4A) - 6. 
• The gain behavior after the IC swap is as expected, both in terms of absolute gain, and the linearity w.r.t. the control voltage.
• Note that in Attachment #2, each color corresponds to a different control voltage to the AD602, varying from -5V DC to +5V DC in 1V steps. 

PZT Capacitance measurement

I confirmed that the PZT capacitance is 225 nF. The measurement was made using an LCR meter connected to the BNC cable delivering the HV to the PZT, at the 1X1 rack end.

OLTF measurement

After re-soldering R23, I put the board back into its Eurocrate, and was able to lock the PMC. For subsequent measurements, the PSL shutter was closed.

• I measured the OLTF using the usual IN1/IN2 prescription, implemented with the help of an SR560.
• At the original PMC Servo gain of +12dB, I found that the feature at ~8kHz results in an OLTF with multiple unity gain crossings.
• So I lowered it to +9dB. This yields an OLTF with ~60deg phase margin, ~2.3 kHz UGF. 
• The feature that sets the gain margin is actually not any of the peaks fit by LISO, but is one of the high frequency features at ~40 kHz. At the new setting of +9dB gain, the gain margin is ~10 dB.
• The measured TF (dots in Attachment #5) was fit with LISO (solid lines in Attachment #5) to allow inferring the out-of-loop servo noise by monitoring the in-loop noise (that plot to follow).

Zeroth order IMC ringdown setup

Following Gautam's IMC ringdown setup, I took the REFL PD form the PMC ringdown experiment and installed it in the IMC REFL path blocking WFS2 (Attachment 1).

I also ran a BNC cable from the transmission PD that Gautam installed on the IMC table to the vertex where the signals are measured on the scope. 

I offloaded the WFS servo output values onto the MC alignment (using the WFS servo relief script) so that its dc values would be correct when the servo is off.

Unfortunately, it seems like the script severely misaligned the MC mirrors at some point when the MC got unlocked. We should fix the script such that it stops when the offloading is complete.

We got the MC realigned but left it in a state where it is not locking easily.

It's fine to block the WFS while doing ringdowns but please return the config to normal so I don't have to spend time every night recovering the interferometer before doing the locking. As I mention in that post, it is possible to do this in a non-invasive way without having to run any extra cables / permanently block any beams. If there is some issue with the data quality, then we can consider a new setup. But I see no reason to re-invent the wheel.

The IMC was also massively misaligned. I had to re-align both MC1 and MC2 to recover the lock. I took this opportunity to reset the WFS offsets. Please do not disturb the alignment of the existing optical layout unless you verify that everything is working as it should be after your changes.

And for whatever reason, ITMX was misaligned. If you do something with the interferometer, no matter how minor it seems, please leave a note on the ELOG. It will save many painful debugging hours.

As I fix these, the seismic activity has gone up . I'll wait around for an hour, but not an encouraging restart to the locking 😢 

To conclude my PMC noise investigations: Attachment #1 shows the PMC noise inferred from the calibrations earlier in this thread and the fitted OLTF for the PMC loop. Attachment #2 compares the frequency noise (inferred from the error point of the PMC servo) when the IMC is locked / unlocked. I don't know what to make of the fact that the PMC suggests improvement from ~20 Hz onwards already - does this mean that the NPRO noise model is wrong by 1 order of magnitude at 30 Hz?

• The IMC was locked for the measurement shown in Attachment #1.
• The in-loop spectra of the error (at the I/F output of the mixer) and control (at TP3) signals were measured with the SR785.
• The control signal voltage monitors don't seem to work - neither the front panel LEMO nor the signals hooked up to the CDS system show me sensible shapes for the spectra between 1-3 Hz.
• To convert in loop to free-running, I multiplied the measured error (control) signal spectra by  (), where L is the OLTF. THe control signal was pre-processed by multiplying by a pole at 11.3 Hz, corresponding to the LPF formed by the 63.3 kohm series resistor and the 225 nF PZT capacitance.
• The "NPRO noise model" curve is 10^4/f Hz/rtHz.

While I initially thought the 1/f^2 rise below ~100 Hz is attributable to the IMC cavity length fluctuations, I found that this profile is present even in the measurement with the PSL shutter closed. I am not embarking on a detailed PMC noise budgeting project for now. Note however that we are not shot noise limited anywhere in this measurement band.

After some discussion with Gautam, I decided to build more spare channels into the new c1psl machine. This is anticipation of adding new laser and ISS channels in the near future, to avoid having to disconnect the installed chassis and pull it out of the rack. The spare channels will be wired to DB37M feedthroughs on the front side of the chassis, with enough wire length to be able to pull the breakout boards out of the front to reconfigure their wiring as needed (e.g., split off channels onto a separate connector).

To have enough overhead, this will require installing 1 additional ADC unit (XT1221) and 1 additional DAC (XT1541). We have enough spare BIO channels among the existing units (both sinking and sourcing). This will give us:

• 13 spare ADC channels
• 14 spare DAC channels
• 16 spare sinking BIO channels
• 12 spare sourcing BIO channels

The updated c1psl chassis wiring assignments are attached. It adds 4 new DB37M connectors for the spare channels (highlighted in yellow) and fixes one typo Jordan found while wiring today. The most current spreadsheet is available here.

A script I was testing errantly set C1:PSL-FSS_INOFFSET => 10 V at about 5:30 pm. I manually reverted the channel value to 0, but I don't know what the value was initially. Someone please check this value if there are problems locking the FSS.

You can trend the data for the past few hours and see what the appropriate value. I think these tests should only be done when whoever is running a test is in the lab.

P.S. I was surprised that the IMC didn't lose lock when this step was applied. I manually stepped this voltage between +/- 10 V and didn't see any response in the FSS readbacks. Either the channel doesn't work, or there is a divide by 40 in the physical circuit or something...

I tested the c1psl AO channels on the electronics bench on Friday. While I found all the wiring to be correct, some of the channels exhibited excess noise with all appearances of a grounding problem.

Today Jordan, Gautam, and I investigated this further. It is indeed a grounding problem, but actually with the Acromag ADCs. The Acromag DAC outputs are single-ended (return is grounded), so (for the purpose of a loopback test) I would expect to leave the ADC inputs ungrounded. This is the configuration I tested Friday. Today we also tested driving the ADC with a floating source. The ADC noise behavior is exactly the same, whether the source end is grounded or not.

However, grounding the minus pin of the ADC channel eliminates the noise. We don't understand why this seems to be required irrespective of the driving source, so there something we're missing about the ADC design. As it turns out, this same fix was made to the AI channels of the previously-upgraded Acromag machines. I know Chub and I had to do this for the AI channels of c1vac, but at the time we thought the source grounding was causing the issue. However, today Jordan and I looked inside c1iscaux, which Chub wired, and confirmed that its AI channels are wired in the same way.

So in any case, Jordan is grounding the c1psl AI channels in the same way as c1iscaux. Once this is done, we'll continue with the bench testing tomorrow.

gautam: here are my notes about this issue when i was doing the c1iscaux testing. As I note there, "previously-upgraded Acromag machines" in the plural may be a bit of a stretch - I have no idea what the grounding situation is in c1susaux / c1auxex for example.

{Yehonathan, Jon, Jordan}

I tested the ai channels of the new PSL Acromag by looping an already-tested ao channel (C2:PSL-FSS-INOFFSET) back to the different ai channels.

I use Jon's IFOTest with /users/jon/ifotest/PSL.yaml.

I created a spreadsheet for the testing based on the current wiring spreadsheet. I added two columns for the high and low readings for each ai channel (attached pdf).

I marked in red the failed channels. Some of them are probably calibration issues, but the ones that show the same voltage for high and low are probably disconnected wires.

I redid the test on the channel that seemed disconnected to confirm.

I created a yaml file with all the failed channels for retesting called /users/jon/ifotest/PSL_failed_channels.yaml.

I checked the failed channels against the EPICS database definitions and the yaml file inputted to IFOTest. The channels where the readings are something other than +10/0 V, but the high/low values do change, I think can be attributed to one of two things:

• An incorrect gain and/or offset conversion parameter in the yaml file
• The EPICS SMOO parameter (smoothing) is set to some long value

I fixed the channel gains/offsets in the master yaml file (PSL.yaml). I also disabled smoothing in the EPICS defintions of the new PSL channels for the purpose of testing. We can uncomment those lines after installing the new chassis if noise is a problem. Please go ahead and re-test the channels again.

{Yehonathan, Jon}

We retested the failed ai channels. Most of them got fixed by applying the inverse calibration in the yaml file.

We still find some anomalous channels, mostly in the DB25 connector. Turns out, their limits were ill-defined in the EPICS database. Specifying the right limit fixed the issue.

We find one miswired channel (BNC4). We connected the BNC to the right channel on the Acromag unit which fixed the issue.

Overall all the ai channels were successfully bench-tested.

Today I engineered the last piece of the new c1psl system: the multi-bit binary output (mbbo) channels that control the MC servo board gains. These 6-bit channels have to be split across two 4-bit Acromag registers. To enforce synchronous switching, I adapted the latch.py script developed by Gautam to address this problem in c1iscaux. Analogously to the c1iscaux implementation, I scripted the code to automatically run as a systemd service which is launched by the main modbusIOC service. I tested this all using the DB37 LED test board and confirmed it to work.

This now completes the electronics bench testing.

There are still several DB37 connectors to be wired, which carry only spare channels for future use and are not interfaced with the EPICS IOC. Jordan and I discussed this today and he or Chub will complete it shortly. To allow time for the spare channel wiring to be completed (as well as for more locking progress before interruption), Gautam and I think Monday/Tuesday next week would be the earliest possible window to install the new system.

I found the PMC unlocked this morning. It was re-locked using the usual procedure. I feel like this has been happening more frequently in the last month than before. In the past, the cause seems to have been the PZT voltage drifting too close to one of the rails - however, in this case, it looks like an IMC unlock event is what triggered the PMC lockloss (admittedly the PZT voltage was somewhat close to the rail). It would be good if someone can re-connect the PMC Transmission photodiode, it was a useful diagnostic channel we had working fine before the ringdowns started.

I also tweaked the input pointing into the PMC and ran the WFS DC offset relief script.

All spare channels on the PSL acromag chassis are connected with ~12in of spare wiring for future use.

[Gautam, Jordan]

Gautam showed me how the PMCTRANSPD signal was reading zero, and he suspected it might have to do with the acromag wiring. Disconnected the acromag box underneath the PSL table and checked the ADC wiring. Side note: When benchtesting the c1psl acromag chassis there was excess noise in the AI channels, and grounding the minus pin of the ADC channel eliminates the noise.

So I grounded the (-) pins on the ADC1 (192.168.113.122), which PMCTRANSPD is connected to and that seemed to fix the problem. As of right now PMCTRANSPD is reading ~.75 V.

See attached pictures

gautam: While this fix seems to have worked, I wonder why this became necessary only in the last month. Note that the problem was a noisy readback on the PMC transmission PD, which also made the FSS_RMTEMP channel noisy, leading me to suspect some kind of ground loop issue.

[many people]

in prep for the install tomorrow, we did the following:

• Install the c1psl Supermicro in the 1X2 rack (Attachment 1). To make room we removed the anti-image filter and mounted it on the OMC rack.
• Set up a local workstation (monitor+mouse+keyboard) for the Supermicro so we can do some local testing (Attachment 2).
• Clear up the immediate area around the 1X1/1X2 rack, setup a cart for the Acromag.
• Make sure there are sufficient adaptor boards cables (DB37, DB15, DB9, DB25, ethernet) etc available at the cart.
• Label cables, connect on Acromag chassis end (Attachment 3).
• Keep some large (A3) printouts of the channel mapping handy by the cart.
• made sure we have open fuse-able DIN rail connectors for +/-15 V DC and +/-24 V DC for the Acromag box (we are waiting on some thinner gauge cabling for the 24V supply, once that arrives, we will power the box from the Sorensens. For now, they are powered by bench supplies on the cart).
• made sure c1psl1 (still this name for the Supermicro) is ssh-able.

Barring objections, tomorrow (Friday 28 Feb 2020) morning I will commence the switch (I still want to work on the IFO tonight).

And so it begins.

Summary:

There are several problems evident already.

1. Several EPICS database entries were missing. WTF.
2. After fixing the missing entries, the PMC could be locked. However, the IMC could not be locked.
3. I think the FSS Interface card is not configured correctly.

For now, I've returned the old c1psl connections, the PMC and IMC are both locked. Need to do some debugging on the bench.

1. The new c1psl Acromag crate is now interfaced to the Eurocrate electronics in 1X1 (formerly VME c1psl) and 1X2 (formerly c1iool0).
2. The PMC and IMC can be locked. We will investigate stability / duty cycle over the weekend.
3. There were a few issues with the wiring - specifically, the worng kind of Acromag BIO unit (sourcing, whereas we want sinking) was used for the FSS board switches. Once Jordan fixed this issue, the IMC could be locked.
4. I began to do the detailed tests of the IMC Servo card channels - there may be some issues with the boost stages, but I ran out of time yesterday, so tbc Monday.

On Monday, we will remove the old c1psl and c1iool0 machines from the electronics rack and install the Acromag crate in a more permanent way. We will also clean up some of the old cabling and cross connects, althoug the situation seems so complicated (some cross connects are also used by the rtcds c1ioo expansion chassis) that I am inclined not to remove any cables.

The area around 1X1/1X2 has a lot of dangling cables and general detritus. Be careful if you are walking around there. We will clean up on monday.

I realigned the input pointing into the PMC this morning. Usually, the way I do this is to minimize any discernible mode structure in the PMC reflection CCD image. Today, I noticed that making the DC reflection go down also makes the DC transmission go down. Possibilities:

• we are sampling slightly different spots inside the PMC cavity which change the buildup by ~2-3%.
• we are misaligned on the transmission/reflection photodiode.
• ??

I investigated the problem reported earlier today with the BIO1 channels. By logging the systemd messages generated when the IOC starts, I was immediately able to determine that the problem was not limited to BIO1. The modbus communications were failing for several other units as well.

Because some in-situ rewiring of a handful of channels had recently been done (more on this soon), I initially suspected that one of the Acromags had been damaged in the process. However, removing BIO1 (or other non-communicating modules) did not restore communications with the rest of the modules. To test whether the chassis was the source of the problem at all, we set up a fresh ADC (new out of the package) and directly connected it to the secondary Ethernet interface of c1psl. With only the one new ADC connected, the modbus IOC failed in exactly the same way.

To confirm that the new ADC did in fact work, we connected it to c1auxex in the same configuration. The unit worked fine connected to c1auxex. This established that the source of the problem was the c1psl host. After some extensive debugging, I traced the problem to a pre-execution script (part of the modbus IOC systemd service) which resets the secondary network interface (the one connected to the Acromag chassis) prior to launching the IOC. This was to ensure the secondary interface always had the correct IP address. It appears this reset was somehow creating a race condition that allowed the modbus initializations (first communications with the Acromags) to sometimes start before the network interface had actually come back up.

I still don't understand how this was happening, or why the pre script worked just fine up until yesterday, but eliminating the network interface reset fixes the problem in 100% of the trials we ran. Unfortunately we lost the entire day to debugging this problem, so the final round of testing is still to be completed. We plan to pick it back up tomorrow afternoon.

We've completed almost all of the in-situ testing of the c1psl channels. During this process, we identified several channels which needed to be rewired to different Acromags (BIO sinking v. sourcing). We also elected to change the connector type of a few channels for practical advantages. Those modifications and other issues found during testing are detailed below. Also attached are the updated channel assignments, with a column indicating the in-situ testing status of each channel.

Post-installation modifications

• All four channels connected to the sourcing BIO module were found to in fact require sinking I/O. They were reassigned to sinking BIO modules. Affected channels:
  • C1:PSL-FSS_FASTSWEEP
  • C1:PSL-FSS_SW1
  • C1:PSL-FSS_SW2
  • C1:PSL-PSL_Shutter
• Added a new AI channel:
  • C1:PSL-FSS_MIXERM
• Removed an unneccessary AI channel:
  • C1:PSL-FSS_LODET
• Moved two AI channels from BNC connectors to a new Dsub connector (labelled DB25M-2 in the spreadsheet).
  • C1:PSL-FSS_RCTEMP
  • C1:PSL-FSS_RMTEMP_VOLTS

Issues identified during testing

• Digital calibration. The following channels work, but we need to verify their EPICS calibration parameters (EGUF/EGUL):
  • C1:PSL-FSS_FASTGAIN
  • C1:PSL-FSS_FAST
  • C1:PSL-PMC_RFADJ
  • C1:PSL-PMC_MODET
• IMC servo board. The Acromag channels themselves were found to work, but the linearity of the mbbo gain stages are in question (i.e., a potential problem with the board). GV is currently testing the servo board.
• PSL QPD board apears to be dead. We connected a scope directly to the test points on the board and measured a high level of noise and no signal (for all four of the QPD channels). I understand this QPD has not been used in some time, so it may not have been noticed before.
• WFS DC channels are saturating when the IMC is unlocked. The acceptance range of the Acromag ADC is only +/-10 V, but we measured sensor voltages as high as ~14 V. It appears that the old ADCs were somehow accepting a range of 0 to +20 V instead of -10 to +10 V. However, the Acromags do not support the input range 0-20 V. Since SNR is not critical for these channels (they're used only for initial alignment), I propose we simply install a voltage divider inside the chassis, just before the Acromag, for each of these signals.

[koji, gautam]

Attachment #1 shows the relevant parts of the schematic of the WFS demod board (not whitening board). 

• The basic problem was that the switchable gain channels were not accounted for in the Acromag channel list 😒.
• What this meant was that the DC gain was set to the default x100 (since the two DG211s that provide the switchable x10 and x1 gain options had their control logic pins pulled up to +5V because these pins weren't connected to any sinking BIO channel).
• Rather than set up new connections to Acromags inside the chassis (though we have plenty of spares), Koji and I decided to make these fixed to x1 gain.
• The actual fix was implemented as shown in the annotated schematic. There are some pictures 📷 of the PCB in the DCC entry linked above.
• Amusingly, this board will now require a sourcing BIO unit if we want to still have the capability of switching gains.

Before removing the boards from the eurocrate: 

• I dialled down the Kepco HV supplies
• disconnected all the cabling to these boards after noting down cable numbers etc.

After Koji effected the fix, the boards were re-installed, HV supplies were dialled back up to nominal voltage/currents, and the PMC/IMC were re-locked. The WFS DC channels now no longer saturate even when the IMC is unlocked 👏 👏 . I leave it to Yehonathan / Jon to calibrate these EPICS channels into physical units of mW of power. We should also fix the MEDM screen and remove the un-necessary EPICS channels.

Later in the evening, I took advantage of the non-saturated readbacks to center the beams better on the WFS heads. Then, with the WFS servos disabled, I manually aligned the IMC mirrors till REFLDC was minimized. Then I centered the beam on the MC2 transmission QPD (looking at individual quadrants), and set the WFS1/2 RF offsets and MC2 Trans QPD offsets in this condition.

[jordan, gautam]

the long DB25 cable to connect the Acromag chassis to the temperature sensor interface box arrived. We laid it out today. This cable does the following:

1. Supply the box with +/- 24 V DC from the chassis. The pin arrangement here is rather unfortunate (the +/-24 V DC and GND pins are in close proximity), so if you're not careful, you'll create a short as you plug this cable in (as we found out today). So the best practise is to power down the crate before connecting/disconnecting this cable. Jordan will label this accordingly tomorrow.
2. Pipe the "TAMB" and "TCAV" signals, corresponding to temperature sensors mounted to the PSL table-top and reference cavity exterior respectively, to the Acromags. We found during some initial testing that the "TAMB" signal was reaching the DB25 connector on the Acromag chassis, but wasn't going to any ADC channel - this was rectified.

Both signals now show up in the EPICS channels, but are noisy - I suspect this is because the return pin of the Acromag is not shorted to ground (this is a problem I've seen on the bench before). We will rectify this tomorrow as well.

We took this opportunity to remvoe the bench supply and temporary Acromag crate (formerly known as c1psl2) from under the PSL table. While trying to find some space to store the bench supply, we came across a damaged Oscilloscope in the second "Electronics" cabinet along the Y-arm, see Attachment #1. 

After this work, I found that the IMC autolocker was reliably failing to run the mcup script at the stage where the FSS gains are ramped up to their final values. I was, however, able to smoothly transition to the low-noise locked state if I was manipulating the EPICS sliders by hand. So I added an extra 2 seconds of sleep time between the increasing of the VCO gain to the final value and the ramping of the FSS gains in the mcup script (where previously there was none). Now the autolocker performs reliably.

I want to monitor the PMC TRANS and REFL levels on the PSL table - previously there were some cables going to the oscilloscope on the shelf but someone had removed these. I re-installed them just now. While there, I disconnected the drive to the AOM - there must've been some DC signal going to it because when I removed the cable, the PMC and IMC transmission were recovered to their nominal levels.

Summary:

On Monday, I hooked up an AG4395 to the PMC error point (using the active probe). The idea was to take a spectrum of the PMC error point every time the FSS PC drive RMS channel indicated an excursion from the nominal value. An initial look at the results don't suggest that this technique is particularly informative. I'll have to think more about a workaround, but please share your ideas/thoughts if you have some.

Also, the feature in the spectrum at ~110 kHz makes me suspect some kind of loop instability. I'll measure the IMC loop OLG at the next opportunity.

Details:

• The PMC servo bandwidth is ~2 kHz, so above this, the PMC error point should be a faithful monitor of the PSL frequency noise, provided the sensing noise is low enough.
• The PMC error point sensing noise is ~100nV/rtHz (I'm monitoring this straight after the Minicircuits mixer+bandpass filter that we are using as a demodulator). This corresponds to ~2 Hz/rtHz, using the ~10 MHz/V PDH discriminant calibration from January. Seems consistent with this elog.
• I was hoping to see if there was a particular frequency band in which the noise gets elevated, and if the crossover frequency is a few kHz and the IMC servo BW is ~110 kHz, I would have expected this to be in the 10-100 kHz region. Possibly my frequency resolution isn't good enough? But with the Agilent, doing a finer grid would mean a longer measurement time, in which case the IMC might lose lock before the measurement is done.
• But, as shown in Attachment #1, there isn't any clear evidence, from the ~20 excursions that were recorded last night. The color of the line is meant to be indicative of the average value of the PC drive RMS channel in the measurement time.
• A significant bottleneck in this whole process is that it takes ~1 minute to initiate the GPIB measurement, and download the data. The pseudo-code I used is:
  • While the IMC is locked, watch PCdrive RMS EPICS channel's "ALARM" state, which becomes non-zero when the PCdrive RMS exceeds 1 V (this is how it is defined in the EPICS db record right now).
  • Make sure this isn't a transient feature - I do this by waiting 5 seconds and checking that the ALARM flag is still flagged.
  • Initiate a AG4395 measurement over GPIB - I use the measurement span of 1 kHz - 1 MHz with a BW/span ratio of 0.1%, 5 averages.
  • Check that the IMC is still locked (if it got unlocked while the measurement was made, presumably the measurement is garbage).
• Is there a better monitor of the laser frequency noise? I can imagine using POX/POY which I think have a lower electronics noise floor but I'm not sure if that's true at 100 kHz and having the arms locked in addition to the IMC seems more complicated...
• Since we are planning a laser upgrade, is this worth spending more time on? I may leave the measurement running on pianosa in a tmux session while I'm not in the lab...

I had set up the 4395 to do this automatically a few years ago, but it looked at the FSS/IMC instead. When the PCDRIVE goes high there is this excess around ~500 kHz in a broad hump.

But the IMC loop gain changes sometimes with alignment, so I don't know if its a loop instability or if its laser noise. However, I think we have observed PCDRIVE to go up without IMC power dropping so my guess is that it was true laser noise.

This works since the IMC is much more sensitive than PMC. Perhaps one way to diagnose would be to lock IMC at a low UGF without any boosts. Then the UGF would be far away from that noise making frequency. However, the PCDRIVE also wouldn't have much activity.

The PMC has been unlocked since September 11 sometime (summary pages are flaky again). I re-locked it just now. I didn't mess with the HEPA settings for now as I'm not using the IFO at the moment, so everything should be running in the configuration reported here. The particulate count numbers (both 0.3um and 0.5um) reported is ~x5-8 of what was reported on Thursday, September 10, after the HEPA filters were turned on. We don't have logging of this information in any automated way so it's hard to correlate things with the conditions in Pasadena. We also don't have a working gauge of the pressure of the vacuum envelope.

The RGA scanning was NOT enabled for whatever reason after the vacuum work. I re-enabled it, and opened VM1 to expose the RGA to the main volume. The unit may still be warming up but this initial scan doesn't look completely crazy compared to the reference trace which is supposedly from a normal time based on my elog scanning (the timestamp is inherited from the c0rga machine whose clock is a bit off).

Update 1500: I checked the particle count on the PSL table and it barely registers on the unit (between 0-20 between multiple trials) so I don't know if we need a better particle coutner or if there is negligible danger of damage to optics due to particulate matter.

The HEPA filters on the PSL enclosure are no longer running. I tried cycling the power switch on the NW corner of the enclosure, and also turned the dial on the Variac down to zero and back up to maximum, no effect. Judging by the indicator LED on it, the power is making it to the Variac - bypassing the Variac and directly connecting the HEPA to the AC power seems to have no effect either.

I can't be sure, but I'm almost certain I heard them running at max speed an hour ago while I was walking around inside the VEA. Probably any damage that can happen has already been done, but I dialled down the Innolight injection current, and closed its shutter till the issue can be resolved.

The PMC servo railed and so I re-locked it at ~half range. I've been noticing that the diurnal drift of the PZT control voltage has been larger than usual - not sure if it's entirely correlated with temperature on the PSL table. Anyway the cavity is locked again so all is good.

PMC lost lock between 21:00 and 22:00 UTC on April 11th as seen in the summary pages:

https://nodus.ligo.caltech.edu:30889/detcharsummary/day/20210411/psl/#gallery-4

That's between 2pm and 3pm on Sunday for us. We're not sure what caused it. We will attempt to lock it back.

Mon Apr 12 08:45:53 2021: we used milind's python script in scripts/PSL/PMC/pmc_autolocker.py. It locked the PMC in about a minute and then IMC catched lock succefully.

However, the PMC transmission PD shows voltage level of about 0.7V. On medm, it is set to turn red below 0.7 and yellow above. In Summary pages in the past, it seems like this value has typically been around 0.74V. Simil;arly, the reflection RFPD DC voltage is around 0.063 V right now while it is supposed to be around 0.04 nominally So the lock is not so healthy.

We tried running this script and the bashscript version too (scripts/PSL/PMC/PMCAutolocker) a couple of times but it was unable to acquire lock.

Then we manually tried to acquire lock by varying the C1:PSL-PMC_RAMP (with gain set to -10 dB) and resetting PZT position by toggling Blank. After a few attempts, we were able to find the lock with transmission PD value around 0.73V and reflection RFPD value around 0.043. PZT control voltage was 30V and shown in red in medm to begin with. So we adjusted the output ramp again to let it come to above 50V (or maybe it just drifted to that value by itself as we could se some slow drift too). At Mon Apr 12 09:50:12 2021 , the PZT voltage was around 58V and shown in green.

We assume this is a good enough point for PMC lock and move on.

PMC lost lock again at around 16:00 April 12. I was able to lock it again but the transmission is only 0.6 now and REFL is 0.14.

Rana came in and realigned the PMC stirring mirrors. Now the transmission is 0.757V, and the REFL is 0.03V.

I noticed that the PZT was around 250V. Given that the PMC got unlocked at 16:00, which is around the peak temperature time in the lab (lagging behind the outside weather), due to the PZT voltage going down to 0V, I figured that the PZT voltage would go up during the night when the lab gets cold and therefore will likely go out of range again.

I found a different working point at 150V and relocked the PMC.

We (rana, yehonathan and i) briefly talked about having high power going into the IFO. I worked on some calcs a couple of years ago, that are summarized here. There is some discussion in the linked page about how much power we even need. In summary, if we can have

• T_PMC ~85% which is what I measured it to be back in 2019
• T_IMC * T_inputFaraday ~60% which is what I estimate it to be now
• 98% mode matching into the IMC
• power recycling gain of 40-45 once we improve the folding mirror situation in the recycling cavities
• and a gain of 270-280 in the arm cavities (20-30ppm round trip loss)

then we can have an overall gain of ~2400 from laser to each arm cavity (since the BS divides the power equally between the two arms). The easiest place to get some improvement is to improve T_IMC * T_inputFaraday. If we can get that up to ~90%, then we can have an overall gain of ~4000, which is I think the limit of what is possible with what we have.

We also talked about the EOM. At the same time, I had also looked into the damage threshold as well as clipping losses associated with the finite aperture of our EOM, which is a NewFocus 4064 (KTP is the Pockel medium). The results are summarized in Attachments #1 and #2 respectively. Rana thinks the EOM can handle factor of ~3 greater power than the rated damage threshold of 20W/mm^2.

{Yehonathan, Rana}

We unpacked the content of the amplifier crate in front of the water fountain (see attachments). Inside we found:

1. Amplifier head. (attachment 1)
2. Amplifier electronics and pump diodes (attachment 2).
3. Optical fiber (attachment 3).
4. 2 Long water hoses (~2m) and 2 short ones.
5. Network cable.
6. A wooden plate.
7. Cable sleeve (attachment 2)?
8. Some manuals will be uploaded to the wiki soon.

Please don't move/touch any of that stuff

Things that we need to consider/obtain:
1. A suitable power cable (attachment 4) with suitable power ratings (800W according to the amplifier specs). The connector head is C19 I believe.
2. A chiller. Rana says Aidan knows where to find one. Should we chill the amplifier head as well?
3. A mounting plate for the amplifier head with good thermal conductivity.
4. The pump wavelength is 808nm, we need to get suitable safety goggles.
5. Where to put the big electronics box. Preferably on 1X1 or 1X2.
6. How do we arrange the different components on the table? We also need to mode match the beam into the amplifier.

A couple of years ago, I got some info about the amplifier setup at the sites from Terra - sharing here in case there is some useful info in there (our setup will be rather different, but it looked to me like our Amp is a 2017 vintage and it may be that the performance is not the same as reported in the 2019 paper).

collection of docs (table layout in 'Proposed....setup') : https://dcc.ligo.org/LIGO-T1700046

LVC 70W presentation: https://dcc.ligo.org/LIGO-G1800538 

I guess we should double check that the beam size everywhere (in vacuum and on the PSL table) is such that we don't exceed any damage thresholds for the mirrors used. 

Some more relevant documents provided by Matt:

Phase III:70W amplifier integration at LIGO

70W amplifier External Shutter

aLIGO PSL high power attenuator

I surveyed a bit the 1X1/2 area to plan for the installation of the laser amplifier.

There is a vacancy at the bottom of 1X2 (attachment 1). I measured the dimensions of the diode box (DB) and it should fit. The optical fiber bundle is 75m long and should reach the amplifier head on the table easily.

According to the specs, the maximum power consumption of the DB is 800W (typically 600W), it should probably have its own circuit breaker. It can easily draw more than a few amps. The rack power strips are connected to this 4 socket box (attachment 2), is this just another power strip? It is connected to a circuit breaker with a 30A rating. How do we proceed from here?

In any case, we will need at least 2 meters of power cable.

I also tried to find a suitable place for a water chiller. A few suggestions are in the attachments. Basically either between the electronics shelves and the small rack next to 1X2 or next to the small rack close to the optical table. Maybe put it where the ladder sits and find another place for the ladder. Other options?

We would also need a windows machine running the Beckhoff software. The idea is that all the different laser components (DB, chillers, interlocks, switches) are connected to the EtherCat (over the ethernet infrastructure) so that the Beckhoff code can recognize a failure and switch off everything.

The things that are monitored:

1. Is the NPRO on?

2. Is the flow rate from the chillers enough?

3. Is the temperature of the diodes in the normal range?

4. Is one of the interlocks open?

5. Was one of the emergency buttons pushed?

6. Was the key switch on the DB turned to OFF?

The DB is EtherCat ready but the rest of the signals need to be interfaced somehow. Do we have to buy these EtherCAT terminals?

• Ideally, we put the chiller outside of the interferometer area. The PSL chiller used to be in the control room near the door by IMC REFL. We could also put it in the drill press room.
• Once we figure out a couple of places where the Diode Box can go, we can ask facilities to make the appropriate power connections. They will have to eval the situation to figure out if the main power to the lab needs to be shut down.
• Can we put the laser diode box in the drill press room too? Then the hoses can be short. Perhaps less EMI getting into our sensitive places.

Shutdown Procedure:
PSL Shutter closed / MC Autolocker disabled / PSL mechanical shutter closed / Laser injection current turned to zero / Laser turn off (red button) / Laser key turned off

The laser stat before the shutdown:
- LD Temp A: Set 22.07 (Untouched)
- LD Temp B: Set 21.03(Untouched)
- Laser Injection Current: Dial 9.53, Actual 2.100 -> Dial was moved to zero upon shutting down
- Laser Crystal Temp: Dial 3.34 (untouched)  Set 30.57 Actual 30.60 (Untouched)

PSL Table covering

- Because of the so many cables going up and down, sealing the PSL table with the metalized sheet was not easy. Therefore, the sheets have been just softly laid above the optics. (Attachment 1)
- The largest sheet which covers the east half of the table was taped to the table at the bottom, so that the air from the chimneys (see below) does not come up to the table

- The large dust could come from the opening of the enclosure during the filter replacement. So it was considered to be easier to seal the openings. (Attachment 2)
- Of course, the HEPAs are going to be tested after the maintenance work. It means that vent paths were needed so that the seals do not explode with the pressure (together with dust).
- Thus, the tubes of the sheets are attached to the seals to form "chimneys" for guiding the airflow beneath the table. (Attachment 2/3/4)
- This configuration was not meant to be sufficiently strong for a continuous run of the fans. Long running of the HEPAs may cause the failure of the seal tapes.
  Therefore the HEPA test should be done with a low flow rate and/or a short period of high flow.

- Once the work has been done, all the sheets should be carefully removed without scattering the fallouts onto the optics.

I also located the (possible) HEPA filters in the lab. (Attachments 1~3)

Oh! This is NO-NO! We can't place anything in front of the mains breakers. (Attachment 2)
I relocated the objects (Attachment 3)

I went to the TCS lab to take a look at the chillers lying around. I spotted two chillers:

1. Thermoflex1400 (attachment 1,2). Spec sheet.

2. Polyscience Recirculator 6000 series (attachment 3,4). Manual.

The Thermoflex has various communication ports. The Recirculator doesn't have any communication ports, but it is connected to a flow meter with what seems to be an electronic readout (attachment 5). Manual.

Both chillers have similar capacity ~ 4 gallons/minute. Thermoflex has 2 times more reservoir capacity than the Recirculator.

None of them seem to be Bechkoff-ready.

I guess we can have interlock code handling mixed signals Beckhoff+Non beckhoffs?

[Anchal, Koji]

Removed the top sheet

• Opened first from the door side so that any dust would spill outside.
• Then rolled the sheet inward to meet in the middle.
• Repeated this twice for the 2 HEPA filters.

Removed the sheets on the table

• Lifted sheet up making sure the top side face outside always.
• Rolled it sideways halfway through.
• Cut down the sheet vertically.
• Slided the doors to the other side and rolled the remaining half.
• On the door side, the sheets above the ALS optics were simply lifted off.

Restarting PSL

• Turned on the HEPAs at the max speed
• Switched on laser to jsut above the threshold
  • Before the 1st eom, power was 20mW 
  • After the EOM/AOM, 18mW. So about 90% transmission through all polarizing optics.
  • We saw the resonances of the PMC but could not lock it even with highest gain available (30 dB).
• Increased the input power to PMC to 100mW
  • Locked the PMC at 30dB gain
  • The transmitted power was ~50-60 mW. (Had to use power meter suspended by hand only.
  • The right before the IMC (after the 2nd EOM) 48mW. So none of the alignment was lost.
• Opened the PSL shutter.
• We were able to see IMC reflection signal.
• We were also able to see IMC catching lock as the servo was left ON earlier.
• Switched off the servo.
• Decided to increase the power while watching PMC Trans/Refl and IMC REFL
• Injection diode current to innolight was increased slowly to 2.10A. Saw a mod hopping region aroun 1.8A.
• We recovered the PMC Trans >0.7 V.
• PZT was near the edge, so moved by one FSR.
• The PMC refelction signal is still shown in red at around 48 mV.

Back to control room

• IMC was locked almost immediately by manually finding the lock while keeping IMC WFS off to preserve the offsets from yesterday.
• Then switch on IMC WFS. Working good.
• Then unlocked the servo and switched on IMC Autolocker. Lock was caught immediately.

Decided to start locking the arms

• The arm transmissions were flashing but at 0.2~0.3 level.
• Decided to adjust TT1 and TT2 Pitch and Yaw to align the light going into the arms.
• This made TRY ~0.6 / TRX ~0.8 at the peak of the flashing
• Locked the arms. (By switching on C1:LSC-MODE_SELECT which engages all servos).
• Used ASS to align Yarm then align Xarm. Procedure:
  • Sitemap > ASC > c1ass
  • Open striptool to look at progress. ! Scripts YARM > striptool.
  • Switch on ASS. ! More Scripts > ON
  • Wait for the TRY to reach to around 0.97.
  • Freeze the outputs. ! Scripts > Freeze Outputs.
  • Offload the offsets to preserve the output. ! More Scripts > OFFLOAD OFFSETS.
  • Switch off ASS. ! More Scripts > OFF
  • Repeted this for XARM.
• At the end, both XARM and YARM were locked with TRX ~ 0.97 and TRY ~ 0.96.