[Kiwamu, Suresh, Rana]

Our goal:

        We wished to determine the performance of the VCO PLL at low frequencies,. 

The procedure we followed:

        The scheme is to use the Marconi (locked to Rb Clock) as an 80MHz reference and lock to it using the PLL. 

        We set up the VCO PLL as in the diagram shown in the attachment and obtained the spectra shown below.

Results:

          We need to figure out the PLL servo gain profile in order to build the Inv PLL filter....

Damn. If this figure is true, we were looking at wrong signal. We should look at the feedback signal to the VCO.

I was getting an excess noise in the C1:IOO-MC_DRUM1 channel - it was a flat spectrum of 10 cts/rHz (corresponding to 600 uV/rHz).

I tried a few things, but eventually had to power cycle the crate with c1iovme in order to recover the standard ADC noise level of 3x10^-3 cts/rHz with a 1/sqrt(f) knee at 10 Hz.

I checked the gain of the channel by injecting a 2 Vpp sine wave at 137.035 Hz. 2Vpp as measured on a scope gives 31919 cts instead of the expected 32768, giving a 2.5% error from what we would have naively calculated.

Even so, the noise in this channel is very surprisingly good: 0.003 / (31919 / 2) = 187 nV /rHz.  The best noise I have previously seen from an ICS-110B channel is 800 nV/rHz. What's going on here?

We are now trying to understand why the coherence between SUS-X_SUSPOS_IN1 and SUS-X_SUSPOS_OUT is lost below 1 Hz for X = MC1, MC2, MC3, BS, ITMX, ITMY, ETMX, ETMY, SRM. It is OKEY only for PRM but the different filteres are used there. For PRM - 30:0.0 and Bounce Roll, for all others - 30:0.0 and Cheby. The transfer functions between these two signals plotted in foton and fft tools are also not the same.

If we switch off all the filters between these 2 signals, than the coherence is one. If one of the filters is switched on, everything is also fine. But if there are several present, than they filter the signal in unexpected way.

Moreover it seems that the coherence is dependent on the input signal. The coherence is better with local dumping than without.

 We have done the following on the c1sus and c1lsc computers : provided excitation, let the signal pass through filters 30:0.0 and Cheby and plotted the coherence between in and out signals.

c1sus computer - coherence is corrupted

c1lsc computer - coherence is not corrupted

[Koji, Rana]

We wanted to track down the excess noise seen in MC_F and other places (see the previous report by Gautam)

Setup1: The IMC was locked and MC_F signal between 500 and 1500Hz was observed. The DTT template was saved as /users/Templates/MC/MCF_noise_201023.xml

- Suspected mech resonance/jitter coupled with clipping or any other imperfections. Poked the various optics and optomechanics on the table. Basically no change. If we tap the laser chassis and the optics close to the laser source, we occasionally unlocked the IMC

- When we touched (lifted) the Innolight controller box from the shelf, for the first time we saw a significant change in the shape of the noise spectrum. The peak around the 700Hz shited towards lower frequency by a few %. Other peaks have no obvious change in the shapes and the heights.

- While observing the MC_F signal on the laptop, we went to the back of the laser controller. Placing a hand close to the fan clearly changes the peak frequency lower. By temporarily disconnecting the fan from the power supply for a short moment, the 700Hz peak could be eliminated. We also tried to see the noise level with the slow thermal servo and diagnosis DB cable disconnected, but we didn't see any significant change of the noise level.

Setup 2: Using the ALS phase tracker, we can observe the relative freq noise of the PSL laser and the ETMY AUX laser without any servo involved. This way we can freely disconnect any cables from the lasers. The measurement template for DTT was saved as /users/Templates/ALS/Y_ALS_FINE_PHASE_OUT_102320.xml

- Noise spectrum before disconnecting the cable (REF0, RMS REF1)

- The Fast PZT input to the PSL was disconnected => This made all the peaks (including the 700Hz) disappeared (REF2, RMS REF3)

- The Fast PZT input was restored as before, then the chain was disconnected at the input of the HV PZT driver (Thorlabs) => Again, this made the peaks disappeared (REF4, RMS REF5)

- The chain was disconnected at the input of the TTFSS box => Again, this made the peaks disappeared (REF6, RMS REF7)

- Disconnected the demod input and the AO cables from the IMC servo board => This made the peaks came back (REF8)

- Disconnected all the input/peripheral cables from the IMC servo board except for the connection to the TTFSS box => Still the excess noise was observed  (REF9)

- In addition to the above, the cable to the FSS box was disconnected but the ground was still touching the MC servo board => This made the peaks disappeared (REF10)

The conclusion is that the noise is injected from the main circuit of the IMC servo board.

Next time we will check if the backplane connection is doing something wrong. Also, we'll test if the presence of the RF signals does something bad to the IMC board via EMI and RFI.

We have reverted the connection and tested if we lock the IMC and Y arm. ==> We saw at least they were locked for a short period. The things are still stabilizing, but left them turned on so they keep trying to lock automatically for the night.

In fact, the problem was the grounding issue (presumably on the IOO racks).
A temporary differential receiver at the TTFSS side was built using an SR560 and a few ponoma cables. This removed the structures ~850Hz.

The MC Servo Output was disconnected from the TTFSS box and monitored with SR785. The 850Hz structure was kept visible no matter what cables, including all the acromag DB cables, were removed. This made me suspicious about the measurement setup. The SR785 was connected to an AC power strip under the SP table and this was too far from the IOO rack.

The SR785 was connected to the AC power strip on 1X2, and now the difference becomes clear. No matter if the acromag cables are connected or not, the connection (particularly ground connection) between the MC servo module and the TTFSS box causes the MC servo output contaminated. (Comparison between Blue and Orange of Attachment #1). During the measurement, the EPICS switch for the fast path was disengaged (=no signal) and the VCO gain (...so called. It's just the MC Servo Gain) was set to be 0dB.

To test if the differential receiving of the MC Servo Output at the PSL helps to reduce this noise, I've built a simple (hacky) differential receiver using an SR560. (Attachment #2)
This kept the noise level same as the disconnected case (Comparison between Green and Orange of Attachment #1, I don't think the difference between them is not significant), while the IMC is locked as before.
Note that we can see that the 36kHz line was significantly reduced. Did we remove this annoying noise?

After talking with Gautam, we decided to leave this configuration while the SE-Diff cable was replaced with a more robust one. (See Attachment #3)

The PSL laser frequency performance was evakluated in the following two ways as we did last week:
1) Use the beat frequency of the free running PSL and the Y-end laser (Attachment #4). The PSL shutter was closed and thus the IMC was not locked.
2) Use the IMC MCF while the IMC was locked. (Attachment #5)

For both cases, the improvement was confirmed.

I also tried to check the reported issue by Gautam on this elog. He used 1Hz BW, but I cheated with 16Hz BW and 10x12.8kHz span PSDs. (Attachment #6)

For the measurement, IN1 GAIN of the IMC Servo was set to be 0dB and the OUT2 was switched to monitor the IN1 noise, while IN1 was terminated by a 50Ohm.

As I mentioned above, the AC power of SR785 was taken from a 1X2 power strip. Is this the reason for the power line forest look less severe compared to the previous case???
Anyway, I tried to use the same differential receiving technique (but with gain of x100) to see if this helps. The differential receiver helped to reduce the structure above 50kHz. The floor noise level was observed to be higher. I didn't pursue this any further, but the forest of the power line looked like a part of the measurement noise. This is indicative that the grounding condition on 1X2 is really not great and we need to review the configuration of the acromag grounding.

Apart from the questionable wiring on the Acromags, one other important difference is in the way the connections were made between the old VME crates to the Eurocrate backplanes, and how we do it now. The thick cables had their sheilds connected to the eurocrate ground (or at least, there was a dedicated ground lug on those cables which we screwed on to the ground terminals on the Eurocrate backplanes). However, in our current configuration, we interface the Acromag ADCs and DACs to the backplane via these adaptor boards. The shields of the DSUB cables are presumably NOT connected to the Eurocrate grounds. This should also be investigated as one potential cause of the grounding issue - while on some of the Eurocrate modules, the P1/P2 connectors may have either the "A" or "C" row of connectors shorted to ground, some may not, and the TTFSS may suffer from such an issue?

Note that we have this problem in all of the slow machines that were upgraded to Acromag (if this turns out to be the issue). 

Summary:

Sometime between 1PM and 6PM on Tuesday, excess laser frequency noise shows up in MCF at around 800 Hz, as shown in Attachment #1. Sigh.

Details:

While I show the MCF spectrum here, I confirmed that this noise is not injected by the IMC loop (with the PSL shutter closed, and the IMC servo board disconnected from the feedback path to the NPRO, the PMC error and control points still show the elevated noise, see Attachment #2). I don't think the problem is from the PMC loop - see Attachment #3 which is the ALS beat out-of-loop noise with the PMC unlocked (the PSL beam doesn't see the cavity before it gets to the ALS setup, and we only actuate on the cavity length for that loop, so this wasn't even really necessary).

Was there some work on the PSL table on Tuesday afternoon that can explain this? 

I did some preliminary debugging of this, and have localized the problem to the output path (after MC slow) on the IMC Servo card. Basically, I monitored the spectrum of the ALS beat frequency fluctuations under a few different conditions: 

• With the BNC to the NPRO PZT disconnected, there is no noise. So the laser and the FSS phase correction EOM (which the ALS beat pickoff sees) are not responsible.
• With the input to the Koji summing box disconnected, still no excess - so the summing box + Thorlabs HV driver are not responsible.
• With the TTFSS output connected to the summing box, but with the input switch to the TTFSS box disabled (isolating the on-PSL table parts of the FSS system), still no excess.
• With the input to the TTFSS enabled, and the BNC output of the IMC Servo card disconnected at 1X2, still no excess.
• Finally, when I connect the BNC cable, the excess starts to show up.

Toggling C1:IOO-MC_FASTSW, which supposedly isolates the post-MC slow (a.k.a. MCL) part of the servo, I see no difference. I am also reasonably confident this switch itself works, because I can break the IMC lock by toggling it. So pending a more detailed investigation, I am forced to conclude that the problem originates in the part of the IMC servo board after the MCL pickoff. Some cabling was removed at 1X2 on Tuesday between the times when there was no excess and when it showed up, but it's hard to imagine how this could have created this particular problem.

[Paco, yuta]

We found excess noise on the YAUX - PSL ALS beat.

While preparing for a PRFPMI lock, we checked the ALS noise first on YAUX. To our surprise there is significant excess noise above 10 Hz as seen in Attachment #1 (dark blue is today's measurement, cyan is reference from the past). For example, the noise floor at around 100 Hz was 1 Hz, now it is ~ 20 Hz, and the 1 - 1000 Hz rms went up from ~ 200 Hz to > 1 kHz. This may be an issue for CARM offset locking, so we decided to investigate further.

Reviewing the elog, the recent significant change was the breakdown and subsequent replacement of the laser controller. Could the swap made by Koji a few weeks back have introduced excess frequency noise? We also noted the YAUX transmission (C1:ALS-TRY_OUT) has an rms level of ~ 10% which seems excessive but this could be a pathological symptom of our frequency noise or viceversa. The first thing we checked was the OLFT of the YAUX, the shape looks ok and the UGF looks typical, maybe a bit low around 6 kHz. We will repeat this measurement in more detail later.

We took an updated YARM cavity noise budget (see Attachment #2) and confirm that the PSL side looks normal (i.e. MC_F noise spectrum is nominal).

[Hiroki, Paco, Yuta, JC]

Hiroki realized the gain on the AUX feedback loop was slightly high, which was at least partially responsible for the excess frequency noise; after this the lock would still break and result in a mode hop so we tried adjusting the loop gains again this morning. The algorithm that seemed to work was reduce the gain on the analog (UPDH) box until the rail indicators (LEDs) stopped flashing and compensate using the SR560 --

[Paco]

Tuning the YAUX laser lowered the excess BEATY noise.

Since as of this post the only change in the YAUX setup was the death and replacement of the NPRO controller, I decided to play with the parameters. I found that increasing the laser power (and compensating for the frequency change by adjusting the temperature) successfully lowers the rms noise of the ALS beat. This is still not as good as it was before (1 Hz/rtHz at 100 Hz), but it is a hint of what may have happened. The initial settings were

ADJ = 0, T=43.6 deg

The final settings were:

ADJ = +6, T=25.8 deg

The maximum power adjustment is +10. Attachment #1 shows a reference (black) before the tuning was made, and after (red and cyan). The cyan trace has the noise eater off, while the red trace had noise eater on. There is no difference as per this measurement, so I left it ON.

With such a big temperature change, do you still get a reasonable beat note frequency? There's some previous elog of Koji I think that explains how we need to tune the lasers to get the 3 lasers to give 2 beat notes that are below 150 MHz.

Paco took the data which means he already had the beat note.

There is some chance that the beat was recovered after some mode "jumps" but usually the temp gap for the same beat frequency is ~2degC.
So my speculation is that there is a big temp gradient in the crystal now and had to compensate it with the struggle of the crystal TEC.

For the past data see https://nodus.ligo.caltech.edu:8081/40m/3759 or http://nodus.ligo.caltech.edu:8080/40m/12078
http://nodus.ligo.caltech.edu:8080/40m/4439 and so on.

For the past couple of days the 0.1 to 0.3 Hz RMS seismic noise along BS-X has increased. Attachment 1 shows the hour trend in the last ~ 10 days. We'll keep monitoring it, but one thing to note is how uncorrelated it seems to be from other frequency bands. The vertical axis in the plot is in um / s

Looks like this increase is correlated for BS/EX/EY. So it is likely to be real.

Comparison between 9/15 (UTC) (Attachment 1) and 9/10 (UTC) (Attachment 2)

On the Green YARM end table the second mirror behind the laser has been exchanged.

Reason: The light is impinging on the mirror under an angle of  about 10 degrees, but the old mirror was coated for angle of incidence (aoi) of 45°.

Thus it was more like a beam splitter. The new mirror is a 1" Goock & Housego mirror which has a better performance for an aoi of 10 degree.

Realignment through Faraday Isolator and SHG cristall as well as 532nm isolator is almost finished.

 In light of recent events and the decision to test the piezo tip-tilts for green beam steering on the X-end table, I have set up 8 excitation points to channels 8 through 15 of the DAC on c1scx (as was done earlier for the DAC at 1Y4 with Jenne's help) in order to verify that the pin-outs of the DAC interface board. I have not yet compiled the model or restarted the computer, and will do these tomorrow, after which I will do the test. The channels are named YYY_CHAN9 etc. 

 I just compiled and installed the model with the excitation points on c1scx and then restarted framebuilder. The channels I set up are now showing up in the awggui dropdown menu. I will do the tests on the DAC channels shortly.

Just to keep things on record, these are the steps I followed:

• opened the model c1scx (path: /opt/rtcds/userapps/release/sus/c1/models) with MATLAB
• Added 8 excitation points and saved the model. A copy has been saved as c1scx.mdl.r2010b because of the recent upgrade to r2013a. 
• ssh to c1iscex (computer running the model c1scx). 
• Entered the following sequence of commands in terminal: rtcds make c1scx ,  rtcds install c1scx , rtcds start c1scx 
• ssh to framebuilder, and restarted the framebuilder by entering telnet fb 8088   and then   shutdown.

So, I'm double-posting, but I figured the last post was long enough as it was, and this is about something different. After double and triple checking the XLR cables, I hooked up the microphone setup (mic---preamp---output) to the oscilloscope to figure out what kind of voltage would register with loud noises. So, I clapped and shouted and forgot to warn the other people in the lab what I was doing (sorry guys) and discovered that, even on the lowest gain setting, my loud noises were generation 2-3 times as much voltage as the ADC can handle (2V). And, since our XLR cables are so freaking long, we probably want to go for a higher gain, which puts us at something like 20 times too much voltage. I doubt this is really necessary, but it's late (early) and I got camera-happy, so I'm going to share anyway:

So, to deal with this issue, I made some nifty voltage dividers. Hopefully they are small enough to fit side-by-side in the ports without needing extra cableage. Anyway, they should prevent the voltage from getting larger than 2V at the output even if the mic setup is producing 50V. Seeing as my screaming as loud as I could about 2mm away from the mic at full gain could only produce 45V, I think this should be pretty safe. I put the ADC in parallel with a 25.5 kOhm resistor, which should have a noise like 10^-8 V/rHz. This is a lot smaller than 1 uV/rHz (the noise in the ADC, if I understood Rana's explanation correctly), so the voltage dividers should pose a noise issue. Now for pictures.

I opened one so you can see its innards.

In case the diagram on the box was too small to decipher...

And finally, I came up with a name scheme for the mics and pre-amps. We now have two Bluebird (bacteriophage) mics named Bonnie and Butch Cassidy. Their preamps are, naturally, Clyde and The Sundance Kid. Sadly, no photos. I know it's disappointing. Also, before anyone gives me crap for putting the labels on the mics upside-down, they are meant to be hung or mounted from high things, and the location (and stiffness) of the cable prevents us from simply standing them up. So they will more than likely be in some kind of upsidedownish position.

Over the past few days I have been trying to understand the existing sensor and heater related hardware by manual inspection and combing elogs. From what I understand, I believe a lot of the hardware was built from scratch by Kira in 2017-18. She put in place the insulation, built the sensor and heater circuits and installed and interfaced everything with EPICS. This let her successfully do step response tests and also execute PID control of the temperature of the can. As suggested by Paco, I am trying to summarise my findings in this elog. This is the block diagram of the overall system.

In the lab I was able to locate two identical temperature sensor boards, one for monitoring ambient temps and another for the can temp, which has a sensor attached to the inner seismometer can. Attachment 1 shows the actual board.

The schematic for the temperature sensor can be found here. It was later clarified that she is using AD590 and not the AD592. I tested the ambient sensor board by hooking it up to an oscilloscope and heating the sensor up. It seems to work like she described, and the output voltage goes down with heat. I will later figure out interfacing it with an ADC and calibrating it if I cannot find Kira's old work related to this. The setup there also has +/-15V Sorensen's for power (I probed these and they work), a BNC for temp readout and a bunch of other wiring for the heater. This wiring comes from the nearby rack which should have the heater circuit, but I haven't yet been able to locate it because there is too much stuff.

Coming to the heater circuit - I found the schematic for it here. I am still unsure about what the power range it can operate over, but this plot seems to imply it saturates at around 55W. This circuit underwent many different iterations so I am unsure what load resistance was actually used finally and it is not clear from the elogs. It might be a pair of heaters combined in series or parallel. Additionally, this elog by Shruti from June 2018 implies that the original circuit that Kira made ended up breaking at several different points. She said there were attempts to fix it, but I cannot find any updates after that and I think her SURF ended. I don't know the current status of the heater circuit. I am not sure where it is stored.

I was hoping to utilise the old EPICS channels set up by Kira for heater control as well as sensor readout to verify if everything was still functional and initially just read out the temperature of the can and find out how much it fluctuates. I planned to later also try to replicate the PID results. But Paco explained how the Acromag system worked and told me that utilising the old channels will not be possible right now, as all the channels have been used up for more critical tasks and none of the old interfaces that Kira had set up can be used now, even if the hardware was fine. 

Also, the can temperature sensor board has a wobbly solder joint at one of the power connectors, and we cannot move it to the lab because the sensor is anchored to the inner can. There is no power indicator LED to signal if the connection is secure so its kind of unusable right now. Paco and JC told me that soldering near the X end is not possible because the fumes would harm the optics. We will look for solutions to this. One option would be to make an entirely new board and keep a mechanical connector for the sensor. 

I picked the boxes up this morning. The inventory per Fil's email looks accurate. Some comments:

1. They shipped the chassis and mounting parts (we should still get rails to mount these on, they're pretty heavy to just be supported on 4 rack nuts on the front). idk if we still need the two empty chassis that were requested from Rich.
2. Regarding the fibers - one of the fibers is pre-2012. These are known to fail (according to Rolf). One of the two that LHO shipped is from 2012 (judging by S/N, I can't find an online lookup for the serial number), the other is 2011. IIRC, Rolf offered us some fibers so we may want to take him up on that. We may also be able to use copper cables if the distances b/w server and expansion chassis are short.
3. The units are fitted with a +24V DC input power connector and not the AC power supplies that we have on all the rest of the chassis. This is probably just gonna be a matter of convenience, whether we want to stick to this scheme or revert to the AC input adaptor we have on all the other units. idk what the current draw will be from the Sorensen - I tested that the boards get power, and with noi ADCs/DACs/BIOs, the chassis draws ~1A (read off from DCPS display, not measured with a DMM). ~Half of this is for the cooling fans It seems like KT @ LLO has offered to ship AC power supplies so maybe we want to take them up on that offer.
4. Without the host side OSSI PCIe card, timing interface board, and supermicro servers that actually have enough PCIe slots, we still can't actually run any meaningful test. I ran just a basic diagnostic that the chassis can be powered on, and the indicator LEDs and cooling fans run.
5. Some photos of the contents are here. The units are stored along the east arm pending installation.

    >     Koji,
    >
    >     Barebones on this order.
    >
    >        1. Main PCIe board
    >        2. Backplane (Interface board)
    >        3. Power Board
    >        4. Fiber (One Stop) Interface Card, chassis side only
    >        5. Two One Stop Fibers
    >        6. No Timing Interface
    >        7. No Binary Cards.
    >        8. No ADC or DAC cards
    >
    >     Fil Clara
    >       

That's fine, we didn't actually request those. We bought and already have in hand new PCIe x4 cables for the chassis-host connection. They're 3 m copper cables, which was based on the assumption of the time that host and chassis would be installed in the same rack.

Koji asked me assemble a detailed breakdown of the parts received from LHO, which I do based on the high-res photos that Gautam posted of the shipment.

Parts in hand:

Parts still needed:

Issue with PCIe slots in new FEs

Also, I looked into the mix-up regarding the number of PCIe slots in the new Supermicro servers. The motherboard actually has six PCIe slots and is on the CDS list of boards known to be compatible. The mistake (mine) was in selecting a low-profile (1U) chassis that only exposes one of these slots. But at least it's not a fundamental limitation.

One option is to install an external PCIe expansion chassis that would be rack-mounted right above the FE. It is automatically configured by the system BIOS, so doesn't require any special drivers. It also supports hot-swapping of PCIe cards. There are also cheap ribbon-cable riser cards that would allow more cards to be connected for testing, although this is not as great for permanent mounting.

It may still be better to use the machines offered by Keith Thorne from LLO, as they're more powerful anyway. But if there is going to be an extended delay before those can be received, we should be able to use the machines we already have in conjunction with one of these PCIe expansion options.

Can you please provide a link to this "list of boards"? The only document I can find is T1800302. In that, under "Basic Requirements" (before considering specific motherboards), it is specified that the processor should be clocked @ >3GHz. The 3 new supermicros we have are clocked at 1.7 GHz. X10SRi-F boards are used according to that doc, but the processor is clocked at 3.6 or 3.2 GHz.

Please also confirm that there are no conflicts w.r.t. the generation of PCIe slots, and the interfaces (Dolphin, OSSI) we are planning to use - the new machines we have are "PCIe 2.0" (though i have no idea if this is the same as Gen 2). 

As for the CX4 cable - I still think it's good to have these on hand. Not good to be in a situation later where FE and expansion chassis have to be in different racks, and the copper cable can't be used.

Indeed T1800302 is the document I was alluding to, but I completely missed the statement about >3 GHz speed. There is an option for 3.4 GHz processors on the X10SRi-F board, but back in 2019 I chose against it because it would double the cost of the systems. At the time I thought I had saved us $5k. Hopefully we can get the LLO machines in the near term---but if not, I wonder if it's worth testing one of these to see whether the performance is tolerable.

I confirm that PCIe 2.0 motherboards are backwards compatible with PCIe 1.x cards, so there's no hardware issue. My main concern is whether the obsolete Dolphin drivers (requiring linux kernel <=3.x) will work on a new system, albeit one running Debian 8. The OSS PCIe card is automatically configured by the BIOS, so no external drivers are required for that one.

I was calculating the power recycling gains we expect for different versions of the PRC, and I am a little concerned that we aren't going to have much gain with the new LaserOptik mirrors.

I'm using

                       t_PRM^2

G =  -------------------------------------------

       (1 - r_PRM * r_PR2 * r_PR3 * r_end)^2

from eqn 11.20 in Siegman.

r_end is either the ITM (for a symmetric Michelson) or the flat mirror that we'll put in (for the PR-flat test case).

r = sqrt( R ) = sqrt( 1 - T ) for mirrors whose power transmission is the quoted value.

Some values: 

t_PRM^2 = T_PRM = 0.055   --------->   r_PRM = sqrt( 1 - 0.055 )

T_G&H = 20e-6   ---->   r_G&H = sqrt( 1 - 20e-6 )

T_LaserOptic = 0.015 (see elog 7624 where Raji measured this...1.5% was the best that she measured for P polarization.  Elog 7644 has more data, with 3.1% for 40deg AoI) -------> r_LasOpt = sqrt( 1 - 0.015 ) or sqrt( 1 - 0.031)

T_ITM = 0.014 -----------> r_ITM = sqrt( 1 - 0.014 )

Some calculations with 1.5% LaserOptik transmission:

G_PRC_2G&H = 45

G_PRC_G&H_LasOpt = 31

G_PRM_flatG&H = 51

With the 3% LaserOptik transmission:

G_PRC_G&H_LasOpt = 22

G_PRM_flatG&H = 30

More ideal case of just PRM, flat mirror (either ITM or G&H), ignoring the folding mirrors:

G_PRM_ITM = 45

G_PRM_flatG&H = 70

Punchline:

If the LaserOptik mirror has 1.5% transmission at ~45 degrees, the regular PRC expected gain goes down to 31, from 45 with both folding mirrors as G&Hs.

Why would we use such a bad optic in our recycling cavity? Is 1.5% the spec for these mirrors? Is this the requirement that Kiwamu calculated somehow? Did anyone confirm this measurement?

I can't believe that we'll have low noise performance in a RC where we dump so much power.

 Yeah, Koji mentioned in response to Raji's measurements several months ago that the LaserOptic mirros were pretty far out of spec. We should probably redo the measurement to confirm.

I experimentally determined the spring constant of a single Vitol spring in order to obtain a rough estimate for the natural frequency of single-stack oscillation.

The procedure basically involved stacking metal bars of known mass onto the Vitol and using a caliper to measure deviations from the equilibrium length.

The plot below shows that, for small compressions, the response is linear to an R-squared of 0.98.

The experimental spring constant came out to be about 270 lb/ft, or 3900 N/m.

Previous documents have listed that the top stack takes on a load of approximately 43 kg. per individual spring. A bit of calculation yields an experimental resonant frequency of 9.5 Hz.

Compared with the theoretical COMSOL first harmonic of about 7.5 Hz, there is a reasonable amount of error. Of course, I used this calculation as a simple ballpark estimate: errors from misplacement onto the Viton were minimized through use of a level, but were still inevitable on the mm scale. Since the two methods yield answers with the same order of magnitude, we are ready to move forward and build the remaining layers of the stack.

So apparently with the current Dolphin drivers, when one of the nodes goes down (say c1lsc), it causes all the other nodes to freeze for up to 20 seconds.

This 20 seconds can force a model to go over the 60 microseconds limit and is sufficiently long enough to force the FE to time out.  Alex and Rolf have been working with the vendors to get this problem fixed, as having all your front ends go down because you rebooted a single computer is bad.

[40184.120912] c1rfm: sync error my=0x3a6b2d5d00000000 remote=0x0
[40184.120914] c1rfm: sync error my=0x3a6b2d5d00000000 remote=0x0
[44472.627831] c1pem: ADC TIMEOUT 0 7718 38 7782
[44472.627835] c1mcs: ADC TIMEOUT 0 7718 38 7782
[44472.627849] c1sus: ADC TIMEOUT 0 7718 38 7782
[44472.644677] c1rfm: cycle 1945 time 17872; adcWait 15; write1 0; write2 0; longest write2 0
[44472.644682] c1x02: cycle 7782 time 17849; adcWait 12; write1 0; write2 0; longest write2 0
[44472.646898] c1rfm: ADC TIMEOUT 0 8133 5 7941


The solution for the moment is to start the computers at exactly the same time, so the dolphin is up before the front ends, or start the models by hand after the computer is up and dolphin running, but after they have timed out.  This is done by:

sudo rmmod c1SYSfe

sudo insmod /opt/rtcds/caltech/c1/target/c1SYS/bin/c1SYSfe.ko

Alex and Rolf have been working with the vendors to get this fixed, and we may simply need to update our Dolphin drivers.  I'm trying to get in contact with them and see if this is the case.

We explore bilinear SRCL to DARM noise coupling mechanisms, and show two cases that by doing BHD readout the noise performance can be improved. In the first case, the bilinear piece is due to residual DHARD motion (see also LHO:45823), and it matters mostly for the low-frequency (<100 Hz) part, and in the second piece the bilinear piece is due to residual SRCL fluctuation and it matters mostly for the a few x 100 Hz part. Details are below:

=================================================

General Model:

We can write the SRCL to DARM transfer function as (Evan Hall's thesis, eq. 2.29)

Z_s2d(f) = C_lf(f) * F^2 * x_D + C_hf(f) * F * dphi_S * x_D    ---- (1)

where

C_lf ~ 1/f^2 and C_hf ~ f are constants at each frequency unless there are major upgrades to the IFO,

F is the finesse of the arm cavity which depends on the alignment, spot position on the TMs, etc., 

dphi_S is the SRCL detuning (wrt the nominal 90 deg value), 

x_D is the DC DARM offset. 

The linear part of this can be removed with feedforward subtractions and it is the bilinear piece that matters, which reads

dZ_s2d = C_lf * <F>^2 * dx_D + C_hf * <F> * <dphi_S> * dx_D

             + 2C_lf * <F> * <x_D>  * dF + C_hf * <dphi_S> * <x_D> * dF

             + C_hf  * <F> * <x_D> * d(dphi_S).     ---- (2)

The first term in (2) is due to residual DARM motion dx_D. This term does not depends on the DC value of DARM offset <x_D> and thus does not depend on doing BHD or DC readout. On the other hand, the typical residual DARM motion is 1 fm << 1 pm of DARM offset. Since the current feedforward reduction factor is about 10 (see both Den Martynov's thesis and Evan Hall's thesis), clearly we are not limited by the residual DARM motion. 

The second term is due to the change in the arm finesse, which can be affected by, e.g., the alignment fluctuation (both increasing the loss due to scattering into 01/10 modes and affecting the spot positon and hence changing the losses), and is likely to be the reason why we see the effect being modulated by DHARD. 

The last term in (2) is due to the residual SRCL fluctuation and is important for the ~ a few x 100 Hz band.

=================================================

DHARD effects. 

As argued above, the DHARD affects the SRCL -> DARM coupling as it changes the finesse in the arm cavity (through scattering into 01/10 modes; in finesse we cannot directly simulate the effects due to spot hitting a rougher location). 

Since in the second term of eq. (2) the LF part depends on the DARM DC offset <x_D>, this effect can be improved by going from DC readout to BHD. 

To simulate it in finesse, at a fixed DARM DC offset, we compute the SRCL->DARM transfer functions at different DHARD offsets, and then numerically compute the derivative \partial Z_s2d / \partial \theta_{DH}. Then multiplying this derivative with the rms value of DHARD fluctuation \theta_{DH} we then know the expected bilinear coupling piece. 

The result is shown in the first attached plot. Here we have assumed a flat SRCL noise of 5e-16 m/rtHz for simplicity (see PRD 93, 112004, 2016). We do not account for the loop effects which further reduces the high frequency components for now. The residual DHARD RMS is assumed to be 1 nrad. 

In the first plot, from top to bottom we show the SRCL noise projection at different DARM DC offsets of (0.1, 1, 10) pm. Since the DHARD alignment only affects the arm finesse starting at quadratic order, it thus matters what DC offset in DHARD we assume. In each pannel, the blue trace is for no DC offset in DHARD and the orange one for a 5 nrad DC offset. As a reference, the A+ sensitivity is shown in grey trace in each plot as a reference. 

We can see if there is a large DC offset in DHARD (a few nrad) and we still do DC readout with a few pm of DARM offset, then the bilinear piece of SRCL can still contaminate the sensitivity in the 10-100 Hz band (bottom panel; orange trace). On the other hand, if we do BHD, then the SRCL noise should be down by ~ x100  even compared to with the top panel. 

(A 5 nrad of DC offset in DHARD coupled with 1 nrad RMS would cause about 0.5% RIN in the arms. This is somewhat greater than the typically measured RIN which is more like <~ 0.2%. See the second plot). 

=================================================

SRCL effect. 

Similarly we can consider the SRCL->DARM coupling due to residual SRCL rms. The approach is very similar to what we did above for DHARD. I.e., we compute Z_s2d at fixed DARM offset and for different SRCL offsets, then we numerically evaluate \partial Z_s2d / \partial dphi_S. A residual SRCL rms of 0.1 nm is then used to generate the projection shown in the third figure. 

Unlike the DHARD effect, the bilinear SRCL piece does not depend on the DC SRCL detuning (for the 50-500 Hz part). It does still depends on the DARM DC offset and therefore could be improved by BHD.

Since we do not include the LP of the SRCL loop in this plot, the HF noise at 1 kHz is artifical as it can be easily filtered out. However, the LP will not be very strong around 100-300 Hz for a SRCL UGF ~ 30 Hz, and thus doing BHD could still have some small improvements for this effect. 

I double checked today and the F2A filters in the output matrices of MC1, MC2 and MC3 in the POS column are ON. I do not get what SDF means? Did we need to add these filters elsewhere?

The SDF system is supposed to help with restoring the correct settings, complementary to burt. My personal opinion is that there is no need to commit these filters to SDF until we're convinced that they help with the locking / noise performance.

Following discussion in this elog thread (40/6004), I used the design of F2A (force to angle(pitch)) decoupling filter as mentioned in this DCC doc T010140. This document is very useful as it talks about the overall control loops of a suspension, including sensor signal conditioning, damping filter shapes, force to pitch decoupling, pitch to position decoupling, and coil strength balancing. In future, if people are working on suspension characterization and damping, this document is a good resource to read.

Force to Angle (Pitch) decoupling filter

The document address this problem with first principle calculation using the geometry of single suspensions. As a first pass, I decided to use the design value of these geometric paramters to create a filter design for upper coils and one for lower coils. The parameters are listed in table 2. I used following:

	Description	Value used
L	Vertical dis-tance  from  the  suspension  point  to  the  wire  take-off  point	247.1 mm
h	Pitch distance (distance above the center-of-mass of the wire take-off point)	0.9 mm
l	L + h	248 mm
D	Vertical distance from a magnet to the center of the optic	24.7 mm
Q	Q value used in poles of the filter (doc says to use 1000)	3, 5, 10
	Position resonant angular frequency given by	6.288 rad/s

Using above parameters, we can define the F2A filter for upper coils as:

and for lower coil:

I used the design values as listed in the table above and got the filters as shown in attachment 1 for Q=3 case. I think the Q is higher than what other f2a filters I have seen for example at ETMY, the filters are as shown in attachment 2.

I tried turning on MC1 f2a filters but the watchdog tripped in about 4 minutes. This was the case when WFS were turned off. Another trial also lead to the same result. I tried this on MC2 and MC3 as well, all of them tripped after som time. See attachment 3 to see MC1 tripping on these filters.

I'll now try to use a lower Q filter.

I calculated the F2A filters for the input mode cleaner optics as described in T010140-01-D eq (4). On Ranas recommendation I added an s/ ( w_0 * Q ) term to the numerator.

The used values are:

w_0 = 2pi / s
h= 0.0009
D= 2.46957E-2
Q=10

I put theses filters into C1:SUS-MC1_TO_COIL_1_1 to _4_1 . For convenience split in Z and P. Well it doesn't work. After a few seconds the optic begins to swing wildly.

Woo. Pretty crazy. The numerators should only be ~10% larger than the denominator below 1 Hz. Let's try again.

 [Rana, Mirko]

I redid this calculation. The idea behind it is to get rid on any pitch that is introduced by applying longitudinal feedback to the mirrors. This coupling happens because the center of percussion for pitch , which is identical with the point where the wires lift off of the mirror, is above the center of mass.

With the same values as before, just less faulty math and Q = 2 instead of 10 we end up with the following filters:

For the lower coils (red), compared to corresponding preexisting BS filters (black):

The upper coils' TF is just mirrored at the 0dB magnitude axis, and have a corresponding frequency response.

I switched the F2a filters on for all MC mirrors. For convenience they are split into F2aZeros and F2aPoles. Everything seems fine. The F2a filters seem to be off for ( all ?) other mirrors.

Our approach to making the F2A or F2P filters for the MC is to use the measured resonant frequencies and then calculating the appropriate mechanical dimensions of each suspension. This is basically because we don't have optical levers with normal incidence on these optics, but the method should be fine.

To find the formulas, I asked Gaby for her old cheat sheet: Its now in the DCC. Its only for Large optics, but you should be able to reconstruct the right ones for SOS by just changing the parameters.

I tired running for a few hours F2A filter with Q=1 and for maybe 30 min Q=0.3 on MC3 today and that keeps the suspension stable. So I'm going to put in Q=0.3 at FM1, Q=0.7 at FM2, and Q=1 filter on FM3. I am setting the test again for tonight with some modifications. Now the separate set of filters will be tried one by one on the three different optics so that we know the best Q filter for each optic. It is set to trigger at 1 am tonight in tmux sessions f2aMC1Test, f2aMC2Test, f2aMC3Test on rossa. To cancel the test or interrupt, do:

[Paco, Anchal]

I changed the script in /opt/rtcds/caltech/c1/Git/40m/scripts/SUS/outMatFilters/createF2Afilters.py to read the measured POS resonant frequencies stored in /opt/rtcds/caltech/c1/Git/40m/scripts/SUS/InMatCalc/resFreqs.yml instead of using the estimate sqrt(g/len). I then added Q = 3 F2A filters into FM1 output filter of LO1, LO2, AS1 and AS4 suspensions in anticipation of BHD locking scheme work.

The f2A filters are set to test on IMC optics. The script used is testF2AFilters.py. The script is running on rossa in a tmux session named f2aTest. It will trigger at 1 am, Nov 4th 2022. First the script will turn off all F2A filters on IMC optics, wait for an hour, then it will try out the three F2A filter sets with different Q values, one at a time, for one hour each. So the test should last for roughly 4 hours. The gpstime stamps will be written in a logfile that can be used later to readback noise performance of IMC with different filter. The script has a try-except failsafe to revert things to original state if something fails. To stop the script from triggering or stop it during runtime, do following on rossa:

This test was not successfull as IMC lost lock during the f2A filter trial. However, we do have 1 hour off data when all f2A filters were turned off in between following GPS times:

start gpstime: 1351584077

stop gpstime: 1351587677

After this gpstime, the f2A filters were turned ON for all IMC optics. After about 2000 seconds of no issues, the MC3 suspension suddenly rung up 1 Hz oscillations around 1351590720 gpstime. See attachment one for noise spectra of local damping error signals for MC3 before and after this event. See attachment 2 for time series of this event.

So, after this point, MC3 remained rung up and IMC remained unlocked, so no WFS signals are meaningful after gpstime 1351590720.

I have seen this happening out of nowhere to MC3 today too when PSL shutter was closed and only thing interacting with MC3 was the local damping loop. This suggests that some glitch event happens in MC3 which is not taken well by the f2a filter on it. The ringing goes down as soon as we turn OFF the f2a filter. The other optics show no such signs.

We'll do more tests in future to figure out the issue. For now, MC3 f2a filters are kept off. Maybe we need custom filter for MC3 rather than the design value default filter we are using right now. I'm attaching foton bode plot for MC3 f2a filters for verification that correct filters are in place.

Already for the second time today all computers loose connection to the framebuilder. When I ssh to framebuilder DAQD process was not running. I started it

controls@fb ~ 130$ sudo /sbin/init q

But I do not know what causes this problem. May be this is a memory issue. For FB

Mem:   7678472k total,  7598368k used,    80104k free

Practically all memory is used. If more is needed and swap is off, DAQD process may die.

Just to be clear, "init q" does not start the framebuilder.  It just tells the init process to reparse the /etc/inittab.  And since init is supposed to be configured to restart daqd when it dies, it restarted it after the reloading of /etc/inittab.  You and Alex must have forgot to do that after you modified the inittab when you're were trying to fix daqd last week.

daqd is known to crash without reason.  It usually just goes unnoticed because init always restarts it automatically.  But we've known about this problem for a while.

This doesn't really mean anything, since the computer always ends up using all available memory.  It doesn't indicate a lack of memory.  If the machine is really running out of memory you would see lots of ugly messages in dmesg.

[jamie, gautam]

In order to test the new daqd config that Jamie has been working on, we felt it would be most convenient for the host name "fb" (martian network IP 192.168.113.202) to point to the physical machine "fb1" (martian network IP 192.168.113.201).

I made this change in /var/lib/bind/martian.hosts on chiara, and then ran sudo service bind9 restart. It seems to have done the job. So as things stand, both hostnames "fb" and "fb1" point to 192.168.113.201.

Now, when starting up DTT or dataviewer, the NDS server is automatically found.

More details to follow.

Over the past few days, I've occasionally been peeking at the framebuilder IO load to see If I could correlate anything with it, but it's usually been low when I looked. I.e. with daqd and all models running, the %wa time was in the few percents at most.

Just now, I was seeing some EPICS sluggishness, and sure enough, the %wa was in the 50-60 range. I used iostat -xmh 5 on the framebuilder to see that /dev/sda, the /frames drive, was at 100% utilization, which means it was reading and writing as fast as it possibliy could. 

I ssh'd over to nodus, and with iotop found that an rsync job was running (rsync -am --exclude .*.gwf full 131.215.114.19::40m/full), and its IO rates corresponded very closely to the data read rates on the framebuilder from /frames. 

I killed the rsync process on nodus, and the %wa time on the framebuilder dropped to near zero. The ASS striptools, where I had noticed the sluggishness, immediately started updating faster.

While rsync is supposed to play nice with a system's IO demands, maybe it only knows about nodus's IO usage, not fb which is the underlying NFS server where the frames live. I think it would be good to throttle the bandwidth of these jobs to a specific bandwidth. 50MB/s seemed like too much, so maybe 10MB/s is ok?