I went inside the ITMX Chamber to read off specs from PR2 edge. This was required to confirm our calculations of LO power for BHR later. The numbers that I could read from the edge were kind of meaningless "0.5 088 or 2.0 088". To make it more worthwhile this opening of the chamber, we decided to remove the PR2 suspension unit so that the optic can be removed and installed on an SOS in the cleanroom. We covered the optic in clean aluminum foil inside the chamber, then placed in on another aluminum foil to cover completely. Then I traveled slowly to the C&B room, where I placed it on a flow bench.

Later on, we decided to use a dummy fixed mount mirror for PR2 initially with the same substrate thickness, so that we get enough LO power in transmission for alignment. In the very end, we'll swap that with the PR2 mounted on an SOS unit.

[Anchal, Paco, Ian]

Before we could start working on Part II, which is to relocate TT2 to new location, we had to clear space in front of injection chamber door and clean the floor which was very dusty. This required us to disconnect everything we could safely from OMC North short electronics rack, remove 10-15 BNC cables, 4-5 power cords and relocate some fiber optic cables. We didn't had caps for fiber optic cables handy, so we did not remove them from the rack mounted unit and just turned it away. At the end, we mopped the floor and dried it with a dry cloth. Before and after photos in attachments.

[Ian, Anchal, Paco]

After the Koji found that there was a problem with the power source Anchal and I fixed the power then reran the measurment. The only change this time around is that I increased the excitation amplitude to 100. In the first run the excitation amplitude was 1 which seemed to come out noise free but is too low to give a reliable value.

link to previous results

The new plots are attached.

[Paco, Ian]

We opened the laser head shutter. Then, we scanned around the PMC resonance and locked it. We then opened the PSL shutter, touched the MC1, MC2 and MC3 alignment (mostly yaw) and managed to lock the IMC. The transmission peaked at ~ 1070 counts (typical is 14000 counts, so at 10% of PSL power we would expect a peak transmission of 1400 counts, so there might still be some room for improvement). The lock was engaged at ~ 16:53, we'll see for how long it lasts.

There should be IR light entering the BSC!!! Be alert and wear laser safety goggles when working there.

We should be ready to move forward into the TT2 + PR3 alignment.

1. We have a rack at the 40m storage. We are free to take it to the lab. If there is a tag, tell the info to Liz. Let's move it to the lab tomorrow right after the meeting.

2. We have a few racks in WB B1 (Attachment 1). Liz and I checked a rack which looks suitable for us. 46U height. Caltech transport will move it to the lab.

Modifications and testing of SatAmp units COMPLETE. Attachments 1 & 2 show all 19 units, one installed unit and the remaining 18 units are stacked and ready for install. Detailed notes of the modification for each unit are presented in the summary document in the dcc.

Things that I need to start suspending optics:

1. Winch adapter plate (D970314). Might need to make one.

2. Quad photodetector

3. Camera and camera mount

4. Beam height target

5. Height gauge

[Anchal, Paco, Ian]

Clean room etiquettes

• Two people in coverall suits, head covers, masks and AccuTech ultra clean gloves.
• One person in just booties to interact with outside "dirty" world.
• Anything that comes in chamber, first cleaned outside with clean cloth and IPA. Then cleaned by the "clean" folks. We followed this for allen keys, camera and beam finder card.
• Once the chamber cover has been removed, cover the annulus with donut. We forgot to do this :(

Optics removal and changes

We removed the following optics from the BSC table and stored them in X-end flowbench with fan on. See attachment 1 and 2.

1. IPPOS SM2
2. GRX SM2
3. PRM OL1
4. PRMOL4
5. IPPOS SM3
6. IPANG SM1
7. PRM OL2
8. Unidentified optic inbetween IPPOS45P and IPPOS SM3
9. Beam block behing PR3
10. Beam block behind GR PBS
11. GR PBS
12. GRPERI1L (Periscope)
13. PRMOL3
14. IPPOS45P
15. Cylindrical counterweight on North-west end of table.
16. Cheap rectangular mirror on South west end of table (probably used for some camera, but not in use anymore)
17. IPANGSM2

We also changed the direction of clamp of MMT1 to move it away from the center of the able (where PRM will be placed)

We screwed in the earthquake stops on PRM and BS from front face and top.

We unscrewed the cable post for BS and PRM oplevs and loved it in between SR3 and BS and screwed it lightly.

We moved the PRM, turned it anti-clockwise 90 degrees and brought it in between TT2 and BS. Now there is a clear line of sight between TT2 and PR2 on ITMY table.

Some next steps:

• We align the input beam to TT2 by opening the "Injection Chamber" (formerly known as OMC chamber). While doing so, we'll clear unwanted optics from this table as well.
• We open ITMX chamber, clear some POP optics. If SOS are ready, we would replace PR2 with SOS and put it in a new position.
• Then we'll replace PR3 with an SOS and align the beam to BS.

These are next few days of work. We need atleast one SOS ready by Thursday.

Photos after today's work: https://photos.app.goo.gl/EE7Mvhw5CjgZrQpG6

[Yehonathan, Anchal]

On thursday Oct 21 2021, Yehonathan and I opened the door to BSC and took some photos. We setup the HEPA stand next to the door with anti-static curtains covering all sides. We spend about 15 minutes trying to understand the current layout and taking photos and a video. Any suggestions on improvement in our technique and approach would be helpful.

Links to photos:

https://photos.app.goo.gl/fkkdu9qAvH1g5boq6

I have done some reading about where would be the best place to put the particle counter. The ISO standard (14644-1:2015) for cleanrooms is one every 1000 m^2 so one for every 30m x 30m space. We should have the particle counter reasonably close to the open chamber and all the manufactures that I read about suggest a little more than 1 every 30x30m. We will have it much closer than this so it is nice to know that it should still get a good reading. They also suggest keeping it in the open and not tucked away which is a little obvious. I think the best spot is attached to the cable tray that is right above the door to the control room. This should put it out of the way and within about 5m of where we are working. I ordered some cables to route it over there last night so when they come in I can put it up there. 

rethinking what I said on Wednesday - its not a good idea to put the particle counter on a vac chamber with optics inside. The rumble from the air pump shows up in the acoustic noise of the interferometer. Let's look for a way to mount it near the BS chamber, but attached to something other than vacuum chambers and optical tables.

SVG doesn't work in my browser(s). Can we use PDF as our standard for all graphics other than photos (PNG/JPG) ?

The particle count channel names were changes yesterday to follow naming conventions used at the sites. Following are the new names:

C1:PEM-BS_DUST_300NM
C1:PEM-BS_DUST_500NM
C1:PEM-BS_DUST_1000NM
C1:PEM-BS_DUST_2000NM
C1:PEM-BS_DUST_5000NM
 

The legacy count channels are kept alive with C1:PEM-count_full copying C1:PEM-BS_DUST_1000NM channel and C1:PEM-count_half copying C1:PEM-BS_DUST_500NM channel.

Attachment one is the particle counter trend since 8:30 am morning today when the HVAC wokr started. Seems like there was some peak particle presence around 11 am. The particle counter even counted 8 counts of particles size above 5um!

I had 8 standoffs made at the Caltech chemistry machine shop to be used as spacers for the side magnets on the 3" Ring assembly. This is to create enough clearance between the magnet and the cap screws directly above on the wire clamp.

These are 0.075" diameter by .10" length. Putting them through clean and bake now.

Probably the hard disk of c0rga is dead. I'll follow up in this elog later today.

Looking at the log in /opt/rtcds/caltech/c1/scripts/RGA/logs , it seemed that the last RGA scan was Sept 2, 2021, the day when we had the disk full issue of chiara.
I could not login to c0rga from control machines.
I was not aware of the presence for c0rga until today, but I could locate it in the X arm.
The machine was not responding and it was rebooted, but could not restart. It made some knocking sound. I am afraid that the HDD failed.

I think we can
- prepare a replacement linux machine for the python scripts
or
- integrate it with c1vac

This was horrible! That's my bad, I should have checked the configuration before assuming that it is right.

I fixed the power supply configuration. Now the strip has two rails of +/- 18V and the GND is referenced to power supply earth GND.

Ian should redo the tests.

I have placed a GT321 particle counter on top of the MC1/MC3 chamber next to the BS chamber. The serial cable is connected to c1psl computer on 1X2 using 2 usb extenders (blue in color) over the PSL enclosure and over the 1X1 rack.

The main serial communication script for this counter by Radhika is present in 40m/labutils/serial_com/gt321.py.

A 40m specific application script is present in the new git repo for 40m scripts, in 40m/scripts/PEM/particleCounter.py. Our plan is to slowly migrate the legacy scripts directory to this repo overtime. I've cloned this repo in the nfs shared directory at /opt/rtcds/caltech/c1/Git/40m/scripts which makes the scripts available at all computers and keep them upto date in all computers.

The particle counter script is running on c1psl through a systemd service, using service file 40m/scripts/PEM/particleCounter.service. Locally in c1psl, /etc/systemd/system/particleCounter.service is symbollically linked to the file in the file.

Following channels for particle counter needed to be created as I could not find any existing particle counter channels.

[C1:PEM-BS_PAR_CTS_0p3_UM]
[C1:PEM-BS_PAR_CTS_0p5_UM]
[C1:PEM-BS_PAR_CTS_1_UM]
[C1:PEM-BS_PAR_CTS_2_UM]
[C1:PEM-BS_PAR_CTS_5_UM]

These are created from 40m/softChansModbus/particleCountChans.db database file. Computer optimus is running a docker container to serve as EPICS server for such soft channels. To add or edit channels, one just need to add new database file or edit database files in thsi repo and on optimus do:

controls@optimus|~> sudo docker container restart softchansmodbus_SoftChans_1
softchansmodbus_SoftChans_1

that's it.

I've added the above channels to /opt/rtcds/caltech/c1/chans/daq/C0EDCU.ini to record them in framebuilder. Starting from 11:20 am Oct 20, 2021 PDT, the data on these channels is from BS chamber area. Currently the script is running continuosly, which means 0.3u particles are sampled every minute, 0.5u twice in 5 minutes and 1u, 2u, and 5u particles are sampled once in 5 minutes. We can reduce the sampling rate if this seems unncessary to us.

(Because of a totally unrelated reason) I was checking the electronics units for the upgrade. And I realized that the electronics units at the test stand have not been properly powered.

I found that the AA/AI stack at the test stand (Attachment 1) has an unusual powering configuration (Attachment 2).
- Only the positive power supply was used / - The supply voltage is only +15V / - The GND reference is not connected to anywhere.

For confirmation, I checked the voltage across the DC power strip (Attachments 3/4). The positive was +5.3V and the negative was -9.4V. This is subject to change depending on the earth potential.

This is not a good condition at all. The asymmetric powering of the circuit may cause damages to the opamps. So I turned off the switches of the units.

The power configuration should be immediately corrected.

1. Use both positive and negative supply (2 power supply channels) to produce the positive and the negative voltage potentials. Connect the reference potential to the earth post of the power supply.
   https://www.youtube.com/watch?v=9_6ecyf6K40   [Dual Power Supply Connection / Serial plus minus electronics laboratory PS with center tap]
2. These units have DC power regulator which produces +/-15V out of +/-18V. So the DC power supplies are supposed to be set at +18V.

I ran a DAC to ADC test on c1sus2 channels where I hooked up the outputs on the DAC to the input channels on the ADC. We used different combinations of ADCs and DACs to make sure that there were no errors that cancel each other out in the end. I took a transfer function across these channel combinations to reproduce figure 1 in T2000188.

As seen in the two attached PDFs the channels seem to be working properly they have a flat response with a gain of 0.5 (-6 dB). This is the response that is expected and is the result of the DAC signal being sent as a single ended signal and the ADC receiving as a differential input signal. This should result in a recorded signal of 0.5 the amplitude of the actual output signal.

The drop off on the high frequency end is the result of the anti-aliasing filter and the anti-imaging filter. Both of these are 8-pole elliptical filters so when combined we should get a drop off of 320dB per decade. I measured the slope on the last few points of each filter and the averaged value was around 347dB per decade. This is slightly steeper than expected but since it is to cut off higher frequencies it shouldn't have an effect on the operation of the system. Also it is very close to the expected value.

The ripples seen before the drop off are also an effect of the elliptical filters and are seen in T2000188.

Note: the transfer function that doesn't seem to match the others is the heartbeat timing signal.

I learned that TP1 was vented through the RGA room in the past. This can be done by opening VM2 and a manual valve ("needle valve")
I checked the setup and realized that this will vent RGA. But it is OK as long as we turns of the RGA during vent and bake it once TP1 is back.

Additional note:

- It'd be nice to take a scan for the current background level before the work.
- Turn RGA EM and filament off, let it cool down overnight. 
- Vent with clean N2 or clean air. (Normal operating temp ~80C is to minimize accumulation of H-C contaminations.)
- There is a manual switch and indicators on the top of the RGA amp. It has auto protection to turn filament off if the pressure increase over ~1e-5.

[Chub, Jordan, Yehonathan, Anchal, Koji]

North door of the BS chamber opened

[S2100738, S2100745, S2100751] Completed three more Sat Amp units modification with seven remaining.

[Chub, Jordan, Anchal, Koji]

- Checked the main volume is isolated.
- TP1 and TP2 were made isolated from other volumes. Stopped TP1. Closed V4 to isolate TP1 from TP2.
- TP3 was made isolated. TP3 was stopped.
- We wanted to vent annuli, but it was not allowed as VA6 was open. We closed VA6 and vented the annuli with VAVEE.
- We wanted to vent the volume between VA6, V5, VM3, V7 together with TP1. So V7 was opened. This did not change the TP1 pressure (P2 = 1.7mmTorr) .
- We wanted to connect the TP1 volume with the main volume. But this was not allowed as TP1 was not rotating. We will vent TP1 through TP2 once the vent of the main volume is done.

- Satrted venting the main volume@Oct 18, 2021 9:45AM PDT

- We started from 10mTorr/min, and increased the vent speed to 200mTorr/min, 700mTorr/min, and now it is 1Torr/min @ 20Torr
- 280Torr @11:50AM
- 1atm  @~2PM

We wanted to vent TP1. We rerun the TP2 and tried to slowly introduce the air via TP2. But the interlock prevents the action.

Right now the magenta volume in the attachment is still ~1mTorr. Do we want to open the gate valves manually? Or stop the interlock process so that we can bypass it?

From the IFO point of view, all look good and we are ready for venting from Mon Oct 18 9AM

I took over the vent prep: I'm going through the list in [ELOG 15649] and [ELOG 15651]. I will also look at [ELOG 15652] at the day of venting.

1. IFO alignment: Two arms are already locking. The dark port beam is well overlapped. We will move PRM/SRM etc. So we don't need to worry about them. [Attachment 1]
   scripts>z read C1:SUS-BS_PIT_BIAS C1:SUS-BS_YAW_BIAS
-304.7661529521767
-109.23924626857811
scripts>z read C1:SUS-ITMX_PIT_BIAS C1:SUS-ITMX_YAW_BIAS
15.534616817500943
-503.4536332290159
scripts>z read C1:SUS-ITMY_PIT_BIAS C1:SUS-ITMY_YAW_BIAS
653.0100945988496
-478.16260735781225
scripts>z read C1:SUS-ETMX_PIT_BIAS C1:SUS-ETMX_YAW_BIAS
-136.17863332517527
181.09285307121306
scripts>z read C1:SUS-ETMY_PIT_BIAS C1:SUS-ETMY_YAW_BIAS
-196.6200333695437
-85.40819256078339
    
2. IMC alignment: Locking nicely. I ran WFS relief to move the WFS output on to the alignment sliders. All the WFS feedback values are now <10. Here is the slider snapshots. [Attachment 2]
    
3. PMC alignmnet: The PMC looked like it was quite misaligned -> aligned. IMC/PMC locking snapshot [Attachment 3]
   Arm transmissions:
   scripts>z avg 10  C1:LSC-TRX_OUT C1:LSC-TRY_OUT
C1:LSC-TRX_OUT 0.9825591325759888
C1:LSC-TRY_OUT 0.9488834202289581
    
4. Suspension Status Snapshot [Attachment 4]
    
5. Anchal aligned the OPLEV beams [ELOG 16407]
   I also checked the 100 days trend of the OPLEV sum power. The trend of the max values look flat and fine. [Attachment 5] For this purpose, the PRM and SRM was aligned and the SRM oplev was also aligned. The SRM sum was 23580 when aligned and it was just fine (this is not so visible in the trend plot).
    
6. The X and Y green beams were aligned for the cavity TEM00s. Y end green PZT values were nulled. The transmission I could reach was as follows.
   >z read C1:ALS-TRX_OUTPUT C1:ALS-TRY_OUTPUT
0.42343354488901286
0.24739624058377277
   It seems that these GTRX and GTRY seemed to have crosstalk. When each green shutters were closed the transmissino and the dark offset were measured to be
   >z read C1:ALS-TRX_OUTPUT C1:ALS-TRY_OUTPUT
0.41822833190834546
0.025039383697636856
>z read C1:ALS-TRX_OUTPUT C1:ALS-TRY_OUTPUT
0.00021112720155274818
0.2249448773499293
   Note that Y green seemed to have significant (~0.1) of 1st order HOM. I don't know why I could not transfer this power into TEM00. I could not find any significant clipping of the TR beams on the PSL table PDs.
    
7. IMC Power reduction
   Now we have nice motorized HWP. sitemap -> PSL -> Power control
   == Initial condition == [Attachment 6]
   C1:IOO-HWP_POS 38.83
   Measured input power = 0.99W
   C1:IOO-MC_RFPD_DCMON = 5.38
   == Power reduction == [Attachment 7]
   - The motor was enabled upon rotation on the screen
   C1:IOO-HWP_POS 74.23
   Measured input power = 98mW
   C1:IOO-MC_RFPD_DCMON = 0.537
   - Then, the motor was disabled
    
8. Went to the detection table and swapped the 10% reflector with the 98% reflector stored on the same table. [Attachments 8/9]
   After the beam alignment the MC REFL PD received about the same amount of the light as before.
   C1:IOO-MC_RFPD_DCMON = 5.6
   There is no beam delivered to the WFS paths.
   CAUTION: IF THE POWER IS INCREASED TO THE NOMINAL WITH THIS CONFIGURATION, MC REFL PD WILL BE DESTROYED.
9. The IMC can already be locked with this configuration. But for the MC Autolocker, the MCTRANS threshold for the autolocker needs to be reduced as well.
   This was done by swapping a line in  /opt/rtcds/caltech/c1/scripts/MC/AutoLockMC.init
# BEFORE
/bin/csh ./AutoLockMC.csh >> $LOGFILE
#/bin/csh ./AutoLockMC_LowPower.csh >> $LOGFILE
--->
# AFTER
#/bin/csh ./AutoLockMC.csh >> $LOGFILE
/bin/csh ./AutoLockMC_LowPower.csh >> $LOGFILE
   Confirmed that the autolocker works a few times by toggling the PSL shutter. The PSL shutter was closed upon the completion of the test
    
10. Walked around the lab and checked all the bellows - the jam nuts are all tight, and I couldn't move them with my hands. So this is okay according to the ancient tale by Steve.

I centered all the optical levers on ITMX, ITMY, ETMX, ETMY, and BS to a position where the single arm lock on both were best aligned. Unfortunately, we are seeing the TRX at 0.78 and TRY at 0.76 at the most aligned positions. It seems less power is getting out of PMC since last month. (Attachment 1).

Then, I tried to lock PRMI with carrier with no luck. But I was able to see flashing of up to 4000 counts in POP_DC. At this position, I centered the PRM optical lever too (Attachment 2).

[Ian, Anchal]

we ran the free swinging test last night and the results match up with in 1/10th of a Hz. We calculated the peak using the getPeakFreqs2 script to find the peaks and they are close to previous values from 2016.

In attachment 1 you will see the results of the test for each optic.

The peak values are as follows:

The results from 2016 can be found at: /cvs/cds/rtcdt/caltech/c1/scripts/SUS/PeakFit/parameters2.m

{Yehonathan, Raj}

We aligned the IFO in the PRMI state and let it swing freely.

Flange Configuration for BHD

We will need total 5 new cable stands. So Qty.6 is the number to be ordered.

Looking at the accuglass drawing, the in-vaccum cables are standard dsub 25pin cables only with two standard fixing threads.

https://www.accuglassproducts.com/sites/default/files/PDF/Partpdf/110070_3.pdf

For SOSs, the standard 40m style cable bracket works fine. https://dcc.ligo.org/D010194-x0

However, for the OMCs, we need to make the thread holes available so that we can mate DB25 male cables to these cables.
One possibility is to improvise this cable bracket to suspend the cables using clean Cu wires or something. I think we can deal with this issue in situ.

Ha! The male side has the 4-40 standoff (jack) screws. So we can hold the male side on the bracket using the standoff (jack) screws and plug the female cables. OK! The issue solved!

https://www.accuglassproducts.com/sites/default/files/PDF/Partpdf/110029_3.pdf

[Ian, Anchal]

We are going to kick the optics tonight at 2am.

The optics we will kick are the PRM BS ITMX ITMY ETMX ETMY

We will kick each one once and record for 2000 seconds and the log files will be placed in users/ian/20211015_FreeSwingTest/logs.

Here is a side by side comparison of the PRMI sensing matrix using PRM/BS actuation (attachment 1) and ITMs actuation (attachment 2). The situation looks similar in both cases. That is, good orthogonality on REFL55 and bad seperation in the rest of the RFPDs.

{Yehonathan, Anchal}

I went to get a sandwich around 10:20 AM and when I came back BS was moving like crazy. We shutdown the watchdog.

We look at the spectra of the OSEMs (attachment 1). Clearly, the UR sensing is bad.

We took the BS sattelite box out. Anchal opened the box and nothing seemed wrong visually. We returned the box and connected it to the fake OSEM box. The sensor spectra seemed normal.

We connected the box to the vacuum chamber and the spectra is still normal (attachment 2).

We turn on the coils and the motion got damped very quickly (RMS <0.5mV).

Either the problem was solved by disconnecting and connecting the cables or it will come back to haunt us.

PMC has been unlocked since ~ 2:30 AM. Seems like the PZT got saturated. I moved the DC output adjuster and the PMC locked immidiatly although with a low transmission of 0.62V (>0.7V is the usual case) and high REFL.

IMC locked immidiately but IFO seems to be completely misaligned. The beams on the AS monitor are moving quite alot syncronously. BS watchdog tripped. I enabled the coil outputs. Waiting for the RMS motion to relax...

Its not relaxing. RMS motion is still high. I disabled the coils again and reenabled them. This seems to have worked. Arms were locked quite easily but the ETMs oplevs were way off and the ASS couldn't get the TRX and TRY more than 0.7. I align the ETMs to center the oplev. I realign everything else and lock the arms. Maximium TR is still < 0.8.

We did a few quick XARM oltf measurements. We excited C1:LSC-ETMX_EXC with a broadband white noise upto 4 kHz. The timestamps for the measurements are: 1318199043 (start) - 1318199427 (end).

We will process the measurement to compute the cavity pole and analog filter poles & zeros later.

Three extra steps (when adding new models, new FE):

• Chris pointed out that the sudo command in c1sus2 is giving error

  sudo: unable to resolve host c1sus2
  

  This error comes in when the computer could not figure out it's own hostname. Since FEs are network booted off the fb1, we need to update the /etc/hosts in /diskless/root everytime we add a new FE.

  controls@fb1:~ 0$ sudo chroot /diskless/root
  fb1:/ 0# sudo nano /etc/hosts
  fb1:/ 0# exit
  

  I added the following line in /etc/hosts file above:

  192.168.113.92  c1sus2 c1sus2.martian
  

  This resolved the issue of sudo giving error. Now, the rtcds make and install steps had no errors mentioned in their outputs.
• Another thing that needs to be done, as Koji pointed out, is to add the host and models in /etc/rtsystab in /diskless/root of fb:

  controls@fb1:~ 0$ sudo chroot /diskless/root
  fb1:/ 0# sudo nano /etc/rtsystab
  fb1:/ 0# exit
  

  I added the following lines in /etc/rtsystab file above:

  c1sus2   c1x07  c1su2
  

  This told rtcds what models would be available on c1sus2. Now rtcds list is displaying the right models:

  controls@c1sus2:~ 0$ rtcds list
  c1x07
  c1su2

• The above steps are still not sufficient for the daqd_ processes to know about the new models. This part is supossed to happen automatically, but does not happen in our CDS apparently. So everytime there is a new model, we need to edit the file /opt/rtcds/caltech/c1/target/daqd/master and add following lines to it:

  # Fast Data Channel lists
  # c1sus2
  /opt/rtcds/caltech/c1/chans/daq/C1X07.ini
  /opt/rtcds/caltech/c1/chans/daq/C1SU2.ini
  
  # test point lists
  # c1sus2
  /opt/rtcds/caltech/c1/target/gds/param/tpchn_c1x07.par
  /opt/rtcds/caltech/c1/target/gds/param/tpchn_c1su2.par
  

  I needed to restart the daqd_ processes in  fb1 for them to notice these changes:

  controls@fb1:~ 0$ sudo systemctl restart daqd_*
  

  This finally lit up the status channels of DC in C1X07_GDS_TP.adl and C1SU2_GDS_TP.adl . However the channels C1:DAQ-DC0_C1X07_STATUS and C1:DAQ-DC0_C1SU2_STATUS both have values 0x2bad. This persists on restarting the models. I then just simply restarted teh mx_stream on c1sus2 and boom, it worked! (see attached all green screen, never seen before!)

So now Ian can work on testing the I/O chassis and we would be good to move c1sus2 FE and I/O chassis to 1Y3 after that. I've also done following extra changes:

• Updated CDS_FE_STATUS medm screen to show the new c1sus2 host.
• Updated global diag rest script to act on c1xo7 and c1su2 as well.
• Updated mxstream restart script to act on c1sus2 as well.

Don't you need to add the new hosts to /diskless/root/etc/rtsystab at fb1? --> There looks many elogs talking about editing "rtsystab".

controls@fb1:/diskless/root/etc 0$ cat rtsystab
#
# host    list of control systems to run, starting with IOP
#
c1iscex  c1x01  c1scx c1asx
c1sus     c1x02  c1sus c1mcs c1rfm c1pem
c1ioo     c1x03  c1ioo c1als c1omc
c1lsc    c1x04  c1lsc c1ass c1oaf c1cal c1dnn c1daf
c1iscey  c1x05 c1scy c1asy
#c1test   c1x10  c1tst2

I connected c1sus2 to the martian network by splitting the c1sim connection with a 5-way switch. I also ran another ethernet cable from the second port of c1sus2 to the DAQ network switch on 1X7.

Then I logged into chiara and added the following in chiara:/etc/dhcp/dhcpd.conf :

host c1sus2 {
  hardware ethernet 00:25:90:06:69:C2;
  fixed-address 192.168.113.92;
}

And following line in chiara:/var/lib/bind/martian.hosts :

c1sus2          A    192.168.113.92

Note that entires c1bhd is already added in these files, probably during some earlier testing by Gautam or Jon. Then I ran following to restart the dhcp server and nameserver:

~> sudo service bind9 reload
[sudo] password for controls:
 * Reloading domain name service... bind9                                                 [ OK ]
~> sudo service isc-dhcp-server restart
isc-dhcp-server stop/waiting
isc-dhcp-server start/running, process 25764

Now, As I switched on c1sus2 from front panel, it booted over network from fb1 like other FE machines and I was able to login to it by first logging to fb1 and then sshing to c1sus2.

Next, I copied the simulink models and the medm screens of c1x06, xc1x07, c1bhd, c1sus2 from the paths mentioned on this wiki page. I also copied the medm screens from chiara(clone):/opt/rtcds/caltech/c1/medm to martian network chiara in the appropriate places. I have placed the file /opt/rtcds/caltech/c1/medm/teststand_sitemap.adl which can be used to open sitemap for c1bhd and c1sus2 IOP and user models.

Then I logged into c1sus2 (via fb1) and did make, install, start procedure:

controls@c1sus2:~ 0$ rtcds make c1x07
buildd: /opt/rtcds/caltech/c1/rtbuild/release
### building c1x07...
Cleaning c1x07...
Done
Parsing the model c1x07...
Done
Building EPICS sequencers...
Done
Building front-end Linux kernel module c1x07...
Done
RCG source code directory:
/opt/rtcds/rtscore/branches/branch-3.4
The following files were used for this build:
/opt/rtcds/userapps/release/cds/c1/models/c1x07.mdl

Successfully compiled c1x07
***********************************************
Compile Warnings, found in c1x07_warnings.log:
***********************************************
***********************************************
controls@c1sus2:~ 0$ rtcds install c1x07
buildd: /opt/rtcds/caltech/c1/rtbuild/release
### installing c1x07...
Installing system=c1x07 site=caltech ifo=C1,c1
Installing /opt/rtcds/caltech/c1/chans/C1X07.txt
Installing /opt/rtcds/caltech/c1/target/c1x07/c1x07epics
Installing /opt/rtcds/caltech/c1/target/c1x07
Installing start and stop scripts
/opt/rtcds/caltech/c1/scripts/killc1x07
/opt/rtcds/caltech/c1/scripts/startc1x07
sudo: unable to resolve host c1sus2
Performing install-daq
Updating testpoint.par config file
/opt/rtcds/caltech/c1/target/gds/param/testpoint.par
/opt/rtcds/rtscore/branches/branch-3.4/src/epics/util/updateTestpointPar.pl -par_file=/opt/rtcds/caltech/c1/target/gds/param/archive/testpoint_211012_174226.par -gds_node=24 -site_letter=C -system=c1x07 -host=c1sus2
Installing GDS node 24 configuration file
/opt/rtcds/caltech/c1/target/gds/param/tpchn_c1x07.par
Installing auto-generated DAQ configuration file
/opt/rtcds/caltech/c1/chans/daq/C1X07.ini
Installing Epics MEDM screens
Running post-build script

safe.snap exists 
controls@c1sus2:~ 0$ rtcds start c1x07
Cannot start/stop model 'c1x07' on host c1sus2.
controls@c1sus2:~ 4$ rtcds list

controls@c1sus2:~ 0$ 

One can see that even after making and installing, the model c1x07 is not listed as available models in rtcds list. Same is the case for c1sus2 as well. So I could not proceed with testing.

Good news is that nothing that I did affect the current CDS functioning. So we can probably do this testing safely from the main CDS setup.

Chris pointed out some information displaying scripts, that show if the DAQ network is working or not. I thought it would be nice to log this information here as well.

controls@fb1:/opt/mx/bin 0$ ./mx_info
MX Version: 1.2.16
MX Build: controls@fb1:/opt/src/mx-1.2.16 Mon Aug 14 11:06:09 PDT 2017
1 Myrinet board installed.
The MX driver is configured to support a maximum of:
    8 endpoints per NIC, 1024 NICs on the network, 32 NICs per host
===================================================================
Instance #0:  364.4 MHz LANai, PCI-E x8, 2 MB SRAM, on NUMA node 0
    Status:        Running, P0: Link Up
    Network:    Ethernet 10G

    MAC Address:    00:60:dd:45:37:86
    Product code:    10G-PCIE-8B-S
    Part number:    09-04228
    Serial number:    423340
    Mapper:        00:60:dd:45:37:86, version = 0x00000000, configured
    Mapped hosts:    3

                                                        ROUTE COUNT
INDEX    MAC ADDRESS     HOST NAME                        P0
-----    -----------     ---------                        ---
   0) 00:60:dd:45:37:86 fb1:0                             1,0
   1) 00:25:90:05:ab:47 c1bhd:0                           1,0
   2) 00:25:90:06:69:c3 c1sus2:0                          1,0

controls@c1bhd:~ 1$ /opt/open-mx/bin/omx_info
Open-MX version 1.5.4
 build: root@fb1:/opt/src/open-mx-1.5.4 Tue Aug 15 23:48:03 UTC 2017

Found 1 boards (32 max) supporting 32 endpoints each:
 c1bhd:0 (board #0 name eth1 addr 00:25:90:05:ab:47)
   managed by driver 'igb'

Peer table is ready, mapper is 00:60:dd:45:37:86
================================================
  0) 00:25:90:05:ab:47 c1bhd:0
  1) 00:60:dd:45:37:86 fb1:0
  2) 00:25:90:06:69:c3 c1sus2:0

controls@c1sus2:~ 0$ /opt/open-mx/bin/omx_info
Open-MX version 1.5.4
 build: root@fb1:/opt/src/open-mx-1.5.4 Tue Aug 15 23:48:03 UTC 2017

Found 1 boards (32 max) supporting 32 endpoints each:
 c1sus2:0 (board #0 name eth1 addr 00:25:90:06:69:c3)
   managed by driver 'igb'

Peer table is ready, mapper is 00:60:dd:45:37:86
================================================
  0) 00:25:90:06:69:c3 c1sus2:0
  1) 00:60:dd:45:37:86 fb1:0
  2) 00:25:90:05:ab:47 c1bhd:0

These outputs prove that the framebuilder and the FEs are able to see each other in teh DAQ network.

Further, the error that we see when IOP model is started which crashes the mx_stream service on the FE machines (see 40m/16391) :

isendxxx failed with status Remote Endpoint Unreachable

This has been seen earlier when Jamie was troubleshooting the current fb1 in martian network in 40m/11655 in Oct, 2015. Unfortunately, I could not find what Jamie did over a year to fix this issue.

should compare side by side with the ITM PRMI radar plots to see if there is a difference. How do your new plots compare with Gautam's plots of PRMI?

Late submission (From Thursday 10/07):

I measured the PRMI sensing matrix to see if the BS and PRMI output matrices tweaking had any effect.

While doing so, I noticed I made a mistake in the analysis of the previous sensing matrix measurement. It seems that I have used the radar plot function with radians where degrees should have been used (the reason is that the azimuthal uncertainty looked crazy when I used degrees. I still don't know why this is the case with this measurement).

In any case, attachment 1 and 2 show the PRMI radar plots with the modified output matrices and and in the normal state, respectively.

It seems like the output matrix modification didn't do anything but REFL55 has good orthogonality. Problem gone??

The teststand has some non-trivial issue with Myrinet card (either software or hardware) which even teh experts are saying they don't remember how to fix it. CDS with mx was iin use more than a decade ago, so it is hard to find support for issues with it now and will be the same in future. We need to wrap up this test procedure one way or another now, so I have following two options moving forward:

Direct integration with main CDS and testing

• We can just connect the c1sus2 and c1bhd FE computers to martian network directly.
• We'll have to connect c1sus2 and c1bhd to the optical fiber subnetwork as well.
• On booting, they would get booted through the exisitng fb1 boot server which seems to work fine for the other 5 FE machines.
• We can update teh DHCP in chiara and reload it so that we can ssh into these FEs with host names.
• Hopefully, presence of these computers won't tank the existing CDS even if they  themselves have any issues, as they have no shared memory with other models.
• If this works, we can do the loop back testing of I/O chassis using the main DAQ network and move on with our upgrade.
• If this does not work and causes any harm to exisitng CDS network, we can disconnect these computers and go back to existing CDS. Recently, our confidence on rebooting the CDS has increased with the robust performance as some legacy issues were fixed.
• We'll however, continue to use a CDS which is no more supported by the current LIGO CDS group.

Testing CDS upgrade on teststand

• From what I could gather, most of the hardware in I/O chassis that I could find, is still used in CDS of LLO and LHO, with their recent tests and documents using the same cards and PCBs.
• There might be some difference in the DAQ network setup that I need to confirm.
• I've summarised the current c1teststand hardware on this wiki page.
• If the latest CDS is backwards compatible with our hardware, we can test the new CDS in teh c1teststand setup without disrupting our main CDS. We'll have ample help and support for this upgrade from the current LIGO CDS group.
• We can do the loop back testing of the I/O chassis as well.
• If the upgrade is succesfull in the teststand without many hardware changes, we can upgrade the main CDS of 40m as well, as it has the same hardware as our teststand.
• Biggest plus point would be that out CDS will be up-to-date and we will be able to take help from CDS group if any trouble occurs.

So these are the two options we have. We should discuss which one to take in the mattermost chat or in upcoming meeting.

However, lspci | grep 'Myri' shows following output on both computers:

controls@fb1:/dev 0$ lspci | grep 'Myri'
02:00.0 Ethernet controller: MYRICOM Inc. Myri-10G Dual-Protocol NIC (rev 01)

Which means that the computer detects the card on PCie slot.

controls@fb1:sudo ./mx_start_stop start
Loading mx driver
Creating mx devices

I tried to add this to /etc/rc.local to run this script at every boot, but it did not work. So for now, I'll just manually do this step everytime. Once the devices are loaded, we get:

controls@fb1:/etc 0$ ls /dev/*mx*
/dev/mx0  /dev/mx4  /dev/mxctl   /dev/mxp2  /dev/mxp6         /dev/ptmx
/dev/mx1  /dev/mx5  /dev/mxctlp  /dev/mxp3  /dev/mxp7
/dev/mx2  /dev/mx6  /dev/mxp0    /dev/mxp4  /dev/open-mx
/dev/mx3  /dev/mx7  /dev/mxp1    /dev/mxp5  /dev/open-mx-raw

The, restarting all daqd_ processes, I found that daqd_dc was running succesfully now. Here is the status:

controls@fb1:/etc 0$ sudo systemctl status daqd_* -l
● daqd_dc.service - Advanced LIGO RTS daqd data concentrator
   Loaded: loaded (/etc/systemd/system/daqd_dc.service; enabled)
   Active: active (running) since Mon 2021-10-11 17:48:00 PDT; 23min ago
 Main PID: 2308 (daqd_dc_mx)
   CGroup: /daqd.slice/daqd_dc.service
           ├─2308 /usr/bin/daqd_dc_mx -c /opt/rtcds/caltech/c1/target/daqd/daqdrc.dc
           └─2370 caRepeater

Oct 11 17:48:07 fb1 daqd_dc_mx[2308]: mx receiver 006 thread priority error Operation not permitted[Mon Oct 11 17:48:06 2021]
Oct 11 17:48:07 fb1 daqd_dc_mx[2308]: mx receiver 005 thread put on CPU 0
Oct 11 17:48:07 fb1 daqd_dc_mx[2308]: [Mon Oct 11 17:48:06 2021] [Mon Oct 11 17:48:06 2021] mx receiver 006 thread put on CPU 0
Oct 11 17:48:07 fb1 daqd_dc_mx[2308]: mx receiver 007 thread put on CPU 0
Oct 11 17:48:07 fb1 daqd_dc_mx[2308]: [Mon Oct 11 17:48:06 2021] mx receiver 003 thread - label dqmx003 pid=2362
Oct 11 17:48:07 fb1 daqd_dc_mx[2308]: [Mon Oct 11 17:48:06 2021] mx receiver 003 thread priority error Operation not permitted
Oct 11 17:48:07 fb1 daqd_dc_mx[2308]: [Mon Oct 11 17:48:06 2021] mx receiver 003 thread put on CPU 0
Oct 11 17:48:07 fb1 daqd_dc_mx[2308]: warning:regcache incompatible with malloc
Oct 11 17:48:07 fb1 daqd_dc_mx[2308]: [Mon Oct 11 17:48:06 2021] EDCU has 410 channels configured; first=0
Oct 11 17:49:06 fb1 daqd_dc_mx[2308]: [Mon Oct 11 17:49:06 2021] ->4: clear crc

● daqd_fw.service - Advanced LIGO RTS daqd frame writer
   Loaded: loaded (/etc/systemd/system/daqd_fw.service; enabled)
   Active: active (running) since Mon 2021-10-11 17:48:01 PDT; 23min ago
 Main PID: 2318 (daqd_fw)
   CGroup: /daqd.slice/daqd_fw.service
           └─2318 /usr/bin/daqd_fw -c /opt/rtcds/caltech/c1/target/daqd/daqdrc.fw

Oct 11 17:48:09 fb1 daqd_fw[2318]: [Mon Oct 11 17:48:09 2021] [Mon Oct 11 17:48:09 2021] Producer thread - label dqproddbg pid=2440
Oct 11 17:48:09 fb1 daqd_fw[2318]: Producer crc thread priority error Operation not permitted
Oct 11 17:48:09 fb1 daqd_fw[2318]: [Mon Oct 11 17:48:09 2021] [Mon Oct 11 17:48:09 2021] Producer crc thread put on CPU 0
Oct 11 17:48:09 fb1 daqd_fw[2318]: Producer thread priority error Operation not permitted
Oct 11 17:48:09 fb1 daqd_fw[2318]: [Mon Oct 11 17:48:09 2021] Producer thread put on CPU 0
Oct 11 17:48:09 fb1 daqd_fw[2318]: [Mon Oct 11 17:48:09 2021] Producer thread - label dqprod pid=2434
Oct 11 17:48:09 fb1 daqd_fw[2318]: [Mon Oct 11 17:48:09 2021] Producer thread priority error Operation not permitted
Oct 11 17:48:09 fb1 daqd_fw[2318]: [Mon Oct 11 17:48:09 2021] Producer thread put on CPU 0
Oct 11 17:48:10 fb1 daqd_fw[2318]: [Mon Oct 11 17:48:10 2021] Minute trender made GPS time correction; gps=1318034906; gps%60=26
Oct 11 17:49:09 fb1 daqd_fw[2318]: [Mon Oct 11 17:49:09 2021] ->3: clear crc

● daqd_rcv.service - Advanced LIGO RTS daqd testpoint receiver
   Loaded: loaded (/etc/systemd/system/daqd_rcv.service; enabled)
   Active: active (running) since Mon 2021-10-11 17:48:00 PDT; 23min ago
 Main PID: 2311 (daqd_rcv)
   CGroup: /daqd.slice/daqd_rcv.service
           └─2311 /usr/bin/daqd_rcv -c /opt/rtcds/caltech/c1/target/daqd/daqdrc.rcv

Oct 11 17:50:21 fb1 daqd_rcv[2311]: Creating C1:DAQ-NDS0_C1X07_CRC_SUM
Oct 11 17:50:21 fb1 daqd_rcv[2311]: Creating C1:DAQ-NDS0_C1BHD_STATUS
Oct 11 17:50:21 fb1 daqd_rcv[2311]: Creating C1:DAQ-NDS0_C1BHD_CRC_CPS
Oct 11 17:50:21 fb1 daqd_rcv[2311]: Creating C1:DAQ-NDS0_C1BHD_CRC_SUM
Oct 11 17:50:21 fb1 daqd_rcv[2311]: Creating C1:DAQ-NDS0_C1SU2_STATUS
Oct 11 17:50:21 fb1 daqd_rcv[2311]: Creating C1:DAQ-NDS0_C1SU2_CRC_CPS
Oct 11 17:50:21 fb1 daqd_rcv[2311]: Creating C1:DAQ-NDS0_C1SU2_CRC_SUM
Oct 11 17:50:21 fb1 daqd_rcv[2311]: Creating C1:DAQ-NDS0_C1OM[Mon Oct 11 17:50:21 2021] Epics server started
Oct 11 17:50:24 fb1 daqd_rcv[2311]: [Mon Oct 11 17:50:24 2021] Minute trender made GPS time correction; gps=1318035040; gps%120=40
Oct 11 17:51:21 fb1 daqd_rcv[2311]: [Mon Oct 11 17:51:21 2021] ->3: clear crc

Now, even before starting teh FE models, I see DC status as ox2bad in the CDS screens of the IOP and user models. The mx_stream service remains in a failed state at teh FE machines and remain the same even after restarting the service.

controls@c1sus2:~ 0$ sudo systemctl status mx_stream -l
● mx_stream.service - Advanced LIGO RTS front end mx stream
   Loaded: loaded (/etc/systemd/system/mx_stream.service; enabled)
   Active: failed (Result: exit-code) since Mon 2021-10-11 17:50:26 PDT; 15min ago
  Process: 382 ExecStart=/etc/mx_stream_exec (code=exited, status=1/FAILURE)
 Main PID: 382 (code=exited, status=1/FAILURE)

Oct 11 17:50:25 c1sus2 systemd[1]: Starting Advanced LIGO RTS front end mx stream...
Oct 11 17:50:25 c1sus2 systemd[1]: Started Advanced LIGO RTS front end mx stream.
Oct 11 17:50:25 c1sus2 mx_stream_exec[382]: Failed to open endpoint Not initialized
Oct 11 17:50:26 c1sus2 systemd[1]: mx_stream.service: main process exited, code=exited, status=1/FAILURE
Oct 11 17:50:26 c1sus2 systemd[1]: Unit mx_stream.service entered failed state.

But  if I restart the mx_stream service before starting the rtcds models, the mx-stream service starts succesfully:

controls@c1sus2:~ 0$ sudo systemctl restart mx_stream
controls@c1sus2:~ 0$ sudo systemctl status mx_stream -l
● mx_stream.service - Advanced LIGO RTS front end mx stream
   Loaded: loaded (/etc/systemd/system/mx_stream.service; enabled)
   Active: active (running) since Mon 2021-10-11 18:14:13 PDT; 25s ago
 Main PID: 1337 (mx_stream)
   CGroup: /system.slice/mx_stream.service
           └─1337 /usr/bin/mx_stream -e 0 -r 0 -w 0 -W 0 -s c1x07 c1su2 -d fb1:0

Oct 11 18:14:13 c1sus2 systemd[1]: Starting Advanced LIGO RTS front end mx stream...
Oct 11 18:14:13 c1sus2 systemd[1]: Started Advanced LIGO RTS front end mx stream.
Oct 11 18:14:13 c1sus2 mx_stream_exec[1337]: send len = 263596
Oct 11 18:14:13 c1sus2 mx_stream_exec[1337]: Connection Made

However, the DC status on CDS screens still show 0x2bad. As soon as I start the rtcds model c1x07 (the IOP model for c1sus2), the mx_stream service fails:

controls@c1sus2:~ 0$ sudo systemctl status mx_stream -l
● mx_stream.service - Advanced LIGO RTS front end mx stream
   Loaded: loaded (/etc/systemd/system/mx_stream.service; enabled)
   Active: failed (Result: exit-code) since Mon 2021-10-11 18:18:03 PDT; 27s ago
  Process: 1337 ExecStart=/etc/mx_stream_exec (code=exited, status=1/FAILURE)
 Main PID: 1337 (code=exited, status=1/FAILURE)

Oct 11 18:14:13 c1sus2 systemd[1]: Starting Advanced LIGO RTS front end mx stream...
Oct 11 18:14:13 c1sus2 systemd[1]: Started Advanced LIGO RTS front end mx stream.
Oct 11 18:14:13 c1sus2 mx_stream_exec[1337]: send len = 263596
Oct 11 18:14:13 c1sus2 mx_stream_exec[1337]: Connection Made
Oct 11 18:18:03 c1sus2 mx_stream_exec[1337]: isendxxx failed with status Remote Endpoint Unreachable
Oct 11 18:18:03 c1sus2 mx_stream_exec[1337]: disconnected from the sender
Oct 11 18:18:03 c1sus2 mx_stream_exec[1337]: c1x07_daq mmapped address is 0x7fe3620c3000
Oct 11 18:18:03 c1sus2 mx_stream_exec[1337]: c1su2_daq mmapped address is 0x7fe35e0c3000
Oct 11 18:18:03 c1sus2 systemd[1]: mx_stream.service: main process exited, code=exited, status=1/FAILURE
Oct 11 18:18:03 c1sus2 systemd[1]: Unit mx_stream.service entered failed state.

This shows that the start of rtcds model, causes the fail in mx_stream, possibly due to inability of finding the endpoint on fb1. I've again reached to the edge of my knowledge here. Maybe the fiber optic connection between fb and the network switch that connects to FE is bad, or the connection between switch and FEs is bad.

But we are just one step away from making this work.

We report here the analysis results for the measurements done in elog:16388. 

Figs. 1 & 2 are respectively measurements of the white noise excitation and the optimized excitation. The shaded region corresponds to the 1-sigma uncertainty at each frequency bin. By eyes, one can already see that the constraints on the phase in the 0.6-1 Hz band are much tighter in the optimized case than in the white noise case. 

We found the transfer function was best described by two real poles + one pair of complex poles (i.e., resonance) + one pair of complex zeros in the right-half plane (non-minimum phase delay). The measurement in fact suggested a right-hand pole somewhere between 0.05-0.1 Hz which cannot be right. For now, I just manually flipped the sign of this lowest frequency pole to the left-hand side. However, this introduced some systematic deviation in the phase in the 0.3-0.5 Hz band where our coherence was still good. Therefore, a caveat is that our model with 7 free parameters (4 poles + 2 zeros + 1 gain as one would expect for an ideal signal-stage L2P TF) might not sufficiently capture the entire physics. 

In Fig. 3 we showed the comparison of the two sets of measurements together with the predictions based on the Fisher matrix. Here the color gray is for the white-noise excitation and olive is for the optimized excitation. The solid and dotted contours are respectively the 1-sigma and 3-sigma regions from the Fisher calculation, and crosses are maximum likelihood estimations of each measurement (though the scipy optimizer might not find the true maximum).

Note that the mean values don't match in the two sets of measurements, suggesting potential bias or other systematics exists in the current measurement. Moreover, there could be multiple local maxima in the likelihood in this high-D parameter space (not surprising). For example, one could reduce the resonant Q but enhance the overall gain to keep the shoulder of a resonance having the same amplitude. However, this correlation is not explicit in the Fisher matrix (first-order derivatives of the TF, i.e., local gradients) as it does not show up in the error ellipse. 

In Fig. 4 we show the further optimized excitation for the next round of measurements. Here the cyan and olive traces are obtained assuming different values of the "true" physical parameter, yet the overall shapes of the two are quite similar, and are close to the optimized excitation spectrum we already used in elog:16388. 

For the oplev, there are DQ channels you can use so that its possible to look back in the past for long measurements. They have names like PERROR

[Raj, Hang]

We did some more measurements on the PRM L2P TF. 

We tried to compare the parameter estimation uncertainties of white vs. optimal excitation. We drove C1:SUS-PRM_LSC_EXC with "Normal" excitation and digital gain of 700.

For the white noise exciation, we simply put a butter("LowPass",4,10) filter to select out the <10 Hz band.

For the optimal exciation, we use butter("BandPass",6,0.3,1.6) gain(3) notch(1,20,8) to approximate the spectral shape reported in elog:16384. We tried to use awg.ArbitraryLoop yet this function seems to have some bugs and didn't run correctly; an issue has been submitted to the gitlab repo with more details. We also noticed that in elog:16384, the pitch motion should be read out from C1:SUS-PRM_OL_PIT_IN1 instead of the OUT channel, as there are some extra filters between IN1 and OUT. Consequently, the exact optimal exciation should be revisited, yet we think the main result should not be altered significantly.

While a more detail analysis will be done later offline, we post in the attached plot a comparison between the white (blue) vs optimal (red) excitation. Note in this case, we kept the total force applied to the PRM the same (as the RMS level matches).

Under this simple case, the optimal excitation appears reasonable in two folds.

First, the optimization tries to concentrate the power around the resonance. We would naturally expect that near the resonance, we would get more Fisher information, as the phase changes the fastest there (i.e., large derivatives in the TF).

Second, while we move the power in the >2 Hz band to the 0.3-2 Hz band, from the coherence plot we see that we don't lose any information in the > 2 Hz region. Indeed, even with the original white excitation, the coherence is low and the > 2 Hz region would not be informative. Therefore, it seems reasonable to give up this band so that we can gain more information from locations where we have meaningful coherence.

The 4 units of Satellite Amp Adapter were done:
- The ears were fixed with the screws
- The handles were attached (The stock of the handles is low)
- The boards are now supported by plastic stand-offs. (The chassis were drilled)
- The front and rear panels were fixed to the chassis
- The front and rear connectors were fixed with the low profile 4-40 stand-off screws (3M 3341-1S)
 

[Tega, Koji]

(S2100737) - Debugging showed that the opamp, AD822ARZ, for PD2 circuit was not working as expected so we replaced with a spare and this fixed the problem. Somehow, the PD1 circuit no longer presents any issues, so everything is now fine with the unit.

(S2100741) - All good.

Note that your tests were done with the output matrix for BS and PRM in the compensated state as done in 40m/16374. The changes made there were supposed to clear out any coil actuation imbalance in the angular degrees of freedom.

[Paco, Hang]

Yesterday afternoon Paco and I measured the PRM L2P transfer function. We drove C1:SUS-PRM_LSC_EXC with a white noise in the 0-10 Hz band (effectively a white, longitudinal force applied to the suspension) and read out the pitch response in C1:SUS-PRM_OL_PIT_OUT. The local damping was left on during the measurement. Each FFT segment in our measurement is 32 sec and we used 8 non-overlapping segments for each measurement. The empirically determined results are also compared with the Fisher matrix estimation (similar to elog:16373).

Results:

Fig. 1 shows one example of the measured L2P transfer function. The gray traces are measurement data and shaded region the corresponding uncertainty. The olive trace is the best fit model. 

Note that for a single-stage suspension, the ideal L2P TF should have two zeros at DC and two pairs of complex poles for the length and pitch resonances, respectively. We found the two resonances at around 1 Hz from the fitting as expected. However, the zeros were not at DC as the ideal, theoretical model suggested. Instead, we found a pair of right-half plane zeros in order to explain the measurement results. If we cast such a pair of right-half plane zeros into (f, Q) pair, it means a negative value of Q. This means the system does not have the minimum phase delay and suggests some dirty cross-coupling exists, which might not be surprising. 

Fig. 2 compares the distribution of the fitting results for 4 different measurements (4 red crosses) and the analytical error estimation obtained using the Fisher matrix (the gray contours; the inner one is the 1-sigma region and the outer one the 3-sigma region). The Fisher matrix appears to underestimate the scattering from this experiment, yet it does capture the correlation between different parameters (the frequencies and quality factors of the two resonances).

One caveat though is that the fitting routine is not especially robust. We used the vectfit routine w/ human intervening to get some initial guesses of the model. We then used a standard scipy least-sq routine to find the maximal likelihood estimator of the restricted model (with fixed number of zeros and poles; here 2 complex zeros and 4 complex poles). The initial guess for the scipy routine was obtained from the vectfit model.  

Fig. 3 shows how we may shape our excitation PSD to maximize the Fisher information while keeping the RMS force applied to the PRM suspension fixed. In this case the result is very intuitive. We simply concentrate our drive around the resonance at ~ 1 Hz, focusing on locations where we initially have good SNR. So at least code is not suggesting something crazy... 

Fig. 4 then shows how the new uncertainty (3-sigma contours) should change as we optimize our excitation. Basically one iteration (from gray to olive) is sufficient here. 

We will find a time very recently to repeat the measurement with the optimized injection spectrum.

[Paco, Rana]

We had a look at the BS actuation. Along the way we created a couple of issues that we fixed. A summary is below.

1. First, we locked MICH. While doing this, we used the /users/Templates/ndscope/LSC/MICH.yml ndscope template to monitor some channels. I edited the yaml file to look at C1:LSC-ASDC_OUT_DQ instead of the REFL_DC. Rana pointed out that the C1:LSC-MICH_OUT_DQ (MICH control point) had a big range (~ 5000 counts rms) and this should not be like that.
2. We tried to investigate the aforementioned thing by looking at the whitening / uwhitening filters but all the slow epics channels where "white" on the medm screen. Looking under CDS/slow channel monitors, we realized that both c1iscaux and c1auxey were weird, so we tried telnet to c1iscaux without success. Therefore, we followed the recommended wiki procedure of hard rebooting this machine. While inside the lab and looking for this machine, we touched things around the 'rfpd' rack and once we were back in the control room, we couldn't see any light on the AS port camera. But the whitening filter medm screens were back up.
3. While rana ssh'd into c1auxey to investigate about its status, and burtrestored the c1iscaux channels, we looked at trends to figure out if anything had changed (for example TT1 or TT2) but this wasn't the case. We decided to go back inside to check the actual REFL beams and noticed it was grossly misaligned (clipping)... so we blamed it on the TTs and again, went around and moved some stuff around the 'rfpd' rack. We didn't really connect or disconnect anything, but once we were back in the control room, light was coming from the AS port again. This is a weird mystery and we should systematically try to repeat this and fix the actual issue.
4. We restored the MICH, and returned to BS actuation problems. Here, we essentially devised a scheme to inject noise at 310.97 Hz and 313.74. The choice is twofold, first it lies outside the MICH loop UGF (~150 Hz), and second, it matches the sensing matrix OSC frequencies, so it's more appropriate for a comparison.
5. We injected two lines using the BS SUS LOCKIN1 and LOCKIN2 oscilators so we can probe two coils at once, with the LSC loop closed, and read back using the C1:LSC-MICH_IN1_DQ channel. We excited with an amplitude of 1234.0 counts and 1254 counts respectively (to match the ~ 2 % difference in frequency) and noted that the magnitude response in UR was 10% larger than UL, LL, and LR which were close to each other at the 2% level.

[Paco]

After rana left, I did a second pass at the BS actuation. I took TF measurements at the oscilator frequencies noted above using diaggui, and summarize the results below:

This procedure should be done with PRM as well and using the PRCL instead of MICH.