40m QIL Cryo_Lab CTN SUS_Lab CAML OMC_Lab CRIME_Lab FEA ENG_Labs OptContFac Mariner WBEEShop
  40m Log, Page 125 of 354  Not logged in ELOG logo
ID Date Authorup Type Category Subject
  16631   Fri Jan 28 11:30:52 2022 KojiSummaryBHDPart III of BHR upgrade - PR2 OSEM finalized, reinsall LO1 OSEMs

All green! Great work, Team!

  16635   Tue Feb 1 15:33:28 2022 KojiSummaryBHDOptomechanics configuration for the in-vacuum aux small optics

I've summarized the optomechanics configuration for the in-vacuum aux small optics

It's not obvious here but the post for POP_SM4 is the stack of BA2V, Newport 9953, PLS-T238, LMR1V. The mirror is a CM254-750-E03 curved mirror. This should go on the suspension base. I hope I did not make a mistake there...

Attachment 1: Invac_opto_mechanics_v2.pdf
Invac_opto_mechanics_v2.pdf Invac_opto_mechanics_v2.pdf Invac_opto_mechanics_v2.pdf Invac_opto_mechanics_v2.pdf
  16640   Wed Feb 2 18:55:58 2022 KojiSummaryBHDOptomechanics configuration for the in-vacuum aux small optics

Here is more detail of the POP_SM4 mount assembly.

It's a combination of BA2V + PLS-T238 + BA1V + TR-1.5 + LMR1V + Mirror: CM254-750-E03
Between BA1V and PLS-T238, we have to do a washer action to fix the post (8-32) with a 1/4-20 slot. Maybe we can use a 1" post shim from thorlabs/newport.
Otherwise, we should be able to fasten the other joints with silver-plated screws we already have/ordered.

I think TR-1.5 (and a shim) has not been given to Jordan for C&B. I'll take a look at these.

Attachment 1: Screen_Shot_2022-02-02_at_6.46.46_PM.png
Screen_Shot_2022-02-02_at_6.46.46_PM.png
  16641   Wed Feb 2 21:16:23 2022 KojiSummaryBHDOptomechanics configuration for the in-vacuum aux small optics

Asked Jordan to clean 2x 1.5" posts (0.5 dia) and a washer with 1" dia.

 

Attachment 1: PXL_20220203_050156031.jpg
PXL_20220203_050156031.jpg
  16651   Mon Feb 7 16:53:02 2022 KojiUpdateGeneralScheduled power outage recovery

I went to the X end and found it was warm. Turned out that not all the A/Cs were on. They were turned on now.

Attachment 1: PXL_20220208_001646282.jpg
PXL_20220208_001646282.jpg
Attachment 2: PXL_20220208_001657871.jpg
PXL_20220208_001657871.jpg
  16653   Wed Feb 9 13:55:05 2022 KojiUpdateGeneralBringing back CDS

Great recovery work and cleaning of the rebooting process.

I'm just curious: Did you observe that the c1sus2 cards have different numbering order than the previous along with the power outage/cycling?

  16656   Thu Feb 10 14:39:31 2022 KojiSummaryComputersNetwork security issue resolved

[Mike P / Koji / Tega / Anchal]

IMSS/LIGO IT notified us that "ILOM ports" of one of our hosts on the "114" network are open. We tried to shut down obvious machines but could not identify the host in question. So we decided to do a bit more systematic search of the host.

[@Network Rack]
- First of all, we disconnected the optical cables coming to the GC router while the ping is running on the AIRLIGO connected laptop (i.e. outside of the 40m network). This made the ping stopped. This means that the issue was definitely in the 40m.
- Secondly, we started to disconnect (and reconnect) the ethernet cables from the GC router one by one. We found that the ping response stops when the cable named "NODUS" was disconnected.

[@40m IFO lab]
- So we tracked the cable down in the 40m lab. After a while, we identified that the cable was really connected to nodus.

- Nodus was supposed to have one network connection to the martian network since the introduction of the bidirectional NAT router (rather than the old configuration with a single direction NAT router).

- In fact, the cable was connected to "non-networking" port of nodus. (Attachment 1). I guess the cable was connected like this long time, but somehow the ILOM (IPMI) port was activated along with the recent power cycling.

- The cable was disconnected at nodus too. (Attachment 2) And a tape was attached to the port so that we don't connect anything to the port anymore.

Attachment 1: PXL_20220210_220816955.jpg
PXL_20220210_220816955.jpg
Attachment 2: PXL_20220210_220827167.jpg
PXL_20220210_220827167.jpg
  16659   Thu Feb 10 19:03:23 2022 KojiUpdateGeneralScheduled power outage recovery - Locking mode cleaner(s)

I came back to the 40m and started the investigation.

If I ping 192.168.113.92, it responds. But telnet (port 23) was rejected. I somehow tried ssh and it responds! I even could login to the host using usual password. Here is the prompt.

controls@nodus|~> ssh 192.168.113.92
controls@192.168.113.92's password:

...
controls@c1sus2:~ 0$

Oh no...

Looks like c1sus2 and the videomux have the IP address conflict.

Here are the useful ELOG links:

https://nodus.ligo.caltech.edu:8081/40m/4498

https://nodus.ligo.caltech.edu:8081/40m/4529

  16660   Thu Feb 10 19:46:37 2022 KojiUpdateGeneralScheduled power outage recovery - Locking mode cleaner(s)

== Assign new IP address to c1sus2 ==

cf: [40m ELOG 16398] [40m ELOG 16396]

- Shutdown c1sus2 (Oh, no. This killed c1lsc/c1sus/c1ioo... This should be taken care of later)

- Confirmed 192.168.113.87 is not alive

- Go to chiara
- Modify /diskless/root/etc/hosts

192.168.113.87  c1sus2 c1sus2.martian

- Modify /etc/dhcp/dhcpd.conf

host c1sus2 {
  hardware ethernet 00:25:90:06:69:C2;
  fixed-address 192.168.113.87;
}

- Modify /var/lib/bind/martian.hosts

c1sus2          A    192.168.113.87
videomux        A    192.168.113.92

- Modify /var/lib/bind/martian.hosts/rev.113.168.192.in-addr.arpa

87            PTR    c1sus2.martian
92            PTR    videomux.martian

- Reload/restart bind9 / dhcpd. Run the following command

sudo service bind9 reload
sudo service isc-dhcp-server restart

- Restart c1sus2 and confirm if the IP address was actually changed

controls@c1sus2:~ 0$ /sbin/ifconfig
eth0      Link encap:Ethernet  HWaddr 00:25:90:06:69:c2
          inet addr:192.168.113.87  Bcast:192.168.113.255  Mask:255.255.255.0
...

== Restart c1lsc / c1sus /c1ioo ==

- Reboot c1lsc/c1sus/c1ioo

- Go to scripts/cds

- Run startC1LSC.sh and follow the instruction

 

  16661   Thu Feb 10 21:10:43 2022 KojiUpdateGeneralVideo Mux setting reset

Now the video matrix is responding correctly and the web interface shows up. (Attachment 1)

Also the video buttons respond as usual. I pushed Locking Template button to bring the setting back to nominal. (Attachment 2)

Attachment 1: Screenshot_2022-02-10_21-11-21.png
Screenshot_2022-02-10_21-11-21.png
Attachment 2: Screenshot_2022-02-10_21-11-54.png
Screenshot_2022-02-10_21-11-54.png
  16662   Thu Feb 10 21:16:27 2022 KojiSummaryCDSchiara resolv.conf wierdo

During the videomux debug, I noticed that the host name resolving on chiara didn't behave well. Basically I could not login to anything from chiara using host names.

I found that there was no /etc/resolv.conf. Instead, there is /etc/resolvconf directory.

According to my research, live resolv.conf is placed in /run/resolveconf/resolv.conf .

# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
#     DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
nameserver 192.168.113.20
nameserver 131.215.125.1
nameserver 8.8.8.8

This 113.20 is directing an old "linux1" machine. Too much obsolete. If I modify this file as

# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
#     DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
nameserver 192.168.113.104
nameserver 131.215.125.1
nameserver 8.8.8.8
search martian

Then the name resolving became reasonable. However, during rebooting / service resetting / etc, resolvconf -u command is executed and /run/resolveconf/resolv.conf is overridden, as indicated in the file.

I have modified /etc/resolvconf/resolv.conf.d/base to include 192.168.113.104 and search martian . The latter was included but the former did not show up.

FInally I figured out that, after the resolv.conf is constructed from base and head files in /etc/resolvconf/resolv.conf.d/ , NetworkManager overrides the nameserver addresses.
The configuration was found in /etc/NetworkManager/system-connections/Wired\ connection\ 1 .

Here is the modified setting (dns entry was modified)

>sudo cat /etc/NetworkManager/system-connections/Wired\ connection\ 1
[sudo] password for controls:
[802-3-ethernet]
duplex=full
mac-address=68:05:CA:36:4E:B4

[connection]
id=Wired connection 1
uuid=ed177e70-d10e-42be-8165-3bf59f8f199d
type=802-3-ethernet
timestamp=1438810765

[ipv6]
method=auto

[ipv4]
method=manual
dns=192.168.113.104;131.215.125.1;8.8.8.8;
addresses1=192.168.113.104;24;192.168.113.2;

And

>cat /etc/resolvconf/resolv.conf.d/base
search martian
# See Also /etc/NetworkManager/system-connections/Wired\ connection\ 1

So complicated...

  16663   Thu Feb 10 21:51:02 2022 KojiUpdateCDS[Solved] Huge random numbers flowing into ETMX/ETMY ASC PIT/YAW

Huge random numbers are flowing into ETMX/ETMY ASC PIT/YAW. Because of this, I could not damp the ETMX/ETMY suspension at the beginning during the recovery from rebooting. (Attachment 1)
By turning off the output of the ASC filters, the mirrors were successfully damped.

Looking at the FE model view of the end RTSs, there were two possibilities: (Attachment 2)

- They are coming from RFM connection
- They are coming from ASXASY

ASX/ASY are not active and I could not see anything producing these numbers. Burtrestore didn't help.

The possibility was something at the other side of the RFM, or corruption of the RFM signal.

- Looking at the RFM model (Attachment 3), the ASC signals are coming from ASS and IOO. The ASS path has the filter module (C1:RFM-ETMX_PIT and etc). This FM is quiet and not guilty.

- Why do we have the RFM from IOO? I went to IOO and found the new ASC (WFS) model is there. I didn't realize the presence of this model. In fact ASC screen showed that these random numbers are flowing into the end SUSs.
So I did burtrestore of c1iooepics. Alas! they are gone.

Now I can go home.

Attachment 1: Screenshot_2022-02-10_21-46-02.png
Screenshot_2022-02-10_21-46-02.png
Attachment 2: Screen_Shot_2022-02-10_at_21.54.21.png
Screen_Shot_2022-02-10_at_21.54.21.png
Attachment 3: Screen_Shot_2022-02-10_at_22.14.23.png
Screen_Shot_2022-02-10_at_22.14.23.png
  16671   Mon Feb 14 21:03:25 2022 KojiUpdateGeneralScheduled power outage recovery

I opened the boxes. Allegra has obvious vent of at least 4 caps. And the power supply did not respond even a paper clip test was performed. https://www.silverstonetek.com/downloads/QA/PSU/PSU-Paper%20Clip-EN.pdf (Paper Clip Test)
=> The mother board and the PSU are dead.

Then Ottavia was also checked. The mother board looked OK, but the PSU did not respond. I quickly opened the PSU and it had a bunch of bulged capacitors in it. => PSU dead

Conclusion: Save the cards/memory etc as much as possible. Migrate the allegra HDD to any other healthy PC or obtain a new used PC from Larry. Otherwise, we just want to buy another WS and copy the disk in it.

 

Attachment 1: PXL_20220215_025325118.jpg
PXL_20220215_025325118.jpg
  16672   Tue Feb 15 19:32:50 2022 KojiUpdateGeneralScheduled power outage recovery - IMC recovery progress

Reduced the IMC power to 100mW

Setup: The power meter was placed right before the final aperture (Attachment 1)

Before the adjustment: Initial position of the HWP was 37.29deg and the input power was 987mW (Attachments 2/3)

After the adjustment: Initial position of the HWP was 74.00deg and the input power was 100mW (Attachments 4/5)

This made the MCREFL reading 0.549.

The MC refl path optics has not been modified.

Attachment 1: PXL_20220216_001731377.jpg
PXL_20220216_001731377.jpg
Attachment 2: Screen_Shot_2022-02-15_at_16.18.16.png
Screen_Shot_2022-02-15_at_16.18.16.png
Attachment 3: PXL_20220216_001727465.jpg
PXL_20220216_001727465.jpg
Attachment 4: Screen_Shot_2022-02-15_at_16.22.16.png
Screen_Shot_2022-02-15_at_16.22.16.png
Attachment 5: PXL_20220216_002229572.jpg
PXL_20220216_002229572.jpg
  16673   Tue Feb 15 19:40:02 2022 KojiUpdateGeneralIMC locking

IMC is locking now. There was nothing wrong: just a careful alignment + proper gain adj

=== Primary Alignment ===

- I used WFS error signals as the indicator of the PDH error signals. Checked C1:IOO-WFS1_(I/Q)n_ERR and ended up C1:IOO-WFS1_I4_ERR as it showed the largest PDH error PP.

- Then used MC2 and MC3 to align the IMC by maximizing the PDH error and the MC trans (C1:IOO-MC_TRANS_SUM_ERR)

=== Locking procedure ===

Note that the MC REFL path is still configured for the full power input

- (Only at the beginning) Run scripts/MC/mcdown for initialization / Run scripts/MC/MC2tickleOFF just in case

- Enable IOO-MC-SW1 (MC SERVO switch right after "IN1 Gain (dB)").
- Disable 40:4000 boost
- Increase VCO Gain from -15 to 0
- Jiggle IN1 Gain from low to +31 until the lock is achieved

- As soon as the lock is acquired, enable 40:4000
- Increase VCO Gain to +10
- Turn up "SUPER BOOST" from 0 to 3

=== Lock loss procedure ===

Note that the MC REFL path is still configured for the full power input

- Disable IOO-MC-SW1
- Disable 40:4000 boost
- Reduce VCO Gain 0
- Turn down "SUPER BOOST" to 0

- Then jiggle IN1 Gain again to lock the IMC

=== MC2 spot ===

- It was obvious that the MC2F spot was not on the center of the optic.
- I tried to move the spot on the camera as much as possible, but this did not make the trans beam to the center of the MC end QPD
- I had the impression that the trans beam started to be clipped when the beam was moved towards the end QPD,

We need to reestablish the reasonable/consistent MC2 spot on the mirror, the MC end optics, and the QPD.
We will need to use MC2 dithering and A2L coupling to determine the center of the mirror

But as long as the transmission is maximized, the transmitted beam thru MC1 and MC3 follows the input beam. So we can continue the vent work

The current maximized transmission was ~1300. MC1 refl CCD view was largely off -> The camera path was adjusted.

=== MC2 alignment note ===

During the alignment, I noticed a sudden change of the MC2 alignment. There might be some hysteresis in the MC2 suspension. If you are locking the IMC and noticed significant misalignment, the first thing to try is to touch MC2 alignment.

  16684   Sat Feb 26 23:48:14 2022 KojiUpdateSUSETMY SUS Electronics Replacement

[Ian, Koji] - Activity on 25th (Fri)

We continued working on the ETMY electronics replacement.

- The units were fixed on the rack along with the rack plan.

- Unnecessary Eurocard modules were removed from the crate.

- Unnecessary IDC cables and the sat amp were removed from the wiring chain. The side cross-connects became obsolete and they also were removed.

- A 18V DC power strip was attached to one of the side DIN rails.

Warning:

- Right now the ETMY suspension is free and not damped. We are relying on the EQ stops.

Next things to do:

- Layout the coil driving cables from the vacuum feedthru to the sat amp (2x D2100675-01 30ft ) [40m wiki]

- Layout DB cables between the units

- Layout the DC power cables from the power strip to the units

- Reassign ADC/DAC channels in the iscey model.

- Recover the optic damping

- Measure the change of the PD gains and the actuator gains.

Attachment 1: PXL_20220226_023111179_2.jpg
PXL_20220226_023111179_2.jpg
  16685   Sun Feb 27 00:37:00 2022 KojiUpdateGeneralIMC Locking Recovery

Summary:

- IMC was locked.
- Some alignment change in the output optics.
- The WFS servos working fine now.
- You need to follow the proper alignment procedure to recover the good alignment condition.

Locking:
- Basically followed the previous procedure 40m/16673.
- The autolocker was turned off. Used MC2 and MC3 for the alignment.
- Once I hit the low order modes, increased the IN1 gain to acquire the lock. This helped me to bring the alignment to TEM00
- Found the MC2 spot was way too off in pitch and yaw.
- Moved MC1/2/3 to bring the MC2 spot around the center of the mirror.
- Found a reasonably good visibility (<90%) at a MC2 spot. Decided this to be the reference (at least for now)

SP Table Alignment Work
- Went to the SP table and aligned the WFS1/2 spots.
- I saw no spot on the camera. Found that the beam for the camera was way too weak and a PO mirror was useless to bring the spot on the CCD.
- So, instead, I decided to catch an AR reflection of the 90% mirror. (See Attachment 1)
- This made the CCD vulnerable to the stronger incident beam to the IMC. Work on the CCD path before increasing the incident power.

MC2 end table alignment work
- I knew that the focusing lens there and the end QPD had inconsistent alignment.
- The true MC2 spot needs to be optimized with A2L (and noise analysis / transmitted beam power analysis / etc)
- So, just aligned the QPD spot using today's beam as the temporary target of the MC alignment. (See Attachment 2)

Resulting CCD image on the quad display (Attachment 3)

WFS Servo
- To activate the WFS with the low transmitted power, the trigger threshold was reduced from 5000 to 500. (See Attachment 4)
- WFS offset was reset with /opt/rtcds/caltech/c1/scripts/MC/WFS/WFS_RF_offsets
- Resulting working state looks like Attachment 5

Attachment 1: PXL_20220226_093809056.jpg
PXL_20220226_093809056.jpg
Attachment 2: PXL_20220226_093854857.jpg
PXL_20220226_093854857.jpg
Attachment 3: PXL_20220226_100859871.jpg
PXL_20220226_100859871.jpg
Attachment 4: Screenshot_2022-02-26_01-56-31.png
Screenshot_2022-02-26_01-56-31.png
Attachment 5: Screenshot_2022-02-26_01-56-47.png
Screenshot_2022-02-26_01-56-47.png
  16686   Sun Feb 27 01:12:46 2022 KojiUpdateGeneralIMC manual alignment procedure

We expect that the MC sus are susceptible to the temperature change and the alignment drifts away with time.

Here is the proper alignment procedure.

0) Assume there is no TEM00 flash or locking, but the IMC is still flashing with higher-order modes.

1) Use the CCD camera and WFS DC spots to bring the beam to the nominal position.

2) Use only MC2 and MC3 to align the cavity to have low-order modes (TEM00,01,02 etc)

3) You should be able to lock the cavity on one of these modes. Minimize the reflection (maximize the transmission) for that mode.

4) This should allow you to jump to a better lower-order mode. Continue alignment optimization only with MC2/3 until you get TEM00.

5) Optimize the TEM00 alignment only with MC2/3

6) Look at the MC end QPD. use one of the scripts in scripts/MC/moveMC2 . Note that the spot moves opposite to the name of the scripts. i.e. MC2_spot_down moves the spot up, MC2_spot_right moved the spot left, etc...
These scripts move MC1/2/3 and try to keep the good MC transmission.

7) moveMC2 scripts are not perfect. As you use them, it makes the MC alignment gradually degraded. Use MC2 and MC3 to recover good transmission.

8) If MC2 spot is satisfactory, you are done.

-------------

Step 6-8 can be done with the WFS on. This way, you can skip step 7 as the WFS servo takes care of it. But if the spot move is too fast, the servo can't keep up with the change. If so, you have to wait for the settling of the servo. Once the spot position is satisfactory, MC servo relief should be run so that the servo offset (in actuation) can be offloaded to the bias slider.

 

Attachment 1: PXL_20220226_100859871.jpg
PXL_20220226_100859871.jpg
  16690   Tue Mar 1 19:26:24 2022 KojiUpdateSUSETMY SUS Electronics Replacement

The replacement key switches and Ne Indicators came in. They were replaced and work fine now.

The power supply units were tested with the X end HeNe display. It turned out that one unit has the supply module for 1350V 4.9mA while the other two do 1700V 4.9mA.
In any case, these two ignited the HeNe Laser (1103P spec 1700V 4.9mA).

The 1350V one is left at the HeNe display and the others were stored in the cabinet together with spare key SWs and Ne lamps.

Attachment 1: PXL_20220302_025723651.jpg
PXL_20220302_025723651.jpg
Attachment 2: PXL_20220302_030102033.MP.jpg
PXL_20220302_030102033.MP.jpg
  16695   Thu Mar 3 04:11:36 2022 KojiUpdateSUSETMY 1Y4 Electronics Replacement

For the Y-end electronics replacement, we want to remove unused power supplies. In fact, we already removed the +/-5V supplies from the stack. I was checking what supply voltages are used by the Eurocard modules. I found that D990399 QPD whitening board had the possible use of +/-5V.

The 40m Y-end version can be found here D1400415. The +/-5V supply voltages are used at the input stage AD620 and the QPD bias voltage of -5V.

AD620 can work with +/-15V. Also the bias voltage can easily be -15V. So I decided to cut the connector legs and connected +5V line to +15V, and -5V line to -15V.

With this modification, I can say that the eurocards only use the +/-15V voltages and nothing else.

The updated schematics can be found as D1400415-v6

Attachment 1: PXL_20220303_082726693.jpg
PXL_20220303_082726693.jpg
Attachment 2: PXL_20220303_082752494.jpg
PXL_20220303_082752494.jpg
Attachment 3: PXL_20220303_082744464.jpg
PXL_20220303_082744464.jpg
  16696   Thu Mar 3 04:24:23 2022 KojiUpdateSUSETMY 1Y4 Electronics Replacement

The DC power strip at Y-end was connected to the bottom two Sorensen power supplies. They are configured to provide +/-18V.

 

Attachment 1: PXL_20220303_094421604.jpg
PXL_20220303_094421604.jpg
Attachment 2: PXL_20220303_094435127.jpg
PXL_20220303_094435127.jpg
  16701   Fri Mar 4 18:12:44 2022 KojiUpdateVACRGA pumping down

1. Jordan reported that the newly installed Pirani gauge for P2 shows 850Torr while PTP2 show 680 Torr. Because of this, the vacuum interlock fails when we try to open V4.

2. Went to c1vac. Copied the interlock setting file interlock_conditions.yaml to interlock_conditions_220304.yaml
3. Deleted diffpressure line and pump_underspeed line for V4
4. Restarted the interlock service

controls@c1vac:/opt/target/python/interlocks$ sudo systemctl status interlock.service  
controls@c1vac:/opt/target/python/interlocks$ sudo systemctl restart interlock.service
controls@c1vac:/opt/target/python/interlocks$ sudo systemctl status interlock.service

5. The above 2~4 was unnecessary. Start over.


Let RP1/3 pump down TP1 section through the pump spool. Then let TP2 pump down TP1 and RGA.

1. Open V7. This made P2 a bit lower (P2 is alive) and P3.
2. Connected the main RP tube to the RP port.
2. Started RP1/3. PRP quickly reaches 0.4Torr.
3. Opened V6 this made P3 and O2 below 1Torr.
4. Close V6. Shutdown RP1/3. Disconnect the RP tube.
5. Turn on auxRP at the wall powe
6. Turn on TP2. Wait for the starting up.
7. Open V4. Once the pressure is below Pirani range, open VM3.
8. Keep it running over the weekend.

9. Once TP2 reached the nominal speed, the "StandBy" button was clicked to lower the rotation speed (for longer life of TP2)

Attachment 1: Screenshot_2022-03-04_19-38-11.png
Screenshot_2022-03-04_19-38-11.png
  16702   Sat Mar 5 01:58:49 2022 KojiSummaryCDSpaola rescue

ETMY end ThinkPad "paola" could not reboot due to "Fan Error". It seems that it is the failure of the CPU fan. I really needed a functional laptop at the end for the electronics work, I decided to open the chassis. By removing the marked screws at the bottom lid, the keyboard was lifted. I found that the CPU fan was stuck because of accumulated dust. Once the fan was cleaned, the laptop starts up as before.

Attachment 1: PXL_20220305_035255834.jpg
PXL_20220305_035255834.jpg
Attachment 2: PXL_20220305_034649120.MP.jpg
PXL_20220305_034649120.MP.jpg
  16703   Sat Mar 5 02:03:46 2022 KojiUpdateSUSETMY 1Y4 Electronics Replacement

Oplev saga

Summary

- The new coil driver had not enough alignment range to bring the oplev beam back to the QPD center
- The coil driver output R was reduced from 1.2k to 1.2k//100 = 92.3 +/- 0.4 Ohm
- Now the oplev spot could be moved to the center of the QPD

- The damping gains (POS/PIT/YAW) and the oplev gains were reduced by a factor of 1/10.
- The damping and the oplev servos work now. Fine gain tuning is necessary.

To Do:
- DC value / TF measurements
- Adjust damping gains
- RFM issue
- Connection check
- Cable labeling


== Alignment Range ==

- Since c1auxey was removed, we no longer have C1:SUS-ETMY_PIT_COMM and C1:SUS-ETMY_YAW_COMM. At this moment, all the alignment is taken with the offset input from the fast real-time system via C1:SUS-ETMY_PIT_OFFSET and C1:SUS-ETMY_YAW_OFFSET.

- The oplev spot could not be moved on the center of the QPD without exceeding the DAC output range (~+ or -32000) for the coils. (Attachment 1)

- This is because the old system had a slow but large current range (Rout = 100) and a small current range for the fast control. Until we commission the new HV BIAS Driver, we have to deal with the large DC current with the HAM-A coil driver.

== Modification to the output resistances ==

The following units and the channels were modified. Each channel had a differential current driver and two output resistances of 1.2K. 100Ohm (OHMITE 43F100, 3W) wire wound resistors were added to them in parallel, making the resulting output R of ~92Ohm.

- ETMY HAM A Coil Driver 1: S2100622 (Attachments 2/3) CH1/2/3
- ETMY HAM A Coil Driver 2: S2100621 (Attachments 4/5) CH3

- This modification allowed me to align the oplev spot to the center of the QPD. C1:SUS-ETMY_PIT_OFFSET and C1:SUS-ETMY_YAW_OFFSE are +2725 (8%FS) and -2341 (7%FS), respectively.
- The previous alignment slider values were -0.9392 and 0.7615 (out of 10). These are the reasonable numbers, considering the change of the Rout from 100 to 92Ohm, and the sign flip.
(By the way, autoBurt files for c1auxex and c1auxey were not properly configured and the history of C1:SUS-ETM*_*_COMM was not recorded.)

== Damping Servos ==

- Now, the POS/PIT/YAW servos experience ~x10 gains. So temporarily these gains were reduced (POS 20->2, PIT 6->0.6, YAW 4->0.4) and the loops are stable when engaged.
- Also the gains of the OPLEV servos were reduced from -4.5 to -0.45. The loops are stable when engaged.

== Snapshot of the working condition ==

Attachent 6 shows the screenshot for the snapshot of the working condition.


To Do

- The damping servos were tested without proper PD whitening compensation.
  -> It turned out this is not necessary as our modified PD whitening has the pole and zero at the same freqs as before.

- Compare the DC values of the OSEM outputs and compensate for the gain increase by the "cts2um" filter.

- The end RTS suffers from the RFM issue. There is no data transmitted from the vertex to the end. I suspect we need to restart the c1rfm process. But this will likely suspend all the vertex real-time machines. Careful execution is necessary.

- c1iscey has all the necessary analog connections. But they are not tested. When we lock the green/IR cavity, we'll need them.

- The cable labeling is only half done.

Attachment 1: oplev_spot.jpg
oplev_spot.jpg
Attachment 2: PXL_20220305_090003837.jpg
PXL_20220305_090003837.jpg
Attachment 3: PXL_20220305_090023436.jpg
PXL_20220305_090023436.jpg
Attachment 4: PXL_20220305_091232290.jpg
PXL_20220305_091232290.jpg
Attachment 5: PXL_20220305_091306604.MP.jpg
PXL_20220305_091306604.MP.jpg
Attachment 6: Screenshot_2022-03-05_01-37-26.png
Screenshot_2022-03-05_01-37-26.png
  16707   Mon Mar 7 14:52:34 2022 KojiUpdateVACOngoing work to get the FRG gauges readouts to EPICs channels

Great trouble shoot!

> I guess, the only remaining issue now is the incorrect atmospheric pressure reading of 1000 Torrs. 

This is just a calibration issue. The controller should have the calibration function.
(The other Pirani showing 850Torr was also a calibration issue although I didn't bother to correct it. I think the pirani's typically has large distribution of the calibration values and requires individual calibration)

  16708   Mon Mar 7 14:55:33 2022 KojiUpdateIOOIMC unlocked again, completely misaligned

Hmm, the bias values were reset at 2022-03-03-20:01UTC which is 2022-03-03-12:01 PST with no apparent disruption of the data acquisition (= no resetting of the RTS). Not sure how this could happen.

 

  16711   Mon Mar 7 18:53:16 2022 KojiUpdateBHDRe-susspension of AS1

Not sure if that small difference can cause the alignment inability. Particularly, the removed metal was just below the wire. This means that there is no misalignment effect at the first order.

Here is my idea:
You may be able to assist the alignment by adding washers on one side of the four holes to this "H" shaped parts. The holes are away from the center line, adding some weight definitely do some misalignment.

 

  16719   Wed Mar 9 12:57:52 2022 KojiUpdateBHDSimplified BHD readout sketch on ITMY table

- BHD beams were already mixed in the chamber. So we don't need a BS on the table. (Probably there is no BS already)

- We don't need to split each BHD beam. One PD per BHD beam is OK for now.

- Check if the BHD paths have reasonable angles from the windows so that the beams do not hit the chamber wall.

- We need the POY path. POY indeed goes to the BS table

  16752   Fri Apr 1 17:02:02 2022 KojiUpdateOptical LeversSimplified sketch on MC table

We are supposed to have BS Oplev Beams. We don't like the shallow angle reflections (i.e. AOI>45deg).

The laser is too big but I suspect the other components are too small. So it'd be check the actual size of the components including the optical mounts that are missing on the figure so far.

  16753   Fri Apr 1 22:22:29 2022 KojiUpdateOptical LeversSimplified sketch on MC table

Possibility to swap BS and ITMX tables:
BS table, which Tega said MC table, is 2ft x 4ft. The ITMX table is 3ft x 5ft and only the central 2ft x 4ft area is used. The area around the BS table is the narrowest for the east arm. We need at least (2+delta) ft of the hallway width so that we can move the instrument. I'm not yet sure if the ITMX table can be placed there without precise investigation.
 

  16758   Wed Apr 6 01:20:48 2022 KojiSummaryBHDPart IIa of BHR upgrade - IR laser alignment on Xarm

PR2/PR3 Output R for fame OSEMS reduced from 1.2K to 1.2K//100Ohm

I put the R=100Ohm for PR3 with the functions of the units mistakenly swapped. This affects imbalanced actuation of PR3 right now as well as too strong SD

PR2 Coil Driver 1 (UL/LL/UR) / S2100616 / PCB S2100520 / R_OUT = (1.2K // 100) for CH1/2/3

PR2 Coil Driver 2 (LR/SD) / S2100617 / PCB S2100519 / R_OUT = (1.2K // 100) for CH3

PR3 Coil Driver 1 (UL/LL/UR) / S2100619 / PCB S2100516 / R_OUT = (1.2K // 100) for CH3 only

PR3 Coil Driver 2 (LR/SD) / S2100618 / PCB S2100518 / R_OUT = (1.2K // 100) for CH1/2/3

----

The output R was reduced from 1.2k to 1.2k//100 = 92 Ohm.

This means that the face coil gains were increased by a factor of 13.

The original gains for PR2 Pos/Pit/Yaw were {0.7, 0.3, 0.2}. To keep the same loop gain, the new gains were supposed to be {0.054, 0.023, 0.015}.
With the new gain, the oscillations were very slowly reduced. Therefore, I increased the gains to have the gain margin of 2. (i.e. increased the gains until I have the oscillation, and then made it half.)
The new values were {0.2, 0.1, 0.05}. The side gain was 20 and unchanged

For PR3 the same operation has been done.

The original gains for PR3 Pos/Pit/Yaw were {1, 0.52, 0.2}. They were supposed to be reduced to  {0.077, 0.04, 0.015}.
The gains were increased to {0.5, 0.1, 0.1}. The side gain was also increased from 1 to 5.

Attachment 1: PXL_20220406_060846581.jpg
PXL_20220406_060846581.jpg
Attachment 2: PXL_20220406_014205726.jpg
PXL_20220406_014205726.jpg
Attachment 3: PXL_20220406_060324812.jpg
PXL_20220406_060324812.jpg
Attachment 4: PXL_20220406_015259326.jpg
PXL_20220406_015259326.jpg
  16760   Wed Apr 6 22:51:47 2022 KojiSummaryBHDPart IIa of BHR upgrade - IR laser alignment on Xarm

[Yuta Koji]

We took out the two coil driver units for PR3 and the incorrect arrangement of the output Rs were corrected. The boxes were returned to the rack.

In order to recover the alignment of the PR3 mirror, C1:SUS_PR3_SUSPOS_INMON / C1:SUS_PR3_SUSPIT_INMON / C1:SUS_PR3_SUSYAW_INMON were monitored. The previous values for them were {31150 / -31000 / -12800}. By moving the alignment sliders, the PIT and YAW values were adjusted to be {-31100 / -12700}. while this change made the POS value to be 52340.

The original gains for PR3 Pos/Pit/Yaw were {1, 0.52, 0.2}. They were supposed to be reduced to  {0.077, 0.04, 0.015}.
I ended up having the gains to be {0.15, 0.1, 0.05}. The side gain was also increased to 50.

----

Overall, the output R configuration for PR2/PR3 are summarized as follows. I'll update the DCC.

PR2 Coil Driver 1 (UL/LL/UR) / S2100616 / PCB S2100520 / R_OUT = (1.2K // 100) for CH1/2/3

PR2 Coil Driver 2 (LR/SD) / S2100617 / PCB S2100519 / R_OUT = (1.2K // 100) for CH3

PR3 Coil Driver 1 (UL/LL/UR) / S2100619 / PCB S2100516 / R_OUT = (1.2K // 100) for CH1/2/3

PR3 Coil Driver 2 (LR/SD) / S2100618 / PCB S2100518 / R_OUT = (1.2K // 100) for CH3

Attachment 1: PXL_20220406_213356237.MP.jpg
PXL_20220406_213356237.MP.jpg
Attachment 2: PXL_20220406_212936502.MP.jpg
PXL_20220406_212936502.MP.jpg
  16771   Mon Apr 11 21:44:14 2022 KojiSummaryBHDPart IIa of BHR upgrade - POY11 debugging

We took out the right most PD interface board D990543 and removed the 74LS04 chips. In fact two out of 4 were already replaced with enabling wires, however it seemed that one of the remaining two got failed.
These remaining two chips were removed and the enabling signals were connected to VCC (+5V). This operation made the status LED light not functional. (We didn't bother to fixed them by connecting them to the GND)

The new schematic diagram was attached here, and the corresponding DCC entry (https://dcc.ligo.org/D1900318-v1) was modified. Attachments #2-3 show the circuit after the changes, and the front view respectively.

Attachment 1: D990543A1.PDF
D990543A1.PDF D990543A1.PDF D990543A1.PDF
Attachment 2: PXL_20220412_023941730.jpg
PXL_20220412_023941730.jpg
Attachment 3: PXL_20220412_023950567.jpg
PXL_20220412_023950567.jpg
  16776   Wed Apr 13 18:55:54 2022 KojiUpdateCamerasCamera Battery Test

I believe that the Nikon has an exposure problem and that's why we bought the Canon.

 

  16787   Mon Apr 18 23:22:39 2022 KojiUpdateGeneralTool box and Work Station Organization

Whoa! Thanks!

Attachment 1: PXL_20220419_062101907.jpg
PXL_20220419_062101907.jpg
  16791   Wed Apr 20 16:11:08 2022 KojiUpdateSUSOutpur resistors updated for LO1 coil drivers / for SR2, LO2, AS1, and AS4 in progress

[JC Koji]

To give more alignment ranges for the SUS alignment, we started updating the output resistors of the BHD SUS coil drivers.
As Paco has already started working on LO1 alignment, we urgently updated the output Rs for LO1 coil drivers.
LO1 Coil Driver 1 now has R=100 // 1.2k ~ 92Ohm for CH1/2/3, and LO1 Coil Driver 2 has the same mod only for CH3. JC has taken the photos and will upload/update an elog/DCC.

We are still working on the update for the SR2, LO2, AS1, and AS4 coil drivers. They are spread over the workbench right now. Please leave them as they're for a while.
JC is going to continue to work on them tomorrow, and then we'll bring them back to the rack.

Attachment 1: IMG_0527.jpeg
IMG_0527.jpeg
Attachment 2: IMG_0528.jpeg
IMG_0528.jpeg
Attachment 3: IMG_0529.jpeg
IMG_0529.jpeg
Attachment 4: IMG_0530.jpeg
IMG_0530.jpeg
Attachment 5: IMG_0531.jpeg
IMG_0531.jpeg
Attachment 6: IMG_0532.jpeg
IMG_0532.jpeg
  16792   Wed Apr 20 18:50:03 2022 KojiUpdateBHDPart V of BHR upgrade - PR2 weirdness

It seemed that it comes from the servo oscillation. This does not happen when the output limitters were set to be 100-ish. But even so the gains looked quite low.

I turned on the Cheby rool-offs for all the DOFs, and this allowed me to increase the damping gain A LOT.
The gains were 2~5 but now they are now 20-25 for the face OSEMs and 150 for SD.

The attached is the example of the damping when all the damping loops are on.

I think we need to tune the servo loops carefully for all the SUSs by actually looking at the openloop transfer functions rather than a personal feeling. => To Do

Attachment 1: Screenshot_2022-04-20_18-46-11.png
Screenshot_2022-04-20_18-46-11.png
  16793   Thu Apr 21 10:35:23 2022 KojiUpdateCDSDAQ seemed down

Yesterday, when I worked on the damping servo, I found that any of the daqvtools (ndscope, dtt, dataviewer,...) is not available.  We may need to restart the fb and rt machines.

  16795   Thu Apr 21 15:22:44 2022 KojiUpdateSUSOutpur resistors updates for SR2, LO2, AS1, and AS4 done

[JC Koji]

Quick report: JC has done all the mods for the coil driver circuit in the morning and we worked on the reinstallation of them in the afternoon.
I'll check the damping loops / sus servo settings. JC is going to make an ELOG entry and DCC updates for more precise record of the mods.

  16801   Thu Apr 21 20:33:31 2022 KojiUpdateBHDAS1 UR OSEM problem localized in the chamber

Tega and Paco reported that the UR OSEM of AS1 lost the response.

- I have checked the LED MON (left) of the satellite amp for AS1. CH1/2/3 had 5V -> This indicates that the OSEM LEDs are (most likely) functioning.
- Then I went to the ITMY flange and connected the OSEM emulator instead of the Dsub25 cable. The attachment shows that the UR OSEM LED/PD worked fine with the OSEM emulator. WIth the vacuum flange connected it lost the response.

This indicates that the AS1 UR OSEM problem is localized in the chamber. Please check if the DSUB pins are touching the table or something else.

 

Attachment 1: Screenshot_from_2022-04-21_20-13-13.png
Screenshot_from_2022-04-21_20-13-13.png
  16809   Mon Apr 25 14:49:02 2022 KojiUpdateGeneralNitrogen Tank

For your (and mine) info:

N2 pressure can be monitored on the 40m summary page: https://nodus.ligo.caltech.edu:30889/detcharsummary/day/20220425/vacuum/
(you need to hit "today" to go to the current status)

 

  16839   Mon May 9 22:19:06 2022 KojiUpdateBHDBHD Platform Progress status

[Don, Koji]

Don is working on finalizing the BHD Platform design. All the components on the BHD platform are almost populated and aligned.

Don is still working on the table legs so that we can detach the legs when we need to float the table in the future.
The BHD BS mount will have a third picomotor so that we can steer 3 dof with the mount while the remaining dof needs to be provided by the OMC.
The BHD BS position is going to be adjusted so that the incident and trans beams have sufficient clearance.
The OMC legs (kinematic mounts) need more work so that we can adjust their positions for initial setup while they can be the reference for the reproducible placement of the OMCs.
The OMCs are rigidly held with the legs. For the damping of the 1-kHz body bode, which has a relatively high Q, there will be a dissipative element touching the glass breadboard.

Attachment 1: Screenshot_2022-05-09_220220.png
Screenshot_2022-05-09_220220.png
Attachment 2: Screenshot_2022-05-09_222405.png
Screenshot_2022-05-09_222405.png
  16840   Mon May 9 23:18:44 2022 KojiUpdateBHDBHD Platform Progress status

I quickly ran the FEA model to check the resonant freqs of the BHD platform.
The boundary conditions were:

  • The platform was not loaded
  • FIxed constraints were given to the five legs

Don has optimized the cut-out size for the OMCs to increase the rigidity of the plate. Also, the ribbed grid is made at the bottom side.

The lowest mode is at 168Hz. Because there is no leg around, it seems reasonable to have this kind of mode as the fundamental mode.
The other mode lined up at 291Hz, 394Hz, 402Hz, ...
The mode freqs will be lower once the platform is loaded. But as the unloaded platform mode, these mode freqs sound pretty good numbers.

Attachment 1: Screenshot_2022-05-09_224145.png
Screenshot_2022-05-09_224145.png
Attachment 2: Screenshot_2022-05-09_224137.png
Screenshot_2022-05-09_224137.png
Attachment 3: bhd_platform_168.png
bhd_platform_168.png
Attachment 4: bhd_platform_291.png
bhd_platform_291.png
Attachment 5: bhd_platform_394.png
bhd_platform_394.png
Attachment 6: bhd_platform_402.png
bhd_platform_402.png
  16895   Mon Jun 6 22:08:55 2022 KojiUpdateIMCMC1 OSEM sensor sign flipped, MC1/2/3 free swinging overnight for inmat diagonalization

Note that MC1 has a new style sat amp because the old one collapsed. The sign flip might have been the result of the replacement

 

  16900   Wed Jun 8 16:03:12 2022 KojiUpdateVACPower Outage 220608: Vacuum Recovery

=== Observation ===

  • TP1/2/3 failed. AUX RP did not survive the power outage (of course)

=== Initial Recovery of TP2/TP3 ===

  • Stopped TP1/2/3
  • Restored RP AUX at the wall breaker module
  • Restarted TP2/3

=== Rough pumping down ===

  • Found RP1/RP3 left connected (unusual)
  • Turned on RP1 and RP3
  • Opened V6 and V7 for roughing of the pump spool. Now 0.3 torr for P2/P3/PA* (annuli) - ready to open V4 and V5 to backup TP1 by TP2 and TP3. Waiting for TP2/3 to be accelerated

=== Towards main volume pumping ===

  • TP2 accelerated and opened V4 (This caused V6 to close). RP1 and RP3 stopped.
  • TP3 hasitated to spin up. Stopped for now.
  • TP1 needed to be reset to clear the displacement errors. Once the errors were cleared, TP1 was restarted (acclerating).
  • TP1 at normal speed. Opened V1 (main volume pumping started)
     
  • Disconnected RP1/RP3 line.
     
  • TP3 was restarted and is now stable at 55k RPM (indicated as soft spin / low speed). Pushed "Stand By On" and "then "Stand By OFF". This makes the TP3 back to "NORMAL". Switched to "Stand By On".
  • TP2 was also switched to "Stand By On".

Now the main volume and the annuli are pumped down TP1 and TP2/TP3 with RP AUX backing.

The attachment is the pressure glitch for the main volume.

 

Attachment 1: Screenshot_2022-06-08_16-43-13.png
Screenshot_2022-06-08_16-43-13.png
  16901   Wed Jun 8 16:33:26 2022 KojiUpdateGeneralPower Outage 220608: HVAC restored

I found the HVACs for the ends were off. They were turned back on.

  16902   Wed Jun 8 16:57:46 2022 KojiUpdateIOOPower Outage 220608: IMC restored

The IMC alignment was restored and the IMC is nicely locking.

Once the vacuum level recovered P<1e-4 torr, the PSL shutter was able to be opened.

The IMC was still flashing, so the lock to TEM00 was possible.

Once it was locked, the MC2 alignment was tweaked and the autolocker and the WFS kicked in to help the locking/alignment.

The transmission is ~13k and seems reasonable considering the low PMC transmission of the PMC (0.672)

  16917   Wed Jun 15 15:03:41 2022 KojiUpdatePSLPMC input beam aligned

The commissioners complained about the PMC alignment. The PMC input beam was aligned. It made the transmission improved from 0.72 to 0.74.

FYI: Which steering mirrors do we use for the PMC beam alignment?
The mounts are indicated with the red arrows in Attachment 2.
You have to move these two in common and differential for each pitch and yaw.
The first steering (the right one in the picture) has the beam going through the immediate back of the mount.
So touching the yaw knob of this mount needs some care so that you don't block the PMC refl beam.

Attachment 1: Screenshot_2022-06-15_15-03-16.png
Screenshot_2022-06-15_15-03-16.png
Attachment 2: PXL_20220615_212304650.jpg
PXL_20220615_212304650.jpg
  16918   Wed Jun 15 15:07:07 2022 KojiUpdateSUSLO2 SUS stuck fixed

I checked the state of the LO2 suspension. I found that the coil driver Enable Mon was all red. Meaning, the actuation signals were not delivered to the coil driver. I wasn't sure if this was intentional or not.

Enabled the coils with "WD Master" Shutdown -> Normal.

Immediately I saw the OSEMS flipped the sign because there was an (non-intentional) alignment offset in pitch. I've adjusted the pitch offset so that all the OSEM PDs have the voltages 4~5V.

That's it.

Attachment 1: Screenshot_2022-06-15_15-00-22.png
Screenshot_2022-06-15_15-00-22.png
Attachment 2: Screenshot_2022-06-15_15-00-40.png
Screenshot_2022-06-15_15-00-40.png
  16928   Fri Jun 17 13:07:08 2022 KojiUpdateBHDComparison of MICH OLTF model with measurement

I'm curious why the actual OLTF included the 60Hz comb...? It is undesirable to have such structure in the OLTF around the UGF as it can cause servo instability.
And if you remove them, you don't need to model them :-)

ELOG V3.1.3-