Photodiode inventory: [OMC ELOG 615]

Dolphin Investigation

- I made a basic description on a wiki page: https://wiki-40m.ligo.caltech.edu/CDS/DolphinSwitch

- Investigation crashed c1lsc/c1sus/c1iscex/c1sus2. Well, it's time to test the dolphin fencing. It seemed successful.

- Rebooted the crashed machines. I accidentally rebooted nodus, but Apache and elog were restarted.

- Burtrestoring to 18:19 snapshots. I suffered from the zero alignment gain issue, but the two arms are aligned and locked.

During the crash, I tried to reboot c1sus2 while the others were running. I actually did not install the script. It seems that it has been there since 2022 Sep.
Here is the instruction:

• Suppose you have one (or multiple) machines are dead (freeze, dolphin glitch, DK, etc).
• From the following list, determine which host you want to restart:
  PORT HOST
1    c1sus
2    c1lsc
3    c1iscex
4    c1iscey
5    c1ioo
6    c1sus2
• ssh into fb1. At the login directory, run the following command with the above port name (replace the "#" with it). If you have multiple hosts, run the command one by one.
  ./dolphin_ix_port_control.sh --disable 192.168.113.40 #
   
• ssh into the problematic machine. Use the following command to reboot it. Now this does not crash other machines!
  sudo reboot
   
• Once the machine starts rebooting, run the dolphin enabling command on fb1.
  ./dolphin_ix_port_control.sh --enable 192.168.113.40 #
  This should be done before the IOP (c1x07 etc) comes up. Otherwise, that IOP fails. It's allright. If the IOP (and other processes fails), just stop them with
  rtcds stop --all
  and enable dolphin with the above command. And then run
  rtcds start --all
   
• Once everything is up, burtrestore appropriate snapshots.

We can improve the process and the location of the script, but this is a good progress I suppose.

OMC Interface Aligner - (It's upside down...)

BHD OFI arrangement

Paco took the data which means he already had the beat note.

There is some chance that the beat was recovered after some mode "jumps" but usually the temp gap for the same beat frequency is ~2degC.
So my speculation is that there is a big temp gradient in the crystal now and had to compensate it with the struggle of the crystal TEC.

For the past data see https://nodus.ligo.caltech.edu:8081/40m/3759 or http://nodus.ligo.caltech.edu:8080/40m/12078
http://nodus.ligo.caltech.edu:8080/40m/4439 and so on.

A bit improved the design of OMC Interface Aligner

The idea is...The OMC I/F aligner covers the OMC for aligning the kinematic mounts (3 pairs of a V-groove and a ball) on the OMC. This makes the kinematic mount of two OMCs identical.

However, the OMC kinematic mount can't be adjusted because all the fasteners of the kinematic mounts are hidden by their counterparts.
We can copy the alignment of the OMC to the aligner, but the opposite is not possible.

Took the backup (snapshot) of nodus' /home/export as of Aug 25, 2023

controls@nodus> cd /cvs/cds/caltech/nodus_backup
controls@nodus> rsync -ah --progress --delete /home/export ./export_230825 >rsync.log&

40m BHD OFI Inventory

• OFI HWP 1
  • Motorized Rotary Stage Thorlabs PDR1V Qty 1
  • 0.5inch HWP: QWPO-1064-05-2 IDEX Optical Tech aka CVI Qty 1
  • Stainless SM5 retainer ring POLARIS-SM05RR (Qty 1 + spare 1)
  • Thorlabs KIM001
  • Power Supply KPS201
  • Post D2300286 (86.69mm = 3.413"), Newport Type https://dcc.ligo.org/LIGO-D2300286
  • Fork, Newport Type
• OFI TFP 1/2
  • Thorlabs LMR1V Qty2
  • Post (84.455mm = 3.325), Newport Type Qty2
  • Fork, Newport Type Qty2
  • 1" TFP obtained from LHO
• OFI FR
  • Already in hand
• OFI HWP2
  • D1600166 mount already in hand
  • 1" HWP already in hand
  • Post D2300287 (78.105mm = 3.075"), Newport Type https://dcc.ligo.org/LIGO-D2300287
  • Fork, Newport Type

Ready / Soon Ready
- BHD OMC Cables ready
- OMC#1 / OMC#4 ready
- BHD Platform parts being cleaned
- Assembly area HEPA being built
=> We will be soon ready to assemble BHD Platform and test with the OMC

In progress
- OMC locking setup (Moku)
- Connectors being attached to the BHD Platform actuators (picos & rotation stage)
- BHD Platform OFI parts drawing/procurement

- 40m BHD Electronics (BHD Adapter / DCPD TIA / Actuator driver I/F)

Other vent items
- In-vac ribbon cable holder (JC)

- Connector holder

- Scattered light control

- Pre-vent work
    * ASS recovery / extension

    * ETMX tuning
    * Vertex Eletronics upgrade
    * Fix PZT amps / PZT
    * Acromag

- Vent work items
    * New PR2
    * Alignment
 

I went to Section Y7 to check the stock DB9-MF cables. The custom dsub cables are also there.

• DB9 1FT / QTY 26
• DB9 2.5FT / QTY 42
• DB9 5FT / QTY 49
• DB9 10FT / QTY 30
• DB9 15FT / QTY 9

Let's say this is (26, 42, 49, 30, 9)

My estimation was:
1Y0/1 (9, 26, 8, 10, 0)
1Y4 or 1X9 (0,8,2,0,0)
1X3/4/5 (11,30,10,12,0)
And the rest was for acromag.

This means that:
No matter how my estimation was wrong, we have prenty of cables for the 1X3/4/5 electronics swap.

The vertex electronics transition work will begin on Monday. We expect the ongoing ASS-X work to be completed by then. But if it needs more time, we must hear a shouting signal from the ASS team.
Is there any other preparation to be done this week to reasonably compensate for changes in gain and TF associated with the transition?

In preparation for the transition, we want to have long custom DSUB25 cables (D2100675) approximately laid out (I mean on the floor, etc) this week. JC takes care of this.

• The lengths of the cables can be found in the attached wiring diagram.
• Both ends of the cables need to be labeled.
• At which side do we want to absorb the slack?

Transition Plan

• Suspension damping and watchdogs are appropriately taken care of, although we soon stop/remove everything.
• We first remove any existing units not going to be used in the circuit (except around the Eurocard crate oplev interface P2 of the wiring diagram).
• The wirings at the side cross-connects are removed. This includes the removal of the thick cables on the cable racks. This would become a heavy work.
• The DC power strips are attached to the racks, and the DC power wiring should be done at this point. We check the DC supply voltages.
• Install the new units as per the above rack layout and proceed with the DSUB connections. We have sufficient number of DSUB cables (this ELOG).
• Turn the units on one by one to detect any unit failure, just in case. If they are all on, we start work on the CDS restoration work.

I came to the lab to see the recovery work from the power glitch this morning 8:15AM. All CDS seems up. The suspensions are somewhat aligned. Some of them were not damped. The oplevs were off. Radhika is working on the recovery of the FP arms.

I noticed that FSS Slow servo is not working. I always forget what is the right way to turn it on. Here is the summary:

How to turn on FSSSlow (2023 Sept version)

• Go to megatron
• sudo systemctl enable FSSSlow
• sudo systemctl start FSSSlow
• sudo systemctl status FSSSlow

  ● FSSSlow.service - Script to run the PID temperature control servo for the PSL
     Loaded: loaded (/opt/rtcds/caltech/c1/Git/40m/scripts/PSL/FSS/FSSSlow.service; enabled; vendor pre
     Active: active (running) since Fri 2023-09-08 17:10:52 PDT; 1s ago
   Main PID: 2088 (python3)
      Tasks: 6 (limit: 4674)
     CGroup: /system.slice/FSSSlow.service
             └─2088 /usr/bin/python3 /opt/rtcds/caltech/c1/Git/40m/scripts/PSL/FSS/PIDLocker.py PIDConf

  Sep 08 17:10:52 megatron systemd[1]: Started Script to run the PID temperature control servo for the

For the preparation of the electronics upgrade, three 18bit DAC AI chassis were transformed to 16bit version.

The power supply connections were touched, so the units were tested with +/-18V, and they work as expected.

We are ready to do the transition from Wed 1PM.

The items for the upgrade was collected around the vertex area (Attached photo).

- aLIGO-style DC power strip (+/-18V) x3
- DC power cables (orange +/-18V)
- Electronics units for the upgrade.
- DSUB (DB9) cables
- Custom DSUB15-DSUB25 cables
- Custom DSUB25 cables

Note on Sorensen:

- Eurocard crate requires +-15V. We can place two 15V Sorensens on 1X4 for Eurocard crate or just leave the current +/-15V supplies.

- The aLIGO units requires +/-18V. We can place two 15V Sorensens on 1X5 or just leave two of the current supplies and set them to +/-18V.

• [OK] Reflected the sorensen setup (minimal change from the conventional config. (See the attachment)

• Before destroying the current setup, bring the alignment biases for the vertex 8 sus to zero and record all the OSEM values.
  => Radhika did it (next ELOG)
  -> This will give us the ratio of the OSEM error signals to know the gain ratios between before and after. Also this will make it easier to bring the alignment back.

• [OK] I suppose the oplevs are still aligned. We don't need to be too nervous about the oplev spot too much.

• How to compensate the coil force cal? Do we know the ratio from the ETM coil driver swap? (What were the coil output Rs? What are they now? Are the ratios reasonable?)

  • Currently:
    - PRM/BS/ITMX/ITMY DAC output for face coils differential
    - SRM/MC2/MC1/MC3 DAC output for face coils single ended
    - All 8 SD coils single ended
    - Coil Output Rs
      According to D1700218
      PRM unknown to be checked
      BS 100Ohm
      ITMX 100Ohm
      ITMY 100Ohm
      SRM 100Ohm
      MC2 430? unknown to be checked
      MC1 430? unknown to be checked
      MC3 430? unknown to be checked

  • New setup
      DAC output differential
      AI has the gain of 1 / HAM coil driver has a gain of 1.2
      Coil output Rs:
      For all the face coils 1.2k // 100 ~ 92Ohm
      For all the side coils 1.2k

- 4x 1064nm NPROs are OFF. The lab hall is laser safe, although the oplev lasers are on.
  The Laser Warning Signs were turned off by the interlock switch at the PSL enclosure (control room side)

- Watch dogs were turned to "disabled"
- Halted c1sus using dolphin fencing. This worked very well.
The previous report of dolphin fencing not working was due to a typo in my instruction (wrong -disable -> correct --disable).

controls@fb1:~ 0$ ./dolphin_ix_port_control.sh --disable 192.168.113.40 1
--disable
Disabling switch_ip 192.168.113.40, port 1
Complete - csr write addr=0x0001C050, val=0x20820090 (with ret=0)

ssh c1sus

controls@c1sus:~$ rtcds stop --all

[Radhika, Paco, Murtaza, Koji]

- Removed all the units that will not be used in the new setup.
- Removed the sidepanel crossconnects
- Removed most of the sidepanel power lines except for the top eurocrate at the top of 1X4 (requires +/-15 pale orange and blue).
- Removed the acromag connections
- Removed the connectors of the long suspension cables

We'll resume the work at 10AM. (We'll have breaks for lunch and the seminar at 3PM)

=== Next steps ===

- Continue to remove the long suspension cables.
- Attach the DC power strip
- Continue to clean up the power lines on the rack side
- Prepare the power lines (Guralp requires +/-15V, the Eurocard crate +/-15V, new power strips (x3) +/-18V)
- Install the units on the racks.

[Radhika, Paco, Murtaza, Koji]

Morning work:

• Power Strip assembly
• Power Strip cable crimping
• c1lsc fiber routing (the PCIE fiber was in danger) / c1lsc machine was stopped after dolphin fencing
• We tried to reroute the c1lsc fiber above the racks such that it does not get pitched by the rack doors,
  but it seemed that the fiber was damaged (Attachment 1) and the c1lsc can't talk with the IO chassis anymore. We need to replace the fiber.
  It seems that P/N is PCIEO-4G3-100.0-11 (Samtec)

Afternoon work:

• Removed the sidepanels of 1X3 and 1X4 for easier work
• Removed the long DB25 cables from the old sat boxes.
• DC power strips are installed.
• Finished cleaning up the side cross-connects
• Checked the DC supply conditions.
  • +/-15V Eurocard crate + Guralp requires 0.5A / 0.6A
  • +/-18V appered on the DC power strips correctly
• Cleaned up the floor a bit
• Murtaza noticed that there was some strange intermittent noise around the 1Y0/1 racks.
  It looks like one of the fans for the c1ioo IO chassis are dying.

Evening work:

• Made coil driver short plugs (Attachment 2). This enables the coils while the coil modes are set to be Acquire mode.
• Rack nuts inserted

Tomorrow plan (10:30AM):

• c1sus IO chassis installation
• Unit installation
• Unit powering tests (before connecting them)
• Cabling between the units
• Long DB cable installation (sat box removal)
• c1lsc fiber replacement
• Some above cable removal (fiber, old custom DB25 for MC1, etc)

[Radhika, Paco, Murtaza, Koji]

We made great progress today. It's going well so far.

Morning work (10:30AM~):

• Installed c1sus IO chassis
• Installed all units
• Removed long (previous) custom DB25 cables for MC1

Afternoon work

• Connected all the units to the DC power strip
• Unit powering test was done before the inter-unit DSUBs were connected.
  • We found one AA chassis don't turn on even though internal +/-15V seems supplied. We pulled the unit out.
  • All the other units were fine.
  • Typical current draw of the units: (unit name, positive supply current, negative supply current)
    • AA 0.5A 0.5A (18W)
    • AI 0.3A 0.3A  (11W)
    • BIO 0A 0A (0W)
    • Trillum I/F 0.1A 0.1A (0.4W)
    • Sat amp 0.3A 0.3A (11W)
    • Coil Driver 0.3A 0.2A (9W)
• Cabling between the units
  • Done except for the extracted AA unit and for the BIO units (not needed until we have Acromag).
• Long DB cable installation (sat box removal)
  • Halfway
• c1lsc fiber tracking/replacement (not yet done)
  • We found the spare box with 3 more cables behind the X-arm tube. The replacement has not been done yet.

Evening work

• c1sus powering up and CDS check
  • After the people had left, I tried to start up c1sus. I had used dolphin fencing, but it worked like a charm!
• ADC1 AA chassis repair

To Do on Mon (10AM~)

• Long DB cable installation (contd)
• DSUB cable labeling
• Sus control system recovery
• c1lsc fiber reinstallation and system recovery
• Tool / Debris cleaning

Eventual needs:

• We need good crimping tools.
• Supply shortage of the crimping connectors.
• Fibers should not be routed together with electronics cables. Fibers should be distributed through tubes hanging on the cable racks
• Cable strain relief
• Move the noisy CDS and DC power supplies to the drill press room.

Rack nut policy

Out rack nut / screws are so much contaminated.
It's a mixture of #10-32 (standard) / M5 (wrong) / M6 (wrong).
Even the labeled bottles are contaminated.
Don't believe the installed rack nuts/screws, even if they seemed to work fine. They may be a metric pair.

Golden standard (Attached photo)

• rack nuts marked 1032
• small washers
• 10-32 tapered-round head screws

If you find other hardware, don't mix them with any stocks. Give them to Rack Nut Police (=Koji). He will hide them to some where secret.

Dolphin Fencing technique

I believe that the dolphin emits some glitches to the other hosts during the host machine shutting down and starting up.
However, if the dolphin is disabled, that FE process will not run.

Therefore, we need some technique:

• When you have a real-time host to be restarted, we can disable the dolphin of that machine.
  e.g. If c1lsc has a problem, run the following command on fb1.
  ./dolphin_ix_port_control.sh --disable 192.168.113.40 2
  This allows us to restart the c1lsc in a safe way.
   
• Restart c1lsc in the above example. Go to c1lsc and run
  sudo reboot; exit
   
• This above brings you back to the previous host you were (suppose it is fb1). Run ping on that restarting machine.
  ping c1lsc
   
• While the c1lsc is shutting down, ping still has the response. Once the restart starts, it makes no response. Then, you can enable dolphin.
  ./dolphin_ix_port_control.sh --enable 192.168.113.40 2
   
• The process comes back automatically. You'll see DK status during the restart. I should disappear once all the models are up.

c1sus ADC1 AA chassis fix.

• Brought the chassis on the workbench.
• Opened the chassis and "alas!" The internal power cables were not connected to the PCBs. This makes sense why there was no current draw at all.
• The cables were connected.
• The unit was tested with +/-18V power. At least, the diff outputs of the AA boards were all 0V.
• A missing connector screw for the external power connector was fixed

The unit was installed on the 1X5 rack, and the DB9 cables were connected.
Repair mission completed.

I've turned on the Eurocard crate (+/-15V) and the AA units (+/-18V) to check if the oplev channels are working. (It seems to be running well)
Also, the AA units are not too hot so far.

[JC, Paco, Radhika, Koji]

Morning/Afternoon work:

• Long DB cable installation (contd)
  • 14 DB25 cables went through the cable-rack bridge above the ITMX chamber. It's twice the previous # of cables.
  • These cables were connected to the chamber flanges. ITMY Flange1 had been having the 2nd connector malfunction. So the cables were connected to the connector 1 and 3 (as before).
• DSUB cable labeling
  • All the (long) cables connected to the units were labeled appropriately.
• c1lsc fiber reinstallation
  • We found one fiber cable (AlpenIO Inc PCIe 4x10G 100m AIO-PCIe4X-100 (2010)) already routed from 1Y4 to the PSL rack. This is the spare JC told us. We routed the host end to 1Y7.
  • c1lsc is up and running as before. All the models are up an running (burtrestore still needed).
• c1sus channel assignment
  • We have the swap of ADC0/1/2 so that the oplev ADC will have ADC0 (duo tone at CH31)
  • The channel assignments were modified:
    • SUS numbering "n": (0-PRM / 1-BS / 2-ITMX / 3-ITMY / 4-SRM / 5-MC2 / 6-MC1 / 7-MC3)
    • Face OSEMs ADC1 CH (n x 4 + 0~3, UL/LL/UR/LR)
    • Side OSEM ADC2 CH n
    • Oplev Ch ADC0 CH (n x 4 + 0~3)
    • Face OSEMs DAC0 CH (n x 4 + 0~3)
    • Side OSEM DAC1 CH n

• Sus damping control recovery
  • We need to lookin to MC3 UL/UR, PRM SD, SRM UL/LL/UR, ITMX all, ITMY face. See next post.
• Tool / Debris cleaning

- Here is the thought how does the factor of 9 come from:

• We are driving OSEM LEDs at 35mA rather than at 25mA. (Honeywell LED SME2470 has quite a linear response for  Irradiance vs. Forward Current.)
• The TIA of the OSEM PD is now 121K instead of the previous 39.2K
• The OSEM output is received by differential AA.

--> Naive estimation is (35/25) x (121k/39.2k) x 2 = 8.64.

- MC1 sat amp has already been replaced with the aLIGO version by Gautam. I wonder where this factor of 2.15 came from (not 2...?).

- Coil driver response:

Previous setup

• BS/PRM/ITMX/ITMY had the differential receiver (cf https://nodus.ligo.caltech.edu:8081/40m/17840)
• All the removed coil drivers have output Rs of 100 Ohm
• The coil resistance is 16~20Ohm.

--> BS/PRM/ITMX/ITMY 2 V_DAC/118 Ohm = 1.7e-2 A/V x V_DAC

--> MC1/MC2/MC3/SRM V_DAC/118 Ohm = 8.5e-3 A/V x V_DAC

--> All the side coils V_DAC/118 Ohm = 8.5e-3 A/V x V_DAC

New setup

• The AIs have the gain of 1.
• The coil driver has a gain of 1.2.
• The output Rs for the face coils are 1.2k//100Ohm = 92Ohm

--> 2 V_DAC * 1.2 / (92+18) Ohm = 2.2e-2 A/V x V_DAC

BS/PRM/ITMX/ITMY face coils will have x1.3 more actuation.

MC1/MC2/MC3/SRM face coils will have x2.6 more actuation.

• The output Rs for the side coils are 1.2k

--> 2 V_DAC * 1.2 / 1200 Ohm = 2.0e-3 A/V x V_DAC

MC1/MC2/MC3/SRM will have less actuation by a factor of 1/4.25.

ITMX 400Ohm
ITMY 400Ohm
BS 100
PRM 100
SRM 100
MC2 427.5/410/411/409.4/410
MC1 434.5/428.4/430.6/432.5/434.0
MC3 432.2/409.4/409.3/410.6/413.8
 

MC1 is ready for the damping test

Trouble shooting plan

• Is the LED on? => Check all the LED mon outputs of the sat amp. It should show 5V if the output current is 35mA. If the constant current loop is open (eg no LED / connection failure etc), it rails at the supply voltage.
• Also the CCD videos should show the status of the LEDs (at least for the TMs and the MC mirrors)
• Then is the PD out responding? => Check all the PD mons.
• If the PD mons are normal, but there is no signal it can be the AA problem. Inject test signal to that channel on the AA and see if we can see some number on the CDS.
   
• Is the coil current flowing? => Check if the coil drv mons are responding.
• If not, check if the AI output has the DAC output in that channel.
• If the DAC signal is there, but no current it can be the driver issue, or the coil/cable/flange connection issue.

[Paco, Murtaza, JC, Koji, Radhika]

MC1/MC2 was working fine.

At this point MC3, SRM, and ITMY are also working fine.

The custom DB25 cables between the sat amps and the flanges are difficult to mate.

• The finger tight was not enough to make all the contacts. Fastening the screws with a screwdriver made MC3 start working fine on CDS.
• The custom cable fastening screw on PRM(1st) was stripped at the flange side. It needs a thread dyeing. The SRM(2st) has the hex nut broken on the sat amp. Need to be fixed.

We checked if the LED mon shows the correct values. When it is connected it shows 5V. If the LED is not connected it shows 0.8V. It goes 0.08V in an unknown state.

• SRM1 all channels were 0V. It turned out that the connection inside the vacuum chamber seemed mirrored. Right now we have temporary mirror ribbon cables to fix this issue. We need two shielded mirror cables for SRM1 and SRM2.
• SRM2 was random (5V, 0.8V, and 0V)
• BS1/2/ITMY1 looked fine.
• ITMY2 was strange.
• ITMX1/2 were completely silent.

We suspected the cable pinouts/ cable mating issue etc, but it turned out that the SRM2 cable was mislabeled and the ITMY2 cable was connected to the SRM sat amp. That's why it was so random. We corrected the connection and SRM and ITMY started working fine on CDS.

We used the OSEM simulation box to test the sat amps. That suggests that the BS/PRM/ITMX problem may be coming from the SAT amps. We need to look into the sat amps.

[Murtaza, Koji]

ITMX / BS / PRM sat amps were removed from the rack and checked on the workbench. They all work fine with the OSEM simulation box.

With the correct circuit, the LED mon should be 5V, and the PD readout should be 2.6~3.0V (i.e. the differential output has twice the voltage difference of this number).

ITMX Sat Amp fixed:

• The internal wiring for CH1-4 was not connected (or disconnected by mechanical impact) (Attachment 1)
• PD1 channel for CH1 had a metal debris on the transimpedance opamp (Attachment 2)
• The internal board for CH5-6 was connected in the opposite direction. This was because both connectors on the board had the wrong genders.
  This was replaced with a spare. (There was two spares and I consumed one now) (Attachment 3)
   
• Put a ventilated lid instead of the solid lid.

BS Sat Amp:

• All the CHs just worked fine.

PRM Sat Amp:

• Found the bias selector jumpers had not been installed. Fixed. (Attachment 4/5)

Plan for Sept 20, 2023

For morning people:

• We don't need to replace the long cables. They seem all fine.
• Close the lid of the repaired sat amps. Use a lid with ventilation slits (there is an extra with the empty unit on the same desk).
• Install the sat amps back to the racks. Connect all the cables us. Check if this makes the OSEM values to positive 10~20k counts.
  • If not, check LED mons and PD mons. If they are OK the sat amp is working fine. Track the signal down to the AI chassis to see if the units after the sat amp are working well.
• The SRM2 and ITM2 cables ( connected to the sat amps at the back of the units) cross (i.e. have twisted) at the rack. Please reroute and nicely coil them up.
• Fix the custom cable issues: "The custom cable fastening screw on PRM(1st) was stripped at the flange side. It needs a thread dyeing. The SRM(2st) has the hex nut broken on the sat amp. Need to be fixed."
• Put the proper labels on the long cables at the flanges and the sat amps. The labels should indicate where the connectors are supposed to be connected.
• Clean up the mess and the tools from the lab.

The

After the weekly meeting, we'll continue to work on the suspension control. The lab will be turned to be LASER HAZARD in the afternoon.

I asked CDS mattermost for help. Chris (Wipf) checked it and reported it is working fine as usual (without fixing anything).

I've reverted the copied C1MCS.txt back in the chans dir (/opt/rtcds/caltech/c1/chans). The filter coefficients were loaded from the GDS screen. The filters were properly updated.

Here is the info from Chris:

Some transient NFS problem, maybe?

One possible clue is that the filter file that the FE actually loads is not chans/<model>.txt,
but chans/tmp/<model>.txt. Before loading, the filter file is copied into the tmp directory,
and a diff of the two files is written to chans/tmp/<model>.diff.
This diff file should normally be an empty file, indicating that the two files match.
But it was not empty at first, so there must have been some issue with the previous load.
After I reloaded, the diff then became empty.

I found the actuator calibration is more complicated. The numbers I reported in the previous elog was not correct.

Here I summarize the numbers of the voltage-to-current conversion.

=== Previous===

=== New ===

e.g. ITMX face coil electronics are x4.6 stronger than the previous coil electronics.

[Koji, Murtaza, JC]

Regarding PRM/BS:

• PRM2 cable and BS2 cable were wrongly connected. This was corrected.
   
• This makes the BS face OSEM values reasonable.
  However, the side signal is still close to zero. We confirmed that the sat amp outputs (LED mon/PD mon/PD diff out) looked reasonable for all five BS OSEMs.
  The side signal issue stays downstream of the Vertex ADC adapter.
   
• PRM2 has no issue with the sat amp.
• PRM1: We found that all the LED mon goes down to 0.17V when the vacuum flange is connected.
  It was found that the reference voltage for the LED (TP11 of D080276) went down to low number (like 0.15V) when the in-vac OSEMs were connected.
  I found that this output was not stable. So, I replaced the U4 chip (AD8672), but this didn't help the voltage sagging issue.
• Murtaza and I started checking the short circuits on the flange. We found that Pin 5 (OSEM PD1 Kathode) and Pin 1 (invac cable shield?) only have 5.1 Ohm. Pin 1 is connected to the vacuum chamber.
  • What does it mean? The PD has the reverse bias voltage of 10V applied on the PD Kathode. This bias voltage is shorted to ground via 5 Ohm. To keep the bias line at 10V, we need 2 A.
  • We don't have many options:
    - We can disconnect the internal wire for pin5 from the cable. (Prepare a ribbon cable). This should make the other OSEM PDs properly biased.
    We may be able to use an independent power supply to provide some amount of reverse bias (10V 2A is too much. Probably 1V 0.2A or 2.5V 0.5A?) so that the UL PD somewhat work.

[JC, Paco, Koji]

We had a power outage on Sept 21, 2023 at ~9AM. This is the third power outage this month as far as I remember.

- JC reported the outage was ~2sec. Some UPS supported machines were affected, while some unsupported machines also survived the incident cf c1psl (what!?)

- Some machines were rebooted by itself (cf the RTS hosts).

- megatron and optimus were powered up. Autolockers (optimus) and FSSSlow (megatron) were restored.

c1vac was still on, but the machine didn't come back online.

- The network adapter was reactivated by running the following commands

> cd /sbin
> sudo ifdown eth0
> sudo ifup eth0

However, the acromag seemed freezer, so c1vac was shutdown, the acromag chassis was power cycled, and the c1vac was rebooted. This brought c1vac fully functional again.
Rebooting made TP1 stop (gracefully)

The vacuum pressure of the main volume was high.

- We found that the vacuum pressure was up to 1e-2 torr in the afternoon when we started the recovery. In fact, the main gate valve was close at the power outage last week. See attachments.

- We made sure the valves were properly open/closed and started TP1 again. Once TP1 reached 33.6K RPM we opened the main volume to recover the vacuum pressure.

- The vacuum pressure came back to <1e5 Torr.

Pumping configuration changed. Now TP2 is backing TP1 and TP3 is pumping annuli.

Fixed IMC/IFO alignment screens

That's not what I'm asking.

Also additional cables are left connected to the signal path. I removed it.

e-log was repeatedly hanging and several attempts to start the daemon failed.

problem was solved after clearing the (firefox) browser cache, cookie, everything!!

The power supply of a lock-in amp, which is on the Y-arm side of PSL clean room, was pulled out by my mistake.

Then I reconnected it, but I don't know whether it is re-adjusted properly.

I'm sorry about this. If you are using that amp, it should be checked.

I checked a broken QPD, which was placed for PSL angle monitor, and finally I cocluded one segment of the quadrant diode was broken.

The broken segment has a offset voltage of -0.7V after 1st I-V amplifier. It means the diode segment has a current offset without any injection of light.

Tomorrow I will check a new QPD for replacement.

Kiwamu IZUMI

Goal is to build a temperature sensor accurate to 1 mK. Schematic is shown below. This does not take into account the DC gain that occurs.

Parts that would be used for this: LM317 regulator, AD592 temperature transducer, OP amp (low input noise and high impedance), 100K (or maybe 10k) resistor. This is what is currently proposed, but the exact parts we use could be changed to better fit the sensor. The resistor and the OP amp will be decided depending on the output of the AD592.

Once this is built, I would like to create a few copies of it and put them into an insulated container and measure the output from each one. This would allow us to calculate the temperature noise of the circuit, as we can take out the variations due to temperature changes inside the container by comparing the outputs.

I can also model the noise in the circuit to see how much noise there is before building it. There are three terms to the noise that we have, and we need to decide which one dominates at low frequencies.

Our final goal is to create an additional circuit that could cancel out the DC gain. I have attached an additional schematic proposed by Rana that would help with this issue. I will leave this second half for when the first part works.

I decided to see what was inside the sensor that had been previously made. According to elog 1102, the temperature sensor is LM34, the specs of which can be found here:

http://www.ti.com/lit/ds/symlink/lm34.pdf

The wiring of this sensor confused me, as it appears that the +Vs end (white) connects to the input, but both the ground (left) and the Vout (middle) pins are connected to the box itself. I don't see how the signal can be read.

Since there seems to be little difference between AD590 and AD592, I guess we could just go with the AD590. The temperature noise spectrum in the first graph are for the AD590, so if we want to reproduce those results, we should use AD590.

For the AD581/AD587, we could go with a few varieties that have the least output voltage drift, although I am not sure what precision we will need. So maybe we could try AD587U and AD581L. We could also try AD587K and AD581K and see if those work as well.

We will also need to calibrate the sensor, as it takes an input of 5V, but the AD581/AD587 provides 10V, which will give about a 1 degree error according to the datasheet. It does state that this is only a calibration error, so it shouldn't be too much of an issue.

I will figure out the packaging once I construct the sensor and verify that it works. Maybe we could use a box similar to the existing sensor, but it depends on the size of the finished circuit.

Quick update: we actually have AD587KRZ and AD592, so we could start by using that and seeing how it works.

Used AD592CNZ and AD586 (5V output) to create a circuit that works and is responsive to temperature changes. At room temp, using ~1K resistor, it showed ~0.3V across it, as expected. The voltage went up when we heated it with a heating gun. Next step will be to add in an OP amp and design some experiments to check to see how accurate it is. Thanks to Gautam for helping me with it!

I have attached the working circuit and a close up of the connections.

Decided to try adding in an OP amp just to see if it would work. Added LT1012 and a 100k resistor to the circuit (I originally wanted to do AD743 as it seems to be the best choice according to Zach's elog here, but it said that they are very precious so I went with LT1012 for testing purposes). When heating it with a heating gun, the output voltage went down by a few 0.01V. The maximum voltage was 0.686V. Similar thing happened when I switched to a 10k resistor, where the maximum was 0.705V and it also went down by a few 0.01V upon heating.

I've attached a few pictures showing the circuit.

I didn't realize that the LT1012 needed an additional input to function. I added in +15V and -15V to pins 7 and 4, respectively and placed a 10k resistor and the numbers make more sense now. The voltage showed a negative value, but it became more negative as I heated it up (it's negative due to how a transimpedance amplifier works).

I have attached the new setup and the value it shows (~-3V). It became more negative by about 0.4V, which translates to about a 40K increase in temperature, which makes sense.

In addition, I have attached an updated sketch of the circuit. I will need to do more testing to determine how accurate this is. The next step would be to calculate how much noise there is currently and figure out how to remove this circuit from the breadboard and use a PCB or something like that for final testing in an insulated container.

The reason I chose AD743 initially for the OP amp is because at low frequencies (which is what we are working with), a FET amp such as AD743 will have a low current noise at high impedance, which is what we have in this case. While a FET amp has high voltage noise compared to other OP amps, the current noise becomes more important at high impedance, so it will work better. According to Zach's graphs, the AD743 is best at high impedances, followed by LT1012.

For the final packaging/mounting of the sensor to the seismometer, I have thought of two options.

1. Attach circuit to a PCB board and place it inside the can, while leaving the AD590 open to the air inside the can.

• This makes sure that the sensor gets a direct measurement of the temperature of the air in the can, as it is exposed to the air. 
• But, it takes a limited area of measurement, so it could be the case that the area we place it in happens to be a hot or cold pocket, and the measurement would be inaccurate.
• This can be solved by placing multiple copies of the circuit in various places of the can and averaging the values.

2. Attach the AD590 to a copper plate with thermal paste and put it into a pomona box.

• This solves the problem of having a limited sample area the first option had, as the copper plate should have a uniform temp distribution, thus we are sampling the temp of that whole area.
• Need to make sure that the response time to the temperature variations of copper is less than the frequency that we are measuring.
• This can be calculated using equations for heat transfer (listed below).

If anyone has input on which method is preferred or any additional options that we may have, I would appreciate it.

Heat transfer:

q = k A dT / s

• k = thermal conductivity
• A = area
• dT = temperature gradient
• s = thickness

For copper, k = 401 W/mK, x = 1.27 mm, A = 2.66x10^-3 m^2 (for the particular copper plate I measured), dT = 1K (assume). Thus the heat transfer will be 839 J/s.

I'm not completely sure what to do with this yet, but it could help us decide whether the copper plate option will be useful for us.

Tested to make sure that even when only the AD586 was heated that there was no change in the reading. I did so by placing the AD586 away from the rest of the circuit and blowing hot air only on it. There was, in fact, no change. 

Tried taking the circuit from the breadboard to the PCB. I attached all the components to adapters that would allow them to be connected to the PCB. From the first picture, the first component is AD586, the second is AD590, and the third is LT1012, along with a resistor across it. I then soldered the connections between the components, as can be seen in the second picture. When I tested out this version of the circuit by hooking it up to the DC source, I got a reading of ~-15V. I will have to check all the connections to make sure there is contact where there should be one, and no contact where there shouldn't be. I had issues attaching the tiny AD590 and LT1012 to its adaptor, so the issue may lie there as well. I'll also check that each component is in working order as well.

Once I figure out where my error is, my plan is to build two more of these and place a metal object such that it contacts only the surface of the AD590s. This would allow me to compare the three values to the actual temperature of the metal, which would then tell me how accurate this setup is.

Note on the resistor: I measured all the resistors and chose three that had exactly 10.00k Ohm. The voltage detected is dependent on the resistor, so if we are to take three identical copies, I ensured that there would be no error due to the resistors being a little different.

Got it to work. One of the connections was faulty. I decided to check the temperature measured against a thermometer. The sensor showed 26.1 C, but the thermometer showed 25.8 C after I let them both cool down after heating them up. The temperature of the thermometer was dropping at the time of measurement, but the temperature of the sensor was not. This is still a rough version of the final sensor, so I'm not sure what exactly causes this discrepancy.

On Friday, I cleaned up the circuit so that there are only three connections needed (+15V, -15V, GND) and a BNC connector for reading the output. Today, I added in bypass capacitors. The small yellow ones are 0.1 microF ceramic, and the large ones are 100 microF electrolytic. They are used to stabilize the +15V and -15V inputs to the OP amp and minimize fluctuations, since it doesn't have a regulator for stability. I have also attached the circuit diagram for the OP amp only, where 1 are the electrolytic and 2 are the ceramic. The temperature is still about 2 degrees off, but if that difference is constant for all temperatures in our range we can just calibrate it later.

Here is a helpful link on bypass capacitors (thanks to Kevin for sending it to me).

As a note, the electrolytic capacitors do have a polarity, so it is important to place them correctly (the negative side is towards the lower voltage potential, and not always towards ground).

I worked with Kevin and Gautam to create a heater circuit. The first attachment is Kevin's schematic of the circuit. The OP amp connects to the gate of the power MOSFET, and the power supply connects to the drain, while the source goes into the heater. We set the power supply voltage to 22V and varied the voltage of the input to the OP amp. At 6V to the OP amp, we got a current of 0.35A flowing through the heater and resistor. This was the peak current we got due to the OP amp being saturated (an increase in either of the power supplies did not change the current), but when we increased the voltage of the supply rails of the OP amp from 15V to 20V, we got a current of 0.5A. We would want a higher current than this, so we will need to get a different OP amp with a higher max voltage rating, and a resistor that can take more power than this one (it currently takes 5W of power, and is the best one we could find).

Kevin and I created a simulation of this circuit using CircuitLab to understand why the current was so low (second attachment). The horizontal axis is the voltage we supply to the OP amp. The blue line shows the voltage at the point between the output of the OP amp and the gate of the MOSFET. The orange line is the voltage at the point between the source of the MOSFET and the heater. The brown line is the voltage at the point between the heater and resistor. Thus, we can see that saturation occurs at about 2.1V. At that point, the gate-source voltage is the difference between the blue curve and the orange curve, which is about 4V, which is what we measured. Likewise, the voltage across the heater is the difference between the orange curve and the brown curve, which comes out to around 8V, which is also what we measured. Lastly, the voltage across the resistor is the brown curve, which is about 2V, which matches our observations. The circuit works as it should, but saturates too soon to get a high enough current out of it.

Gautam noted that it is important to measure the current correctly. We can't just use an ammeter and place it across the resistor or heater, because the internal resistance of the ammeter (~0.5 ohm) is comparable to the resistance we want to measure, so the current gets split between the circuit and the ammeter and we get an equivalent resistance of 1/R = 1/R0 + 1/Ra, where R0 is the resistance of the part we want to measure the current across, and Ra is the ammeter resistance. Thus, the new resistance will be lower and the ammeter will show a higher current value than what is actually there. So to accurately measure the current, we must place the ammeter in series with the part we want to measure. We initially got a 1A reading on the heater, which was not correct, and our setup did not heat up at all basically. When we placed the ammeter in series with the heater, we got only 0.35A.

The last two images are the setup for testing of the heater. We wrapped it around an aluminum piece and covered it with a few layers of insulating material. We can stick a thermometer in between the insulation and heater to see the temperature change. In later tests, we may insulate the whole piece so that less heat gets dissipated. In addition, we used a heat sink and thermal paste to secure the MOSFET to it, as it got very hot.

Our next steps will be to get a resistor and an OP amp that are better suited for our purposes. We will also run simulations with components that we choose to make sure that it can provide the desired current of 1A (the maximum output of the power supply is 24V, and the heater is 24 ohm, so max current is 1A). Kevin is working on that now.