40m QIL Cryo_Lab CTN SUS_Lab TCS_Lab OMC_Lab CRIME_Lab FEA ENG_Labs OptContFac Mariner WBEEShop
  40m Log, Page 247 of 339  Not logged in ELOG logo
ID Date Author Type Category Subjectup
  3343   Sat Jul 31 22:35:01 2010 KojiUpdateVACVac-P1 still 1.2mtorr

I resumed the pumping from 19:00.

Now the valve RV1 is full open. But the pumping is really slow as we are using only one RP.

After 3hrs of pumping, P1 reached 1.2mmtorr but still we need 2hrs of pumping...

I stopped pumping at 22:30.

  14452   Thu Feb 14 15:37:35 2019 gautamUpdateVACVacromag failure

[chub, gautam]

Sumary:

One of the XT1111 units (XT1111a) in the new vacuum system has malfunctioned. So all valves are closed, PSL shutter is also closed, until this is resolved.

Details:

  1. Chub alerted me he had changed the main N2 line pressure, but this did not show up in the trend data. In fact, the trend data suggested that all 3 N2 gauges had stopped logging data (they just held the previous value) since sometime on Monday, see Attachment #1.
  2. We verified that the gauges were being powered, and that the analog voltage output of the gauges made sense in the drill press room ---> So this suggested something was wrong at the Vacuum rack electronics rack.
  3. Went to the vacuum rack, saw no obvious indicator lights signalling a fault.
  4. So I restarted the modbus process on c1vac using sudo systemctl restart modbusIOC.service. The way Jon has this setup, this service controls all the sub-processes talking to gauges and TPs, so resatrting this master process should have brought everything back.
  5. This tripped the interlock, and all valves got closed.
  6. Once the modbus service restarted, most things came back normally. However, V1, V3, V4 and V5 readbacks were listed as "UNDEF".
  7. The way the interlock code works, it checks a valve state change request against the monitor channel, so all these valves could not be opened.
  8. We confirmed that the valves themselves were operational, by bypassing the itnerlock logic and directly actuating on the valve - but this is not a safe way of running overnight so we decided to shut everything down.
  9. We also confirmed that the problem is with one particular Acromag unit - switching the readback Dsub connector to another channel (e.g. V1 --> VM2) showed the expected readback.
  10. As a further check - I connected a windows laptop with the Acromag software installed, to the suspected XT1111 - it reported an error message saying "USB device may be damaged". Plugging into another XT111 in the crate, I was able to access the unit in the normal way.
  11. The phoenix connector architecture of the Acromags makes it possible to replace this single unit (we have spare XT1111 units) without disturbing the whole system - so barring objections, we plan to do this at 9am tomorrow. The replacement plan is summarized in Attachment #2.

Pressure of the main volume seems to have stabilized - see Attachment #3, so it should be fine to leave the IFO in this state overnight.

Questions:

  1. What caused the original failure of the writing to the ADC channels hooked up to the N2 gauges? There isn't any logging setup from the modbus processes afaik.
  2. What caused the failure of the XT1111? What is the failure mode even? Because some other channels on the same XT1111 are working...
  3. Was it user error? The only operation carried out by me was restarting the modbus services - how did this damage the readback channels for just four valves? I think Chub also re-arranged some wires at the end, but unplugging/re-connecting some cables shouldn't produce this kind of response...

The whole point of the upgrade was to move to a more reliable system - but seems quite flaky already.

Attachment 1: Screenshot_from_2019-02-14_15-40-36.png
Screenshot_from_2019-02-14_15-40-36.png
Attachment 2: IMG_7320.JPG
IMG_7320.JPG
Attachment 3: Screenshot_from_2019-02-14_20-43-15.png
Screenshot_from_2019-02-14_20-43-15.png
  14453   Thu Feb 14 18:16:24 2019 JonUpdateVACVacromag failure

I sent Gautam instructions to first try stopping the modbus service, power cycling the Acromag chassis, then restarting the service. I've seen the Acromags go into an unresponsive state after a strong electrical transient or shorted signal wires, and the unit has to be power cycled to be reset.

If this doesn't resolve it, I'll come in tomorrow to help with the Acromag replacement. We have plenty of spares.

Quote:

[chub, gautam]

Sumary:

One of the XT1111 units (XT1111a) in the new vacuum system has malfunctioned. So all valves are closed, PSL shutter is also closed, until this is resolved.

 

  14309   Mon Nov 19 23:38:41 2018 JonOmnistructure Vacuum Acromag Channel Assignments

I've completed bench testing of all seven vacuum Acromags installed in a custom rackmount chassis. The system contains five XT1111 modules (sinking digital I/O) used for readbacks of the state of the valves, TP1, CP1, and the RPs. It also contains two XT1121 modules (sourcing digital I/O) used to pass 24V DC control signals to the AC relays actuating the valves and RPs. The list of Acromag channel assignments is attached.

I tested each input channel using a manual flip-switch wired between signal pin and return, verifying the EPICS channel readout to change appropriately when the switch is flipped open vs. closed. I tested each output channel using a voltmeter placed between signal pin and return, toggling the EPICS channel on/off state and verifying the output voltage to change appropriately. These tests confirm the Acromag units all work, and that all the EPICS channels are correctly addressed.

Attachment 1: Binary_IO_Channel_Assignments.pdf
Binary_IO_Channel_Assignments.pdf
  14296   Wed Nov 14 21:34:44 2018 JonOmnistructure Vacuum Acromags installed and tested

All 7 Acromag units are now installed in the vacuum chassis. They are connected to 24V DC power and Ethernet.

I have merged and migrated the two EPICS databases from c1vac1 and c1vac2 onto the new machine, with appropriate modifications to address the Acromags rather than VME crate.

I have tested all the digital output channels with a voltmeter, and some of the inputs. Still more channels to be tested.

I’ll follow up with a wiring diagram for channel assignments.

Attachment 1: IMG_3003.jpg
IMG_3003.jpg
  14375   Thu Dec 20 21:29:41 2018 JonOmnistructureUpgradeVacuum Controls Switchover Completed

[Jon, Chub, Koji, Gautam]

Summary

Today we carried out the first pumpdown with the new vacuum controls system in place. It performed well. The only problem encountered was with software interlocks spuriously closing valves as the Pirani gauges crossed 1E-4 torr. At that point their readback changes from a number to "L OE-04, " which the system interpreted as a gauge failure instead of "<1E-4." This posed no danger and was fixed on the spot. The main volume was pumped to ~10 torr using roughing pumps 1 and 3. We were limited only by time, as we didn't get started pumping the main volume until after 1pm. The three turbo pumps were also run and tested in parallel, but were isolated to the pumpspool volume. At the end of the day, we closed every pneumatic valve and shut down all five pumps. The main volume is sealed off at ~10 torr, and the pumpspool volume is at ~1e-6 torr. We are leaving the system parked in this state for the holidays. 

Main Volume Pumpdown Procedure

In pumping down the main volume, we carried out the following procedure.

  1. Initially: All valves closed (including manual valves RV1 and VV1); all pumps OFF.
  2. Manually connected roughing pump line to pumpspool via KF joint.
  3. Turned ON RP1 and RP2.
  4. Waited until roughing pump line pressure (PRP) < 0.5 torr.
  5. Opened V3.
  6. Waited until roughing pump line pressure (PRP) < 0.5 torr.
  7. Manually opened RV1 throttling valve to main volume until pumpdown rate reached ~3 torr/min (~3 hours on roughing pumps).
  8. Waited until main volume pressure (P1a/P1b) < 0.5 torr.

We didn't quite reach the end of step 8 by the time we had to stop. The next step would be to valve out the roughing pumps and to valve in the turbo pumps.

Hardware & Channel Assignments

All of the new hardware is now permanently installed in the vacuum rack. This includes the SuperMicro rack server (c1vac), the IOLAN serial device server, a vacuum subnet switch, and the Acromag chassis. Every valve/pump signal cable that formerly connected to the VME bus through terminal blocks has been refitted with a D-sub connector and screwed directly onto feedthroughs on the Acromag chassis.

The attached pdf contains the master list of assigned Acromag channels and their wiring.

Attachment 1: 40m_vacuum_acromag_channels.pdf
40m_vacuum_acromag_channels.pdf 40m_vacuum_acromag_channels.pdf 40m_vacuum_acromag_channels.pdf
  14493   Thu Mar 21 18:36:59 2019 JonOmnistructureUpgradeVacuum Controls Switchover Completed

Updated vac channel list is attached. There are several new ADC channels.

Quote:

Hardware & Channel Assignments

All of the new hardware is now permanently installed in the vacuum rack. This includes the SuperMicro rack server (c1vac), the IOLAN serial device server, a vacuum subnet switch, and the Acromag chassis. Every valve/pump signal cable that formerly connected to the VME bus through terminal blocks has been refitted with a D-sub connector and screwed directly onto feedthroughs on the Acromag chassis.

The attached pdf contains the master list of assigned Acromag channels and their wiring.

Attachment 1: 40m_Vacuum_Acromag_Channels_20190321.pdf
40m_Vacuum_Acromag_Channels_20190321.pdf 40m_Vacuum_Acromag_Channels_20190321.pdf 40m_Vacuum_Acromag_Channels_20190321.pdf
  14315   Sun Nov 25 17:41:43 2018 JonOmnistructure Vacuum Controls Upgrade - Status and Plans

New hardware has been installed in the vacuum controls rack. It is shown in the below post-install photo.

  • Supermicro server (c1vac) which will be replacing c1vac1 and c1vac2.
  • 16-port Ethernet switch providing a closed local network for all vacuum devices.
  • 16-port IOLAN terminal server for multiplexing/Ethernetizing all RS-232 serial devices.

Below is a high-level summary of where things stand, and what remains to be done.

Completed:

 Set up of replacement controls server (c1vac).

  • Supermicro 1U rackmount server, running Debian 8.5.
  • Hosting an EPICS modbus IOC, scripted to start/restart automatically as a system service.
  • First Ethernet interface put on the martian network at 192.168.113.72.
  • Second Ethernet interface configured to host a LAN at 192.168.114.xxx for communications with all vacuum electronics. It connects to a 16-port Ethernet switch installed in the vacuum electronics rack.
  • Server installed in vacuum electronics rack (see photo).

 Set up of Acromag terminals.

  • 6U rackmount chassis frame assembled; 15V DC, 24V DC, and Ethernet wired.
  • Acromags installed in chassis and configured for the LAN (5 XT1111 units, 2 XT1121 units).

 EPICS database migration.

  • All vacuum channels moved to the modbus IOC, with the database updated to address the new Acromags. [The new channels are running concurrently at "C1:Vac2-...." to avoid conflict with the existing system.]
  • Each hard channel was individually tested on the electronics bench to confirm correct addressing and Acromag operation.

 Set up of 16-port IOLAN terminal server (for multiplexing/Ethernetizing the serial devices).

  • Configured for operation on the LAN. Each serial device port is assigned a unique IP address, making the terminal server transparent to client TCP applications.
  • Most of the pressure gauges are now communicating with the controls server via TCP.

Ongoing this week:

  • [Jon] Continue migrating serial devices to ports on the terminal server. Still left are the turbo pumps, N2 gauge, and RGA.
  • [Jon] Continue developing Python code for communicating with gauges and pumps via TCP sockets. A beta version of gauge readout code is running now.
  • [Chub] Install feedthrough panels on the Acromag chassis. Connect the wiring from feedthrough panels to the assigned Acromag slots.
  • [Chub/Jon] Test all the hard EPICS channels on the electronics bench, prior to installing the crate in the vacuum rack.
  • [Chub/Jon] Install the crate in the vacuum rack; connect valve/pump readbacks and actuators; test each hard EPICS channel in situ.
  • [Jon] Once all the signal connections have been made, in situ testing of the Python interlock code can begin.
Attachment 1: rack_photo.jpg
rack_photo.jpg
  13179   Wed Aug 9 16:34:46 2017 ranaUpdateVACVacuum Document recovered

Steve and I found the previous draft of the 40m Vacuum Document. Someone in 2015 had browsed into the Docs history and then saved the old 2013 version as the current one.

We restored the version from 2014 which has all of Steve's edits. I have put that version (which is now the working copy) into the DCC:  https://dcc.ligo.org/E1500239.

The latest version is in our Google Docs place as usual. Steve is going to have a draft ready for us to ready be Tuesday, so please take a look then and we can discuss what needs doing at next Wednesday's 40m meeting.

  16508   Wed Dec 15 15:06:08 2021 JordanUpdateVACVacuum Feedthru Install

Jordan, Chub

We installed the 4x DB25 feedthru flange on the North-West port of ITMX chamber this afternoon. It is ready to go.

  9017   Fri Aug 16 09:35:18 2013 SteveUpdateVACVacuum Normal state recognition is back

Quote:

Quote:

Quote:

Quote:

Apparently all of the ION pump valves (VIPEE, VIPEV, VIPSV, VIPSE) opened, which vented the main volume up to 62 mTorr.  All of the annulus valves (VAVSE, VAVSV, VAVBS, VAVEV, VAVEE) also appeared to be open.  One of the roughing pumps was also turned on.  Other stuff we didn't notice?  Bad. 

 Several of the suspensions were kicked pretty hard (600+ mV on some sensors) as a result of this quick vent wind.  All of the suspensions are damped now, so it doesn't look like we suffered any damage to suspensions.

CLOSE CALL on the vacuum system:

Jamie and I disabled V1, VM2 and VM3 gate valves by disconnecting their 120V solenoid actuator before the swap of the VME crate.

The vacuum controller unexpectedly lost control over the swap as Jamie described it. We were lucky not to do any damage! The ion pumps were cold and clean. We have not used them for years so their outgassing possibly  accumulated to reach ~10-50 Torr

I disconnected_ immobilized and labelled the following 6 valves:  the 4 large ion pump gate valves and VC1,  VC2  of the cryo pump. Note: the valves on the cryo pump stayed closed. It is crucial that a warm cry pump is kept closed!

This will not allow the same thing to happen again and protect the IFO from warm cryo contamination.

The down side of this that the computer can not identify vacuum states any longer.

This vacuum system badly needs an upgrade. I will make a list.

 While I was doing the oil change of the roughing pumps I accidentally touched the 24 V adjustment knob on the power supply.

All valve closed to default condition. I realized that the current indicator was red at 0.2A  and the voltage fluctuated from 3-13V

Increased current limiter to 0.4A and set voltage to 24V     I think this was the reason for the caos of valve switching during the VME swap.

 

 Based on the facts above I reconnected VC1 and VC2 valves.  State recognition is working.  Ion pumps are turned off and their gate valves are disabled. 

We learned that even with closed off gate valves while at atmosphere  ion pumps outgass hydrocarbons at 1e-6 Torr level.  We have not used them for this reason in the passed 9 rears.

 

I need help with implementing V1 interlock triggered by Maglev failure signal  and-or P2 pressure.

MEDM screen agrees with vacuum rack signs.

Attachment 1: VacuumNormal.png
VacuumNormal.png
Attachment 2: vacValvesDisabled.jpg
vacValvesDisabled.jpg
  16980   Fri Jul 8 14:03:33 2022 JCHowToVACVacuum Preparation for Power Shutdown

[Koji, JC]

Koji and I have prepared the vacuum system for the power outage on Saturday.

  1. Closed V1 to isolate the main volume.
  2. Closed of VASE, VASV, VABSSCI,VABS, VABSSCO, VAEV, and VAEE.
  3. Closed V6, then close VM3 to isolate RGA
  4. Turn off TP1 (You must check the RPMs on the TP1 Turbo Controller Module)
  5. Close V5
  6. Turn off TP3 (There is no way to check the RPMs, so be patient)
  7. Close V4 (System State changes to 'All pneumatic valves are closed)
  8. Turn off TP2 (There is no way to check the RPMs, so be patient)
  9. Close Vacuum Valves (on TP2 and TP3) which connect to the AUX Pump.
  10. Turn of AUX Pump with the breaker switch wall plug.

From here, we shutdown electronics.

  1. Run /sbin/shutdown -h now on c1vac to shut the host down.
  2. Manually turn off power to electronic modules on the rack.
    • GP316a
    • GP316b
    • Vacuum Acromags
    • PTP3
    • PTP2
    • TP1
    • TP2 (Unplugged)
    • TP3 (Unplugged)

 

Attachment 1: Screen_Shot_2022-07-12_at_7.02.14_AM.png
Screen_Shot_2022-07-12_at_7.02.14_AM.png
  14308   Mon Nov 19 22:45:23 2018 JonOmnistructure Vacuum System Subnetwork

I've set up a closed subnetwork for interfacing the vacuum hardware (Acromags and serial devices) with the new controls machine (c1vac; 192.168.113.72). The controls machine has two Ethernet interfaces, one which faces outward into the martian network and another which faces the internal subnetwork, 192.168.114.xxx. The second network interface was configured via the following procedure.

1. Add the following lines to /etc/network/interfaces:

allow-hotplug eth1
iface eth1 inet static
address 192.168.114.9
netmask 255.255.255.0

2. Restart the networking services:

$sudo /etc/init.d/networking restart

3. Enable DNS lookup on the martian network by adding the following lines to /etc/resolv.conf:

search martian
nameserver 192.168.113.104

4. Enable IP forwarding from eth1 to eth0:

$sudo echo 1 > /proc/sys/net/ipv4/ip_forward

5. Configure IP tables to allow outgoing connections, while keeping the LAN invisible from outside the gateway (c1vac):

$sudo iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
$sudo iptables -A FORWARD -i eth0 -o eth1 -m state --state RELATED,ESTABLISHED -j ACCEPT
$sudo iptables -A FORWARD -i eth1 -o eth0 -j ACCEPT

6. Finally, because the EPICS 3.14 server binds to all network interfaces, client applications running on c1vac now see two instances of the EPICS server---one at the outward-facing address and one at the LAN address. To resolve this ambiguity, two additional enviroment variables must be set that specify to local clients which server address to use. Add the following lines to /home/controls/.bashrc:

EPICS_CA_AUTO_ADDR_LIST=NO
EPICS_CA_ADDR_LIST=192.168.113.72

A list of IP addresses so far assigned on the subnetwork follows.

Device IP Address
Acromag XT1111a 192.168.114.1
Acromag XT1111b 192.168.114.2
Acromag XT1111c 192.168.114.3
Acromag XT1111d 192.168.114.4
Acromag XT1111e 192.168.114.5
Acromag XT1121a 192.168.114.6
Acromag XT1121b 192.168.114.7
Perle IOLAN SDS16 192.168.114.8
c1vac 192.168.114.9
  5180   Wed Aug 10 22:47:22 2011 ranaSummaryVACVacuum Workstation (linux3) re-activated

For some reason the workstation at the vac rack was off and unplugged. Nicole and I plugged its power back in to the EX rack.

I turned it on and it booted up fine; its not dead. To get it on to the network I just made the conversion from 131.215 to 192.168 that Joe had done on all the other computers several months ago.

Now it is showing the Vacuum overview screen correctly again and so Steve no longer has to monopolize one of the Martian laptops over there.

  11352   Wed Jun 10 15:54:14 2015 SteveUpdateVACVacuum comp. rebooted

Koji and Steve succeded rebooting C1vac1, C1vac2 and pressure reading is working now

More tomorrow .........

 

Attachment 1: afterReb061015.png
afterReb061015.png
  11353   Thu Jun 11 19:40:59 2015 KojiUpdateVACVacuum comp. rebooted

The serial connections to the vacuum gauges were recovered by rebooting c1vac1 and c1vac2.

Steve claimed that the vacuum screen had showed "NO COMM" at the vacuum pressure values.
The epics connection to c1vac was fine. We could logged in to c1vac1 with telnet too although c1vac2 had no response.

After some inspection, we decided to reboot the slow machines. Steve manually XXXed YYY valves (to be described)
to prepare for any possible unwanted switching. Initially Koji thought only c1vac2 can be rebooted. But it was wrong.
If the reset button is pushed, all of the modules on the same crate is reset. So everything was reset. After ~3min we still
don't have the connection to c1vac1 restored. We decided to another reboot. This time I pushed c1vac1 reset button.
After waiting about two minutes, the ADCs started to show green lights and the switch box started scanning.
We recovered the telnet connection to c1vac1 and epics functions. c1vac2 is still note responding to telnet, and
the values associated with c1vac2 are still blank.

Steve restored the valves and everything was back to normal.

  11354   Fri Jun 12 08:40:17 2015 SteveUpdateVACVacuum comp. rebooted

Koji and Steve,

One computer expert and one vacuum expert required.

Quote:

The serial connections to the vacuum gauges were recovered by rebooting c1vac1 and c1vac2.

Steve claimed that the vacuum screen had showed "NO COMM" at the vacuum pressure values.
The epics connection to c1vac was fine. We could logged in to c1vac1 with telnet too although c1vac2 had no response.

After some inspection, we decided to reboot the slow machines. Steve manually XXXed YYY valves (to be described)
to prepare for any possible unwanted switching. Initially Koji thought only c1vac2 can be rebooted. But it was wrong.
If the reset button is pushed, all of the modules on the same crate is reset. So everything was reset. After ~3min we still
don't have the connection to c1vac1 restored. We decided to another reboot. This time I pushed c1vac1 reset button.
After waiting about two minutes, the ADCs started to show green lights and the switch box started scanning.
We recovered the telnet connection to c1vac1 and epics functions. c1vac2 is still note responding to telnet, and
the values associated with c1vac2 are still blank.

Steve restored the valves and everything was back to normal.

Atm 1,  problem condition: gauges are not reading for a week, error message "NO COMM" and all computer LEDs are green

Atm 2, prepare to safe reboot:

            a, close V1, disconnect it's power cable and turn off Maglev, wait till rotation stops

            b, close PSL shutter ( take adrenaline if needed )

            c, close V4, V5, VA6 valves and disconnect their cables. "Moving" error message indicating this condition.

               V1 is not showing "Moving" because its power cable disconnected only! It will show it if its position indicator cable is disconnected too. There is no need for that.

               These valves closed and disabled will not allow accidental venting of main volume.

            d, push reset, reseting c1vac2 will reset c1vac1 also, wait ~ 6 minutes

"Vacuum Normal" valve configuration was restored after succesful reboot as follows:

             a, reconnect cable and open V4 and V5 at P2 & P3 <1e-1 Torr

             b, observe that P2  < 1e-3 Torr and retsart Maglev

             c, wait till Maglev reaches full speed of 560 Hz and reconnect-open V1

             d, reconnect-open VA6 at P3 <1e-3 Torr

NOTE: VM1 valve was locked in open position and it was not responding before and after reboot

          Error message on Atm2 is indicating this locked condition: "opening VM1 will vent IFO"

          This is a fauls message. The valve is frozen in open position. We need a softwear expert help.

 

 

Attachment 1: vacMonNoGauges.png
vacMonNoGauges.png
Attachment 2: prepReboot.png
prepReboot.png
  1673   Mon Jun 15 15:17:33 2009 josephb, SteveConfigurationVACVacuum control and monitor screens

We updated the vacuum control and monitor screens  (C0VAC_MONITOR.adl and C0VAC_CONTROL.adl).  We also updated the /cvs/cds/caltech/target/c1vac1/Vac.db file.

1) We changed the C1:Vac-TP1_lev channel to C1:Vac-TP1_ala channel, since it now is an alarm readback on the new turbo pump rather than an indication of levitation.  The logic on printing the "X" was changed from X is printed on a 1 = ok status) to X is printed on a 0 = problem status.  All references within the Vac.db file to C1:Vac-TP1_lev were changed.  The medm screens also now are labeled Alarm, instead of Levitating.

2) We changed the text displayed by the CP1 channel (C1:Vac-CP1_mon in Vac.db) from "On" and "Off" to "Cold - On" and "Warm - OFF".

3) We restarted the c1vac1 front end as well as the framebuilder after these changes.

  15499   Thu Jul 23 15:58:24 2020 JonSummaryVACVacuum controls refurbishment plan

This year we've struggled with vacuum controls unreliability (e.g., spurious interlock triggers) caused by decaying hardware. Here are details of the vacuum refurbishment plan I described on the 40m call this week.

 Refurbish TP2 and TP3 dry pumps. Completed [ELOG 15417].

 Automated notifications of interlock-trigger events. Email to 40m list and a new interlock flag channel. Completed [ELOG 15424].

Replace failing UPS.

  • Two new Tripp Lite units on order, 110V and 230V [ELOG 15465].
  • Jordan will install them in the vacuum rack once received.
  • Once installed, Jon will come test the new units, set up communications, and integrate them into the interlock system following this plan [ELOG 15446].
  • Jon will move the pumps and other equipment to the new UPS units only after completing the above step.

Remove interlock dependencies on TP2/TP3 serial readbacks. Due to persistent glitching [ELOG 15140, ELOG 15392].

Unlike TP2 and TP3, the TP1 readbacks are real analog signals routed to Acromags. As these have caused us no issues at all, the plan is to eliminate dependence on the TP2/3 digital readbacks in favor of the analog controller outputs. All the digital readback channels will continue to exist, but the interlock system will no longer depend on them. This will require adding 2 new sinking BI channels each for TP2 and TP3 (for a total of 4 new channels). We have 8 open Acromag XT1111 channels in the c1vac system [ELOG 14493], so the new channels can be accommodated. The below table summarizes the proposed changes.

Channel Type Status Description Interlock
C1:Vac-TP1_current AI exists Current draw (A) keep
C1:Vac-TP1_fail BI exists Critical fault has occurred keep
C1:Vac-TP1_norm BI exists Rotation speed is within +/-10% of set point new
C1:Vac-TP2_rot soft exists Rotation speed (krpm) remove
C1:Vac-TP2_temp soft exists Temperature (C) remove
C1:Vac-TP2_current soft exists Current draw (A) remove
C1:Vac-TP2_fail BI new Critical fault has occurred new
C1:Vac-TP2_norm BI new Rotation speed is >80% of set point new
C1:Vac-TP3_rot soft exists Rotation speed (krpm) remove
C1:Vac-TP3_temp soft exists Temperature (C) remove
C1:Vac-TP3_current soft exists Current draw (A) remove
C1:Vac-TP3_fail BI new Critical fault has occurred new
C1:Vac-TP3_norm BI new Rotation speed is >80% of set point new
  14419   Fri Jan 25 16:14:51 2019 gautamUpdateVACVacuum interlock code, N2 warning

I reset the remote of this git repo to the 40m version instead of Jon's personal one, to ensure consistency between what's on the vacuum machine and in the git repo. There is now a N2 checker python mailer that will email the 40m list if all the tank pressures are below 600 PSI (>12 hours left for someone to react before the main N2 line pressure drops and the interlocks kick in). For now, the script just runs as a cron job every 3 hours, but perhaps we should integrate it with the interlock process?

Quote:

All the python code running on c1vac is archived to the git repo: 

https://git.ligo.org/40m/vacpython

  15283   Wed Mar 25 15:15:55 2020 gautamUpdateVACVacuum interlock code, N2 warning update

The email address in the N2 checking script wasn't right - I now updated it to email the 40m list if the sum of reserve tank pressures fall below 800 PSI. The checker itself is only run every 3 hours (via cron on c1vac).

Quote:

I reset the remote of this git repo to the 40m version instead of Jon's personal one, to ensure consistency between what's on the vacuum machine and in the git repo. There is now a N2 checker python mailer that will email the 40m list if all the tank pressures are below 600 PSI (>12 hours left for someone to react before the main N2 line pressure drops and the interlocks kick in). For now, the script just runs as a cron job every 3 hours, but perhaps we should integrate it with the interlock process

  15501   Mon Jul 27 15:48:36 2020 JonSummaryVACVacuum parts ordered

To carry out the next steps of the vac refurbishment plan [ELOG 15499], I've ordered parts necessary for interfacing the UPS units and the analog TP2/3 controller outputs with c1vac. The purchase list is appended to the main BHD list and is located here. Some parts we already had in the boxes of Acromag materials. Jordan is gathering what we do already have and staging it on the vacuum controls console table - please don't move them or put them away.

Quote:

Replace failing UPS.

Remove interlock dependencies on TP2/TP3 serial readbacks. Due to persistent glitching [ELOG 15140, ELOG 15392].

  16312   Thu Sep 2 21:21:14 2021 KojiSummaryComputersVacuum recovery 2

Attachment 1:
We are pumping the main volume with TP2. Once P1a reached the pressure ~2.2mtorr, we could open the PSL shutter. The TP2 voltage went up once but came down to ~20V. It's close to nominal now.
We wondered if we should use TP3 or not. I checked the vacuum pressure trends and found that the annulus pressures were going up. So we decided to open the annulus valves.

Attachment 2:
The current vacuum status is as shown in the MEDM screenshot.

There is no trend data of the valve status (sad)

Attachment 1: Screenshot_2021-09-02_21-20-24.png
Screenshot_2021-09-02_21-20-24.png
Attachment 2: Screenshot_2021-09-02_21-20-48.png
Screenshot_2021-09-02_21-20-48.png
  7272   Fri Aug 24 16:03:39 2012 SteveUpdateVACVacuum related work at atm

Vacuum related work at atmosphere:

Atm1,  Check all chamber dog clamps tightness with torque wrench,

Atm2,  Replace old, black molibdenum disulfite bolts -nut with new silicon bronze nuts and clean SS bolts.

Atm3,  Replace CC1 cold cathode gauges: horizontal and vertical.

Attachment 1: IMG_1563.JPG
IMG_1563.JPG
Attachment 2: IMG_1565.JPG
IMG_1565.JPG
Attachment 3: IMG_1566.JPG
IMG_1566.JPG
  15526   Fri Aug 14 10:10:56 2020 JonConfigurationVACVacuum repairs today

The vac system is going down now for planned repairs [ELOG 15499]. It will likely take most of the day. Will advise when it's back up.

  15527   Sat Aug 15 02:02:13 2020 JonConfigurationVACVacuum repairs today

Vacuum work is completed. The TP2 and TP3 interlocks have been overhauled as proposed in ELOG 15499 and seem to be performing reliably. We're now back in the nominal system state, with TP2 again backing for TP1 and TP3 pumping the annuli. I'll post the full implementation details in the morning.

I did not get to setting up the new UPS units. That will have to be scheduled for another day.

Quote:

The vac system is going down now for planned repairs [ELOG 15499]. It will likely take most of the day. Will advise when it's back up.

  8752   Wed Jun 26 01:30:31 2013 ranaSummaryPEMVariation in 10-30 Hz seismic RMS

For quite a while (no one knows how long), we've seen fluctuations in the 10-30 Hz seismic motion. This shows up as the purple trace on the seismic BLRMS on the wall projector.

The second plot shows that this is not only a periodic increase in the usual 29.5 Hz HVAC peak, but also an anomolous 32.2 Hz peak. Probably some malfunctioning machinery - maybe in the 40m or maybe on the roof.

Attachment 1: gur1z_rms.pdf
gur1z_rms.pdf
Attachment 2: gur1z.pdf
gur1z.pdf
  828   Tue Aug 12 12:21:13 2008 josephbConfigurationCamerasVariation in fit over 140 images for GC650 and GC750
Used matlab to calculate Gaussian fits on 145 GC650 images and 142 GC750 images. These were individual images (no averaging) looking at the PSL output from May 29th 2008. The GC650 and GC750 were looking at a split, but had different exposure values, slightly different distances to the nominal waist of the beam, and were not centered on the beam identically. Mostly this is a test of the fluctuations in the fit from image to image.

Note the mm refer to the size or position on the CCD or CMOS detector itself.
GC650

Mean
Amplitude  X center       Y center       X waist  Y waist  Background offset from zero
           position (mm)  position (mm)  (mm)     (mm)
0.3743     1.7378         2.6220         0.7901   0.8650  0.0047

Standard Deviation
Amplitude  X center       Y center       X waist  Y waist  Background offset from zero
           position (mm)  position (mm)  (mm)     (mm)
0.0024     0.0006         0.0005         0.0005   0.0003  0.00001

Std/Mean x100 (percent)
Amplitude  X center       Y center       X waist  Y waist  Background offset from zero
           position (mm)  position (mm)  (mm)     (mm)
0.6%       0.03%          0.02%          0.06%    0.04%    0.29%


GC750

Mean
Amplitude  X center       Y center       X waist  Y waist  Background offset from zero
           position (mm)  position (mm)  (mm)     (mm)
0.2024     2.5967         1.4458         0.8245   0.9194  0.0418

Standard Deviation
Amplitude  X center       Y center       X waist  Y waist  Background offset from zero
           position (mm)  position (mm)  (mm)     (mm)
0.0011     0.0005         0.0005         0.0003   0.0005  0.00003

Std/Mean x100 (percent)
Amplitude  X center       Y center       X waist  Y waist  Background offset from zero
           position (mm)  position (mm)  (mm)     (mm)
0.6%       0.02%          0.04%          0.04%    0.05%    0.07%
  8882   Fri Jul 19 22:35:06 2013 KojiSummaryLSCVarious Arm signal (Yarm)

The StripTool plot attached below shows various arm signals measured with the Y arm cavity swept using ALS.

Yellow: TRY

Blue: ALS additive OFFSET to the error signal

Red: Raw PDH error signal (POY11I)

Purple: Linearized PDH error (POY11/TRY)

Green: 1/Sqrt(TRY)-5 (No normalization)

Inverse Sqrt of the TRY had been implemented when this LSC controller was first coded.
It is confirmed that the calculation is working correctly.

Attachment 1: various_arm_signal.png
various_arm_signal.png
  9685   Mon Mar 3 17:35:10 2014 KojiUpdateLSCVarious demod phase measurement

I wanted to check how the refl signals looked like.
I decided to measure the demod phase where PRCL and MICH appear, one by one.

The method I used is to actuate PRCL or MICH at a fixed frequency and rotate the demod phase such that
the signal at the actuating frequency disappears.

For the PRCL actuation, PRM was actuated by the lock-in oscillator with the amplitude of 100cnt.
For MICH, the ITMX and ITMY was actuate at the amplitude of 1000cnt and 1015cnt respectively.

The script I used was something like this

ezcaread C1:LSC-REFL11_PHASE_R
ezcaservo -r C1:CAL-SENSMAT_CARM_REFL11_Q_I_OUTPUT C1:LSC-REFL11_PHASE_R -g 100 -t 60
ezcaread C1:LSC-REFL11_PHASE_R

"11" should be changed according to the PD you want to test.
"Q" should be changed to "I" depending on form which quadrature you want to eliminate the signal

The option "-g" specifies the servo gain. This specifies which slope (up or down) of the sinusoidal curve the signal is locked.
Therefore, it is important to flip the signal angle 180degree if a negative gain is used.


Note: Original phase settings before touching them

REFL11  - 19.2
REFL33   135.4
REFL55    48.0
RELF165 -118.5

 

Here in the measurement PRMI was locked with AS55Q (MICH) and REFL55I (PRCL)


Without no serious reason I injected a peak at 503.1Hz. This peak is not notched out by the servo. There may have been
some residual effect of the feedback loops.

PRCL: By elliminating the peak from the Q quadrature, we optimize the I phase for PRCL.

REFL11,   minimize PRCL in "Q", gain, -1, -19.3659 deg
REFL33,   minimize PRCL in "Q", gain, -1, 132.813 deg
REFL55,   minimize PRCL in "Q", gain, -1, 20.9747 deg
REFL165, minimize PRCL in "Q", gain, -1, -119.004 deg

MICH: By elliminating the peak from the I quadrature, we optimize the Q phase for MICH.
If PRCL and MICH appears at the same phase, the resulting angles shows an identical number.

REFL11,   minimize PRCL in "I", gain, -1, -28.4526 deg
REFL33,   minimize PRCL in "I", gain, -1, 65.9148 deg
REFL55,   minimize PRCL in "I", gain, -1, 12.4051 deg
REFL165, minimize PRCL in "I", gain, -0.1, -143.75 deg


Then, the signal frequency was changed to 675Hz where the notch filters in the servo is active.

PRCL: By elliminating the peak from the Q quadrature, we optimize the I phase for PRCL.

REFL11,   minimize PRCL in "Q", gain, 1, -19.5224 deg
REFL33,   minimize PRCL in "Q", gain, -1, 135.868 deg
REFL55,   minimize PRCL in "Q", gain, 1, 48.5716 deg
REFL165, minimize PRCL in "Q", gain, 1, -122.398 deg

MICH: By elliminating the peak from the I quadrature, we optimize the Q phase for MICH.
If PRCL and MICH appears at the same phase, the resulting angles shows an identical number.

REFL11,   minimize PRCL in "I", gain, -10, -73.7153 deg
REFL33,   minimize PRCL in "I", gain, -10, 135.5 deg
REFL55,   minimize PRCL in "I", gain, 10, -2.55868 deg
REFL165, minimize PRCL in "I", gain, -5, -156.135 deg


 

 

This is just a test of the REFL channels for the arms signals. ETMX or ETMY were actuated.

YARM

REFL11, minimize ETMY in "Q", gain 100 => C1:LSC-REFL11_PHASE_R = 145.694
REFL55, minimize ETMY in "Q", gain 100 => C1:LSC-REFL11_PHASE_R = -60.1512

XARM

REFL11, minimize ETMX in "Q", gain 100 => C1:LSC-REFL11_PHASE_R = 142.365
REFL55, minimize ETMX in "Q", gain 100 => C1:LSC-REFL55_PHASE_R = -68.6521

  2556   Mon Feb 1 18:33:10 2010 steveUpdateMOPAVe half the lazer!

The 2W NPRO from Valera arrived today and I haf hidden it somewere in the 40m lab!

 

Rana was so kind to make this entry for me

Attachment 1: inno2w.JPG
inno2w.JPG
Attachment 2: inno2Wb.JPG
inno2Wb.JPG
  17036   Tue Jul 26 19:50:25 2022 DeekshaUpdateComputer Scripts / ProgramsVector fitting

Trying to vectfit to the data taken from the DFD previously but failing horribly. I will update this post as soon as I get anything semi-decent. For now here is this fit.

Attachment 1: data.png
data.png
Attachment 2: fit_attempt.png
fit_attempt.png
  17038   Tue Jul 26 21:16:41 2022 KojiUpdateComputer Scripts / ProgramsVector fitting

I think the fit fails as the measurement quality is not good enough.

 

  16955   Tue Jun 28 16:26:58 2022 CiciSummaryGeneralVector fitting open loop transfer function/Audio cancellation of optical table enclosure

[Deeksha, Cici]

We attempted to use vectfit to fit our earlier transfer function data, and were generally unsuccessful (see vectfit_firstattempt.png), but are much closer to understanding vectfit than before. Couple of problems to address - finding the right set of initial poles to start with has been very hard, and also however vectfit is plotting the phase data is unwrapping it, which makes it generally unreadable. Still working on how to mess with the vectfit automatically-generated plots. In general, our data is very messy (this is old data of the transfer function from last week), so we took more data today to see if our coherence was the problem (see TFSR785_28-06-2022_161937.pdf). As is visible from the graph, our coherence is terrible, and above 1kHz is almost entirely below 0.5 (or 0.2) on both channels. Figuring out why this is and fixing it is our first priority.

In the process of taking new data, we also found out that the optical table enclosure at the end of the X-arm does a decent job of sound isolation (see enclosure_open.mp4 and enclosure_closed.mp4). The clicking from the shutter is visible on a spectrogram at high frequencies when the enclosure is open, but not when it is closed. We also discovered that the script to toggle the shutter can run indefinitely, which can break the shutter, so we need to fix that problem!

Attachment 1: vectfit_firstattempt.png
vectfit_firstattempt.png
Attachment 2: TFSR785_28-06-2022_161937.pdf
TFSR785_28-06-2022_161937.pdf
Attachment 3: enclosure_open.MP4
Attachment 4: enclosure_closed.MP4
  14064   Fri Jul 13 10:54:55 2018 aaronUpdateVACVent 80

[aaron, steve]

Steve gave me a venting tutorial. I'll record this in probably a bit more detail than is strictly necessary, so I can keep track of some of the minor details for future reference.

Here is Steve's checklist:

  • Check that all jam nuts are tightened
  • all viewports are closed
  • op levs are off
  • take a picture of the MEDM screens
  • Check particle counts
  • Check that the cranes work & wiped
  • Check that HV is off

Gautam already did the pre-vent checks, and Steve took a screenshot of the IFO alignment, IMC alignment, master op lev screen, suspension condition, and shutter status to get a reference point. We later added the TT_CONTROL screen. Steve turned off all op levs.

We then went inside to do the mechanical checks

  • N2 cylinders in the 40m antechamber are all full enough (have ~700psi/day of nitrogen)
  • We manually record the particle count
    • this should be <10,000 on the 0.5um particles to be low enough to vent, otherwise we will contaminate the system
    • note: need to multiply the reading on the particle counter by 10 to get the true count
    • the temperature inside the PSL enclosure should be 23-24C +/- 3 degrees
    • We recorded the particle counts at ~830 and ~930, and the 0.5um count was up to ~3000
  • We put a beam stop in front of the laser at the PSL table
  • Checked that all HV supplies are either off or supplying something in air
    • we noticed four HV supplies on 1X1 that were on. Two were accounted for on the PSL table (FSS), and the other two were for C1IOO_ASC but ran along the upper cable rack. We got ahold of Gautam (sorry!) and he told us these go to the TT driver on OMC_SOUTH, where we verified the HV cables are disconnected. We took this to mean these HV supplies are not powering anything, and proceeded without turning these HV off.
    • There are HV supplies which were all either off or supplying something in-air at: 1Y4, 1Y2, OMC N rack, 1X9 (green steering HV)
  • Checked that the crane works--both move up and down
    • vertex crane switch is on the wall at the inner corner of the IFO
    • y arm crane switch is on the N wall at the Y end
    • turn off the cranes at the control strip after verifying they work
  • While walking around checking HV, we checked that the jam nuts and viewports are all closed
    • we replaced one viewport at the x arm that was open for a camera

After completing these checks, we grabbed a nitrogen cylinder and hooked it up to the VV1 filter. Steve gave me a rundown of how the vacuum system works. For my own memory, the oil pumps which provide the first level of roughing backstream below 500mtorr, so we typically turn on the turbo pumps (TP) below that level... just in case there is a calibrated leak to keep the pressure above 350mtorr at the oil pumps. TP2 has broken, so during this vent we'll install a manual valve so we can narrow the aperture that TP1 sees at V1 so we can hand off to the turbo at 500mtorr without overwhelming it. When the turbos have the pressure low enough, we open the mag lev pump. Close V1 if things screw up to protect the IFO. This 6" id manual gatevalve will allow us throttle the load on the small turbo while the maglev is taking over the pumping  The missmatch in pumping speed is 390/70 l/s [ maglev/varian D70 ]  We need to close down the conductive intake of the TP1 with manual gate valve so the 6x smaller turbo does not get overloaded...

We checked CC1, which read 7.2utorr.

Open the medm c0/ce/VacControl_BAK.adl to control the valves.

Steve tells me we are starting from vacuum normal state, but that some things are broken so it doesn't exactly match the state as described. In particular, VA6 is 'moving' because it has been disconnected and permanently closed to avoid pumping on the annulus. During this v ent, we will also keep pumping on the RGA since it is a short vent; steve logged the RGA yesterday.

We began the vent by following the vacuum normal to chamber open procedure.

  1. VM1 closed
  2. We didn't open VM3, because we want to keep the RGA on
  3. Closed V1
  4. Connect the N2 to the VV1 filter
    1. first puged the line with nitrogen
    2. We confirmed visually that V1 is closed
  5. We opened VM2 to pump on the RGA with the mag lev pump.
    1. This is a nonstandard step because we are keeping the RGA pumped down.
    2. The current on TP3 is ~0.19A, which is a normal, low load on the pump
  6. VV1 opened to begin the vent at ~10:30am
    1. use crescent wrench to open, torque wrench wheel to close
    2. Keep the pressure regulator below 10 psi for the vent. We started the vent with about 2psi, then increased to 8psi after confirming that the SUS sensors looked OK.
  7. We checked the pressure plot and ITMX/ETMX motion to make sure we weren't venting too quickly or kicking the optics
    1. Should look at eg C1:SUS-ITMX_SENSOR_LL, as well as C1:Vac-P1_pressure
  8. Once the pressure reaches 25torr, we switched over to dry air
    1. wipe off the outside dolly wheels with a wet rag, and exit through the x-arm door to get the air. Sweep off the area outside the door, and wipe off new air containers with the rag.
    2. Bring the cylinder inside, get the regulator ready/purged, and swap relatively quickly.
    3. We increased the vent speed to 10psi. 
    4. Steve says the vents typically take 4 of 300 cf cylinders from Airgas "Ultra Zero" AI  UZ300 that contains 0.1 PPM of THC

Everything looks good, so I'm monitoring the vent and swapping out cylinders.

At 12:08pm, the pressure was at 257 torr and I swapped out in a new cylinder.

Steve: Do not overpressurize the vacuum envelope! Stop around 720 Torr and let lab air do the rest. Our bellows are thin walled for seismic isolation.

Attachment 1: vent80wtiptilts.png
vent80wtiptilts.png
  14066   Fri Jul 13 16:26:52 2018 SteveUpdateVACVent 80 is completing...

Steve and Aaron,

6 hrs vent is reaching equlibrium to room air. It took 3 and a half instrument grade air cilynders [ AI UZ300 as labelled ] at 10 psi pressure. Average vent speed ~ 2 Torr/min

Valve configuration: IFO at atm and RGA is pumped through VM2 by TP1 maglev.

 

Attachment 1: @atm.png
@atm.png
Attachment 2: vent80_7h.png
vent80_7h.png
Attachment 3: ventregN2&Air_c.jpg
ventregN2&Air_c.jpg
  14081   Wed Jul 18 03:14:48 2018 AnnalisaUpdateGeneralVent 80 recovery

[Gautam, Johannes, Koji, Annalisa]

Tonight we increased the power of the PSL laser and we achieved the lock of both arms with high power.

The AUX beam alignment to the Y arm was recovered and the PLL restored (using the Marconi as LO).

We made a quick measurement of the phase noise and the results will be posted tomorrow.

The beam on the PSL has been blocked, as well as the AUX beam on the AS table. The Marconi has been switched off.


gautam:

  1. Before turning up PSL power, I placed a block in front of MC refl to avoid any PD burning. Replaced HR Y1 2" optic with the usual 10% reflective BS to direct MC REFL to the locking PD.
  2. Waveplate was rotated back to 180 deg (original position before the vent). After optimizing PMC transmission, I measured 1.05 W going into the IMC (pre-vent value was 1.07 W, prolly within power meter absolute accuracy).
  3. IMC autolocker restored to usual high power version on megatron.
  4. There seems to be some kind of vacuum interlock in effect that prevents me from opening the PSL shutter via EPICS - I had to toggle the position on the shutter controller under the table. After tonight's work, I returned the controller to the NC state, to avoid any further interference with this interlock code that may prevent pumping in the AM.
  5. PLL gain was re-adjusted to achieve maximum stability (judged by eye) of the beat-note in lock triggered on the Marconi LO signal. Alignment onto the NF beatPD was also tweaked to squeeze out as much beat as possible.
  6. The main objective tonight was to send AUX beam in, recover transmission beat, scan the AUX frequency, and resolve some peaks (MAX HOLD scanning technique, magnitude only for now, no phase info). Thanks to JE's expert fiber alignment and beatnote maximization, we achieved this yes. Annalisa will post a plot tmr. 
  7. For unknown reasons, the Y arm ASS does not maximize TRY. So we are in the unfortunate situation of neither arm having a working ASS servo. To be worked on later.
  14084   Wed Jul 18 23:43:50 2018 KojiUpdateGeneralVent 80 recovery

Is the reflector too close to the beam and causing clipping?

Quote:

For unknown reasons, the Y arm ASS does not maximize TRY. So we are in the unfortunate situation of neither arm having a working ASS servo. To be worked on later.

Attachment 1: IMG_5868.JPG
IMG_5868.JPG
Attachment 2: IMG_5382.JPG
IMG_5382.JPG
  14305   Mon Nov 19 14:59:48 2018 ChubUpdateVACVent 81

Vent 80 is nearly complete; the instrument is almost to atmosphere.  All four ion pump gate valves have been disconnected, though the position sensors are still connected,and all annulus valves are open.  The controllers of TP1 and TP3 have been disconnected from AC power. VC1 and VC2 have been disconnected and must remained closed. Currently, the RGA is being vented through the needle valve and the RGA had been shut off at the beginning of the vent preparations.  VM1 and VM3 could not be actuated.  The condition status is still listed as Unidentified because of the disconnected valves. 

  14306   Mon Nov 19 17:09:00 2018 SteveUpdateVACVent 81

Gautam, Aaron, Chub and Steve,

Quote:

Vent 80 is nearly complete; the instrument is almost to atmosphere.  All four ion pump gate valves have been disconnected, though the position sensors are still connected,and all annulus valves are open.  The controllers of TP1 and TP3 have been disconnected from AC power. VC1 and VC2 have been disconnected and must remained closed. Currently, the RGA is being vented through the needle valve and the RGA had been shut off at the beginning of the vent preparations.  VM1 and VM3 could not be actuated.  The condition status is still listed as Unidentified because of the disconnected valves. 

The vent 81 is completed.

4 ion pumps and cryo pump are at ~ 1-4 Torr (estimated as we have no gauges there), all other parts of the vacuum envelope are at atm. P2 & P3 gauges are out of order.

V1 and VM1 are in a locked state. We suspect this is because of some interlock logic.

TP1 and TP3 controllers are turned off.

Valve conditions as  shown: ready to be opened or closed or moved or rewired. To re-iterate: VC1, VC2, and the Ion Pump valves shouldn't be re-connected during the vac upgrade.

Thanks for all of your help.

Attachment 1: beforeVent82.png
beforeVent82.png
Attachment 2: vent81completed.png
vent81completed.png
  14318   Mon Nov 26 15:58:48 2018 SteveUpdateVACVent 81

Gautam, Aaron, Chub & Steve,

ETMY heavy door replaced by light one.

We did the following:  measured 950 particles/cf min of 0.5 micron at SP table, wiped crane and it's cable, wiped chamber,

                                placed heavy door on clean merostate covered stand, dry wiped o-rings and isopropanol wiped Aluminum light cover

                              

Quote:

Gautam, Aaron, Chub and Steve,

Quote:

Vent 80 is nearly complete; the instrument is almost to atmosphere.  All four ion pump gate valves have been disconnected, though the position sensors are still connected,and all annulus valves are open.  The controllers of TP1 and TP3 have been disconnected from AC power. VC1 and VC2 have been disconnected and must remained closed. Currently, the RGA is being vented through the needle valve and the RGA had been shut off at the beginning of the vent preparations.  VM1 and VM3 could not be actuated.  The condition status is still listed as Unidentified because of the disconnected valves. 

The vent 81 is completed.

4 ion pumps and cryo pump are at ~ 1-4 Torr (estimated as we have no gauges there), all other parts of the vacuum envelope are at atm. P2 & P3 gauges are out of order.

V1 and VM1 are in a locked state. We suspect this is because of some interlock logic.

TP1 and TP3 controllers are turned off.

Valve conditions as  shown: ready to be opened or closed or moved or rewired. To re-iterate: VC1, VC2, and the Ion Pump valves shouldn't be re-connected during the vac upgrade.

Thanks for all of your help.

 

  14398   Mon Jan 14 10:06:53 2019 gautamUpdateVACVent 82 complete

[chub, gautam]

  • IFO pressure was ~2e-4 torr when we started, on account of the interlock code closing all valves because the N2 line pressure dropped below threshold (<65 psi)
  • Chub fixed the problem on the regulator in the drill-press area where the N2 tanks are, the N2 line is now at ~75 psi so that we have the ability to actuate valves if we so desire
  • We decided that there is no need to vent the pumpspool this time - avoiding an unnecessary turbo landing, so the pumpspool is completely valved off from the main volume and TPs 1-3 are left running
  • Went through the pre-vent checklist:
    • Chub measured particle count, deemed it to be okay (I think we should re-locate the particle counter to near 1X8 because that is where the air enters the IFO anyways, and that way, we can hook it up to the serial device server and have a computerized record of this number as we had in the past, instead of writing it down in a notebook)
    • Checked that the PSL was manually blocked from entering the IFO
    • Walked through the lab, visually inspected Jam Nuts and window covers, all was deemed okay
  • Moved 2 tanks of N2 into the lab on account of the rain
  • Started the vent at ~930am PST
    • There were a couple of short bursty increases in the pressure as we figured out the right valve settings but on average, things are rising at approx the same rate as we had in vent 81...
    • There was a rattling noise coming from the drypump that is the forepump for TP2 (Agilent) - turned out to be the plastic shell/casing on the drypump, moreover, the TP2 diagnostics (temperature, current etc) are all normal.
    • The CC1 gauge (Hornet) is supposed to have an auto-shutoff of its High Voltage when the pressure exceeds 10 mTorr, but it was reporting pressures in the 1 mTorr range even when the adjacent Pirani was at 25 torr. To avoid risk of damage, we manually turned the HV off. There needs to be a python script that can be executed to transition control between the remote and local control modes for the hornet, we had to Power Cycle the gauge because it wouldn't give us local control over the HV.
    • Transitioned from N2 to dry air at P1a=25 torr. We had some trouble finding the correct regulator (left-handed thread) for the dry air cylinders, it was stored in a cabinet labelled green optics no
    • Disconnected dry air from VV1 intake once P1b reached 700 torr, to let lab air flow into the IFO and avoid overpressuring.
    • VA* and VAV* valves were opened so as to vent the annuli as we anticipate multiple chamber openings for this vent.

As of 8pm local time, the IFO seems to have equilibriated to atmospheric pressure (I don't hear the hiss of in-rushing air near 1X8 and P1a reports 760 torr). The pumpspool looks healthy and there are no signs in the TP diagnostics channels that anything bad happened to the pumps. Chub is working on getting the N2 setup more robust, we plan to take the EY door off at 9am tomorrow morning with Bob's help.

* I took this opportunity to follow instructions on pg 29 of the manual and set the calibration for the SuperBee pirani gauge to 760 torr so that it is in better agreement with our existing P1a Pirani gauge. The correction was ~8% (820-->760).

Attachment 1: Vent82Summary.png
Vent82Summary.png
  12233   Thu Jun 30 16:11:57 2016 ericqUpdateGeneralVent Prep

I have updated the vent prep checklist on the wiki. Gautam and I did the following things from it:

  • Center all oplevs, transmon QPDs 
    • ETMX oplev has not been centered, since it's moving around so much, and we're going to immediately move the suspension anyways.
  • Align the arm cavities for IR and align the green lasers to the arms. 
    • AUX X Green was aligned while the X arm was well aligned. Soon thereafter, ETMX wandered away, but the green will remain a good reference
  • Update the SUS Dritmon values 
  • Reconcile all SDF differences 
  • Reduce input power to no more than 100mW by adjusting wave plate+PBS setup on the PSL table BEFORE the PMC. (Using the WP + PBS that already exist after the laser.) 

  • Replace 10% BS before MC REFL PD with Y1 mirror and lock MC at low power.
    • I don't think we've vented since the most recent slew of changes to the IMC servo, so its not surprising that the current low power scripts don't work. I'm working on locking the IMC, but this does not prevent us from initiating the vent tomorrow.

 

The following bullets have not yet been executed:

  • Close shutter of PSL-IR and green shutters at the ends 
  • Make sure the jam nuts are protecting bellows 
  • Check crane functionality & cleanliness 

  • Turn off HV into vacuum: OMC is not wired this time 
  • Particle count must be under 10,000 counts / cf min for 0.5 micron 
  • Check all metal window covers are on. 
  • Check 5 cylinders (24 cft size) of instrument grade air, called Alfa Gas 1 in stock.
  12235   Thu Jun 30 21:24:36 2016 ericqUpdateGeneralVent Prep

Steve has ordered some teflon parts to take the place of the metal parts in his acetone-soaking jig. They should arrive tomorrow. 

So, we will be begin the venting process tomorrow. Doors to come off on Tuesday.

  15649   Wed Oct 28 20:42:53 2020 KojiUpdateGeneralVent Prep
  1. Two arms / BS / PRM / SRM were aligned. (Attachment 1)
  2. IMC was aligned by WFS and the WFS offsets were offloaded.
  3. Suspension Status Snapshot (Attachment 2)
  4. Oplevs are aligned (Attachment 3)
  5. Xarm green was aligned in the daytime. Xarm green refl DC (C1:ALS-X_REFL_DC_OUTPUT) was 620 (aligned) ~1300 (drifted). When unlocked, it was 3750.
  6. Yarm green: I saw no flash. We don't have functional PZT alignment since the ASY M2 PZT got broken. I went to the Yend. Something funky is going on with the Yend green. I struggled to have any flash of the cavity. The apertures were not so precise. I finally got TEM00 locked, but the modematching seems exteremely low (like 1/1000?). Basically I saw no power reduction of the refl when the cavity is locked. So at least the cavity was locked but we might need to revisit when we open the chamber
    ==> Gautam thinks it was not like that. So he will check the green alignment tomorrow (Thu).
  7. Item checking: I familialized myself with the yend crane operation. Today I learned that there is a power switch on the wall (Attachment 4). The yend has two heavy door storages (Attachments 4/5). The slings to lift the heavy door are in the crane cabinet along with the y arm (Attachment 6). I didn't yet try to find the "hammer" to hit the door if the o-ring stuck too strong, although that's optional.
  8. We want to reduce the PSL power. But Gautam wants to use the arm locking with the nominal power, it will be done tomorrow by him.
     
  9. The last thing is to check the green trans power. I noticed that the green trans beams are blocked by an HWP for the BHD LO path on the PSL table. (Attachment 7)
    The HWP was moved and the process was recorded in the movie (Attachment 8). The fiber output was monitored by the BHD DC (aka AS110 DC) with the AS path blocked. The fiber output of 22.6mV (offset -2.5mV) was improved tio 29.1mV after the HWP move and the alignment adjustment.
  10. Now the green transmissions are visible by the green PDs. Attachment 9 shows the trans and ref of each green beams with and without locking to TEM00. The questionable green TRY was ~0.3. If we compare this with the histrical data (Attachment 10), it is about 1/4 of the value in the past. It's not too crazy but still quite low.

At this point, I'm leaving the lab. All the suspensions (incl SRM) are aligned. PSL/GRX/GRY shutters were left open.

Attachment 1: Screen_Shot_2020-10-28_at_19.42.27.png
Screen_Shot_2020-10-28_at_19.42.27.png
Attachment 2: Screen_Shot_2020-10-28_at_19.42.38.png
Screen_Shot_2020-10-28_at_19.42.38.png
Attachment 3: Screen_Shot_2020-10-28_at_20.42.26.png
Screen_Shot_2020-10-28_at_20.42.26.png
Attachment 4: P_20201028_222839.jpg
P_20201028_222839.jpg
Attachment 5: P_20201028_223020.jpg
P_20201028_223020.jpg
Attachment 6: P_20201028_223212.jpg
P_20201028_223212.jpg
Attachment 7: 2020-10-28-233038.jpeg
2020-10-28-233038.jpeg
Attachment 8: HWP_moving.mp4
Attachment 9: Screenshot_from_2020-10-29_01-04-06.png
Screenshot_from_2020-10-29_01-04-06.png
Attachment 10: Screenshot_from_2020-10-29_01-16-53.png
Screenshot_from_2020-10-29_01-16-53.png
  15651   Thu Oct 29 12:43:35 2020 gautamUpdateGeneralVent Prep
  1. Oplev HeNe at ETMY was replaced, see here for my earlier discussion on this.
    • I thought this is a good idea since we want the Oplev as a coarse reference and it'd not be ideal if this HeNe dies during the time the optic is out of the chamber.
    • New HeNe had 2.8mW of power output as measured with Ophir power meter. This is in line with what is expected from these Lumentum heads.
    • I labelled the head with the power output and today's date, re-aligned the Oplev reflection onto its QPD. 
    • After this work, the Y-arm could be locked without the huge angular fluctuations that was visible earlier, 👍 .
  2. GTRY anomaly
    • I actually judged that there is no anomaly.
    • The GTRY CDS indicator is actually quite useless - the ADC saturates at ~3500 cts (and not 32768 as you would expect from a 16 bit ADC but that's the well-known whitening filter saturation problem). This should be fixed, but this is a task for later.
    • I measured with a DMM the voltage when the TEM00 is locked to the cavity and GTRY is 0.3 (the nominal value these days), the DC voltage was ~5.6 V. The prompt reflection from the ETM registers ~6.5 V DC. So the mode-matching isn't stellar, but this is again a known issue, and can be fixed later.
  3. Other pre-vent checks
    • The Oplevs had drifted significantly, I re-centered ITMs, ETMs and BS after aligning the arm cavities and green beams in the POX/POY lock state. See Attachment #1.
    • I locked the PRMI on carrier and used this config to re-center the PRM Oplev, see Attachment #2.
    • No further action was taken regarding SRM oplev.
    • I checked the ALS noise, see Attachment #3. The X arm ALS has excess noise at ~100 Hz that certainly wasn't there previosuly - sigh. There is nothing I can find about any changes made at EX in the elog.
    • Updated the "DriftMon" values, though I guess we don't even really use this anymore these days?
    • Re-relieved the IMC WFS offsets.
    • Cut the input power to the IMC from 1.007W to 100.1 mW (both numbers measured with Ophir power meter).
    • Replaced the 10% beamsplitter in the MCREFL path on the AS table with a Y1 HR mirror. Note that there is no beam on the IMC WFS in this configuration.
    • Was able to lock the IMC on low power to a TEM00 mode - need to set up the low power autolocker. The IMC autolocker is now set to the low power settings, and I've tested it locks a couple of times. Attachment #5 shows the low power lock in StripTool.
    • Walked around and looked at all the bellows - the jam nuts are up against their stops, and I can't move them with my hands, so I think that's okay.

If everything else looks good, I'll start letting the dry N2 into the main volume after lunch.

Quote:

Now the green transmissions are visible by the green PDs. Attachment 9 shows the trans and ref of each green beams with and without locking to TEM00. The questionable green TRY was ~0.3. If we compare this with the histrical data (Attachment 10), it is about 1/4 of the value in the past. It's not too crazy but still quite low.

BTW, nice video! @ Koji, How difficult was it to edit it into this form? 

Attachment 1: preVentAlignment.png
preVentAlignment.png
Attachment 2: PRMIcarr_PRMalignment.png
PRMIcarr_PRMalignment.png
Attachment 3: ALS_ool.pdf
ALS_ool.pdf
Attachment 4: lowPowerMC.png
lowPowerMC.png
  16407   Fri Oct 15 16:46:27 2021 AnchalSummaryOptical LeversVent Prep

I centered all the optical levers on ITMX, ITMY, ETMX, ETMY, and BS to a position where the single arm lock on both were best aligned. Unfortunately, we are seeing the TRX at 0.78 and TRY at 0.76 at the most aligned positions. It seems less power is getting out of PMC since last month. (Attachment 1).

Then, I tried to lock PRMI with carrier with no luck. But I was able to see flashing of up to 4000 counts in POP_DC. At this position, I centered the PRM optical lever too (Attachment 2).

Attachment 1: Screen_Shot_2021-10-15_at_4.34.45_PM.png
Screen_Shot_2021-10-15_at_4.34.45_PM.png
Attachment 2: Screen_Shot_2021-10-15_at_4.45.31_PM.png
Screen_Shot_2021-10-15_at_4.45.31_PM.png
Attachment 3: Screen_Shot_2021-10-15_at_4.34.45_PM.png
Screen_Shot_2021-10-15_at_4.34.45_PM.png
Attachment 4: Screen_Shot_2021-10-15_at_4.34.45_PM.png
Screen_Shot_2021-10-15_at_4.34.45_PM.png
  16408   Fri Oct 15 17:17:51 2021 KojiSummaryGeneralVent Prep

I took over the vent prep: I'm going through the list in [ELOG 15649] and [ELOG 15651]. I will also look at [ELOG 15652] at the day of venting.

  1. IFO alignment: Two arms are already locking. The dark port beam is well overlapped. We will move PRM/SRM etc. So we don't need to worry about them. [Attachment 1]
    scripts>z read C1:SUS-BS_PIT_BIAS C1:SUS-BS_YAW_BIAS
    -304.7661529521767
    -109.23924626857811
    scripts>z read C1:SUS-ITMX_PIT_BIAS C1:SUS-ITMX_YAW_BIAS
    15.534616817500943
    -503.4536332290159
    scripts>z read C1:SUS-ITMY_PIT_BIAS C1:SUS-ITMY_YAW_BIAS
    653.0100945988496
    -478.16260735781225
    scripts>z read C1:SUS-ETMX_PIT_BIAS C1:SUS-ETMX_YAW_BIAS
    -136.17863332517527
    181.09285307121306
    scripts>z read C1:SUS-ETMY_PIT_BIAS C1:SUS-ETMY_YAW_BIAS
    -196.6200333695437
    -85.40819256078339

     
  2. IMC alignment: Locking nicely. I ran WFS relief to move the WFS output on to the alignment sliders. All the WFS feedback values are now <10. Here is the slider snapshots. [Attachment 2]
     
  3. PMC alignmnet: The PMC looked like it was quite misaligned -> aligned. IMC/PMC locking snapshot [Attachment 3]
    Arm transmissions:
    scripts>z avg 10  C1:LSC-TRX_OUT C1:LSC-TRY_OUT
    C1:LSC-TRX_OUT 0.9825591325759888
    C1:LSC-TRY_OUT 0.9488834202289581

     
  4. Suspension Status Snapshot [Attachment 4]
     
  5. Anchal aligned the OPLEV beams [ELOG 16407]
    I also checked the 100 days trend of the OPLEV sum power. The trend of the max values look flat and fine. [Attachment 5] For this purpose, the PRM and SRM was aligned and the SRM oplev was also aligned. The SRM sum was 23580 when aligned and it was just fine (this is not so visible in the trend plot).
     
  6. The X and Y green beams were aligned for the cavity TEM00s. Y end green PZT values were nulled. The transmission I could reach was as follows.
    >z read C1:ALS-TRX_OUTPUT C1:ALS-TRY_OUTPUT
    0.42343354488901286
    0.24739624058377277

    It seems that these GTRX and GTRY seemed to have crosstalk. When each green shutters were closed the transmissino and the dark offset were measured to be
    >z read C1:ALS-TRX_OUTPUT C1:ALS-TRY_OUTPUT
    0.41822833190834546
    0.025039383697636856
    >z read C1:ALS-TRX_OUTPUT C1:ALS-TRY_OUTPUT
    0.00021112720155274818
    0.2249448773499293

    Note that Y green seemed to have significant (~0.1) of 1st order HOM. I don't know why I could not transfer this power into TEM00. I could not find any significant clipping of the TR beams on the PSL table PDs.
     
  7. IMC Power reduction
    Now we have nice motorized HWP. sitemap -> PSL -> Power control
    == Initial condition == [Attachment 6]
    C1:IOO-HWP_POS 38.83
    Measured input power = 0.99W
    C1:IOO-MC_RFPD_DCMON = 5.38
    == Power reduction == [Attachment 7]
    - The motor was enabled upon rotation on the screen

    C1:IOO-HWP_POS 74.23
    Measured input power = 98mW
    C1:IOO-MC_RFPD_DCMON = 0.537
    - Then, the motor was disabled
     
  8. Went to the detection table and swapped the 10% reflector with the 98% reflector stored on the same table. [Attachments 8/9]
    After the beam alignment the MC REFL PD received about the same amount of the light as before.
    C1:IOO-MC_RFPD_DCMON = 5.6
    There is no beam delivered to the WFS paths.
    CAUTION: IF THE POWER IS INCREASED TO THE NOMINAL WITH THIS CONFIGURATION, MC REFL PD WILL BE DESTROYED.
  9. The IMC can already be locked with this configuration. But for the MC Autolocker, the MCTRANS threshold for the autolocker needs to be reduced as well.
    This was done by swapping a line in  /opt/rtcds/caltech/c1/scripts/MC/AutoLockMC.init
    # BEFORE
    /bin/csh ./AutoLockMC.csh >> $LOGFILE
    #/bin/csh ./AutoLockMC_LowPower.csh >> $LOGFILE
    --->
    # AFTER
    #/bin/csh ./AutoLockMC.csh >> $LOGFILE
    /bin/csh ./AutoLockMC_LowPower.csh >> $LOGFILE

    Confirmed that the autolocker works a few times by toggling the PSL shutter. The PSL shutter was closed upon the completion of the test
     
  10. Walked around the lab and checked all the bellows - the jam nuts are all tight, and I couldn't move them with my hands. So this is okay according to the ancient tale by Steve.
Attachment 1: Screenshot_2021-10-15_17-36-00.png
Screenshot_2021-10-15_17-36-00.png
Attachment 2: Screenshot_2021-10-15_17-39-58.png
Screenshot_2021-10-15_17-39-58.png
Attachment 3: Screenshot_2021-10-15_17-42-20.png
Screenshot_2021-10-15_17-42-20.png
Attachment 4: Screenshot_2021-10-15_17-46-13.png
Screenshot_2021-10-15_17-46-13.png
Attachment 5: Screenshot_2021-10-15_18-05-54.png
Screenshot_2021-10-15_18-05-54.png
Attachment 6: Screen_Shot_2021-10-15_at_19.45.05.png
Screen_Shot_2021-10-15_at_19.45.05.png
Attachment 7: Screen_Shot_2021-10-15_at_19.47.10.png
Screen_Shot_2021-10-15_at_19.47.10.png
  16409   Fri Oct 15 20:53:49 2021 KojiSummaryGeneralVent Prep

From the IFO point of view, all look good and we are ready for venting from Mon Oct 18 9AM

  12299   Wed Jul 13 15:35:56 2016 LydiaUpdateGeneralVent Progress - ETMY repositioned and removed

[Lydia, Johannes]

Took photos to document the original OSEM orientation and wrote down the serial numbers for each position. We removed the OSEMs, moved the suspension to the accessible side of the table and took out the optic, which was brought to the clean room to have the magnets reglued. The ETMY chamber is now closed up with the OSEMs and clamps inside on the table, and should not need to be reopened until the magnets have been reattached. 

ELOG V3.1.3-