40m QIL Cryo_Lab CTN SUS_Lab TCS_Lab OMC_Lab CRIME_Lab FEA ENG_Labs OptContFac Mariner WBEEShop
  40m Log, Page 334 of 344  Not logged in ELOG logo
New entries since:Wed Dec 31 16:00:00 1969
ID Date Author Type Categoryup Subject
  16692   Wed Mar 2 11:50:39 2022 TegaUpdateVACOngoing work to get the FRG gauges readouts to EPICs channels

Here is the IOLAN SDS TCP socket setting and the USBserial setting for comparison.

I have also included the python script and output from the USBserial test from earlier.

Attachment 1: XGS600_IOLAN_settings_1.png
XGS600_IOLAN_settings_1.png
Attachment 2: XGS600_IOLAN_settings_2.png
XGS600_IOLAN_settings_2.png
Attachment 3: XGS600_USBserial_settings.png
XGS600_USBserial_settings.png
Attachment 4: XGS600_comm_test.py
#!/usr/bin/env python

#Created 2/24/22 by Tega Edo
'''Script to read/write to the XGS-600 Gauge Controller'''

import serial
import sys,os,math,time

ser = serial.Serial('/dev/cu.usbserial-1410') # open serial port 

... 74 more lines ...
Attachment 5: XGS600_comm_test_result.txt
----- Multiple Sensor Read Commands -----

Sent to XGS-600 -> #0001\r : Read XGS contents
response : >FE4CFE4CFE4C

Sent to XGS-600 -> #0003\r : Read Setpoint States
response : >0000

Sent to XGS-600 -> #0005\r : Read software revision
response : >0206,0200,0200,0200
... 69 more lines ...
  16693   Wed Mar 2 12:40:08 2022 TegaUpdateVACOngoing work to get the FRG gauges readouts to EPICs channels

Connector Test:

A quick test to rule out any issue with the Ethernet to Serial adapter was done using the setup shown in Attachment 1. The results rule out any connector problem.

 

IOLAN COMM test (as per Koji's suggestion):

The next step is to swap the controller with a laptop set up to receive serial commands using the same settings as the XGS600 controller. Basically, run a slightly modified version of python script where we go into listening mode. Then send commands to the TCP socket on the IOLAN SDS unit using c1vac and check what data makes its way to the laptop USBserial terminal. After working on this for a bit, I realized that we do not need to do anything on the c1vac machine. We only need to start the service as it would work normally. So I wrote a small python code for a basic XGS-600 controller emulator, see Attachment 4. The outputs from the laptop and c1vac terminals are Attachments 5 and 6 respectively. 

These results show that we can communicate via the assigned IP address "192.168.114.22" and the commands that are sent from c1vac reaches the laptop in the correct format. Furthermore, the serial_XSG service, a part modbusIOC_XGS service, which usually exits with an error seems fine now after successfully communicating with the laptop. I don't know why it did not die after the tests. I also found a bug in my code as a result of the test, where the status field for the fourth gauge didn't get written to. 

 

Pressure reading issue:

I noticed that the pressure reading was not giving the atmospheric value of ~760 Torrs as expected. Looking through my previous readouts, it seems the unit showed this atm value of ~761 Torrs when the first gauge was attached. However, a closer look at the issue revealed a transient behavior, i.e. when the unit is turned on the reading dips to atm value but eventually rises up to 1000 Torrs. I don't think this is a calibration problem bcos the value of 1000 Torrs is the maximum value for the gauge range. I also found out that when the XGS-controller has been running for a while, a power cycle does not have this transient behavior. So maybe a faulty capacitor somewhere? I have attached a short video clip that shows what happens when the XGS-controller unit is turned on.

Attachment 1: IMG_20220302_123529382.jpg
IMG_20220302_123529382.jpg
Attachment 2: XGS600_Serial2Ethernet2Serial2USB_comm_test_result.txt
$ python3 XGS600_comm_test.py  

----- Multiple Sensor Read Commands -----

Sent to XGS-600 -> #0001\r : Read XGS contents
response : >FE4CFE4CFE4C

Sent to XGS-600 -> #0003\r : Read Setpoint States
response : >0000

... 73 more lines ...
Attachment 3: VID-20220302-WA0001.mp4
Attachment 4: comm_test_c1vac_to_laptop_via_iolansds.py
#!/usr/bin/env python

#Created 3/2/22 by Tega Edo
'''Script to emulate XGS-600 controller using laptop USBserial port'''

import serial
import sys,os,math,time

ser = serial.Serial('/dev/cu.usbserial-1410') # open serial port 

... 19 more lines ...
Attachment 5: laptop_terminal.txt
(base) tega.edo@Tegas-MBP serial % python3 comm_test_c1vac_to_laptop_via_iolansds.py

----- Listen for USBserial command and asynchronously send data in XGS600 format -----

Command received from c1vac [1] : 

Data sent to c1vac [1] : >1.000E+00,NOCBL    ,NOCBL    ,NOCBL    ,2.00E+00,NOCBL\r
Command received from c1vac [2] : 

Data sent to c1vac [2] : >2.000E+00,NOCBL    ,NOCBL    ,NOCBL    ,3.00E+00,NOCBL\r
... 54 more lines ...
Attachment 6: c1vac_terminal.txt
controls@c1vac:/opt/target/python/serial$ caget C1:Vac-FRG1_status && caget C1:Vac-FRG2_status && caget C1:Vac-FRG3_status && caget C1:Vac-FRG4_status && caget C1:Vac-FRG5_status
C1:Vac-FRG1_status             1.530E+02
C1:Vac-FRG2_status             OFF
C1:Vac-FRG3_status             OFF
C1:Vac-FRG4_status             NO COMM
C1:Vac-FRG5_status             1.55E+02
controls@c1vac:/opt/target/python/serial$ caget C1:Vac-FRG1_status && caget C1:Vac-FRG2_status && caget C1:Vac-FRG3_status && caget C1:Vac-FRG4_status && caget C1:Vac-FRG5_status
C1:Vac-FRG1_status             1.630E+02
C1:Vac-FRG2_status             OFF
C1:Vac-FRG3_status             OFF
... 70 more lines ...
  16701   Fri Mar 4 18:12:44 2022 KojiUpdateVACRGA pumping down

1. Jordan reported that the newly installed Pirani gauge for P2 shows 850Torr while PTP2 show 680 Torr. Because of this, the vacuum interlock fails when we try to open V4.

2. Went to c1vac. Copied the interlock setting file interlock_conditions.yaml to interlock_conditions_220304.yaml
3. Deleted diffpressure line and pump_underspeed line for V4
4. Restarted the interlock service

controls@c1vac:/opt/target/python/interlocks$ sudo systemctl status interlock.service  
controls@c1vac:/opt/target/python/interlocks$ sudo systemctl restart interlock.service
controls@c1vac:/opt/target/python/interlocks$ sudo systemctl status interlock.service

5. The above 2~4 was unnecessary. Start over.


Let RP1/3 pump down TP1 section through the pump spool. Then let TP2 pump down TP1 and RGA.

1. Open V7. This made P2 a bit lower (P2 is alive) and P3.
2. Connected the main RP tube to the RP port.
2. Started RP1/3. PRP quickly reaches 0.4Torr.
3. Opened V6 this made P3 and O2 below 1Torr.
4. Close V6. Shutdown RP1/3. Disconnect the RP tube.
5. Turn on auxRP at the wall powe
6. Turn on TP2. Wait for the starting up.
7. Open V4. Once the pressure is below Pirani range, open VM3.
8. Keep it running over the weekend.

9. Once TP2 reached the nominal speed, the "StandBy" button was clicked to lower the rotation speed (for longer life of TP2)

Attachment 1: Screenshot_2022-03-04_19-38-11.png
Screenshot_2022-03-04_19-38-11.png
  16704   Sun Mar 6 18:14:45 2022 TegaUpdateVACOngoing work to get the FRG gauges readouts to EPICs channels

Following repeated failure to establish communication between c1vac and the XGS600 controller via Perle IOLAN serial device server, I decided to monitor the signal voltage of the communication channels (pin#2, pin#3 and pin#5) using an oscilloscope. The result of this investigation is presented in the attached pdf document. In summary, it seems I have used a crossed wired RS232 serial cable instead of a normal RS232 serial cable, so the c1vac read request command is being relayed on the wrong comm channel (pin#2 instead of pin#3). I will swap out the cable to see if this resolves the problem.  

Attachment 1: iolan_xgs_comm_investigation.pdf
iolan_xgs_comm_investigation.pdf iolan_xgs_comm_investigation.pdf iolan_xgs_comm_investigation.pdf iolan_xgs_comm_investigation.pdf
  16706   Mon Mar 7 13:53:40 2022 TegaUpdateVACOngoing work to get the FRG gauges readouts to EPICs channels

So it appears that my deduction from the pictures of needing a cable swap was correct, however, it turns out that the installed cable was actually the normal RS232 and what we need instead is the RS232 null cable. After the swap was done, the communication between c1vac and the XGS600 controller became active. Although, the data makes it all to the to c1vac without any issues, the scope view of it shows that it is mainly utilizing the upper half of the voltage range which is just over 50% of the available range. I don't know what to make of this.

 

I guess, the only remaining issue now is the incorrect atmospheric pressure reading of 1000 Torrs. 

 

Quote:

Following repeated failure to establish communication between c1vac and the XGS600 controller via Perle IOLAN serial device server, I decided to monitor the signal voltage of the communication channels (pin#2, pin#3 and pin#5) using an oscilloscope. The result of this investigation is presented in the attached pdf document. In summary, it seems I have used a crossed wired RS232 serial cable instead of a normal RS232 serial cable, so the c1vac read request command is being relayed on the wrong comm channel (pin#2 instead of pin#3). I will swap out the cable to see if this resolves the problem.  

 

Attachment 1: iolan_xgs_comm_live.pdf
iolan_xgs_comm_live.pdf
  16707   Mon Mar 7 14:52:34 2022 KojiUpdateVACOngoing work to get the FRG gauges readouts to EPICs channels

Great trouble shoot!

> I guess, the only remaining issue now is the incorrect atmospheric pressure reading of 1000 Torrs. 

This is just a calibration issue. The controller should have the calibration function.
(The other Pirani showing 850Torr was also a calibration issue although I didn't bother to correct it. I think the pirani's typically has large distribution of the calibration values and requires individual calibration)

  16713   Tue Mar 8 12:08:47 2022 TegaUpdateVACOngoing work to get the FRG gauges readouts to EPICs channels

OMG, it worked! It was indeed a calibration issue and all I had to do was press the "OK" button after selecting the "CAL" tab beside the pressure reading. Wow.

Quote:

Great trouble shoot!

> I guess, the only remaining issue now is the incorrect atmospheric pressure reading of 1000 Torrs. 

This is just a calibration issue. The controller should have the calibration function.
(The other Pirani showing 850Torr was also a calibration issue although I didn't bother to correct it. I think the pirani's typically has large distribution of the calibration values and requires individual calibration)

 

Attachment 1: XGS600_calibration.pdf
XGS600_calibration.pdf
  16717   Wed Mar 9 10:23:28 2022 JordanUpdateVACLeak Testing of New Manual Gate Valve - Attempt #1

Jordan, Chub, Paco

Chub and I went into the lab this morning to leak check the new gate valve after pumping over the weekend. This would be done through the RGA and spraying helium around the newly installed flanges. The RGA is set to monitor the partial pressure of helium versus time and we visually watch for any spikes in pressure to indicate an air leak.

So, I mistakenly thought the gate valve was opened and both sides were being pumped on, this was not the case. The vavle was closed so there was a pressure differential whenI turned the handle it tripped the interlocks and closed V4. I closed VM3 to the RGA volume to prevent the filament from being damaged.

Then the medm screen for the vac controls started flashing rapidly, I closed the window and reopened the controls to find all the panels were white, but we could still see the read only vac screen. So Paco restarted c1vac and the controls were restored. V7 closed, but was then reopened.

We now need to restart pumping with the odd pressure differentials between V4 and VM3.

Plan

- TP2 turned off along with the AUX pump

- VM3 opened to vent RGA volume (RGA turned off, filament cooled)

- Open the manual vent screw on TP1 to bring the RGA+TP1 volume back to atmosphere, now no pressure diff. on V4

- Open V4, restart AUX pump to rough out the volume

- Connect RP1/3 line to the pump scroll, turn on RP1/3 RGA+TP1 volume went to mtorr

- Slowly open the manual gate valve

- Connect AUX pump to TP2 and rough out

- Restart TP2, once at speed open V4, close V6 and turn off RP1/3

Now we are pumping on the RGA/TP1 volume with TP2, leak check attempt #2 will happen tomorrow morning

  16721   Thu Mar 10 09:39:59 2022 JordanUpdateVACLeak Testing of New Manual Gate Valve - Attempt #2

This morning Chub and I leak checked the manual gate valve with the RGA and helium. There was no change in the helium partial pressure while spraying helium around the flanges, all looks good.

 

I also took a 100 AMU analog scan, after the filament had warmed up overnight and the plot was quite noisy even with the lowest scan speed. I recommend this unit go back to SRS for a filament replacement/recalibration. I am worried yesterday's "vent" of the RGA volume may have burned the filament. See the comparison of yesterday's analog scan to today's below.

Attachment 1: 3-09-2022.PNG
3-09-2022.PNG
Attachment 2: 3-10-2022.PNG
3-10-2022.PNG
  16747   Tue Mar 29 15:07:11 2022 TegaSummaryVACOpening of MC and ETM chambers

[Anchal, Chub, Ian, Paco, Tega]

Today, we opened the MC chamber, ETMX chamber and ETMY chamber.

  16769   Mon Apr 11 11:00:30 2022 JancarloUpdateVACC1VAC Reboot and Nitrogen tanks

[Paco, JC, Ian, Jordan, Chub]

Checking in the morning, I walked over to the Nitrogen tanks to check the levels. Noticed one tank was empty, so I swapped it out. Chub came over to check the levels and to take note of how many tanks were left available for usage (None). Chub continued to put in a work order for a set of full Nitrogen tanks. We should be set on Nitrogen until Thursday this week (4/14/22).

As for C1VAC, this morning, Paco and I attempted to open the PSL shutter, but the interlock system was tripped so we didn't get any light into the IFO. We traced the issue down to C1VAC being unresponsive. We discussed this may have interlocked as a result of the Nitrogen tanks running out, but we do not believe this was the issue since we would have recieved an email. We tried troubleshooting as much as possible avoiding a reboot, but were unable to solve the issue. In response, we ran the idea of a reboot across Jordan and Ian, where everyone was in agreement, and fixed the system. Restarting c1vac seems to have closed V4, but this didn't cause any issues with the current state of the vacuum system.

After opening the PSL shutter again, we see the laser down the IFO, so we resume alignment work

  16772   Tue Apr 12 09:05:21 2022 JordanUpdateVACNew Pressure Gauge Install/Pump Spool Vent

Today, Tega and I would like to vent the pump spool an dinstall the new FRG-400 Agilent Pressure Gauges (per elog 15703). The attached picture shows the volume needed to be vented highlighted in red, and the gauges that need to be replaced/removed (purple dot next to the name).

The vent plan is as follows:

Open RV2

Open VM3

Open V7

Open V4

Shut down TP2

Install new gauges

Will add to post with updates post vent.

Attachment 1: Screenshot_2022-04-12_08-42-33.png
Screenshot_2022-04-12_08-42-33.png
Attachment 2: 81CB0936-1B19-4722-8A32-C3DC1D1FBC21.heic
Attachment 3: 2262ECEF-6200-4E95-8E32-C83CB9EB4F17.heic
Attachment 4: B5712B34-ECF6-43BE-BCB2-38CD775CF653.heic
Attachment 5: BEE14A07-976A-45C2-82A0-1774D377941E.heic
Attachment 6: 84557D6E-6AAE-47CF-A3AD-1DF329FEB550.heic
  16773   Wed Apr 13 12:50:07 2022 TegaUpdateVACNew Pressure Gauge Install/Pump Spool Vent

[Jordan, JC, Tega]

We have installed all the FRGs and updated the VAC medm screens to display their sensor readings. The replacement map is CC# -> FRG#, where # in [1..4] and PRP1 -> FRG5. We now need to clean up the C1VAC python code so that it is not overloaded with non-function gauges (CC1,CC2,CC3,CC4,PRP1). Also, need to remove the connection cables for the old replaced gauges.

Attachment 1: VAC_FRG_UPGRADE.png
VAC_FRG_UPGRADE.png
  16777   Thu Apr 14 09:04:30 2022 JordanUpdateVACRGA Volume RGA Scans

Prior to venting the RGA volume on Tuesday (4/12/2022) I took an RGA scan of the volume to be vented (RGA+TP1 volume+Manual Gate Valve) to see if there was a difference after replacing the manual gate valve. Attached is the plot from 4/12/22, and an overlay plot to complare 4/12/22 to 12/10/2021, when the same volume was scanned with the old (defective) manual gate valve.

There is a significant drop in the ratio O2 compared the the nitrogen peak and reduced Argon (AMU 40) which indicates there is no longer a large air leak.

12/10/21 N2/O2 ratio ~ 4 (Air 78%N2 / 21%O2)

4/12/22 N2/O2 ratio ~ 10      

There is one significant (above noise level) peak above AMU 46, which is at AMU 58. This could possibly be acetone (AMU 43 and 58) but overall the new RGA Volume scans look significantly better after the manual gate valve replacement. Well done!

Attachment 1: 40mRGA_Overlay.pdf
40mRGA_Overlay.pdf
Attachment 2: RGAVolume_4_12_22.PNG
RGAVolume_4_12_22.PNG
  16794   Thu Apr 21 11:31:35 2022 JCUpdateVACGauges P3/P4

[Jordan, JC]

It was brought to attention during yesterday's meeting that the pressures in the vacuum system were not equivalent althought the valve were open. So this morning, Jordan and I reviewed the pressure gauges P3 and P4. We attempted to recalibrate, but the gauges were unresponsive. Following this, we proceeded to connect new gauges on the outside to test for a calibration. The two gauges successfully calibrated at atmosperic pressure. We then proceeded to remove the old gauges and install the new ones. 

Attachment 1: IMG_0560.jpeg
IMG_0560.jpeg
Attachment 2: IMG_0561.jpeg
IMG_0561.jpeg
  16796   Thu Apr 21 16:36:56 2022 TegaUpdateVACcleanup work for vacuum git repo

git repo - https://git.ligo.org/40m/vac

Finally incorporated the FRGs into the main modbusIOC service and everything seems to be working fine. I have also removed the old sensors (CC1,CC2,CC3,CC4,PTP1,IG1) from the serial client list and their corresponding EPICS channels. Furthermore, the interlock service python script has been updated so that all occurrence of old sensors (turns out to be only CC1) were replaced by their corresponding new FRG sensor (FRG1) and a redundnacy was also enacted for P1a where the interlock condition is replicated with P1a being replaced with FRG1 because they both sense the main volume pressure.

  16806   Fri Apr 22 14:14:33 2022 JordanUpdateVACTP3 Forepump tip seal replacement

Jordan, JC

While the pumpspool is vented, I thought it would be a convenient time to change out the tip seal on the TP3 forepump. This one had not been changed since 2018, so as preventative maintence I had JC remove the pump and begin cleaning/installing the new tip seal.

Unfortunately the tip seal broke, but I have ordered another. We should have this pump ready to go late next week. If one is needed sooner, there is a spare IDP 7 pump we can install as the TP3 forepump.

Attachment 1: IMG_0572.jpeg
IMG_0572.jpeg
Attachment 2: IMG_0573.jpeg
IMG_0573.jpeg
  16820   Fri Apr 29 08:34:40 2022 JCUpdateVACRGA Pump Down

Jordan and I, in order to start pumpig down the RGA Volume, we began by opening V7 and VM. Afterwards, we started RP1 and RP3. After this, the pressure in the line between RP1, RP3, and V6 dropped to 3.4 mTorr. Next, we tried to open V6, although an error message popped up. We haven't been able to erase it since. But we were able to turn on TP2 with V4 closed. The pressure in that line is reporting 1.4 mTorr.

 

PRP on the sitemap is giving off an incorrect pressure for the line between RP1, RP3, and V6. This is verified by the pressure by the control screen and the physical controller as well. 

Attachment 1: Screen_Shot_2022-04-29_at_8.46.53_AM.png
Screen_Shot_2022-04-29_at_8.46.53_AM.png
  16822   Mon May 2 10:00:34 2022 JordanUpdateVACTP3 Forepump tip seal replacement

[JC, Jordan]

Jordan recieved the new tip seal Friday afternoon and I continued the replacement process in the morning. Finishing up, we proceeded to test the pump in the Clean and Bake room. The pump's pressure lowered to 110 mTorr, and we continue pumping so the seal can recieve a good fitting.

Update: We have confirmed the pump is working great and have reinstalled this back into the vacuum system.                                                                                                                   Note: The same O-Rings were used.

Quote:

Jordan, JC

While the pumpspool is vented, I thought it would be a convenient time to change out the tip seal on the TP3 forepump. This one had not been changed since 2018, so as preventative maintence I had JC remove the pump and begin cleaning/installing the new tip seal.

Unfortunately the tip seal broke, but I have ordered another. We should have this pump ready to go late next week. If one is needed sooner, there is a spare IDP 7 pump we can install as the TP3 forepump.

 

  16825   Tue May 3 13:18:47 2022 JCUpdateVACRGA Pump Down

Jordan, Tega, JC

Issue has been resolved. Breaker on RP1 was tripped so the RP1 button was reporting ON, but was not actually on which continuously tripped the V6 interlock. Breaker was reset, RP1 and RP3 turned on. The V6 was opened to rough out the RGA volume. Once, pressure was at ~100mtorr, V4 was opened to pump the RGA with TP2. V6 was closed and RP1/3 were turned off.

RGA is pumping down and will take scans next week to determine if a bakeout is needed

Quote:

Jordan and I, in order to start pumpig down the RGA Volume, we began by opening V7 and VM. Afterwards, we started RP1 and RP3. After this, the pressure in the line between RP1, RP3, and V6 dropped to 3.4 mTorr. Next, we tried to open V6, although an error message popped up. We haven't been able to erase it since. But we were able to turn on TP2 with V4 closed. The pressure in that line is reporting 1.4 mTorr.

 

PRP on the sitemap is giving off an incorrect pressure for the line between RP1, RP3, and V6. This is verified by the pressure by the control screen and the physical controller as well. 

 

  16867   Fri May 20 12:42:23 2022 TegaUpdateVACDoor installation on the end stations

[JC, Tega, Chub]

Today we installed the 200 lbs doors on the end station chambers.

  16870   Tue May 24 10:37:09 2022 TegaUpdateVACadded FRG channels to slow channel ini file

[Vacuum gauge sensors]

Paco informed me that the FRG sensor EPICS channels are not available on dataviewer, so I added them to slow channels ini file (/opt/rtcds/caltech/c1/chans/daq/C0EDCU.ini). I also commented out the old CC1, CC2, CC3 and CC4 gauges. A service restart is required for them to become available but this cannot be done right now because it would adversely affect the progress of the upgrade work. So this would be done at a later date.

Quote:

git repo - https://git.ligo.org/40m/vac

Finally incorporated the FRGs into the main modbusIOC service and everything seems to be working fine. I have also removed the old sensors (CC1,CC2,CC3,CC4,PTP1,IG1) from the serial client list and their corresponding EPICS channels. Furthermore, the interlock service python script has been updated so that all occurrence of old sensors (turns out to be only CC1) were replaced by their corresponding new FRG sensor (FRG1) and a redundnacy was also enacted for P1a where the interlock condition is replicated with P1a being replaced with FRG1 because they both sense the main volume pressure.

 

  16871   Tue May 24 11:04:53 2022 JCUpdateVACBeginning Pumpdown

[JC, Jordan, Paco, Chub]

We began with the pumpdown this morning. We started with the annulus volume and proceeded by using the following:

1. Isolate the RGA Volume by closing of valves VM3 and V7.

2. Opened valves VASE, VASV, VABSSCT, VABS, VABSSCO, VAEV, and VAEE, in that order.

3. Open VA6 to allow P3, FRG3, and PAN to equalize.

4. Turn on RP1 and RP3, rough out annulus volume, once <1 torr turn on TP3. Close V6. Open V5 to pump the annulus volume with TP3.

5. Re route pumping from RP1 and RP3 to the main volume by opening V3 and slowly opening RV1.

6. After ~3.5 hours the pressure in the arms was <500mtorr on both FRG1 and P1a. Turn on TP1 and wait to reach full speed 560 Hz

7. Open V1 with RV2 barely open. The pressure diff between P1a and P2/FRG2 needs to be below 1 torr. This took a couple attempts with the manual valve in different positions. The interlocks were tripped for this reason. Repeat step 7 until the manual gate valve was in a position that throttled pumping enough to maintain the <1 torr differential.

8. Slowly open the manual gate valve over the course of ~ 1 hour. Once the manual gate valve fully opened, pressure in the arms was <1mtorr.

9. V7 was closed, leaving only TP2 to back TP1, while TP3 was used to continue pumping the annuli. Left in that configuration overnight (see attached)

 

We did have to replace gauge PAN becuase it was reading a signal error. In addition, we found the cable is a bit sketchy and has a sharp bend. The signal comes in and out when the cable is fiddled with.

Attachment 1: PUMPDOWN-2022-05-24_16-57-59.png
PUMPDOWN-2022-05-24_16-57-59.png
Attachment 2: C1VAC_Screenshot_2022-05-24_16-59-27.png
C1VAC_Screenshot_2022-05-24_16-59-27.png
  16900   Wed Jun 8 16:03:12 2022 KojiUpdateVACPower Outage 220608: Vacuum Recovery

=== Observation ===

  • TP1/2/3 failed. AUX RP did not survive the power outage (of course)

=== Initial Recovery of TP2/TP3 ===

  • Stopped TP1/2/3
  • Restored RP AUX at the wall breaker module
  • Restarted TP2/3

=== Rough pumping down ===

  • Found RP1/RP3 left connected (unusual)
  • Turned on RP1 and RP3
  • Opened V6 and V7 for roughing of the pump spool. Now 0.3 torr for P2/P3/PA* (annuli) - ready to open V4 and V5 to backup TP1 by TP2 and TP3. Waiting for TP2/3 to be accelerated

=== Towards main volume pumping ===

  • TP2 accelerated and opened V4 (This caused V6 to close). RP1 and RP3 stopped.
  • TP3 hasitated to spin up. Stopped for now.
  • TP1 needed to be reset to clear the displacement errors. Once the errors were cleared, TP1 was restarted (acclerating).
  • TP1 at normal speed. Opened V1 (main volume pumping started)
     
  • Disconnected RP1/RP3 line.
     
  • TP3 was restarted and is now stable at 55k RPM (indicated as soft spin / low speed). Pushed "Stand By On" and "then "Stand By OFF". This makes the TP3 back to "NORMAL". Switched to "Stand By On".
  • TP2 was also switched to "Stand By On".

Now the main volume and the annuli are pumped down TP1 and TP2/TP3 with RP AUX backing.

The attachment is the pressure glitch for the main volume.

 

Attachment 1: Screenshot_2022-06-08_16-43-13.png
Screenshot_2022-06-08_16-43-13.png
  16916   Wed Jun 15 07:26:35 2022 JCUpdateVACBeginning Pumpdown

[Jordan, JC]

Jordan and I went in to retore the Vacuum System back to it's original state before the power loss on June 8, 2022. The process went smoothly as we first closed V7 and opened VM3 (in that order).

The RP1/3 line did not have the KF blank installed. That was added and the RP flex line was capped off.

Quote:

[JC, Jordan, Paco, Chub]

We began with the pumpdown this morning. We started with the annulus volume and proceeded by using the following:

1. Isolate the RGA Volume by closing of valves VM3 and V7.

2. Opened valves VASE, VASV, VABSSCT, VABS, VABSSCO, VAEV, and VAEE, in that order.

3. Open VA6 to allow P3, FRG3, and PAN to equalize.

4. Turn on RP1 and RP3, rough out annulus volume, once <1 torr turn on TP3. Close V6. Open V5 to pump the annulus volume with TP3.

5. Re route pumping from RP1 and RP3 to the main volume by opening V3 and slowly opening RV1.

6. After ~3.5 hours the pressure in the arms was <500mtorr on both FRG1 and P1a. Turn on TP1 and wait to reach full speed 560 Hz

7. Open V1 with RV2 barely open. The pressure diff between P1a and P2/FRG2 needs to be below 1 torr. This took a couple attempts with the manual valve in different positions. The interlocks were tripped for this reason. Repeat step 7 until the manual gate valve was in a position that throttled pumping enough to maintain the <1 torr differential.

8. Slowly open the manual gate valve over the course of ~ 1 hour. Once the manual gate valve fully opened, pressure in the arms was <1mtorr.

9. V7 was closed, leaving only TP2 to back TP1, while TP3 was used to continue pumping the annuli. Left in that configuration overnight (see attached)

 

We did have to replace gauge PAN becuase it was reading a signal error. In addition, we found the cable is a bit sketchy and has a sharp bend. The signal comes in and out when the cable is fiddled with.

 

  16980   Fri Jul 8 14:03:33 2022 JCHowToVACVacuum Preparation for Power Shutdown

[Koji, JC]

Koji and I have prepared the vacuum system for the power outage on Saturday.

  1. Closed V1 to isolate the main volume.
  2. Closed of VASE, VASV, VABSSCI,VABS, VABSSCO, VAEV, and VAEE.
  3. Closed V6, then close VM3 to isolate RGA
  4. Turn off TP1 (You must check the RPMs on the TP1 Turbo Controller Module)
  5. Close V5
  6. Turn off TP3 (There is no way to check the RPMs, so be patient)
  7. Close V4 (System State changes to 'All pneumatic valves are closed)
  8. Turn off TP2 (There is no way to check the RPMs, so be patient)
  9. Close Vacuum Valves (on TP2 and TP3) which connect to the AUX Pump.
  10. Turn of AUX Pump with the breaker switch wall plug.

From here, we shutdown electronics.

  1. Run /sbin/shutdown -h now on c1vac to shut the host down.
  2. Manually turn off power to electronic modules on the rack.
    • GP316a
    • GP316b
    • Vacuum Acromags
    • PTP3
    • PTP2
    • TP1
    • TP2 (Unplugged)
    • TP3 (Unplugged)

 

Attachment 1: Screen_Shot_2022-07-12_at_7.02.14_AM.png
Screen_Shot_2022-07-12_at_7.02.14_AM.png
  16985   Mon Jul 11 15:26:12 2022 JCHowToVACStartup after Power Outage

[Koji, Jc]

Koji and I began starting that Vacuum system up.

  1. Reverse order step 2 of shutting down electronics. Anthing after, turn on manually.
  2. If C1vac does not come back, then restart by holding the reset button.
  3. Open VA6
  4. Open VASE, VASV,VABSSCI, VABS, VABSSCO, VAEV, and VAEE
  5. Open V7
  6. Check P3 and P2, if they are at high pressure, approx. 1 Torr range, then you must use the roughing pumps.
  7. Connect Rotary pump tube. (Manually)
  8. Turn on AUX Pump
  9. Manually open TP2 and TP3 valves.
  10. Turn on TP2 and TP3, when the pumps finish startup, turn off Standby to bring to nominal speed.
  11. Turn on RP1 and RP3
  12. Open V6
  13. Once P3 reaches <<1 Torr, close V6 to isolate the Roughing pumps.
  14. When TP2 and TP3 are at nominal speed, open V5 and V4.
  15. Now TP1 is well backed, turn on TP1.
  16. When TP1 is at nominal speed, Open V1.
  16986   Mon Jul 11 17:25:43 2022 TegaUpdateVACfixed obsolete reference bug in serial_XGS600 service

Koji noticed that the FRG sensors were not updating due to reference to an obsolete modbusIOC_XGS service, which was used temporarily to test the operation of the serial XGS sensor readout to EPICS. The information in this service was later moved into modbusIOC.service but the dependence on the modbusIOC_XGS.service was not removed from the serial_XGS600.service. This did not present any issue before the shutdown, probably bcos the obsolete service was already loaded but after the restart of c1vac, the obsolete service file modbusIOC_XGS.service was no longer available. This resulted in  serial_XGS600.service throwing a failure to load error for the missing obsolete modbusIOC_XGS service. The fix involved replacing two references to 'modbusIOC_XGS' with 'modbusIOC' in  /opt/target/services/serial_XGS600.service.

I also noticed that the date logged in the commit message was Oct 2010 and that I could not do a push from c1vac due to an error in resolving git.ligo.org. I was able to push the commit from my laptop git repo but was unable to do a pull on c1vac to keep it synced with the remote repo.

  16987   Mon Jul 11 17:41:52 2022 KojiHowToVACStartup after Power Outage

- Once the FRG gauge readings are back (see next elog by Tega), I could open V1 to pump down the main vacuum manifold.
- TP2/TP3 were brought back to stand-by mode (slower spinning)
- V7 was closed to separate the annuli side and TP1

During the vacuum recovery, I saw TPs were automatically turned on as soon as the backing pumps were engaged. I could not figure out what caused this automation.

Also, I saw some gate valve states changed while I was not touching them. e.g. V7 was close / VM3 was open / etc
I really had no idea what/who was handling these.

As of ~18:00 local, the main volume pressure is ~2e-5 torr and ready to open the PSL shutter.

Attachment 1: Screen_Shot_2022-07-11_at_18.13.00.png
Screen_Shot_2022-07-11_at_18.13.00.png
  17154   Fri Sep 23 10:05:38 2022 JCUpdateVACN2 Interlocks triggered

[Chub, Anchal, Tega, JC]

After replacing an empty tank this morning, I heard a hissing sound coming from the nozzle area. It turns out that this was from the copper tubing. The tubing we slightly broken and this was comfirmed with soapy water bubbles. This caused the N2 pressure to drop and the Vac interlocks to be triggered. Chub and I went ahead and replaced the fitting in connected this back to normal. Anchal and Tega have used the Vacuum StartUp procedures to restore the vacuum to normal operation.

Adding screenshot as the pressure is decreasing now.

Attachment 1: Screenshot_2022-09-23_10-19-39.png
Screenshot_2022-09-23_10-19-39.png
Attachment 2: Screenshot_2022-09-23_10-22-45.png
Screenshot_2022-09-23_10-22-45.png
  17283   Fri Nov 18 09:00:27 2022 JCUpdateVACPressure Gauge Information

I bought the spare Full-Range Pirani Gauge a while ago and realized that I never logged this. The Pirani Gauges we are using is described below.

QTY Product Description Serial No.
1 FRG702CF35 -702 FULL RANGE PIRANI/IMG GA.,2.75CF

LI2218F003

I purchased this gauge from Agilent through TechMart. The spare is located inside the Vac Equipment cabinet (The only brown cabinet.) along the X-Arm.

  17307   Wed Nov 23 17:21:44 2022 JCUpdateVACInterlocks may have been tripped due to N2 pressure loss

[Paco, JC]

While changing out one of the N2 tanks today, one of the fitting stripped. This caused a major loss of pressure. I replaced one fitting then realized there was a second leak around the area of the gauge. Paco and I changed this and everything should be back up and running. Thhe interlocks may have been tripped  within the last 2 hours.

 

Attachment 1: Screen_Shot_2022-11-23_at_5.22.57_PM.png
Screen_Shot_2022-11-23_at_5.22.57_PM.png
Attachment 2: Screen_Shot_2022-11-23_at_5.23.09_PM.png
Screen_Shot_2022-11-23_at_5.23.09_PM.png
  17312   Fri Nov 25 12:15:46 2022 AnchalUpdateVACVacuum Gate valves restored

I came today to find that PSL shutter was closed. I orginially thought some shimmer obersvations are underway in the quiet state. But that was not the case. When I tried to open the shutter, it closed back again indicating a hard compliance condition making it close. This normally happens when vacuum level is not sufficient, so I opened the vacuum screena dn indeed all gate valves were closed. This most probably happend during this interlock trip. So the main volume was just slowly leaking and reached to milli torr level today.

Lesson for future: Always check vacuum status when interlock trips.


[Paco, Anchal]

Paco came by to help. We went to asia (the Asus laptop at vacuum workstation) but could not open the medm or find the nfs mounted files. The chiara change did something and nfs mounted directories are not available on asia of c1vac. We rebooted asia and the nfs mount was working again. We can't simply restart c1vac because it runs acromag channels for vacuum system and needs to be done more carefully, a task for Monday.

After restarting asia, we opened the the vacuum control medm screen and followed the vaccum pump down instructions (mainly opening of the gate vales as the pumps were already on). Point to keep in mind, rule of thumb, do not open valve between a turbo pump and a volume if the pressure differential is more than 3 orders of magnitude. Saving turbo pumps is the priority.. Now the main volume is pumping down.

Attachment 1: Screenshot_2022-11-25_12-29-23.png
Screenshot_2022-11-25_12-29-23.png
  4274   Fri Feb 11 16:43:09 2011 steveUpdateVIDEOMC1 & 3 video monitor

I set up video monitoring of MC1 and MC3

Attachment 1: P1070415.JPG
P1070415.JPG
  4314   Thu Feb 17 13:20:06 2011 SureshUpdateVIDEOSome more labels

[Larisa, Kiwamu, Steve and Suresh]

 

  We continued the labeling of video cables. All exiting cables which are going to be used used in the new scheme have been labeled.

We also labeled the cables running from the video mux to the TV monitors in the computer room. Some of these will be removed or reallocated.

We will continue next Wednesday (after the meeting) and will lay cables that are most urgently required. 

 i

  4346   Wed Feb 23 16:56:17 2011 Larisa ThorneUpdateVIDEOCable laying...continued

Having finished labeling the existing cables to match their new names, we (Steve, Kiwamu and Larisa) moved on to start laying new cables and labeling them according to the list.

 

Newly laid cables include: ETMXT (235'), ETMX (235'), POP (110') and MC2 (105').  All were checked by connecting a camera to a monitor and checking the clarity of the resulting image. Note that these cables were only laid, so they are not plugged in.

 

The MC2 cable needs to be ~10' longer; it won't reach to where it's supposed to. It is currently still in its place. 

The three other cables were all a lot longer than necessary.

  4390   Wed Mar 9 16:07:42 2011 kiwamuUpdateVIDEOcable session

[Koji, Steve, Suresh, Kiwamu]

The following video cables have been newly laid down :

  - MC1F/MC3F (65 ft.)

  - PMCR (100 ft.)

  - PSL spare (100 ft.)

  - PSL1  (100 ft.)

  - PSL2  (100 ft.)

 

  4462   Wed Mar 30 17:01:08 2011 Larisa ThorneUpdateVIDEOCable laying...continued

[Steve, Suresh, Kiwamu, Larisa]

 

Only the PRM/BS cable was laid today.

In one of the previous updates on cable laying, it was noted that the MC2 cable needed an additional 10' and the MC2T needed an additional 15' to reach their destinations.  We cut and put BNC ends on 10' and 15' cables and connected them to the original cables in order to make them long enough.

 

This concludes the laying of new cables. Suresh is currently working on the QUADs...

  4474   Thu Mar 31 08:31:44 2011 SureshUpdateVIDEOCable laying...continued

The video work has crossed a milestone.    

Kiwamu and Steve have shifted the three quads from the control room to the Video MUX rack (1Y1) and have wired them to the MUX.

The monitors in the control room have been repositioned and renumbered.  They are now connected directly to the MUX. 

Please see the new cable list for the input and output channels on the MUX.

As of today, all cables according the new plan are in place.  Their status   indicated on the wiki page above is not verified .  Please ignore that column for now, we will be updating that soon.

I shifted the MC1F/MC3F camera and the MC2F cameras onto the new cables.  Also connected the monitors at the BS chamber and end of the X arm to their respective cables.  I have removed the RG58B BNC (black) cables running from MC2 to BS and from ETMXF to the top of the Flow Bench. 

Some of the old video cables are still in place but are not used.   We might consider removing them to clear up the clutter. 

Some of the video cables in use are orange and if the lab's  cable color code is to be enforced these will have to be replaced with blue ones..

Some of the cables in use running from the MUX to the monitor in the control room are the white 50 Ohm variety.  There are also black RG59 Cables running the same way ( we have surplus cables in that path)  and we have to use those instead of the white ones. 

There are a number of tasks remaining:

a)  The inputs from the various existing cameras have to be verified. 

b) There are quite a few cameras which are yet to be installed.

c) The Outputs may not not be connected to their monitors.  That the monitors may still be connected to an old cable which is not connected to the MUX.  The new cable should be lying around close by.  So if you see a blank monitor please connect it to its new cable. 

d) The status column on the wiki page has to be updated.

e) Some of the currently in place may need to be replaced and some need to be removed.  We need to discuss our priorities and come up with a plan for that.

After checking everything we can certify that the video cabling system is complete.

I would like Joon Ho to take care of this verification+documenting process and declaring that the job is complete. 

 

Steve attached these two pictures.

Attachment 1: P1070489.JPG
P1070489.JPG
Attachment 2: P1070494.JPG
P1070494.JPG
  4498   Thu Apr 7 13:12:23 2011 KojiHowToVIDEOVideo switching tip

Long time ago, I looked at the manual of the video switcher.
http://media.extron.com/download/files/userman/Plus_Ultra_MAV_C.pdf
Here is the summary. This will be the basic of the more sophisticated switching program which may have GUI.

In principle, you can manually control the matrix via telnet. At the console machines, you can connect to the matrix using telnet

telnet 192.168.113.92

This opens TCP/IP port 23 of the specified machine. You will receive some messages.
Then type some command like:
--------------------

  • 1*2!       (connect input#1 to output#2)
  • 1,           (save the current setting into preset1)
  • 1.           (restore the setting from preset1)

--------------------

Basicaly that's all. There are many other features but I don't think we need them.

We can create a simple program with any of the language as any of the language has the capability of the TCP/IP connection.
e.g. C, Perl, Python. Tcl/Tk
Any of them are fine.

Now what we have to think about is how to implement the interface in the epics screen (or whatever).
It needs some investigation how the people is thinking as the ideal interface.
But, first of all, you should make the above three operations available as a simple UNIX command like:

videoswitch -i 192.168.113.92 1 2
videoswitch -i 192.168.113.92 -store 1
videoswitch -i 192.168.113.92 -recall 1
(There is no such command yet. These are showing what it should be!)

This can be done by a single day work and our life will be much better.

  4529   Fri Apr 15 02:30:24 2011 KojiHowToVIDEOVideo switching tip

I have made a small python script to handle the video matrix.

It is too far from the perfection, but I release it as it is already useful in some extent.

The script is in the /cvs/cds/rtcds/caltech/c1/scripts/general directory.

usage:

videoswitch.py in_ch_name out_ch_name

in_ch_name is one of the followings

MC2F, IFOPO, OMCR, FI, AS_Spare, ITMYF, ITMXF, ETMYF, ETMXF,
PMCR, RCR, RCT, PSL_Spare, PMCT, ETMXT, MC2T, POP, IMCR, REFL,
MC1F, SRMF, AS, ETMYT, PRM, OMCT, Quad1, Quad2, Quad3

out_ch_name is one of the followings

Mon1, Mon2, Mon3, Mon4, Mon5, Mon6, Mon7,
ETMY, MC1, PSL1, PSL2, ETMX, MC2, CRT9,CRT10,Projector,
Quad1_1, Quad1_2, Quad1_3, Quad1_4,
Quad2_1, Quad2_2, Quad2_3, Quad2_4,

Quad3_1, Quad3_2, Quad3_3, Quad3_4

  4546   Wed Apr 20 20:42:55 2011 kiwamuUpdateVIDEOtoday's video session

[Steve / Suresh / Kiwamu]

90 % of unused video cables have been removed.

Still a couple of video cables are floating around the video MUX. They will be removed in the next week's session.

DSC_2931_ss.jpg

DSC_2938_ss.jpg

 

  5265   Thu Aug 18 22:24:08 2011 jamieOmnistructureVIDEOUpdated 'videoswitch' script

I have updated the 'videoswitch' program that controls the video MUX.  It now includes the ability to query the video mux for the channel mapping:

controls@pianosa:~ 0$ /opt/rtcds/caltech/c1/scripts/general/videoswitch -h
Usage:
videoswitch [options] [OUT]      List current output/input mapping [for OUT]
videoswitch [options] OUT IN     Set output OUT to be input IN

Options:
  -h, --help            show this help message and exit
  -i, --inputs          List input channels and exit
  -o, --outputs         List output channels and exit
  -l, --list            List all input and output channels and exit
  -H HOST, --host=HOST  IP address/Host name
controls@pianosa:~ 0$

  6991   Thu Jul 19 02:14:37 2012 JenneUpdateVIDEOMade a video gui

I learned a little bit of python scripting while looking at the videoswitch script, and I made a video medm screen. 

Each monitor has a dropdown menu for all the common cameras we use (medm only lets you put a limited # of lines on a dropdown menu...when we want to add things like OMCR or RCT, we'll need to add another dropdown set)

Each monitor also has a readback to tell you what is on the TV.  So far, the quads only say "QUAD#", not what the 4 components are. 

I put a set of epics inputs in the PEM model, under a subsystem with top-names VID to represent the different monitors.  The readbacks on the video screen look at these, with the names corresponding to the numbers listed in the videoswitch script.  The videoswitch script now does an ezcawrite to these epics inputs so that even if you change the monitors via command line, the screen stays updated.

For example, since MC2F's camera is plugged in to Input #1 of the video matrix, if you type "./videoswitch MON1 MC2F", the script will write a "1" to the channel "C1:VID-MON1", and the screen will show "MC2F" in the Mon1 cartoon.

This required a quick recompile of the PEM model, but not the framebuilder since these were just epics channels.

There is also a dropdown menu for "Presets", which right now only include my 2 arm locking settings.

All of the dropdowns just call an iteration of the videoswitch script.

  7020   Tue Jul 24 18:33:12 2012 YaakovUpdateVIDEOCentering the MCR camera

Jenne and I centered the MCR camera on the AP table.

We moved the camera as far as it could go on its mount without shifting its screws, and adjusted the optic that the camera looks at for the rest of the way.

  7026   Wed Jul 25 11:41:00 2012 YaakovUpdateVIDEOCentering the MCR camera

Quote:

Jenne and I centered the MCR camera on the AP table.

We moved the camera as far as it could go on its mount without shifting its screws, and adjusted the optic that the camera looks at for the rest of the way.

 The spot was still off center on the quad display in the control room, so I re-recentered it today.

  135   Wed Nov 28 19:02:41 2007 AndreyBureaucracyWIKI-40M UpdateNew WIKI-40M page describing Matlab Suspension Modeling

I created the WIKI-40m page with some details about my today's talk on the 40-m lab meeting.

The address is:

http://lhocds.ligo-wa.caltech.edu:8000/40m/Modeling_of_suspensions

(or you can go to the main page, http://lhocds.ligo-wa.caltech.edu:8000/40m/ , and click on the link "Modeling of suspensions").

The WIKI-40m page describes my transfer functions and contains the pdf-file of my presentation.
  219   Wed Jan 2 11:38:56 2008 tobinSummaryWIKI-40M Updatewiki spam
Our wiki keeps getting spammed. It turns out that you can upload attachments without a password, which is a bug. I'll ask David Barker whether there's a fix for this.
  1941   Tue Aug 25 03:30:23 2009 YoichiSummaryWIKI-40M UpdateGreen lock and phase noise
While Koji and I were discussing about the green laser lock, we wondered if the common motion of the cavity mirrors,
which won't be suppressed by the green laser servo, will cause any problem to the locking.

Since the common motion of the cavity mirrors is equivalent to the change of the path length from the laser to the
input mirror, it will show up as a phase noise in the error signal.
Unfortunately, since we inject the green laser from the end mirror, this phase noise has opposite sign for the
PSL and the green laser.

I calculated the magnitude of the phase noise using an extremely rough estimate of the common motion of the mirrors.
It is explained in the 40m wiki.
http://lhocds.ligo-wa.caltech.edu:8000/40m/Upgrade_09/GreenLock

The result plot is attached.
(Probably the seismic noise I used is an over estimate.)
Attachment 1: PhaseNoise.png
PhaseNoise.png
  2127   Wed Oct 21 11:41:29 2009 ZachUpdateWIKI-40M UpdatePSL Table Diagram wiki entry

 I made a wiki entry for the PSL table diagram under the PSL directory on the 40mHomePage. I tried to use the ImageLink macro to use a resized (smaller) version of the diagram as a link to the full image, which it is designed to do if there is no target given, but it didn't seem to work. Instead, I had to create a second page that had the full-sized diagram, and I used ImageLink with a smaller version to link to that page.

The inventory that is shown is clearly incomplete. Part of this is due to the fact that many labels were either missing or impossible to read without touching stuff. For those components with labels missing, I tried to infer what they were to the best of my knowledge, but I wasn't able to for all of them. In true wiki spirit, everyone is encouraged to fill in any additional information they might have on these components. 

ELOG V3.1.3-