40m QIL Cryo_Lab CTN SUS_Lab TCS_Lab OMC_Lab CRIME_Lab FEA ENG_Labs OptContFac Mariner WBEEShop
  40m Log, Page 326 of 339  Not logged in ELOG logo
ID Date Author Type Categoryup Subject
  13702   Mon Mar 26 09:25:18 2018 SteveUpdateVACVM1 opened

CC1 old MKS cold cathode gauge randomly turns on- off. This makes software interlock close VM1 to protect RGA  So the closed off RGA region pressure goes up and the result is distorted RGA scan.

CC1 MKS gauge is disconnected and VM1 opened. This reminds me that we should connect our interlocks to CC1 Hornet Pressure gauge.

Quote:

Pumpdown 80 at 511 days and pd80b at 218 days

Valve configuration:  special vacuum normal, annuloses are not pumped at 3 Torr, IFO pressure 7.4e-6 Torr at vac envelope temp 22 +- 1C degrres

 

 

Attachment 1: CC4VM1.png
CC4VM1.png
  13709   Tue Mar 27 08:58:21 2018 SteveUpdateVACVM1 opened.......scan fine

 

Quote:

CC1 old MKS cold cathode gauge randomly turns on- off. This makes software interlock close VM1 to protect RGA  So the closed off RGA region pressure goes up and the result is distorted RGA scan.

CC1 MKS gauge is disconnected and VM1 opened. This reminds me that we should connect our interlocks to CC1 Hornet Pressure gauge.

Quote:

Pumpdown 80 at 511 days and pd80b at 218 days

Valve configuration:  special vacuum normal, annuloses are not pumped at 3 Torr, IFO pressure 7.4e-6 Torr at vac envelope temp 22 +- 1C degrres

 

 

 

Attachment 1: rga2018march27.png
rga2018march27.png
  13918   Wed Jun 6 10:02:52 2018 SteveUpdateVACRGA scan
Attachment 1: RGA302d.png
RGA302d.png
Attachment 2: annuloses_NOT_pumped.png
annuloses_NOT_pumped.png
Attachment 3: temp_vac.png
temp_vac.png
  13919   Wed Jun 6 10:44:52 2018 gautamUpdateVACAnnulus pressure channels added to frames

[steve, gautam]

We added the following channels to C0EDCU.ini and restarted the daqd processes. Channels seem to have been added successfully, we will check trend writing later today. Motivation is to have a long term record of annulus pressure (even though we are not currently pumping on the annulus).

C1:Vac-PASE_status

C1:Vac-PASV_status

C1:Vac-PABS_status

C1:Vac-PAEV_status

C1:Vac-PAEE_status

plot next day

Attachment 1: AnsPressureLogged.png
AnsPressureLogged.png
  14059   Thu Jul 12 16:18:22 2018 SteveUpdateVACVent preparations

We are getting ready to vent.

 

Attachment 1: before_Vent.png
before_Vent.png
Attachment 2: before_Vent_cond.png
before_Vent_cond.png
  14064   Fri Jul 13 10:54:55 2018 aaronUpdateVACVent 80

[aaron, steve]

Steve gave me a venting tutorial. I'll record this in probably a bit more detail than is strictly necessary, so I can keep track of some of the minor details for future reference.

Here is Steve's checklist:

  • Check that all jam nuts are tightened
  • all viewports are closed
  • op levs are off
  • take a picture of the MEDM screens
  • Check particle counts
  • Check that the cranes work & wiped
  • Check that HV is off

Gautam already did the pre-vent checks, and Steve took a screenshot of the IFO alignment, IMC alignment, master op lev screen, suspension condition, and shutter status to get a reference point. We later added the TT_CONTROL screen. Steve turned off all op levs.

We then went inside to do the mechanical checks

  • N2 cylinders in the 40m antechamber are all full enough (have ~700psi/day of nitrogen)
  • We manually record the particle count
    • this should be <10,000 on the 0.5um particles to be low enough to vent, otherwise we will contaminate the system
    • note: need to multiply the reading on the particle counter by 10 to get the true count
    • the temperature inside the PSL enclosure should be 23-24C +/- 3 degrees
    • We recorded the particle counts at ~830 and ~930, and the 0.5um count was up to ~3000
  • We put a beam stop in front of the laser at the PSL table
  • Checked that all HV supplies are either off or supplying something in air
    • we noticed four HV supplies on 1X1 that were on. Two were accounted for on the PSL table (FSS), and the other two were for C1IOO_ASC but ran along the upper cable rack. We got ahold of Gautam (sorry!) and he told us these go to the TT driver on OMC_SOUTH, where we verified the HV cables are disconnected. We took this to mean these HV supplies are not powering anything, and proceeded without turning these HV off.
    • There are HV supplies which were all either off or supplying something in-air at: 1Y4, 1Y2, OMC N rack, 1X9 (green steering HV)
  • Checked that the crane works--both move up and down
    • vertex crane switch is on the wall at the inner corner of the IFO
    • y arm crane switch is on the N wall at the Y end
    • turn off the cranes at the control strip after verifying they work
  • While walking around checking HV, we checked that the jam nuts and viewports are all closed
    • we replaced one viewport at the x arm that was open for a camera

After completing these checks, we grabbed a nitrogen cylinder and hooked it up to the VV1 filter. Steve gave me a rundown of how the vacuum system works. For my own memory, the oil pumps which provide the first level of roughing backstream below 500mtorr, so we typically turn on the turbo pumps (TP) below that level... just in case there is a calibrated leak to keep the pressure above 350mtorr at the oil pumps. TP2 has broken, so during this vent we'll install a manual valve so we can narrow the aperture that TP1 sees at V1 so we can hand off to the turbo at 500mtorr without overwhelming it. When the turbos have the pressure low enough, we open the mag lev pump. Close V1 if things screw up to protect the IFO. This 6" id manual gatevalve will allow us throttle the load on the small turbo while the maglev is taking over the pumping  The missmatch in pumping speed is 390/70 l/s [ maglev/varian D70 ]  We need to close down the conductive intake of the TP1 with manual gate valve so the 6x smaller turbo does not get overloaded...

We checked CC1, which read 7.2utorr.

Open the medm c0/ce/VacControl_BAK.adl to control the valves.

Steve tells me we are starting from vacuum normal state, but that some things are broken so it doesn't exactly match the state as described. In particular, VA6 is 'moving' because it has been disconnected and permanently closed to avoid pumping on the annulus. During this v ent, we will also keep pumping on the RGA since it is a short vent; steve logged the RGA yesterday.

We began the vent by following the vacuum normal to chamber open procedure.

  1. VM1 closed
  2. We didn't open VM3, because we want to keep the RGA on
  3. Closed V1
  4. Connect the N2 to the VV1 filter
    1. first puged the line with nitrogen
    2. We confirmed visually that V1 is closed
  5. We opened VM2 to pump on the RGA with the mag lev pump.
    1. This is a nonstandard step because we are keeping the RGA pumped down.
    2. The current on TP3 is ~0.19A, which is a normal, low load on the pump
  6. VV1 opened to begin the vent at ~10:30am
    1. use crescent wrench to open, torque wrench wheel to close
    2. Keep the pressure regulator below 10 psi for the vent. We started the vent with about 2psi, then increased to 8psi after confirming that the SUS sensors looked OK.
  7. We checked the pressure plot and ITMX/ETMX motion to make sure we weren't venting too quickly or kicking the optics
    1. Should look at eg C1:SUS-ITMX_SENSOR_LL, as well as C1:Vac-P1_pressure
  8. Once the pressure reaches 25torr, we switched over to dry air
    1. wipe off the outside dolly wheels with a wet rag, and exit through the x-arm door to get the air. Sweep off the area outside the door, and wipe off new air containers with the rag.
    2. Bring the cylinder inside, get the regulator ready/purged, and swap relatively quickly.
    3. We increased the vent speed to 10psi. 
    4. Steve says the vents typically take 4 of 300 cf cylinders from Airgas "Ultra Zero" AI  UZ300 that contains 0.1 PPM of THC

Everything looks good, so I'm monitoring the vent and swapping out cylinders.

At 12:08pm, the pressure was at 257 torr and I swapped out in a new cylinder.

Steve: Do not overpressurize the vacuum envelope! Stop around 720 Torr and let lab air do the rest. Our bellows are thin walled for seismic isolation.

Attachment 1: vent80wtiptilts.png
vent80wtiptilts.png
  14066   Fri Jul 13 16:26:52 2018 SteveUpdateVACVent 80 is completing...

Steve and Aaron,

6 hrs vent is reaching equlibrium to room air. It took 3 and a half instrument grade air cilynders [ AI UZ300 as labelled ] at 10 psi pressure. Average vent speed ~ 2 Torr/min

Valve configuration: IFO at atm and RGA is pumped through VM2 by TP1 maglev.

 

Attachment 1: @atm.png
@atm.png
Attachment 2: vent80_7h.png
vent80_7h.png
Attachment 3: ventregN2&Air_c.jpg
ventregN2&Air_c.jpg
  14073   Mon Jul 16 15:07:19 2018 KojiSummaryVACOven C vent

[Steve Koji]

- Attachment1: Removed the thermal cap. Checked the temperature of the oven. It was totally cold.

- Attachment2: Confirmed the RGA section was isolated. The pumps for the RGA was left running.

- Attachment3: Closed the main valve. The pumps for the main volume was left running.

- Attachment4: Started removing the rid. This did not change the gause readings as they were isolated from the venting main volume.

- Attachment5: Opened the rid. Took the components out on a UHV foil bag. The rid was replaced but loosely held by a few screws with the old gasket, just to protect the frange and the volume from rough dusts.

Attachment 1: P_20180716_141512.jpg
P_20180716_141512.jpg
Attachment 2: P_20180716_141601.jpg
P_20180716_141601.jpg
Attachment 3: P_20180716_141610.jpg
P_20180716_141610.jpg
Attachment 4: P_20180716_141827.jpg
P_20180716_141827.jpg
Attachment 5: P_20180716_143901.jpg
P_20180716_143901.jpg
  14074   Mon Jul 16 18:12:00 2018 KojiUpdateVACAdding a manual gate valve between TP1 and V1/VM2

[Steve Koji]

We are in the process of adding a manual gate valve between TP1 (Osaka Maglev) and the other gate valves (I suppose V1 and VM2).
The work is still on going and we will continue to work on this tomorrow. Because this section is isolated from the main volume, this work does not hold off the possible rough pumping tomorrow morning.

The motivation of this work is as follows:
- Since TP2 failed, the main vacuum volume has been pumped down by TP1 and TP3. However TP3 is not capable to handle the large pressure difference at the early stage of the turbo pumping. This cause TP3 to have excessive heating or even thermal shutdown.
- The remedy is to put a gate valve between TPs and the main vacuum to limit the amount of gas flowing into the TPs. This indeed slows down the pumping speed of turbo, but this is not the dominant part of the pumping time.

Actual work:
- Comfirmed TP1 is isolated.
- Unscrewed the flange of TP1.
- Remove TP1. This required to lift up TP1 with some shim as the nuts interferes with the TP1 body. (Attachment1, 2, 3)
- Now remove 10inch flange adapter. (Attachment4)
-
Attach 10"-8" adapter and 8" rotational sleeve. (Attachment5)

Attachment 1: P_20180716_155413.jpg
P_20180716_155413.jpg
Attachment 2: P_20180716_155645.jpg
P_20180716_155645.jpg
Attachment 3: P_20180716_155738.jpg
P_20180716_155738.jpg
Attachment 4: P_20180716_162307.jpg
P_20180716_162307.jpg
Attachment 5: P_20180716_172000.jpg
P_20180716_172000.jpg
  14079   Tue Jul 17 18:16:38 2018 SteveSummaryVACpumpdown 81 at 6 hrs

Precondition:  4 days at atm.   Atm5

HEPA tent used during the vent at ETMY  It reduced partical count 10 fold of 0.5 and 0.3 micron particals. Atm6

New items in vacuum:  Clean manual gate valve [Cetec made] from John Worden with 6" id....as it came from Hanford... [ Throttle able gate valve- TGV ] Atm3

                                 ( note: we have 3 more identical in the lab. The original intention was to use them for purging gates )

                                  Optiform Au plated reflector , Induceramics heating elements, similar as existing Cooner cables and related lenses, hardwear. see 14078

                                  OMC related item : none......... 14,110

 

The pumpdown is at 510 mTorr with RP1 & RP3 still pumping. Koji will shut it down the roughing later tonight. Tomorrow morning I will start the pumping by switching over to TP1 maglev.

Thanks for Koji and Gautam'  help of the installation of the manual gate valve. Atm4  This will allow us to control the load on our Varian foreline D70 turbo TP3

 

Attachment 1: pd81@6hrs.png
pd81@6hrs.png
Attachment 2: before_c.jpg
before_c.jpg
Attachment 3: tgv_c.jpg
tgv_c.jpg
Attachment 4: TGVinstalled.jpg
TGVinstalled.jpg
Attachment 5: 4_days_vent.png
4_days_vent.png
Attachment 6: tentHEPA.jpg
tentHEPA.jpg
  14080   Tue Jul 17 22:25:41 2018 KojiSummaryVACpumpdown 81 at 6 hrs

10:20PM

  • Opened VM2 to pump down the RGA section with TP1
  • Stopped rotary roughing pumps
    • Manually closed RV1
    • Closed V3
    • Stopped RP1 and RP3
    • Vented the RP hose

The P1 pressure is 380mTorr. I allowed Gautam to use the full PSL power (~1W).

  14082   Wed Jul 18 12:49:08 2018 SteveSummaryVACpumpdown 81 at 6 +4.5hrs

The manual gate valve scan was clean. Atm1     TP1 was pumping on it overnight.

                                                Pumpdown continued to hand over the pumping to TP1 maglev turbo

V1 was opened at P1 400 mTorr  with manual gate at 3/4 turn open position as Magev at 560 Hz rotation.

Two aux fans on to hold tubo temps TP1 & TP3 . Atm3

This is the first time we pumping down from atm with ONE small "beer can" turbo  and throttled gate valve to control load on small turbo forepump

The 70 l/s turbo is operating at 50k RPM, 0.7 A and 31 C,  pumping speed  ~ 44 mTorr/h at 200-400 mTorr range with aux drypump in the foreline of TP3

Watching foreline pressures and current one can keep opening gate valve little by little the so the load is optimized. It is working but not fast.

Let's keep small turbo at 0.8 Amp and 32 C max at this pumpdown. 

Quote:

10:20PM

  • Opened VM2 to pump down the RGA section with TP1
  • Stopped rotary roughing pumps
    • Manually closed RV1
    • Closed V3
    • Stopped RP1 and RP3
    • Vented the RP hose

The P1 pressure is 380mTorr. I allowed Gautam to use the full PSL power (~1W).

Attachment 1: manlGateScan.png
manlGateScan.png
Attachment 2: handing_over_Mag.png
handing_over_Mag.png
Attachment 3: TGVw2auxfans_.jpg
TGVw2auxfans_.jpg
  14083   Wed Jul 18 17:36:50 2018 SteveSummaryVACpumpdown 81 at 6 +9 hrs completed

IFO P1 6e-4 Torr,  manual gate valve is fully open

The annuloses will be pumped down tomorrow.

Valve configuration: vacuum normal, RGA and annuloses are not pumped

Quote:

The manual gate valve scan was clean. Atm1     TP1 was pumping on it overnight.

                                                Pumpdown continued to hand over the pumping to TP1 maglev turbo

V1 was opened at P1 400 mTorr  with manual gate at 3/4 turn open position as Magev at 560 Hz rotation.

This is the first time we pumping down from atm with one small "beer can" turbo  and throttled gate to control load on small turbo forepump

The 70 l/s turbo is operating at 50k RPM, 0.7 A and 31 C,  pumping speed  ~ 44 mTorr/h at 200-400 mTorr range.

Watching foreline pressures and current one can keep opening gate valve little by little the so the load is optimized. It is working but not fast.

Let's keep small turbo at 0.8 Amp and 32 C max at this pumpdown. 

Quote:

10:20PM

  • Opened VM2 to pump down the RGA section with TP1
  • Stopped rotary roughing pumps
    • Manually closed RV1
    • Closed V3
    • Stopped RP1 and RP3
    • Vented the RP hose

The P1 pressure is 380mTorr. I allowed Gautam to use the full PSL power (~1W).

Attachment 1: pd81completed.png
pd81completed.png
Attachment 2: pd81@30hrs.png
pd81@30hrs.png
  14085   Thu Jul 19 01:56:25 2018 gautamSummaryVACAUX pump shutdown

[koji, gautam]

Per Steve's instructions, we did the following:

  • TP3fl pressure reading was 65 torr.
  • TP3 controller reported pumping current of ~0.18A, temperature of 24C.
  • We throttled the manual valve which was connecting the "AUX" pump to the TP3fl.
  • The TP3fl pressure went up to 330 torr.
  • TP3fl controller reported current of 0.22A, temperature of 24C.
  • After ~5mins, we shut the AUX pump off.
  • We have monitored it over the last 1hour, no red flags.
    • (Before stopping AUX RP)
      0:56AM TP3 I=0.18A, P=6W, 23degC, TP3FL: 66
    • 0:59AM TP3 I=0.22A, P=7W, 23degC, TP3FL: 336
    • 1:15AM TP3 I=0.21A, P=7W, 23degC, TP3FL: 320
    • 1:31AM TP3 I=0.21A, P=7W, 23degC, TP3FL: 310
    • 2:06AM TP3 I=0.21A, P=7W, 23degC, TP3FL: 301
    • 5:06AM TP3 I=0.21A, P=7W, 23degC, TP3FL: 275
  14087   Thu Jul 19 11:01:03 2018 SteveSummaryVACpd81 @ 2e-5 Torr

Cold cathode gauge just turned on.

Attachment 1: pd81@2days.png
pd81@2days.png
  14088   Thu Jul 19 13:35:30 2018 SteveSummaryVACannuloses pumped

Roughing down the annuloses required closing V1 for 13 minutes

IFO is 2.2e-5 Torr

Attachment 1: AnsPumped.png
AnsPumped.png
  14098   Mon Jul 23 09:58:52 2018 SteveSummaryVACRGA scan at day 6

 

 

Attachment 1: pd81-560Hz-d6.png
pd81-560Hz-d6.png
  14137   Mon Aug 6 09:34:02 2018 SteveUpdateVACRGA scan at day 20

 

 

Attachment 1: pd81d20.png
pd81d20.png
  14167   Thu Aug 16 07:50:28 2018 SteveUpdateVACpumpdown 81 at day 30

 

 

Attachment 1: pd81d30.png
pd81d30.png
  14168   Thu Aug 16 14:48:14 2018 SteveUpdateVACwhy do we need a root pump?

Basic Pump Throughput Concepts

What is Pump Throughput?

The manufacturer of a vacuum pump supplies a chart for each pump showing pumping speed (volume in unit time) vs pressure. The example, for a fictitious pump, shows the pumping speed is substantially constant over a large pressure range.

By multiplying pumping speed by pressure at which that pumping speed occurs, we get a measure called pump throughput. We can tabulate those results, as shown in the table below, or plot them as a graph of pressure vs pump throughput. As is clear from the chart,  pump throughput (which might also be called mass flow) decreases proportionally with PRESSURE, at least over the pressure range where pumping speed is constant.

 

Pumping Speed Pressure Pressure x Pumping Speed
100 L/sec 10 torr 1000 torr.liter/sec
100 L/sec 1 torr 100 torr.liter/sec
100 L/sec 0.1 torr 10 torr.liter/sec
100 L/sec 0.01 torr

1 torr.liter/sec

The roughing pump speed actually will reach 0 l/s  at it's ultimate pressure performance.

Our roughing pump  pumping speed will slowly drop  as chamber pressure drops. Below 10 Torr this decrease is accelerated and bottoms out. This where the Root pump can help. See NASA evaluation of dry rough pumps...What is a root pump 

We have been operating succsessfully with a narrow margin. The danger is that the Maglev forline peaks at 4 Torr. This puts load on the small turbo TP2, TP3 &  large TP1

The temperature of these TP2 & 3  70 l/s drag turbos go up to 38 C and their  rotation speed slow to 45K rpm from 50K rpm because of the large volume 33,000 liters

Either high temp or low rotation speed of drag turbo or long time of overloading  can shut down the small turbo pumps......meaning: stop pumping, wait till they cool down

The manual gate valve installed helped to lower peak temp to 32C It just took too long.

We have been running with 2 external fans [one on TP1 & one on TP3]  for cooling and one aux drypump to help lowering the foreline pressure of TP2 & 3

The vacuum control upgrade should include adding root pump into the zero pumping speed range.

 

Atm1,   Pump speed chart:   TP1  turbo -red, root pump -blue and mechanical pump green. Note green color here representing an oily rotory pump. Our small drypumps [SH-100] typically run above 100 mTorr

                                           They are the forepump of TP2 & 3     Our pumpdown procedure: Oily Leybold rotory pumps ( with safety orifice 350 mT to atm ) rough to 500 mTorr

                                                                                                 Here we switch over to TP2 & 3 running at 50k RPM with drypumps SH-100 plus Aux Triscroll

                                                                                                 TP1- Maglev rotating full speed when V1 is opened at full volume at 500 mTorr 

                         History: the original design of the early 1990s had no dry scroll pumps. Oil free dry scrools replaced the oily forepumps of TP2 & TP3 in ~2002  at the cost of degrading the forline pressure somewhat.

                                     We had 2 temperature related Maglev failers in 2005 Aug 8 and 2006 April 5  Osaka advised us to use AUX fan to cool TP1  This helped. 

Atm2,   Wanted Root pump - Leybold EcoDry 65 plus  

Atm3,   Typical 8 hrs pumpdown from 2007 with TP2 & 3 

Atm4,   Last pumpdown zoomed in from 400 mT to 1mT with throttled gate  valve took 9 hrs  The foreline pressure of TP1 peaked at 290 mT, TP3 temperature peaked at 32C

            This technic is workable, but 9 hrs is too long.

Atm5,   The lowest pressure achived in  the 40m Vacuum Envelope 5e-7 Torr with pumps Maglev  ~300 l/s,  Cryo 1500 l/s  and 3 ion pumps of 500 l/s      [ in April 2002 at pumpdown 53 day 7 ] with annuloses at ~ 10 mTorr

Atm6,  Osaka TG390MCAB Throughput with screen ~300 L/s at 12 cfm backing pump

Attachment 1: PUMPSPEED_CHAR.pdf
PUMPSPEED_CHAR.pdf
Attachment 2: Leybold_Broschuere_8Seiten_EN_ANSICHT.pdf
Leybold_Broschuere_8Seiten_EN_ANSICHT.pdf
Attachment 3: pd65.jpg.png
pd65.jpg.png
Attachment 4: pd81completed.png
pd81completed.png
Attachment 5: best_.pdf
best_.pdf
Attachment 6: Osaka390.pdf
Osaka390.pdf
  14183   Fri Aug 24 10:51:23 2018 SteveUpdateVACpumpdown 81 at day 38

 

 

Attachment 1: d38.png
d38.png
  14189   Wed Aug 29 09:56:00 2018 SteveUpdateVACMaglev controller needs service

TP-1 Osaka maglev controller  [  model TCO10M,  ser V3F04J07 ]  needs maintenance. Alarm led  on indicating  that we need Lv2 service.

The turbo and the controller are in good working order.

*****************************

Hi Steve,

Our maintenance level 2 service price is $...... It consists of a complete disassembly of the controller for internal cleaning of all ICB’s, replacement of all main board capacitors, replacement of all internal cooling units, ROM battery replacement, re-assembly, and mandatory final testing to make sure it meets our factory specifications. Turnaround time is approximately 3 weeks.

  RMA 5686 has been assigned to Caltech’s returning TC010M controller. Attached please find our RMA forms. Complete and return them to us via email, along with your PO, prior to shipping the cont

Best regards,

Pedro Gutierrez

Osaka Vacuum USA, Inc.

510-770-0100 x 109

*************************************************

our TP-1 TG390MCAB is 9 years old. What is the life expectancy of this turbo?

                        The Osaka maglev turbopumps are designed with a 100,000 hours(or ~ 10 operating years) life span but as you know most of our end-users are

                        running their Osaka maglev turbopumps in excess of 10+, 15+ years continuously.     The 100,000 hours design value is based upon the AL material being rotated at

                        the given speed.   But the design fudge factor have somehow elongated the practical life span.  

We should have the cost of new maglev & controller in next year budget. I  put the quote into the wiki.

 

                         

 

  14207   Fri Sep 21 16:51:43 2018 gautamUpdateVACc1vac1 is unresponsive

Steve pointed out that some of the vacuum MEDM screen fields were reporting "NO COMM". Koji confirmed that this is a c1vac1 problem, likely the same as reported here and can be fixed using the same procedure.

However, Steve is worried that the interlock won't kick in in case of a vacuum emergency, so we are leaving the PSL shutter closed over the weekend. The problem will be revisited on Monday.

  14215   Mon Sep 24 15:06:10 2018 gautamUpdateVACc1vac1 reboot + TP1 controller replacement

[steve, gautam]

Following the procedure in this elog, we effected a reset of the vacuum slow machines. Usually, I just turn the key on these crates to do a power cycle, but Steve pointed out that for the vacuum machines, we should only push the "reset" button.

While TP1 was spun down, we took the opportunity to replace the TP1 controller with a spare unit the company has sent us for use while our unit is sent to them for maintenance. The procedure was in principle simple (I only list the additional ones, for the various valve closures, see the slow machine reset procedure elog):

  • Turn power off using switch on rear.
  • Remove 4 connecting cables on the back.
  • Switch controllers.
  • Reconnect 4 cables on the back panel.
  • Turn power back on using switch on rear.

However, we were foiled by a Philips screw on the DB37 connector labelled "MAG BRG", which had all its head worn out. We had to make a cut in this screw using a saw blade, and use a "-" screwdriver to get this troublesome screw out. Steve suspects this is a metric gauge screw, and will request the company to send us a new one, we will replace it when re-installing the maintaiend controller. 

Attachments #1 and #2 show the Vacuum MEDM screen before and after the reboot respectively - evidently, the fields that were reading "NO COMM" now read numbers. Attachment #3 shows the main volume pressure during this work.

Quote:

The problem will be revisited on Monday.

Attachment 1: beforeReboot.png
beforeReboot.png
Attachment 2: afterReboot.png
afterReboot.png
Attachment 3: CC1.png
CC1.png
  14217   Wed Sep 26 10:07:16 2018 SteveUpdateVACwhy reboot c1vac1

Precondition: c1vac1 & c1vac2 all LED warning lights green [ atm3 ], the only error message is in the gauge readings NO COMM, dataviewer will plot zero [ atm1 ], valves are operational

When our vacuum gauges read " NO COMM " than our INTERLOCKS  do  NOT communicate either.

So V1 gate valve and PSL output shutter can not be triggered to close if the the IFO pressure goes up.                        

   [ only CC1_HORNET_PRESSURE reading is working in this condition because it goes to a different compuer ] 

Quote:

[steve, gautam]

Following the procedure in this elog, we effected a reset of the vacuum slow machines. Usually, I just turn the key on these crates to do a power cycle, but Steve pointed out that for the vacuum machines, we should only push the "reset" button.

While TP1 was spun down, we took the opportunity to replace the TP1 controller with a spare unit the company has sent us for use while our unit is sent to them for maintenance. The procedure was in principle simple (I only list the additional ones, for the various valve closures, see the slow machine reset procedure elog):

  • Turn power off using switch on rear.
  • Remove 4 connecting cables on the back.
  • Switch controllers.
  • Reconnect 4 cables on the back panel.
  • Turn power back on using switch on rear.

However, we were foiled by a Philips screw on the DB37 connector labelled "MAG BRG", which had all its head worn out. We had to make a cut in this screw using a saw blade, and use a "-" screwdriver to get this troublesome screw out. Steve suspects this is a metric gauge screw, and will request the company to send us a new one, we will replace it when re-installing the maintaiend controller. 

Attachments #1 and #2 show the Vacuum MEDM screen before and after the reboot respectively - evidently, the fields that were reading "NO COMM" now read numbers. Attachment #3 shows the main volume pressure during this work.

Quote:

The problem will be revisited on Monday.

 

Attachment 1: NOcomm.png
NOcomm.png
Attachment 2: Reboot_&_sawp.png
Reboot_&_sawp.png
Attachment 3: c1vac1&2_.jpg
c1vac1&2_.jpg
  14229   Thu Oct 4 08:25:50 2018 SteveUpdateVACrga scan pd81 at day 78

 

 

Attachment 1: pd81d78.png
pd81d78.png
  14244   Fri Oct 12 08:27:05 2018 SteveUpdateVACdrypump

Gautam and Steve,

Our TP3 drypump seal is at 360 mT [0.25A load on small turbo]  after one year.  We tried to swap in old spare drypump with new tip  seal. It was blowing it's fuse, so we could not do it.

Noisy aux drypump turned on and opened to TP3 foreline [ two drypumps are in  the foreline now ]  The pressure is 48 mT and 0.17A load on small turbo.

Attachment 1: forepump.png
forepump.png
  14247   Fri Oct 12 17:37:03 2018 SteveUpdateVACpressure gauge choices

We want to measure the pressure gradient in the 40m IFO

Our old MKS cold cathodes are out of order. The existing working gauge at the pumpspool is InstruTech CCM501

The plan is to purchase 3 new gauges for ETMY, BS and MC2 location.

Basic cold cathode     or    Bayard-Alpert Pirani

    

 

  14256   Mon Oct 15 13:59:42 2018 SteveUpdateVACdrypump replaced

Steve & Bob,

Bob removed the head cover from the housing to inspect the condition of the the tip seal. The tip seal was fine but the viton cover seal had a bad hump. This misaligned the tip seal and it did not allow it to rotate.

It was repositioned an carefully tithened. It worked. It's starting current transiant measured 28 A and operational mode 3.5 A

This load is normal with an old pump. See the brand new DIP7 drypump as spare was 25 A at start and  3.1 A in operational mode. It is amazing how much punishment a slow blow ceramic 10A  fuse can take [ 0215010.HXP ]

In the future one should measure the current pick up [ transient <100ms ] after the the seal change with Fluke 330 Series Current Clamp

 

It was swapped in and the foreline pressure dropped to 24 mTorr after 4 hours. It is very good. TP3 rotational drive current  0.15 A at 50K rpm   24C

Quote:

Gautam and Steve,

Our TP3 drypump seal is at 360 mT [0.25A load on small turbo]  after one year.  We tried to swap in old spare drypump with new tip  seal. It was blowing it's fuse, so we could not do it.

Noisy aux drypump turned on and opened to TP3 foreline [ two drypumps are in  the foreline now ]  The pressure is 48 mT and 0.17A load on small turbo.

 

Attachment 1: drypump_swap.png
drypump_swap.png
  14262   Mon Oct 22 15:19:05 2018 SteveUpdateVACMaglev controller serviced

Gautam & Steve,

Our controller is back with Osaka maintenace completed. We swapped it in this morning.

Quote:

TP-1 Osaka maglev controller  [  model TCO10M,  ser V3F04J07 ]  needs maintenance. Alarm led  on indicating  that we need Lv2 service.

The turbo and the controller are in good working order.

*****************************

Hi Steve,

Our maintenance level 2 service price is $...... It consists of a complete disassembly of the controller for internal cleaning of all ICB’s, replacement of all main board capacitors, replacement of all internal cooling units, ROM battery replacement, re-assembly, and mandatory final testing to make sure it meets our factory specifications. Turnaround time is approximately 3 weeks.

  RMA 5686 has been assigned to Caltech’s returning TC010M controller. Attached please find our RMA forms. Complete and return them to us via email, along with your PO, prior to shipping the cont

Best regards,

Pedro Gutierrez

Osaka Vacuum USA, Inc.

510-770-0100 x 109

*************************************************

our TP-1 TG390MCAB is 9 years old. What is the life expectancy of this turbo?

                        The Osaka maglev turbopumps are designed with a 100,000 hours(or ~ 10 operating years) life span but as you know most of our end-users are

                        running their Osaka maglev turbopumps in excess of 10+, 15+ years continuously.     The 100,000 hours design value is based upon the AL material being rotated at

                        the given speed.   But the design fudge factor have somehow elongated the practical life span.  

We should have the cost of new maglev & controller in next year budget. I  put the quote into the wiki.

 

                         

 

 

Attachment 1: our_controller_is_back.png
our_controller_is_back.png
  14264   Wed Oct 31 17:54:25 2018 gautamUpdateVACCC1 hornet power connection restored

Steve reported to me that the CC1 Hornet gauge was not reporting the IFO pressure after some cable tracing at EX. I found that the power to the unit had been accidentally disconnected. I re-connected the power and manually turned on the HV on the CC gauge (perhaps this can be automated in the new vacuum paradigm). IFO pressure of 8e-6 torr is being reported now.

Attachment 1: cc1_Hornet.png
cc1_Hornet.png
  14279   Tue Nov 6 23:19:06 2018 gautamUpdateVACc1vac1 FAIL lights on (briefly)

Jon and I stuck a extender card into the eurocrate at 1X8 earlier today (~5pm PT), to see if the box was getting +24V DC from the Sorensen or not. Upon sticking the card in, the FAIL LEDs on all the VME cards came on. We immediately removed the extender card. Without any intervention from us, after ~1 minute, the FAIL LEDs went off again. Judging by the main volume pressure (Attachment #1) and the Vacuum MEDM screen (Attachment #2), this did not create any issues and the c1vac1 computer is still responsive.

But Steve can perhaps run a check in the AM to confirm that this activity didn't break anything.

Is there a reason why extender cards shouldn't be stuck into eurocrates?

Attachment 1: Screenshot_from_2018-11-06_23-18-23.png
Screenshot_from_2018-11-06_23-18-23.png
Attachment 2: Screenshot_from_2018-11-06_23-19-26.png
Screenshot_from_2018-11-06_23-19-26.png
  14281   Wed Nov 7 08:32:32 2018 SteveUpdateVACc1vac1 FAIL lights on (briefly)...checked

The vacuum and MC are OK

Quote:

Jon and I stuck a extender card into the eurocrate at 1X8 earlier today (~5pm PT), to see if the box was getting +24V DC from the Sorensen or not. Upon sticking the card in, the FAIL LEDs on all the VME cards came on. We immediately removed the extender card. Without any intervention from us, after ~1 minute, the FAIL LEDs went off again. Judging by the main volume pressure (Attachment #1) and the Vacuum MEDM screen (Attachment #2), this did not create any issues and the c1vac1 computer is still responsive.

But Steve can perhaps run a check in the AM to confirm that this activity didn't break anything.

Is there a reason why extender cards shouldn't be stuck into eurocrates?

 

Attachment 1: Vac_MC_OK.png
Vac_MC_OK.png
  14291   Tue Nov 13 16:15:01 2018 SteveUpdateVACrga scan pd81 at day 119

 

 

Attachment 1: pd81-d119.png
pd81-d119.png
Attachment 2: pd81-560Hz-d119.png
pd81-560Hz-d119.png
  14299   Fri Nov 16 10:26:12 2018 SteveUpdateVAC single viton O-rings

The 40m vacuum envelope has one large single O-ring on the OOC west side. All other doors have double O-ring with annuloses.

There are 3  spacers to protect o-ring. They should not be removed!

 

The Cryo-pump static seal to VC1 also viton. All gate valves and right angle valve plates have single viton o-ring seal.

Small single viton o-rings on all optical quality viewports.

Helium will permiate through these fast. Leak checking time is limited to 5-10 minutes.

All other seals are copper gaskits. We have 2 manual right angle with METAL-dynamic seal [ VATRING ] as  VV1 & RV1

 

Attachment 1: Single-O-ring.png
Single-O-ring.png
  14301   Fri Nov 16 15:09:31 2018 SteveConfigurationVACnot venting cryo and ion pumps

Notes on the ion pumps and cryo pump:

  • Our 4 ion pumps were closed off for a lomg time. I estmated their pressure to be around ~1 Torr. After talking with Koji we decided not to vent them.

  • It'd be still useful to wire their position sensors. But make sure we do not actuate the valves. 

  • The cryo pump was regenerated to 1e-4 Torr about 2 years ago. It's pressure can be ~ 2 Torr with charcoal powder. It is a dirty system at room temperature.

  • Do not actuate VC1 and VC2, and keep its manual valve closed.

  • IF someone feels we should vent them for some reason, let us know here in the elog before Monday morning.

 

Quote:

Wiring of the power, Ethernet, and indicator lights for the vacuum Acromag chassis is complete. Even though this crate will only use +24V DC, I wired the +/-15V connector and indicator lights as well to conform to the LIGO standard. There was no wiring diagram available, so I had to reverse-engineer the wiring from the partially complete c1susaux crate. Attached is a diagram for future use. The crate is ready to begin software developing on Monday. 

 

  14305   Mon Nov 19 14:59:48 2018 ChubUpdateVACVent 81

Vent 80 is nearly complete; the instrument is almost to atmosphere.  All four ion pump gate valves have been disconnected, though the position sensors are still connected,and all annulus valves are open.  The controllers of TP1 and TP3 have been disconnected from AC power. VC1 and VC2 have been disconnected and must remained closed. Currently, the RGA is being vented through the needle valve and the RGA had been shut off at the beginning of the vent preparations.  VM1 and VM3 could not be actuated.  The condition status is still listed as Unidentified because of the disconnected valves. 

  14306   Mon Nov 19 17:09:00 2018 SteveUpdateVACVent 81

Gautam, Aaron, Chub and Steve,

Quote:

Vent 80 is nearly complete; the instrument is almost to atmosphere.  All four ion pump gate valves have been disconnected, though the position sensors are still connected,and all annulus valves are open.  The controllers of TP1 and TP3 have been disconnected from AC power. VC1 and VC2 have been disconnected and must remained closed. Currently, the RGA is being vented through the needle valve and the RGA had been shut off at the beginning of the vent preparations.  VM1 and VM3 could not be actuated.  The condition status is still listed as Unidentified because of the disconnected valves. 

The vent 81 is completed.

4 ion pumps and cryo pump are at ~ 1-4 Torr (estimated as we have no gauges there), all other parts of the vacuum envelope are at atm. P2 & P3 gauges are out of order.

V1 and VM1 are in a locked state. We suspect this is because of some interlock logic.

TP1 and TP3 controllers are turned off.

Valve conditions as  shown: ready to be opened or closed or moved or rewired. To re-iterate: VC1, VC2, and the Ion Pump valves shouldn't be re-connected during the vac upgrade.

Thanks for all of your help.

Attachment 1: beforeVent82.png
beforeVent82.png
Attachment 2: vent81completed.png
vent81completed.png
  14307   Mon Nov 19 22:01:50 2018 gautamUpdateVACLoose nut on valve

As I was turning off the lights in the VEA, I heard a rattling sound from near the PSL enclosure. I followed it to a valve - I couldn't see a label on this valve in my brief effort to find one, but it is on the south-west corner of the IMC table, so maybe VABSSCI or VABSSCO? The power cable is somehow spliced with an attachment that looks to be bringing gas in/out of the valve (See Attachment #1), and the nut on the bottom was loose, the whole power cable + mettal attachment was responsible for the rattling. I finger-tightened the nut and the sound went away.

Attachment 1: IMG_7171.JPG
IMG_7171.JPG
  14310   Tue Nov 20 13:13:01 2018 gautamUpdateVACIMC alignment is okay

I checked the IMC alignment following the vent, for which the manual beam block placed on the PSL table was removed. The alignment is okay, after minor touchup, the MC Trans was ~1200 cts which is roughly what it was pre-vent. I've closed the PSL shutter again.

  14318   Mon Nov 26 15:58:48 2018 SteveUpdateVACVent 81

Gautam, Aaron, Chub & Steve,

ETMY heavy door replaced by light one.

We did the following:  measured 950 particles/cf min of 0.5 micron at SP table, wiped crane and it's cable, wiped chamber,

                                placed heavy door on clean merostate covered stand, dry wiped o-rings and isopropanol wiped Aluminum light cover

                              

Quote:

Gautam, Aaron, Chub and Steve,

Quote:

Vent 80 is nearly complete; the instrument is almost to atmosphere.  All four ion pump gate valves have been disconnected, though the position sensors are still connected,and all annulus valves are open.  The controllers of TP1 and TP3 have been disconnected from AC power. VC1 and VC2 have been disconnected and must remained closed. Currently, the RGA is being vented through the needle valve and the RGA had been shut off at the beginning of the vent preparations.  VM1 and VM3 could not be actuated.  The condition status is still listed as Unidentified because of the disconnected valves. 

The vent 81 is completed.

4 ion pumps and cryo pump are at ~ 1-4 Torr (estimated as we have no gauges there), all other parts of the vacuum envelope are at atm. P2 & P3 gauges are out of order.

V1 and VM1 are in a locked state. We suspect this is because of some interlock logic.

TP1 and TP3 controllers are turned off.

Valve conditions as  shown: ready to be opened or closed or moved or rewired. To re-iterate: VC1, VC2, and the Ion Pump valves shouldn't be re-connected during the vac upgrade.

Thanks for all of your help.

 

  14322   Tue Nov 27 17:06:51 2018 SteveConfigurationVACAgilent 84FS turbo installed as TP2

Chub & Steve,

We swapped in our  replacement of Varian V70D "bear-can" turbo as factory clean.

The new Agilent TwisTorr 84 FS  turbo pump [ model x3502-64002,  sn IT17346059 ]  with intake screen, fan, vent valve. The controller  [ model 3508-64001, sn IT1737C383 ] and a larger drypump IDP-7,  [ model x3807-64010, sn MY17170019 ] was installed.

Next things to do:

  1. implement hardware interlock to close V4 at 80% pumping speed slowdown of "standby" rotation speed, estimated to be ~ 40,000 RPM ( when Standby 50K RPM  )
  2. set up isolation valve in the foreline of TP2, with delayed start of the IDP-7 and/or use relay to power drypump.  This turbo controller can not switch off or start of the dry pump. [ Agilent isolation valve #X3202-60055, with position indicator, pneumatic actuation, 115V solenoid ]..........as a second thought, we do not need isolation valve if we go with the relay option. The IDP-7 has built in delay of 10-15 sec
  3. test performance of new turbo
  14367   Wed Dec 19 14:19:15 2018 KojiSummaryVACPlan for pumpoing down test

We still need elaborated test procedure posted

12/29 Wed

  • Jon continues to work on valve actuator tests.
  • Chub continues to work on wiring / fixing wiring.
  • At the end of the day Jon is going to send out a notification email of "GO"/"NO GO" for pumping.

 

12/30 Thu

  • 9AM: Start closing two doors unless Jon gives us NO GO sign.
  • 10AM: Start pumping down
    • Test roughing pump capability via new control system
    • (Independently) Test turbo rotating procedure. This time we will not open the gate valve between the TP1 and the main volume. This is because we want to take care of the backing turbo loads while we gradually open the gate valve. This will take more hours to be done and we will not be able to finish this test by the end of Thu.
    • At the end of the procedure, we isolate the main volume, stop all the pumps, and vent the roghing pumps to save them from the oil backstream.

gautam: Koji and I were just staring at the vacuum screen, and realized that the drypumps, which are the backing pumps for TP2 and TP3, are not reflected on the MEDM screen. This should be rectified.

Steve also mentioned that the new small turbo controller does not directly interface with the drypump. So we need some system to delay the starting of the turbo itself, once the drypump has been engaged. Does this system exist?

Attachment 1: Screenshot_from_2018-12-19_14-49-34.png
Screenshot_from_2018-12-19_14-49-34.png
  14370   Wed Dec 19 21:14:50 2018 gautamUpdateVACPumpdown tomorrow

I just spoke to Jon who asked me to make this elog - we will be ready to test one or more parts of the pumpdown procedure tomorrow (12/20), so we should proceed as planned to put the heavy doors back on EY and OMC chambers at 9am tomorrow morning. Jon will circulate a more detailed procedure about the pumpdown steps later today evening.

  14372   Thu Dec 20 08:38:27 2018 JonUpdateVACPumpdown tomorrow

Linked is the pumpdown procedure, contained in the old 40m documentation. The relevant procedure is "All Off --> Vacuum Normal" on page 11.

Quote:

I just spoke to Jon who asked me to make this elog - we will be ready to test one or more parts of the pumpdown procedure tomorrow (12/20), so we should proceed as planned to put the heavy doors back on EY and OMC chambers at 9am tomorrow morning. Jon will circulate a more detailed procedure about the pumpdown steps later today evening.

 

  14373   Thu Dec 20 10:28:43 2018 gautamUpdateVACHeavy doors back on for pumpdown 82

[Chub, Koji, Gautam]

We replaced the EY and IOO chamber heavy doors by 10:10 am PST. Torquing was done first oen round at 25 ft-lb, next at 45 ft-lb (we trust the calibration on the torque wrench, but how reliable is this? And how important are these numbers in ensuring a smooth pumpdown?). All went smooth. The interior of the IOO chamber was found to be dirty when Koji ran a wipe along some surfaces.

For this pumpdown, we aren't so concerned with having the IFO in an operating state as we will certainly vent it again early next year. So we didn't follow the full close-up checklist.

Jon and Chub and Koji are working on starting the pumpdown now... In order to not have to wear laser safety goggles while we closed doors and pumped down, I turned off all the 1064nm lasers in the lab.

  14377   Fri Dec 21 11:13:13 2018 gautamOmnistructureVACN2 line valved off

Per the discussion yesterday, I valved off the N2 line in the drill press room at 11 am PST today morning so as to avoid any accidental software induced gate-valve actuation during the holidays. The line pressure is steadily dropping...

Attachment #1 shows that while the main volume pressure was stable overnight, the the pumpspool pressure has been steadily rising. I think this is to be expected as the turbo pumps aren't running and the valves can't preserve the <1mtorr pressure over long timescales?

Attachment #2 shows the current VacOverview MEDM screen status.

Attachment 1: VacGauges.png
VacGauges.png
Attachment 2: Screenshot_from_2018-12-21_13-02-06.png
Screenshot_from_2018-12-21_13-02-06.png
  14379   Fri Dec 21 12:57:10 2018 KojiOmnistructureVACN2 line valved off

Independent question: Are all the turbo forelines vented automatically? We manually did it for the main roughing line.

 

  14380   Thu Jan 3 15:08:37 2019 gautamOmnistructureVACVac status unknown

Larry W came by the 40m, and reported that there was a campus-wide power glitch (he was here to check if our networking infrastructure was affected). I thought I'd check the status of the vacuum.

  • Attachment #1 is a screenshot of the Vac overview MEDM screen. Clearly something has gone wrong with the modbus process(es). Only the PTP2 and PTP3 gauges seem to be communicative.
  • Attachment #2 shows the minute trend of the pressure gauges for a 12 day period - it looks like there is some issue with the frame builder clock, perhaps this issue resurfaced? But checking the system time on FB doesn't suggest anything is wrong.. I double checked with dataviewer as well that the trends don't exist... But checking the status of the individual daqd processes indeed showed that the dates were off by 1 year, so I just restarted all of them and now the time seems correct. How can we fix this problem more permanently? Also, the P1b readout looks suspicious - why are there periods where it seems like we are reading values better than the LSB of the device?

I decided to check the systemctl process status on c1vac:

controls@c1vac:~$ sudo systemctl status modbusIOC.service
● modbusIOC.service - ModbusIOC Service via procServ
   Loaded: loaded (/etc/systemd/system/modbusIOC.service; enabled)
   Active: active (running) since Thu 2019-01-03 14:53:49 PST; 11min ago
 Main PID: 16533 (procServ)
   CGroup: /system.slice/modbusIOC.service
           ├─16533 /usr/bin/procServ -f -L /opt/target/modbusIOC.log -p /run/...
           ├─16534 /opt/epics/modules/modbus/bin/linux-x86_64/modbusApp /opt/...
           └─16582 caRepeater

Jan 03 14:53:49 c1vac systemd[1]: Started ModbusIOC Service via procServ.

Warning: Unit file changed on disk, 'systemctl daemon-reload' recommended.

So something did happen today that required restart of the modbus processes. But clearly not everything has come back up gracefully. A few lines of dmesg (there are many more segfaults):

[1706033.718061] python[23971]: segfault at 8 ip 000000000049b37d sp 00007fbae2b5fa10 error 4 in python2.7[400000+31d000]
[1706252.225984] python[24183]: segfault at 8 ip 000000000049b37d sp 00007fd3fa365a10 error 4 in python2.7[400000+31d000]
[1720961.451787] systemd-udevd[4076]: starting version 215
[1782064.269844] audit: type=1702 audit(1546540443.159:38): op=linkat ppid=21820 pid=22823 auid=4294967295 uid=1000 gid=1000 euid=1000 suid=1000 fsuid=1000 egid=1000 sgid=1000 fsgid=1000 tty=pts0 ses=4294967295 comm="git" exe="/usr/bin/git" res=0
[1782064.269866] audit: type=1302 audit(1546540443.159:39): item=0 name="/cvs/cds/caltech/target/c1vac/.git/objects/85/tmp_obj_uAXhPg" inode=173019272 dev=00:21 mode=0100444 ouid=1001 ogid=1001 rdev=00:00 nametype=NORMAL
[1782064.365240] audit: type=1702 audit(1546540443.255:40): op=linkat ppid=21820 pid=22823 auid=4294967295 uid=1000 gid=1000 euid=1000 suid=1000 fsuid=1000 egid=1000 sgid=1000 fsgid=1000 tty=pts0 ses=4294967295 comm="git" exe="/usr/bin/git" res=0
[1782064.365271] audit: type=1302 audit(1546540443.255:41): item=0 name="/cvs/cds/caltech/target/c1vac/.git/objects/58/tmp_obj_KekHsn" inode=173019274 dev=00:21 mode=0100444 ouid=1001 ogid=1001 rdev=00:00 nametype=NORMAL
[1782064.460620] audit: type=1702 audit(1546540443.347:42): op=linkat ppid=21820 pid=22823 auid=4294967295 uid=1000 gid=1000 euid=1000 suid=1000 fsuid=1000 egid=1000 sgid=1000 fsgid=1000 tty=pts0 ses=4294967295 comm="git" exe="/usr/bin/git" res=0
[1782064.460652] audit: type=1302 audit(1546540443.347:43): item=0 name="/cvs/cds/caltech/target/c1vac/.git/objects/cb/tmp_obj_q62Pdr" inode=173019276 dev=00:21 mode=0100444 ouid=1001 ogid=1001 rdev=00:00 nametype=NORMAL
[1782064.545449] audit: type=1702 audit(1546540443.435:44): op=linkat ppid=21820 pid=22823 auid=4294967295 uid=1000 gid=1000 euid=1000 suid=1000 fsuid=1000 egid=1000 sgid=1000 fsgid=1000 tty=pts0 ses=4294967295 comm="git" exe="/usr/bin/git" res=0
[1782064.545480] audit: type=1302 audit(1546540443.435:45): item=0 name="/cvs/cds/caltech/target/c1vac/.git/objects/e3/tmp_obj_gPI4qy" inode=173019277 dev=00:21 mode=0100444 ouid=1001 ogid=1001 rdev=00:00 nametype=NORMAL
[1782064.640756] audit: type=1702 audit(1546540443.527:46): op=linkat ppid=21820 pid=22823 auid=4294967295 uid=1000 gid=1000 euid=1000 suid=1000 fsuid=1000 egid=1000 sgid=1000 fsgid=1000 tty=pts0 ses=4294967295 comm="git" exe="/usr/bin/git" res=0
[1783440.878997] systemd[1]: Unit serial_TP3.service entered failed state.
[1784682.147280] systemd[1]: Unit serial_TP2.service entered failed state.
[1786407.752386] systemd[1]: Unit serial_MKS937b.service entered failed state.
[1792371.508317] systemd[1]: serial_GP316a.service failed to run 'start' task: No such file or directory
[1795550.281623] systemd[1]: Unit serial_GP316b.service entered failed state.
[1796216.213269] systemd[1]: Unit serial_TP3.service entered failed state.
[1796518.976841] systemd[1]: Unit serial_GP307.service entered failed state.
[1796670.328649] systemd[1]: serial_Hornet.service failed to run 'start' task: No such file or directory
[1797723.446084] systemd[1]: Unit serial_MKS937b.service entered failed state.

 

I don't know enough about the new system so I'm leaving this for Jon to debug. Attachment #3 shows that the analog readout of the P1 pressure gauge suggests that the IFO is still under vacuum, so no random valve openings were effected (as expected, since we valved off the N2 line for this very purpose).

Attachment 1: Screenshot_from_2019-01-03_15-19-51.png
Screenshot_from_2019-01-03_15-19-51.png
Attachment 2: Screenshot_from_2019-01-03_15-14-14.png
Screenshot_from_2019-01-03_15-14-14.png
Attachment 3: 997B13A9-CAAF-409C-A6C2-00414D30A141.jpeg
997B13A9-CAAF-409C-A6C2-00414D30A141.jpeg
  14383   Fri Jan 4 10:25:19 2019 JonOmnistructureVACN2 line valved off

Yes, for TP2 and TP3. They both have a small vent valve that opens automatically on shutdown.

Quote:

Independent question: Are all the turbo forelines vented automatically? We manually did it for the main roughing line.

 

 

ELOG V3.1.3-