Joe, Alberto and Steve
We tested gate valve V1 interlock by :
1, decelerated rotation by brake from maglev controller unit.
2, turned maglev controller off from controller unit.
3, unpluged 220VAC plug from wall socket
None of the above action triggered V1 to close. This needs to be corrected in the future.
The MEDM monitor screen of maglev indicated the correct condition changes.
APC Smart-UPS (uninterruptible power supply) batteries RBC12 replaced at 1Y8 vacuum rack.
Their life span were 22 months.
The 40m-IFO vacuum envelope doors are sealed with dual viton O-rings and they are pumped through the annulos lines.
This allows easy access into the chambers. The compression of the o-rings are controlled by the o-ring grooves.
The OOC (output optic chamber)'s west side door has no such groove and it is sealed by just one single O-ring.
We have to protect this O-ring from total compression by 3 shims as shown below.
There were control shims in place before and they disappeared.
Let's remember that these shims are essential to keep our vacuum system in good condition.
IFO pressure was 2.3 mTorr this morning,
The Maglev's foreline valve V4 was closed so P2 rose to 4 Torr. The Maglev was running fine with V1 open.
This is a good example for V1 to be closed by interlock, because at 4 Torr foreline pressure the compression ratio for hydrocarbones goes down.
V4 was closed by interlock when TP2 lost it's drypump. The drypump's AC plug was lose.
To DO: set up interlock to close V1 if P2 exceeds 1 Torr
We added C1:Vac-CC1_pressure to the alarm handler, with the minor alarm at 5e-6 torr and the major alarm at 1e-5 torr.
Steve, Rana, Ben, Jenne, Alberto, Rob
UPS in the vacuum rack failed this afternoon, cutting off power to the vacuum control system. After plugging all the stuff that had been plugged into the UPS into the wall, everything came back up. It appears that V1 closed appropriately, TP1 spun down gracefully on its own battery, and the pressure did not rise much above 3e-6 torr.
The UPS fizzed and smelled burnt. Rana will order a new, bigger, better, faster one.
We've had a devil of a time getting V1 to open, due to the Interlock code.
The short story is that if C1:Vac-PTP1_pressure > 1.0, the interlock code won't let you push the button to open V1 (but it won't close V1).
PTP1 is broken, so the interlock was frustrating us. It's been broken for a while, but this hasn't bitten us till now.
We tried swapping out the controller for PTP1 with one of Bob's from the Bake lab, but it didn't work.
It said "NO COMM" in the C1:Vac-PTP1_status, so I figured it wouldn't update if we just used tdswrite to change C1:Vac-PTP1_pressure to 0.0. This actually worked, and V1 is now open. This is a temporary fix.
The Maglev is running on single phase 220V and that voltage was not interrupted. TP1 was running undisturbed with V1 and V4 closed.
It is independent of the UPS 120V.
The swapped in 307 convectron gauge controller is very likely to have the RS232 connection wired differently from the old one.
PRP gauge has now the same error message as the PTP1: "no comm" I would look at RS232 wiring of the PRP gauge on the broken
controller and adapt the swapped in one to communicate. The PRP was reading 620 Torr before the swap.
As Rob noted last Friday, the UPS which powers the Vacuum rack failed. When we were trying to move the plugs around to debug it, it made a sizzling sound and a pop. Bad smells came out of it.
Ben came over this week and measured the quiescent power consumption. The low power draw level was 11.9 A and during the reboot its 12.2 A. He measured this by ??? (Rob inserts method here).
So what we want is a 120 V * 12.2 A ~ 1.4 kVA UPS with ~30-50% margin. We look for this on the APC-UPS site:
On Monday, we will order the SUA2200 from APC. It should last for ~25 minutes during an outage. Its $1300. The next step down is $200 cheaper and gives 10 minutes less uptime.
The new APC Smart -UPS 2200VA is now running at the vacuum rack. There are 2 load monitoring leds on out of 5
Maglev, dry pumps and roughing pumps are not using UPS.
The switch over went smoothly with Yoichi's help.
First we closed all vacuum valves and stopped the two small turbos.
Than turned power off to instruments in the vac-rack and VME: c1vac1 & c1vac2
Maglev was left running.
Now we moved the AC plugs from the wall receptacles over to the back of the UPS and powered them up.
Varian turbos were restarted and vacuum valves were restored in order to reach vacuum normal condition.
See 40m Vacuum System States and Sequences Manual of 10-24-2001
Linux 3 desk top computer is out of order at the pump spool. We should replace it.
The vacuum control screen can be pulled up on a lap top: /cvs/cds/caltech/medm/c0/ve/VacControl_BAK.adj
Steve noticed the RGA was not working today. It was powered on but no other lights were lit.
Turns out the c0rga machine had not been rebooted when the file system on linux1 was moved to the raid array, and thus no longer had a valid mount to /cvs/cds/. Thus, the scripts that were run as a cron could not be called.
We rebooted c0rga, and then ran ./RGAset.py to reset all the RGA settings, which had been reset when the RGA had lost power (and thus was the reason for only the power light being lit).
Everything seems to be working now. I'll be adding c0rga to the list of computers to reboot in the wiki.
The RGA isolation valve VM1 was closed since Aug 24, 2009 I installed the new UPS that time.
The last RGA scan in the log is from Aug 7, 2009 The vacuum rack UPS failed on Aug 15, 2009
I opened VM1 today so we can have ifo rga scan tomorrow.
Pump down #66 is 435 days old. RGA scan is normal. New maglev is fine. New UPS is in place but not hooked up to communicate.
V1 has bare minimum interlock. Pirani vacuum gauges PTP1 and PRP do not communicate with readout system.
There is no emergency dial out in case of vacuum loss. Our existing vacuum dedicated desk top computer is dead.
New cold cathodes, Pirani gauges and gauge controller should be added.
In general: vacuum system needs an upgrade !
Steve, Jenne, Koji
09:30 25 torr
10:30 180 torr
11:00 230 torr
12:00 380 torr
13:00 520 torr
14:30 680 torr - Finish. It is already over pressured.
[Everybody: Alberto, Kiwamu, Joe, Koji, Steve, Bob, Jenne]
The last heavy door was put on after lunch. We're now ready to pump.
last 20 days - including the pounding from next door
The vacuum system is at 760 torr All chambers with doors on and their annuloses are pumped down.
PSL output shutter is still closed. We are fully prepared to star slow pump down tomorrow.
The plan is to reach 1 torr ~ in 6 hrs without a creating a sand storm.
Oplev positions before and after drag wiped arm TMs as of yesterday. Slow-mode pumpdown has started with 3/4 turn opened RV1 valve at 8am today.
I'm leaving the lab now for less than 2 hours. I should be back in time for when the pumping is finished so that I can measure the finesse again.
Wait, Wait, Wait. You are moving too fast. Go one by one.
Check the PZTs, the MC, initial pointings, IFO mirrors, some of the partial locks, and maybe some momentary full locks?
Once the recover of the IFO is declared, you can proceed to the measurements.
Pump down is completed. Valve configuration is VACUUM NORMAL. CC1 pressure is in the ~8 e-5 torr PSL output shutter is opened.
This morning I and Steve replaced the dry fore pump of TP3, which is located under the y-arm.
After replacing it we confirmed vacuum normal condition. The fore line pressure of TP3 went down to 11 mTorr from 750 mTorr
Attached picture is new pump after setup.
I have installed a slow start throttle valve in front of V3 This spring loaded valve will cut down on the flow at high pressures. There will be no more sand storme
and static built-up during pump down.
Joe and Alex are working on the computers. Our vacuum system is temporary "All off" condition: meaning all valves are closed, so there is no pumping. cc1 = 1.6e-6 Torr
Designated vacuum control lap top is trouble some to use. Joe finally fixed it and I switched valve configuration back to vacuum normal. Shutter is open
As preparation for the upcoming planned power outage we turned turbos, RGA off and closed valves.
IFO chamber is not pumped now. Small leaks and out gassing will push the pressure up slowly. At 3 mTorr of P1 the PSL output shutter
will be closed by the interlock.
It is OK to use light in the IFO up to this point.
The Vac pressure measured at P1 is at 2.5mTorr. I expect we'll hit 3mTorr sometime this afternoon, at which point (according to Steve) the interlock will shut the shutter, and we won't have light in the IFO. Anything which needs to happen with light in the IFO before Monday needs to happen fairly soon.
Morning condition: vacuum rack power is still off, no MEDM screen reading.....meaning unknown vacuum pressure.We closed PSL shutter immediately.
Joe restored c1iscepis and Jenne powered up the vac-rack UPS. Now the rest of the vac-rack power were restored from starting at the top to bottom.
P1 was reading 15 mTorr. We restarted pumps and set vacuum valve positions. V1 opening required Rob's recipe of elog # 1863 to defeat interlock that
has a non communicating gauge: PTP1
CC1 pressure just reached 1e-6 Torr at VAC NORMAL configuration.
Kiwamu and Steve have started venting the 40m vacuum envelope.
centered oplevs at resonating cavities,
ITM references were set by green pointer from the ends by Koji,
closed PSL shutter and placed manual block into beam path,
checked jamnuts in locked positions on bellows,
turned HV off at PZT-Jena "steering mirror" power supply and OMC HV ps
checked particle counts,
switched oplev servos off,
set up N2 cylinder to start vent from 1e-6 Torr to 25 Torr,
have ~ 6 cylinders of instrument grade compressed air to bring envelope from 25 Torr to 760 Torr
All three cranes were wiped off today.
Our vacuum was not pumped for 4 days. The computers were not up when we started pumping again. The manual reading on P1 was 15 mTorr.
This means that our outgassing plus leakrate is ~3.8 mTorr /day
3-5 mTorr / day is normal
The vent has been finished.
Now the pressure inside the chamber is 760 torr, and it's getting equilibrium with the atmospheric pressure.
Therefore we are ready and can open the door of the chamber tomorrow.
Kiwamu has completed the vent.
There is a planned power outage tomorrow, Saturday from 7am till midnight.
I vented all annulies and switched to ALL OFF configuration. The small region of the RGA is still under vacuum.
The vac-rack: gauges, c1vac1 and UPS turned off.
Vac- rack is powered back up. UPS first, than all other power switches from top to bottom of the rack, except Maglev
Manually started one by one TP2 and TP3 to accelerate to 50 KRPM
Brought up vac.control screen on lap-top at /cvs/cds/caltech/medm/c0/ve/VacControl_BAK.adl
V5 and VM3 were opened so TP3 can pump on the RGA
V4 was opened so TP2 can pump on the Maglev-TP1. The Maglev power was turned on and started acceleration.
The vac control screen positions indicators were checked for true position and annulies vent valves were opened.
RGA manual on/off switch was turned at the top of the RGA-head. Ubuntu copmuter was started at cc4 1.1e-6 Torr
The RGA communication was started with: ssh c0rga from control room
The rga-script was started ./RGAset.py This script turns on the filament, rs-232 and scan parameters etc
Vac -configuration: IFO-P1 at atm, RGA is pumped and running in background mode, all annulos at atm
RGA scan of rga-region only at day 18 This is the back ground of the rga with some calibration gas.
Steve and Koji
We aligned the peeping mirrors to look at the surface of the ITMs.
They had been misligned as we move the positions of the ITMs, but now they are fine.
[Jenne, Kiwamu, and Steve via phone]
Around 9:30pm, Kiwamu and I came back from dinner, and were getting ready to begin the beam scan measurements. I noticed that one of the vacuum pumps was being very loud. Kiwamu noted that it is the fore pump for TP3's turbo, which he and Steve replaced in January (elog 2538). We had not noticed these noises before leaving for dinner, around 8pm.
We called Steve at home, and he could hear the noise through the phone. He said that even though it was really loud, since it was reading 3.3mTorr (on the display of the controller, in the vacuum rack just above head-height) which is close to the nominal value, it should be fine to leave. He will check it out in the morning. If it had been reading at or above ~1Torr, that's indicative of it being really bad, and we would have needed to shut it off.
For future reference, in case we need to turn it off, Steve said to use the following procedure:
1. Close VM3, to isolate the RGA, which is what this pump is currently (while we're at atmosphere) pumping on. I don't know if there are other things which would need to be shut at this stage, if we were at vacuum nominal.
2. Close VM5, which is right in front of TP3, so TP3's pump is just pumping on itself.
3. Push the "Stop" button on the Turbo controller for TP3, in the vacuum rack, about waist level. Turning off the turbo will also turn off the fore pump.
UPDATE, 1am: The controller in the rack is reading 3.1mTorr, so the pump, while still noisy, still seems to be working.
The foreline pressure of TP3 is 2.9 mTorr The drypump is loosing it's bearing and it is very noisy.
V3, V5 closed and TP3 small turbo controller off. This turned off the noisy forepump that has to be replaced.
RGA is running at cc4 2e-6 Torr
The RGA was turned off at cc4 1e-5 Torr
Bob replaced the tipseal in an other drypump and I swapped it in. TP3 turbo is running again, it's foreline pressure is 40 mTorr. The RGA is still off
We started a vacuum work in this morning. And still it's going on.
Although the last night the green team replaced a steering mirror by an 80% reflector on the PLS table, the beam axis to the MC looks fine.
The MC refl beam successfully goes into the MCrefl PD, and we can see the MC flashing as usual.
We started measuring the distance of the optics inside the vacuum chamber, found the distance from MC3 to MMT1(curved mirror) is ~13cm shorter than the design.
We moved the positions of the flat mirror after the Faraday and the MMT1, but could not track the beam very well because we did not completely lock the MC.
Now we are trying to get the lock of the MC by steering the MC mirrors.
Kevin suceeded in locking it !!
I just finished redoing the calc based on the measurements that happened last week. Using the average of the Vert and Horz measurements in Kevin's elog 2986, I find that we need to make the MMT telescope ~8cm longer. So, can you please place the flat mirror after the Faraday in the same place as the drawing, but move the MMT1 79mm farther away from that flat mirror? Looking at the table layouts that Koji has on the wiki, this should still (barely) fit.
d2a = 884.0mm (no change) ------ MC3 to Flat after Faraday
d2b = 1123.2mm (move MMT1 farther toward center of BS table) -------- Flat after Faraday (SM1) to MMT1
d3 = 1955.0mm (result of moving MMT1) --------- MMT1 to MMT2
d4a = 1007.9mm (no chnage) ----------- MMT2 to SM2
d4b = 495.6mm (no change) ------------ SM2 to PRM
I just got off the phone with Alberto and Kiwamu, and I'm going to try to recalculate things based on their measurements of the distances between MC3 and SM1. It sounds like the CAD drawings we have aren't totally correct. I know that when we opened doors just before Christmas we measured the distances between the BS table and the ITM tables, but I don't think we measured the distance between the IOO table and the BS table. Hopefully we can fit everything in our chambers.....
Some unused optics in the BS chamber were removed.
After that the beam splitter has been drag wiped.
So now the BS chamber is waiting for the installation of the other core optics i.e. PRM, SRM and Tip-Tilts.
-- removing of unused optics
There were some unused optics, mainly 1.5 inch optics which had been used for the oplevs in the chamber.
Kathaine, Shamila (Team Magnet) and Kiwamu took those optics out from the chamber.
And then we carefully wrapped each of them by aluminum foils and put them in some clear boxes.
In fact, before wrapping them, we gently attached lens papers on their HR surfaces such that aluminum foils can not damage it.
Now there are only three 1.5 inch optics in the chamber, and they are supposed to be used for the oplevs.
We didn't remove any of the 2 inch optics and the PZT mirrors because they are still going to be used.
These are the pictures of the BS chamber after we cleaned up them.
-- wiping of the BS
Rana and Kiwamu drag wiped the HR surface of the BS by using lens paper with the solvents.
The below is the procedure we did. You can find some details about the wiping technique for suspended optics in this entry.
In this time we could wipe the beam splitter without removing the front earthquake stops because the beam splitter was brought close enough to us.
(1). put some blocks attaching the edge of the bottom plate of the tower in order to record the original position.
(2). locked the beam splitter to the frame by screwing the earthquake stops.
(3). made sure if it is really locked by seeing the output signal of the OSEMs in dataviewer. If it's locked successfully, the resonant frequency gets higher and the Q-value gets lower.
(3). moved the BS tower close to the door in order to reach the beam splitter easily.
(4). inspected the surface by using a fiber light. There were about 10 bright spots on the HR surface.
(5). wiped the surface three times by using the lens paper with Aceton.
(6). wiped it several times with Isopropyl.
(7). inspected the surface again, found there were no big bright spots near the center. Thumbed up
(8). put the tower back to the original place and released the beam splitter from the earthquake stops.
[Jenne, Kiwamu, Steve, Sharmila, Katherine, Joe]
We finished bolting the door on the new ITMX (old ITMY) and putting the access connector section back into place. We finished with torquing all the bolts to 40 foot-pounds.
The folding crane was fixed and tested this morning by the NNN rigging company. Pictures will be posted by Steve in the morning.
Afterwards, the ITM-east door was installed, jam-nuts checked. No high voltage was on for the in-vac PZTs.
The annulus spaces were roughed down to 350mTorr by Roughing Pump RP1. For this operation, we removed the low flow valve from the RP1 line.
After the spaces came down to ~400 mTorr, we closed their individual valves.
Warning: The VABSSC1 and VABSSC0 valves are incorrect and misleadingly drawn on the Vacuum overview screen.
Our idea is to have a much slower pumpdown this time than the last time when we had a hurricane kick up the dust. Looks like it worked, but next time we should do only 1/2 turn.
The pumpdown started at 4 PM (2300 UTC). At 10 PM, we (Jenne, Jan, and I) opened up the RV1 valve to full open. That's the second inflection point in the plot.
As per Steve's instructions, at 12:43 AM, I used the following steps to stop the pumpdown until the morning: