Cold cathode gauge CC4 is reading normal.
CC1 is glitching, it is probably dirty.
CC2 is fluctuating too much and it is cutting out for 6-7 minutes. It must be insulated by deposits and there is no emission current.
I think the same goes for P1
They will have to be replaced at the next vent
Vacuum normal valve condition was changed to accommodate SRS-RGA calibration.
VM1 was closed to isolate the RGA from the IFO.
Vacuum valve configuration is back to VACUUM NORMAL condition. RGA calibration completed.
RGA scan attached is the backgroud of the rga with std cal leak open, sn 08581
Krypton at amu 84 and Argon at amu 40 are the cal signals.
Our Osaka TG360MB maglev failed with CSB error message. This means that the dry emergency landing bearing has to be replaced.
I will consalt with Osaka about the choice of replacing bearing or installing new spare tomorrow.
Mean while V1 is closed and the vac envelope is not pumped.
Valve configuration: BG -background, pumping on the RGA-only
High voltage to IOO PZT steering mirrors and OMC are turned off.
PSL output shutter is closed and manual block is in place.
I will start cooling the CYO pump in the morning, so the IFO will be pumped by noon.
Outgassing plus leakrate after 10 hrs the pressure is 2.3 mTorr
This rate of rise is normal and it is safe to work with the ifo.
The CRYO pump cooled down and VC1 was opened. This valve configuration is Cryo-only
PSL output shutter opened at 4pm
PZT HV power supplies turned on for OMC and IOO steering mirrors.
There positions were not corrected to strain gauge values.
Ben helped us to conclude that the FAILURE led indicator is working correctly and
has nothing to do with the one lose, dangling wire#258 in the side connects of the vac rack.
I reset the CSB switch inside the Maglev controller and tried to start accelerating.
It fails after 2-3 sec and failure led light comes on.
Now we can say the MAGLEV 360 is DEAD and the new OSAKA TG420M can be swapped in.
However it requires new interface with our epics based MEDM or better...?
Ben and I found this vacuum valve relay box intermittently shorthing yesterday.
It effects V4, V5, VA6 and VM1........ Please do not touch this box under the beam pipe next to the vac rack!
The function of this box to send 120VAC to the vacuum valve to move.
I tested the cryopump interlock today. It is touchy. I do not have full confidence in it.
I'm proposing that VC1 gate valve should be kept closed while nobody is working in the 40m lab.
How to open gate valve:
1, confirm temp of 12K on the gauge at the bottom of the cryopump
2, if medm screen cryo reads OFF( meaning warm) hit reset will result reading ON (meaning cold 12K )
3, open VC1 gate valve if P1 is not higher than 20 mTorr
VC1 was closed at 18:25,
IFO condition: not pumped,
expected leak plus out gassing should be less than 5 mTorr/day
The RGA is in bg-mode, annuloses are closed off
In response to Steve's elog entry, and for 40m posterity, I provide the Paschen Curve.
CC1 5e-7 Torr, VC1 closed at 18:25, IFO is not pumped, RGA is in bg-mode
At about 1am or so Yoichi and I opened VC1. CC1 had fallen to about 5e-5 torr.
Congratulation Yoichi and Peter for full rf locking at night. Let's remember that the cryopump was shaking the hole vac envelope and ifo during this full lock.
The Cryo pump is running reliably since April 22 hence there is no need to close VC1 repeatedly.
The photo switch interlock was put back onto the H2 vapor pressure gauge and it is working.
Our spare Osaka maglev purchased in Oct 2005 turned out to be having a viton o-ring seal connection on the intake.
It was shipped back to San Jose for retrofitting it with 6" conflat flange ( CF ) This CF is using copper gasket so there will be no permeation of He when you leak check the IFO
The digital controller and cable are here. The controller needs to be interfaced with the interlocks and computer system; those have been in a neglected condition lately.
see elog #1505 Historically after every REBOOT of c1vac2 the readbacks works for 3-4 days only. Fixing of this was postponed many times in the past as low priority or lack of knowledgeable
The maglev TG390MCAB wil be back on Tuesday, May 4, 2009. The mourning of our fateful 360 will only end at the first levitation of the 390.
ETMY damping restored.
Cryo interlock closed VC1 ~2 days ago. P1 is 6.3 mTorr. Cryo temp 12K stable, reset photoswitch and opened VC1
CYAN - cryo ON
BLACK - cryo OFF
BLUE - no crappy lens + mount
I found some neat signal analysis software for my mac (http://www.faberacoustical.com/products/), and took a spectrum of the ambient noise coming from the cryopump. The two main noise peaks from that bad boy were nowhere near 3.7 kHz.
Joe and Steve
The retrofitted Osaka 390 was installed on the pumpspool yesterday.
V1 gate valve is disabled for safety by disconnected pneumatic power plug.
The foreline of this maglev now have a KF25 size viton o-ring directly on the turbo.
This is bad for leak hunting.
Joe is ready with new interface cable. Power supply and cables are in place.
The maglev was pumped down this morning.
All new gas kits and metal hose were leak checked by sprayed methanol.
There is no obvious sign of leak. I was expecting the pressure to drop below 1e-5 Torr in one hour.
TP2 is drying out the levitating coils of the turbo at ~7 l/s for N2
We'll start the pump as soon as Joe is in.
Joe and Steve
The Maglev is running at 680 Hz, 40,800 RPM with V1 gate valve closed and valve disabled to change position. C1vac2 was rebooted before starting.
Interlocks are not tested yet, but the medm COVAC_MONITOR.adl screen is reading correctly. RGA scan will determine the need for baking on Monday
The foreline pressure is still ~2e-5 Torr
Acceleration takes 3 minutes 30sesconds without load. There is no observabale temp effect on the body of the turbo during braking and acceleration.
The IFO is still pumped by the CRYO only
We updated the vacuum control and monitor screens (C0VAC_MONITOR.adl and C0VAC_CONTROL.adl). We also updated the /cvs/cds/caltech/target/c1vac1/Vac.db file.
1) We changed the C1:Vac-TP1_lev channel to C1:Vac-TP1_ala channel, since it now is an alarm readback on the new turbo pump rather than an indication of levitation. The logic on printing the "X" was changed from X is printed on a 1 = ok status) to X is printed on a 0 = problem status. All references within the Vac.db file to C1:Vac-TP1_lev were changed. The medm screens also now are labeled Alarm, instead of Levitating.
2) We changed the text displayed by the CP1 channel (C1:Vac-CP1_mon in Vac.db) from "On" and "Off" to "Cold - On" and "Warm - OFF".
3) We restarted the c1vac1 front end as well as the framebuilder after these changes.
The new Maglev fore line pressure is at 4e-6 torr at day 3
Valve VM1 was closed to isolate IFO from RGA and valve VM2 was opened so the RGA can scan the Maglev only.
The maglev fore line pressure is 3e-6 torr at CC2 after 4 days of pumping. Varian turbo V-70 is pumping on it through V4 and VM2
Actual pumping speed ~10 l/s for N2. There was no baking. The maglev performance looks good : 3e-9 torr on CC4 with RGA -region only.
The Maglev is running for 10 days with V1 closed. The pressure at the RGA-region is at 2e-9 torr on CC4 cold cathode gauge.
Valve VM2 to Rga-only was opened 6 days ago. The foreline pressure is still 2.2e-6 torr with small Varian turbo ~10 l/s on cc2
Daily scans show small improvement in large amu 32 Oxygen and large amu 16, 17 and 18 H20 water peaks.
Argon calibration valve is leaking on our Ar cylinder and it is constant.
The good news is that there are no fragmented hydrocarbons in the spectrum.
The Maglev is soaked with water. It was seating in the 40m for 4 years with viton o-ring seals
However I can not explan the large oxygen peak, either Rai Weiss can not.
The Maglev scans are indicating cleanliness and water. I'm ready to open V1 to the IFO
V1 valve is open to IFO now. V1 interlock will be tested tomorrow.
Valve configuration: VAC NORMAL with CRYO and Maglev are both pumping on the IFO
The IFO RGA scan is normal.
The Cryo needs to be regenerated next. It has been pumping for 36 days since last regenerated.
This has to be done periodically, so the Cryo's 14 K cold head is not insulated by by ice of all things pumped away from the IFO
Joe, Alberto and Steve
We tested gate valve V1 interlock by :
1, decelerated rotation by brake from maglev controller unit.
2, turned maglev controller off from controller unit.
3, unpluged 220VAC plug from wall socket
None of the above action triggered V1 to close. This needs to be corrected in the future.
The MEDM monitor screen of maglev indicated the correct condition changes.
APC Smart-UPS (uninterruptible power supply) batteries RBC12 replaced at 1Y8 vacuum rack.
Their life span were 22 months.
The 40m-IFO vacuum envelope doors are sealed with dual viton O-rings and they are pumped through the annulos lines.
This allows easy access into the chambers. The compression of the o-rings are controlled by the o-ring grooves.
The OOC (output optic chamber)'s west side door has no such groove and it is sealed by just one single O-ring.
We have to protect this O-ring from total compression by 3 shims as shown below.
There were control shims in place before and they disappeared.
Let's remember that these shims are essential to keep our vacuum system in good condition.
IFO pressure was 2.3 mTorr this morning,
The Maglev's foreline valve V4 was closed so P2 rose to 4 Torr. The Maglev was running fine with V1 open.
This is a good example for V1 to be closed by interlock, because at 4 Torr foreline pressure the compression ratio for hydrocarbones goes down.
V4 was closed by interlock when TP2 lost it's drypump. The drypump's AC plug was lose.
To DO: set up interlock to close V1 if P2 exceeds 1 Torr
We added C1:Vac-CC1_pressure to the alarm handler, with the minor alarm at 5e-6 torr and the major alarm at 1e-5 torr.
Steve, Rana, Ben, Jenne, Alberto, Rob
UPS in the vacuum rack failed this afternoon, cutting off power to the vacuum control system. After plugging all the stuff that had been plugged into the UPS into the wall, everything came back up. It appears that V1 closed appropriately, TP1 spun down gracefully on its own battery, and the pressure did not rise much above 3e-6 torr.
The UPS fizzed and smelled burnt. Rana will order a new, bigger, better, faster one.
We've had a devil of a time getting V1 to open, due to the Interlock code.
The short story is that if C1:Vac-PTP1_pressure > 1.0, the interlock code won't let you push the button to open V1 (but it won't close V1).
PTP1 is broken, so the interlock was frustrating us. It's been broken for a while, but this hasn't bitten us till now.
We tried swapping out the controller for PTP1 with one of Bob's from the Bake lab, but it didn't work.
It said "NO COMM" in the C1:Vac-PTP1_status, so I figured it wouldn't update if we just used tdswrite to change C1:Vac-PTP1_pressure to 0.0. This actually worked, and V1 is now open. This is a temporary fix.
The Maglev is running on single phase 220V and that voltage was not interrupted. TP1 was running undisturbed with V1 and V4 closed.
It is independent of the UPS 120V.
The swapped in 307 convectron gauge controller is very likely to have the RS232 connection wired differently from the old one.
PRP gauge has now the same error message as the PTP1: "no comm" I would look at RS232 wiring of the PRP gauge on the broken
controller and adapt the swapped in one to communicate. The PRP was reading 620 Torr before the swap.
As Rob noted last Friday, the UPS which powers the Vacuum rack failed. When we were trying to move the plugs around to debug it, it made a sizzling sound and a pop. Bad smells came out of it.
Ben came over this week and measured the quiescent power consumption. The low power draw level was 11.9 A and during the reboot its 12.2 A. He measured this by ??? (Rob inserts method here).
So what we want is a 120 V * 12.2 A ~ 1.4 kVA UPS with ~30-50% margin. We look for this on the APC-UPS site:
On Monday, we will order the SUA2200 from APC. It should last for ~25 minutes during an outage. Its $1300. The next step down is $200 cheaper and gives 10 minutes less uptime.
The new APC Smart -UPS 2200VA is now running at the vacuum rack. There are 2 load monitoring leds on out of 5
Maglev, dry pumps and roughing pumps are not using UPS.
The switch over went smoothly with Yoichi's help.
First we closed all vacuum valves and stopped the two small turbos.
Than turned power off to instruments in the vac-rack and VME: c1vac1 & c1vac2
Maglev was left running.
Now we moved the AC plugs from the wall receptacles over to the back of the UPS and powered them up.
Varian turbos were restarted and vacuum valves were restored in order to reach vacuum normal condition.
See 40m Vacuum System States and Sequences Manual of 10-24-2001
Linux 3 desk top computer is out of order at the pump spool. We should replace it.
The vacuum control screen can be pulled up on a lap top: /cvs/cds/caltech/medm/c0/ve/VacControl_BAK.adj
Steve noticed the RGA was not working today. It was powered on but no other lights were lit.
Turns out the c0rga machine had not been rebooted when the file system on linux1 was moved to the raid array, and thus no longer had a valid mount to /cvs/cds/. Thus, the scripts that were run as a cron could not be called.
We rebooted c0rga, and then ran ./RGAset.py to reset all the RGA settings, which had been reset when the RGA had lost power (and thus was the reason for only the power light being lit).
Everything seems to be working now. I'll be adding c0rga to the list of computers to reboot in the wiki.
The RGA isolation valve VM1 was closed since Aug 24, 2009 I installed the new UPS that time.
The last RGA scan in the log is from Aug 7, 2009 The vacuum rack UPS failed on Aug 15, 2009
I opened VM1 today so we can have ifo rga scan tomorrow.
Pump down #66 is 435 days old. RGA scan is normal. New maglev is fine. New UPS is in place but not hooked up to communicate.
V1 has bare minimum interlock. Pirani vacuum gauges PTP1 and PRP do not communicate with readout system.
There is no emergency dial out in case of vacuum loss. Our existing vacuum dedicated desk top computer is dead.
New cold cathodes, Pirani gauges and gauge controller should be added.
In general: vacuum system needs an upgrade !
Steve, Jenne, Koji
09:30 25 torr
10:30 180 torr
11:00 230 torr
12:00 380 torr
13:00 520 torr
14:30 680 torr - Finish. It is already over pressured.
[Everybody: Alberto, Kiwamu, Joe, Koji, Steve, Bob, Jenne]
The last heavy door was put on after lunch. We're now ready to pump.
last 20 days - including the pounding from next door
The vacuum system is at 760 torr All chambers with doors on and their annuloses are pumped down.
PSL output shutter is still closed. We are fully prepared to star slow pump down tomorrow.
The plan is to reach 1 torr ~ in 6 hrs without a creating a sand storm.