I replaced an empty N2 cylinder, there are now two empty tanks in the outside rack.
I will be at the 40m today from 9:30am to 4pm.
I removed the backing pumps for TP2 and TP3 today to test ultimate pressure and determine if they need a tip seal replacement. This was done with Jon backing me on Zoom. We closed off TP3 and powered down TP3 and the auxilliary pump, in order to remove the forepumps from the exhaust line.
Once pumps were removed I connected a Pirani gauge to the pump directly and pumped down, results as follows:
TP2 Forepump (Agilent IDP 7):
TP3 Forepump (Varian SH 110):
TP3 forepump defintely needs a new tip seal, and while the pressure on TP2 Forepump was good there was a significant amount of particulate that came out of the exhaust line, so a new tip seal might not be needed but it is recommeded.
I agree with your assessment, Jordan. If I'm not mistaken the scroll pump for TP2 is new; we had a very early failure with the last new scroll pump (the forepump for TP3) tip seals at just over 5000 hours. Glad to see my replacement seals held up for over 60K hours. If this is the trend with these pumps, we can simply run them to around 60000 hours and replace the seals at that time, rather than waiting for failure! - Chub
I will be at the 40m today from 9am to 3pm.
Tip Seals were replaced on the forepumps for TP2 and TP3, and both are ready to be installed back onto the forelines.
TP2 Forepump Ultimate Pressure: 180 mtorr
TP3 Forepump Ultimate Pressure: 120 mtorr
I will be at the 40m today from 11am to 4pm.
I will be in the Clean and Bake lab today from 10am to 4pm.
I will be in the Clean and Bake lab from 11pm to 4pm
I will be in the Clean and Bake lab today from 11am to 4pm.
I will be in the Clean and Bake lab today from 11:30am to 4pm
I will be in the clean and bake lab today from 9am to 4pm.
I will be in the Clean and Bake lab from 9am to 4pm today.
As part of an ongoing effort to improve airflow in workspaces/bathrooms on campus, I have installed an air scrubber unit in each of the bathrooms at the 40m lab.
I will be in the Clean and Bake lab today from 8:30am to 4pm
I will be in the clean and bake lab today from 9am to 3pm.
I will be in the Clean and Bake lab today from 9am to 3pm
I will be in the Clean and Bake lab today from 9am to 4pm
I will be in the Clean and Bake Lab today from 9am to 4pm.
I will be in the Clean and Bake lab today from 9am to 4pm.
I installed the Tripp Lite SMX1000RT2U and Tripp Lite Smart1000LCD at the bottom of the 1x8 electronics rack. These are plugged in to power, and are ready for testing. All other cables (serial, usb, etc.) have been left on the table next to the 1x8 rack.
I removed the forepump (Varian SH-110) for TP3 today to see why it had failed over the weekend. I tested it in the C&B lab and the ultimate pressure was only ~40torr. I checked the tip seals and they were destroyed. The scroll housing also easily pulled off of the motor drive shaft, which is indicative of bad bearings. The excess travel in the bearings likely led to significant increase in tip seal wear. This pump will need to be scrapped, or rebuilt.
I tested the spare Varian SH-110 pump located at the X-end and the ultimate pressure was ~98 mtorr. This pump had tip seals replaced on 11/5/18, and is currently at 55163 operating hours. It has been installed as the TP3 forepump.
Once installed, restarting the pump line occured as follows: V5 Closed, VA6 closed, VASE Closed, VASV closed, VABSSCI closed, VABS closed, VABSSCO closed, VAEV closed, VAEE closed,TP3 was restarted and once at normal operation, valves were opened in same order.
The pressure differential interlock condition for V5 was temporaily changed to 10 torr (by Gautam), so that valves could be opened in a controlled manner. Once, the vacuum system was back to normal state the V5 interlock condition was set back to the nominal 1 torr. Vacuum system is now running normally.
I removed the forepump to TP2 this morning after the vacuum failure, and tested in the C&B lab. I pumped down on a small volume 10 times, with no issue. The ultimate pressure was ~30 mtorr.
I re-installed the forepump in the afternoon, and restarted TP2, leaving V4 closed. This will run overnight to test, while TP3 backs TP1.
In order to open V1, with TP3 backing TP1, the interlock system had to be reset since it is expecting TP2 as a backing pump. TP2 is running normally, and pumping of the main volume has resumed.
Unclear why the TP2 foreline pump failed in the first place, it has been running fine for several hours now (although TP2 has no load, since V4 isolates it from the main volume). Koji's plots show that the TP2 foreline pressure did not recover even after the interlock tripped and V4 was closed (i.e. the same conditions as TP2 sees right now).
I have placed 3 new in box, IDP 7 forepumps along the x arm of the interferometer. These are to be used as spares for both the 40m and Clean and Bake.
As it currently stands the Center of Mass of the Adapter Ring/Optic assembly is 0.0175" out of the plane formed by the suspension wire. See Attachments. The side plate, along with the EQ stops are hidden to show the CoM and the plane.
Note: The changes discussed in the meeting with Calum have not been added and are a work in progress. These changes include:
- Adding a 45 deg chamfer to the both parallel faces of the adapter ring. This along with a modified bracket for the EQ stops will allow for easier adjustment of the screws.
- Potentially changing material of adapter ring to stainless stell to more accurately emulate the mass of a 3" optic.
- Different adjustment mechanism of the "dumbell" at bottom of adapter ring to something similar to the VOPO suspension (will need to consult Calum further)
Adjusting the thickness of the cylindrical hole for the mirror on the 2" optic sleeve, from .6875" to .61" thick, moves the CoM to 0.0003" out of plane from the suspension wire. This is with the dumbell at its neutral point.
How close to zero do we need this to be? More fine tuning of that thickness can get it to zero, but this would require much tighter machining tolerance on that hole depth.
Moving the dumbell towards the back of the SOS assembly (noted as negative direction, with origin at the plane formed by the wires), moves the CoM to -0.002" from the plane.
Moving the dumbell towards the front of the SoS assmebly (positive direction wrt the plane formed by the suspension wire), moves the CoM to +0.0022" from the plane.
So the total adjustment range with the dumbell is -0.002"to 0.0022", with the plane formed by the wires as the origin.
I have added a .1", 45deg chamfer to the bottom of the adapter ring. This was added for a new placement of the eq stops, since the barrel screws are hard to access/adjust.
This also required a modification to the eq stop bracket, D960008-v2, with 1/4-20 screws angled at 45 deg to line up with the chamfer.
The issue I am running into is there needs to be a screw on the backside of the ring as well, otherwise the ring would fall backwards into the OSEMs in the event of an earthquake. The only two points of contact are these front two angled screws, a third is needed on the opposite side of the CoM for stability. This would require another bracket mounted at the back of the SOS tower, but there is very little open real estate because of the OSEMs.
Instead of this whole chamfer route, is it possible/easier to just swap the screws for the barrel eq stops? Instead of a socket head cap screw, a SS thumb screw such as this, will provide more torque when turning, and remove the need to use a hex wrench to turn.
Adding the chamfer around the edge of the optic ring did not change the center of mass relative to the plane from the suspension wires.
The CoM was .0003" away from the plane. Adding the chamfer moved it closer by .0001". See the attached photo.
I've also attached the list of the Moments of Inertia of the SOS Assembly.
When I came into the lab this morning, I noticed that both N2 tanks were empty. I had swapped one on Friday (4-16-21) before I left the lab. Looking at the logs, the right tank (T2) sprung a leak shortly shortly after install. I leak checked the tank coupling after install but did not see a leak. There could a leak further down the line, possibly at the pressure transducer.
The left tank (T1) emptied normally over the weekend, and I quickly swapped the left tank for a full one, and is curently at ~2700 psi. It was my understanding that if both tanks emptied, V1 would close automatically and a mailer would be sent out to the 40m group. I did not receive an email over the weekend, and I checked the Vac status just now and V1 was still open.
I will keep an eye on the tank pressure throughout the day, and will try to leak check the T2 line this afternoon, but someone should check the vacuum interlocks and verify.
Installed T2 today, and leaked checked the entire line. No issues found. It could have been a bad valve on the tank itself. Monitored T2 pressure for ~2 hours to see if there was any change. All seems ok.
Here are the mass properties for the only the test mass assembly (optic, 3" ring, and wire block). (Updated with g*mm^2)
No, this is the property of the suspension assembly. The mass says 10kg
Could you do the same for the testmass assembly (only the suspended part)? The units are good, but I expect that the values will be small. I want to keep at least three significant digits.
The current vertical distance between the CoM and the wire clamping point on the 3" Ring assembly is 0.33mm. That is the CoM is .33 mm below the clamping point of the wire. I took the clamping point to be the top edge of the wire clamp piece. see the below attachments.
I am now modifying the dumbell mechanism at the bottom of the ring to move the CoM to the target distance of 1.1mm.
After changing the material of the Balance Mass from 6061 Al to 304 Steel, and changing the thickness to 0.21" from 0.25". The CoM is now 1.11mm below the clamping point.
Koji expected a mass change of ~ 4g to move the mass to 1.1mm. The 6061 mass weighed ~1.31g and the 304 mass weighs 4.1g.
A potential issue with this is the screw used the adjust the position of these balance masses, threads through both the aluminum ring and this now 304 steel mass. A non silver plated screw could cold weld at the mass, but a silver plated screw will gall in the aluminum threads.
8 of the 2"->3" adapter rings (D2100377) arrived from RDL yesterday. I have not tested the threads but dimensional inspection on SN008 cleared. Parts look very good. The rest of the parts should be shipping out in the next week.
The right N2 tank had a bad/loose valve and did not fully open. This morning the left tank was just about empty and the right tank showed 2000+ psi on the gauge. Once the changeover happened the copper line emptied but the valve to the N2 tank was not fully opened. I noticed the gauges were both reading zero at ~1pm just before the meeting. I swapped the left tank, but not in time. The vacuum interlocks tripped at 1:04 pm today when the N2 pressure to the vacuum valves fell below 65psi. After the meeting, Chub tightened the valve, fully opened it and refilled the lines. I will monitor the tank pressures today and make sure all is ok.
There used to be a mailer that was sent out when the sum pressure of the two tanks fell <600 psi, telling you to swap tanks. Does this no longer exist?
Yehonathan noticed today that the silver plated hardware on the assembled SOS towers had some pretty severe discoloration on it. See attached picture.
These were all brand new screws from UC components, and have been sitting on the flow bench for a couple months now. I believe this is just oxidation and is not an issue, I spoke to Calum as well and showed him the attached picture and he agreed it was likely oxidation and should not be a problem once installed.
He did mention if there is any concern from anyone, we could take an FTIR sample and send it to JPL for analysis, but this would cost a few hundred dollars.
I don't believe this to be an issue, but it is odd that they oxidized so quickly. Just wanted to relay this to everyone else to see if there was any concern.
The remaining machined parts for the SOS adapter ring have arrived. I will inspect these today and get them ready for C&B.
I had 8 standoffs made at the Caltech chemistry machine shop to be used as spacers for the side magnets on the 3" Ring assembly. This is to create enough clearance between the magnet and the cap screws directly above on the wire clamp.
These are 0.075" diameter by .10" length. Putting them through clean and bake now.
After an overnight pumpdown/RGA warm up, I took a 100 amu scan of the RGA volume and subsequent pumping line. Attached is a screenshot along with the .txt file. Given the high argon peak (40) and the N2/O2 ratio, it looks like there is a decent sized air leak somehwere in the volume.
Are we interested in the hydrocarbon leak rates of this volume? That will require another scan with one of the calibrated leaks opened.
Edit: Added a Torr v AMU plot to see the partial pressures
So that Jordan can run the RGA scan this afternoon, I ran TP3 and started pumping down the RGA section.
- Same 1~4
- Same 5
- 6 Opened only the backing path for TP3
- 7 Turned on TP3 only
- TP3 reached the nominal full speed @75kRPM
- 11 Opened V5 to pump the pump spool -> Immediately reached P3<1e-4
- 12 Opened VM3 to pump the RGA section -> Immediately reached P4<1e-4
The pumps are kept running. I'll come back later to shut down the pumps.
=> Jordan wants to heat the filament (?) and to run the scan tomorrow.
So we decided to keep TP3 running overnight. I switched TP3 to the stand-by mode (= lower rotation speed @50kRPM)
We installed the 4x DB25 feedthru flange on the North-West port of ITMX chamber this afternoon. It is ready to go.
Today, Chub and I removed TP1 and the failed manual gate valve off of the pumping spool.
First, P2 needed to be vented in order to remove TP1. TP1 has a purge valve on the side of the pump which we slowly opened bringing the P2 volume up to atmosphere. Although, this was not vented using the dry air/N2, using this purge valve eliminated the need to vent the RGA volume.
Then we disconnected TP1 foreline, removed TP1+8" flange reducer, then the gate valve. All of the removed hardware looked good, so no need to replace bolts/nuts, only needs new gaskets. TP1 and the failed valve are sitting on a cart wrapped in foil next to the pumping station.
Chub and I installed the new manual gate valve (Nor-Cal GVM-6002-CF-K79) and reinstalled TP1. The new gate valave was placed with the sealing side towards the main 40m volume, then TP1 was installed on top and the foreline reattched to TP1.
This valve has a hard stop in the actuator to prevent over torquing.
Jordan, Chub, Paco
Chub and I went into the lab this morning to leak check the new gate valve after pumping over the weekend. This would be done through the RGA and spraying helium around the newly installed flanges. The RGA is set to monitor the partial pressure of helium versus time and we visually watch for any spikes in pressure to indicate an air leak.
So, I mistakenly thought the gate valve was opened and both sides were being pumped on, this was not the case. The vavle was closed so there was a pressure differential whenI turned the handle it tripped the interlocks and closed V4. I closed VM3 to the RGA volume to prevent the filament from being damaged.
Then the medm screen for the vac controls started flashing rapidly, I closed the window and reopened the controls to find all the panels were white, but we could still see the read only vac screen. So Paco restarted c1vac and the controls were restored. V7 closed, but was then reopened.
We now need to restart pumping with the odd pressure differentials between V4 and VM3.
- TP2 turned off along with the AUX pump
- VM3 opened to vent RGA volume (RGA turned off, filament cooled)
- Open the manual vent screw on TP1 to bring the RGA+TP1 volume back to atmosphere, now no pressure diff. on V4
- Open V4, restart AUX pump to rough out the volume
- Connect RP1/3 line to the pump scroll, turn on RP1/3 RGA+TP1 volume went to mtorr
- Slowly open the manual gate valve
- Connect AUX pump to TP2 and rough out
- Restart TP2, once at speed open V4, close V6 and turn off RP1/3
Now we are pumping on the RGA/TP1 volume with TP2, leak check attempt #2 will happen tomorrow morning
This morning Chub and I leak checked the manual gate valve with the RGA and helium. There was no change in the helium partial pressure while spraying helium around the flanges, all looks good.
I also took a 100 AMU analog scan, after the filament had warmed up overnight and the plot was quite noisy even with the lowest scan speed. I recommend this unit go back to SRS for a filament replacement/recalibration. I am worried yesterday's "vent" of the RGA volume may have burned the filament. See the comparison of yesterday's analog scan to today's below.
Today, Tega and I would like to vent the pump spool an dinstall the new FRG-400 Agilent Pressure Gauges (per elog 15703). The attached picture shows the volume needed to be vented highlighted in red, and the gauges that need to be replaced/removed (purple dot next to the name).
The vent plan is as follows:
Shut down TP2
Install new gauges
Will add to post with updates post vent.
Prior to venting the RGA volume on Tuesday (4/12/2022) I took an RGA scan of the volume to be vented (RGA+TP1 volume+Manual Gate Valve) to see if there was a difference after replacing the manual gate valve. Attached is the plot from 4/12/22, and an overlay plot to complare 4/12/22 to 12/10/2021, when the same volume was scanned with the old (defective) manual gate valve.
There is a significant drop in the ratio O2 compared the the nitrogen peak and reduced Argon (AMU 40) which indicates there is no longer a large air leak.
12/10/21 N2/O2 ratio ~ 4 (Air 78%N2 / 21%O2)
4/12/22 N2/O2 ratio ~ 10
There is one significant (above noise level) peak above AMU 46, which is at AMU 58. This could possibly be acetone (AMU 43 and 58) but overall the new RGA Volume scans look significantly better after the manual gate valve replacement. Well done!
While the pumpspool is vented, I thought it would be a convenient time to change out the tip seal on the TP3 forepump. This one had not been changed since 2018, so as preventative maintence I had JC remove the pump and begin cleaning/installing the new tip seal.
Unfortunately the tip seal broke, but I have ordered another. We should have this pump ready to go late next week. If one is needed sooner, there is a spare IDP 7 pump we can install as the TP3 forepump.
Jordan recieved the new tip seal Friday afternoon and I continued the replacement process in the morning. Finishing up, we proceeded to test the pump in the Clean and Bake room. The pump's pressure lowered to 110 mTorr, and we continue pumping so the seal can recieve a good fitting.
Update: We have confirmed the pump is working great and have reinstalled this back into the vacuum system. Note: The same O-Rings were used.
The NDS2 server on megatron was unresponsive for what i think was the last couple of days.
The NDS the log file (~nds2mgr/logs/nds2-201407151045.log) started reporting "Stage: parser output queue is full." at 2014.7.24 14:47:54 also there are 16 connections still not closed with LindmeierLaptop.cacr.caltech.edu (18.104.22.168) with 15 of them in CLOSE_WAIT.
To identify these zombie sockets we use "netstat -an | grep 31200"
The server was in a condition that /etc/init.d/nds2 stop didn't work and the process had to be manually kill -9'ed and then about 3 or 4 minutes later the zombie sockets were gone at /etc/init.d/nds2 start was used to restart the server.
The LindemejerLaptop was using pynds to get a bunch of channels at once to test drive a streaming visualization code for glitches. It's unclear whether this bumped into a server limitation. We have seen similar states in ldvw that seem to be the result of errors which result in client-server connections not being closed properly, leaving data in an output buffer causing Linux to wait for the other side to empty the buffer.