40m QIL Cryo_Lab CTN SUS_Lab TCS_Lab OMC_Lab CRIME_Lab FEA ENG_Labs OptContFac Mariner WBEEShop
  40m Log, Page 233 of 339  Not logged in ELOG logo
ID Date Author Type Category Subjectup
  11593   Mon Sep 14 10:41:03 2015 SteveUpdateVACTP2 dry pump replaced

TP2 dry fore pump sn PLE10082 was replaced at pressure 717 mTorr,  TP2 50K rpm 0.33A @ 112,677 hrs

Top seal life was 8,160 hrs

Model  SH110, Sn LP1007L556 was installed. It's fore line pressure after 30 minutes of running 38 mTorr, TP2 turbo at 50K rpm 0.18A

 

 

Attachment 1: TP2_dry_pump_replaced.png
TP2_dry_pump_replaced.png
  15602   Wed Sep 23 15:06:54 2020 JordanUpdateVACTP2 Forepump Re-install

I removed the forepump to TP2 this morning after the vacuum failure, and tested in the C&B lab. I pumped down on a small volume 10 times, with no issue. The ultimate pressure was ~30 mtorr.

I re-installed the forepump in the afternoon, and restarted TP2, leaving V4 closed. This will run overnight to test, while TP3 backs TP1.

In order to open V1, with TP3 backing TP1, the interlock system had to be reset since it is expecting TP2 as a backing pump. TP2 is running normally, and pumping of the main volume has resumed.


gautam 2030:

  1. The monitor (LCD display) at the vacuum rack doesn't work - this has been the case since Monday at least. I usually use my laptop to ssh in so I didn't notice it so it could have been busted from before. But for anyone wishing to use the workstation arrangement at 1X8, this is not great. Today, we borrowed the vertex laptop to ssh in, the vertex laptop has since been returned to its nominal location.
  2. The modification to the interlock condition was made by simply commenting out the line requiring V4 to be open for V1 to be opened. I made a copy of the original .yaml file which we can revert to once we go back to the normal config.
  3. I also opened VM1 to allow the RGA scans to continue to be meaningful.
  4. At the time of writing, all systems seem nominal. See Attachment #2. The vertical line indicates when we started pumping on the main volume again earlier today, with TP3 backing TP1.

Unclear why the TP2 foreline pump failed in the first place, it has been running fine for several hours now (although TP2 has no load, since V4 isolates it from the main volume). Koji's plots show that the TP2 foreline pressure did not recover even after the interlock tripped and V4 was closed (i.e. the same conditions as TP2 sees right now).

Attachment 1: Screenshot_from_2020-09-23_15-15-43.png
Screenshot_from_2020-09-23_15-15-43.png
Attachment 2: MainVolPumpDown.png
MainVolPumpDown.png
  15409   Thu Jun 18 15:25:08 2020 JordanUpdateVACTP2 and TP3 Forepump removal

I removed the backing pumps for TP2 and TP3 today to test ultimate pressure and determine if they need a tip seal replacement. This was done with Jon backing me on Zoom. We closed off TP3 and powered down TP3 and the auxilliary pump, in order to remove the forepumps from the exhaust line.

  1. Close V1
  2. Close V5
  3. Turn off TP3
  4. Turn off aux dry pump (manually)
  5. Once the PTP3 foreline pressure has come up to atmosphere, you can disconnect the TP3 dry pump and cap the exhaust line with a KF blank.
  6. Restore the vac configuration in reverse order: dry pump ON, TP3 ON, open V5, open V1

Once pumps were removed I connected a Pirani gauge to the pump directly and pumped down, results as follows:

TP2 Forepump (Agilent IDP 7):

  • Ultimate Pressure: 123 mtorr
  • Hours: 10903

TP3 Forepump (Varian SH 110):

  • Ultimate pressure: ~70 torr
  • Hours: 60300

TP3 forepump defintely needs a new tip seal, and while the pressure on TP2 Forepump was good there was a significant amount of particulate that came out of the exhaust line, so a new tip seal might not be needed but it is recommeded.

  15411   Thu Jun 18 16:56:34 2020 JordanUpdateVACTP2 and TP3 Forepump removal
Quote:

I removed the backing pumps for TP2 and TP3 today to test ultimate pressure and determine if they need a tip seal replacement. This was done with Jon backing me on Zoom. We closed off TP3 and powered down TP3 and the auxilliary pump, in order to remove the forepumps from the exhaust line.

  1. Close V1
  2. Close V5
  3. Turn off TP3
  4. Turn off aux dry pump (manually)
  5. Once the PTP3 foreline pressure has come up to atmosphere, you can disconnect the TP3 dry pump and cap the exhaust line with a KF blank.
  6. Restore the vac configuration in reverse order: dry pump ON, TP3 ON, open V5, open V1

Once pumps were removed I connected a Pirani gauge to the pump directly and pumped down, results as follows:

TP2 Forepump (Agilent IDP 7):

  • Ultimate Pressure: 123 mtorr
  • Hours: 10903

TP3 Forepump (Varian SH 110):

  • Ultimate pressure: ~70 torr
  • Hours: 60300

TP3 forepump defintely needs a new tip seal, and while the pressure on TP2 Forepump was good there was a significant amount of particulate that came out of the exhaust line, so a new tip seal might not be needed but it is recommeded.

I agree with your assessment, Jordan.  If I'm not mistaken the scroll pump for TP2 is new; we had a very early failure with the last new scroll pump (the forepump for TP3) tip seals at just over 5000 hours.  Glad to see my replacement seals held up for over 60K hours. If this is the trend with these pumps, we can simply run them to  around 60000 hours and replace the seals at that time, rather than waiting for failure! - Chub

  8978   Wed Aug 7 15:36:29 2013 SteveUpdateVACTP2 drypump replaced

 

 TP2's fore line - dry pump replaced at performance level 600 mTorr after 10,377 hrs of continuous operation.

Where are the foreline pressure gauges? These values are not on the vac.medm screen.

The new tip seal dry pump lowered the small turbo foreline pressure 10x

TP2fl after 2 day of pumping 65mTorr

Attachment 1: forelinesPs.jpg
forelinesPs.jpg
  9876   Tue Apr 29 16:42:29 2014 SteveUpdateVACTP2 drypump replaced

Quote:

 

 TP2's fore line - dry pump replaced at performance level 600 mTorr after 10,377 hrs of continuous operation.

Where are the foreline pressure gauges? These values are not on the vac.medm screen.

The new tip seal dry pump lowered the small turbo foreline pressure 10x

TP2fl after 2 day of pumping 65mTorr

 TP2 dry pump replaced at fore pump pressure 1 Torr,  TP2 50K_rpm 0.34A

 Top seal life 6,362 hrs

 New seal performance at 1 hr  36 mTorr, 

 Maglev at 560 Hz, cc1 6e-6 Torr

 

Attachment 1: dryforepumpreplaced.png
dryforepumpreplaced.png
  13407   Mon Oct 30 10:09:41 2017 SteveUpdateVACTP2 failed

 IFO pressure 1.2e-5 Torr at 9:30am

Quote:

Valve configuration: Vacuum normal

Note: Tp2 running at 75Krpm 0.25A 26C has a  load high pitch sound today. It's fore line pressure 78 mTorr. Room temp 20C

 

Atm. 1,  This was the vacuum condition  this morning.

               IFO P1  9.7 mTorr, V1 openV4 was in closed position , ~37 C warm Maglev at normal 560Hz rotation speed with foreline pressure 3.9 Torr because V4 closed 2 days ago when TP2 failed .....see Atm.3

               The error messege at TP2  controller was: fault overtemp.

I did the following to restored IFO pumping: stopped pumping of the annulose with TP3 and valves were configured so TP3 can be the forepump of the Maglev.

closed VM1 to protect the RGA,  close PSL shutter .....see Gautam  entry

aux fan on to cool down Maglev-TP1, room temp 20 C,

aux drypump turned on and opend to TP3 foreline to gain pumping speed,

closed PAN to isolate annulos pumping,

opened V7 to pump Maglev forline with TP3 running at 50Krpm, It took 10 minutes to reach P2  1mTorr from 3.9 Torr

aux drypump closed off at P2  1 mTorr, TP3 foreline pressure 362 mTorr.......see Atm.2

As we are running now:

IFO pressure 7e-6 Torr at Hornet cold cathode gauge at 15:50  We have no IFO CC1 logging now.  Annuloses are in 3-5 mTorr range are not pumped.

TP3 as foreline pump of TP1 at 50 Krpm, 0.24 A, 24 C, it's drypump forline pressure 324 mTorr

V4 valve cable is disconnected.

I need help with wiring up the logging of the Hornet cold cathode gauge.

 

 

 

Attachment 1: tp2failed.png
tp2failed.png
Attachment 2: ifo_1.0E-5_Torrit.png
ifo_1.0E-5_Torrit.png
Attachment 3: tp2failed2dago.png
tp2failed2dago.png
Attachment 4: 4days.png
4days.png
  14512   Wed Apr 3 10:42:36 2019 gautamUpdateVACTP2 forepump replaced

Bob and Chub concluded that the drypump that serves as TP2's forepump had failed. Steve had told me the whereabouts of a spare Agilent IDP-7. This was meant to be a replacement for the TP3 foreline pump when it failed, but we decided to swap it in while diagnosing the failed drypump (which had 2182 hours continuous running according to the hour counter). Sure enough, the spare pump spun up and the TP2fl pressure dropped at a rate consistent with what is expected. I was then able to spin up TP1, TP2 and TP3. 

However, when opening V4 (the foreline of TP1 pumped by TP2), I heard a loud repeated click track (~5Hz) from the electronics rack. Shortly after, the interlocks shut down all the TPs again, citing "AC power loss". Something is not right, I leave it to Jon and Chub to investigate.

  14514   Wed Apr 3 16:17:17 2019 JonUpdateVACTP2 forepump replaced

I can't explain the mechanical switching sound Gautam reported. The relay controlling power to the TP2 forepump is housed in the main AC relay box under the arm tube, not in the Acromag chassis, so it can't be from that. I've cycled through the pumpdown sequence several times and can't reproduce the effect. The Acromag switches for TP2 still work fine.

In any case, I've made modifications to the vacuum interlocks that will help with two of the issues:

  1. For the "AC power loss" over-triggering: New logic added requiring the UPS to be out of the "on line power, battery OK" state for ~5 seconds before tripping the interlock. This will prevent electrical transients from triggering an emergency shutdown, as seems to be the case here (the UPS briefly isolates the load to battery during such events).
  2. PSL interlocking: New logic added which directly sets C1:AUX-PSL_ShutterRqst --> 0 (closes the PSL shutter) when the main volume pressure is 3 mtorr-500 torr. Previously there was a channel exposed for this interlock (C1:Vac-interlock_high_voltage), but c1aux was not actually monitoring it. Following the convention of every vac interlock, after the PSL shutter has been closed, it has to be manually reopened. Once the pressure is out of this range, the vac system will stop blocking the shutter from reopening, but it will not perform the reopen action itself. gautam: a separate interlock logic needs to be implemented on c1aux (the shutter machine) that only permits the shutter to be opened if the Vac pressure range is okay. The SUS watchdog style AND logic in the EPICS database file should work just fine.

After finishing this vac work, I began a new pumpdown at ~4:30pm. The pressure fell quickly and has already reached ~1e-5 torr. TP2 current and temp look fine.

Quote:

However, when opening V4 (the foreline of TP1 pumped by TP2), I heard a loud repeated click track (~5Hz) from the electronics rack. Shortly after, the interlocks shut down all the TPs again, citing "AC power loss". Something is not right, I leave it to Jon and Chub to investigate.

Attachment 1: IMG_3180.jpg
IMG_3180.jpg
  15599   Wed Sep 23 08:57:18 2020 gautamUpdateVACTP2 running HOT

The interlocks tripped at ~630am local time. Jordan reported that TP2 was supposedly running at 52 C (!).

V1 was already closed, but TP2 was still running. With him standing by the rack, I remotely exectued the following sequence:

  • VM1 closed (isolates RGA volume).
  • VA6 closed (isolates annuli from being pumped).
  • V7 opened (TP3 now backs TP1, temporarily, until I'm in the lab to check things out further).
  • TP2 turned off.

Jordan confirmed (by hand) that TP2 was indeed hot and this is not just some serial readback issue. I'll do the forensics later.

Attachment 1: Screen_Shot_2020-09-23_at_8.55.39_AM.png
Screen_Shot_2020-09-23_at_8.55.39_AM.png
  15600   Wed Sep 23 10:06:52 2020 KojiUpdateVACTP2 running HOT

Here is the timeline. This suggests TP2 backing RP failure.

1st line: TP2 foreline pressure went up. Accordingly TP2 P, current, voltage, and temp went up. TP2 rotation went down.

2nd line: TP2 temp triggered the interlock. TP2 foreline pressure was still high (10torr) so TP2 struggled and was running at 1 torr.

3rd line: Gautam's operation. TP2 was isolated and stopped.

Between the 1st line and 2nd line, TP2 pressue (=TP1 foreline pressure) went up to 1torr. This made TP1 current increased from 0.55A to 0.68A (not shown in the plot), but TP1 rotation was not affected.

Attachment 1: Screen_Shot_2020-09-23_at_10.00.43.png
Screen_Shot_2020-09-23_at_10.00.43.png
  5887   Mon Nov 14 15:46:14 2011 steveUpdateVACTP2's forepump changed

The foreline pressure of TP2 was 1.4 Torr this morning. This drypump worked well for ten months.

Recently rebuilt  drypump with new seal was swapped in.

This is how you do it: close  V1,  V4 and turn off TP2. Replace drypump and start up TP2

Set pump speed to 50 K rpm and open V4 to TP1 Note that the Maglev was not turned off  because V4 was closed off only 5-10 minutes.

Open V1 the status is Vac Normal.

TP2 is rotating at  50K rpm, current pick up 0.2A,  the temp is 26C and its foreline pressure 33 mTorr

Attachment 1: drypump.png
drypump.png
  4654   Fri May 6 15:59:40 2011 steveUpdateVACTP3 fore pump replaced

Dry fore pump of TP3 replaced by brand new Varian SH-110  at  1.1 Torr_75,208 hrs

The annuloses were closed off for 25 minutes. We are back to VACUUM NORMAL mode.

The TP3 fore line pressure dropped to 44 mTorr at 25 minutes in operation.......9.4mTorr at day 2 with full annulos load

  15591   Mon Sep 21 15:57:08 2020 JordanUpdateVACTP3 Forepump Replacement and Vac reset

I removed the forepump (Varian SH-110) for TP3 today to see why it had failed over the weekend. I tested it in the C&B lab and the ultimate pressure was only ~40torr. I checked the tip seals and they were destroyed. The scroll housing also easily pulled off of the motor drive shaft, which is indicative of bad bearings. The excess travel in the bearings likely led to significant increase in tip seal wear. This pump will need to be scrapped, or rebuilt.

I tested the spare Varian SH-110 pump located at the X-end and the ultimate pressure was ~98 mtorr. This pump had tip seals replaced on 11/5/18, and is currently at 55163 operating hours. It has been installed as the TP3 forepump.

Once installed, restarting the pump line occured as follows: V5 Closed, VA6 closed, VASE Closed, VASV closed, VABSSCI closed, VABS closed, VABSSCO closed, VAEV closed, VAEE closed,TP3 was restarted and once at normal operation, valves were opened in same order.

The pressure differential interlock condition for V5 was temporaily changed to 10 torr (by Gautam), so that valves could be opened in a controlled manner. Once, the vacuum system was back to normal state the V5 interlock condition was set back to the nominal 1 torr. Vacuum system is now running normally.

Attachment 1: Screenshot_from_2020-09-21_15-56-04.png
Screenshot_from_2020-09-21_15-56-04.png
Attachment 2: 20200921_145043.jpg
20200921_145043.jpg
Attachment 3: 20200921_145040.jpg
20200921_145040.jpg
  16806   Fri Apr 22 14:14:33 2022 JordanUpdateVACTP3 Forepump tip seal replacement

Jordan, JC

While the pumpspool is vented, I thought it would be a convenient time to change out the tip seal on the TP3 forepump. This one had not been changed since 2018, so as preventative maintence I had JC remove the pump and begin cleaning/installing the new tip seal.

Unfortunately the tip seal broke, but I have ordered another. We should have this pump ready to go late next week. If one is needed sooner, there is a spare IDP 7 pump we can install as the TP3 forepump.

Attachment 1: IMG_0572.jpeg
IMG_0572.jpeg
Attachment 2: IMG_0573.jpeg
IMG_0573.jpeg
  16822   Mon May 2 10:00:34 2022 JordanUpdateVACTP3 Forepump tip seal replacement

[JC, Jordan]

Jordan recieved the new tip seal Friday afternoon and I continued the replacement process in the morning. Finishing up, we proceeded to test the pump in the Clean and Bake room. The pump's pressure lowered to 110 mTorr, and we continue pumping so the seal can recieve a good fitting.

Update: We have confirmed the pump is working great and have reinstalled this back into the vacuum system.                                                                                                                   Note: The same O-Rings were used.

Quote:

Jordan, JC

While the pumpspool is vented, I thought it would be a convenient time to change out the tip seal on the TP3 forepump. This one had not been changed since 2018, so as preventative maintence I had JC remove the pump and begin cleaning/installing the new tip seal.

Unfortunately the tip seal broke, but I have ordered another. We should have this pump ready to go late next week. If one is needed sooner, there is a spare IDP 7 pump we can install as the TP3 forepump.

 

  15582   Sat Sep 19 18:07:35 2020 KojiUpdateVACTP3 RP failure

I came to the campus and Gautam notified that he just had received the alert from the vac watchdog.

I checked the vac status at c1vac. PTP3 went up to 10 torr-ish and this made the diff pressure for TP3 over 1torr. Then the watchdog kicked in.

To check the TP3 functionality, AUX RP was turned on and the manual valve (MV in the figure) was opened to pump the foreline of TP3. This easily made PTP3 <0.2 torr and TP3 happy (I didn't try to open V5 though).

So the conclusion is that RP for TP3 has failed. Presumably, the tip-seal needs to be replaced.

Right now TP3 was turned off and is ready for the tip-seal replacement. V5 was closed since the watchdog tripped.

Attachment 1: vac.png
vac.png
Attachment 2: Screen_Shot_2020-09-19_at_17.52.40.png
Screen_Shot_2020-09-19_at_17.52.40.png
  15586   Sat Sep 19 19:37:16 2020 not KojiUpdateVACTP3 RP failure

Disconcerting because those tip seals were just replaced [15417]. Maybe they were just defective, but if there is a more serious problem with the pump, there is a spare Varian roughing pump (the old TP2 dry pump) sitting at the X-end.

I reset the interlock error to unfreeze the vac controls (leaving V5 closed).

Quote:

So the conclusion is that RP for TP3 has failed. Presumably, the tip-seal needs to be replaced.

Right now TP3 was turned off and is ready for the tip-seal replacement. V5 was closed since the watchdog tripped.

  15140   Wed Jan 22 16:19:47 2020 JonUpdateVACTP3 controller errors

Gautam and I debugged a communications problem with TP3 that was causing its python service to fail. We traced the problem back to the querying of the pump controller for its operational parameters (speed, voltage, temp). Some small percentage of the time (~5%, indeterministically), the pump controller is returning an invalid response which causes the service to shut itself down and signal a NO COMM error.

As a temporary fix, I wrapped the failing query in an exception handler to continue past this particular error. However, we suspect the microprocessor in the TP3 controller may be beginning to fail. There is a spare controller sitting right next to it in the vacuum rack. We will ask Chub to install the replacement in the near future.

gautam: this pump is responsible for pumping the annular volume under normal operations. while this problem is being resolved, the annular volume is valved off (as it has been since July 2019 anyway which is when this problem first manifested).

  13423   Fri Nov 10 08:52:21 2017 SteveUpdateVACTP3 drypump replaced

PSL shutter closed at 6e-6 Torr-it    

   The foreline pressure of the drypump is 850 mTorr at 8,446 hrs of seal life

V1 will be closed for ~20 minutes for drypump replacement..........

9:30am dry pump replaced, PSL shutter opened at 7.7E-6 Torr-it

  Valve configuration: vacuum normal as  TP3 is the forepump of the Maglev  & annuloses are not pumped.

Quote:

TP3 drypump replaced at 655 mTorr, no load, tp3 0.3A 

This seal lasted only for 33 days at  123,840 hrs

The replacement is performing well: TP3 foreline pressure is 55 mTorr, no load, tp3 0.15A at 15 min  [ 13.1 mTorr at d5 ]

 

Valve configuration: Vacuum Normal, ITcc 8.5E-6 Torr

Quote:

Dry pump of TP3 replaced after 9.5 months of operation.[ 45 mTorr d3 ]

The annulosses are pumped.

Valve configuration: vac normal, IFO pressure 4.5E-5 Torr [1.6E-5 Torr d3 ] on new ITcc gauge, RGA is not installed yet.

Note how fast the pressure is dropping when the vent is short.

Quote:

IFO pressure 1.7E-4 Torr on new not logged cold cathode gauge. P1 <7E-4 Torr

Valve configuration: vac.normal with anunulossess closed off.

TP3 was turned off with a failing drypump. It will be replaced tomorrow.

All time stamps are blank on the MEDM screens.

 

 

  13637   Fri Feb 16 15:57:58 2018 SteveUpdateVACTP3 drypump replaced

The forline pressure of TP3 was 399 mTorr

It was replaced this morning at  TP3 controller 134,638hrs with the "failed TP2 station" drypump. The foreline pressure now at TP3 is 100 mTorr at 6 hrs of operation.[ at day 3  63 mT ]

IFO pressure at CC Hornet 7.9e - 6 Torr

Valve configuration: vacuum normal as TP3 is the forepump of the Maglev & the annuloses are not pumped

Quote:

PSL shutter closed at 6e-6 Torr-it    

   The foreline pressure of the drypump is 850 mTorr at 8,446 hrs of seal life

V1 will be closed for ~20 minutes for drypump replacement..........

9:30am dry pump replaced, PSL shutter opened at 7.7E-6 Torr-it

  Valve configuration: vacuum normal as  TP3 is the forepump of the Maglev  & annuloses are not pumped.

Quote:

TP3 drypump replaced at 655 mTorr, no load, tp3 0.3A 

This seal lasted only for 33 days at  123,840 hrs

The replacement is performing well: TP3 foreline pressure is 55 mTorr, no load, tp3 0.15A at 15 min  [ 13.1 mTorr at d5 ]

 

Valve configuration: Vacuum Normal, ITcc 8.5E-6 Torr

Quote:

Dry pump of TP3 replaced after 9.5 months of operation.[ 45 mTorr d3 ]

The annulosses are pumped.

Valve configuration: vac normal, IFO pressure 4.5E-5 Torr [1.6E-5 Torr d3 ] on new ITcc gauge, RGA is not installed yet.

Note how fast the pressure is dropping when the vent is short.

Quote:

IFO pressure 1.7E-4 Torr on new not logged cold cathode gauge. P1 <7E-4 Torr

Valve configuration: vac.normal with anunulossess closed off.

TP3 was turned off with a failing drypump. It will be replaced tomorrow.

All time stamps are blank on the MEDM screens.

 

 

 

  13454   Sun Nov 26 19:38:40 2017 SteveUpdateVACTP3 drypump replaced again

The TP3 foreline pressure was 4.8 Torr, 50K rpm 0.54A and 31C........Maglev rotation normal 560 Hz.......    IFO pressure 7.2e- 6 Torrit was not effected

V1 closed ......replaced drypump.........V1 opened

IFO 6.9e-6 Torrit at 19:55, TP3fl 18 mT,  50Krpm 0.15A 24C

VM1 is still closed

 

Attachment 1: after_replacement.png
after_replacement.png
  10838   Tue Dec 23 15:37:32 2014 SteveUpdateVACTP3 drypump replaced

Quote:

Quote:

 

 TP2's fore line - dry pump replaced at performance level 600 mTorr after 10,377 hrs of continuous operation.

Where are the foreline pressure gauges? These values are not on the vac.medm screen.

The new tip seal dry pump lowered the small turbo foreline pressure 10x

TP2fl after 2 day of pumping 65mTorr

 TP2 dry pump replaced at fore pump pressure 1 Torr,  TP2 50K_rpm 0.34A

 Top seal life 6,362 hrs

 New seal performance at 1 hr  36 mTorr, 

 Maglev at 560 Hz, cc1 6e-6 Torr

 

 TP3 dry pump  replaced at 540 mT as TP3 50K_rpm 0.3A with annulos load. It's top seal life time was 11,252 hrs

 

 

  12636   Wed Nov 23 11:32:13 2016 SteveUpdateVACTP3 drypump replaced again

TP3 drypump replaced at 655 mTorr, no load, tp3 0.3A 

This seal lasted only for 33 days at  123,840 hrs

The replacement is performing well: TP3 foreline pressure is 55 mTorr, no load, tp3 0.15A at 15 min  [ 13.1 mTorr at d5 ]

 

Valve configuration: Vacuum Normal, ITcc 8.5E-6 Torr

Quote:

Dry pump of TP3 replaced after 9.5 months of operation.[ 45 mTorr d3 ]

The annulosses are pumped.

Valve configuration: vac normal, IFO pressure 4.5E-5 Torr [1.6E-5 Torr d3 ] on new ITcc gauge, RGA is not installed yet.

Note how fast the pressure is dropping when the vent is short.

Quote:

IFO pressure 1.7E-4 Torr on new not logged cold cathode gauge. P1 <7E-4 Torr

Valve configuration: vac.normal with anunulossess closed off.

TP3 was turned off with a failing drypump. It will be replaced tomorrow.

All time stamps are blank on the MEDM screens.

 

  2963   Fri May 21 00:30:44 2010 JenneUpdateVACTP3 fore pump is very loud

[Jenne, Kiwamu, and Steve via phone]

Around 9:30pm, Kiwamu and I came back from dinner, and were getting ready to begin the beam scan measurements.  I noticed that one of the vacuum pumps was being very loud.  Kiwamu noted that it is the fore pump for TP3's turbo, which he and Steve replaced in January (elog 2538).  We had not noticed these noises before leaving for dinner, around 8pm.

We called Steve at home, and he could hear the noise through the phone. He said that even though it was really loud, since it was reading 3.3mTorr (on the display of the controller, in the vacuum rack just above head-height) which is close to the nominal value, it should be fine to leave.  He will check it out in the morning.  If it had been reading at or above ~1Torr, that's indicative of it being really bad, and we would have needed to shut it off.

For future reference, in case we need to turn it off, Steve said to use the following procedure:

1. Close VM3, to isolate the RGA, which is what this pump is currently (while we're at atmosphere) pumping on.  I don't know if there are other things which would need to be shut at this stage, if we were at vacuum nominal.

2. Close VM5, which is right in front of TP3, so TP3's pump is just pumping on itself.

3. Push the "Stop" button on the Turbo controller for TP3, in the vacuum rack, about waist level.  Turning off the turbo will also turn off the fore pump.

UPDATE, 1am: The controller in the rack is reading 3.1mTorr, so the pump, while still noisy, still seems to be working.

  14517   Fri Apr 5 01:10:18 2019 gautamUpdateVACTP3 forepump is also noisy

Is this one close to failure as well?

  2965   Fri May 21 08:28:29 2010 steveConfigurationVACTP3 turned off

Quote:

[Jenne, Kiwamu, and Steve via phone]

Around 9:30pm, Kiwamu and I came back from dinner, and were getting ready to begin the beam scan measurements.  I noticed that one of the vacuum pumps was being very loud.  Kiwamu noted that it is the fore pump for TP3's turbo, which he and Steve replaced in January (elog 2538).  We had not noticed these noises before leaving for dinner, around 8pm.

We called Steve at home, and he could hear the noise through the phone. He said that even though it was really loud, since it was reading 3.3mTorr (on the display of the controller, in the vacuum rack just above head-height) which is close to the nominal value, it should be fine to leave.  He will check it out in the morning.  If it had been reading at or above ~1Torr, that's indicative of it being really bad, and we would have needed to shut it off.

For future reference, in case we need to turn it off, Steve said to use the following procedure:

1. Close VM3, to isolate the RGA, which is what this pump is currently (while we're at atmosphere) pumping on.  I don't know if there are other things which would need to be shut at this stage, if we were at vacuum nominal.

2. Close VM5, which is right in front of TP3, so TP3's pump is just pumping on itself.

3. Push the "Stop" button on the Turbo controller for TP3, in the vacuum rack, about waist level.  Turning off the turbo will also turn off the fore pump.

UPDATE, 1am: The controller in the rack is reading 3.1mTorr, so the pump, while still noisy, still seems to be working.

 The foreline pressure of TP3 is 2.9 mTorr  The drypump is loosing it's bearing and it is very noisy.

V3, V5 closed and TP3 small turbo controller off. This turned off the noisy forepump that has to be replaced.

RGA is running at cc4 2e-6 Torr

The RGA was turned off at cc4 1e-5 Torr

  9120   Tue Sep 10 15:43:01 2013 SteveUpdateVACTP3' dry pump is replaced

Quote:

 

 TP3 foreline's dry pump is getting noisier and noisier.  Turbo TP3 is pumping on the annulos. The foreline pressure is 7.2 mTorr and it is not degrading. It was swapped in March 5, 2013

The seal is very good, but the bearing is dying.

 

 

 

 The drypump is replaced at 95,781 hrs on TP3 controller time. The foreline pressure is 30 mTorr and dropping.

 It is 13 mTorr after 17 hours of pumping.

  1374   Mon Mar 9 12:04:18 2009 YoichiUpdateComputersTPs and AWG are back
I had to do one more reboot of tpman and daqd to get the TPs working.
I confirmed the alignment scripts run fine.

Now the oplevs of some optics are largely mis-centered. Alberto and I will center them after lunch.
  13920   Wed Jun 6 14:36:15 2018 gautamUpdateLSCTRX clipping

For some time now, I've been puzzled by the unreliability of the ASS_X dither alignment servo. Leaving the servo on, TRX often begins to decay to a lower value, and even after freezing the dither at the maximum TRX values, I can manually align the mirrors to increase TRX. We have suspected some kind of clipping in the TRX path that is responsible for this behaviour. Today I decided to investigate this a bit further. To have the arm locked and to inspect the beam, we have to change the locking trigger - TRX is what is normally used, but I misaligned the Y arm completely, and used AS110 as a trigger instead. There is some strangeness in the triggering topology, but this deserves a separate elog.

Once the arm was locked (and relocks using the AS110 trigger in the event of an unlock), I was able to trace the beampath on the EX table with an IR card. The TRX beam is rather large and weak, so it is hard to see, but as best as I can tell, the only real danger of clipping (or perhaps the beam is already clipped) is on the final steering mirror before the beam hits the (Thorlabs) PD. Steve/Pooja are working on getting a photo of this, and will upload it here shortly. Options to mitigate this:

  1. Use the harmonic separator to steer the beam lower, and center it on the 1" steering mirror. However, this could possibly lead to clipping on some of the upstream lenses.
  2. Raise the height of the 1" steering mirror by 0.25". However, this would require a custom 3/4" dia post height or some shims, which I am not sure is in line with our optomechanic mounting practises.
  3. Use a 2" mirror instead of a 1" mirror.

The EX QPD has stopped working since the Acromag install. If it were working, we wouldn't have to rely on the alternate triggering with AS110 and instead just use the QPD as TRX, while we debug the Thorlabs PD path.

  13927   Thu Jun 7 16:15:03 2018 gautamUpdateLSCTRX clipping

I opted for the quickest fix - I raised the height of the offending steering mirror using a 0.25" shim. In the long term, we can get a taller post machined. After raising the mirror height, I then checked the DC centering of the spot on the DC PD using a scope.

Looking at the performance of the X arm ASS, I no longer see the strange oscillatory behaviour I described in my previous post yes. Moreover, the TRX level was ~1 before be raising the steering mirror - but it is now ~1.2. So we were certainly losing some power.

  1878   Mon Aug 10 17:27:47 2009 robConfigurationLSCTRX, TRY gain

 

These are the settings which determine the transmon (eg, TRX) amplitude, and which are updated by the matchTransMon scripts.

For the X arm

 

op440m:AutoDither>tdsread C1:LSC-TRX_GAIN C1:LSC-LA_PARAM_FLT_01 C1:LSC-LA_PARAM_FLT_00
0.0023
0.155
119.775

 

For the Y arm

op440m:AutoDither>tdsread C1:LSC-TRY_GAIN C1:LSC-LA_PARAM_FLT_04 C1:LSC-LA_PARAM_FLT_03
0.00237196
-0.116
19.9809


  10862   Wed Jan 7 03:04:13 2015 JenneUpdateLSCTRY (thorlabs pd) weird noise

[Jenne, Diego, Rana]

This is a note about work done last night. 

We were starting to lock, and saw glitches in the Thorlabs TRY PD about once every 1/60th of a second.  It is not a sine wave, so it is not 60Hz line noise directly.  It looks like this:

TRY_60Hz_peaks_5Jan2015.pdf

Rana pointed out that this looks like it could be from a power supply that is converting AC to DC. 

We went down to the Yend, and noticed some weird symptoms.  So far, we do not know where the noise is coming from.  Rather, we are just using the QPD for locking.

* The noise comes and goes, particularly if someone is moving around at the end station.

* Moving the Thorlabs power supply farther from the HeNe power supply didn't do much.  Turning off and disconnecting the HeNe supply didn't make the noise go away, so we conclude that it is not the HeNe's fault.

* We suspected the loops of excess cable that were sitting on top of iscey, but moving the coils away from the computer did not make the noise go away.

* We removed a few disconnected BNC cables that were near or touching the end table, but that didn't fix things.

* We disconnected the PD's signal cable and pulled it out of the table enclosure, and then put it back.  Noise was gone when cable was disconnected (good), but it was back after plugging the cable back in.

* The noise still comes and goes, but we don't have to use the Thorlabs PD for locking, so we leave it for another day.

RXA: also moved the Thorlabs power supply to a different power strip and tried putting it closer/farther to the Uniblitz shutter controller. Another suspect is that its some PWM type noise from the doubler crystal temperature driver. Need to try turning off the heater and the Raspberry PI to if it effects the noise.

  14459   Fri Feb 15 18:42:57 2019 gautamUpdateLSCTRY 60 Hz solved

A more permanent fix than a crocodile clip was implemented. Should probably look to do this for the X end unit as well.

Attachment 1: IMG_7323.JPG
IMG_7323.JPG
  14446   Mon Feb 11 15:41:49 2019 gautamUpdateLSCTRY 60 Hz solved - but clipping persists

Rich came by the 40m to photocopy some pages from Hobbs, and saw me working on the 60 Hz hunting. As I suspected, the problem was being generated in the D040060. This board receives the photodiode signal single-ended, but has a different power ground than the photodiode (even though the PD is plugged into a power strip that claims to come from 1Y4). The mechanism is not entirely clear - the presence of these 60 Hz features seemed to be dependent on the light level on the TRY photodiode (i.e. they were absent when the PSL shutter is closed, and were more prominent when TRY was 0.9 rather than 0.5) but the PD certainly wasn't saturated - the DC signal was only ~100 mV when viewed on a scope. In any case, Rich suggested the simplest test would be to ground the BNC shield bringing TRY to the rack, to the local ground on the board, which I did using a crocodile clip. This did the trick, the TRY signal RMS is now dominated by the ~1 Hz seismic-driven variation.

 On a more pessimistic note - it looks like the elliptical reflector moving did not work, and the clipping in the Y arm persists no. I am able to recover TRY~1 with the yaw offset on the ETM (which is still lower than the 1.06-1.07 Koji reported in Aug 2018, but I can believe that being down to the MC transmission being a few % lower at 15000cts rather than 15500), while the maximum I see without it is ~0.9. This is puzzling, because when the chamber was open, we saw that there was ~1.5" clearance between the edge of the reflector and the beam on an IR card. I suppose the input pointing could have been off by a small amount. So one of the primary vent objectives wasn't acheieved... But I will push ahead with the loss measurement.

  9838   Tue Apr 22 01:11:42 2014 JenneUpdateLSCTRY 60Hz noise

Quote:

 

P.S. I realigned the Y green to the arm and brought GTRY to 0.93

This evening, I was not able to successfully transition CARM from ALS to 1/sqrt(trans) signals.  The TRY time series looked odd, so I took a spectra, and we have huge 60Hz noise in TRY. 

I found a lock stretch from around 6:30pm that did not show the 60Hz noise, and then there was a lock stretch around 8pm that did have the noise.  So, something happened at the Yend between 6:30 and 8pm tonight.

Asking around, this was the time frame in which Manasa was down at the Yend to realign the green beam, and to check cabling for the PZT_OUT and ERR_MON signals to the ADC.

Looking at the spectra, Rana noted that we have even as well as odd harmonics of the 60Hz line, which is unusual.

TRY_60Hz_noise_21Apr2014.pdf

To try to diagnose the problem, Rana and I tried to make sure no cables' connectors were touching, and that no equipment was plugged in that shouldn't be.  We noticed that none of: the shutter, the Thorlabs TRY PD, or the QPD TRY are isolated from the table.  To see if perhaps the shutter was the problem, I turned off the power to the Yend green shutter, and unplugged the cable.  The cable is laying on the table, with the connector sitting on a piece of plastic to isolate it.  Removing the shutter from the system did not change anything. 

We don't see the 60Hz noise in the Xarm, so it's not on the laser light itself.  Also, we don't see the 60Hz lines in the Yarm feedback signal, so we're not putting the lines onto the mirror, and thus onto the Yarm's light. 

Manasa, can you please take a look, and see if you can figure out what is going on?  We need TRY so that we can transition to 1/sqrt(trans) signals for CARM.  Thanks!!

  9841   Tue Apr 22 21:54:50 2014 manasaUpdateLSCTRY 60Hz noise

Quote:

Quote:

 

P.S. I realigned the Y green to the arm and brought GTRY to 0.93

This evening, I was not able to successfully transition CARM from ALS to 1/sqrt(trans) signals.  The TRY time series looked odd, so I took a spectra, and we have huge 60Hz noise in TRY. 

Manasa, can you please take a look, and see if you can figure out what is going on?  We need TRY so that we can transition to 1/sqrt(trans) signals for CARM.  Thanks!!

 I went to the Y end to look at the TRY 60Hz noise situation this morning. While looking at TRY noise on dtt, I found that just lifting the cable away from the cable bunch that runs out of the table suppressed the noise drastically. 

Attachment 1

I removed the unwanted bnc connector in the path of the already long TRY cable running from the PD to the 1Y4 rack and isolated it from the bunch. TRY became less noisy.

But the noise was back again earlier in the evening and it looks like the noise is very much related to the TRY cable. TRY cable might have moved from its sweet spot while I was around checking cable connections yesterday.

I couldn't find a spare to replace it right away today (We need a BNC to 4 pin lemo).

Attachment 1: 60HzTRY.jpg
60HzTRY.jpg
  9842   Tue Apr 22 22:49:10 2014 ranaUpdateLSCTRY 60Hz noise

 

 The detectors and electronics on this table are not properly isolated. To reduce the 60 Hz and ground loops, photodiodes and shutter must be isolated by using plastic spacers as we usually do elsewhere - this table just seems to have a few oversights.

Steve can start assembling all of the pieces to do this in the morning and then we can start the swapping after the meeting.

The high gain Transmon cable should be a regular BNC. There's no need for 4-pin LEMO in this usage, so the best move is to modify the board and replace the 4-pin LEMO connector with an isolated panel mount BNC female.

The AC adapter for this diode (and all of the detectors on the table) should get their power from a power strip which gets plugged into the rack with the whitening boards. The SHG oven, the Uniblitz shutter, and any cameras can get their power from another power strip if needed/wanted.

  9843   Wed Apr 23 19:58:00 2014 manasaUpdateLSCTRY 60Hz noise

[Steve, Manasa]

To find noise source

1. Swapped the power cable of the PD and checked that it is connected to the right power source.

2. Changed the aluminium base of the post holding the diode so that the diode is floating

3. Grounded the table and the rack

4. Routed the cable on the other side of the beam tube to isolate it from other cables.

After all the above, we still found that shaking the cable was making TRY noisy.

I pulled out the PD whitening board to replace the 4 pin lemo connector with a bnc connector so that we can swap the cable with a new one. So there is no TRY right now.

 

  9918   Tue May 6 18:32:14 2014 steveUpdateLSCTRY 60Hz noise hunt

This is an effort to get rid of our ground loops  by isolating the electronic components from the optical table.

Aluminum mounting base plates of Thorlabs BA2 and Newport B-2 were replaced by plates or post made out of delrin material.

This is an insulator. DELRIN base plates were installed 6 places. The oplev-qpd has Nylon base plate.

The NPRO and HE/NE lasers are not isolated from the table. S8 and S9

I'm not sure about the doubling oven S10 

The optical table is grounded at G11  through  ~1 Mohms to the ETMY chamber.

Alignment touch up needed   at all D-marked component!

 

Attachment 1: ETMY-ISCT_EISOL.jpg
ETMY-ISCT_EISOL.jpg
  9921   Wed May 7 14:01:36 2014 steveUpdateLSCTRY 60Hz noise hunt

Quote:

This is an effort to get rid of our ground loops  by isolating the electronic components from the optical table.

Aluminum mounting base plates of Thorlabs BA2 and Newport B-2 were replaced by plates or post made out of delrin material.

This is an insulator. DELRIN base plates were installed 6 places. The oplev-qpd has Nylon base plate.

The NPRO and HE/NE lasers are not isolated from the table. S8 and S9

I'm not sure about the doubling oven S10 

The optical table is grounded at G11  through  ~1 Mohms to the ETMY chamber.

Alignment touch up needed   at all D-marked component!

 

 Attachment appendix:

 

D: component delrin isolated

N: component nylon isolated ( or Delrin )

S: component shell is shorting to optical table (except oven)

G:  optical table ground

 

I failed to maximize TRY the pds.

  9933   Thu May 8 17:25:17 2014 steveUpdateLSCTRY 60Hz noise hunt

 I worked at the ETMY-ISCT this morning and late afternoon.  I will continue the 60 Hz noise hunt tomorrow. 

  8885   Fri Jul 19 23:35:52 2013 KojiSummaryLSCTRY DC locking

The Y arm was locked with the TRY DC signal.

The handing off process is too complicated because there is no path from ALS to the LSC error.


 The TRY DC error signal & the gain determination

- The error signal was produced by the operation 1/SQRT(TRY) - OFFSET. The initial offset was -5.

- The sign of the TRY DC error signal depends on which side of the resonance the arm is.
  By looking at the strip chart, I determined that the sign is opposite of the ALS.
  The ALS had the gain of -25, so the TRY control gain was to be positive.

- From the strip chart on the previous entry , the slope difference between the PDH error and the TRY DC error was x500.
  The arm control with POY11 PDH had the gain of 0.2. So the target gain for the TRY DC was determined to be +100.

Handing off

- The arm was stabilized by ALS. The ALS gain was -25 with FM2/3/5/6/7/10

- YARM configuration: no trigger / no FM trigger / gain =+0 / FM5 ON / OFFSET -5

- Start handing off:
  YARM: Turned up the gain to +50

- ALS: Turned off FM6/7

- YARM: Turned on FM6/7

- ALS: Turned off FM2

- YARM: Turned on FM4

- ALS: Turned off FM3/10

- YARM: Turned on FM2/3/8/9 ON

- ALS: Reduced the gain to -15

- YARM: Increased the gain to +70

- ALS: Reduced the gain to 0

- YARM: Increased the gain to +100

HANDING OFF - DONE

Changing the offset

The offset of -5 gave the TRY of <0.1.

The detuning was reduced by giving the offset of -4. TRY went up to ~.1

The offset of -3 made TRY 0.13

The offset of -2 made TRY 0.25

The offset of -1.5 made TRY 0.4. And the arm could not be held by this error signal anymore.

 

 

Attachment 1: TRY_locking.png
TRY_locking.png
  10395   Thu Aug 14 22:31:12 2014 JenneUpdateLSCTRY gets mystery offset

I don't know why, but TRY has somehow gotten a 0.3 count offset in the last hour. 

Rana and I are witnesses for each other that neither of us has gone into the IFO room in the last several hours (and we're the only ones here).  For some reason though, the TRY PD now has a 0.3 count offset.  We have been doing some ALS locks, but we have not run the offset script in the last several hours.  Closing the green shutter doesn't change things, and we still see the offset when the MC loses lock, so it's not to do with the end or the PSL laser.  We haven't been in there, so there hasn't been a change in the room lights. 

TRY_0pt3_offset.pdf

  8438   Thu Apr 11 02:00:21 2013 JenneUpdateLockingTRY gone???

TRY signals are all gone!  Both the PD and the camera show no signal.  I went down there to turn off the lights, and look to see what was up, and I don't see any obvious things blocking the beam path on the table.  However, Steve has experimentally bungeed the lids down, so I didn't open the box to really look to see what the story is.

Absent TRY, I redid the IFO alignment.  Yarm locked, so I assumed it was close enough.  I redid Xarm alignment pretty significantly.  Transmission was ~0.5, which I got up to ~0.85 (which isn't too bad, since the PMC transmission is 0.74 instead of the usual 0.83).  I then aligned MICH, and PRM.  After fixing up the BS alignment, the POP beam wasn't hitting the POP PD in the center any more.  I centered the beam on the PD, although as Gabriele pointed out to me a week or two ago, we really need to put a lens in front of POP, since the beam is so big.  We're never getting the full beam when the cavity flashes, which is not so good.

Den is still working on locking, so I'll let him write the main locking report for the night.

We see that the PRC carrier lock seems to be more stable when we lock MICH with +1 for ITMY and -1 for ITMX, and PRCL with -1 for both ITMs.  This indicates that we need to revisit the systematic problem with using the PRM oplev to balance the coils, since that oplev has a relatively wide opening angle.  I am working on how to do this.

  10401   Fri Aug 15 14:09:21 2014 JenneUpdateLSCTRY mystery offset gone

Again unknown, but about 6 hours ago (so ~8am) the offset disappeared. 

Here's a 1-day trend:

TRY_0pt3_offset_gone.pdf

  10402   Fri Aug 15 14:35:57 2014 ericqUpdateLSCTRY mystery offset gone

One question answered, but another raised. The offset came from LSC-TRY switching to the ETMY-QPD signal from ETMY-TRY (Hi gain pd). 

BUT WHY

TRYmystery.png

  5654   Wed Oct 12 00:35:42 2011 kiwamuUpdateLSCTRY path realigned

The TRY (TRansmitted light from Y arm ) path was a bit realigned because there had been a small clipping.

This clipping was introducing offsets on the error signals of the C1ASS servo.

(Story)

During I was running the C1ASS servo on the Y arm I found every time after the auto-alignment is done there still remained a slight offset in the beam pointing,

I looked at the CCD camera which looks at the transmitted light and then introduced an intentional misalignment in ETMY in order to find an obvious clipping.

Indeed there was a clipping in horizontal direction. I checked through the optics on the Y end optical bench.

On the second mirror (beam splitter) the beam was on a very edge. So I steered the first steering mirror to fix it,

In addition to that an iris which is placed between the first and second mirror was also clipping the beam,

So I fully opened the aperture of the iris.

  4656   Sat May 7 02:52:53 2011 kiwamuUpdateLSCTRY photo diode installed

[Suresh / Kiwamu]

 We installed the TRY photo diode (Thorlabs one) and the ETMYT CCD camera in place on the ETMY table.

Now we can see a signal on the TRY digital channel.

It will be quite useful for the Y arm locking, for instance we can do a triggered locking and the maximization of the intracavity power.

Someone has to install the EMTY trans QPD at some point.

  8568   Tue May 14 01:13:35 2013 JenneUpdate40m UpgradingTRY realigned

Koji noticed that earlier this afternoon the Yarm ASS was working, but then after dinner it was no longer working.  I saw that the ETMY trans camera beam was clipped.  These things precipitated a visit to the Yend station. 

I saw that the beam on the optic that steers the camera beam to the camera was very, very low, almost falling off the optic.  The only mirror which steers to this optic is the harmonic separator which reflects the IR, and transmits the green.  I turned the pitch knobs on the harmonic separator until the beam was roughly centered on all 3 optics between the separator and the camera (BS to QPD, BS to TRYDC and Y1 for camera).  The yaw was fine, so I didn't touch it.

I then adjusted the steering mirror to the camera, and the BS pointing to the DC PD.  I have not touched the BS pointing to the QPD.  Once the beam was on the TRY PD, Koji ran the ASS script, and I recentered the beam on the DC PD.  During this time, Koji had the Yarm triggering using -1 in the POYDC element of the matrix.

The harmonic separator is not mounted in a nice way (I'm assuming that Annalisa is in the middle of things, and she'll get back to it after the green work), so the TRY PD and camera will need to be aligned again, so I didn't do any ASS-recentering-ASS iteration tonight.

The Yarm ASS works nicely again, getting TRY to ~0.89 . 

ELOG V3.1.3-