40m QIL Cryo_Lab CTN SUS_Lab TCS_Lab OMC_Lab CRIME_Lab FEA ENG_Labs OptContFac Mariner WBEEShop
  40m Log, Page 234 of 341  Not logged in ELOG logo
ID Date Author Type Category Subjectup
  356   Tue Mar 4 19:14:09 2008 ranaConfigurationCDSTDS & SVN
Matt, Rob, Rana

Today we added the TDS software to the 40m SVN repo.

First we rationalized things by deleting all the old TDS directories and taking
the tds_mevans dir and making it be the main one (apps/linux/tds).

We also deleted all of the TDS directories in the project area. It is now very
likely that several scripts will not work.
We're going to have the teething
problems of repointing everything to the nominal paths (in the apps areas).

Finally we did:
svn import tds https://40m.ligo.caltech.edu/svn/40m/tds --username rana

to stick it in. To check it out do:
svn checkout https://40m.ligo.caltech.edu/svn/40m/tds --username rana

We'll get a couple of the O'Reilly SVN books as well to supplement our verion control knowledge.
Unitl then you can use the SVN cheat sheets available at:
http://www.digilife.be/quickreferences/quickrefs.htm
  12770   Mon Jan 30 18:41:41 2017 jamieUpdateCDSTEST ABORTED of new daqd code on fb1

I just aborted the fb1 test and reverted everything to the nominal configuration.  Everything looks to be operating nominally.  Front ends are mostly green except for c1rfm and c1asx which are currently not being acquired by the DAQ, and an unknown IPC error with c1daf.  Please let me know if any unusual problems are encountered.

The behavior of daqd on fb1 with the latest release (3.2.1) was not improved.  After turning on the full pipe it was back to crashing every 10 minutes or so when the full and second trend frames were being written out.  lame.  back to the drawing board...

  13628   Fri Feb 9 13:37:44 2018 gautamUpdateALSTHD measurement trial

I quickly put together some code that calculates the THD from CDS data and generates a plot (see e.g. Attachment #1).

Algorithm is:

  1. Get data (for now, offile file, but can be readily adapted to download data live or from NDS).
  2. Compute power spectrum using scipy.signal.periodogram. I use a Kaiser window with beta=38 based on some cursory googling, and do 10 averages (i.e. nfft is total length / 10), and set the scaling to "spectrum" so as to directly get a power spectrum as opposed to a spectral density.
  3. Find fundamental (assumed highest peak) and N harmonics using scipy.signal.find_peaks_cwt. I downsample 16k data to 2k for speed. A 120second time series takes ~5 seconds.
  4. Compute THD as \mathrm{THD} = \frac{\sqrt{\sum_{i=2}^{N}\mathrm{V}_i^2}}{V_1}where V_i denotes an rms voltage, or in the case of a power spectrum, just the y-axis value.

I conducted a trial on the Y arm ALS channel whitening board (while the X arm counterpart is still undergoing surgery). With the whitening gain set to 0dB, and a 1Vpp input signal (so nothing should be saturated), I measure a THD of ~0.08% according to the above formula. Seems rather high - the LT1125 datasheet tells us to expect <0.001% THD+N at ~100Hz for a closed loop gain of ~10. I can only assume that the digitization process somehow introduces more THD? Of course the FoM we care about is what happens to this number as we increase the gain.

Quote:
 

I'm going to work on putting together some code that gives me a quick readback on the measured THD, and then do the test for real with different amplitude input signal and whitening gain settings.

 

Attachment 1: THD_trial.pdf
THD_trial.pdf
  3243   Mon Jul 19 13:51:09 2010 kiwamuUpdateCDSTIming card at X end

[Joe, Kiwamu]

 

 The timing slave in the IO chassis on the new X end was not working with symptoms of no front "OK" green light, no "PPS" light, 3.3V testpoint not working and  ERROR testpoint bouncing between 5-6V.

We took out the timing slave from the X end IO chassis put in to the new Y end IO chassis .

It worked perfectly there. We took the working one from Y end put in the X end IO chassis.

We slowly added cables. First we added power , it worked fine and we saw green "OK" light. Then we added 1PPS signal by a fiber and it also worked.

We turned everything off and then we added 40pin IPC cable from the chassis and infiniband cable from the  computer.

When we turned ON it we didn't see the green light.

This means something in the computer configuration might be wrong not in the timing card, we now are trying to make contact with Alex.

We are comparing the setup of the C1SCX  machine and the working C1ISCEX machine.

  1796   Mon Jul 27 14:12:14 2009 ranaSummarySUSTM Coil Noise Spectra
Rob noticed that the ITMY DAC channels were saturating occassionally. Looking at the spectrum we can see why.
With an RMS of 10000 cts, the peak excursions sometimes cause saturations.

There's a lot of mechanical noise which is showing up on the ITM oplevs and then going to the mirror via
the oplev servo. We need to reduce the mechanical noise and/or modify the filters to compensate. The ITM
COIL_OUT RMS needs to be less than ~3000.
Attachment 1: Coils.pdf
Coils.pdf
  750   Mon Jul 28 17:58:05 2008 SharonUpdate TOP screen changes
I wanted to test the adaptive code with a downsampling rate of 32 instead of 16. To do this I entered a 32 Hz ((2048/32)/2 - match Nyquist Freq.) low pass filter on the ERROR EMPH, MC1 and the relevant PEM channels.
  16643   Thu Feb 3 10:25:59 2022 JordanUpdateVACTP1 and Manual Gate Valve Install

Jordan, Chub

Chub and I installed the new manual gate valve (Nor-Cal GVM-6002-CF-K79) and reinstalled TP1. The new gate valave was placed with the sealing side towards the main 40m volume, then TP1 was installed on top and the foreline reattched to TP1.

This valve has a hard stop in the actuator to prevent over torquing.

 

Attachment 1: 20220203_101455.jpg
20220203_101455.jpg
Attachment 2: 20220203_094831.jpg
20220203_094831.jpg
Attachment 3: 20220203_094823.jpg
20220203_094823.jpg
  16634   Mon Jan 31 10:39:19 2022 JordanUpdateVACTP1 and Manual Gate Valve Removal

Jordan, Chub

Today, Chub and I removed TP1 and the failed manual gate valve off of the pumping spool.

First, P2 needed to be vented in order to remove TP1. TP1 has a purge valve on the side of the pump which we slowly opened bringing the P2 volume up to atmosphere. Although, this was not vented using the dry air/N2, using this purge valve eliminated the need to vent the RGA volume.

Then we disconnected TP1 foreline, removed TP1+8" flange reducer, then the gate valve. All of the removed hardware looked good, so no need to replace bolts/nuts, only needs new gaskets. TP1 and the failed valve are sitting on a cart wrapped in foil next to the pumping station.

Attachment 1: 20220131_100637.jpg
20220131_100637.jpg
Attachment 2: 20220131_102807.jpg
20220131_102807.jpg
Attachment 3: 20220131_102818.jpg
20220131_102818.jpg
Attachment 4: 20220131_100647.jpg
20220131_100647.jpg
  11593   Mon Sep 14 10:41:03 2015 SteveUpdateVACTP2 dry pump replaced

TP2 dry fore pump sn PLE10082 was replaced at pressure 717 mTorr,  TP2 50K rpm 0.33A @ 112,677 hrs

Top seal life was 8,160 hrs

Model  SH110, Sn LP1007L556 was installed. It's fore line pressure after 30 minutes of running 38 mTorr, TP2 turbo at 50K rpm 0.18A

 

 

Attachment 1: TP2_dry_pump_replaced.png
TP2_dry_pump_replaced.png
  15602   Wed Sep 23 15:06:54 2020 JordanUpdateVACTP2 Forepump Re-install

I removed the forepump to TP2 this morning after the vacuum failure, and tested in the C&B lab. I pumped down on a small volume 10 times, with no issue. The ultimate pressure was ~30 mtorr.

I re-installed the forepump in the afternoon, and restarted TP2, leaving V4 closed. This will run overnight to test, while TP3 backs TP1.

In order to open V1, with TP3 backing TP1, the interlock system had to be reset since it is expecting TP2 as a backing pump. TP2 is running normally, and pumping of the main volume has resumed.


gautam 2030:

  1. The monitor (LCD display) at the vacuum rack doesn't work - this has been the case since Monday at least. I usually use my laptop to ssh in so I didn't notice it so it could have been busted from before. But for anyone wishing to use the workstation arrangement at 1X8, this is not great. Today, we borrowed the vertex laptop to ssh in, the vertex laptop has since been returned to its nominal location.
  2. The modification to the interlock condition was made by simply commenting out the line requiring V4 to be open for V1 to be opened. I made a copy of the original .yaml file which we can revert to once we go back to the normal config.
  3. I also opened VM1 to allow the RGA scans to continue to be meaningful.
  4. At the time of writing, all systems seem nominal. See Attachment #2. The vertical line indicates when we started pumping on the main volume again earlier today, with TP3 backing TP1.

Unclear why the TP2 foreline pump failed in the first place, it has been running fine for several hours now (although TP2 has no load, since V4 isolates it from the main volume). Koji's plots show that the TP2 foreline pressure did not recover even after the interlock tripped and V4 was closed (i.e. the same conditions as TP2 sees right now).

Attachment 1: Screenshot_from_2020-09-23_15-15-43.png
Screenshot_from_2020-09-23_15-15-43.png
Attachment 2: MainVolPumpDown.png
MainVolPumpDown.png
  15409   Thu Jun 18 15:25:08 2020 JordanUpdateVACTP2 and TP3 Forepump removal

I removed the backing pumps for TP2 and TP3 today to test ultimate pressure and determine if they need a tip seal replacement. This was done with Jon backing me on Zoom. We closed off TP3 and powered down TP3 and the auxilliary pump, in order to remove the forepumps from the exhaust line.

  1. Close V1
  2. Close V5
  3. Turn off TP3
  4. Turn off aux dry pump (manually)
  5. Once the PTP3 foreline pressure has come up to atmosphere, you can disconnect the TP3 dry pump and cap the exhaust line with a KF blank.
  6. Restore the vac configuration in reverse order: dry pump ON, TP3 ON, open V5, open V1

Once pumps were removed I connected a Pirani gauge to the pump directly and pumped down, results as follows:

TP2 Forepump (Agilent IDP 7):

  • Ultimate Pressure: 123 mtorr
  • Hours: 10903

TP3 Forepump (Varian SH 110):

  • Ultimate pressure: ~70 torr
  • Hours: 60300

TP3 forepump defintely needs a new tip seal, and while the pressure on TP2 Forepump was good there was a significant amount of particulate that came out of the exhaust line, so a new tip seal might not be needed but it is recommeded.

  15411   Thu Jun 18 16:56:34 2020 JordanUpdateVACTP2 and TP3 Forepump removal
Quote:

I removed the backing pumps for TP2 and TP3 today to test ultimate pressure and determine if they need a tip seal replacement. This was done with Jon backing me on Zoom. We closed off TP3 and powered down TP3 and the auxilliary pump, in order to remove the forepumps from the exhaust line.

  1. Close V1
  2. Close V5
  3. Turn off TP3
  4. Turn off aux dry pump (manually)
  5. Once the PTP3 foreline pressure has come up to atmosphere, you can disconnect the TP3 dry pump and cap the exhaust line with a KF blank.
  6. Restore the vac configuration in reverse order: dry pump ON, TP3 ON, open V5, open V1

Once pumps were removed I connected a Pirani gauge to the pump directly and pumped down, results as follows:

TP2 Forepump (Agilent IDP 7):

  • Ultimate Pressure: 123 mtorr
  • Hours: 10903

TP3 Forepump (Varian SH 110):

  • Ultimate pressure: ~70 torr
  • Hours: 60300

TP3 forepump defintely needs a new tip seal, and while the pressure on TP2 Forepump was good there was a significant amount of particulate that came out of the exhaust line, so a new tip seal might not be needed but it is recommeded.

I agree with your assessment, Jordan.  If I'm not mistaken the scroll pump for TP2 is new; we had a very early failure with the last new scroll pump (the forepump for TP3) tip seals at just over 5000 hours.  Glad to see my replacement seals held up for over 60K hours. If this is the trend with these pumps, we can simply run them to  around 60000 hours and replace the seals at that time, rather than waiting for failure! - Chub

  8978   Wed Aug 7 15:36:29 2013 SteveUpdateVACTP2 drypump replaced

 

 TP2's fore line - dry pump replaced at performance level 600 mTorr after 10,377 hrs of continuous operation.

Where are the foreline pressure gauges? These values are not on the vac.medm screen.

The new tip seal dry pump lowered the small turbo foreline pressure 10x

TP2fl after 2 day of pumping 65mTorr

Attachment 1: forelinesPs.jpg
forelinesPs.jpg
  9876   Tue Apr 29 16:42:29 2014 SteveUpdateVACTP2 drypump replaced

Quote:

 

 TP2's fore line - dry pump replaced at performance level 600 mTorr after 10,377 hrs of continuous operation.

Where are the foreline pressure gauges? These values are not on the vac.medm screen.

The new tip seal dry pump lowered the small turbo foreline pressure 10x

TP2fl after 2 day of pumping 65mTorr

 TP2 dry pump replaced at fore pump pressure 1 Torr,  TP2 50K_rpm 0.34A

 Top seal life 6,362 hrs

 New seal performance at 1 hr  36 mTorr, 

 Maglev at 560 Hz, cc1 6e-6 Torr

 

Attachment 1: dryforepumpreplaced.png
dryforepumpreplaced.png
  13407   Mon Oct 30 10:09:41 2017 SteveUpdateVACTP2 failed

 IFO pressure 1.2e-5 Torr at 9:30am

Quote:

Valve configuration: Vacuum normal

Note: Tp2 running at 75Krpm 0.25A 26C has a  load high pitch sound today. It's fore line pressure 78 mTorr. Room temp 20C

 

Atm. 1,  This was the vacuum condition  this morning.

               IFO P1  9.7 mTorr, V1 openV4 was in closed position , ~37 C warm Maglev at normal 560Hz rotation speed with foreline pressure 3.9 Torr because V4 closed 2 days ago when TP2 failed .....see Atm.3

               The error messege at TP2  controller was: fault overtemp.

I did the following to restored IFO pumping: stopped pumping of the annulose with TP3 and valves were configured so TP3 can be the forepump of the Maglev.

closed VM1 to protect the RGA,  close PSL shutter .....see Gautam  entry

aux fan on to cool down Maglev-TP1, room temp 20 C,

aux drypump turned on and opend to TP3 foreline to gain pumping speed,

closed PAN to isolate annulos pumping,

opened V7 to pump Maglev forline with TP3 running at 50Krpm, It took 10 minutes to reach P2  1mTorr from 3.9 Torr

aux drypump closed off at P2  1 mTorr, TP3 foreline pressure 362 mTorr.......see Atm.2

As we are running now:

IFO pressure 7e-6 Torr at Hornet cold cathode gauge at 15:50  We have no IFO CC1 logging now.  Annuloses are in 3-5 mTorr range are not pumped.

TP3 as foreline pump of TP1 at 50 Krpm, 0.24 A, 24 C, it's drypump forline pressure 324 mTorr

V4 valve cable is disconnected.

I need help with wiring up the logging of the Hornet cold cathode gauge.

 

 

 

Attachment 1: tp2failed.png
tp2failed.png
Attachment 2: ifo_1.0E-5_Torrit.png
ifo_1.0E-5_Torrit.png
Attachment 3: tp2failed2dago.png
tp2failed2dago.png
Attachment 4: 4days.png
4days.png
  14512   Wed Apr 3 10:42:36 2019 gautamUpdateVACTP2 forepump replaced

Bob and Chub concluded that the drypump that serves as TP2's forepump had failed. Steve had told me the whereabouts of a spare Agilent IDP-7. This was meant to be a replacement for the TP3 foreline pump when it failed, but we decided to swap it in while diagnosing the failed drypump (which had 2182 hours continuous running according to the hour counter). Sure enough, the spare pump spun up and the TP2fl pressure dropped at a rate consistent with what is expected. I was then able to spin up TP1, TP2 and TP3. 

However, when opening V4 (the foreline of TP1 pumped by TP2), I heard a loud repeated click track (~5Hz) from the electronics rack. Shortly after, the interlocks shut down all the TPs again, citing "AC power loss". Something is not right, I leave it to Jon and Chub to investigate.

  14514   Wed Apr 3 16:17:17 2019 JonUpdateVACTP2 forepump replaced

I can't explain the mechanical switching sound Gautam reported. The relay controlling power to the TP2 forepump is housed in the main AC relay box under the arm tube, not in the Acromag chassis, so it can't be from that. I've cycled through the pumpdown sequence several times and can't reproduce the effect. The Acromag switches for TP2 still work fine.

In any case, I've made modifications to the vacuum interlocks that will help with two of the issues:

  1. For the "AC power loss" over-triggering: New logic added requiring the UPS to be out of the "on line power, battery OK" state for ~5 seconds before tripping the interlock. This will prevent electrical transients from triggering an emergency shutdown, as seems to be the case here (the UPS briefly isolates the load to battery during such events).
  2. PSL interlocking: New logic added which directly sets C1:AUX-PSL_ShutterRqst --> 0 (closes the PSL shutter) when the main volume pressure is 3 mtorr-500 torr. Previously there was a channel exposed for this interlock (C1:Vac-interlock_high_voltage), but c1aux was not actually monitoring it. Following the convention of every vac interlock, after the PSL shutter has been closed, it has to be manually reopened. Once the pressure is out of this range, the vac system will stop blocking the shutter from reopening, but it will not perform the reopen action itself. gautam: a separate interlock logic needs to be implemented on c1aux (the shutter machine) that only permits the shutter to be opened if the Vac pressure range is okay. The SUS watchdog style AND logic in the EPICS database file should work just fine.

After finishing this vac work, I began a new pumpdown at ~4:30pm. The pressure fell quickly and has already reached ~1e-5 torr. TP2 current and temp look fine.

Quote:

However, when opening V4 (the foreline of TP1 pumped by TP2), I heard a loud repeated click track (~5Hz) from the electronics rack. Shortly after, the interlocks shut down all the TPs again, citing "AC power loss". Something is not right, I leave it to Jon and Chub to investigate.

Attachment 1: IMG_3180.jpg
IMG_3180.jpg
  15599   Wed Sep 23 08:57:18 2020 gautamUpdateVACTP2 running HOT

The interlocks tripped at ~630am local time. Jordan reported that TP2 was supposedly running at 52 C (!).

V1 was already closed, but TP2 was still running. With him standing by the rack, I remotely exectued the following sequence:

  • VM1 closed (isolates RGA volume).
  • VA6 closed (isolates annuli from being pumped).
  • V7 opened (TP3 now backs TP1, temporarily, until I'm in the lab to check things out further).
  • TP2 turned off.

Jordan confirmed (by hand) that TP2 was indeed hot and this is not just some serial readback issue. I'll do the forensics later.

Attachment 1: Screen_Shot_2020-09-23_at_8.55.39_AM.png
Screen_Shot_2020-09-23_at_8.55.39_AM.png
  15600   Wed Sep 23 10:06:52 2020 KojiUpdateVACTP2 running HOT

Here is the timeline. This suggests TP2 backing RP failure.

1st line: TP2 foreline pressure went up. Accordingly TP2 P, current, voltage, and temp went up. TP2 rotation went down.

2nd line: TP2 temp triggered the interlock. TP2 foreline pressure was still high (10torr) so TP2 struggled and was running at 1 torr.

3rd line: Gautam's operation. TP2 was isolated and stopped.

Between the 1st line and 2nd line, TP2 pressue (=TP1 foreline pressure) went up to 1torr. This made TP1 current increased from 0.55A to 0.68A (not shown in the plot), but TP1 rotation was not affected.

Attachment 1: Screen_Shot_2020-09-23_at_10.00.43.png
Screen_Shot_2020-09-23_at_10.00.43.png
  5887   Mon Nov 14 15:46:14 2011 steveUpdateVACTP2's forepump changed

The foreline pressure of TP2 was 1.4 Torr this morning. This drypump worked well for ten months.

Recently rebuilt  drypump with new seal was swapped in.

This is how you do it: close  V1,  V4 and turn off TP2. Replace drypump and start up TP2

Set pump speed to 50 K rpm and open V4 to TP1 Note that the Maglev was not turned off  because V4 was closed off only 5-10 minutes.

Open V1 the status is Vac Normal.

TP2 is rotating at  50K rpm, current pick up 0.2A,  the temp is 26C and its foreline pressure 33 mTorr

Attachment 1: drypump.png
drypump.png
  4654   Fri May 6 15:59:40 2011 steveUpdateVACTP3 fore pump replaced

Dry fore pump of TP3 replaced by brand new Varian SH-110  at  1.1 Torr_75,208 hrs

The annuloses were closed off for 25 minutes. We are back to VACUUM NORMAL mode.

The TP3 fore line pressure dropped to 44 mTorr at 25 minutes in operation.......9.4mTorr at day 2 with full annulos load

  15591   Mon Sep 21 15:57:08 2020 JordanUpdateVACTP3 Forepump Replacement and Vac reset

I removed the forepump (Varian SH-110) for TP3 today to see why it had failed over the weekend. I tested it in the C&B lab and the ultimate pressure was only ~40torr. I checked the tip seals and they were destroyed. The scroll housing also easily pulled off of the motor drive shaft, which is indicative of bad bearings. The excess travel in the bearings likely led to significant increase in tip seal wear. This pump will need to be scrapped, or rebuilt.

I tested the spare Varian SH-110 pump located at the X-end and the ultimate pressure was ~98 mtorr. This pump had tip seals replaced on 11/5/18, and is currently at 55163 operating hours. It has been installed as the TP3 forepump.

Once installed, restarting the pump line occured as follows: V5 Closed, VA6 closed, VASE Closed, VASV closed, VABSSCI closed, VABS closed, VABSSCO closed, VAEV closed, VAEE closed,TP3 was restarted and once at normal operation, valves were opened in same order.

The pressure differential interlock condition for V5 was temporaily changed to 10 torr (by Gautam), so that valves could be opened in a controlled manner. Once, the vacuum system was back to normal state the V5 interlock condition was set back to the nominal 1 torr. Vacuum system is now running normally.

Attachment 1: Screenshot_from_2020-09-21_15-56-04.png
Screenshot_from_2020-09-21_15-56-04.png
Attachment 2: 20200921_145043.jpg
20200921_145043.jpg
Attachment 3: 20200921_145040.jpg
20200921_145040.jpg
  16806   Fri Apr 22 14:14:33 2022 JordanUpdateVACTP3 Forepump tip seal replacement

Jordan, JC

While the pumpspool is vented, I thought it would be a convenient time to change out the tip seal on the TP3 forepump. This one had not been changed since 2018, so as preventative maintence I had JC remove the pump and begin cleaning/installing the new tip seal.

Unfortunately the tip seal broke, but I have ordered another. We should have this pump ready to go late next week. If one is needed sooner, there is a spare IDP 7 pump we can install as the TP3 forepump.

Attachment 1: IMG_0572.jpeg
IMG_0572.jpeg
Attachment 2: IMG_0573.jpeg
IMG_0573.jpeg
  16822   Mon May 2 10:00:34 2022 JordanUpdateVACTP3 Forepump tip seal replacement

[JC, Jordan]

Jordan recieved the new tip seal Friday afternoon and I continued the replacement process in the morning. Finishing up, we proceeded to test the pump in the Clean and Bake room. The pump's pressure lowered to 110 mTorr, and we continue pumping so the seal can recieve a good fitting.

Update: We have confirmed the pump is working great and have reinstalled this back into the vacuum system.                                                                                                                   Note: The same O-Rings were used.

Quote:

Jordan, JC

While the pumpspool is vented, I thought it would be a convenient time to change out the tip seal on the TP3 forepump. This one had not been changed since 2018, so as preventative maintence I had JC remove the pump and begin cleaning/installing the new tip seal.

Unfortunately the tip seal broke, but I have ordered another. We should have this pump ready to go late next week. If one is needed sooner, there is a spare IDP 7 pump we can install as the TP3 forepump.

 

  15582   Sat Sep 19 18:07:35 2020 KojiUpdateVACTP3 RP failure

I came to the campus and Gautam notified that he just had received the alert from the vac watchdog.

I checked the vac status at c1vac. PTP3 went up to 10 torr-ish and this made the diff pressure for TP3 over 1torr. Then the watchdog kicked in.

To check the TP3 functionality, AUX RP was turned on and the manual valve (MV in the figure) was opened to pump the foreline of TP3. This easily made PTP3 <0.2 torr and TP3 happy (I didn't try to open V5 though).

So the conclusion is that RP for TP3 has failed. Presumably, the tip-seal needs to be replaced.

Right now TP3 was turned off and is ready for the tip-seal replacement. V5 was closed since the watchdog tripped.

Attachment 1: vac.png
vac.png
Attachment 2: Screen_Shot_2020-09-19_at_17.52.40.png
Screen_Shot_2020-09-19_at_17.52.40.png
  15586   Sat Sep 19 19:37:16 2020 not KojiUpdateVACTP3 RP failure

Disconcerting because those tip seals were just replaced [15417]. Maybe they were just defective, but if there is a more serious problem with the pump, there is a spare Varian roughing pump (the old TP2 dry pump) sitting at the X-end.

I reset the interlock error to unfreeze the vac controls (leaving V5 closed).

Quote:

So the conclusion is that RP for TP3 has failed. Presumably, the tip-seal needs to be replaced.

Right now TP3 was turned off and is ready for the tip-seal replacement. V5 was closed since the watchdog tripped.

  15140   Wed Jan 22 16:19:47 2020 JonUpdateVACTP3 controller errors

Gautam and I debugged a communications problem with TP3 that was causing its python service to fail. We traced the problem back to the querying of the pump controller for its operational parameters (speed, voltage, temp). Some small percentage of the time (~5%, indeterministically), the pump controller is returning an invalid response which causes the service to shut itself down and signal a NO COMM error.

As a temporary fix, I wrapped the failing query in an exception handler to continue past this particular error. However, we suspect the microprocessor in the TP3 controller may be beginning to fail. There is a spare controller sitting right next to it in the vacuum rack. We will ask Chub to install the replacement in the near future.

gautam: this pump is responsible for pumping the annular volume under normal operations. while this problem is being resolved, the annular volume is valved off (as it has been since July 2019 anyway which is when this problem first manifested).

  13423   Fri Nov 10 08:52:21 2017 SteveUpdateVACTP3 drypump replaced

PSL shutter closed at 6e-6 Torr-it    

   The foreline pressure of the drypump is 850 mTorr at 8,446 hrs of seal life

V1 will be closed for ~20 minutes for drypump replacement..........

9:30am dry pump replaced, PSL shutter opened at 7.7E-6 Torr-it

  Valve configuration: vacuum normal as  TP3 is the forepump of the Maglev  & annuloses are not pumped.

Quote:

TP3 drypump replaced at 655 mTorr, no load, tp3 0.3A 

This seal lasted only for 33 days at  123,840 hrs

The replacement is performing well: TP3 foreline pressure is 55 mTorr, no load, tp3 0.15A at 15 min  [ 13.1 mTorr at d5 ]

 

Valve configuration: Vacuum Normal, ITcc 8.5E-6 Torr

Quote:

Dry pump of TP3 replaced after 9.5 months of operation.[ 45 mTorr d3 ]

The annulosses are pumped.

Valve configuration: vac normal, IFO pressure 4.5E-5 Torr [1.6E-5 Torr d3 ] on new ITcc gauge, RGA is not installed yet.

Note how fast the pressure is dropping when the vent is short.

Quote:

IFO pressure 1.7E-4 Torr on new not logged cold cathode gauge. P1 <7E-4 Torr

Valve configuration: vac.normal with anunulossess closed off.

TP3 was turned off with a failing drypump. It will be replaced tomorrow.

All time stamps are blank on the MEDM screens.

 

 

  13637   Fri Feb 16 15:57:58 2018 SteveUpdateVACTP3 drypump replaced

The forline pressure of TP3 was 399 mTorr

It was replaced this morning at  TP3 controller 134,638hrs with the "failed TP2 station" drypump. The foreline pressure now at TP3 is 100 mTorr at 6 hrs of operation.[ at day 3  63 mT ]

IFO pressure at CC Hornet 7.9e - 6 Torr

Valve configuration: vacuum normal as TP3 is the forepump of the Maglev & the annuloses are not pumped

Quote:

PSL shutter closed at 6e-6 Torr-it    

   The foreline pressure of the drypump is 850 mTorr at 8,446 hrs of seal life

V1 will be closed for ~20 minutes for drypump replacement..........

9:30am dry pump replaced, PSL shutter opened at 7.7E-6 Torr-it

  Valve configuration: vacuum normal as  TP3 is the forepump of the Maglev  & annuloses are not pumped.

Quote:

TP3 drypump replaced at 655 mTorr, no load, tp3 0.3A 

This seal lasted only for 33 days at  123,840 hrs

The replacement is performing well: TP3 foreline pressure is 55 mTorr, no load, tp3 0.15A at 15 min  [ 13.1 mTorr at d5 ]

 

Valve configuration: Vacuum Normal, ITcc 8.5E-6 Torr

Quote:

Dry pump of TP3 replaced after 9.5 months of operation.[ 45 mTorr d3 ]

The annulosses are pumped.

Valve configuration: vac normal, IFO pressure 4.5E-5 Torr [1.6E-5 Torr d3 ] on new ITcc gauge, RGA is not installed yet.

Note how fast the pressure is dropping when the vent is short.

Quote:

IFO pressure 1.7E-4 Torr on new not logged cold cathode gauge. P1 <7E-4 Torr

Valve configuration: vac.normal with anunulossess closed off.

TP3 was turned off with a failing drypump. It will be replaced tomorrow.

All time stamps are blank on the MEDM screens.

 

 

 

  13454   Sun Nov 26 19:38:40 2017 SteveUpdateVACTP3 drypump replaced again

The TP3 foreline pressure was 4.8 Torr, 50K rpm 0.54A and 31C........Maglev rotation normal 560 Hz.......    IFO pressure 7.2e- 6 Torrit was not effected

V1 closed ......replaced drypump.........V1 opened

IFO 6.9e-6 Torrit at 19:55, TP3fl 18 mT,  50Krpm 0.15A 24C

VM1 is still closed

 

Attachment 1: after_replacement.png
after_replacement.png
  10838   Tue Dec 23 15:37:32 2014 SteveUpdateVACTP3 drypump replaced

Quote:

Quote:

 

 TP2's fore line - dry pump replaced at performance level 600 mTorr after 10,377 hrs of continuous operation.

Where are the foreline pressure gauges? These values are not on the vac.medm screen.

The new tip seal dry pump lowered the small turbo foreline pressure 10x

TP2fl after 2 day of pumping 65mTorr

 TP2 dry pump replaced at fore pump pressure 1 Torr,  TP2 50K_rpm 0.34A

 Top seal life 6,362 hrs

 New seal performance at 1 hr  36 mTorr, 

 Maglev at 560 Hz, cc1 6e-6 Torr

 

 TP3 dry pump  replaced at 540 mT as TP3 50K_rpm 0.3A with annulos load. It's top seal life time was 11,252 hrs

 

 

  12636   Wed Nov 23 11:32:13 2016 SteveUpdateVACTP3 drypump replaced again

TP3 drypump replaced at 655 mTorr, no load, tp3 0.3A 

This seal lasted only for 33 days at  123,840 hrs

The replacement is performing well: TP3 foreline pressure is 55 mTorr, no load, tp3 0.15A at 15 min  [ 13.1 mTorr at d5 ]

 

Valve configuration: Vacuum Normal, ITcc 8.5E-6 Torr

Quote:

Dry pump of TP3 replaced after 9.5 months of operation.[ 45 mTorr d3 ]

The annulosses are pumped.

Valve configuration: vac normal, IFO pressure 4.5E-5 Torr [1.6E-5 Torr d3 ] on new ITcc gauge, RGA is not installed yet.

Note how fast the pressure is dropping when the vent is short.

Quote:

IFO pressure 1.7E-4 Torr on new not logged cold cathode gauge. P1 <7E-4 Torr

Valve configuration: vac.normal with anunulossess closed off.

TP3 was turned off with a failing drypump. It will be replaced tomorrow.

All time stamps are blank on the MEDM screens.

 

  2963   Fri May 21 00:30:44 2010 JenneUpdateVACTP3 fore pump is very loud

[Jenne, Kiwamu, and Steve via phone]

Around 9:30pm, Kiwamu and I came back from dinner, and were getting ready to begin the beam scan measurements.  I noticed that one of the vacuum pumps was being very loud.  Kiwamu noted that it is the fore pump for TP3's turbo, which he and Steve replaced in January (elog 2538).  We had not noticed these noises before leaving for dinner, around 8pm.

We called Steve at home, and he could hear the noise through the phone. He said that even though it was really loud, since it was reading 3.3mTorr (on the display of the controller, in the vacuum rack just above head-height) which is close to the nominal value, it should be fine to leave.  He will check it out in the morning.  If it had been reading at or above ~1Torr, that's indicative of it being really bad, and we would have needed to shut it off.

For future reference, in case we need to turn it off, Steve said to use the following procedure:

1. Close VM3, to isolate the RGA, which is what this pump is currently (while we're at atmosphere) pumping on.  I don't know if there are other things which would need to be shut at this stage, if we were at vacuum nominal.

2. Close VM5, which is right in front of TP3, so TP3's pump is just pumping on itself.

3. Push the "Stop" button on the Turbo controller for TP3, in the vacuum rack, about waist level.  Turning off the turbo will also turn off the fore pump.

UPDATE, 1am: The controller in the rack is reading 3.1mTorr, so the pump, while still noisy, still seems to be working.

  14517   Fri Apr 5 01:10:18 2019 gautamUpdateVACTP3 forepump is also noisy

Is this one close to failure as well?

  2965   Fri May 21 08:28:29 2010 steveConfigurationVACTP3 turned off

Quote:

[Jenne, Kiwamu, and Steve via phone]

Around 9:30pm, Kiwamu and I came back from dinner, and were getting ready to begin the beam scan measurements.  I noticed that one of the vacuum pumps was being very loud.  Kiwamu noted that it is the fore pump for TP3's turbo, which he and Steve replaced in January (elog 2538).  We had not noticed these noises before leaving for dinner, around 8pm.

We called Steve at home, and he could hear the noise through the phone. He said that even though it was really loud, since it was reading 3.3mTorr (on the display of the controller, in the vacuum rack just above head-height) which is close to the nominal value, it should be fine to leave.  He will check it out in the morning.  If it had been reading at or above ~1Torr, that's indicative of it being really bad, and we would have needed to shut it off.

For future reference, in case we need to turn it off, Steve said to use the following procedure:

1. Close VM3, to isolate the RGA, which is what this pump is currently (while we're at atmosphere) pumping on.  I don't know if there are other things which would need to be shut at this stage, if we were at vacuum nominal.

2. Close VM5, which is right in front of TP3, so TP3's pump is just pumping on itself.

3. Push the "Stop" button on the Turbo controller for TP3, in the vacuum rack, about waist level.  Turning off the turbo will also turn off the fore pump.

UPDATE, 1am: The controller in the rack is reading 3.1mTorr, so the pump, while still noisy, still seems to be working.

 The foreline pressure of TP3 is 2.9 mTorr  The drypump is loosing it's bearing and it is very noisy.

V3, V5 closed and TP3 small turbo controller off. This turned off the noisy forepump that has to be replaced.

RGA is running at cc4 2e-6 Torr

The RGA was turned off at cc4 1e-5 Torr

  9120   Tue Sep 10 15:43:01 2013 SteveUpdateVACTP3' dry pump is replaced

Quote:

 

 TP3 foreline's dry pump is getting noisier and noisier.  Turbo TP3 is pumping on the annulos. The foreline pressure is 7.2 mTorr and it is not degrading. It was swapped in March 5, 2013

The seal is very good, but the bearing is dying.

 

 

 

 The drypump is replaced at 95,781 hrs on TP3 controller time. The foreline pressure is 30 mTorr and dropping.

 It is 13 mTorr after 17 hours of pumping.

  1374   Mon Mar 9 12:04:18 2009 YoichiUpdateComputersTPs and AWG are back
I had to do one more reboot of tpman and daqd to get the TPs working.
I confirmed the alignment scripts run fine.

Now the oplevs of some optics are largely mis-centered. Alberto and I will center them after lunch.
  13920   Wed Jun 6 14:36:15 2018 gautamUpdateLSCTRX clipping

For some time now, I've been puzzled by the unreliability of the ASS_X dither alignment servo. Leaving the servo on, TRX often begins to decay to a lower value, and even after freezing the dither at the maximum TRX values, I can manually align the mirrors to increase TRX. We have suspected some kind of clipping in the TRX path that is responsible for this behaviour. Today I decided to investigate this a bit further. To have the arm locked and to inspect the beam, we have to change the locking trigger - TRX is what is normally used, but I misaligned the Y arm completely, and used AS110 as a trigger instead. There is some strangeness in the triggering topology, but this deserves a separate elog.

Once the arm was locked (and relocks using the AS110 trigger in the event of an unlock), I was able to trace the beampath on the EX table with an IR card. The TRX beam is rather large and weak, so it is hard to see, but as best as I can tell, the only real danger of clipping (or perhaps the beam is already clipped) is on the final steering mirror before the beam hits the (Thorlabs) PD. Steve/Pooja are working on getting a photo of this, and will upload it here shortly. Options to mitigate this:

  1. Use the harmonic separator to steer the beam lower, and center it on the 1" steering mirror. However, this could possibly lead to clipping on some of the upstream lenses.
  2. Raise the height of the 1" steering mirror by 0.25". However, this would require a custom 3/4" dia post height or some shims, which I am not sure is in line with our optomechanic mounting practises.
  3. Use a 2" mirror instead of a 1" mirror.

The EX QPD has stopped working since the Acromag install. If it were working, we wouldn't have to rely on the alternate triggering with AS110 and instead just use the QPD as TRX, while we debug the Thorlabs PD path.

  13927   Thu Jun 7 16:15:03 2018 gautamUpdateLSCTRX clipping

I opted for the quickest fix - I raised the height of the offending steering mirror using a 0.25" shim. In the long term, we can get a taller post machined. After raising the mirror height, I then checked the DC centering of the spot on the DC PD using a scope.

Looking at the performance of the X arm ASS, I no longer see the strange oscillatory behaviour I described in my previous post yes. Moreover, the TRX level was ~1 before be raising the steering mirror - but it is now ~1.2. So we were certainly losing some power.

  1878   Mon Aug 10 17:27:47 2009 robConfigurationLSCTRX, TRY gain

 

These are the settings which determine the transmon (eg, TRX) amplitude, and which are updated by the matchTransMon scripts.

For the X arm

 

op440m:AutoDither>tdsread C1:LSC-TRX_GAIN C1:LSC-LA_PARAM_FLT_01 C1:LSC-LA_PARAM_FLT_00
0.0023
0.155
119.775

 

For the Y arm

op440m:AutoDither>tdsread C1:LSC-TRY_GAIN C1:LSC-LA_PARAM_FLT_04 C1:LSC-LA_PARAM_FLT_03
0.00237196
-0.116
19.9809


  10862   Wed Jan 7 03:04:13 2015 JenneUpdateLSCTRY (thorlabs pd) weird noise

[Jenne, Diego, Rana]

This is a note about work done last night. 

We were starting to lock, and saw glitches in the Thorlabs TRY PD about once every 1/60th of a second.  It is not a sine wave, so it is not 60Hz line noise directly.  It looks like this:

TRY_60Hz_peaks_5Jan2015.pdf

Rana pointed out that this looks like it could be from a power supply that is converting AC to DC. 

We went down to the Yend, and noticed some weird symptoms.  So far, we do not know where the noise is coming from.  Rather, we are just using the QPD for locking.

* The noise comes and goes, particularly if someone is moving around at the end station.

* Moving the Thorlabs power supply farther from the HeNe power supply didn't do much.  Turning off and disconnecting the HeNe supply didn't make the noise go away, so we conclude that it is not the HeNe's fault.

* We suspected the loops of excess cable that were sitting on top of iscey, but moving the coils away from the computer did not make the noise go away.

* We removed a few disconnected BNC cables that were near or touching the end table, but that didn't fix things.

* We disconnected the PD's signal cable and pulled it out of the table enclosure, and then put it back.  Noise was gone when cable was disconnected (good), but it was back after plugging the cable back in.

* The noise still comes and goes, but we don't have to use the Thorlabs PD for locking, so we leave it for another day.

RXA: also moved the Thorlabs power supply to a different power strip and tried putting it closer/farther to the Uniblitz shutter controller. Another suspect is that its some PWM type noise from the doubler crystal temperature driver. Need to try turning off the heater and the Raspberry PI to if it effects the noise.

  14459   Fri Feb 15 18:42:57 2019 gautamUpdateLSCTRY 60 Hz solved

A more permanent fix than a crocodile clip was implemented. Should probably look to do this for the X end unit as well.

Attachment 1: IMG_7323.JPG
IMG_7323.JPG
  14446   Mon Feb 11 15:41:49 2019 gautamUpdateLSCTRY 60 Hz solved - but clipping persists

Rich came by the 40m to photocopy some pages from Hobbs, and saw me working on the 60 Hz hunting. As I suspected, the problem was being generated in the D040060. This board receives the photodiode signal single-ended, but has a different power ground than the photodiode (even though the PD is plugged into a power strip that claims to come from 1Y4). The mechanism is not entirely clear - the presence of these 60 Hz features seemed to be dependent on the light level on the TRY photodiode (i.e. they were absent when the PSL shutter is closed, and were more prominent when TRY was 0.9 rather than 0.5) but the PD certainly wasn't saturated - the DC signal was only ~100 mV when viewed on a scope. In any case, Rich suggested the simplest test would be to ground the BNC shield bringing TRY to the rack, to the local ground on the board, which I did using a crocodile clip. This did the trick, the TRY signal RMS is now dominated by the ~1 Hz seismic-driven variation.

 On a more pessimistic note - it looks like the elliptical reflector moving did not work, and the clipping in the Y arm persists no. I am able to recover TRY~1 with the yaw offset on the ETM (which is still lower than the 1.06-1.07 Koji reported in Aug 2018, but I can believe that being down to the MC transmission being a few % lower at 15000cts rather than 15500), while the maximum I see without it is ~0.9. This is puzzling, because when the chamber was open, we saw that there was ~1.5" clearance between the edge of the reflector and the beam on an IR card. I suppose the input pointing could have been off by a small amount. So one of the primary vent objectives wasn't acheieved... But I will push ahead with the loss measurement.

  9838   Tue Apr 22 01:11:42 2014 JenneUpdateLSCTRY 60Hz noise

Quote:

 

P.S. I realigned the Y green to the arm and brought GTRY to 0.93

This evening, I was not able to successfully transition CARM from ALS to 1/sqrt(trans) signals.  The TRY time series looked odd, so I took a spectra, and we have huge 60Hz noise in TRY. 

I found a lock stretch from around 6:30pm that did not show the 60Hz noise, and then there was a lock stretch around 8pm that did have the noise.  So, something happened at the Yend between 6:30 and 8pm tonight.

Asking around, this was the time frame in which Manasa was down at the Yend to realign the green beam, and to check cabling for the PZT_OUT and ERR_MON signals to the ADC.

Looking at the spectra, Rana noted that we have even as well as odd harmonics of the 60Hz line, which is unusual.

TRY_60Hz_noise_21Apr2014.pdf

To try to diagnose the problem, Rana and I tried to make sure no cables' connectors were touching, and that no equipment was plugged in that shouldn't be.  We noticed that none of: the shutter, the Thorlabs TRY PD, or the QPD TRY are isolated from the table.  To see if perhaps the shutter was the problem, I turned off the power to the Yend green shutter, and unplugged the cable.  The cable is laying on the table, with the connector sitting on a piece of plastic to isolate it.  Removing the shutter from the system did not change anything. 

We don't see the 60Hz noise in the Xarm, so it's not on the laser light itself.  Also, we don't see the 60Hz lines in the Yarm feedback signal, so we're not putting the lines onto the mirror, and thus onto the Yarm's light. 

Manasa, can you please take a look, and see if you can figure out what is going on?  We need TRY so that we can transition to 1/sqrt(trans) signals for CARM.  Thanks!!

  9841   Tue Apr 22 21:54:50 2014 manasaUpdateLSCTRY 60Hz noise

Quote:

Quote:

 

P.S. I realigned the Y green to the arm and brought GTRY to 0.93

This evening, I was not able to successfully transition CARM from ALS to 1/sqrt(trans) signals.  The TRY time series looked odd, so I took a spectra, and we have huge 60Hz noise in TRY. 

Manasa, can you please take a look, and see if you can figure out what is going on?  We need TRY so that we can transition to 1/sqrt(trans) signals for CARM.  Thanks!!

 I went to the Y end to look at the TRY 60Hz noise situation this morning. While looking at TRY noise on dtt, I found that just lifting the cable away from the cable bunch that runs out of the table suppressed the noise drastically. 

Attachment 1

I removed the unwanted bnc connector in the path of the already long TRY cable running from the PD to the 1Y4 rack and isolated it from the bunch. TRY became less noisy.

But the noise was back again earlier in the evening and it looks like the noise is very much related to the TRY cable. TRY cable might have moved from its sweet spot while I was around checking cable connections yesterday.

I couldn't find a spare to replace it right away today (We need a BNC to 4 pin lemo).

Attachment 1: 60HzTRY.jpg
60HzTRY.jpg
  9842   Tue Apr 22 22:49:10 2014 ranaUpdateLSCTRY 60Hz noise

 

 The detectors and electronics on this table are not properly isolated. To reduce the 60 Hz and ground loops, photodiodes and shutter must be isolated by using plastic spacers as we usually do elsewhere - this table just seems to have a few oversights.

Steve can start assembling all of the pieces to do this in the morning and then we can start the swapping after the meeting.

The high gain Transmon cable should be a regular BNC. There's no need for 4-pin LEMO in this usage, so the best move is to modify the board and replace the 4-pin LEMO connector with an isolated panel mount BNC female.

The AC adapter for this diode (and all of the detectors on the table) should get their power from a power strip which gets plugged into the rack with the whitening boards. The SHG oven, the Uniblitz shutter, and any cameras can get their power from another power strip if needed/wanted.

  9843   Wed Apr 23 19:58:00 2014 manasaUpdateLSCTRY 60Hz noise

[Steve, Manasa]

To find noise source

1. Swapped the power cable of the PD and checked that it is connected to the right power source.

2. Changed the aluminium base of the post holding the diode so that the diode is floating

3. Grounded the table and the rack

4. Routed the cable on the other side of the beam tube to isolate it from other cables.

After all the above, we still found that shaking the cable was making TRY noisy.

I pulled out the PD whitening board to replace the 4 pin lemo connector with a bnc connector so that we can swap the cable with a new one. So there is no TRY right now.

 

  9918   Tue May 6 18:32:14 2014 steveUpdateLSCTRY 60Hz noise hunt

This is an effort to get rid of our ground loops  by isolating the electronic components from the optical table.

Aluminum mounting base plates of Thorlabs BA2 and Newport B-2 were replaced by plates or post made out of delrin material.

This is an insulator. DELRIN base plates were installed 6 places. The oplev-qpd has Nylon base plate.

The NPRO and HE/NE lasers are not isolated from the table. S8 and S9

I'm not sure about the doubling oven S10 

The optical table is grounded at G11  through  ~1 Mohms to the ETMY chamber.

Alignment touch up needed   at all D-marked component!

 

Attachment 1: ETMY-ISCT_EISOL.jpg
ETMY-ISCT_EISOL.jpg
  9921   Wed May 7 14:01:36 2014 steveUpdateLSCTRY 60Hz noise hunt

Quote:

This is an effort to get rid of our ground loops  by isolating the electronic components from the optical table.

Aluminum mounting base plates of Thorlabs BA2 and Newport B-2 were replaced by plates or post made out of delrin material.

This is an insulator. DELRIN base plates were installed 6 places. The oplev-qpd has Nylon base plate.

The NPRO and HE/NE lasers are not isolated from the table. S8 and S9

I'm not sure about the doubling oven S10 

The optical table is grounded at G11  through  ~1 Mohms to the ETMY chamber.

Alignment touch up needed   at all D-marked component!

 

 Attachment appendix:

 

D: component delrin isolated

N: component nylon isolated ( or Delrin )

S: component shell is shorting to optical table (except oven)

G:  optical table ground

 

I failed to maximize TRY the pds.

  9933   Thu May 8 17:25:17 2014 steveUpdateLSCTRY 60Hz noise hunt

 I worked at the ETMY-ISCT this morning and late afternoon.  I will continue the 60 Hz noise hunt tomorrow. 

ELOG V3.1.3-