There is currently no table at the X end!

We have moved the vast majority of the optics to a temporary storage breadbord, and moved the end table itself to the workbench at the end. 

Steve says Transportation is coming at 1PM to put the new table in.

Given the similarities between the MDT694B (single channel piezo controller) and TC200 (temperature controller) serial interfaces, I added the pyserial driver here. 

*Warning* this first version of the driver remains untested

FYI, there is this. Seems pretty well maintained, and so might be more useful in the long run. The available catalog of instruments is quite impressive - TC200 temp controller and SRS345 func gen are included and are things we use in the lab. maybe you can make a pull request to add MDT694B (there is some nice API already built I think). We should also put our netgpibdata stuff and the vacuum gauge control (basically everything that isn't rtcds) on there (unless there is some intellectual property rights issues that the Caltech lawyers have to sort out).

svn import tds https://40m.ligo.caltech.edu/svn/40m/tds --username rana

svn checkout https://40m.ligo.caltech.edu/svn/40m/tds --username rana

http://www.digilife.be/quickreferences/quickrefs.htm

I just aborted the fb1 test and reverted everything to the nominal configuration.  Everything looks to be operating nominally.  Front ends are mostly green except for c1rfm and c1asx which are currently not being acquired by the DAQ, and an unknown IPC error with c1daf.  Please let me know if any unusual problems are encountered.

The behavior of daqd on fb1 with the latest release (3.2.1) was not improved.  After turning on the full pipe it was back to crashing every 10 minutes or so when the full and second trend frames were being written out.  lame.  back to the drawing board...

I quickly put together some code that calculates the THD from CDS data and generates a plot (see e.g. Attachment #1).

Algorithm is:

1. Get data (for now, offile file, but can be readily adapted to download data live or from NDS).
2. Compute power spectrum using scipy.signal.periodogram. I use a Kaiser window with beta=38 based on some cursory googling, and do 10 averages (i.e. nfft is total length / 10), and set the scaling to "spectrum" so as to directly get a power spectrum as opposed to a spectral density.
3. Find fundamental (assumed highest peak) and N harmonics using scipy.signal.find_peaks_cwt. I downsample 16k data to 2k for speed. A 120second time series takes ~5 seconds.
4. Compute THD as where V_i denotes an rms voltage, or in the case of a power spectrum, just the y-axis value.

I conducted a trial on the Y arm ALS channel whitening board (while the X arm counterpart is still undergoing surgery). With the whitening gain set to 0dB, and a 1Vpp input signal (so nothing should be saturated), I measure a THD of ~0.08% according to the above formula. Seems rather high - the LT1125 datasheet tells us to expect <0.001% THD+N at ~100Hz for a closed loop gain of ~10. I can only assume that the digitization process somehow introduces more THD? Of course the FoM we care about is what happens to this number as we increase the gain.

[Joe, Kiwamu]

 The timing slave in the IO chassis on the new X end was not working with symptoms of no front "OK" green light, no "PPS" light, 3.3V testpoint not working and  ERROR testpoint bouncing between 5-6V.

We took out the timing slave from the X end IO chassis put in to the new Y end IO chassis .

It worked perfectly there. We took the working one from Y end put in the X end IO chassis.

We slowly added cables. First we added power , it worked fine and we saw green "OK" light. Then we added 1PPS signal by a fiber and it also worked.

We turned everything off and then we added 40pin IPC cable from the chassis and infiniband cable from the  computer.

When we turned ON it we didn't see the green light.

This means something in the computer configuration might be wrong not in the timing card, we now are trying to make contact with Alex.

We are comparing the setup of the C1SCX  machine and the working C1ISCEX machine.

Jordan, Chub

Chub and I installed the new manual gate valve (Nor-Cal GVM-6002-CF-K79) and reinstalled TP1. The new gate valave was placed with the sealing side towards the main 40m volume, then TP1 was installed on top and the foreline reattched to TP1.

This valve has a hard stop in the actuator to prevent over torquing.

Jordan, Chub

Today, Chub and I removed TP1 and the failed manual gate valve off of the pumping spool.

First, P2 needed to be vented in order to remove TP1. TP1 has a purge valve on the side of the pump which we slowly opened bringing the P2 volume up to atmosphere. Although, this was not vented using the dry air/N2, using this purge valve eliminated the need to vent the RGA volume.

Then we disconnected TP1 foreline, removed TP1+8" flange reducer, then the gate valve. All of the removed hardware looked good, so no need to replace bolts/nuts, only needs new gaskets. TP1 and the failed valve are sitting on a cart wrapped in foil next to the pumping station.

TP2 dry fore pump sn PLE10082 was replaced at pressure 717 mTorr,  TP2 50K rpm 0.33A @ 112,677 hrs

Top seal life was 8,160 hrs

Model  SH110, Sn LP1007L556 was installed. It's fore line pressure after 30 minutes of running 38 mTorr, TP2 turbo at 50K rpm 0.18A

I removed the forepump to TP2 this morning after the vacuum failure, and tested in the C&B lab. I pumped down on a small volume 10 times, with no issue. The ultimate pressure was ~30 mtorr.

I re-installed the forepump in the afternoon, and restarted TP2, leaving V4 closed. This will run overnight to test, while TP3 backs TP1.

In order to open V1, with TP3 backing TP1, the interlock system had to be reset since it is expecting TP2 as a backing pump. TP2 is running normally, and pumping of the main volume has resumed.

gautam 2030:

1. The monitor (LCD display) at the vacuum rack doesn't work - this has been the case since Monday at least. I usually use my laptop to ssh in so I didn't notice it so it could have been busted from before. But for anyone wishing to use the workstation arrangement at 1X8, this is not great. Today, we borrowed the vertex laptop to ssh in, the vertex laptop has since been returned to its nominal location.
2. The modification to the interlock condition was made by simply commenting out the line requiring V4 to be open for V1 to be opened. I made a copy of the original .yaml file which we can revert to once we go back to the normal config.
3. I also opened VM1 to allow the RGA scans to continue to be meaningful.
4. At the time of writing, all systems seem nominal. See Attachment #2. The vertical line indicates when we started pumping on the main volume again earlier today, with TP3 backing TP1.

​Unclear why the TP2 foreline pump failed in the first place, it has been running fine for several hours now (although TP2 has no load, since V4 isolates it from the main volume). Koji's plots show that the TP2 foreline pressure did not recover even after the interlock tripped and V4 was closed (i.e. the same conditions as TP2 sees right now).

I removed the backing pumps for TP2 and TP3 today to test ultimate pressure and determine if they need a tip seal replacement. This was done with Jon backing me on Zoom. We closed off TP3 and powered down TP3 and the auxilliary pump, in order to remove the forepumps from the exhaust line.

1. Close V1
2. Close V5
3. Turn off TP3
4. Turn off aux dry pump (manually)
5. Once the PTP3 foreline pressure has come up to atmosphere, you can disconnect the TP3 dry pump and cap the exhaust line with a KF blank.
6. Restore the vac configuration in reverse order: dry pump ON, TP3 ON, open V5, open V1

Once pumps were removed I connected a Pirani gauge to the pump directly and pumped down, results as follows:

TP2 Forepump (Agilent IDP 7):

• Ultimate Pressure: 123 mtorr
• Hours: 10903

TP3 Forepump (Varian SH 110):

• Ultimate pressure: ~70 torr
• Hours: 60300

TP3 forepump defintely needs a new tip seal, and while the pressure on TP2 Forepump was good there was a significant amount of particulate that came out of the exhaust line, so a new tip seal might not be needed but it is recommeded.

I agree with your assessment, Jordan.  If I'm not mistaken the scroll pump for TP2 is new; we had a very early failure with the last new scroll pump (the forepump for TP3) tip seals at just over 5000 hours.  Glad to see my replacement seals held up for over 60K hours. If this is the trend with these pumps, we can simply run them to  around 60000 hours and replace the seals at that time, rather than waiting for failure! - Chub

 TP2's fore line - dry pump replaced at performance level 600 mTorr after 10,377 hrs of continuous operation.

Where are the foreline pressure gauges? These values are not on the vac.medm screen.

The new tip seal dry pump lowered the small turbo foreline pressure 10x

TP2fl after 2 day of pumping 65mTorr

 TP2 dry pump replaced at fore pump pressure 1 Torr,  TP2 50K_rpm 0.34A

 Top seal life 6,362 hrs

 New seal performance at 1 hr  36 mTorr, 

 Maglev at 560 Hz, cc1 6e-6 Torr

 IFO pressure 1.2e-5 Torr at 9:30am

Atm. 1,  This was the vacuum condition  this morning.

               IFO P1  9.7 mTorr, V1 open,  V4 was in closed position , ~37 C warm Maglev at normal 560Hz rotation speed with foreline pressure 3.9 Torr because V4 closed 2 days ago when TP2 failed .....see Atm.3

               The error messege at TP2  controller was: fault overtemp.

I did the following to restored IFO pumping: stopped pumping of the annulose with TP3 and valves were configured so TP3 can be the forepump of the Maglev.

closed VM1 to protect the RGA,  close PSL shutter .....see Gautam  entry

aux fan on to cool down Maglev-TP1, room temp 20 C,

aux drypump turned on and opend to TP3 foreline to gain pumping speed,

closed PAN to isolate annulos pumping,

opened V7 to pump Maglev forline with TP3 running at 50Krpm, It took 10 minutes to reach P2  1mTorr from 3.9 Torr

aux drypump closed off at P2  1 mTorr, TP3 foreline pressure 362 mTorr.......see Atm.2

As we are running now:

IFO pressure 7e-6 Torr at Hornet cold cathode gauge at 15:50  We have no IFO CC1 logging now.  Annuloses are in 3-5 mTorr range are not pumped.

TP3 as foreline pump of TP1 at 50 Krpm, 0.24 A, 24 C, it's drypump forline pressure 324 mTorr

V4 valve cable is disconnected.

I need help with wiring up the logging of the Hornet cold cathode gauge.

Bob and Chub concluded that the drypump that serves as TP2's forepump had failed. Steve had told me the whereabouts of a spare Agilent IDP-7. This was meant to be a replacement for the TP3 foreline pump when it failed, but we decided to swap it in while diagnosing the failed drypump (which had 2182 hours continuous running according to the hour counter). Sure enough, the spare pump spun up and the TP2fl pressure dropped at a rate consistent with what is expected. I was then able to spin up TP1, TP2 and TP3. 

However, when opening V4 (the foreline of TP1 pumped by TP2), I heard a loud repeated click track (~5Hz) from the electronics rack. Shortly after, the interlocks shut down all the TPs again, citing "AC power loss". Something is not right, I leave it to Jon and Chub to investigate.

I can't explain the mechanical switching sound Gautam reported. The relay controlling power to the TP2 forepump is housed in the main AC relay box under the arm tube, not in the Acromag chassis, so it can't be from that. I've cycled through the pumpdown sequence several times and can't reproduce the effect. The Acromag switches for TP2 still work fine.

In any case, I've made modifications to the vacuum interlocks that will help with two of the issues:

1. For the "AC power loss" over-triggering: New logic added requiring the UPS to be out of the "on line power, battery OK" state for ~5 seconds before tripping the interlock. This will prevent electrical transients from triggering an emergency shutdown, as seems to be the case here (the UPS briefly isolates the load to battery during such events).
2. PSL interlocking: New logic added which directly sets C1:AUX-PSL_ShutterRqst --> 0 (closes the PSL shutter) when the main volume pressure is 3 mtorr-500 torr. Previously there was a channel exposed for this interlock (C1:Vac-interlock_high_voltage), but c1aux was not actually monitoring it. Following the convention of every vac interlock, after the PSL shutter has been closed, it has to be manually reopened. Once the pressure is out of this range, the vac system will stop blocking the shutter from reopening, but it will not perform the reopen action itself. gautam: a separate interlock logic needs to be implemented on c1aux (the shutter machine) that only permits the shutter to be opened if the Vac pressure range is okay. The SUS watchdog style AND logic in the EPICS database file should work just fine.

After finishing this vac work, I began a new pumpdown at ~4:30pm. The pressure fell quickly and has already reached ~1e-5 torr. TP2 current and temp look fine.

The interlocks tripped at ~630am local time. Jordan reported that TP2 was supposedly running at 52 C (!).

V1 was already closed, but TP2 was still running. With him standing by the rack, I remotely exectued the following sequence:

• VM1 closed (isolates RGA volume).
• VA6 closed (isolates annuli from being pumped).
• V7 opened (TP3 now backs TP1, temporarily, until I'm in the lab to check things out further).
• TP2 turned off.

Jordan confirmed (by hand) that TP2 was indeed hot and this is not just some serial readback issue. I'll do the forensics later.

Here is the timeline. This suggests TP2 backing RP failure.

1st line: TP2 foreline pressure went up. Accordingly TP2 P, current, voltage, and temp went up. TP2 rotation went down.

2nd line: TP2 temp triggered the interlock. TP2 foreline pressure was still high (10torr) so TP2 struggled and was running at 1 torr.

3rd line: Gautam's operation. TP2 was isolated and stopped.

Between the 1st line and 2nd line, TP2 pressue (=TP1 foreline pressure) went up to 1torr. This made TP1 current increased from 0.55A to 0.68A (not shown in the plot), but TP1 rotation was not affected.

The foreline pressure of TP2 was 1.4 Torr this morning. This drypump worked well for ten months.

Recently rebuilt  drypump with new seal was swapped in.

This is how you do it: close  V1,  V4 and turn off TP2. Replace drypump and start up TP2

Set pump speed to 50 K rpm and open V4 to TP1 Note that the Maglev was not turned off  because V4 was closed off only 5-10 minutes.

Open V1 the status is Vac Normal.

TP2 is rotating at  50K rpm, current pick up 0.2A,  the temp is 26C and its foreline pressure 33 mTorr

Dry fore pump of TP3 replaced by brand new Varian SH-110  at  1.1 Torr_75,208 hrs

The annuloses were closed off for 25 minutes. We are back to VACUUM NORMAL mode.

The TP3 fore line pressure dropped to 44 mTorr at 25 minutes in operation.......9.4mTorr at day 2 with full annulos load

I removed the forepump (Varian SH-110) for TP3 today to see why it had failed over the weekend. I tested it in the C&B lab and the ultimate pressure was only ~40torr. I checked the tip seals and they were destroyed. The scroll housing also easily pulled off of the motor drive shaft, which is indicative of bad bearings. The excess travel in the bearings likely led to significant increase in tip seal wear. This pump will need to be scrapped, or rebuilt.

I tested the spare Varian SH-110 pump located at the X-end and the ultimate pressure was ~98 mtorr. This pump had tip seals replaced on 11/5/18, and is currently at 55163 operating hours. It has been installed as the TP3 forepump.

Once installed, restarting the pump line occured as follows: V5 Closed, VA6 closed, VASE Closed, VASV closed, VABSSCI closed, VABS closed, VABSSCO closed, VAEV closed, VAEE closed,TP3 was restarted and once at normal operation, valves were opened in same order.

The pressure differential interlock condition for V5 was temporaily changed to 10 torr (by Gautam), so that valves could be opened in a controlled manner. Once, the vacuum system was back to normal state the V5 interlock condition was set back to the nominal 1 torr. Vacuum system is now running normally.

Jordan, JC

While the pumpspool is vented, I thought it would be a convenient time to change out the tip seal on the TP3 forepump. This one had not been changed since 2018, so as preventative maintence I had JC remove the pump and begin cleaning/installing the new tip seal.

Unfortunately the tip seal broke, but I have ordered another. We should have this pump ready to go late next week. If one is needed sooner, there is a spare IDP 7 pump we can install as the TP3 forepump.

[JC, Jordan]

Jordan recieved the new tip seal Friday afternoon and I continued the replacement process in the morning. Finishing up, we proceeded to test the pump in the Clean and Bake room. The pump's pressure lowered to 110 mTorr, and we continue pumping so the seal can recieve a good fitting.

Update: We have confirmed the pump is working great and have reinstalled this back into the vacuum system.                                                                                                                   Note: The same O-Rings were used.

I came to the campus and Gautam notified that he just had received the alert from the vac watchdog.

I checked the vac status at c1vac. PTP3 went up to 10 torr-ish and this made the diff pressure for TP3 over 1torr. Then the watchdog kicked in.

To check the TP3 functionality, AUX RP was turned on and the manual valve (MV in the figure) was opened to pump the foreline of TP3. This easily made PTP3 <0.2 torr and TP3 happy (I didn't try to open V5 though).

So the conclusion is that RP for TP3 has failed. Presumably, the tip-seal needs to be replaced.

Right now TP3 was turned off and is ready for the tip-seal replacement. V5 was closed since the watchdog tripped.

Disconcerting because those tip seals were just replaced [15417]. Maybe they were just defective, but if there is a more serious problem with the pump, there is a spare Varian roughing pump (the old TP2 dry pump) sitting at the X-end.

I reset the interlock error to unfreeze the vac controls (leaving V5 closed).

Gautam and I debugged a communications problem with TP3 that was causing its python service to fail. We traced the problem back to the querying of the pump controller for its operational parameters (speed, voltage, temp). Some small percentage of the time (~5%, indeterministically), the pump controller is returning an invalid response which causes the service to shut itself down and signal a NO COMM error.

As a temporary fix, I wrapped the failing query in an exception handler to continue past this particular error. However, we suspect the microprocessor in the TP3 controller may be beginning to fail. There is a spare controller sitting right next to it in the vacuum rack. We will ask Chub to install the replacement in the near future.

gautam: this pump is responsible for pumping the annular volume under normal operations. while this problem is being resolved, the annular volume is valved off (as it has been since July 2019 anyway which is when this problem first manifested).

PSL shutter closed at 6e-6 Torr-it    

   The foreline pressure of the drypump is 850 mTorr at 8,446 hrs of seal life

V1 will be closed for ~20 minutes for drypump replacement..........

9:30am dry pump replaced, PSL shutter opened at 7.7E-6 Torr-it

  Valve configuration: vacuum normal as  TP3 is the forepump of the Maglev  & annuloses are not pumped.

The forline pressure of TP3 was 399 mTorr

It was replaced this morning at  TP3 controller 134,638hrs with the "failed TP2 station" drypump. The foreline pressure now at TP3 is 100 mTorr at 6 hrs of operation.[ at day 3  63 mT ]

IFO pressure at CC Hornet 7.9e - 6 Torr

Valve configuration: vacuum normal as TP3 is the forepump of the Maglev & the annuloses are not pumped

The TP3 foreline pressure was 4.8 Torr, 50K rpm 0.54A and 31C........Maglev rotation normal 560 Hz.......    IFO pressure 7.2e- 6 Torrit was not effected

V1 closed ......replaced drypump.........V1 opened

IFO 6.9e-6 Torrit at 19:55, TP3fl 18 mT,  50Krpm 0.15A 24C

VM1 is still closed

 TP3 dry pump  replaced at 540 mT as TP3 50K_rpm 0.3A with annulos load. It's top seal life time was 11,252 hrs

TP3 drypump replaced at 655 mTorr, no load, tp3 0.3A 

This seal lasted only for 33 days at  123,840 hrs

The replacement is performing well: TP3 foreline pressure is 55 mTorr, no load, tp3 0.15A at 15 min  [ 13.1 mTorr at d5 ]

Valve configuration: Vacuum Normal, ITcc 8.5E-6 Torr

[Jenne, Kiwamu, and Steve via phone]

Around 9:30pm, Kiwamu and I came back from dinner, and were getting ready to begin the beam scan measurements.  I noticed that one of the vacuum pumps was being very loud.  Kiwamu noted that it is the fore pump for TP3's turbo, which he and Steve replaced in January (elog 2538).  We had not noticed these noises before leaving for dinner, around 8pm.

We called Steve at home, and he could hear the noise through the phone. He said that even though it was really loud, since it was reading 3.3mTorr (on the display of the controller, in the vacuum rack just above head-height) which is close to the nominal value, it should be fine to leave.  He will check it out in the morning.  If it had been reading at or above ~1Torr, that's indicative of it being really bad, and we would have needed to shut it off.

For future reference, in case we need to turn it off, Steve said to use the following procedure:

1. Close VM3, to isolate the RGA, which is what this pump is currently (while we're at atmosphere) pumping on.  I don't know if there are other things which would need to be shut at this stage, if we were at vacuum nominal.

2. Close VM5, which is right in front of TP3, so TP3's pump is just pumping on itself.

3. Push the "Stop" button on the Turbo controller for TP3, in the vacuum rack, about waist level.  Turning off the turbo will also turn off the fore pump.

UPDATE, 1am: The controller in the rack is reading 3.1mTorr, so the pump, while still noisy, still seems to be working.

Is this one close to failure as well?

 The foreline pressure of TP3 is 2.9 mTorr  The drypump is loosing it's bearing and it is very noisy.

V3, V5 closed and TP3 small turbo controller off. This turned off the noisy forepump that has to be replaced.

RGA is running at cc4 2e-6 Torr

The RGA was turned off at cc4 1e-5 Torr

 The drypump is replaced at 95,781 hrs on TP3 controller time. The foreline pressure is 30 mTorr and dropping.

 It is 13 mTorr after 17 hours of pumping.

For some time now, I've been puzzled by the unreliability of the ASS_X dither alignment servo. Leaving the servo on, TRX often begins to decay to a lower value, and even after freezing the dither at the maximum TRX values, I can manually align the mirrors to increase TRX. We have suspected some kind of clipping in the TRX path that is responsible for this behaviour. Today I decided to investigate this a bit further. To have the arm locked and to inspect the beam, we have to change the locking trigger - TRX is what is normally used, but I misaligned the Y arm completely, and used AS110 as a trigger instead. There is some strangeness in the triggering topology, but this deserves a separate elog.

Once the arm was locked (and relocks using the AS110 trigger in the event of an unlock), I was able to trace the beampath on the EX table with an IR card. The TRX beam is rather large and weak, so it is hard to see, but as best as I can tell, the only real danger of clipping (or perhaps the beam is already clipped) is on the final steering mirror before the beam hits the (Thorlabs) PD. Steve/Pooja are working on getting a photo of this, and will upload it here shortly. Options to mitigate this:

1. Use the harmonic separator to steer the beam lower, and center it on the 1" steering mirror. However, this could possibly lead to clipping on some of the upstream lenses.
2. Raise the height of the 1" steering mirror by 0.25". However, this would require a custom 3/4" dia post height or some shims, which I am not sure is in line with our optomechanic mounting practises.
3. Use a 2" mirror instead of a 1" mirror.

The EX QPD has stopped working since the Acromag install. If it were working, we wouldn't have to rely on the alternate triggering with AS110 and instead just use the QPD as TRX, while we debug the Thorlabs PD path.

I opted for the quickest fix - I raised the height of the offending steering mirror using a 0.25" shim. In the long term, we can get a taller post machined. After raising the mirror height, I then checked the DC centering of the spot on the DC PD using a scope.

Looking at the performance of the X arm ASS, I no longer see the strange oscillatory behaviour I described in my previous post . Moreover, the TRX level was ~1 before be raising the steering mirror - but it is now ~1.2. So we were certainly losing some power.

These are the settings which determine the transmon (eg, TRX) amplitude, and which are updated by the matchTransMon scripts.

For the X arm

op440m:AutoDither>tdsread C1:LSC-TRX_GAIN C1:LSC-LA_PARAM_FLT_01 C1:LSC-LA_PARAM_FLT_00
0.0023
0.155
119.775

For the Y arm

op440m:AutoDither>tdsread C1:LSC-TRY_GAIN C1:LSC-LA_PARAM_FLT_04 C1:LSC-LA_PARAM_FLT_03
0.00237196
-0.116
19.9809

[Jenne, Diego, Rana]

This is a note about work done last night. 

We were starting to lock, and saw glitches in the Thorlabs TRY PD about once every 1/60th of a second.  It is not a sine wave, so it is not 60Hz line noise directly.  It looks like this:

Rana pointed out that this looks like it could be from a power supply that is converting AC to DC. 

We went down to the Yend, and noticed some weird symptoms.  So far, we do not know where the noise is coming from.  Rather, we are just using the QPD for locking.

* The noise comes and goes, particularly if someone is moving around at the end station.

* Moving the Thorlabs power supply farther from the HeNe power supply didn't do much.  Turning off and disconnecting the HeNe supply didn't make the noise go away, so we conclude that it is not the HeNe's fault.

* We suspected the loops of excess cable that were sitting on top of iscey, but moving the coils away from the computer did not make the noise go away.

* We removed a few disconnected BNC cables that were near or touching the end table, but that didn't fix things.

* We disconnected the PD's signal cable and pulled it out of the table enclosure, and then put it back.  Noise was gone when cable was disconnected (good), but it was back after plugging the cable back in.

* The noise still comes and goes, but we don't have to use the Thorlabs PD for locking, so we leave it for another day.

RXA: also moved the Thorlabs power supply to a different power strip and tried putting it closer/farther to the Uniblitz shutter controller. Another suspect is that its some PWM type noise from the doubler crystal temperature driver. Need to try turning off the heater and the Raspberry PI to if it effects the noise.

A more permanent fix than a crocodile clip was implemented. Should probably look to do this for the X end unit as well.

Rich came by the 40m to photocopy some pages from Hobbs, and saw me working on the 60 Hz hunting. As I suspected, the problem was being generated in the D040060. This board receives the photodiode signal single-ended, but has a different power ground than the photodiode (even though the PD is plugged into a power strip that claims to come from 1Y4). The mechanism is not entirely clear - the presence of these 60 Hz features seemed to be dependent on the light level on the TRY photodiode (i.e. they were absent when the PSL shutter is closed, and were more prominent when TRY was 0.9 rather than 0.5) but the PD certainly wasn't saturated - the DC signal was only ~100 mV when viewed on a scope. In any case, Rich suggested the simplest test would be to ground the BNC shield bringing TRY to the rack, to the local ground on the board, which I did using a crocodile clip. This did the trick, the TRY signal RMS is now dominated by the ~1 Hz seismic-driven variation.

 On a more pessimistic note - it looks like the elliptical reflector moving did not work, and the clipping in the Y arm persists . I am able to recover TRY~1 with the yaw offset on the ETM (which is still lower than the 1.06-1.07 Koji reported in Aug 2018, but I can believe that being down to the MC transmission being a few % lower at 15000cts rather than 15500), while the maximum I see without it is ~0.9. This is puzzling, because when the chamber was open, we saw that there was ~1.5" clearance between the edge of the reflector and the beam on an IR card. I suppose the input pointing could have been off by a small amount. So one of the primary vent objectives wasn't acheieved... But I will push ahead with the loss measurement.

This evening, I was not able to successfully transition CARM from ALS to 1/sqrt(trans) signals.  The TRY time series looked odd, so I took a spectra, and we have huge 60Hz noise in TRY. 

I found a lock stretch from around 6:30pm that did not show the 60Hz noise, and then there was a lock stretch around 8pm that did have the noise.  So, something happened at the Yend between 6:30 and 8pm tonight.

Asking around, this was the time frame in which Manasa was down at the Yend to realign the green beam, and to check cabling for the PZT_OUT and ERR_MON signals to the ADC.

Looking at the spectra, Rana noted that we have even as well as odd harmonics of the 60Hz line, which is unusual.

To try to diagnose the problem, Rana and I tried to make sure no cables' connectors were touching, and that no equipment was plugged in that shouldn't be.  We noticed that none of: the shutter, the Thorlabs TRY PD, or the QPD TRY are isolated from the table.  To see if perhaps the shutter was the problem, I turned off the power to the Yend green shutter, and unplugged the cable.  The cable is laying on the table, with the connector sitting on a piece of plastic to isolate it.  Removing the shutter from the system did not change anything. 

We don't see the 60Hz noise in the Xarm, so it's not on the laser light itself.  Also, we don't see the 60Hz lines in the Yarm feedback signal, so we're not putting the lines onto the mirror, and thus onto the Yarm's light. 

Manasa, can you please take a look, and see if you can figure out what is going on?  We need TRY so that we can transition to 1/sqrt(trans) signals for CARM.  Thanks!!

 I went to the Y end to look at the TRY 60Hz noise situation this morning. While looking at TRY noise on dtt, I found that just lifting the cable away from the cable bunch that runs out of the table suppressed the noise drastically. 

Attachment 1

I removed the unwanted bnc connector in the path of the already long TRY cable running from the PD to the 1Y4 rack and isolated it from the bunch. TRY became less noisy.

But the noise was back again earlier in the evening and it looks like the noise is very much related to the TRY cable. TRY cable might have moved from its sweet spot while I was around checking cable connections yesterday.

I couldn't find a spare to replace it right away today (We need a BNC to 4 pin lemo).

 The detectors and electronics on this table are not properly isolated. To reduce the 60 Hz and ground loops, photodiodes and shutter must be isolated by using plastic spacers as we usually do elsewhere - this table just seems to have a few oversights.

Steve can start assembling all of the pieces to do this in the morning and then we can start the swapping after the meeting.

The high gain Transmon cable should be a regular BNC. There's no need for 4-pin LEMO in this usage, so the best move is to modify the board and replace the 4-pin LEMO connector with an isolated panel mount BNC female.

The AC adapter for this diode (and all of the detectors on the table) should get their power from a power strip which gets plugged into the rack with the whitening boards. The SHG oven, the Uniblitz shutter, and any cameras can get their power from another power strip if needed/wanted.

[Steve, Manasa]

To find noise source

1. Swapped the power cable of the PD and checked that it is connected to the right power source.

2. Changed the aluminium base of the post holding the diode so that the diode is floating

3. Grounded the table and the rack

4. Routed the cable on the other side of the beam tube to isolate it from other cables.

After all the above, we still found that shaking the cable was making TRY noisy.

I pulled out the PD whitening board to replace the 4 pin lemo connector with a bnc connector so that we can swap the cable with a new one. So there is no TRY right now.