C1:PSL-PMC_PMCTRANSPD was increased from 0.72 to 0.731

I don't know why, but as you can see in Steve's plot from earlier this morning, the PMC transmission has been going down significantly all weekend.  The PMC refl camera was very bright.  I tweaked up the alignment (mostly pitch), and now we're back to normal. 

The IMC was having trouble staying locked all morning, and I'm hoping that this PMC adjustment will help - the MC already looks better, although it's only been a few minutes.

Summary:

1. The input beam to the PMC cavity was changed back to the zeroth order beam from the AOM. 
2. The PMC was locked and nominal transmission levels were recovered.
3. The AOM driver voltage was set to 0V DC. 
4. A razor beam dump was placed to catch the first (and higher order) beams from the AOM (see Attachment #1), but allow the zeroth order beam to reach the PMC cavity.
5. Some dangling cabling was cleared from the PSL enclosure.

Details

• HEPA turned to 100% while work was going on in the PSL enclosure.
• Input power to the PMC cut from ~1.3 W to ~20 mW using the first available HWP downstream of the laser head, before any realignment work was done.
• Next, the beam dump blocking the undeflected zeroth order beam was removed.
• Triangle wave was applied to the PZT servo board "EXT DC" input to sweep the cavity length to make the alignment easier.
• After some patient alignment, I could see a weak transmitted beam locked to some high order mode, at which point I increased the input power to 200mW, and did the fine alignment by looking at the mode shape of the transmitted beam.
• Once I could lock to a TEM00 mode, I bumped the power back up to the nominal 1.3W, I fine tuned the alignment further by minimizing PMC REFL's DC level. 
• Dialled the power back down (using HWP) for installation of the beam block to catch the AOM's first (and higher order) beams.
• Checked that the reflected beam from the PMC cavity is well centered on the PMC REFL PDH photodiode. The ghost from the AR coating of the high-T beamsplitter is blocked by the iris installed by yehonathan on Friday. 
• The beam was a little low on the PMC REFL CCD camera - I raised the camera by ~1cm.
• With the beam axis well matched to the PMC, I measured 1.33 mW going into the cavity, and 1.1 W transmitted, so . Whatever loss numbers we extract should be consistent with this fact.
• HEPA turned back down to 30% shortly after noon.

Note that for all the alignment work, only the two steering mirrors immediately upstream of the PMC cavity were touched.

PZT monitor is not lying to us, I've measured it with a voltmeter. But PMC SERVO is still interesting. If I break the loop after PMCERR signal monitor (C1:PSL-PMC_BLANK=0), I can change PZT voltage from 0V (C1:PSL-PMC_RAMP = -7.3) to 132V (C1:PSL-PMC_RAMP=-10.0). If I break the loop by enabling TEST1, PZT voltage goes up to 294 V, though voltage on the TEST1 and MIXER MON is 0.

{Jon, Yehonathan}

We burt-restored the PSL and the PMC locked immediately.

The PMC is now locked on the AOM first order mode.

{Yehonathan, Jon}

We are able to lock the PMC on the TEM00 mode of the deflected beam.

However when we turn off the driving voltage to the AOM and back on the lock is not restored. It get stuck on some higher order mode.

There are plethora of modes present when the PZT is scanned, which makes us believe the cavity is misaligned.

To lock again on the TEM00 mode again we disconnect the loop (FP Test point 1), find a TEM00 mode using the DC output adjust and close the loop again.

Make sure to measure the power drop of the beam downstream of the AOM but before the PMC. Need to plot both together to make sure the chop time is much shorter than the 1/e time.

 The PMC PZT voltage slider seemed sticky.  First it would not do anything, than after moving slider back an forth a few times, it had a range of 60V and later it had full range and it locked

The PMC was locking right the way, but it's transmission would not go up.  Finally I get it back up by moving the "sticky" DC Gain slider up and down a few times.

 The FSS was -2.9, and the PMC won't lock happily unless you bring this back to 0.  The symptom that this is happening is that the PMC reflection camera is totally saturated, but the PMC still looks like it's locked on 00.

- PMC suddenly refused to lock.

- Investigated what's wrong

- Finally, I touched RF Output Adjust (C1:PSL-PMC_RFADJ). Then it started locking.

- C1:PSL-PMC_RFADJ was set to 2.0 by rana when we looked at the PMC LO issue.
  Now PMC does not lock with this value. I set it to 6.0 so that the lock is robust.

- Right before I lost PMC locking, I had some difficulty in locking IMC. Of course,  the robustness of the PMC is related to the robustness of the IMC.
  We definitely need to investigate this. (RF powers, open loop TF, etc)

I went down to the IFO hall and checked the PMC situation. The freq mixer fixed on the rack 1X1 had BNC-T, so the raw error was checked while the PMC was swept around the resonance.
The error signal was ~1mVpp while there is ~1mV offset.

There are a few possibilities: modulation, beam, PD, demodulator, or something else.

I checked if the PD was busted, misaligned, unpowered, etc, but there was no clear sign of what was wrong.
The beam was not well steered on the PD but realigning the PD didn't help the size of the error signal.

Modulation... a possibly high-power RF amp is usually delicate so without consulting with the circuit schematic, I did not want to disconnect the output from a proper load.
Instead, I just checked the modulation level monitor on the PMC Phaseshifter screen. C1:PSL-PMC_MODET said 0~-0.004, which didn't mean anything.
By the way, the LO output had ~1Vpp observed with 50ohm termination on an oscilloscope. That looked OK.

But the history of this channel told us that there was some negative number, which we lost ~3.5 days ago. So it's suspicious.

I checked the circuit schematic of the oscillator box D980353. As I said, the LO was OK. So the oscillator would also be OK. Maybe the EOM path has something.
There is a notorious ERA-5SM in the path. So the first thing to check is this chip. Of course, we have many spares in the blue tower.
 

The attached plots show the PMC cavity line width measurement with 1 mW and 160 mW into the PMC. The two curves on each plot are the PMC transmitted power and the ramp of the fast input of the NPRO. The two measurements are consistent within errors - a few %. The PMC line width  3.5 ms (FWHM) x 4 V / 20 ms (slope of the ramp) x 1.1 MHz / V (NPRO fast actuator calibration from Innolight spec sheet) = 0.77 MHz.

Here is the output of the calculation using Malik Rakhmanov code:

modematching =  8.4121e-01

transmission1 =   2.4341e-03

transmission2 =   2.4341e-03

transmission3 =   5.1280e-05

averageLosses =  6.1963e-04

visibility =  7.7439e-01

The PMC local oscillator is going a little weird dying.  We need to check out why the level is fluctuating so much.

Here's a 6 month plot, where you can see that the lower level keeps getting lower (y-axis is dBm):

This LHO entry from 2008 shows where we first discovered this effect. As Rick Savage and Paul Schwinberg later found out, the ERA-5SM+ amplifier slowly degrades over several years and was replaced for both of the eLIGO interferometers. We have spares in the Blue box and can replace this sometime during the day.

Our PMC LO is made by this obsolete crystal oscillator circuit: D000419. There are many versions of this floating around, but they all have the ERA-5 issue.

It seems that there is no better chip in MiniCircuits line-up with the same form factor.
ERA-5 is the most powerful one in the ERA (or MAR) series.

If the output is ~0dBm we have MAR-6SM in stock. But I suspect that ERA-5 was driven at the power level close to its saturation (~18dBm).

If we allow different form factors, we have GVA-** or GALI-** in the market and also in the blue tower, in order to gain more performance margin.
If it is difficult to apply them, I would rather use another ERA-5 with enhanced heat radiation.

I'm sure that Downs has EAR-5 replacement.

Koushik and Koji try to fix the PMC oscillator issue. So we remove the module from the rack.
This means we don't have the PMC transmission during the work.

Koushik replaced an ERA-5 in the PC path. We put the module back to the rack and found no change.
The epics LO level monitor monitor is still fluctuating from 6~11dBm. We need more thorough investigation
by tracing the signals everywhere on the board.

Despite the poor situation of the modulation, PMC was locking (~9PM). Rana suspect that the PMC demodulation
phase was not correctly adjusted long time. 

Koushik has the measured power levels and the photos of the board. I'll ask him to report on them.

 After the ERA-5 was replaced (see Koushik elog) we relocked the PMC.

The new LO level going into the PMC servo card is +11.5 dBm. The LO mon on the PMC card reads 9 dBm and seems so flat I now suspect the monitor circuit.

I also measured the RF drive to the EOM as a function of C1:PSL-PMC_RFADJ on the Phase Shifter screen.

The phase shifter slider gives ~75 deg/V in phase shift of the RF out to the EOM. I tried to optimize the loop gain quickly using the fluctuations in the reflected power. The loop oscillates at high frequency with the slider at 21 dB and also at 9 dB. So I set the gain at +14 dB. Needs to be optimized correctly in the daytime.

 Updates from Koushik:

The power levels measured (before and after relacement of ERA-5) are as follows:

LO to Servo : Vout = 2.3 Vpp / Pout = 11.21 dBm at f = 35.5 MHz

RF to PC   :   Vout = 354 mVpp / Pout = -5.1 dBm at f= 35.5 MHz

The measurements were done using an oscilloscope with 50 ohms load impedance. Unfortunately the photos are not available from the camera.

The PMC HV drops off more offen lately.

The PMC acted like it was sleeping. The HV slider was dead. The MC locked instantly as the PMC had transmission.

I locked the PMC and the MC followed instantly.

 The PMC  behavior is not changed.

 The PMC was locked and the MC followed intantly

The PMC locked manually. MC  grabbed lock instantaniously

 PMC locked

 PMC locked.

PMC needed to be locked manually.

The PMC was unlocked earlier this morning, for ~20min, presumably from the rocks next door.  I relocked it.

Then, a few min ago, the PMC suddenly decided that it wouldn't lock with a transmission greater than ~0.7 .  I found that the laser temp adjust on the FSS screen was at -1.9ish.  I put it back to zero, and now the PMC locks happily again.  I think we got into a PSL mode-hopping temperature region on accident.

I made a python script for relocking PMC.
It currently lives in /opt/rtcds/caltech/c1/scripts/PSL/PMC/PMClocker.py.

I think the hardest part for this kind of locker is the scan speed. I could make the scan relatively fast by using pyNDS.
The basic algorithm is as follows.

1. Turns off the servo by C1:PSL-PMC_SW1.

2. Scans C1:PSL-PMC_RAMP using ezcastep.bin. Default settings for ezcastep is

ezcastep.bin C1:PSL-PMC_RAMP -s 0.1 0.01 10000

So, it steps by 0.01 for 10000 times with interval of 0.1 sec.

3. Get C1:PSL-PMC_PMCTRANSPD and C1:PSL-PMC_RAMP online 1 sec data using pyNDS.

4. If it finds a tall peak in C1:PSL-PMC_PMCTRANSPD, kills ezcastep.bin process, sets C1:PSL-PMC_RAMP to the value where the tall peak was found, and then turns on the servo.

5. If tall peak wasn't found, go back to 3 and get data again.

6. If C1:PSL-PMC_RAMP reaches near -7 V or 0 V, it kills previous ezcastep.bin process and turns the sign of the scan.

I tested this script several times. It sometimes passes over TEM00 (because of the dead time in online pyNDS?), but it locks PMC with in ~10 sec.
Currently, you have to run this to relock PMC because I don't know how to make this an autolocker.

I think use of pyNDS can be applied for finding IR resonance using ALS, too.
I haven't checked it yet becuase c1ioo is down, but ALS version lives in /users/yuta/scripts/findIRresonance.py. ALS may be easier in that we can use fast channels and nice filter modules.

Other scripts:
 I updated /opt/rtcds/caltech/c1/scripts/general/toggler.py. It now has "lazymode". When lazymode, it toggles automatically with interval of 1 sec until you Ctrl-c.

 Also, I moved damprestore.py from my users directory to /opt/rtcds/caltech/c1/scripts/SUS/damprestore.py. It restores suspension damping of a specified mirror when watchdog shuts down the damping.

I thought we rewrite auto lockers once per year, but this time it took us only a month. I wrote it for PMC on May 24. Is it not working?

Could someone make it more clear why some scripts are written on bash, others on sh or python? I think we should elaborate a strict order. Masha and I can work on it if anyone else considers this issue as a problem.

I know.
I just wanted to use pyNDS for this kind of scanning & locking situation.
c1ioo was down for the weekend and I couldn't test my script for ALS, so I used it for PMC.

But I think PMClocker.py can relock PMC faster because it can sweep C1:PSL-PMC_RAMP continuously and can get continuous data of C1:PSL-PMC_PMCTRANSPD.

 In May of 2013 Den wrote a PMC Autolocker because he ignored / didn't want to read anyone else's code. Later that year Yuta also wrote another one from scratch for the same reasons.

I tried to use both today, but neither one runs. Yuta's one doesn't run because he was using a bunch of private yuta library stuff in the yuta directory. That kind of programming style is pretty useless for us since it never works after some time.

So I re-activated and tested the PMCAutolock bash script (it is actually a symbolic link called "PMCAutolock" which points to AutoLock.sh). These scripts are all basically the same:

They turn off the loop (or turn down the gain) and then scan the PZT, look for a resonance, and then activate the loop.

One problem with the logic has been that turning off the loop makes the gain so low that the peak flashes by too fast. But leaving the loop ON and just sweeping with the gain turned down to -10 dB is also not good. That only reduces the UGF from 1 kHz to ~100 Hz. What we want is more like a 10 Hz UGF while scanning the length. SO, I edited the script to turn down the modulation depth on the EOM by that factor. After acquiring lock, it returns all settings to the nominal levels as defined on the PSL_SETTINGS screen.

I've tested it a few times and it seems to work OK. You can run it from the yellow shabang button on the PMC screen.

I also changed the .bashrc aliases for the MEDM command so that if you type medm_good at the command line you get MEDM screens with scalable fonts. So you can stretch the screens.

I used a script (~PSL/PMC/testAutoLocker.sh) to unlock the PMC and run autlocker ~100 times to see how robust the new autlocker is.

It failed to grab it 2 out of 137 times. During those times it just went on trying to ramp the PZT even after it had gone to a rail. Once someone resurrects Rob's 'trianglewave'  script we should be OK. Even so, I think this is good enough. Please try this out via the yellow button next time the PMC needs to be locked.

It usually takes 10-30 seconds to lock, depending upon where the fringe is compared to the upper voltage rail. Good enough.

PMC wasn't locking. Had to power down c1psl. Did burt restore. Still not great.

I think many of the readbacks on the PMC MEDM screen are now bogus and misleading since the PMC RF upgrade that Gautam did awhile ago. We ought to fix the screen and clearly label which readbacks and actuators are no longer valid.

Also, the locking procedure is not so nice. The output V adjust doesn't work anymore with BLANK enabled. Would be good to make an autolocker script if we find a visitor wanting to do something fun.

After I recovered the lock of PMC, I found that the PMC transmission was quite low. It was about 0.26 in the EPICS display.

I zeroed the PSL temperature feedback value which had been -2.3 and then the PMC transmission went back to a normal value of 0.83.

I believe it was because the PSL was running with two different oscillation modes due to the big temperature offset.

Kiwamu and I found that the first lens in the PMC mode matching telescope was mislabeled. It is supposed to be PLCX-25.4-77.3-C and was labeled as such but in fact it was PLCX-25.4-103.0-C. This is why the PMC mode matching was bad. We swapped the lens for the correct one and got the PMC visibility of 82%. The attached plot shows the beam scans before and after the PMC. The data were taken with the wrong lens. The ABCD model shown in the plot uses the lens that was there at the time - PLCX-25.4-103.0-C. The model for the PMC is just the waist of 0.371 mm at the nominal location. The snap shot of the ABCD file is attached. The calculation includes the KTP for FI and LiNb for EOM with 4 cm length. The distances are as measured on the table.

 The PMC mode matching was initially done at low power ~150 mW. It was expected and found that at full power ~2 W (injection current 2.1 A) the mode matching got much worse:

the visibility degraded from 80% to 50% (1 - refl locked/refl unlocked) . The thermal lensing could be in the laser, EOM, or FI.

The first attached plot shows the scan of the beam after the EOM at low and full laser power. At full power the waist position is 10 mm after the turning mirror after the EOM and the waist size is 310 um.

The second plot shows the ABCD calculation for the mode matching solution.

I removed the MM lens PLCX-25.4-77.3-C and placed the PLCX-25.4-180.3-UV about 20 mm after the first PMC periscope mirror (the second mirror after the EOM).

The PMC visibility improved to 94% and the power through the PMC, as measured by the PMC transmission PD, went up by a factor of 2.

I moved the lens just before the PMC to check the mode matching landscape. The PMC trans went up from ~6.5 to ~6.8. That's 5% with ~1 hour of work.

As per the micrometer, this took ~7-8 mm of travel. Since there's so much power left in the HOMs, we we we will have to do a proper mode scan and re-calculate the solution.

The measured transmission is now ~610 mW. The power reflected from the PMC with it unlocked is ~1400 mW.

Summary:

I estimate the PMC servo modulation depth to be approximately 50 mrad. This is only 15% lower than what was measured in Jan 2018, and cannot explain the ~x50 reduction of optical gain measured earlier in this thread. Later in the day, I also confirmed that the LO input to the ZAD-6 mixer is +7 dBm. So the crystal is not to blame.

Details:

1. PSL frequency is locked to the IMC length.
2. Arm lengths are locked to the PSL frequency using POX/POY.
3. EX green laser locked to the X arm length using end PDH servo. GTRX was ~0.4 in this measurement, which is the nominal value.
4. The 20dB coupled port of the beat between the EX and PSL lasers was monitored using the AG4395A in "Spectrum" units.
5. The beat was set at ~90 MHz, and a spectrum was taken for ~100 MHz span centered at the beat frequency.
6. The modulation depth is estimated by considering the ratio of power at the beat frequency relative to that 35.5 MHz away. See Attachment #1.

Assuming a finesse of 700 for the PMC, we expect an optical gain of 2*Pin*J0(50e-3)*J1(50e-3)/fp  ~ 1.2e-7 W/Hz (=0.089 GW/m). I can't find a measurement of the PMC RFPD transimpedance to map this onto a V/Hz value. 

 The PMC power degrading on this 3 days plot. MC2 -T = 14,200 counts. C1:IOO-MC_TRANS_SUM can not be ploted in dataviewer. The MEDM screen has a valid number.

Initial pointing is not so bad (what does "not so bad" mean ???)

C1iscey comes and goes again.

The PMC is losing power.

I'm just now realizing that the PMC has also not been locked since noon today, and doesn't seem to be responding to anything right now.

wtf is going on here?

{JC, Yehonathan}

When JC came this morning the IMC was completely misaligned. After tweaking the MC alignment we realized the MC WFS where comletely off the rail. We cleared their history and returned the suspensions to their original position. MC locked immediately.

However there still issues:

1. When WFS where turned on it (now its at 8.194) would slowly misalign the IMC.

2. PMC would get unlocked very quickly everytime the IMC tweaked due the PMC input offset being at the slider's edge at 10V.

First we dealt with the PMC issue. We brought the offset down and tried locking the PMC. When it didn't work we went to the PMC table and connected a triangular wave to EXT DC in the PMC servo. PMC reflection and tansmission where observed using a scope while triggering on the triangular wave.

We changed the input volatge until we could see transmission peaks during the voltage ramp. We aligned the PMC so that the HOMs are minimized. We were then able to lock the PMC at an offset input voltage of 8.2V. Transmission is at 0.67V REFL is at 0.03V.

WFS Servo fix:

We went to the AS table and centered the WFS. We run correct DC and AC WFS offset scripts. When then we locked the IMC and turned on the WFS servo. This time the WFS didn't rail, at least not for the last 1 hour. We also realigned MC2 transmission QPD.

1. Well, it seems that the PMC error offset issue still remains.

2. The IMC transmon QPD had been aligned for some reference e.g. Max transmission, Center of the MC2 mirror, lowest noise, or else. It's not arbitrary.
    Find the appropriate spot poisiton and then align the QPD for the given spot.

[Kevin Koji]

- The PBS alignment increased the transmitted power

- The first faraday and the PMC EOM were realigned.

- The transmission of the PMC increased from ~5.4V to ~6.5V.

Thus we need to pay attention to the incident beam power on to the MC
so that it does not exceed the power of 20-40mW.

Kevin will give us the detail of the work.

[Jenne, Yuta]

I looked at PMCR camera on the MC1 tv, and tweaked up the beam going into the PMC - it only needed a little bit of pitch.

Yuta and I measured the MC spots, determined (consistent with my measurements this morning) that they were only off in yaw.  We touched the 2nd steering mirror in the zigzag on the PSL table in yaw a small amount (top of knob away from me), realigned the MC, and things were good.  The plot is zoomed in to show only measurements taken today.  2 in the morning, before anything in the IFO room was touched.  1 this afternoon after tweaking PMC.  1st attempt at PSL beam tweaking was successful, 2nd measurement confirms it wasn't a fluke.