Baja 4.9 m earth quake tripped suspentions, except ETMX Sus damping recovered. MC is locking.
 

This has been edited several times over the last several hours, as I try to change different parameters, to see if they affect the movement of ETMX.  So far, I don't know what is causing the motion.  If it is there, it is only present when the LSC is engaged, so I don't think it's wobbling constantly on a twisted wire.

FINAL EDIT, 9:10pm:  The arm ASC was turning itself on when the arms were locked.  Whelp, that was only 3 hours of confusion.  Blargh.

For his penance for leaving the arm ASC engaged, Q has made a set of warning lights on the LSC screen, right next to the ASS warning lights.

ETMX might be having one of those days today, which is lame. 

So far tonight, I have run the LSC offset script, set the FSS slow value to +0.2, and run the arm ASS scripts.  Nothing too crazy I think.

Sometimes when I lock the single arms, the ETMs move around like crazy.  Other times, not.  What is going on here???  The ETMs don't move at all when they are not being actuated on with the LSC.

In this screenshot you can see the end of a POX/POY lock stretch where everything was nice and good.  Then, the arms were unlocked, and they have a bit of a DC offset.  After settling from that step, they continue sitting nice and still.  Then, I relock the cavities on POX and POY a little before -4 minutes.  ETMY takes a moment to pull itself together, but then it's steady.  ETMX just wobbles around for several minutes, until I turn off the LSC enable switch (happened after the end of this plot). 

I'm not going to be able to lock like this.  Eeek! 

This is somehow related to light being in the Xarm.  This next plot was taken while the arms were held with ALS in CARM/DARM mode. 

• Between -4 and -3, and again between -2 and -1.5, the arms were held with ALS "on resoanance".  Yarm power (green) is fine, Xarm power (purple) is not fine.  The ETMX OSEM pitch and yaw signals are bumpy at the same times.
• Between -3 and -2 minutes, the arms were held with ALS off resonance.  Obviously you can't see anything about IR transmission, but you can see that the SUSPIT and SUSYAW signals quiet down. 
• Between -1 and 0, I moved the Yarm ALS offset such that only it was resonating (the jags at the beginning of the resonance is me finding it, not to do with ETMY misbehaving). 
  • The ETMX SUS signals stay quiet. 
  • ETMY YAW has a DC offset when there is IR power in the arm.
• After 0 minutes on this scale, I move the offsets such that only Xarm is "on resonance". 
  • The ETMX osem signals go bumpy again.

I closed and re-opened all 3 green shutters.  Now (at least the last 8 arm locks in the last 6 mintues) ETMX has never gone wobbly, except for a little bit right after acquisition, to deal with whatever the DC offset it. Why is this changing? 

The arms were fine for one long ~30 minute lock while I stepped out for dinner.  At some point after returned, the MC lost lock. When the arms came back, ETMX was being fussy again.  Then, it decided that it was done.

In this plot, at -1 minute I started the ASS.  Other than that, I did not touch any buttons at all, just observed.  I have no idea why at about -3 minutes the bad stuff seems to go away. 

I was curious if it had to do with the DC pointing of the optics, so I unlocked the arms, put ETMX about where it was during the long good lock stretch, then reaquired lock.  I had to undo a little of that so that it would lock on TEM00, but at the beginning of the lock stretch (starting at about -3) the pitch is about the same spot.  But, the oscillations persist.  This time it was clear that the oscillations were around 80 mHz, and they started getting bigger until they settled to an amplitude they seemed to like.

Seems pretty independent from FSS temp.  There are 3 lock stretches in the next plot (easier to see by looking at the Yarm transmission, green trace).  The first one, the FSS slow was at 0.35.  the middle one, it was around 0.05.  The last one, it was around -0.4.  Other than the different DC pointings (which I don't know if they are related), I don't see anything qualitatively different in the movement of ETMX.

The temperature of the east and south ends  are normal, they are about the same.

The BS oplev servo was kicking up the BS. It was turned off

I just realized that the "damprestore" script that can be called from the watchdog screen did not have the new oplev names.  I have updated it, and added it to the svn.

The oplev situation still seems unresolved - notice this DTT. I guess there are still inconsistencies in the screens / models etc.

Could use some some investigation and ELOGGING from Eric.

           March  19, 2015   2  new  JDSU 1103P, sn P919645 & P919639 received from Thailand through Edmond Optics. Mfg date 12/2014............as spares

In addition to (and probably related to) the XARM ASS not working today, the ITMX has been jumping around kind of like ETMX sometimes does.  It's very disconcerting. 

Earlier today, Q and I tried turning off both the LSC and the oplev damping (leaving the local OSEM damping on), and ITMX still jumped, far enough that it fell off the oplev PD. 

I'm not sure what is wrong with ITMX, but probably ASS won't work well until we figure out what's up.

I tried a few lock stretches (after realigning the Xgreen on the PSL table) after hand-aligning the Xarm, but the overall alignment just isn't good enough.  Usually POPDC gets to 400 or 450 while the arms are held off resonance, but today (after tweaking BS and PRM alignment), the best I can get POPDC is about 300 counts. 

Den and I are looking at the ASS and ITMX now.

So, was there real shifting in the ITMX alignment as seen in the DV trend or just mis-diagnosis from the ETMX violin mode? Or how would the ETMX violin mode drive the ITMX with the LSC feedback disabled?

I've been poking around the oplev situation. One thing I came across regarding ITMX was that the gain on segment 4 seems to be about higher than the other segments. I was led to believe this by steering the optic around, and looking at the counts on each quadrant when the other 3 were dark.

Putting a gain of 0.86 (the ratio of the other segments' max counts over segment 4's max counts) in the segment 4 FM flattens the 1 Hz peak in the ITMX_OL_SUM spectrum, as well as significantly reducing the sub-Hz coherence of the sum with the individual quandrant counts. This is what I would expect from reducing the coupling of angular motion due to quadrant gain mismatch into the sum. 

Here are the ITMX_OL_SUM spectra before and after (oplev servos are off).

The "burps" and control filter saturations are still unexplained. Investigations continue...

ITMX, ETMY, BS and SRM are oscillating ?

The BS oplev has been misbehaving and kicking the optic from time to time since noon. The kicks are not strong enough to trip the watchdogs (current watchdog max counts for the sensors is 135).

I took a look at the spectrum of the BS oplev error in pit and yaw with both loops enabled while the optic was stable. There is nothing alarmingly big except for some additional noise above 4Hz.

I have turned the BS oplev servo OFF for now.

I saw this kicking before  

I think that this happens when the beam gets too close to the edge of the QPD.  We see this regularly in the ETMs, if they've been kicked a bit, but not enough to trip the watchdogs.  I think it might be the step/impulse response of the RES3.3 filter, which rings for almost 20 seconds. 

Anyhow, I've just recentered the BS oplev.  It was at -21urad in pitch, and had more than 400 counts on the top two quadrants, but only about 100 counts on the bottom two.  Now it's around 300 counts on all 4 quadrants.

As a totally unrelated aside, I have installed texlive on Donatella, so that I could run pdflatex.

The laser below is dead. JDSU 1103P, SN P845655 lived for 3.5 years.

It was replaced by JDSU P/N 22037130,( It has a new name for 1103P Uniphase ) sn P919639 of mfg date 12-2014

Beam shape at 5 m nicely round. Output power 2.8 mW of 633 nm

BS spot size on qpd ~1 mm &  60 micro W

PRM spot size on qpd ~1 mm & 50 micro W

ETMX sus damping restored.

Recently, Steve replaced the HeNe which was sourcing the BS & PRM OL. After replacement, no one checked the beam sizes and we've been living with a mostly broken BS OL. The beam spot on the QPD was so tiny that we were seeing the 'beam is nearly the size of the segment gap' effect.

Today I removed 2 of the lenses which were in the beam path: one removed from the common PRM/BS path, and one removed from the PRM path. The beams on both the BS & PRM got bigger. The BS beam is bigger by a factor of 7. I've increased the loop gains by a factor of 6 and now the UGFs are ~6 Hz. The loop gains were much too high with the small beam spots that Steve had left there. I would prefer for the beams to be ~1.5-2x smaller than they are now, but its not terrible.

Many of the mounts on the table are low quality and not constructed stably. One of the PRM turning mirror mounts twisted all the way around when I tried to align it. This table needs some help this summer.

In the future: never try locking after an OL laser change. Always redo the telescope and alignment and check the servo shape before the OL job is done.

Also, I reduced the height of the RG3.3 in the OL loops from 30 to 18 dB. The BS OL loops were conditionally stable before and thats a no-no. It makes it oscillate if it saturates.

After last week's work on the BS/PRM oplev table, I think the PRM oplev got centered while the PRM was misaligned.  With the PRM aligned, the oplev spot was not on the QPD.  It has been centered.

PRM watchdog tripped, but the damprestore.py script wouldn't run. 

It turns out the script tries to import some ezca stuff from /users/yuta (), which had been moved to /users/OLD/yuta (). 

I've moved the yuta directory back to /users/ until I fix the damprestore script. 

I will move it back. We need to fix our scripts to not use any users/ libraries ever again.

Local earthquake 3.8 Mag tripped only PRM

Vac monitor is not communicating.

PSL HEPA turned on
 

Lenses removed from oplev beam path at elog entry 11246

This afternoon, I showed Eric Gustafson some of the basics of making swept sine measurements with DTT. We turned off the f2a filters and oplev damping on the BS and made a cursory measurement of the transfer function from position drive to the oplev signals. 

He will be in the lab periodically to continue this line of investigations. 

Today I noticed the box around the ETMY oplev sum flashing red, as it dipped below 1k. I don't recall seeing this recently, so I wanted to look up the history.

However, we've been having trouble with our minute (and longer) trend data, so I had to hack it out a bit... Here is the unfortunate result:

I think we can be fairly confident that this is not due to alignment drifts, we generally keep the QPD reasonably well centered. I also recentered it today, and the counts remained at ~1k. 

Details of the hack that got me this data:

I ended up looking at the BURT snapshots from every night at midnight, which report a number for ETMY_OL_SUM_OUT16, and making a text file with dates and values with the following BASH spaghetti:

find /opt/rtcds/caltech/c1/burt/autoburt/snapshots/2015 -wholename "*00*/*scy*" |xargs ack --nogroup "ETMY_OL_SUM_OUT16 1" |  sed -e 's/.*2015/2015/g' -e 's/\/c1.*\([0-9]\..*$\)/, \1/g' -e 's/\//-/g'  > ETMYsum.txt

This produces a file full of unsorted lines like: 2015-Aug-23-00:07, 1.106459228515625e+03

The python package pandas is good at parsing dates and automatically plotting time series: 

Repeated for all optics, ETMY seems like the only one sharply dropping for now (PRM is all over the place and hard to gauge, since we often leave it partially- or mis-aligned):

Hacky bits:

Bash:

qontrols@pianosa|~ > head allOL.txt 
2015-Apr-10-00:07, BS, 1.146766113281250e+03
2015-Apr-10-00:07, ETMX, 1.597261328125000e+04
2015-Apr-10-00:07, ETMY, 4.331762207031250e+03
2015-Apr-10-00:07, ITMX, 6.488521484375000e+03
2015-Apr-10-00:07, ITMY, 1.387590234375000e+04
2015-Apr-10-00:07, PRM, 8.352053833007812e+02
2015-Apr-10-00:07, SRM, 6.099560928344727e+01
2015-Apr-1-00:07, BS, 1.180478149414062e+03
2015-Apr-1-00:07, ETMX, 1.584842480468750e+04

Python:

Rana, Steve

We investigated the ETMY oplev table set up and did not find a red herring.

Two 2 years  vs one day plot below.

ps: thanks  Q for fixing  DTT, the auto scaling is not working at sampling rate 10 min and 1 hr period??

Since Andrey's SUS Drift mon screen back in 2007, we've had several versions which used different schemes and programming languages. Diego made an update back in January.

Today I added his stuff to the SVN since it was lost in the NFS disks somewhere. Its in SUS/DRIFT_MON/.

Since we've been updating our userapps directory recently to pull in the screens and scripts from the sites, we also got a copy of the Thomas Abbott drift mon stuff which is better (Diego actually removed the yellow/red functionality as part of the 'upgrade'), but more complicated. For now we have the old one. I updated the good values with all optics roughly aligned just a few minutes ago.
 

Our last effort to change the existing Al-6061 wire standoffs was at April 2012

We requested sapphire and/or ruby materials with smaller R at the bottom of the  groove. Groove polishing was asked for.

Insaco Inc. quote 84740 as " best effort " NO POLISHING.  The groove cut to be with eximer laser.

Jeff Lewis as 9-12-2012:  the LIGO sapphire prisms grooves were NOT POLISHED  but Resonatics used the corner of a rasor blade to scrape off the

ablated material wich was redeposited in and around the grooves.
 

We have talked about the drift of ETMX sus on the Wednesday meeting.

It has stopped moving on Jan 8, 2015 and it has been reasanable stable since than.

I don't see any evidence of it getting more stable. It seems there was a big step in January, but the problem we were talking about - the suspension shifting when it gets a big kick - can't be proven to be gone or not by just looking at the trends. The real issue is whether or not it slips when we put in a large step in the LSC.

Gautam alerted me that the Y arm looked like it was being dithered, even though the ASS was turned off. I found that the ETMY OL signals were garbage, leading to the servos flipping back and forth between their rails. 

We went out to the ETMY table, and found the HeNe laser to be emitting a paltry <0.5mW; the OL QPD could not register the puny beam incident on it.

Here is the last 30 days of OL_SUM:

Steve will replace the laser this afternoon. 

This JDSU 1103P laser, sn P892324 lived for 2 years. It's power output is 0.05 mW now

It was replaced with brand new JDSU 1103P,  sn P919645, Mfg date 12/2014 with 2.75 mW output.

There is 0.14 mW  light returning to the qpd = 7,250 counts without AR 632 lenses

Using a modified version of Hang's deMod_deCoup scripts, I tuned the MC2 coil output matrix to minimize the appearance of POS drive in the SUSPIT signal at 28Hz. Up until now, there was no F2P compensation. This reduced the force to pitch coupling at 28Hz by 8dB. 

Old: POS -> 1 x UL, 1 x UR, 1 x LL, 1x LR

New: POS -> 1.1054 x UL, 1.1054 x UR, 0.8946 x LL, 0.8946 LR

I checked the MCL spectrum before and after this change with OAF on, this did not spoil the feedforward length subtraction in any noticible way. 

The script lives in userapps/release/isc/c1/scripts/decoup, but I've symlinked it to /opt/rtcds/caltech/c1/scripts/decoup.

The script modification I made had to do with how the I and Q data is collected. Before, it was sporadically probing the I and Q FM output monitor EPICS values; I changed it to use the avg function of cdsutils, which calculates the mean and std from the 16kHz data and have seen it improve results by 1dB or so. I've been in touch with Jenne to propagate this to the sites. 

The OSEMs cannot be used for coil balancing above ~10 Hz. The main coupling path from OSEM drive to sensor is not through the mirror motion, but instead direct electrical coupling of the drive wires to the sensing wires.

I put this in the elog every ~1-2 years since people keep trying it, but it keeps coming back like a zombie.

Better to use the MC angular sensors for L2A decoupling. Not perfect, but better than OSEMs. For the TMs we can use the OLs.

D'oh. Good point.

Reverted for now; I'm thinking about doing laser pointer->MC2 QPD...

The oplev and  the LSC are off.

I was trying to take a few more IR transmission scans with ALS when the ETMX got kicked again. I'm not sure how to fix this, so for the time being, I'm leaving the Oplev servo and the LSC turned off. The oplev spot looks really far off center especially in yaw, the yaw error is ~ -80.

I've turned the ETMX oplev servos off for the time being. (At the input side, so that no scripts will accidently turn it on).

Thus, only the local damping is being applied, let's see if we see any kicks...

During this time period, it looks like there was maybe one excursion. Here are the individual OSEM signals, which I think to be calibrated to microns. 

When I'm doing other things, I'm going to intentionally leave the watchdog tripped, and see if anything happens. 

During a ETMX kick that just occured, with only local damping on, the slow VMon channels didn't show any noticable change. 

How many volts does it take to pitch the ETM by 5 urad?

PMC

- Before any of the following work, I went to the PSL table and aligned the PMC. In fact, it has not been misaligned.

IMC

- It was claimed in the meeting today that the IMC had not been happy thesedays. I checked out what's happening.

- I found the IMC was still well aligned. Autolocker frequently stuck on a weak higher-order mode and couldn't recover TEM00 locking without help.

- I modified /opt/rtcds/caltech/c1/scripts/MC/mcdown for easier relocking on TEM00.

The MC_REFL_GAIN and MC_VCO_GAIN for relocking was set to be 27 and -3, in stead of 0 and 10, respectively.
This means that REFL_GAIN is not changed before and after the locking. Only VCO_GAIN is lowered for lock acquisition.
The corresponding lines in mcdown are excerpted here.

#set servo and boost gains for re-acquisition
#${ewrite} C1:IOO-MC_REFL_GAIN 0 &
${ewrite} C1:IOO-MC_REFL_GAIN 27 &
#${ewrite} C1:IOO-MC_VCO_GAIN 10 &
${ewrite} C1:IOO-MC_VCO_GAIN -3 &

We still have some chance of locking on higher-order modes. If I jiggle the VCO gain slider from -31 to 0, eventually I find TEM00.
I don't know how to do it in the script yet. For now, I increased tickle amplitude from 300 to 500.

/opt/rtcds/caltech/c1/scripts/MC/MC2tickleON

amp=300
=>
amp=500

- IMC was locked and aligned with the WFS. The WFS feedback offsets were offloaded to the alignment slider (via the MEDM button as usual).

TM SUS

- I wanted to use ASS. => The OL damping and ASC inputs are enabled for ETMX. The filter bank output of the ASS servos are all turned on.
- The arms were locked and aligned with ASS. The ASS servo offsets were offloaded to the ASC offset sliders (as usual).

- I found the X arm spot positionis moving slowly in pitch. I wanted to know what is causing this.

- Turned off all the OPLEV dampings for the four test masses.
- Took the power spectra of the OSEM output (e.g. C1:SUS-ITMX_LLSEN_OUT)

See attachment 1 (The DTT XML file can be found as /users/koji/151111/TM_SUS.xml )

- It seems that something is wrong with ITMX UL OSEM
  The signal level seems to be identical with the others. However, the noise level is huge. We need to check if the cable connection is OK.

- ITMY LL shows remarkably higher bounce mode although I can't tell if this is normal or not.

- The OPLEV dampings have been restored.

The ITMX OSEMS are oscillating.

By looking at the oplev and Vmons before and after a step of 90 counts in SUS-ETMX_PIT_OFFSET, I observe:

• UL: +1.53mV / urad
• UR: -1.94mV / urad
• LR: -1.54mV / urad
• LL: +1.92mv / urad

The random error associated with these measurements is ~0.02mV

So, the ~7urad urad shift seen in my earlier post would mean a change of around 10mV in the Vmon signals, which isn't evident in the traces. So, this is possibly a piece of evidence in favor of a real mechanical shift, rather than an electronic glitch. 

Are they oscillating or not ?

ETMX, ETMY and MC2 POS _biases are off. Why ?

The Epics MEDMs screens are going blank in  ~3-5 minutes reluctantly.

After running dither alignment for all mirrors, all oplevs were recentered. (Except ETMY, since we did that earlier today.)

Looking at Koji's template for OSEM signals, the ITMX UL sensor noise floor seems more in line with the LL sensor, though there continues to be more noise than in other mirrors. 

Trending the sensor signals over the past 7 days, Koji's measurement looks to have been taken during a time when the UL sensor voltage had jumped down. Did someone squish the satellite box cable? I have not done so. 

I think that the step right at the end is due to a new POS offset of -2k counts, which I think Koji put into place earlier today. 

According to the wiki, the Vmax/2 values and the current values are: 

Although this noise is bad, we have always had these kind of humps around the bounce mode. Our interpretation in the past was that this was due to poor alignment of the OSEM in the frame, leading to a large vertical to horizontal coupling. Once you implement the BLRMS for the SUS channels, we'll be able to trend the noise over long periods of time.

Two companies are willing to  make the ruby grooves and the third one is still working on their quote.

The price is ~$100 each. The cost goes down 10% if we  order 50 instead of 30 pieces.

How many should we get ?