40m QIL Cryo_Lab CTN SUS_Lab CAML OMC_Lab CRIME_Lab FEA ENG_Labs OptContFac Mariner WBEEShop
  40m Log, Page 205 of 354  Not logged in ELOG logo
ID Date Authorup Type Category Subject
  10797   Mon Dec 15 12:53:13 2014 ericqUpdateComputer Scripts / ProgramsStatus of the new nodus

 Nodus (solaris) is dead, long live Nodus (ubuntu).

Diego and I are smoothing out the Kinks as they appear, but the ELOG is running smoothly on our new machine. 

SVN is working, but your checkouts may complain because they expect https, and we haven't turned SSL on yet...

  10809   Wed Dec 17 13:14:38 2014 ericqUpdateLSCPRMI loops need help

Quote:

However, the PRMI would not acquire lock with the arms held off resonance. 

This is entirely my fault. 

Last week, while doing some stuff with PRY, I put this filter in SUS_PRM_LSC, to stop some saturations from high frequency sensing noise

oops.pdf

After the discussion at today's meeting, it struck me that I might have left it on. Turns out I did. 

20 degree phase lag at 200Hz can explain the instability, some non-flat shape at few hundreds of Hz explains the non 1/f shape.

Sorry about all that...

  10811   Wed Dec 17 18:14:36 2014 ericqUpdateASCTransmon QPD -> ASC servos ready for comissioning

 I have completed all of the model modifications and medm screen updates to allow for feedback from the transmon QPD pitch and yaw signals to the ITMs. Now, we can design and test actual loops...

newASCscreen.png

The signals come from c1sc[x/y] to c1rfm via RFM, and then go to c1ass via dolphin. 


Out of curiosity about the RFM+dolphin delay, I took a TF of an excitation at the end SUS model (C1:SUS-ETM[X/Y]_QPD_[PIT/YAW]_EXC) to the input FM in the ASC model (C1:ASC-ETM[X/Y]_QPD_[PIT/YAW]_IN1). All four signals exhibit the same delay of 122usec. I saved the dtt file in Templates/ASC/transmonQPDdelay.xml

This is less than a degree under 20Hz, so we don't have to worry about it. 

  10814   Thu Dec 18 02:23:34 2014 ericqUpdateLSCLocking update

 [ericq, Diego]

Some locking efforts tonight; many locklosses due to PRC angular motion. Furthest progress was arm powers of 15, and I've stared at the corresponding lockloss plot, with little insight into what went wrong. (BTW, lastlock.sh seems to catch the lock loss reliably in the window)

15lockloss.png

CARM and DARM loops were measured not long before this lock loss, and had nominal UGFs (~120Hz, ~20deg PM). However, there was a reasonably clear 01 mode shape at the AS camera, which I did nothing to correct. Here's a spectrum from *just* before the lockloss, recovered via nds. Nothing stands out to me, other than a possible loss of DARM optical gain. (I believe the references are the error signal spectra taken in ALS arms held away + PRMI on 3F configuration)

15spec.pdf

The shape in the DARM OLTF that we had previously observed and hypothesized as possible DARM optical spring was not ever observed tonight. I didn't induce a DARM offset to try and look for it either, though.

Looking into some of the times when I was measuring OLTFs, the AS55 signals do show coherence with the live DARM error signal at the excitation frequencies, but little to no coherence under 30Hz, which probably means we weren't close enough to swap DARM error signals yet. This arm power regime is where the AS55 sign flip has been modeled to be...


A fair amount of time was spent in pre-locking prep, including:

  • The usual X green beat alignment tweak, to make the X ALS control not terrible
  • Both ITM oplevs were centered
  • For some reason, I had to flip the signs of the REFL165 matrix elements for the PRMI...
  • "Restore PRMI SB" has ASC automatically enabled, which results in unexpected kicks even with LSC off, which caused a few minutes head-scratching
  10815   Thu Dec 18 15:41:30 2014 ericqUpdateComputer Scripts / ProgramsOffsite backups of /cvs/cds going again

Since the Nodus switch, the offsite backup scripts (scripts/backup/rsync.backup) had not been running successfully. I tracked it down to the weird NFS file ownership issues we've been seeing since making Chiara the fileserver. Since the backup script uses rsync's "archive" mode, which preserves ownership, permissions, modification dates, etc, not seeing the proper ownership made everything wacky. 

Despite 99% of the searches you do about this problem saying you just need to match your user's uid and gid on the NFS client and server, it turns out NFSv4 doesn't use this mechanism at all, opting instead for some ID mapping service (idmapd), which I have no inclination of figuring out at this time. 

Thus, I've configured /etc/fstab on Nodus (and the control room machines) to use NFSv3 when mounting /cvs/cds. Now, all the file ownerships show up correctly, and the offsite backup of /cvs/cds is churning along happily. 

  10816   Thu Dec 18 16:21:08 2014 ericqUpdateComputer Scripts / Programsscripts not being backed up!

 I just stumbled upon this while poking around:

Since the great crash of June 2014, the scripts backup script has not been workingon op340m. For some reason, it's only grabbing the PRFPMI folder, and nothing else. 

Megatron seems to be able to run it. I've moved the job to megatron's crontab for now.

  10819   Fri Dec 19 16:39:25 2014 ericqUpdateComputer Scripts / Programselog autostart

I've set up nodus to start the ELOG on boot, through /etc/init/elog.conf. Also, thanks to this, we don't need to use the start-elog.csh script any more. We can now just do:

controls@nodus:~  $ sudo initctl restart elog

I also tweaked some of the ELOG settings, so that image thumbnails are produced at higher resolution and quality. 

  10820   Fri Dec 19 16:59:32 2014 ericqUpdateComputer Scripts / ProgramsFSS Slow servo moved to megatron

Given that op340m showed some undesired behavior, and that the FSS slow seems prone to railing lately, I've moved the FSS slow servo job over to megatron in the same way I did for the MC autolocker. 

Namely, there is an upstart configuration (megatron:/etc/init/FSSslow.conf), that invokes the slow servo. Log file is in the same old place (/cvs/cds/caltech/logs/scripts), and the servo can be (re)started by running: 

controls@megatron|~ > sudo initctl start FSSslow

Maybe this won't really change the behavior. We'll see

  10825   Sat Dec 20 00:00:03 2014 ericqUpdateComputer Scripts / ProgramsFSS Slow servo moved to megatron

I ssh'd in, and was able to run each script manually successfully. I ran the initctl commands, and they started up fine too. 

We've seen this kind of behavior before, generally after reboots; see ELOGS 10247 and 10572

  10830   Mon Dec 22 16:21:15 2014 ericqUpdateelogStrange ELOG search

In order to fix ELOG search, I have started running ELOG v2.9.2 on Nodus.

Sadly, due to changes in the software, we can no longer use one global write password. Instead, we must now operate with registered users.

Based on recent elog users, I'll be creating user accounts with the following names, using the same old ELOG write password. (These will be valid across all logbooks)

  • ericq
  • rana
  • koji
  • diego
  • jenne
  • manasa
  • Steve
  • Kate
  • Zach
  • Evan
  • Aidan
  • Chris
  • Dmass
  • nicolas
  • Gabriele
  • xiaoyue

All of these users will be "Admins" as well, meaning they can add new users and change settings, using the "Config" link.

Let me know if I neglected to add someone, and sorry for the inconvenience.

RXA: What Eric means to say, is that "upgrading" from Solaris to Linux broke the search and made us get a new elog software that;s worse than what we had.

  10835   Tue Dec 23 14:27:16 2014 ericqUpdateCDSChiara moved to UPS

 Steve and I switched chiara over to the UPS we bought for it, after ensuring the vacuum system was in a safe state. Everything went without a hitch. 

Also, Diego and I have been working on getting some of the new computers up and running. Zita (the striptool projecting machine) has been replaced. One think pad laptop is missing an HD and battery, but the other one is fine. Diego has been working on a dell laptop, too. I was having problems editing the MAC address rules on the martian wifi router, but the working thinkpad's MAC was already listed. 

  10836   Tue Dec 23 14:30:11 2014 ericqUpdateCDSChiara moved to UPS

Quote:

 Steve and I switched chiara over to the UPS we bought for it, after ensuring the vacuum system was in a safe state. Everything went without a hitch. 

Also, Diego and I have been working on getting some of the new computers up and running. Zita (the striptool projecting machine) has been replaced. One think pad laptop is missing an HD and battery, but the other one is fine. Diego has been working on a dell laptop, too. I was having problems editing the MAC address rules on the martian wifi router, but the working thinkpad's MAC was already listed. 

 Turns out that, as the martian wifi router is quite old, it doesn't like Chrome; using Firefox worked like a charm and now also giada (the Dell laptop) is on 40MARS.

  10839   Tue Dec 23 16:49:32 2014 ericqUpdateelogStrange ELOG search

 So, despite having registered users, it turns out that the "Author" field is still open for editing when making posts. I.e. we don't really need to make new accounts for everyone. 

Thus, I've made a user named "elog" with the old write password that can write to all ELOGs. 

(Also, I've added a user called "jamie")

  10852   Mon Jan 5 12:42:09 2015 ericqUpdateSUSBS misbehaving

The BS was showing some excess motion. I think I've fixed it. Order of operations:

  • The DC PIT bias from previous ASS runs was at ~500, I zeroed this and aligned the BS to be centered on its oplev QPD with DC alignment sliders
  • I squished the gold box cables. This changed the alignment slightly, and brought the UR voltage back to a normal value. Excess motion still existed
  • I found that the the C1:SUS-BS_LRSEN filter had HOLD OUTPUT enabled. I turned it off. All seems well. 

I'm not sure how this might have gotten switched on...

  10855   Mon Jan 5 23:36:47 2015 ericqUpdateIOOAO cable reconnected

Quote:

 I lost the connecting cable from the CM to the AO input (unlabeled). 

 This afternoon, I labelled both ends of this cable, and reconnected it to the MC servo board. 

  10857   Tue Jan 6 03:13:09 2015 ericqUpdateLSCPRFPMI status & IFO status

Two plots from tonight:

Lock loss. Based on the fact that it looked like the DARM servo was running away, Rana posited an effective sign flip in the DARM loop, perhaps due to a parasitic angular feedback mechanism.

Jan6lockloss1_zoom.png

 


While Jenne was probing the IFO at lower powers, we noticed a sudden jump in ASDC. Found the GPS time and fed it to the lockloss plotter. Seems fairly evident that some sudden ETMX motion was to blame. (~2urad kick in yaw)

Jan6_asdcjump.png

  10864   Wed Jan 7 09:44:33 2015 ericqUpdateLSCDARM phase budget

As Jenne mentioned, I created a model of the DARM OLG to see why we have so little phase margin. However, it turns out I can explain the phase after all.

Chris sent me his work for the aLIGO DARM phase budget, which I adapted for our situation. Here's a stacked-area plot that shows the contributions of various filters and delays on our phase margin, and a real measurement from a few days ago . 

DarmPhaseBudget.pdf

This isn't so great! Informed by Chris's model, the digital delays look like: (Here I'm only listing pure delays, not phase lags from filters)

  • 64k cycle (End IOP)
  • 16k cycle (End isce[x/y])
  • 16k cycle x 2 (end to LSC through RFM) [See ELOG 10811]
  • 16k cycle (LSC)
  • 16k cycle (LSC to SUS through dolphin) [See ELOG 9881]
  • 16k cycle (SUS)
  • 16k cycle x2 (SUS to end through RFM)
  • 16k cycle (End isce[x/y])
  • 64k cycle (SUS IOP)
  • DAC zero order hold

This adds up to about 570usec, 20.5 degrees at 100Hz, largely due to the sheer number of computer hops the transmission loops involve. 

As a check, I divided the measured OLG by my model OLG, to see if there is any shape to the residual, that my model doesn't explain. It looks like it fits pretty well. Plot:

 DarmOLGresidual.pdf

So, unless we undertake a bunch of computer work, we can only improve our transmission loops through our control filter design. 

Everything I used to generate these plots is attached. 

Attachment 3: 2015-01-DARMphase.zip
  10875   Thu Jan 8 02:52:09 2015 ericqUpdateLSCUGF Servo for LSC

I added UGF servos for the DRMI DoFs, after creating a library block for the servos. I also deleted the FMs before the phase rotation, since we can just do it afterwards in other existing FMs. I've only added the MICH and PRCL buttons to the LSC screen because in the end, I feel like a dropdown is better, but I just wanted to get it running quickly tonight. The LSC model and the UGF block have been committed to the svn. 

We were able to use the PRCL UGF servo successfully, as Jenne was exploring MICH offset space. 

  10877   Thu Jan 8 03:40:50 2015 ericqUpdateComputer Scripts / ProgramsELOG 3.0

I've installed the very fresh ELOG 3.0, for nothing else than the new built in text editor which has a LATEX capable equation editor built right in. 

Check out this sweet limerick: 

\int_{1}^{\sqrt[3]{3}}t^2 dt\, \textbf{cos}(\frac{3\pi}{9}) = \textbf{ln}(\sqrt[3]{e})

  10880   Thu Jan 8 19:41:50 2015 ericqUpdateGeneralinane LSCoffsets script removed

The restored offset script used old tdsavg calls that our workstations can't do, and didn't include things like the transmon QPDs. I've written yet another offset script that uses cdsutils averaging to do the thing, and committed to the svn. 

  10886   Mon Jan 12 18:11:25 2015 ericqUpdateASCTest Mass -> Transmon QPD TFs measured

We want to have some angular control of the arms during lock acquistion. 

In single arm lock, Diego and I shook the TMs and measured how the QPDs responded. (I would've liked to do a swept sine in DTT, but the user envelope function still isnt' working!)

For now, we can close simple loops with QPD sensor and ITM actuator, but, as Rana pointed out to Diego and me today, this will drive some amount of the angular cavity degree of freedom that the QPD doesn't sense. So, ideally, we want to come up with the right combination of ITM and ETM motion that lies entirely within the DoF that the QPD senses.

I created a rudimentary loop for Yarm yaw, was able to get ~20Hz for the upper UGF, a few mHz for the lower, but it was starting to leak into the length error signal. Further tweaking will be neccesary...

Attachment 1: Jan12_singleArmSensing.pdf
Jan12_singleArmSensing.pdf
Attachment 2: Jan12_singleArmSensing.xml.zip
  10889   Tue Jan 13 01:58:16 2015 ericqUpdateComputer Scripts / ProgramsCdsutils upgraded to 382

I've upgraded our cdsutils installation to v382; there have been some changes to pydv which will allow me to implement the auto y-scaling on our lockloss plots. 

After some brief testing, things seem to still work...

  10890   Tue Jan 13 03:47:46 2015 ericqUpdateCDSInstalled new ethernet driver on Chiara

Chiara threw another network hissy fit. Dmesg was spammed with a bunch of messages like eth0: link up appearing rapidly. 

Some googling indicated that this error message in conjuction with the very ethernet board and driver that Chiara had in use could be solved by updating with an appropriate driver from the manufacturer.

In essence, I followed steps 1-7 from here: http://ubuntuforums.org/showthread.php?t=1661489

So far, so good. We'll keep an eye out to see how it works...

  10892   Tue Jan 13 04:57:26 2015 ericqUpdateCDSAll FE diagnostics back to green

I was looking into the status of IPC communications in our realtime network, as Chris suggested that there may be more phase missing that I thought. However, the recent continual red indicators on a few of the models made it hard to tell if the problems were real or not. Thus, I set out to fix what I could, and have achieved full green lights in the CDS screen. 

This required:

  • Fixing the BLRMS block, as was found to be a problem in ELOG 9911 (There were just some hanging lines not doing anything)
  • Cleaning up one-sided RFM and SHMEM communications in C1SCY, C1TST, C1RFM and C1OAF

The frontend models have been svn'd. The BLRMs block has not, since its in a common cds space, and am not sure what the status of its use at the sites is...

  10901   Wed Jan 14 19:27:09 2015 ericqUpdateLSCUGF servo now linear again

The UGF servos have been moved to the control point, are are once again totally linear!

  10908   Thu Jan 15 18:57:41 2015 ericqUpdateASCTransmon QPD loops live

I've measured the sensing for each of the arms, by using our calibrated oplevs, in terms of QPD counts per micron. It is:

YARM
ETMY: QPD PIT / OPLEV PIT =   22.0 count/urad
      QPD YAW / OPLEV YAW =   17.1 count/urad
ITMY: QPD PIT / OPLEV PIT =   -6.0 count/urad
      QPD YAW / OPLEV YAW =    5.9 count/urad
XARM
ETMX: QPD PIT / OPLEV PIT =   16.6 count/urad
      QPD YAW / OPLEV YAW =   -9.3 count/urad
ITMX: QPD PIT / OPLEV PIT =    4.0 count/urad
      QPD YAW / OPLEV YAW =   -6.0 count/urad

In the absence of a lens, the QPD would be significantly more sensitive to cavity axis translation than tilt, and thus about equally sensitive to ITM and ETM angle. However, there are lenses on the end tables. I didn't go out and look at them, but found some elogs from Annalisa that mentioned 1m focal length lenses. Back-of-the-envelope calculations convince me that this can plausibly lead to the above sensitivity ratios.

I used these quantities to come up with an actuation matrix for the ASC loops, and measured the effective plant seen by the FM, fitted it to some poles( looks like zpk([],-2*pi*[1.47+3.67i,1.47-3.67i],160); ), and designed a control servo. Here is the designed loop:

The servo works on both arms, both DoFs. A DTT measurement agrees with the designed loop shape, up to a few degrees, which are probably due to the CDS delay. The RMS of the QPD error signals goes down by about 20dB, and are currently dominated by the bounce mode, so maybe we can try to sneak in some resonant gain...?

Once we confirm that they work when locking, we can write up and down lines into the locking scripts...

Attachment 1: loopDesign.pdf
loopDesign.pdf
  10909   Thu Jan 15 19:01:30 2015 ericqUpdatePSLPMC realigned

PMC realigned again... The transmission was down to 0.70, and the MC was having a hard time trying to autolock.

  10912   Fri Jan 16 04:14:38 2015 ericqUpdateCDSUnexpected CDS behavior

EDIT: Sleepy Eric doesn't understand loops. The conditions for this observation included active oplev loops. Thus, obviously, looking at the in-loop signal after the ASC signl joins the oplev signal will produce this kind of behavior. 


After some talking with Rana, I set out on making an even better-er QPD loop. I made some progress on this, but a new mystery halted my progress. 

I sought to have a more physical undertanding of the plant TF I had measured. Earlier, I had assumed that the 4Hz plant features I had measured for the QPD loops were coming from the oplev-modified pendulum response, but this isn't actually consistent with the loop algebra of the oplev servos. I had seen this feature in both the oplev and qpd error signals when pushing an excitation from the ASC-XARM_PIT (and so forth) FMs. 

However, when exciting via the SUS-ETMX-OLPIT FMs (and so forth), this feature would not appear in either the QPD or oplev error signals. That's weird. The outputs of these two FMs should just be summed, right before the coil matrix. 

I started looking at the TF from ASC-YARM_PIT_OUT to SUS-ETMY_TO_COIL_1_2, which should be a purely digital signal routing of unity, and saw it exhibit the phase shape at 4Hz that I had seen in earlier measurements. Here it is:

I am very puzzled by all of this. indecision Needs more investigation.

Attachment 1: digitalProblem.pdf
digitalProblem.pdf
  10920   Mon Jan 19 18:27:16 2015 ericqUpdateASCQPD ASC saga continues.

Herein, I will try to paint a more thorough picture of this whole QPD ASC mess. 


Motivation for QPD ASC loops:

  • We would like to use the QPDs as a DC arm pointing reference during locking attempts, or over multiple days, if possible. 
  • It would be nice if the QPDs could complement the oplevs to reduce angular motion of the cavity. 
  • We must not make the single arm longnitudinal noise or RIN worse, because anything observable in the single arm case will be catastrophic at full sensitivity

Actuation design:

As mentioned in a previous ELOG, in single arm lock, I measured the QPD response with respect to the calibrated oplev signals. They were:

YARM
ETMY: QPD PIT / OPLEV PIT =   22.0 count/urad
      QPD YAW / OPLEV YAW =   17.1 count/urad
ITMY: QPD PIT / OPLEV PIT =   -6.0 count/urad
      QPD YAW / OPLEV YAW =    5.9 count/urad
XARM
ETMX: QPD PIT / OPLEV PIT = 16.6 count/urad
      QPD YAW / OPLEV YAW = -9.3 count/urad
ITMX: QPD PIT / OPLEV PIT =  4.0 count/urad
      QPD YAW / OPLEV YAW = -6.0 count/urad

For reference, one microradian of either ITM or ETM motion produces about 60um of ETM beam spot displacement, compared to the spot size of ~5mm. 

However, given the lenses on the end tables that are used for green mode matching, that the IR transmitted beam also passes through, the QPDs are not directly imaging the ETM spot position; if they were, they would have equal sensitivity to ITM and ETM motion due to our flat/curved arm geometry. 

From this data, I calculated the actuation coefficients for each DoF as A_{ETM} = \frac{d_{ETM}}{\sqrt {d_{ETM}^2 + d_{ITM}^2}}, and similarly for the ITMs, where the d's come from the table above. However, it occurs to me that maybe this isn't the way to go... I'll mention this later. 


Loop design:

Up until now, at every turn, I had not properly been thinking about how the oplev loop plays into all of this. I went to the foton filters, and grabbed the loop and plant models for the ETMY oplev, and constructed the closed loop gain, 1/1+G, and the modified plant, P/1+G, which is what the ASC loop sees as its plant. 

Here, the purple trace explains all of the features I was confused about earlier. 

With this in hand, I set up to design a loop to satisfy our motivations. 

  • Bounce/roll mode notches to avoid exciting them
  • 1/f UGF crossing at a few Hz, limited by the gain margin at ~10Hz, which is where the phase will hit 180, due to the notches
  • 4th order Elliptic lowpass at 100, to avoid contaminating the longnitudinal signals
  • 1/f^2 at low frequencies for DC gain

To do this, I inverted the oplev closed loop plant pole around 4Hz to smooth the whole thing out. Here's a comparison of the measured OLG with what I modelled. 

There's a little bit of phase discrepency around 10Hz, but I think it looks about right overall. 


Evaluation:

So, here's the part that counts: How does this actually perform? I took spectra of the QPD error signals, the relevant OpLev signals as out of loop sensors, the PDH error signal and transmitted RIN while single arm locked, with this loop off, and on for all 4 DoFs simultaneously. 

Verdict:

  • In-loop signals get small, unsurprisingly.
  • Cavity signals unchanged. yes
  • ITM oplev signals are unchanged (and not plotted, to not clutter the plots (This isn't surprising since the loops mostly actuate on the ETMs).
  • ETM oplev signals get smaller around the 3Hz peak, but are louder in other bands.no 

This is what makes me think I may need to revisit the actuation matrix. If I did it wrong, I am driving the "invisible" quadrant of the cavity angular DoFs, and this could be what is injecting noise into the oplevs. 


Conclusion:

In the end, I have a better understanding of what is going on, and I don't think we're quite there yet.  

Attachment 1: oplevPlant.pdf
oplevPlant.pdf
Attachment 2: loopDesignComparison.pdf
loopDesignComparison.pdf
  10921   Tue Jan 20 02:39:49 2015 ericqUpdateASCQPD ASC saga continues.

Although the QPD loops are less than ideal right now, I made changes to the ASC model to trigger the QPD loops on and off politely, depending on TRX and TRY. The settings are exposed on the ASC screen. However, I have not yet exposed the FM triggering that I also set up to make sure the integrator doesn't misbehave if the arm loses lock. We probably don't want to trigger them on at anything lower than arm powers of about 1.0. 

I've tested the triggering by randomly turning LSC mode on and off, and making sure that the optics don't recieve much of a kick as the QPD loops engage a few seconds after the LSC boosts do, or when lock is lost. This works as long as there isn't much of a DC offset befire the loops are engaged. (Under 20 counts or so is fine)

As a side note, I was going to use the TRIG_SIG signals sent via the LSC model via SHMEM blocks for the ASC triggering, but oddly, the data streams that made it over were actually the MICH and SRCL TRIG_SIGs, instead of XARM and YARM as labelled. I double checked the simulink diagrams; everything seemed fine to me. In any case, ASS was already recieving TRX and TRY directly via RFM, so I just piped those over to the ASC block. This way is probably better anyways, because it directly references the arm powers, instead of the less obvious LSC triggering matrix. 

  10928   Thu Jan 22 02:56:07 2015 ericqUpdateIOOFSS input offset adjusted

Since Rana's overhaul of the IMC, the FSS input offset had been sitting at zero. 

Over the last day or so, I had noticed the MC refl wall striptool trace looking noisier, and earlier this evening, we were suffering from a fair amount of downtime due to IMC unlocks, and failure to autolock for times on the order of ten minutes. 

While we had used ezcaservo for this in the past, I set the FSS offset manually tonight. Namely, I popped open a dataviewer trace of MC_F, and scanned the FSS offset to make MC_F go to zero. It required a good amount of offset, 4.66 V according to the FSS screen. I did this while the FSS slow servo was on, which held the FSS Fast output at zero. 

That was four hours ago; MC_F is still centered on zero, and we have not had a single IMC unlock since then. The MC refl trace is thinner too, though this may be from nighttime seismic. 

  10936   Fri Jan 23 15:46:21 2015 ericqUpdateCDSNew netgpib scripts for SR785
Quote:

Does netgpibdata/TFSR785 work at the 40m currently? 

It does appear to work here. However, I've since supplanted TFSR785 and SPSR785 with SRmeasure, which has some simpler command line options for directly downloading a manually configured measurement. I've also set up a git repository for the gpib scripts I've done at https://github.com/e-q/netgpibdata, which could be easier than grabbing the whole 40m directory. 

  10940   Mon Jan 26 17:43:52 2015 ericqConfigurationIOOMC Autolocker update

The MC autolocker hasn't been so snappy recently, and has been especially fussy today. Previously, the mcup script was triggered immediately once the transmission was above a certain threshold. However, this could waste time if it was just an errant flash. Hence, I've added a 0.5 second delay and a second threshold check before mcup is triggered. 

After breaking the lock 5ish times, it does seem to come back quicker.

  10953   Thu Jan 29 04:27:35 2015 ericqUpdateLSCCARM on REFL11

[ericq, Diego]

Tonight, we transitioned CARM from ALS directly to REFL11 I at 25% Mich Offset. yes

We attempted the transition twice, the first time worked, but we lost lock ~5 seconds after full transition due to a sudden ~400Hz ringup (see attached lockloss plot). The second barfed halfway, I think because I forgot to remove the CARM B offset from the first time frown

The key to getting to zero CARM offset with CARM and DARM on ALS is ekeing out every bit of PRMI phase margin that you can. Neither MICH nor PRCL had their RG filters on and I tweaked the MICH LP to attenuate less and give back more phase (the HF still isn't the dominant RMS source.) PRCL had ~60 degrees phase margin at 100Hz UGF, MICH had ~50 deg at 47Hz UGF. The error signals were comparitively very noisy, but we only cared that they held on. Also important was approaching zero slooooooooowly, and having the CARM and DARM UGF servo excitations off, because they made everything go nuts. Diego stewarded the MICH and PRCL excitation amplitudes admirably. 

Oddly, and worringly, the arm powers at zero CARM offset were only around 10-12. Our previous estimations already include the high Xarm loss, so I'm not sure what's going on with this. Maybe we need to measure our recycling gain?

I hooked up the SR785 by the LSC rack to the CM board after the first success. For the second trial, I also took TFs with respect to CM slow, but they looked nowhere near as clean as the normal REFL11 I channel; I didn't really check all the connections. I will be revisiting the whole AO situation soon. 

In any case, I think we're getting close...

Attachment 1: Jan29_REFL11_lockloss.png
Jan29_REFL11_lockloss.png
  10969   Tue Feb 3 16:36:33 2015 ericqUpdateLSCCM servo & AO path status

I have removed REFLDC and the SR560 offsetter from the CM board IN2. Now, analog AS55 I lives there, for our single arm testing. (Analog I has more of the single arm Y PDH signal in it). REFL11 has been reconnected to IN1. 


With ITMX super misaligned, Diego and I locked the Y-arm with the AO path on AS55, ultimately at 4kHz bandwidth, but with plenty of gain margin. We didn't allocate the gains too intelligently, and had the CM board input gain slider maxed out, but plenty of headroom in the digital and AO sliders, making it inconvenient to up the UGF even more, to engage the super boosts. However, since this is just a test case to make sure we still can AO lock, I'm not too worried about this. 

Since LSC FMs and such had changed around, old recipies didn't neccesarily work 1:1. Diego is writing a script for the current recipe, and will post an elog with the steps. 

Gains and signs are able to be tracked by loop TFs, the real sticking point is a stable crossover. We used the 1.6k:80 hardware filter in the CM board to give the AO Path a 1/f shape in the crossover region, and undid it digitally in the CM_SLOW input FM. However, we do use a 300:80 in the MC2 sus FM to make the digital loop like 1/f^2 around the crossover, once a little bit of AO has come in to pull up the digital loop's phase. We used the CARM filter bank to do this, so I think we should be able to use a similar technique to do it in the PRFPMI case, as long as the coupled cavity pole is around ~100Hz. 

Attached are a few OLTFs from the progression.

Attachment 1: yarmAO.pdf
yarmAO.pdf
  10972   Wed Feb 4 14:30:05 2015 ericqUpdateLSCASDC Whitening Gain

At the lunch meeting, we were thinking about the exess high frequency content of the MICH control signal, which leads to railing against the FM output limiter and lock loss. I propsed that POPDC sensor/ADC noise was the culprit. 

In short, I was wrong. Just now, I locked the PRMI with a MICH offset as we normally do, and then froze the POPDC output. The MICH spectrum did not change in any noticible way. 

However, increasing the analog ASDC whitening gain showed a direct improvement of the error signal noise floor, and thus a reduction in the control signal spectrum. I.e. we have been suffereing from ASDC ADC noise.

We had previously set the ASDC whitening gain so that half fringe of the PRMI would be well within the ADC range, but since we're actually only ever going to 25%, I feel fine upping this gain. Interestingly, when increasing the whitening gain by 12dB,  the control signal spectrum has fallen by more like 20 dB in the 400Hz-1kHz region, which is great. 

The only potential hurdle I can think of is that when we start reducing the MICH offset at zero CARM offset, we may approach ADC saturation in ASDC before we can hand off to RF signals, in which case we would have to dynamically lower the whitening gain, which introduces offsets, which could get hairy. But, since MICH railing has been directly seen to lead to lock-loss, I'd rather solve that problem first. 

  10984   Fri Feb 6 15:29:29 2015 ericqUpdateCDSNetwork Shenanigans

Just now we had another EPICS freeze. The network was still up; i.e. I could ssh to chiara and fb, who both seemed to be working fine.

I could ping c1lsc successfully, but ssh just hung. fb's dmesg had some daqd segfault messages, so I telnet'ed to daqd and shut it down. Soon after, EPICS came back, but this is not neccesarily because of the daqd restart...

  10994   Tue Feb 10 03:09:02 2015 ericqUpdateLSCSome locking thoughts on PRMI

Unfortunately, we only had one good CARM offset reduction to powers of about 25, but then my QPD loop blew it. We spent the vast majority of the night dealing with headaches and annoyances. 

Things that were a pain:

  • If TRX is showing large excursions after finding resonance, there is no hope. These translate into large impulses while reducing the CARM offset, which the PRMI has no chance of handling. The first time aligning the green beat did not help this. For some reason, the second time did, though the beatnote amplitude wasn't increased noticibly. 
    • NOTICE: We should re-align the X green beatnote every night, after a solid ASS run, before any serious locking work. 
    • Afterwards, phase tracker UGFs (which depend on beatnote amplitude, and thereby frequency) should be frequently checked. 
  • We suffered some amount from ETMX wandering. Not only for realigning between lock attempts, but on one occasion, with CARM held off, GTRX wandered to half its nominal value, leading to a huge effective DARM offset, which made it impossible to lock MICH with any reasonble power in the arms. Other times, simply turning off POX/POY locking, after setting up the beatnotes, was enough to significantly change the alignment. 
  • IMC was mildly tempermental, at its worst refusing to lock for ~20min. One suspicion I have is that when the PMC PZT is nearing its rail, things go bad. The PZT voltage was above 200 when this was happening, after relocking the PMC to ~150, it seems ok. I thing I've also had this problem at PZT voltages of ~50. Something to look out for. 

Other stuff:

  • We are excited for the prospect of the FOL system, as chasing the FSS temperature around is no fun. 
  • UGF servo triggering greatly helps the PRMI reacquire if it briefly flashes out, since the multipliers don't run away. This exacerbated the ALS excursion problem. 
  • Using POPDC whitening made it very tough to hold the PRMI. Maybe because we didn't reset the dark offset...?
  10997   Tue Feb 10 18:37:17 2015 ericqUpdateASCQPD ASC ready to go

I've remeasured the QPD ASC sensing coefficients, and figured out what I did weird with the actuation coefficients. I've rearranged the controller filters to be able to turn on the boost in a triggered way, and written Up/Down scripts that I've tested numerous times, and Jenne has used as well; they are exposed on the ASC screen. 

All four loops (2 arms * pit,yaw), have their gains set for 8Hz UGF, and have 60 degrees of phase margin. The loop shape is the same as the previous ELOG post. Here is the current on/off performance. The PDH signals (not shown, but in attached xml) show no extra noise, and the low frequency RIN goes down a bit, whic is good. The oplevs error signals are a bit noisier, but I suppose that's unavoidable. The Y-arm performs a bit better than the X-arm. 

The up/down scripts don't touch the filters' trigger settings at all, just handles switching the input and output and clearing history. FM1 contains the boost which is intended to have a longer trigger delay than the filters themselves.

Attachment 1: Feb10_loops_offOn.png
Feb10_loops_offOn.png
Attachment 2: Feb10_newLoops_offOn.xml.zip
  11003   Wed Feb 11 17:31:11 2015 ericqUpdateLSCRFPD spectra

For future reference, I've taken spectra of our various RFPDs while the PRMI was sideband locked on REFL33, using a 20dB RF coupler at the RF input of the demodulator boards. The 20dB coupling loss has been added back in on the plots. Data files are attached in a zip.

Exceptions: 

  • The REFL165 trace was taken at the input of the amplifier that immediately preceeds the demod board. 
  • The 'POPBB' trace was taken with the coupler at the input of the bias tee, that leads to an amplifier, then splitter, then the 110 and 22 demod boards. 

I also completely removed the cabling for REFLDC -> CM board, since it doesn't look like we plan on using it anytime in the immediate future. 

Attachment 1: REFL.png
REFL.png
Attachment 2: AS.png
AS.png
Attachment 3: POP.png
POP.png
Attachment 4: 2015-02-PDSpectra.zip
  11004   Wed Feb 11 18:07:42 2015 ericqUpdateLSCRFPD spectra

After some discussion with Koji, I've asked Steve to order some SBP-30+ bandpass filters as a quick and cheap way to help out REFL33. (Also some SBP-60+ for 55MHz, since we only have 1*fmod and 2*fmod bandpasses here in the lab). 

  11010   Thu Feb 12 03:43:54 2015 ericqUpdateLSC3F PRMI at zero ALS CARM

I have been able to recover the ability to sit at zero CARM offset while the PRMI is locked on RELF33 and CARM/DARM are on ALS, effectively indefinitely. However, I feel like the transmon QPDs are not behaving ideally, because the reported arm powers freqently go negative as the interferometer is "buzzing" through resonance, so I'm not sure how useful they'll be as normalizing signals for REFL11. I tried tweaking the DARM offset to help the buildup, since ALS is only roughly centered on zero for both CARM and DARM, but didn't have much luck.

Example:

Turning off the whitening on the QPD segments seems to make everything saturate, so some thinking with daytime brain is in order.


How I got there:

It turns out triggering is more important than the phase margin story I had been telling myself. Also, I lost a lot of time to needing demod angle change in REFL33. Maybe I somehow caused this when I was all up on the LSC rack today?

We have previously put TRX and TRY triggering elements into the PRCL and MICH rows, to guard against temporary POP22 dips, because if arm powers are greater than 1, power recylcing is happening, so we should keep the loops engaged. However, since TRX and TRY are going negative when we buzz back and forth through the resonsnace, the trigger row sums to a negative value, and the PRMI loops give up. 

Instead, we can used the fortuitously unwhitened POPDC, which can serve the same function, and does not have the tendancy to go negative. Once I enabled this, I was able to just sit there as the IFO angrily buzzed at me. 

Here are my PRMI settings

REFL33 - Rotation 140.2 Degrees, -89.794 measured diff

PRCL = 1 x REFL33 I; G = -0.03; Acquire FMs 4,5; Trigger FMs 2, 9; Limit: 15k ; Acutate 1 x PRM

MICH = 1 x REFL33 Q, G= 3.0, Acquire FMs 4,5,8; Trigger FM 2, 3; Limit: 30k; Actuate -0.2625 x PRM + 0.5 x BS

Triggers = 1 x POP22 I + 0.1 * POPDC, 50 up 5 down


Just for kicks, here's a video of the buzzing as experienced in the control room

Attachment 1: Feb12_negativeTR.png
Feb12_negativeTR.png
  11015   Thu Feb 12 15:21:37 2015 ericqUpdateComputer Scripts / Programsnetgpib updates

I've fixed the gpib scripts for the SR785 and AG4395A to output data in the same format as expected by older scripts when called by them. In addition, there are now some easier modes of operation through the measurement scripts SRmeasure and AGmeasure. These are on the $PATH for the main control room machines, and live in scripts/general/netgpib


Case 1: I manually set up a measurement on the analyzer, and just want to download / plot the data. 

Make sure you have a yellow prologix box plugged in, and can ping the address it is labeled with. (i.e. 'vanna'). Then, in the directory you want to save the data, run:

SRmeasure -i vanna -f mydata --getdata --plot

This saves mydata_(datetime).txt and mydata_(datetime).pdf in the current directory. 

In all cases, AGmeasure has the identical syntax. If the GPIB address is something other than 10, specifiy it with -a, but this is rarely the case.


Case 2: I want to remotely specify a measurement

Rather than a series of command line arguments, which may get lost to the mists of time, I've set the scripts up to use parameter files that serve as arguments to the scripts. 

Get the templates for spectrum and TF measurements in your current directory by running

SRmeasure --template

Set the parameters with your text editor of choice, such as frequency span, filename output, whether to create a plot or not, then run the measurement:

SRmeasure SR785template.yml


Case 3: I want to compare my data with previous measurements

In the template parameter files, there is an option 'plotRefs', that will automatically plot the data from files whose filenames start with the same string as the current measurement.

If, in the "#" commented out header of the data file, there is a line that contains "memo:" or "timestamp:", it will include the text that follows in the plot legend. 


There are also methods to remotely trigger an already configured measurement, or remotely reset an unresponsive instrument. Options can be perused by looking at the help in SRmeasure -h

I've tested, debugged, and used them for a bit, but wrinkles may remain. They've been svn40m committed, and I also set up a separate git repository for them at github.com/e-q/netgpibdata

  11022   Fri Feb 13 14:27:30 2015 ericqUpdateElectronicsSecond QPD Whitening Switch enabled

I have re-enabled the second whitening stage switching on each quadrant of each end's QPD whitening board, to try and avoid saturations at full power. Looking at the spectra while single arm locked, I confirmed that the FM2 whitening switch works as expected. FM1 should be left on, as it is still hard-wired to whiten. 

The oscillations in the Y QPD still exist. Jenne is updating the schematics on the DCC.

  11023   Fri Feb 13 14:59:13 2015 ericqUpdateElectronicsSecond QPD Whitening Switch enabled

Went to zero CARM offset on ALS; transmission QPDs are still saturating :(

Maybe we need to switch off all whitening.

  11026   Fri Feb 13 19:41:13 2015 ericqUpdateGeneralRF amplifier for ALS

As Koji found one of the spare channels of the ALS/FOL RF amplifier box nonfunctional yesterday, I pulled it out to fix it. I found that one of the sma cables did not conduct.

It was replaced with a new cable from Koji. Also, I rearranged the ports to be consistent across the box, and re-labeled with the gains I observed. 

It has been reinstalled, and the Y frequency counter that is using one of the channels shows a steady beat freq.

I cannot test the amplitude of the green X beat at this time, as Koji is on the PSL table with the PSL shutter closed, and is using the control room spectrum analyzer. However, the dataviewer trace for the fine_phase_out_Hz looks like free swinging cavity motion, so its probably ok.

  11046   Wed Feb 18 01:58:51 2015 ericqUpdateLSCALS Fool single arm performance

I'm playing around with the lastest ALS fool feedforward on the Yarm, and I like what I'm seeing. 

First, I verified that I could reproduce the TF shapes in ELOG 11041, which I was able to do with a gain of +9.3 and FMs 5 and 6 in the FF module. 

Then, I locked the arm on ALS with full bandwidth, and on POY with the settings currently used the MC module, and took their spectra as references. (I put an excitation on the arm at 443Hz to line them up to the same arbitrary units.)

Then, with ALS at its usual 100Hz UGF and boosts on, the Fool path on, and the MC FM set to trigger on/off at 0.8/0.5, I slowly brought ALS towards zero offset and saw it pop right into resonance. cool I then manually triggered the PDH boosts. 

Here are some spectra, showing that, with the Fool path on:

  • POY unsurprisingly picks up the high frequency noise of the ALS. (Could be mitigated by judicious lowpassing?)
  • The in-fool-loop POY noise is WAY more supressed at low frequencies, so the loops are definitely working together. RMS is about 2x smaller too.

Once the PDH loop is running, the ALS loop can be switched out at the CARM FM output without much of an effect beyond a small kick.


However, looking at the loop shapes, maybe I got lucky here. I took the usual injection TFs at the MC FM, the CARM FM, and at ETMY, to get the overall OLG; all of them have >0.9 coherence pretty much everywhere except the first two points.

 As desired, the PDH loop looks pretty normal.

I have no intuition about how the fooled CARM loop should look, since this is even more complicated than a two-loop system. 

I don't currently know what is causing the odd feature in the overall at ~50Hz, and it spooked me out when I saw the multiple UGF crossings. The only thing I could think of happening there is the pole in the ALS phase tracker boost. I turned it off, and remeasured; the feature persists...


To wrap it up, here's something I think is pretty cool. Here's what happens when I give ETMY a 1000 count position step impulse (no ramp). [Here, CARM is on ALS with G=12, but only FM5 on]

Although the arm was controlled with IR before the kick, ALS maintained control. As soon as ALS brought the arm back towards resonance, the PDH loop picked it right back up.

Coooool.


Some random notes:

  • We should really DQ the output of the feedforward FM. I'll try to remember to do this tonight
  • I was having problems with the zero crossing triggering, so I didn't end up using it, desipte trying to. Maybe we should implement the Schmitt-y style of "Am I below the threshold? Wait. Am I closer to zero now? Ok, Go!"
  • The 1Hz pole in the FF FM rings, unsurprisingly, when the MC FM triggers on briefly, which can be a pain. I made a FM4 which is a Q=1 (instead of 7) pole pair for less ringy. This probably hurts the cancellation somewhat, but I was impatient. 
    • Alternatively, we could try to figure out how to force history clear when the MC filter is triggered off. 

DTT data is attached, in case it's useful to anyone!

Attachment 1: ALSfool_spectra.png
ALSfool_spectra.png
Attachment 2: ALSfool_kick.png
ALSfool_kick.png
Attachment 3: ALSfool_LoopShapes.png
ALSfool_LoopShapes.png
Attachment 4: ALSfool_Feb182015.zip
  11047   Wed Feb 18 17:51:40 2015 ericqUpdateLSCALS Fool impulse response

Koji raised a good question about the step response I wrote about. Namely, if the UGF of the ALS servo is around 100Hz, we would expect it to settle with a characteristic time on the order of tens of milliseconds, not seconds, as was seen in the plot I posted. 

I claim that the reason for the slow response was the fact that the feedforward FM stayed on after the kick, despite the MC filter bank being triggered off. Since it has filters that look like a oscillator at 1Hz, the ringdown or exponential decay of this filter's output in response to the large impulsive output of the PDH control signal just before being triggered down would slowly push the ALS error signal around through the feedforward path. 

Given this reasoning, this should be helped by adding output triggering to the FF filter that uses the MC trigger matrix row, as I wanted to do anyways. I have now added this into the LSC model (as well as DQ at 2kHz for the MC_CTRL_FF_OUT), and the impulse response is indeed much quicker. 

In the following plot, I hit ETMY with a five sample, 5000 amplitude, impulse (rather than a step, as I did yesterday). The system comes back to PDH lock after ~40ms. 

Attachment 1: ALSfool_fasterKick.png
ALSfool_fasterKick.png
  11049   Thu Feb 19 04:09:21 2015 ericqUpdateElectronicsSecond QPD Whitening Switch enabled

X end QPD has recieved 0.2+0.4 absorptive ND filters. Y end QPD got one at 0.6. This appears to have mitigated the saturations for now; the unwhitened signals no longer go negative. The digital gains have been reset. 

 

  11051   Thu Feb 19 15:45:43 2015 ericqUpdateCDSBad CDS behavior

At about 10AM, the C1LSC frontend stopped reporting any EPICS information. The arms were locked at the time, and remained so for some hours, until I noticed the totally whited-out MEDM screens. The machine would respond to pings, but did not respond to ssh, so we had to manually reboot.

Soon thereafter, we had a global 15min EPICS freeze, and have been in a weird state ever since. Epics has come back (and frozen again), but the fast frontends are still wonky, even when EPICS is not frozen. Intermittantly, the status blinkers and GPS time EPICS values will freeze for multiple seconds at a time, sporadically updating. Looking at a StripTool trace of an IOPs GPS time value shows a line with smooth portions for about 30 seconds, about 2 minutes apart. Between this is totally jagged step function behavior. C1LSC needed to be power cycled again; trying to restart the models is tough, because the EPICS slowdown makes it hard to hit the BURT button, as is needed for the model to start without crashing.

The DAQ network switch, and martian switch inside were power cycled, to little effect. I'm not sure how to diagnose network issues with the frontends. Using iperf, I am able to show hundreds of Mbit/s bandwidth betweem the control room machines and the frontends, but their EPICS is still totally wonky. 

What can we do??? indecision

ELOG V3.1.3-