We will vent tomorrow (12/18)

After lengthy discussion, I have determined that we should vent now, with the primary goal of this vent to replace the input steering PZTs with the new active tip-tilts.  Since we still don't understand exactly what's going on with the PRC, it's unclear what we would do to attack the problem.  We need to do more modeling and measurements first.  We should limit the goal of this vent to replacing the PZTs, and then close up and do more measurements with better modeling and improved input point in hand.

There is limited time this week before everyone leaves for two-week holidays on Thursday or Friday.  The reason to not vent would be that we don't want to leave the IFO at air during the two week holiday.  People seem to think that this is not a problem, so we don't gain anything by waiting.  Therefore we vent now and do what we can before people take off.

The goal for this week is to replace PZT1 only:

Now: Jenne and Manasa are doing vent prep.  Manasa is lowering the input power and preparing the mode cleaner.  Jenne centered IPPOS and IPANG.  This will allow us to check how the input alignment changed during the vent.

12/18: Steve is going to start the vent as soon as he gets back from the dentist, at around 10am.  He will regulate the vent such that we are ready to lock the mode cleaner by 4pm.  At that time we will lock the MC and recheck the input alignment with IPPOS/ANG, with the idea being to see if anything moves during the vent.

12/19: First thing in the morning we take off the access connector ONLY.  The access connector is all we'll need to replace PZT1.    Put light door on the OMC chamber side immediately, since we won't need to get in there at all.  We won't need the light access connector.

For the rest of Wednesday we'll remove PZT1 and install the new active TT.  We'll be using the current out-of-vac cable for PZT1 for the new TT.  We should only have to modify the rack end of the cable to accommodate the coil driver.  This should be a small modification.  Given that we have no wiring diagrams we'll have to pin it out in situ.

12/20: Hopefully finish up TT installation.

Jenne leaves 12/20, Manasa and Jamie leave 12/21.  We will either leave light doors on access connector holes, or possibly Rana, Koji, and Steve will replace access connector on Friday so that we can pump down to 1 Torr or something so that we leave it there over the holiday.

After we return from vacation:

PZT2/TT2 installation.  This will be less straight forward since the new TT has a bigger foot print than PZT2 and will block the PRM optical levers.  We'll need one additional steering mirror to redirect the oplev around the TT.  See elog 7815.

Once the new TTs are installed, we'll reevaluate where we're at.  If PRC modeling has progressed and we have an idea of something to work on with the PRC, we can.  Otherwise, we'll just button up, pump down, and get on with some better PRC measurements.

[Jenne, Manasa, Jamie]

Now that we're up to air we relocked the mode cleaner, tweaked up the alignment, and looked at the spot positions:

The measurements from yesterday were made before the input power was lowered.  It appears that things have not moved by that much, which is pretty good.

We turned on the PZT1 voltages and set them back to their nominal values as recorded before shut-down yesterday.  Jenne had centered IPPOS before shutdown (IPANG was unfortunately not coming out of the vacuum).  Now we're at the following value: (-0.63, 0.66).  We need to calibrate this to get a sense of how much motion this actually is, but this is not insignificant.

Nic and I discovered a problem with the in-vac wiring from the feed-thru to the top of the table.  Pin 13 at the top of the stack, which is one of the coil pins on the tip-tilt quadrapus cables, is *the* shield braid on the cable that goes to the feed-thru.  This effectively shorts one of our coil signals.

There are three solutions as we see it:

* swap pin 13 for something else at the top of the stack, and then swap it back somewhere else outside of the vacuum.

* swap *all* the pins at the top of the table to be the mirror.  We would then need to mirror our cables on the outside, but that's less of an issue.

* make a mirror adapter that sits at the top.  This would obviously need to be cleaned/baked.

None of these solutions is particularly good or fast.

So this is obviously a general problem for all the TTs.  Our in-vacuum wiring is unfortunately mirrored relative to that of aLIGO, or at least:

• aLIGO: in-vacuum pin 1 tied to shield (T1200131)
• 40m: in vacuum pin 13 tied to shield

And again, the problem is that pin 13 on the TT quadrapus cables is the one of the coil pins for one of the OSEMs.

I think the right solution is to make mirroring adapter cables for the TTs.  Modifying the pins on the stack-top brackets for just the TTs would leave us with a bunch of brackets that are different than all the rest, which I think is a bad idea.  Therefore we leave all the feed-thru-->bracket wiring the same, and make adapters.  I'll describe the adapters in a follow-up post.

The silver lining to this whole thing, if there is one, is that I wired the polarity on the out-of-vac adapter cable at the coil driver in such a way that the drive/send signal went to the grounded in-vac pin.  If I had by chance wired the polarity oppositely everything would probably have worked, except that the return for one of the coils would have gone through the cable shield and the chamber, rather than the return pin to the coil driver.  I'll let you image the problems that would have caused.

Making new adapters will take a little while, but I think we can proceed with the installation and alignment with a temporary setup in the mean time by taking advantage of the polarity I mention above.  We can temporarily swap the polarity so that we can drive current to the coil using pin 13.  This will allow us to complete the installation and do all the alignment.  Once the in-vac adapter cable arrives, we just put it in, fix the out-of-vac polarity, check that everything works as expected, and button up.

We'll pick up all this when we're back on Jan 7.  Steve will put in an order for the in-vac adapter cables ASAP.

I take full responsibility for this fuck up.  We've been unable to find any in-vac wiring diagrams, but I should have checked all of the wiring during the last vent so that we could have prepared for this ahead of time.  Sorry.

We need short cables that mirror the pins:

The male side will plug into the 25-pin female on the stack-top bracket.  The tip-tilt quadrapus cable will plug into the female side. This will match up pin 1 on the tip-tilt cable, which is connected to it's shield, to pin 13 on the bracket, which is the shield of the cable that runs to the stack.

They need to be vacuum compatible.  Shorter length is preferred, and there is no minimum length (something like an all-in-one gender changer would be ideal, but probably expensive to have made).

There's definitely a timing issue with this machine.  I looked at it a bit yesterday.  I'll try to get to it by the end of the week.

I replaced the BS1 between the POPDC PD and the camera with a 98 reflector, and moved the 50 up before the BS to dump half the light.  Still saturating POPDC, but hopefully the ratio between POPDC and the camera should be better.  We just need to dump more of the power before we get there.  I'll come back to this after C&D if no one else has already gotten to it.

I don't know why I didn't pay more attention last night, but things look way WAY better.  The beams are much cleaner and the power level is much much higher.

Here's a sort of rough analysis of the aligned PRM-PR2 cavity mode scan.

On Friday we took some mode scan measurements of the PRM-PR2 cavity by pushing PRM (C1:SUS-PRM_LSC_EXT) with a 0.01 Hz, 300 count sine wave.  We looked at the transmitted power on the POP DC PD and the error signal on REFL11_I.

Below is a detail of the scan, chosen because the actuation was in its linear region and there were three relatively ok looking transmission peaks with nice PDH response curves:

The vertical green lines on the bottom plot indicate the rough averaged separation of the 11 MHz side-band resonances from the carrier, at +- 0.0275 s.  If we take this for our calibration, we get roughly 400 MHz / second.

The three peaks in top plot have an average FWHM of 0.00381 s.  Given the calibration above, the average FWHM = ~1.52 MHz.

If we assume a cavity length of 1.91 m, FSR = 78.5 MHz.

Putting this together we get a finesse = ~51.6.

Analysis of misaligned mode scans to follow.

I've added a new program called getdata (to scripts/general/getdata) that will conveniently pull arbitrary data from an NDS server, either DQ or online (ie. testpoints).

Start times and durations may be specified.  If past data is requested, you must of course be requesting DQ channels.  If no start time is specified, data will be pulled "online", in which case you can specify testpoints.

If an output directory is specified, the retrieved data will be stored in that directory in files named after the channels.  If an output directory is not specified, no output will be

Help usage:

controls@pianosa:~ 0$ /opt/rtcds/caltech/c1/scripts/general/getdata --help
usage: getdata [-h] [-s START] [-d DURATION] [-o OUTDIR] channel [channel ...]

Pull online or DQ data from an NDS server. Use NDSSERVER environment variable
to specify host:port.

positional arguments:
  channel               Acquisition channel. Multiple channels may be
                        specified acquired at once.

optional arguments:
  -h, --help            show this help message and exit
  -s START, --start START
                        GPS start time. If omitted, online data will be
                        fetched. When specified must also specify duration.
  -d DURATION, --duration DURATION
                        Length of data to acquire.
  -o OUTDIR, --outdir OUTDIR
                        Output directory. Data from each channel stored as
                        '.txt'. Any existing data files will be
                        automatically overwritten.
controls@pianosa:~ 0$ 

I noticed that Koji used a high reflector for the ITMs for his full PRC arbcav calculation. I just redo it here with the correct ITM transmission and RoC for completeness.

In this case the finesse is 95, instead of 121.

I ran Zach's arbcav on our SRC with curved TTs and the situation looks much worse than the PRC.

I used the following parameters

SRM: RoC = 142 m, T = 10%
ITM: RoC = 83.1e3 m, T = 1.4%
SRC length: 5.37 m

In this case, with TT RoC of -600, the combined cavity g-factor = 0.9986, and astigmatism from SR3 makes the cavity patently not stable.  You have to go up to an RoC of -710 before the cavity is just over the edge.

This is just a simple rerun of arbcav from #7995 but with the PR2/3 RoCs set to 600, instead of -600.  Overall g-factor = 0.922, and the modes are well separated:

This doesn't take into account the effect of traveling through the substrates (still working on it).  It assumes the PR2/3 have been moved such that the cavity fold lengths remain the same.

This is something that we need to keep in mind: we will need to adjust the position of the PR2/3 to keep the fold lengths the same.

I intended to post a long analysis of the PRC/arm mode matching for the various TT situations using Nic's a la mode mode matching program, but I seem to have encountered what I think might be a bug.  I'll talk to Nic about it first thing in the AM.  Once the issue is resolved I should be able to post the full analysis fairly quickly.  Sorry about the delay.

NOTE: There was a small bug in my initial calculation.  The plots and numbers have been updated with the fixed values.  The conclusion remains the same.

Using Nic's a la mode mode matching program, I've calculated the PRC mode and g-parameter for various PR2/3 scenarios.  I then looked at the overlap of the resultant PRC eigenmodes with the ARM eigenmode.  Here are the results:

NOTE: each optical element below (PR2, ITM, etc.) is represented by a compound M matrix.  The z axis in these plots is actually just the free space propagation between the elements, not the overall optical path length.

ARM

This is the ARM mode I used for all comparisons:

PRC, nominal design (flat PR2/3)

This is the nominal "as designed" PRC, with flat PR2/3 folding mirrors.

 ARM mode matching: 0.9998

PRC, both PR2/3 flipped

This assumes both PR2 and PR3 have a RoC of -600 when not flipped, and includes the affect of propagation through the substrates.

ARM mode matching: 0.9806

PRC, only PR2 flipped

In this case we only flip PR2 and leave PR3 with it's bad -600 RoC:

ARM mode matching: 0.9859

Discussion

I left out the current situation (PR2/3 with -600 RoC) and the case where only PR3 is flipped, since those are both unstable according to a la mode.

I guess the main take away is that we get slightly better PRC stability and mode matching to the arms by only flipping PR2.

I completely agree with Koji.  We definitely should have locked the half PRC first.  We were all set up for that.  Why go through all this work to align MICH when we haven't confirmed with the half PRC that the flipping is helping us?  The first rule of debugging is to only make one change at a time.  We have measurements from the half PRC, so we could have made a direct comparison with those to see how things have changed.  If we jump the gun we're going to end up wasting more time when we have to back-track.

Also, we never talked about moving PR2 to adjust optical path length, although I can understand why we would think that should be done.  My calculations were all done assuming the free-space separation between PRM/PR2 and PR2/PR3 were unchanged.  It's possible changing the position is better, but again, it's more work and it changes multiple things at one.  I can redo my calculations for this new scenario, but we need to update our drawings with this new configuration.  Please note precisely where PR2 has moved to.

We should have just flipped PR2 and that's it.  Then we could have run the exact same measurements we had previously.  Only then, once we understood this new simple cavity, should we have done further adjustments.

You would guess, and I would have guessed too, but the calculations tell the story.   Flipping both does not increase the stability.  The main issue is that flipping PR3 induces considerable astigmatism.  This is why flipping PR3 alone does not make the cavity stable.  I will do some simple calculations today that will demonstrate this effect.

But again, we should only change one thing at a time and understand that before moving on.  Given that the calculations show that flipping only PR2 should alone have a positive affect, we should do just that first, and verify that we understand what's going on, before we move on to making more changes.

I will try to make some more arbcav runs as well, for just the flipped PR2.

Arbcav with half PRC (flat temporary mirror in front of BS), PR2 RoC = 600m, PR3 RoC = -600m:

NOTE: this does NOT include the affect of the PR2 substrate in the cavity.  Arbcav does not handle that.  It would have to be modified to accept arbitrary ABCD matrices.

NOTE: I added to the mode plot the frequency separation of the first HOMs from the carrier (\omega_{10/01}), in units of carrier FSR.

This is exactly why I added the higher order mode spacing, so you could calculate the g parameter.  For TEM order N = n + m with spacing f_N, the overall cavity g parameter should be:

g = (cos( (f_N/f_FSR) * (\pi/N) ))^2

The label on the previous plat should really be f_N/FSR, not \omega_{10,01}

BUT, arbcav does not currently handle arbitrary ABCD matrices for the mirrors, so it's going to be slightly less accurate for our more complex flipped mirrors.  The affect would be bigger for a flipped PR3 than for a flipped PR2, because of the larger incidence angle, so arbcav will be a little more correct for our flipped PR2 only case (see below).

This is not correct.  Multiplying the RoC by -N would be a very large change.  For an arbitrary ABCD matrix:

R_eff = -2 / C

When the incident angle in non-zero:

tangential: R_eff = R_eff / cos(\theta)
sagittal:   R_eff = R_eff * cos(\theta)

For flipped PR2, with small 1.5 degree incident angle and RoC of -706 at HR:

M_t = M_s = [1.0000, 0.0131; -0.0028, 1.0000]
R_eff = 705.9

For flipped PR3, with large 41 degree incident angle and RoC of -700 at HR:

M_t = [1.0000, 0; 0.0038, 1.0000]
M_s = [1.0000, 0; 0.0022, 1.0000]
R_eff = 592.4

The affect of the substrate is negligible for flipped PR2 but significant for flipped PR3.

The current half-PRC setup

OK, I have now completely reconciled my alamode and arbcav calculations.  I found a small bug in how I was calculating the ABCD matrix for non-flipped TTs that made a small difference.  I now get the exact same g parameter values with both with identical input parameters.

Sooooo, I have redone my alamode and arbcav calculations with these updated values.  Here are the resulting g parameters

So the sagittal values all agree pretty well, but the tangential measurement does not.  Maybe there is an actual astigmatism in one of the optics, not due to angle of incidence?

arbcav HOM plot:

Password for nodus and all control room workstations has been changed.  Look for new one in usual place.

We will try to change the password on all the RTS machines soon.  For the moment, though, they remain with the old passwerd.

Given that we're measuring different g parameters in the tangential and sagittal planes, I went back to alamode to see what astigmatism I could put into PR2 and/or PR3 to match what we're measuring.  I looked at three cases: only PR2 is astigmatic, only PR3 is, or where we split the difference.  Since the sagittal measurement matches, I left all the sagittal curvatures the same in

case 1: PR3 only

case 2: PR3 only

case 3: PR2 and PR3

From Koji's post about the scans of the G&H mirrors, it looks entirely reasonable that we could have these levels of astigmatism in the optics.

What this means for full PRC

These all make the same full PRC situation:

     g (tangential):  0.966

     g (sagittal):  0.939

     ARM mode matching:  0.988

Is it possible to calculate astigmatism with your tools?  Can we get curvature in X/Y direction, preferably aligned with some axis that we might align to in the vacuum?

Yuta, Manasa, Jamie, Jenne, Steve, Rana

Starting this morning, we removed the temporary half PRC mirror in front of BS and started to align the IFO in prep for an in-air lock of the PRMI.

This morning, using the new awesome steerable active input TTs, Jenne and I centred the beam on PRM, PR2/3, BS, ITMY and ETMY.

After lunch, Yuta and Manasa aligned the Y ARM, by looking at the multi-pass beam.  The X-end door was still on, so they roughly aligned to the X ARM by centring on ITMX with BS.  They then got fringes at the BS, and tweaked the ITMs and PRM to get full fringes at BS.

We're currently stuck because the REFL beam appears to be clipped coming out of the faraday, even though the retro-reflected beam from PRM is cleanly going through the faraday output aperture.  The best guess at the moment is that the beam is leaving MC at an angle, so the retro-reflected beam is coming out of the faraday at an angle.  We did not center spots on MC mirrors before we started the alignment procedure today.  That was dumb.

We may be ok to do our PRMI characterization with the clipped REFL, though, then we can fix everything right before we close up.  We're going to need to go back to touch up alignment before we close up anyway (we need to get PR2 centered).

Yuta and Manasa are finishing up now by making sure the AS and REFL beams are cleanly existing onto the AS table.

Tomorrow we will set up the PRM oplev, and start to look at the in-air PRMI.  Hopefully we can be ready to close up by the end of the week.

After using alamode to calculate the round-trip mode of the beam at the Faraday exit after retro-reflection form the PRM, I'm not able to blame the MMT and TT curvature for the beam ellipticity.

I assume an input waist at the mode cleaner of [0.00159, 0.00151] (in [T, S]).  Propagating this through the MMT to PRM, then retro-reflecting back with flat TTs I get

w_t/w_s = 0.9955,  e = 0.0045

If I give the TTs a -600 m curvature, I get:

w_t/w_s = 1.0419,  e = 0.0402

That's just a 4% ellipticity, which is certainly less than we see.  I would have to crank up the TT curvature to -100m or so to see an ellipticity of 20%.  We're seeing something that looks bigger than 50% to me.

Below are beam size through MMT + PRM retro-reflection, TT RoC = -600m:

I appears that the c1iscex IO-chassis is either dead or in a very bad state.  The PCIe interface card in the IO-chassis is showing four red lights, where it's supposed to be showing a dozen or so green lights.  Obviously this is going to prevent anything from running.

We've had power issues with this chassis before, so possibly that's what we're running into now.  I'll pull the chassis and diagnose asap.

In order to address the issue of low MC1 OSEM voltages, Yuta and I looked at the IMC table levelling.  Looking with the bubble level, Yuta confirmed that the table was indeed out of level in the direction that would cause MC1 to move closer to it's cage, and therefore lower it's OSEM voltages.  Looking at the trends, it looks like the table was not well levelled after TT1 installation.  We should have been more careful, and we should have looked at the MC1/3 voltages after levelling.

Yuta moved weights around on the table to recover level with the bubble level.  Unfortunately this did not bring us back to good MC1 voltages.  We speculate that the table was maybe not perfectly level to begin with.  We decided to try to recover the MC1 OSEM voltages, rather than go solely with the bubble level, since we believe that the MC suspensions should be a good reference.  Yuta then moved weights around until we got the MC1/3 voltages back into an acceptable range.  The voltages are still not perfect, but I believe that they're acceptable.

The result is that, according to the bubble level, the IMC table is low towards MC2.  We are measuring spot positions now.  If the spot positions look ok, then I think we can live with this amount of skew.  Otherwise, we'll have to physically adjust the MC1 OSEMS.

c1iscex is back up.  It is communicating with it's IO chassis, and all of it's models (c1x01, c1scx, c1spx) are running again.

The problem was that the IO chassis had no connection to the computer.  The One Stop card in the IO chassis, which is the PCIe bridge from the front-end machine and the IO chassis, was showing four red lights instead of the dozen or so green lights that it usually shows.  Upon closer inspection, the card appeared to be complaining that it had no connection to the host card in the front-end machine.  Un-illuminated lights on the host card seemed to be pointing to the same thing.

There are two connector slots on the expansion card, presumably for a daisy chain situation.  Looking at other IO chassis in the lab I determined that the cable from the front-end machine was plugged into the wrong slot in the One Stop card.  wtf.

Did someone unplug the cable connecting c1iscex to it's IO chassis, and then replug it in in the wrong slot?  A human must have done this.

The c1iscex machine itself wasn't dead, the models were just not running.  Here are the last messages in dmesg:

[130432.926002] c1spx: ADC TIMEOUT 0 7060 20 7124
[130432.926002] c1scx: ADC TIMEOUT 0 7060 20 7124
[130433.941008] c1x01: timeout 0 1000000 
[130433.941008] c1x01: exiting from fe_code()

I'm guessing maybe the timing signal was lost, so the ADC stopped clocking.   Since the ADC clock is the everything clock, all the "fe" code (ie. models) aborted. Not sure what would have caused it.

I restarted all the models ("rtcds restart all") and everything came up fine. Obviously we should keep our eyes on things, and note if anything strange was happening if this happens again.

At around 2:30pm today something brought down most of the martian network.  All control room workstations, nodus, etc. were unresponsive.  After poking around for a bit I finally figured it had to be linux1, which serves the NFS filesystem for all the important CDS stuff.  linux1 was indeed completely unresponsive.

Looking closer I noticed that the Fibrenetix FX-606-U4 SCSI hardware RAID device connected to linux1 (see #1901), which holds cds network filesystem, was showing "IDE Channel #4 Error Reading" on it's little LCD display.  I assumed this was the cause of the linux1 crash.

I hard shutdown linux1, and powered off the Fibrenetix device.  I pulled the disk from slot 4 and replaced it with one of the spares we had in the control room cabinets.  I powered the device back up and it beeped for a while.  Unfortunately the device was requiring a password to access it from the front panel, and I could find no manual for the device in the lab, nor does the manufacturer offer the manual on it's web site.

Eventually I was able to get linux1 fully rebooted (after some fscks) and it seemed to mount the hardware RAID (as /dev/sdc1) fine.  The brought the NFS back.  I had to reboot nodus to get it recovered, but all the control room and front-end linux machines seemed to recover on their own (although the front-ends did need an mxstream restart).

The remaining problem is that the linux1 hardware RAID device is still currently unaccessible, and it's not clear to me that it's actually synced the new disk that I put in it.  In other words I have very little confidence that we actually have an operational RAID for /opt/rtcds.  I've contacted the LDAS guys (ie. Dan Kozak) who are managing the 40m backup to confirm that the backup is legit.  In the mean time I'm going to spec out some replacement disks onto which to copy /opt/rtcds, and also so that we can get rid of this old SCSI RAID thing.

[Jamie, Brett, Jenne]

We made some small modifications to the sus_single_control suspension controller library part to get in/out the signals that Brett needs for his "global damping" work.  We brought out the POS signal before the SUSPOS DOF filter, and we added a new GLOBPOS input to accommodate the global damping control signals.  We added a new EPIC input to control a switch between local and global damping.  It's all best seen from this detail from the model:

The POSOUT goto goes to an additional output.  As you can see I did a bunch of cleanup to the spaghetti in this part of the model as well.

As the part has a new input and output now we had to modify c1sus, c1scx, c1scy, and c1mcs models as well.  I did a bunch of cleanup in those models as well.  The models have all been compiled and installed, but a restart is still needed.  I'll do this first thing tomorrow morning.

All changes were committed to the userapps SVN, like they should always be.

We still need to update the SUS MEDM screens to display these new signals, and add switches for the local/global switch.  I'll do this tomorrow.

During the cleanup I found multiple broken links to the sus_single_control library part.  This is not good.  I assume that most of them were accidental, but we need to be careful when modifying things.  If we break those links we could think we're updating controller models when in fact we're not.

The one exception I found was that the MC2 controller link was clearly broken on purpose, as the MC2 controller has additional stuff added to it ("STATE_ESTIMATE"):

I can find no elog that mentions the words "STATE" and "ESTIMATE".  This is obviously very problematic.  I'm assuming Den made these modifications, and I found this report: 7497, which mentions something about "state estimation" and MC2.  I can't find any other record of these changes, or that the MC2 controller was broken from the library.  This is complete mickey mouse bullshit.  Shame shame shame.  Don't ever make changes like this and not log it.

I'm going to let this sit for a day, but tomorrow I'm going to remove replace the MC2 controller with a proper link to the sus_single_control library part.  This work was never logged so it didn't happen as far as I'm concerned.

[jamie, brett]

Yesterday we added some new control logic to the sus_single_control part to allow for global damping.  Today we decided that a binary switch between local/global damping was probably a bit extreme since we might want to smoothly ramp between them, instead of just hard switching.  So we removed this switch and are now just summing the control inputs from global and local damping right before the output matrix.

Changes were committed to the SVN, and all suspension models were recompiled/installed/restarted.

I made good on my threat from yesterday to convert the MC2 suspension controller to the library part.  Whatever changes were in MC2 were thrown out, although they are archived in the SVN.  Again, this kind of undocumented breaking is forbidden.

Change was committed to SVN, and c1mcs was recompiled/installed/restarted.

I updated the getdata script so that it can now handle downloading long stretches of data.

  /opt/rtcds/caltech/c1/scripts/general/getdata

It now writes the data to disk incrementally while it's downloading from the server, so it doesn't fill up memory.

I also added a couple new options:

* --append allows for appending to existing data files

* --noplot suppresses plotting during download

I did a little bit of cleanup of the suspension POS inputs today, both in model and MEDM land.

model

sus_single_control.mdl was simplified such that now there are just two position inputs:

• POS: with LSC filter
• ALTPOS: unfiltered

The regular LSC inputs go into POS, and any optic-specific extra pos inputs go into ALTPOS after being properly filtered and summed.

So for instance, MC2_MCL and MC2_ALS are filtered and summed then go into MC2 ALTPOS.  The ETM ALS inputs go into ALTPOS.

I modified the GLOBAL DAMPING outputs so that they are filtered in the GLOBAL block before being sent to be summed before going into ALTPOS for {I,E}TM{X,Y}.

All suspension models were rebuilt/installed/restarted.

MEDM

The SUS_SINGLE.adl template screen was modified such that the POS button now points to optic-specific POS filter screens at:

/opt/rtcds/caltech/c1/medm/master/sus/SUS_$(OPTIC)_POS.adl

For MC1, MC3, PRM, BS, SRM these are links to SUS_SINGLE_POS.adl.  The rest of the suspensions (MC2, {I,E}TM{X,Y}) now have custom screens that are variations of SUS_SINGLE_POS but with their extra filter screens added in.  For instance, here is the new SUS_ETMX_POS.adl:

This gets rid of all the white screen crap that was in here before.

All of this has been committed to the SVN.  NOTE: symlinks were heavily used when sorting this stuff out, so check for symlinks when modifying in the future.

I spent some time trying to figure out how to get a record of the pointing of the input pointing tip-tilt (TT) channels.

Frames

Currently the TT pointing is done via the offset in the PIT/YAW filter banks, ie. C1:IOO-TT1_PIT_OFFSET, which is an EPICS record.  I added these channels to the C0EDCU.ini, which (I'm pretty sure) specifies which EPICS channels are recorded to frames.

controls@pianosa:~ 0$ grep C1:IOO-TT /opt/rtcds/caltech/c1/chans/daq/C0EDCU.ini 
[C1:IOO-TT1_PIT_OFFSET]
[C1:IOO-TT1_YAW_OFFSET]
[C1:IOO-TT2_PIT_OFFSET]
[C1:IOO-TT2_YAW_OFFSET]
controls@pianosa:~ 0$ 

I then confirmed that the data is being recorded:

controls@pianosa:~ 0$ FrChannels /frames/full/10466/C-R-1046657424-16.gwf | grep TT
C1:IOO-TT1_PIT_OFFSET 16
C1:IOO-TT1_YAW_OFFSET 16
C1:IOO-TT2_PIT_OFFSET 16
C1:IOO-TT2_YAW_OFFSET 16
controls@pianosa:~ 0$ 

BURT

The EPICS records for these channels *should* be recorded by autoburt, but Yuta noticed they were not:

controls@pianosa:~ 0$ grep -R C1:IOO-TT1_PIT_OFFSET /opt/rtcds/caltech/c1/burt/autoburt/snapshots/2013/Mar/6/
controls@pianosa:~ 1$

The autoburt log seems to indicate some sort of connection problem:

controls@pianosa:! 130$ grep C1:IOO-TT1_PIT_OFFSET /opt/rtcds/caltech/c1/burt/autoburt/logs/c1assepics.log
pv >C1:IOO-TT1_PIT_OFFSET< nreq=-1
pv >C1:IOO-TT1_PIT_OFFSET.HSV< nreq=-1
pv >C1:IOO-TT1_PIT_OFFSET.LSV< nreq=-1
pv >C1:IOO-TT1_PIT_OFFSET.HIGH< nreq=-1
pv >C1:IOO-TT1_PIT_OFFSET.LOW< nreq=-1
C1:IOO-TT1_PIT_OFFSET ... ca_search_and_connect() ... OK
C1:IOO-TT1_PIT_OFFSET.HSV ... ca_search_and_connect() ... OK
C1:IOO-TT1_PIT_OFFSET.LSV ... ca_search_and_connect() ... OK
C1:IOO-TT1_PIT_OFFSET.HIGH ... ca_search_and_connect() ... OK
C1:IOO-TT1_PIT_OFFSET.LOW ... ca_search_and_connect() ... OK
C1:IOO-TT1_PIT_OFFSET ... not connected so no ca_array_get_callback()
C1:IOO-TT1_PIT_OFFSET.HSV ... not connected so no ca_array_get_callback()
C1:IOO-TT1_PIT_OFFSET.LSV ... not connected so no ca_array_get_callback()
C1:IOO-TT1_PIT_OFFSET.HIGH ... not connected so no ca_array_get_callback()
C1:IOO-TT1_PIT_OFFSET.LOW ... not connected so no ca_array_get_callback()
controls@pianosa:~ 0$ 

This is in contrast to successfully recorded channels:

controls@pianosa:~ 0$ grep C1:LSC-DARM_OFFSET /opt/rtcds/caltech/c1/burt/autoburt/logs/c1lscepics.log
pv >C1:LSC-DARM_OFFSET< nreq=-1
pv >C1:LSC-DARM_OFFSET.HSV< nreq=-1
pv >C1:LSC-DARM_OFFSET.LSV< nreq=-1
pv >C1:LSC-DARM_OFFSET.HIGH< nreq=-1
pv >C1:LSC-DARM_OFFSET.LOW< nreq=-1
C1:LSC-DARM_OFFSET ... ca_search_and_connect() ... OK
C1:LSC-DARM_OFFSET.HSV ... ca_search_and_connect() ... OK
C1:LSC-DARM_OFFSET.LSV ... ca_search_and_connect() ... OK
C1:LSC-DARM_OFFSET.HIGH ... ca_search_and_connect() ... OK
C1:LSC-DARM_OFFSET.LOW ... ca_search_and_connect() ... OK
C1:LSC-DARM_OFFSET ... ca_array_get_callback() nreq 1 ... OK
C1:LSC-DARM_OFFSET.HSV ... ca_array_get_callback() nreq 1 ... OK
C1:LSC-DARM_OFFSET.LSV ... ca_array_get_callback() nreq 1 ... OK
C1:LSC-DARM_OFFSET.HIGH ... ca_array_get_callback() nreq 1 ... OK
C1:LSC-DARM_OFFSET.LOW ... ca_array_get_callback() nreq 1 ... OK
controls@pianosa:~ 0$ 

In fact all the records in the c1assepics log are showing the same "not connected so no ca_array_get_callback()" error.  I don't know what the issue is.  I have no problem reading the values from the command line, with e.g. ezcaread.  So I'm perplexed.

If anyone has any idea why the c1ass EPICS records would fail to autoburt, let me know.

I pulled the beatbox from the 1X2 rack so that I could try to hack in some output whitening filters.  These are shamefully absent because of my mis-manufacturing of the power on the board.

Right now we're just using the MON output.  The MON output buffer (U10) is the only chip in the output section that's stuffed:

The power problem is that all the AD829s were drawn with their power lines reversed.  We fixed this by flipping the +15 and -15 power planes and not stuffing the differential output drivers (AD8672).

It's possible to hack in some resistors/capacitors around U10 to get us some filtering there.  It's also possible to just stuff U9, which is where the whitening is supposed to be, then just jump it's output over to the MON output jack.  That might be the cleanest solution, with the least amount of hacking on the board.

In any event, we really need to make a v2 of these boards ASAP.  Before we do that, though, we need to figure out what we're going to do with the "disco comparator" stage back near the RF input.  (There are also a bunch of other improvements that will be incorporated into v2).

The framebuilder RAID is back online.  The disk had been mounted read-only (see below) so daqd couldn't write frames, which was in turn causing it to segfault immediately, so it was constantly restarting.

The jetstor RAID unit itself has a dead power supply.  This is not fatal, since it has three.  It has three so it can continue to function if one fails.  I have removed the bad supply and gave it to Steve so he can get a suitable replacement.

Some recovery had to be done on fb to get everything back up and running again.  I ran into issues trying to do it on the fly, so I eventually just rebooted.  It seemed to come back ok, except for something going on with daqd.  It was reporting the following error upon restart:

[Tue Mar 12 11:43:54 2013] main profiler warning: 0 empty blocks in the buffer

It was spitting out this message about once a second, until eventually the daqd died.  When it restarted it seemed to come back up fine.  I'm not exactly clear what those messages were about, but I think it has something to do with not being able to dump it's data buffers to disk.  I'm guessing that this was a residual problem from the umounted /frames, which somehow cleared on it's own.  Everything seems to be ok now.

DO NOT DO THIS.  This is what caused all the problems.  The unit has three redundant power supplies, for just this reason.  It was probably continuing to function fine.  The beeping was just to tell you that there was something that needed attention.  Rebooting the device does nothing to solve the problem.  Rebooting in an attempt to silence beeping is not a solution.  Shutting of the RAID unit is basically the equivalent of ripping out a mounted external USB drive.  You can damage the filesystem that way.  The disk was still functioning properly.  As far as I understand it the only problem was the beeping, and there were no other issues.  After you hard rebooted the device, fb lost it's mounted disk and then went into emergency mode, which was to remount the disk read-only.  It didn't understand what was going on, only that the disk seemed to disappear and the reappear.  This was then what caused the problems.  It was not the beeping, it was the restarting the RAID that was mounted on fb.

Computers are not like regular pieces of hardware.  You can't just yank the power on them.  Worse yet is yanking the power on a device that is connected to a computer.  DON"T DO THIS UNLESS YOU KNOW WHAT YOU"RE DOING.  If the device is a disk drive, then doing this is a sure-fire way to damage data on disk.

I modified the beatbox according to this plan.  I stuffed the whitening filter stage (U9) as indicated in the schematic (I left out the C26 compensation cap which, according to the AD829 datasheet, is not actually needed for our application).  I also didn't have any 301 ohm resistors so I stuffed R18 with 332 ohm, which I think should be fine.

Instead of messing with the working monitor output that we have in place, I stuffed the J5 SMA connector and wired U9 output to it in a single-ended fashion (ie. I grounded the shield pins of J5 to the board since we're not driving it differentially).  I then connected J5 to the I/Q MON outputs on the front panel.  If there's a problem we can just rewire those back to the J4 MON outputs and recover exactly where we were last week.

It all checks out: 0 dB of gain at DC, 1 Hz zero, 10 Hz pole, with 20 dB of gain at high frequencies.

I installed it back in the rack, and reconnected X/Y ARM ALS beatnote inputs and the delay lines.  The I/Q outputs are now connected directly to the DAQ without going through any SR560s (so we recover four SR560s). 

Below are new alamode calculations of the PRC and SRC g-factors and arm mode matchings.  These include fixes to the ABCD matrices for the flipped folding mirrors that properly (hopefully) take into account the focusing effect of passing through the optic substrates.

I've used nominal curvatures of -600 m for the G&H PR2/SR2 optics, and -700 m for the Laseroptik PR3/SR3 dichroics.

An interesting and slightly disappointing note is that it looks like it actually would have been better to flip PR3 instead of PR2, although the difference isn't too big.  We should considering flipping SR3 instead or SR2 when dealing with the SRC.  I'll take responsibility for messing up the calculation for the flipped TTs.

PRC

SRC

RXA: maybe nominal, but we don't actually have measurements of the installed optics' curvatures, so there could be ~10-15% errors in the RoC. Which translates into a 1-2% error in the g-factor.

Is there a reason to use non-45 degree incident angles on the steering mirrors between the laser and the PD?  I would always use 45 degree incident angles unless there is a really good reason not to.

I'm not exactly sure what the problem was here, but I think it had to do with a stuck mx_stream process that wasn't being killed properly.  I manually killed the process and it seemed to come up fine after that.  The regular restart mechanisms should work now.

No idea what caused the process to hang in the first place, although I know the newer RCG (2.6) is supposed to address some of these mx_stream issues.

In case anyone is curious how I got the numbers for the effective radius of curvature of the flipped TT mirrors, I include the code below.  Now you can calculate at home!

Here's the calculation for the effective RoC of a flipped SR2 with nominal un-flipped HR RoC of -600:

>> [Mt, Ms] = TTflipped(600, 5);
>> M2Reff('t', Mt, 5)

ans =

  412.9652

>>

Steve's suggestion for how to level the end table using "swivel leveling mounts":

Steve ordered a replacement power supply for the FB JetStor power supply that failed a couple weeks ago.  I just installed it and it looks fine.

This turned out to just be a loose connection of the ribbon cable from Contec board in the LSC IO chassis at the BIO break-out box.  The DSUB connector at the break-out box was not strain relieved!  I reseated the connector and strain relieved it and now everything is switching fine.

I wonder if we'll ever learn to strain relieve...

I seem to be able to retrieve these channels ok from the past:

controls@pianosa:/opt/rtcds/caltech/c1/scripts 0$ tconvert 1049050000
Apr 03 2013 18:46:24 UTC
controls@pianosa:/opt/rtcds/caltech/c1/scripts 0$ ./general/getdata -s 1049050000 -d 10 --noplot C1:LSC-PRCL_TRIG_MON
Connecting to server fb:8088 ...
nds_logging_init: Entrynds_logging_init: Exit
fetching... 1049050000.0
Hit any key to exit: 
controls@pianosa:/opt/rtcds/caltech/c1/scripts 0$ 

Maybe DTT just needed to be reloaded/restarted?

I investigated the situation of the two Sorensen supplies in the LSC rack (1Y2).  They are there solely to supply power to the LSC LO RF distribution box.  One is +18 V and the other is +28 V.  All we need to do is make a new longer cable with the appropriate plug on one end (see below), long enough to go from the bottom of the 1Y3 rack to the top of 1Y2, and we could move them over quickly.  Some sort of non-standard circular socket connector is used on the distribution box:

It could probably use thicker conduction wire as well.

If someone else makes the cable I'll move everything over.

I started to look into putting together a 110 MHz demod board to be used as POP110 (see #8399).

We have five spare old-skool EuroCard demod boards (LIGO-D990511).  From what I gather (see #4538, #4708) there are two modifications we do to these boards to make them ready for prime time:

• appropriate LP filter at PD RF input (U5 -> MC SCLF-*)
• swap out T1 transformer network with a commercial phase shifting power splitter (MC PQW/PSCQ)

#4538 also describes some other modifications but I'm not sure if those were actually implemented or not:

• removal of the attenuator/DC block/ERA-5 amp sections at the I/Q outputs
• swap ERA-5 amp with "Cougar"(?) amp at LO input.

What we'll need for a 110 demod:

• U5 -> SCLF-135
• T1 network -> PSCQ-2-120

I'll scrounge or order.

This SCPQ-150+, which is surface mount, might also work in place of the PSCQ-2-120, which is through-mount.  Would need to be reconciled with the board layout.

I'm having Steve order the following:

2x  SXBP-100+
2x  SCLF-135+
2x  PSCQ-2-120+

If you want him to add anything to the order let him know ASAP.

Folks were complaining that they were getting zeros whenever they tried to open fast channels in DTT or Dataviewer.  It turned out that the problem was that all available test points were in use in the c1lsc model:

There is a limit to how many test points can be open to a single model (in point of fact I think the limit is on the data rate from the model to the frame builder, not the actual number of open test points).  In any event, they was all used up.  The grid at the bottom right of the C1LSC GDS screen was all full of non-zeros, and the FE TRATE number was red, indicating that the data rate from this model had surpassed threshold.

The result of this overbooking is that any new test points just get zeros.  This is a pretty dumb failure mode (ideally one would not be able to request the TP at all with an appropriate error message), but it is what it is.  This usually means that there are too many dtt/dataviewers left with open connections.

We tried killing all the open processes that we could find that might be holding open test points, but that didn't seem to clear them up.  Stuck open test points is another known problem.  Referencing the solution in #6968 I opened the diag shell and killed all test points everywhere:

controls@pianosa:~ 0$ diag -l -z
Set new test FFT
NDS version = 12
supported capabilities: testing  testpoints  awg  
diag> tp clear * *
test point cleared
diag> quit
EXIT KERNEL
controls@pianosa:~ 0$