Yuta retrieved the IR card that had fallen to the bottom of the IOO chamber, just before we put on the access connector yesterday.  The clean "pickle picker" long grabber tool worked wonders.

controls@rosalba:/opt/rtcds/caltech/c1/medm/c1ioo/master/master/master 0$ ls
C1IOO_LKIN_OUT_MTRX.adl   C1IOO_MC_ASS_LOCKIN5.adl     C1IOO_WFS1_I.adl             C1IOO_WFS_LKIN.adl
C1IOO_LOCKMC.adl          C1IOO_MC_ASS_LOCKIN6.adl     C1IOO_WFS1_Q.adl             C1IOO_WFS_MASTER.adl
C1IOO_LOCKMC_BAK.adl      C1IOO_MC_ASS_PIT_LOCKIN.adl  C1IOO_WFS1_SETTINGS.adl      C1IOO_WFS_MASTER.adl~
C1IOO_MC_ALIGN.adl        C1IOO_MC_ASS_YAW_LOCKIN.adl  C1IOO_WFS1_SETTINGS.adl.old  C1IOO_WFS_MASTER_BAK.adl
C1IOO_MC_ALIGN.adl~       C1IOO_MC_LOCKINS.adl         C1IOO_WFS2_I.adl             C1IOO_WFS_OUTMATRIX.adl
C1IOO_MC_ALIGN_BAK.adl    C1IOO_MC_SERVO.adl           C1IOO_WFS2_Q.adl             C1IOO_WFS_QPD.adl
C1IOO_MC_ASS.adl          C1IOO_MC_TRANS_QPD.adl       C1IOO_WFS2_SETTINGS.adl      C1IOO_WFS_QPD.adl.old
C1IOO_MC_ASS_LOCKIN1.adl  C1IOO_Mech_Shutters.adl      C1IOO_WFS2_SETTINGS.adl.old  fmX
C1IOO_MC_ASS_LOCKIN2.adl  C1IOO_MODECLEANER.adl        C1IOO_WFS_HEADS.adl          junk
C1IOO_MC_ASS_LOCKIN3.adl  C1IOO_QPDS.adl               C1IOO_WFS_HEADS.adl.old      master
C1IOO_MC_ASS_LOCKIN4.adl  C1IOO_QPDS_BAK.adl           C1IOO_WFS_INMATRIX.adl       svn-commit.tmp~
controls@rosalba:/opt/rtcds/caltech/c1/medm/c1ioo/master/master/master 0$ cd master
controls@rosalba:/opt/rtcds/caltech/c1/medm/c1ioo/master/master/master/master 0$ cd master
controls@rosalba:/opt/rtcds/caltech/c1/medm/c1ioo/master/master/master/master/master 0$ cd master
controls@rosalba:/opt/rtcds/caltech/c1/medm/c1ioo/master/master/master/master/master/master 0$ cd master
controls@rosalba:/opt/rtcds/caltech/c1/medm/c1ioo/master/master/master/master/master/master/master 0$ cd master
controls@rosalba:/opt/rtcds/caltech/c1/medm/c1ioo/master/master/master/master/master/master/master/master 0$ cd master
controls@rosalba:/opt/rtcds/caltech/c1/medm/c1ioo/master/master/master/master/master/master/master/master/master 0$ cd master
controls@rosalba:/opt/rtcds/caltech/c1/medm/c1ioo/master/master/master/master/master/master/master/master/master/master 0$ cd master
controls@rosalba:/opt/rtcds/caltech/c1/medm/c1ioo/master/master/master/master/master/master/master/master/master/master/master 0$ cd master
controls@rosalba:/opt/rtcds/caltech/c1/medm/c1ioo/master/master/master/master/master/master/master/master/master/master/master/master 0$ cd master
controls@rosalba:/opt/rtcds/caltech/c1/medm/c1ioo/master/master/master/master/master/master/master/master/master/master/master/master/master 0$ cd master
controls@rosalba:/opt/rtcds/caltech/c1/medm/c1ioo/master/master/master/master/master/master/master/master/master/master/master/master/master/master 0$ cd master
controls@rosalba:/opt/rtcds/caltech/c1/medm/c1ioo/master/master/master/master/master/master/master/master/master/master/master/master/master/master/master 0$ cd master
controls@rosalba:/opt/rtcds/caltech/c1/medm/c1ioo/master/master/master/master/master/master/master/master/master/master/master/master/master/master/master/master 0$ cd master
controls@rosalba:/opt/rtcds/caltech/c1/medm/c1ioo/master/master/master/master/master/master/master/master/master/master/master/master/master/master/master/master/master 0$ cd master
controls@rosalba:/opt/rtcds/caltech/c1/medm/c1ioo/master/master/master/master/master/master/master/master/master/master/master/master/master/master/master/master/master/master 0$ cd master
controls@rosalba:/opt/rtcds/caltech/c1/medm/c1ioo/master/master/master/master/master/master/master/master/master/master/master/master/master/master/master/master/master/master/master 0$ cd master
controls@rosalba:/opt/rtcds/caltech/c1/medm/c1ioo/master/master/master/master/master/master/master/master/master/master/master/master/master/master/master/master/master/master/master/master 0$ cd master
controls@rosalba:/opt/rtcds/userapps/trunk/isc/c1/medm/c1ioo 0$ cd master
controls@rosalba:/opt/rtcds/userapps/trunk/isc/c1/medm/c1ioo/master 0$ cd master
controls@rosalba:/opt/rtcds/userapps/trunk/isc/c1/medm/c1ioo/master/master 0$ cd master
controls@rosalba:/opt/rtcds/userapps/trunk/isc/c1/medm/c1ioo/master/master/master 0$ cd master
controls@rosalba:/opt/rtcds/userapps/trunk/isc/c1/medm/c1ioo/master/master/master/master 0$ helppp
helppp: command not found
controls@rosalba:/opt/rtcds/userapps/trunk/isc/c1/medm/c1ioo/master/master/master/master 127$ help me
bash: help: no help topics match `me'.  Try `help help' or `man -k me' or `info me'.

its an acquired taste, but its a must since we're sending an interferometer to India

I *finally* figured out how to bend Matlab to my will, and close a very simple oplev loop using LQR technology. 

This is super, super simple, but it's a step in the right direction.  There is no noise, just a simple pendulum with a resonant frequency of 0.75Hz, and a Q of 10.  The LQR tries to keep the position of the pendulum at a minimum, and does not care at all about the velocity.  You cannot (with Matlab's LQR, at least that I can find) make it care "0" about the control output, so it cares about the control output a factor of 1e-4 as much as the position.

Here are some plots:

The first plot has the open loop system (just the pendulum, no control at all), as well as the closed loop system (the pendulum under control).

Plot 2 is the open loop gain of the controller that the LQR designed.

Plots 3 and 4 are the step and impulse responses of the open loop (pendulum with no control), and closed loop (pendulum with feedback) systems.

The conclusion from the plots (if this were an interesting system) is that it doesn't take much to damp an ideal pendulum.  The real conclusion here is that I think I now know how to use the Matlab LQR function, and can make a loop with some noise now.

From Linda and Bram:

 I hereby confess to having a secret script.  But it is secret no longer!

It's a "goLock" script, and it is now in the path from any terminal.  It kills any open medm sessions (to clean up desktops), and then opens a palette of screens that I find useful.  It also starts up the CARM and DARM ALS watch scripts, and the toggle shutter scripts.  It then leaves the terminal in .../scripts/PRFPMI/ , which is where the carm_cm_up.sh script that we've been using lives.

I also made tonight a "goHome" script, but all that one does so far is set the LSC mode to OFF.  The other thing that this could / should do is restore all optics so we don't have hysteresis problems.

Also, also, my "new" misalign / restore scripts had a bug, in that they were always switching oplevs for the PRM, no matter what optic was requested.  This sometimes caused the PRM oplev to be engaged while the optic was misaligned, so the PRM would get rung up.  This has been fixed.

I show here the matrix formalism to calculate analytically the loop TF relationships for the IMC w/ both FSS actuators so that it would be easier to interperet the results.

The attached PDF shows the Mathematica notebook and the associated block diagram.

In the notebook, I have written the single hop connection gains into the K matrix. P is the optical plant, C is the Common electronic gain, F is the 'fast' NPRO PZT path, and M is the phase Modulator.

G is the closed loop gain matrix. The notation is similar to matlab SS systems; the first index is the row and the second index is the column. If you want to find the TF from node 2 to node 3, you would ask for G[[3,2]].

As examples, I've shown how to get the FAST gain TF that I recently made with the Koji filter box as well as the usual OLG measurement that we make from the MC servo board front panel.

Welcome to your new abode, Donatella!

SP table has been a mess because Q and I had let our SURF leave without cleaning up.

I cleaned up the SP table, put things back where they belong and did some sorting. I will put back the optomechanics where they belong sometime later.

For now, check out the SP table next time you are looking for a Y1  or lens or BS.

Following this entry, I have made the same change in the controls account on rossa:

In the ~/.grace/gracerc file (create one if it doesn't exist), put in a line which reads:

PAGE LAYOUT FREE

Now we can scale our dataviewer live and playback plots by stretching the window with our mouse. The attached screenshot shows how I filled up one of the vertical monitors with a DV window for arm locking.

Just in case there was some confusion, the champagne on my desk is not to be opened before I get back, no matter how many signals are transitioned to RF.

Last supper before departing at  "Grazie" El Portal. All the best on your journey!

   Beautifully Done

   Chirp

  what is next?

Atm 3, Ron Drever could not celebrate with us because of health issues.

 Celebration

I say just fix the clipping. Don't worry about the PRM OSEM filters. We can do that next time when we put in the ITM baffles. No need for them on this round.

In this video: https://youtu.be/iphcyNWFD10, the comments focus on the orange crocs, my wrinkled shirt, and the first aid kit.

Very, very cool!  

Les Guthman interviews  gradstudent Graig.

Main laser emergency shut off was acuated by accident during this fiming. The laser is turned on.

I have recommissioned the Zojirushi coffee pot lid. You may, once again, align the dots in order to make the carafe pourable.

Details:

The Zojirushi lid is a two part mechanism:

1. The top part of the lid must be removed for cleaning.
2. When replacing the lid the two components must be aligned to < 3 mrad precision so that the "teeth" are able to land in the groove.
3. There is a 4-fold degeneracy in this process. To break the degeneracy, align the dot on top with the spout gap (visible from the bottom view).
4. After proper alignment and mating, the two parts should snap together and the relative alignment wiggle available should be < 2 mrad.
5. After screwing the two-piece lid onto the carafe, ensure that the 2 dots are separated by < 170 deg in the closed position.

I 've just found this time capsule note from Nov. 26, 2000 by Kip Thorne:  LIGO will discover gravitational waves by Dec.31, 2007

     It took at least ten years to rust away.

We have no coffee machine.

We are dreaming about it

We still do not have it.

New all organic machine.

IFOcad model/video of the AEI 10m interferometer:

https://10m.aei.mpg.de/design-and-sensitivity/

I have received the new screwdriver bits that will work with the two electric screwdrivers we have. I have distributed them in the 40m. Some are in the electronics stations and some are in the toolbox in the lab. The new electric screwdriver (which looks like a drill but takes typical screwdriver bits) is in the room with the workshop. It is in the blue Makita box. For some reason, lots of the old bits were rounded because of incorrect use. I have thrown the unuseable ones out.

I also requested some screw extractors in case we need them. the one we have now is really big and may not work on smaller screws.

There are brand new empty plastic containers located inside the shed that is outside in the cage. These can be used for organizing new equipment for CDS, or cleanup after Wednesday meetings

I've started putting together a list of things we'll need to buy to do BHD readout. I'm still messing around with more detailed optics layouts, but wanted to get a list started here so people can let me know if I'm missing any big, obvious categories of goods.

My current plan makes minimal changes to the signal path going to the OMC, and tries to just get the LO beam into the OMC with minimal optics. I'm not thinking of any of the optics as suspended, and it requires several reflections of the LO beam, so probably this is not an excellent configuration, but it's a start for getting the parts list:

1. My current thought is to use the MC reflection, the beam that heads from MC1 to MCR1, as the LO beam
   1. From MCR1, send the LO to a BS that directs it into an MMT, oriented along x (and lets us keep the MC refl PO)
   2. After the two MMT optics, the beam will be traveling along -x, and can be directed to a mirror that sends it towards -y to two steering mirrors that send it along -x then +x near the top of the table (perpendicular to the signal MMT.
   3. Then, send it to a PBS, which replaces the mirror directly after the signal MMT. This is where it combines 
2. Beam is then sent to the steering mirrors to bring it into the OMC
3. In parallel, the signal beam is going through the same path it has now, including the pickoff beam, with the one change that we need a HWP somewhere before the PBS, and the PBS replaces the mirror directly after the MMT (and needs to move a bit closer to have the beam properly directed)

I started making a layout of this scheme, but it's probably not going to work so I'm going to make a quick layout of this more major modification instead:

1. Both the MCR beam and the AS beam come in about parallel. Send each to a PO mirror.
2. The PO mirror directs both beams into parallel MMT aligned along x
3. From the MMT, each is directed to a pair of steering mirrors located where the OMC MMT is located now
4. From the steering mirrors go to the PBS that combines the signal and LO
5. Then to two more steering mirrors to get into the OMC, which may be moved towards +x
6. From the OMC go to the BHD PBS

What we need

Optics

• HWP for just before the LO combines with the signal
• HWP for just before the signal combines with the LO (is this necessary?)
• PBS to replace OM5 (combines the LO and the signal)
• Two MMT optics
• Two piezo-driven TT optics for steering the LO to the PBS
• One additional non-piezo optic for between the LOMMT and the LO-TTs
• One additional BS to get the LO into the MMT (and to let us still have the PO)
• -1 optic—we pick up one mirror that we replace with the PBS

Optomechanics

• 2x HWP mounts
• 1x PBS mount
• 2x mounts for piezo-driven TT
• 2x MMT optic mounts—I didn’t take a close enough look at these during the vent to know what we need here
• 2x mounts for ordinary optics
• 9x clamps for optics mounts (maybe fewer if some are on blocks)
• 9x posts for optics mounts

Electronics

• Additional TT driver
• HV supply for the new TTs
• Are the HWP actively controlled? We might need something to drive those?
• Do we have enough DAC/ADC channels?

Questions

These are mostly just miscellaneous

1. What of these optics need to be suspended? If we need suspensions on all of the LO optics, including the MMT, I’m not sure we’re going to be able to fit all of this on the table as I envision it…..
2. What if anything can we put out of vacuum (HWP for example)?
3. Do we actually need two MMT?

Can we use the leakage beam from MMT2 on the OMC table as the LO beam? I can't find the spec for this optic, but the leakage beam was clearly visible on an IR card even with the IMC locked with 100 mW input power so presumably there's enough light there, and this is a cavity transmission beam which presumably has some HOM content filtered out.

That seems fine, I wasn't thinking of that beam. in that case could we just have a PBS directly behind MMT2 and send both beams to the same OMMT?

Alternatively we can move OM5 and the beam path OMPO-OMMTSM towards -y, then put the LO-OMMT parallel to the existing OMMT but displaced in +x... we'd have to move the existing OMC and BHD towards +x as well. 

I've attached the diagram of what I mean.

There are a couple caveats and changes that would have to be made that are not included in this diagram, because they would be made on different tables.

1. I moved MMT2, which means the other MMT optics would have to be adjusted to accomodate this. I checked out the configuration on the BS table and this seems doable with a small rotation of MMT1 and maybe PJ2.
2. I don't know the best way to get the OMC REFL beam out... thoughts?
3. This diagram is kind of crappy after my edits, someone should tell me how to avoid collapsing all layers when I open these layout diagrams in inkscape, because I ended up editing the layout in Acrobat instead, where the lack of object grouping caused a bunch of the optics to lose one or two lines along the way.
4. I didn't include all beam paths explicitly but can if this looks like a good configuration.
5. The optic that picks off the transmission through MMT2 will need to move a bit, but there was a clamp in the way; this should be a minor change.
6. The optic just before the OMC needs to be moved up a bit.
7. The optic after the signal OMMT should be changed to a PBS and translated a bit; this is where the LO and signal beams will combine

Gautam also had some questions about the BHD/OMC timeline and plan. I feel somewhat on shaky ground with the answers, but figured I'd post them so I can be corrected once and for all.

1. Is the plan really to use the current OMC setup to make a homodyne measurement? 
   1. ​I'm not sure where on the timeline the new OMC and BHD switchover are relative to each other. I have been imagining doing the swap to BHD before having a new OMC.
2. I thought the current OMC resurrection plan was to do DC readout and not homodyne?
   1. ​I think the OMC resurrection plan is leading to DC readout, but when we switch over to BHD we'll be able to operate at the dark fringe. Is that right?
3. Is it really possible to use our single OMC to clean both the LO and dark port beams? Isn't this the whole raging debate for A+?
   1. ​My understanding is yes, with the LO and DP in orthogonal polarizations. It's not clear to me why we expect to be able to do this while there is a debate for A+, perhaps our requirements are weaker. It is something I should calculate, I'll talk to Koji.
4. A layout diagram would be really useful.
   1. ​Attached now.
5. Where in the priority list does this come in?
   1. ​I am a leaf in the wind. I think this comes well after we have the OMC resurrected, we just want to get a sense for what parts we need so we can order them before the fiscal year closes.

Rana, Aaron, Gautam

The old Zojirushi has died. We have received and comissioned our new Technivoorm Mocha Master today. It is good.

Below is an inventory of the signal feedthroughs that need to be installed on the vacuum Acromag crate this week.

**The original documentation lists five satellite boxes (one for each test mass chamber and one for the beamsplitter chamber), but Chub reports not all of them are in use. We may remove the ones not used.

Based on new input from Chub, attached is the revised list of signal cable feedthroughs needed on the vacuum system Acromag crate. I believe this list is now complete.

There turned out to be a few analog signals for the vacuum system after all. The TP2/3 foreline pressure gauges were never part of the digital system, but we wanted to add them, as some of the interlock conditions should be predicated on their readings. Each gauge connects to an old Granville-Phillips 375 controller which only has an analog output. Interfacing these signals with the new system required installing an Acromag XT1221 8-channel A/D unit. Taking advantage of the extra channels, I also moved the N2 delivery line pressure transducer to the XT1221, eliminating the need for its separate Omega DPiS32 controller. When the new high-pressure transducers are added to the two N2 tanks, their signals can also be connected.

The XT1221 is mounted on the DIN rail inside the chassis and I have wired a DB-9 feedthrough for each of its three input signals. It is assigned the IP 192.168.114.27 on the vacuum subnet. Testing the channels in situ revealed a subtley in calibrating them to physical units. It was first encountered by Johannes in a series of older posts, but I repeat it here in one place.

An analog-input EPICS channel can be calibrated from raw ADC counts to physical units (e.g., sensor voltage) in two ways:

1. Via LINR="LINEAR" by setting the engineering-units fields EGUF="[V_max_adc]", EGUL="[V_min_adc]"
2. Via LINR="NO CONVERSION" by manually setting the gain ASLO="[V/count]" and offset AOFF="[V_offset]"

From the documentation, under the engineering-units method EPICS internally computes:

where EGUF="eng units full scale", EGUL="eng units low", and "full scale A/D counts" is the full range of ADC counts. EPICS automatically infers the range of ADC counts based on the data type returned by the ADC. For a 16-bit ADC like the XT1221, this number is 2^16 = 65,536.

The problem is that, for unknown reasons, the XT1221 rescales its values post-digitization to lie within the range +/-30,000 counts. This corresponds to an actual "full scale A/D counts" = 60,001. If a multiplicative correction factor of 65,536/60,000 is absorbed into the values of EGUF and EGUL, then the first term in the above summation can be corrected. However, the second term (the offset) has no dependence on "full scale A/D counts" and should NOT absorb a correction factor. Thus adjusting the EGUF and EGUL values from, e.g., 10V to 10.92V is only correct when EGUL=0V. Otherwise there is a bias introduced from the offset term also being rescaled.

The generally correct way to handle this correction is to use the manual "NO CONVERSION" method. It constructs calibrated values by simply applying a specified gain and offset to the raw ADC counts:

calibrated val = (measured A/D counts)  x ASLO + AOFF

The gain ASLO="[(V_max_adc - V_min_adc) / 60,001]" and the offset AOFF="0". I have tested this on the three vacuum channels and confirmed it works. Note that if the XT1221 input voltage range is restricted from its widest +/-10V setting, the number of counts is not necessarily 60,001. Page 42 of the manual gives the correct counts for each voltage setting.

[Jon, Chub, Koji, Gautam]

Summary

Today we carried out the first pumpdown with the new vacuum controls system in place. It performed well. The only problem encountered was with software interlocks spuriously closing valves as the Pirani gauges crossed 1E-4 torr. At that point their readback changes from a number to "L OE-04, " which the system interpreted as a gauge failure instead of "<1E-4." This posed no danger and was fixed on the spot. The main volume was pumped to ~10 torr using roughing pumps 1 and 3. We were limited only by time, as we didn't get started pumping the main volume until after 1pm. The three turbo pumps were also run and tested in parallel, but were isolated to the pumpspool volume. At the end of the day, we closed every pneumatic valve and shut down all five pumps. The main volume is sealed off at ~10 torr, and the pumpspool volume is at ~1e-6 torr. We are leaving the system parked in this state for the holidays. 

Main Volume Pumpdown Procedure

In pumping down the main volume, we carried out the following procedure.

1. Initially: All valves closed (including manual valves RV1 and VV1); all pumps OFF.
2. Manually connected roughing pump line to pumpspool via KF joint.
3. Turned ON RP1 and RP2.
4. Waited until roughing pump line pressure (PRP) < 0.5 torr.
5. Opened V3.
6. Waited until roughing pump line pressure (PRP) < 0.5 torr.
7. Manually opened RV1 throttling valve to main volume until pumpdown rate reached ~3 torr/min (~3 hours on roughing pumps).
8. Waited until main volume pressure (P1a/P1b) < 0.5 torr.

We didn't quite reach the end of step 8 by the time we had to stop. The next step would be to valve out the roughing pumps and to valve in the turbo pumps.

Hardware & Channel Assignments

All of the new hardware is now permanently installed in the vacuum rack. This includes the SuperMicro rack server (c1vac), the IOLAN serial device server, a vacuum subnet switch, and the Acromag chassis. Every valve/pump signal cable that formerly connected to the VME bus through terminal blocks has been refitted with a D-sub connector and screwed directly onto feedthroughs on the Acromag chassis.

The attached pdf contains the master list of assigned Acromag channels and their wiring.

The base Acromag vacuum system is running and performing nicely. Here is a list of remaining questions and to-do items we still need to address.

Safety Issues

• Interlock for HV supplies. The vac system hosts a binary EPICS channel that is the interlock signal for the in-vacuum HV supplies. The channel value is OFF when the main volume pressure is in the arcing range, 3 mtorr - 500 torr, and ON otherwise. Is there something outside the vacuum system monitoring this channel and toggling the HV supplies?
• Exposed 30-amp supply terminals. The 30-amp output terminals on the back of the Sorensen in the vac rack are exposed. We need a cover for those.
• Interlock for AC power loss. The current vac system is protected only from transient power glitches, not an extended loss. The digital system should sense an outage and put the IFO into a safe state (pumps spun down and critical valves closed) before the UPS battery is fully drained. However, it presently has no way of sensing when power has been lost---the system just continues running normally on UPS power until the battery dies, at which point there is a sudden, uncontrolled shutdown. Is it possible for the digital system to communicate directly with the UPS to poll its activation state?

Infrastructure Improvements

• Install the new N2 tank regulator and high-pressure transducers (we have the parts; on desk across from electronics bench). Run the transducer signal wires to the Acromag chassis in the vacuum rack.
• Replace the kludged connectors to the Hornet and SuperBee serial outputs with permanent ones (we need to order the parts).
• Wire the position indicator readback on the manual TP1 valve to the Acromag chassis.
• Add cable tension relief to the back of the vac rack.
• Add the TP1 analog readback signals (rotation speed and current) to the digital system.  Digital temperature, current, voltage, and rotation speed signals have already been added for TP2 and TP3.
• Set up a local vacuum controls terminal on the desk by the vac rack.
• Remove gauges from the EPICS database/MEDM screens that are no longer installed or functional. Potential candidates for removal: PAN, PTP1, IG1, CC2, CC3, CC4.
• Although it appeared on the MEDM screen, the RGA was never interfaced to the old vac system. Should it be connected to c1vac now?

Everything is set for a second pumpdown tomorrow. We'll plan to start pumping after the 1pm meeting. Since the main volume is already at 12 torr, the roughing phase won't take nearly as long this time.

I've added new channels for the TP1 analog readings (current and speed) and for the two N2 tank pressure readings. Chub finished installing the new regulator and has run the transducer signal cable to the vacuum rack. In the morning he will terminate the cable and make the final connection to the Acromag.

Gautam and I updated the framebuilder config file, adding the newly-added channels to the list of those to be logged. We also set up a git repo containing all of the Python interlock/interfacing code: https://git.ligo.org/40m/vacpython. The idea is to use the issue tracker to systematically document any changes to the interlock code.

Updated vac channel list is attached. There are several new ADC channels.

Now that the Acromag upgrade of c1vac is complete, the next system to be upgraded will be c1susaux. We chose c1susaux because it is one of the highest-priority systems awaiting upgrade, and because Johannes has already partially assembled its Acromag replacement (see photos below). I've assessed the partially-assembled Acromag chassis and the mostly-set-up host computer and propose we do the following to complete the system.

Documentation

As I go, I'm writing step-by-step documentation here so that others can follow this procedure for future systems. The goal is to create a standard procedure that can be followed for all the remaining upgrades.

Acromag Chassis Status

The bulk of the remaining work is the wiring and testing of the rackmount chassis housing the Acromag units. This system consists of 17 units: 10 ADCs, 4 DACs, and 3 digitial I/O modules. Johannes has already created a full list of channel wiring assignments. He has installed DB37-to-breakout board feedthroughs for all the signal cable connections. It looks like about 40% of the wiring from the breakout boards to Acromag terminals is already done.

The Acromag units have to be initially configured using the Windows laptop connected by USB. Last week I wasn't immediately able to check their configuration because I couldn't power on the units. Although the DC power wiring is complete, when I connected a 24V power supply to the chassis connector and flipped on the switch, the voltage dropped to ~10V irrespective of adjusting the current limit. The 24V indicator lights on the chassis front and back illuminated dimly, but the Acromag lights did not turn on. I suspect there is a short to ground somewhere, but I didn't have time to investigate further. I'll check again this week unless someone else looks at it first.

Host Computer Status

The host computer has already been mostly configured by Johannes. So far I've only set up IP forwarding rules between the martian-facing and Acromag-facing ethernet interfaces (the Acromags are on a subnet inaccessible from the outside). This is documented in the link above. I also plan to set up local installations of modbus and EPICS, as explained below. The new EPICS command file (launches the IOC) and database files (define the channels) have already been created by Johannes. I think all that remains is to set up the IOC as a persistent system service.

Host computer OS

Recommendation from Keith Thorne:

For CDS lab-wide, Jamie Rollins and Ryan Blair have been maintaining Debian 8 and 9 repos with some of these.  
They have somewhat older EPICS versions and may not include all the modules we have for SL7.
One worry is whether they will keep up Debian 9 maintained, as Debian 10 is already out.

I would likely choose Debian 9 instead of Ubuntu 18.04.02, as not sure of Ubuntu repos for EPICS libraries.

Based on this, I propose we use Debian 9 for our Acromag systems. I don't see a strong reason to switch to SL7, especially since c1vac and c1susaux are already set-up using Debian 8. Although Debian 8 is one version out of date, I think it's better to get a well-documented and tested procedure in place before we upgrade the working c1vac and c1susaux computers. When we start building the next system, let's install Debian 9 (or 10, if it's available), get it working with EPICS/modbus, then loop back to c1vac and c1susaux for the OS upgrade.

Local vs. central modbus/EPICS installation

The current convention is for all machines to share a common installation which is hosted on the /cvs/cds network drive. This seems appealing because only a single central EPICS distribution needs to be maintained. However, from experience attempting this on c1vac, I'm convinced this is a bad design for the new Acromag systems.

The problem is that any network outage, even routine maintenance or brief glitches, wreaks havoc on Acromags set up this way. When the network is interrupted, the modbus executable disappears mid-execution, crashing the process and hanging the OS (I think related to the deadlocked NFS mount), so that the only way to recover is to manually power-cycle. Still worse, this can happen silently (channel values freeze), meaning that, e.g., watchdog protections might fail.

To avoid this, I'm planning to install a local EPICS distribution from source on c1susaux, just as I did for c1vac. This only takes a few minutes to do, and I will include the steps in the documented procedure. Building from source also better protects against OS-dependent buginess.

Main TODO items

• Debug issue with Acromag DC power wiring
• Complete wiring from chassis feedthroughs to Acromag terminals, following this wiring diagram
• Check/set the configuration of each Acromag unit using the software on the Windows laptop
• Set the analog channel calibrations in the EPICS database file
• Test each channel ex situ. Chub and I discussed an idea to use two DB-37F breakout boards, with the wiring between the board terminals manually set. One DAC channel would be calibrated and driven to test other ADC channels. A similar approach could be used for the digital input/output channels.

Just a few remarks, since I heard from Gautam that c1susaux is next in line for upgrade.

All units have already been configured with IP addresses and settings following the scheme explained on the slow controls wiki page. I did this while powering the units in the chassis, so I'm not sure where the short is coming from. Is the power supply maybe not sourcing enough current? Powering all units at the same time takes significant current, something like >1.5 Amps if I remember correctly. These are the IPs I assigned before I left:

I used black/white twisted-pair wires for A/D, red/white for D/A, and green/white for BIO channels. I found it easiest to remove the blue terminal blocks from the Acromag units for doing the majority of the wiring, but wasn't able to finish it. I had also done the analog channel calibrations using the windows untility using multimeters and one of the precision voltage sources I had brought over from the Bridge labs, but it's probably a good idea to check it and correct if necessary. I also recommend to check that the existing wiring particularly for MC1 and MC2 is correct, as I had swapped their order in the channel assignment in the past.

While looking through the database files I noticed two glaring mistakes which I fixed:

1. The definition of C1SUSAUX_BIO2 was missing in /cvs/cds/caltech/target/c1susaux2/C1SUSAUX.cmd. I added it after the assignments for C1SUSAUX_BIO1
2. Due to copy/paste the database files /cvs/cds/caltech/target/c1susaux2/C1_SUS-AUX_<OPTIC>.db files were still pointing to C1AUXEX. I overwrote all instances of this in all database files with C1SUSAUX.

Thanks to new info from Johannes, I was able to finish setting up the modbus IOC on c1susaux2. It turns out the 17 Acromags draw ~1.9 A, which is way more than I had expected. Hence the reason I had suspected a short. Adding a second DC supply in parallel solves the problem. There is no issue with the wiring.

With the Acromags powered on, I carried out the following:

• Confirmed c1susaux2 can communicate with each Acromag at its assigned IP address
• Modified the EPICS .cmd file to point to the local modbus installation (not the remote executable on /cvs/cds)
• Debugged several IOC initialization errors. All were caused by minor typos in the database files.
• Scripted the modbus IOC to launch as a systemd service (will add implementation details to the documentation page)

The modbusIOC is now running as a peristent system service, which is automatically launched on boot and relaunched after a crash. I'm able to access a random selection of channels using caget.

What's left now is to finish the Acromag-to-feedthrough wiring, then test/calibrate each channel.

I found the current bias output channels, C1:SUS-<OPTIC>_<DOF>BiasAdj, were all pointed at C1:SUS-<OPTIC>_ULBiasSet for every degree of freedom. This same issue appeared in all eight database files (one per optic), so it looks like a copy-and-paste error. I fixed them to all reference the correct degree of freedom.

For future reference:

• The updated list of c1susaux channel wiring (includes the "coil enable" --> "coil test" digital outputs change)
• Step-by-step instructions on how to set up an Acromag system from scratch

22:05:02 UTC Jordan refilled his water bottle at the water dispenser in the control room.