I did just this, and it looks okay to me:

 I tried shifting the notch frequencies on the D000186-revision D board given to me by Koji. The existing notches were at ~16 kHz and ~32 kHz. I shifted these to notches at ~64 kHz and ~128 kHz by effecting the following changes (see schematic for component numbering) on Channel 8 of the board-I decided to check things out on one channel before implementing changes en masse:

• R6 and R7 replaced with 511 ohm smts
• R8 replaced with 255 ohm smt
• R14 and R15 replaced with 549 ohm smts
• R16 replaced with 274 ohm smt

=> New notches should be at 66.3 kHz and 131.7 kHz.

I then measured the frequency response of the modified channel using the SR785, and compared it to the response I had measured before switching out the resistors. The SR785 only goes up to 102 kHz, so I cannot verify the 128 kHz notch at this point. The position of the 64 kHz notch looks alright though. I think I will go ahead and switch out the remaining resistors in the evening.

Note 1: These plots are just raw data from the SR785, I have not tried to do any sort of fitting to poles and zeros. I will do this at some point. 

Note 2: All these smts were taken from Downs. Todd helped me locate the non-standard value resistors. I also got a plastic 25-pin D-sub backshells (the spares are in the rack), with which I have fashioned the required custom ribbon cables (40 pin IDC to 25 pin D-sub with twisted ribbon wire, and a short, 10pin IDC to 10pin IDC with straight ribbon wire).

 [Eric, Alex]

We added our reference photodetector (Newport 1611, REF DET) to the southern edge of the AS table, as pictured. The detector's power supply is located under the southwest corner of the table, as pictured. We have connected the detector to its power supply, and will connect the detector's fiber input and RF output tomorrow.

EDIT: this is about the RFPD frequency response setup...

[Eric, Alex]

For the RFPD frequency response project, we routed the fiber that will connect our REF DET (on the AS table) to our 1x16 optical splitter (in the OMC_North rack), as pictured. (The new fiber is the main one in the picture, which ends at the right edge near REF DET) Note that we secured the fiber to the table in two places to ensure the fiber would remain immobile and out of other optical paths already in place.

At 2:00 we plan to run fiber from our laser module (in rack 1Y1) to our 1x16 optical splitter (in the OMC_North rack) and measure the power output at one of the splitter's output ports. We plan to keep the output power limited to less than 0.5 mW per optical splitter output.

[Eric, Alex]

We decided that the POY Table would be a better home for our REF DET (Newport 1611 FC-AC) than the AS Table. We moved the PD to the POY Table (1st attachment) and routed a fiber from our 1x16 Optical Splitter in the OMC_North rack to the POY Table. REF DET's power supply is now located under the POY table (2nd attachment). We left the fiber described in the previous post on the AS Table.

Afterwards, we hooked a fiber up to our laser module to test it (3rd attachment). The laser was not being distributed, just going to one fiber with a power meter at its end. Everything turns out, but we realized we need to read the power supply's manual before continuing. 

Summary 

I have been working on setting up a QPD which can eventually be used to calibrate the PZT, and also orient the PZT in the mount such that the pitch and yaw axes roughly coincide with the vertical and horizontal.

The calibration constants have been determined to be:

X-axis: -3.69 V/mm

Y-axis: -3.70V/mm

Methodology:

I initially tried using the QPD setup left behind by Chloe near MC2, but this turned out to be dysfunctional. On opening out the QPD, I found that the internal circuitry had some issues (shorts in the wrong places etc.) Fortunately, Steve was able to hand me another working unit. For future reference, there are a bunch of old QPDs which I assume are functional in the cabinet marked 'Old PDs' along the Y-arm. 

I then made a circuit with which to read out the X and Y coordinates from the QPD. This consists of 4 buffer amplifiers (one for each quadrant), and 3 summing amplifiers (outputs are A+B+C+D = sum, B+C-A-D = Y-coordinate, and A+B-C-D = X-coordinate) that take the appropriate linear combinations of the 4 quadrants to output a voltage that may be calibrated against displacement of the QPD. 

The output from the QPD is via a sub-D connector on the side of the pomona box enclosing the PD and the circuitry, with 7 pins- 3 for power lines, and 4 for the 4 quadrants of the QPD. It was a little tricky to figure the pin-out for this connector, as there was no way to use continuity checking to map the pins to quadrants. Therefore, I used a laser pointer, and some trial and error (i.e. shine the light on a given quadrant, and check the sign of the X and Y voltages on an oscilloscope) to map the pin outs. Steve tells me that these QPDs were made long before colour code standardisation, but I note here the pin outs in any case for future reference (the quadrant orientations are w.r.t the QPD held with all the circuitry above it, with the active surface facing me):

Red= +Vcc

Black= -Vcc

Green = GND

Blue = Upper Left Quadrant

White = Upper Right Quadrant

Purple = Lower Left Quadrant

Grey = Lower Right Quadrant

Chloe had noted that there was some issue with the voltage regulators on her circuit (overheating) but I suspect this may have been due to the faulty internal circuitry. Also, she had used 12 V regulators. I checked the datasheet of the QPD, Op-Amp LF347 (inside the pomona box) and the OP27s on my circuit, and found that they all had absolute maximum ratings above 18V, so I used 15V voltage regulators. The overheating problem was not a problem anymore.

I then proceeded to arrange a set up for the calibration (initially on the optical bench next to MC2, but now relocated to the SP table, and a cart adjacent to it). It consists of the following:

• He-Ne laser source
• Y2 2-inch mirror (AR and HR coated for 532nm) glued onto the PZT and mounted on a machined Newport U100P  - see this elog for details.
• QPD mounted on a translational stage whose micrometers are calibrated in tenths of an inch (in the plots I have scaled this to mm)
• A neutral density filter (ND = 2.0) which I added so that the QPD amplifier output did not saturate. I considered using a lens as well to reduce the spot size on the QPD but found that after adding the ND filter, it was reasonably small.
• High-voltage power supply (on cart)
• Two SR power supplies (for the PZT driver board and my QPD amplifier
• SR function generator
• Laser power source
• Two oscilloscopes
• Breadboard holding my QPD amplifier circuit

Having set everything up and having done the coarse alignment using the mirror mount, I proceeded to calibrate the X and Y axes of the QPD using the translational stage. The steps I followed were:

• Centre spot on QPD using coarse adjustment on the mirror mount: I gauged this by monitoring the X and Y voltage outputs on an oscilloscope, and adjusted things till both these went to zero.
• Used the tilt knob on the translational stage to roughly decouple the X and Y motion of the QPD.
• Kept Y-coordinate fixed, took the X-coordinate to close to its maximum value (I gauged this by checking where the voltage stopped changing appreciably for changes in the QPD position.
• Using this as a starting point, I moved the QPD through its X range, noting voltage output of the X-coordinate (and also the Y) on an oscilloscope.
• Repeated the procedure for the Y-coordinate.
• Analysis follows largely what was done in these elogs. I am attaching the script I used to fit an error function to the datapoints, this is something MATLAB should seriously include in cftool (note that it is VERY sensitive to the initial guess. I had to do quite a bit of guessing).

The plots are attached, from which the calibration values cited above are deduced. The linear fits for the orthogonal axis were done using cftool. There is some residual coupling between the X and Y motions of the QPD, but I think this os okay my purposes. 

My next step would be to first tweak the orientation of the PZT in the mount while applying a small excitation to it in order to decouple the pitch and yaw motion as best as possible. Once this is done, I can go ahead and calibrate the angular motion of the PZT in mrad/V.

                                                            X-Axis                                                                                                                                Y-axis                                                

 Yesterday, I mounted the first PZT in one of the modified mounts, and then glued a 2-inch Y2 mirror on it using superglue.

Details:

-The mirror is a 2-inch, Y2 mirror with HR and AR coatings for 532 nm light.

-The AR side of the mirror had someone's fingerprint on it, which I removed (under Manasa's guidance) using tweezers wrapped in lens cleaning paper, and methanol.

-Before gluing the mirror, I had to assemble the modified mount. Manasa handed over the remaining parts of the mounts (which are now in my newly acquired tupperware box along with all the other Piezo-related hardware). I took the one labelled A, and assembled the holder part. I then used one of the new mounts (2.5 inches, these are with the clean mounts in a cardboard box in the cupboard holding the green optics along the Y-arm) and mounted the holder on it. 

-Having assembled the mount, I inserted the piezo tip-tilt into the holder. The wedge that the machine shop supplied is useful (indeed required) for this. 

-I then cleaned the AR surface of the mirror and the top-surface of the tip-tilt. 

-The gluing was done using superglue which Steve got from the bookstore (the remaining tube is in the small fridge). We may glue the other mirror using epoxy. I placed 4 small drops of superglue on the tip-tilt's top surface, placed the mirror with its AR face in contact with the piezo, and applied some pressure for a short while until the glue spread out fairly evenly. I then left the whole setup to dry for about half an hour.

-Steve suggested using a reference piece (I used two small bolts) to verify when the glue had dried.

-Finally, I attached the whole assembly to a base.

Here it is in action in my calibration setup (note that it has not been oriented yet. i.e. the two perpendicular axes of the piezo are for the time being arbitrarily oriented. And maybe the spreading of the glue wasn't that even after all...):

Sidenote:

Yesterday, while setting stuff up, I tested the piezo with a 0.05 Hz, 10Vpp input from the SR function generator just to see if it works, and also to verify that I had set up all my electronics correctly. Though the QPD was at this point calibrated, I did observe periodic motion of both the X and Y outputs of my QPD amp! Next step- calibration... 

 I carried some further modifications and tests to the AI Board. Details and observations here:

• I switched out the resistors for all the remaining 7 channels, using the same substitutions as detailed here. 
• I then verified that the modified transfer function for all 8 channels using the SR785. I did not collect data for all the channels as netgpib was taking ages, but I did use the cursor on the screen to verify the position of the first notch at ~64 kHz. I noticed that all the channels did not have the lowest point of the notch at the same frequency. Rather, (at least on the screen), this varied between 63kHz and 67kHz. I would put this down to component tolerance. Assuming 5% tolerance shifts the theoretical notch frequency from 66268 Hz to 63112 Hz. 
• After verifying the transfer functions, I went to 1Y4 and plugged the AI board into the eurocrate. I then connected the input of the AI board to the DAC output using my custom ribbon cable. Next, I used the excitation points set up earlier to send a 1 kHz, 32000 counts amplitude sine wave through the channels one at a time. I monitored the output using an oscilloscope and the LEMO monitor channels on the front panel of the board.
• I found that the single-ended output of the AI board swings between -10 V and 10 V (w.r.t ground, oscilloscope trace attached). This is good because this is the range of input voltage to the PZT driver boards required to realize the full actuation range of the PZTs.
• I also verified that the connections on the custom ribbon cable are correct (channel map was right) and that there were no accidental shorts (I checked other channels' output monitor while driving one channel). 

I think the board is okay to be used now.

 I have managed to orient the PZT in the mount such that its axes are approximately aligned with the vertical and the horizontal. 

In the process, I discovered that the 4 screws on the back face of the PZT correspond to the location of the piezoelectric stacks beneath the tip-tilt platform. The PZT can therefore be oriented during the mounting process itself, before the mirror is glued onto the tip-tilt platform.

In order to verify that the pitch and yaw motion of the mirror have indeed been roughly decoupled, I centred the spot on the QPD, fed to the 'pitch' input of the PZT driver board (connected to channel 1 of the PZT) a 10 Vpp, 1 Hz sine wave from the SR function generator (having turned all the other relevant electronics, HV power supply etc ON. The oscilloscope trace of the output observed on the QPD is shown. The residual fluctuation in the Y-coordinate (blue trace) is I believe due to the tilt in the QPD, and also due to the fact that the PZT isnt perfectly oriented in the mount.

It looks like moving the tip-tilt through its full range of motion takes us outside the linear regime of the QPD calibration. I may have to rethink the calibration setup to keep the spot on the QPD in the linear range if the full range is to be calibrated, possibly decrease the distance between the mirror and the QPD. Also, in the current orientation, CH1 on the PZT controls YAW motion, while CH2 controls pitch.

Oscilloscope Trace:

Yellow: X-coordinate

Blue: Y-coordinate

I mounted the second PZT in a modified mount, and then glued a 1-inch Y2 mirror on it using superglue.

 Details:

-The mirror is a Laseroptik 1-inch, Y2 mirror with HR and AR coatings for 532 nm light.

-The procedure for mounting the mirror was the same as detailed in elog 8874. This time, I tried to orient the Piezo such that the four screws on the back face coincided with the horizontal and vertical axes, as this appeared to (somewhat) decouple the pitch and yaw motion of the tip-tilt on the first PZT.

-One thing I forgot to mention in the earlier elog: it is best to assemble the mount fully before inserting the tip-tilt into it and gluing the mirror to the tip-tilt. In particular, the stand should be screwed onto the mount before inserting the tip-tilt into the holder, as once it is in, it will block the hole through which one can screw the stand onto the mount. 

 I recalibrated the QPD today as I had shifted its position a little. I then identified the linear range of the QPD and performed a preliminary calibration of the Piezo tip-tilt within this range.

Details:

-I recalibrated the QPD as I had shifted it around a little in order to see if I could move it to a position such that I could get the full dynamic range of the piezo tilt within the linear regime of the QPD. This proved difficult because there are two reflections from the mirror (seeing as it is AR coated for 532nm and I am using a red laser). At a larger separation, these diverge and the stray spot does not bother me. But it does become a problem when I move the QPD closer to the mirror (in an effort to cut down the range in which the spot on the QPD moves). In any case, I had moved the QPD till it was practically touching the mirror, and even then, could not get the spot motion over the full range of the PZTs motion to stay within the QPD's linear regime (as verified by applying a 20Vpp 1Hz sine wave to the PZT driver board and looking at the X and Y outputs from the QPD amplifier. 

-So I reverted to a configuration in which the QPD was ~40cm away from the mirror (measured using a measuring tape).

-The new calibration constants are as follows (see attached plots):

X-Coordinate: -3.43 V/mm
Y-Coordinate: -3.41 V/mm

-I then determined the linear range of the QPD to be when the output was in the range [-0.5V 0.5V]. 

-Next, at Jenne's suggestion, I decided to do a preliminary calibration of the PZT within this linear range. I used an SR function generator to supply an input voltage to the PZT driver board's input (connected to Channel 1 of the piezo). In order to supply a DC voltage, I set a DC offset, and set the signal amplitude to 0V. I then noted the X and Y-coordinate outputs, being sure to run through the input voltages in a cyclic fashion as one would expect some hysteresis. 

-I did this for both the pitch and yaw inputs, but have only superficially analysed the latter case (I will put up results for the former later). 

Comments:

-There is indeed some hysteresis, though the tilt seems to vary linearly with the input voltage. I have not yet included a calibration constant as I wish to perform this calibration over the entire dynamic range of the PZT. 

-There is some residual coupling between the pitch and yaw motion of the tip tilt, possibly due to its imperfect orientation in the holder (I have yet to account for the QPD's tilt).

-I have not included a graphical representation here, but there is significantly more pitch to yaw coupling when my input signal is applied to the tip-tilts pitch input (Channel 2), as compared to when it is input to channel 1. It is not clear to me why this is so.

-I have to think of some smart way of calibrating the PZT over its entire range of motion, keeping the spot in the QPD's linear regime throughout. One idea is to start at one extreme (say with input voltage -10V), and then perform the calibration, re-centering the spot to 0 on the QPD each time the QPD amp output reaches the end of its linear regime. I am not sure if this will work, but it is worth a shot. The other option is to replace the red laser with a green laser (from one of the laser pointers) in the hope that multiple reflections will be avoided from the mirror. Then I will have to recalibrate the set up, and see if I can get the QPD close enough to the mirror such that the spot stays within the linear regime of the QPD. More investigation needs to be done.

Plots:

QPD Calibration Plots:
            

Piezo tilt vs input voltage plots:

                                                          Yaw Tilt                                                                                                                                         Pitch Tilt

 Given that the green beam is to be used as the reference during the vent, it was decided to first test the PZT mounted mirrors at the X-endtable rather than the Y-endtable as originally planned. Yesterday, I prepared a second PZT mounted mirror, completed the full range calibration, and with Manasa, installed the mirrors on the X-endtable as mentioned in this elog. The calibration constants have been determined to be (see attached plots for aproximate range of actuation):

M1-pitch: 0.1106 mrad/V

M1-yaw: 0.143 mrad/V

M2-pitch: 0.197 mrad/V

M2-yaw: 0.27 mrad/V

Second 2-inch mirror glued to tip-tilt and mounted:

• The spot sizes on the steering mirrors at the X-end are fairly large, and so two 2-inch steering mirrors were required.
• The mirrors already glued to the PZTs were a CVI 2-inch and a Laseroptik 1-inch mirror.
• I prepared another Laseroptik 2-inch mirror (45 degree with HR and AR coatings for 532 nm) and glued it to a PZT mounted in a modified mount as before.
• Another important point regarding mounting the PZTs: there are two perforated rings (see attached picture) that run around the PZT about 1cm below the surface on which the mirror is to be glued. The PZT has to be pushed in through the mount till these are clear of the mount, or the actuation will not be as desired. In the first CVI 2-inch mirror, this was not the the case, which probably explains the unexpectedly large pitch-yaw coupling that was observed during the calibration [Thanks Manasa for pointing this out]. 

Full range calibration of PZT:

Having prepared the two steering mirrors, I calibrated them for the full range of input voltages, to get a rough idea of whether the tilt varied linearly and also the range of actuation. 

Methodology:

• The QPD setup described in my previous elogs was used for this calibration. 
• The linear range of the QPD was gauged to be while the output voltage lay between -0.5V and 0.5V. The calibration constants are as determined during the QPD calibration, details of which are here.
• In order to keep the spot always in the linear range of the QPD, I stared with an input signal of -10V or +10V (ie. one extreme), and moved both the X and Y micrometers on the translational stage till both these coordinates were at one end of the linear range (i.e -0.5V or 0.5V). I then increased the input voltage in steps of ~1V through the full range from -10V to +10V DC. The signal was applied using a SR function generator with the signal amplitude kept to 0, and a DC offset in the range -5V to 5V DC, which gave the desired input voltages to the PZT driver board (between -10V DC and 10V DC).
• When the output of the QPD amp reached the end of the linear regime (i.e 0.5V or -0.5V), I moved the appropriate micrometer dial on the translational stage to take it to the other end of the linear range, before continuing with the measurements. The distance moved was noted. 
• Both the X and Y coordinates were noted in order to investigate pitch-yaw coupling.

Analysis and remarks:

• The results of the calibration are presented in the plots below. 
• Though the measurement technique was crude (and maybe flawed because of a possible z-displacement while moving the translational stage), the calibration was meant to be rough, and I think the results obtained are satisfactory. 
• Fitting the data linearly is only an approximation, as there is evidence of hysteresis. Also, PZTs appear to have some drift, though I have not been able to quantify this (I did observe that the output of the QPD amp shifted by an amount equal to ~0.05mm while I left the setup standing for an hour or so).  
• The range of actuation seems to be different for the two PZTs, and also for each degree of freedom, though the measured data is consistent with the minimum range given in the datasheet (3.5 mrad for input voltages in the range -20V to 120V DC). 

PZT Calibration Plots

The circles are datapoints for the degree of freedom to which the input is applied, while the 'x's are for the other degree of freedom. Different colours correspond to data measured with the position of the translational stage at some value.

                                            M1 Pitch                                                                                             M1 Yaw

                                              M2 Pitch                                                                                        M2 Yaw 

Installation of the mirrors at the X-endtable:

The calibrated mirrors were taken to the X-endtable for installation. The steering mirrors in place were swapped out for the PZT mounted pair. Manasa managed (after considerable tweaking) to mode-match the green beam to the cavity with the new steering mirror configuration. In order to fine tune the alignment, Koji moved ITMx and ETMx in pitch and yaw so as to maximise green TRX. We then got an idea of which way the input pointing had to be moved in order to maximise the green transmission.

I have updated the schematic of the D980323 PZT driver boards to reflect the changes made. The following changes were made (highlighted in red on the schematic):

• Gain of all four HV amplifier stages changed from ~15 to ~5 by swapping 158k resistors R43, R44, R69 and R70 for 51k resistors.
• Electrolytic 10 uF capacitors C11, C12, C29 and C31 swapped for 470pF, 500V mica capacitors.
• Fixed resistor in voltage divider (R35, R40, R59 and R64) replaced with 0 ohm resistors so as to be able to apply a bias of -10V to the HV amplifier
• The DC-DC Series components, which I think were originally meant to provide the 100V DC voltage, have been removed.
• The path between the point at which +100V DC is delivered and jumpers J3 and J6 has been shorted (bypassing R71 and R11 for J3, R73 and R12 for J6).
• Tantalum capacitors C38 and C39 have been replaced with electrolytic capacitors (47 uF, 25V). One of the original tantalum capacitors had burned out when I tried installing the board in the eurocrate, shorting out -15V to ground. At Koji's suggestion, I made this switch. The AD797s do not seem to be oscillating after the switch.
  
  
  

I have also changed the routing of the 100V from the HV power supply onto the board, it is now done using an SMA T-connector and two short lengths of RG58 cable with SMA connectors crimped on.

The boards are functional (output swings between 0 and 100V as verified with a multimeter for input voltages in the range -10V to +10V applied using a function generator.

Revised schematics:

 The following hardware has been installed on rack 1X9;

• KEPCO high voltage power supply (kept in a plastic box at the bottom of the rack, with the 3m SMA cable carrying 100V running along the inside side wall of the rack). The HV supply has not been connected to the driver board yet.
• AI board D000186 installed in top eurocrate. The board does not seem to fit snugly into the slot, so I used a longish screw to bolt the front panel to the eurocrate.
• PZT driver board D980323 installed in top eurocrate adjacent to the AI board.
• Six 11m SMB-LEMO cables have been laid out from 1X9 to the endtable. I have connected these to the PZT driver board, but the other end (to the PZTs) is left unconnected for now. They have been routed through the top of the rack, and along the cable tray to the endtable. All the cables have been labelled at both ends. 
  

I have also verified that the AI board is functional in the eurocrate by using the LEMO monitoring points on the front panel.

The driver boards remain to be verified, but this cannot be done until we connect the HV supply to the board. 

 [Alex, Gautam]

The signal chain from the DAC output to the output of the PZT driver board (including the HV supply) has been verified. 

I had installed the two boards in the eurocrate yesterday and laid out the cables from 1X9 to the endtable. The output of the AI board had been verified using the monitor port on the front panel, but the output from the PZT driver board was yet to be checked because I had not connected the HV supply yesterday.

When I tried this initially today, I was not getting the expected output from the monitor channels on the front panel of the PZT driver board, even though the board was verified to be working. Alex helped debug the problem, which was identified as the -15V supply voltage not making it onto the board.

I changed the slot the board was sitting in, and used a long screw to bolt the board to the crate. Both the AI board and the PZT driver board seem to be slightly odd-sized, and hence, will not work unless firmly pushed into the eurocrate and bolted down. This would be the first thing to check if a problem is detected with this system. 

In any case, I have bolted both boards to the eurocrate, and the output from the PZT driver board is as expected when I sent a 10Vp sine wave out from the DAC. I think the cables can now be hooked up to the PZTs once we are pumped down.

 I have glued a fourth mirror to a PZT (using superglue) and inserted it into a modified mount. This is to be used together with the 1-inch Laseroptik mirror I had glued a couple of weeks back at the Y-endtable. I will be calibrating both these mirrors tonight such that these are ready to put in as soon as we are pumped down.

The mirror was one of those removed from the X-endtable during the switch of the steering mirrors. It is a CVI 2-inch mirror, with HR and AR coatings for 532 nm. 

 I had prepared two more PZT mounted mirrors for the Y-end some time back. These are:

• A 2-inch CVI mirror (45 degree, HR and AR for 532nm, was originally one of the steering mirrors at the X-endtable, and was removed while switching those out for the PZT mounted mirrrors).
• A 1-inch Laseroptik mirror (45 degree, HR and AR for 532nm).

I used the same QPD set-up and the methodology described here to do a full-range calibration of these PZTs. Plots attached. The calibration constants have been determined to be:

CVI-pitch: 0.316 mrad/V

CVI-yaw:  0.4018 mrad/V

Laseroptik pitch: 0.2447 mrad/V

Laseroptik yaw:  0.2822 mrad/V

Remarks:

• These PZTs, like their X-end counterparts, showed evidence of drift and hysteresis. We just have to deal with this.
• One of the PZTs (the one on which the CVI mirror is mounted) is a used one. While testing it, I thought that its behaviour was a little anomalous, but the plots do not seem to suggest that anything is amiss.

Plots:

                                                        CVI YAW                                                                                                                         CVI PITCH

                                                        Laseroptik YAW                                                                                                             Laseroptik PITCH

   

For the photodetector frequency response project, I finished the construction of our baluns chassis and mounted it in rack 1Y1 (1st picture).

After consulting with Jenne, I mounted the fiber launcher for REFL165 on the AS table such that it would not cause an obstruction. I aligned the launcher using a multimeter to monitor the DC output of REFL165, but looking at the data I got, it seems I need to do a better alignment/focusing job to get rid of a bunch of noise.

For the photodetector frequency response project, our new RF Switch Chassis (NI pxie-1071) arrived today. I took the switches out of the old chassis (Note for future generations: you have to yank pretty darn hard) and put them in the new chassis, which I mounted in rack 1Y1 as pictured. 

The point of this new chassis is that its controller is compatible with our control room computer setup. We will be able to switch the chassis using TCP/IP or telnet, aiding in our automation of the measurement of photodetector frequency response.

 I routed cables (RG405 SMA-SMA) from several demodulator boards in rack 1Y2 to the RF Switch in rack 1Y1 using the overhead track. Our switch chassis contains two 8x1 switches. The COM of the "right" switch goes to channel 7 of the "left" switch to effectively form a 16x1 switch. The following is a table of correspondences between PD and RF Switch input.

ThePOP110 demod board has not yet had a cable routed from it to the switch because I ran out of RG405.

We should also consider how important it is to include MCREFL in our setup. Doing so would require fabrication of a ~70 ft RG405 cable. 

I have modified one of the spare demod boards that was sitting above the electronics bench (the one which was unlabeled - the others say 33MHz, 55MHz and 165MHz) to be the new AS110 demod board.  In place of the T1 coil, and the C3 and C6 resistors, I have put the commercial splitter PSCQ-2-120+.  In place of U5 (the low pass for the PD input) I have put an SCLF-135+. 

In order to figure out how to make the pinout of the PSCQ match up with the available pads from T1, I first pulled the "AS11" board (it's not something that we use, so it would be less of a tragedy if something happened while I had the board pulled).  However, while the PCB layout is the same, the splitter for the low frequencies (PSCQ-2-51W) has a different pinout than the one I need for the 110MHz.  So, I put AS11 back, and pulled the POP110 board. (After I noted the pinout on POP110, I reinstalled that board.  To get it out, I had to unplug the I and Q outs of POP22, but I have also replugged those in).

For my new AS110 demod board, I copied the pin connections on POP110.  I have made a little diagram, so you can see what pins went where.  The top 2 rectangles are the "before" installation cartoon, and the bottom is the "as installed" cartoon.

The one thing that must be noted is that, because of the pinout of the splitter and the constraints of the board layout, the +0 degrees (I-phase) output of the splitter is connected to the Q channel for the rest of the demod board.  This means that the +90 degrees (Q-phase) output of the splitter is connected to the I channel for the rest of the demod board.  This is not noted for POP110, but is true for both:  The I and Q channels of the 110 MHz demod boards are switched.  In practice, we can handle this with our digital phase rotation.

Daytime tomorrow, I will test my new board as Suresh did in elog 4736.  Before we get to use AS110, we need (a) some LO juice from the RF distribution box, and (b) a spot to plug the board in, in the LSC rack.  Meditating on how those are going to happen are also tasks for daytime tomorrow.

OK, but what kind of filter should we be actually using? i.e. what purpose the 135 MHz low pass serve in contrast to a PHP-100+ ?

 Hmmm. Indeed. This is just cutting off higher frequency stuff, but anything from other lower sidebands still gets through.  I should actually stick in the SXBP-100's, which will band pass from 87-117 MHz.  These have an insertion loss at 100 MHz of 1.64 dB. 

Jamie ordered 2 of these, so I can put one in each of AS110 and POP110. 

I measured the phase split between the I and Q signals of my AS110 board.  To do so, I plugged the board into an empty slot next to the PD DC readout / whitening board in the LSC rack.  I borrowed the POP110 local oscillator, and used a Marconi to generate a "PD input".  (I'm roughly following what Suresh did in elog 4736).  Our 11MHz is currently 11.066134MHz, so I had the Marconi going at 110.662340 MHz (1kHz from 10*11MHz), and I had the Marconi source at -13dBm.  

I took a transfer function using the SR785 between the I and Q outs of the AS110 demod board, and got a magnitude misbalance of 0.809 dB, and a phase split of 110.5 degrees.  This isn't so close to 90 degrees, but this may be a problem with the splitter that we're using, as Suresh detailed in elog 4755.  In that elog, he measured a phase split of POP110 of 105 degrees, unless the power going into the splitter was pretty high.  As with POP110, since I expect that we'll usually only look at one channel (I, for instance), this isn't such a big deal for AS110. 

I have left, for now, the board in the empty slot.  It looks like (I'm going to go check) there are 3 open channels on the whitening board that has the PD DC signals.  So, the only thing left to figure out is how we want to get some local oscillator action for this new board.

EDIT: Yes, those channels are available.  Right now (as a remnant from testing the whitening filters waaaay back in the day) they are called C1:LSC-PDXXX I, Q, DC.  I'll use 2 of those for the AS110 I and Q. 

As I see it, we have a few options for getting the 110 MHz LO to both the POP110 and AS110 demod boards.  

The current situation is described by Kiwamu in elog 5746.  The 55 MHz signal comes into the box, and is split 4 ways, with each path having 19.7 dBm.  One of these 4 is for 110.  It has a 2dB attenuator (giving us ~17.7 dBm), and then it goes to an MK-2 frequency multiplier.  I'm a little lost on why we're giving the MK-2 17 dBm, since it says that it can handle an input power between 1 - 15 dBm.  It has ~16 dB conversion loss, so the 110 output of the distribution board has (according to the drawing) 1.9 dBm.  The demod boards have a 10 dB attenuator as the first element on the LO path, so we're giving the ERA-5 -8 dBm. 

We can either amplify the 110 leaving the distribution box, split it, and then attenuate it to the appropriate level for the demod boards, or we can change the attenuators on the POP110 and AS110 demod boards. 

Since we seem to be over driving the 2x frequency multiplier, I think I should change the 2dB attenuator to a 5dB attenuator, so we're giving the 2x multiplier ~15 dBm.  The conversion loss of ~16 dB means we'll have -1 dBm of 110 MHz.  I want to amplify that by ~10 dB, to give 9 dBm.  Attenuate by 5 dB to get to 4 dBm, then split into 2, giving me 2 110 MHz spigots, each of ~1 dBm.  Since the demod boards expect between 0-2 dBm for the LO's, this should be just fine.

Thoughts, before I start scrounging parts, and pulling the RF distribution box?

- Do we have an appropriate amplifier?

- True challenge could be to find a feedthrough for the new port. (or to find a space for the amplifier in the box)

- PDXXX channels is on the DC whitening filter module. There could be some modification on this module (like diabling the whitening gain selector).

- We don't have AS11 and AS165, and so far it is unlikely to use AS11. i.e. The feedthrough, the slot on the crate, the whitening, and the channels can be trasnsition from 11 to 110.

So the IPC situation on the front end network is not so great right now.  For various no-longer-valid reasons, c1lsc had no RFM card, all the IPC connections were routed through the c1rfm model on c1sus, and routed to c1lsc via dolphin PCIe as needed.  As things grew, c1rfm became overloaded.  Koji tried to fix the situation by breaking things out of c1rfm to make direct connections where we could.  This cleared up c1rfm a bit, but not c1mcs is overloading.

Reminder: PCIe (dolphin) is faster and higher bandwidth than RFM.  The more things we can put on PCIe the better.

Attached is a graph of my rough accounting of the intended direct IPC connections between the front ends.  By "intended direct" I mean what should be direct connections if we had all the appropriate hardware.  Right now the actual connection graph is more convoluted than this since things are passing through c1rfm.  I note this graph was NOT particularly easy to make, which is very unfortunate.  I had to manually look through every model and determine the ultimate source of every incoming IPC.  Kind of a pain in the butt.  It would be nice if there was a simple way to represent this.

Here are some various solutions to the problem as I see it:

a) put c1lsc on the RFM network

This would allow c1lsc to talk to c1ioo, c1iscex, and c1iscey without having to go through c1sus, thereby eliminating c1rfm altogether.  I'm not sure why we didn't just do this originally.

Requires:

• One RFM card for c1lsc

b) put c1ioo on the PCIe network (and move c1sus's RFM card to c1lsc)

This is probably the most robust solution.

b1) There are roughly 8 IPCs going from c1ioo to c1sus, and 4 going the other way, and 3 IPCs from c1ioo to c1lsc.  If we put c1ioo on PCIe all of these now RFM connections would become direct PCIe connections, which would be a big win.

At this point only the end station front ends would be on RFM, and most of the connections to those come from c1lsc, so it would make sense to give c1lsc the RFM card, thereby eliminating a lot of stuff from c1rfm.

Requires:

• dolphin card for c1ioo (do the old sun machines support these?  if they don't we could swap the old sun machine with a new spare aLIGO-approved supermicro machines, which we have spares of)
• dolphin fibre to go to dolphin switch in 1X3 rack

b2) OR, we could move c1ioo to 1X4 with c1lsc and c1sus, and get a OneStop fibre cable to connect to its IO chassis.  We would still need a dolphin card, but we could use coper instead of fibre.  This is my preferred solution, since it moves c1ioo out of 1X1, where it's really in the way and making a lot of noise.  It would also be easier to manage all the machines if they're together in one rack.

Requires:

• dolphin card for c1ioo
• dolphin coper cable for c1ioo
• OneStop fibre for c1ioo

c) put another cpu in c1sus

c1sus is (I believe) able to support another 6-core cpu.  If we added more cores to c1sus, we could break up c1rfm into c1rfm0, c1rfm1, etc.  This is a less elegant solution imho, but it would probably do the job.

Requires:

• one new CPU for c1sus

I have checked out the new cdsutils repository at:

/opt/rtcds/cdsutils/release

This is a new repository that is intended to hold all of our python libraries and command-line utilities for interacting with the IFO, things like:

• get/write values EPICS channels
• interact with filter module switches
• average a test point for some amount of time
• etc.

Basically everything that used to be ez* or tds*.

There's not much in there at the moment, but hopefully it will start to get filled in soon.

WARNING:

This code in here will be used by the sites to interact with the real aLIGO IFOs.  Please be careful as you develop things in here, and o so conscientiously.  If you do bad things here and it messes things up at the sites people will be angry.  Particularly me, since I have to support everything in here for Guardian use.

Usage

<cdsutils>/lib/cdsutils is the primary python library.  For each function you want to add, put it in a new file named after the function.  So for instance function "foo" should be in a file called <cdsutils>/lib/cdsutils/foo.py.

There is a command line utility at <cdsutils>/bin/cdsutils.  It will automatically find anything you add to the library and expose it as a sub command (e.g. "cdsutils foo")

We'll try to put together a wiki page describing development and usage of this soon.

The reason we had the PCIe/RFM system was to test this mixed configuration in prior to the actual implementation at the sites.
Has this configuration been intesively tested at the site with practical configuration?

It's hard to believe that c1lsc -> c1sus only has 4 channels. We actuate ITMX/Y/BS/PRM/SRM for the length control.
In addition to these, we control the angles of ITMX/Y/BS/PRM (and SRM in future) via c1ass model on c1lsc.
So there should be at least 12 connections (and more as I ignored MCL).

I personally prefers to give the PCIe card to c1ioo and move the RFM card to c1lsc.
But in either cases, we want to quantitatively compare what the current configuration is (not omitting the bridging by c1rfm),
and what the future configuration will be including the addtional channels we want add in close future,
because RFM connections are really costly and moving the RFM card to c1lsc may newly cause the timeout of c1lsc
just instead of c1sus.

Koji was correct that I missed some connections from c1lsc to c1sus.  I corrected the graph in the original post.

Also, I should have noted, that that graph doesn't actually include everything that we now have.  I left out all the simplant stuff, which adds extra connections between c1lsc and c1sus, mostly because the sus simplant is being run on c1lsc only because there was no space on c1sus.  That should be corrected, either by moving c1rfm to c1lsc, or by adding a new core to c1sus.

I also spoke to Rolf today and about the possibility of getting a OneStop fiber and dolphin card for c1ioo.  The dolphin card and cable we should be able to order no problem.  As for the OneStop, we might have to borrow a new fiber-supporting card from India, then send our current card to OneStop for fiber-supporting modifications.  It sounds kind of tricky.  I'll post more as I figure things out.

Rolf also said that in newer versions of the RCG, the RFM direct memory access (DMA) has improved in performance considerably, which reduces considerably the model run-time delay involved in using the RFM.  In other words, the long awaited RCG upgrade might alleviate some of our IPC woes.

We need to upgrade the RCG to the latest release (2.7)

 I have removed the 135 MHz low pass from my new AS110 demod board, but these SXBPs have different feet than the SCLFs, so I want to confirm with Koji or someone that I can solder them in the same way, before I get carried away and destroy anything.  I should be able to finish this up tomorrow, plug in the demod board and the distribution box, and try out AS110 triggering, etc, tomorrow night.

We now have a proper install of cdsutils:

 controls@rossa:~ 0$ cdsutils
 usage: cdsutils <cmd> <args>

 Advanced LIGO Control Room Utilites

 Available commands:

   read         read EPICS channel value
   write        write EPICS channel value
   switch       switch buttons in standard LIGO filter module
   avg          average NDS channels for some amount of time
   servo        simple integrator (pole at zero)

 Add '-h' after individual commands for command help.
 controls@rossa:~ 0$ 

It is installed in /ligo/apps/cdsutils, and should be in the path on all workstations.

The "development" source working directory is currently checked out at /opt/rtcds/cdsutils/trunk.

 controls@rosalba:~ 0$ cdsutils
Traceback (most recent call last):
  File "/ligo/apps/cdsutils/lib/cdsutils/__main__.py", line 7, in <module>
    from cdsutils import CMDS
  File "/ligo/apps/cdsutils/lib/cdsutils/__init__.py", line 4, in <module>
    from servo import servo
  File "/ligo/apps/cdsutils/lib/cdsutils/servo.py", line 1, in <module>
    from epics import PV
ImportError: No module named epics
controls@rosalba:~ 1$ 

Mon Sep 16 19:40:32 2013 

 controls@rosalba:~ 0$ cdsutils
Traceback (most recent call last):
  File "/ligo/apps/cdsutils/lib/cdsutils/__main__.py", line 7, in <module>
    from cdsutils import CMDS
  File "/ligo/apps/cdsutils/lib/cdsutils/__init__.py", line 4, in <module>
    from servo import servo
  File "/ligo/apps/cdsutils/lib/cdsutils/servo.py", line 1, in <module>
    from epics import PV
ImportError: No module named epics
controls@rosalba:~ 1$ 

Mon Sep 16 19:40:32 2013 

 I properly installed the python-pyepics package on all the workstations, so this should be working now.

And for posterity, the pyepics source is at:

  pianos:/home/controls/src/pyepics

From this debian packages were built:

  controls@pianosa:~/src/pyepics 0$ debuild -uc -us

The .deb was then moved into the /ligo/apps/deb nfs:

  controls@pianosa:~/src 0$ cp python-pyepics_*_all.deb /ligo/apps/debs/pyepics/

It was then installed on the various workstations:

  controls@rosalba:~ 0$ sudo dpkg -i /ligo/apps/debs/pyepics/python-pyepics*.deb

This will probably need to be repeated any time we upgrade the EPICS install.

 Today I noticed that there was a lot of noise at the Bounce and Roll eigenfrequencies for ETMY. I found that the bandstop filter were set at completely the wrong frequencies, so I've remade them.

The filters were last tuned by Leo in May of 2011. Even so, he left the frequencies at the frequencies of the old MOS suspensions which had f_bounce ~ 12 Hz.

 The FOTON plot shows the OLD ones versus the NEW ones. The DTT spectra shows the oplev error signals in the usual state. I have also copied these over to the SUSPOS,PIT,YAW, and SIDE filter banks and turned them all ON.

I also turned OFF and deleted the 3 Hz RG filter that was there. There's no such peak in the error signal and even if one wanted to compensate for the stack mode, it should be a low Q filter, not this monster.

1. Found the low pass filters OFF in a couple of the MC SUS damping loops. This allows injection of OSEM noise all the way up to ~100 Hz. I deleted unused ones and turned on the 'Cheby' for all SUS DOFs: cheby1("LowPass",2,0.1,3)cheby1("LowPass",6,1,12)gain(1.13501)
2. Tuned the Bounce/Roll filters to catch the bounce and roll modes for all 3 MC SUS (based on the Mech Res Wiki page). These are now engaged on ALL MC SUS DOFs. They have been deleted from the ASCPIT/YAW filter banks and moved into the WFS screen into the MC1,2,3 filter banks there to be more in line with our knowledge of multi loop instability from notches in individual loops:ellip("BandStop",4,1,40,15.9,17.2)ellip("BandStop",4,1,40,23.5,24.7)gain(1.25893)

Jenne just discovered an issue with the python-matplotlib package that I knew was coming but forgot about.

We pull packages from the LSCSOFT Debian "squeeze" archive, which is a convenient way for us to install LIGO data analysis software.  There are no LSCSOFT archives for Ubuntu, and Debian "squeeze" is the closest supported distribution to Ubuntu 10.04 "lucid", which is what we are using.

DASWG recently added python-matplotlib to the LSCSOFT squeeze archive.  The version they added (1.0.1-3) supersedes the version in lucid, so by default apt wants to install it.  However, the LSCSOFT version is compiled against a newer version of some standard libraries, so it won't function on our system and seg faults.

The solution (a solution) is to use apt "pinning" to pin the package to the lucid version that works.  I've added the following file on all the 10.04 workstations to prevent the package from upgrading to the LSCSOFT version:

controls@pianosa:~ 0$ cat /etc/apt/preferences.d/pin_python-matplotlib
Package: python-matplotlib
Pin: release a=lucid
Pin-Priority: 1000

controls@pianosa:~ 0$ cat /etc/apt/preferences.d/pin_python-matplotlib
Package: python-matplotlib
Pin: release a=lucid
Pin-Priority: 1000 

I forgot that there were a couple of different matplotlib packages that all needed to be pinned.  To be safe I decided to just pin all packages to the lucid versions.  This will still allow us to install lscsoft packages that are not ubuntu, but it will always prefer packages from lucid instead.  Here's the new pinning file:

controls@pianosa:~ 0$ cat /etc/apt/preferences.d/pinning 
Package: *
Pin: release a=lucid
Pin-Priority: 1000
controls@pianosa:~ 0$ 

[Rana, Nic, Evan]

We did some work today on getting the AOM back up and running so that we can implement an SR560-based ISS.

We've removed the 18 AWG wire that was previously used to power the driver and have replaced it with a 12 AWG twisted pair (black and white, enclosed in a single gray cladding). This pair runs into the PSL rack's 24 V terminal block with a 2 A fuse. We've also replaced the cable connecting the AOM to the driver; it's now RG405.

Also disconnected the power to the old Kalmus FSS crystal driver box and turned it off. It was powered illegally. Also disconnected the power connection between the Sorensen and the old ISS AA chassis since it was wired directly without any fuse (which is a code violation). It will stay off until someone uses a proper fuse and wiring to hook it back up.

 While looking at the PMC REFL beam for the AOM diffracted beam, we noticed that although only one beam exists between the PMC and the first steering mirror, there are two afterwards and they both go to the PMC REFL  RFPD!!! This is madness. We only want one beam on our PDH diode.

The reason that we have two beams is that that first steering mirrors is actually a (W1-PW-1025-UV-1064-45P) non-wedged window with an AR coating on only one side. So two beams come out of it. There is a terrible and floppy and illegal anodized aluminum dump close to this beam which *someone* probably intended to use as a "scraper" to get rid of one of the beams.

Black anodized aluminum is a horrible beam dump material at 1064 - its about as grey as Steve's chair. And its so soft that it scatters light back into the PMC and makes more acoustic noise. And it is mounted so poorly (only one screw) that it can easily be bumped and twist and miss the beam. Punchline: only use anodized aluminum dumps for stray light around cameras or for HeNe for OL. Its NOT allowed anywhere where we care about interferometry of NIR beams.

It was also set to dump the dimmer beam. On Monday, we should order ~5 W1 and get them with a wedge of 1-2 deg. Then we use a black glass dump for the dim beam and orient the bright one to hit the REFL camera and the PMC REFL PD.

For the weekend, I have adjusted the crappy grey aluminum flapper to catch the bright beam so that the PMC REFL image no longer shows the interference fringe of two beams. Lets see how the PMC drifts over the next 3 days.

The first picture shows that there is indeed a DAC next to the ADC in the LSC IO chassis. The second picture shows how there are two cables, each one carrying 8 channels of DAC. The third one shows how these come out of the coil drivers to handle the Tip/Tilt mirrors which point the beam from the IMC into the PRC. It should be the case that the second Dewhitening filter board can give us access to the next 8 channels for use in driving an audio signal into the control room or an ISS excitation.

Back in 2009, Jenne replaced the PMC board mixer with a Level 13 one. Today I noticed that the LO level on the PMC screen was showing a LO level of ~5-10 dBm and fluctuating a lot. I think that it is related to the well known failure of the Mini-Circuits ERA-5SM amplifier which is on the D000419-A schematic (PMC Frequency Reference Card). The Hanford one was dying for 12 years and we found it in late 2008. If we don't have any in the blue bin, we should ask Steve to order 10 of them.

The attached trend shows 2000 days of hour trend of the PMC LODET channel. The big break in 2009 is when Jenne changed the mixer and then attenuated the input by 3 dB. The slow decay since then is the dying amplifier I guess.

Since the LOCALC channel was not in the trend, I added it to the C0EDCU file tonight and restarted the FB DAQD process. Its now in the dataviewer list.

I went out and took out the 3 dB attenuator between the LO card and the PMC Mixer. The LO monitor now reads 14.9 dBm (??!!). The SRA-3MH mixer data sheet claims that the mixer works fine with an LO between 10 and 16 dBm, so I'll leave it as is. After we get the ERA-5, lets fix the LODET monitor by upping its gain and recalibrating the channel.

Now netgpibdata is working again.

Usage:

cd /cvs/cds/rtcds/caltech/c1/scripts/general/netgpibdata   
./netgpibdata -i 192.168.113.108 -d AG4395A -a 10 -f meas01
./netgpibdata -i 192.168.113.105 -d SR785 -a 6 -f meas01   

Jenne witnessed and certified that they were working fine.
Now no one can say "it does not work"!

These weeks I was annoyed by the fact that netgpibdata was messed up by dummies.
A terrible situation was found:

1. Someone pushed the factory reset of one of the wifi bridges (LINKSYS WET54G).
2. Someone gave wrong IPs to the bridges (192.168.1.* instead of 192.168.113.*)
3. Someone left a default IP to the bridges. This means the routers had the same IPs.

-------

I gave the IPs to the bridges. According lines of /etc/hosts in linux1 were updated.

192.168.113.230 WET54G1
192.168.113.231 WET54G2

All of the network settings are taped on the bridge now.
The reset switch of each bridge was covered by a tape so that dummies can't randomly push the button.

The command was tested with each device.

Just wanted to point out that the correct "modern" path to this stuff is:

/opt/rtcds/caltech/c1/scripts/general/netgpibdata

This is, of course, the same directory, but under the correct "/opt/rtcds", instead of the old, incorrect "/cvs/cds".

The previous LSC whitening filters for the DCPDs were in an unknown state (although the transfer functions were actually measured and fit a while ago)
They had no DC gain control and some of the channels had modifications.

To make the setup clear, the filter module was replaced with the spare module without any modification.

The channels are now respoding to the whitening gain switches. So far there is no screen for the new  whitening gains yet.

Also I found that POX11 DC cable has not been connected. Now it is connected.

 Yesterday night I plugged back the REFL11 RF cable into the corresponding demodulation board.

[Koji Rana]

We are looking at NTP settings. I looked at nodus.

- xntpd is running

- We decided to start over the configuration file /etc/inet/ntp.conf

    - The old configuration was moved to ntp.conf.bak

    - The server configuration file was copied from ntp.server to ntp.conf

    - Caltech NTP servers 131.215.239.14 and 131.215.220.22 were selected as the servers we are reffering

    - Commented out the lines for "fudge" and "broadcast"

- xntpd was restarted

    - sudo /etc/init.d/xntpd stop
    - sudo /etc/init.d/xntpd start

- check how the daemon is running
      tail -50 /var/adm/messages

   Mar 28 23:00:49 nodus xntpd[27800]: [ID 702911 daemon.notice] xntpd 3-5.93e Mon Sep 20 15:47:11 PDT 1999 (1)
   Mar 28 23:00:49 nodus xntpd[27800]: [ID 301315 daemon.notice] tickadj = 5, tick = 10000, tvu_maxslew = 495, est. hz = 100
   Mar 28 23:00:49 nodus xntpd[27800]: [ID 798731 daemon.notice] using kernel phase-lock loop 0041
   Mar 28 23:00:49 nodus last message repeated 1 time
   Mar 28 23:00:49 nodus xntpd[27800]: [ID 132455 daemon.error] trusted key 0 unlikely
   Mar 28 23:00:49 nodus xntpd[27800]: [ID 581490 daemon.error] 0 makes a poor request keyid

- check the syncing staus by ntpq -p

        remote           refid      st t when poll reach   delay   offset    disp
   ==============================================================================
   *ntp-02.caltech. .CDMA.           1 u   37   64  377     0.56    3.010    0.08
   +ntp-03.caltech. ntp1.symmetrico  2 u   36   64  377     0.52    2.727    0.12
      this * means nodus is synced to ntp-02. "+" means it is stand by as a valid secondary server.  "when" increases every second.
      When "when" reaches "poll" the clock is synced to the server. These marks are not set at the beginning.
      It was necessary to wait for sometime to get synced to the server.

- Once nodus became a synced server, the realtime systems starts syncing to nodus automatically.

   controls@c1sus ~ 0$ cat /var/log/ntpd
   25 Mar 01:41:00 ntpd[4443]: logging to file /var/log/ntpd
   (...)
   28 Mar 23:13:46 ntpd[4983]: synchronized to 192.168.113.200, stratum 2
   28 Mar 23:14:25 ntpd[4983]: time reset +39.298455 s
   28 Mar 23:14:25 ntpd[4983]: kernel time sync status change 2001
   28 Mar 23:25:19 ntpd[4983]: synchronized to 192.168.113.200, stratum 2
   controls@c1sus ~ 0$ ntpq -p
        remote           refid      st t when poll reach   delay   offset  jitter
   ==============================================================================
   *nodus.martian   131.215.239.14   2 u   42   64  377    0.140   42.222  11.373


FB: /etc/ntp.conf was updated as below such that it refers nodus and also caltech server when nodus is not available

server 192.168.113.200
server 131.215.239.14

ntpd was restarted and

diskless RT machines: they are booted from /diskless/root on fb.
Therefore /diskless/root/etc/ntp.conf was updated in the same way as above.
When the machines are rebooted, this setting will be active.

control machines:

now they are referring nodus and other external servers too.

As payment for borrowing 2 of our seismometers, Zach has made us a new Trillium cable, to go from the granite station to the readout box, which we can put into 1X7, where the PEM ADC is.  To put the T-240 in side the can, and seal it, we need a little jumper cable from the seismometer to the granite, but for now, we can just pass this cable underneath the can.