I moved the epics IOC server process for the single Acromag ADC that monitors the PSL signals from megatron to c1auxex2.
First, I disabled the legacy support on all channels as explained in elog 13565. Then I copied the files npro_config.cmd and NPRO.db from /opt/rtcds/caltech/c1/scripts/Acromag to /cvs/cds/caltech/target/c1psl2/ following the pattern of the old Motorola machines and the new c1auxex2. I had to make some edits for correct paths and expanded the epics records to the standard we're using for ETMX.
I then added a service to systemd on c1auxex2 that runs the epics IOC for the Acromag PSL channels: /etc/systemd/system/modbusPSL.service. No more tmux on megatron.
Running two IOCs on a signle machine at the same time did not produce any errors and seems fine so far.
I've often gotten confused by the labeling on the SUS MEDM screens about the coil "Vmon" fields - they're labelled as "30 Hz HPF", and indeed this is one of the many readbacks available on the coil driver board. But the actual EPICS channel that is being displayed in this field is from the "EPICS VMON" monitor point on the coil driver board. It has a gain of 1/2, so the actual voltage going to the coil is twice the channel value. Today, I fixed the SUS master screen to avoid this confusion - new labeling is shown in Attachment #1.
pd80b rga scan at 175 day. IFO pressure 7.3e-6 Torr-it
Condition: vacuum normal, annuloses not pumped. Rga turned on yesterday.
The rga was not on since last poweroutage Jan 2, 2018 It is warming up and outgassing Atm2
I think part of the problem was that the rejected beam from the PBS was not really very Gaussian - looking at the spot on the beam profiler, I saw at least 3 local maxima in the intensity profile. So I'm now switching strategies to use a leakage beam from one of the PMC input steering optics- this isn't ideal as it already has the PMC modulation sideband on it, and this field won't be attenuated by the PMC transmission - but at least we can use a pre-doubler pickoff. This beam looks beautifully Gaussian with the beam profiler. Pics to follow shortly...
I tried to couple the PSL pickoff into the fiber today for several hours, but got nowhere really, achieved a maximum coupling efficiency of ~10%. TBC tomorrow... Work done yesterday and today
I tried to couple the PSL pickoff into the fiber today for several hours, but got nowhere really, achieved a maximum coupling efficiency of ~10%. TBC tomorrow... Work done yesterday and today:
The final temperature reached in about 4.5 hours is 30.5C, while the starting temperature is about 24C. I can't seem to screenshot the data for some reason.
Also, I will calibrate the lab temperature sensor to Celcius in the near future so that we would have a working sensor inside the lab.
After almost 3 hours the temperature rose by about 3.5C. Seems a bit slow, but we can drive it more if necssary. The heating curve itself is exponentiial, which is a good sign.
We started the actual heating test today and it seems to be working so far. Hoping to heat it to about 40C. We also set up another temperature sensor to measure the lab temperature and connected it to J7, bottom.
I was looking at this a little more closely. As I understand it, the purpose of the audio differential IF amplifier is:
Attachment #1 shows, the changes to the TF of this stage as a result of changing R19->50ohm, R17->500ohm. For the ALS application, we expect the beat signal to be in the range 20-100MHz, so the 2f frequency component of the mixer output will be between 40-200MHz, where the proposed change preserves >50dB attenuation. The Q of the ~500kHz resonance because of the series LCR at the input is increased as a result of reducing R17, so we have slightly more gain there.
At the meeting yesterday, Koji suggested incorporating some whitening in the preamp itself, but I don't see a non-hacky way to use the existing PCB footprint and just replace components to get whitening at audio frequencies. I'm going to try and measure the spectrum of the I and Q demodulated outputs with the actual beat signal to see if the lack of whitening is going to limit the ALS noise in some frequency band of interest.
Does this look okay?
The demod circuit board is configured to have gain of x100 post demod (conversion loss of the mixer is ~-8dB). This works well for the PDH cavity locking type of demod scheme, where the loop squishes the error signal in lock, so most of the time, the RF signal is tiny, and so a gain of x100 is good. For ALS, the application needs are rather different. So we lowered the gain of the "Audio IF amplifier" stage of the circuit from x100 to x10, by effecting the resistor swaps 10ohms->50ohms, 1kohm->500ohms (more details about this later).
Gautam and I set up the insulated seismometer can in the lab today. I had previously wired up the two heaters I placed onto the sides of the can in parallel to get a total resistance of 12.5 ohms and then I wrapped the whole can in 3 layers of insulation (k-factor 0.25). We placed it on a large sheet of insulation as to not crush the wires leading out the bottom of the can. I stuck on one of my AD590 sensors to the inside of the can onto the copper lining using duct tape, though this is only a temporary solution. In the future, it would be nice to have some sort of thermal clamp to secure the sensor to the can. To provide power to the heater circuit board and the temperature sensor board, we got a powerstrip and plugged in two power supplies and a function generator into it. The heater circuit (attachment 3) is powered by one of the power supplies and the function generator, while the temperature sensor (attachment 5) is stuck to the side of the can and is powered by the second power supply. The heater circuit's MOSFET (IRF640, attachment 4) is placed on a metal block and sandwiched between two more to make sure it doesn't move around. The temperature sensor is connected by a long BNC cable to the channels in attachment 6.
gautam: we plugged the BNC output of Kira's temperature sensor circuit to J7 on the AA input chassis in 1X2 - this corresponds to ADC1 input 12 in c1ioo. I then made a "PEM" namespace block inside the c1als model, and placed a single CDS filter module inside it (this can be used for calibration purposes). The filter module is named "C1:PEM-SEIS_EX_TEMP", and has the usual CDSfilt channels available. I DQ'ed the output of the filter module (@256 Hz, probably too high, but I'm holding off on a recompile for now). Recompilation and model restart of c1als went smoothly.
2 bench power supplies are being used for this test, we can think of a more permanent solution later.
**25 Jan noon: Added another filter module, "C1:PEM-SEIS_EX_TEMP", to which Kira is hooking up a second temperature sensor, which will serve as a monitor of the "Ambient" lab temperature. Added DQ channel for the output of this filter module, fixed sampling to 32Hz. Compile and restart went smooth.
M4 local earthquake at 10:10 UTC There is no sign of damage.
....here is an other one.........M5.8 Ferndale, CA at 16:40 UTC
The result is a smooth transition from idling to the controlled state with no sudden or large offset changes.
While checking how smooth the transition is we still noticed significant motion of ETMX by looking at the locked green laser and OpLevs. We found that this motion was not caused by interruption of the slow offset adjust, but rather the Watchdog being re-initialized to its OFF state, which cuts the fast channels OFF. On other optics this is observed too, but not as severe. The cause is a rather large offset on the LR coil coming from the fast DAQ, which was reported as 50mV by the slow readback channel (while other readback values are <10mV). It is present even when turning the output of the CDS model OFF, but vanishes when the watchdog is triggered. This helped us trace it to an offset of the DAC output itself: it is present at the output of the AI board but vanishes when the DAC is disconnected. The actual offset is ~40mV, as opposed to other channels on the same board, which ahve offsets in the range 3-7mV.
While we can compensate for this offset in software - it made us wonder if the DAC channel is somehow busted and if that's what causing the 'wandering' of ETMX that we have been observing recently. There are two free DAC channels on the AI chassis that has the side coil and the green temperature control signals. We could re-route the LR signal through a different DAC channel to fix this.
gautam: 40mV offset at the AI board output gets multiplied by 3 in the dewhitening board, so there is a 120mV DC offset going to the coil (measured at dewhite board output with DMM). The offset itself isn't hurting us, but the fact that it is several times larger than other channels led us to wonder if it could be drifting around as well. From my SOS pitch balancing forays, in my head I have the number 30mrad as being the full range of the OSEM actuation - so if the offset swings by 120mV, that's ~150urad of motion, which is quite large, and is of the order of magnitude I'm used to seeing ETMX move around by.
I compiled the burt binaries on c1auxex2 which took a little fiddling with dependencies and paths but nothing too major. The complete local epics folder (/opt/epics/) which contains the base epics binaries, modbus and burt for 32-bit linux has been copied to the shared drive at /opt/rtapps/epics-3.15.5. They belong to the most recent stable release. This was so we can now automatically call burt after the IOC initialization on c1auxex2 to restore the backed-up channel values.
I also copied the database definition and modbus instruction files to /cvs/cds/caltech/target/c1auxex2, from where they are now being read upon IOC initialization. This is an excerpt of the service file:
#ExecStart=/usr/bin/procServ -f -L /home/controls/modbusIOC/modbusIOC.log -p /run/modbusioc.pid 8008 /opt/epics/modules/modbus/bin/linux-x86/modbusApp /cvs/cds/caltech/target/c1auxex2/ETMXaux2.cmd <-- Contains logging to file, see note 1)
ExecStart=/usr/bin/procServ -f -p /run/modbusioc.pid 8008 /opt/epics/modules/modbus/bin/linux-x86/modbusApp /cvs/cds/caltech/target/c1auxex2/ETMXaux2.cmd <-- Initializes the EPICS IOC with Modbus support
ExecStop=/bin/kill -9 ` cat /run/modbusioc.pid` <-- Kills the detached process by its process ID
ExecStartPost=/bin/bash -c "/opt/epics/extensions/bin/linux-x86/burtwb -f /opt/rtcds/caltech/c1/burt/autoburt/latest/c1auxex.snap" <-- Restores general channel values
ExecStartPost=/bin/bash -c "/opt/epics/extensions/bin/linux-x86/burtwb -f /opt/rtcds/caltech/c1/medm/MISC/ifoalign/burt/ETMX.snap" <-- Restores PIT and YAW values from align MEDM screen
ExecStartPost=/bin/bash -c ". /home/controls/modbusIOC/ETMXaux2.sh" <-- Enables writing to PIT and YAW DAC channels, see note 2)
Note 1) I removed the logging to file for now because I noticed that if there are Acromag communication issues the logfile tends to grow in size VERY fast. In the cryo lab is had gotten to over 70GB just over the winter break. I don't think it's absolutely necessary to have it, and if diagnostics are needed we can easily uncomment it temporarily.
Note 2) I modified the static EPICS records of the four OSEM bias adjust channels so they won't start updating as soon as the IOC starts up (and before the channel defaults are restored by burt). This was done by setting the OMSL (output mode select) field from "closed_loop" to "supervisory". Sample record:
field(DESC,"Bias Adjust for ETMX UL Coil Output")
field(OUT, "@asynMask(C1AUXEX_XT1541A_DAC, 0, -16)MODBUS_DATA")
field(OMSL,"supervisory") <-- Used to be "closed_loop"
field(DOL, "C1:SUS-ETMX_ULBiasSet PP")
Now, on reboort/IOC re-initialization the physical DAC channels are performing a one-time readback of the last stored value in the Acromag's register, then idle until the last StartPost statement executes the script ETMXaux.sh, which changes their OMSL field back to "closed_loop". This causes them to start updating their output from the calc records defined in their DOL field (which have by then recovered their default values curtesy of burt). The result is a smooth transition from idling to the controlled state with no sudden or large offset changes.
I replaced the two remaining D-Sub M/M cables that still had gender-changers with M/F cables today, completing the mechanical and wiring work on the ETMX rack. All backplane adapter boards were secured to a cross-strut of the crate using zip ties. This was necessary because the adapter boards don't fit the crate with their panels attached ( the ETMX eurocrate is the only one with slightly different dimensions from all the others), and the we can't mount them to the strut using the panels. This won't be an issue on any of the other crates.
In other news:
I disabled the legacy support in the three Acromag ADC units and set the input averaging to 10 samples via the USB configuration utility. The documentation is unfortunately a little sparse about what this actually means. The manual states that "fresh input data is available to the network every 10ms", so the sampling rate is for sure faster than 100Hz. Since the IOC updates its channels every .1 seconds I assume that an averaging value of 10 to reduce the input noise is safe. The maximum value the configuration tool permits is 200. I tried this using the CryoLab DAQ and set all input channels to 200 and used StripTool to look at the time series of a slow oscillation (.1Hz) with a large amplitude (16Vpp) and looked for missed data points, indicating too long wait times for channels updates. There was no such qualitative difference between 1 sample, 10 samples, and 200 samples, so even pushing the averaging value to max seemed okay. I went with the conservative value of 10 for the ETMX DAQ, but we can likely increase this if noise on the slow inputs becomes an issue.
The input scaling of the ADC channels has been corrected. I changed the conversion method in the EPICS records from manual using the ASLO and AOFF fields to using engineering units via EGUF and EGUL. This required a little attention. The Acromags scale the dynamic input range of +/- 10V to +/- 30,000 raw value, but the EPICS IOC interprets the data type as ranging from -32767 to +32768, so the EGUF and EGUL fields must be set to -10.923 and +10.923 to achieve proper scaling. I also changed the SCAN field on all ADC channels to 0.1 seconds. This has been done for all ADC and DAC channel records.
I was looking into the physics of polarization maintaining fibers, and then I was trying to remember whether the fibers we use are actually polarization maintaining. Looking up the photos I put in the elog of the fibers when I cleaned them some months ago, at least the short length of fiber attached to the PD doesn't show any stress elements that I did see in the Thorlabs fibers. I'm pretty sure the fiber beam splitters also don't have any stress elements (see Attached photo). So at least ~1m of fiber length before the PD sensing element is probably not PM - just something to keep in mind when thinking about mode overlap and how much beat we actually get.
On Friday, while Udit and I were doing some characterization of the EX+PSL IR beat at the LSC rack, I noticed that there were sidebands around the main beat peak at 20dBm lower level. These were offset from the main peak by ~200kHz - I didn't do a careful characterization but because of the symmetric nature of these sidebands and the fact that they appeared with the same offset from the main peak for various values of the central beat frequency, I hypothesize that these are from the modulation sidebands we use for PDH locking the EX laser to the arm cavity. So we can estimate the modulation depth from the relative powers of the main beat peak and the ~200kHz offset sidebands.
Since the IR light is used for the beat and we directly couple it to the fiber to make the beat, there is no green or IR cavity pole involved here. 20dBm in power means . And so the modulation depth, . I will do a more careful meaurement of this, but this method of measuring the modulation depth can give us a precise estimate - for what it's worth, this number is in the same ballpark as the measurement I quote in elog12105.
What is the implication of having these sidebands on our ALS noise? I need to think about this, effectively the phase noise of the SR function generators we use to do the phase modulation of the EX laser is getting imprinted on the ALS noise? Is this hurting us in any frequency range that matters?
I plan to do some characterization of this problem. The plan is to use THD as a metric for whether we are having hidden saturations. Pg 9 of the LT1125 datasheet tells us what fraction of THD to expect. I will use one of the several unused DAC channels available at the LSC rack to drive a 100Hz sine wave into one of the inputs of the whitening chassis, and measure the THD up to a reasonable harmonic number (will probably be set by the ADC noise) for (i) various whitening gain settings and (ii) various input signal amplitudes.
The motivation is to attempt to quantify the problem better:
Then we can decide what, if anything, to do about this issue.
I did some work on the PSL table today. Main motivations were to get a pickoff for the BeatMouth PSL beam before any RF modulations are imposed on it, and to improve the mode-matching into the fiber. Currently, we use the IR light reflected by the post doubling oven harmonic separator. This has the PMC modulation sideband on it, and also some green leakage.
So I picked off ~8.5mW of PSL light from the first PBS (pre Faraday rotator), out of the ~40 mW available here, using a BS-80-1064-S. I dumped the 80% reflected light into the large beam dump that was previously being used to dump this PBS reflection. Initially, I used a R=10% BS for S-pol that I found on the SP table, but Koji tipped me off on the fact that these produce multiple reflected beams, so I changed strategy to use the R=80% BS instead.
The transmitted 20% is routed to the West edge of the PSL table via 2 1" Y1-1037-45S optics, towards the rough vicinity of the fiber coupler. For now it is just dumped, tomorrow I will work on the mode matching. We may want to cut the power further - ideally, we want ~2.5mW of power in the fiber - this is then divided by 4 inside the beat mouth before reaching the beat PD, and with other losses, I expect ~500mW of PSL power and comparable AUX light, we will have a strong >0dBm beat.
Attachment #1 is a picture of my modifications. For this work, I
1500 and 2000 lbs load cells arrived from MIT to measure the vertical loads on each leg.
We've been thinking about putting in a blade spring / wire based aluminum breadboard on top of the ETM & ITM stacks to get an extra factor of 10 in seismic attenuation.
Today Koji and I wondered about whether we could instead put something on the outside of the chambers. We have frozen the STACIS system because it produces a lot of excess noise below 1 Hz while isolating in the 5-50 Hz band.
But there is a small gap between the STACIS and the blue crossbeams that attache to the beams that go into the vacuum to support the stack. One possibility is to put in a small compliant piece in there to gives us some isolation in the 10-30 Hz band where we are using up a lot of the control range. The SLM series mounts from Barry Controls seems to do the trick. Depending on the load, we can get a 3-4 Hz resonant frequency.
Steve, can you please figure out how to measure what the vertical load is on each of the STACIS?
I have acquired 5 pieces of the Teledyne AP1053 from Koji - these are now at the 40m. I will determine an appropriate location for storage of these and update. We are also looking to acquire 5 more of these. The combination of high power output (26dBm), low gain (10dB), and low noise figure (1.5dB) are quite uncommon in an amplifier and so these should be used only when such properties are required simultaneously.
*Steve informs me that these amps have been stored in the RF cabinet E6 along the east arm.
Steve's note: Teledyne rf amp product selection guide
Teledyne rf low noise amp guide
After discussing with Koji, we looked at the aLIGO incarnation of this board. Interestingly, it too has a similar topology of 4 switchable gain stages with gains of 24, 12, 6 and 3dB. The main differences are that they use single Op27 ICs instead of the quad LT1125s, and also, they use a different combination of feedback resistors to realize the various gains.
We considered upping the feedback resistance (R15, R143) on the 24dB gain stage of our boards from (1k, 66.5ohms) to (3k, 200ohms) as on the aLIGO boards - but this doesn't really help? Because KCL demands that the same current flow in R15 and R143, and so the output Vsat of the op amp and its max current driving capabilities in combination determine if the inverting input can follow the non inverting input?
As Hartmut points out in his note, he was able to access the full range of ADC voltages when the gain was set to 3dB, despite the fact that the LT1125 was still getting internally saturated. Operating with minimum 24dB whitening gain doesn't really solve the problem either because the problem just gets shifted to the next gain stage in the chain, and we still have saturation. I also don't have a feeling for how much differential voltage these LT1125s can sustain before they are damaged - I guess the planned THD check will reveal if they are okay or not.
It seems to me like the only way to truly fix this problem of one stage saturating and screwing up the others is to use single Op27s (or equivalent) in place of the quad LT1125s. The aLIGO design also has a series resistance to the non-inverting input - this can help prevent current overdraw from the previous stage (due to a lowered input impedance of the OpAmp - but I wonder how low this can go?).
this is the note from Hartmut Grote on this topic from 2004
The beat setup has been made on the PSL table. The BS and the PD were setup. The beat was found at 29.42degC and 50.58degC for the PSL and AUX crystal temperatures, respectively.
We are ready for the EOM test. I have instruments stacked around the PSL table. Please leave them as they are for a while. If you need to move them, please contact with me. Thanks.
A picked-off PSL after the main modulator was used as the PSL beam. This was already introduced close to the setup thanks to the previous 3f cancellation test ELOG 11029. The AUX beam was obtained from the transmission of 90% mirror. Both paths have S polarization. The beams are combined with a S-pol 50% BS. The combined beam is detected by a new focus 1GHz PD.
The PSL crystal temp (actual) was 50.58degC. The AUX crystal temp was swept upward and the string beat was found at 50.58degC. After a bit of alignment, the beat strength was -18dBm (at 700V/A RF transimpedance of NF1611) .
I'm planning to construct a beat setup between the PSL and AUX beams. I am going to make it in the area shown in a blue square in the attached photo. This does not disturb Johannes' and PSL setups. The beams are obtained from the PBS reflection of the PSL and the dumped beam of the aux path (0th or 1st order beam of the AOM).
After some research: -the- reason for the reduced +/- 20,000 swing in raw values is a default setting for having support for legacy devices enabled when using the acromag proprietary i2o peer-to-peer protocol. So this is doubly unnecessary because a) we don't have any legacy devices at all and b) we're using pure modbus/TCP and no i2o. To change the setting I have to connect via the USB configuration utility. In addition, I want to understand the averaging feature of the acromag units better, which is also configured via USB, and lets one set a fixed amount of samples to be averaged before updating the read-register value. The documentation says that the 8 channels are multiplexed into a single ADC, and that new input data is available after 10 ms for each channel, suggesting a sampling rate of 100 Hz per channel and that the multiplexing happens faster, but is not super-clear about this, so I want to test it in the cryo lab first before unleashing it onto c1auxex2.
Furthermore, the standard timing options for updating epics records are 10s, 5s, 2s, 1s, 0.5s, 0,2s, and 0.1s. On the previous c1auxex, the monitoring channels were set to 0.1s, but that clashes with the 16 Hz global EPICS rate, resulting in partial double-sampling. One can manually provide the option 0.0625s for 16Hz update rate. I will test this and how it deals with the averaging in the cryolab too.
We use D990694 in various places. Today, Rana alerted me to an important consideration to be kept in mind when we use this board, which I found quite interesting. I still don't understand the problem at the BJT level, but I think one can appreciate the problem without going to the transistor design of the LT1125. I'm attaching an annotated schematic of the whitening section in question. If the following assumptions are valid, then I think my picture is valid.
Then, as one can see in the attached schematic, when we set the gain of any input to <24dB, we must ensure that the input voltage is less than approximately 2V. Otherwise, by asking too much of the first stage op-amp in the quad IC LT1125, we may be messign around with all the 4 op amps in the quad! Even the 0dB setting is not immune to this problem, as it uses one of the 4 op amps.
Now that I think about this a bit more - this problem shouldn't be significant for the usual LSC degrees of freedom when in lock, as the huge DC gain of the loop should squish large DC values of the error signals, and so there shouldn't be any danger of overloading the LT1125. But I don't know if we are being hurt by this effect when flashing through resonances, when the PDH horn-to-horn voltage can be quite high (which is in principle a good thing?). I don't know if there is any "hysterisis" effect where the overloaded quad IC has some relaxation time before it returns to normal operation, and if we are being limited in our ability to catch lock because if this effect.
The concerns remain valid for th ALS demodulated error signals though, for which the signals will remain large throughout.
[rana, kevin, udit, gautam]
quick notes of some discussions we had today:
RXA: 0805 size SMD thin film resistors have been ordered from Mouser, to be shipped on Monday. **note that these thin film resistors are black; i.e. it is NOT true that all black SMD resistors are thick film**
Rendered the SOS assembly (D960001) with correct materials and all and it looks very nice. Will extend this to the building cad later.
40m CAD Project
I swapped the inputs to the ZHL-3A at the PSL table - so now the X beat RF signals from the beat mouth are going through what was previously the Y arm ALS electronics. From Attachment #1, you can see that the Y arm beat is now noisier than the X. The ~5kHz peak has also vanished.
So I will pursue this strategy of switching to try and isolate where the problem lies...
Somebody had forgotten to turn the HEPA variac on the PSL table down. It was set at 70. I set it at 20, and there is already a huge difference in the ALS spectra
c1psl, c1susaux, and c1auxey today
MC autolocker got stuck (judging by wall StripTool traces, it has been this way for ~7 hours) because c1psl was unresponsive so I power cycled it. Now MC is locked.
I am facing two problems:
Status of the AS-port auxiliary laser injection
This happened because there are multiple ways to scale the raw value of an EPICS channel to the desired output range. In the CryoLab I was using one way, but the EPICS records I copied from c1auxex were doing it differently. Basically this:
If the "LINR" field is set to "LINEAR", the fields EGUF and EGUL are used to convert the raw value to the channel value VAL. To use them, one needs to enter the voltages that return the maximum and minimum values expected for the given data type. It used to be +10V and -10V, respectively, and was copied that way but that doesn't work with the data type required for the Acromag units. For -some- reason, while the the ADC range is -10V to +10V, this corresponds to values -20000 to +20000, while for the DAC channels it's -30000 to +30000. I had observed this before when setting up the DAQ in the CryoLab, but there we were using "NO CONVERSION", which skips the EGUF and EGUL fields, and used the ASLO and AOFF for manual scaling to get it right. When I mixed the records from there with the old ones from c1auxex this got lost in translation. Gautam and I confirmed by eye that this indeed explains the different levels well. This means that the VMon channelsfor the coils are also showing the wrong voltages, which will be corrected, but the readback still definitely works and confirms that the enable switches do their job.
Some suggestions of checks to run, based on the rightmost colum in the wiring diagram here - I guess some of these have been done already, just noting them here so that results can be posted.
With Johannes' help, I re-installed the box in the LSC electronics rack. In the end, I couldn't find a good solution to thermally insulate the inside of the box with foam - the 2U box is already pretty crowded with ~100m of cabling inside of it. So I just removed all the haphazardly placed foam and closed the box up for now. We can evaluate if thermal stability of the delay line is limiting us anywhere we care about and then think about what to do in this respect. This box is actually rather heavy with ~100m of cabling inside, and is right now mounted just by using the ears on the front - probably should try and implement a more robust mounting solution for the box with some rails for it to sit on.
I then restored all the cabling - but now, the delayed part of the split RF beat signal goes to the "RF in" input of the demod board, and the non-delayed part goes to the back-panel "LO" input. I also re-did the cabling at the PSL table, to connect the two ZHL3-A amplifier inputs to the IR beat PDs in the BeatMouth instead of the green BBPDs.
I didn't measure any power levels today, my plan was to try and get a quick ALS error signal spectrum - but looks like there is too much beat signal power available at the moment, the ADC inputs for both arm beat signals are overflowing often. The flat gain on the AS165 (=ALS X) and POP55 (=ALS Y) channels have been set to 0dB, but still the input signals seem way too large. The signals on the control room spectrum analyzer come from the "RF mon" ports on the demod board, and are marked as -23dBm. I looked at these peak heights with the end green beams locked to the arm cavities, as per the proposed new ALS scheme. Not sure how much cable loss we have from the LSC rack to the network analyzer, but assuming 3dB (which is the Google value for 100ft of RG58), and reading off the peak heights from the control room analyzer, I figure that we have ~0dBm of RF signal in the X arm. => I would expect ~3dBm of signal to the LO input. Both these numbers seem well within range of what the demod board is designed to handle so I'm not sure why we are saturating.
Note that the nominal (differential) I and Q demodulated outputs from the demod board come out of a backplane connector - but we seem to be using the front panel (single-ended) "MON" channels to acquire these signals. I also need to update my Fiber ALS diagram to indicate the power loss in cabling from the PSL table to the LSC electronics rack, expect it to be a couple of dB.
After labeling cables I would disconnect, I pulled the box out of the LSC rack. Attachment #1 is a picture of the insides of the box - looks like it is indeed just two lengths of cabling. There was also some foam haphazardly stuck around inside - presumably an attempt at insulation/isolation.
Since I have the box out, I plan to measure the delay in each path, and also the signal attenuation. I'll also try and neaten the foam padding arrangement - Steve was showing me some foam we have, I'll use that. If anyone has comments on other changes that should be made / additional tests that should be done, please let me know.
20180111_2200: I'm running some TF measurements on the delay line box with the Agilent in the control room area (script running in tmux sesh on pianosa). Results will be uploaded later.
Johannes informed me that he touched up the PMC REFL camera alignment. I am holding off on re-soldering the AOM driver power as I could use another pair of hands getting the power cable disentangled and removed from the 1X2 rack rails, so that I can bring it out to the lab and solder it back on.
Is anyone aware of a more robust connector solution for the type of power pins we have on the AOM driver?
While moving the RefCav to facilitate the PSL shelf install, I bumped the power cable to the AOM driver. I will re-solder it in the evening after the shelf installation. PMC and IMC have been re-locked. Judging by the PMC refl camera image, I may also have bumped the camera as the REFL spot is now a little shifted. The fact that the IMC re-locked readily suggests that the input pointing can't have changed significantly because of the RefCav move.
[ Johannes, Rana, Mark and Steve ]
On the second trial the shelf was installed. Plastic cover removed. South end door put back on and 2W Inno turned on.
Shelf 10 " below the existing one: 92" x 30" x 3/4" melamine (or MDF) covered with white Formica. 200 lbs it's max load. Working distance to top of the table 18"
I checked the calibration of the Oplevs for both ITMs, both ETMs and the BS. The table below summarizes the old and new cts->urad conversion factors, as well as the factor describing the scaling applied. Attachment #1 is a zip file of the fits performed to calculate these calibration factors (GPS times of the sweeps are in the titles of these plots). Attachment #2 is the spectra of the various Oplev error signals (open loop, so a measure of seismic induced angular motion for a given optic, and DoF) after the correction. Loop TF measurements post calibration factor update and loop gain adjustment to be uploaded tomorrow.
Now that the ETMX calibration has been updated, let's keep an eye out for a wandering ETMX.
Looks like this problem presisted over the weekend - Attachment #1 is the wall StripTool trace for PSL diagnostics, seems like the control signal to the NPRO PZT and FSS EOM were all over the place, and saturated for the most part.
I traced down the problem to an unresponsive c1iool0. So looks like for the IMC autolocker to work properly (on the software end), we need c1psl, c1iool0 and megatron to all be running smoothly. c1psl controls the FSS box gains through EPICS channels, c1iool0 controls the MC servo board gains through EPICS channels, and megatron runs the various scripts to setup the gains for either lock acquisition or in lock states. In this specific case, the autolocker was being foiled because the mcdown script wasn't running properly - it was unable to set the EPICS channel C1:IOO-MC_VCO_GAIN to its lock acquisition value of -15dB, and was stuck at its in-lock value of +7dB. Curiously, the other EPICS channels on c1iool0 seemed readable and were reset by mcdown. Anyways, keying the c1iool0 crate seems to have fixed the probelm.
I've noticed this a couple of times today - when the autolocker runs the mcdown script, sometimes it doesn't seem to actually change the various gain sliders on the PSL FSS. There is no handshaking built in to the autolocker at the moment. So the autolocker thinks that the settings are correct for lock re-acquisition, but they are not. The PCdrive signal is often railing, as is the PZT signal. The autolocker just gets stuck waiting to re-acquire lock. This has happened today ~3 times, and each time, the Autolocker has tried to re-acquire lock unsuccessfully for ~1hour.
Perhaps I'll add a line or two to check that the signal levels are indicative of mcdown being successfully executed.
I know it, and I don't like it. DokuWiki seems to allow us to use an external server for notification emails. That would be the way to go.
I think sendmail is required on nodus since that's how the dokuwiki works. That's why the dokuwiki was trying to send an email to Umakant.
Steve and I removed c1auxex from 1X9 today to make space for the DAQ chassis. Steve installed rails for mounting. To install the box I had to remove all cabling, for which I used the usual precautions (disconnect satellite box etc.)
On reconnect c1auxex2 didn't initialize the physical EPICS channels (the 'actual' acromag channels), apparently it had trouble communicating. A reboot fixed this. It's possible that this is because of the direct cable connection without a network switch that exists
between the Acromags and c1auxex. The EPICS server was started automatically on reboot.
Currently the channel defaults need to be loaded manually after every EPICS server script restart with burt. I'm looking for a good way to automate this, but the only compiled burt binaries for x86 (that we can in principle run on c1auxex2 itself) on the martian network are from EPICS version 3.14.10 and throw a missing shared object error. Could be that simply some path variable is missing.
The burt binaries are not distributed by the lscsoft or cdssoft packages, so alternatively we would need to compile it ourselves for x86 or get it working with the older epics version.
Okay I will port awade's python mailer stuff for this purpose.
gautam 11Feb2018 1730: The python mailer had never once worked successfully in automatically sending the message. I realized this may be because I had put the script on the root user's crontab, but had setup the authentication keyring with the password for the mailer on the controls user. So I have now setup a controls user crontab, which for now just runs the pizza mailing. let's see if this works next Sunday...
I personally don't like the idea of having sendmail (or something similar like postfix) on a personal server as it requires a lot of maintenance cost (like security update, configuration, etc). If we can use external mail service (like gmail) via gmail API on python, that would easy our worry, I thought.
After much googling, I figured out how to install pip on SL7:
sudo easy_install pip
Next, I installed git:
sudo yum install git A
Turns out, actually, pip can be installed via yum using
sudo yum install python-pip
I'm having trouble getting the sendmail service going on nodus since the Christmas day power failure - for some reason, it seems like the mail server that sendmail uses to send out emails on nodus (mx1.caltech.iphmx.com, IP=18.104.22.168) is on a blacklist! Not sure how exactly to go about remedying this.
Running sudo systemctl status sendmail.service -l also shows a bunch of suspicious lines:
Jan 12 10:15:27 nodus.ligo.caltech.edu sendmail: STARTTLS=client, relay=cluster6a.us.messagelabs.com., version=TLSv1/SSLv3, verify=FAIL, cipher=DHE-RSA-AES256-GCM-SHA384, bits=256/256
Jan 12 10:15:45 nodus.ligo.caltech.edu sendmail: w0A7QThE032091: to=<firstname.lastname@example.org>, ctladdr=<email@example.com> (1001/1001), delay=2+10:49:16, xdelay=00:00:39, mailer=esmtp, pri=5432408, relay=cluster6a.us.messagelabs.com. [22.214.171.124], dsn=4.0.0, stat=Deferred: 421 Service Temporarily Unavailable
Jan 12 11:15:23 nodus.ligo.caltech.edu sendmail: STARTTLS=client, relay=cluster6a.us.messagelabs.com., version=TLSv1/SSLv3, verify=FAIL, cipher=DHE-RSA-AES256-GCM-SHA384, bits=256/256
Jan 12 11:15:31 nodus.ligo.caltech.edu sendmail: w0A7QThE032091: to=<firstname.lastname@example.org>, ctladdr=<email@example.com> (1001/1001), delay=2+11:49:02, xdelay=00:00:27, mailer=esmtp, pri=5522408, relay=cluster6a.us.messagelabs.com. [126.96.36.199], dsn=4.0.0, stat=Deferred: 421 Service Temporarily Unavailable
Jan 12 12:15:25 nodus.ligo.caltech.edu sendmail: STARTTLS=client, relay=cluster6a.us.messagelabs.com., version=TLSv1/SSLv3, verify=FAIL, cipher=DHE-RSA-AES256-GCM-SHA384, bits=256/256
Jan 12 12:15:42 nodus.ligo.caltech.edu sendmail: w0A7QThE032091: to=<firstname.lastname@example.org>, ctladdr=<email@example.com> (1001/1001), delay=2+12:49:13, xdelay=00:00:33, mailer=esmtp, pri=5612408, relay=cluster6a.us.messagelabs.com. [188.8.131.52], dsn=4.0.0, stat=Deferred: 421 Service Temporarily Unavailable
Why is nodus attempting to email firstname.lastname@example.org?
Measurements for good fit were made. The new shelf will be installed on next Tuesday at 2pm
The reference cavity ion pump is in the way so the cavity will be moved 5" westward. The shelf height space will be 10" Under shelf working height 18" to optical table.
I have just received the scheduling of the PSL self work for tomorrow. Gautam and I agreed that if it is needed I will shut the laser off and cover the hole table with plastic.