Next, I will work on commissioning the BEAT MOUTH for ALS beat generation.
Note: In the ~40mins that I've been typing out these elogs, the IR lock has been stable for both the X and Y arms. But the X green has dropped lock twice, and the Y green has been fluctuating rather more, but has mangaged to stay locked. I think the low frequency Y-arm GTRY fluctuations are correlated with the arm cavity alignment drifting around. But the frequent X arm green lock dropouts - not sure what's up with that. Need to look at IR arm control signals and ALS signals at lock drop times to see if there is some info there.
After some more tweaking, I feel like I may be getting closer to a cost-function definition that works.
Some things to figure out:
Five mechcanical traps set inside of boxes. Red-white warning tape on top of each.
Last jump at rack Y2.
MC autolocker got stuck (judging by wall StripTool traces, it has been this way for ~7 hours) because c1psl was unresponsive so I power cycled it. Now MC is locked.
I've been observing this for a few days: ETMX's DC alignment seems to drift by so much that the previously well aligned X arm cavity is now totally misaligned.
The wall StripTool trace shows that both the X and Y arms were locked with arm transmissions around 1 till c1psl conked out - so in the attached plot, around 1400 UTC, the arm cavity was well aligned. So the sudden jump in the OSEM sensor signals is the time at which LSC control to the ETM was triggered OFF. But as seen in the attached plot, after the lockloss, the Oplev signals seem to show that the mirror alignment drifted by >50urad. This level of drift isn't consistent with the OSEM sensor signals - of course, the Oplev calibration could be off, but the tension in values is almost an order of magnitude. The misalignment seems real - the other Oplev spots have stuck around near the (0,0) points where I recentered them last night, only ETMX seems to have undergone misalignment.
Need to think about what's happening here. Note that this kind of "drift" behaviour seems to be distinct from the infamous ETMX "glitching" problem that was supposed to have been fixed in the 2016 vent.
I should've put in the SUSPIT and SUSYAW channels in the previous screenshot. I re-aligned ETMX till I could see IR flashes in the arm, and also was able to lock the green beam on a TEM00 mode with reasonable transmission. As I suspected, this brought the Oplev spot back near the center of it's QPD. But the answer to the question "How much did I move the ETM by" still varies by ~1 order of magnitude, depending on if you believe the OSEM SUSPIT and SUSYAW signals, or the Oplev error signals - I don't know which, if any, of these, are calibrated.
Best to just calibrate the ETM OL in the usual way. I bet the OSEM outputs have a cal uncertainty of ~50% since the input matrix changes as a function of the DC alignment. Still, a 30 urad pitch mis-alignment gives a (30e-6 rad)(40 m) ~ 1 mm beam spot shift. This would be enough to flash other modes, but it would still be easy to lock on a TEM00 like this. I also doubt that the OL calibration is valid outside of some region near zero - can easily check by moving the ETM bias sliders.
What we still don't know is if this is due to Johannes/Aaron working at the ETMX rack (bumping some of the flaky coil cables and/or bumping the blue beams which support the stack). Adding or substracting weight from the stack supports will give us an ETM mis alignment.
This evening I transitioned the slow controls to c1auxex2.
Gautam and I then proceeded to test basic functionality
Arms are locked, have been for ~1hour with no hickups. We will leave it like this overnight to observe, and debug further tomorrow.
Good going Johannes!
I did a cursory check of the ALS signal chain in preparation for commissioning the IR ALS system. The main elements of this system are shown in my diagram in the previous elog in this thread.
Questions I have:
I have just received the scheduling of the PSL self work for tomorrow. Gautam and I agreed that if it is needed I will shut the laser off and cover the hole table with plastic.
I've noticed this a couple of times today - when the autolocker runs the mcdown script, sometimes it doesn't seem to actually change the various gain sliders on the PSL FSS. There is no handshaking built in to the autolocker at the moment. So the autolocker thinks that the settings are correct for lock re-acquisition, but they are not. The PCdrive signal is often railing, as is the PZT signal. The autolocker just gets stuck waiting to re-acquire lock. This has happened today ~3 times, and each time, the Autolocker has tried to re-acquire lock unsuccessfully for ~1hour.
Perhaps I'll add a line or two to check that the signal levels are indicative of mcdown being successfully executed.
After labeling cables I would disconnect, I pulled the box out of the LSC rack. Attachment #1 is a picture of the insides of the box - looks like it is indeed just two lengths of cabling. There was also some foam haphazardly stuck around inside - presumably an attempt at insulation/isolation.
Since I have the box out, I plan to measure the delay in each path, and also the signal attenuation. I'll also try and neaten the foam padding arrangement - Steve was showing me some foam we have, I'll use that. If anyone has comments on other changes that should be made / additional tests that should be done, please let me know.
20180111_2200: I'm running some TF measurements on the delay line box with the Agilent in the control room area (script running in tmux sesh on pianosa). Results will be uploaded later.
For completeness, I'd like to temporarily pull the box out of the rack, open it up, take photos, and make a diagram unless there are any objections.
Some suggestions of checks to run, based on the rightmost colum in the wiring diagram here - I guess some of these have been done already, just noting them here so that results can be posted.
We'd like to setup the recording of the PSL diagnostic connector Acromag channels in a more robust way - the objective is to assess the long term performance of the Acromag DAQ system, glitch rates etc. At the Wednesday meeting, Rana suggested using c1ioo to run the IOC processes - the advantage being that c1ioo has the systemd utility, which seems to be pretty reliable in starting up various processes in the event of the computer being rebooted for whatever reason. Jamie pointed out that this may not be the best approach however - because all the FEs get the list of services to run from their common shared drive mount point, it may be that in the event of a power failure for example, all of them try and start the IOC processes, which is presumably undesirable. Furthermore, Johannes reported the necessity for the procServ utility to be able to run the modbusIOC process in the background - this utility is not available on any of the FEs currently, and I didn't want to futz around with trying to install it.
One alternative is to connect the PSL Acromag also to the Supermicro computer Johannes has set up at the Xend - it currently has systemd setup to run the modbusIOC, so it has all the utilities necessary. Or else, we could use optimus, which has systemd, and all the EPICS dependencies required. I feel less wary of trying to install procServ on optimus too. Thoughts?
There is some problem with the routing of the fast BIO channels through the new chassis - so the ANALOG de-whitening filter seems to be always engaged, despite our toggling the software BIO bits . Something must be wrongly wired, as we confirmed by returning only the FAST BIO wiring to the pre-acromag state (but everything else is now controlled by acromag) and didn't have the problem anymore. Or some electrical connection is not made (I had to use gender changers on these connectors due to lack of proper cabling)
The switches for the QPD gain stages did not work. I suspect a wiring problem, since the switching of the coil enables did work.
Both issues were fixed. In both cases it was two separate causes that prevented them from working.
The QPD gain stage switch software channels were assigned to wrong physical pins of the Acromag, and additionally their DSub cable was swapped with a different one.
The BIO switching had its signal and ground wires swapped on ALL connections, and part of it was also suffering from the cable-mixup.
Both issues were fixed. All backplane signals are now routed through the Acromag chassis.
Gautam and I did notice that occasionally ETMX alignment will start drifting as evident from the OpLev. I want to set up a diagnostic channel to see if the DAC voltages coming from the Acromag are responsible for this.
Measurements for good fit were made. The new shelf will be installed on next Tuesday at 2pm
The reference cavity ion pump is in the way so the cavity will be moved 5" westward. The shelf height space will be 10" Under shelf working height 18" to optical table.
After much googling, I figured out how to install pip on SL7:
sudo easy_install pip
Next, I installed git:
sudo yum install git A
Turns out, actually, pip can be installed via yum using
sudo yum install python-pip
Steve and I removed c1auxex from 1X9 today to make space for the DAQ chassis. Steve installed rails for mounting. To install the box I had to remove all cabling, for which I used the usual precautions (disconnect satellite box etc.)
On reconnect c1auxex2 didn't initialize the physical EPICS channels (the 'actual' acromag channels), apparently it had trouble communicating. A reboot fixed this. It's possible that this is because of the direct cable connection without a network switch that exists
between the Acromags and c1auxex. The EPICS server was started automatically on reboot.
Currently the channel defaults need to be loaded manually after every EPICS server script restart with burt. I'm looking for a good way to automate this, but the only compiled burt binaries for x86 (that we can in principle run on c1auxex2 itself) on the martian network are from EPICS version 3.14.10 and throw a missing shared object error. Could be that simply some path variable is missing.
The burt binaries are not distributed by the lscsoft or cdssoft packages, so alternatively we would need to compile it ourselves for x86 or get it working with the older epics version.
Looks like this problem presisted over the weekend - Attachment #1 is the wall StripTool trace for PSL diagnostics, seems like the control signal to the NPRO PZT and FSS EOM were all over the place, and saturated for the most part.
I traced down the problem to an unresponsive c1iool0. So looks like for the IMC autolocker to work properly (on the software end), we need c1psl, c1iool0 and megatron to all be running smoothly. c1psl controls the FSS box gains through EPICS channels, c1iool0 controls the MC servo board gains through EPICS channels, and megatron runs the various scripts to setup the gains for either lock acquisition or in lock states. In this specific case, the autolocker was being foiled because the mcdown script wasn't running properly - it was unable to set the EPICS channel C1:IOO-MC_VCO_GAIN to its lock acquisition value of -15dB, and was stuck at its in-lock value of +7dB. Curiously, the other EPICS channels on c1iool0 seemed readable and were reset by mcdown. Anyways, keying the c1iool0 crate seems to have fixed the probelm.
While moving the RefCav to facilitate the PSL shelf install, I bumped the power cable to the AOM driver. I will re-solder it in the evening after the shelf installation. PMC and IMC have been re-locked. Judging by the PMC refl camera image, I may also have bumped the camera as the REFL spot is now a little shifted. The fact that the IMC re-locked readily suggests that the input pointing can't have changed significantly because of the RefCav move.
[ Johannes, Rana, Mark and Steve ]
On the second trial the shelf was installed. Plastic cover removed. South end door put back on and 2W Inno turned on.
Shelf 10 " below the existing one: 92" x 30" x 3/4" melamine (or MDF) covered with white Formica. 200 lbs it's max load. Working distance to top of the table 18"
Johannes informed me that he touched up the PMC REFL camera alignment. I am holding off on re-soldering the AOM driver power as I could use another pair of hands getting the power cable disentangled and removed from the 1X2 rack rails, so that I can bring it out to the lab and solder it back on.
Is anyone aware of a more robust connector solution for the type of power pins we have on the AOM driver?
With Johannes' help, I re-installed the box in the LSC electronics rack. In the end, I couldn't find a good solution to thermally insulate the inside of the box with foam - the 2U box is already pretty crowded with ~100m of cabling inside of it. So I just removed all the haphazardly placed foam and closed the box up for now. We can evaluate if thermal stability of the delay line is limiting us anywhere we care about and then think about what to do in this respect. This box is actually rather heavy with ~100m of cabling inside, and is right now mounted just by using the ears on the front - probably should try and implement a more robust mounting solution for the box with some rails for it to sit on.
I then restored all the cabling - but now, the delayed part of the split RF beat signal goes to the "RF in" input of the demod board, and the non-delayed part goes to the back-panel "LO" input. I also re-did the cabling at the PSL table, to connect the two ZHL3-A amplifier inputs to the IR beat PDs in the BeatMouth instead of the green BBPDs.
I didn't measure any power levels today, my plan was to try and get a quick ALS error signal spectrum - but looks like there is too much beat signal power available at the moment, the ADC inputs for both arm beat signals are overflowing often. The flat gain on the AS165 (=ALS X) and POP55 (=ALS Y) channels have been set to 0dB, but still the input signals seem way too large. The signals on the control room spectrum analyzer come from the "RF mon" ports on the demod board, and are marked as -23dBm. I looked at these peak heights with the end green beams locked to the arm cavities, as per the proposed new ALS scheme. Not sure how much cable loss we have from the LSC rack to the network analyzer, but assuming 3dB (which is the Google value for 100ft of RG58), and reading off the peak heights from the control room analyzer, I figure that we have ~0dBm of RF signal in the X arm. => I would expect ~3dBm of signal to the LO input. Both these numbers seem well within range of what the demod board is designed to handle so I'm not sure why we are saturating.
Note that the nominal (differential) I and Q demodulated outputs from the demod board come out of a backplane connector - but we seem to be using the front panel (single-ended) "MON" channels to acquire these signals. I also need to update my Fiber ALS diagram to indicate the power loss in cabling from the PSL table to the LSC electronics rack, expect it to be a couple of dB.
This happened because there are multiple ways to scale the raw value of an EPICS channel to the desired output range. In the CryoLab I was using one way, but the EPICS records I copied from c1auxex were doing it differently. Basically this:
If the "LINR" field is set to "LINEAR", the fields EGUF and EGUL are used to convert the raw value to the channel value VAL. To use them, one needs to enter the voltages that return the maximum and minimum values expected for the given data type. It used to be +10V and -10V, respectively, and was copied that way but that doesn't work with the data type required for the Acromag units. For -some- reason, while the the ADC range is -10V to +10V, this corresponds to values -20000 to +20000, while for the DAC channels it's -30000 to +30000. I had observed this before when setting up the DAQ in the CryoLab, but there we were using "NO CONVERSION", which skips the EGUF and EGUL fields, and used the ASLO and AOFF for manual scaling to get it right. When I mixed the records from there with the old ones from c1auxex this got lost in translation. Gautam and I confirmed by eye that this indeed explains the different levels well. This means that the VMon channelsfor the coils are also showing the wrong voltages, which will be corrected, but the readback still definitely works and confirms that the enable switches do their job.
I am facing two problems:
c1psl, c1susaux, and c1auxey today
I swapped the inputs to the ZHL-3A at the PSL table - so now the X beat RF signals from the beat mouth are going through what was previously the Y arm ALS electronics. From Attachment #1, you can see that the Y arm beat is now noisier than the X. The ~5kHz peak has also vanished.
So I will pursue this strategy of switching to try and isolate where the problem lies...
Somebody had forgotten to turn the HEPA variac on the PSL table down. It was set at 70. I set it at 20, and there is already a huge difference in the ALS spectra
Rendered the SOS assembly (D960001) with correct materials and all and it looks very nice. Will extend this to the building cad later.
[rana, kevin, udit, gautam]
quick notes of some discussions we had today:
RXA: 0805 size SMD thin film resistors have been ordered from Mouser, to be shipped on Monday. **note that these thin film resistors are black; i.e. it is NOT true that all black SMD resistors are thick film**
We use D990694 in various places. Today, Rana alerted me to an important consideration to be kept in mind when we use this board, which I found quite interesting. I still don't understand the problem at the BJT level, but I think one can appreciate the problem without going to the transistor design of the LT1125. I'm attaching an annotated schematic of the whitening section in question. If the following assumptions are valid, then I think my picture is valid.
Then, as one can see in the attached schematic, when we set the gain of any input to <24dB, we must ensure that the input voltage is less than approximately 2V. Otherwise, by asking too much of the first stage op-amp in the quad IC LT1125, we may be messign around with all the 4 op amps in the quad! Even the 0dB setting is not immune to this problem, as it uses one of the 4 op amps.
Now that I think about this a bit more - this problem shouldn't be significant for the usual LSC degrees of freedom when in lock, as the huge DC gain of the loop should squish large DC values of the error signals, and so there shouldn't be any danger of overloading the LT1125. But I don't know if we are being hurt by this effect when flashing through resonances, when the PDH horn-to-horn voltage can be quite high (which is in principle a good thing?). I don't know if there is any "hysterisis" effect where the overloaded quad IC has some relaxation time before it returns to normal operation, and if we are being limited in our ability to catch lock because if this effect.
The concerns remain valid for th ALS demodulated error signals though, for which the signals will remain large throughout.
this is the note from Hartmut Grote on this topic from 2004
After some research: -the- reason for the reduced +/- 20,000 swing in raw values is a default setting for having support for legacy devices enabled when using the acromag proprietary i2o peer-to-peer protocol. So this is doubly unnecessary because a) we don't have any legacy devices at all and b) we're using pure modbus/TCP and no i2o. To change the setting I have to connect via the USB configuration utility. In addition, I want to understand the averaging feature of the acromag units better, which is also configured via USB, and lets one set a fixed amount of samples to be averaged before updating the read-register value. The documentation says that the 8 channels are multiplexed into a single ADC, and that new input data is available after 10 ms for each channel, suggesting a sampling rate of 100 Hz per channel and that the multiplexing happens faster, but is not super-clear about this, so I want to test it in the cryo lab first before unleashing it onto c1auxex2.
Furthermore, the standard timing options for updating epics records are 10s, 5s, 2s, 1s, 0.5s, 0,2s, and 0.1s. On the previous c1auxex, the monitoring channels were set to 0.1s, but that clashes with the 16 Hz global EPICS rate, resulting in partial double-sampling. One can manually provide the option 0.0625s for 16Hz update rate. I will test this and how it deals with the averaging in the cryolab too.
After discussing with Koji, we looked at the aLIGO incarnation of this board. Interestingly, it too has a similar topology of 4 switchable gain stages with gains of 24, 12, 6 and 3dB. The main differences are that they use single Op27 ICs instead of the quad LT1125s, and also, they use a different combination of feedback resistors to realize the various gains.
We considered upping the feedback resistance (R15, R143) on the 24dB gain stage of our boards from (1k, 66.5ohms) to (3k, 200ohms) as on the aLIGO boards - but this doesn't really help? Because KCL demands that the same current flow in R15 and R143, and so the output Vsat of the op amp and its max current driving capabilities in combination determine if the inverting input can follow the non inverting input?
As Hartmut points out in his note, he was able to access the full range of ADC voltages when the gain was set to 3dB, despite the fact that the LT1125 was still getting internally saturated. Operating with minimum 24dB whitening gain doesn't really solve the problem either because the problem just gets shifted to the next gain stage in the chain, and we still have saturation. I also don't have a feeling for how much differential voltage these LT1125s can sustain before they are damaged - I guess the planned THD check will reveal if they are okay or not.
It seems to me like the only way to truly fix this problem of one stage saturating and screwing up the others is to use single Op27s (or equivalent) in place of the quad LT1125s. The aLIGO design also has a series resistance to the non-inverting input - this can help prevent current overdraw from the previous stage (due to a lowered input impedance of the OpAmp - but I wonder how low this can go?).
I have acquired 5 pieces of the Teledyne AP1053 from Koji - these are now at the 40m. I will determine an appropriate location for storage of these and update. We are also looking to acquire 5 more of these. The combination of high power output (26dBm), low gain (10dB), and low noise figure (1.5dB) are quite uncommon in an amplifier and so these should be used only when such properties are required simultaneously.
*Steve informs me that these amps have been stored in the RF cabinet E6 along the east arm.
Steve's note: Teledyne rf amp product selection guide
Teledyne rf low noise amp guide
I did some work on the PSL table today. Main motivations were to get a pickoff for the BeatMouth PSL beam before any RF modulations are imposed on it, and to improve the mode-matching into the fiber. Currently, we use the IR light reflected by the post doubling oven harmonic separator. This has the PMC modulation sideband on it, and also some green leakage.
So I picked off ~8.5mW of PSL light from the first PBS (pre Faraday rotator), out of the ~40 mW available here, using a BS-80-1064-S. I dumped the 80% reflected light into the large beam dump that was previously being used to dump this PBS reflection. Initially, I used a R=10% BS for S-pol that I found on the SP table, but Koji tipped me off on the fact that these produce multiple reflected beams, so I changed strategy to use the R=80% BS instead.
The transmitted 20% is routed to the West edge of the PSL table via 2 1" Y1-1037-45S optics, towards the rough vicinity of the fiber coupler. For now it is just dumped, tomorrow I will work on the mode matching. We may want to cut the power further - ideally, we want ~2.5mW of power in the fiber - this is then divided by 4 inside the beat mouth before reaching the beat PD, and with other losses, I expect ~500mW of PSL power and comparable AUX light, we will have a strong >0dBm beat.
Attachment #1 is a picture of my modifications. For this work, I
I plan to do some characterization of this problem. The plan is to use THD as a metric for whether we are having hidden saturations. Pg 9 of the LT1125 datasheet tells us what fraction of THD to expect. I will use one of the several unused DAC channels available at the LSC rack to drive a 100Hz sine wave into one of the inputs of the whitening chassis, and measure the THD up to a reasonable harmonic number (will probably be set by the ADC noise) for (i) various whitening gain settings and (ii) various input signal amplitudes.
The motivation is to attempt to quantify the problem better:
Then we can decide what, if anything, to do about this issue.
On Friday, while Udit and I were doing some characterization of the EX+PSL IR beat at the LSC rack, I noticed that there were sidebands around the main beat peak at 20dBm lower level. These were offset from the main peak by ~200kHz - I didn't do a careful characterization but because of the symmetric nature of these sidebands and the fact that they appeared with the same offset from the main peak for various values of the central beat frequency, I hypothesize that these are from the modulation sidebands we use for PDH locking the EX laser to the arm cavity. So we can estimate the modulation depth from the relative powers of the main beat peak and the ~200kHz offset sidebands.
Since the IR light is used for the beat and we directly couple it to the fiber to make the beat, there is no green or IR cavity pole involved here. 20dBm in power means . And so the modulation depth, . I will do a more careful meaurement of this, but this method of measuring the modulation depth can give us a precise estimate - for what it's worth, this number is in the same ballpark as the measurement I quote in elog12105.
What is the implication of having these sidebands on our ALS noise? I need to think about this, effectively the phase noise of the SR function generators we use to do the phase modulation of the EX laser is getting imprinted on the ALS noise? Is this hurting us in any frequency range that matters?
I was looking into the physics of polarization maintaining fibers, and then I was trying to remember whether the fibers we use are actually polarization maintaining. Looking up the photos I put in the elog of the fibers when I cleaned them some months ago, at least the short length of fiber attached to the PD doesn't show any stress elements that I did see in the Thorlabs fibers. I'm pretty sure the fiber beam splitters also don't have any stress elements (see Attached photo). So at least ~1m of fiber length before the PD sensing element is probably not PM - just something to keep in mind when thinking about mode overlap and how much beat we actually get.
I replaced the two remaining D-Sub M/M cables that still had gender-changers with M/F cables today, completing the mechanical and wiring work on the ETMX rack. All backplane adapter boards were secured to a cross-strut of the crate using zip ties. This was necessary because the adapter boards don't fit the crate with their panels attached ( the ETMX eurocrate is the only one with slightly different dimensions from all the others), and the we can't mount them to the strut using the panels. This won't be an issue on any of the other crates.
In other news:
I disabled the legacy support in the three Acromag ADC units and set the input averaging to 10 samples via the USB configuration utility. The documentation is unfortunately a little sparse about what this actually means. The manual states that "fresh input data is available to the network every 10ms", so the sampling rate is for sure faster than 100Hz. Since the IOC updates its channels every .1 seconds I assume that an averaging value of 10 to reduce the input noise is safe. The maximum value the configuration tool permits is 200. I tried this using the CryoLab DAQ and set all input channels to 200 and used StripTool to look at the time series of a slow oscillation (.1Hz) with a large amplitude (16Vpp) and looked for missed data points, indicating too long wait times for channels updates. There was no such qualitative difference between 1 sample, 10 samples, and 200 samples, so even pushing the averaging value to max seemed okay. I went with the conservative value of 10 for the ETMX DAQ, but we can likely increase this if noise on the slow inputs becomes an issue.
The input scaling of the ADC channels has been corrected. I changed the conversion method in the EPICS records from manual using the ASLO and AOFF fields to using engineering units via EGUF and EGUL. This required a little attention. The Acromags scale the dynamic input range of +/- 10V to +/- 30,000 raw value, but the EPICS IOC interprets the data type as ranging from -32767 to +32768, so the EGUF and EGUL fields must be set to -10.923 and +10.923 to achieve proper scaling. I also changed the SCAN field on all ADC channels to 0.1 seconds. This has been done for all ADC and DAC channel records.
I compiled the burt binaries on c1auxex2 which took a little fiddling with dependencies and paths but nothing too major. The complete local epics folder (/opt/epics/) which contains the base epics binaries, modbus and burt for 32-bit linux has been copied to the shared drive at /opt/rtapps/epics-3.15.5. They belong to the most recent stable release. This was so we can now automatically call burt after the IOC initialization on c1auxex2 to restore the backed-up channel values.
I also copied the database definition and modbus instruction files to /cvs/cds/caltech/target/c1auxex2, from where they are now being read upon IOC initialization. This is an excerpt of the service file:
#ExecStart=/usr/bin/procServ -f -L /home/controls/modbusIOC/modbusIOC.log -p /run/modbusioc.pid 8008 /opt/epics/modules/modbus/bin/linux-x86/modbusApp /cvs/cds/caltech/target/c1auxex2/ETMXaux2.cmd <-- Contains logging to file, see note 1)
ExecStart=/usr/bin/procServ -f -p /run/modbusioc.pid 8008 /opt/epics/modules/modbus/bin/linux-x86/modbusApp /cvs/cds/caltech/target/c1auxex2/ETMXaux2.cmd <-- Initializes the EPICS IOC with Modbus support
ExecStop=/bin/kill -9 ` cat /run/modbusioc.pid` <-- Kills the detached process by its process ID
ExecStartPost=/bin/bash -c "/opt/epics/extensions/bin/linux-x86/burtwb -f /opt/rtcds/caltech/c1/burt/autoburt/latest/c1auxex.snap" <-- Restores general channel values
ExecStartPost=/bin/bash -c "/opt/epics/extensions/bin/linux-x86/burtwb -f /opt/rtcds/caltech/c1/medm/MISC/ifoalign/burt/ETMX.snap" <-- Restores PIT and YAW values from align MEDM screen
ExecStartPost=/bin/bash -c ". /home/controls/modbusIOC/ETMXaux2.sh" <-- Enables writing to PIT and YAW DAC channels, see note 2)
Note 1) I removed the logging to file for now because I noticed that if there are Acromag communication issues the logfile tends to grow in size VERY fast. In the cryo lab is had gotten to over 70GB just over the winter break. I don't think it's absolutely necessary to have it, and if diagnostics are needed we can easily uncomment it temporarily.
Note 2) I modified the static EPICS records of the four OSEM bias adjust channels so they won't start updating as soon as the IOC starts up (and before the channel defaults are restored by burt). This was done by setting the OMSL (output mode select) field from "closed_loop" to "supervisory". Sample record:
field(DESC,"Bias Adjust for ETMX UL Coil Output")
field(OUT, "@asynMask(C1AUXEX_XT1541A_DAC, 0, -16)MODBUS_DATA")
field(OMSL,"supervisory") <-- Used to be "closed_loop"
field(DOL, "C1:SUS-ETMX_ULBiasSet PP")
Now, on reboort/IOC re-initialization the physical DAC channels are performing a one-time readback of the last stored value in the Acromag's register, then idle until the last StartPost statement executes the script ETMXaux.sh, which changes their OMSL field back to "closed_loop". This causes them to start updating their output from the calc records defined in their DOL field (which have by then recovered their default values curtesy of burt). The result is a smooth transition from idling to the controlled state with no sudden or large offset changes.
The result is a smooth transition from idling to the controlled state with no sudden or large offset changes.
While checking how smooth the transition is we still noticed significant motion of ETMX by looking at the locked green laser and OpLevs. We found that this motion was not caused by interruption of the slow offset adjust, but rather the Watchdog being re-initialized to its OFF state, which cuts the fast channels OFF. On other optics this is observed too, but not as severe. The cause is a rather large offset on the LR coil coming from the fast DAQ, which was reported as 50mV by the slow readback channel (while other readback values are <10mV). It is present even when turning the output of the CDS model OFF, but vanishes when the watchdog is triggered. This helped us trace it to an offset of the DAC output itself: it is present at the output of the AI board but vanishes when the DAC is disconnected. The actual offset is ~40mV, as opposed to other channels on the same board, which ahve offsets in the range 3-7mV.
While we can compensate for this offset in software - it made us wonder if the DAC channel is somehow busted and if that's what causing the 'wandering' of ETMX that we have been observing recently. There are two free DAC channels on the AI chassis that has the side coil and the green temperature control signals. We could re-route the LR signal through a different DAC channel to fix this.
gautam: 40mV offset at the AI board output gets multiplied by 3 in the dewhitening board, so there is a 120mV DC offset going to the coil (measured at dewhite board output with DMM). The offset itself isn't hurting us, but the fact that it is several times larger than other channels led us to wonder if it could be drifting around as well. From my SOS pitch balancing forays, in my head I have the number 30mrad as being the full range of the OSEM actuation - so if the offset swings by 120mV, that's ~150urad of motion, which is quite large, and is of the order of magnitude I'm used to seeing ETMX move around by.
M4 local earthquake at 10:10 UTC There is no sign of damage.
....here is an other one.........M5.8 Ferndale, CA at 16:40 UTC
We started the actual heating test today and it seems to be working so far. Hoping to heat it to about 40C. We also set up another temperature sensor to measure the lab temperature and connected it to J7, bottom.
Gautam and I set up the insulated seismometer can in the lab today. I had previously wired up the two heaters I placed onto the sides of the can in parallel to get a total resistance of 12.5 ohms and then I wrapped the whole can in 3 layers of insulation (k-factor 0.25). We placed it on a large sheet of insulation as to not crush the wires leading out the bottom of the can. I stuck on one of my AD590 sensors to the inside of the can onto the copper lining using duct tape, though this is only a temporary solution. In the future, it would be nice to have some sort of thermal clamp to secure the sensor to the can. To provide power to the heater circuit board and the temperature sensor board, we got a powerstrip and plugged in two power supplies and a function generator into it. The heater circuit (attachment 3) is powered by one of the power supplies and the function generator, while the temperature sensor (attachment 5) is stuck to the side of the can and is powered by the second power supply. The heater circuit's MOSFET (IRF640, attachment 4) is placed on a metal block and sandwiched between two more to make sure it doesn't move around. The temperature sensor is connected by a long BNC cable to the channels in attachment 6.
gautam: we plugged the BNC output of Kira's temperature sensor circuit to J7 on the AA input chassis in 1X2 - this corresponds to ADC1 input 12 in c1ioo. I then made a "PEM" namespace block inside the c1als model, and placed a single CDS filter module inside it (this can be used for calibration purposes). The filter module is named "C1:PEM-SEIS_EX_TEMP", and has the usual CDSfilt channels available. I DQ'ed the output of the filter module (@256 Hz, probably too high, but I'm holding off on a recompile for now). Recompilation and model restart of c1als went smoothly.
2 bench power supplies are being used for this test, we can think of a more permanent solution later.
**25 Jan noon: Added another filter module, "C1:PEM-SEIS_EX_TEMP", to which Kira is hooking up a second temperature sensor, which will serve as a monitor of the "Ambient" lab temperature. Added DQ channel for the output of this filter module, fixed sampling to 32Hz. Compile and restart went smooth.
I was looking at this a little more closely. As I understand it, the purpose of the audio differential IF amplifier is:
Attachment #1 shows, the changes to the TF of this stage as a result of changing R19->50ohm, R17->500ohm. For the ALS application, we expect the beat signal to be in the range 20-100MHz, so the 2f frequency component of the mixer output will be between 40-200MHz, where the proposed change preserves >50dB attenuation. The Q of the ~500kHz resonance because of the series LCR at the input is increased as a result of reducing R17, so we have slightly more gain there.
At the meeting yesterday, Koji suggested incorporating some whitening in the preamp itself, but I don't see a non-hacky way to use the existing PCB footprint and just replace components to get whitening at audio frequencies. I'm going to try and measure the spectrum of the I and Q demodulated outputs with the actual beat signal to see if the lack of whitening is going to limit the ALS noise in some frequency band of interest.
Does this look okay?
The demod circuit board is configured to have gain of x100 post demod (conversion loss of the mixer is ~-8dB). This works well for the PDH cavity locking type of demod scheme, where the loop squishes the error signal in lock, so most of the time, the RF signal is tiny, and so a gain of x100 is good. For ALS, the application needs are rather different. So we lowered the gain of the "Audio IF amplifier" stage of the circuit from x100 to x10, by effecting the resistor swaps 10ohms->50ohms, 1kohm->500ohms (more details about this later).
After almost 3 hours the temperature rose by about 3.5C. Seems a bit slow, but we can drive it more if necssary. The heating curve itself is exponentiial, which is a good sign.
The final temperature reached in about 4.5 hours is 30.5C, while the starting temperature is about 24C. I can't seem to screenshot the data for some reason.
Also, I will calibrate the lab temperature sensor to Celcius in the near future so that we would have a working sensor inside the lab.
I tried to couple the PSL pickoff into the fiber today for several hours, but got nowhere really, achieved a maximum coupling efficiency of ~10%. TBC tomorrow... Work done yesterday and today:
I think part of the problem was that the rejected beam from the PBS was not really very Gaussian - looking at the spot on the beam profiler, I saw at least 3 local maxima in the intensity profile. So I'm now switching strategies to use a leakage beam from one of the PMC input steering optics- this isn't ideal as it already has the PMC modulation sideband on it, and this field won't be attenuated by the PMC transmission - but at least we can use a pre-doubler pickoff. This beam looks beautifully Gaussian with the beam profiler. Pics to follow shortly...
I tried to couple the PSL pickoff into the fiber today for several hours, but got nowhere really, achieved a maximum coupling efficiency of ~10%. TBC tomorrow... Work done yesterday and today