CDSutlils has been updated to the newest version, 474; there are some matrix interface methods that will make our locking scripts easier to read, modify, and maintain.
I've tested the ALS and CARM down scripts, and the LSC offsets script, and they all work fine.
I'm seeing LOTS OF STUFF NOT CHECKED INTO THE SVN!!! both modified things that haven't been updated, and things that looked like they haven't been checked in at all.
Look through EVERYTHING that you think you might have touched, or even care about, and make sure it's checked in.
I have added a switch in series with the WFS_GAIN. And I have also added a LKIN_OUT_MATRX between the lockin-outputs and the MC suspensions. This will enable us to drive the MC mirrors in any combination so that we can (in principle) attain pure translations and rotations of the MC axis.
I will compile the model later during the day. This is just in case anyone one else were to compile c1ioo.mdl before then.
We tested the CM board by implementing the high bandwidth IR lock (single arm). In preparation for this test we temporarily connected the POY11_Q_MON output to the CM board IN1 input and checked the YARM POY transfer function by running the AA_YARM_TEMPLATE under users/Templates/LSC/LSC_loops/YARM_POY/. We made sure the YARM dither optimized TRY so as to maximize the optical gain stage. Then we proceeded as follows:
Ultimately, our ability to progressively increase the control bandwidth of the YARM is a proxy that the CM board is working properly. Attachment 1 shows the OLTF progression as we increased the loop's UGF. Note how as we approached the maximum measured UGF of ~ 22 kHz, our phase margin decreased signifying poor stability.
At the end of this measurement, at about ~ 15:45 I restored the CM board IN1 input and disconnected the POY11_Q_MON
gautam: the conclusion here is that the CM board seems to work as advertised, and it's not solely responsible for not being able to achieve the IR handoff.
Koji pointed out an important subtlety pertaining to the "LATCH ENABLE" signal line on the CM board. The purpose of this line is to smoothly facilitate the transition of a change in the "multi-bit-binary-outputs", a.k.a. "mbbo", that are controlled by MEDM gain sliders, to the analog electronics on the CM board. Why is this necessary? Imagine changing the gain from 7dB (=0111 in mbbo representation) to 8dB (=1000 in mbbo representation). In order to realize this change, all 4 bits have to change their state. But this almost certainly doesn't happen synchronously, because our EPICS interface isn't synchronous. So at some intermediate times, the mbbo representation could be 0100 (=4dB), or 1111 (=15dB), or many other possible values, which are all significantly different from either the initial value or the desired final state. This is clearly undesirable.
In order to protect against this kind of error, a Latched output part, 74ALS573, is used to buffer the physical digital logic levels from the switches in the analog gain stages. So in the default state, the "LATCH ENABLE" signal line is held "LOW". When a change happens in the EPICS value corresponding to a gain slider, the "LATCH ENABLE" state is quickly toggled to "HIGH", so as to enable the appropriate analog gain stages to be switched, and then again to "LOW", at which point the latch holds its output state. This logic is currently implemented by a piece of code called "latch.o", which is the compiled version of "latch.st", which may be found in /cvs/cds/caltech/target/c1iool0 where it presumably was written for the IMC servo board, but not in /cvs/cds/caltech/target/c1iool0 , which is where the CM board database files reside. The only elog reference I can find pertaining to this particular piece of code is from Alan, and doesn't say anything about the actual logic.
For the new c1iscaux, we need to implement this logic somehow. After discussion between Koji and me, we feel that a piece of python code is sufficient. This would continuously run in the background on the supermicro server machine. The channel hierarchy for each gain channes is as follows (I've taken the example of C1:LSC-CM_REFL1_GAIN):
So the logic will be that it continuously scans the EPICS channel C1:LSC-CM_REFL1_GAIN for a change in set value. When a change is detected, it has to update the C1:LSC-CM_REFL1_SET channel. In the next EPICS refresh cycle, this would result in the mbbo bits, C1:LSC-CM_REFL1_BITS , all changing to the appropriate values. After these changes have happened, we need to toggle the LATCH ENABLE in order to allow the changes to propagate to the analog gain stage switches. Need to think about what's the best way to do this.
DATED, SEE ELOG14941 for the most up-to-date info on latch.py.
I wrote (/cvs/cds/caltech/target/c1iscaux3/latch.py) and tested the logic illustrated in Attachment #1. Results of a test are shown in Attachment #2, the various channels change as expected. Note that for negative values of the gain channel, the corresponding "BITS" channel will take on values like 65536 - this is because the mbboDirect data type is a 16 bit data type, and presumably the MSB is the sign bit. A bit mask is applied to this channel before the actual BIO unit bits are set - we should verify that the correct behavior happens, but I don't immediately see any problems.
To me, this is a robust logic, but it will benefit from more sets of eyes giving it a look over. The idea is to run this continuously on the Supermicro machine.
Apart from this, I also fixed some errors in the mbboDirect record syntax - so now I am able to start up the EPICS server without it throwing any error messages. It remains to verify that changing an EPICS gain slider results in the appropriate gain bits being flipped in the correct way (on the hardware side, I think the correct behavior is happening on the software end). For this testing, I turned off the old c1iscaux crate at ~10am, and started up the server on c1iscaux3. I am reverting to the nominal config now (~1pm).
Further testing will require the wiring inside the Acromag chassis to be completed. This should be the priority task for next week.
*Update 1130 22 July 2019: I've now installed the required dependencies on c1iscaux3 and setup the latch.py script to run as a systemctl process dependent on modbusIOC.service.
As part of checking the common mode board before we get too carried away with using it, I looked at the time series of the AO servo output when I turned on various boosts, or changed gain values. As it turns out, basically anything that I did caused glitches. Oooops.
I plugged a function generator to the IN1 port of the CM board, with a freq of 400Hz, and a voltage of 10mVpp (which is the smallest value that it would allow). I plugged the BNC version of the servo output into a 300MHz 'scope.
First I looked at "boost" and "super boost", and then I looked at various steps of the AO gain slider. For all of the button presses that gave me glitches, I saved .png's of the 'scope screen (on a floppy, so I'll have to fetch the data tomorrow...).
Both enabling, and disabling the "Boost" button gave me glitches.
For "Super Boost", I saw glitches for all of the steps, 0->1, 1->2, 2->3.
For the AO path, I only started at 0dB, and only captured screenshots of glitches when I increased the gain, since presumably that's when we'll care the most during acquisition. I found that going down in gain caused glitches at every step! For increasing the gain, steps from an odd number of dBs to an even number consistently caused glitches. Steps from an even number to an odd number occasionally caused glitches, but they weren't very common. For the steps that did cause glitches, some were worse than others (7dB to 8dB, 15 dB to 16 dB, and 23 dB to 24 dB seemed the worst.)
After my work, I put all of the cables back, so that we should be ready to utilize the CM board for locking this evening.
For posterity, here are the notes that I took while I was working - I'll make them more coherent when I fold them in with my images tomorrow. The "first .png, next, etc." are because the 'scope numbers them in order as a default.
1st png = boost enable, then disable
2nd png = super boost, start at 0, then 1, then 2, then 3
3rd png = AO gain from 1 to 0
4th is AO gain from 0 to 1 (happens less often than 1->0, which is every time I get a glitch)
Next is AO gain 1->2, got 2 glitches!
3->2 glitch often, 2->3 much less often
next is 2->3
next png is 3->4, 2 glitches with weird dip
next png is 5->6
6->7 is rare
next png is 7->8, which is nasty!!
8->9 is rare
10->11 is rare
png 11-> 12, 3 glitches
png 13->14, 2 glitches
png 15->16, kind of nasty
png 17->18, 2 glitches
png 19->20, 3 glitches
png 21->22, 2 glitches
png 23->24, kind of nasty
png 25->26, 2 glitches
png 27->28, 3 glitches, at least
png 29->30, 2 glitches
Somehow, the images got put into a whole new entry, even though I thought I was editing this one. Anyhow, please see elog 9938.
Note: I thought I was editing elog 9935, but somehow this became a whole new entry. Either way, all the info is in here.
The screenshot of the Boost enable / disable I'll have to re-take. Apparently I instead caught a screenshot of the list of files on the floppy...ooops.
This is a shot of enabling the Super Boosts. At the beginning, it's at "0", so no superboosts (also, regular boost was off). Then, I switch to "1", and the trace gets a little fuzzy. Then I switch to "2", and it gets very fuzzy. Then I switch to "3", and a lot of the fuzz goes away. There's a glitch at each transition.
The following screenshots are all of various steps of the AO gain slider. For all of these, both the "boost" and "super boosts" were off. Each screenshot is a single gain step, even if there are several glitches captured.
First, 0dB to 1dB:
Next, 1dB to 2dB:
2dB to 3dB:
3dB to 4dB:
While increasing the gain, I didn't find any more steps from an even to an odd number where I got a glitch. They would glitch when I undid that step (decreased the gain), but over ~5 trials for each increase, I didn't ever catch a glitch. The odd to even steps still had glitches while increasing the gain though.
5dB to 6dB:
7dB to 8dB:
9dB to 10dB:
11dB to 12dB:
13dB to 14dB:
15dB to 16dB:
17dB to 18dB:
19dB to 20dB:
21dB to 22dB:
23dB to 24dB:
25dB to 26dB:
27dB to 28dB:
29dB to 30dB:
I think the boosts that are currently stuffed on the CM board are too aggressive to be usable for locking the interferometer. I propose some changes.
[CM board schematic]
[CM board transfer function measurement]
[Measurement of the AO path TF]. Empirically, I have observed that the CARM OLTF has ~90 degrees phase margin available at the UGF when no boosts are engaged, which is consistent with Koji's measurement. Assuming we want at least 30 degrees phase margin in the final configuration, and assuming a UGF to be ~10 kHz, the current boosts eat up way too much phase at 10 kHz. Attachment #1 shows the current TFs (dashed lines), as the boosts are serially engaged. I have subtracted the 180 degrees coming from the inverting input stage. The horizontal dash-dot line on the lower plot is meant to indicate the frequency at which the boost stages eat up 60 degrees of phase, which tells us if we can meet the 30 degree PM requirement.
In solid lines on Attachment #1, I have plotted the analogous TFs, with the following changes:
These changes will allow possibly two super boosts to be engaged if we can bump up the CARM UGF to ~15 kHz. We sacrifice some DC gain - I have not yet done the noise analysis of the full CARM loop, but it may be that we don't need 120 dB gain at DC to be sensing noise limited. I suppose the pole frequencies can also be halved if we want to keep the same low frequency gain. In any case, in the current form, we can't access all that gain anyways because we can't enable the boosts without the loop going unstable.
The input referred noise gets worse by a factor of 2 as a result of these changes, but the IN1 gain stage noise is maybe already higher? If this sounds like a reasonable plan, I'll implement it the next time I'm in the lab.
We just changed the input to the CM board from REFL11 to AS55.
In preparation for trying out some high-bandwidth Y arm cavity locking using the CM board, I hooked up the POY11_Q_Mon channel of the POY11 demod board to the IN1 of the CM board (and disconnected the usual REFL11 cable that goes to IN1). The digital phase rotation for usual POY Yarm locking is 106 degrees, so the analog POY11_Q channel contains most of the signal. I then set the IN1 gain of the servo to 0dB, and looked at the CM_Slow signal - I changed the whitening gain of this channel to +18dB (to match that used for POY11_I and POY11_Q), and found that I had to apply a digital gain of 0.5 to get the PDH horns in the usual POY11_I signal and the CM_Slow signal to line up. There was also a sign inversion. Then I was able to use the digital LSC system and lock the Y arm cavity length to the PSL frequency by actuating on ETMY, using CM_Slow as an error signal. A comparison of the in-loop POY11_I ASD when the arm is locked is shown in Attachment #1 - CMslow seems to be dominated by some kind of electroncis noise above ~100 Hz, so possibly needs more whitening (even though the nominal whitening filter is engaged)?
Anyway, now that I have this part of the servo working, the next step is to try and engage the AO path and achieve a higher BW lock of the Y arm cavity to the PSL frequency (= IMC length). Maybe it makes more sense to actuate on MC2 for the slow path.
One of the differences between the direct POY and the CM_SLOW POY is the presence of the CM Servo gain stages. So this might mean that you need to move some of the whitening gain to the CM IN1 gain.
Tonight we worked on the CM board and AO path:
The BLUE plot is at MC Gain = 0.10 and REFL1 Gain = 4dB; the GREEN plot is for MC Gain = 0.10 and REFL1 Gain = 3dB, which seemed a more stable configuration; after this last configuration, we increased the MC Gain to 0.15 and the AO Gain from 8dB to 9dB and took another measurement, the RED plot; this is as far as we got as of now. We also couldn't increase the REFL11 Gain because it made things unstable and more prone to unlock.
So, some little progress on the AO path procedure, but we are very low on our UGF and we have to find a way to increase our gains without breaking the lock and avoiding the gain peaking we have witnessed tonight.
I have removed REFLDC and the SR560 offsetter from the CM board IN2. Now, analog AS55 I lives there, for our single arm testing. (Analog I has more of the single arm Y PDH signal in it). REFL11 has been reconnected to IN1.
With ITMX super misaligned, Diego and I locked the Y-arm with the AO path on AS55, ultimately at 4kHz bandwidth, but with plenty of gain margin. We didn't allocate the gains too intelligently, and had the CM board input gain slider maxed out, but plenty of headroom in the digital and AO sliders, making it inconvenient to up the UGF even more, to engage the super boosts. However, since this is just a test case to make sure we still can AO lock, I'm not too worried about this.
Since LSC FMs and such had changed around, old recipies didn't neccesarily work 1:1. Diego is writing a script for the current recipe, and will post an elog with the steps.
Gains and signs are able to be tracked by loop TFs, the real sticking point is a stable crossover. We used the 1.6k:80 hardware filter in the CM board to give the AO Path a 1/f shape in the crossover region, and undid it digitally in the CM_SLOW input FM. However, we do use a 300:80 in the MC2 sus FM to make the digital loop like 1/f^2 around the crossover, once a little bit of AO has come in to pull up the digital loop's phase. We used the CARM filter bank to do this, so I think we should be able to use a similar technique to do it in the PRFPMI case, as long as the coupled cavity pole is around ~100Hz.
Attached are a few OLTFs from the progression.
I wrote the script with the recipe we used, using the Yarm and AS55 on the IN2 of the CM board; however, the steps where the offset should be reduced are not completely deterministic, as we saw that the initial offset (and, therefore, the following ones) could change because of different states we were in. In the script I tried to "servo" the offset using C1:LSC-POY11_I_MON as the reference, but in the comments I wrote the actual values we used during our best test; the main points of the recipe are:
I tried the procedure and it seems fine, as it did during the tries Q and I made; however, since it touches many things in many places, one should be careful about which state the IFO is into, before trying it.
The script is in scripts/CM/CM_Servo_OneArm_CARM_ON.py and in the SVN.
The logic chips 74ALS573 were replaced. And now the gain sliders are working properly.
== Test Status ==
[done] Whitening gain switching test
[done] AA enable/disable switching
[0th order] LO Det Mon channel check
[none] PD I/F board check
[done] QPD I/F board check
[done] CM Board
[none] ALS I/F board
Last week we found that the logic chip for the REFL1 gain switching was not transmitting the input logic. I went to Downs and obtained the chips. After some inspection some other latch chips were suspicious. Therefore U46, U47, and U48 (#1, #3, and #4 from the top) were replaced. After the replacement, the gain measurements were repeated. This time the test for the AO gain was also performed. Now all three slideres show the gain as expected except for the consistent -0.2dB deficit.
Note that the transfer functions for the REFL gains were measured with the input at IN1 or IN2 and the output at TESTA1. The TFs for the AO gain was measured with the excitation at EXC B, the input at TESTB2 and the output at the SERVO output. The gain and phase variantions for the AO gain at low frequency is the effect of AC coupling existing between the excitation and the servo output.
[Update on Oct 14, 2019]
The measured transfer functions show the phase delay determined by the opamps involved. The phase delay well below the pole frequencies can be represented well by a simple time delay (a phase delay linear to the frequency). Attachment 7 shows the time delay estimated by LISO for each gain setting of each gain stage. REFL2 has particularly large phase delay because of the use of OP27s. The delay is even larger when the gain is high presunmably because of the limited GBW.
The boost filters of the CM servo board were tested. Their ZPK models were made.
The transfer functions of the boost filters were measured with the SG output of a SR785 connected to IN1. The IN1 gain was set to be 0dB. The transfer function was taken between the IN1 input and the TEST1A output.
With no boost and normal boost, the input signal amplitude was fixed to 20mVpk. For the other boosts, however, I could expect large gain variation through a single sweep. Therefore automatic SG amplitude tracking was used. The target was to have the output to be 1V with maximum amplitude of 100mV.
Attachment 1 shows the measured transfer functions.
The pole and zero frequencies of the boosts were estimated using LISO. Here the TFs were normalized by the TF of 'no boost' to cancel the delay of the other stages including that of the monitor channel.
ZPK model of Normal Boost:
ZPK model of Super Boost (State1):
ZPK model of Super Boost (State2):
ZPK model of Super Boost (State3):
CM Board Slow out (digital length control) path transfer function / pole-zero filter pair (79Hz/1.6kHz) transfer function
The excitation was given from EXC A. The denominator was TESTA2, and the numerator was OUT1.
Attachment 1 shows the measured transfer function with and without PZ filter off and on. The PZ filter provides ~26dB attenuation at high frequency. The output stage has a single order 100kHz LPF and it is visible in the transfer function.
The transfer function without the PZ filter was modelled by LISO as the following PZK representation. There looked a small step in the TF which caused the additional PZ pair (66~67Hz) but has very minor effect in the mag and phase.
The transfer function of the PZ filter was separately analyzed. The TF with the switch ON was normalized by the one with the switch OFF. Thus it revealed the pure effect of the switch. The PZK model of the stage was estimated to be
For the CM board modeling purpose, the transfer function from TESTA2 to TESTB2 was needed. (Attachment 1)
The ZPK model of this part is
The output stage (and AO GAIN stage) of the MC board was modelled. The transfer function was measured with the injection from EXC B. The denominator was TESTB2, and the numerator was SERVO OUT.
This stage is AC coupled by 2x 1st order HPFs. Firstly, this transfer function was measured with AO GAIN set to be 0dB. (Attachment 1)
This TF was used to characterize the cutoffs of the HPF stages, represented as the following ZPK:
Then the AO GAIN was already measured as seen in [ELOG 14948]. The AO gain TF was then modeled by LISO with the above HPF as the preset. This allows us to characterize the time delay of the AO GAIN part.
Input referred offsets on the IN1/IN2 were tested with different gain settings. The two inputs were plugged by the 50 ohm terminators. The output was monitored at OUT1 (SLOW Length Output). The fast path is AC coupled and has no sensitivity to the offset.
There is the EPICS monitor point for OUT1. With the multimeter it was confirmed that the EPICS monitor (C1:LSC-CM_REFL1_GAIN) has the right value except for the opposite sign because the output stage of OUT1 is inverting. The previous stages have no sign inversion. Therefore, the numbers below does not compensate the sign inversion.
Attachment 1 shows the output offset observed at C1:LSC-CM_REFL1_GAIN. There is some gain variation, but it is around the constant offset of ~26mV. This suggested that the most of the offset is not from the gain stages but from the later stages (like the boost stages). Note that the boost stages were turned off during the measurements.
Attachment 2 shows the input refered offset naively calculated from the above output offset. In dependent from which path was used, the offset with low gain was hugely enhanced.
Since the input referred offset without subtracting the static offset seemed useless, a constant offset of -26mV was subtracted from the calculation (Attachment 2). This shows that the input refered offset can go up to ~+/-20mV when the gain is up to -16dB. Above that, the offset is mV level.
I don't think this level of offset by whichever OP27 or AD829 becomes an issue when the input error signal is the order of a volt.
This suggests that it is more important to properly set the internal offset cancellation as well as to keep the gain setting to be high.
== Test Status ==
The photos of the latest board can be found as Attachments 3/4
With some input signals, the functionarities of the CM servo switches were tested.
After the tests the LSC cables were reconnected (Attachment 6)
Now the module is inserted at the 2nd crate from the top of 1Y2 (LSC analog rack). It is next to the DCPD whitening module.
I found the backplane cable for the Common Mode servo module.
I traced a cable form XY220 at the right most module on the crate where iscaux2 is living.
This cable was connected to the upper backplane connector.
Switching of the module is tested. All the switches and gain control are doing their job.
It was found that the offset and slow readback are not responding.
I checked the schematic of the CM servo module (D040180).
It seems that there is another cable for the offset and read back voltages.
I found another backplane cable for the CM servo module. It is plugged to the module now.
I can see that C1:LSC-CM_SLOW_MON is responding to C1:LSC-CM_REFL_OFFSET.
But C1:LSC-CM_SUM_MON and C1:LSC-CM_FAST_MON are not replated to the given offset.
I probably need to check the cross connects.
Den just plugged an output from the common mode board into an LSC whitening board (the spare channel that used to be called "PD_XXX_I" in the LSC model). I have modified the LSC model to reflect the new name of the new signal ("CM_SLOW"), and have added this channel to the LSC input matrix. Koji is, I believe, adding this channel to the LSC screen in the auxiliary error signals section. I am also adding the _OUT of the filter bank to the DAQ channels block.
We spent some time tuning CM slow servo such that fast path would be stable in the AO gain range -32db -> 29dB (UGF=20kHz) when all boosts are turned off and common gain is 25dB. Current filters that we use for locking are not good enough - AO can not be engaged due to oscillations around 1kHz. This is clearly seen from slow path closed loop transfer function. I will attach servo shapes tomorrow.
Attached plot "EOM" shows EOM rms voltage while changing AO gain from -10dB to 4dB. For UGF of 20kHz we need AO gain of 29dB.
It seems we can start using CM servo for CARM offset but the sensor should be at least factor of 30 better than POY. Add another factor of 10 if we would like to use BOOST 2 and BOOST 3.
I've changed the LSC rack wiring a little bit, to give us some flexibility when it comes to REFL11.
Previous, the REFL11 demod I output was fed straight to the CM servo board, and the slow CM board output was hooked up to the REFL11I ADC channel. Thus, it wasn't really practical to ever even look at sensing angles in REFL11, since the I and Q inputs were subject to different signal paths/gains. (Also, doing LSC offsets would do wonky things to refl11 depending on the state of the switches on the CM board screen.)
Thus, I've hooked up the CM board slow output into the the previously existing, aptly named, CM_SLOW channel. The REFL11 demod board I output is split to IN1 of the CM board, and the REFL11 I ADC channel.
So, there is no longer hidden behavior in behind the REFL11 input filters, channels are what they claim to be, and the CM board output is just as easily accessible to the LSC filters as before.
The images that I just posted were taken with the CMOS camera. We switched from the CCD to the CMOS because the CCD was exhibiting much higher blooming effects. Unlike the CCD, there is a slight background structure if you look carefully in the amplitude image, but I can correct for this consistent background by taking a uniformly exposed image by placing a convex lens in front of the CMOS. I will then divide each frame taken of the laser wavefront by the background image.
I've set up network with a CNN encoder (front end) feeding into a single LSTM cell followed by the output layer (see attachment #1). The network requires significantly more memory than the previous ones. It takes around 30s for one epoch of training. Attached are the predicted yaw motion and the fft of the same. The FFT looks rather curious. I still haven't done any tuning and these are only the preliminary results.
Rana also suggested I try LSTMs today. I'll maybe code it up tomorrow. What I have in mind- A conv layer encoder, flatten, followed by an LSTM layer (why not plain RNNs? well LSTMs handle vanishing gradients, so why the hassle).
Well, what about the previous conv nets?
What I did:
What I observed:
What I think is going wrong:
What I will try now:
I did a whole lot of hyperparameter tuning for convolutional networks (without 3d convolution). Of the results I obtained, I am attaching the best results below.
The lower the power of the error signal (difference between the true and predicted X and Y positions), essentially mse, on the test data, the better the performance of the model. Of the trained models I had, I chose the one with the lowest mse.
(Note: Attachment 6 and 7 present information regarding a fraction of the data. However, the behaviour remains the same for the rest of the data.)
Observations and analysis:
P.S. I will also try the 2D convolution followed by the 1D convolution thing now.
P.P.S. Gabriele suggested that I try average pooling instead of max pooling as this is a regression task. I'll give that a shot.
Experiment file: train_both.py
See Attachment #2.
Make the MSE a subplot on the same axes as the time series for easier interpretation.
Do I need to provide any more details here?
Describe the training dataset - what is the pk-to-pk amplitude of the beam spot motion you are using for training in physical units? What was the frequency of the dither applied? Is this using a zoomed-in view of the spot or a zoomed out one with the OSEMs in it? If the excursion is large, and you are moving the spot by dithering MC2, the WFS servos may not have time to adjust the cavity alignment to the nominal maximum value.
What is the minimum detectable motion given the CCD resolution?
see attachment #4.
I wrote what I think is a handy script to observe if the frames are saturated. I thought this might be handy for if/when I collect data with higher exposure times. I assumed there was no saturation in the images because I'd set the exposure value to something low. I thought it'd be useful to just verify that. Attachment #3 has log scale on the x axis.
What is the significance of Attachment #6? I think the x-axis of that plot should also be log-scaled.
At the lab meeting today, Rana suggested that I use the Pylon app to collect more data if that's what I need. Following this, Jon helped me out by updating the pylon version and installing additional software to record video. Now I am collecting data at
Consequently I have dithered the MC2 optic from around 9:00 PM.
Today Joe and I undertook a FULL rebuild of all front end systems with the head of the 2.1 branch of the RCG. Here is the full report of what we did:
--- Makefile (revision 2451)+++ Makefile (working copy)@@ -346,7 +346,7 @@ #MDL_MODELS = x1cdst1 x1isiham x1isiitmx x1iss x1lsc x1omc1 x1psl x1susetmx x1susetmy x1susitmx x1susitmy x1susquad1 x1susquad2 x1susquad3 x1susquad4 x1x12 x1x13 x1x14 x1x15 x1x16 x1x20 x1x21 x1x22 x1x23 #MDL_MODELS = $(wildcard src/epics/simLink/l1*.mdl)-MDL_MODELS = $(shell cd src/epics/simLink; ls m1*.mdl | sed 's/.mdl//')+MDL_MODELS = $(shell cd src/epics/simLink; ls c1*.mdl | sed 's/.mdl//') World: $(MDL_MODELS) showWorld:
cd src/gdsmakecp awgtpman /opt/rtcds/caltech/c1/target/gds/bin
The c1iscex models (c1x01 and c1scx) did not come back up. c1x01 was running long on every cycle, until the model crashed and brought down the computer. After many hours, and with Alex's help, we managed to track down the issue to a patch from Rolf at r2361. The code included in that patch should have been wrapped in an "#ifndef RFM_DIRECT_READ". This was fixed and committed to branches/branch-2.1 at r2460 and to trunk at r2461.
Everything is now green, and things seem to be working. Mode cleaner is locked. X arm locked.
I'm planning to start removing components from the X endtable tomorrow morning at ~10AM - if anyone thinks I should hold off and do some further checks/planning, let me know before this so that I can do the needful.
It seems to work, sort of.
I added COMSOL example files to the 40m svn to demonstrate how to make transfer function measurements in COMSOL.
The directory also contains an (incomplete) explanation of the method in a PDF file.
WE tried to use the new conlog today and discovered that:
1) No one at the 40m uses conlog because they don't know that it ever ran and don't know how to use regexp.
2) It has not been running since the last time Megatron was rebooted (probably a power outage).
3) We could not get it to run using the instructions that Syracuse left in our wiki.
Emails are flying.
Often people say "I don't use conlog because its real slow". Its a little like not driving because your car has no gas.
I looked into what's going on with conlog. No one has fixed its channel list in ~1 year so it didn't make much sense. Also since Oct 1 of this year, it expired the leap seconds epoch and has been waiting for someone to look at the log file and update the list of leap seconds.
There are a bunch of bad channels which are screwing up various tools (DV, DTT, etc.):
Examples: C1:LSC-Subsystem_NPRO_SW1, C1:-DOF2PD_MTRX_0_0_SW1, C1:BAD-BAR_CRAZY_2_RSET, C1:C1L-DOF2PD_MTRX_3_14_SW2, C1:DUB-SEIS_GUR2_Z_LIMIT, etc.
To fix up some of these issues, I have deleted several MEDM directories which I thought were old (there are several extras left from Aidan's Green time). I also have put a bunch of exclude variables into the conlog 'scan_adls' script to prevent it from adding some of the new worthless channels. Finally, I have started this command
../bin/strip_out_channels '.*STAT.*','.*_ALIVE.*','C1:PEM.*','.*_Name.*','C1:UCT.*','C1:MCP.*','C1:SP.*','C1:DU.*','C1:RF.*','C1:NIO.*','C1:TST.*','C1:SUP.*','C1:X.*','C1:FEC.*','.*_LFSERVO.*','.*FSS_SLOWDC.*','C1:LSC-LA_MTRX_21','C1:LSC-PD.*OFFSET','C1:LSC-ETM.*OFFSET' conlog*.log
which should strip lots of the excess conlog data out of the conlog directory. The only downside is that its setting all of the timestamps of the .log files to today instead of the historical times but I don't think we'll care about this too much. Hopefully it will speed things up to have less than 450 GB of conlog files...
update: 12 hours later, its still running and has removed ~100 GB so far. It will probably take the rest of the weekend to finish.
[Yehonathan, Paco, Yuta, JC]
As we were cleaning up this morning, we heard a high pitch sound that turned into a buzz. After searching for where the sound came from, we noticed the CRT TV went out. We swapped this out with a moniter and used a BNC to VGA adapter to display the cameras.
Paco and I fixed the ethernet cable which was hanging. We stopped models c1x07 and c1su2, realigned the cable to follow the shelf from top, and returned to turn on the computers.
Note: There was not a long enough ethernet cable, so we used a female to female adapter and attached 2 ethernet cables.
I've finished, for now, the CDS network tests that I was conducting. Everything should be back to normal.
I wanted to see if I could make the EPICS glitches we've been seeing go away if I unplugged everything from the CDS martian switch in 1X6 except for:
What I unplugged were things like megatron, nodus, the slow computers, etc. The control room workstations were still connected, so that I could monitor.
I then used StripTool to plot the output of a front end oscillator that I had set up to generate a 0.1 Hz sine wave (see elog 11662). The slow sine wave makes it easy to see the glitches, which show up as flatlines in the trace.
More tests are needed, but there was evidence that unplugging all the extra stuff from the switch did make the EPICS glitches go away. During the duration of the test I did not see any EPICS glitches. Once I plugged everything back in, I started to see them again. However, I'm currently not seeing many glitches (with everything plugged back in) so I'm not sure what that means. I think more tests are needed. If unplugging everything did help, we still need to figure out which machine is the culprit.
Here's an example of the glitches we've been seeing, as seen in the StripTool trace of the front end oscillator:
You can clearly see the glitch at around T = -18. Obviously during non-glitch times the sine wave is nice and cleanish (there are still the very small discretisation from the EPICS sample times).
I'm about to start conducting some tests on the CDS network. Things will probably be offline for a bit. Will post when things are back to normal.
I've installed Control System Studio (CSS) on pianosa, from the version 3.0.2 Red Hat binary zip. It should be available as "css" from the command line.
CSS is a new MEDM replacement. It's output is .opi files, instead of .adl files. It's supposed to include some sort of converter, but I didn't play with it enough to figure it out.
Please play around with it and let me know if there are any issues.
So far I've only given it about half an hour of my time, but it is *really* frustrating so far. There don't seem to be any instructions on how to tell it what our channels are / how to link CSS to our EPICS databases. Or, the instructions that are there say "do it!", but they neglect to mention how... Also, there exists (maybe?) an ADL->BOY converter, but I can't find any buttons to click, or how to import an .adl, or what I'm supposed to do. Also, it's not clear how to get to the editor to start making screens from scratch.
It looks like it has lots of nifty indicators and buttons, but I would have felt better if I had been able to do anything.
Another thing that is going to be a problem: the Shell Command button that we use all over the place in our MEDM screens is not supported by this program. It's listed in the "limitations" of the ADL2BOY converter. This may kill the CSS program immediately. Jamie: did Rolf/anyone mention a game plan for this? It's super nice to be able to run scripts from the screens.
Moral of the story: I'm annoyed, and going to continue making my OAF screens in MEDM for now.
How to run/use CSS/BOY
0) Everything runs on pianosa for now.
1) type css to launch CSS IDE.
2) You may want to create your own project folder as generally everything happens below this folder.
=== How to make a new project ===
- Right-click on tree view of Navigator pane
- You are asked to select a wizard. Select "General -> Project". Click "Next".
- Type in an appropriate project name (like KOJI). Click "Finish"
- The actual location of the project is /home/controls/CSS-Workspaces/Default/KOJI/ in the above example
(PLAY WITH "Data Browser", THE STRIPTOOL ALTERNATIVE)
1) Select the menu "CSS -> Trends -> Data Browser". A new data browser window appears.
2) Right-click on the data browser window. Select the menu "Add PV". Type in the channel name (e.g. "C1:LSC-ASDC_OUT16")
3) Once the plot configuration is completed, it can be saved as a template. Select the menu "File -> Save" and put it in your project folder.
4) Everything else is relatively straightforward. You can add multiple channels. Log scaling is also available.
I still don't find how to split the vertical axis to make a stacked charts, but I don't surprise even if it is not available.
(HOW TO MAKE A NEW "BOY" SCREEN)
0) Simply to say BOY is the alternative of MEDM. The screen file of BOY is named as "*.opi" similar to "*.adl" for MEDM.
1) To create a new opi file, right-click on the navigator tree and select the menu "New -> Other".
2) You are asked to select a wizard. Select "BOY -> OPI file" and click "Next".
3) Type in the name of the opi file. Also select the location of the file in the project tree. Click "Finish".
4) Now you are in OPI EDITOR. Place your widgets as you like.
5) To test the OPI screen, push the green round button at the top right. The short cut key is "CTRL-G".
(HOW TO EDIT AN EXISTING OPI FILE)
1) Right click an OPI file in the navigator tree. Select the menu "Open WIth -> OPI Editor". That's it.
(HOW TO CONVERT AN EXISTING ADL FILE INTO OPI FILE)
1) You need to copy your ADL file into your project folder. In this example, it is /home/controls/CSS-Workspaces/Default/KOJI/
2) Once the ADL file is in the project folder, it should appear in the navigator tree. If not, right-click the navigator pane and select "Refresh".
3) Make sure "ADL Parser" button at the left top part is selected, although this has no essential function.
This button just changes the window layout and does nothing. But the ADL Tree View pane would be interesting to see.
3) If you select the ADL file by clicking it,the tree structure of the ADL file is automatically interpreted and appears in the ADL Tree View pane.
But it is just a display and does nothing.
4) Right-click the ADL file in the navigator pane. Now you can see the new menu "BOY". Select "BOY -> Convert ADL File to OPI".
5) Now you get the opi version of that file.
The conversion is not perfect as we can imagine. It works fine for the simple screens. (e.g. matrix screens)
But the filter module screens get wierd. And the new LSC screen did not work properly (maybe too heavy?)
(HOW TO RUN OUR SHELL/PERL/PYTHON SCRIPTS FROM BOY)
I suspect that we can make a wrapper to run external commands from python script although it is not obvious yet.
Previously in elog 8959, I gave a very simple method for determining the noise suppression behavior of the ISS. Recently, I recalculated this requirement in a more correct fashion and again redesigned the ISS to be used in the CTN experiment.
Just as before, the data from PSL elog 1270 is necessary to infer a noise suppression requirement. The data presented there by Evan consists of two noise spectra, 1) the unstabilized RIN presently observed in the CTN experiment readout and 2) the theoretical brownian noise produced by thermal processes in the mirror coating+substrate. The statement "TF_mag = (Unstabilized RIN) / (Calculated Brownian Noise Limit)", where TF_mag refers to the required open-loop gain of the ISS, is actually a first order approximation of the 'required' noise suppression. In fact if we wanted the laser noise to be suppressed below the calculated brownian noise level, it is more correct to say
Closed-loop ISS gain = (Calculated Brownian Noise Limit) / (Unstabilized RIN)
As this essentially gives a noise suppression spectrum i.e. a closed-loop gain in linear control theory. Below is a very simple block diagram showing how the ISS fits into the CTN experiment. The F(f) block represents my full servo board.
Some of the relevant quantities involved:
So looking at the block diagram, our full closed-loop transfer function is given by,
So then to determine the required F(f), i.e. the required transfer function for my servo, we consider the expression
The plant transfer function is simply Plant = (C(f) * a * P * A) ~ 0.014 V/V, where I have ignored the cavity pole around 97 kHz as our open-loop transfer function ends up crossing unity gain around 10 kHz. In the above, I have included what I call a 'safety factor' of 10. Essentially, I want to design my servo such that it suppresses noise well beyond what is actually required so that we can be sure noise contributions to experiment readouts are not significantly influenced by the laser intensity noise.
Using the data Evan reported for the brownian noise and free-running RIN, I came up with an F(f) to the meet the requirement as shown below.
Where the blue curve includes the safety factor mentioned before. This plot just demonstrates that using my modular ISS design, I can meet the given noise suppression requirements.
To be complete, I'll say a little more about the final design. As usual, the servo consists of three stages. The first is the usual LP filter that is always 'on' when the ISS loop is closed. The boosts I have chosen to use consist of an integrator with a single zero and a filter that looks somewhat like a de-whitening filter. The simulated open-loop transfer functions are shown below.
In PSL elog 1270, Evan elucidated the explicit requirements for the CTN ISS board. Essentially, the transfer function of the ISS should be something like:
TF_mag = (Unstabilized RIN) / (Calculated RIN Requirement)
I took Evan's data and did exactly this. I then designed a servo (using the general design I proposed here) to meet this requirement with a safety factor of ~10. By safety factor, I mean that if the ISS operates exactly according to theory, it should suppress the noise by a factor of 10 more than what is necessary/set out by the requirement. Below is a plot of the loop gain obtained directly from the requirement (the above expression for TF_mag) and the transfer function of the servo I am proposing.
I don't have the actual schematics attached as I was working with a LISO file and have yet to update the corresponding Altium schematic. The LISO file is attached and I will add the schematics later, although one can reference the second link to find a simple drawing.
# Stage 1
r R31 1.58k in n_inU3
op U3 ad829 p_inU3 n_inU3 outU3
r R35 1k p_inU3 gnd
c C33 1u p_inU3 gnd
c C32 10n n_inU3 outU3
r R34 158k n_inU3 outU3
# Stage 2
#r R41 15.8 outU3 n_inU4U5