Updated A, B, C, D matrices for the state-space model to remove bugs in the previous estimate of the system dynamics. Updated the last post to represent the current matrixes.
We used MatLab to get the correct time-series filter coefficients in ZPK format and added them to the filters running in the TM_RESP filter matrix.
Get the pos-pos transfer function from the CDS model. Strangely, this seems to take a lot longer than anticipated to generate the transfer function, even though we are mainly probing the low-frequency behavior of the system.
For example, a test that should be taking approximately 6 minutes is taking well over an hour to complete. This swept sine (results below) was on the low settings to get a fast answer and it looks bad. This is a VERY basic system it shouldn't be taking this long to complete a Swept sine TF.
Noticed that we need to run eval $(./env_cymac) every time we open a new terminal otherwise CDS doesn't work as expected. Since this has been the source of quite a few errors already, we have decided to put it in the startup .bashrc script.
I went through the optics list (in the BHD procurement google spreadsheet) and summarized how to build them.
The red ones are what we need to purchase. Because of the strange height of the LMR mounts, the post needs to have none half-integer inch heights.
They need to be designed as the usual SS posts are not designed to be vac compatible (not because of the material but the design like screw hole venting).
We also need to check how many clean forks we have.
-> The components were ordered except for the custom posts.
[Ian, Raj, Tega]
Here is the comparison between the results of Raj's python model and the transfer function measurement done on the plant model by Tega and me.
As You can see in the graphs there are a few small spots of disagreement but it doesn't look too serious. Next we will measure the signals flowing through the entire plant and controller.
For a nicer (and printable) version of these plots look in the zipped folder under Plots/Plant_TF_Individuals.pdf
1. Investigate cross-coupling btw the various degrees of freedom (dof) - turn on noise for each dof in the plant model and measure the transfer function of the other dofs.
2. Get a closed-loop transfer function using noise injection and give a detailed outline of the procedure in elog - IN1/IN2 for each TM_RESP filter while the others are turned off.
3. Derive analytic model of the closed-loop transfer functions for comparison.
4. Adapt control filters to fit optimized analytical solutions.
I added mpmath to the quantization noise code. mpmath allows me to specify the precision that I am using in calculations. I added this to both the IIR filters and the State-space models although I am only looking at the IIR filters here. I hope to look at the state-space model soon.
I also added a new notebook which you can find HERE. This notebook creates a signal by summing two sine waves and windowing them.
Then that signal is passed through our filter that has been limited to a specific precision. In our case, we pass the same signal through a number of filters at different precisions.
Next, we take the output from the filter with the highest precision, because this one should have the lowest quantization noise by a significant margin, and we subtract the outputs of the lower precision filters from it. In summary, we are subtracting a clean signal from a noisy signal; because the underlying signal is the same, when we subtract them the only thing that should be left is noise. and since this system is purely digital and theoretical the limiting noise should be quantization noise.
Now we have a time series of the noise for each precision level (except for our highest precision level but that is because we are defining it as noiseless). From here we take a power spectrum of the result and plot it.
After this, we can calculate a frequency-dependent SNR and plot it. I also calculated values for the SNR at the frequencies of our two inputs.
This is the procedure taken in the notebook and the results are shown below.
Analysis of Results:
The first thing we can see is that the precision levels 256 and 128 bits are not shown on our graph. the 256-bit signal was our clean signal so it was defined to have no noise so it cant be plotted. The 128-bit signal should have some quantization noise but I checked the output array and it contained all zeros. after further investigation, I found that the quantization noise was so small that when the result was being handed over from mpmath to the general python code it was rounding those numbers to zero. To overcome this issue I would have to keep the array as a mpmath object the entire time. I don't think this is useful because matplotlib probably couldn't handle it and it would be easier to just rewrite the code in C.
The next thing to notice is sort of a sanity check thing. In general, low precision filters yield higher noise than high precision. This is a good quick sanity check. However, this does not hold true at the low end. we can see that 16-bit actually has the highest noise for most of the range. Chris pointed out that at low precisions that quantization noise can become so large that it is no longer a linearly coupled noise source. He also noted that this is prone to happen for low precision coefficients with features far below the Nyquist frequency like I have here. This is one explanation that seems to explain the data especially because this ambiguity is happening at 16-bit and lower as he points out.
Another thing that I must mention, even if it is just a note to future readers, is that quantization noise is input dependent. by changing the input signal I see different degrees of quantization noise.
Analysis of SNR:
One of the things we hoped to accomplish in the original plan was to play around with the input and see how the results changed. I mainly looked at how the amplitude of the input signal scaled the SNR of the output. Below I include a table of the results. These results were taken from the SNR calculated at the first peak (see the last code block in the notebook) with the amplitude of the given sine wave given at the top of each column. this amplitude was given to both of the two sine waves even though only the first one was reported. To see an example, currently, the notebook is set up for measurement of input amplitude 10.
As we can see from the table above the SNR does not seem to relate to the amplitude of the input. in multiple instances, the SNR dips or peaks in the middle of our amplitude range.
This looks great. I think what we want to see mainly is just the noise in the 32 bit IIR filtering subtracted from the 64 bit one.
It would be good if Tega can look through your code to make sure there's NO sneaky places where python is doing some funny casting of the numbers. I didn't see anything obvious, but as Chris points out, these things can be really sneaky so you have to be next level paranoid to really be sure. Fox Mulder level paranoia.
And, we want to see a comparison between what you get and what Denis Martynov put in an appendix of his thesis when comparing the Direct Form II, with the low-noise form (also some slides from Matt Evans on thsi from a ~decade agoo). You should be able to reproduce his results. He used matlab + C, so I am curious to see if it can be done all in python, or if we really need to do it in C.
And then...we can make this a part of the IFOtest suite, so that we point it at any filter module anywhere in LIGO, and it downloads the data and gives us an estimate of the digital noise being generated.
Tega and I have gone through the IIR Filter code and optimized it to make sure there aren't any areas that force high precision to be down-converted to low precision.
For the new biquad filter we have run into the issue where the gain of the filter is much higher than it should be. Looking at attachments 1 and 2, which are time series comparisons of the inputs and outputs from the different filters, we see that the scale for the output of the Direct form II filter shown in attachment 1 on the right is on the order of 10^-5 where the magnitude of the response of the biquad filter is on the order of 10^2. other than this gain the responses look to be the same.
I am not entirely sure how this gain came into the system because we copied the c code that actually runs on the CDS system into python. There is a gain that affects the input of the biquad filter as shown on this slide of Matt Evans Slides. This gain, shown below as g, could decrease the input signal and thus fix the gain. However, I have not found any way to calculate this g.
With this gain problem we are left with the quantization noise shown in Attachment 4.
I have controlled the state space filter to act with a given precision level. However, my code is not optimized. It works by putting the input state through the first state-space equation then integrating the result, which finally gets fed through the second state-space equation.
This is not optimized and gives us the resulting quantization noise shown in attachment 5.
However, the state-space filter also has a gain problem where it is about 85 times the amplitude of the DF2 filter. Also since the state space is not operating in the most efficient way possible I decided to port the code chris made to run the state-space model to python. This code has a problem where it seems to be unstable. I will see if I can fix it
I am trying to replicate the simulation done by Matt Evans in his presentation (see Attachment 1 for the slide in particular).
He defines his input as so he has two inputs one of amplitude 1 at 1 Hz and one of amplitude 10^-9 at 1/4th the sampling frequency in this case: 4096 Hz
For his filter, he uses a fourth-order notch filter. To achieve this filter I cascaded two second-order notch filters (signal.iirnotch) both with locations at 1 Hz and quality factors of 1 and 1e6. as specified in slide 13 of his presentation.
I used the same procedure outlined here. My results are posted below in attachment 2.
Analysis of results:
As we can see from the results posted below the results don't match. there are a few problems that I noticed that may give us some idea of what went wrong.
First, there is a peak in the noise around 35 Hz. this peak is not shown at all in Matt's results and may indicate that something is inconsistent.
the second thing is that there is no peak at 4096 Hz. This is clearly shown in Matt's slides and it is shown in the input spectrum so it is strange that it does not appear in the output.
My first thought was that the 4kHz signal was being entered at about 35Hz but even when you remove the 4kHz signal from the input it is still there. The spectrum of the input shown in Attachment 3 shows no features at ~35Hz.
The Input filter, Shown in attachment 4 shows the input filter, which also has no features at ~35Hz. Seeing how the input has no features at ~35Hz and the filter has no features at ~35Hz there must be either some sort of quantization noise feature there or more likely there is some sort of sampling effect or some effect of the calculation.
To figure out what is causing this I will continue to change things in the model until I find what is controlling it.
I have included a Zip file that includes all the necessary files to recreate these plots and results.
4 units of Vertex SUS DAC adapter (https://dcc.ligo.org/LIGO-D2100035) ready.
The units are completely passive right now and has option to extend to have a dewhitening board added inside.
So the power switch does nothing.
Some of the components for the dewhitening enhancement are attached inside the units.
[Anchal, Yehonathan, Paco]
Today we opened ITMX chamber and removed the following optics and placed them in the Xend flow bench (see attachment 1):
Yehonathan brought his first SOS baby next to ITMX chamber. The suspension was carried by hands throughout. He gave me the suspension over the IMC beam tube from where I placed it on the table. I checked through the OSEMs and the face magnets were still on. I could not verify the side magnet but nothing seemed out of place.
I then moved LO1 near its planned place. I had to bolt it at 1 inch North and 0.5 inch West of its planned position because the side OSEM on ITMX is long and protrudes out of the base footprint. Even if it was small, the current layout would make the OSEM pins of the side OSEMs of ITMX and LO1 very near each other. So we can not place LO1 closer to ITMX from current position. This means the layout needs to be redesigned a bit for the modified position of LO1. I believe it will significantly shift and turn the beam from LO1 to LO2, so we might need to change the beam upstream from TT2 onwards. More discussion is required.
Unfortunately, what I thought was clicking photos was just changing modes between video and image mode, so I have no photos from today but only a video that I recorded in the end.
If ITMX already has another side magnet, we can migrate the side OSEM of ITMX to the other side. This way, the interference of the OSEMs can be avoided.
[Anchal, Yehonathan, Chub]
We today laid down 14 70 ft long DB25 cables from 1Y1 (6), 1Y0 (8) to ITMY Chamber (4), BS Chamber (6) and ITMX Chamber (4). The cables have been connected to respective satellite amplifier on the racks and the other ends are connected to the vacuum flange feedthru on ITMX for LO1 and PR2, while the others have been kept near the planned flange postions. LO1 is now ready to be connected to CDS by connecting the in-vacuum cable inside ITMX chamber to the OSEMs.
I've updated the c1su2 model today with model suspension blocks for the 7 new SOSs (LO1, LO2, AS1, AS4, SR2, PR2 and PR3). The model is running properly now but we had some difficulty in getting it to run.
Initially, we were getting 0x2000 error on the c1su2 model CDS screen. The issue probably was high data transmission required for all the 7 SOSs in this model. Koji dug up a script /opt/rtcds/caltech/c1/userapps/trunk/cds/c1/scripts/activateDQ.py that has been used historically for updating the data rate on some of theDQ channels in the suspension block. However, this script was not working properly for Koji, so he create a new script at /opt/rtcds/caltech/c1/chans/daq/activateSUS2DQ.py.
[Ed by KA: I could not make this modified script run so that I replaces the input file (i.e. C1SU2.ini). So the output file is named C1SU2.ini.NEW and need to manually replace the original file.]
With this, Koji was able to reduce acquisition rate of SUSPOS_IN1_DQ, SUSPIT_IN1_DQ, SUSYAW_IN1_DQ, SUSSIDE_IN1_DQ, SENSOR_UL, SENSOR_UR, SENSOR_LL,SENSOR_LR, SENSOR_SIDE, OPLEV_PERROR, OPLEV_YERROR, and OPLEV_SUM to 2048 Sa/s. The script modifies the /opt/rtcds/caltech/c1/chans/daq/C1SU2.ini file which would get re-written if c1su2 model is remade and reinstalled. After this modification, the 0x2000 error stopped appearing and the model is running fine.
We notice that all our suspension models need to go through this weird python script modifying auto-generated .ini files to reduce the data rate. Ideally, there is a simpler solution to this by simply adding the datarate 2048 in the '#DAQ Channels' block in the model library part /cvs/cds/rtcds/userapps/trunk/sus/c1/models/lib/sus_single_control.mdl which is the root model in all the suspensions. With this change, the .ini files will automatically be written with correct datarate and there will be no need for using the activateDQ script. But we couldn't find why this simple solution was not implemented in the past, so we want to know if there is more stuff going on here then we know. Changing the library model would obviously change every suspension model and we don't want a broken CDS system on our head at the begining of holidays, so we'll leave this delicate task for the near future.
We want to maintain the 16 kHz sample rate for the COIL DAQ channels, but nothing wrong with reducing the others.
I would suggest setting the DQ sample rates to 256 Hz for the SUS DAMP channels and 1024 Hz for the OPLEV channels (for noise diagnostics).
Maybe you can put these numbers into a new library part and we can have the best of all worlds?
[Paco (Vacuum Work), Anchal]
Today we opened the ITMY Chamber and installed suspended AS1 and AS4 in their planned positions. In doing so, we removed the razor or plate mounted on a pico motor at the south end of the table (see 40m/16450). We needed to make way for AS4 to be installed.
We need more dog clamps for installing the suspensions, we have used temporary clamps for now. However, knows where new C&B clamps are, please let us know.
[Anchal, Koji] Part of elog: 40m/16549.
The magnets on the mirror face are arranged in a manner that the overall magnetic dipole moment is nullified faraway. Because of this, the coil output gains in all such optics need to have positive and negative signs in a butterfly mode pattern (eg. UL, LR: +ve and UR, LL: -ve).
In the particular case of LO1, we chose following coil output gains:
This ensures that all damping gains have positive signs. Following damping gain values were chosen:
Having said that, this is a convention and we need to discuss more on what we want to set a convention (or follow a previous one if it exists). My discussion with Koji came up with the idea of fixing the motion response of an OSEM with respect to coil offset by balancing the coil gains across all optics and use same servo gains for all optics afterwards. But it is a complicated thought coming out of tired minds, needs more discussion.
Added input filters, input matrix, damping filters, output matrix, coil filters, and copy the state over from LO1 into AS1 screen in anticipation for damping.
Added input filters, input matrix, damping filters, output matrix, coil filters, and copy the state over from LO1 into AS4 screen in anticipation for damping.
Added input filters, input matrix, damping filters, output matrix, coil filters, and copy the state over from ITMX into LO2 screen in anticipation for damping.
I used the open light level output of 908 for ITMX side OSEM from 40m/16549 to roughly calibrate cts2um filter module in LO1 OSEM input filters. All values were close to 0.033. As the calibration reduces the signal value by about 30 times, I increased all damping gains by a factor of 30. None of loops went into any unstable oscillations and I witnessed damping of kicks to the optic.
I also compared in-loop power spectrum of ETMX and LO1 while damping. ETMX was chosen because it is one of the unaffected optics by the upgrade work. ITMX is held by earthquake stops to avoid unnecessary hits to it while doing chamber work.
Attachment 1 and 2 show the power spectrum of in-loop OSEM values (calibrated in um). At high frequencies, we see about 6 times less noise in LO1 OSEM channel noise floor in comparison to ETMX. Some peaks at 660 Hz and 880 Hz are also missing. At low frequencies, the performance of LO1 is mostly similar to EMTX except for a peak (might be loop instability oscillation) at 1.9 Hz and another one at 5.6 Hz. I'll not get into noise hunting or loop optimization at this stage for the suspension. For now, I believe the new electronics are damping the suspensions as good as the old electronics.
LO1 is set to go through a free swinging test at 1 am tonight. We have used this script (scripts/SUS/InMatCalc/freeSwing.py) reliably in the past so we expect no issues, it has a error catching block to restore all changes at the end of the test or if something goes wrong.
To access the test, on rossa, type:
Then you can kill the script if required by Ctrl-C, it will restore all changes while exiting.
The frree swinging test was successful. I ran the input matrix diagonalization code (scripts/SUS/InMAtCalc/sus_diagonalization.py) on the LO1 free swinging data collected last night. The logfile and results are stroed in scripts/SUS/InMatCalc/LO1 directory. Attachment 1 shows the power spectral density of the DOF bassis data (POS, PIT, YAW, SIDE) before and after the diagonalization. Attachment 2 shows the fitted peaks.
The new matrix was loaded on LO1 input matrix and this resulted in no control loop oscillations at least. I'll compare the performance of the loops in future soon.
[Anchal, Paco, Yehonathan]
Connected in-cir cable to new flange on ITMY Chamber
Connected OSEM one-by-one. Starting from top right to left (PIn1)
1st connector: LL -> UR -> UL
2nd connector: LR -> SD
Loosening all OSEMs and taking them out and noting full bright readings:
:( We had to stop here as we were unable to actuate on the side coils. We checked the signal chain and found that the monitor output of AS1 LL/SD coil driver is responding to offset changes in the coil output filter module of AS1 side. However, when we connected the output of the coil driver through a breakout board to the AS1 Sat Amp, we saw no signal. We tried switching the coil driver bo with another one one the rack but we found the exact same issue. This led us to believe that something must be wrong with the AS1 Sat Amp. We checked the output of the AS1 LL/SD coil driver without connecting it to the sat amp and found that the output was responding to our CDS changes. Then we checked the second "Coil Input" port of the AS1 Sat Amp, and found that pins 2-7 and pins 3-8 are shorted. This means channel 5 and 8 on this box would be shorted. This is the reason why we were unable to actuate on the coils. I'll work on debugging this box, my first guess is that the ribbon cable is bad.
AS1 Sat Amp (S2100741) has a critical PCB issue on it's Ch5-8 board S2100548. This board is supposed to just feed through the coil driver signal from the front DB9 connector to the back DB25 connector but it has a short between pins 2 and 7 at the "Coil Input" end (connector J1). The short persists even after I disconnect the sat amp to the flange connector on the back of this board, which definitely means the short is present in the passive channeling through the PCB or at the soldering points of the two DB connectors. I just flipped the board and found that the soldering connections are clean and separate. I think we'll have to use one of the spare sat amp boxes for AS1 for now, while we either declare this one manufacture defected or fix the issue.
I actually found the short on the PCB trace by just looking carefully at it. Attachment 1 shows the photo of it. Maybe we can fix this by simply cutting the tumor between the two traces (why are these traces so close together in such a large board anyways!!!), but I'm not sure if that is a reliable way of fixing this issue. I'll wait for Koji's comments on what to do with this. We'll recommence OSEM tuning for AS1 tomorrow with fixed electronics.
I fixed the issue in AS1 Sat Amp (S2100741) by using a razor blade. I cut the short between the two places, cleaned up the area and covered it with electrical tape. However, later feedback from Rana was to not use electrical tape as it dries up and becomes brittle and lfaky in long run. So after the AS1 OSEM tuning is over, I'll open this box again and use something else to insulate the exposed area. See attached pictures for current status.
[Anchal (vacuum work), Paco (outside)]
After the AS1 Sat Amp fix (40m/16579), we today were able to tune all OSEMs to the mid-bright level. But when we were about to call it, we were told that the new PEEK earthquake stop screw and bolts need to go on the thin suspended optics. Against better judgment, we decided to install the new back earthquake stop in-situ since we had tuned all OSEMs already. I installed the new stop but after that found that in the process I have broken off the side magnet and LR magnet from the optic adaptor and they are inside the OSEM coils now. This means we'll have to redo the AS1 suspension almost from scratch again . We have transported AS1 to the cleanroom where the work on re-suspension has begun.
After the debacle with AS1 (40m/16580), we decided the put the PEEK earthquake stop by first removing the lower OSEM plate and then doing it. So I unfastened AS4 from its position with the earthquake stops in place and moved the suspension to the center of the table. Then I carefully removed the bottom OSEM plate. But I found out that the LR magnet is broken and lying on the floor of the suspension . Given my past on the same day, it could be me breaking it again during the moving of the suspension of taking off the OSEM plate or there is a small chance that this break happened before. Regardless of fault, this meant we have to resuspend AS4 again as well. So we transported AS4 back to the clean room and the work on it's re-suspension has begun.
AS4 was succesfully suspended and trasported to ITMY chamber (40m/16589). We placed it near the door to make it easy to tune the OSEMs. We connected the OSEMs and found that the LL OSEM is not responding. Even though the the OSEMs are completely out right now, there was no signal on this OSEM. This could be an issue in either at the LED driver circuit or the PD circuit in AS4 Sat Amp box, or it could be the OSEM that is bad. We'll investigate further next day. For now, we recorded the full brightness reading for the OSEMs:
Another thing to note is that UL value above is not changing at all. I checked the CDS screen and the the ADC input is overflowing in complete bright position of the OSEM.
Added input filters, input matrix, damping filters, output matrix, coil filters, and copy the state over from LO1 into SR2, PR2, PR3 screens in anticipation for damping.
[Tega, Anchal, Paco]
We started working on SR2 installation. Preliminary work involved
That was pretty much it. After identifying the cabling situation, we proceeded to bring SR2 from the cleanroom. The magnets and wires remained well through their travel.
2nd connector: LR -> SD** (we had some trouble here where the first time we made a connection we didn't see any signal, after a brief review of cables, sat amp unit, cables again with Koji, and sat amp again, we found out a connection was not done in the front of the SR2 SatAmp box, after which we saw the sensor signals).
After finishing the initial SD osem tuning, we moved onto UL, and then to UR, but we noticed that the UR was not able to drop to its target value of ~13000 counts, even when the OSEM face was < 1 mm from the adapter (see Attachments #1-2). Apart from becoming harder to push in, it became apparent that the dark level (full shadow) is not consistent with ~ 0 counts; is there an offset coming from SatAmp? We quickly checked the OSEM by replacing it in-situ with another working one from the cleanroom batch, but the issue persisted. We decided to stop here, as we suspect the SatAmp box might have some issue.
I tested the monitor ports on the SR2 Sat Amp Box but found that all LED Mon and PD Mon are giving expected values. I disconnected the cable to OSEM and checked the PD monitors and found no offsets in case of no PD current. I realised that PD transimpedance offset should be checked with PD inputs shorted instead. So I created a male DB 25 connector with pinds 2-3, 50-6, 8-9 and 11-12 shorted. This on connecting to the OSEM cable at the back of sat amp boxes should short the PD inputs. On using this plug, I found no offsets in any of the Sat Amp PD output channels.
It is possible that the issue is there because the magnet is missing the LED-PD path way because it is offset sideways. In fact, in my limited memory, I do not recall seeing the UL OSEM signal ever going to complete darkness either. Tomorrow, we should take a photo of the OSEMs from the back and see if any sideways adjustment of the top OSEM plate is required. If any adjustment is required, we must take the OSEMs out and then do the adjustment. Do not attempt to adjust OSEM plate with OSEMs inserted in-situ. That will most probably knock off the magnets.
AS4 satellite amplifier D1002818 / D080276 troubleshoot
I dug into the circuit to see what/where things were wrong.
- UL saturation issue: The open light voltage at the TIA output (I-V out) was 10.4V. It seemed that the photocurrent of 86uA was simply a little too much for the transimpedance gain of 121kOhm. So the R18 was replaced to 100kOhm. This made the I-V out to be 8.6V and the ADC input count to be 28200 (Attachment 1). This modification was done on the unit S2100742 CH1 (LEFT CH1)
- Non responding LL issue: Now moved on to LL (LEFT CH2). The basic circuit test didn't reveal any problem. So the DSUB25 cables were swapped at the vacuum feedthru flange. The result is shown in Attachment 2. LL OSEM issue was moved to the 2nd ch of the right channel of the sat amp (CH6). This is means that the problem is somewhere in the vacuum chamber (including the vacuum feedthru). We need to check the in-vacuum cable and the OSEM. We can test the OSEM by swapping the position of the OSEM connector between LL and UL (for example).
It was indeed the issue of the top OSEM plate not being in the right place horizontally. But the issue was more non-trivial. I believe because of the wedge in thick optics, there is a YAW offset in the optic in the free hanging position. I had to readjust the OSEM plate 4 times to be able to get full dark to bright range in both upper OSEMs. After doing that, I tuned the four OSEMs somewhat near the halfway point and once I was sure I'm inside the sensitive region in all face OSEMs, I switched on POS, PIT, and YAW damping. Then I was able to finely tune the positions of both upper OSEMs.
However, on reaching to lower right OSEM, I found again the same issue. I had to stop to go to the 40m meeting, I'll continue this work in the afternoon. But OSEM plate adjustment in the horizontal direction, particularly for thick optics is required to be done before transporting them. I achieved the best position by turning the OSEM 90 degrees and using the OSEM LED/PD plates to determine the position. This was the final successful trial I did in adjusting the plate position horizontally.
[Paco, Tega, Anchal]
Today, we started work on AS4 SOS by checking the OSEM and cable. Swapping the connection preserved the failure (no counts) so we swapped the long OSEM for a short one that we knew was working instead, and this solved the issue. We proceeded to swap in a "yellow label", long OSEM in place and then noticed the top plate had issues with the OSEM threads. We took out the bolt and inspected its thread, and even borrowed the screw from PR2 plate but saw the same effect. Even using a silver plated setscrew such as the SD OSEM one resulted in trouble... Then, we decided to not keep trying weird things, and took our sweet time to remove the UL, UR OSEMs, top earthquake stops, and top plate carefully in-situ. Then, we continued the surgery by installing a new top plate which we borrowed from the clean room (the only difference is the OSEM aperture barrels are teflon (?) rather than stainless. The operation was a success, and we moved on to OSEM installation.
After reaching a good place with the OSEM installation, where most sensors were at 50% brightness level and we were happy with the damping action (!!), we fixed all EQ stops and proceeded to push the SOS to its nominal placement. Then upong releasing the EQ stops, we found out that the sensor readings were shifted.
The main issue with SR2 OSEMs, now that I think of it, was that the BS table was very inclined due to the multiple things we removed (including counterweights). Today the first I did was level the BS table by placing some counterweights in the correct positions. I placed the level in two directions right next to SR2 (clamped in its planned place), and made the bubble center.
While doing do, at one point, I was trying to reach the far South-West end of the table with the 3x heavy 6" cylindrical counterweight in my hand. The counterweight slipped off my hand and fell from the table. See the photo in attachment 1. It went to the bottommost place and is resting on its curved surface.
This counterweight needs to be removed but one can not reach it from over the table. So to remove it, we'll have to open one of the blank flanges on the South-west end of BS chamber and remove the counterweight from there. We'll ask Chub to help us on this. I'm sorry for the mistake, I'll be more careful with counterweights in the future.
Moving on, I tuned all the SR2 OSEMs. It was fairly simple today since the table was leveled. I closed the chamber with the optic free to move and damped in all degrees of freedom.
SUSPENSION STATUS UPDATED HERE
SR2 is set to go through a free swinging test at 3 am tonight. We have used this script (Git/40m/scripts/SUS/InMatCalc/freeSwing.py) reliably in the past so we expect no issues, it has a error catching block to restore all changes at the end of the test or if something goes wrong.
To access the test, on allegra, type:
The free swinging test was successful. I ran the input matrix diagonalization code (/opt/rtcds/caltech/c1/Git/40m/scripts/SUS/InMAtCalc/sus_diagonalization.py) on theSR2 free-swinging data collected last night. The logfile and results are stored in /opt/rtcds/caltech/c1/Git/40m/scripts/SUS/InMAtCalc/SR2 directory. Attachment 1 shows the power spectral density of the DOF basis data (POS, PIT, YAW, SIDE) before and after the diagonalization. Attachment 2 shows the fitted peaks.
The new matrix was loaded on SR2 input matrix and this resulted in no control loop oscillations at least. I'll compare the performance of the loops in future soon.
Connected the New SUS screens to the controller for the simplant model. Because of hard-coded links in the medm screen links, it was necessary to create the following path in the c1sim computer, where the new medm screen files are located:
We noticed a few problems:
1. Some of the medm files still had C1 hard coded, so we need to replace them with $IFO instead, in order for the custom damping filter screen to be useful.
2. The "Load coefficient" button was initially blank on the new sus screen, but we were able to figure out that the problem came from setting the top-level DCU_ID to 63.
medm -x -macro "IFO=X1,OPTIC=OPT_CTRL,DCU_ID=63" SUS_SINGLE_OVERVIEW.adl
Get the data showing the controller damping the pendulum. This will involve tweaking some gains and such to fine-tune the settings in the controller medm screen. Then we will be able to post some data of the working controller.
We should have a single place with all the instructions that are currently spread over multiple elogs so that we can better navigate the simplant computer.
I placed LO2 in its planned position in BS chamber, inserted the OSEMs, and tuned their position to halfway brightness. At the end of the work, I was able to damp the optic successfully. The full open (full brightness) OSEM ADC counts are:
UL 25743. -> 12876
UR 27384. -> 13692
LL 25550. -> 12775
LR 27395 -> 13697
SD 28947 -> 14473
Today's OSEM tuning was relatively unhappening. I have only following two remarks:
LO2 is set to go through a free swinging test at 10 pm tonight. We have used this script (Git/40m/scripts/SUS/InMatCalc/freeSwing.py) reliably in the past so we expect no issues, it has a error catching block to restore all changes at the end of the test or if something goes wrong.
AS1 was installed in the ITMY chamber today. For this I moved AS4 to its nominal final placement and clamped it down with a single dog clamp. Then, I placed AS1 near the center of the table, and quickly checked AS4 could still be damped. After this, I leveled the table using a heavier/lighter counterweight pair.
Once things were leveled, I proceeded to install AS1 OSEMs. The LL, UL, UR OSEMs had a bright level of 27000 counts, while SD and LR were at 29500, and 29900 respectively. After a while, I managed to damp all degrees of freedom around the 50% thousand count levels, and decided to stop.
UL 27000. -> 16000
UR 27000. -> 13800
LL 27000 -> 14600
LR 29900 -> 14900
SD 29500 -> 12900
Free swinging test set to trigger
AS1 is set to go through a free swinging test at 3 am this evening. We have used this script (Git/40m/scripts/SUS/InMatCalc/freeSwing.py) reliably in the past so we expect no issues, it has a error catching block to restore all changes at the end of the test or if something goes wrong.
The free swinging test was successful. I ran the input matrix diagonalization code (/opt/rtcds/caltech/c1/Git/40m/scripts/SUS/InMAtCalc/sus_diagonalization.py) on the LO2 free-swinging data collected last night. The logfile and results are stored in /opt/rtcds/caltech/c1/Git/40m/scripts/SUS/InMAtCalc/LO2 directory. Attachment 1 shows the power spectral density of the DOF basis data (POS, PIT, YAW, SIDE) before and after the diagonalization. Attachment 2 shows the fitted peaks.
The new matrix was loaded on LO2 input matrix and this resulted in no control loop oscillations at least. I'll compare the performance of the loops in future soon.
For some reasonf the free swing test showed only one resonance peak (see attachment 1). This probably happened because one of the earthquake stops is touching the optic. Maybe after the table balancing, the table moved a little over its long relazation time and by the time the free swing test was performed at 3 am, one of the earthquake stops was touching the optic. We need to check this when we open the chamber next.
I noticed that our current suspension damping loops for the new SOS were railing the DAC outputs. The reason being that cts2um module has not been updated for most optics and thus teh OSEM signal (with the new Sat Amps) is about 30 times stronger. That means our usual intuition of damping gains is too high without applying correct conversion cts2um filter module. I reduced all these gains today and nothing is overflowing the c1su2 chassis now. I also added two options in the "!" (command running drop down menu) in the sus_single medm screens for opening ndscope for monitoring coil outputs or OSEM inputs of the optic whose sus screen is used.
This morning, I went into ITMY chamber to inspect AS1 after the free swinging test failed. Indeed, as forecasted by Anchal, the top front EQ stop was slightly touching, which means AS1 was not properly installed before. I proceeded by removing it well behind any chance of touching the optic, and did the same for all the other stops, of which most were already recessed significantly. Finally, the OSEMs changed accordingly to produced a PITCHed optic (top front EQ was slightly biasing the pitch angle), so I did a reinstallation until the levels were around the 14000 count region. After damping AS1 relatively quickly, I closed the ITMY chamber.
[Ian, Paco, Tega]
Last night we set up the four main matrices that handle the conversion between the degrees of freedom bases and the sensor bases. We also wrote a bash script to automatically set up the system. The script sets the four change of bases matrices and activates the filters that control the plant. this script should fully set up the plant to its most basic form. The script also turns off all of the built-in noise generators.
After this, we tried damping the optic. The easiest part of the system to damp is the side or y motion of the optic because it is separate from the other degrees of freedom in both of the bases. We were able to damp that easily. in attachment 1 you can see that the last graph in the ndscope screen the side motion of the optic is damped. Today we decided to revisit the problem.
Anyways, looking at the problem with fresh eyes today, I noticed the in pit2pit coupling has the largest swing of all the plant filters and thought this might be the reason why the inputs (UL,UR,LR,LL) to the controller was hitting the rails for pit DoF. I reduce the gain of the pit2pit filter then slowly increased it back to one. I also reduced the gain in the OSEM input filter from 1 to 1/100. The attached image (Attachment2) is the output from this trial. This did not solve the problem. The output when all OSEM input filter gain set to one is shown in Attachment2.
We will try to continue to tweak the coefficients. We are probably going to ask Anchal and Paco to sit down with us and really hone in on the right coefficients. They have more experience and should be able to really get the right values.
We removed the old PR3 housed in a tip-tilt style suspension and put it on the North flow bench in the cleanroom. I put PR3 in an accessible position near the North West edge of the BS chamber and balanced the table again with many weights. The OSEM tuning was very uneventful and easy. Following are the full brightness ADC counts for the OSEMs:
UL 25693. -> 12846
UR 24905. -> 12452
LL 23298. -> 11649
LR 24991. -> 12495
SD 26357. -> 13178
I was able to damp the optic easily after the OSEM installation with no issues.
[Paco, Anchal, Tega]
After installing the short OSEMs into PR2, we moved it into ITMX Chamber. While Tega loaded some of the damping filters and other settings, we took time to balance the heavily tilted ITMX chamber. After running out of counterweigths, Anchal had to go into the cleanroom and bring the SOS stands, two of which had to be stacked near the edge of the breadoard. Finally, we connected the OSEMs following the canonical order
LL -> UR-> UL
LR -> SD
But found that UR was reading -14000 counts. So, we did a quick swap of the UR and UL sensors and verified that the OSEM itself is working, just in a different channel... So it's time to debug the electronics (probably PR2 Sat Amp?)...
PR2 Sat Amp preliminary investigation:
Thanks to Koji's hotfix on the PR2 SatAmp box last evening, this morning I was able to finish the OSEM installation for PR2. PR2 is now fully damped. Then, I realized that with the extreme rebalancing done in ITMX chamber, LO1 needed to be reinstalled, so I proceeded to do that. I verified all the degrees of freedom remained damped.
I think all SOS are nominally damped, so we are 90% done with suspension installation!
AS1, PR2 and PR3 are set to o go through a free swinging test at 3 am. We have used this script (Git/40m/scripts/SUS/InMatCalc/freeSwing.py) reliably in the past so we expect no issues, it has a error catching block to restore all changes at the end of the test or if something goes wrong.