Installed two new 2TB disks into the cymac3. Also, the main disk has a 1TB partition with the operating system, so I created a new 1TB partition. I created a logic volume that spans the three partitions, for a total of about 5TB. This partition is mounted in /mnt/data and linked to the /frames folder. Frames are written to this new logic volume.
I cross checked the calibration of the sum channel with a power meter. Now all quadnant signals X3:Q?_OUT and the sum signal X3:CR1-SUM_OUT are correctly calibrated in microwatts.
Compiled and installed an updated real time model. It acquires the four QPD segments, compute sum and differences, and normalize. I also added three experimental peak tracking components, that will be used to track in real time the amplitude and frequency of the disk modes.
I also created a summary medm screen that can be used to control all the QPD readout. It can be opened from any terminal with the command 'cr1'. I haven't added the peak tracking parts yet.
The X_NORM_OUT and Y_NORM_OUT signals are high passed at 500 Hz, to get rid of low frequency stuff that can affect the peak tracking.
I installed a dielectric laser line filter (Thorlabs FLH05633-5, center wavelength 633 nm, FWHM 5 nm) in front of the QPD. In this way we are no more affected by the room light. In the plot below blue is without filter, red with filter. A lot of peaks at high frequency are eliminated by the filter.
The plot below shows the QPD signal quadrant signals in a few different configurations: blue with the room light on and laser off, red with the room light off and laser off, green with the laser on. With the filter installed, when the laser is on we are dominated by its intensity noise, which shows a lot of peaks at high frequency. Those peaks are not completely eliminated by the difference of the quadrants.
Yesterday we received the prototype of the disk suspension and retain system. Everything looks good. I checked that the disk fits in the holder, and all dimensions are good. The coil holders are out for winding, so I couldn't test the movimentation yet.
Yesterday I cloned the cymac2 disk and installed it into the cymac3.
Jamie tweaked a few things (I can't really give more details) and now cymac3 is up an running with the same software as cymac2.
I compiled and installed the CR1 model, to readout the QPD. No more jumps in the signals!
To be able to access testpoints and have AWG working I had to follow the hack explained here: Cryo_Lab/781
I tested the following features:
The plot below shows the spectrum of the QPD. For the moment being I'm just sending a straight HeNe beam into the QPD, since the test setup with the disk is no more available. Units are arbitrary
I ran a set of COMSOL simulations to determine the dependency of the frequency of each eigenmode on the disk thickness and diameter, within the tolerances. I chose wide ranges: diameter 75.0 +- 0.1 mm and thickness 1.0 +- 0.1 mm, much more than the expected tolerances. It turns out that the frequencies depends almost exactly linearly on both variables: mostly on the thickness and negligibly on the diameter. The following plots shows: the mode shape and frequency (left), the frequency dependency on the two variables (center), the residual of a linear fit and the functional form of the fit (right).
I'm including only the modes that will be measurable by our system (no motion in the center, frequency below 32kHz. Since the disks in my simulation is completely round, I'm showing only one mode for each doublet.
So I’m sending a 1.123 Hz sinusoid into the ADC, and generating 3.123 Hz sines and cosines in the model. Frequencies are reasonably incommensurable with one second.
I finished populating the new four QPD boards, and fixed the first one I populated weeks ago. I tested all five new boards: the output of the transimpendance respond correctly to the ambient light; the output of the whitening also respond correctly and has increased high frequency noise; the differential driver stages are all functional and balanced.
In summary, we have six QPD circuits ready: serial 02 is installed into the box and it has been used for the previous tests. Serial number 01, 03, 04, 05, 06 are not yet into a box, but fully functional. Boxes are ready.
For testing purposed, I also built another ADC interface board: it's complete with the exception of the connector that goes to the ADC.
Today I populated four more QPD boards. All components are installed, except for the QPD themselves.
I wrote a C function to reconstruct the amlplitude and frequency of a line. It can be added as a block into a real time monitor. The idea is to use it to track in real time the frequency and amplitude of the disk modes, during the ring down. I did some tests and finally managed to get the function to compile and run on the cymac2 (the crackling lab cymac).
The following plot shows a simulation, since I can't run the code on the new cymac and I don't have the disk installed anymore. The top panel gives the amplitude of a decaying line, and the bottom panel the frequency offset from a reference local oscillator (more below). The nominal values are an initial amplitude of 1, frequency of 1109.375 Hz to be compared to a 1109.0 local oscillator. The fitted decay time is 10.005 seconds, to be comapred with 10 seconds nominal. There is some additive gaussian noise, that causes the ring down to be unmesurable after about 70 seconds of data.
This code will be used for real time estimation of the disk modes, once theot frequency has been roughly estimated with FFTs. The estimation of the frequency work remarkably well. In the first 20 seconds the mean value is 0.3747 Hz, with a standard deviation of 1.5 mHz. When the SNR gets worse (between t=30s and t=50s) the mean value is 0.3745 with a standard deviation of 20 mHz.
Because of the way it's built (see below) the code is sensitive to DC offsets, so the input signal must be high-passed.
The code is based on demodulation of the input signal with a reference local oscillator thta must have a frequency as close as possible to the line we want to track. The inputs to the block are: the signal to be monitored, sine of the local oscillator, cosine of the local oscillator. The outputs are: amplitude squared of the peak, frequency offset in Hz from the local oscillator.
Here's the math. Let's assume that the signal is
and the local oscillator has a frequency f0:
The code multiply the signal by the two local oscillators and average the result over 65kHz / 8 Hz samples. Therefore we get two output streams at 8 Hz which are
Then the sum of the two squared 8 Hz streams give an estimate of the amplitude squared. The code computes this every second
while the arctangent of the ratio gives a phase that varies linearly with time.
For each of the 8Hz samples the code computes the arctangent (using a home-brewed function based on a lookup table, since we can't import math.h in the RCG). It unwraps it, and then every second fit a line to the unwrapped arctangent, to estimate the frequency offset with respect to the local oscillator.
The C function has some parameters hard coded: the main sampling frequency (65536 Hz), the number of points per second to use for the frequency estimation (8 Hz), the fact that the output is computed every second. The first two parameters can be changed, the third one cannot for the moment being.
Installed the ADC and DAC boards into a proper box. Also, swapped the temporary DAC board (with cale hack) with the final one. Schematics and PCB are in the DCC: D1600196 and D1600301
The box is sitting on top of the cymac computer, on the back, since I don't have any long cable to connect the ADC.
Ordered the clean room (hardware+hepa filters) and vacuum gauges
Cleaned the chamber in the washing machine at 40m and started 48 baking at 120 C
As decided at out Red Door meeting, we're going to clean the vacuum chamber and move it to the large table, which will be enclosed in a clean room.
So today we disassembled and packaged the vacuum chamber, which is now ready to be trasnspoted to be cleaned and baked.
11:25am LT: closed valve between roughing and turbo pumps, switched off both pumps. Turbo pump is slowing down
After lunch I opened the chamber and removed everything from the inside.
The chamber around the vacuum gauge is really dirty now, see picture:
In addition, the electrostatic driver shows some signs of "burn" even though it was still working quite well. Unfortunately, whatever happened contaminated our sample:
This plot shows a signal which is generated purely digitally inside a code that I was testing.
Jumps are present even in this case.
Another addition: I was testing some simple code I was writing. Forget about OUT1 and OUT2, but OUT3 and OUT4 should be simple counters. Here's what I get by running the code on cymac3:
Since it didn't make any sense, I ran the same code on cymac2. Here's the (correct) result:
So there's something wrong even when all-digital signals are generated.
The old channel X3:CR1-PRESSURE_LOGTORR does not exist anymore. The new channel is now directly in torr and it is called X3:CR1-PRESSURE_TORR.
I had to write a C function to compute the 10^x operation, since it is not included in the RCG routines. Also it's not possible to include library functions, so I had to write an ad-hoc function, which first compute the integer part of the exponent, and then approximate the fractional part with linear interpolation and a look-up table. Code is in /opt/rtcds/userapps/release/models/pow10.c
Just to confirm that my noise estimates make sense, here's a plot of the not-normalized QPD signal that gives the X motion (sum and difference of all four quadrants):
This is the signal after compensating for the whitening filter. If I remove this compensation, the following plot gives the noises in terms of the voltage directly in input to the ADC (or in output of the analog board):
So the total "dark" electronic noise is about 13 uV/rHz.
I did a roughly estimate of the sources of electronic noise:
So the total noise at the outoput of the first stage is about 84 nV/rHz. The second stage adds a gain of 30 at high frequency, and negligible noise. So at the output of the whitening we have 2.5 uV/rHz. The DRV135 adds another gain of 2 and a neglegible output noise.
So the total electronic noise at the output of each quadrant is 5 uV/rHz. Since we are combining four of them, the total expected electronic noise is 10 uV/rHz, which is not too far from the measured value.
We are basically dominated equally by the Johnson-Nyquist noise of the TI resistor and by the input current noise of the LT1124. No gain to be obtained by changing the whitening.
I measured the noise sources limiting the QPD sensitivity. Unfortunately, I had to do some MATLAB tricks to get rid of the glitches: basically I load the data directly from the raw frames (NDS access to data is not working yet) and remove all jumps in the signals that happen in one single sample and are larger than a manually tuned threshold. This is not perfect, but it's enough to give us a rough idea of the spectrum of the QPD signal. The following plot shows the QPD_X signal (in units of disk motion, radians) in a few situations:
The total power on the QPD is 30 uW, which correspond to a shot noise limited sensitivity of 4.3e-12 W/rHz. Considering that the signal is the quadrant asymmetry normalized by the total power, the shot noise limited sensitivity is sqrt(2) * SN / Power which once calibrated corresponds to 1.1e-10 rad/rHz.
The following plot shows that shot noise is the dominant source, followed closely by the electronics dark noise. The total agrees perfectly with the measured background noise above 2 kHz. Below that we have some leakage due to the large turbopump peak: this is due to FFT limitations but mostly to unsuppressed glitches.
From the QPD datasheet (Hamamatsu S5981) I learn that the noise equivalent power should be of the order of 2e-14 W/rHz at the sensitivity peak, so probably a factor of two or so worse at the HeNe frequency. It's still much lower than the measured dark noise.
This sensitivity is already pretty good, but we can improve it by increasing the power on the diode. Indeed, 30 uW corresponds to about 2.7 V after the transimpedance, so we could increase the power by a factor 4 and win a factor 2 in the shot noise to dark noise ratio. Probably not worth it, since it will give us only a 30% gain in high frequency noise.
I measured the properties of the beam on the QPD. The total power is 31 uW. The beam shape is not gaussian, since we are seeing the interference of the reflection from the two surfaces:
The X and Y diameters are 1400 and 1300 microns, so I take the average of the two as an estimate of the beam size: 1300 +- 100 um. I also estimated the lever arm length to be 1.03 +- 0.02 m.
This allows me to esitmate the response of the normalized QPD signal to a tilt of the disk surface:
Plugging in the numbers gives a gain of (1900 +- 300) /rad for the normalized signals. I implemented those numbers in the filter banks: now X_NORM and Y_NORM have units of radians, and measure the disk surface angular motion. I also calibrated the SUM channel in microwatts, using the nominal responsivity of 0.45 A/W and the transimpedance of 200k (gain 11.1 uW/V)
Here's teh calibrated spectrum: note that the background noise is much larger than the real one because of the signal jumps.
Here are two more tests I did this morning
So in conclusion: the problem is in the cymac3 computer, either software or hardware. I tend to excluse an I/O hardware problem, since I used two different ADCs and removed the DAC, without improving the situation.
[Rolf, Ben, Rich, Gabriele]
Rolf couldn't find any good explanation on the software side for the signals jumps. He investigated a bit the reason why IOP takes a long execution time, without success. It's still mysterious why it ran with low time for a while.
One effect of Rolf activity is that now the signal jumps happen at ~0.8 seconds (after the start of each second) instead that ~0.45. This is suspiciously pointing to a software issue...
Ben and I spent a few hours trying to better understand the origin of the glitches. Finally, we plugged in a function generator to the four ADC channels, and we could find the glitches again, at the expected time. So we could rule out completely that it's a problem of the QPD analog electronics.
Some more ivnestigations:
On a bright side, Rolf recompiled the awgman software, and now excitation channels are working.
So here's the final proof that the glitches I see are digital:
I swapped the ADC board with a second one in the new cymac, but no change: glitches are still there.
Yesterday I could clearly see the glitches as jumps in the time domain plot of X and Y signals, and trace them to somewhat harder to see jumps in the quadrant signals.
One suspect was an intermittend oscillation of the transimpendance amplifier, so I looked into the schematics (D1600196) to see what could be the optimal value of the compensating capacitor C7. Following some useful notes online I computed the optimal value of C7 to be close to 2pF (instead of 10pF). I used 30-35 pF as the QPD capacitance, and 10 MHz has the gain-bandwidth product of the opamp. I swapped all 10pF capacitors with 2pF. After this I can still see the glitches in the spectra, but I can't find them anymore in the time domain. So things seem to have improved, although I still have annoying glitches.
Rich suggested to test the stability of the transimpendance stage by driving the output with a square wave and looking for the signal ringing. Here's his note:
I tried this for both the TI stage and the whitening stage, using 1k and 1uF and a square wave at 10 kHz. Here are the results, which look reasonable to me (firts is the TI, ringing at about 0.5 MHz, second is the whitening, almost no ringing):
So now I'm quite confident that the electronics is working. In the first trace you can see some intermittend background noise, due to the ambient light leaking into the QPD.
More investigations will follow.
Not so sure anymore...
I was suspecting dust crossing the beam, so I build a very rough enclosure, that should help with dust. I don't think I saw any change in the glitches.
So one might conclude that the glictches are produced by the analog QPD electronics. However, I plugged in a scope and I couldn't see any in the analog signal. But I checked only before the DRV135 stages. I'll need some sort of breakout board to test the output of the DRV135.
The glitches I saw in the data happens roughly every second, even though not exactly on the second. They are suddend jumps on the signal values over one sample, so of clear digital origin
We can't generate any arbitary signal with the real time model, since awg is not working properly yet. For the moment being I added a uniform random number generator in the model (only option I found for noise) and send it into the ESD filter bank. In this way I can generate band-passed noise.
I plugged in the DAC output to the HV amplifier input, and I could send white noise to the electrostatic drive. Behold: I was able to excite quite a few modes. In the following trace blue is a reference and red is right after I sent white noise (3 V peak to peak) to the disk for a while (less than 1 minute). Excitation stopped at 4:56pm LT.
Using COMSOL and tuning the disk thickness at 1.018 mm I could hit the frequency of the first butterfly mode (1109 Hz) and get a reasobly good estimate of the other modes.
After about half a hour of ring down, most of the modes are gone but the two lowest are still going strong.
Note that the flattish background noise seems to be generated by some sort of glitches. I tried to swap the laser and the power supply, without change. More investigations are needed.
Note that the roughing pump was still on during the test.
We connected the analog output of the vacuum gauge controller to one of the ADC channels. The signal is calibrated so that the pressure is 10^(X3:CR1-PRESSURE_LOGTORR_OUT). Unfortunately the RTG does not know how to compute 10^x...
Small modifications to the optical setup:
The real time system seems to be working properly, except for the excitations: we can't activate any excitation using awggui or diaggui
Eric rebuilt the workstation from scratch installing Debian 8.5. All CDS software seem to be working. We setup a ssh-key for ssh'ing into cymac3 and configured the automatic mount of the remote /opt/rtcds.
Here's the first spectrum of the QPD X and Y signals, acquired with the digital system. Roughing and turbo pumps are still on.
The noise floor seems quite non stationary. To be investigated.
I checked that the QPD electronics works as expected, and that I can acquire the signals using the ADCs. A new model (x3cr1) is up and running. It acquires the four quadrants, convert them from counts to volts, and compensate for the analog whitening filter. The four quadrant signals are X3:CR1-Q1_OUT, X3:CR1-Q2_OUT, X3:CR1-Q3_OUT, X3:CR1-Q4_OUT.
A matrix is used to compute the X and Y signals, defined as X = (Q1+Q4-Q2-Q3) and Y = (Q2+Q4-Q1-Q3). The SUM signal is also computed as SUM = (Q1+Q2+Q3+Q4).
Finally, the X and Y signals are normalized with the sum to produce X3:CR1-X_NORM_OUT and X3:CR2-Y_NORM_OUT.
A filter bank (ESD) is connected to the DAC channel #0 to produce the excitation that will be sent to the high voltage amplifier. I checked that the DAC is working properly (adding offsets). The input to the ESD filter bank is in volts.
The normalized X and Y signals, the sum of all four quadrants and the output of the ESD driver filter bank are saved to frames. The model runs at 65kHz.
The QPD quadrants are wired accoridng to the following convention
Today at 1:40pm pressure is 8.5e-7 Torr
Following Alena's procedure, at about 1:30pm LT I started the chamber pump down. At 14:15pm LT the pressure was still 240 mTorr
At 6:20pm the pressure was about 70 mTorr, so I started the turbo pump.
I connected the QPD to the ADC interface with a temporary cable running on the floor.
I could get signals. I still have a problem with the digital system: I can't access test points with dataviewer, but I get them with DTT. This will have to be fixed.
Using the already installed high voltage feedthrough, I cabled one of the electrostatic actuators (1mm gap between electrodes) and installed it into the chamber. One of the electrodes is connected to the feedthrough cenral pin, the other is grounded on the bottom of the chamber.
The electrostatic actuator is mounted at about 1 mm above the disk, see pictures.
As a preliminary test, I checked that switching the HV amplifier on and off with about 1.5kV produces a visible motion (~2-3 mm) of the optical lever beam. So the actuator is working.
The laboratory workstation is coatings.ligo.caltech.edu
The RTS is cymac3.ligo.caltech.edu
I set up a ssh-mount of the /opt/rtcds/userapps folder in the workstation. I also created shared ssh keys for the controls user, so we can ssh into the cymac3 without password
Please see prosidures for pumping down and venting with air for the test vacuum chamber here https://dcc.ligo.org/T1600304
We played around a bit with the cymac, in efforts to make things better.
As I see it, the main problems that persist are:
This afternoon I completed the assembly of the electronics boards to interface the ADC and DAC. The ADC is interfaced with a new custom board, which accepts up to eight QPD inputs, the syncronization signal, and it's connected to the ADC:
For the DAC I used one spare board from the Crackle experiment. However, that board had a wrong pinout for the DAC side connector, so I had to implemented again the same hack I did for the crackling noise experiment.
All boards are connected to the ADC and DACs, and to the syncronization signal generated with a SR DS345. No boxes for the moment being, I'll figure out a better organization of the boards in the future if needed. I still haven't tested if the real time system is able to communicate properly with the new interfaces.
At 5pm the pressure was 6.5e-6 torr.
Checked again at:
Installed a gate valve between the roughing and tubbo pump. See below a pump down curve. The convection gauge is not calibrated. the turbo started at 14th min (at about 3 torr)
We attached one of the silicon lenses to a 1" optical post using some kapton tape, and installed it into the vacuum chamber. We built a simple periscope using standard optical component, and managed to send the optical level beam into the disk and back out.
To set a reference for the horizontal position of the disk we used the LMA method: we put a small container with water in place of the disk, and mark on a reference where the reflected beam hits out of the chamber:
We then put back the disk, and aligned it to have the beam hitting the same position. During pumdown we couldn't see any shift of the disk, judging from the position of the optical lever beam.
I copy here parts of an email from Jamie with instructions on how to run the RTS on the cymac:
I had to change the site/ifo to be "caltech/x3" to avoid EPICS
collisions with other cymacs. You'll need to update your models to
reflect this change (change names, site/ifo parameters in the model,
etc.). So for instance, change x1iop -> x3iop, and update the params
daqd is installed and running, although I haven't really stress tested
it yet (no models/channels/etc). It's running under systemd, which is
the Debian service management system. You can control daqd with the
# systemctl restart daqd
# systemctl stop daqd
# systemctl start daqd
# systemctl status daqd
# journalctl --unit=daqd
controls@cymac3:~ 0$ systemctl status daqd
● daqd.service - Advanced LIGO RTS daqd service
Loaded: loaded (/etc/systemd/system/daqd.service; enabled)
Active: active (running) since Sat 2016-07-09 11:22:38 PDT; 23h ago
Main PID: 20827 (daqd-standiop)
└─20827 /usr/bin/daqd-standiop -c /etc/advligorts/daqdrc
controls@cymac3:~ 0$ sudo journalctl --unit=daqd
Remember to restart daqd ("systemctl restart daqd") after you add/change
I've gotten all the RCG components working, but not without some small
kinks. The RCG currently expects a specific EPICS install, different
than the system install we're using right now. I've hacked a way to
make this work seemlessly for model builds, but it requires sourcing the
following file before *starting* models:
So in the mean time, you can start/stop models via the following:
$ . /opt/rtapps/epics/etc/epics-user-env.sh
In the last days Jamie installed the patched kernel to run the real time system (RTS) on the new CyMAC. Today (with Jamie's remote advices) I managed to get a IOP (input output process) model compiled and running. There is still no timing input (to be fixed at the beginning of the netx week, I'm presently missing a connector which is on order).
Running the x1iop model gives:
x1iopepics: no process found
Number of ADC cards on bus = 1
Number of DAC16 cards on bus = 1
Number of DAC18 cards on bus = 0
Specified filename iocX1.log does not exist.
x1iopepics X1 IOC Server started
controls@cymac3:/opt/rtcds/tst/x1/scripts$ awg_server Version $Id: awg_server.c 2917 2012-05-22 22:33:39Z alexander.ivanov@LIGO.ORG $
channel_client Version $Id: gdschannel.c 4170 2016-04-05 21:24:46Z jonathan.hanks@LIGO.ORG $
testpoint_server Version $Id: testpoint_server.c 3303 2013-03-05 23:33:45Z alexander.ivanov@LIGO.ORG $
/opt/rtcds/tst/x1/target/gds/bin/awgtpman -s x1iop -4 -l /opt/rtcds/tst/x1/target/gds/awgtpman_logs/x1iop.log started on host cymac3 hostid ffffffffd783d97b
awgtpman Version $Id: awgtpman.c 4170 2016-04-05 21:24:46Z jonathan.hanks@LIGO.ORG $
1) install MATLAB and add to startup.m the following line
2) copied from cymac2 the file /opt/rtcds/rtcds-user-env.sh and changed the content to match the right folders
3) installed readline library which was missing: apt-get install
sudo apt-get install libreadline-dev
4) installed linux headers sudo apt-get install linux-headers-3.2.0-rts-amd64
5) created symbolic link to linux headers:
sudo ln -s /usr/src/linux-headers-3.2.0-rts-common /usr/src/linux
6) changed host name to cymac3
sudo vi hostname
7) created symbolic link to lspci
sudo ln -s /usr/bin/lspci /usr/sbin/lspci
8) created a symbolic link to awgtpman
ln -s /usr/bin/awgtpman /opt/rtcds/tst/x1/target/gds/bin/awgtpman
Fixed the 307 gauge controller (a missing contact on the rair panel). The low pressure gauge was connected to 1G port and has measured 1.7E-6 torr. We are not sure since how long the turbo was operating (no vacuum logger yet).
This afternoon we opened the tall belljar vacuum chamber, and took everything out of it. All the stuff that was inside the chamber is "temporarily" stored on the floor beside the optical table.
We installed a "spacer" into the chamber, made from one of the optical table legs that were sitting in the hallway. We installed one of the aluminum base plates on top of it, so that the optical components will be at the level of the viewport. Another leg and a thinner base plate are installed out of the chamber, at a similar level.
After this we closed the chamber with one of the flats used for the old chamber, and a rubber o-ring. We started the roughing pump, quickly reached a pressure below 1 mTorr and switched on the turbo pump. Unfortunately, it seems that the low pressure gauge is not working properly, so we don't know what's the pressure right now. We'll check the gauge and controller tomorrow morning and swap it out if needed.