Looks like 2 different MEDM Snapshot functiions (at least) are broken.
The regular update of the screens here as well as the usual "Update Snapshot" and view "previous snapshot" button on all of the auto-generated screens.
Also, how do we add the snapshot button to the custom made screens?
There has been some input matrix diagonalization in the past by Yuta and Kiwamu, but I find the automation to be not totally satisfactory.
It would be better if we could automatically fit the data to find the Suspended optic eigenfrequencies and then use that to get the matrix. So I wrote a peak fitter to get the matrix.
It gets the data from mafalda with NDS2, then it makes the PSDs, and then starts with some initial guesses (based on looking at the plots) and them uses fminsearch to get the peak frequencies and Q's.
Using the output of this, we can use Yuta's method and take the passive transfer functions with the free swing data (from April 30, so we got do do it quick) to get the input matrix.
Doing the SUS input matrix is nice for having good damping (as long as we remember to include SIDE), but my motivation is to produce a good null stream from the 4 face sensors so that we can estimate the sensor noises at all times.
Just FYI, the 3113 is a 12-bit ADC, so we won't get very good resolution out of these. We should get one of those purple boxes.
Also, I corresponded some with Dave Barker. The 40m Wiki at LHO is down because of disk errors. He's working on it and will let us know. But suggests that we move it to Caltech since he'll turn the box off at the end of the year.
Mon May 16 13:57:49 2011 Wiki is back up.
This is OK....but, the input matrix should come from the same place as the regular input matrix: i.e. it should be just another row like CARM, DARM, etc. rather than have its own screen.
Also, I think a nice mod to all the matrices would be if the ORANGE triangle was only visible when there's a signal flowing through it.
Aka, from a hotel in Pisa.
Restarted Thu May 19 00:21:49 2011 to recover from Jenne's Italian terrorism.
I've moved all of my SOS peak fitting stuff into the scripts area so that Leo can make it better:
findPeaks.m gets the data and makes the fitted spectra that I put in the previous entry.
findMatrix.m is the barely started script that ought to take the TF data and output the matrix to the MEDM screen.
I've finished tuning POY11 and it is now sitting on top of the analyzer waiting for Koji to test its noise.
Some weeks ago, Joe, Jamie, and I reworked the ETMY controls.
Today we found that the model rebuilds and BURT restores have conspired to put the SUS damping into a bad state.
1) The FM1 files in the XXSEN modules should switch the analog shadow sensor whitening. I found today that, at least on ETMY and ETMX, they do nothing. This needs to be fixed before we can use the suspensions.
2) I found all of the 3:30 and cts2um buttons OFF AGAIN. There's something certainly wrong with the way the models are being built or BURTed. All of our suspension tuning work is being lost as a consequence. We (Joe and Jamie) need to learn to use CONLOG and check that the system is not in a nonsense state after rebuilds. Just because the monitors have lights and the MEDM values are fluctuating doesn't mean that "ITS WORKING". As a rule, when someone says "it seems to work", that basically means that they have no idea if anything is working.
3) We need a way to test that the CDS system is working...
I focused these lenses so that we could get a clean image of the mirrors and the OSEMs.
Our goal is to have an image where the optic diameter almost fills the entire monitor. We want the focus to be adjusted for the YAG beam (which is almost the same as focusing for the OSEMs). This will NOT produce a nice image of the cage using visible light, but that is just fine.
When Justin Garofoli was here he found a nice lens combo that did the job, so if anyone can find his old email or elog, lets just go back to that.
For now, we do not need a camera/lens system to focus very tightly on the center of the optic.
While answering a quick question by Kiwamu, I noticed we only had second trends going back to 99050000 GPS time, May 27th 2011.
Trends (I thought) were intended to be kept forever, and certainly longer than full data, which currently goes back several months.
Jamie will need to look into this.
Our concept is to keep second trends for 1-2 months and minute trends forever. The scheme that Alan had worked out many years ago had it such that we could look back to 1998 and that the minute trends would be backed up somehow.
If its not working, we need to get Alan's help to recover the previous configuration.
Koji and I found 2 RFPD boxes to send to LLO. We've put them onto Steve's desk to be overnighted to Valera.
One of them is our old 21.5 MHz gold box RFPD from the FSS (which we don't use). The other one is a 2mm gold box one which was previously tuned for 66 MHz.
They shipped out on Friday
There was a rogue, undocumented, gateway process running on NODUS since ~4 PM. This guy was broadcasting channels back into the Martian and causing lockups in the IOO controls. I did a kill -9 on its process.
Someone will pay for this.
I put a paper Peet's bag with half of the Mini-Moos into George.
All the suspensions are bad until you fix them. But, ... there is a script which can be used to diagnose them today:
When I try to get minute trend, it says "word too long".
I have updated the scripts/SUS/peakFit/ directory so that it now finds the SUS input matrix coefficients in addition to just finding the free-swinging peaks.
The attached PDF shows how much rejection of the unwanted DOFs we get between the existing diagonal input matrix and this new empirical matrix. Previously, the decoupling was only a factor of a few for some of the modes. Now the decoupling is more like orders of magnitude (at least according to this calculation). It will be worse when we load it and then try another free swinging run. However, the fact that the suppression can be this good means that the variation in the coefficients at the ~hours time scale is at least this small (~< 0.1%)
That's the basic procedure, but there are a lot of important but mainly technical details:
I'll set the optics to be aligned and then swing tonight.
I used scripts/SUS/freeswing-all.csh to give the optics a kick and then turn off their watchdogs and collect the free swinging data. Final script end time = 993173551. Start taking data ~ 993173751
I had to fix up the script a little: it had amateur stuff in there, such as undefined variables.
It still doesn't work that well. On the new Ubuntu workstations, pianosa, it fails by just not setting some of the EPICS variables using the EZCA stuff.
On Allegra, it failed on ~1 out of 10 commands by returning "epicsThreadOnce0sd epicsMutexLock failed" ???
On Pianosa, it sometimes says, instead, "epicsThreadOnceOsd: pthread_mutex_lock returned Invalid argument.". Ah...now I understand?
epicsThreadOnceOsd: pthread_mutex_lock returned Invalid argument.
So finally, I had to run the script on op340m to get it to actually run all of its commands. That's right; I used a 15 year old Solaris 9 Blade 150 because none of our fancy new Linux machines could do the job reliably.
Fixing our EZCA situation is a pretty high priority; if the locking scripts fail to run ~1 command every hour its going to completely derail the lock acquisition attempts.
If you want to use the IFO tonight, just run the script again on op340m again when you're done.
Since the MC1 LRSEN channel is not wasn't working, my input matrix diagonalization hasn't worked today wasn't working. So I decided to fix it somehow.
I went to the rack and traced the signal: first at the LEMO monitor on the whitening card, secondly at the 4-pin LEMO cable which goes into the AA chassis.
The signal existed at the input to the AA chassis but not in the screen. So I pressed the jumper wire (used to be AA filter) down for the channel corresponding to the MC1 LRSEN channel.
It now has come back and looks like the other sensors. As you can see from this plot and Joe's entry from a couple weeks ago, this channel has been dead since May 17th.
993190663 = free swinging ringdown restarted again
The slow readback of the ETMX side seems to also have something flaky and bi-stable. This is not an issue for damping, but it disables the SIDE watchdog for ETMX and makes it unsafe if we accidentally use the wrong damping sign.
Chris Wipf tells me that the EPICS Mutex Jumbo Mumbo can be overcome by upgrading our EPICS. We should get one of Jamie's assistants to get this going on one of the Ubuntu workstations.
Today I wanted to investigate the MC Length path situation for obscure reasons.
Jamie has started to revert the "ALTPOS" effect on the MC mirrors. So far, the screens and SLOW channels have been fixed, but the fast channels still say "ALTPOS" in the dataviewer instead of "MCL".
I also noticed that all of our old ADCU channels for diagnosing the PSL, MC, ISS, PMC ,etc. are completely AWOL. Let's blame Joe.
I think that there are probably some ADC channels available and that we'll just have to figure out what Joe intended for this. We certainly need it if we want to diagnose our PMC, ISS, FSS, MC, etc. Kiwamu tells us that the old PSL/IOO AA chassis is being used for some of the GCV signals, so its likely that we just have to do the appropriate channel name mapping in the DAQCONFIG tool.
Forging ahead with no data, I made up some filters in the MC2-MCL filter bank so that there could be a stable crossover between the laser path. I was able to turn it on and get some suppression of the FSS-FAST control signal, but there's no way to be sure without the fast channels. We gotta get Jamie to help us out once he finished the ETM BO mess.
As I recently had trouble getting all of the SUS SENSOR channels at once from NDS2, I asked J.Z. for help. He found that the number of buffers on mafalda was set to only allow a small amount of data to be requested at one time.
He's going to have to figure out a more permanent fix, but for now he's increased the data buffer size to allow somewhat larger chunks to be gotten. I have made a work around in matlab, which gets smaller chunks and then cats them together.
Its in SUS/peakFit/.
What is implicit in Suresh's entry is that we decided to run the WFS with the 10 dB internal attenuation set to ON as the nominal. In the past, we have always had all the attenuation OFF for max gain. The layout of the WFS is such that we get that nasty 200 MHz oscillation due to crosstalk between the 2 MAX4106 opamps for each quadrant. The 10 dB attenuator is able to reduce the positive feedback enough to damp the oscillation.
In principle, this is still OK noise-wise. I think the thermal noise of the resonant circuit should be ~2-3 nV/rHz. Then the first opamp has a gain of 5, then the -10 dB attenuator, then another gain of 5. The noise going to the demod board is then ~10-15 nV.
The real noise issue will be the input noise of the demod board. As you may recall, the output of the AD831 mixer goes to a AD797. The AD797 is a poor choice for this application. It has low noise only at high frequencies. At 10 Hz, it has an input voltage noise of 10 nV/rHz and a current noise of 20 pA/rHz. If we wanted to use the AD797 here, at least the RC filter's resistor should be reduced to ~500 Ohms. Much better is to use an OP27 and then choose the R so as to optimize the noise.
We should also be careful to keep the filter frequency low enough so as not to rate limit the OP27. From the schematic, you can see that this circuit is also missing the 50 Ohm termination on the output. There ought to be the usual high-order LC low pass at the mixer output. The simple RC is just not good enough for this application.
As a quick fix, I recommend that when we next want to up the WFS SNR, we just replace the RC with an RLC (R = 500 Ohms, L = 22 uH, C = 1 uF).
Actually, ETMY was the only good one. They should all have the 30 Hz High pass as the damping filter. I think these details are in the elog entry that we originally made while doing ETMY.
They should all also have a 3:30 in the XXSEN to compensate the whitening. The logic is supposed to be that FM1 is ON when the hardware whitening is ON. This is the opposite of the old logic and its why the damping filter has to be moved from 3 to 30 Hz.
MC1 MC2 MC3 ETMX ETMY ITMX ITMY PRM SRM BS mean std
Pitch 0.671 0.747 0.762 0.909 0.859 0.513 0.601 0.610 0.566 0.747 0.698 0.129
Yaw 0.807 0.819 0.846 0.828 0.894 0.832 0.856 0.832 0.808 0.792 0.831 0.029
Pos 0.968 0.970 0.980 1.038 0.983 0.967 0.988 0.999 0.962 0.958 0.981 0.024
Side 0.995 0.993 0.971 0.951 1.016 0.986 1.004 0.993 0.973 0.995 0.988 0.019
There is a large amount of variation in the frequencies, even though the suspensions are nominally all the same. I leave it to the suspension makers to ponder and explain.
This CSHRC mangling on Feb 4 did more than re-arrange FB binaries.
It broke the path to MEDM for the 32-bit machines in the lab (e.g. mafalda) and stopped the MEDM snapshots from being posted onto our MEDM Status Web Page.
This is because, in addition to the paths mentioned in the above elog, the paths to the EPICS directories were also commented out. I've re-inserted them into our
.cshrc file in the 32-bit section; the statScreen CRON that Yoichi set up is now back in business.
* for some reason, the 'cronjob.sh' script is wiping out its own log file. It would be great if someone who understands stderr output re-direction can fix it so that the log-file from each run is retained until the next time cron runs.
This is getting closer, but with the whitening left OFF and the cts2um filter also OFF, none of the suspensions are working correctly. I'm shutting down all the watchdogs until someone gets around to setting the damping gains and filters correctly.
I'm attaching a screenshot of some of the problems I see so far with MC3.
I'm going to try to get the MC suspensions working OK for tonight so that we can use them for the PRMI locking work.
Update #1: None of the MC SUS DAQ channels are found by dataviewer....SUS debugging speed reduced by 10x. Tue Jul 05 21:38:17 2011
Update #2: POS/PIT/YAW BIAS sliders now seem to work, but are ~1000x too weak to do anything. Tue Jul 05 21:41:38 2011
In addition to the OL quadrants, you need to plot the OPLEV_PERROR and OPLEV_YERROR signals since these are the real signals we use for finding the mirror motion. If they're not in the Dataviewer, Jamie should add them as 256 Hz DAQ channels (using these names so that we have the continuity with the past). These DAQ channels correspond to the IN1 channels for the OL filter banks.
Also JPG are banned from the elog - you should put all of the plots into a single, multipage PDF file in honor of the new Wagonga.
At 0 dB, the resistor noise is only 30 nV/rHz, whereas the ADC noise is more like 10000 nV/rHz...
I installed Virtual Box on rossa. Then I put Windows 7 in there and am now installing Altium.
You can run Windows on rossa by just clicking Applications -> System Tools -> Virtual Box.
I guess Valera forgot to elog it. Steve, please email him and get the info.
I started to check out the OL servos today so that our whole interferometer is not too floppy.
This is sort of OK, except the capacitor connects across the (+) terminals of the two input opamps, and does not connect to ground.
Also, we don't care about the CMRR at 64 kHz. We care about it at up to 10 kHz, but not above. The sample frequency of the ADC is 64 kHz, but all of the models run at 16 kHz or less, so the Nyquist frequency is 8 kHz.
And doesn't the value depend on the resistors?
Looks like either the LR OSEM is totally mis adjusted in its holder or the whitening eletronics are broken.
Also looks like the ETMY is just not damped at 1 Hz? How can this be?
I look at the SUS_SUMMARY screen which apparently only Steve and I look at:
Looks like the suspensions have factor of 10-100 different gains. Why?
** The ETMY just doesn't behave correctly when I bias it. Both pitch and yaw seem to make it do yaw. I leave this for Jamie to debug in the morning.
*** Also, the BIAS buttons are still broken - the HOPR/LOPR limits ought to be 5000 and the default slider increment be 100. Also the YAW button readback doesn't correctly show the state of the BIAS.
**** And.....we have also lost the DAQ channels that used to be associated with the _IN1 of the SUSPOS/PIT/YAW filter modules. Please put them back; our templates don't work without them.
pianosa:gpib 0> ./readSR785.csh rb2
netgpibdata.py: Command not found.
Hmm. Should have only been +/- 1 GHz. Some setting got changed apparently...
This is a part of the RefCav temperature measurement setup. You'll get an elog from Jenny very soon.
I turned the RefCav heater and servo back on a couple days ago. At first it was stabilizing again at a low setpoint, but in reality the right temperature (~40 C). After fixing the in-loop signal offsets, the setpoint now correctly reflects the actual temperature.
Jenny is going to calibrate the sensors using some kind of dunking cannister next week.
wow. This Monte-Carlo matrix is one of the most advanced optical modeling things I have ever seen. We never had this for any of the interferometers before.
I made some attempts to measure the current length of the arm cavities by using the mass-kicking technique.
Why not just scan the Green laser to measure the arm lengths instead? The FSR of the arm is ~3.75 MHz and so all you have to do is lock the arm green and then sweep the PZT until the resonance is found at 3.75 MHz.
Remember that the Oplevs are not good references because of the temperature sensitivity. The week long trend shows lots of 24 hour fluctuations.
This OSEM placement is just the OPPOSITE of what the proper placement is.
Usually, we want to put them in so that the LED beam is vertical. This makes the OSEM immune to the optic's vertical mode.
The orientation with the horizontal LED beam makes the immunity to the side mode better, but may spoil the vertical.
In reality, neither of these assumptions is quite right. The LED beam doesn't come out straight. That's why Osamu and I found that we have to put in some custom orientations.
Also, the magnet gluings relative to the OSEM bracket centers are not perfectly aligned. So...I am saying that the OSEMs have to be oriented empirically to reduce the couplings which we want to reduce.
I've finally completed the SUS/peakFit/ scripts which find the new input matrix for the SUS. MC1, MC2, MC3, and ITMX have been matrix'd.
I tried to do the BS, but it came out with very funny matrix elements. Also the BS is missing its DAQ channels again (JAMIE !) so we can't diagnose it with the free swinging method.
To continue, we have to get some good data and try this again. Right now there are some weird issues with a lot of the optics. I've also set the damping gains for the optics with the new matrices.
new_matrix = findMatrix('ITMX')
this script writes the values to the MEDM input SUS matrix. To do the writing, I used the low level 'caput' command instead of ezcawrite since the ezca libraries are getting deprecated.
caput doesn't really have good diagnostics, so I use matlab to check the return status and then display to the terminal. You can just rerun it if it gives you an error.
A coupled of normalization notes:
1) The POS/PIT/YAW rows are scaled so that the mean of abs(FACE elements) = 1. Previously, I had the max element = 1.
2) The SIDE row is scaled so that the SIDE element = +1.
3) I then normalized the ROWS according to the geometrical factors that Jamie has calculated and almost put into the elog.
All these scripts have been added to the SVN. I've removed the large binary data files from the directory though. You can just rsync them in to your laptop if you want to run this stuff remotely.
Besides the purpose of correctly tuning the suspensions, my hidden goal in the input matrix diagonalization has been to figure out what the 'true' sensing noise of the OSEMs is so that we can accurately predict the noise impact on the OAF.
The attached plot shows the DOFs of ITMX calibrated into microns or microrad as per Jamie's ethereal input matrix calculations.
The main result is in the ratio of POS to BUTTER. It tells us that even at nighttime (when this data was taken) we should be able to get some reduction in the arms at 1 Hz.
Whether we can get anything down to 0.1 Hz depends on how the arm control signal compares to the POS signal here. I leave it to Jenne to overlay those traces using a recent Arm lock.
A plot showing that the daily variation in the OLs is sometimes almost as much as the full scale readout (-1 to +1).
I'm pretty sure that don't have any ADC's with this gain. It should be +/- 10V for 16 bits.
For some reason the workstation at the vac rack was off and unplugged. Nicole and I plugged its power back in to the EX rack.
I turned it on and it booted up fine; its not dead. To get it on to the network I just made the conversion from 131.215 to 192.168 that Joe had done on all the other computers several months ago.
Now it is showing the Vacuum overview screen correctly again and so Steve no longer has to monopolize one of the Martian laptops over there.
In the previous elog of mine, I looked at the nullstream (aka butterfly mode) to find out if the intrinsic OSEM noise is limiting the displacement noise of the interferometer or possibly the Wiener FF performance.
The conclusion was that its not above ~0.2 Hz. Due to the fortuitous breaking of the ITMX magnet, we also have a chance to check the 'bright noise': what the noise is with no magnet to occlude the LED beam.
As expected, the noise spectra with no magnets is less than the calculated nullstream. The attached plot shows the comparison of the LL OSEM (all the bright spectra look basically alike) with the damped
optic spectra from 1 month week ago.
From 0.1 - 10 Hz, the motion is cleanly larger than the noise. Below ~0.2 Hz, its possible that the common mode rejection of the short cavity lengths are ruined by this. We should try to see if the low frequency
noise in the PRC/SRC is explainable with our current knowledge of seismicity and the 2-dimensional 2-poiint correllation functions of the ground.
I just tried to adjust the ETMY camera and its not very user friendly = NEEDS FIXING.
* Camera view is upside down.
* Camera lens is contacting the lexan viewport cover; this means the focus cannot be adjusted without misaligning the camera.
* There's no strain relief of the camera cables at the can. Needs a rubber cable grommet too.
* There's a BNC "T" in the cable line.
Probably similar issues with some of the other setups; they've had aluminum foil covers for too long. We'll have a camera committee meeting tomorrow to see how to proceed.
(What we did)
* Moved SUS to edge of table for OSEM adjustment.
* Leveled the table in this temporary tower position.
* Rotated all OSEMs to give some seperation between magnets and LED/PD packages.
* Moved the upper OSEM bracket a little bit upward.
* All the OSEM holding set screws were short with flat heads; this is annoying since we would like to use them more like thumbscrews. Steve took the long set-screws out of the old ITMX cage and we swapped them. Need to order ~100 silver-plated socket head spare/replacements.
* Took pictures of OSEMs.
* Moved tower back to old position.
* Releveled the table (added one rectangular weight in the NW corner of the table).
* Find that ITMX OSEMs were a couple 100 micron out of position; we adjusted them in-situ in the final position of the tower, trying not to rotate them. All mean voltages now are within 100 mV of ideal half-light.
* Back/front EQ positions adjusted by the screw method. bottom/top stops adjusted earlier.
* OSEM cables tied down with copper wire.
* Increased the incident power up to 91 mW going into MC to temporarily make the POX beam more visible.
* The POX beam was checked. It was exiting from the chamber and going through about the center of the viewport.
After Steve pointed out the 'deep hoop' issue, we decided to examine putting an RF Amp on the PSL table, between the RF combiner and the triple resonant box.
This will reduce the chances of standing waves in the cables and reduce the radiation induced pick-up in the RF PD and Demod electronics.
We would like to send ~10 dBm from the distribution box to the combiner. We also want to able to get as much as ~33 dBm of drive at 11 and 55 MHz. So the amp should have a gain of ~20-30 dB and an operating range of 10-100 MHz.
Also desirable are low distortion (high IP3) and good reverse isolation ( > 40 dB).
Some possibilities so far (please add your RF Google Results here):
1) Mini-Circuits ZHL-1-2W-S: G = +32 dB, Max Out = +33 dBm, NF = 6 dB, Directivity = 25 dB
2) Mini-Circuits TIA-1000-1R8: G=+40 dB, Max Out = +36 dBm, NF = 15 dB (AC Powered, Inst. Amp), Directivity = 58 dB.
3) Mini-Circuits ZHL-2-8: G = +27dB, Max out = +29 dBm, NF = 6dB, Directivity = 32 dB
4) RFbay MPA-10-40: G = +40dB, Max Out = + 30 dBm, NF = 3.3 dB, Rev Iso = 23 dB
5) No proper stuff from Teledyne Couger
By looking at what Daniel used in the low noise EOM Driver for aLIGO, we found the A2CP2596 from Cougar.
G = +24 dB, NF = 5 dB, Max Out = +37 dBm. It comes in a 2-stage SMA connector package. I've asked Steve to order 2 of them with the appropriate heatsinks.