Sun Jun 21 00:06:43 PDT 2009
Mar 6 15:46:32 nodus sshd: [ID 800047 auth.crit] fatal: Timeout before authentication for 184.108.40.206
Mar 10 11:11:32 nodus sshd: [ID 800047 auth.crit] fatal: Timeout before authentication for 220.127.116.11
Mar 11 13:27:37 nodus sshd: [ID 800047 auth.error] error: connect_to 18.104.22.168 port 7000: Connection refused
Mar 11 13:27:37 nodus sshd: [ID 800047 auth.error] error: connect_to nodus port 7000: failed.
Mar 11 13:31:40 nodus sshd: [ID 800047 auth.error] error: connect_to 22.214.171.124 port 7000: Connection refused
Mar 11 13:31:40 nodus sshd: [ID 800047 auth.error] error: connect_to nodus port 7000: failed.
Mar 11 13:31:45 nodus sshd: [ID 800047 auth.error] error: connect_to 126.96.36.199 port 7000: Connection refused
Mar 11 13:31:45 nodus sshd: [ID 800047 auth.error] error: connect_to nodus port 7000: failed.
Mar 11 13:34:58 nodus sshd: [ID 800047 auth.error] error: connect_to 188.8.131.52 port 7000: Connection refused
Mar 11 13:34:58 nodus sshd: [ID 800047 auth.error] error: connect_to nodus port 7000: failed.
Mar 12 16:09:23 nodus sshd: [ID 800047 auth.crit] fatal: Timeout before authentication for 184.108.40.206
Mar 14 20:14:42 nodus sshd: [ID 800047 auth.crit] fatal: Timeout before authentication for 220.127.116.11
Mar 25 19:47:19 nodus sudo: [ID 702911 local2.alert] controls : 3 incorrect password attempts ; TTY=pts/2 ; PWD=/cvs/cds ; USER=root ; COMMAND=/usr/bin/rm -rf kamioka/
Mar 25 19:48:46 nodus su: [ID 810491 auth.crit] 'su root' failed for controls on /dev/pts/2
Mar 25 19:49:17 nodus last message repeated 2 times
Mar 25 19:51:14 nodus sudo: [ID 702911 local2.alert] controls : 1 incorrect password attempt ; TTY=pts/2 ; PWD=/cvs/cds ; USER=root ; COMMAND=/usr/bin/rm -rf kamioka/
Mar 25 19:51:22 nodus su: [ID 810491 auth.crit] 'su root' failed for controls on /dev/pts/2
Jun 8 16:12:17 nodus su: [ID 810491 auth.crit] 'su root' failed for controls on /dev/pts/4
12:06am up 150 day(s), 11:52, 1 user, load average: 0.05, 0.07, 0.07
12:06am up 150 day(s), 11:52, 1 user, load average: 0.05, 0.07, 0.07
I set the MC back to its good alignment (June 21st) using this procedure. The trend of the OSEM values over the last 40 days and 40 nights is attached.
Then I aligned the periscope to that beam. This took some serious periscope knob action. Without WFS, the transmission went to 2.7 V and the reflection down to 0.6V.
Then I re-aligned the MC_REFL path as usual. The beam was far enough off that I had to also re-align onto the MC LSC PD as well as the MC REFL camera (~2 beam radii).
Beams are now close to their historical positions on Faraday and MC2. I then restored the PZT sliders to their April snapshot and the X-arm locked.
Steve - please recenter the iris which is on the periscope. It has been way off for a long time.
So it looks OK now. The main point here is that we can trust the MC OSEMs.
Afterwards I rebooted c1susvme1 and c1susvme2 because they were skewed.
Just now found it dead. Restarted it. Is our elog backed up in the daily backups?
we have a tester cable, but you don't want it. Instead the problem is probably at the cross-connect. The D-cable goes to a cross-connect and you can probe there with a voltmeter. If the signal is good there, trace it to the ADC. Also trend for several years to see when this happened - Yoichi may know the history better.
Also, we still need to complete the FSS RFPD task list from last year.
I have added/modified SMOO settings to all of the records in psl.db appropriately. Changes checked in to SVN.
As a reminder, you should check in to the SVN all changes you make to any of the .db files or any of the .ini files in chans.
I found that several of the cables are unlabeled so I'm not sure what's plugged in. In the end, I found that the TEMP_2, _3, & _44 channels were working and so I plugged in anything that looked seismic into there.
TEMP_2 is now apparently the X channel of the 2nd Guralp. If someone can figure out which cable belongs to the Y channel, please plug it into TEMP_3 and then we can fix the channel names.
I also removed (gently) all of the accelerometers from MC2's chamber. This didn't break the lock, but I intentionally broke it to make sure it reacquired fine. It did and the MC TRANS QPD showed no significant shift afterwards.
Friday, we were seeing a 2 Hz harmonic series in all of the PEM channels. Today I found that some bad person had put in a 4V (!) signal into one of the channels with a signal generator. The generator was also sneakily stuck way back inside the DCU rack. NO SECRET SIGNAL INJECTIONS!
Since the ADC has a 2Vpk range, this was saturating and putting in harmonics in all the adjacent channels. I disconnected it and turned off the function generator.
This is very nice. We have, for the first time, a real time plant with which we can test our changes of the control system. From my understanding, we have a control system with the usual POS/PIT/YAW matrices and filter banks. The outputs go to a separate real-time system which is running something similar and where we have loaded the pendulum TF as a filter. Cross-couplings, AA & AI filters, and saturations to come later.
The attached plot is just the same as what Peter posted earlier, but with more resolution. I drove at the input to the SUSPOS filter bank and measured the open loop with the loop closed. The loop wants an overall gain of -0.003 or so to be stable.
Looks like someone rebooted nodus at ~3 PM today but did not elog it. Also the SVN is not running. Why?
Rana, Jan, Jenne
We noticed that the Ranger data was all bogus at low frequencies. So we checked it and found that the proper procedure had not been used when changing it from horizontal to vertical last week. So the huddle test data from the weekend is not valid for the ranger; we will have to repeat it sometime.
So we used the manual, and extended the hanger rod on top of the Ranger to free the mass. It now has good response and coherence with the Guralps down to 0.1 Hz. See attached plot soon.
As Rob noted last Friday, the UPS which powers the Vacuum rack failed. When we were trying to move the plugs around to debug it, it made a sizzling sound and a pop. Bad smells came out of it.
Ben came over this week and measured the quiescent power consumption. The low power draw level was 11.9 A and during the reboot its 12.2 A. He measured this by ??? (Rob inserts method here).
So what we want is a 120 V * 12.2 A ~ 1.4 kVA UPS with ~30-50% margin. We look for this on the APC-UPS site:
On Monday, we will order the SUA2200 from APC. It should last for ~25 minutes during an outage. Its $1300. The next step down is $200 cheaper and gives 10 minutes less uptime.
Added the conlog directory to the SVN, minus the enormous data directory. We are now free to make changes to the conlog code.
There were no injuries...Now we need to get some new chairs.
Looks like all of the accelerometers and seismometers have been disconnected since early AM last Monday when Clara disconnected them for her sensor noise measurement.
The RAID array servicing the Frame builder was finally switched over to JetStor Sata 16 Bay raid array. Each bay contains a 1 TB drive. The raid is configured such that 13 TB is available, and the rest is used for fault protection.
The old Fibrenetix FX-606-U4, a 5 bay raid array which only had 1.5 TB space, has been moved over to linux1 and will be used to store /cvs/cds/.
This upgrade provides an increase in look up times from 3-4 days for all channels out to about 30 days. Final copying of old data occured on August 5th, 2009, and was switched over on that date.
Sadly, this was only true in theory and we didn't actually check to make sure anything happened.
We are not able to get lookback of more than ~3 days using our new huge disk. Doing a 'du -h' it seems that this is because we have not yet set the framebuilder to keep more than its old amount of frames. Whoever sees Joe or Alex next should ask them to fix us up.
The PSL Temperature Box (D980400-B-C, what kind of numbering scheme is that?) modified at LHO/LLO ~8 years ago to have better resolution on the in-loop temperature sensors.
I haven't been able to find a DCN / ECN on this, but there's an elog entry from Hugh Radkins here. I'm also attaching the PDF of the latest drawing (circa 2000) from the DCC.
The schematic doesn't show it, but I am guessing that the T_SENSE inputs are connected to the AD590 chips, and that 4 of these are attached somehow to the RefCav can. IF this is true, I don't understand why there are input resistors on the LT1125 of U1; the AD590 is supposed to be a current source ?
Peter King is supposed to be coming over to work on this today so whoever spots him should force/cajole/entice him to elog what he's done. Film him if necessary.
I also think R1-8 should be swapped into metal film resistors for stability. The datasheet says that it puts out 1 uA/K, so the opamps put out 10 mV/K.
J8 and JP1 should be shorted to disable both the tidal and VME control input. Both are unused and a potential source of drift.
There's no elog entry about what work has gone on today, but it looks like Peter took apart the reference cavity temperature control around 2PM.
I touched the reference cavity by putting my finger up underneath its sweater and it was nearly too hot to keep my finger in there. I looked at the heater power supply front panel and it seems that it was railed at 30 V and 3 A. The nominal value according to the sticker on the front is 11.5 V and 1 A.
So I turned down the current on the front panel and then switched it off. Otherwise, it would take it a couple of days to cool down once we get the temperature box back in. So for tonight there will definitely be no locking. The original settings are in the attached photo. We should turn this back on with its 1A setting in the morning before Peter starts so that the RC is at a stable temp by the evening. Its important NOT to turn it back on and let it just rail. Use the current limit to set it to 1 A. After the temperature box is back in the current limit can be turned back up to 2A or so. We never need the range for 3A, don't know why anyone set it so high.
I made the changes to the psl.db to handle the new Temperature box hardware. The calibrations (EGUF/EGUL) are just copied directly from the LHO .db file (I have rsync'd their entire target area to here).
allegra:c1psl>diff psl.db~ psl.db
< field(DESC,"TIDALOUT- drive to the reference cavity heater")
< field(SCAN,".5 second")
< field(INP,"#C0 S28 @")
< field(DESC,"TIDALINPUT- tidal actuator input")
< field(SCAN,".5 second")
< field(INP,"#C0 S3 @")
> field(DESC,"TIDALINPUT- tidal actuator input")
> field(SCAN,".5 second")
> field(INP,"#C0 S3 @")
> field(DESC,"TIDALOUT- drive to the reference cavity heater")
> field(SCAN,".5 second")
> field(INP,"#C0 S28 @")
I stepped the TIDALSET and looked at what happened. Loop was closed with the very low gain.
The RED guy tells us the step/impulse response of the RC can to a step in the heater voltage.
The GREY SLOWDC tells us how much the actual glass spacer of the reference cavity lags the outside can temperature.
Since MINCOMEAS is our error signal, I have upped his SCAN period from 0.5 to 0.1 seconds in the database and reduced its SMOO from 0.9 to 0.0. I've also copied over the Fricke SLOW code and started making a perl PID loop for the reference cavity.
Since ~Aug. 27, the reference cavity has been running with no thermal control. This is not really a problem at the 40m; a 1 deg change of the glass cavity
will produce a 5 x 10-7 strain in the arm cavity. That's around 20 microns of length change.
This open loop time gave us the opportunity to see how good our cavity's vacuum can insulation is.
The first plot below shows the RCTEMP sensors and the RMTEMP sensor. RMTEMP is screwed down to the table close to the can and RCTEMP is on the can, underneath the insulation. I have added a 15 deg offset to RMTEMP so that it would line up with RCTEMP and allow us to see, by eye, what's happening.
There's not enough data here to get a good TF estimate, but if we treat the room temperature as a single frequency (1 / 24 hours) sine wave source, then we can measure the delay and treat it as a phase shift. There's a ~3 hour delay between the RMTEMP and RCTEMP. If the foam acts like a single pole low pass filter, then the phase delay of (3/24)*360 = 45 deg implies a pole at a ~3 hour period. I am not so sure that this is a good foam model, however.
The colorful plot is a scatter plot of RCTEMP v. RMTEMP. The color denotes the time axis - it starts out blue and then becomes red after ten days.
I have added the records for the RC thermal PID servo into the psl/slowpid.db file which also holds the records for the SLOW servo that uses the NPRO-SLOW to minimize the NPRO-FAST. This new database will take effect upon the next PSL boot.
The perl script which runs the servo is scripts/PSL/FSS/RCthermalPID.pl. Right now it is using hard-coded PID parameters - I will modify it to use the on-screen values after we reboot c1psl.
The new screen C1PSL_FSS_RCPID.adl, the script, and the .db have been added to the SVN.
I have got some preliminary PID parameters which seem to be pretty good: The RCTEMP recovers in ~10 minutes from a 1 deg temperature step and the closed loop system is underdamped with a Q of ~1-2.
I'm leaving it running on op340m for now - if it goes crazy feel free to do a 'pkill RCthermalPID.pl'.
I have wiped out the 2008a install of 64-bit linux matlab and installed 2009a in its place. Enjoy.
The RC thermal PID is now controllable from its own MEDM screen which is reachable from the FSS screen. The slowpid.db and psl.db have been modified to add these records and all seems to be working fine.
Also, I've attached the c1psl startup output that we got on the terminal. This is just for posterity.
I'm also done tuning the PID for now. Using Kp = -1.0, Ki = -0.01, and Kd = 0, the can servo now has a time constant of ~10 minutes and good damping as can be seen in the StripTool snap below. These values are also now in the saverestore.req so hopefully its fully commissioned.
I bet that its much better now than the MINCO at holding against the 24 hour cycle and can nicely handle impulses (like when Steve scans the table). Lets revisit this in a week to see if it requires more tuning.
I added a new database record (C1:PSL-FSS_RCPID_SETPOINT) to allow for changing of the RC setpoint while the loop is on. This will enable us to step the can's temperature and see the result in the NRPO's SLOWDC.
I stepped the RC can temperature to see the response in the laser frequency. This gives a true measure of the thermal time constant of the RC. Its ~4 hours.
Since the RCPID screen now has a setpoint field, I can remotely type in 1 deg steps. The NPRO SLOW actuator locks the NPRO to the RC at long time scales and so we can use C1:PSL-FSS_SLOWDC to measure the RC length. By knowing what the step response time constant is, we can estimate the transfer function from can temperature to frequency noise and thereby make a better heater circuit.
Does the observed temperature shift make any sense? Well, John Miller and I measured the SLOW calibration to be 1054 +/- 30 MHz / V.
We know that the thermal expansion coefficient of fused silica, alpha = 5.5 x 10-7 (dL/L)/deg. So the frequency shift ought to be alpha * c / lambda = 155 MHz / deg.
Instead we see something like 110 MHz / deg. We have to take more data to see if the frequency shift will actually asymptote to the right value or not. If it doesn't, one possibility is that we are seeing the effect of temperature on the reflection phase of the mirror coatings through the dn/dT and the thermal expansion of the dielectric layers. I don't know what these parameters are for the Ta2O5 layers.
A more useful measure of the frequency noise can be gotten by just looking at the derivative in the first 30 minutes of the step, since that short time scale is much more relevant for us. Its 0.04 V / hour / (2 deg) => 860 (Hz/s)/deg.
In the frequency domain this comes out to be dnu/dT = 860 Hz/deg @ 0.16 Hz or dnu/dT = 137 *(1/f) Hz / deg.
Our goal for the reference cavity frequency noise is 0.01 * (1/f) Hz/rHz. So the temperature noise of the can needs to be < 0.1 mdeg / rHz.
This first plot shows the RC temperature channels' performance from 40 days ago, before we disabled the MINCO PID controller. Although RCTEMP is supposed to be the out of loop sensor, what we really care about is the cavity length and so I've plotted the SLOW. To get the SLOW on the same scale, I've multiplied the channel by 10 and then adjusted the offset to get it on the same scale.
The second plot shows a period after that where there is no temperature control of the can at all. Same gain scaling has been applied to SLOW as above, so that instead of the usual 1 GHz/V this plot shows it in 0.1 GHz/V.
The third plot shows it after the new PID was setup.
Summary: Even though the PID loop has more gain, the true limit to the daily fluctuations in the cavity temperature and the laser frequency are due to the in-loop sensors measuring the wrong thing. i.e. the out-of-loop temperature is too different from the in-loop sensor. This can possibly be cured with better foam and better placement of the temperature sensors. Its possible that we're now just limited by the temperature gradients on the can.
I've created a 40m Google account. Please post all the 40m related photos to this site. If you don't already have it, download Picasa to make this easier.
40m Installation Photos">
the password is in the usual password place.
>/etc/init.d/xntpd start -c /etc/inet/ntp.conf
[root@c1dcuepics sbin]# ./ntpq -p
remote refid st t when poll reach delay offset jitter
nodus.ligo.calt 0.0.0.0 16 u - 64 0 0.000 0.000 4000.00
*nodus.ligo.calt usno.pa-x.dec.c 2 u 29 64 377 1.688 -65.616 6.647
-lime7.adamantsy clock.trit.net 3 u 32 64 377 37.448 -72.104 4.641
-montpelier.ilan .USNO. 1 u 19 64 377 18.122 -74.984 8.305
+spamd-0.gac.edu nss.nts.umn.edu 3 u 28 64 377 72.086 -66.787 0.540
-mighty.poclabs. time.nist.gov 2 u 30 64 377 71.202 -61.127 4.067
+monitor.xenscal clock.sjc.he.ne 2 u 16 64 377 11.855 -67.105 6.368
This is a trend of the last 20 days. After our work with the NPRO, we have recovered only 5% in PMC trans power, although there's an apparent 15% increase in AMPMON.
The AMPMON increase is partly fake; the AMPMON PD has too much of an ND filter in front of it and it has a strong angle dependence. In the future, we should not use this filter in a permanent setup. This is not a humidity dependence.
The recovery of the refcav power mainly came from tweaking the two steering mirrors just before and just after the 21.5 MHz PC. I used those knobs because that is the part of the refcav path closest to the initial disturbance (NPRO).
BTW, the cost of a 1W Innolight NPRO is $35k and a 2W Innolight NPRO is $53k. Since Jenne is on fellowship this year, we can afford the 2W laser, but she has to be given priority in naming the laser.
What I like to try:
a) Change the NPRO temp to more cold side.
b) Change the MOPA head temp to a bit hot side.
c) Tweak the MOPA current (is it difficult?)
I think that the AMPMON ND problem was just that the responsivity changes with angle. So when I aligned it a little we got some few% improvement in the signal which is not a real power increase.
I don't think we can adjust any of the MOPA parameters because the controller is broken, but we can try the NPRO crystal temperature.
While trying to make the OAF work, I found that the XYCOM switches for MC1/3 have been set in the bad way for awhile. This means that the hardware filters were bypassed and that MC1 & MC3 were moving around too much at high frequency and possibly causing trouble with the locking. I have put them back into the default position.
On Friday, Jenne and I were playing around with turning the dewhitening off/on to see if it efffected the OAF stability. At the time, I didn't pay too much attention to what the state was. Looks like it was in the wrong state (hardware bypassed) when we found it. For the OAF work, we generally want it in that bypassed state, but its bad because it makes noise in the interferometer. The bits in question are bits 16-23 on the XYCOM screen.
I have updated the snapshot and set the screen in the appropriate settings. I used a swept sine measurement to verify the filter state. In the attached plot, green corresponds to XYCOM green and red corresponds to red.
Since we used to run with a gain slider setting of +15 dB on the MZ, I wanted to check that the new setting of +30dB was OK. It is.
To check it I turned it up and looked for some excess noise in the ISS or in the MC. There was none. I also set the input offset slider by unlocking the PMC and zeroing the mixer monitor readback. The new slider setting is -6.5V.
I don't know why we would need more gain on the MZ loop, but we can have some if we want it by turning up the gain before the servo (optical or RF). The attached plot shows the MC_F and ISS signals with the ISS loop on and off. There was no change in either of these spectra with the MZ gain high or low.
cosmic rays in cars
The attached shows the 200 day '10-minute' trend of the CPU meters and also the room temperature.
To my eye there is no correlation between the signals. Its clear that c1susvme2 (SRM LOAD) is going up and no evidence that its temperature.