We had a unexpected power shutdown for 5 sec at ~ 9:15 AM.
Chiara had to be powered up and am in the process of getting everything else back up again.
Steve checked the vacuum and everything looks fine with the vacuum system.
There was an equipment malfunction in one of Pasadena's substation that caused the outage. After about an 8 second delay, back up circuits restored power. This affected about 1/2 of the campus.
From: Steve Vass [mailto:email@example.com]
Sent: Tuesday, October 07, 2014 2:18 PM
To: Anchondo, Michael
Can you tell me about yesterday's power outage?
Chiara doesn't seem to be responding and I guess something happened 7 hrs ago.
I tried to hook up chiara to a monitor to reboot or atleast look for error messages; but it is not even detecting the external monitor (Tried changing monitors and vga cables; still see nothing).
I tried to ssh into it and only received errors :
NFS lookup failed for server XXX.XXX.XXX.XXX : error 5 (RPC: Timed out)
ssh: chiara: host/servname not known
Steve had the vacuum checked and everything seems fine with the status of the vacuum system atleast.
No exciting progress today. I did PSL green alignment for the Yarm, although I now think that the Xarm green needs realigning too.
Also, I was foiled for a while by ETMX jumping around. I think it's because the adapter board on the Xend rack didn't have any strain relief. So, I zip tied the heavy cable in a few places so that it's no longer pulling on the connector. Hopefully we won't see ETMX misbehaving as often now, so we won't have to go squish cables as often.
After Q brought back the IR, I went to check the green situation.
1. The end lasers had to be turned ON.
2. The heaters for the doubler crystals had to be enabled. The heaters are at the set values.
3. The X arm PZTs for the steering mirrors had to be powered up (Set voltage 100V and current 6.7mA)
4. I aligned the green to the already IR-aligned arms.
Green PSL alignment has to be done after Q finishes his work on the MC WFS.
The autolocker is now working, but I didn't change anything to make it so. I was just putting in some echo statements, to see where it was getting hung up, and it started working... This isn't the first time I've had this experience.
It turns out IOO had a bad BURT restore. I restored from 5AM this morning, the WFS are ok now.
I brought back the PMC, MC and Arms.
As per other slow computers, which Chris figured out in elog 10189, I added all the rest of the slow computers to Chiara's /etc/hosts file, so that they would come up when Manasa went and keyed the crates.
Computers that were already there:
Computers that I added today:
Manasa keyed all of these crates *except* for the vac computer, since Steve said that the vacuum system is up and running fine.
The last time we had a power failure IFO recovery elog
We had an unexpected power shutdown for 5 sec at ~ 9:15 AM.
PSL Innolight laser and the 3 units of IFO air conditions turned on.
The vacuum system reaction to losing power: V1 closed and Maglev shut down. Maglev is running on 220VAC so it is not connected to VAC-UPS. V1 interlock was triggered by Maglev "failure" message.
Maglev was reset and started. After Chiara was turned on manually I could bring up the vac control screen through Nodus and opened V1
"Vacuum Normal" valve configuration was recovered instantly.
It is arriving Thursday
There are several non scientific reasons.
It seems clever, but I wonder why use DTT and command line perl, instead of using the FE lockins or just demod the offline data or all of the other sensing matrix scripts made for the LSC (at 40m) or ASC (at LLO) ?
Yesterday, Koji and I measured the transfer function of pitch and yaw excitations of each MC mirror, directly to each quadrant of each WFS QPD.
When I last touched the WFS settings, I only used MC2 excitations to set the individual quadrant demodulation phases, but Koji pointed out that this could be incomplete, since motion of the curved MC2 mirror is qualitatively different than motion of the flat 1&3.
We set up a DTT file with twenty TFs (the excitation to I & Q of each WFS quadrant, and the MC2 trans quadrants), and then used some perl find and replace magic to create an xml file for each excitation. These are the files called by the measurement script Koji wrote.
I then wrote a MATLAB script that uses the magical new dttData function Koji and Nic have created, to extract the TF data at the excitation frequency, and build up the sensing elements. I broke the measurements down by detector and excitation coordinate (pitch or yaw).
The amplitudes of the sensing elements in the following plots are normalized to the single largest response of any of the QPD's quadrants to an excitation in the given coordinate, the angles are unchanged. From this, we should be able to read off the proper digital demodulation angles for each segment, confirm the signs of their combinations for pitch and yaw, and construct the sensing matrix elements of the properly rotated signals.
The axes of each quadrant look consistent across mirrors, which is good, as it nails down the proper demod angle.
The xml files and matlab script used to generate these plots is attached. (It requires the dttData functions however, which are in the svn (and the dttData functions require a MATLAB newer than 2012b))
We have two cold cathode gauges at the pump spool and one signal cable to controller. CC1 in horizontal position and CC1 in vertical position.
CC1 h started not reading so I moved cable over to CC1 v
I haven't been able to lock the DRMI tonight, neither with 1F and no arms nor 3F and arms held off with ALS... I tried previous recipes, and new combinations informed by simulations I've run, to no avail.
I touched the alignment of the green beat PD on the PSL table, since the X beatnote was rather low, but wasn't able to improve it by much. I never took a spectrum, since it wasn't my main focus tonight, but the low frequency motion of both arms on ALS, as observed by RIN, was good as I've ever seen it.
In our WFS work earlier today, Koji and I reset the WFS offsets, and it actually seems to have helped a good deal, in terms of the "fuzz" of MC REFL on the wall striptool. I had previously presumed this to be due to excess angular motion, but perhaps it is more accurately described as an alignment offset that let the nominal angular motion couple into the RIN more.
We made sensing matrix measurements for the IMC WFS and the MC2 QPD.
The data is under further analysis but here is some record of the current state to show
IMC Trans RIN and the ASC error signals with/without IMC ASC loops
The measureents were done automatically running DTT. This can be done by
The analysis is in preparation so that it provides us a diagnostic report in a PDF file.
KroneCrane Fred inspected and certified the 3 40m cranes for 2014. The vertex crane crane was load tested at fully extended position.
Small oil drops were found during prevent inspection of the vertex crane. They were wiped off. It took 231 days to grow this size.
Pump down reached "vacuum normal" state. IFO _P1 pressure 1e-4 torr
PSL shutter is opened.
IFO_P1 pressure 1.6e-5 torr after 6 days at atm
PS: PSL sliding door 11 was left open overnight. The PSL particle count will reach room counts in 20 seconds at low speed of HEPA
The IFO is ready for 3F DRMI comissioning
Pump down reached "vacuum normal" state. IFO _P1 pressure 1e-4 torr in 8 hrs actual pumping time
PSL shutter is opened.
We stopped pumping just short of 3 hours at 320 Torr. Pumping speed was 2.7 Torr / min with partially closed RV1 and butterfly valve/
RP1&3 roughing pump hose is disconnected. Butterfly valve removed. The vac envelope is closed.
This is our second stop. I will be back this afternoon. IFO P1 3.5 Torr
I've installed a new 2pin lemo cable going from the CM servo out to in2 of the MC servo board, and removed the temporary BNC. I used some electrical tape to give the cable some thickness where the lemo head screws on to try to strain relieve the solder joints; hopefully this cable is more robust than the last.
I put an excitation into the CM board, and saw it come out of MC_F, so I think we're set.
Q checked the earth quake stops of SRM and we put the ITMY & BS doors on.
Photos have been taken of the ITMY chamber, and uploaded to picasa. Here's a slideshow:
I closed the PSL shutter as we didn't want to burn the mirror surface when we are not working.
Everything is aligned, AS and POY make it out of vacuum unclipped, OSEM readings look good.
I set up the SRM oplev, centered all oplevs.
Tomorrow, we just have to take pictures of the ITMY chamber before we put the heavy doors on.
We have completed the above points; the ITMY table is still level.
Despite what the wiki says, the SRM LR OSEM open voltage is ~1.97V instead of ~1.64, so we shot for half of that.
The in-air steering of the SRM oplev return beam needs adjustment. I'll estimate the beam path length when I'm taking pictures and closing up.
Left to do:
Related In-Air work:
The RGA time stamp was correct last at 20140527
Rga stopped scanning at 20140530
POY has >2 inches of clearance from the SRM cage.
Distance reconstruction indicates an SRC length of 5399mm, which was exactly our target.
Today so far:
Here's my quick brain dump of things to do before we can pump down (anyone see anything missing?):
I figured out that didn't change the initial guess for the fit routine in Gabriele's code. I also changed the fminsearch criteria to least squares fitting, instead of minimax. The consistency checks now look just as good as the previous time we did these kind of measurements, no disagreements bigger than 1.6mm.
Thus, the current estimate of the SRC length after yesterday's motions is 5402mm, where we desire 5399mm. So, we will try to move SRM 3mm closer to SR2, after confirming that we are not clipping the POY beam. After all that, we will level the table.
Larry and Nicolas
Larry's transfer function measurements suddenly started returning 0dB 0degrees when before there was some fake filter in the C1:ALS-OFFSETTER2 filter bank.
We looked in the filter bank and his filter was gone. So I created a new filter called LARRYP in FM2. We also disabled the output so he could drive the filter bank and test his TF code.
ITMY in vac table needs leveling.
[q, Jenne, Manasa]
I think I found out why rossa was mad.
An apt-get update on the 18th downloaded kernel 2.6.32-65-generic, so 2.6.32-58-generic, which what was previously chosen as a working kernel, had moved down in the grub ordering.
It turns out the grub configuration accepts strings, so I changed it to GRUB_DEFAULT="Ubuntu, with Linux 2.6.32-58-generic", ran sudo update-grub, and Rossa now seems to boot happily.
GRUB_DEFAULT="Ubuntu, with Linux 2.6.32-58-generic",
[Zach, Jenne, Steve]
This work happened on Tuesday. Bad Jenne for forgetting to elog it!
Zach brought the 40m's seismometers back (one Guralp and one T-240). We have set the seismometers on their slabs. Also, we ran the T240 cable from 1X5 over to the vertex slab. Also, also, Zach and Steve mounted the T-240 readout box in the 1X5 rack. We have not yet hooked it up to power, although there are fused power blocks available on that rack.
So, the T-240 box needs power, and then we need to connect the seismometers to their respective boxes. Also, we need to run medium-short BNC cables from the T-240 readout box to the PEM AA board over in 1X7.
I looked at the CAD layout and it seems like we will clearly be clipping POY if we move SRM by 7.5cm. Since POY is not visible at low power, we cannot be sure about the clipping.
I was bad and forgot to elog this yesterday (bad grad student!), but I setup a laser pointer to show us where the POY beam is.
To do this, I removed the tiny mirror that sends the beam to the POY RF PD (so we do not have POY to lock the Yarm right now. I think Q has successfully been using AS). The laser pointer goes through 2 temporary steering mirrors, then passes through the place that the tiny mirror usually sits, and then travels along the POY path into the vacuum system. The idea here is that we should be able to adjust the laser pointer and the temp steering mirrors, and not touch any of the actual POY mirrors, but still get the green beam to go all the way to ITMY. Yesterday I confirmed that the laser pointer was hitting the in-vac POY pickoff mirror, and today Q and Manasa are doing final adjustment to get the beam all the way to the ITM.
[Steve, EricQ, Jenne]
ITMY and BS heavy doors are off, light doors are on. Q is aligning the IFO.
Ottavia was having some severe interaction latency today. Xorg was taking up >90% of the CPU, just sitting around. The machine was logged in to a desktop session with lots of graphical effects turned on. I changed the system default session to "gnome-fallback" in /etc/lightdm/lightdm.conf, which was already set as the default for controls, but wouldn't get chosen for the autologin that happens on boot.
Hopefully this helps ottavia stay usable...
Interferometer alignment is restored
ASS has been run on each arm, recycling mirrors were aligned by overlapping on AS camera.
ETMY was not getting its ASC pitch and yaw signals. C1SCY had a red RFM bit (although, it still does now...)
I took a look at the c1rfm simulink diagram and found that C1RFM had an RFM block called C1:RFM-TST_ETMY_[PIT/YAW] and C1SCY had one called C1:TST-SCY_ETMY_[PIT/YAW].
It seems that C1TST was illegally being used in a real signal chain, and Jenne's recent work with c1tst broke it. I renamed the channels in C1RFM and C1SCY to C1:RFM-SCY_ETMY_[PIT/YAW], saved, compiled, installed, restarted. All was well.
There are still some in SCY that have this TST stuff going on, however. They have to do with ALS, it seems, but are SHMEM blocks, not RFM. Namely:
The vent is completed. ITMX was kicked up accidentally. Valve configuration: chamber open, RGA is pumped through VM2 Maglev
We should have a plan B before we move everything. I suggest we move a combination of SRM and SR2 to get the desired SRC length.
Moving SR2 will require extra effort to walk the beam unclipped through all the 6 output steering mirrors that follow; but there will be little room for error if we use irides to propagate the beam through the first 4 mirrors that are in the BS and ITMY chamber.
Jam nuts checked. Oplev servos turned off. Particle count checked. Vertex crane functionality checked.
As Q mentioned in elog 10527, (prompted by Koji's email this afternoon) we are prepping the IFO for vent. Here is a copy of the pre-vent checklist from the wiki, updated as we work:
1 & 2: Locked arms on IR, ran ASS. Unlocked IFO, aligned PRM for good POP flashes, aligned SRM for symmetric AS flashes. Aligned all oplevs. Used PZTs to align Xgreen to arm. Used knobs to align Ygreen to arm. With PS:L green shutter closed, Xgreen = 0.520, Ygreen = 0.680.
3: Moved MC servo output cable that goes to ADC from OUT2 (which we had been using for monitoring AO path signals) back to its usual OUT1 (which is MC_L). This is used in the spot position measurement script. Spots at: [2.32, -0.50, 1.97, -1.11, 0.26, -1.86] mm.
4: Done -Q
5: Removed a PD that was monitoring the light coming backwards through the Faraday that sits just after the laser, just in case (confirmed that beam dump behind PD was catching beam). Other port of PBS just had regular black hole dump. Adjusted half wave plate until we had ~90mW just before injection into the vacuum.
6: Completed. Locked MC manually at transmission of ~1150, but low power autolocker isn't working. This isn't a critical thing, and can be fixed at any point during the vent. -Q
7: Shutters closed. Ready for Steve to check nuts and begin venting! -Q
Rather than using a CAD drawing, I used Gabriele's code from ELOG 9590 to try and judge if we could shorten the SRC by the appropriate length, without clipping the SR3-SR2 beam.
Specifically, I used these lines:
% Move SRM 7.5 towards SR2, parallel to beam
dAS = BS2-AS; % Vector from SRM to SR2
dASmag = sqrt(dAS(1)^2+dAS(2)^2);
dMove = delta*dAS/dASmag; % delta times unit vector
CS = CS+dMove;
As a reminder, Gabriele's code used the following logic:
In my opinion, this is the best estimate of beam trajectory that we currently have.
Thus, from looking at the plot above, I claim we can correct the SRC length without clipping the beam by moving the SRM forward by the required 7.5cm.
Although the measured distance may be off on the order of a cm (since our PRC correction had a 0.5cm disagreement between interferometric and hand distance measurements), this will nevertheless markedly improve our 3F DRMI sensing, based on my previous ELOG.
Hence, given our discussions last week, Jenne and I will proceed to ready the interferometer for venting in the morning, by following the vent checklist.
Our sole objective for this vent is this move of the SRM.
Steve, please check the jam nuts, and begin the vent when you get in. Thanks!