The corrected drawing base for tip tilt with coils are going to the shop. The will be back by the end of the week.
They are synchronised tiny glitches. They are not mechanical.
I got a call from Koji and Yuta that something was wrong with the CDS system. I somehow had an immediate suspicion that it had something to do with the recent leap second.
It took a while for nodus to respond, and once he finally let me in I found a bunch of the following in his dmesg, repeated and filling the buffer:
Jul 3 22:41:34 nodus xntpd: [ID 774427 daemon.notice] time reset (step) 0.998366 s
Jul 3 22:46:20 nodus xntpd: [ID 774427 daemon.notice] time reset (step) -1.000847 s
Looking at date on all the front end systems, including fb, I could tell that they all looked a second fast, which is what you would expect if they had missed the leap second. Everything syncs against nodus, so given nodus's problems above, that might explain everything.
I stopped daqd and nds on fb, and unloaded the mx drivers, which seemed to be showing problems. I also stopped nodus's xntp:
sudo /etc/init.d/xntpd stop
His ntp config file is in /etc/inet/ntp.conf, which is definitely the WRONG PLACE, given that the ntp server is not, as far as I can tell, being controlled by inetd. (nodus is WAY out of date and desperately needs an overhaul. it's nearly impossible to figure out what the hell is going on in there). I found an old elog of Rana's that mentioned updating his config to point him to the caltech NTP server, which is now listed in the config, so I tried manually resyncing against that:
sudo ntpdate -s -b -u 22.214.171.124
Unfortunately that didn't seem to have any effect. This was making me wonder if the caltech server is off? Anyway, I tried resyncing against the global NTP pool:
sudo ntpdate -s -b -u pool.ntp.org
This seemed to work: the clock came back in sync with others that are known good. Once nodus time was good I reloaded the mx drivers on fb and restarted daqd and nds. They seemed come up fine. At this point front ends started coming back on their own. I went and restarted all the models on the machines that didn't (c1iscey and c1ioo). Currently everything is looking ok.
I'm worried that there is still a problem with one of the NTP servers that nodus is sync'ing against, and that the problem might come back. I'll check in again later tonight.
There is definitely a timing distribution malfunction at the c1iscex IO chassis. There is no timing link between the "Master Timer Sequencer D050239" at the 1X6 and the c1iscex IO chassis. Link lights at both ends are dead. No timing, no running models.
It does not appear to be a problem with the Master Timer Sequencer. I moved the c1iscey link to the J15 port on the sequencer and it worked fine. This means its either a problem with the fiber or the timing card in the IO chassis. The IO timing card is powered and does have what appear to be normal status lights on (except for the fiber link lights). It's getting what I think is the nominal 4V power. The connection to the IO chassis backplane board look ok. So maybe it's just a dead fiber issue?
I do not know what could have been the problem with c1auxex, or if it's related to the fast timing issue.
Steve and Koji looked around, and called around, and there seem to be no spare fibers that are long enough to reach the end, so Steve has ordered
"Tripp Lite N520-30M 100' Multimode Duplex 50/125 Fiber Optic Patch Cable LC/LC"
and it should be here tomorrow.
The new fiber arrived today, and we tried it out. No luck. We think it is the timing card, so we'll need to get one, since we can't find a spare.
Order of operations:
* Lay new fiber on floor, plugged it in at both ends, saw no fiber link lights.
* From control room, killed all models running on c1iscex, shutdown computer. Still no link lights.
* Power cycled computer and IO chassis.
* Tried plugging new fiber into different port on Master Timing Sequencer, with other end still plugged in to c1iscex. Still no link lights.
* Looked around with flashlight at Xend IO chassis. The board that the fiber is connected to does not have a power light, although the board next to it has 2. We compared with the SUS IO chassis, and the board there with the fiber has one power light, plus the fiber link lights, as well as 2 on the board next to the fiber. So, perhaps there's a problem with power distribution on the timing board at the Xend?
* Unplugged and replugged the power connector to the timing board, inside the IO chassis, board next to the fiber's board got lights back, but the fiber's board did not. However, power must be going through the board with the fiber attached, to the next board, so there's power at least on some part of the timing board, just not the whole thing.
From this, we conclude that the blue fiber that was in place is probably fine (or is not found guilty), and that we need a replacement timing board. Koji didn't find one in the "CDS stuff" boxes underneath the Jenne Laser, and I feel like I recall Jamie saying that we would have to get a spare from somewhere else. We rolled up the new spare fiber, and put it in the box with other "CDS Stuff" under the Jenne Laser table.
Front ends seem to be experiencing a timing issue. I can visibly see a difference in the GPS time ticks between models running on c1ioo and c1sus.
In addition, the fb is reporting a 0x2bad to all front ends. The 0x2000 means a mismatch in config files, but the 0xbad indicates an out of sync problem between the front ends and the frame builder.
As there are plans to work on the optic tables today and suspension damping is needed, we are holding off on working on the problem until this afternoon/evening, since suspensions are still damping. It does mean the RFM connections are not available.
At that point I'd like to do a reboot of the front ends and framebuilder and see if they come back up in sync or not.
I have recently been running into hitting the 4MB/s data rate limit on testpoints - basically, I can't run DTT TF and spectrum measurements that I was able to while locking the interferometer, which I certainly was able to this time last year. AFAIK, the major modification made was the addition of 4 DQ channels for the in-air BHD experiment - assuming the data is transmitted as double precision numbers, i estimate the additional load due to this change was ~500KB/s. Probably there is some compression so it is a bit more efficient (as this naive calc would suggest we can only record 32 channels and I counted 41 full rate channels in the model), but still, can't think of anything else that has changed. Anyway, I removed the unused parts and recompiled/re-installed the models (c1lsc and c1omc). Holding off on a restart until I decide I have the energy to recover the CDS system from the inevitable crash. For documentation I'm also attaching screenshot of the schematic of the changes made.
Anyway, the main point of this elog is that at the compilation stage, I got a warning I've never seen before:
Building front-end Linux kernel module c1lsc...
make: Warning: File 'GNUmakefile' has modification time 13 s in the future
make: warning: Clock skew detected. Your build may be incomplete.
This prompted me to check the system time on c1lsc and FB - you can see there is a 1 minute offset (it is not a delay in me issuing the command to the two machines)! I am suspecting this NTP action is the reason. So maybe a model reboot is in order. Sigh
Why do we have these timing blanks?
I 've just found this time capsule note from Nov. 26, 2000 by Kip Thorne: LIGO will discover gravitational waves by Dec.31, 2007
what is next?
Atm 3, Ron Drever could not celebrate with us because of health issues.
I powered down 1Y1 and 1Y2 instrument racks of MC and RF-systems this morning.
Both racks, SP table, clean tools-parts boxes and MC2 area are also covered with plastics.
Bertin Bros. Construction, Inc is working on the floor preparation. Their head man is Robby Bertin, cell 626. 831.2264
remove tiles, clean off tile glue, drill for ribbed steel bars, rough clean- surface for good bounding. paint floor with bounding agent
Tiles and adhesive removed. There were no dust storms created. Solvent paint- removal made the hole IFO -lab smell like the paint shop.
The south door is open with a fan blowing air inside. The north-west door is open through control room to outside to let air rinse the drying
paint-removal. Koji is taking lunch break while test riding our new PSL table legs.
Could not get past arm power of ~11 or so. I was suspicious of the transmon high-gain/low-gain PD handover, so I ran the matchTransMon scripts, but that did not help. I also removed the line in the cm_step script that increased the CM gain by 18dB at an arm power of 4. The gain of the CM servo will increase naturally as the power in the IFO builds up, so it may not be good to crank it right away. I tried several other CM gains, and watched the DARM loop, but still could not get past an arm power of ~10-11. I'm not sure what's wrong, but it may be that mysterious CM-servo/McWFS conspiracy, so we can try turning down the McWFS gain next time.
There are two new Matlab files on the svn in /mDV/extra/C1. 'mycsd.m' is to calculate the cross-spectral density between two channels, 'csd_40T_40T_SS1.m' calls this function with the available seismic channels and derives a self-noise spectrum for the vertical axis using all three seismometers. The method requires that there are no correlations between two instruments only which is a bad idealization for certain frequencies if you have seismometers of totally different types.
'mycsd.m' uses the high-gain, low-resolution Nuttall window (built-in Matlab function 'nuttallwin.m'). High-gain windows are used for broad-band spectra like seismic spectra, but it should be exchanged by another window if you plan to look at small-bandwidth features like peaks.
Since the three-channel analysis does not require knowledge of response functions, it could be used to evaluate the performance of the adaptive filter. For example, if three channels responding to the same signal are available, then the ratio of any two csds corresponds to one of the relative transfer functions. You can then compare this function with the result produced by the adaptive filter.
c1sus and c1iscey were reset. The PMC needed to be locked. MC locked instantly.
The FSS ion pump power supply was turned on.
c1sus needed reset.
c1sus and c1ioo were restarted. PMC locked.
The new cold cathode gauge CC1 is in place. We were at atmosphere for 28 days ......more later
cc1 = 2.3e-5 Torr at day 6 vacuum normal
Suresh and I tweaked the OSEM angles in ETMX yesterday. Last night the ETMs were left free swinging, and today I ran Rana's peakFit scripts on ETMX to check the input diagnolization:
It's well inverted, but the matrix elements are not great:
pit yaw pos side butt
UL 0.3466 0.4685 1.6092 0.3107 1.0428
UR 0.2630 -1.5315 1.7894 -0.0706 -1.1859
LR -1.7370 -1.5681 0.3908 -0.0964 0.9392
LL -1.6534 0.4319 0.2106 0.2849 -0.8320
SD 1.0834 -2.6676 -0.9920 1.0000 -0.1101
The magnets are all pretty well centered in the OSEMS, and we worked at rotating the OSEMS such that the bounce mode was minimized.
Rana and Koji are working on ETMY now. Maybe they'll come up with a better procedure.
I added 600 cc of Arrowhead Distilled Water to the chiller.
60 days plot shows that about every ~ 10 days I have to add some.
Please check the water level yourself.
Summary: Third unsuccessful attempt at getting ETMX suspended. I think we should dial the torque wrench back down to 1.0 N m from 1.5 N m for tightening the primary clamp at the top of the SOS tower. No damage to magnets, standoff successfully retrieved (it is sitting in the steel bowl)
Unfortunately I don't know of a more deterministic way of deciding on a "safe" torque with which to tighten the bolts except by trial and error. It is also possible that the clamping piece is damaged in some way and is responsible for these breakages, but short of getting the edges chamfered, I am not sure what will help in this regard.
Unrelated to this work: earlier today before the first wire failure, while I was optimistic about doing fine pitch balancing and gluing the standoff, I set up an optical lever arm ~3m in length, with the beam from the HeNe on the clean bench at 5.5 in above the table, and parallel to it (verified using Iris close to the HeNe and at the end of the lever arm). I also set up the PZT buzzer - it needs a function generator as well for our application, so I brought one into the cleanroom from the lab, isopropanol wiped it. The procedure says apply 5Vrms triangular wave at 1000Hz, but our SR function generators can't put out such a large signal, the most they could manage was ~2Vrms (we have to be careful about applying an offset as well so as to not send any negative voltages to the PZT voltage unit's "External input". All the pieces we need for the fine pitch balancing should be in the cleanroom now.
Gautam and Steve,
The clamp's left side was jammed onto the left guide pin. It was installed slit facing left. Gautam had to use force to remove it. The clamp should move freely seating on the guide rods till torque aplied. Do not move on with the hanging of optic with a jammed clamp. Fix it.
Never use force as you are hanging - aligning optic. The clamp is in the shop for resurfacing and slit opening.
Shade 12 black glass-green lines: 1/8" thick, 6" wide x 7 " high mounted on angle bracket below. Aperture 1" diameter. Let me know what do you think.
Things to do after making a new Simulink model.
1. ssh c1sus, go to /opt/rtcds/caltech/c1/core/advLigoRTS/ and run
bash ./makestuff.sh c1SYS
sed -i 's/RO \(.*SWR.*\)/\1/' /opt/rtcds/caltech/c1/target/$*/$*epics/autoBurt.req
If you don't need to re-install DAQs or screens, just run line 2,3,4, and 7 and go to step #6.
2. ssh c1sus, go to /opt/rtcds/caltech/c1/scripts/ and run
For now, you have to put "1" in "BURT Restore" in GDS screens with in 5-10 secs.
3. Now the DAQ channel numbers are changed. So, go to /cvs/cds/rtcds/caltech/c1/chans/daq/ and run
activateDAQ will activate the following DAQ channels for every optics with data rate 2048Hz;
4. Restart fb. See this wiki page.
Basically you what have to do is kill and restart daqd in fb and restart mx_streams in c1sus.
5. DONE! Burt restore if you want.
6. If you don't need to re-install DAQs or screens;
a. Go to /opt/rtcds/caltech/c1/target/c1SYS/c1SYSepics and run
sudo rmmod c1SYSfe
b. Go to /opt/rtcds/caltech/c1/target/c1SYS/bin/ and run
sudo insmod c1SYSfe.ko
bash ./makestuff.sh c1
sed -i 's/RO \(.*SWR.*\)/\1/' /opt/rtcds/caltech/c1/target/$*/$*epics/autoBurt.req
sudo rmmod c1SYSfe
sudo insmod c1SYSfe.ko
Thank you for the summary.
I think now you are getting into a phase where you should start doing some quantitative and careful checks.
1. Calculate how much light will be reflected from the cavity if the alignment is perfect.
2. Investigate where those kHz oscillations are coming from.
3. Estimate how much the 1.1 kHz corner frequency in LPF will reduce the phase margin around 10 kHz.
4. Calculate the estimated amplitude of the PDH signal.
5. Compute how big the gain of the PDH box should be.
This is a kind of summary of what I have worked on this week.
CVI broadband AR coating was measured at the PSL-enclosure table around 9-10am today. The 2W Innolight first PBS S polarization beam was used with an other 1/2 wave plate and PBS.
W2-PW1-1004-C-633-1064-0 This 0.045" thick window has 0.7- 0.8 % reflected beam on each sides at 5 degrees of incidence, P polarization.
The specification is R avg <0.5 % per surface at 0 degree
Rana wants The device would be useless with such a high R, but R 0.1% is OK so I will get V coating.
CVI V-AR coating at 1064 nm, 0 degree, catalog item is R< 0.25% on each sides,
R <0.1 % is custom at much higher prices.
This custom order should go with other orders that has similar need.
From CVI: 5-6-2014
I checked the trace info on the W2-PW1-1004-C-633-1064-0, BBAR coated window that you received. It is side 1, 0.42%R & side 2, 0.53%R @ 1064nm. And with the shift, I’m not too surprised you ended up with 0.7%. A V coat would start with <0.25% (and more typically coming in at ~0.1%) per surface. As far as stock options, I have a 1”dia x 4mmT, fused silica window that is recorded as side 1, R=0.09 and Side 2, R=0.08% @ 1064. Is this too think or will it work for you?
Pasadena reported the Sunday night event as a power transient caused by the trip of a power transmission line. This affected the entire city. Once the loss was detected, the system automatically switches to an alternate source. It was about one second for the system to recover.
2W Innolight, both Lightwaves at the ends, PSL HEPA, Ref Cavity and 3 AC units turned on.
The 40m vacuum did not trip. It is vacuum normal.
The Jetstore computer is indicating power failer.
There are 4 oscilloscopes left on the AP optical table top.... It's only 25 lbs... Do not leave anything on the optical table tops!
It takes 18 months to double the computational power of microprocessors but it took man thousands of years to invent the zipper. I never really understood that till these days.
Here is a sample of my latest results from Optickle simulations of the locking signal for the Power Recycling Cavity.
Thanks also to Rob's revolutionary bidimensional rotating matrix idea (I can see entire books of linear algebra going to be rewritten now because of that) I could find the way to determine the optimal demodulation phases for the demod signals.
There were also an other couple of missing details. But that came easily along.
The parfor function for the parallel computation in Matlab sped up some loops by a factor of 100.
In these particular plots there's still no CARM offset scan. That's what I'm going to post next on the elog, together with the signals for the other degrees of freedom.
Just to show that I'm confident I'm getting reasonable results, I'll post two PRC scans for different CARM. One set of plots is for the current 40m with -19.78 deg of SRM detuning phase, the other is for the Old Upgrade (9 Mhz vs the 11 currently planned) with no detuning phase.
I'm going to put together the results and get some conclusion about the 3f locking scheme for the current 40m and the upgrade.
The 40m IFO has reached atmospher in 5 hours. It is ready to open chamber condition. The RGA is pumped with the maglev.
P1 pirani gauge is contact dependent as you see it on the linear plot It will be replaced during this vent.
The venting speed was 2-4 Torr / min
Atm2 shows how the BS is sensing the venting air cylinder changes.
The 4th cylinder of instrument grade air bump is overlapping with our janitor working at the BS chamber.
Steve, Yuta and Jamie
Jam nuts were checked and oplev servos were turned off. Sus summery is below with strain gauge values. Are the strain gauge values have any meaning when the PZT contorrels are off??????????????????
You said that the use of FAXST was forbidden for phds and graduate students. I had to swear on the promise of not ever buying an other FAXST
I drained the water and removed side covers from the Neslab RTE 140 refrigerated water cooler unit this morning. The hoses to the laser were disconnected.
This abled you to see the little window of refregerant R404A was free of bubles, meaning: no recharge was needed.
The circulator bath was refilled with 7 liters of Arrowhead distilled water and the unit was turned on.
The water temp was kept 20.00+- .05C without any load. Finally the AC-repair man Paul showed up.
He measured the R404A level to be as specified: 23-24 PSI on the suction side and 310 PSI on the discharge side.
The unit was working fine. Paul found an intermittently functioning starting capacitor on the compressor that was removed.
The 240 micro Farad 120VAC cap will arrive tomorrow
Five mechcanical traps set inside of boxes. Red-white warning tape on top of each.
Last jump at rack Y2.
- The tanks are open
[done] - Remove the PZT cable currently underlying between BS and ITMY chambers
[done] - Put this PZT cable between BS and IMC chambers. Connect it on the PZT on the IMC table (SM1)
[done]- Put the two OSEM cables between BS and ITMY chambers. Connect this cable to SRM.
The connector for this cable at the BS side is coming from Bob's place on Wednesday. We left it disconnected for now.
- Energize all of four PZTs and check the functionality.
Steve, Rob and Alberto
Starting capacitor 216 miroFarad was installed on the compressor. Water lines were connected to the MOPA as corrected, so the flow meter readings are logical.
Now IN means flowing water in the direction of black arrow on the hose.
We struggled with the Neslab presetting: temp, bauds rate and other unknowns till Rob found the M6000 manual on Peter king's website.
Alberto realized that the chiller temp had to be reset to 20C on water chiller.
I put 1mg of Chloramin T into the water to restrict the growth of algae in the bath.
The NPRO heat sink was around ~20C without flow meter wheel rotation and the PA body ~25C by touch of a finger
I just opened up the needle valve a litle bit so the flow meter wheel would started rotating slowly.
That small glitch at the end of this 3 hrs plot shows this adjustment.