Why do we have these timing blanks?
I have recently been running into hitting the 4MB/s data rate limit on testpoints - basically, I can't run DTT TF and spectrum measurements that I was able to while locking the interferometer, which I certainly was able to this time last year. AFAIK, the major modification made was the addition of 4 DQ channels for the in-air BHD experiment - assuming the data is transmitted as double precision numbers, i estimate the additional load due to this change was ~500KB/s. Probably there is some compression so it is a bit more efficient (as this naive calc would suggest we can only record 32 channels and I counted 41 full rate channels in the model), but still, can't think of anything else that has changed. Anyway, I removed the unused parts and recompiled/re-installed the models (c1lsc and c1omc). Holding off on a restart until I decide I have the energy to recover the CDS system from the inevitable crash. For documentation I'm also attaching screenshot of the schematic of the changes made.
Anyway, the main point of this elog is that at the compilation stage, I got a warning I've never seen before:
Building front-end Linux kernel module c1lsc...
make: Warning: File 'GNUmakefile' has modification time 13 s in the future
make: warning: Clock skew detected. Your build may be incomplete.
This prompted me to check the system time on c1lsc and FB - you can see there is a 1 minute offset (it is not a delay in me issuing the command to the two machines)! I am suspecting this NTP action is the reason. So maybe a model reboot is in order. Sigh
Front ends seem to be experiencing a timing issue. I can visibly see a difference in the GPS time ticks between models running on c1ioo and c1sus.
In addition, the fb is reporting a 0x2bad to all front ends. The 0x2000 means a mismatch in config files, but the 0xbad indicates an out of sync problem between the front ends and the frame builder.
As there are plans to work on the optic tables today and suspension damping is needed, we are holding off on working on the problem until this afternoon/evening, since suspensions are still damping. It does mean the RFM connections are not available.
At that point I'd like to do a reboot of the front ends and framebuilder and see if they come back up in sync or not.
There is definitely a timing distribution malfunction at the c1iscex IO chassis. There is no timing link between the "Master Timer Sequencer D050239" at the 1X6 and the c1iscex IO chassis. Link lights at both ends are dead. No timing, no running models.
It does not appear to be a problem with the Master Timer Sequencer. I moved the c1iscey link to the J15 port on the sequencer and it worked fine. This means its either a problem with the fiber or the timing card in the IO chassis. The IO timing card is powered and does have what appear to be normal status lights on (except for the fiber link lights). It's getting what I think is the nominal 4V power. The connection to the IO chassis backplane board look ok. So maybe it's just a dead fiber issue?
I do not know what could have been the problem with c1auxex, or if it's related to the fast timing issue.
Steve and Koji looked around, and called around, and there seem to be no spare fibers that are long enough to reach the end, so Steve has ordered
"Tripp Lite N520-30M 100' Multimode Duplex 50/125 Fiber Optic Patch Cable LC/LC"
and it should be here tomorrow.
The new fiber arrived today, and we tried it out. No luck. We think it is the timing card, so we'll need to get one, since we can't find a spare.
Order of operations:
* Lay new fiber on floor, plugged it in at both ends, saw no fiber link lights.
* From control room, killed all models running on c1iscex, shutdown computer. Still no link lights.
* Power cycled computer and IO chassis.
* Tried plugging new fiber into different port on Master Timing Sequencer, with other end still plugged in to c1iscex. Still no link lights.
* Looked around with flashlight at Xend IO chassis. The board that the fiber is connected to does not have a power light, although the board next to it has 2. We compared with the SUS IO chassis, and the board there with the fiber has one power light, plus the fiber link lights, as well as 2 on the board next to the fiber. So, perhaps there's a problem with power distribution on the timing board at the Xend?
* Unplugged and replugged the power connector to the timing board, inside the IO chassis, board next to the fiber's board got lights back, but the fiber's board did not. However, power must be going through the board with the fiber attached, to the next board, so there's power at least on some part of the timing board, just not the whole thing.
From this, we conclude that the blue fiber that was in place is probably fine (or is not found guilty), and that we need a replacement timing board. Koji didn't find one in the "CDS stuff" boxes underneath the Jenne Laser, and I feel like I recall Jamie saying that we would have to get a spare from somewhere else. We rolled up the new spare fiber, and put it in the box with other "CDS Stuff" under the Jenne Laser table.
I got a call from Koji and Yuta that something was wrong with the CDS system. I somehow had an immediate suspicion that it had something to do with the recent leap second.
It took a while for nodus to respond, and once he finally let me in I found a bunch of the following in his dmesg, repeated and filling the buffer:
Jul 3 22:41:34 nodus xntpd: [ID 774427 daemon.notice] time reset (step) 0.998366 s
Jul 3 22:46:20 nodus xntpd: [ID 774427 daemon.notice] time reset (step) -1.000847 s
Looking at date on all the front end systems, including fb, I could tell that they all looked a second fast, which is what you would expect if they had missed the leap second. Everything syncs against nodus, so given nodus's problems above, that might explain everything.
I stopped daqd and nds on fb, and unloaded the mx drivers, which seemed to be showing problems. I also stopped nodus's xntp:
sudo /etc/init.d/xntpd stop
His ntp config file is in /etc/inet/ntp.conf, which is definitely the WRONG PLACE, given that the ntp server is not, as far as I can tell, being controlled by inetd. (nodus is WAY out of date and desperately needs an overhaul. it's nearly impossible to figure out what the hell is going on in there). I found an old elog of Rana's that mentioned updating his config to point him to the caltech NTP server, which is now listed in the config, so I tried manually resyncing against that:
sudo ntpdate -s -b -u 188.8.131.52
Unfortunately that didn't seem to have any effect. This was making me wonder if the caltech server is off? Anyway, I tried resyncing against the global NTP pool:
sudo ntpdate -s -b -u pool.ntp.org
This seemed to work: the clock came back in sync with others that are known good. Once nodus time was good I reloaded the mx drivers on fb and restarted daqd and nds. They seemed come up fine. At this point front ends started coming back on their own. I went and restarted all the models on the machines that didn't (c1iscey and c1ioo). Currently everything is looking ok.
I'm worried that there is still a problem with one of the NTP servers that nodus is sync'ing against, and that the problem might come back. I'll check in again later tonight.
They are synchronised tiny glitches. They are not mechanical.
The corrected drawing base for tip tilt with coils are going to the shop. The will be back by the end of the week.
Now that I read Koji's last elog about phase maps, I am not sure if these are still required, but here they are (the tilt-removed phase maps of the Laser Optik mirrors), first 1, 2, 3:
So they all have an elevated center. I am not sure why the phase maps of mirrors 5 and 6 are slightly smaller in dimension. Anyhow, all mirrors have quite strong aberrations. Also, there is no big difference between the mirrors. Check for yourself, but be careful with the colors since the scales are all different.
Are these maps drawn from the data we extracted using Image SXM??
Indeed. So the only manipulation that I did was to remove the tilt (since this should usually be seen as an artifact of the measurement, or better, we can assume that tilt is compensated by alignment). I did not remove the curvature.
Below are phasemaps for the tip-tilts with both tilt and RoC removed. We have not used Koji's code; but tweaked the earlier code to remove curvature.
The RoC values matched approximately to that quoted by Gari Lynn ~700m.
The color scale for height are not the same for all mirrors.
SN1, SN2 and SN3
SN4, SN5 and SN6
Below are phasemaps for the tip-tilts with both tilt and RoC removed. We have not used Koji's code; but tweaked the earlier code to remove curvature.
The posted residual phase maps show circular contours since the data came with relatively low resolution in height. This is ok for what we want to do with these phase maps (i.e. simulating higher-order mode content in the PRC using Finesse). Better resolution is only required if you want to understand in detail optical scattering out of the cavity. Anyhow, the circular artifacts can be removed by first interpolating the phase maps to a higher lateral resolution, and then performing tilt and curvature subtraction. So we will soon have better looking phase maps posted. Then we should think about what type of Finesse simulation we could run. Certainly one simulation is to look at the beam shape in the PRC, but more interesting could be how sensitive the shape is to mirror alignments. The current simulation shows a mode that resembles the TEM01, but I have not yet tried to find optimal alignment of the mirrors (in simulation) to search for the TEM00 mode.
* mode matching
* epics LO HI values
* recover FSS
* make ISS working
- Put priority on the list
- Put names on the items
- Where is the CDS TO DO ==> Joe
- Remote disconnection of the greeen PDH
- What is the situation of the PD DC for the LSC PDs?
- SUS Satelite box Resister replacement ==> Jamie
- IMC mode matching ==> Jamie/Larisa
- Mechanical shutters everywhere
- SRM OPLEV Connection
- MC OAF
- Better LSC whitening boards
1) Get ETMY working - figure out why signals are not getting past the AI board (D000186) to the coils.
2) Get TDS and command line AWG stuff working
3) Get c1ass and new c1lsc (with Koji) fully integrated with the rest of the system.
4) Get CDS software instructions up to date and well organized.
5) Redo cabling and generally make it a permanent installation instead of hack job:
a) Measure cable lengths, check connectors, wire with good routes and ensure strain relief. Ensure proper labeling
b) Get correct length fiber for c1sus RFM and timing.
c) Fix up final BO adapter box and DAC boxes.
d) Make boxes for the AA filter adapters which are currently just hanging.
e) Get two "faceplates" for the cards in the back of the ETMY IO chassis so they can screwed down properly.
f) Remove and properly store old, unused cables, boards, and anything else.
6) Create new documentation detailing the current 40m setup, both DCC documents and interactive wiki.
7) Setup an Ubuntu work station using Keith's wiki instructions
1)Create simulated plant to interface with current end mass controls (say scx).
2) Create proper filters for pendulum and noise generation, test suspension.
3) Propagate to all other suspensions.
4) Working on simulated IFO plant to connect to LSC. Create filters for near locked (assume initial green control perhaps) state.
5) Test LSC controls on simulated IFO.
6) Fix c code so there's seamless switching between simulated and real controls.
- more precise F2P measurement and modify lockin simlink model (Kiwamu)
- run C1ASS to check it (Valera)
- take care of CDS (Joe)
- MC mode matching (Jenne/Koji)
- RF stuff (Suresh)
- mode matching for doubling crystal at PSL table (low priority)
- OPLEV (low priority)
- update the noise spectra of green locking
- make noise budgets
. mode matching for MC (Jenne/Koji)
. mode matching for doubling crystal on PSL table (Suresh/Koji)
. f2p adjustment (Kiwamu)
. fix daq and CDS issues (Joe)
. increase oplev gain (low priority)
. make ITMY camera nicer (Steve)
. c1ass simlink model (Valera/Joe)
. Bounce Roll notches (Suresh)
. align everything (at first green beam, then X arm cavity and finally IR beam)
. update the noise spectrum of the green locking
. estimate the noise from angle to length coupling
PMC locked manually and PRM sus damping restored.
Here's an update of the suspensions, after yesterdays in-vacuum work and OSEM tweaking:
pit yaw pos side butt
UL 0.828 1.041 1.142 -0.135 1.057
UR 1.061 -0.959 1.081 -0.063 -1.058
LR -0.939 -0.956 0.858 -0.036 0.849
LL -1.172 1.044 0.919 -0.108 -1.035
SD 0.196 -0.024 1.861 1.000 0.043
pit yaw pos side butt
UL 1.141 0.177 1.193 -0.058 0.922
UR 0.052 -1.823 0.766 -0.031 -0.974
LR -1.948 -0.082 0.807 -0.013 1.147
LL -0.859 1.918 1.234 -0.040 -0.957
SD -1.916 2.178 3.558 1.000 0.635
pit yaw pos side butt
UL 1.589 0.694 0.182 0.302 1.042
UR 0.157 -1.306 1.842 -0.176 -0.963
LR -1.843 -0.322 1.818 -0.213 0.957
LL -0.411 1.678 0.158 0.265 -1.038
SD 0.754 0.298 -3.142 1.000 0.053
[Anamaria / Kiwamu]
The incident beam pointing was improved by using PZT1 and PZT2.
With some triggers the lock of PRMI became smoother.
For the DRMI lock, the MICH and SRCL signals on AS55 are not quite decoupled, so we should find cleaner signals for them.
(what we did)
+ locked the Y arm
+ aligned incident beam by using PZT1(#5450) and PZT2. The spot positions on ITMY and ETMY are now well-centered.
+ tried activating C1ASS but failed. It needs some gain changes due to the new PZT1 response.
+ locked the X arm
+ aligned the TRX PD (Thorlab signal PD) and set the trigger.
+ C1ASS also doesn't work for the X arm
+ realigned the PRM and BS oplevs. the PRM oplev was clipped at a steering mirror on the optical bench
+ locked PRMI and aligned the PRM mirror such that the optical gain was maximized
+ optimized the demod phase of AS55 and REFL11
+ checked the UGF of the MICH and PRCL lock. The UGF of MICH is about 100Hz with gain of -20, and the UGF of PRCL was 85 Hz with gain of 0.1
+ adjusted the output matrix such that the MICH control doesn't couple into the PRCL control.
+ set the triggers for the MICH and PRCL control to make the lock acquisition smoother.
+ tried locking DRMI and it was sort of locked. However the SRCL signal showed up a lot in AS55_Q, where the MICH signal is extracted.
I think this week is going to be an "alignment week".
The goal is to get a good alignment on X arm for both the green and the IR beam in order to reduce a2l couplings.
Today's missions are :
- fixing the oplev channel names (see here)
- fixing the oplev gain issue (see here)
- engage the oplev servos
- f2p adjustment
- make a realtime lockin model for the f2p measurement and the dithering technique
- alignment of the MC incident beam (because we installed a new PMC this morning)
- manual alignment of the IR beam by steering PZT1 and PZT2 (this procedure will be replaced by an automatic way soon)
- bounce roll filters (see here)
0. have a coffee and then dress up the clean coat.
1. level the MC table
2. lock and align MC
3. run A2L script to see how much off-centering of the spots
4. steer the periscope mirror <--- We are here
5. move the pick off mirror which is used for monitoring of MCT CCD
6. check the leveling and move some weights if it's necessary
7. shut down
[Steve / Suresh / Kiwamu]
90 % of unused video cables have been removed.
Still a couple of video cables are floating around the video MUX. They will be removed in the next week's session.
The main goal of today is to extract the pick-off beams
Today's menu :
+ If necessary steer ETMs and ITMs to make the X and Y green beam flashing.
+ Open the IOO and OMC chamber and lock MC.
=> cover the place of the access connector by a large piece of aluminum foil. It will give a robust lock of MC.
+ Check the beam pointing down to Y arm by looking at the ETMY face camera.
=> If it's necessary align PZT1 and PZT2 from EPICS to make the IR beam flashing in the X arm.
+ Align BS and let the beam hit the center of ETMX to make the X arm flashing.
=> These alignment procedure will automatically gives us the MICH fringes on the AS CCD camera.
+ Rotate the SRM tower to get the SRMI fringes on the AS CCD camera.
=> This is because the required amount of the YAW correction on SRM is currently beyond the range of the DC bias.
+ Align PRM to get the PRMI fringes on AS CCD camera. Also make sure the beam comes back to the REFL CCD camera.
+ Lock the PRC to make POP/POX/POY bright enough.
=> Probably the REFL11 RFPD needs more power. To increase the power, just rotate the HWP, which is sitting before the RFPDs on the AS table.
=> If the signal on AS55 is too small, we can use REFL11_Q or REFL55 for the MICH lock.
+ OR inject and align the ABSL laser to make those pick-off beams bright enough.
=> This case we don't have to make the DRMI exactly on the resonance, what we need is just the DRMI flashing.
+ Align necessary optics for those pick-off beams.
=> In our definition (aLIGO definition) POP is the beam propagating from BS to PRM. Don't be confused by another one, which propagates from PRM to BS.
+ Steer two mirrors on the ETMY table for aligning IPANG. Also, steer some mirrors on the BS table for IPPOS.
=> IPANG has already reached the ETMY table, so ideally we don't have to steer a mirror on the BS table.
=> IPPOS/ANG are very visible with a sensor card.
+ Align some oplevs.
=> This work is relatively independent from the other tasks. Steve can take care of it.
+ Adjust the OSEM to their mid-range.
=> This work can be done anytime, but everytime we have to recover the alignment of the interferometer.
>If necessary steer ETMs and ITMs to make the X and Y green beam flashing.
Green is now flashing in both X and Y arms
>Open the IOO and OMC chamber and lock MC.
Open, and cover in place. MC is flashing and locking.
Kiwamu aligned things for me, and I rotated the SRM tower so that the reflected beam was pretty much totally overlapping the incident beam. The SRC was aligned to make sure things were good. Now the DC bias for SRM Yaw is ~1.4, so we're totally good.
To rotate SRM, Jamie had the idea of using 2 screws so I could push the tower on one side, and back off the screw an equal amount on the other side and push the tower to be touching both screws again, to ensure that I was rotating about the center of the tower and wasn't introducing any Pos action.
While I was at it, I also moved the OSEM connector tower back to its normal place on the table, so it's not in the way of oplev beams. It had been moved previously to accommodate ITMY near the door.
Our toilets were replaced by a green Gerber 1.6 gpf ( 6 lpf ) last week.
Electrician is coming to fix one of the fluorenent light fixture holder in the east arm tomorrow morning at 8am. He will be out by 9am.
The job did not get done. There was no scaffolding or ladder to reach troubled areas.
The lock of PRMI wasn't so robust although it could stay locked for more than 10 minutes.
There have been 2-3Hz spikes in everywhere. It needs to be investigated.
+ Diagnosis on the suspensions.
+ Check the beam centering on the RFPDs.
+ Check the f2a filters on PRM and BS.
+ Health check of the suspensions by locking some cavities and measuring the noise spectra for comparison.
+ Trying to use another signal port other than AS55.
The attached picture below is an example of the REFLDC and POXDC signals in time series.
This was when PRCL and MICH were locked by REFL33_I and AS55_Q respectively.
Note that when PRMI is unlocked, REFLDC goes to ~ 5000 counts and POXDC goes within ADC noise of ~ 1 counts.
According to the POP camera it looked like something was oscillating in the YAW direction which coincided with the spikes.
I tried finding any suspicious angular motions in the ITMs, BS and PRM olevs, but none of them showed the 2-3 Hz feature.
I found that all the Heliax cables landing on the bottom of 1Y2 were too loose.
Due to this loose connection the RF power at 55 MHz varies from -34 dBm to 3 dBm, depending on the angle of the Heliax's head.
The looseness basically comes from the fact the black plate is too thick for the Heliax cable to go all the way. It permits the Heliax's heads to rotate freely.
What we should do is to make countersinks on the black plate like this:
The countersink gives rise to another problem when we mount the N-type-to-SMA bulkhead adaptor. As we are making a circular hole in the plastic strip (instead of a hole with two flat sections) the adaptor is free to turn when we tighten it with a wrench. We currently hold the smooth circular part on the other side with a gripping pliers and while tightening. If that part disappears into the countersink (as seen in the pics) we will not be able to tighten the adaptor sufficiently and consequently we will also not be able to get the heliax connector to be tight.
A better solution would be to use the 1/4-inch plastic L-angle beam which Steve has used on the AS table. In addition to solving this loose connector problem, the beam is also more rigid than the plastic strip.
New Wiha 28504 torque wrench for SOS wire clamping. It's range 7.5 - 20 in-lb in 0.5 steps [ 0.9 - 2.2 Nm ] Audible and perceptible click when the pre-set torque has been attained at ±6% accuracy.
The new ETMX sus wire torqued to ~ 11.5 in-lb [1.3 Nm ]
Gautam and Steve,
The clamp's left side was jammed onto the left guide pin. It was installed slit facing left. Gautam had to use force to remove it. The clamp should move freely seating on the guide rods till torque aplied. Do not move on with the hanging of optic with a jammed clamp. Fix it.
Never use force as you are hanging - aligning optic. The clamp is in the shop for resurfacing and slit opening.
Few 1/4 -20 socket cap head screw with washers were tested for optimum torque.
QJR 117E Snap On torque wrench was used. I found that 40 lb in was enough.
These numbers will varie with washers, material it's going into and so on!
The standard among high-strength fasteners, these screws are stronger than Grade 8 steel screws. They have a minimum tensile strength of 170,000 psi. and a minimum Rockwell hardness of C37. Length is measured from under the head.
Inch screws have a Class 3A thread fit. They meet ASTM A574.
Black Oxide—Screws have been heat-treated for hardness, which results in a dark surface color.
Rana is next to calibrate his feelings and declare the right number.
Than Koji....and so on
Once we a number, than I buy more torque wrenches to fit it.
For 1/4-20 bolts made of 18-8 Stainless Steel, the recommended torque varies from 65-100 inch-pounds, depending upon the application, the lubrication, how loose the bolt is, if there's a washer, etc.
For our case, where we are going into a tapped, ferromagnetic stainless table, its less clear, but it will certainly by in the 60-80 range. This is close to the 5-6 foot-lbs that I recommended on Wednesday.
I've ordered 3 torque wrenches with 1/4" drive so that we can have one at each end and one in the toolbox near MC2. We'll indicate the recommended torque on there so that we can tighten everything appropriately.
Tonight I measured seismic noise coupling to beam spot on PR2. There is coherence of 0.9 from X to PIT and Y to YAW around the stack resonances. TF was fited using vectfit and put into static matrix of oaf in the elements T240X -> PRM PIT, T240Y -> PRM YAW. I think we should actuate on the error point of the PRM OL but I decided not to go for a model change tonight. Data from seismometers and POP QPD was obtained during the UTC time 04:06:00 - 04:50:00 when PRMI was locked on sideband
Interferometer was locking rather robustly and every lock lasted on the everage of 3 minutes. During these lock periods I incresed bandwidth of optical lever servos of BS and test masses from 4Hz up to 10Hz and then closed transmission QPD loops. It seems from the camera that lock losses correspond to strong motion of the beam on pop camera. Scripts that change OPLEV bandwidth are in /users/den "increase_ol_bandwidth.sh" "decrease_ol_bandwidth.sh". Script "engage_qpd_servos" turns off ETM oplevs and turns on ETM -> trans QPD servos. These scripts can be copied to locking directrly if are useful.
Please, note that transition from 3f to 1f should still be tuned. Only PRCL was stably controlled using 1f so far
Bob replaced the tipseal in an other drypump and I swapped it in. TP3 turbo is running again, it's foreline pressure is 40 mTorr. The RGA is still off
Does anyone know if this master file is the real thing that's in use now? Are we really using a file called tpchn_C1_new.par? If anyone sees Alex, please get to the bottom of this.
"Yes. This master file is used."
Now that both end transmission QPDs have the line filters, I aligned them.
I locked and aligned the IR using the ASS, then went to each end table and put the beam in the center of the QPD.
I have assembled the circuit and the control box for the quadrant magnetic levitation yesterday. The final setup is shown
in the figure below:
Due to my carelessness, I I connected the wrong ends of the power supply. I damaged 4 op-amp and one voltage
regulator during this assembly. This stupid mistake spent me several hours to fix, and I got a bitter lesson;-(
Afterwards, I replaced those op-amps and reconnected the power supply . Kiwamu helped me and we measured
the transfer function of this circuit. The transfer function agrees with the specification in the schematics which
has a integrator below 1 Hz and a differentiator from 5 to 20 Hz. The bode plot for the measured transfer function
is the following:
Today I tested the photodetector parts and found that there is a mysterious oscillation. Whenever I connect the
photodector input A of the circuit (as indicated in the figure below),
the output of the op-amp has a 500k Hz oscillation shown up in the oscilloscope.This happens even A is disconnected from
the photodetector and connected to an open end wire. I don't know how to eliminate it, and its amplitude is so large (peak to
peak is around 2.5 V) which completely dominates the photodetector output. Does anybody has some ideas? Thanks.
1. Why do all of the BNCs have no GND connection? Each should have the individual cables to the ground. Each signal line and the corresponding ground line should be twisted together.
2. This looks the (usual) oscillation of the V-I conversion stage but I can't tell anything as I don't have the circuit diagram of the whole circuit.
3. In a certain case, putting some capacitance at the feedback may help. Read P.11 of the data sheet of LT1125. Try to put some capacitors from 20pF to some larger one whether it changes the situation or not. I suppose the bandwidth of your sensor can be ~1kHz. So putting a capacitance less than 10nF still has no effect to the servo.
1. They are all connected to the box which has a single connection to the board ground. If I connect each of them to the ground, there would be many small loops
of ground. Of course, I should have replaced all the connectors such that the they are disconnected to the box as point out by Robert.
2. The oscillation disappears after I add 5 nF capacitor in parallel to the existing resistor. Thank you very much for pointing this out.