I'm not sure what's going on today but we're seeing ~80% packet loss on the 40MARS wireless network. This is obviously causing big problems for all of our wirelessly connected machines. The wired network seems to be fine.
I've tried power cycling the wireless router but it didn't seem to help. Not sure what's going on, or how it got this way. Investigating...
Here's an example of the total horribleness of what's happening right now:
controls@rossa:~ 0$ ping 192.168.113.222
PING 192.168.113.222 (192.168.113.222) 56(84) bytes of data.
From 192.168.113.215 icmp_seq=2 Destination Host Unreachable
From 192.168.113.215 icmp_seq=3 Destination Host Unreachable
From 192.168.113.215 icmp_seq=4 Destination Host Unreachable
From 192.168.113.215 icmp_seq=5 Destination Host Unreachable
From 192.168.113.215 icmp_seq=6 Destination Host Unreachable
From 192.168.113.215 icmp_seq=7 Destination Host Unreachable
From 192.168.113.215 icmp_seq=9 Destination Host Unreachable
From 192.168.113.215 icmp_seq=10 Destination Host Unreachable
From 192.168.113.215 icmp_seq=11 Destination Host Unreachable
64 bytes from 192.168.113.222: icmp_seq=12 ttl=64 time=10341 ms
64 bytes from 192.168.113.222: icmp_seq=13 ttl=64 time=10335 ms
--- 192.168.113.222 ping statistics ---
35 packets transmitted, 2 received, +9 errors, 94% packet loss, time 34021ms
rtt min/avg/max/mdev = 10335.309/10338.322/10341.336/4.406 ms, pipe 11
Note that 10 SECOND round trip time and 94% packet loss. That's just beyond stupid. I have no idea what's going on.
I'm still seeing some problems with this - some laptops are losing and not recovering any connection. What's to be done next? New router?
We had the same problem yesterday. However the Vacuum Dedicated laptop worked with fewer disconnects. Christian is coming over this after noon to look at this issue.
This happened a few weeks ago and it recovered misteriously. Jamie did not understand it.
Temporary solution: I ssh'd to nodus from the 40m wifi network and was able to connect to the FE machines.This works but the bandwidth is limited this way as expected.
40m MARS network needs to be fixed.
Mike and Christian brought over a Mac laptop for surf Alex.
They power cycled the wireless router of 40Marsh and labtops are working. Seeing 75-80% signals on all 3 Dell lab top sisters at both end of the lab
i added my laptop's mac address to teh martian at port 13 today.
No personal laptop is allowed to the martian network. Only access to the General Computing Side is permitted.
Please disconnect it.
Jenne, Mike and I installed all of the post holders we could today including: REFL11, REFL33, REFL55, AS55, MCRef, POX11 and POP55. We did not install AS110, POY or REFL165 because there are interferences that will require moving stuff around. We also did not mount POP22 because it is a peely wally ThorLabs PD that will be replaced by a strong, straight and right thinking LIGO PD in the fullness of time. We did move it out of the way however which is no more than it deserves. Next step this afternoon Mike and I will install all of the telescopes and launching hardware. Then with the help of Steve we will begin routing the fibers. The splitter module will be here by next Monday, the laser by the following Friday and then we will light up the fibers.
Mike and I installed all of the telescopes and launching hardware for REFL11, REFL33, REFL55, AS55, MCRef, POX11 and POP55. On Monday afternoon Steve will work with us on the fiber routing. Steve is buying some protective covers for the fibers.
[ian, anchal, paco]
After our second attempt of locking PRFPMI tonight, we tried to resotre XARM and YARM locks to IR by clicking on IFO_CONFIGURE>Restore XARM (POX) and IFO_CONFIGURE>Restore YARM (POY) but the arms did not lock. The green lasers were locked to the arms at maximum power, so the relative alignments of each cavity was ok. We were also able to lock PRMI using IFO_CONFIGURE>Restore PRMI carrier.
This was very weird to us. We were pretty sure that the aligment is correct, so we decided to cehck the POX POY signal chain. There was essentially no signal coming at POX11 and there was a -100 offset on it. We could see some PDH signal on POY11 but not enough to catch the locks.
We tried running IFO_CONFIGURE>LSC OFFSETS to cancel out any dark current DC offsets. The changes made by the script are shown in attachment 1.
We went to check the tables and found no light visible on beam finder cards on POX11 or POY11. We found that ITMX was stuck on one of the coils. We unstuck it using the shaking method. The OPLEVs on ITMX after this could not be switched on as the OPLEV servo were railing to limits. But when we ran Restore XARM (POX) again, they started working fine. Something is done by this script that we are not aware of.
We're stopping here. We still can not lock any of the single arms.
Wed Jul 28 11:19:00 2021 Update:
Gautam found that the restoring of POX/POY failed to restore the whitening filter gains in POX11 / POY11. These are meant to be restored to 30 dB and 18 dB for POX11 and POY11 respectively but were set to 0 dB in detriment of any POX/POY triggering/locking. The reason these are lowered is to avoid saturating the speakers during lock acquisition. Yesterday, burt-restore didn't work because we restored the c1lscepics.snap but said gains are actually in c1lscaux.snap. After manually restoring the POX11 and POY11 whitening filter gains, gautam ran the LSCOffsets script. The XARM and YARM were able to quickly lock after we restored these settings.
The root of our issue may be that we didn't run the CARM & DARM watch script (which can be accessed from the ALS/Watch Scripts in medm). Gautam added a line on the Transition_IR_ALS.py script to run the watch script instead.
Alan and Alberto conducted a tour of 40 high-school students.
It may be the same tour that Rana found a spare PMC during the tour explanation as far as I remember...
the inside temperature is alarming at the red level today - should check if the HIHI value is set correctly
SRM, ITMX, ETMX, ITMY and ETMY lost damping at 4:55am this morning from 4.8 magnitude earthquake.
Their damping were restored.
C1:SUS-ITMX_URSEN_OUTPUT swich was found in off position. It was turned on.
MZehnder and MC were locked.
The WFS qpd spot needs recentering
20180508 4:49am Cabazon earth quake 4.5M at 79 miles away. ETMX is in load cell measurment condition.
There was an earthquake, all watchdogs were tripped, ITMX was stuck, and c1psl was dead so MCautolocker was stuck.
Watchdogs were reset (except ETMX which remains shutdown until we finish with the stack weight measurement), ITMX was unstuck using the usual jiggling technique, and the c1psl crate was keyed.
IFO restored after 4.5 Mag Banning, Ca earthquake.
It was little bit surprising to me but Rana's professorial rock'n roll excitation released its sticking on the unconfirmed thing by unconfirmed reason.
I aligned the Xarm manually and via ASS.
Now we are back in the normal state.
This recovery proceeder deserves a pattern
Note: IR shield glass position variations, Atm4
It looks like that ETMX have 2 sticky magnets.
I am really, really happy to hear that it was just a sticking situation. Really happy.
Suspensions recovered after 4.4 Mag EQ
It looks like there was a 4.4 magnitude earthquake near Fontana, CA around 1:30am today. This tripped all of the suspension watchdogs, which Q has just now re-enabled.
Earth quake shake down yesterday Atm1
Atm2, today's shake
Suspensions are recovered after 4.2 Mag earth quake. No obvoius sign of damage.
Sus dampings recovered. ETMY oplev needs to be recentered.
To achive the same beam height each components needs their specific post height.
We have 2.625" tall, 3/4" OD SS posts for Polaris K1 mirror mounts: 20 pieces
Ordered Newport LH-1 lens mounts with axis height 1.0
Are 4 of these spring loaded pins enough? I'm not sure how one pin can hold 2 lids at each point. It seems like we need 8 pins.
Steve has explained to me that the pins will go in between the 2 lids, with a big washer, so that one pin holds both lids at the same time. 4 is the right number.
I'm looking for some movement indicators of the vent-pump down events.
33MHz sidebands can be elliminated by careful choice of the modulation depths and the relative phase between the modulation signals.
If this condition is realized, the REFL33 signals will have even more immunity to the arm cavity signals because the carrier signal will lose
its counterpart to produce the signal at 33MHz.
Formulation of double phase modulation
m1: modulation depth of the f1 modulation
m2: modulation depth of the f2 (=5xf1) modulation
The electric field of the beam after the EOM
Therefore what we want to realize is the following "extinction" condition
We are in the small modulation regime. i.e. J0(m) = 1, J1(m) = m/2, J2(m) = m2/8, J3(m) = m3/48
Therefore we can simplify the above exitinction condition as
m2 < 0 means the start phase of the m2 modulation needs to be 180deg off from the phase of the m1 modulation.
- I built another beat setup on the PSL table at the South East side of the table.
- The main beam is not touched, no RF signal is touched, but recognize that I was present at the PSL table.
- The beat note is found. The 3rd order sideband was not seen so far.
- A PLL will be built tomorrow. The amplifier box Manasa made will be inspected tomorrow.
- One of the two beams from the picked-off beam from the main beam line was introduced to the beat setup.
(The other beam is used of for the beam pointing monitors)
There is another laser at that corner and the output from this beam is introduced into the beat setup.
The combined beam is introduced to PDA10CF (~150MHz BW).
- The matching of the beam there is poor. But without much effort I found the beat note.
The PSL laser had 31.33 deg Xtal temp. When the beat was found, the aux laser had the Xtal temp of 40.88.
- I could observe the sidebands easily, with a narrower BW of the RF analizer I could see the sidebands up to the 2nd order.
The 3rd order was not seen at all.
- The beat note had the amplitude of about -30dBm. One possibility is to amplify the signal. I wanted to use a spare channel
of the ALS/FOLL amplifier box. But it gave me rather attenuation than any amplification. I'll look at the box tomorrow.
- Also the matching of two beams are not great. The PD also has clipping I guess. These will also be improved tomorrow
- Then the beat note will be locked at a certain frequency using PLL so that we can reduce the measurement BW more.
[SUCCESS] The 3f sideband cancellation seemed worked nicely.
- Beat effeciency improved: ~30% contrast (no need for amplification)
- PLL locked
- 3f modulation sideband was seen
- The attenuation of the 55MHz modulation and the delay time between the modulation source was adjusted to
have maximum reduction of the 3f sidebands as much as allowed in the setup. This adjustment has been done
at the frequency generation box at 1X2 rack.
- The measurement and receipe for the sideband cancellation come later.
- This means that I jiggled the modulation setup at 1X2 rack. Now the modulation setup was reverted to the original,
but just be careful to any change of the sensing behavior.
- The RF analyzer was returned to the control room.
- The HEPA speed was reduced from 100% (during the action on the table) to 40%.
Right before the PSL beam goes into the vacuum chamber, it goes through an AR-wedged plate.
This AR plate produces two beams. One of them is for the IO beam angle/position monitor.
And the other was usually dumped. I decided to use this beam.
A G&H mirror reflects the beam towards the edge of the table.
A 45deg HR mirror brings this beam to the beat set up at the south side of the table.
This beam is S-polarlized as it directly comes from the EOM.
The beam from the PSL goes through a HWP and some matching lenses before the combining beam splitter (50% 45deg P).
The AUX laser beam is attenuated by a HWP and a PBS. The transmitted beam from the PBS is supposed
to have P-polarization. The beam alignment is usually done at the PSL beam side.
The combined beam is steered by a HR mirror and introduced to Thorlabs PDA10CF. As the PD has small diameter
of 0.5mm, the beam needed to be focused by a strong lens.
After careful adjustment of the beam mode matching, polarization, and alignment, the beatnote was ~1Vpp for 2.5Vdc.
In the end, I reduced the AUX laser power such that the beat amplitude went down to ~0.18Vpp (-11dBm at the PD,
-18dBm at the mixer, -27dBm at the spectrum analyzer) in order to minimize nonlinearity of the RF system and
in order that the spectrum analyzer didn't need input attenuation.
The PD signal is mixed with a local oscillator signal at 95MHz, and then used to lock the PLL loop.
The PLL loop allows us to observe the peaks with more integration time, and thus with a better signal-to-noise ratio.
The signal from the PD output goes through a DC block, then 6dB attenuator. This attenuator is added to damp reflection
and distortion between the PD and the mixer. When the PLL is locked, the dominant signal is the one at 95MHz. Without this attenuator,
this strong 95MHz signal cause harmonic distortions like 190MHz. As a result, it causes series of spurious peaks at 190MHz +/- n* 11MHz.
10dB coupler is used to peep the PD signal without much disturbing the main line. Considering we have 6dB attanuator,
we can use this coupler output for the PLL and can use the main line for the RF monitor, next time.
The mixer takes the PD signal and the LO signal from Marconi. Marconi is set to have +7dBm output at 95MHz.
FOr the image rejection, SLP1.9 was used. The minicirsuit filters have high-Z at the stop band, we need a 50Ohm temrinator
between the mixer and the LPF.
The error signal from the LPF is fed to SR560 (G=+500, 1Hz 1st-order LPF). I still don't understand why I had to use a LPF
for the locking. As the NPRO PZT is a frequency actuator, and the PLL is sensitive to the phase, we are supposed to use
a flat response for PLL locking. But it didn't work. Once we check the open loop TF of the system, it will become obvious (but I didn't).
The actuation signal is fed to the fast PZT input of the AUX NPRO laser.
- PD response [Attachment 1]
The AUX laser temperature was swept along with the note by Annalisa [http://nodus.ligo.caltech.edu:8080/40m/8369]
It is easier to observe the beat note by closing the PSL shutter as the MC locking yields more fluctuation of the PSL
laser freuqency at low frequency. Once I got the beat note and maximized it, I immediately noticed that the PD response
is not flat. For the next trial, we should use Newfocus 1611. For the measurement today, I decided to characterize the
response by sweeping the beat frequency and use the MAXHOLD function of the spectrum analyzer.
The measured and modelled response of the PD are shown in the attachment 1. It has non-intuitive shape.
Therefore the response is first modelled by two complex pole pair at 127.5MHz with Q of 1, and then the residual was
empirically fitted with 29th polynomial of f.
- Modulation profile of the nominal setting [Attachment 2]
Now the spectrum of the PD output was measured. This is a stiched data of the spectrum between 1~101MHz and 99~199MHz
that was almost simultaneously measured (i.e. Display 1 and Display 2). The IF bandwidth was 1kHz. The PD response correction
described above was applied.
It obviously had the peaks associated with our main modulations. In addition, there are more peaks seen.
The attachment 2 breaks down what is causing the peaks.
From the measured peak height, we are able to estimate the modulation depths for 11MHz, 55MHz, IMC modulations, as well as
the relative phase of the 11MHz and 55MHz modulation. (It is not yet done).
- 3f modulation reduction [Attachment 3]
Now, the redcution of the 3f modulation was tried. The measured modulation levels for the 11MHz and 55MHz were almost the same.
The calculation predicts that the modulation for the 55MHz needs to be 1/3 of the 11MHz one. Therefore the attenuation of 9dB and 10dB
of the modulation attenuation knob at the frequency generation box were tried.
To give the variable delay time in the 55MHz line, EG&G ORTEC delay line unit was used. This allows us to change the delay time from
0ns to 63.5ns with the resolution of 0.5ns. The frequency of 55MHz yields a phase sensitivity of ~20deg/ns (360deg/18ns).
Therefore we can adjust the phase with the precision of 10deg over 1275deg.
The 3rd-order peak at 61.8MHz was observed with measurement span of 1kHz with very narrow BW like 30Hz(? not so sure). The delay
time was swept while measuring the peak height each time. For both the atteuation, the peak height clearly showed the repeatitive dependence
with the period of 18ns, and the 10dB case gave the better result. The difference between the best (1.24e-7 Vpk) and the worst (2.63e-6 Vpk)
was more than a factor of 20. The 3rd-order peak in the above broadband spectrum measurement was 6.38e-6 Vpk. Considering the attenuation
of the 55MHz modulation by 10dB, we were at the exact unluck phase difference. The improvement expected from the 3f reduction (in the 33MHz signal)
will be about 50, assuming there is no other coupling mechanism from CARM to REFL33.
I decided to declare the best setting is "10dB attenuation & 28ns delay".
- Resulting modulation profile [Attachment 4]
As a confirmation, the modulation profie was measured as done before the adjustment.
It is clear that the 3rd-order modulation was buried in the floor noise. 10dB attenuation of the 55MHz modulation yields corresponding reduction of the sidebands.
This will impact the signal quality for the 55MHz series error signals, particularly 165MHz ones. We should consider to install the Teledyne Cougar amplifier
next to the EOM so that we can increase the over all modulation depth.
Both the Upgrade and the Old40m's signals look anomalous since the zero-crossing point does not change with the demodulation phases.
I suspect there's is a problem with the optickle model of the 40m.
To check the demodulation boards for REFL33 and REFL165, a long cable from ETMY (SUS-ETMY-SDCOIL-EXT monitor) is pulled to the rack on Y side.
(1) A filter just after the RF input and (2) transfer function from the RF input to the demodulated signal will be checked for the two 3f demod boards to confirm that they are appropriate for 33 and 165 MHz.
There is a LP filter just after the RF input of an demodulation board (its schematic can be found as D990511-00-C on DCC). I have checked if the 3f freq, 33MHz, can pass this filter. The filter TF from the RF input to RF monitor (the filter is between the input and monitor) on REFL33 demo-board was measured as shown in Fig. 1. At 33MHz, the magnitude is still flat and OK, but the phase is quite steep. I am going to consider if it is ok for the PDH method or not.
Fig. 1 Transfer function from the RF input to RF monitor on the REFL33 demodulation board. At 33MHz, a very steep phase is applied on the input signal.
The phase delay due to the RF input filter on the demodulation board will not bother the resulting PDH signals.
I quickly calculated the below question (see the blue sentence in the quote below). I applied an arbitrary phase delay (theta) due to the filter I measured, on the detected RF signal by the photo detector. Then the filtered RF signal is multiplied by cos(omega_m) then filter the higher (2 omega_m) freqency as the usual mixing operation for the PDH signal. As a result, the I signal is delayed by cos(theta) and the Q signal is delayed by sin(theta). Therefore the resulting signals and its orthogonalitity is kept ok. From the sideband point of view, theta is applied on both upper and lower and seems to make the unbalance, however, as it is like a fixed phase offset on both SBs at the modulation frequency, the resulting signals is just multiplied by cos or sin theta for I and Q, respectively. It won't make any strange effect (it is difficult to explain by sentence not using equations!).
Filters at the RF inputs of REFL33 and REFL165 demodulation boards were measured again. The filters will be totally fine for 33MHz and 165MHz.
Last time I forgot to calibrate the cable lengths, therefore the phase delay of the measurement included the cable lengths. This time the measurements were done for REFL33 and REFL165 demod board with calibration. As the cable lengths were calibrated, the shown plots (Fig.1 and Fig.2) do not include the phase delay dues to measurement cables. Please note that the x-axis is in linear. The phase delays of both boards seems to be not too steep (it will not affect anyway, as Kiwamu pointed out in his comment on the previous post). You can see that the two filters do not filter 33MHz and 165MHz component out.
Fig.1 A response of a filter which is placed just after the RF input of the demodulation board for REFL33. X-axis is shown in linear (~50MHz).
Fig.2 A response of a filter which is placed just after the RF input of the demodulation board for REFL165.
I also quickly checked the orthogonality of the demodulation board for REFL33 and REFL165 using function generators and oscilloscope. I checked the frequencies at 1,10,100,1K,10KHz of the demodulated signals. They are fine and ready for 3f signal extraction.
Wait. I am checking the whitening filters of the 33 and 165 demodulation boards.
Also, LSC-REFL33-I-IN1(IN2, OUT) and LSC-REFL165-Q-IN1(IN2,OUT) channels may not be working??
LSC-REFL33-I-IN1(IN2, OUT) and LSC-REFL165-Q-IN1(IN2,OUT) channels are back!
We disconnected and connected again the AA filters then the channels are fixed. Apparently the AA filters just before the digital world were somhow charged and not working... Thank you Kiwamu!
Whitening filters for the REFL33 & 165 demodulated channels were measured and confirmed that they are working. They can be turned on and off by un-white filter switches on the MEDM screen because they are properly linked. The measured filter responses are showen below. (Sorry, apparentyl the thumbnails are not shown here. Please click the attachments.)
Attachments: (top) Whitening filter for REFL33 demodulation board. (bottom) Whitening filter response for REFL 165 demodulation board.
Keiko, Jamie , Kiwamu
The I and Q orthogonalities of REFL33 and 165 demodulation board were measured by "orthogonality.py" Python package scipy were addied on Pianosa to run this code. Please note that "orthogonality.py" can be run only on Pianosa.
The results were:
ABS = 1.070274, PHASE = -81.802479 [deg]
if you wanna change epics values according to this result, just copy and execute the following commands
ezcawrite C1:LSC-REFL165_Q_GAIN 0.934340 && ezcawrite C1:LSC-REFL165_PHASE_D -81.802479
- - - - - - - - - - - - - - - - - -
ABS = 1.016008 , PHASE = -89.618724 [deg]
ezcawrite C1:LSC-REFL33_Q_GAIN 0.984244 && ezcawrite C1:LSC-REFL33_PHASE_D -89.618724
Fig.1 and 2 are the resulting plots for 33 and 165 MHz demod baoards, respectively.You should look at the 3Hz in x axis, as the demodulated signal frequency was set as 3 Hz.
Fig. 1 REFL33 I and Q orthogonality at 3 Hz.
Fig. 2 REFL165 I and Q orthogonality at 3 Hz.
In addition to REFL 33 ans 165, I checked the orthogonality for the other existing three channels.
ABS = 1.025035 PHASE = -93.124929 [deg]
ABS = 0.920984 PHASE = -88.824691 [deg]
ABS = 1.029985 , PHASE = -90.901123 [deg]
The demodulated signal was set as 50 Hz (for example LO 11MHz and RF 11MHz+50Hz from function generators.) These AS11, REFL11, REL55, REFL33m REFL165 are the current available channels in terms of the connection to the data system from the demodulation board. I am going to estimate the error next.
The demodulation phases and gains for the all existing channels, AS11, REFL11,REFL55, REFL165, and REFL33, were adjusted by the command "ezcawrite" commands.
REFL165 ezcawrite C1:LSC-REFL165_Q_GAIN 0.934340 && ezcawrite C1:LSC-REFL165_PHASE_D -81.802479
ezcawrite C1:LSC-REFL33_Q_GAIN 0.984244 && ezcawrite C1:LSC-REFL33_PHASE_D -89.618
So, I really should have done this as soon as Manasa measured the arm lengths... I've updated my MIST model with the real arm lengths, but still am using assumed identical losses of 75ppm on each mirror. (I've tried measuring the arm losses for real, but got numbers in the hundreds of ppms, so I need to reexamine things...)
Here's a simulation of the fields in a perfectly locked PRC when CARM is swept (Normalized to input power = 1).
More importantly, here's the latest simulation of MICH vs. PRCL demodulation angle separation in the 3F signals. It seems that we may be getting burned by using REFL33 for the PRC lock. REFL165, on the other hand looks much more robust. We should try this out.
(Some of my previous simulations incorrectly implemented MICH excitations; I only moved the ITMS, not the ETMS along with them, so some other stuff slipped in... )