40m QIL Cryo_Lab CTN SUS_Lab TCS_Lab OMC_Lab CRIME_Lab FEA ENG_Labs OptContFac Mariner WBEEShop
  40m Log, Page 93 of 327  Not logged in ELOG logo
ID Date Author Type Category Subject
  11820   Sat Nov 28 11:46:40 2015 yutaroUpdateLSCpossible error source of loss map measurement

I found that TRY level degraded and the beam shape seen with CCD camera at AS port was splitted when the beam spot on ETMY was not close to the center. This was because dither started not working well. I suspect so because in such a case TRY level went up when I did iteration with TT1 and TT2 after freezing dither. Splitted beam shape indicates that incident light did not match well with the cavity mode.

TRY level for each point was this:

TRYDC
[[ 0.6573      0.8301      0.8983      0.8684      0.6773    ]
 [ 0.7555      0.8904      0.9394      0.8521      0.6779    ]
 [ 0.6844      0.8438      0.9318      0.8834      0.6593    ]
 [ 0.7429      0.8688      0.9254      0.8427      0.6474    ]
 [ 0.7034      0.8447      0.8834      0.8147      0.6966    ]]

 In the worst case, TRY level was 70 % of the maximum level. Assuming that this degrade was totally due to the mode mismatch, this corresponds to ~50 urad difference between the angle of incident light and resonant lighe in the arm (see elog 11819).

  11819   Fri Nov 27 22:20:24 2015 yutaroUpdateLSCround trip loss of Y arm

Here, I upload data I took last night, including the power of reflected power (locked/misaligned) and transmitted power for each point (attachement 1).

 

And I would like to write about possible reason why the loss I measured with POYDC and the loss I measured with ASDC are different by about 60 - 70 ppm (elog 11810 and 11818). The conclusion I have reached is: 

It could be due to the strange bahavior of ASDC level. 

This difference corresponds to the error of ~2% in the value of P_L/P_M. As reported in elog 11815, ASDC level changes when angle of the light reflected by ITMY changes, and 2% change of ASDC level corresponds to 10 urad change of the angle of the light according to my rough estimation with the figure shown in elog 11815 and attachment 2. This means that 2% error in P_L/P_M could occur if the angle of the light incident to YARM and that of resonant light in YARM differ by 10 urad. Since the waist width w_0 of the beam is ~3 mm, with the 10 urad difference, the ratio of the power of TEM10 mode is (10\,\mu \mathrm{rad}/ \theta_0)^2\sim0.01, where \theta_0=\lambda/\pi w_0. This value is reasonable; in elog 11743 Gautam reported that the ratio of the power of TEM10 was ~ 0.03, from the result of cavity scan. Therefore it is possible that the angle of the light incident to YARM and that of resonant light in YARM differ by 10 urad and this difference causes the error of ~2% in P_L/P_M, which could exlain the 60 - 70 ppm difference. 

Attachment 1: log.txt.zip
Attachment 2: 17.png
17.png
  11818   Fri Nov 27 03:38:23 2015 yutaroSummaryLSCround trip loss of Y arm

Tonight I measured "loss map" of ETMY. The method to calculate round trip loss is same as written in elog 11810, except that I used POYDC instead of ASDC this time.

How I changed beam spot on ETMY is: elog 11779.

I measured round trip loss for 5 x 5 points. The result is below.

(unit: ppm)

494.9 +/- 7.6       356.8 +/- 6.0       253.9 +/- 7.9       250.3 +/- 8.2       290.6 +/- 5.1      
215.7 +/- 4.8       225.6 +/- 5.7       235.1 +/- 7.0       284.4 +/- 5.4       294.7 +/- 4.5      
205.2 +/- 6.0       227.9 +/- 5.8       229.4 +/- 7.2       280.5 +/- 6.3       320.9 +/- 4.3      
227.9 +/- 5.7       230.5 +/- 5.5       262.1 +/- 5.9       315.3 +/- 4.7       346.8 +/- 4.2      
239.7 +/- 4.5       260.7 +/- 5.3       281.2 +/- 5.8       333.7 +/- 5.0       373.8 +/- 4.9 

The correspondence between the loss shown above and the beam spot on ETMY is shown in the following figure. In the figure, "downward" and "left" indicate direction of shift of the beam spot when you watch it via the camera (ex. 494.9 ppm corresponds to the lowest and rightest point). 

Edited below on 28th Nov. 

To shift the beam spot on ETMY, I added offset in YARM dither loop. The offset was [-30,-15,0,15,30]x[-10,-5,0,5,10] for pitch and yaw, respectively.  How I calibrated the beam spot is basically based on elog 11779, but I multiplied 5.3922 for vertical direction and 4.6205 for horizontal direction which I had obtained by caliblation of oplev (elog 11785).

Edited above on 28 th Nov.    

 

   

 

I will report the detail later. 

  11817   Thu Nov 26 19:39:27 2015 KojiUpdateLSCCurrent state of the frequency source, and possible improvement

Uploaded on T1000461 too.

Attachment 1: RF_Frequency_Source.pdf
RF_Frequency_Source.pdf RF_Frequency_Source.pdf
  11816   Wed Nov 25 23:34:52 2015 yutaroUpdateLSCround trip loss of Y arm

[yutaro, Koji]

Due to the strange behavior (elog 11815) of ASDC level, we checked if it is possible to use POYDC instead of ASDC to measure the power of reflected light of YARM. Attached below is the spectrum of them when the arm is locked. This spectrum shows that it is not bad to use POYDC, in terms of noise. The spectrum of them when ETMY is misaligned looked similar.

So I am going to use POYDC instead of ASDC to measure arm loss of YARM. 


Ed by KA:
The spectra of POYDC and ASDC were measured. We foudn that they have coherence at around 1Hz (good).
It told us that POYDC is about 1/50 smaller than ASDC. Therefore in the attached plot, POYDC x50 is shown.
That's the meaning of the vertical axis unit "ASDC".

Attachment 1: 14.png
14.png
  11815   Wed Nov 25 23:17:34 2015 yutaroUpdateLSCstrange behavior of ASDC

[yutaro, Koji]

I noticed that ASDC level changes depending on the angle of ITMY when trying to take some data for loss map of YARM. We finally found that ASDC level behaves strangely when the angle of ITMY in yaw direction is varied, as you can see in Attachment 1. Now, AS port recieved only the reflection of ITMY. 

NOTE: This behavior indicates that angular motion could couple to length signal in AS port.      

Koji suggested that this behavior might be caused by interference at SR2 or SR3 between main path light and the light reflected by the AR surface. By rough estimation, we confirmed that this scenario would be possible. So it would be better to measure AR reflection of the same mirror to ones used for SR2 and SR3 in term of incident angle.     

Ed by KA: This senario could be true if the AR reflection of teh G&H mirrors have several % due to large angle of incidence. But then we still need think about the overlap between the ghost beam and the main beam. It's not so trivial.

Attachment 1: 14.png
14.png
  11814   Wed Nov 25 22:59:42 2015 yutaroUpdateLSCAS table optics realignment

I slightly changed the orientation of a few mirrors on AS table that are used to make the AS light get into PDs, in order to confirm that the strange behavior of ASDC (I will report later) is not caused by clipping related to these mirrors or miscentering on PDs.

Then output level of ASDC, AS55, and AS165 could have changed.   

So take care of this possible change when you do something related to them. But the relative change of them would be at most several %, I think.

 

  11813   Wed Nov 25 22:37:12 2015 yutaroUpdateLSCremoval of Gautam's cable in 1Y2 and restoration of POYDC

[yutaro, Koji]

We disconnected the cable that was connected to CH5 of the whitening filter in 1Y2, then connected POYDC cable to there (CH5). This channel is where POYDC used to connect.

Then we turned on the whitening filter for POYDC (C1:LSC-POYDC FM1) and changed the gain of analog whitening filter for POYDC from 0 dB to 39 dB (C1:LSC-POYDC_WhiteGain).

  11812   Wed Nov 25 20:07:35 2015 ranaUpdateIOOLO level check for the IMC demod board

Perhaps we can replace T1 with a mini-circuits hybrid 0-90 deg splitter and then remove the trim caps. (JSPQ-80, JYPQ-30, SCPQ-50)

  11811   Wed Nov 25 16:46:10 2015 KojiUpdateIOOLO level check for the IMC demod board

Awwww. I found that the demod board has the power splitter (PSCJ-2-1) with one output unterminated.
This power splitter should be removed.

https://dcc.ligo.org/D1500443

Attachment 1: D990511-40m_151123.pdf
D990511-40m_151123.pdf
  11810   Wed Nov 25 16:40:32 2015 yutaroUpdateLSCround trip loss of Y arm

I measured round trip loss of Y arm. The alignment of relevant mirrors was set ideal with dithering (no offset).

Summary:

 round trip loss of Y arm: 166.2 +/- 9.3 ppm

(In the error, only statistic error is included.)

How I measured it: 

I compared the power of light reflected by Y arm (measured at AS) when the arm was locked (P_L) and when ETMY was misaligned (P_M). P_L and P_M can be described as 

P_M=P_0(1-T_\mathrm{ITM})

P_L=P_0\left[1-(1-\alpha)\frac{4T_\mathrm{ITM}}{T_\mathrm{tot}^2}T_\mathrm{loss}\right].

The reason why P_L takes this form is: (1-alpha)*4T_ITM/(T_tot)^2 is intracavity power and then product of intracavity power and loss describes the power of light that is not reflected back. Here, alpha is power ratio of light that does not resonate in the arm (power of mismatched mode and modulated sideband), and T_tot is T_ITM+T_loss. Transmissivity of ETM is included in T_loss. I assumed alpha = 7%(mode mismatch) + 2 % (modulation) (elog 11745)

After some calculation we get

1-P_L/P_M\simeq \frac{4(1-\alpha) T_\mathrm{loss}}{T_\mathrm{ITM}}-T_\mathrm{ITM}.

Here, higher order terms of T_ITM and (T_loss/T_ITM) are ignored. Then we get

(1-\alpha) T_\mathrm{loss}=\frac{T_\mathrm{ITM}}{4}(1-P_L/P_M+T_\mathrm{ITM}).

Using this formula, I calculated T_loss. P_L and P_M were measured 100 times (each measurement consisted of 1.5 sec ave.) each and I took average of them. T_ETM =13.7 ppm is used.

Discussion: 

-- This value is not so different from the value ericq reported in July (elog 10248).

-- This method of measuring arm loss is NOT sensitive to T_ITM.  In contrast, the method in which loss is obtained from finesse (for example, elog 11740) is sensitive to T_ITM.

In the method I'm now reporting, 

\Delta T_\mathrm{loss}/T_\mathrm{loss}\simeq\Delta T_\mathrm{ITM}/T_\mathrm{ITM},

but in the method with finesse,

\Delta T_\mathrm{loss}\simeq\Delta T_\mathrm{ITM}.

In the latter case, if relative error of T_ITM is 10%, error of T_loss would be 1000 ppm.

So it would be better to use power of reflected light when you want to measure arm loss.  

  11809   Wed Nov 25 14:46:53 2015 gautamUpdateCDSc1lsc models restarted

I noticed that all the models running on C1LSC had crashed when I came in earlier today. I restarted all of them by ssh-ing into C1LSC and running rtcds restart all. The models seem to be running fine now.

  11808   Wed Nov 25 10:07:10 2015 KojiUpdateIOOLO level check for the IMC demod board

I didn't finish making the DCC entry for this module yet.

But the attenuators are
- AT1: 10dB. There is a sign that it was 3dB before ---  a 3dB chip was also attached on the boardnext to 10dB.
- AT2/3: Removed. They were replaced with 0Ohm resisters.

Currently the input is -8dBm. The input and output of the first ERA-5 are -17dBm and +7dBm, respectively.
Then the input and output of the second ERA-5 are -2dBm and 17dB, respectively.

In order to remove the second amplification stages, the first stage has to produce 26dBm. This is too much for either ERA-5 or any chips that fits on the foot print. If we use low gain but high output amp like GVA-81 (G=10dB, DF782 package), it is doable

Input 0dBm - [ATTN 3] - -3dBm - [ERA-5 G=20dB] - +16~+17dBm - [Circuits -9dB] - +7dBm - [Attn 0dB] - +7dBm - [GVA-81 G=10dB] - +17dBm

I think we should check the conditions of all the LSC demods.

  11807   Wed Nov 25 04:24:21 2015 ranaUpdateIOOLO level check for the IMC demod board

Hmmm. Very non-standard demod. From the photo, looks like someone did some surgery with the attenuators (AT1, AT2, AT3) in the LO path. (might be me from a long time ago).

-8 dBm input to a circuit is a not a low noise situation. It would be best to remove the amplifiers in the I&Q paths and just have a single amplifier in the main path. Ideally we want the LO to never go below -3 dBm and certainly not below 0 dBm while outside of the board.

I doubt that all of the LSC demods were modified in this way - this one ought to get some sharpie or stickers to show its difference.

  11806   Tue Nov 24 14:58:40 2015 yutaroUpdateLSCITMX misaligned

I misaligned ITMX. The oplev servo for ITMX is now turned off. You can restore ITMX alignment by running "restore".

  11805   Tue Nov 24 11:18:47 2015 yutaroUpdateOptical LeversBeam centering for the oplev of ITMY

I made the beam spot on QPD for the oplev of ITMY centered by changing the orientation of the mirror just before the QPD.

Before doing this, I ran dithering for Y arm and froze the output of ASS for Y arm.

  11804   Tue Nov 24 01:14:23 2015 KojiUpdateLSCALSY recovered

Sorry, I completely forgot to turn the Marconi on...

  11803   Mon Nov 23 23:42:56 2015 ericqUpdateLSCALSY recovered

[ericq, gautam]

Gautam couldn't observe a Y green beatnote earlier, so we checked things out, fixed things up, and performance is back to nominal based on past references. 

Things done:

  • Marconi carrier output switched back on after Koji's excellent RF maintence
  • BBPD power supplies switched on
  • Removed a steering mirror from the green beatY path to do near/far field alignment. 
  • Aligned PSL / Y green beams 
  • Replaced mirror, centered beam on BBPD, moved GTRY camera to get the new spot.
  • POY locked, dither aligned, beatnote found, checked ALS out-of-loop noise, found to be in good shape. 
  11802   Mon Nov 23 22:12:10 2015 KojiUpdateIOOLO level check for the IMC demod board

In order to check the proper LO level, the IMC demod board was checked. As a short summary, -8dBm is the proper input for the IMC demod board. This was realized when the variable attenuator of the RF AM Stabilizer was set up be -7dB.


Initially, I tried to do the measurement using the extender board. But every board had the issue of +15V not working. After several extender boards were tried, I noticed that the current draw of the demod board burned the 15V line of the extender board.

Then I moved to the work bench. The signals were checked with the 10:1 probe. It's not properly the 50Ohm system, exactly to say.

I found that the LO signals at the mixers have huge distortion as it reaches the nominal 17dBm, and I wondered if ERA-5s were gone. Just in case I replaced the ERA-5s but didn't see any significant change. Then I thought it is due to the mixer itself. The mixer was removed and replaced with a 50Ohm SMD resister. Then the output of the last ERA-5 became sinusoidal, and the level was adjusted to be ~17dBm (4.52 Vpp) when the input power was measured to be -7.7dBm with the RF power meter. Once the mixer was reinstalled, it was confirmed that the waveform becase rectangular like, with the similar amplitude (4.42Vpp).

Now the module was returned to the rack. The RF level at the LO input was adjusted to be -8dBm by setting the attenuator level to be 7dBm.

Once the IMC is locked with this setting, the open loop transfer function was measured. The optical gain seemed almost unchanged compared with the recent nominal. The UGF and PM were measured to be 144kHz and 30deg.

Attachment 1: IMG_2137.JPG
IMG_2137.JPG
Attachment 2: IMG_2138.JPG
IMG_2138.JPG
  11801   Mon Nov 23 21:48:49 2015 KojiUpdateLSCFrequency source fixed, IMC LO level adjusted

The trouble we had: the 29.5 MHz source had an output of 6 dBm instead of 13 dBm.

The cause of the issue: A short cable inside had its shield cut and had no connection of the return.


- The frequency source box was dismantled.
- The power supply voltages of +28 and +18 were provided from bench supplies.
- The 29.5 MHz output of 5~6 dBm was confirmed on the work bench.
- The 11 MHz OCXO out (unused) had an output of 13 dBm.

- Once the lid was opened, it was immediately found that the output cable for the 29.5 MHz source had a sharp cut of the shield (Attachment1).
- OK. This cable was replaced. The output of 13 dBm was recovered.

- But wait. Why is the decoupling capacitor on the 29.5 MHz OCXO bulging? The polarity of the electrolytic capacitor was wrong!
- OK. This capacitor was replaced. It was 100 uF 35 V but now it is 100 uF 50 V.

- I further found some cables which had flaky shields. Some of them were twisted. When the panel cable s connected, the feedthroughs were rotated. This twists internally connected cables. Solder balls were added to the connector to reinforce the cable end.

- When the box was dismantled, it was already noticed that some of the plastic screws to mount the internal copper heat sinks for ZHL-2's were broken.
They seemed to be degraded because of the silicone grease. I didn't try to replace all as it was expected to take too much time, so only the broken screws
were replaced with steel screws with shoulder washers
at the both side of the box.

- After confirming the circuit diagram, the box was returned to the rack. The 29.5 MHz output of 13 dBm there was confirmed.

Attachment 1: IMG_2136.JPG
IMG_2136.JPG
Attachment 2: IMG_2129.JPG
IMG_2129.JPG
Attachment 3: IMG_2133.JPG
IMG_2133.JPG
  11800   Mon Nov 23 20:32:43 2015 KojiUpdateLSCFrequency source fixed, IMC LO level adjusted

The frequency source was fixed. The IMC LO level was adjusted.

IMC is locked => OLTF measured UGF 144kHz PM 30deg.

  11799   Mon Nov 23 14:45:39 2015 ericqUpdateComputer Scripts / ProgramsNew software

COMSOL 5.1 has been installed at: /cvs/cds/caltech/apps/linux64/comsol51/bin/comsol

MATLAB 2015b has been installed at: /cvs/cds/caltech/apps/linux64/matlab15b/bin/matlab 

This has not replaced the default matlab on the workstations, which remains at 2013a. If some testing reveals that the upgrade is ok, we can rename the folders to switch. 

  11798   Sun Nov 22 12:12:17 2015 KojiUpdateIOOIMC OLTF

Well. I thought a bit more and now I think it is likely that this is just the servo bump as you can see in the closed-loop TF.

Quote:

6. Broad feature from 40kHz to 200kHz. The IMC loop is adding the noise. This is the frequency range of the PC drive. Is something in the PC drive noisy???

  11797   Sun Nov 22 12:07:09 2015 KojiUpdateIOOIMC fix

TO DO

Done (Nov 23, 2015) - Check if the attenuator is still there in the input chain

Done (Nov 23, 2015) - Check if the actual LO levels at the 17dBm mixers are reasonable.

- Check if the actual LO levels for the LSC demods are OK too

  11796   Sun Nov 22 07:09:01 2015 ranaUpdateIOOIMC fix

On the demod board there is a 10 dB attenuator (AT1), which lowers the level to -10 dBm before the ERA-5. Then it should be 10 dBm before going to the rest of the parts. But I guess the ERA-5 chips which come later on in the circuit could be decaying like the ones in the PMC LO board.

Quote:

Later at home, I thought this nominal LO level of 0dBm could have been wrong.

  11795   Sat Nov 21 00:46:33 2015 KojiUpdateIOOIMC OLTF

Here is the comparison before and after the fix.

Before the work, the UGF was ~40kHz. The phase margin was ~5deg. This caused huge bump of the frequency noise.

After the LO power increase, I had to reduce the MC loop gain (VCO Gain) from 18dB to 6dB. This resulted 4dB (x2.5) increase of the OLTF. This means that my fix increased the optical gain by 16dB (x6.3). The resulting UGF and phase mergin were measured to be 117kHz and 31deg, respectively.

 

Now I was curious to see if the PMC err shows reasonable improvement when the IMC is locked. Attachment 2 shows the latest comparison of the PMC err with and without the IMC locked. The PMC error has been taken up to 500kHz. The errors were divided by 17.5kHz LPF and 150kHz LPF to compensate the sensing response. The PMC cavity pole was ignored in this calculation. T990025 saids the PMC finesse is 4400 and the cavity pole is 174kHz. If this is true, this also needs to be applied.

Observations:

1. Now we can see improvement of the PMC error in the region between 10kHz to 70kHz.

2. The sharp peak at 8kHz is due to the marginally stable PMC servo. We should implement another notch there. T990025 suggests that the body resonance of the PMC spacer is somewhere around there. We might be able to damp it by placing a lossy material on it.

3. Similarly, the features at 12kHz and 28kHz is coming from the PMC. They are seen in the OLTF of the PMC loop.

4. The large peak at 36kHz does not change with the IMC state. This does mean that it is coming from the laser itself, or anything high-Q of the PMC. This signal is seen in the IMC error too.

5. 72kHz, 108kHz, 144kHz: Harmonics of 36kHz?

6. Broad feature from 40kHz to 200kHz. The IMC loop is adding the noise. This is the frequency range of the PC drive. Is something in the PC drive noisy???
 

7. The feature at 130kHz. Unknown. Seems not related to IMC. The laser noise or the PMC noise.


Remaining IMC issues:

Done (Nov 23, 2015) - 29.5MHz oscillator output degraded. Possibly unstable and noisy. Do we have any replacement? Can we take a Marconi back from one of the labs?

Done (Nov 23, 2015) - Too high LO?

- Large 36kHz peak in the IMC

- IMC loop shape optimization

- IMC locking issue. The lock streatch is not long.

- IMC PC drive issue. Could be related to the above issue.

Maybe not relevant - PC drive noise?

Attachment 1: IMC_OLTF.pdf
IMC_OLTF.pdf
Attachment 2: PMC_noise_comparison.pdf
PMC_noise_comparison.pdf
  11794   Sat Nov 21 00:45:30 2015 KojiUpdateIOOIMC fix

Based on the observation of the PMC error signal, I started measuring the IMC OLTF. Immediately, it was found that the overall IMC loop gain was too low.
The UGF was ~40kHz, which was really marginal. It had been >100kHz when I have adjusted it about a year ago. (Next entry for the detail)

The first obvious thing was that the SMA cables around the IMC servo have visible degradation (Attatched photos).
I jiggled the signal cable from the demodulator Q_out to the MC servo. The openloop gain seemed fluctuating (increased) based on the cabling.
I decided to repair these cables by adding solder on the shield.

Even after the repair, the open loop TF didn't show any improvement. I checked the LO level and found that it was -16.7dBm.

I traced the problem down to the frequency generator unit (T1000461). The front panel of the unit indicates the output power for the 29.5MHz output is 13dBm,
while measurement showed it was 6~8dBm (fluctuating). The T1000461 document describes that there is only a wenzel oscillator inside. Does this mean the oscillatorwas degraded??? We need to open the box.

I was not sure what was the LO level. I naively assumed the input is 0dBm. Reducing the attenuation of the dial on the AM Stabilizer unit from 12dB attn to 0dB.
This made the LO level -3.3dBm.


Later at home, I thought this nominal LO level of 0dBm could have been wrong.

The demodulator circuit (D990511) has the amplifier ERA-5 (G=~20dB) at the input. Between the input and the ERA-5 there is a pattern for an attenuator.
Assuming we have no attenuator, the ERA-5 has to spit out 20dBm. That is too much for this chip. I need to pull out the box to see how much is the nominal LO for this box using an active probe.

This decrease/increase of the LO level affects the WFS demod too. According to D980233-B, the input stage has the comparator chip AD96687, which can handle differential voltage of 5.5V.
Therefore the effect is minimal.

Attachment 1: PDRFsignal_cable.JPG
PDRFsignal_cable.JPG
Attachment 2: Qsignal_cable1.JPG
Qsignal_cable1.JPG
Attachment 3: Qsignal_cable2.JPG
Qsignal_cable2.JPG
Attachment 4: Qsignal_cable3.JPG
Qsignal_cable3.JPG
  11793   Fri Nov 20 15:44:12 2015 KojiSummaryPSLAdded 17.5kHz LPF to the PMC servo

As a final tune of the PMC servo, I've added 1nF cap at the error signal amplification stage. The diagram has been updated and uploaded to DCC. https://dcc.ligo.org/D1400221

It should be noted that this modification yielded the error signal to have 17.5kHz roll off.


The openloop TF after the modification has been measured. (Attachment 1)

With the new nominal gain of 9dB, almost the same gain margin for the 28kHz peak has been realized.
=> We have 6dB (factor of 2) more gain at low frequency. Currently, the feature at 8kHz causes the oscillation when the gain is further increased.
 

Here is the model function for the OLTF.

function cmpOLTFc = PMC_OLTF_model(freqOLTFc)

cmpOLTFc = -9.5e5*pole1(freqOLTFc,0.162).*zero1(freqOLTFc,491)... % from the circuit diagram
    .*pole1(freqOLTFc,17.5e3)... % Newly implemented input filter => GBW pole was replaced with this
    .*zero2(freqOLTFc,12.5e3,100)... % eye-fit
    .*pole2(freqOLTFc,12.2e3,6)... % eye-fit
    .*pole2(freqOLTFc,28.8e3, 12)... % eye-fit
    .*pole1(freqOLTFc,150e3)... % Mixer LPF estimated from Circuit Lab Simulation
    .*pole1(freqOLTFc,11.3)... % Output Impedance + PZT LPF
    .*pole1(freqOLTFc,3e4); % Unknown
   
end


The free-running round-trip displacement (roundtrip) / frequency noise is shown in Attachments. There we compare the spectra with and without IMC locked.

i.e. When the IMC is not locked, we are measuring the laser frequency noise with the sensor (PMC cavity) that is noisy due to the PMC displacement.
When the IMC is locked, the laser frequency is further stabilized while the sensor (PMC) noise is not changed.

- Without IMC locked

Can we see the laser freq noise? It seems that it is visible above 100Hz.
The red curve is the measured noise level. The NPRO (although it is LWE NPRO) noise level from S. Nagano's thesis (see our wiki) is shown there.

- With IMC locked

When the IC is locked, we see the increase of the noise between 1~4Hz. It means that the IMC is not only noisier than the laser, but also noisier than the PMC cavity.
Sounds reasonable. And the PMC is capable to handle this motion.

The reduction of the frequency noise is seen from 100Hz to 30kHz.

The interesting point is that we can see the noise increase above 30kHz when the IMC is locked.
I believe that the phase correction EOM is shared with the PMC modulation. i.e. PMC sees the corrected laser frequency.

We expect that the frequency noise is reduced at this frequency. But in reality not.

In addition, there is a sharp peak at ~35kHz. I wonder If this is caused by the IMC servo. It is worse to investigate.

Attachment 1: PMC_OLTF.pdf
PMC_OLTF.pdf
Attachment 2: PMC_noise_comparison.pdf
PMC_noise_comparison.pdf
  11792   Fri Nov 20 09:45:39 2015 steveSummarySUSoplev laser summary

 

Quote:

 

Quote:

 

      May  13, 2014             ETMX,  .............laser in place 90 d

      May  22, 2012             ETMY, 

     Oct.  7,  2013             ETMY,  LT  503 d  or  1.4 y............bad beam quality ?

     Aug. 8,  2014              ETMY,  .............laser in place   425 days  or  1.2 y

 

      Sept. 5, 2014              new 1103P, sn P893516  installed at SP table for aLIGO oplev use qualification

 

Attachment 1: oplev_summery.png
oplev_summery.png
  11791   Thu Nov 19 17:06:57 2015 KojiConfigurationCDSDisabled auto-launching RT processes upon FE booting

We want to startup the RT processes one by one on boot of FE machines.

Therefore /diskless/root/etc/rc.local on FB was modified as follows. The last sudo line was commented out.

for sys in $(/etc/rt.sh); do
    #sudo -u controls sh -c ". /opt/rtapps/rtapps-user-env.sh && /opt/rtcds/cal\
tech/c1/scripts/start${sys}"

    # NOTE: we need epics stuff AND iniChk.pl in PATH
    # we use -i here so that the .bashrc is sourced, which should also
    # source rtapps and rtcds user env (for epics and scripts paths)

    # commented out Nov 19, 2015, KA
    # see ELOG 11791 http://nodus.ligo.caltech.edu:8080/40m/11791
    # sudo -u controls -i /opt/rtcds/caltech/c1/scripts/start${sys}
done

  11790   Thu Nov 19 16:06:54 2015 yutaroUpdateOptical LeversCalibration of oplevs for ITMY/ETMY

I'm sorry. I will be careful about that. And I updated the plots in elog 11785.

Quote:

OMG. Please try to use larger fonts and PDF so that we can read the plots.

 

Quote:
Quote:

Based on elog 1403, I calibrated the oplevs for ITMY/ETMY.

I'm not sure that these calibration measurements are reliable. I would feel better if Steve can confirm them using our low accuracy method of moving the QPD by 1 mm and doing trigonometry.

 In this morning, Steve and I looked at the ETMY table and we found that the measurement you suggested might interfere with other optics or detectors because of space constraint. So, before doing this measurement, I roughly estimated the calibration factors for ETMY oplev by using the rough value of the arm length of the optical lever and the beam width of the light just before the QPD.

 

How I got the arm length and the beam width:

I measured the length of the optical path between ETMY and the QPD. Then I measured the beam width with an iris to screen the beam. To get the beam width from the decrease of the power of the beam detected by QPD, I used this formula: P/P_{max}=1-\exp(-2r^2/w^2).

Then I got:   (arm length) = 1.8 +/-0.2 m,    w= 0.56 +/- 0.5 mm.

 

How I estimated the calibration factors from these:

The calibration factors (such as C1:SUS-ETMY_OL_PIT_CALIB; (real angle) / (normalized output of QPDXorY)) can be calculated with: \sqrt{\pi/32}\times w\,/\,(\mathrm{arm\,length}). Then, I got

130\,\pm\,20 \,\mu \mathrm{rad},

though the calibration factors, C1:SUS-ETMY_OL_PIT_CALIB C1:SUS-ETMY_OL_YAW_CALIB, right now are 26.0 and 31.0, respectively. (If I express this in the same way as elog 11785, 5.0 and 4.2 for ETMY_PIT and ETMY_YAW, respectively. they are consistent with yesterday's results.) 

I believe that the calibration factors I estimated today are not different from the true values by a factor of 2 or something, so this estimation indicates that the oplev calibration measurements I did yesterday are reliable, at least for the oplev for ETMY. 

 

 

  11789   Thu Nov 19 15:16:24 2015 ericqUpdateCDSc1iscey IO chassis now has brackets

[Steve, ericq]

Brackets for the c1iscey IO chassis cards have been installed. Now, I can't unseat the cards by wiggling the ADC or DAC cable. yes

  11788   Thu Nov 19 14:50:34 2015 Max IsiUpdateGeneralNew 2D histogram plot for summary pages

A new type of plot is now available for use in the summary pages, based on EricQ's 2D histogram plots (elog 11210). I have added an example of this to the SandBox tab (https://nodus.ligo.caltech.edu:30889/detcharsummary/day/20151119/sandbox/). The usage is straighforwad: the name to be used in config files is histogram2d; the first channel corresponds to the x-axis and the second one to the y-axis; the options accepted are the same as numpy.histogram2d and pyploy.pcolormesh (besides plot limits, titles, etc.). The default colormap is inferno_r and the shading is flat.

Attachment 1: C1-ALL_AB3834_HISTOGRAM2D-1131926417-86400.png
C1-ALL_AB3834_HISTOGRAM2D-1131926417-86400.png
  11787   Wed Nov 18 23:40:01 2015 ranaUpdateOptical LeversCalibration of oplevs for ITMY/ETMY

OMG. Please try to use larger fonts and PDF so that we can read the plots.

Quote:

Based on elog 1403, I calibrated the oplevs for ITMY/ETMY.

I'm not sure that these calibration measurements are reliable. I would feel better if Steve can confirm them using our low accuracy method of moving the QPD by 1 mm and doing trigonometry.

  11786   Wed Nov 18 23:18:07 2015 ericqUpdateComputer Scripts / Programsnodus /boot cleared up

The /boot partition was filling up with old kernels. Nodus has automatic security updates turned on, so new kernels roll in and the old ones don't get removed. 

I ran apt-get autoremove, which removed several old kernels. (apt is configured by default to keep two previous kernels around when autoremoving, so this isn't so risky)

Now: /dev/sda1                    236M   94M  130M  42% /boot

In principle, one should be able change a setting in /etc/apt/apt.conf.d/50unattended-upgrades that would do this cleanup automatically, but this mechanism has a bug whose fix hasn't propagated out yet (link). So, I've added a line to nodus' root crontab to autoremove once a week, Sunday morning. 

  11785   Wed Nov 18 22:26:33 2015 yutaroUpdateOptical LeversCalibration of oplevs for ITMY/ETMY

Based on elog 1403, I calibrated the oplevs for ITMY/ETMY.

Summary of this method is following:

We lock an arm, and slightly misalign one mirror of the arm. Then, the transmission of the arm starts to decrease quadratically as the misalign angle of the mirror changes. Here, how much the transmission decreases in terms of the misalign angle is determined by geometry of optics, so we can see how much the angle really changes from this quadratic curve.

 

RESULTS  

These are the relationship between misalign angles measured by oplev (the units are based on the calibration for now) and transmission power.

(I updated following figures on Nov 19 2015. You can find the figures I attached once here in the zipped folder attached.) 

 

 

According to this measurement, ratio of the calibration factor derived with this measurement (NEW) and the calibration factor for now (OLD), i.e. NEW/OLD was:  

ETMY_PIT: 5.0265  --->> 5.3922 (without an outlier; the outlier I removed is shown in the figure in zipped folder attached.)

ETMY_YAW: 4.6205

ITMY_PIT: 1.5010

ITMY_YAW: 1.2970

This results show that calibration of oplevs for ITMY is kind of OK, but that for ETMY is so BAD and the calibration factors should be updated.

 

NOTE

The calibration factors of the oplevs for ETMY/ITMY are   NOT UPDATED YET.  I updated on Dec 11, 2015

If these results are reliable, I will update them tomorrow.   

Attachment 1: calib_etmypit2.pdf
calib_etmypit2.pdf
Attachment 2: calib_etmyyaw2.pdf
calib_etmyyaw2.pdf
Attachment 3: calib_itmypit2.pdf
calib_itmypit2.pdf
Attachment 4: calib_itmyyaw2.pdf
calib_itmyyaw2.pdf
Attachment 5: calib_oplev.zip
  11784   Wed Nov 18 20:49:05 2015 ranaUpdateComputer Scripts / Programsnodus boot getting full

controls@nodus|~ > df -h
Filesystem                   Size  Used Avail Use% Mounted on
/dev/mapper/nodus2--vg-root  355G   69G  269G  21% /
udev                         5.9G  4.0K  5.9G   1% /dev
tmpfs                        1.2G  308K  1.2G   1% /run
none                         5.0M     0  5.0M   0% /run/lock
none                         5.9G     0  5.9G   0% /run/shm
/dev/sda1                    236M  210M   14M  94% /boot
chiara:/home/cds             2.0T  1.5T  459G  77% /cvs/cds
fb:/frames                    13T   11T  1.6T  88% /frames

  11783   Wed Nov 18 17:32:36 2015 yutaroUpdateOptical LeversBeam centering for the oplev of ITMY

[yutaro, Koji]

We made the beam spot on QPD for the oplev of ITMY centered by changing the orientation of the mirror just before the QPD.

Before doing this, we ran dithering for Y arm and froze the output of ASS for Y arm.

  11782   Wed Nov 18 17:09:22 2015 KojiSummaryPSLPMC Servo analysis

Summary

The PMC servo was analysed. OLTF was measured and modeled by ZPK (Attachment 1). The error and actuator signals were calibrated in m/rtHz (Attachment 2)

Measurement methods

OLTF:

- The PMC servo board does not have dedicated summing/monitor points for the OLTF measurement. Moreover the PZT HV output voltage is monitored with 1/49.6 attenuation.
  Therefore we need a bit of consideration.

- The noise injection can be done at EXT DC.
- Quantity (A): Transfer function between HV OUT MON and MIX OUT MON with the injection.
  We can measure the transfer function between the HV OUT (virtual) and the MIX OUT. (HV OUT->MIX OUT). In reality, HV OUT is attenuated by factor of 49.6.
  i.e. A = (HV_OUT->MIX_OUT)*49.6
- Quantity (B): Transfer function between HV OUT MON and MIX OUT MON without the injection.
 
This is related to the transfer function between the MIX OUT and HV OUT. In reality, HV OUT is attenuated. 
  i.e. B = 1/((MIX_OUT->HV_OUT)/49.6)

- What we want to know is HV_OUT->MIX_OUT->HV_OUT. i.e. A/B = (HV_OUT->MIX_OUT*49.6)*((MIX_OUT->HV_OUT)/49.6) = HV_OUT->MIX_OUT->HV_OUT

PSD:

- The MIX OUT and HV OUT spectra have been measured. The MIX OUT was calibrated with the calibration factor in the previous entry. This is the inloop stability estimation.
  From the calibrated MIX OUT and HV OUT, the free running stability of the cavity was estimated, by mutiplying with |1-OLTF| and |1-1/(1-OLTF)|, respectively, in order to recover
  the free running motion.

OLTF Modeling

Here is the model function for the open loop TF. The first line comes from the circuit diagram. The overall factor was determined by eye-fit.
The second and third lines are to reproduce the peak/notch feature at 12kHz. The fourth line is to reproduce 28kHz feature.
The LPF right after the mixer was analyzed by a circuit simulation (Circuit Lab). It can be approximated as 150kHz LPF as the second pole
seems to come at 1.5MHz.

The sixth line comes from the LPF formed by the output resistance and the PZT capacitance.

The seventh line is to reproduce the limit by the GBW product of OP27. As the gain is 101 in one of the stages,
it yields the pole freq of ~80kHz. But it is not enough to explain the phase delay at low frequency. Therefore this
discrepancy was compensated by empirical LPF at 30kHz.

function cmpOLTFc = PMC_OLTF_model(freqOLTFc)

cmpOLTFc = -7e5*pole1(freqOLTFc,0.162).*zero1(freqOLTFc,491)... % from the circuit diagram
    .*zero2(freqOLTFc,12.5e3,100)... % eye-fit
    .*pole2(freqOLTFc,12.2e3,6)... % eye-fit
    .*pole2(freqOLTFc,27.8e3, 12)... % eye-fit
    .*pole1(freqOLTFc,150e3)... % Mixer LPF estimated from Circuit Lab Simulation
    .*pole1(freqOLTFc,11.3)... % Output Impedance + PZT LPF
    .*pole1(freqOLTFc,8e6/101)... % GBW OP27
    .*pole1(freqOLTFc,3e4); % Unknown

end

Result

Attachment 1:

The nominal OLTF (Nov 17 data) shows the nominal UGF is ~1.7kHz and the phase margin of ~60deg.

The measured OLTF was compared with the modelled OLTF. In the end they show very sufficient agreement for further calibration.
The servo is about to be instable at 28kHz due to unknown series resonance. Later in the same day, the gain of the PMC loop had to be
reduced from 7dB to 3dB to mitigate servo oscillation. It is likely that this peak caused the oscillation. The notch frequency was measured
next day and it showed no sign of frequncy drift. That's good.

We still have some phase to reduce the high freq peaks by an LPF in order to increase the over all gain.

Attachment 2:

The red curve shows the residual floor displacement of 2~10x10-15 m/rtHz. Below 4Hz there is a big peak. I suspect that I forgot to close
the PSL shutter and the IMC was locked during the measurement. Then does this mean the measured noise corresponds to the residual laser
freq noise or the PMC cavity displacement? This is interesting to see.

The estimated free running motion from the error and actuation signals agrees very well. This ensures the precision of the caibration in the precious entries.
 

 

Attachment 1: PMC_OLTF.pdf
PMC_OLTF.pdf
Attachment 2: PMC_DSP.pdf
PMC_DSP.pdf
  11781   Wed Nov 18 16:53:17 2015 KojiSummaryPSLPMC PZT capacitance

The PMC PZT capacitance was measured.
- Turn off the HV supplies. Disconnect HV OUT cable.
- Make sure the cable is discharged.
- Measure the capacity at the cable end with an impedance meter.
=> The PMC PZT capacitance at the cable end was measured to be 222nF
Combined with the output impedance of 63.3kOhm, the LPF pole is at 11.3Hz

  11780   Wed Nov 18 16:51:58 2015 KojiSummaryPSLPMC servo calibration

Summary

The PMC servo error (MIX OUT MON on the panel) and actuation (HV OUT MON) have been calibrated using the swept cavity.

Error signal slope in round-trip displacement: 2.93e9 +/- 0.05e9 [V/m]
HV OUT calibration (round-trip displacement): 5.36e-7 +/- 0.17e-7 [m/V]
PZT calibration (round-trip displacement): 10.8 +/- 0.3 [nm/V]
=> corresponds to ~2.5 fringes for 0~250V full range => not crazy

Measurement condition

The transmission level: 0.743V (on the PMC MEDM screen)
LO level: ~13dBm (after 3dB attenuation)
Phase setting: 5.7
PMC Servo gain: 7dB during the measurement (nominal 3dB)

Method

- Chose PMC actuation "BLANK" to disable servo
- Connect DS345 function generator to EXT DC input on the panel
- Monitor "MIX OUT MON" and "HV OUT MON" with an oscilloscope
- Inject a triangular wave with ~1Vpp@1 or 2Hz with appropriate offset to see the cavity resonance at about the middle of the sweep.
  The frequency of the sweep was decided considering the LPF corner freq formed by the output impedance and the capacitance of the PZT. (i.e. 11.3Hz, see next entry)

Result

- 4 sweep was taken (one 2Hz seep, three 1Hz sweep)
- The example of the sweep is shown in the attachment.
- The input triangular wave and the PDH slopes were fitted by linear lines.
- Spacing between the sideband zero crossing corresponds to twice of the modulation frequency (2x35.5MHz = 71MHz)
- The error signal slope was calibrated as V/MHz
- FSR of the PMC is given by google https://www.google.com/search?q=LIGO+pmc.m
  => Cavity round trip length is 0.4095m, FSR is 732.2MHz
- Convert frequency into round-trip displacement

- Convert HV OUT MON signal into displacement in the same way.
- The voltage applied to the PZT element is obtained considering the ratio of 49.6 between the actual HV and the HV OUT MON voltage.

Attachment 1: PMC_err_cal.pdf
PMC_err_cal.pdf
  11779   Wed Nov 18 11:22:13 2015 yutaroUpdateASCLoss maps of arm cavities

I got linear relation between these. The results and method are below.

Quote:

In preparation for the measurement of loss maps of arm cavities, I measured the relationship between:

the offset just after the demodulation of dithering loop (C1:ASS-YARM_ETM_PIT_L_DEMOD_I_OFFSET and C1:ASS-YARM_ETM_YAW_L_DEMOD_I_OFFSET)

and

the angle of ETMY measured with oplev (C1:SUS-ETMY_OL_PIT_INMONC1:SUS-ETMY_OL_PIT_INMON and C1:SUS-ETMY_OL_PIT_INMONC1:SUS-ETMY_OL_PIT_INMON)

while the dithering script is running. With the angle of ETMY, we can calculate the beam spot on the ETMY assuming that the beam spot on the ITMY is not changed thanks to the dithering. What we have to do is to check the calbration of oplev with another way to measure the angle, to see if the results are reliable or not.

I will report the results later.

 

RESULTS

 

 

METHOD

I added offset (C1:ASS-YARM_ETM_PIT_L_DEMOD_I_OFFSET and C1:ASS-YARM_ETM_YAW_L_DEMOD_I_OFFSET) to shift the beam spot on ETMY. For each data point, I measured the difference in angle of ETMY with oplev before and after adding offset. The precedure for each measurement I employed is following:

-- run dither

-- wait until error signals of dithering settle down

-- freeze dither

-- measure angle (10s avg)

-- add offset

-- wait until error signals of dithering settle down

-- freeze dither

-- measure angle (10s avg)

The reason why I measured the angle without offset for each measurement is that the angle which oplev shows changed with time (~several tens of minutes or something). 

 

DISCUSSION

At the maximum values of offsets, the arm transmission power started to degrade, so the range where I can do this measurement is limited by these values as for now. However, we have to shift the beam spot more in order to make loss maps of sufficiently broad area on the mirror (the beam width (w) on ETM; w ~ 5 mm). Then, we have to find out how to shift the beam spot more.  

  

Attachment 1: ETM_PIT_up.png
ETM_PIT_up.png
  11778   Wed Nov 18 10:10:53 2015 ericqUpdateCDSc1iscey IO chassis missing brackets

Steve and I inadvertently discovered that the c1iscey IO chassis doesn't have brackets to secure the cards where the ADC/DAC cables are connected, making them very easy to knock loose. All other IO chassis have these brackets. Pictures of c1iscey and c1lsc IO chassis to compare:

  11777   Tue Nov 17 20:57:43 2015 ericqUpdateCDS16Hz frame writing running again

Back to nominal FB configuration at 1131857782, aka Nov 18 2015 04:56:05 UTC.

Weirdly, during this time, the script watching MC_TRANS_SUM from pianosa saw tons of freezes, but another instance  watching LSC-TRY_OUT16 on optimus saw no freezes. 

  11776   Tue Nov 17 20:40:08 2015 yutaroSummaryASCLoss maps of arm cavities

In preparation for the measurement of loss maps of arm cavities, I measured the relationship between:

the offset just after the demodulation of dithering loop (C1:ASS-YARM_ETM_PIT_L_DEMOD_I_OFFSET and C1:ASS-YARM_ETM_YAW_L_DEMOD_I_OFFSET)

and

the angle of ETMY measured with oplev (C1:SUS-ETMY_OL_PIT_INMONC1:SUS-ETMY_OL_PIT_INMON and C1:SUS-ETMY_OL_PIT_INMONC1:SUS-ETMY_OL_PIT_INMON)

while the dithering script is running. With the angle of ETMY, we can calculate the beam spot on the ETMY assuming that the beam spot on the ITMY is not changed thanks to the dithering. What we have to do is to check the calbration of oplev with another way to measure the angle, to see if the results are reliable or not.

I will report the results later.

  11775   Tue Nov 17 16:21:10 2015 KojiSummaryPSLPMC servo circuit review, follow up measurements

I'm still analyzing the open loop TF data. Here I report some nominal settings of the PMC servo

Nominal phase setting: 5.7
Nominal gain setting: 3dB

After the tuning of the notch frequency, I thought I could increase the gain from 5dB to 9dB.
However, after several hours of the modification, the PMC servo gradually started to have oscillation.
This seemed to be mitigated by reducing the gain down to 4dB. This may mean that the notch freq got drifted away
due to themperature rise in the module. PA85 produce significant amount of heat.

(The notch frequency did not change. Just the 22kHz peak was causing the oscillation.)

  11774   Tue Nov 17 15:59:23 2015 ranaUpdateSUSITMX UL calmed?

Although this noise is bad, we have always had these kind of humps around the bounce mode. Our interpretation in the past was that this was due to poor alignment of the OSEM in the frame, leading to a large vertical to horizontal coupling. Once you implement the BLRMS for the SUS channels, we'll be able to trend the noise over long periods of time.

  11773   Tue Nov 17 15:49:23 2015 KojiConfigurationIOOMC Autolocker modified

/opt/rtcds/caltech/c1/scripts/MC/AutoLockMC.csh  was modified last night.

1. Autolocker sometimes forget to turn off the MC2Tickle. I added the following lines to make sure to turn it off.

    echo autolockMCmain: MC locked, nothing for me to do >> ${lfnam}
    echo just in case turn off MC2 tickle >> ${lfnam}
    ${SCRIPTS}/MC/MC2tickleOFF

2. During the lock acquisition, Autolocker frequently stuck on a weak mode. So the following lines were added
so that the Autolocker toggles the servo switch while waiting for the lock.

    echo autolockMCmain: Mon=$mclockstatus, Waiting for MC to lock .. >> ${lfnam}
    # Turn off MC Servo Input button
    ezcawrite C1:IOO-MC_SW1 1
    date >> ${lfnam}
    sleep 0.5;
    # Turn on MC Servo Input button
    ezcawrite C1:IOO-MC_SW1 0
    sleep 0.5;

  11772   Tue Nov 17 14:31:25 2015 ericqUpdateCDS16Hz frame writing temporarily disabled

To test the effect on EPICS latency, I've restarted daqd with modified ini files which disable all frame writing of 16Hz channels. 

This happened at GPS:1131835955 aka Nov 17 2015 22:52:18 UTC

Last night, I started running a script written by Dave Barker that monitors a specified EPICS channel (in this case C1:IOO-MC_TRANS_SUM), to look for seconds in which it does not update the expected number of times. This is still running, so I will be able to compare the rate of EPICS slowdowns before and after this change. 

I will revert back to the nominal state of things in a few hours, or until someone asks me to. 

  11771   Tue Nov 17 10:06:53 2015 SteveUpdateGeneralwater leak in east arm
Quote:

A Caltech maintenance staff dropped by at around noon today, and told me that he had seen a small puddle of water on the other side of the door along the Y-arm that is kept locked (about 10m from the end-table, on the south side of the arm). He suspected a leak in the lab. Koji and I went down to the said door and observed that there was indeed a small puddle of water accumulated there. There isn't any obvious source of a leak on our side of the door, although the walls tiles in the area suggest that there could be a leak in one of the pipes running through the wall/under the floor. In any case, the leak doesn't seem too dramatic, and we have decided to consult Steve as to what is to be done about this once he is back on Wednesday.

The leak was found inside the wall. Fortunately the plumbers were able to access it from CES room 108

This has been leaking for sometimes. The damaged wall area is about 18 ft long and 1 ft high.

Attachment 1: eastArmLeakyPipeInsideWall.jpg
eastArmLeakyPipeInsideWall.jpg
Attachment 2: wd18ft.jpg
wd18ft.jpg
ELOG V3.1.3-