40m QIL Cryo_Lab CTN SUS_Lab TCS_Lab OMC_Lab CRIME_Lab FEA ENG_Labs OptContFac Mariner WBEEShop
  40m Log, Page 70 of 344  Not logged in ELOG logo
ID Date Author Type Category Subjectdown
  7802   Sun Dec 9 17:51:53 2012 ranaUpdateLSCbeam inside DRMI is clipping on PR3 Tip-Tilt

 

 Some explanation of how you define power buildup please. Also some plots showing the evidence.

  7803   Mon Dec 10 03:02:03 2012 DenUpdateLSCbeam inside DRMI is clipping on PR3 Tip-Tilt

Quote:

 

 Some explanation of how you define power buildup please. Also some plots showing the evidence.

 I think about power buildup as a ratio of the power in the cavity when it is locked and unlocked = (POYDC_LOCKED - POYDC_OFFSET) / (POYDC_UNLOCKED - POYDC_OFFSET). I do not multiply this number by PRM transmission.

POYDC_OFFSET = -0.006

POYDC_UNLOCK = 0.063

For example, on the plot below power buildup is 15.

PRCL_LOCK.png

  7804   Mon Dec 10 10:13:41 2012 DenUpdateLSCbeam inside DRMI is clipping on PR3 Tip-Tilt

 

That's OK, but its best to use standard notation. The power recycling gain is defined as the power incident on the BS divided by the power incident on the PRM from the laser side. You should also compare it with the PRC gain that you expect from mirror transmissions.

  7806   Mon Dec 10 22:34:34 2012 DenUpdateLSCbeam inside DRMI is clipping on PR3 Tip-Tilt

Quote:

 

That's OK, but its best to use standard notation. The power recycling gain is defined as the power incident on the BS divided by the power incident on the PRM from the laser side. You should also compare it with the PRC gain that you expect from mirror transmissions.

 I've made snapshots of PR2, PRM, ITMY and ITMX mirrors. Power buildup recycling gain (POWER BS / POWER PRM) was equal to 3-4.

PR2.bmp    PRM_LOCK.bmp    ITMY.png    ITMX.png

  7820   Thu Dec 13 03:20:48 2012 DenUpdateLSCbeam inside DRMI is clipping on PR3 Tip-Tilt

Quote:

 

 I've made snapshots of PR2, PRM, ITMY and ITMX mirrors. Power buildup recycling gain (POWER BS / POWER PRM) was equal to 3-4.

           

 We've looked at PR2 face camera when PRM, BS and one of the ITMs were aligned. We saw an extra beam at PR2 when ITMX was aligned (right plot). This spot stays on the PR2 when prcl is locked.

PR2_ITMX.png   PR2_ITMY.png

Then we looked at PR3 transmission mirror and saw that the main beam is not on the edge of the mirror. Secondary beam is clipping on the mirror mount of PR3 that we see on BS_PRM camera.

PR3_LOCK.png

Measured beam spot positions:

Optics Pitch, mm Yaw, mm
ITMX 5.6 1.5
ETMX -1.5 1.5
ITMY 4.8 -1.5
ETMY -1.4 5.6
PRM 2.7 4.1

"+" for pitch means that the beam is too high, "-" too low

"+" for yaw means that the beam is left if you look from the back, "-" is right

Beam spots were measured using x, y arm and prcl locking to the carrier.

  9325   Fri Nov 1 09:45:32 2013 SteveUpdateIOObeam dumps to be find

Quote:

Quote:

Steve, can you please put 2 mounted  razor dumps near the WFS for this purpose??    

            Tuesday: Razor dumps are waiting for you.

 I couldn't find any dumps near the WFS. Koji looked. I looked twice. Maybe they are spooky and absorbing all of the light?

The MC alignment was bad and the WFS were making it drift. Koji aligned the beam into the PMC. I then restored the MC suspensions to where they were 8 days ago (back when the transmission and reflection were good). With the WFS OFF, this gave us a MC trans ~ 16000. With WFS ON it goes to 17500 which is about as good as its been over the last 80 days.

I centered the beam on the WFS with the MC unlocked and also centered the beam on the whole WFS path (it was near clipping between WFS 1 & 2). Also for some reason that beamsplitter which steers the beam onto WFS1 is a R=33% (!? why is this not a R=50% ??).

Steve, please swap this out to a BS1-1064-50-1025-45S if we have one sitting around. If not, we want to add this to the CVI purchase list, but not buy until we get a bigger list together.

I also centered this newly aligned beam into the IMC onto the PSL QPDs. We should now use these as a pointing reference for the beam into the IMC.

While doing this I noticed that the beam was almost clipping on the Uniblitz shutter used to block the PSL beam. That shutter is mounted too short and was also not centered horizontally. I removed it for now so that Steve can find a more adjustable mount for it and put it back into play. The beam going into the IMC is BIG, so you have to very careful when centering the shutter. Might be that we cannot leave it at 45 deg and still get a big enough aperture.

Note #3 for Steve: please also replace the mount for last steering mirror into the IMC with a Polanski or a Superman, that black Ultima is no good. Also the dogs must be steel - no aluminum dogs for our sensitive places.

No wonder they could not find the beam dumps. Last night was Haloween. They should of just said: Trick or treat! where are the beam dumps?

  9974   Tue May 20 11:48:22 2014 SteveUpdateSUSbeam dumps added to ETMX_ISCT

Anodized aluminum dumps replaced by 6 razor beam dumps.

Two more razor beam dumps added this afternoon.   The picture will updated tomorrow.

  9448   Fri Dec 6 15:57:41 2013 SteveUpdateIOObeam dumps for PSL pointing monitoring

Quote:

Since the pointing has gone bad again, I went to the PSL to investigate. Found some bad things and removed them:

1) There was a stopped down iris AGAIN in the main beam path, after the newly installed mirror mount. I opened it. Stop closing irises in the beam path.

2) The beam dump for the IOO QPD reflection was just some black aluminum. That is not a real dump. I removed it. We need two razor blade dumps for this.

3) There was an ND filter wheel (???) after one of the PMC steering mirrors. This is not good noise / optics practice. I removed it and dumped the beam in a real dump. No elog about this ?!#?

 

The attached trend shows the last 20 days. The big step ~2 weeks ago is when Steve replaced the steering mirror mount with the steel one. I don't understand the drift that comes after that.

 

Today I also spent ~1 hour repairing the Aldabella laptop. Whoever moved it from the PSL area to the SP table seems to have corrupted the disk by improper shutdown. Please stop shutting the lid and disconnecting it from the AC power unless you want to be fixing it. Its now running in some recovery mode. Lets leave it where it is next to the PSL and MC1.

I steered the MC suspensions back to where they were on the trends before the PSL mirror mount swap and then aligned the PSL beam into it by touching the last 2 steel mounts. Once the alignment was good without WFS, I centered the beams on the IOO QPDs. If it behaves good overnight, I will center the unlocked beams on the MC WFS.

 

Please stay off the PSL for a couple days if you can so that we can watch the drift. This means no opening the doors, turning on the lights, or heavy work around there.

 IOO pointing monitoring qpds received razor beam dumps on their refs.

The Pos QPD was rotated and recentered.

The Ang QPD was left untouched.

TREND plot of 23 days is attached.

  9420   Thu Nov 21 10:24:50 2013 KojiUpdateSUSbeam dumps

You don't need the fourth glass piece on the diamond beam dump.

  893   Thu Aug 28 18:56:14 2008 ranaConfigurationPSLbeam block distorted
There was a beam block after the Mach Zender. Who or what put this there?

The going to the MC now looks distorted as if someone has left something funny in the beam or maybe the new PMC has started to degrade??

Use the ELOG people...its good for you.
  896   Fri Aug 29 10:20:32 2008 YoichiConfigurationPSLbeam block distorted

Quote:
There was a beam block after the Mach Zender. Who or what put this there?

The going to the MC now looks distorted as if someone has left something funny in the beam or maybe the new PMC has started to degrade??

Use the ELOG people...its good for you.


I put the block. I was frequently reaching to the FSS box to change the test point probes. I put the block to protect my hands/clothes from being burnt accidentally.
  902   Fri Aug 29 16:35:18 2008 YoichiConfigurationPSLbeam block distorted

Quote:
There was a beam block after the Mach Zender. Who or what put this there?

The going to the MC now looks distorted as if someone has left something funny in the beam or maybe the new PMC has started to degrade??

Use the ELOG people...its good for you.


The apparent distortion of the MC refl. was caused by mis-alignment of the MC mirrors.
Because the MC1 was mis-aligned, the reflected light was clipped by a steering mirror.
I restored the MC angle bias values from the conlog history and now the MC locks.
According to conlog, the MC alignment was changed at around 18:30 on Thursday PDT.
It could have been caused by the computer reboots.
  5062   Fri Jul 29 16:25:06 2011 kiwamuUpdateASCbeam axis and Y arm aligned

Last night I aligned the incident beam axis and the Yarm by touching the PZT mirrors and the suspensions.

I didn't estimate how good they were aligned, but I guess the Y arm is now ready for the Y green light.

 Next : Y green alignment and the MC spots measurement / alignment.

 

 ++ Motivation ++

Prior to the coming vent we want to have the Y arm, incident beam axis and Y green light aligned so that we can align some necessary optics in the chamber.

Also alignment of the incident beam will allow us to re-position the incident beam alignment monitor (i.e. IPPOS and IPANG).

Our plan was to first align the Y arm using the ASS system and then align the Y green light to the Y arm.

 

++ what I failed ++

First I was trying to measure the spot positions on the MC mirrors to make me sure the beam axis has/hasn't changed.

Also I was going to align the MC suspensions to have nice spot position on each suspension using the MCASS system

because this will help us checking the beam clearance in the Faraday and perhaps re-positioning of the Faraday during the coming vent.

But essentially I failed and eventually gave up because MCASS didn't work. It seems that MCASS needs some modifications in the scripts.

Then, to make me feel better I moved on to the Y arm and beam axis alignment.

 

++ what I did ++

I tried using C1ASS to align the incident beam and suspensions on the Y arm, but it didn't work.

However the drive signals from ASS and its demodulated signals looked fine. Only the feedback did not work correctly.

Every time I enabled the feedback paths, the arm just lost the lock. Something is wrong in the feedback paths.

Then I started to align the cavity by my hands while looking at the demodulated signal from each LOCKIN module.

I aligned the things until each demodulated signal fluctuates around zero.

At the end the beam spots on the ETMY and ITMY camera looked well-aligned and the transmitted light became larger by a factor of 2ish.

  620   Wed Jul 2 00:59:13 2008 MashaUpdateAuxiliary lockingbalanced detection, noise plots, progress
Progress report submitted today(!). It is on the 40m wiki page. Below is a figure of some estimated noise sources.

Made voltage divider that acts as an attenuator for one of the paths in the Mach Zehnder, which should help to balance the detection and reduce noise.

First tested using a 636 Hz Matlab generated audio signal (thanks to John inspiration on portable headphones). Figure is attached, with plots of noise spectra of original and optimized signal with and without added acoustic noise (visible as peaks as 636 and 1272 Hz, linewidth approx 4 Hz). My first try at optimization reduces the noise by nearly an order of magnitude for most frequencies.

Will work on finding different noise source to better see what happens at low frequency, and try to get finer control of tunable gain.
  7375   Wed Sep 12 17:02:00 2012 SteveUpdateCamerasbaffle plate hole getting larger

Quote:

Quote:

The alignment of the pick-off mirror near ETMX is done. Everything turned out to be easy once we realized that there is no sense getting the alignment laser (going through viewport to pick-off to ITMX) back to ETMX. It is only necessary to hit ITMX somehow, since this makes sure that there is one scattered beam that will make it from ITMX to pick-off through viewport.

After the auxiliary optic (that we never used in the end) was removed again, we levelled the optical table.

So in the current setup, we can have small-angle scattering measurements on ITMX and large-angle scattering measurements on ETMX.

 This is how it was envisioned. The video camera was in nobodies mind to look through the 40 mm  diameter hole than.

 Rana is proposing 50 mm hole in the baffle plate that is attached to the tower.  Atm1

Atm2 is showing the back side where the solid line is 40 mm

  7358   Fri Sep 7 09:37:20 2012 SteveUpdateCamerasbaffle plate for SOS

Quote:

The alignment of the pick-off mirror near ETMX is done. Everything turned out to be easy once we realized that there is no sense getting the alignment laser (going through viewport to pick-off to ITMX) back to ETMX. It is only necessary to hit ITMX somehow, since this makes sure that there is one scattered beam that will make it from ITMX to pick-off through viewport.

After the auxiliary optic (that we never used in the end) was removed again, we levelled the optical table.

So in the current setup, we can have small-angle scattering measurements on ITMX and large-angle scattering measurements on ETMX.

 This is how it was envisioned. The video camera was in nobodies mind to look through the 40 mm  diameter hole than.

  7356   Fri Sep 7 00:08:10 2012 janoschMetaphysics baffle clipping loss

With a curvature radius of about 57m for the ETMs, flat ITMs at the beam waist, and using 39m for the arm lengths, one finds that the beam radius at the ETMs is about 5.3mm. The clipping power loss of a 5.3mm beam through a 20mm radius baffle hole would be less than a ppm of a ppm if the beam was perfectly centered. If the baffle hole had 15mm radius, the clipping loss would be 0.01ppm. If the baffle hole had 10mm radius, the loss would be 810ppm. The loss values are calculated using the formula of the  "Gaussian beam" Wikipedia article, "Power through an aperture" section. So I did not check if that one is ok.

  2975   Mon May 24 14:28:35 2010 kiwamuUpdateElectronicsbad power supply of a vme rack

In this morning I found daqawg didn't work.

After looking for the cause, I found one of the vme racks mounted on 1Y6 doesn't work correctly.

It looks like the vme rack mounting c0daqawg could not supply any power to the frontends.

 

Now Steve and I are trying to look for a spare for it.

  2978   Tue May 25 07:22:59 2010 kiwamuUpdateElectronicsbad power supply of a vme rack

Notes on May 25th

 Don't do the following things !! This causes bad cross-talking of CPUs mounted on the crate.

 


I moved c0daqawg and c1pem1 from 1Y6 vme crate to 1Y7 crate due to the bad power supply.

Another problem: c0dcu1 doesn't come back to the network. 

After moving them, I tried to get back them into the RFM network. However  c0dcu1 never came back, it still indicates red in C0DAQ_DETAIL.adl screen.

Alberto and I did even "nuclear option" (as instructed), but no luck.

  2981   Tue May 25 10:06:09 2010 kiwamuUpdateElectronicsbad power supply of a vme rack

 I got a VME crate from Peter's lab. It is already installed in 1Y6 instead of the old broken one.

I checked its power supply, and it looked fine. It successfully supplies +5, +12 and -12 V. And then I put c0daqawg and c1pem1 back from 1Y7.

Now I am trying to reboot all the front end computers with Peter's VME crate. A picture of the VME crate will be updated later.

  5521   Thu Sep 22 17:48:20 2011 kiwamuUpdateSUSbad oplev on ETMY

It turned out the oplev controls on ETMY were just bad.

It looks like the whitening filters have been OFF and because of that the resultant open-loop was not crossing the unity gain.

I will check the whitening filters.

 

(open-loop transfer function)

The blue dots are the measured data points and the green curve is the fit.

Apparently the open-loop doesn't go above the unity gain, so the oplev had been doing nothing.

If we try to increase the overall gain it will oscillate because of the phase delay of more than 180 deg around 3 Hz.

The red curve is the expected one with the whitening filters (WFs) properly engaged.

Note that WF are supposed to have two zeros at 1 Hz and two poles at 10 Hz.

 OLETMY.png

Quote from #5518
(to do)
 + optimization of the ETMY oplevs and OSEM damping.

  3173   Wed Jul 7 22:52:38 2010 rana, nancyConfigurationIOObad length control offset for the MC

Rana found out that a connection was bad in the shown place, due to which the MEDM screen was showing bad offset for length control.

Basically, the offset slider value would not go into the system because of that bad connection, and was locking the mode cleaner at the wrong location.

  3742   Tue Oct 19 21:10:27 2010 yutaUpdateCDSbad filter coefficients for every suspensions!

(Koji, Yuta)

Summary:
 The damping servos of the MC suspensions has not been working since yesterday. They had worked on Friday.
 We found that some of the filter coefficients somehow changed from the correct value during dealing with the sampling frequency shift (from 2048Hz to 16384Hz). (see elog #3659)

What we did:
We thought that the problem occurred from the CDS update on Monday. So, we restored them using svn.
 1. Went to /opt/rtcds/caltech/c1/core/advLigoRTS and ran svn update using the revision number 2125.
   svn update -r 2125
 2. Rebuilt all the front ends.
   ssh -X c1sus
   cd /opt/rtcds/caltech/c1/core/advLigoRTS
   make clean-SYSNAME
   make SYSNAME
   make install-SYSNAME 
   (where SYSNAME=c1x02,c1sus,c1mcs,c1rms)
 3. Restarted the front end models.
   ssh -X c1sus
   cd /opt/rtcds/caltech/c1/scripts
   sudo startc1SYS 
   (where SYS=sus,mcs,rms)
 4. Restarted mx_streams.
   sudo /etc/restart_streams
See this wiki page for the restart procedures.

At this point we judged that the realtime system and "dataviewer" worked fine (by poking some suspensions / by measuring PSDs/TFs of the signals).

However, just restoring the RT system didn't fix the damping servos.

After that, we measured the transfer functions of the servo filters using DTT(diaggui on rosalba).
 5. The filter at MC1_SUSPOS, "3:0.0" should have a cut-off frequency at 3Hz, but it was around 0.3Hz.
 6. We used foton in order to look at the filter bank files (in /cvs/cds/rtcds/caltech/c1/chans). Then we found that the filter design was somehow changed to
   zpk([0],[0.375],2.66666,"n")
  instead of the correct one
   zpk([0],[3],0.333333,"n")

Why the filter designs changed?:
 We suspect that the filter designs were changed because of the replacement of sampling frequency in filter bank files(see elog #3659). We only replaced the lines like
  # SAMPLING SUS_MC3_SUSPOS 2048
 to
  # SAMPLING SUS_MC3_SUSPOS 16384
 and didn't changed the coefficients.
 This may caused a problem when opening the files with foton, and foton changed 3Hz to 0.375Hz (2048/16384) and other coefficients.

Note:
 The damping servo for SIDE has been working for every MCs. POS, PIT, YAW are the ones not working.

Next work:
 - Fix filters for every suspensions.
   There are backup files in /cvs/cds/rtcds/caltech/c1/chans/filter_archive directory, so this should help.
   But I don't think just fixing filters would fix the damping servo because the filter design of MC suspensions were changed this morning and the damping had worked until Friday.

  4937   Tue Jul 5 08:32:38 2011 steveUpdatePEMbad air quality in the lab

The fireworks of yesterday showing up in the lab. Pasadena out side air 6.6 million  cfm for 0.3 micron particles  and 1.5 million cfm for 0.5 micron size.

  11329   Thu May 28 10:42:19 2015 SteveUpdatePEMbad Guralp X-cable
Quote:

Tried swapping cables at the Guralp interface box side. It seems that all of our seismic signal problems have to do with the GUR2 cable being flaky (not surprising since it looks like it was patched with Orange Electrical tape!! rather than proper mechanical strain relief).

After swapping the cables today, the GUR2 DAQ channels all look fine: i.e. GUR1 (the one at the Y end) is fine, as is its cable and the GUR2 analog channels inside the interface box.

OTOH, the GUR1 DAQ channels (which have GUR2 (EX) connected into it) are too small by a factor of ~1000. Seems like that end of the cable will need to be remade. Luckily Jenne is still around this week and can point us to the pinout / instructions. Looks like there could be some shorting inside the backshell, so I've left it disconnected rather than risk damaging the seismometer. We should get a GUR1 style backshell to remake this cable. It might also be possible that the end at the seismometer is bad - Steve was supposed to swap the screws on the granite-aluminum plate on Thursday; I'll double check.

The Guralps were swapped.

What I did:   turned DC power off at 1X1,  hand carried them,  centered  each leveling bubbles, gently locked the jack screws and turned power back on.

ETMY at the east end now has CMG-T40-0008, sn T4157 as channel C1:PEM-SEIS_STS_1_Y_OUT_DQ.........

ETMX at south end now has CMG-T40-0053, sn T4Q17 as channel C1:PEM-SEIS_STS_2_Y_OUT_DQ.........

Conclusion:  Guralps are fine.  X  cable is bad. It was bad 6 months ago when it was made.

We can still swap the 3ft short cables at the granite base if Rana has not done it.

 

  980   Mon Sep 22 21:30:06 2008 ranaConfigurationPSLbad FSS
The MC refl power was going up and the FSS PC drive was so large that I had to turn up the FSS
common gain from 1.5 dB to 10.5 dB to get it to be better. Attached are the before (REF) and
after plots of frequency noise. Is the FSS gain really supposed to be 1.5 dB?? How did we gain
so many dB's of optical gain? Is there a loop measurement from after Peter's oscillator change?
  982   Mon Sep 22 22:24:19 2008 ranaConfigurationPSLbad FSS

Quote:
The MC refl power was going up and the FSS PC drive was so large that I had to turn up the FSS...


Looks like I bumped the PS for the 21.5 MHz test setup and changed the supply voltage of the amplifier
from +24 to +38 V. This made the amplifier go hot after a few hours and the output eventually dropped.

Yoichi and I walked out there now and it was too hot to touch. We turned it off and put it on a heat
sink to make it chill out and it came back after a few minutes. We have set the input to the amp to
be -7 dBm instead of -8 dBm after deciding that we should take into account the 1 dB loss in the cable
run and deliver a real +17 dBm to the mixer.

The right way is to calibrated the LO mon of the FSS.
  953   Wed Sep 17 12:58:12 2008 robUpdateLockingbad

Locking was pretty unsuccessful last night. All the subparts were locked (ARMs, PRM, DRM) and
aligned, but no DRMI+2ARMs locks. The alignment may have drifted significantly by the time I
got around to working the full shebang, however.

We should get back into the habit of clicking the
yellow "Restore last auto-alignment" button when we finish using the interferometer.
  1014   Wed Oct 1 02:54:03 2008 robUpdateLockingbad

Tried the spring-y side tonight with a discouraging lack of progress. There were several locks of DRMI+2ARMs with
the +f2 (springy) sideband resonating in the DRM, but they weren't very stable. Moving to just the DRMI and resonating
the +f2, in order to tune up the acquisition and the handoff to the double demod signals, revealed the problem that the
DRM just won't stay locked on the +f2 sideband. It locks quickly, but only for a few seconds. This is different from the
behaviour with the -f2 sideband, which locks quickly and stably. In theory, the two sidebands should behave similarly.
It could be problems with HOMs in the recycling cavities, and so we may try changing the modulation frequency slightly.
  2141   Mon Oct 26 03:57:06 2009 robUpdateLockingbad

Lock acquisition has gone bad tonight. 

The initial stage works fine, up through handing off control of CARM to MCL.  However, when increasing the AO path (analog gain), there are large DC shifts in the C1:IOO-MC_F signal.  Eventually this causes the pockels cell in the FSS loop to saturate, and lock is lost. 

  2152   Tue Oct 27 18:19:14 2009 robUpdateLockingbad

Quote:

Lock acquisition has gone bad tonight. 

The initial stage works fine, up through handing off control of CARM to MCL.  However, when increasing the AO path (analog gain), there are large DC shifts in the C1:IOO-MC_F signal.  Eventually this causes the pockels cell in the FSS loop to saturate, and lock is lost. 

 This problem has disappeared.  I don't know what it was. 

The first plot shows one of the symptoms.  The second plot is a similar section taken from a more normal acquisition sequence the night before.

All is not perfect, however, as now the handoff to RF CARM is not working.

  2162   Thu Oct 29 21:51:07 2009 robUpdateLockingbad

Quote:

Quote:

Lock acquisition has gone bad tonight. 

The initial stage works fine, up through handing off control of CARM to MCL.  However, when increasing the AO path (analog gain), there are large DC shifts in the C1:IOO-MC_F signal.  Eventually this causes the pockels cell in the FSS loop to saturate, and lock is lost. 

 This problem has disappeared.  I don't know what it was. 

The first plot shows one of the symptoms.  The second plot is a similar section taken from a more normal acquisition sequence the night before.

All is not perfect, however, as now the handoff to RF CARM is not working.

 

The problem has returned.  I still don't know what it is, but it's making me angry. 

  77   Wed Nov 7 10:55:21 2007 ajwConfigurationComputersbackup script restarted
Following the reboot of computers on 10/31/07, the backup script required restart (which unfortunately "can't" be automated because a password needs to be typed in). I restarted, following the instructions in /cvs/cds/caltech/scripts/backup/000README.txt and verified that it more-or-less worked last night (the rsync sometimes times out; it gets through after a couple of days of trying.)
  1854   Fri Aug 7 13:42:12 2009 ajwOmnistructureComputersbackup of frames restored

Ever since July 22, the backup script that runs on fb40m has failed to ssh to ldas-cit.ligo.caltech.edu to back up our trend frames and /cvs/cds.

This was a new failure mode which the scripts didn't catch, so I only noticed it when fb40m was rebooted a couple of days ago.

Alex fixed the problem (RAID array was configured with the wrong IP address, conflicting with the outside world), and I modified the script ( /cvs/cds/caltech/scripts/backup/rsync.backup ) to handle the new directory structure Alex made.

Now the backup is current and the automated script should keep it so, at least until the next time fb40m is rebooted...

 

  453   Sat Apr 26 11:21:15 2008 ajwOmnistructureComputersbackup of /cvs/cds restarted
The backup of /cvs/cds (which runs as a cron job on fb40m; see /cvs/cds/caltech/scripts/backup/000README.txt) 
has been down since fb40m was rebooted on March 3.
I was unable to start it because of conflicting ssh keys in /home/controls/.ssh .
With help from Dan Kozak, we got it to work with both sets of keys
( id_rsa, which allows one to ssh between computers in our 113 network without typing a password,
 and backup2PB which allows the cron job to push the backup files to the archive in Powell-Booth).

It still goes down every time one reboots fb40m, and I don't have a solution.
A simple solution is for the script to send an email whenever it can't connect via ssh keys
(requiring a restart of ssh-agent with a passphrase), but email doesn't seem to work on fb40m.
I'll see if I can get help on how to have sendmail run on fb40m.
  8181   Wed Feb 27 11:22:54 2013 yutaUpdateComputersbackup crontab

I made a simple script to backup crontab (/opt/rtcds/caltech/c1/scripts/crontab/backupCrontab).

#!/bin/bash

crontab -l > /opt/rtcds/caltech/c1/scripts/crontab/crontab_$(hostname).$(date '+%Y%m%d%H%M%S')


I put this script into op340m crontab.

00 8 * * * /opt/rtcds/caltech/c1/scripts/crontab/backupCrontab

It took me 30 minutes to write and check this one line script. I hate shell scripts.

  11868   Wed Dec 9 19:01:45 2015 jamieUpdateCDSback to fb1

I spent this afternoon trying to debug fb1, with very little to show for it.  We're back to running from fb.

The first thing I did was to recompile EPICS from source, so that all the libraries needed by daqd were compiled for the system at hand.  I compiled epics-3.14-12-2_long from source, and installed it at /opt/rtapps/epics on local disk, not on the /opt/rtapps network mount.  I then recompiled daqd against that, and the framecpp, gds, etc from the LSCSoft packages.  So everything has been compiled for this version of the OS.  The compilation goes smoothly.

There are two things that I see while running this new daqd on fb1:

instability with mx_streams

The mx stream connection between the front ends and the daqd is flaky.  Everything will run fine for a while, the spontaneously one or all of the mx_stream processes on the front ends will die.  It appears more likely that all mx_stream processes will die at the same time.  It's unclear if this is some sort of chain reaction thing, or if something in daqd or in the network itself is causing them all to die at the same time.  It is independent of whether or not we're using multiple mx "end points" (i.e. a different one for each front end and separate receiver threads in the daqd) or just a single one (all front ends connecting to a single mx receiver thread in daqd).

Frequently daqd will recover from this.  The monit processes on the front ends restart the mx_stream processes and all will be recovered.  However occaissionally, possibly if the mx_streams do not recover fast enough (which seems to be related to how frequently the receiver threads in daqd can clear themselves), daqd will start to choke and will start spitting out the "empty blocks" messages that are harbirnger of doom:

Aborted 2 send requests due to remote peer 00:30:48:be:11:5d (c1iscex:0) disconnected
00:30:48:d6:11:17 (c1iscey:0) disconnected
mx_wait failed in rcvr eid=005, reqn=182; wait did not complete; status code is Remote endpoint is closed
disconnected from the sender on endpoint 005
mx_wait failed in rcvr eid=001, reqn=24; wait did not complete; status code is Remote endpoint is closed
disconnected from the sender on endpoint 001
[Wed Dec  9 18:40:14 2015] main profiler warning: 1 empty blocks in the buffer
[Wed Dec  9 18:40:15 2015] main profiler warning: 0 empty blocks in the buffer
[Wed Dec  9 18:40:16 2015] main profiler warning: 0 empty blocks in the buffer

My suspicion is that this time of failure is tied to the mx stream failures, so we should be looking at the mx connections and network to solve this problem.

frame writing troubles

There's possibly a separate issue associated with writing the second or minute trend files to disk.  With fair regularity daqd will die soon after it starts to write out the trend frames, producing the similar "empty blocks" messages.

  2582   Tue Feb 9 10:10:58 2010 AlbertoUpdateABSLback to analog

I want to try to do the measurement with the network analyzer used as local oscillator, instead of the Marconis that I'm using now. Tha could give me better noise rejection. It would also give me information about the phase.

Also I wouldn't dislike abandoning the GPIB interfaces to acquire data.

  5142   Mon Aug 8 15:52:39 2011 steveUpdateVACback ground rga scan

The RGA region is beeing monitored during the vent. This will tell us how clean the RGA itself is.

 

  5331   Thu Sep 1 11:03:22 2011 steveUpdateVACback ground rga scan

The RGA back ground at day 29 of this vent.

  2154   Wed Oct 28 05:02:28 2009 robUpdateLockingback

LockAcq is back on track, with the full script working well.  Measurements in progress.

  1989   Thu Sep 17 14:17:04 2009 robUpdateComputersawgtpman on c1omc failing to start

[root@c1omc controls]# /opt/gds/awgtpman -2 &
[1] 16618
[root@c1omc controls]# mmapped address is 0x55577000
32 kHz system
Spawn testpoint manager
no test point service registered
Test point manager startup failed; -1

[1]+  Exit 1                  /opt/gds/awgtpman -2

 

 

 

  1990   Thu Sep 17 15:05:47 2009 robUpdateComputersawgtpman on c1omc failing to start

Quote:

[root@c1omc controls]# /opt/gds/awgtpman -2 &
[1] 16618
[root@c1omc controls]# mmapped address is 0x55577000
32 kHz system
Spawn testpoint manager
no test point service registered
Test point manager startup failed; -1

[1]+  Exit 1                  /opt/gds/awgtpman -2

 

 

 

 

 

 

This turned out to be fallout from the /cvs/cds transition.  Remounting and restarting fixed it.

  6207   Tue Jan 17 16:09:20 2012 kiwamuUpdateCDSawg not working on the c1sus machine

Actually awg works fine without any problems when the excitation channels belong to the c1lsc machine.

It seems that the awg doesn't inject signals on the channels of the c1sus machine, for example C1:SUS-BS_LSC_EXC and so on.

Quote from #6204

AWG is not working. This needs to be fixed.

  1366   Fri Mar 6 18:14:58 2009 YoichiUpdateComputersawg not working
Starting from this afternoon, the awg is not working.
I rebooted FE computers, c0daqawg as well as tpman and daqd processes on fb40m several times.
But the problem is still there.
I sent an email to Alex.
  6204   Tue Jan 17 02:44:59 2012 kiwamuUpdateCDSawg not working

AWG is not working. This needs to be fixed.

I could set the channel and the parameters in the AWGGUI screen, but it never inject signals to the realtime system.

  16085   Mon Apr 26 18:52:52 2021 Anchal, PacoHowToComputer Scripts / Programsawg free slot

Today we had some trouble launching an excitation on C1:IOO-MC_LSC_EXC from awggui. The error read:

awgSetChannel: failed getIndexAWG C1:SUS-MC2_LSC_EXC ret=-3

What solved this was the following :

  1. launch the dtt command line interface
  2. Anchal remembers a slot number 37008
  3. We issue >> awg free 37008
  4. Slot freed, launch a new instance of awggui
  13911   Sun Jun 3 22:48:59 2018 johannesUpdatePSLaux laser replacement

I brought the NPRO from the Crackle experiment over to the 40m Lab and set it up on the PSL table to replace the slowly dying AUX laser. I also brought along a Faraday isolator, broadband EOM, and an ISOMET AOM with driver electronics from the optics storage in the Crackle Lab.

This laser is a much newer model, made in 2008, and still has all its mojo, but we should probably keep up the practice of turning it off when it's not going to be used for a while. I measured 320 mW leaving the laser, and 299mW of that going through the Faraday isolator, whose Brewster-angle polarizer I had to clean because they were a little dusty. While the laser output is going strong, the controller displays a power output of only 10 mW, which makes me think that the power monitoring PD is busted. This is a completely different failure mode from what we've seen with the other NPROs that we can hopefully get repaired at some point, particularly because the laser is newer, but for now it's installed on the PSL table. This likely means that the noise eater isn't working on this unit either, for different reasons, but at least we have plenty of optical power.

The setup is very similar to before, with the addition of a Faraday isolator and a broadband EOM, in case we decide to get more bandwidth in the PLL. I changed the Crystal Technologies 3200-113 200 MHz AOM for an ISOMET 80 MHz AOM with RF driver from the Crackle lab's optics storage and sized the AUX beam to a diameter of 200 micron. I couldn't locate an appropriate heat sink for the driver, which is still in factory condiction, but since the PSL AOM also runs on 80MHz I used that one instead. The two AOMs saturate at different RF powers, so care must be taken to not drive the AUX AOM too high. At 600 mV input to the driver the deflection into the first order was maximal at 73 % of the input power, with the second order beam and the first order on the other side cleary visible.

In order to speed things up I didn't spend too much time on mode-matching, but the advantage of the fiber setup is that we can always improve later if need be without affecting things downstream. I coupled the first order beam into the fiber to the AS table with 58% efficiency, and restored the beat with the PSL laser on the NewFocus 1611. The contrast there is only about 20%, netting a -20 dBm beat note. This is only a marginal improvement from before, so the PLL will work as usual, but if we get the visibility up a little in the future we won't need to amplify the PD signal for the PLL anymore.

Some more things I wanted to do but didn't get to today are

  • Measure intensity noise of aux laser
  • Measure rise and fall times of new AOM
  • Get PLL back up and running
  • Place 90/10 beamsplitter in AS path and couple IFO output into fiber (= couple fiber output into IFO)

I'll resume this work tomorrow. I turned the aux laser and the AOM driver input off. For the PSL beat the AOM drive is not needed, and the power in the optical fiber should not exceed 100 mW, so the offset voltage to the AOM RF driver has to remain below 300 mV.

  13912   Mon Jun 4 02:52:52 2018 johannesUpdatePSLaux laser replacement

> While the laser output is going strong, the controller displays a power output of only 10 mW, which makes me think that the power monitoring PD is busted.

NPRO internal power monitor often shows smaller value than the actual due to a broken PD or misalignment. I don't think we need to fix it.

STEVE: Aux Lightwave M126-1064-200, sn259 [July 2009] 1.76A, ADJ 9,  9mW on it's display should not mislead you. It's output  320mW

Quote:

I brought the NPRO from the Crackle experiment over to the 40m Lab and set it up on the PSL table to replace the slowly dying AUX laser. I also brought along a Faraday isolator, broadband EOM, and an ISOMET AOM with driver electronics from the optics storage in the Crackle Lab.

This laser is a much newer model, made in 2008, and still has all its mojo, but we should probably keep up the practice of turning it off when it's not going to be used for a while. I measured 320 mW leaving the laser, and 299mW of that going through the Faraday isolator, whose Brewster-angle polarizer I had to clean because they were a little dusty. While the laser output is going strong, the controller displays a power output of only 10 mW, which makes me think that the power monitoring PD is busted. This is a completely different failure mode from what we've seen with the other NPROs that we can hopefully get repaired at some point, particularly because the laser is newer, but for now it's installed on the PSL table. This likely means that the noise eater isn't working on this unit either, for different reasons, but at least we have plenty of optical power.

The setup is very similar to before, with the addition of a Faraday isolator and a broadband EOM, in case we decide to get more bandwidth in the PLL. I changed the Crystal Technologies 3200-113 200 MHz AOM for an ISOMET 80 MHz AOM with RF driver from the Crackle lab's optics storage and sized the AUX beam to a diameter of 200 micron. I couldn't locate an appropriate heat sink for the driver, which is still in factory condiction, but since the PSL AOM also runs on 80MHz I used that one instead. The two AOMs saturate at different RF powers, so care must be taken to not drive the AUX AOM too high. At 600 mV input to the driver the deflection into the first order was maximal at 73 % of the input power, with the second order beam and the first order on the other side cleary visible.

In order to speed things up I didn't spend too much time on mode-matching, but the advantage of the fiber setup is that we can always improve later if need be without affecting things downstream. I coupled the first order beam into the fiber to the AS table with 58% efficiency, and restored the beat with the PSL laser on the NewFocus 1611. The contrast there is only about 20%, netting a -20 dBm beat note. This is only a marginal improvement from before, so the PLL will work as usual, but if we get the visibility up a little in the future we won't need to amplify the PD signal for the PLL anymore.

Some more things I wanted to do but didn't get to today are

  • Measure intensity noise of aux laser
  • Measure rise and fall times of new AOM
  • Get PLL back up and running
  • Place 90/10 beamsplitter in AS path and couple IFO output into fiber (= couple fiber output into IFO)

I'll resume this work tomorrow. I turned the aux laser and the AOM driver input off. For the PSL beat the AOM drive is not needed, and the power in the optical fiber should not exceed 100 mW, so the offset voltage to the AOM RF driver has to remain below 300 mV.

 

  13913   Mon Jun 4 11:00:37 2018 gautamUpdatePSLaux laser replacement
Quote:

I couldn't locate an appropriate heat sink for the driver, which is still in factory condiction, but since the PSL AOM also runs on 80MHz I used that one instead.

We have the appropriate heatsink - I'd like to minimize interference with the main beam wherever possible.

Quote:

For the PSL beat the AOM drive is not needed, and the power in the optical fiber should not exceed 100 mW, so the offset voltage to the AOM RF driver has to remain below 300 mV.

If damage to the fiber is a concern, I think it's better to use a PBS + waveplate to attenuate the power going into the fiber. When the AOM switching is hooked up to CDS, it's easy to imagine a wrong button being pressed or a wrong value being typed in.

It would probably also be good to have a pickoff monitor for the NPRO DC power so that we can confirm its health (in the short run, we can hijack a PSL Acromag channel for this purpose, as we now do for FSS_RMTEMP). I don't know that we need an EOM for the PLL, as in order to get that going, we probably need some fast electronics for the EOM path, like an FSS box. 

STEVE: I ordered the right heatsink for the acousto after Koji pointed out that the vertical fins are 20% more efficient. Why? Because hot air rises. It will be here in 3-4 days.

ELOG V3.1.3-