40m QIL Cryo_Lab CTN SUS_Lab TCS_Lab OMC_Lab CRIME_Lab FEA ENG_Labs OptContFac Mariner WBEEShop
  40m Log, Page 335 of 344  Not logged in ELOG logo
ID Date Authordown Type Category Subject
  1194   Fri Dec 19 11:18:52 2008 AlbertoConfigurationGeneralMode Cleaner Temperature Monitor
I reduced from 10 to 5 the gain of the SR560 that Caryn has set up after the lock-in amplifier nest to the PSL rack because the overload LED was flashing.
  1244   Thu Jan 22 11:54:09 2009 AlbertoConfigurationPSLMach Zehnder Output Beam QPD
I rotated by 180 degrees the 10% beam splitter that it is used to fold the beam coming from the Mach Zehnder (directed to the MC) on to the QPD.

The alignment of the beam with that QPD has so been lost. I'll adjust it later on.

The rotation of the BS had the (surprising) effect of amplifying the Absolute Length experiment's beat by 9 times. Maybe because of a polarizing effect of the Beam Splitter which could have increased the beating efficiency between the PSL and the NPRO beams?
  1252   Sat Jan 24 11:50:24 2009 AlbertoConfigurationElectronicsPhotodiode Filters' Transfer Functions
I found an elog entry by Jenne with the measurement of the transfer functions of the filters of some of our photodetectors. Since it might turn useful to us these days, while I'm thinking about posting them on the wiki sometime, I also copy the link here:
Jenne's on the PD's TF

If we still had the data for those plots, it would be great. Do we have it?
  1263   Mon Feb 2 12:35:22 2009 AlbertoConfigurationGeneralNew Elog 2.7.5 in Service on Nodus
I moved the 40m, the Adhikari Lab and the SUS elogs from Dziban (located in Millikan's 6th floor) to our gateway server Nodus. In this way we should the complete control of it. I also updated the elog manager from the 2.6.5 version to the 2.7.5. Some smoothing of its interface might still be needed these days. We'll be testing it for a while before killing the old one. from now on everybody is invited to use only the new elog address since there will be no record of entries posted in the old one. Let me know of any possible difficulty in having access to it.
  1264   Mon Feb 2 17:25:44 2009 AlbertoConfigurationGeneralSome little problem with the new elog
For some reason the new elog does not look exactly like it should. 1) Some of the editing features are not available. 2) The Reply option opens the HTML of the message by default. I think this is happening because Nodus is a Sun of platform and things are a little bit different from linux. I'm working to fix these things. If I make any change and need to restart the elog, I'll try to be very quick. I apologize for the inconveniences.
  1265   Mon Feb 2 18:32:54 2009 AlbertoConfigurationGeneralSome little problem with the new elog

Quote:
For some reason the new elog does not look exactly like it should. 1) Some of the editing features are not available. 2) The Reply option opens the HTML of the message by default. I think this is happening because Nodus is a Sun of platform and things are a little bit different from linux. I'm working to fix these things. If I make any change and need to restart the elog, I'll try to be very quick. I apologize for the inconveniences.

 I think I solved the problem (as you can probably see).

The cause was that this WYSIWYG interface for HTML is provided by an independent text editor called FCKeditor which is included in the elog. Although the elog installer has a bug and does not unzip properly the relative package. One has to do it by hand. (going to /elog/scripts/ and unzipping fckeditor.zip by hand in the same directory).

  1266   Mon Feb 2 18:51:02 2009 AlbertoConfigurationGeneralNew Elog 2.7.5 in Service on Nodus

Quote:
I moved the 40m, the Adhikari Lab and the SUS elogs from Dziban (located in Millikan's 6th floor) to our gateway server Nodus. In this way we should the complete control of it. I also updated the elog manager from the 2.6.5 version to the 2.7.5. Some smoothing of its interface might still be needed these days. We'll be testing it for a while before killing the old one. from now on everybody is invited to use only the new elog address since there will be no record of entries posted in the old one. Let me know of any possible difficulty in having access to it.

 As a reference. The elog runs on background in nodus.

To kill the process:

1) pkill -3 elogd

2) rm -f /var/run/elogd.pid

To restart it:

elogd -p 8080 -c /export/elog/elog-2.7.5/elogd.cfg -D

  1268   Tue Feb 3 15:01:38 2009 AlbertoFrogsComputersmegatron slow?

I notice that Megatron is slower than any other computer in running code that invokes optickle or looptickle (i.e. three times slower than Ottavia). Even without the graphics.

Has anyone ever experienced that?

  1308   Mon Feb 16 10:18:13 2009 AlbertoUpdateLSCFE system rebooted

Quote:

Quote:

I didn't get a chance to do much testing since the sus controller (susvme1) went nuts. In retrospect, this could be due to something in the script, so maybe we should try a burt restore to Friday afternoon next time someone wants to look at it.


I tried the burtrestore today, it didn't work. Also tried some switching of timing cables, and multiple reboots, to no avail. This will require some more debugging. We might try diagnosing the clock driver and fanout modules, the penteks, and we can also try rebooting the whole FE system.


I rebooted the whole FE system and now c1susvme1 and c1susvme2 are back on.

I can't restart the MC autolocker on c1susvme2 because it doesn't let me ssh in. I tried to reboot it a few times but it didn't work. Once you restart it, it becomes inaccessible and doesn't even respond to pinging. Although the controls for the MC mirrors are on.

The mode cleaner stays unlocked.
  1345   Mon Mar 2 16:27:40 2009 AlbertoConfigurationElectronicsMC Drum Mode SR560 Preamplifier Replaced

Today I checked out the SR560 around the lab. I confirmed that the one with the label "channel A noisy" is effectively mulfuctioning.

It was coonected to the lock-in amplifier set up for the drum mode peak readout.

I repleaced that with a working one.

  1348   Tue Mar 3 10:39:07 2009 AlbertoUpdateLSCc1lsc discontinued functioning

The c1lsc has been unstable since last night. Its status on the DAQ screen was oscillating from green to red every minute.

Yesterday, I power recycled it. That brought it back but the MC got unclocked and the autolocker could not get engaged. I think it's because the power recycling also turned c1iscaux2 off which occupies the same rack crate.

Killing the autolocker on op340 e restarting didn't work. So I rebooted also c1dcuepis and burt-restored almost all snapshot files. To do that, as usual, I had to edit the snapshot files of c1dcuepics to move the quotes from the last line.

After that I restarted the autolocker that time it worked.

This morning c1lsc was again in the same unstable status as yesterday. This time I just reset it (no power recycling) and then I restarted it. It worked and now everything seems to be fine.

  1354   Wed Mar 4 12:38:07 2009 AlbertoUpdateComputer Scripts / Programs3f locking simulations
I simulated the REFL signals demodulated at the differential frequencies of the sidebands (f2-f1), at their summed frequencies (f2+f1). I also simulated their combination as in the Double Demodulation.
 
I repeated the simulation for:
- Old (current) 40m
- 40m Upgrade
- AdvLIGO
 
I'm attaching the results to this elog entry.
 
The plots show how the signal varies exploring the two-dimensional space of the demodulation frequencies (differential and sum).

 Both the Upgrade and the Old40m's signals look anomalous since the zero-crossing point does not change with the demodulation phases.

I suspect there's is a problem with the optickle model of the 40m.

  1373   Mon Mar 9 11:09:33 2009 AlbertoUpdateComputersRe: Not even data retrieval working

Quote:
Although getting the regular DAQ data works, we can't get any testpoints.

I tried restarting tpman several times; there's no inittab on fb40m for this so we should get Alex to set one up when he comes.
I also tried various power cycles and reboots: daqawg, daqctrl, etc. I also notice that Osamu's setup of new stuff is connected to
the same rack and power strips as all of our sensitive DAQ machines. We should find out if there was any hardware installed in the
last couple days; it would be easy to accidentally unplug or damage on of our fibers.

I moved the old tpman.log over to tpman.log.090308. It starts out with a header and then just lists when each TP is requested.

When restarting tpman it puts the following into the terminal:
fb:controls>./tpman &
[1] 1037
fb:controls>VMIC RFM 5565 (0) found, mapped at 0x2868c90
VMIC RFM 5579 (1) found, mapped at 0x2868c90
Could not open 5565 reflective memory in /dev/daqd-rfm1
16 kHz system
Spawn testpoint manager
Channel list length for node 0 is 4168
Test point manager (31001001 / 1): node 0
which is OK?; its the same startup outputs that are in the old log file. It would be nice if there was not and error message about the RFM.
Requesting new testpoints via tdsdata, dtt, or the diag command line doesn't seem to work. tpman doesn't spit anything out although 'tp show 0'
does show that the TP is selected.

Once Alex fixes the 'tpman' issue, we should make sure to put an inittab or startup script in there so that tpman writes a log
file and also archives its old log files upon a restart.


Alex fixed the problem. It was caused by the awgtpman running on kami1.martian which conflicted with the tpman in fb0.

Killing awgtpman on kami1 allowed for the tpman on tp0 to work properly again.

If more test points are needed, Alex suggested to tune the GDS settings accordingly.
What this actually means, I still have to understand it.
  1377   Mon Mar 9 17:11:38 2009 AlbertoConfigurationoplevsoptical levers centering

Yoichi, Alberto

this afternoon we centered the optical levers for all the optics.

To do that we first ran the alignment scripts for all the cavities.

  1421   Tue Mar 24 13:10:07 2009 AlbertoConfigurationPSLnew mirror installed on the AP table

New flipping mirror installed on the AP table on the beam path to the REFL199 PD.

If you're missing the double demod signal, please check that it is actually down.

  1479   Mon Apr 13 18:57:03 2009 AlbertoFrogsComputersGPIB/ETH Interface Troubles

I really don't understand why my programs that I used to use to get data from the HP Spectrum Analyzer and the Marconi frequency generator don't work anymore.

I spent hours trying to debug the code but I can't sort the problem out.

The main problem seem to be with the function recv from the socket library. Somehow it can't anymore get any data from the instruments. The thing I can't understand, though, is that if called directly from the python terminal it works fine!

In particular the problem is with the following lines in my code:

netSock.send("mkpk;mka?\n")
netSock.send("++read eoi\n")
tmp = netSock.recv(1024)

Tried a lot of tickering but it didn't work.

I attach the two scripts I've been using. One (sweepfrequencyPRC.py) calls the other (HP4395PRC.py).

They worked egregiously for weeks in the past. Don't know what happened since then.

  1481   Tue Apr 14 12:10:11 2009 AlbertoFrogsComputersGPIB/ETH Interface Troubles

Quote:

I really don't understand why my programs that I used to use to get data from the HP Spectrum Analyzer and the Marconi frequency generator don't work anymore.

I spent hours trying to debug the code but I can't sort the problem out.

The main problem seem to be with the function recv from the socket library. Somehow it can't anymore get any data from the instruments. The thing I can't understand, though, is that if called directly from the python terminal it works fine!

In particular the problem is with the following lines in my code:

netSock.send("mkpk;mka?\n")
netSock.send("++read eoi\n")
tmp = netSock.recv(1024)

Tried a lot of tickering but it didn't work.

I attach the two scripts I've been using. One (sweepfrequencyPRC.py) calls the other (HP4395PRC.py).

They worked egregiously for weeks in the past. Don't know what happened since then.

This morning Joe looked at my code and made me notice that for some reason the query to the Spectrum Analyzer made by netSock.recv(1024) contained two answers. It was like the buffer contained the answer two different queries.

After some experiment I found that basically the GPIB interface wasn't switching from the "auto 1" to the "auto 0" mode as it should. I rewrote part of the code and that seemed have solved the problem.

Still don't understand why it used to work in the past and then it stopped.

  1490   Thu Apr 16 16:37:42 2009 AlbertoUpdateAuxiliary lockingthe zipper

It takes 18 months to double the computational power of microprocessors but it took man thousands of years to invent the zipper. I never really understood that till these days.

Here is a sample of my latest results from Optickle simulations of the locking signal for the Power Recycling Cavity.

Thanks also to Rob's revolutionary bidimensional rotating matrix idea (I can see entire books of linear algebra going to be rewritten now because of that) I could find the way to determine the optimal demodulation phases for the demod signals.

There were also an other couple of missing details. But that came easily along.

The parfor function for the parallel computation in Matlab sped up some loops by a factor of 100.

 

In these particular plots there's still no CARM offset scan. That's what I'm going to post next on the elog, together with the signals for the other degrees of freedom.

  1491   Thu Apr 16 17:19:44 2009 AlbertoUpdateAuxiliary lockingthe zipper

Quote:

It takes 18 months to double the computational power of microprocessors but it took man thousands of years to invent the zipper. I never really understood that till these days.

Here is a sample of my latest results from Optickle simulations of the locking signal for the Power Recycling Cavity.

Thanks also to Rob's revolutionary bidimensional rotating matrix idea (I can see entire books of linear algebra going to be rewritten now because of that) I could find the way to determine the optimal demodulation phases for the demod signals.

There were also an other couple of missing details. But that came easily along.

The parfor function for the parallel computation in Matlab sped up some loops by a factor of 100.

 

In these particular plots there's still no CARM offset scan. That's what I'm going to post next on the elog, together with the signals for the other degrees of freedom.

 Just to show that I'm confident I'm getting reasonable results, I'll post two PRC scans for different CARM. One set of plots is for the current 40m with -19.78 deg of SRM detuning phase, the other is for the Old Upgrade (9 Mhz vs the 11 currently planned) with no detuning phase.

I'm going to put together the results and get some conclusion about the 3f locking scheme for the current 40m and the upgrade.

  1538   Fri May 1 18:24:36 2009 AlbertoSummaryGeneraljitter of REFL beam ?
Some loud thinking.
 
For the measurement of the length of the PRC, I installed a fast photodiode in the path of the beam reflected by PRM which goes to the 199 PD on the AS table. I picked up the beam by a flipping mirror on the same table.
I have the problem that the DC power that I measure at the PD when the PRC is locked is not constant but fluctuates. This fluctuation is irregular and has a frequency of about one Hz or so. I.e. it makes the 33 Mhz line on the PD oscillate by +/- 10 dBm.
 
Since this fluctuation does not appear at the REFL 33 PD, which has a much larger surface, but also shows up on the REFL 199 PD, I suspected that it was due to the very small size of the fast PDs. If the spot is too large, I thought, the power on the PD should be affected by the beam jitter.
 
Trying to avoid any beam jitter, I placed two lenses with focal lengths one ten times the other on the optical path in such a way to reduce the spot size on my fast PD by the same factors. The DC power was still fluctuating, so I'm not sure it's beam jitter anymore.
 
SPOB is definitely not constant when the PRC is normally locked, even with high loop gains, so maybe the reflected power really fluctuates that much.
Although, if it's actually the DC power that is fluctuating, shouldn't it appear also at the REFL 33 and shouldn't it be a problem that it shows up also in REFL 199? The elog doesn't say anything about that.
 

It's crucial that I get a stable transmitted power to have an accurate measurement of the PRC transmissivity and thus of its macroscopic length.

  1539   Fri May 1 18:51:34 2009 AlbertoSummaryEnvironmentearthquake

Earthquake 4.4 Leo Carrillo Beach.

Some of the watchdogs tripped out.

  1543   Mon May 4 16:49:56 2009 AlbertoUpdateMOPAlaser power is dropped

Quote:

As PSL-126MOPA_DTEC went up, the power out put went down yesterday

Alberto, Jenne, Rob, Steve,
 
later on in the afternoon, we realized that the power from the MOPA was not recovering and we decided to hack the chiller's pipe that cools the box.
 
Without unlocking the safety nut on the water valve inside the box, Jenne performed some Voodoo and twisted a bit the screw that opens it with a screw driver. All the sudden some devilish bubbling was heard coming from the pipes.
The exorcism must have freed some Sumerian ghost stuck in our MOPA's chilling pipes (we have strong reasons to believe it might have looked like this) because then the NPRO's radiator started getting cooler.
I also jiggled a bit with the valve while I was trying to unlock the safety nut, but I stopped when I noticed that the nut was stuck to the plastic support it is mounted on.
 
We're now watching the MOPA power's monitor to see if eventually all the tinkering succeeded.

 

[From Jenne:  When we first opened up the MOPA box, the NPRO's cooling fins were HOT.  This is a clear sign of something badbadbad.  They should be COLD to the touch (cooler than room temp).  After jiggling the needle valve, and hearing the water-rushing sounds, the NPRO radiator fins started getting cooler.  After ~10min or so, they were once again cool to the touch.  Good news.  It was a little worrisome however that just after our needle-valve machinations, the DTEC was going down (good), but the HTEMP started to rise again (bad).  It wasn't until after Alberto's tinkering that the HTEMP actually started to go down, and the power started to go up.  This is probably a lot to do with the fact that these temperature things have a fairly long time constant. 

Also, when we first went out to check on things, there was a lot more condensation on the water tubes/connections than I have seen before.  On the outside of the MOPA box, at the metal connectors where the water pipes are connected to the box, there was actually a little puddle, ~1cm diameter, of water. Steve didn't seem concerned, and we dried it off.  It's probably just more humid than usual today, but it might be something to check up on later.]

  1556   Thu May 7 17:59:23 2009 AlbertoConfiguration MC WFS
This afternoon the MC could not get locked.
I first checked the Osems values at the MC mirrors and compared them to the trend of the last few hours. That showed that the alignment of the mirrors had slightly changed. I then brought each mirror back to its old alignment state.
 
That let the LSC loop lock the MC, although the reflected power was still high (1.5V) and the WFS control wouldn't engage.
 
Since earlier during the day I was working on the AS table, it is possible that I inadvertently hit the MC REFL beam splitter misaligning the beam to the MC WFS.
To exclude that there was a problem in the suspensions, before touching the WFS, I checked that the cables at the MC's ends and those going to the ADC in the rack were well pushed in.
 
Then I proceeded in centering the beam on both the WFS, balancing the power over the QPDs.
 
In the end the MC could lock again properly.
 
  1636   Mon Jun 1 13:56:52 2009 AlbertoUpdatePSLLaser Power after fixing the laser chiller

The laser power seems to have become more stable after fixing the laser chiller. The power is lower than it used to be (MOPA amplitude 2.5 versus 2.7) but, as shown in the attchement, it became more steady.

  1644   Tue Jun 2 23:55:45 2009 AlbertoUpdateoplevsoplevs centerd

Tonight I centered the oplevs for ITMX/Y, SRM, PRM, BS.

After doing that I noticed that the BS drifted a little from where I had set it.

  1661   Mon Jun 8 18:22:27 2009 AlbertoDAQLSCAdded PD11 I amd Q slow channels

I just added two slow channels to C0EDCUEPICS to monitor the input of PD11. The names are:

[C1:LSC-PD11_I_INMON]
[C1:LSC-PD11_Q_INMON]

  1664   Wed Jun 10 01:52:34 2009 AlbertoUpdateElectronicsMC length and Marconis' frequencies

Pete, Rob, Alberto,

yesterday we thought that some of the problems we were having in locking the IFO might be related to a change of the length of the mode cleaner. So today we decided to measure it again.

We followed the Sigg-Frolov technique (see 40m Wiki, Waldman, Fricke). For the record, the MC_AO input corresponds to IN2 on the MC Servo board.

We obtained: L = 27.092 +/- 0.001 m

From the new measurement we reset the frequencies of the Marconis to the following values:

33196450 Hz

132785800 Hz

165982250 Hz

199178700 Hz

 

  1667   Thu Jun 11 03:15:15 2009 AlbertoUpdateLSCDD Handoff attempts

Pete, Alberto,

tonight we worked on the tuning of the double demod phases for the handoff of the short DOFs control signals.

Only MICH can now undergo the handoff. PRC can't make it.

Basically, we tuned the PD6 demod phase and reduced the offset in PD6_I. Then we tuned the relative gain of PD6_I and PD2_I so that the two open loop transfer function of the control loops would match. We tried that in several ways and several times but without success.

I guess we're missing to do/check something.

  1669   Thu Jun 11 22:14:10 2009 AlbertoUpdateLSCDD Handoff for the Short DOFs completed

This afternoon I tuned the handoff script for the SRC, after that Rob eralier during the day had already adjusted that for PRC. To do that, I followed the procedure in the Wiki.

  • I measured the OL transfer function of the single demod path and of the double demod path and tuned thier gains so that they matched
  • I tuned the double demod pahses of PD_7 and PD_8 in order to reduce the offset in the PD_x_I signals

After that the SRC could get locked with the double demod signals. the open loop transfer function emasurement on the PRC loop showed that it was nearly unstable. Rob reduced a little its gain to improve the stability.

The DD handoff is now working and we can get back to locking the interferometer.

  1681   Tue Jun 16 20:03:41 2009 AlbertoUpdateElectronicsRequirements on Wenzel Multiplier

For the 40m Upgrade, we plan to eliminate the Mach-Zehnder and replace it with a single EOM driven by all three modulation frequencies that we'll need: f1=11MHz, f2=5*f1=55MHz, fmc=29.5MHz.

A frequency generator will produce the three frequencies and with some other electronics we'll properly combine and feed them to the EOM.

The frequency generator will have two crystals to produce the f1 and fmc signals. The f2 modulation will be obtained by a frequency multiplier (5x) from the f1.

The frequency multiplier, for the way it works, will inevitably introduce some unwanted harmonics into the signals. These will show up as extra modulation frequencies in the EOM.

In order to quantify the effects of such unwanted harmonics on the interferometer and thus to let us set some limits on their amplitude, I ran some simulations with Optickle. The way the EOM is represented is by three RF modulators in series. In order to introduce the unwanted harmonics, I just added an RF modulator in series for each of them. I also made sure not to leave any space in between the modulators, so not to introduce phase shifts.

To check the effect at DC I looked at the sensing matrix and at the error signals. I considered the 3f error signals that we plan to use for the short DOFs and looked at how they depend on the CARM offset. I repeated the simulations for several possible amplitude of the unwanted harmonics. Some results are shown in the plots attached to this entry. 'ga' is the amplitude ratio of the unwanted
harmonics relative to the amplitude of the 11 & 55 MHz modulations.

Comparing to the case where there are no unwanted harmonics (ga = 0), one can see that not considerable effect on the error signals for amplitudes 40dB smaller than that of the main sidebands. Above that value, the REFL31I signals, that we're going to use to control PRCL, will start to be distorted: gain and linearity range change.

So 40 dB of attenuation in the unwanted harmonics is probably the minimum requirement on the frequency multiplier, although 60dB would provide a safer margin.

I'm still thinking how to evaluate any AC effect on the IFO.

 

** TODO: Plot DC sweeps with a wider range (+/- 20 pm). Also plot swept sines to look for changes in TFs out to ~10 kHz.

  1686   Fri Jun 19 13:38:42 2009 AlbertoConfigurationComputerselog rebooted

Today I found the elog down, so I rebooted it following the instructions in the wiki.

  1688   Fri Jun 19 14:30:47 2009 AlbertoConfigurationComputerselog rebooted

Quote:

Today I found the elog down, so I rebooted it following the instructions in the wiki.

 I have the impression that Nodus has been rebooted since last night, hasn't it?

  1709   Tue Jun 30 23:09:40 2009 AlbertoUpdateLockingchronicles of some locking attempts

Tonight I tried to lock the interferometer. At the first attempts the arm power didn't go above about 4. The mode cleaner seemed to be not well aligned and it lost lock or got stuck on a wrong mode. I had to run the MC_UP and MC_DOWN scripts to lock it again.

After that the locking proceed more smoothly; at least till a power level in the arms of about 60. Then again the mode cleaner lost lock and I had to run the scripts again. Without the MCWFS servo off the MC reflected power is still rather high (about 1.7); also even when the WFS servo is engaged the reflected power is about 0.5, versus 0.3 that it should be.

Those are both signs of a not very good alignment. Tomorrow I'll have to work on the injection periscope on the PSL table to try to fix that.

  1722   Wed Jul 8 11:13:36 2009 AlbertoOmnistructureComputerswireless router disconnected

Once again, this morning I found the wireless router disconnected from the LAN cable. No martian WiFi was available.

I wonder who is been doing that and for what reason.

  1725   Wed Jul 8 19:13:19 2009 AlbertoUpdatePSLPSL beam aligned to the Mode Cleaner

Today I tuned the periscope on the PSL table to align the beam to the Mode Cleaner. With the Wave Front Sensor control off, I minimized the reflection from the MC and maximized the transmission. While doing that I also checked that the transmitted beam after the MC didn't lose the alignment with the interferometer's main Faraday isolator.

In this way, I've got a reflection, as read from the MC_REFLPD_MC, of about 0.6. Then I centered the WFS on the AS table. After that the WFS alignment control brought the reflection to 0.25 and a nice centered bull-eye spot showed on the monitor.

  1735   Mon Jul 13 00:34:37 2009 AlbertoDAQComputersAll computers down

Quote:

I popped by the 40m, and was dismayed to find that all of the front end computers are red (only framebuilder, DAQcontroler, PEMdcu, and c1susvmw1 are green....all the rest are RED).

 

I keyed the crates, and did the telnet.....startup.cmd business on them, and on c1asc I also pushed the little reset button on the physical computer and tried the telnet....startup.cmd stuff again.  Utter failure. 

 

I have to pick someone up from the airport, but I'll be back in an hour or two to see what more I can do.

 I think the problem was caused by a failure of the RFM network: the RFM MEDM screen showed frozen values even when I was power recycling any of the FE computers. So I tried the following things:

- resetting the RFM switch
- power cycling the FE computers
- rebooting the framebuilder
 
but none of them worked.  The FEs didn't come back. Then I reset C1DCU1 and power cycled C1DAQCTRL.
 
After that, I could restart the FEs by power recycling them again. They all came up again except for C1DAQADW. Neither the remote reboot or the power cycling could bring it up.
 
After every attempt of restarting it its lights on the DAQ MEDM  screen turned green only for a fraction of a second and then became red again.
 
So far every attempt to reanimate it failed.
  1736   Mon Jul 13 00:53:50 2009 AlbertoDAQComputersAll computers down

Quote:

Quote:

I popped by the 40m, and was dismayed to find that all of the front end computers are red (only framebuilder, DAQcontroler, PEMdcu, and c1susvmw1 are green....all the rest are RED).

 

I keyed the crates, and did the telnet.....startup.cmd business on them, and on c1asc I also pushed the little reset button on the physical computer and tried the telnet....startup.cmd stuff again.  Utter failure. 

 

I have to pick someone up from the airport, but I'll be back in an hour or two to see what more I can do.

 I think the problem was caused by a failure of the RFM network: the RFM MEDM screen showed frozen values even when I was power recycling any of the FE computers. So I tried the following things:

- resetting the RFM switch
- power cycling the FE computers
- rebooting the framebuilder
 
but none of them worked.  The FEs didn't come back. Then I reset C1DCU1 and power cycled C1DAQCTRL.
 
After that, I could restart the FEs by power recycling them again. They all came up again except for C1DAQADW. Neither the remote reboot or the power cycling could bring it up.
 
After every attempt of restarting it its lights on the DAQ MEDM  screen turned green only for a fraction of a second and then became red again.
 
So far every attempt to reanimate it failed.

 

After Alberto's bootfest which was more successful than mine, I tried powercycling the AWG crate one more time.  No success.  Just as Alberto had gotten, I got the DAQ screen's AWG lights to flash green, then go back to red.  At Alberto's suggestion, I also gave the physical reset button another try.  Another round of flash-green-back-red ensued.

When I was in a few hours ago while everything was hosed, all the other computer's 'lights' on the DAQ screen were solid red, but the two AWG lights were flashing between green and red, even though I was power cycling the other computers, not touching the AWG at the time.  Those are the lights which are now solid red, except for a quick flash of green right after a reboot.

I poked around in the history of the curren and old elogs, and haven't found anything referring to this crazy blinking between good and bad-ness for the AWG computers.  I don't know if this happens when the tpman goes funky (which is referred to a lot in the annals of the elog in the same entries as the AWG needing rebooting) and no one mentions it, or if this is a new problem.  Alberto and I have decided to get Alex/someone involved in this, because we've exhausted our ideas. 

  1737   Mon Jul 13 15:14:57 2009 AlbertoUpdateComputersDAQAWG

Today Alex came over, performed his magic rituals on the DAQAWG computer and fixed it. Now it's up and running again.

I asked him what he did, but he's not sure of what fixed it. He couldn't remember exactly but he said that he poked around, did something somewhere somehow, maybe he tinkered with tpman and eventually the computer went up again.

Now everything is fine.

  1742   Tue Jul 14 00:57:11 2009 AlbertoUpdateLockingphotodiode alignment check

Since lately the alignment of the input beam to the interferometer has changed, I went checking the alignment of the beam on the photodiodea. They were all fine except for pd9, that is AS DD 199. Here the DC is totally null. The beam seems to go right on the diode but the scope on the PD's DC output shows no power. This is really strange and bad.

  1749   Wed Jul 15 12:14:08 2009 AlbertoUpdateLockingphotodiode alignment check

Quote:

Since lately the alignment of the input beam to the interferometer has changed, I went checking the alignment of the beam on the photodiodea. They were all fine except for pd9, that is AS DD 199. Here the DC is totally null. The beam seems to go right on the diode but the scope on the PD's DC output shows no power. This is really strange and bad.

After inspecting PD9 with the viewer and the cards, the beam looks like it is aligned to the photodiode althought there is no signal at the DC output of the photodetector. So I checked the spectrum for PD9_i and Q (see attachments) and it seems that those channels are actually seeing the beam. I'm going to check the alignemtn again and see the efefct on the spectra to make sure that the beam is really hitting the PD.

 

  1755   Thu Jul 16 01:00:56 2009 AlbertoUpdateLockingPD9 aligned

Quote:

Quote:

Since lately the alignment of the input beam to the interferometer has changed, I went checking the alignment of the beam on the photodiodea. They were all fine except for pd9, that is AS DD 199. Here the DC is totally null. The beam seems to go right on the diode but the scope on the PD's DC output shows no power. This is really strange and bad.

After inspecting PD9 with the viewer and the cards, the beam looks like it is aligned to the photodiode althought there is no signal at the DC output of the photodetector. So I checked the spectrum for PD9_i and Q (see attachments) and it seems that those channels are actually seeing the beam. I'm going to check the alignemtn again and see the efefct on the spectra to make sure that the beam is really hitting the PD.

 

 I aligned PD9. Here are the spectra confirming that.

p.s.
Ants, theyre everywhere, even inside the AS table. They're taking over the lab, save yourself!
  1772   Wed Jul 22 01:57:19 2009 AlbertoConfigurationComputerscomputers are down

Quote:

All suspentions are kicked up. Sus dampings and  oplev servos turned off.

c1iscey and c1lsc are down. c1asc and c1iovme are up-down

 The computers and RFM network are up working again. A boot fest was necessary.Then I restored all the parameters with burtgooey.

The mode cleaner alignment is in a bad state. The autolocker can't get it locked. I don't know what caused it to move so far from the good state that it was till this afternoon.  I went tuning the periscope but the cavity alignment is so bad that it's taking more time than expected. I'll continue working on that tomorrow morning.

  1773   Wed Jul 22 09:04:10 2009 AlbertoConfigurationComputerscomputers are down

Quote:

Quote:

All suspentions are kicked up. Sus dampings and  oplev servos turned off.

c1iscey and c1lsc are down. c1asc and c1iovme are up-down

 The computers and RFM network are up working again. A boot fest was necessary.Then I restored all the parameters with burtgooey.

The mode cleaner alignment is in a bad state. The autolocker can't get it locked. I don't know what caused it to move so far from the good state that it was till this afternoon.  I went tuning the periscope but the cavity alignment is so bad that it's taking more time than expected. I'll continue working on that tomorrow morning.

 I now suspect that after the reboot the MC mirrors didn't really go back to their original place even if the MC sliders were on the same positions as before.

  1774   Wed Jul 22 10:49:40 2009 AlbertoUpdatePSLMC Alignment Trend

Trying to track the MC positions back for a few days, it seems that the data hasn't been recorded properly for a while. Something happened yesterday after my boot fest and then the record got restored. Attached here are the readbacks showing the event for MC1.

Is anything wrong with the data record?

  1776   Wed Jul 22 11:14:11 2009 AlbertoConfigurationComputerscomputers are down

Quote:

Quote:

Quote:

All suspentions are kicked up. Sus dampings and  oplev servos turned off.

c1iscey and c1lsc are down. c1asc and c1iovme are up-down

 The computers and RFM network are up working again. A boot fest was necessary.Then I restored all the parameters with burtgooey.

The mode cleaner alignment is in a bad state. The autolocker can't get it locked. I don't know what caused it to move so far from the good state that it was till this afternoon.  I went tuning the periscope but the cavity alignment is so bad that it's taking more time than expected. I'll continue working on that tomorrow morning.

 I now suspect that after the reboot the MC mirrors didn't really go back to their original place even if the MC sliders were on the same positions as before.

 Alberto, Rob,

we diagnosed the problem. It was related with sticky sliders. After a reboot of C1:IOO the actual output of the DAC does not correspond anymore to the values read on the sliders. In order to update the actual output it is necessary to do a change of the values of the sliders, i.e. jiggling a bit with them.

  1783   Thu Jul 23 10:05:38 2009 AlbertoUpdatePSLSummary of the latest adventures with the alignment of the mode cleaner

Alberto, Koji,

Summary of the latest adventures with the alignment of the mode cleaner

Prior events.

  • Last week, on July 12th the RFM network crashed (elog entry 1736). I don't know for sure which one was the cause and which one the effect, but also C1DAQADW was down and it didn't want to restart. Alex fixed it the day after.
  • On the evening of Sunday July 20th I noticed that the mode cleaner was unlocked. A closer inspection showed me that MCL was frozen at -32768 counts. To fix that I rebooted C1DCUEPICS and burtrestored to snapshots from the day before.
  • On Tuesday July 21st another failure of the RFM Network made necessary a reboot of the frame builder and of all front end computers (entry 1772). As a consequence, the mode cleaner couldn't get locked anymore, even if the mirror's sliders in the MC-Align MEDM screen were in the proper positions. At that time I missed to check the MC suspension positions as a way to ensure that the MC hadn't really changed. Although later, as it turned out, that would have been useless anyway since all the data record prior to the computers crash of that day somehow had been corrupted (entry 1774). Neither the MC2 LSC control or MC ASC control could engage so I (erroneously) thought that some tune of the periscope might help. So I did but, since the Mode Cleaner was misaligned, that had the effect of spoiling the good matching of the periscope to the MC cavity.
  • Yesterday, Wednesday July 22nd, I found out about the sticky slider effect (entry 1776). At that point we didn't have anymore a way to know that the MC optics were actually in their proper original alignment state because of the lack of a reference for those in the data record (as I wrote above). I had to go back to the periscope and fix the alignment.


Chronicles of periscope and MC alignment

Yesterday morning I started aligning the periscope but it turned out to be trickier than usual. With the ASC (Alignment Sensing Control) off and only the length controls on, the Mode Cleaner didn't lock easily, although I knew I wasn't very far from the sweet spot.

In the afternoon the struggle continued and the matching of the the beam to the MC cavity became just worse. At some point I noticed that the ASC inputs somehow had got on - although the ASC still looked disabled from the MClock MEDM main screen. So I was actually working against the Wave Front Sensors and further worsening the periscope alignment.

That hurled me to the weeds. After hours of rowing across the stormy waters of a four-dimensional universe I got to have occasional TEM00 flashes at the transmission but still, surprisingly, no MC locking. Confused, I kept tuning the periscope but that just kicked me off road again.

Then at about 7pm Koji came to my rescue and suggested a more clever and systematic way to solve the problems. He suggested to keep record of the MC mirrors alignment state and re-align the cavity to the periscope. Then we would gradually bring the cavity back to the original good position changing the periscope alignment
at the same time.

 

That would have worked straight away, if we hadn't been fighting against a subtle and cruel enemy: the 40m computer network. But I (as John Connor), and Koji (as the Terminator) didn't pull back.

Here's a short list of the kinds of weapons that the computers threw to us:

  1. After a while the FSS entered a funny state. It showed transmission: we had light at the MC (and even flashes) but the MEDM readout of the FSS transmitted power after the cavity was low (~0.019). Also the spot on the monitor showed a slightly different pattern from how I remembered it. On the other side the transmission camera didn't show that typical halo as usual.
  2. MCL was frozen at 32768. I ran the MCDown and MCUp script a couple of times and that unstuck it.
  3. On op340m we found that the MC autolocker script wasn't running. So I restarted it. Still nothing changed: bright and sharp flashes appeared on the monitor (sign of a not too bad alignment) but no lock.
  4. I rebooted C1IOO. No change.
  5. I rebooted C1DCUEPICS and burtrestored the EPICS computers to Jul 19th. No change.
  6. Then I burtrestored the c1psl.snapshot and that finally did something. The FSS reflected spot changed and the halo appeared again at its transmission camera. Soon after the MC got locked.


We then proceeded with Koji's plan. In an iterative process, we aligned the MC cavity maximizing the transmission and tuned the periscope in order to match the Faraday input of the interferometer. The last thing we did it by looking at the camera pointing at the Faraday isolator.

We found that we didn't have to tune the periscope much. That means that all afternoon I didn't really go too far, but the autolocker wasn't working properly, or it wasn't working at all.

Then we ran the alignment script for the X arm but it didn't work before we aligned the steering mirrors.

Then we ran it three times but could not get more than 0.87 at TRX. That means that there we still have to work on the alignment to the Faraday. That's job for today in the trenches of the lab.

 

  1784   Thu Jul 23 20:30:23 2009 AlbertoUpdatePSLBeam aligned to the Farady

After yesterday's changes in the MC cavity state today it was necessary to optimize the alignment to the Faraday.

The way I did it was by tuning the PSL periscope in pitch and yaw trying to maximize TRX with the arm locked. After a small change in either one of the two directions I first maximized the MC transmitted power and then I ran the alignment script for the X arm.

I explored the space for both pitch and yaw and the max that I could get from TRX was 0.91. I'm not sure whether the increase in TRX is entirely due to a better alignment to the Farady rather than to a higher MC transmitted power.

Also I'm not sure I'm well interpreting the image from the camera pointing at the Farady. I guess I need someone more familiar with it to tell me if it shows any sign of clipping.

Anyway, last week, even before the MC got misaligned, TRX didn't go above 0.90. So now I wonder whether it's the MC's fault or something else's if we have that value..

  1785   Fri Jul 24 11:04:11 2009 AlbertoConfigurationComputerselog restarted

This morning I found the elog down. I restarted it using the procedure in the wiki.

  1788   Fri Jul 24 21:02:46 2009 AlbertoUpdatePSLAligning the beam to the Faraday

This afternoon I kept working on the alignment of the beam so that it matches at the same the PSL periscope, the Mode Cleaner and the Faraday isolator at the input of the IFO.

The camera looking at the Farady showed a beam quite low from the center of the Faraday's entrance. I wanted to move it up.

After working on the periscope alignment and on the MC mirrors, I think I managed to moved it up a bit. To know whether that was enough or not I wanted to evaluate the alignment to the X arm by checking the value of TRX.

In order for the MC to be finely matched to the input beam from the periscope, the WFS controls have to be on. Before turning them on, I centered the beam on their QPDs and run the WFS_zero_offset script.

When I turned them on, the control signal in Pitch from WFS2 started going up with no stop. It was like the integrator in the loop was fed with a DC bias. The effect of that was to misalign the MC cavity from the good state in which it was with the only length control on (that is, transmission ~2.7, reflection ~ 0.4).

I don't know why that is happening. To exclude that it was due to a computer problem I first burtrestored C1IOO to July the 18th, but since that did not help, I even restarted it. Also that didn't solve the problem.

 

Flashes at ETMX show at least that the beam is going through the Farady. How well, I can't tell untill the MC is under full control.

 

I have to leave the lab now, but I can be back tomorrow to keep working on that.

  1794   Sun Jul 26 16:05:17 2009 AlbertoUpdatePSLAligning the mode cleaner

Quote:

I set the MC back to its good alignment (June 21st) using this procedure. The trend of the OSEM values over the last 40 days and 40 nights is attached.

Then I aligned the periscope to that beam. This took some serious periscope knob action. Without WFS, the transmission went to 2.7 V and the reflection down to 0.6V.

Then I re-aligned the MC_REFL path as usual. The beam was far enough off that I had to also re-align onto the MC LSC PD as well as the MC REFL camera (~2 beam radii).

Beams are now close to their historical positions on Faraday and MC2. I then restored the PZT sliders to their April snapshot and the X-arm locked.

Steve - please recenter the iris which is on the periscope. It has been way off for a long time.

So it looks OK now. The main point here is that we can trust the MC OSEMs.

Afterwards I rebooted c1susvme1 and c1susvme2 because they were skewed.

 

 It is really surprising that we now have again the data from the MC OSEMs since up to two days ago the record looked corrupted (see the attachments in my entry 1774).

The reason I ended up severely misaligning the the MC is exactly that there wasn't anymore a reference position that I could go back to and I had to use the camera looking a the Faraday.

ELOG V3.1.3-