40m QIL Cryo_Lab CTN SUS_Lab TCS_Lab OMC_Lab CRIME_Lab FEA ENG_Labs OptContFac Mariner WBEEShop
  40m Log, Page 96 of 339  Not logged in ELOG logo
ID Date Author Typeup Category Subject
  2011   Mon Sep 28 02:24:05 2009 ranaUpdateLockingMC1/3 Dewhitening found OFF: Turned back ON

While trying to make the OAF work, I found that the XYCOM switches for MC1/3 have been set in the bad way for awhile. This means that the hardware filters were bypassed and that MC1 & MC3 were moving around too much at high frequency and possibly causing trouble with the locking. I have put them back into the default position.

On Friday, Jenne and I were playing around with turning the dewhitening off/on to see if it efffected the OAF stability. At the time, I didn't pay too much attention to what the state was. Looks like it was in the wrong state (hardware bypassed) when we found it. For the OAF work, we generally want it in that bypassed state, but its bad because it makes noise in the interferometer. The bits in question are bits 16-23 on the XYCOM screen.

I have updated the snapshot and set the screen in the appropriate settings. I used a swept sine measurement to verify the filter state. In the attached plot, green corresponds to XYCOM green and red corresponds to red.

Attachment 1: C1SUS_SRM_XYCOM1.png
C1SUS_SRM_XYCOM1.png
Attachment 2: Untitled.png
Untitled.png
  2012   Mon Sep 28 11:52:23 2009 JenneUpdateTreasureOAF screen added to the screenshots webpage

I used Kakeru's instructions in elog 1221 to add the C1OAF screen (still called C1ASS_TOP) to the medm screenshots webpage.  The tricky part of this is figuring out that the file that needs editing is in fact in /cvs/cds/projects/statScreen, not /cvs/cds/caltech/statScreen, as claimed in the entry. 

  2013   Mon Sep 28 17:39:34 2009 robUpdatePSLproblems

The PSL/IOO combo has not been behaving responsibly recently. 

The first attachment is a 15 day trend of the MZ REFL, ISS INMON, and MC REFL power.  These show two separate problems--recurring MZ flakiness, which may actually be a loose cable somewhere which makes the servo disengage.  Such disengagement is not as obvious with the MZ as it is with other systems, because the MZ is relatively stable on its own.  The second problem is more recent, just starting in the last few days.  The MC is drifting off the fringe, either in alignment, length, or both.  This is unacceptable.

The second attachment is a two-day trend of the MC REFL power.  Last night I carefully put the beam on the center of the MC-WFS quads.  This appears to have lessened the problem, but it has not eliminated it. 

It's probably worth trying to re-measure the MCWFS system to make sure the control matrix is not degenerate. 

Attachment 1: problems.png
problems.png
Attachment 2: problems2.png
problems2.png
  2017   Tue Sep 29 10:44:29 2009 KojiUpdateMZMZ investigation

Rana, Jenne, Koji

Last night we checked MZ. The apparent thing we found was the gain slider does not work.
The slider actually changes the voltage at the cross connection of 1Y2 (31 pin4?), the gain does not change.
The error spectrum didn't change at all even when the slider was moved.

Rana poked the flat cable at the bottom of 1Y2, we had no imporvement.

We coudn't find the VME extender board, so we just replaced AD602 (=VGA) and LT1125 (=Buffer for the ctrl voltage).
Even after the replacement, the gain slider is not working yet.

Today, I will put a lead or probe to the board to see whether the slider changes the voltage on the board or not.

Somehow the gain is sitting at a intermediate place that is not to low not to high. So I still don't know the gain slider
is the cause of the MZ instability or not.

  2018   Tue Sep 29 12:47:08 2009 KojiUpdateMZMZ unlocked

12:45 I started the work on MZ. Thus the MZ was unlocked.

Found the bad connection on the FLKM 64pin cross connection board. We need a replacement.

I went to Wilson and got the replacement, two VME extender boards, three 7815, and three 7915. Thanks, Ben!

  2020   Tue Sep 29 18:21:41 2009 KojiUpdateMZMZ work done

The MZ work completed. I replaced the bad cross connection terminal. The gain slider is working now.

I looked at the error spectrum on an FFT analyzer. I could see the lock was more tight.

Then I proceeded to the MZ epics panel.

1) C1:PSL-MZ_MZTRANSPD has no meaning (not connected). So I put  C1:PSL-ISS_INMONPD as the MZ trans monitor.

2) The EPICS setting for the MZ gain slider was totaly wrong.
    Today I learned from the circuit, the full scale of the gain slider C1:PSL-MZ_GAIN gave us +/-10V at the DAC.
    This yield +/-1V to V_ctrl of the AD602 after the internal 1/10 attenuation stage.
    This +/-1V didn't correspond to -10dB~+30dB, but does -22dB~+42dB and is beyond the spec of the chip.

    The gain of AD602 is calculated by

G [dB] = 32 V_crtl + 10,  for -0.625 [V]< V_ctrl < +0.625 [V].

    In order to fix this I used the following commands which overrode the EPICS parameters.
    The tip of EGUF/EGUL is to know how much the gain (virtually) goes for the full scale of the DAC output. 

ezcawrite C1:PSL-MZ_GAIN.EGUF 42
ezcawrite C1:PSL-MZ_GAIN.EGUL -22
ezcawrite C1:PSL-MZ_GAIN.DRVH 30
ezcawrite C1:PSL-MZ_GAIN.DRVL -10
ezcawrite C1:PSL-MZ_GAIN.HOPR 30
ezcawrite C1:PSL-MZ_GAIN.LOPR -10

   and for the permanent change I modified the db file /cvs/cds/caltech/target/c1iool0/c1iooMZservo.db
   This will be active when cliool0 is rebooted.

# This yields the output limited to -6.25V ~ +6.25V, which corresponds to -10dB ~ +30dB
# modified by Koji Arai (29-Sept-2009)
grecord(ao,"C1:PSL-MZ_GAIN")
{
        field(DESC,"GAIN- overall pre-modecleaner servo loop gain")
        field(SCAN,"Passive")
        field(PINI,"YES")
        field(DISV,"1")
        field(DTYP,"VMIVME-4116")
        field(OUT,"#C3 S5 @")
        field(EGUF,"42")
        field(EGUL,"-22")
        field(PREC,"1")
        field(EGU,"dB")
        field(HOPR,"30")
        field(LOPR,"-10")
        field(DRVH,"30")
        field(DRVL,"-10")
        field(LINR,"LINEAR")
        field(OROC,"0")
        field(DOL,"0")
}

# previous code
grecord(ao,"C1:PSL-MZ_GAIN")
{
        field(DESC,"GAIN- overall pre-modecleaner servo loop gain")
        field(SCAN,"Passive")
        field(PINI,"YES")
        field(DISV,"1")
        field(DTYP,"VMIVME-4116")
        field(OUT,"#C3 S5 @")
        field(EGUF,"30")
        field(EGUL,"-10")
        field(PREC,"4")
        field(EGU,"Volts")
        field(HOPR,"30")
        field(LOPR,"-10")
        field(LINR,"LINEAR")
        field(OROC,"0")
        field(DOL,"0")
}

Quote:

12:45 I started the work on MZ. Thus the MZ was unlocked.

Fond the bad connection on the FLKM 64pin cross connection board. We need the replacement.

I went to Wilson and got the replacement, two VME extender boards, three 7815, and three 7915. Thanks, Ben!

 

  2021   Tue Sep 29 21:37:09 2009 ranaUpdateMZMZ work done : some noise checking

Since we used to run with a gain slider setting of +15 dB on the MZ, I wanted to check that the new setting of +30dB was OK. It is.

To check it I turned it up and looked for some excess noise in the ISS or in the MC. There was none. I also set the input offset slider by unlocking the PMC and zeroing the mixer monitor readback. The new slider setting is -6.5V.

I don't know why we would need more gain on the MZ loop, but we can have some if we want it by turning up the gain before the servo (optical or RF). The attached plot shows the MC_F and ISS signals with the ISS loop on and off. There was no change in either of these spectra with the MZ gain high or low.

Attachment 1: fsm.pdf
fsm.pdf
  2022   Tue Sep 29 21:51:32 2009 KojiUpdateMZMZ work done : some noise checking

The previous "+15" was Vctrl = 0.25 [V]. Which was +18 dB.

Quote:

Since we used to run with a gain slider setting of +15 dB on the MZ, I wanted to check that the new setting of +30dB was OK.

 

  2023   Tue Sep 29 22:51:20 2009 KojiUpdateMZPossible gain mis-calibration at other places (Re: MZ work done)

Probably there is the same mistake for the PMC gain slider. Possibly on the FSS slider, too???

Quote:

2) The EPICS setting for the MZ gain slider was totaly wrong.
    Today I learned from the circuit, the full scale of the gain slider C1:PSL-MZ_GAIN gave us +/-10V at the DAC.
    This yield +/-1V to V_ctrl of the AD602 after the internal 1/10 attenuation stage.
    This +/-1V didn't correspond to -10dB~+30dB, but does -22dB~+42dB and is beyond the spec of the chip.

  2024   Tue Sep 29 23:43:46 2009 robUpdateSUSITMY UL OSEM

We had a redo of elog entry 975 tonight.  The noisy OSEM was fixed by jiggling the rack end of the long cable.  Don't know exactly where--I also poked around the OSEM PD interface board.

In the attached PDF the reference trace is the noisy one.

Attachment 1: ITMYosemBAD.pdf
ITMYosemBAD.pdf
  2026   Wed Sep 30 01:04:56 2009 robUpdateComputersgrief

much grief.  somehow a burt restore of c1iscepics failed to work, and so the LSC XYCOM settings were not correct.  This meant that the LSC whitening filter states were not being correctly set and reported, making it difficult to lock for at least the last week or so.

Attachment 1: C1LSC_XYCOM.jpg
C1LSC_XYCOM.jpg
  2027   Wed Sep 30 02:01:28 2009 robUpdateLockingweek

It's been a miserable week for lock acquisition, with each night worst than the last.  The nadir was around Sunday night, when I couldn't even get a PRM to lock stably, which meant that the auto-alignment scripts could not finish successfully.  It now appears that was due to some XYCOM mis-settings. 

We've also been having problems with timing for c1susvme2.  Attached is a one-hour plot of timing data for this cpu, known as SRM.  Each spike is an instance of lateness, and a potential cause of lock loss.  This has been going on for a quite a while.

Tonight we also encountered a large peak in the frequency noise around 485 Hz.  Changing the MZ lock point (the spot in the PZT range) solved this.

 

Attachment 1: srmcpu.png
srmcpu.png
  2028   Wed Sep 30 12:21:08 2009 JenneUpdateComputersrestarted the elog

I'm blaming Zach on this one, for trying to upload a pic into the ATF elog (even though he claims it was small....)  Blame assigned: check.  Elog restarted: check.

  2029   Wed Sep 30 17:49:21 2009 JenneUpdateAdaptive FilteringNew UP/DOWN scripts for OAF

[Sanjit, Jenne]

The up and down scripts accessible from the OAF (still C1:ASS-TOP) screen are now totally functional and awesome.  They are under the blue ! button.  The up script can either be for the Seismometers, or the Accelerometers at this time.  The only difference between these 2 is which burt restore file they look at:  the seismometer version puts all 7 seismometer channels in the PEM selecting matrix, while the accelerometer version puts the 6 accelerometer channels in that matrix.  Both scripts also turn on HP_1mHz filters in the ERR_EMPH filter bank and all of the witness filter banks, and the AA32 and AI32 filters in ERR_EMPH, CORR and PEM filter banks.  This makes all of the starting filters the same between the witness paths and the error path.

If you want to use a different combination of sensors, run one of the up scripts, then change the PEM matrix by hand. 

The down script disables the output to the optics, and resets the adapted filter coefficients.  DO NOT use this script if you're trying to "pause" the filter to take some nice long averages.

  2030   Thu Oct 1 03:12:56 2009 robUpdateLockingsome progress

Good progress in IFO locking tonight, with the arm powers reaching about half the full resonant maximum. 

Still to do is check out some weirdness with the OMC DAC, fix the wireless network, and look at c1susvme2 timing.

  2031   Thu Oct 1 08:37:43 2009 steveUpdateSUSsuspention damping restored and MZ HV stuck

Earthquake  of magnitude 5.0  shakes ETMY loose.

MC2 lost it's damping later.

Attachment 1: eq5oct1.jpg
eq5oct1.jpg
  2032   Thu Oct 1 09:36:09 2009 KojiUpdateMZMZ relocked (Re:suspention damping restored and MZ HV stuck)

MZ stayed unlocked. Now It was relocked.

Quote:

Earthquake  of magnitude 5.0  shakes ETMY loose.

MC2 lost it's damping later.

 

  2033   Thu Oct 1 10:23:00 2009 steveUpdatePSLMC2 damping restored again

 The EQ did not change the input beam pointing. All back to normal, except MC2 wachdogs tripped again.

Attachment 1: mc2again.jpg
mc2again.jpg
  2034   Thu Oct 1 11:39:47 2009 JenneUpdateSUSMC2 damping restored again

Quote:

 The EQ did not change the input beam pointing. All back to normal, except MC2 wachdogs tripped again.

 Round 3 for the day of MC2 watchdogs tripping.

  2035   Thu Oct 1 13:12:41 2009 KojiUpdateMZMZ Work from 13:00-

I will investigate the MZ board. I will unlock MZ (and MC).

  2036   Thu Oct 1 14:22:28 2009 robUpdateSUSall suspensions undamped

Quote:

Quote:

 The EQ did not change the input beam pointing. All back to normal, except MC2 wachdogs tripped again.

 Round 3 for the day of MC2 watchdogs tripping.

 I've watchdogged all the suspensions while I mess around with computers.  If no one else is using the IFO, we can leave them undamped for a couple of hours to check the resonant frequencies, as long as I don't interrupt data streams with my computer hatcheting.

  2037   Thu Oct 1 15:42:55 2009 robUpdateLockingc1susvme2 timing problems update

Quote:

We've also been having problems with timing for c1susvme2.  Attached is a one-hour plot of timing data for this cpu, known as SRM.  Each spike is an instance of lateness, and a potential cause of lock loss.  This has been going on for a quite a while.

 

 

Attached is a 3 day trend of SRM CPU timing info.  It clearly gets better (though still problematic) at some point, but I don't know why as it doesn't correspond with any work done.  I've labeled a reboot, which was done to try to clear out the timing issues.  It can also be seen that it gets worse during locking work, but maybe that's a coincidence.

Attachment 1: srmcpu2.png
srmcpu2.png
  2038   Thu Oct 1 19:04:05 2009 KojiUpdateMZMZ work done (Re: MZ Work from 13:00-)

MZ work has been done. I did not change anything on the circuit.

Recently we observed that the MZ PZT output was sticking at a certain voltage. I found the reason.
Shortly to say "we must return the PZT Ramp offset to 0, after the lock"

I am going to write a MZ auto lock script someday, to do it automatically.


According to the resister values used in the circuit, the PZT HV output voltage is determined by the following formula:

V_PZT = 150 - 12 V_ctrl - 24 Vramp

Here the ramp voltage Vramp moves from -10V to +10V, the feedback control voltage V_ctrl moves from -13V to +13V.
The baseline offset of 150V is provided in the circuit board.

When V_ramp is 0, V_PZT runs from 0 to 300. This is just enough for the full scale of the actual V_PZT range,
that is 0V~280V.

If any Vramp offset is given, V_PZT rails at either side of the edges. This limits the actual range of the PZT out.

This is not nice, but is what happened recently.

Quote:

I will investigate the MZ board. I will unlock MZ (and MC).

 

Attachment 1: MZ_PZT.pdf
MZ_PZT.pdf
  2039   Thu Oct 1 19:18:24 2009 KojiUpdateSUSall suspensions undamped

Ops. I restored the damping of the suspensions at around 16:30.

Quote:

Quote:

Quote:

 The EQ did not change the input beam pointing. All back to normal, except MC2 wachdogs tripped again.

 Round 3 for the day of MC2 watchdogs tripping.

 I've watchdogged all the suspensions while I mess around with computers.  If no one else is using the IFO, we can leave them undamped for a couple of hours to check the resonant frequencies, as long as I don't interrupt data streams with my computer hatcheting.

 

  2040   Fri Oct 2 02:55:07 2009 robUpdateLockingmore progress

More progress with locking tonight, with initial acquisition and power ramps working.  The final handoff to RF CARM still needs work.

I found the wireless router was unplugged from the network--just plugging in the cable solved the problem.  For some reason that RJ45 connector doesn't actually latch, so the cable is prone to slipping out of the jack.

 

  2041   Fri Oct 2 14:52:55 2009 ranaUpdateComputersc1susvme2 timing problems update

The attached shows the 200 day '10-minute' trend of the CPU meters and also the room temperature.

To my eye there is no correlation between the signals. Its clear that c1susvme2 (SRM LOAD) is going up and no evidence that its temperature.

 

Attachment 1: Untitled.png
Untitled.png
  2042   Fri Oct 2 15:11:44 2009 robUpdateComputersc1susvme2 timing problems update update

It got worse again, starting with locking last night, but it has not recovered.  Attached is a 3-day trend of SRM cpu load showing the good spell.

Attachment 1: srmcpu3.png
srmcpu3.png
  2044   Fri Oct 2 17:00:06 2009 AlbertoUpdateGeneral40m Update - Requirements on the 5x Frequency Multiplier

Here's the gist of the requirements on the 5x frequency multiplier for the upgrade (see attachemnt - despite the preview down here in the elog, they're 3 pages).

An extended version is available on the wiki.

A more complete document is under work and will soon be available on the wiki as well.

Attachment 1: DesignOfTheMultiplier1-14-15.pdf
DesignOfTheMultiplier1-14-15.pdf DesignOfTheMultiplier1-14-15.pdf
  2045   Fri Oct 2 18:04:45 2009 robUpdateCDSDTT no good for OMC channels

I took the output of the OMC DAC and plugged it directly into an OMC ADC channel to see if I could isolate the OMC DAC weirdness I'd been seeing.  It looks like it may have something to do with DTT specifically.

Attachment 1 is a DTT transfer function of a BNC cable and some connectors (plus of course the AI and AA filters in the OMC system).  It looks like this on both linux and solaris.

Attachment 2 is a transfer function using sweepTDS (in mDV), which uses TDS tools as the driver for interfacing with testpoints and DAQ channels. 

Attachment 3 is a triggered time series, taken with DTT, of the same channels as used in the transfer functions, during a transfer function.  I think this shows that the problem lies not with awg or tpman, but with how DTT is computing transfer functions. 

 

I've tried soft reboots of the c1omc, which didn't work.   Since the TDS version appears to work, I suspect the problem may actually be with DTT.

Attachment 1: omc_dac_dtt.png
omc_dac_dtt.png
Attachment 2: omc_dac_sweepTDS.png
omc_dac_sweepTDS.png
Attachment 3: omc_dac_dtt_ts.png
omc_dac_dtt_ts.png
  2047   Sat Oct 3 14:53:24 2009 robUpdateLockingmore progress

Late last night after the ETMY settled down from the quake I made some more progress in locking, with the handoff to RF CARM succeeding once.  The final reduction of the CARM offset to zero didn't work, however.

  2048   Mon Oct 5 02:51:08 2009 robUpdateLockingalmost there

Working well tonight: the handoff of CARM to RF (REFL2I), successful reduction of CARM offset to zero, and transition control of MCL path to the OUT1 from the common mode board.  All that's left in lock acquisition is to try and get the common mode bandwidth up and the boost on.

  2050   Mon Oct 5 10:41:31 2009 KojiUpdatePSLPSL laser accidentally turned off

The PSL output looks smaller than the incident. Try to FSS Slow actuator adj of -5.6 (nominal), instead of -3.5.

Quote:

Alberto, Steve,

While I was moving a cart near by the PSL table I pushed the red emergency button that turns off the PSL laser. We had to unlock the button and then power cycle the laser driver to turn the laser back on.

I relocked MZ, FSS, PMC and I'm now waiting for the power to finish ramping up back to the previous value.

 

  2051   Mon Oct 5 13:43:37 2009 ZachUpdatePSLPSL table photos

I have been commissioned to take pictures of the PSL table so that it can be diagrammed. I am starting now (1:42 pm, 10/5/09).

  2052   Mon Oct 5 14:18:41 2009 ZachUpdatePSLPSL table photos

Quote:

I have been commissioned to take pictures of the PSL table so that it can be diagrammed. I am starting now (1:42 pm, 10/5/09).

All done (for now). That wasn't so bad, was it?

  2053   Mon Oct 5 14:37:29 2009 AlbertoUpdateABSLAbsolute Length Meaasurement NPRO is on

In the revival of the experiement length measurement for the recycling cavities, I turned the auxiliary NPRO back on. The shutter is closed.

I also recollected all the equipment of the experiment after that during the summer it had been scattered around the lab to be used for other purposes (Joe and Zach's cameras and Stephanie and Koji's work with the new EOM).

  2054   Mon Oct 5 18:34:26 2009 JenneUpdateAdaptive FilteringAttempts to take a TF of the OAF system

[Jenne, Sanjit]

As per Matt's instructions in his OAF document (elog 395) in the Tuning section, Sanjit and I took a transfer function measurement from the output of the OAF system, to the input.  i.e. we're trying to measure what happens out in the real world when we push on MC1, and how that is fed back to the input of our filter as MC_L.  The game plan is to measure this transfer function, and read off the phase at the nyquist frequency, and use this value to calculate the appropriate sample-and-hold delay to be used in the OAF.  The downsample rate for the OAF is 32, so that we're running at 64Hz instead of the 2048Hz of the front-end.  Thus, our Nyquist frequency is 32Hz.

                            DownSampleRate

Phase@Nyquist * ------------------------

                                    180

In the attached figure we do a swept sine from CORR_EXC to ERR_EMPH_OUT to determine the transfer function.  Here, we turn off all of the filters in both the CORR and EXC banks, because those are already matched/taken into account in the PEM filter banks. 

Using the cursor on DTT, we find that the phase at 29.85Hz is -228.8deg, and at 37.06Hz is -246.0deg.  Extrapolating, this means that at 32Hz, we expect about -234deg phase.  Using our handy-dandy formula, this means that we should try a delay of 41 or 42 (41.6 is between these two...) 

We'll give this a shot!

Attachment 1: OAF_TF_CORRexc_EMPHout_2.png
OAF_TF_CORRexc_EMPHout_2.png
  2055   Mon Oct 5 19:39:26 2009 KojiUpdatePSLPSL laser accidentally turned off

I set the FSS slow actuator adj to -5.6 at the lunch time. It gave a little help at that time. Now max of the MC Trans is comming back somehow. I hope the MC Trans level is as good as before, if the HEPA is slowed down.

Quote:

The PSL output looks smaller than the incident. Try to FSS Slow actuator adj of -5.6 (nominal), instead of -3.5.

Quote:

Alberto, Steve,

While I was moving a cart near by the PSL table I pushed the red emergency button that turns off the PSL laser. We had to unlock the button and then power cycle the laser driver to turn the laser back on.

I relocked MZ, FSS, PMC and I'm now waiting for the power to finish ramping up back to the previous value.

 

 

  2056   Tue Oct 6 01:41:20 2009 robUpdateLockingDC Readout

Lock acquisition working well tonight.  Was able to engage CM boost (not superboost) with bandwidth of ~10kHz.  Also succeeded once in handing off DARM to DC readout.

  2057   Tue Oct 6 10:09:55 2009 ZachUpdatePSLMore pictures

I'm back to terrorize the PSL table again. The pictures I took yesterday were rubbish--today I'm using a clamp that Steve was nice enough to loan me. I'm starting now, at 10:09 am.

  2058   Tue Oct 6 11:13:53 2009 JenneUpdateAdaptive FilteringAttempts to take a TF of the OAF system

Quote:

[Jenne, Sanjit]

As per Matt's instructions in his OAF document (elog 395) in the Tuning section, Sanjit and I took a transfer function measurement from the output of the OAF system, to the input.  i.e. we're trying to measure what happens out in the real world when we push on MC1, and how that is fed back to the input of our filter as MC_L.  The game plan is to measure this transfer function, and read off the phase at the nyquist frequency, and use this value to calculate the appropriate sample-and-hold delay to be used in the OAF.  The downsample rate for the OAF is 32, so that we're running at 64Hz instead of the 2048Hz of the front-end.  Thus, our Nyquist frequency is 32Hz.

                            DownSampleRate

Phase@Nyquist * ------------------------  =  Delay

                                    180

In the attached figure we do a swept sine from CORR_EXC to ERR_EMPH_OUT to determine the transfer function.  Here, we turn off all of the filters in both the CORR and EXC banks, because those are already matched/taken into account in the PEM filter banks. 

Using the cursor on DTT, we find that the phase at 29.85Hz is -228.8deg, and at 37.06Hz is -246.0deg.  Extrapolating, this means that at 32Hz, we expect about -234deg phase.  Using our handy-dandy formula, this means that we should try a delay of 41 or 42 (41.6 is between these two...) 

We'll give this a shot!

 As Rana pointed out to me last night, I was using continuous phase, which is not good.  When using regular phase, I find: (29.85Hz, 131.216deg), (37.06Hz, 113.963deg), so extrapolating gives (32Hz, 126.07deg).  Plugging this in to our handy-dandy formula, we get a delay of 22.4, so we should try both 22 and 23.

  2061   Wed Oct 7 03:49:49 2009 ranaUpdateAdaptive FilteringAttempts to take a TF of the OAF system

 Here's a plot of the spectra of the seismometers and MCL. The coherence shows which axes are aligned right now: MC1_X is coherent with GUR_NS which means that its mis-oriented.

I've now swapped the "MC1" cables: so the old "NS" now goes into EW and the old EW now goes into NS. VERT is unchanged.

Also fixed the channel names - the Guralp previously named MC1 is now GUR1 and the other one is GUR2. Also no more EW, NS, & VERT. Its all XYZ.

DAQD restarted with the new channel names.

Attachment 1: Untitled.png
Untitled.png
  2063   Wed Oct 7 07:42:55 2009 ranaUpdateAdaptive FilteringAttempts to take a TF of the OAF system

I remeasured the OAF time delay using the OAF-TF template from the Templates/ directory.

Troublingly, I found the MC1 dewhitening switches set OFF - please make sure that the MC1 dewhitening is back ON after each OAF tuning so that the interferometer locking is not hosed.

The OAF-TF template had the excitation amplitude set ~20x too high. I reduced it and the coherence was still > 0.95. The phase at 32 Hz was still ~126 deg as Jenne had measured, but since the phase at DC is 180 deg, the overall phase lag is just 180-126 = 54 deg. So the delay should be 54/180 * 32 = 9.7 => 10. Luckily, Jenne is working on an instructional manual for OAF that will make all of this crystal clear.

  2065   Wed Oct 7 19:23:49 2009 JenneUpdateAdaptive Filtering(Final?) PEM cabling changes

Quote:

 Here's a plot of the spectra of the seismometers and MCL. The coherence shows which axes are aligned right now: MC1_X is coherent with GUR_NS which means that its mis-oriented.

I've now swapped the "MC1" cables: so the old "NS" now goes into EW and the old EW now goes into NS. VERT is unchanged.

Also fixed the channel names - the Guralp previously named MC1 is now GUR1 and the other one is GUR2. Also no more EW, NS, & VERT. Its all XYZ.

DAQD restarted with the new channel names.

 I spiffed up the order of the cables / sensors plugged into the PEM ADCU.  Now all of the seismometers are labeled as Rana left them, and the 2 Guralp's have their sets of 3 channels next to eachother in channel-number-land.  None of the accelerometer names/cabling have changed recently.  In the table, Cable-label refers to the physical tag tied to the end of the cables plugged into the ADCU...they are meant to be descriptive of what seismometer channels they are hooked up to, and then the names change to something useful for us when they come into the DAQ system.  Also, the labels of input channels on the ASS_TOP_PEM screen have been updated accordingly.

 

Channel Name Channel Number on ADCU and OAF PEM list Cable-label .ini channel number
C1:SEIS-GUR2_X 2 Gur2 EW 15001
C1:SEIS-GUR2_Y 3 Gur2 NS 15002
C1:SEIS-GUR2_Z 4 Gur2 Vert 15003
C1:SEIS-GUR1_X 10 Gur1 EW 15009
C1:SEIS-GUR1_Y 11 Gur1 NS 15010
C1:SEIS-GUR1_Z 12 Gur1 Vert 15011
C1:SEIS-RANGER_Y 24 Ranger

15023

 

  2066   Wed Oct 7 20:32:21 2009 ranaUpdateAdaptive Filteringextra delay and noise in PEM -> ASS/OAF system

[Rana, Jenne]

There is some craziness going on with the delay in the PEM path for the OAF.  We plot the difference between the C1:PEM-SEIS_GUR1_X and C1:ASS-TOP_PEM_10.  These are physically the same channel, plugged into the PEM ADCU, and then the signal is used as a regular PEM channel, and is also sent to the ASS computer and used there for the OAF system.  As you can see in the blue trace on the bottom plot, there is a huge amount of delay, and it's very noisy.  We also plot the _GUR2_X / ASS-TOP_PEM_2 pair (red), and it has a similar amount of delay, but it is not nearly as fuzzy and noisy.  For comparison, we plot the SUS-MC2_MCL (which is identical to IOO-MC_L) and ASS-TOP_ERR_MCL pair (green), and they don't have any big overall delay problems, so it's not totally a problem with the signals getting to the ASS computer.

This problem was present during/after all of the following attempts to fix it:

* The sample rate on the ASS computer is 2048.  The PEM channels were being acquired the ADCU at 512.  We changed the ADCU sampling rate to 2048 to match.

* We soft rebooted the ASS computer, in case it was a timing problem.

* Doing a "sudo shutdown -r now" while logged in as controls.

We might also try resetting/power cycling c0dcu in the morning.  Alex has been emailed to help us try to figure this out.

 

In other news, the time delay that we measure from the plot gives us 180degrees in ~210Hz.  This corresponds to a little more than 2msec of delay, with the C1:ASS version lagging behind the C1:PEM version.  (2 samples at 840Hz) Converting to the 2048 sampling rate, we have a delay of 4.8, so 5 front-end cycles.  Since Rana measured this morning that the delay indicated by the transfer function is 10 cycles, and this delay shows that the ASS lags the actual seismometer signal by 5 cycles, we should subtract this 5 from the 10 from the transfer function, giving us a final sample-and-hold delay of 5.  Coincidentally(?), 5 is the delay that was found in the C1:ASS-TOP screen, after it's one year of dormancy.  The point of the delay feature in the code is to help match the delay in the two signal paths: the PEM path and the output path of the filter.  Since the output has a lag of 10, and the PEM path has a lag of 5, to make them match, we artificially put in a delay of 5.

Attachment 1: a.gif
a.gif
  2067   Thu Oct 8 11:10:50 2009 josephb, jenneUpdateComputersEPICs Computer troubles

At around 9:45 the RFM/FB network alarm went off, and I found c1asc, c1lsc, and c1iovme not responding. 

I went out to hard restart them, and also c1susvme1 and c1susvme2 after Jenne suggested that.

c1lsc seemed to have a promising come back initially, but not really.  I was able to ssh in and run the start command.  The green light under c1asc on the RFMNETWORK status page lit, but the reset and CPU usage information is still white, as if its not connected.   If I try to load an LSC channel, say like PD5_DC monitor, as a testpoint in DTT it works fine, but the 16 Hz monitor version for EPICs is dead.  The fact that we were able to ssh into it means the network is working at least somewhat.

I had to reboot c1asc multiple times (3 times total), waiting a full minute on the last power cycle, before being able to telnet in.  Once I was able to get in, I restarted the startup.cmd, which did set the DAQ-STATUS to green for c1asc, but its having the same lack of communication as c1lsc with EPICs.

c1iovme was rebooted, was able to telnet in, and started the startup.cmd.  The status light went green, but still no epics updates.

The crate containing c1susvme1 and c1susvme2 was power cycled.  We were able to ssh into c1susvme1 and restart it, and it came back fully.  Status light, cpu load and channels working.  However I c1susvme2 was still having problems, so I power cycled the crate again.  This time c1susvme2 came back, status light lit green, and its channels started updating.

At this point, lacking any better ideas, I'm going to do a full reboot, cycling c1dcuepics and proceeding through the restart procedures.

  2068   Thu Oct 8 11:37:59 2009 josephbUpdateComputersReboot of dcuepics helped, c1susvme1 having problems

Power cycling c1dcuepics seems to have fixed the EPICs channel problems, and c1lsc, c1asc, and c1iovme are talking again.

I burt restored c1iscepics and c1Iosepics from the snapshot at 6 am this morning.

However, c1susvme1 never came back after the last power cycle of its crate that it shared with c1susvme2.  I connected a monitor and keyboard per the reboot instructions.  I hit ctrl-x, and it proceeded to boot, however, it displays that there's a media error, PXE-E61, suggests testing the cable, and only offers an option to reboot.  From a cursory inspection of the front, the cables seem to look okay.  Also, this machine had eventually come back after the first power cycle and I'm pretty sure no cables were moved in between.

 

  2069   Thu Oct 8 14:41:46 2009 jenneUpdateComputersc1susvme1 is back online

Quote:

Power cycling c1dcuepics seems to have fixed the EPICs channel problems, and c1lsc, c1asc, and c1iovme are talking again.

I burt restored c1iscepics and c1Iosepics from the snapshot at 6 am this morning.

However, c1susvme1 never came back after the last power cycle of its crate that it shared with c1susvme2.  I connected a monitor and keyboard per the reboot instructions.  I hit ctrl-x, and it proceeded to boot, however, it displays that there's a media error, PXE-E61, suggests testing the cable, and only offers an option to reboot.  From a cursory inspection of the front, the cables seem to look okay.  Also, this machine had eventually come back after the first power cycle and I'm pretty sure no cables were moved in between.

 

 I had a go at trying to bring c1susvme1 back online.  The first few times I hit the physical reset button, I saw the same error that Joe mentioned, about needing to check some cables.  I tried one round of rebooting c1sosvme, c1susvme2 and c1susvme1, with no success.  After a few iterations of jiggle cables/reset button/ctrl-x on c1susvme1, it came back.  I ran the startup.cmd script, and re-enabled the suspensions, and Mode Cleaner is now locked.  So, all systems are back online, and I'm crossing my fingers and toes that they stay that way, at least for a little while.

  2074   Fri Oct 9 03:53:56 2009 JenneUpdateAdaptive FilteringRemaking the ASS

The c1ass computer, which is now used for the OAF system, has many remnants from the days when it was actually used as an ASS.  These PIT and YAW filter banks and other things were taking up a lot of unnecessary space, so I deleted them in the ass.mdl file.  These files are all backed up, so we can always revert back to an older version when we want some Alignment Stabilization again someday.  I then did a make ass, following the instructions on the 40m Wiki -> Computers and Scripts -> Simulink to Front-End Code page.  Rana moved some things around, most notably all of the things (like the ASS screens) which were only in ...../users/alex/.... are now in ....../caltech/cds/advLigo/..... .  This required a few restarts of the c1ass machine (after a couple different versions of the simulink diagram....one to make sure we knew how to do it, and then again actually deleting the unused portions).

The big lesson of the night was that there are 2 signal paths for the PEM channels.  As is shown in Figure 3 in the mevans document, the PEM channels get the matching filters when they go to the adaptation algorithm, but when they go to the FIR filter, they do not get the matching filters. This is implemented by taking the output of the giant PEM matrix, and having a duplicate of each of the channels "selected for adaptation", one which gets filtered through the PEM_N_ADPT banks, and one which goes straight (in code-land) to the FIR filter.  So, it seems like all the filters which we had been including in the input side of the matrix for matching purposes need to be put in the output side.  One of the AA32 filters needs to stay in the input side, for actual anti imaging of the PEM channels, then we put the AA32 and AI32 which are for matching the ERR_EMPH and CORR filter banks up in the PEM_N_ADAPT banks.  Rana and I made these filters, and they are now turned on appropriately with the OAF down script (so that all the filters are ready and waiting for the OAF to be turned on).

A little success with getting the 3Hz peak reduced, but not a lot beyond that.  Tomorrow I'll put the accelerometers back where they used to be to see if they help out at all.

  2076   Fri Oct 9 16:36:13 2009 rob UpdateIOOfrequency noise problem

I used the XARM as a reference to measure the frequency noise after the MC.  It's huge around 4kHz--hundreds of times larger than the frequency noise the MC servo is actually squashing.  This presents a real problem for our noise performance.

An elog search reveals that this noise has been present (although not calibrated till now) for years.  We're not sure what's causing it, but suspicion falls on the piezojena input PZTs. 

I didn't bother too much about it before because we previously had enough common mode servo oomph to squash it below other DARM noises, and I didn't worry too much about stuff at 4kHz..  Now that we have a weaker FSS and thus much weaker CM servo, we can't squash it, and the most interesting feature of our IFO is at 4kHz. 

I'll measure the actual voltage noise going to the PZTs.  I remember doing this before and concluding it was ok, but can't find an elog entry.  So this time maybe I'll  do it right.

Attachment 1: freqnoiseaftermc.png
freqnoiseaftermc.png
  2077   Fri Oct 9 17:37:21 2009 JenneUpdateIOOMC2 trans beam is dumped

Last night while noise hunting, Rana found that the MC2 trans beam has been left for an unknown length of time just hitting the side of the black enclosure box.  Today I put a brand-spankin'-new razor dump on the MC2 table, to dump the beam.

ELOG V3.1.3-