40m QIL Cryo_Lab CTN SUS_Lab TCS_Lab OMC_Lab CRIME_Lab FEA ENG_Labs OptContFac Mariner WBEEShop
  40m Log, Page 131 of 339  Not logged in ELOG logo
ID Dateup Author Type Category Subject
  6546   Wed Apr 18 19:59:48 2012 JamieUpdateCDSCDS upgrade success

The upgrade is nearly complete:

  • new daqd code is running on fb
  • the fe/daqd timing issue was resolved by adjusting the GPS offset in the daqdrc.  I will document this more later.
  • the power outage conveniently rebooted all the front-end machines, so they're all now running new caRepeater
  • all models have been successfully recompiled with RCG 2.5 (with only a couple small glitches)
  • all new models are running on all front-end machines (with a couple exceptions)
  • all suspension models seem to be damping under local control (PRM is having troubles that are likely unrelated to the upgrade).
  • a lot of cleanup has been done

Remaining tasks/issues:

  • more testing OF EVERYTHING needs o be done
  • I did not yet update the DIS dolphin code, so we're running with the old code.  I don't think this is a problem, but it would be nice to get us running what they're running at the sites
  • I tried to cleanup/simplify how front-end initialization is done.  However, there is a problem and models are not auto-starting after reboot.  This needs to be fixed.
  • the userapps directory is in a new place (/opt/rtcds/userapps).  Not everything in the old location was checked into the repository, so we need to check to make sure everything that needs to be is checked in, and that all the models are running the right code.
  • the c1oaf model seems to be having a dolphin issue that needs to be sorted
  • the c1gfd model causes c1ioo to crash immediately upon being loaded.  I have removed it from the rtsystab.  That model needs to be fixed.
  • general model cleanup is in order.
  • more front-end cleanup is needed, particularly in regards to boot-up procedure.
  • document the entire upgrade procedure.

I'll finish up these remaining tasks tomorrow.

  6547   Wed Apr 18 23:12:49 2012 DenUpdateCDSoaf

Adaptive filter outputs some non-zero signal in the OFF position

Screenshot.png

I turned "ON" one of them and c1lsc suspended, I've rebooted it and restarted models on c1lsc and c1sus.

Now it also outputs something non-zero though the first line of the adaptive code is "if(OFF) output=0.0; return;" May be another version of the code has been compiled.

Edit: Old version (~september) of the code and oaf model is running now. In the 2.1 code there was a link from src/epics/simLink to oaf code for each DOF. It seems that 2.5 version finds models and c codes in standard directories. I need to move working code to the proper directory.

  6548   Thu Apr 19 08:43:16 2012 JamieUpdateCDSoaf

Quote:

Edit: Old version (~september) of the code and oaf model is running now. In the 2.1 code there was a link from src/epics/simLink to oaf code for each DOF. It seems that 2.5 version finds models and c codes in standard directories. I need to move working code to the proper directory.

 Yes, things have changed.  Please wait until I have things cleaned up before working on models.  I'll explain what the new setup is.

  6549   Thu Apr 19 09:45:43 2012 SureshUpdateGeneralMC2 Oplev signals redirected for use in WFS servo

Quote:

 
Quote:
  • there are no oplev signals in MC1, MC2, and MC3

 None of the 3 MC optics have oplevs, so there shouldn't be any oplev signals. Although MC2 has the trans QPD, which was once (still is??) going through the MC2 oplev signal path.

.......

 

The MC2 Oplev signal path has been modified in the c1mcs model.  The ADC channels have been sent over the rfm into c1ioo model and are currently used in the WFS servo loops.  Please see this elog 5397.

 

  6550   Thu Apr 19 16:21:04 2012 ZachUpdateComputer Scripts / ProgramsArbcav updated, made badass

I have modified Arbcav to be way cooler than it used to be.

Main modifications:

  • Can now truly model an arbitrary cavity geometry
    • The previous version could only handle a few different topologies. In each case, it would unfold the cavity into the equivalent linear cavity and use the g-parameter method to calculate gouy phases, etc.
    • The new model uses the closed cavity propagation matrix to find the supported mode, and then explicitly calculates the accumulated gouy phase by propagating the beam through the full cavity. This is done analytically with zR, so there is negligible slow-down.
  • Now plots a diagram of the cavity geometry, both to help you and for you to verify that it is calculating the right thing (<-- this is the cool part)
    • Plots the beam path and mirror locations
    • Specifies whether mirrors are curved or flat
    • Prints mirror parameters next to them
    • Finds all intracavity waist locations and plots them
    • Gives waist information (size in X, Y)

Since the information is already there, I will have the output structure include things like the input beam q parameter, which could then be fed directly to mode matching tools like ModeMatchr.

The function takes as input the same arguments as before. Example for a square cavity:

out = arbcav([200e-6 50e-6 200e-6 50e-6],[0.75 0.75 0.75 0.75],[1e10 9 1e10 9],[45 45 45 45],29.189e6,10e-6,1064e-9,1000);

i.e.,

out = arbcav(transmissivity_list, length_list, RoC_list, angle_list, modulation_freq, loss_list_or_loss_per_mirror, wavelength, num_pts_for_plot);

If you don't give it a modulation frequency, it will just plot carrier HOMs. If you don't give it RoCs and angles, it will just plot the transmission spectrum.

 

I'm still fine-tuning some functionality, but I should have it up on the SVN relatively soon. Comments or suggestions are welcome!

 

Some screenshots:

Cavity geometry plots (linear, triangular, square, bowtie):

linear.pngtriangular.pnggyro.pngbowtie.png

 

Transmission and HOM spectra (these correspond to the square cavity at lower left, above):

gyro_spect.pnggyro_HOM.png

  6551   Thu Apr 19 22:18:24 2012 DenUpdateAdaptive Filteringoaf algorithm: old vs new

 Here are the issues that I found not quite accurate in the old oaf code:

1. There is no need to calculate the norm of the witness signal every time from zero: 

norm += (*pBufAdapt) * (*pBufAdapt); // add to the norm 

Every step the witness signal vector is the same except the first and last values

wit[i].norm += histAdpt[nCoeff]*histAdpt[nCoeff] - histAdpt[0]*histAdpt[0];

 This step will reduce the number of multiplications and summations from 3*M/k to 2*M/k, M - filter length, k - downsample ratio.

2. Old code filter corrects filter coefficients with a delay equal to k=downsample ratio (pretty big):

witness       o o o o o o o o o o o o o o o o o o o o o

error           o o o o o o o o o o o o o o o o o o o o o

We want the filter to work at green points and skip red points computing output and correcting coefficients at this time (downsample ratio in this example is 4). Old code

  • grabs error signal
  • calculates output during next k-1 red points and 1 green point
  • corrects coefficients using this error during next k-1 red points and 1 green point

But LMS algorithm should correct coefficients according to the latest error. As we calculate output and correct coefficients before the latest error signal will be available, we should change the order:

  • grabs error signal
  • corrects coefficients using this error during next k-1 red points and 1 green point
  • calculates output during next k-1 red points and 1 green point

This scheme is completely equivalent to the ordinary LMS algorithm, as now we correct coefficients according to the latest error signal and so do not add any delay.

3. So long soft start is probably not needed for the LMS filter, it makes the filter to converge longer 

// modify adaptation gain for soft start

    if( state.iWait < state.nFIR )

    {

      adaptGain = 0.0;

      decayRate = 1.0;  // clear FIR coeffs after reset

    }

As far as I understand this is done to prevent the filter from huge coefficients variations in the beginning when norm is not calculated yet. Instead we can just introduce some small

epsilon = 10.0;

to prevent the filter from divergence in the beginning 

delta = mu * error / (wit[i].norm + epsilon);

Though some soft start might be needed by not so long - it will take several minutes before the adaptation gain will take it's specified value. 

  6552   Fri Apr 20 19:54:57 2012 JamieUpdateCDSCDS upgrade problems

I ran into a couple of snags today.

A big one is that the framebuilder daqd started going haywire when I told it to start writing frames.  After restart the logs started showing this:

[Fri Apr 20 17:23:40 2012] main profiler warning: 0 empty blocks in the buffer
[Fri Apr 20 17:23:41 2012] main profiler warning: 0 empty blocks in the buffer
[Fri Apr 20 17:23:42 2012] main profiler warning: 0 empty blocks in the buffer
[Fri Apr 20 17:23:43 2012] main profiler warning: 0 empty blocks in the buffer
[Fri Apr 20 17:23:44 2012] main profiler warning: 0 empty blocks in the buffer
[Fri Apr 20 17:23:45 2012] main profiler warning: 0 empty blocks in the buffer
GPS time jumped from 1019002442 to 1019003041
FATAL: exception not rethrown
FATAL: exception not rethrown
FATAL: exception not rethrown

and the network seemed like it started to get really slow.  I wasn't able to figure out what was going on, so I shut the frame writing off again.  I'll have to work with Rolf on that next week.

Another big problem is the workstation application upgrades.  The NDS protocol version has been incremented, which means that all the NDS client applications have to be upgraded.  The new dataviewer is working fine (on pianosa), but dtt is not:

controls@pianosa:~ 0$ diaggui
diaggui: symbol lookup error: /ligo/apps/linux-x86_64/gds-2.15.1/lib/libligogui.so.0: undefined symbol: _ZN18TGScrollBarElement11ShowMembersER16TMemberInspector
controls@pianosa:~ 127$ 

I don't know what's going on here.  All the library paths are ok.  Hopefully I'll be able to figure this out soon.  The old version of dtt definitely does not work with the new setup.

I might go ahead and upgrade some more of the workstations to Ubuntu in the next couple of days as well, so everything is more on the same page.

I also tried to cleanup the front-end boot process, which has it's own problems (models won't auto-start).  I haven't figured that out yet either.  It really needs to just be completely overhauled.

  6553   Fri Apr 20 23:02:25 2012 DenUpdateAdaptive Filteringfrequency domain filter

 DFT-LMS is a frequency domain adaptive filter that demonstrates faster convergence compared to the time-domain LMS filter. I've tested Discrete Fourier Transform (DFT-LMS) filter. It converts witness signal to the frequency domain using DFT and corrects the eigenvalues of the covariance matrix to make them as equal to each other as possible (does pre-whitenning of the witness signal).

Left plot compares learning curves for time domain LMS and DFT-LMS algorithms on the simulated data from seismometers and mcl (number of averages  = 30) Right plot shows the evolution of the filter coefficients norm (Euclidean norms of the coefficient vector). Though LMS algorithm works in the time domain and DFT-LMS in the frequency domain, the coefficient vectors must have the same length, because we Fourier Transform is achieved by applying a unitary operator => vector norm must not change.

dft.png   norm.png

Plots show that both algorithms converge to the same coefficients vector norm, but DFT-LMS does it much faster then LMS. 

Online realization: 

Good news: algorithm complexity is linear in filter length. Though the algorithm does Fourier transform, its complexity is still O(M), M - number of coefficients. Simulations show that DFT-LMS is ~8-9 times slower then LMS. This is not so bad, may be we can do even slightly better.

Bad news: downsample process is not simple. Due to Fourier transform, the filter needs the whole witness signal vector before calculating the output. This is sad and in contrast with LMS algorithm where we could start to calculate the new output immediately after computing the previous output. We either need to calculate the whole output immediately or introduce delay in the output or approximate Fourier transform with some previous witness signal values.

Realization in the kernel: I asked Alex about complex numbers, exponents, sin and cos functions in the kernel c and he answers that we do not have complex numbers, about exp, cos, sin he is not sure. But for DFT-LMS algorithm we are able to get round of these difficulties. Complex numbers will be presented as  2 real numbers. Then exp (a) = cos(a) + i*sin(a). All what we need for DFT-LMS are sin(2 * pi * k / M) and cos(2 * pi * k / M), k=0,1,2,...,M-1. Fortunately, M - (filter length) is big enough, typical value pi/M ~ 0.001 and we can calculate sin(2*pi/M) and cos(2*pi/M) using Taylor series. As the argument is small, 5-6 terms will be enough to get precision ~1e-20. Then we build the whole table of cos and sin according to induction cos(2*pi/M*k) = cos(2*pi/M*(k-1))cos(2*pi/M) - sin(2*pi/M*(k-1))sin(2*pi/M), sin(2*pi/M*k) = cos(2*pi/M*(k-1))sin(2*pi/M) + sin(2*pi/M*(k-1))cos(2*pi/M). We should do it only once, so the algorithm will build these values in the beginning during first several iterations, then will use them.

The main problem is downsampling. I need to think more about it.

  6554   Sat Apr 21 17:38:19 2012 JamieUpdateCDSdtt, dataviewer working; problem with trend frames

Quote:

Another big problem is the workstation application upgrades.  The NDS protocol version has been incremented, which means that all the NDS client applications have to be upgraded.  The new dataviewer is working fine (on pianosa), but dtt is not:

controls@pianosa:~ 0$ diaggui
diaggui: symbol lookup error: /ligo/apps/linux-x86_64/gds-2.15.1/lib/libligogui.so.0: undefined symbol: _ZN18TGScrollBarElement11ShowMembersER16TMemberInspector
controls@pianosa:~ 127$

 dtt (diaggui) and dataviewer are now working on pianosa to retrieve realtime data and past data from DQ channels.

Unfortunately it looks like there may be a problem with trend data, though.  If I try to retrieve 1 minute of "full data" with dataviewer for channel C1:SUS-ITMX_SUSPOS_IN1_DQ around GPS 1019089138 everything works fine:

Connecting to NDS Server fb (TCP port 8088)
Connecting.... done
T0=12-04-01-00-17-45; Length=60 (s)
60 seconds of data displayed

but if I specify any trend data (second, minute, etc.) I get the following:

Connecting to NDS Server fb (TCP port 8088)
Connecting.... done
Server error 18: trend data is not available
datasrv: DataWriteTrend failed in daq_send().
T0=12-04-01-00-17-45; Length=60 (s)
No data output.

Alex warned me that this might have happened when I was trying to test the new daqd without first turning off frame writing.

I'm not sure how to check the integrity of the frames, though.  Hopefully they can help sort this out on Monday.

  6555   Sat Apr 21 20:40:28 2012 JamieUpdateCDSframebuilder frame writing working again

Quote:

A big one is that the framebuilder daqd started going haywire when I told it to start writing frames.  After restart the logs started showing this:

[Fri Apr 20 17:23:40 2012] main profiler warning: 0 empty blocks in the buffer
[Fri Apr 20 17:23:41 2012] main profiler warning: 0 empty blocks in the buffer
[Fri Apr 20 17:23:42 2012] main profiler warning: 0 empty blocks in the buffer
[Fri Apr 20 17:23:43 2012] main profiler warning: 0 empty blocks in the buffer
[Fri Apr 20 17:23:44 2012] main profiler warning: 0 empty blocks in the buffer
[Fri Apr 20 17:23:45 2012] main profiler warning: 0 empty blocks in the buffer
GPS time jumped from 1019002442 to 1019003041
FATAL: exception not rethrown
FATAL: exception not rethrown
FATAL: exception not rethrown

and the network seemed like it started to get really slow.  I wasn't able to figure out what was going on, so I shut the frame writing off again.  I'll have to work with Rolf on that next week.

So the framebuilder/daqd frame writing issue seems to have been a transient one.  Alex said he tried enabling frame writing manually and it worked fine, so I tried re-enabling the frame writing config lines and sure enough it worked fine.  So it's a mystery.  Alex said the "main profiler warning" lines tend to show up when data is backed up in the buffer, although he didn't explain why exactly that would have been the issue here.

daqdrc frame writing directives:

start frame-saver;
sync frame-saver;
start trender;
start trend-frame-saver;
sync trend-frame-saver;
start minute-trend-frame-saver;
sync minute-trend-frame-saver;
start raw_minute_trend_saver;
  6556   Sat Apr 21 21:10:34 2012 JamieUpdateCDStrend frame issue partially resolved

Quote:

Unfortunately it looks like there may be a problem with trend data, though.  If I try to retrieve 1 minute of "full data" with dataviewer for channel C1:SUS-ITMX_SUSPOS_IN1_DQ around GPS 1019089138 everything works fine:

Connecting to NDS Server fb (TCP port 8088)
Connecting.... done
T0=12-04-01-00-17-45; Length=60 (s)
60 seconds of data displayed

but if I specify any trend data (second, minute, etc.) I get the following:

Connecting to NDS Server fb (TCP port 8088)
Connecting.... done
Server error 18: trend data is not available
datasrv: DataWriteTrend failed in daq_send().
T0=12-04-01-00-17-45; Length=60 (s)
No data output.

Alex warned me that this might have happened when I was trying to test the new daqd without first turning off frame writing.

Alex told me that the "trend data is not available" message comes from the "trender" functionality not being enabled in daqd.  After re-enabling it (see #6555) minute trend data was available again.  However, there still seems to be an issue with second trends.  When I try to retrieve second trend data from dataviewer for which minute trend data *is* available I get the following error message:

Connecting to NDS Server fb (TCP port 8088)
Connecting.... done
No data found

read(); errno=9
read(); errno=9
T0=12-04-04-02-14-29; Length=3600 (s)
No data output.

Awaiting more help from Alex...

  6557   Mon Apr 23 23:20:07 2012 DenUpdatePEMmicrophones

Tonight I wanted to measure the ambient noise level using Blue Bird mics and figure out if Panasonic WM61a or Primo EM172/173 will be good enough or not. Blue Bird that is in the control room does not seem to work. May be the problem is with the pre-amplifier. The output measured by ADC/Oscilloscope is noise ( amplitude=5mV ). I will return to this issue tomorrow.

  6558   Tue Apr 24 09:58:37 2012 steveUpdatePEMearthquake 3.9 magnitude

Local eq shakes the lab

Attachment 1: eq3.9.png
eq3.9.png
  6559   Tue Apr 24 11:27:51 2012 steveUpdatePEMnew chairs

We received 6 new  chairs stools for the work benches that have 36" heights.

The control room east side and general office desks require 18" seat height. The new chairs stools have  minimum seat heights  26"

Please label chairs that we are getting rid of.

P4241026.JPG

  6560   Tue Apr 24 14:30:08 2012 JamieUpdateCDSIntroducing: rtcds, a utility for interacting with the CDS RTS/RCG

The new rtcds utility

I have written a new utility for interacting with the CDS RTS/RCG system: "rtcds".  It should be available on all workstations and front-end machines, but certain commands are restricted to run on certain front end machines (build, start, stop, etc.).  Here's the help:

controls@c1lsc ~ 0$ rtcds help
Usage: rtcds <command> [arg]

Available commands:

  build|make SYS      build model
  install SYS         install model
  start SYS|all       start model
  restart SYS|all     restart running model
  stop|kill SYS|all   stop running model
  list                list all models for current host

controls@c1lsc ~ 0$ 

Please use this new utility from now on when interacting with RTS.

I'll be improving and expanding it as we go.   Please let me know if you encounter any problems.

 

  6561   Tue Apr 24 14:35:37 2012 JamieUpdateCDSlimited second trend lookback

Quote:

Alex told me that the "trend data is not available" message comes from the "trender" functionality not being enabled in daqd.  After re-enabling it (see #6555) minute trend data was available again.  However, there still seems to be an issue with second trends.  When I try to retrieve second trend data from dataviewer for which minute trend data *is* available I get the following error message:

Connecting to NDS Server fb (TCP port 8088)
Connecting.... done
No data found

read(); errno=9
read(); errno=9
T0=12-04-04-02-14-29; Length=3600 (s)
No data output.

Awaiting more help from Alex...

It looks like this is actually just a limit of how long we're saving the second trends, which is just not that long.  I'll look into extending the second trend look-back.

  6562   Tue Apr 24 14:55:00 2012 JamieUpdateCDScds code paths (rtscore, userapps) have moved

NOTE: Unless you really care about what's going on under the hood, please ignore this entire post and go here: USE THE NEW RTCDS UTILITY

Quote:

  • the userapps directory is in a new place (/opt/rtcds/userapps).  Not everything in the old location was checked into the repository, so we need to check to make sure everything that needs to be is checked in, and that all the models are running the right code.

 An important new aspect of this upgrade is that we have some new directories and some of the code paths have moved to correspond with the "standard" LIGO CDS filesystem hierarchy:

  • rtscore:      /opt/rtcds/rtscore/release       -->  RTS RCG source code (svn: adcLigoRTS)
  • userapps: /opt/rtcds/userapps/release  -->  CDS userapps source for models, RCG C code, medm screens, scripts, etc (svn: cds_user_apps)
  • rtbuild:       /opt/rtcds/caltech/c1/rtbuild    -->  RCG build directory

All work should be done in the "userapps" directory, and all builds should be done in the build directory.  Some important points:

WARNING: DO NOT MODIFY ANYTHING IN RTSCORE

This is important.  The rtscore directory is now just where the RCG source code is stored.  WE NO LONGER BUILD MODELS IN THE RTS SOURCE  Please use the rtbuild directory instead.

NO MORE MODEL/CODE SYMLINKS

You don't need to link you model or code anywhere to get it to compile.  The RCG now uses proper search paths to source in the RCG_LIB_PATH to find the needed source.  This has been configured by the admin, so as long as you put your code in the right place it should be found.

ALL CODE/MODELS/ETC GO IN USERAPPS

All RTS code is now stored ENTIRELY in the userapps directory (e.g. /opt/rtcds/userapps/release/isc/c1/models/c1lsc.mdl).  This is more-or-less the same as before, except that symlinking is no longer necessary.  I have placed a symlink at the old location for convenience.

BUILD ALL MODELS IN RTSCORE

You can run "make c1lsc" in the rtbuild directory, just as you used to in the old core directory.  However, don't do that.  USE THE NEW RTCDS UTILITY instead.

  6563   Tue Apr 24 16:15:24 2012 DenUpdatePEMmicrophones

I've installed Blue Bird microphone to listen to the acoustic noise at the PSL near PMC.

DSC_4271.JPG     DSC_4272.JPG

Coherence between MC_F and Blue Bird output (C1:PEM-ACC_MC2_Z for now) is changing from low to high value at frequencies 20 - 200 Hz with period ~1 min. Maybe HEPA works with some periodicity. Now it works pretty hard, ~80% of max.

micro_mc_low.png    micro_mc_high.png

  6564   Tue Apr 24 22:16:59 2012 DenUpdatePEMBlue Bird Pre-Amplifier

 I detached Clyde (pre-amp that was in the control room) to understand why it is not working. I seems that the board circuit is burnt near R40, R47, R45.

DSC_4273.JPG     DSC_4274.JPG

  6565   Wed Apr 25 00:20:01 2012 DenUpdatePEMacoustic noise at 40m

 Blue Bird Mic is suspended close to PMC now and outputs ~10 counts when pre-amp gain is 8 dB. This means that the mic outputs ~2.42 mV. Its sensitivity is 27 mV/Pa => acoustic noise is ~0.1 Pa or ~75 dB SPL.

If we buy Panasonic WM61A with their sensitivity -35 dB => they will output ~1.7 mV. We can amplify this signal without adding significant noise. For WM61A S/N ratio is given to be 62 dB. This is for some standard signal that is not specified. For Blue Bird mic it is specified according to IEC 651. So I assume SPL of the standard signal = 94 dB => noise level of WM61A is 32 dB (pretty bad compared to 7 dB-A of Blue Bird). But in our case for PSL S/N ratio is ~43 dB that is not too bad. PSL is noisy due to HEPA, acoustic noise level close to MC2 stack will be less. So we may want to consider Primo EM172/173 where the noise level is claimed to be 18 dB less. I think we should buy several WM61A and EM172.

  6566   Wed Apr 25 11:45:34 2012 JenneUpdatePEMBlue Bird Pre-Amplifier

It looks like we should change the "R40" and "R47" diodes.  If you do it this week, ask Jamie or Koji to check that you've got them oriented correctly before soldering them and plugging it in.

  6567   Wed Apr 25 18:59:47 2012 ranaUpdatePEMBlue Bird Pre-Amplifier

Usually R is for resistors and D is for diodes. Do you think from the schematic that we should put diodes into the R slots?

  6568   Wed Apr 25 19:32:57 2012 DenUpdatePEMBlue Bird Pre-Amplifier

Quote:

Usually R is for resistors and D is for diodes. Do you think from the schematic that we should put diodes into the R slots?

 That guys are resistors.

  6569   Wed Apr 25 19:36:19 2012 DenUpdatePSLPMC aligned

[Koji, Den]

We have aligned PMC,  the WFS are not working yet.

  6570   Wed Apr 25 21:24:10 2012 DenUpdateComputer Scripts / Programsc1oaf

C1OAF model, codes and medm screens are updated. All proper files are commited to svn and updated at the new model path.

  6571   Thu Apr 26 09:22:17 2012 steveUpdate under the east end optical table

Quote:

I added an U channel based bottom shelf at the south end today.

 I'm working on similar shelf at ETMY.  Precondition: NPRO in bypass mode, heater for doubling in bypass........since power outage?  Optical level servo turned off.......

U-channel based shelf in place.  Oplev servo is back on at 11:15am    The table may moved.  The oplev return is missing the quad by a few milimeter.

  6572   Thu Apr 26 11:56:10 2012 DenUpdatePEMBlue Bird Pre-Amplifier

Quote:

Quote:

Usually R is for resistors and D is for diodes. Do you think from the schematic that we should put diodes into the R slots?

 That guys are resistors.

 You are right, they just looked like they were too small to be resistors when I glanced at them. 

  6573   Thu Apr 26 16:35:34 2012 JamieSummaryCDSrosalba now running Ubuntu 10.04

This morning I installed Ubuntu 10.04 on rosalba.  This is the same version that is running on pianosa.  The two machines should be identically configured, although rosalba may be missing some apt-getable packages.

  6574   Thu Apr 26 18:15:59 2012 JamieUpdateCDSpossible issue with mx_stream on front ends

I'm noticing what appears to be occasional failures of mx_stream on the front end machines.  It doesn't happen that frequently, but I've noticed it a couple of times already since the upgrade.

The symptom is that the DC Status goes to "0xbad" (red) and the "FE NET" goes red for all models on a given front end.

The solution seems to be restarting mx_stream on the given front end:    sudo  /etc/init.d/mx_stream restart"

There is nothing in the mx_stream log:

 controls@c1sus ~ 0$ cat /opt/rtcds/caltech/c1/target/fb/mx_stream_logs/c1sus.log 
 c1x02
 c1sus
 c1mcs
 c1rfm
 c1pem
 mmapped address is 0x7f43740ec000
 mapped at 0x7f43740ec000
 mmapped address is 0x7f43700ec000
 mapped at 0x7f43700ec000
 mmapped address is 0x7f436c0ec000
 mapped at 0x7f436c0ec000
 mmapped address is 0x7f43680ec000
 mapped at 0x7f43680ec000
 mmapped address is 0x7f43640ec000
 mapped at 0x7f43640ec000
 send len = 263596
 Connection Made

but I do see some funny messages in the front end dmesg:

 [200341.317912] DXH Adapter 0 : Heartbeat alive-check for node=12 failed (cnt=8387 state=0x1 deb=0 val=0).
 [200341.318670] DXH Adapter 0 : Session for node 12 is disabled - Status = 0x5
 [200341.319062] Session callback reason=1 status=5 target_node=12
 [200341.319069] Session callback reason=3 status=0 target_node=12
 [200341.359534] (map_table_check_access:752):my id 1 ->  remote id 2 : entry was valid - is now tentatively valid
 [200341.859584] DXH Adapter 0 : Probe failure for node=12 - disabling session probeStatus=0x40000f02
 [200341.860335] DXH Adapter 0 : Session for node 12 is disabled - Status = 0x3
 [200341.860728] Session callback reason=1 status=3 target_node=12
 [200374.006111] DXH Adapter 0 : Set reachable remote node list.
 [200409.020670] DXH Adapter 0 : Set reachable remote node list.
 [200409.021076] DXH Adapter 0 : Session for node 12 is deleted - Status = 0x0
 [200409.021468] Session callback reason=5 status=0 target_node=12
 [200412.362824] (map_table_insert:648):** successfully inserted **(valid unicast) inst 0 node 1->0 fwd 0 fwd_tp 4 egress 0
 [200418.025994] (map_table_check_access:752):my id 1 ->  remote id 0 : entry was valid - is now invalid
 [200418.025998] (map_table_insert:648):** successfully inserted **(valid unicast) inst 0 node 1->2 fwd 0 fwd_tp 4 egress 0
 [200421.743916] Session callback reason=0 status=0 target_node=12
 [200422.073776] DXH Adapter 0 : Set reachable remote node list.
 [200422.342446] Session callback reason=7 status=0 target_node=12
 [200422.342454] DXH Adapter 0 : Session for node 12 is ok.

I'm awaiting feedback from experts.

 

  6575   Thu Apr 26 18:17:56 2012 SureshUpdateIOOMC WFS: Tweaked the WFS offsets

[Jamie, Suresh]

Yesterday Den and Koji reported that the WFS loops were causing the MC to become unlocked.  They had aligned the PMC.   The input beam into the MC seems to be well aligned.  MCREFL DC close to minimum it gets while MC is locked (~0.45 V).

I checked and saw that the WFS heads and the MC2_TRANS_QPD had picked up DC offsets.  To reset them I turned off the MC_autolocker and closed the PSL shutter.

The ADC offsets were set using this script /cvs/cds/rtcds/caltech/c1/scripts/MC/WFS/WFS_QPD_offsets.  (Jamie fixed the paths to ezcaservo to get this script to work)

The WFS sensor head offsets were manually set to adjust the Q and I signals from the sensor head to zero.  (This operation is supposed to be done by a script which is available, but I will check it out before I direct people to it).

Then we noticed that the ASC outputs were turned off.  (Presumably Koji turned them off yesterday, when the MC was repeatedly unlocking due to the WFS loops).

We turned on the ASC outputs and the MC stayed locked with reasonable outputs on the WFS output channels.  (+/-100)

However, engaging the WFS servo increases the MCREFL DC signal to 0.7 V from the 0.45 V value when the servo is not engaged.  This could be because of DC offsets in the WFS servo filters.   I will adjust these offsets to maintain good MC transmission when the WFS servo is engaged.

 

 

 

  6576   Thu Apr 26 20:44:23 2012 JamieUpdateCDSdaq network failure, c1ioo failing to start models

Den tried adding a SINGLE acquire channel to the c1ioo, which for some reason hung c1ioo and took down the entire DAQ network (at least all communication between the front ends and the fb).  We recovered by restarting c1ioo and restarting mx_stream on all the rest of the front-ends

After "recovering", though, c1ioo is failing to load models, or at least it's IOP. Here is the tail of dmesg when trying to start the IOP:

[ 1751.140283] c1x03: Initializing space for daqLib buffers
[ 1751.140284] c1x03: Initializing Network
[ 1751.140285] c1x03: Found 1 frameBuilders on network
[ 1751.250658] CPU 2 is now offline
[ 1751.250657] c1x03: Sync source = 4
[ 1751.250657] c1x03: Waiting for EPICS BURT Restore = 1
[ 1751.310008] c1x03: Waiting for EPICS BURT 0
[ 1751.310008] c1x03: BURT Restore Complete
[ 1751.310008] c1x03: Initialized servo control parameters.
[ 1751.311699] c1x03: DAQ Ex Min/Max = 1 3
[ 1751.311699] c1x03: DAQ XEx Min/Max = 3 53
[ 1751.311733] c1x03: DAQ Tp Min/Max = 10001 10007
[ 1751.311733] c1x03: DAQ XTp Min/Max = 10007 10507
[ 1751.311737] c1x03: DIRECT MEMORY MODE of size 64
[ 1751.311737] c1x03: daqLib DCU_ID = 33
[ 1751.311737] c1x03: Invalid num daq chans = 0
[ 1751.311737] c1x03: DAQ init failed -- exiting

The chan file for this model (/opt/rtcds/caltech/c1/chans/daq/C1X03.ini) looks totally fine, has two un-acquired channels uncommented, and has otherwise not been touched. The C1:FEC-33_MSGDAQ is also reading: "ERROR reading DAQ file!"

I'm at a loss for what is going on. I've tried restarting every CDS process on the machine, restarting the model multiple times, restarting fb, and even restarting the entire c1ioo machine, all to no affect.

  6577   Fri Apr 27 08:11:19 2012 steveUpdateCDSc1ioo condition

 

 

Attachment 1: c1ioo.png
c1ioo.png
  6578   Fri Apr 27 09:00:15 2012 JamieUpdateCDSsus watchdogs?

Why are all the suspension watchdogs tripped?  None of the suspension models are running on c1ioo, so they should be completely unaffected.  Steve, did you find them tripped, or did you shut them off?

In either event they should be safetly turned back on.

  6579   Fri Apr 27 09:27:48 2012 DenUpdateCDSsus watchdogs?

Quote:

Why are all the suspension watchdogs tripped?  None of the suspension models are running on c1ioo, so they should be completely unaffected.  Steve, did you find them tripped, or did you shut them off?

In either event they should be safetly turned back on.

 I've turned off the coils. Though non of them are on the c1ioo, who knows what can happen when we'll try to run the models again.

  6580   Fri Apr 27 12:12:14 2012 DenUpdateCDSc1ioo is back

Rolf came to the 40m today and managed to figure out what the problem is. Reading just dmesg was not enough to solve the problem. Useful log was in

>> cat /opt/rtcds/caltech/c1/target/c1x03/c1x03epics/iocC1.log

Starting iocInit
The CA server's beacon address list was empty after initialization?
iocRun: All initialization complete
sh: iniChk.pl: command not found
Failed to load DAQ configuration file

iniChk.pl checks the .ini file of the model.

>> cat /opt/rtcds/rtscore/release/src/drv/param.c


int loadDaqConfigFile(DAQ_INFO_BLOCK *info, char *site, char *ifo, char *sys)
{

  strcpy(perlCommand, "iniChk.pl ");
  .........
  strcat(perlCommand, fname); // fname - name of the .ini file
  ..........
}

So the problem was not in the C1X03.ini. The code could not find the perl script though it was in the /opt/rtcds/caltech/c1/scripts directory. Some environment variables are not set. Rolf added /opt/rtcds/caltech/c1/scripts/ to $PATH variable and c1ioo models (x03, ioo, gcv) started successfully. He is not sure whether this is a right way or not, because other machines also do not have "scripts" directory in their PATH variable.

>> cat /opt/rtcds/caltech/c1/target/c1x03/c1x03epics/iocC1.log

Starting iocInit
The CA server's beacon address list was empty after initialization?
iocRun: All initialization complete

Total count of 'acquire=0' is 2
Total count of 'acquire=1' is 0
Total count of 'acquire=0' and 'acquire=1' is 2

Counted 0 entries of datarate=256     for a total of 0
Counted 0 entries of datarate=512     for a total of 0
Counted 0 entries of datarate=1024     for a total of 0
Counted 0 entries of datarate=2048     for a total of 0
Counted 0 entries of datarate=4096     for a total of 0
Counted 0 entries of datarate=8192     for a total of 0
Counted 0 entries of datarate=16384     for a total of 0
Counted 0 entries of datarate=32768     for a total of 0
Counted 2 entries of datarate=65536     for a total of 131072

Total data rate is 524288 bytes - OK

Total error count is 0

Rolf mentioned about automatic set up of variables - kiis of smth like that - probably that script is not working correctly. Rolf will add this problem to his list.

  6581   Fri Apr 27 13:32:06 2012 DenUpdatePEMseism channels

A few weeks ago I found that GUR2_X signal is biased from 0 to 800 counts in average. I decided that the corresponding channel in the readout box is bad - adds DC voltage to the signal. I stopped using GUR2_XYZ channels of the seism readout box. Now the same thing happened with the GUR1_XYZ channels. I checked the signals coming out from the seism box with the oscilloscope and they were fine. So the problem is not in the readout box. Then I applied 1 V sine wave to the input of AA board to the GUR1_X and ACC_MC1_Z channels. GUR1_X channel still shows noise. Something is wrong with these channels inside the AA board or in the ADC.

pem.png

 

Edited by Den: GUR1_XYZ_IN1 signals are empty though GUR1_XYZ are fine. So the problem is just that GUR1_XYZ_IN1 are not acquired for now though some of the ACC_IN1 channels contain the signal. I need to correct .ini files.

  6582   Mon Apr 30 13:00:50 2012 SureshUpdateCDSFrame Builder is down

Frame builder is down.  PRM has tripped its watch dogs.  I have reset the watch dog on PRM and turned on the OPLEV. It has damped down.  Unable to check what happened since FB is not responding.

There was an minor earthquake yesterday morning which people could feel a few blocks away.  It could have caused the the PRM to unlock.

Jamie,Rolf,  is it okay or us to restart the FB?  

  6583   Mon Apr 30 13:58:25 2012 JamieUpdateCDSFrame Builder is down

Quote:

Frame builder is down.  PRM has tripped its watch dogs.  I have reset the watch dog on PRM and turned on the OPLEV. It has damped down.  Unable to check what happened since FB is not responding.

There was an minor earthquake yesterday morning which people could feel a few blocks away.  It could have caused the the PRM to unlock.

Jamie,Rolf,  is it okay or us to restart the FB?  

 If it's down it's alway ok to restart it.  If it doesn't respond or immediately crashes again after restart then it might require some investigation, but it should always be ok to restart it.

  6584   Mon Apr 30 16:56:05 2012 SureshUpdateCDSFrame Builder is down

Quote:

Quote:

Frame builder is down.  PRM has tripped its watch dogs.  I have reset the watch dog on PRM and turned on the OPLEV. It has damped down.  Unable to check what happened since FB is not responding.

There was an minor earthquake yesterday morning which people could feel a few blocks away.  It could have caused the the PRM to unlock.

Jamie,Rolf,  is it okay or us to restart the FB?  

 If it's down it's alway ok to restart it.  If it doesn't respond or immediately crashes again after restart then it might require some investigation, but it should always be ok to restart it.

I tried restarting the fb in two different ways.  Neither of them re-established the connection to dtt or epics.

1) I restarted the fb from the control room console with the 'shutdown' command.  No change.

2) I halted the machine with 'shutdown -h now' and restarted it with the hardware reset button on its front-panel. No change.

The console connected to the fb showed that the network file systems did not load.  Could this have resulted in failure to start several services since it could not find the files which are stored on the network file system?

The fb is otherwise healthy since I am able to ssh into it and browse the directory structure.

  6585   Mon Apr 30 18:46:34 2012 ranaUpdateComputersmegatron

Last week I found that megatron was still off after the power outage. Apparently, the power recovery checklist had not been followed during the recovery.

Please remember to use the checklist and post the checklist results after each power outage. Megatron is now on and functioning.

  6586   Mon Apr 30 20:43:33 2012 SureshUpdateCDSFrame Builder is down

Quote:

Quote:

Quote:

Frame builder is down.  PRM has tripped its watch dogs.  I have reset the watch dog on PRM and turned on the OPLEV. It has damped down.  Unable to check what happened since FB is not responding.

There was an minor earthquake yesterday morning which people could feel a few blocks away.  It could have caused the the PRM to unlock.

Jamie,Rolf,  is it okay or us to restart the FB?  

 If it's down it's alway ok to restart it.  If it doesn't respond or immediately crashes again after restart then it might require some investigation, but it should always be ok to restart it.

I tried restarting the fb in two different ways.  Neither of them re-established the connection to dtt or epics.

1) I restarted the fb from the control room console with the 'shutdown' command.  No change.

2) I halted the machine with 'shutdown -h now' and restarted it with the hardware reset button on its front-panel. No change.

The console connected to the fb showed that the network file systems did not load.  Could this have resulted in failure to start several services since it could not find the files which are stored on the network file system?

The fb is otherwise healthy since I am able to ssh into it and browse the directory structure.

[Mike, Rana]

The fb is okay.  Rana found that it works on Pianosa, but not on Allegra or Rossa.  It also works on Rosalba, on which Jamie recently installed Ubuntu.

The white fields on the medm  'Status' screen for fb are an unrelated problem.

 

 

  6587   Mon Apr 30 21:05:32 2012 JenneUpdateSUSETMY oplev power supply dead

[Jenne, MikeJ]

The ETMY oplev quad sum was ~0, so we went to investigate.  The power supply was dead.  Keying it on and off didn't fix things, and it was certainly getting power since the front indicator light was flickering.  We replaced it with a different power supply, and the oplev is back.

Also, the Y-green AUX laser was off, presumably from the power outage a while back.  I turned it back on.

 

EDIT JD:  The first power supply we used didn't work....the oplev was on for a few minutes, then went off again.  We switched to one of the new (still JDSU) power supplies from Edmund Optics, and it's been happily working for at least an hour now.

  6588   Mon Apr 30 21:24:43 2012 KojiUpdateSUSETMY oplev power supply dead

And has the temp controller of the green been recovered too? (Push enable button)

 

  6589   Mon Apr 30 22:41:34 2012 JenneUpdateSUSETMY oplev power supply dead

Quote:

And has the temp controller of the green been recovered too? (Push enable button)

 

 Just now, yes.

  6590   Mon Apr 30 22:58:57 2012 JenneUpdateElectronics11MHz Marconi set to default after power outage

After a power outage, a Marconi comes back to it's defaults.  It needed to be reset to the values in elog 5530.  I'm putting a label on the Marconi so we don't have to look it up next time.

Before fixing the Marconi, POY11, AS11 and AS55 all looked like noise, no real signals, even though the arm is flashing.  Now they all look PDH-y, so things are better.

  6591   Tue May 1 08:18:50 2012 JamieUpdateCDSFrame Builder is down

Quote:

 

I tried restarting the fb in two different ways.  Neither of them re-established the connection to dtt or epics.

 Please be conscious of what components are doing what.  The problem you were experiencing was not "frame builder down".  It was "dtt not able to connect to frame builder".  Those are potentially completely different things.  If the front end status screens show that the frame builder is fine, then it's probably not the frame builder.

Also "epics" has nothing whatsoever to do with any of this.  That's a completely different set of stuff, unrelated to DTT or the frame builder.

  6592   Tue May 1 17:42:15 2012 Mike J.UpdateComputersSensoray

I've upgraded the Sensoray GUI so it can now switch the video channel it receives, thanks to the videoswitch script.

V4L2_Capture_Demo_r01.png

  6593   Wed May 2 19:47:20 2012 DenUpdatePEMEM 172 nonlinearities

I've checked new small EM172 microphones for nonlinearities using Koji's Mac Book speakers. EM172 + Mac nonlinearities are presented at the figure

500Hz_noise.png

The Mac's sound frequency was specified to be 500 Hz. Harmonics are seen at 1k, 1.5k, but their amplitude is ~30 times less.

  6594   Wed May 2 21:04:06 2012 DenUpdatePEMEM 172 coherence

I measured coherence between 2 EM 172 microphones in a "quiet room" with SR785

em_coh.png

High-frequency noise (>2k) is SR785 noise - I'm not using any amplifier now, the signal from microphone is sent directly to SR785 and is weak at high frequencies.

  6595   Thu May 3 13:07:05 2012 KojiBureaucracyGeneralInterferometer check-up session

From 2PM, Jenne/Den/Koji. Finished at 10:30PM

  • General
    • More LCD screens! Wider screens!
      • Allegra - only can handle single head right now, but should prepare for the eventual upgrade of that videocard/whole computer and get 2 monitors.
      • Every workstation should have 2 24" monitors.  We currently have 2 good ones (one on Rossa, one on Pianosa), so we need 6.
    • Remove Control room shelf currently holding OSA 'scopes
    • Adjust both TV shelves so that they're at the same height, ~6" higher than the current left shelf.
      This gives space for stacking the 24" monitors for each computer.
    • Audio system for the signals!!!! Even a crappy one!
           /cvs/cds/ligo/centos5.5/cds/project/gds-2.14.2/Monitors/rockIFO
           /cvs/cds/gds/gds/Monitors/rockIFO
           (really /cvs/cds, not /opt)

  • IOO
    • PZT or Picomotor mounts for PSL/ALS beams
    • ALS on the both arm simultaneously / common/diff ALS scripts
  • SUS
    • PRM OPLEV servos are sometimes not stable. (fixed, JCD/DM/KA, 5/2/2012)
    • ETMX oplev - replace 1064nm lens with 632nm lens (KBC037, -100)
  • IFO
    • It seems that there is an occasional common-mode power transient in the arm transmissions.
      -> Track down the cause and fix.
    • Fix arm ASS even on CentOS
    • Drift of the green incident axis -> Assess the amount of the drift / replace the mount
    • Calibration of POP22 / AS110
    • PMC/IMC/ARM characterization (loss, finesse, reflectivity, etc)
  • CDS
    • Capture OSA signals in CDS (the 'scope TDS1001B has a USB port in the back for connecting to the computer)
    • Transmon (arms) for high and low power
    • POX11 whitening is not toggling the analog whitening???
    • CentOS machines cannot open simulink models properly (get "Bad Link"s everywhere, so you can't do anything useful).
    • Dataviewer and DTT can't connect to framebuilder from CentIOS machines
  • Electronics
    • Actuator noise level characterization (coil driver response in V/m & coil driver noise level V/rtHz)
    • Beat box
    • I/Q DFD
    • Improvement of POP22/110/AS110 RF circuits.
  • MEDM
    • OL alert in the watch dog screen (this is awesome!) should have small text saying "OL" (done, JCD, 5May2012)
    • Complete 40m overview screen - everything should be clickable with pseudo 3D icons
    • BETTER SUSPENSION SCREEN!!!
    • Script to generate a MEDM tree
    • Resurrect MEDM snapshots
       
    • The LSC screen has some "white" boxes -> investigate and fix them (done, KA, 5/2/2012)
    • Make the LSC control screen (compact) in the vertical arrangement (done, KA, 5/2/2012)
    • The signals of the watch dog screen should go red if any of the WDs are not enabled.
      =>Copy the logic of the signals in LSC screen.
      (done, KA, 5/2/2012)
    • Remove c1gfd from the cds status screen (done, KA, 5/2/2012)
    • Add "BURT" status indicators on the cds status screen (done, KA, 5/2/2012)
  • SCRIPT
    • Locking scripts with integrator triggers / disabling when unlocked
    • Daily diagnosis of the MC spot positions (there must be something already...)
    • Daily/occasional adjustment of the incident axis on the MC
    • Suspension "misalign/restore" script is too wild => make a new script (Done, JCD, 7May2012)
    • IFO_CONFIGURE scripts still do a burt restore, so step the optics.  Need to remove optic alignment from config .snap files, use reg restore script instead.
    • OPLEV/OSEM trending script before the IFO work for diagnosis.
    • Auto-locker for PMC/Arm/etc
  • Video
    • If each video screen has a caption, that would be great
    • GUI interface of "videoswitch" Mike!
    • Mouse Jiggler for Zita (called Swinput?)
  • Ubuntu
    • burttoday is not aliased in ubuntu. burttoday:      aliased to cd /opt/rtcds/caltech/c1/burt/autoburt/today/
    • ASS doesn't run on Ubuntu!
      ezcawrite: error while loading shared libraries: libca.so: cannot open shared object file: No such file or directory
      ezcaswitch: error while loading shared libraries: libca.so: cannot open shared object file: No such file or directory
       
ELOG V3.1.3-