40m QIL Cryo_Lab CTN SUS_Lab TCS_Lab OMC_Lab CRIME_Lab FEA ENG_Labs OptContFac Mariner WBEEShop
  40m Log, Page 214 of 344  Not logged in ELOG logo
ID Date Author Type Category Subject
  6576   Thu Apr 26 20:44:23 2012 JamieUpdateCDSdaq network failure, c1ioo failing to start models

Den tried adding a SINGLE acquire channel to the c1ioo, which for some reason hung c1ioo and took down the entire DAQ network (at least all communication between the front ends and the fb).  We recovered by restarting c1ioo and restarting mx_stream on all the rest of the front-ends

After "recovering", though, c1ioo is failing to load models, or at least it's IOP. Here is the tail of dmesg when trying to start the IOP:

[ 1751.140283] c1x03: Initializing space for daqLib buffers
[ 1751.140284] c1x03: Initializing Network
[ 1751.140285] c1x03: Found 1 frameBuilders on network
[ 1751.250658] CPU 2 is now offline
[ 1751.250657] c1x03: Sync source = 4
[ 1751.250657] c1x03: Waiting for EPICS BURT Restore = 1
[ 1751.310008] c1x03: Waiting for EPICS BURT 0
[ 1751.310008] c1x03: BURT Restore Complete
[ 1751.310008] c1x03: Initialized servo control parameters.
[ 1751.311699] c1x03: DAQ Ex Min/Max = 1 3
[ 1751.311699] c1x03: DAQ XEx Min/Max = 3 53
[ 1751.311733] c1x03: DAQ Tp Min/Max = 10001 10007
[ 1751.311733] c1x03: DAQ XTp Min/Max = 10007 10507
[ 1751.311737] c1x03: DIRECT MEMORY MODE of size 64
[ 1751.311737] c1x03: daqLib DCU_ID = 33
[ 1751.311737] c1x03: Invalid num daq chans = 0
[ 1751.311737] c1x03: DAQ init failed -- exiting

The chan file for this model (/opt/rtcds/caltech/c1/chans/daq/C1X03.ini) looks totally fine, has two un-acquired channels uncommented, and has otherwise not been touched. The C1:FEC-33_MSGDAQ is also reading: "ERROR reading DAQ file!"

I'm at a loss for what is going on. I've tried restarting every CDS process on the machine, restarting the model multiple times, restarting fb, and even restarting the entire c1ioo machine, all to no affect.

  6575   Thu Apr 26 18:17:56 2012 SureshUpdateIOOMC WFS: Tweaked the WFS offsets

[Jamie, Suresh]

Yesterday Den and Koji reported that the WFS loops were causing the MC to become unlocked.  They had aligned the PMC.   The input beam into the MC seems to be well aligned.  MCREFL DC close to minimum it gets while MC is locked (~0.45 V).

I checked and saw that the WFS heads and the MC2_TRANS_QPD had picked up DC offsets.  To reset them I turned off the MC_autolocker and closed the PSL shutter.

The ADC offsets were set using this script /cvs/cds/rtcds/caltech/c1/scripts/MC/WFS/WFS_QPD_offsets.  (Jamie fixed the paths to ezcaservo to get this script to work)

The WFS sensor head offsets were manually set to adjust the Q and I signals from the sensor head to zero.  (This operation is supposed to be done by a script which is available, but I will check it out before I direct people to it).

Then we noticed that the ASC outputs were turned off.  (Presumably Koji turned them off yesterday, when the MC was repeatedly unlocking due to the WFS loops).

We turned on the ASC outputs and the MC stayed locked with reasonable outputs on the WFS output channels.  (+/-100)

However, engaging the WFS servo increases the MCREFL DC signal to 0.7 V from the 0.45 V value when the servo is not engaged.  This could be because of DC offsets in the WFS servo filters.   I will adjust these offsets to maintain good MC transmission when the WFS servo is engaged.

 

 

 

  6574   Thu Apr 26 18:15:59 2012 JamieUpdateCDSpossible issue with mx_stream on front ends

I'm noticing what appears to be occasional failures of mx_stream on the front end machines.  It doesn't happen that frequently, but I've noticed it a couple of times already since the upgrade.

The symptom is that the DC Status goes to "0xbad" (red) and the "FE NET" goes red for all models on a given front end.

The solution seems to be restarting mx_stream on the given front end:    sudo  /etc/init.d/mx_stream restart"

There is nothing in the mx_stream log:

 controls@c1sus ~ 0$ cat /opt/rtcds/caltech/c1/target/fb/mx_stream_logs/c1sus.log 
 c1x02
 c1sus
 c1mcs
 c1rfm
 c1pem
 mmapped address is 0x7f43740ec000
 mapped at 0x7f43740ec000
 mmapped address is 0x7f43700ec000
 mapped at 0x7f43700ec000
 mmapped address is 0x7f436c0ec000
 mapped at 0x7f436c0ec000
 mmapped address is 0x7f43680ec000
 mapped at 0x7f43680ec000
 mmapped address is 0x7f43640ec000
 mapped at 0x7f43640ec000
 send len = 263596
 Connection Made

but I do see some funny messages in the front end dmesg:

 [200341.317912] DXH Adapter 0 : Heartbeat alive-check for node=12 failed (cnt=8387 state=0x1 deb=0 val=0).
 [200341.318670] DXH Adapter 0 : Session for node 12 is disabled - Status = 0x5
 [200341.319062] Session callback reason=1 status=5 target_node=12
 [200341.319069] Session callback reason=3 status=0 target_node=12
 [200341.359534] (map_table_check_access:752):my id 1 ->  remote id 2 : entry was valid - is now tentatively valid
 [200341.859584] DXH Adapter 0 : Probe failure for node=12 - disabling session probeStatus=0x40000f02
 [200341.860335] DXH Adapter 0 : Session for node 12 is disabled - Status = 0x3
 [200341.860728] Session callback reason=1 status=3 target_node=12
 [200374.006111] DXH Adapter 0 : Set reachable remote node list.
 [200409.020670] DXH Adapter 0 : Set reachable remote node list.
 [200409.021076] DXH Adapter 0 : Session for node 12 is deleted - Status = 0x0
 [200409.021468] Session callback reason=5 status=0 target_node=12
 [200412.362824] (map_table_insert:648):** successfully inserted **(valid unicast) inst 0 node 1->0 fwd 0 fwd_tp 4 egress 0
 [200418.025994] (map_table_check_access:752):my id 1 ->  remote id 0 : entry was valid - is now invalid
 [200418.025998] (map_table_insert:648):** successfully inserted **(valid unicast) inst 0 node 1->2 fwd 0 fwd_tp 4 egress 0
 [200421.743916] Session callback reason=0 status=0 target_node=12
 [200422.073776] DXH Adapter 0 : Set reachable remote node list.
 [200422.342446] Session callback reason=7 status=0 target_node=12
 [200422.342454] DXH Adapter 0 : Session for node 12 is ok.

I'm awaiting feedback from experts.

 

  6573   Thu Apr 26 16:35:34 2012 JamieSummaryCDSrosalba now running Ubuntu 10.04

This morning I installed Ubuntu 10.04 on rosalba.  This is the same version that is running on pianosa.  The two machines should be identically configured, although rosalba may be missing some apt-getable packages.

  6572   Thu Apr 26 11:56:10 2012 DenUpdatePEMBlue Bird Pre-Amplifier

Quote:

Quote:

Usually R is for resistors and D is for diodes. Do you think from the schematic that we should put diodes into the R slots?

 That guys are resistors.

 You are right, they just looked like they were too small to be resistors when I glanced at them. 

  6571   Thu Apr 26 09:22:17 2012 steveUpdate under the east end optical table

Quote:

I added an U channel based bottom shelf at the south end today.

 I'm working on similar shelf at ETMY.  Precondition: NPRO in bypass mode, heater for doubling in bypass........since power outage?  Optical level servo turned off.......

U-channel based shelf in place.  Oplev servo is back on at 11:15am    The table may moved.  The oplev return is missing the quad by a few milimeter.

  6570   Wed Apr 25 21:24:10 2012 DenUpdateComputer Scripts / Programsc1oaf

C1OAF model, codes and medm screens are updated. All proper files are commited to svn and updated at the new model path.

  6569   Wed Apr 25 19:36:19 2012 DenUpdatePSLPMC aligned

[Koji, Den]

We have aligned PMC,  the WFS are not working yet.

  6568   Wed Apr 25 19:32:57 2012 DenUpdatePEMBlue Bird Pre-Amplifier

Quote:

Usually R is for resistors and D is for diodes. Do you think from the schematic that we should put diodes into the R slots?

 That guys are resistors.

  6567   Wed Apr 25 18:59:47 2012 ranaUpdatePEMBlue Bird Pre-Amplifier

Usually R is for resistors and D is for diodes. Do you think from the schematic that we should put diodes into the R slots?

  6566   Wed Apr 25 11:45:34 2012 JenneUpdatePEMBlue Bird Pre-Amplifier

It looks like we should change the "R40" and "R47" diodes.  If you do it this week, ask Jamie or Koji to check that you've got them oriented correctly before soldering them and plugging it in.

  6565   Wed Apr 25 00:20:01 2012 DenUpdatePEMacoustic noise at 40m

 Blue Bird Mic is suspended close to PMC now and outputs ~10 counts when pre-amp gain is 8 dB. This means that the mic outputs ~2.42 mV. Its sensitivity is 27 mV/Pa => acoustic noise is ~0.1 Pa or ~75 dB SPL.

If we buy Panasonic WM61A with their sensitivity -35 dB => they will output ~1.7 mV. We can amplify this signal without adding significant noise. For WM61A S/N ratio is given to be 62 dB. This is for some standard signal that is not specified. For Blue Bird mic it is specified according to IEC 651. So I assume SPL of the standard signal = 94 dB => noise level of WM61A is 32 dB (pretty bad compared to 7 dB-A of Blue Bird). But in our case for PSL S/N ratio is ~43 dB that is not too bad. PSL is noisy due to HEPA, acoustic noise level close to MC2 stack will be less. So we may want to consider Primo EM172/173 where the noise level is claimed to be 18 dB less. I think we should buy several WM61A and EM172.

  6564   Tue Apr 24 22:16:59 2012 DenUpdatePEMBlue Bird Pre-Amplifier

 I detached Clyde (pre-amp that was in the control room) to understand why it is not working. I seems that the board circuit is burnt near R40, R47, R45.

DSC_4273.JPG     DSC_4274.JPG

  6563   Tue Apr 24 16:15:24 2012 DenUpdatePEMmicrophones

I've installed Blue Bird microphone to listen to the acoustic noise at the PSL near PMC.

DSC_4271.JPG     DSC_4272.JPG

Coherence between MC_F and Blue Bird output (C1:PEM-ACC_MC2_Z for now) is changing from low to high value at frequencies 20 - 200 Hz with period ~1 min. Maybe HEPA works with some periodicity. Now it works pretty hard, ~80% of max.

micro_mc_low.png    micro_mc_high.png

  6562   Tue Apr 24 14:55:00 2012 JamieUpdateCDScds code paths (rtscore, userapps) have moved

NOTE: Unless you really care about what's going on under the hood, please ignore this entire post and go here: USE THE NEW RTCDS UTILITY

Quote:

  • the userapps directory is in a new place (/opt/rtcds/userapps).  Not everything in the old location was checked into the repository, so we need to check to make sure everything that needs to be is checked in, and that all the models are running the right code.

 An important new aspect of this upgrade is that we have some new directories and some of the code paths have moved to correspond with the "standard" LIGO CDS filesystem hierarchy:

  • rtscore:      /opt/rtcds/rtscore/release       -->  RTS RCG source code (svn: adcLigoRTS)
  • userapps: /opt/rtcds/userapps/release  -->  CDS userapps source for models, RCG C code, medm screens, scripts, etc (svn: cds_user_apps)
  • rtbuild:       /opt/rtcds/caltech/c1/rtbuild    -->  RCG build directory

All work should be done in the "userapps" directory, and all builds should be done in the build directory.  Some important points:

WARNING: DO NOT MODIFY ANYTHING IN RTSCORE

This is important.  The rtscore directory is now just where the RCG source code is stored.  WE NO LONGER BUILD MODELS IN THE RTS SOURCE  Please use the rtbuild directory instead.

NO MORE MODEL/CODE SYMLINKS

You don't need to link you model or code anywhere to get it to compile.  The RCG now uses proper search paths to source in the RCG_LIB_PATH to find the needed source.  This has been configured by the admin, so as long as you put your code in the right place it should be found.

ALL CODE/MODELS/ETC GO IN USERAPPS

All RTS code is now stored ENTIRELY in the userapps directory (e.g. /opt/rtcds/userapps/release/isc/c1/models/c1lsc.mdl).  This is more-or-less the same as before, except that symlinking is no longer necessary.  I have placed a symlink at the old location for convenience.

BUILD ALL MODELS IN RTSCORE

You can run "make c1lsc" in the rtbuild directory, just as you used to in the old core directory.  However, don't do that.  USE THE NEW RTCDS UTILITY instead.

  6561   Tue Apr 24 14:35:37 2012 JamieUpdateCDSlimited second trend lookback

Quote:

Alex told me that the "trend data is not available" message comes from the "trender" functionality not being enabled in daqd.  After re-enabling it (see #6555) minute trend data was available again.  However, there still seems to be an issue with second trends.  When I try to retrieve second trend data from dataviewer for which minute trend data *is* available I get the following error message:

Connecting to NDS Server fb (TCP port 8088)
Connecting.... done
No data found

read(); errno=9
read(); errno=9
T0=12-04-04-02-14-29; Length=3600 (s)
No data output.

Awaiting more help from Alex...

It looks like this is actually just a limit of how long we're saving the second trends, which is just not that long.  I'll look into extending the second trend look-back.

  6560   Tue Apr 24 14:30:08 2012 JamieUpdateCDSIntroducing: rtcds, a utility for interacting with the CDS RTS/RCG

The new rtcds utility

I have written a new utility for interacting with the CDS RTS/RCG system: "rtcds".  It should be available on all workstations and front-end machines, but certain commands are restricted to run on certain front end machines (build, start, stop, etc.).  Here's the help:

controls@c1lsc ~ 0$ rtcds help
Usage: rtcds <command> [arg]

Available commands:

  build|make SYS      build model
  install SYS         install model
  start SYS|all       start model
  restart SYS|all     restart running model
  stop|kill SYS|all   stop running model
  list                list all models for current host

controls@c1lsc ~ 0$ 

Please use this new utility from now on when interacting with RTS.

I'll be improving and expanding it as we go.   Please let me know if you encounter any problems.

 

  6559   Tue Apr 24 11:27:51 2012 steveUpdatePEMnew chairs

We received 6 new  chairs stools for the work benches that have 36" heights.

The control room east side and general office desks require 18" seat height. The new chairs stools have  minimum seat heights  26"

Please label chairs that we are getting rid of.

P4241026.JPG

  6558   Tue Apr 24 09:58:37 2012 steveUpdatePEMearthquake 3.9 magnitude

Local eq shakes the lab

Attachment 1: eq3.9.png
eq3.9.png
  6557   Mon Apr 23 23:20:07 2012 DenUpdatePEMmicrophones

Tonight I wanted to measure the ambient noise level using Blue Bird mics and figure out if Panasonic WM61a or Primo EM172/173 will be good enough or not. Blue Bird that is in the control room does not seem to work. May be the problem is with the pre-amplifier. The output measured by ADC/Oscilloscope is noise ( amplitude=5mV ). I will return to this issue tomorrow.

  6556   Sat Apr 21 21:10:34 2012 JamieUpdateCDStrend frame issue partially resolved

Quote:

Unfortunately it looks like there may be a problem with trend data, though.  If I try to retrieve 1 minute of "full data" with dataviewer for channel C1:SUS-ITMX_SUSPOS_IN1_DQ around GPS 1019089138 everything works fine:

Connecting to NDS Server fb (TCP port 8088)
Connecting.... done
T0=12-04-01-00-17-45; Length=60 (s)
60 seconds of data displayed

but if I specify any trend data (second, minute, etc.) I get the following:

Connecting to NDS Server fb (TCP port 8088)
Connecting.... done
Server error 18: trend data is not available
datasrv: DataWriteTrend failed in daq_send().
T0=12-04-01-00-17-45; Length=60 (s)
No data output.

Alex warned me that this might have happened when I was trying to test the new daqd without first turning off frame writing.

Alex told me that the "trend data is not available" message comes from the "trender" functionality not being enabled in daqd.  After re-enabling it (see #6555) minute trend data was available again.  However, there still seems to be an issue with second trends.  When I try to retrieve second trend data from dataviewer for which minute trend data *is* available I get the following error message:

Connecting to NDS Server fb (TCP port 8088)
Connecting.... done
No data found

read(); errno=9
read(); errno=9
T0=12-04-04-02-14-29; Length=3600 (s)
No data output.

Awaiting more help from Alex...

  6555   Sat Apr 21 20:40:28 2012 JamieUpdateCDSframebuilder frame writing working again

Quote:

A big one is that the framebuilder daqd started going haywire when I told it to start writing frames.  After restart the logs started showing this:

[Fri Apr 20 17:23:40 2012] main profiler warning: 0 empty blocks in the buffer
[Fri Apr 20 17:23:41 2012] main profiler warning: 0 empty blocks in the buffer
[Fri Apr 20 17:23:42 2012] main profiler warning: 0 empty blocks in the buffer
[Fri Apr 20 17:23:43 2012] main profiler warning: 0 empty blocks in the buffer
[Fri Apr 20 17:23:44 2012] main profiler warning: 0 empty blocks in the buffer
[Fri Apr 20 17:23:45 2012] main profiler warning: 0 empty blocks in the buffer
GPS time jumped from 1019002442 to 1019003041
FATAL: exception not rethrown
FATAL: exception not rethrown
FATAL: exception not rethrown

and the network seemed like it started to get really slow.  I wasn't able to figure out what was going on, so I shut the frame writing off again.  I'll have to work with Rolf on that next week.

So the framebuilder/daqd frame writing issue seems to have been a transient one.  Alex said he tried enabling frame writing manually and it worked fine, so I tried re-enabling the frame writing config lines and sure enough it worked fine.  So it's a mystery.  Alex said the "main profiler warning" lines tend to show up when data is backed up in the buffer, although he didn't explain why exactly that would have been the issue here.

daqdrc frame writing directives:

start frame-saver;
sync frame-saver;
start trender;
start trend-frame-saver;
sync trend-frame-saver;
start minute-trend-frame-saver;
sync minute-trend-frame-saver;
start raw_minute_trend_saver;
  6554   Sat Apr 21 17:38:19 2012 JamieUpdateCDSdtt, dataviewer working; problem with trend frames

Quote:

Another big problem is the workstation application upgrades.  The NDS protocol version has been incremented, which means that all the NDS client applications have to be upgraded.  The new dataviewer is working fine (on pianosa), but dtt is not:

controls@pianosa:~ 0$ diaggui
diaggui: symbol lookup error: /ligo/apps/linux-x86_64/gds-2.15.1/lib/libligogui.so.0: undefined symbol: _ZN18TGScrollBarElement11ShowMembersER16TMemberInspector
controls@pianosa:~ 127$

 dtt (diaggui) and dataviewer are now working on pianosa to retrieve realtime data and past data from DQ channels.

Unfortunately it looks like there may be a problem with trend data, though.  If I try to retrieve 1 minute of "full data" with dataviewer for channel C1:SUS-ITMX_SUSPOS_IN1_DQ around GPS 1019089138 everything works fine:

Connecting to NDS Server fb (TCP port 8088)
Connecting.... done
T0=12-04-01-00-17-45; Length=60 (s)
60 seconds of data displayed

but if I specify any trend data (second, minute, etc.) I get the following:

Connecting to NDS Server fb (TCP port 8088)
Connecting.... done
Server error 18: trend data is not available
datasrv: DataWriteTrend failed in daq_send().
T0=12-04-01-00-17-45; Length=60 (s)
No data output.

Alex warned me that this might have happened when I was trying to test the new daqd without first turning off frame writing.

I'm not sure how to check the integrity of the frames, though.  Hopefully they can help sort this out on Monday.

  6553   Fri Apr 20 23:02:25 2012 DenUpdateAdaptive Filteringfrequency domain filter

 DFT-LMS is a frequency domain adaptive filter that demonstrates faster convergence compared to the time-domain LMS filter. I've tested Discrete Fourier Transform (DFT-LMS) filter. It converts witness signal to the frequency domain using DFT and corrects the eigenvalues of the covariance matrix to make them as equal to each other as possible (does pre-whitenning of the witness signal).

Left plot compares learning curves for time domain LMS and DFT-LMS algorithms on the simulated data from seismometers and mcl (number of averages  = 30) Right plot shows the evolution of the filter coefficients norm (Euclidean norms of the coefficient vector). Though LMS algorithm works in the time domain and DFT-LMS in the frequency domain, the coefficient vectors must have the same length, because we Fourier Transform is achieved by applying a unitary operator => vector norm must not change.

dft.png   norm.png

Plots show that both algorithms converge to the same coefficients vector norm, but DFT-LMS does it much faster then LMS. 

Online realization: 

Good news: algorithm complexity is linear in filter length. Though the algorithm does Fourier transform, its complexity is still O(M), M - number of coefficients. Simulations show that DFT-LMS is ~8-9 times slower then LMS. This is not so bad, may be we can do even slightly better.

Bad news: downsample process is not simple. Due to Fourier transform, the filter needs the whole witness signal vector before calculating the output. This is sad and in contrast with LMS algorithm where we could start to calculate the new output immediately after computing the previous output. We either need to calculate the whole output immediately or introduce delay in the output or approximate Fourier transform with some previous witness signal values.

Realization in the kernel: I asked Alex about complex numbers, exponents, sin and cos functions in the kernel c and he answers that we do not have complex numbers, about exp, cos, sin he is not sure. But for DFT-LMS algorithm we are able to get round of these difficulties. Complex numbers will be presented as  2 real numbers. Then exp (a) = cos(a) + i*sin(a). All what we need for DFT-LMS are sin(2 * pi * k / M) and cos(2 * pi * k / M), k=0,1,2,...,M-1. Fortunately, M - (filter length) is big enough, typical value pi/M ~ 0.001 and we can calculate sin(2*pi/M) and cos(2*pi/M) using Taylor series. As the argument is small, 5-6 terms will be enough to get precision ~1e-20. Then we build the whole table of cos and sin according to induction cos(2*pi/M*k) = cos(2*pi/M*(k-1))cos(2*pi/M) - sin(2*pi/M*(k-1))sin(2*pi/M), sin(2*pi/M*k) = cos(2*pi/M*(k-1))sin(2*pi/M) + sin(2*pi/M*(k-1))cos(2*pi/M). We should do it only once, so the algorithm will build these values in the beginning during first several iterations, then will use them.

The main problem is downsampling. I need to think more about it.

  6552   Fri Apr 20 19:54:57 2012 JamieUpdateCDSCDS upgrade problems

I ran into a couple of snags today.

A big one is that the framebuilder daqd started going haywire when I told it to start writing frames.  After restart the logs started showing this:

[Fri Apr 20 17:23:40 2012] main profiler warning: 0 empty blocks in the buffer
[Fri Apr 20 17:23:41 2012] main profiler warning: 0 empty blocks in the buffer
[Fri Apr 20 17:23:42 2012] main profiler warning: 0 empty blocks in the buffer
[Fri Apr 20 17:23:43 2012] main profiler warning: 0 empty blocks in the buffer
[Fri Apr 20 17:23:44 2012] main profiler warning: 0 empty blocks in the buffer
[Fri Apr 20 17:23:45 2012] main profiler warning: 0 empty blocks in the buffer
GPS time jumped from 1019002442 to 1019003041
FATAL: exception not rethrown
FATAL: exception not rethrown
FATAL: exception not rethrown

and the network seemed like it started to get really slow.  I wasn't able to figure out what was going on, so I shut the frame writing off again.  I'll have to work with Rolf on that next week.

Another big problem is the workstation application upgrades.  The NDS protocol version has been incremented, which means that all the NDS client applications have to be upgraded.  The new dataviewer is working fine (on pianosa), but dtt is not:

controls@pianosa:~ 0$ diaggui
diaggui: symbol lookup error: /ligo/apps/linux-x86_64/gds-2.15.1/lib/libligogui.so.0: undefined symbol: _ZN18TGScrollBarElement11ShowMembersER16TMemberInspector
controls@pianosa:~ 127$ 

I don't know what's going on here.  All the library paths are ok.  Hopefully I'll be able to figure this out soon.  The old version of dtt definitely does not work with the new setup.

I might go ahead and upgrade some more of the workstations to Ubuntu in the next couple of days as well, so everything is more on the same page.

I also tried to cleanup the front-end boot process, which has it's own problems (models won't auto-start).  I haven't figured that out yet either.  It really needs to just be completely overhauled.

  6551   Thu Apr 19 22:18:24 2012 DenUpdateAdaptive Filteringoaf algorithm: old vs new

 Here are the issues that I found not quite accurate in the old oaf code:

1. There is no need to calculate the norm of the witness signal every time from zero: 

norm += (*pBufAdapt) * (*pBufAdapt); // add to the norm 

Every step the witness signal vector is the same except the first and last values

wit[i].norm += histAdpt[nCoeff]*histAdpt[nCoeff] - histAdpt[0]*histAdpt[0];

 This step will reduce the number of multiplications and summations from 3*M/k to 2*M/k, M - filter length, k - downsample ratio.

2. Old code filter corrects filter coefficients with a delay equal to k=downsample ratio (pretty big):

witness       o o o o o o o o o o o o o o o o o o o o o

error           o o o o o o o o o o o o o o o o o o o o o

We want the filter to work at green points and skip red points computing output and correcting coefficients at this time (downsample ratio in this example is 4). Old code

  • grabs error signal
  • calculates output during next k-1 red points and 1 green point
  • corrects coefficients using this error during next k-1 red points and 1 green point

But LMS algorithm should correct coefficients according to the latest error. As we calculate output and correct coefficients before the latest error signal will be available, we should change the order:

  • grabs error signal
  • corrects coefficients using this error during next k-1 red points and 1 green point
  • calculates output during next k-1 red points and 1 green point

This scheme is completely equivalent to the ordinary LMS algorithm, as now we correct coefficients according to the latest error signal and so do not add any delay.

3. So long soft start is probably not needed for the LMS filter, it makes the filter to converge longer 

// modify adaptation gain for soft start

    if( state.iWait < state.nFIR )

    {

      adaptGain = 0.0;

      decayRate = 1.0;  // clear FIR coeffs after reset

    }

As far as I understand this is done to prevent the filter from huge coefficients variations in the beginning when norm is not calculated yet. Instead we can just introduce some small

epsilon = 10.0;

to prevent the filter from divergence in the beginning 

delta = mu * error / (wit[i].norm + epsilon);

Though some soft start might be needed by not so long - it will take several minutes before the adaptation gain will take it's specified value. 

  6550   Thu Apr 19 16:21:04 2012 ZachUpdateComputer Scripts / ProgramsArbcav updated, made badass

I have modified Arbcav to be way cooler than it used to be.

Main modifications:

  • Can now truly model an arbitrary cavity geometry
    • The previous version could only handle a few different topologies. In each case, it would unfold the cavity into the equivalent linear cavity and use the g-parameter method to calculate gouy phases, etc.
    • The new model uses the closed cavity propagation matrix to find the supported mode, and then explicitly calculates the accumulated gouy phase by propagating the beam through the full cavity. This is done analytically with zR, so there is negligible slow-down.
  • Now plots a diagram of the cavity geometry, both to help you and for you to verify that it is calculating the right thing (<-- this is the cool part)
    • Plots the beam path and mirror locations
    • Specifies whether mirrors are curved or flat
    • Prints mirror parameters next to them
    • Finds all intracavity waist locations and plots them
    • Gives waist information (size in X, Y)

Since the information is already there, I will have the output structure include things like the input beam q parameter, which could then be fed directly to mode matching tools like ModeMatchr.

The function takes as input the same arguments as before. Example for a square cavity:

out = arbcav([200e-6 50e-6 200e-6 50e-6],[0.75 0.75 0.75 0.75],[1e10 9 1e10 9],[45 45 45 45],29.189e6,10e-6,1064e-9,1000);

i.e.,

out = arbcav(transmissivity_list, length_list, RoC_list, angle_list, modulation_freq, loss_list_or_loss_per_mirror, wavelength, num_pts_for_plot);

If you don't give it a modulation frequency, it will just plot carrier HOMs. If you don't give it RoCs and angles, it will just plot the transmission spectrum.

 

I'm still fine-tuning some functionality, but I should have it up on the SVN relatively soon. Comments or suggestions are welcome!

 

Some screenshots:

Cavity geometry plots (linear, triangular, square, bowtie):

linear.pngtriangular.pnggyro.pngbowtie.png

 

Transmission and HOM spectra (these correspond to the square cavity at lower left, above):

gyro_spect.pnggyro_HOM.png

  6549   Thu Apr 19 09:45:43 2012 SureshUpdateGeneralMC2 Oplev signals redirected for use in WFS servo

Quote:

 
Quote:
  • there are no oplev signals in MC1, MC2, and MC3

 None of the 3 MC optics have oplevs, so there shouldn't be any oplev signals. Although MC2 has the trans QPD, which was once (still is??) going through the MC2 oplev signal path.

.......

 

The MC2 Oplev signal path has been modified in the c1mcs model.  The ADC channels have been sent over the rfm into c1ioo model and are currently used in the WFS servo loops.  Please see this elog 5397.

 

  6548   Thu Apr 19 08:43:16 2012 JamieUpdateCDSoaf

Quote:

Edit: Old version (~september) of the code and oaf model is running now. In the 2.1 code there was a link from src/epics/simLink to oaf code for each DOF. It seems that 2.5 version finds models and c codes in standard directories. I need to move working code to the proper directory.

 Yes, things have changed.  Please wait until I have things cleaned up before working on models.  I'll explain what the new setup is.

  6547   Wed Apr 18 23:12:49 2012 DenUpdateCDSoaf

Adaptive filter outputs some non-zero signal in the OFF position

Screenshot.png

I turned "ON" one of them and c1lsc suspended, I've rebooted it and restarted models on c1lsc and c1sus.

Now it also outputs something non-zero though the first line of the adaptive code is "if(OFF) output=0.0; return;" May be another version of the code has been compiled.

Edit: Old version (~september) of the code and oaf model is running now. In the 2.1 code there was a link from src/epics/simLink to oaf code for each DOF. It seems that 2.5 version finds models and c codes in standard directories. I need to move working code to the proper directory.

  6546   Wed Apr 18 19:59:48 2012 JamieUpdateCDSCDS upgrade success

The upgrade is nearly complete:

  • new daqd code is running on fb
  • the fe/daqd timing issue was resolved by adjusting the GPS offset in the daqdrc.  I will document this more later.
  • the power outage conveniently rebooted all the front-end machines, so they're all now running new caRepeater
  • all models have been successfully recompiled with RCG 2.5 (with only a couple small glitches)
  • all new models are running on all front-end machines (with a couple exceptions)
  • all suspension models seem to be damping under local control (PRM is having troubles that are likely unrelated to the upgrade).
  • a lot of cleanup has been done

Remaining tasks/issues:

  • more testing OF EVERYTHING needs o be done
  • I did not yet update the DIS dolphin code, so we're running with the old code.  I don't think this is a problem, but it would be nice to get us running what they're running at the sites
  • I tried to cleanup/simplify how front-end initialization is done.  However, there is a problem and models are not auto-starting after reboot.  This needs to be fixed.
  • the userapps directory is in a new place (/opt/rtcds/userapps).  Not everything in the old location was checked into the repository, so we need to check to make sure everything that needs to be is checked in, and that all the models are running the right code.
  • the c1oaf model seems to be having a dolphin issue that needs to be sorted
  • the c1gfd model causes c1ioo to crash immediately upon being loaded.  I have removed it from the rtsystab.  That model needs to be fixed.
  • general model cleanup is in order.
  • more front-end cleanup is needed, particularly in regards to boot-up procedure.
  • document the entire upgrade procedure.

I'll finish up these remaining tasks tomorrow.

  6545   Wed Apr 18 11:54:31 2012 JenneUpdateGeneralPower outage last night

Quote:

None of the 3 MC optics have oplevs, so there shouldn't be any oplev signals. Although MC2 has the trans QPD, which was once (still is??) going through the MC2 oplev signal path.

 Duh.  Unawake brain.  The MCs look fine.

  6544   Wed Apr 18 11:46:14 2012 JenneUpdateGeneralPower outage last night

Quote:

  • there are no oplev signals in MC1, MC2, and MC3
  • Something is wrong with PRM.  He is very noisy.  Turning on his oplev servo makes him go crazy.

 None of the 3 MC optics have oplevs, so there shouldn't be any oplev signals. Although MC2 has the trans QPD, which was once (still is??) going through the MC2 oplev signal path.

PRM was noisy last week too.  But turning on his oplev shouldn't make him crazy. That's not so good. Try restoring PRM (the ! near the PRM label on the IFO align screen), then checking if his oplev is ~centered.  Maybe PRM wasn't in the nominal position at the time you restored from.

  6543   Wed Apr 18 10:05:40 2012 JamieUpdateGeneralPower outage last night

All of the front-ends are back up and I've been able to recover local control of all of the opitcs (snapshots from saturday). Issues:

  • I can't lock the PMC.  Still unclear why.
  • there are no oplev signals in MC1, MC2, and MC3
  • Something is wrong with PRM.  He is very noisy.  Turning on his oplev servo makes him go crazy.
  • There are all sorts of problems with the oplevs in general.  Many of the optics have no oplev settings.  This is probably not related to the power outage.

On a brighter note, ETMX is damped with it's new RCG 2.5 controller!  yay!

  6542   Wed Apr 18 08:53:50 2012 JamieUpdateGeneralPower outage last night

Apparently there was a catastrophic power failure last night.  Bob says it took out power in most of Pasadena.

Bob checked the vacuum system when he got in first thing this morning and everything's back up and checks out.  The laster is still off and most of the front-end computers did not recover.

I'm going to start a boot fest now.  I'll be able to report more once everything is back on.

  6541   Tue Apr 17 19:03:09 2012 JamieUpdateCDSCDS upgrade in progress

Upgrade progresses, but not complete.  There are some relatively minor issues, and one potentially big issue.

All new software has been installed, including the new epics that supports long channel names.

I've been doing a LOT of cleanup.  It was REALLY messy in there.

The new framebuilder/daqd code is running on fb.

Models are compiling with the new RCG and I am able to get them running.  Some of them are not compiling for relatively minor reasons (the simulink models need updating).  I'm also running into compile problems with IOPs that are using the dolphin drivers.

The major issue is that the framebuilder and the models are not syncing their timing, so there's no data collection.  I've spoken to Alex and he and Rolf are going to come over tomorrow to sort it out.  It's possible that we're missing timing hardware that the new code is expecting.

There are still some stability issues I haven't sorted out yet, and I have a lot more cleanup to do.

At this rate I'm going to shoot for being done Thursday.

  6540   Tue Apr 17 11:05:04 2012 JamieUpdateCDSCDS upgrade in progress

I am continuing to attempt to upgrade the CDS system to RTS 2.5.  Systems will continue to be up and down for the rest of the day.

  6539   Tue Apr 17 10:55:50 2012 RyanUpdateComputersUpdating aLIGO Conlog

Quote:

Over the next few days, I will be working on upgrading the aLIGO Conlog install to include new bugfixes distributed by Patrick T.  The currently running conlog *should* not be affected, but please let me know if it is (ryan.fisher@ligo.org).

 The upgrade to the aLIGO Conlog is completed.  The conlog is once again running on megatron in a screen session. (see http://nodus.ligo.caltech.edu:8080/40m/6396)

  6538   Mon Apr 16 14:37:01 2012 DenUpdatePEMgur2_x

Quote:

Quote:

Already not for the first time I notice that GUR2 readjusts its X zero position

As a result the coherence between GUR1X and GUR2X is lost, but between GUR2X and GUR2Y shows up. It seems that these two signals mix at some point.

 Can you go back in time before the X-position jumped to plot the x1-x2 and x2-y2 coherences?  Just to see what things look like?

DAQ is not working now but ordinary the coherence between GUR1X and GUR2X is ~1 at 0.1 - 10 Hz and between GUR2X and GUR2Y is ~0.

  6537   Mon Apr 16 09:44:54 2012 JenneUpdatePEMgur2_x

Quote:

Already not for the first time I notice that GUR2 readjusts its X zero position

As a result the coherence between GUR1X and GUR2X is lost, but between GUR2X and GUR2Y shows up. It seems that these two signals mix at some point.

 Can you go back in time before the X-position jumped to plot the x1-x2 and x2-y2 coherences?  Just to see what things look like?

  6536   Mon Apr 16 09:10:37 2012 DenUpdatePEMgur2_x

Already not for the first time I notice that GUR2 readjusts its X zero position

seisms.png

 

As a result the coherence between GUR1X and GUR2X is lost, but between GUR2X and GUR2Y shows up. It seems that these two signals mix at some point.

gur12_coh.jpg

  6535   Sat Apr 14 00:19:35 2012 SureshOmnistructureLSCOptical Fibers for insitu RFPD characterisation

   I have worked out the fibers we need to get for the following distribution scheme:

1) We have a laser placed at the 1Y1 rack.  A part of the power is split off for monitoring the laser output and sent to a broadband PD also placed in the same rack.  The RF excitation applied to the laser is split and sent to LSC rack (1Y2) and used to calibrate the full PD+Demod board system for each RFPD.

2) A single fiber goes from the laser to a 11+ way switch located in the OMC electronics cabinet next to the AP table.  From here the fibers branch out to three different tables.

Table / Rack   RF PDs on the table Number of PDs Fiber Length from OMC
The AP table AS11,AS55,AS165,REFL11,REFL33,REFL55,REFL165 7 6 m
The ITMY table POY11 1 12 m
The ITMX table POX11, POP22/110 and POP55 3 20 m

 

Cable for the laser source to the OMC table:

The 1Y1 Rack to OMC rack AM Laser Source to Switch 25 m

We also require a cable going from PSL table to the ETMY table:   This is not a part of the RFPD characterisation.  It is a part of the PSL to Y-end Aux laser lock  which is a part of the green locking scheme.  But it is  fiber we need and might as well order it now along with the rest.

PSL Table to ETMY Table PSL to ETMY Aux laser 75m

 

If you would like to add anything to this layout / scheme, please comment.  On Monday Eric is going to take a look at this and place orders for the fibers.

(I have included the lengths required for routing the fibers and added another 20% to that ) 

 

  6534   Fri Apr 13 16:09:43 2012 SureshUpdateComputer Scripts / ProgramsACAD 2002 installed on C21530

I have installed ACAD 2002 on one of the Windows machines in the Control Room.    It is on the machine which has Solid Works (called C21530). 

The installation files are in MyDocuments under Acad2002.  This a shared LIGO license which Christian Cepada had with him.

I hope we will be able to open our optical layout diagrams with this and update them even though it is an old version.

 

 

  6533   Fri Apr 13 01:27:10 2012 JenneUpdateSUSOplevs recentered

It's been a while since I think anyone has done it, and several optics were pretty far from centered, so I centered all of the oplevs except for SRM.

I am confident about my arm alignment, MICH and PRC (so BS, ITMX, ITMY, PRM, ETMX and ETMY), but I wasn't sure if I was getting SRC right, so I didn't touch the SRM's oplev.

Suresh removed all of the IPPOS measurement optics, so there was nothing blocking the BS and PRM oplevs.

However, the PRM oplev was ridiculously bad, and I don't know how long it's been that way.  Some of the optics shooting the beam into the chamber weren't optimally aligned, so the beam coming out of the chamber was hitting the lowest edge of the optic mount, for the first optic the beam encountered.  I adjusted the mirror launching the beam into the chamber by a teeny bit, so that the outcoming beam was ~horizontal and hitting the center of the first steering mirror in pitch.  I had to move that steering mirror a little to the right (if you are staring at the HR face of the mirror), to get the beam to come close to the horizontal center of the optic.  Then I proceeded to do normal oplev alignment.

Also, I've noticed lately that ITMX is noisier than all the other optics.  It's kind of annoying.  The sensor RMS values reported for the ITMX watchdogs for UL and LL are rarely below 2, and are often (~70% of the time?) above 3.  The SD RMS is a normal 1-ish.

  6532   Thu Apr 12 23:52:49 2012 JenneUpdateLockingPRMI locked - 'bouncy'

I am locking some things, and have the PRM aligned, and it will stay locked for short periods of time, but as Kiwamu warned me, when the PRM alignment is better, the lock is more "crazy" and unstable.  This should go on our list of mysteries.

 

  6531   Thu Apr 12 23:12:16 2012 SureshUpdateIOOBeam Profile measurement: IPPOS beam: Possible Clipping

WiQuote:

[Suresh, Jenne]

  The input beam is most probably being clipped at the Faraday Isolator.  

Evidence: 

.....

We plan to investigate this further to be sure.. 

.....

 

I tried to determine an optimal WFS2YAW offset to be used so that we may avoid clipping.

Initially, I just measured the beam diameter as a function of offset.  If the beam diameter would become independent of offset if it is not clipped.  However a systematic effect became apparent when I shifted the beam on the detector to a slightly different location.  So I repeated the measurements while recentering the beam to the same location everytime  (centered at -1650+/- 50 for both H and V directions).

I have attached plots of the scans for both cases, with recentering and without.    I have not been able to figure out what is going on since the beam diameter does not become independent of the offset.  While the beam profile becomes more gaussian beyond offsets of about 7 or so, the beam diameter does not seem to follow a clear pattern.  The measurements are repeatable (within one sigma) so the experimental errors are smaller than 1 sigma.

The photographs below show the improvement of Horizontal beam profile with WFS2Yaw offset.  These seem to indicate a good gaussian beam for offsets beyond 7 or so.  At offsets more than 12 the MC unlocks.

 

Hor_OSet-2.png Hor_OSet0.png Hor_OSet2.png Hor_OSet8.png
 Offset = -2   Offset = 0   Offset = 2   Offset = 8

 

 

HorizontalNoRecenter.png  HorizontalRecentered.png
  This seems to indicate that the beam diameter does not vary for WFS2Yaw offset > 8  But if we recenter the beam for each measurement this effect seems to vanish

        

 Will continue tomorrow.   Jenne wants to do some IFO locking now.

 

  6530   Thu Apr 12 22:04:17 2012 Mike J.UpdateComputersNew Hysteresis Model & Plots

The new hysteresis model uses a triangle wave with offset zero points as the position function and a sinusoidal force function, creating a loop similar to this. Model is at /users/mjenson/matlab/ferro_hyst.mdl.

ferro_hyst.pnghyst_combo.png

  6529   Thu Apr 12 20:56:07 2012 DenUpdatePEMdaq

GUR1 XYZ, GUR2 XYZ, MC_F channels are now recorded at 256 Hz.

EDIT by JCD:  What Den means to say here is that (a) he modified some .ini files, and (b) he restarted the fb.

  6528   Thu Apr 12 14:48:44 2012 SureshUpdateSUSlocal damping and WFS

    WFS servo is moving the MC mirror angles to minimise TEM01 and TEM10 modes within the MC cavity.    This means it will compensate not only for angular noise in the mirrors but also for the PSL beam pointing fluctuations.  So the extra "noise" we see when WFS loops are on is because they are active below the WFS UGF of about 2 Hz.  Also if the HEPA airflow is above 20% (of its max), the PSL beam jitter (caused by the airflow) will add broadband noise into the WFS servo loops and this will show up in the OSEM signals.  See elog 5943 for details.

 

 

Quote

    ......            

Then I compared MC1, MC2, MC3 suspos, suspit, susyaw and susside positions with WFS on (black curve) and off (red curve). WFS add noise to MC1 and MC3 measured by osems (MC2 is fine though). WFS should change osem readings but is it a correct way to do this below 0.5 Hz (?) It looks like just a flat noise. Need to think about the conclusion.

 

 wfsmc1.png                      

 

 

 

  6527   Thu Apr 12 08:49:14 2012 DenUpdateSUSlocal damping and WFS

I tried to figure out what can add noise below 0.5 Hz to the MC_F. I compared MC1, MC2, MC3 suspos, suspit, susyaw and susside positions with damping (black curves) and without (red curves). Local damping is fine.

 mc1.png        mc2.png          mc3.png

Then I compared MC1, MC2, MC3 suspos, suspit, susyaw and susside positions with WFS on (black curve) and off (red curve). WFS add noise to MC1 and MC3 measured by osems (MC2 is fine though). WFS should change osem readings but is it a correct way to do this below 0.5 Hz (?) It looks like just a flat noise. Need to think about the conclusion.

 

 wfsmc1.png           wfsmc2.png            wfsmc3.png

ELOG V3.1.3-