40m QIL Cryo_Lab CTN SUS_Lab TCS_Lab OMC_Lab CRIME_Lab FEA ENG_Labs OptContFac Mariner WBEEShop
  40m Log, Page 301 of 344  Not logged in ELOG logo
ID Date Author Type Categorydown Subject
  11891   Thu Dec 17 16:44:03 2015 gautamUpdateCDSALS Slow control MEDM screen updated
Quote:

I've not updated the MEDM screens to reflect the two new paths yet, but will do so soon. It also remains to install appropriate filters for the servo path that takes the frequency readout as the input.

A few more related changes:

  1. The couplers that used to sit on the green beat PDs on the PSL table have now been shifted to the IR broadband PDs in the FOL box so that I can get the IR beat frequency over to the frequency counters. The FOL box itself, along with the fibers that bring IR light to the PSL table, have been relocated to the corner of the PSL table where the green beat PDs sit because of cable length constraints.
  2. I've updated the ALS slow control MEDM screen to allow for slow control of the beat frequency. The servo shape for now is essentially just an integrator with a zero at 1 Hz. The idea is to set an offset in the new filter module, which is the desired beat frequency, and let the integrator maintain this beat frequency. One thing I've not taken care of yet is automatically turning this loop off when the IMC loses lock. Screenshot of the modified MEDM screen is attached. 
  3. I checked the performance by using the temperature sliders to introduce an offset. The integrator is able to bring the beat frequency back to the setpoint in a few seconds, provided the step I introduced was not two big (~20 counts, but this is a pretty large shift in beat frequency, nearly 20MHz).

To do:

  1. Figure out how to deal with the IMC losing lock. I guess this is important if we want to use the IR beatnote as a diagnostic for the state of the X AUX laser.
  2. Optimize the servo gains a little - I still see some ringing when I introduce an offset, this could be avoided...
  11972   Wed Feb 3 08:39:17 2016 SteveUpdateCDSblank daily summary pages

Daily summary pages are blank today. Yesterday is ok

  12024   Sun Mar 6 15:24:05 2016 gautamUpdateCDSFB down again

I came in to check the status of the nitrogen and noticed that the striptool panels in the control room were all blank.

  • PMC was unlocked but I was able to relock it using the usual procedure
  • FB seems to be down: I was unable to ssh into it (or any of the FEs for that matter). I checked the lights on the RAID array, they are all green. I am holding off on doing a hard reboot of FB in case there is some other debugging that can be done first
  • None of the watchdogs were tripped, but judging by the green spots on the mirrors, all of them are moving quite a bit. I've shutdown the watchdogs on all the optics except the MC mirrors, but the ITMs and ETMs still seem to be moving quite a bit.

I am leaving things in this state for now. It is unclear why this should have happened, it doesn't seem like there was a power glitch?

  12025   Mon Mar 7 20:40:02 2016 ericqUpdateCDSFB down again

We went and looked at the monitor plugged into FB. All kinds of messages were being spammed to the screen (maybe RAM errors), and nothing could be done to interrupt. Sadly, a hard reboot of FB was neccesary.

Video of error messages: https://youtu.be/7rea_kokhPY

After the reboot, it just took a couple of model restarts to get the CDS screen happy.

  12064   Tue Apr 5 14:16:34 2016 gautamUpdateCDSBLRMS for optics suspensions - library block UPDATED

As discussed in a Wednesday meeting some time ago, we don't need to be writing channels from BLRMS filter modules to frames at 16k (we suspect this is leading to the frequent daqd crashes which were seen the last time we tried setting BLRMS up for all the suspensions). EricQ pointed out to me that there conveniently exists a library block that is much better suited to our purposes, called BLRMS_2k. I've replaced all the BLRMS library blocks in the sus_single_BLRMS library block that I made with there BLRMS_2k blocks. I need to check that the filters used by the BLRMS_2k block (which reside in /opt/rtcds/userapps/release/cds/common/src/BLRMSFILTER.c) are appropriate, after which we can give setting up BLRMS for all the suspensions a second try...

  12140   Mon May 30 18:19:50 2016 JohannesUpdateCDSASS medm screen update

I noticed that the TRY button in the ASS main screen was linking to LSC_TRX instead of LSC_TRY. Gautam fixed it.

  12148   Fri Jun 3 13:05:18 2016 ericqUpdateCDSCDS Notes

Some CDS related things:


Keith Thorne has told us about a potential fix for our framebuilder woes. Jamie is going to be at the 40m next week to implement this, which could interfere with normal interferometer operation - so plan accordingly. 


I spent a little time doing some plumbing in the realtime models for Varun's audio processing work. Specifically, I tried to spin up a new model (C1DAF), running on the c1lsc machine. This included:

  • Removing the unused TT3 and TT4 parts from the IOO block in c1ass.mdl, freeing up some DAC outputs on the LSC rack
  • Adding an output row to the LSC input matrix which pipes to a shared memory IPC block. (This seemed like the simplest way for the DAFI model to have access to lots of signals with minimal overhead).
  • Removing two unused ADC inputs from c1lsc.mdl (that went to things like PD_XXX), to give c1daf.mdl the required two ADC inputs - and to give us the option of feeding in some analog signals.
  • Editing the rtsystab file to include c1daf in the list of models that run on c1lsc
  • Editing the existing DAFI .mdl file (which just looked like an old recolored cut-n-paste of c1ioo.mdl) to accept the IPC and ADC connections, and one DAC output that would go to the fibox. 

The simple DAFI model compiled and installed without complaint, but doesn't succesfully start. For some reason, the frontend never takes the CPU offline. Jamie will help with this next week. Since things aren't working, these changes have not been commited to the userapps svn. 

  12151   Mon Jun 6 16:41:36 2016 ericqUpdateCDSFB upgrade work

Barring objections, starting tomorrow morning, Jamie will be testing the new FB code. The IFO will not be available for other use while this is ongoing.

  12152   Tue Jun 7 11:12:47 2016 jamieUpdateCDSDAQD UPGRADE WORK UNDERWAY

I am re-starting work on the daqd upgrade again now.  Expect the daqd to be offline for most of the day.  I will report progress.

  12155   Tue Jun 7 20:49:50 2016 jamieUpdateCDSDAQD work ongoing

Summary: new daqd code running overnight test on fb1.  Stability issues persist.

The code is from Keith's "tests/advLigoRTS-40m" branch, which is a branch of the current trunk.  It's supposed to include patches to fix the crashes when multiple frame types are written to disk at the same time.  However, the issue is not fixed:

2016-06-07_20:38:55 about to write frame @ 1149392336
2016-06-07_20:38:55 Begin Full WriteFrame()
2016-06-07_20:38:57 full frame write done in 2seconds
2016-06-07_20:39:11 about to write frame @ 1149392352
2016-06-07_20:39:11 Begin Full WriteFrame()
2016-06-07_20:39:13 full frame write done in 2seconds
2016-06-07_20:39:27 about to write frame @ 1149392368
2016-06-07_20:39:27 Begin Full WriteFrame()
2016-06-07_20:39:29 full frame write done in 2seconds
2016-06-07_20:39:43 about to write second trend frame @ 1149391800
2016-06-07_20:39:43 Begin second trend WriteFrame()
2016-06-07_20:39:43 about to write frame @ 1149392384
2016-06-07_20:39:43 Begin Full WriteFrame()
2016-06-07_20:39:44 full frame write done in 1seconds
2016-06-07_20:39:59 about to write frame @ 1149392400
2016-06-07_20:40:04 Begin Full WriteFrame()
2016-06-07_20:40:04 Second trend frame write done in 21 seconds
2016-06-07_20:40:14 [Tue Jun  7 20:40:14 2016] main profiler warning: 1 empty blocks in the buffer
2016-06-07_20:40:15 [Tue Jun  7 20:40:15 2016] main profiler warning: 0 empty blocks in the buffer
2016-06-07_20:40:16 [Tue Jun  7 20:40:16 2016] main profiler warning: 0 empty blocks in the buffer
2016-06-07_20:40:17 [Tue Jun  7 20:40:17 2016] main profiler warning: 0 empty blocks in the buffer
2016-06-07_20:40:18 [Tue Jun  7 20:40:18 2016] main profiler warning: 0 empty blocks in the buffer
2016-06-07_20:40:19 [Tue Jun  7 20:40:19 2016] main profiler warning: 0 empty blocks in the buffer
2016-06-07_20:40:20 [Tue Jun  7 20:40:20 2016] main profiler warning: 0 empty blocks in the buffer
2016-06-07_20:40:21 [Tue Jun  7 20:40:21 2016] main profiler warning: 0 empty blocks in the buffer
2016-06-07_20:40:22 [Tue Jun  7 20:40:22 2016] main profiler warning: 0 empty blocks in the buffer
2016-06-07_20:40:23 [Tue Jun  7 20:40:23 2016] main profiler warning: 0 empty blocks in the buffer

This failure comes when a full frame (1149392384+16) is written to disk at the same time as a second trend (1149391800+600).  It seems like every time this happens daqd crashes.

I have seen other stability issues as well, maybe caused by mx flakiness, or some sort of GPS time synchronization issue caused by our lack of IRIG-B cards.  I'm going to look to see if I can get the GPS issue taken care of so we take that out of the picture.

For the last couple of hours I've only seen issues with the frame writing every 20 minutes, when the full and second trend frames happen to be written at the same time.  Running overnight to gather more statistics.

  12156   Wed Jun 8 08:34:55 2016 jamieUpdateCDSDAQD work ongoing

38 restarts overnight.  Problem definitely not fixed.  I'll be reverting back to old daqd and fb this morning.  Then regroup and evaluate options.

  12158   Wed Jun 8 13:50:39 2016 jamieConfigurationCDSSpectracom IRIG-B card installed on fb1

[EDIT: corrected name of installed card]

We just installed a Spectracom TSyc-PCIe timing card on fb1.  The hope is that this will help with the GPS timeing syncronization issues we've been seeing in the new daqd on fb1, hopefully elliminating some of the potential failure channels.

The driver, called "symmetricom" in the advLigoRTS source (name of product from competing vendor), was built/installed (from DCC T1500227):

controls@fb1:~/rtscore/tests/advLigoRTS-40m 0$ cd src/drv/symmetricom/
controls@fb1:~/rtscore/tests/advLigoRTS-40m/src/drv/symmetricom 0$ ls
Makefile  stest.c  symmetricom.c  symmetricom.h
controls@fb1:~/rtscore/tests/advLigoRTS-40m/src/drv/symmetricom 0$ make
make -C /lib/modules/3.2.0-4-amd64/build SUBDIRS=/home/controls/rtscore/tests/advLigoRTS-40m/src/drv/symmetricom modules
make[1]: Entering directory `/usr/src/linux-headers-3.2.0-4-amd64'
  CC [M]  /home/controls/rtscore/tests/advLigoRTS-40m/src/drv/symmetricom/symmetricom.o
/home/controls/rtscore/tests/advLigoRTS-40m/src/drv/symmetricom/symmetricom.c:59:9: warning: initialization from incompatible pointer type [enabled by default]
/home/controls/rtscore/tests/advLigoRTS-40m/src/drv/symmetricom/symmetricom.c:59:9: warning: (near initialization for ‘symmetricom_fops.unlocked_ioctl’) [enabled by default]
/home/controls/rtscore/tests/advLigoRTS-40m/src/drv/symmetricom/symmetricom.c: In function ‘get_cur_time’:
/home/controls/rtscore/tests/advLigoRTS-40m/src/drv/symmetricom/symmetricom.c:89:2: warning: ISO C90 forbids mixed declarations and code [-Wdeclaration-after-statement]
/home/controls/rtscore/tests/advLigoRTS-40m/src/drv/symmetricom/symmetricom.c: In function ‘symmetricom_init’:
/home/controls/rtscore/tests/advLigoRTS-40m/src/drv/symmetricom/symmetricom.c:188:2: warning: ISO C90 forbids mixed declarations and code [-Wdeclaration-after-statement]
/home/controls/rtscore/tests/advLigoRTS-40m/src/drv/symmetricom/symmetricom.c:222:3: warning: label ‘out_remove_proc_entry’ defined but not used [-Wunused-label]
/home/controls/rtscore/tests/advLigoRTS-40m/src/drv/symmetricom/symmetricom.c:158:22: warning: unused variable ‘pci_io_addr’ [-Wunused-variable]
/home/controls/rtscore/tests/advLigoRTS-40m/src/drv/symmetricom/symmetricom.c:156:6: warning: unused variable ‘i’ [-Wunused-variable]
  Building modules, stage 2.
  MODPOST 1 modules
  CC      /home/controls/rtscore/tests/advLigoRTS-40m/src/drv/symmetricom/symmetricom.mod.o
  LD [M]  /home/controls/rtscore/tests/advLigoRTS-40m/src/drv/symmetricom/symmetricom.ko
make[1]: Leaving directory `/usr/src/linux-headers-3.2.0-4-amd64'
controls@fb1:~/rtscore/tests/advLigoRTS-40m/src/drv/symmetricom 0$ sudo make install
#remove all old versions of the driver
find /lib/modules/3.2.0-4-amd64 -name symmetricom.ko -exec rm -f {} \; || true
find /lib/modules/3.2.0-4-amd64 -name symmetricom.ko.gz -exec rm -f {} \; || true
# Install new driver
install -D -m 644 symmetricom.ko /lib/modules/3.2.0-4-amd64/extra/symmetricom.ko
/sbin/depmod -a || true
/sbin/modprobe symmetricom
if [ -e /dev/symmetricom ] ; then \
        rm -f /dev/symmetricom ; \
    fi
mknod /dev/symmetricom c `grep symmetricom /proc/devices|awk '{print $1}'` 0
chown controls /dev/symmetricom
controls@fb1:~/rtscore/tests/advLigoRTS-40m/src/drv/symmetricom 0$ ls /dev/symmetricom
/dev/symmetricom
controls@fb1:~/rtscore/tests/advLigoRTS-40m/src/drv/symmetricom 0$ ls -al /dev/symmetricom
crw-r--r-- 1 controls root 250, 0 Jun  8 13:42 /dev/symmetricom
controls@fb1:~/rtscore/tests/advLigoRTS-40m/src/drv/symmetricom 0$ 
  12161   Thu Jun 9 13:28:07 2016 jamieConfigurationCDSSpectracom IRIG-B card installed on fb1

Something is wrong with the timing we're getting out of the symmetricom driver, associated with the new spectracom card.

controls@fb1:~/rtscore/tests/advLigoRTS-40m/src/drv/symmetricom 127$ lalapps_tconvert 
1149538884
controls@fb1:~/rtscore/tests/advLigoRTS-40m/src/drv/symmetricom 0$ cat /proc/gps 
704637380.00
controls@fb1:~/rtscore/tests/advLigoRTS-40m/src/drv/symmetricom 0$ 

The GPS time is way off, and it's counting up at something like 900 seconds/second.  Something is misconfigured, but I haven't figured out what yet.

The timing distribution module we're using is spitting out what appears to be an IRIG B122 signal (amplitude moduled 1 kHz carrier), which I think is what we expect.  This is being fed into the "AM IRIG input" connector on the card.

Not sure why the driver is spinning so fast, though, with the wrong baseline time.  Reboot of the machine didn't help.

  12162   Thu Jun 9 15:14:46 2016 jamieUpdateCDSold fb restarted, test of new daqdon fb1 aborted for time being

I've restarted the old daqd on fb until I can figure out what's going on with the symmetricom driver on fb1.

Steve:  Jamie with hair.... long time ago
 

  12166   Fri Jun 10 12:09:01 2016 jamieConfigurationCDSIRIG-B debugging

Looks like we might have a problem with the IRIG-B output of the GPS receiver.

Rolf came over this morning to help debug the strange symmetricom driver behavior on fb1 with the new Spectracom card.  We restarted the machine againt and this time when we loaded the drive rit was clocking at a normal rate (second/second).  However, the overall GPS time was still wrong, showing a time in October from this year.

The IRIG-B122 output is supposed to encode the time of year via amplitude modulation of a 1kHz carrier.  The current time of year is:

controls@fb1:~ 0$ TZ=utc date +'%j day, %T'
162 day, 18:57:35
controls@fb1:~ 0$ 

The absolute year is not encoded, though, so the symmetricon driver has the year offset hard coded into the driver (yuck), to which it adds the time of year from the IRIG-B signal to get the correct GPS time.

However, loading the symmetricom module shows the following:

...
[ 1601.607403] Spectracom GPS card on bus 1; device 0
[ 1601.607408] TSYNC PIC BASE 0 address = fb500000
[ 1601.607429] Remapped 0xffffc90017012000
[ 1606.606164] TSYNC NOT receiving YEAR info, defaulting to by year patch
[ 1606.606168] date = 299 days 18:28:1161455320
[ 1606.606169] bcd time = 1161455320 sec  959 milliseconds 398 microseconds  959398630 nanosec
[ 1606.606171] Board sync = 1
[ 1606.616076] TSYNC NOT receiving YEAR info, defaulting to by year patch
[ 1606.616079] date = 299 days 18:28:1161455320
[ 1606.616080] bcd time = 1161455320 sec  969 milliseconds 331 microseconds  969331350 nanosec
[ 1606.616081] Board sync = 1
controls@fb1:~ 0$ 

Apparently the symmetricom driver thinks it's the 299nth day of the year, which of course corresponds to some time in october, which jives with the GPS time the driver is spitting out.

Rolf then noticed that the timing module in the VME crate in the adjacent rack, which also receives an IRIG-B signal from the distribution box, was also showing day 299 on it's front panel display. We checked and confirmed that the symmetricom card and the VME timing module both agree on the wrong time of year, strongly suggesting that the GPS receiver is outputing bogus data on it's IRIG-B output, even though it's showing the correct time on it's front panel.  We played around with setting in the GPS receiver to no avail.  Finally we rebooted the GPS receiver, but it seemed to come up with the same bogus IRIG-B output (again both symmetricom driver and VME timing module agree on the wrong day).

So maybe our GPS receiver is busted?  Not sure what to try now.

 

  12167   Fri Jun 10 12:21:54 2016 jamieConfigurationCDSGPS receiver not resetting properly

The GPS receiver (EndRun Technologies box in 1Y5? (rack closest to door)) seems to not coming back up properly after the reboot.  The front pannel says that it's "LKD", but the "sync" LED is flashing instead of solid, and the time of year displayed on the front panel is showing day 6.  The fb1 symmetricom driver and VME timing module are still both seeing day 299, though.  So something may definitely be screwy with the GPS receiver.

  12173   Mon Jun 13 20:01:30 2016 varunUpdateCDSDAFI GUI update

Summary: I am implementing digital audio filtering on various interferometer signals in order to listen to the processed audio which will help in characterizing and noise reduction in the interferometer. following is a summary of the gui i have made towards a general purpose DAF module linked to the LSC. 

Details:  attachment 1 shows the top level overview of the daf module.

The "INPUTS" button shown redirects to the medm screen shown in attachment 2, which is a collection of inputs going into the module.

Each of the buttons shown in "C1DAFI_INPUTS.png" is further linked to various i/o boxes like adc1, adc2, lsc signal and exitation. An example is shown in attachment 3. This is the specific I/O box for the LSC signal.

The field labelled "INPUT_MTRX" is linked to a matrix which routes these 4 inputs to various DSP blocks. Similarly, the "OUTPUT_MTRX" tab is useful for choosing which output goes to the speaker. 

Time and computational load monitoring is done in the "GDS_TP" tab which links to the medm screen shown in attachment 4.

Currently the AGC is successfully implemented as one of the DSP block. The details of the AGC implementation were given in a previous elog: https://nodus.ligo.caltech.edu:8081/40m/12159

I need to make a few changes to the code for Frequency Shifting and Whitening before uploading them on the FE. I will put the details soon.

Some more things that I think need to be added: 

1) "Enable" buttons for each of the DSP blocks.

2) Labels for each of the matrix elements.

3) Further headers and other description for each of the tabs 

  12179   Tue Jun 14 19:37:40 2016 jamieUpdateCDSOvernight daqd test underway

I'm running another overnight test with new daqd software on fb1.  The normal daqd process on fb has been shutdown, and the front ends are sending their signals to fb1.

fb1 is running separate data concentrator (dc) and  frame writer (fw) processes, to see if this is a more stable configuration than the all-in-one framebuilder (fb) that we have been trying to run with.  I'll report on the test tomorrow.

  12180   Tue Jun 14 20:10:19 2016 varunUpdateCDSDAFI GUI update

I have added Enable buttons for each of the DSP blocks, and labels for the matrix elements. The input matrix takes inputs from each of the 4 channels: ADC1, ADC2, LSC and EXC, and routes them to the audio processing blocks (attachment 2). The output matrix (attachment 3) takes the outputs of the various DSP blocks and routes them to the output and then to the speakers. 

  12181   Wed Jun 15 09:52:02 2016 jamieUpdateCDSVery encouraging results from overnight split daqd test

laughVery encouraging results from the test last night.  The new configuration did not crash once overnight, and seemed to write out full, second trend, and minute trend frames without issueyes.  However, full validity of all the written out frames has not been confirmed.

overview

The configuration under test involves two separate daqd binaries instead of one.  We usually run with what is referred to as a "framebuilder" (fb) configuration:

  • fb: a single daqd binary that:
    • collect the data from the front ends
    • coallate full data into frame file format
    • calculates trend data
    • writes frame files to disk.

The current configuration separates the tasks into multiple separate binaries: a "data concentrator" (dc) and a "frame writer" (fw):

  • dc:
    • collect data from front ends
    • coallate full data into frame file format
    • broadcasts frame files over local network
  • fw:
    • receives frame files from broadcast
    • calculates trend data
    • writes frame files to disk

This configuration is more like what is run at the sites, where all the various components are separate and run on separate hardware.  In our case, I tried just running the two binaries on the same machine, with the broadcast going over the loopback interface.  None of the systems that use separated daqd tasks see the failures that we've been seeing with the all-in-one fb configuration (and other sites like AEI have also seen).

My guess frown is that there's some busted semaphore somewhere in daqd that's being shared between the concentrator and writer components.  The writer component probably aquires the lock while it's writing out the frame, which prevents the concentrator for doing what it needs to be doing while the frame is being written out.  That causes the concentrator to lock up and die if the frame writing takes too long (which it seems to almost necessarily do, especially when trend frames are also being written out).

results

The current configuration hasn't been tweaked or optimized at all.  There is of course basically no documentation on the meaning of the various daqdrc directives.  Hopefully I can get Keith Thorne to help me figure out a well optimized configuration.

There is at least one problem whereby the fw component is issuing an excessively large number of re-transmission requests:

2016-06-15_09:46:22 [Wed Jun 15 09:46:22 2016] Ask for retransmission of 6 packets; port 7097
2016-06-15_09:46:22 [Wed Jun 15 09:46:22 2016] Ask for retransmission of 8 packets; port 7097
2016-06-15_09:46:22 [Wed Jun 15 09:46:22 2016] Ask for retransmission of 3 packets; port 7097
2016-06-15_09:46:22 [Wed Jun 15 09:46:22 2016] Ask for retransmission of 5 packets; port 7097
2016-06-15_09:46:22 [Wed Jun 15 09:46:22 2016] Ask for retransmission of 5 packets; port 7097
2016-06-15_09:46:22 [Wed Jun 15 09:46:22 2016] Ask for retransmission of 5 packets; port 7097
2016-06-15_09:46:22 [Wed Jun 15 09:46:22 2016] Ask for retransmission of 5 packets; port 7097
2016-06-15_09:46:22 [Wed Jun 15 09:46:22 2016] Ask for retransmission of 6 packets; port 7097
2016-06-15_09:46:23 [Wed Jun 15 09:46:23 2016] Ask for retransmission of 1 packets; port 7097

It's unclear why.  Presumably the retransmissions requests are being honored, and the fw eventually gets the data it needs.  Otherwise I would hope that there would be the appropriate errors.

The data is being written out as expected:

 full/11500: total 182G
drwxr-xr-x  2 controls controls 132K Jun 15 09:37 .
-rw-r--r--  1 controls controls  69M Jun 15 09:37 C-R-1150043856-16.gwf
-rw-r--r--  1 controls controls  68M Jun 15 09:37 C-R-1150043840-16.gwf
-rw-r--r--  1 controls controls  68M Jun 15 09:37 C-R-1150043824-16.gwf
-rw-r--r--  1 controls controls  69M Jun 15 09:36 C-R-1150043808-16.gwf
-rw-r--r--  1 controls controls  69M Jun 15 09:36 C-R-1150043792-16.gwf
-rw-r--r--  1 controls controls  68M Jun 15 09:36 C-R-1150043776-16.gwf
-rw-r--r--  1 controls controls  68M Jun 15 09:36 C-R-1150043760-16.gwf
-rw-r--r--  1 controls controls  69M Jun 15 09:35 C-R-1150043744-16.gwf

 trend/second/11500: total 11G
drwxr-xr-x  2 controls controls 4.0K Jun 15 09:29 .
-rw-r--r--  1 controls controls 148M Jun 15 09:29 C-T-1150042800-600.gwf
-rw-r--r--  1 controls controls 148M Jun 15 09:19 C-T-1150042200-600.gwf
-rw-r--r--  1 controls controls 148M Jun 15 09:09 C-T-1150041600-600.gwf
-rw-r--r--  1 controls controls 148M Jun 15 08:59 C-T-1150041000-600.gwf
-rw-r--r--  1 controls controls 148M Jun 15 08:49 C-T-1150040400-600.gwf
-rw-r--r--  1 controls controls 148M Jun 15 08:39 C-T-1150039800-600.gwf
-rw-r--r--  1 controls controls 148M Jun 15 08:29 C-T-1150039200-600.gwf
-rw-r--r--  1 controls controls 148M Jun 15 08:19 C-T-1150038600-600.gwf

 trend/minute/11500: total 152M
drwxr-xr-x 2 controls controls 4.0K Jun 15 07:27 .
-rw-r--r-- 1 controls controls  51M Jun 15 07:27 C-M-1150023600-7200.gwf
-rw-r--r-- 1 controls controls  51M Jun 15 04:31 C-M-1150012800-7200.gwf
-rw-r--r-- 1 controls controls  51M Jun 15 01:27 C-M-1150002000-7200.gwf

The frame sizes look more or less as expected, and they seem to be valid as determined with some quick checks with the framecpp command line utilities.

  12182   Wed Jun 15 10:19:10 2016 SteveConfigurationCDSGPS antennas... debugging

Visual inspection of rooftop GPS antennae:

The 2 GPS antennas are located on the north west corner of CES roof. Their condition looks well weathered. I'd be surprised if they work.

The BNC connector of the 1553 head is inside of the conduit so it is more likely to have a better connection than the other one.

I have not had a chance to look into the "GPS Time Server" unit.

  12183   Wed Jun 15 11:21:51 2016 jamieUpdateCDSstill work to do to transition to new configuration/code

Just to be clear, there's still quite a bit of work to fully transition the 40m to this new system/configuration.  Once we determine a good configuration we need to complete the install, and modify the setup to run the two binaries instead of just the one.  The data is also being written to a raid on the new fb1, and we need to decide if we should use this new raid, or try to figure out how to move the old jetstor raid to the new fb1 machine.

  12185   Wed Jun 15 22:12:55 2016 varunUpdateCDSDAFI update: stereo output

I wish to have stereo audio output for the DAF module. Hence, there needs to be a second output from the DAF. I added this second output to the model. Following are the details:

FiBox: It consists of two analog inputs which are digitized and multiplexed and transmitted optically. (only 1 fiber is needed due to multiplexing). Attachment 1 shows the fibox with its 2 analog inputs (one of which, is connected), and 1 fiber output. The output of the DAF goes to the FiBox. Until today, the Fibox recieved only 1 analog input. This analog signal comes from the DAC-8 (count starting from 0), which is located at "CH 1 OUT" SMA output in the "MONITORS" bin on the racks (attachment 2).

I have added another output channel to the DAF model both in software and in hardware. The DAF now also uses DAC-9 analog output which goes to the second analog input of the FiBox. The DAC-9 output is located at "CH 2 OUT" SMA output in the "MONITORS" bin on the racks (attachment 4).

After making the changes, the Fibox is shown in attacment 3.

Testing: The LSC input on passing through the DAF block is given through two different DAC outputs, to the same Fibox channel (one after the other), and the output is heard. More concrete testing will be done tomorrow. It will be as follows:

1) Currently, I need to search for a suitable cable that would connect the second channel of the output fibox to the audio mixer. After doing this, end to end testing of both channels will be done.

2) I could not access the AWG, probably because the DAQ was offline today afternoon. Using a signal from the AWG will give a more concrete testing of the stereo output.

3) After this, I will separate the two channels of the stereo completely (currectly they are seperated only at the DAF output stage)

4) I also will edit the medm gui appropriately.

 

Quote:

I have added Enable buttons for each of the DSP blocks, and labels for the matrix elements. The input matrix takes inputs from each of the 4 channels: ADC1, ADC2, LSC and EXC, and routes them to the audio processing blocks (attachment 2). The output matrix (attachment 3) takes the outputs of the various DSP blocks and routes them to the output and then to the speakers. 

 

  12191   Thu Jun 16 16:11:11 2016 jamieUpdateCDSupgrade aborted for now

After poking at the new configuration more, it also started to show instability.  I couldn't figure out how to make test points or excitations available in this configuration, and adding in the full set of test point channels, and trying to do simple things like plotting channels with dtt, the frame writer (fw) would fall over, apparetnly unable to keep up with the broadcast from the dc.

I've revered everything back to the old semi-working fb configuration, and will be kicking this to the CDS group to deal with.

  12200   Mon Jun 20 11:11:20 2016 SteveConfigurationCDSGPS receiver not resetting properly

I called https://www.endruntechnologies.com/pdf/USM3014-0000-000.pdf  and they said it's very likely just needs a software update. They will email Jamie the details.

Quote:

The GPS receiver (EndRun Technologies box in 1Y5? (rack closest to door)) seems to not coming back up properly after the reboot.  The front pannel says that it's "LKD", but the "sync" LED is flashing instead of solid, and the time of year displayed on the front panel is showing day 6.  The fb1 symmetricom driver and VME timing module are still both seeing day 299, though.  So something may definitely be screwy with the GPS receiver.

 

  12201   Mon Jun 20 11:19:41 2016 jamieConfigurationCDSGPS receiver not resetting properly
Quote:

I called https://www.endruntechnologies.com/pdf/USM3014-0000-000.pdf  and they said it's very likely just needs a software update. They will email Jamie the details.

I got the email from them.  There was apparently a bug that manifested on February 14 2016.  I'll try to software update today.

http://endruntechnologies.com/pdf/FSB160218.pdf
http://endruntechnologies.com/upgradetemplx.htm

  12202   Mon Jun 20 14:03:04 2016 jamieConfigurationCDSEndRun GPS receiver upgraded, fixed

I just upgraded the EndRun Technologies Tempus LX GPS receiver timing unit, and it seems to have fixed all the problems.  cool

Thanks to Steve for getting the info from EndRun.  There was indeed a bug in the firmware that was fixed with a firmware upgrade.

I upgraded both the system firmware and the firmware of the GPS subsystem:

Tempus LX GPS(root@Tempus:~)-> gntpversion 
Tempus LX GPS 6010-0044-000 v 5.70 - Wed Oct 1 04:28:34 UTC 2014
Tempus LX GPS(root@Tempus:~)-> gpsversion 
F/W 5.10 FPGA 0416
Tempus LX GPS(root@Tempus:~)->

After reboot the system is fully functional, displaying the correct time, and outputting the correct IRIG-B data, as confirmed by the VME timing unit.

I added a wiki page for the unit: https://wiki-40m.ligo.caltech.edu/NTP

 

Steve added this picture

  12207   Tue Jun 21 11:26:42 2016 varunFrogsCDSmedm command not working

"medm: command not found" error when run through command line both in pianosa and rossa in both editing and execution modes. It however gets executed and edited through the sitemap button. Don't know the source of the problem. Gautam did check the .bashrc file. aliases for SITEMAP and m40m are intact in the .bashrc file.

  12208   Tue Jun 21 11:49:29 2016 ericqFrogsCDSmedm command not working

The workstations' .bashrc is a symbolic link to /users/controls/.bashrc

In it, someone commented out the critical line:

#source /ligo/cdscfg/workstationrc.sh

I uncommented it. medm (and all of the other things like cdsutils) work again.

I blame jamie.

  12211   Wed Jun 22 10:15:45 2016 varunUpdateCDSDAFI update: stereo output

I have updated the DAFI with the following changes:

1) Separated both the channels of stereo output completely, as well as in the GUI.

2) Added text monitors for the inputs and outputs.

The stereo output is now ready except for a cable going from the second channel of the output fibox to the audio mixer.

Attached is the main DAF_OVERVIEW screen and its link button from the LSC screen labelled "DAFI"

Quote:

I wish to have stereo audio output for the DAF module. Hence, there needs to be a second output from the DAF. I added this second output to the model. Following are the details:

FiBox: It consists of two analog inputs which are digitized and multiplexed and transmitted optically. (only 1 fiber is needed due to multiplexing). Attachment 1 shows the fibox with its 2 analog inputs (one of which, is connected), and 1 fiber output. The output of the DAF goes to the FiBox. Until today, the Fibox recieved only 1 analog input. This analog signal comes from the DAC-8 (count starting from 0), which is located at "CH 1 OUT" SMA output in the "MONITORS" bin on the racks (attachment 2).

I have added another output channel to the DAF model both in software and in hardware. The DAF now also uses DAC-9 analog output which goes to the second analog input of the FiBox. The DAC-9 output is located at "CH 2 OUT" SMA output in the "MONITORS" bin on the racks (attachment 4).

After making the changes, the Fibox is shown in attacment 3.

Testing: The LSC input on passing through the DAF block is given through two different DAC outputs, to the same Fibox channel (one after the other), and the output is heard. More concrete testing will be done tomorrow. It will be as follows:

1) Currently, I need to search for a suitable cable that would connect the second channel of the output fibox to the audio mixer. After doing this, end to end testing of both channels will be done.

2) I could not access the AWG, probably because the DAQ was offline today afternoon. Using a signal from the AWG will give a more concrete testing of the stereo output.

3) After this, I will separate the two channels of the stereo completely (currectly they are seperated only at the DAF output stage)

4) I also will edit the medm gui appropriately.

 

Quote:

I have added Enable buttons for each of the DSP blocks, and labels for the matrix elements. The input matrix takes inputs from each of the 4 channels: ADC1, ADC2, LSC and EXC, and routes them to the audio processing blocks (attachment 2). The output matrix (attachment 3) takes the outputs of the various DSP blocks and routes them to the output and then to the speakers. 

 

 

  12215   Mon Jun 27 15:12:09 2016 varunUpdateCDSDAFI update: stereo output

Using an RC to BNC connector from the inner drawer, I have added a second output cable going from the output Fibox in the control room to the audio mixer.

Quote:

I have updated the DAFI with the following changes:

1) Separated both the channels of stereo output completely, as well as in the GUI.

2) Added text monitors for the inputs and outputs.

The stereo output is now ready except for a cable going from the second channel of the output fibox to the audio mixer.

Attached is the main DAF_OVERVIEW screen and its link button from the LSC screen labelled "DAFI"

Quote:

I wish to have stereo audio output for the DAF module. Hence, there needs to be a second output from the DAF. I added this second output to the model. Following are the details:

FiBox: It consists of two analog inputs which are digitized and multiplexed and transmitted optically. (only 1 fiber is needed due to multiplexing). Attachment 1 shows the fibox with its 2 analog inputs (one of which, is connected), and 1 fiber output. The output of the DAF goes to the FiBox. Until today, the Fibox recieved only 1 analog input. This analog signal comes from the DAC-8 (count starting from 0), which is located at "CH 1 OUT" SMA output in the "MONITORS" bin on the racks (attachment 2).

I have added another output channel to the DAF model both in software and in hardware. The DAF now also uses DAC-9 analog output which goes to the second analog input of the FiBox. The DAC-9 output is located at "CH 2 OUT" SMA output in the "MONITORS" bin on the racks (attachment 4).

After making the changes, the Fibox is shown in attacment 3.

Testing: The LSC input on passing through the DAF block is given through two different DAC outputs, to the same Fibox channel (one after the other), and the output is heard. More concrete testing will be done tomorrow. It will be as follows:

1) Currently, I need to search for a suitable cable that would connect the second channel of the output fibox to the audio mixer. After doing this, end to end testing of both channels will be done.

2) I could not access the AWG, probably because the DAQ was offline today afternoon. Using a signal from the AWG will give a more concrete testing of the stereo output.

3) After this, I will separate the two channels of the stereo completely (currectly they are seperated only at the DAF output stage)

4) I also will edit the medm gui appropriately.

 

Quote:

I have added Enable buttons for each of the DSP blocks, and labels for the matrix elements. The input matrix takes inputs from each of the 4 channels: ADC1, ADC2, LSC and EXC, and routes them to the audio processing blocks (attachment 2). The output matrix (attachment 3) takes the outputs of the various DSP blocks and routes them to the output and then to the speakers. 

 

 

 

  12251   Wed Jul 6 10:58:29 2016 VarunUpdateCDSDAFI review

The DAFI block was reviewed by Rana yesterday. The following changes/improvements were suggested: (Updated on 20th July 2016 with tasks taat remain in red)

1) include all the various channels like PEM, LSC, ASC, SUS, SEI, etc. as the inputs. Currently the inputs are only the LSC.
2) include all the control signals.
3) create a very detailed diagram of the entire signal flow and plan tasks accordingly.
4) Enable cascading of various DSP processes.
5) Adjusting the gain of the AGC such that the amplitude of the output signal comes to about half the peak amplitude offered by the ADC. This will help taking advantage of the entire dynamic range of the ADC.
6) change the enable button styles from a text input based controller to a button controller.
7) Currently, disabling a particular signal terminates the signal. Instead, it should turn into a unity gain block on disabling.
8) Check if the Fibox does AC coupling or not. If not, add an AC coupling arrangement in the DAFI.
9) Check the nature of the ADC1 and ADC2 inputs to the DAFI. I checked them yesterday, and they are channels 25 and 26 of ADC0, which are empty.

  12266   Thu Jul 7 12:44:52 2016 varunUpdateCDSDAFI update

Attached is a diagram, showing the entire (planned) signal flow of the DAF model. Some thoughts on the implementation after discussion with eric:

1) Since the LSC control signals and ASC signals are running on the c1lsc FE at the same rate as DAFI (16kHz), it would be wise to start from these.

   Current implementation: has a matrix at the end of the LSC PD signals, which selects one of the PD signals and outputs it to the DAFI via IPC communication.

    Proposed Changes: Add another matrix at the end of the LSC PD signals, to give to the second stereo output. Similarly, add two matrices each at the end of the LSC control signals and     the ASC signals. Each matrix must select one of the signals and output it to the DAF via IPC.

2) The PEM running on the c1sus FE system will have to be brought to DAFI in a similar fashon. However, since c1sus runs at 2kHz, there is a possibility of imaging while the signal is    transfered to the DAFI. This could be taken care of by an anti imaging filter, or inserting zeros between two samples coming at to the 16 kHz system from the 2kHz system and then low-passing it to remove the aliased parts. (similar to upsampling)

3) For the SUS control signals, input can be given from a matrix prepared for each optic seperately.

  12282   Fri Jul 8 22:26:03 2016 varunUpdateCDSDAFI Update: Changes in LSC model

I have added the control signals DARM_CTRL, MICH_CTRL, PRCL_CTRL, SRCL_CTRL, CARM_CTRL, XARM_CTRL, YARM_CTRL, MC_CTRL to the DAFI model from the LSC model via IPC commn.

The changes done to the LSC model include addition of an extra block going to DAFI (attachment 2, red rectangle in attachment 1), and addition of an extra overall output from the LSC, called DAFI_OUT2, which goes to DAFI through IPC link C1:LSC-DAF_2 (attach. 3). Now two distinct inputs can be given to the DAFI, whose intended purpose is to act as two distinct audio signals in the stereo output, but can also be used for arbitrary math. 

I am going to add the following PEM channels as DAF inputs subsequently, in a similar 2 input fashon.

SEIS_GUR1_X_OUT
SEIS_GUR1_Y_OUT
SEIS_GUR1_Z_OUT
SEIS_GUR2_X_OUT
SEIS_GUR2_Y_OUT
SEIS_GUR2_Z_OUT
SEIS_STS_1_X_OUT
SEIS_STS_1_Y_OUT
SEIS_STS_1_Z_OUT
ACC_MC1_X_OUT 
ACC_MC1_Y_OUT
ACC_MC1_Z_OUT
ACC_MC2_X_OUT
ACC_MC2_Y_OUT
ACC_MC2_Z_OUT

 

 

  12284   Sun Jul 10 15:54:23 2016 ericqUpdateCDSAll models restarted

For some reason, all of the non-IOP models on the vertex frontends had crashed.

To get all of the bits green, I ended up restarting all models on all frontends. (This means Gautam's coil tests have been interrupted.)

  12287   Sun Jul 10 20:08:44 2016 varunUpdateCDSDAFI Update: Changes in LSC and PEM models

I have added the PEM signals mentioned in the previous elog as DAF inputs through PCIE IPC, and compiled and restarted the c1pem and c1daf models. 

Attached are the pictures of the simulink diagram of the addition in the PEM and the DAF. 

Since the signals are moving from a 2kHz clock rate machine to a 16kHz clock rate machine, some imaging effects are possible, which I have to look into.

  12303   Thu Jul 14 23:38:59 2016 varunUpdateCDSc1lsc FE unresponsive

Today, at around 10:30, c1lsc machine froze and stopped responding to ping and ssh after I compiled and restarted c1daf. I think it is due to a large array in one of my codes. The daqd.log file shows the following:


..................................................................
CA.Client.Exception...............................................
    Warning: "Virtual circuit unresponsive"
    Context: "c1lsc.martian.113.168.192.in-addr.arpa:5064"
    Source File: ../tcpiiu.cpp line 945
    Current Time: Thu Jul 14 2016 22:27:42.102649102
..................................................................

I think the c1lsc FE may need a hard reboot.

  12304   Fri Jul 15 12:21:28 2016 varunUpdateCDSc1lsc FE unresponsive

c1lsc is up and running, Eric did a manual reboot today.

Quote:

Today, at around 10:30, c1lsc machine froze and stopped responding to ping and ssh after I compiled and restarted c1daf. I think it is due to a large array in one of my codes. The daqd.log file shows the following:


..................................................................
CA.Client.Exception...............................................
    Warning: "Virtual circuit unresponsive"
    Context: "c1lsc.martian.113.168.192.in-addr.arpa:5064"
    Source File: ../tcpiiu.cpp line 945
    Current Time: Thu Jul 14 2016 22:27:42.102649102
..................................................................

I think the c1lsc FE may need a hard reboot.

 

  12307   Sat Jul 16 00:30:42 2016 varunUpdateCDSDAFI update: Frequency warping | c1lsc unresponsive

Summary: I am trying to implement frequency warping/pitch shifting on the real time FE. Here is a description long overdue:

Description: The overall idea is as follows:

The DFT of a frame x_i[n]  is given by X_i[k] = \sum_{n=0}^{m-1}x_i[n]e^{-j2\pi \frac{kn}{m}}. A matrix W containing all W_{kn} = e^{-j2\pi \frac{kn}{m}} for k, n = 1, 2, ..., m can be calculated and predefined in the code. The input arrival rate is 16384 Hz, i.e. once in every 60 $\mu$s time window. Hence, the fourier coefficients can be updated cumulatively in each cycle using the current value of the input, previous value of the fourier coefficient and the components of the W matrix. This will distribute the computational load of the FFT into all the time windows. Similar operations can be carried out for the inverse STFT.

I have written and run a pseudo-real time code on my CPU. The following is the essence:
 Let the frame-length be M, and the intended scale factor of the frequency warping be 'r'. The frame overlap is 50%. At each clock cycle, the following tasks are performed:  (1 to 3 are routine tasks performed at every clock cycle, 4 is a special task performed only when a frame is filled.)
 1) Take input and apply hanning window to it.
 2) Cumulate X_i[k] for every k using the value of x_i[n] (the input) at that particular instant. Also start to cumulate X_{i+1}[k], which will be later transfered to X_i[k].
 3) Because of 4), we now have 'r+1' filled frames corresponding to output fft. Now take the ifft using two consecutive frames corresponding to only two time series points. The computations required for this task are the same as the computations required for calculation of the fourier coefficients iteratively, since the entire time series ifft is not computed.
 4) Do these special tasks after each frame gets filled:
      At this point, the ffts of the current frame and a previous frame is ready. Let us call them X1 and X2.
      Calculate phase difference between the two.
      Calculate all the interpolated |Y_i| in between these two frames depending upon the scale factor.
      Assign phase of X1 to first Y frame and assign increasing phase to all the other Y frames.
      and also do all the usual non-special tasks.

This code takes about 9-10 microseconds for a cycle with special tasks, and 5-6 microseconds for a cycle with routine tasks on my laptop (brought down from 100 microseconds peak time in the earlier offline implementation due to elemination of explicit dft and reduction in fft size), for a frame size of 32 samples. However, when fed into the c1lsc FE, it crashes, as it has done once again today evening, in the same fashon as yesterday. There could be 2 possible reasons:

1) Size of the array containing the W_{kn} matrix elements is too large for the FE memory,

2) the computations are taking up more than 60 microseconds.

Since there are already a few codes with similar array sizes, I am more inclined to think that 2) is more likely.

Another problem that I am anticipating is that for a 32 point dft and a sampling rate of 16kHz, the frequency resolution achieved is about 500 Hz, which is not sufficient if we need to represent seismic signals. The only way I can think of, for representing such signals with a small number of fft points, is to reduce the effective sampling rate, i.e. do DSP on inputs at a much lower rate than 16kHz (say 1kHz, which will give a resolution of ~30 Hz, or 2kHz giving a resolution of ~60Hz). Another advantage of this method is that it frees up more clock cycles for computation, thus the computational load can be further distributed.  The problem in this implementation is that it will increase the delays.

  12309   Mon Jul 18 18:44:52 2016 varunUpdateCDSc1lsc FE recovered

c1lsc FE is up and running.

Details:

2) The machine was manually rebooted.

3) c1daf was recompiled and installed, with the problematic piece of code removed.

4) NTP timing was adjusted.

5) Frame Builder was restarted.

6) All models on c1lsc machine were restarted.

Attachment 1 shows the CDS status after the recovery. I wont be trying to run frequency warping immediately, I will first finish implementing the other harmless modules first.

  12319   Thu Jul 21 12:03:35 2016 varunUpdateCDSDAFI update: Humming noise in DAFI output

 

Summary: There was always a constant humming noise in the output of speakers of both the audio channels. Tried to resolve the problem. Details are given below:

Details: The source of the noise was the typical 60 Hz (and harmonics), ~13 mV peak to peak output, in at least three channels of the DAC. (two coming from the DAF module, and one not related to the DAF.) Attachment 1 shows the noise in both the DAF channels. As compared to that, the signal coming through the AGC weak, about 6 mV RMS, about the same order as noise. In order to resolve this, the gain of the AGC was increased, so that the RMS output voltage of the Fibox (FBAO, the one at the output) was about 1.23 V RMS. It is approximately equal to +4 dBu, which is the typical expected output of the Fibox, according to the datasheet.

  12320   Thu Jul 21 14:27:24 2016 varunUpdateCDSDAFI Update: Arbitrary Math block

Summary: I have added an arbitrary math block to the DAFI model, which takes two inputs, say X and Y, and can perform various unary and binary operations on them:

Details:

  • Unary Operations:
  1. Delay - There exists a text-based input to specify the amount of delay to be given to a particular signal.
  2. sin()
  3. cos()
  • Binary Operations:
  1. Weighted addition and multiplication: The output is calculated according to the relation: A*X + B*Y + C*X*Y. A, B, C are constant inputs, which can be given through text-based inputs in the GUI.
  2. MAX{X,Y}
  3. MIN{X,Y}

Attachment 1 shows the existing DAFI gui, updated with cascading of various DSP blocks, upto three levels, button-based ENABLE and DISABLE controls for all blocks except arb. math (the control on arb. math. is achieved by clicking on the block.) On clicking the arb. math block one is taken to the dedicated arb. math screen, which has enable buttons for all the processes listed above. A screenshot of this screen is in attachment 2. There is one control input, which controls all the unary operations on X and the binary operations on X and Y, and another control input which controls the unary operations on Y. switching on a particular arb. math process gives a particular control input, which choses the appropriate section of the code. At a time, only one process from the top grey block (corresponding to unary operations on X and binary operations on X and Y) and one process from the bottom grey block (corresponding to unary operations on Y) can be selected. Thus, the outputs which go from the arb. math block to the intermediate matrices (MATRIX1L or MATRIX2L) are:

a) Either an output of unary operation on X or a binary operation on X and Y, the specific one depending upon the control input,

    and

b) Output of a unary operation on Y, again the specific one depending upon the control input

Thus there is apparent asymmetricity in the action of the arb. math block on the two inputs. However, this is done in order to reduce to total number of outputs and control signals, and this can be easily taken care of by interchanging the inputs before the block. 

While compiling this code, the c1lsc machine had crashed once, it was found that this was due to a stray "printf()" command in the c code. This glich in the code now stands rectified  There is a possibility that the previous incidents of the code crashing could also be due to the existence of a printf() command. 

Preliminary Testing: I have done a preliminary testing of the arb math block, i.e. verified that on enabling the sin and cos processes, the output is less that 1, on swithching on the process of weighted avarage and multiplication, the output looks like it is right, for a few simple values of A, B, C, like 0, 1, etc. The delay block however is giving zero output for delay of more than 6 samples.

 

 

 

 

  12321   Thu Jul 21 15:03:13 2016 varunUpdateCDSDAFI Update

1) I have added the status summary of the DAFI block to the main FE status overview screen in the c1lsc cloumn. (attachment 1)

2) I have edited all the kissel matrix buttons appropriately, and given them appropriate lables. (attachment 2)

  12324   Thu Jul 21 22:02:35 2016 varunUpdateCDSDAFI update: Frequency warping

The code for frequency warping contained a "printf()" command, which had caused the system to crash in one another instance (refer elog 12320) . Hence, I tried running the code tody by removing this line. Unfortunately, this did not work. the model still crashed. Attached is the screenshot of the FE status.

  12336   Tue Jul 26 09:56:34 2016 ericqUpdateCDSc1susaux restarted

c1susaux (which controls watchdogs and alignments for all non-ETM optics) was down, the last BURT was done yesterday around 2PM. 

I restarted via keying the crate. I restored the BURT snapshot from yesterday.

  12361   Mon Aug 1 20:09:37 2016 ranaUpdateCDSDAFI Update

I found the DAFI screen as a button inside of the LSC screen - I think its more logically found from the sitemap, so I'll move it into there as well.

Quote:

1) I have added the status summary of the DAFI block to the main FE status overview screen in the c1lsc cloumn. (attachment 1)

2) I have edited all the kissel matrix buttons appropriately, and given them appropriate lables. (attachment 2)

Gautam and I noticed a 60 Hz + harmonics hum which comes from the DAFI. Its the noisiest thing in the control room. It goes away when we unplug the fiber coming into the control room FiBox receiver, so its not a ground loop on this end. Probably a ground loop at the LSC rack.

Upon further investigation we notice that the Fibox at the LSC rack had its gain turned all the way up to +70 dB. This seemed too much, we reduced it to ~20 (?) so that we could use more of the DAC range.  Also, it is powered by a AC/DC converter plugged in to the LSC rack power strip. We cannot use this for a permanent install - must power the FiBox using the same power supplies as are used for the LSC electronics. Probably we'll have to make a little box that takes the fused rack power of 15 V and turns it into +12 V with a regulator (max current of 0.15 A). Making sure that the FiBox doesn't pollute the rest of the LSC stuff with its nasty internal DC-DC converters.

We also put a high pass in the output filter banks of DAFI. For the PEM channels we put in a 60 Hz comb. WE then routed the Y-end Guralp in through the boxes and out the output, mostly bypassing the frequency shifting and AGC. It seems that there is still a problem with GUR2.

Does anyone know which one is GUR1 and which one is GUR2? I don't remember the result of the Guralp cable switching adventures - maybe Koji or Steve does. According to the trend it was totally dead before March and in March it became alive enough for us to see ~30 ADC counts of action, so way smaller than GUR111 or GUR snoopy or whatever its called.

  12539   Fri Oct 7 20:25:14 2016 KojiUpdateCDSPower-cycled c1psl and c1iool0

Found the MC autolocker kept failing, It turned out that c1iool0 and c1psl went bad and did not accept the epics commands.

Went to the rack and power cycled them. Burt resotred with the snapshot files at 5:07 today.

The PMC lock was restored, IMC was locked, WFS turned on, and WFS output offloaded to the bias sliders.

The PMC seemed highly misaligned, but I didn't bother myself to touch it this time.

  12542   Mon Oct 10 11:48:05 2016 gautamUpdateCDSPower-cycled c1susaux, realigned PMC, spots centered on WFS1 and WFS2

[Koji, Gautam]

We did the following today morning:

  1. I re-aligned the PMC - transmission level on the scope on the PSL table is now ~0.72V which is around what I remember it being
  2. The spot had fallen off WFS 2 - so we froze the output of the MC WFS servo, and turned the servo off. Then we went to the table to re-center the spot on the WFS. The alignment had drifted quite a bit on WFS2, and so we had to change the scale on the grid on the MEDM screen to +/-10 (from +/- 1) to find the spot and re-center it using the steering mirror immediately before the WFS. It would appear that the dark offsets are different on WFS1 and WFS2, so the "SUM" reads ~2.5 on WFS1 and ~0.3 on WFS2 when the spots are well centered
  3. Coming back to the control room, we ran the WFSoffsets script and turned on the WFS servo again. Trying to run the relief servo, we were confronted by an error message that c1susaux needed to be power cycled (again). This is of course the slow machine that the ITMX suspension is controlled by, and in the past, power cycling c1susaux has resulted in the optic getting stuck. An approach that seems to work (without getting ITMX stuck)  is to do the following:
    • Save the alignment of the optic, turn off Oplev servo
    • Move the bias sliders on IFO align to (0,0) slowly
    • Turn the watchdog for ITMX off
    • Unplug the cables running from the satellite box to the vacuum feedthrough
    • Power cycle the slow machine. Be aware that when the machine comes back on, the offset sliders are reset to the value in the saved file! So before plugging the cables back in, it would be advisable to set these to (0,0) again, to avoid kicking the optic while plugging the cables back in
    • Plug in the cables, restore alignment and Oplev servos, check that the optic isn't stuck
  4. Y green beat touch up - I tweaked the alignment of the first mirror steering the PSL green (after the beam splitter to divide PSL green for X and Y beats) to maximize the beat amplitude on a fast scope. Doing so increased the beat amplitude on the scope from about 20mVpp to ~35mVpp. A detailed power budget for the green beats is yet to be done

It is unfortunate we have to do this dance each time c1susaux has to be restarted, but I guess it is preferable to repeated unsticking of the optic, which presumably applies considerable shear force on the magnets...


After Wednesday's locking effort, Eric had set the IFO to the PRMI configuration, so that we could collect some training data for the PRC angular feedforward filters and see if the filter has changed since it was last updated. We should have plenty of usable data, so I have restored the arms now.

  12592   Wed Nov 2 22:56:45 2016 gautamUpdateCDSc1pem revamped

installing the BLRMS 2k blocks turned out to be quite non-trivial due to a whole host of CDS issues that had to be debugged, but i've restored everything to a good state now, and the channels are being logged. detailed entry with all the changes to follow.

  12595   Thu Nov 3 12:38:42 2016 gautamUpdateCDSc1pem revamped

A number of changes were made to C1PEM and some library parts. Recall that the motivation was to add BLRMS channels for all our suspension coils and shadow sensor PDs, which we are first testing out on the IMC mirrors.

Here is the summary:

BLRMS_2k library block

  • The name of the custom C code block in this library part was named 'BLRMSFILTER' which conflicted with the name of the function call in the C code it is linked to, which lead to compilation errors
  • Even though the part was found in /opt/rtcds/userapps/release/cds/c1/models and not in the common repository, just to be safe, I made a copy of the part called BLRMS_2k_40m which lives in the above directory. I also made a copy of the code it calls in /opt/rtcds/userapps/release/cds/c1/src

C1PEM model + filter channels

  • Adding the updated BLRMS_2k_40m library part still resulted in some compilation errors - specifically, it was telling me to check for missing links around the ADC parts
  • Eric suggested that the error messages might not be faithfully reporting what the problem is - true enough, the problem lay in the fact that c1pem wasn't updated to follow the namespace convention that we now use in all the RT models - the compiler was getting confused by the fact that the BLRMS stuff was in a namespace block called 'SUS', but the rest of the PEM stuff wasn't in such a block
  • I revamped c1pem to add namespace blocks called PEM and DAF, and put the appropriate stuff in the blocks, after which there were no more compilation errors
  • However, this namespace convention messed up the names of the filter modules and associated channels - this was resolved with Eric's help (find and replace did the job, this is a familiar problem that we had encountered not too long ago when C1IOO was similarly revamped...)
  • There was one last twist in that the model would compile and install, but just would not start. I tried the usual voodo of restarting all the models, and even did a soft reboot of c1sus, to no avail. Looking at dmesg, I tracked the problem down to a burt restore issue - the solution was to press the little 'BURT' button next to c1pem on the CDS overview MEDM screen as soon as it appeared while restarting the model

All the channels seem to exist, and FB seems to not be overloaded judging by the performance overnight up till the power outage. I will continue to monitor this...

GV Edit 3 Nov 2016 7pm:

I had meant to check the suitability of the filters used - there is a detailed account of the filters implemented in BLRMSFILTER.c here, and I quickly looked at the file on hand to make sure the BP filters made sense (see Attachment #1). These the BP filters are 8th order elliptical filters and the lowpass filters are16th order elliptical filters scaled for the appropriate frequency band, which are somewhat different from what we use on the seismometer BLRMS channels, where the filters are order 4, but I don't think we are significantly overloaded on the computational aspect, and the lowpass filters have sufficiently steep roll-off, these should be okay...

ELOG V3.1.3-