40m QIL Cryo_Lab CTN SUS_Lab TCS_Lab OMC_Lab CRIME_Lab FEA ENG_Labs OptContFac Mariner WBEEShop
  40m Log, Page 89 of 339  Not logged in ELOG logo
ID Date Author Type Categoryup Subject
  5558   Tue Sep 27 15:33:03 2011 JenneUpdateComputersMaking models, wreaking havoc

[Jenne, Mirko, Den]

We have entered into an adventure in model compiling.  What follows is a stream-of-consciousness report of what the hell we're doing, so Jamie can figure it out and fix it if everything goes to hell.

Note that for the first part of things, we have used a new version of the Adaptive XFCODE, which Mirko and Den modified last night to be able to handle multiple control signal inputs.  

On c1lsc, make uninstall-daq-c1oaf, make clean-c1oaf, make c1oaf.

***ERROR: The following IPCx RECEIVER module(s) not found in the file /opt/rtcds/caltech/c1/chans/ipc/C1.ipc:

                C1:RFM_OAF_MCL

On c1sus, make uninstall-daq-c1sus, make clean-c1sus, make c1sus.  (This was an accident.  I should have been making c1rfm.  Oops.)  Then make install-c1sus.  It looks like this automatically did make install-daq-c1sus and make install-screens-c1sus, so I'm not doing those. 

On c1sus, make uninstall-daq-c1rfm, make clean-c1rfm, make c1rfm.

***ERROR: The following IPCx RECEIVER module(s) not found in the file /opt/rtcds/caltech/c1/chans/ipc/C1.ipc:

                C1:IOO-RFM_MCL


On c1ioo, make uninstall-daq-c1ioo, make clean-c1ioo, make c1ioo.  No errors.

On c1lsc, make c1oaf.  Here's some of the ouptut, with some of the error stuff:

: warning: ISO C90 forbids mixed declarations and code
/opt/rtcds/caltech/c1/core/branches/branch-2.1/src/fe/c1oaf/../controller.c:2954: warning: ISO C90 forbids mixed declarations and code
make[3]: *** [/opt/rtcds/caltech/c1/core/branches/branch-2.1/src/fe/c1oaf/c1oaffe.o] Error 1
make[2]: *** [_module_/opt/rtcds/caltech/c1/core/branches/branch-2.1/src/fe/c1oaf] Error 2
make[2]: Leaving directory `/usr/src/linux-2.6.34.1-cs'
make[1]: *** [default] Error 2
make[1]: Leaving directory `/opt/rtcds/caltech/c1/core/branches/branch-2.1/src/fe/c1oaf'
make: *** [c1oaf] Error 2

Again on c1lsc, make clean-c1oaf, make c1oaf.  Here are some things:

Warning:  variable "sysnum" is used but not declared.

In file included from build/c1oafepics/c1oaf.i:38:
src/include/fmReadCoeff.h:4:1: warning: "NO_FM10GEN_C_CODE" redefined
<command-line>: warning: this is the location of the previous definition

build/c1oafepics/c1oaf.i:5156: warning: passing argument 2 of 'strcpy' discards qualifiers from pointer target type
/usr/include/string.h:127: note: expected 'const char * __restrict__' but argument is of type 'volatile char *'

/opt/rtcds/caltech/c1/core/branches/branch-2.1/src/fe/c1oaf/../../include/drv/inputFilterModule1.h:5: note: expected 'double *' but argument is of type 'long unsigned int *'

/opt/rtcds/caltech/c1/core/branches/branch-2.1/src/fe/c1oaf/../controller.c:2780: warning: ISO C90 forbids mixed declarations and code

make[3]: *** [/opt/rtcds/caltech/c1/core/branches/branch-2.1/src/fe/c1oaf/c1oaffe.o] Error 1
make[2]: *** [_module_/opt/rtcds/caltech/c1/core/branches/branch-2.1/src/fe/c1oaf] Error 2
make[1]: *** [default] Error 2
make: *** [c1oaf] Error 2

Again, on c1lsc, make clean-c1oaf, make c1oaf.  More errors, pretty similar.  Then we changed the name of the adaptive filtering code, so maybe it will work now?  We had called the block "TOP_XFCODE", but that was the name of the old .c code.  The block used to be called "XFCODE", in a subsystem "TOP".  So now we named the .c code "ADAPT_XFCODE" since the block is "XFCODE", and the subsystem is "TOP".

Again, on c1lsc, make clean-c1oaf, make c1oaf.  Errors, they look the same.

Mirko is now modifying c1oaf.mdl to look more like the old version, with only one control signal input, so that we can use the old XFCODE that has been around for years.

First, we completely took out the .c code entirely.  Now the c1oaf.mdl is just signals and matricies, no c-code is called.  We did make clean-c1oaf, make c1oaf, and pretty much all of the same errors are present.
 

We took out the buses, and did make clean-c1oaf, make c1oaf, and we got a whole lot of warnings, but no "Error 2"'s.  This seems good.  We're going to try replacing those buses with Muxes, and see how that goes.

Now we're going to try to install the c1oaf, because maybe all the errors and shit we're seeing is just useless crap, and there aren't actually problems...here we go!

That seemed to work, and the c1oaf model on the GDS status screen seems happy.  Numbers are moving around, which is my only current diagnostic.

Okay, now Mirko is going back to the full, new c1oaf, but replacing the Buses with Muxes. 

Did a make clean-c1oaf, make c1oaf, got errors again.

Once again removed the .c code.  Just put in a matrix instead. Did make clean-c1oaf, make c1oaf.  No errors. 

Den did a reorganization of the .c code, and we put it back in to the simulink model.  Trying again the making stuff.  Fail..Basically the same errors as before.

Next up:  Putting in .c-code, but something which basically does nothing.  Just defines all the outs as zeros.  Make stuff.  Still had problems, same errors.  Grrrr, argh. 

Found the RCG manual:  T080135-v4.  In it, when talking about including c-code, it had an example of totally simple code.  We tried out their version of simple code, and it worked.  No errors!  Now to figure out what is same and different between our simple code and theirs.

 PUT THE RIGHT STUFF in the Block Properties for the c-code, including name of the .c file, and path to the .c file.  This is critical!!  Now we can make some of our simple versions work, but not all.  We're slowly increasing complexity of our c-code...

 At some point in the last hour, I tried a make install-c1oaf, and then checked the screens, and they all had bad white boxes.  So even though the model seemed to compile (at one point), the channels and screens aren't happy yet.  But that's really a project for after the code compiles happily.

Okay, some progress was made to get the c-code running, and compiling, but it's not all there yet.  We're putting in a simple, useless version of the c-code so we can try compiling everything else.

Everything is compiled, installed, there are no red lights on the GDS_STATUS screen.  All seems okay for locking for tonight.

 

  5565   Wed Sep 28 14:15:40 2011 JenneUpdateComputersEdits to c1pem, c1oaf

[Mirko, Jenne]

Mirko edited c1pem to have some new BLRMS channels.

I added a master Enable switch to the c1oaf.

Both were compiled, and restarted.  fb rebooted.  All looks okay (hopefully)

  5568   Wed Sep 28 21:23:23 2011 MirkoUpdateComputersISCY FE network card / cable not ok

[Mirko,Jenne]

We discovered that the left network cable is not rigidly connected to the back of the ISCY FE computer. You can easily pull it out a mm disconnecting it. It should click rigidly in place. Not clear if it's the cable or the network card.

  5569   Wed Sep 28 21:28:34 2011 MirkoUpdateComputersTorturing control computers. Fine again now

[Mirko, Jenne]

We tried to run an extended version of Matt's LMS adaptive filter c-code. We got the extension to compile separately in gcc first. Then after some tweaking of the code we could make-install c1oaf with the c-code.

This killed c1lsc (the FE computer running c1oaf). Not responding to ssh or even pings. We replaced the bad c-code with harmless code, then reset c1lsc via the hardware button. While looking for c1lsc we discovered the problem with c1iscy network card (see previous entry).

After c1lsc reboot, restart of the FB, and a BURT restore not ok yet

  5571   Wed Sep 28 22:25:25 2011 JenneUpdateComputersTorturing control computers. Fine again now

Quote:

[Mirko, Jenne]

We tried to run an extended version of Matt's LMS adaptive filter c-code. We got the extension to compile separately in gcc first. Then after some tweaking of the code we could make-install c1oaf with the c-code.

This killed c1lsc (the FE computer running c1oaf). Not responding to ssh or even pings. We replaced the bad c-code with harmless code, then reset c1lsc via the hardware button. While looking for c1lsc we discovered the problem with c1iscy network card (see previous entry).

After c1lsc reboot, restart of the FB, and a BURT restore not ok yet

 We had lots of trouble damping the Vertex Suspensions after this craziness.  A symptom was that even if all of the damping servos on an optic were OFF, and I turned the watchdog on (LSC is disabled, so no OAF siganls, no LSC signals), there were signals going to the coils. 

We did a reboot of the c1sus computer, did another BURT restore, and the optics started damping happily.   Burt restore, at least for c1susepics and c1mcsepics, seems to not be happening automatically.  I thought it was supposed to happen when the model was restarted?

Things now appear to be normal again.

  5573   Thu Sep 29 00:16:35 2011 DenUpdateComputersSegmentation fault fixed.

The OAF c-code crashed because of the segmentation fault. We've created arrays of static variables

    static int pst[nDOF];
    static int isFirst[nDOF];
    static adaptive_filter_state state[nDOF];

an tried to give in the to the ITERATE - function their current values

        datOut[i] = ITERATE(iterateDatIn, iterateNIn, pst[i], isFirst[i], state[i]);

ITERATE function was declared as

double ITERATE(double *datIn, int nIn, int pst, int isFirst, adaptive_filter_state state) {}

Here the segmentation fault comes out. Static variables are meant to be created only once but here in the function ITERATE we try to create them once again in a local form, because we give the variables by their values.

Instead, we must give the variables by their pointer, then the variables won't be created again during the function call and will be changed in the function.

        datOut[i] = ITERATE(iterateDatIn, iterateNIn, &pst[i], &isFirst[i], &state[i]);

       double ITERATE(double *datIn, int nIn, int *pst_s, int *isFirst_s, adaptive_filter_state *state_s)

In order not to change significantly Matt's code and use his notations we can add in the ITERATE function

    int pst = *pst_s;
    int isFirst = *isFirst_s;
    static adaptive_filter_state state;
    state = *state_s;

..................................Matt's code.........................................

    *pst_s = pst;
    *isFirst_s = isFirst;
    *state_s = state;

I've tested the program, now it does not give any segmentation faults and conserves memory that it uses.

  5577   Thu Sep 29 20:37:12 2011 MirkoUpdateComputersNew c1oaf c-code: Breaking in new way

[Mirko, Jenne]

Programmed a new implementation of the LMS in C. Compiles fine in gcc. The full code still kills c1lsc computer. Tried to go through the code uncommenting more and more. Not perfect in reproducability. The attached version should compile and keep c1oaf running, but not actually produce an adaptive filter. At some point the code just keeps c1oaf from starting up. Leaves the c1lsc computer in working order. At some point I got error messages like ..................................................................
CA.Client.Exception...............................................
    Warning: "Identical process variable names on multiple servers"
    Context: "Channel: "C1:OAF-ADAPT_CORR_BS_Name08", Connecting to: 192.168.113.62:50970, Ignored: c1lsc:34533"
    Source File: ../cac.cpp line 1209
    Current Time: Thu Sep 29 2011 19:18:17.219208306
..................................................................
CA.Client.Exception...............................................
    Warning: "Identical process variable names on multiple servers"
    Context: "Channel: "C1:OAF-ADAPT_CORR_BS_Name09", Connecting to: 192.168.113.62:50970, Ignored: c1lsc:34533"
    Source File: ../cac.cpp line 1209
    Current Time: Thu Sep 29 2011 19:18:17.225999915
..................................................................
CA.Client.Exception...............................................
    Warning: "Identical process variable names on multiple servers"
    Context: "Channel: "C1:OAF-ADAPT_CORR_ETMX_SW1R", Connecting to: 192.168.113.62:50970, Ignored: c1lsc:34533"
    Source File: ../cac.cpp line 1209
    Current Time: Thu Sep 29 2011 19:18:17.243037042
..................................................................

This usually indicates that there are multiple carepeater running. Didn't find where that would be. After rebooting c1lsc and c1sus a couple of times everything seems fine.

  5733   Tue Oct 25 01:19:17 2011 SureshUpdateComputersfb restarted and c1ioo model committed to svn

When I installed the new model I restarted the fb between 1 and 1:30 AM PDT Oct 25, 2011

  5757   Fri Oct 28 15:33:06 2011 JenneUpdateComputersNifty screen generator

Suresh showed me a cool script that Mirko made, but didn't elog about.

You tell the script what filter banks you want, and it creates a screen for each with a bunch of different filter module display formats.  Then you can copy the format you like into the actual screen you're modifying. 

Currently PEM, LSC and IOO (and maybe others?) have "fmX" folders inside their medm/c1.../master folders.  For each subsystem, we need to copy this folder, and modify the generic .adl file so that it puts in the correct subsystem letters.  Once this is done, you can just run the generateFMscreens.py after putting in your filter bank names.

  5758   Fri Oct 28 15:45:52 2011 MirkoUpdateComputersNifty screen generator

Quote:

Suresh showed me a cool script that Mirko made, but didn't elog about.

You tell the script what filter banks you want, and it creates a screen for each with a bunch of different filter module display formats.  Then you can copy the format you like into the actual screen you're modifying. 

Currently PEM, LSC and IOO (and maybe others?) have "fmX" folders inside their medm/c1.../master folders.  For each subsystem, we need to copy this folder, and modify the generic .adl file so that it puts in the correct subsystem letters.  Once this is done, you can just run the generateFMscreens.py after putting in your filter bank names.

 Wasn"t me.

  5778   Tue Nov 1 18:45:48 2011 JenneUpdateComputersAllegra's screens

I was trying to give Allegra a second head, and it didn't quite work.  It's still in progress.  Steve, you might not like how I've 'mounted' the second monitor, so we can talk about something that might work tomorrow.

  5780   Tue Nov 1 23:13:28 2011 ZachUpdateComputersNDS2 channel files

I did some messing around with the NDS2 config and channel files and things seem to be working as expected... for now. SENSOR channel data can be acquired for all sensors on all hanging optics.

What I did:

  • NDS2 gets its channel lists from .../users/jzweizig/nds2-mafalda/nds2.conf, which is called in the start-nds2 script. Within this, there are channel.file lines that specify which channels are available for raw and trend data. The four files that were listed were:
    • C-R-raw-channel_list.txt
    • C-M-ChanList.txt
    • C-T-ChanLIst.txt
    • C-R-online-channel_list.txt (this one was listed after a hashed line, which was suspicous---see below)
  • I noticed that a grep for SENSOR only returned lines for non-MC mirrors in both "R" files
  • I also noticed that calling NDS2_GetChannels('mafalda.martian:31200') did not return any non-suffixed (i.e., raw) channel names for MCx_SENSOR channels, while non-MC SENSOR channels each had two non-suffixed listings. I thought this was strange.
  • I manually added the line "C1:SUS-MC1_SENSOR_UL 0 real_4 2048 C-R" to one of the "R" channel files, then restarted the NDS2 server, and that channel was still not served. I figured that the second "R" channel file might have been left in the config file as a mistake, so I commented it out, restarted the NDS2 server, and was able to get MC1_SENSOR_UL data. I have left the comment-out there, with a signed EDIT.
  • Wary of (and too lazy to) manually add lines for all 5 sensors for each MC mirror, I decided to try generating a channel file using the most recent .gwf file in the frames, as indicated in Joe and John's elog post. To do this, while in .../nds2-mafalda/, I ran:
    • /cvs/cds/caltech/users/jzweizig/nds2-server/bin/buildChannelList /frames/full/10042/C-R-1004246528-16.gwf > C-R-raw-channel_list.txt
  • A grep for SENSOR in the new C-R-raw-channel_list.txt now returned lines for all MC mirror sensors... BUT NOT FOR ETMY(?!). I tried some slightly older .gwf files (all from today), but the ETMY files never showed up. I had no choice but to enter them manually. Another odd thing is that the channel file generated this way seems to be fairly jumbled up, in the sense that there is no clear top-town order (e.g. SUS-BS_blah then SUS-ETMX_blah). Instead, some SUS-BS channels are here, some are after SUS-ETMX or SUS-PRM channels, etc. Look at the file to know what I mean.
  • The original raw channel file is there as C-R-raw-channel_list.txt.bak.1004247481.

In any case, as I said, everything appears to be working now, but as soon as we try to generate a new channel file using the prescribed means, there will inevitably be channels omitted. Someone who knows more than me should get to the bottom of this and wiki a strict, detailed procedure for how this is to be done.

  5830   Mon Nov 7 14:50:19 2011 JenneUpdateComputersISCEX is having a bad day

I clicked on the FE status screen, just to check on things, and everything on the c1iscex section was red (the IOP and c1scx).  Upon deciding that was probably a bad thing, I did a soft reboot from the control room.  Now the IOP says "NO SYNC", and the c1scx status thing is totally frozen. 

I have sent Jamie a whiny email. He promises to be here soon to fix it.

  5836   Mon Nov 7 17:27:28 2011 jamieUpdateComputersISCEX IO chassis timing slave dead

It appears that the timing slave in the c1iscex IO chassis is dead.  It's front "link" lights are dark, although there appears to be power to the board (other on-board leds are lit).  These front lights should either be on and blinking steadily if the board is talking to the timing system, or blinking fast if there is no connection to the timing distribution box.  This likely indicates that the board has had some sort of internal failure.

Unfortunately Downs has no spare timing slave boards lying around at the moment; they're all stuffed in IO chassis awaiting shipping.  I'm going to email Rolf about stealing one, and if he agrees we'll work with Todd Etzel to pull one out for a transplant

  5843   Tue Nov 8 19:08:21 2011 MirkoHowToComputersNew DV

Quote:
To use the new ligoDV (previously GEO DV) to look at 40m data, open up a matlab, set up for mDV as usual,
and then from the /cvs/cds/caltech/apps/ligoDV/ directory, type 'ligoDV'.

Then select which NDS server you want to look at and then start clicking to get some plots.

To start ligodv go in matlab to /cvs/cds/caltech/apps/ligoDV/ and call ligodv. Ligodv will start up when you are in another directory, but will give strange errors. Only seems to work with NDS2 server mafalda port 31200. This doesn't have all channels. When pointing it to fb port 8088 it freezes when you try to adjust the start/stop time. Make sure to ask for the correct UTC time, not the local time.
  5847   Wed Nov 9 13:44:04 2011 MirkoUpdateComputersNDS1 missing channels in matlab

The Matlab NDS1 access seems to only work for some channels. With other channels it just hangs 'Busy' and does nothing.
Below you can see some channels that make matlab hang. Everyting happened on allegra. I tried compiling the NDS1 sources (from https://www.gravity.phy.syr.edu/dokuwiki/doku.php?id=ligodv:nds1_ligodv_install ) into mex files myself. Same result. I
 

a=NDS_GetChannels('fb:8088'); %/cvs/cds/caltech/apps/linux64/share/matlab/NDS_GetChannels.m
%data=NDS_GetData({'C1:IOO-MC_F_DQ'},1004826500,100,'fb:8088',a)     %Works
%data=NDS_GetData({'C1:IOO-WFS1_PIT_IN1_DQ'},1004826500,100,'fb:8088',a)     %Works
data=NDS_GetData({'C1:LSC-AS11_I_OUT'},1004826500,100,'fb:8088',a)         %Doesn't work, hangs
%%%which NDS_GetData.m: /cvs/cds/caltech/apps/linux64/share/matlab/NDS_GetData.m

  5853   Wed Nov 9 17:41:17 2011 JenneUpdateComputersETMX restored

Jamie did computer magic.  I burt restored scxepics, and restored ETMX damping.

  5863   Thu Nov 10 16:26:46 2011 MirkoUpdateComputersFirefox kills elog

Had to restart the elog many times. For some reason firefox 8 on Win 7 kills the elog pretty consistently when trying to make a new entry. IE9 works fine ....

  5880   Sat Nov 12 02:27:00 2011 SureshUpdateComputersC1IOO front end suddenly froze. Was restarted remotely

[Koji Suresh]

No one was messing with the c1ioo or any other machine.   The medm screens for WFS and MC alignment froze while I was working on Rossa.

There were number of red lights pertaining to c1ioo machine on the CDS_FR_STATUS screen.  So we logged into c1ioo   from Rossa and restarted it with 'sudo shutdown -r now'.  It came back up but the C1IOO_MC_TRANS_SUM, P and Y signals were not available on the C1IOO LOCKMC screen.

I saw several messages similar to the one here


Sat Nov 12 02:09:14 PST 2011

  medmCAExceptionHandlerCb: Channel Access Exception:
  Channel Name: Unavailable
  Native Type: Unavailable
  Native Count: 0
  Access: Unavailable
  IOC: Unavailable
  Message: Virtual circuit disconnect
  Context: c1ioo.martian:43553
  Requested Type: TYPENOTCONN
  Requested Count: 0
  Source File: ../cac.cpp
  Line number: 1126

 

The MC autolocker script wasnt running.  The heartbeat bit was not blinking on the MC_LOCKMC screen.  So we manually restarted the script.  Hopefully it will return to normal operation.

I restarted the fb at Sat Nov 12 02:12:19 PST 2011  in an attempt to see this resolves the problem.

It didnt.

 

  5881   Sat Nov 12 02:44:18 2011 SureshUpdateComputersC1IOO front end suddenly froze. Was restarted remotely

Quote:

[Koji Suresh]

No one was messing with the c1ioo or any other machine.   The medm screens for WFS and MC alignment froze while I was working on Rossa.

There were number of red lights pertaining to c1ioo machine on the CDS_FR_STATUS screen.  So we logged into c1ioo   from Rossa and restarted it with 'sudo shutdown -r now'.  It came back up but the C1IOO_MC_TRANS_SUM, P and Y signals were not available on the C1IOO LOCKMC screen.

I saw several messages similar to the one here


Sat Nov 12 02:09:14 PST 2011

  medmCAExceptionHandlerCb: Channel Access Exception:
  Channel Name: Unavailable
  Native Type: Unavailable
  Native Count: 0
  Access: Unavailable
  IOC: Unavailable
  Message: Virtual circuit disconnect
  Context: c1ioo.martian:43553
  Requested Type: TYPENOTCONN
  Requested Count: 0
  Source File: ../cac.cpp
  Line number: 1126

 

The MC autolocker script wasnt running.  The heartbeat bit was not blinking on the MC_LOCKMC screen.  So we manually restarted the script.  Hopefully it will return to normal operation.

I restarted the fb at Sat Nov 12 02:12:19 PST 2011  in an attempt to see this resolves the problem.

It didnt.

 

 The problem was resolved after I burtrestored (c1mcs c1ioo and c1rfm) epics snapshots.

 

  5886   Mon Nov 14 12:16:41 2011 JenneUpdateComputersOAF model died for unknown reason

I am meditating on the OAF, and had it running and calculating things.  I had the outputs disabled so I could take reference traces in DTT, but the Adapt block was calculating for MCL.  At some point, all the numbers froze, and the CPU meter had gone up to ~256ms.  Usually it's around ~70 or so for the configuration I had (2 witness sensors and one degree of freedom enabled....no c-code calculations on any other signals).  The "alive" heartbeat was also frozen.

I ssh'ed into c1lsc, ran ./startc1oaf in the scripts directory, and it came back just fine.

Anyhow, I don't know why it got funny, but I wanted to record the event for posterity.  I'm back to OAFing now.

  5889   Mon Nov 14 21:22:48 2011 ranaConfigurationComputersprimetime RSYNC slowing down NODUS

nodus:elog>w; who ; date
  9:20pm  up 44 day(s),  5:14,  5 users,  load average: 0.29, 1.04, 1.35
User     tty           login@  idle   JCPU   PCPU  what
controls pts/1         9:18pm            5         -tcsh
controls pts/2         2:37pm  6:39  25:02  25:02  /opt/rsync/bin/rsync -avW /cvs/c
controls pts/3         9:14pm                      w
controls pts/4         4:20pm  1:56   5:02   5:02  ssh -X rosalba
controls pts/8         8:23pm    47   4:03         -tcsh
controls   pts/1        Nov 14 21:18    (pianosa.martian)
controls   pts/2        Nov 14 14:37    (ldas-cit.ligo.caltech.edu)
controls   pts/3        Nov 14 21:14    (rosalba)
controls   pts/4        Nov 14 16:20    (192.168.113.128)
controls   pts/8        Nov 14 20:23    (gwave-103.ligo.caltech.edu)
Mon Nov 14 21:20:48 PST 2011

we will ask the man to stop running backups at this time of night...

  6016   Sat Nov 26 07:22:20 2011 sureshUpdateComputers 

c1sus has been shutdown so that the optics dont bang around.  This is because the watch dogs are not working.

  6108   Mon Dec 12 16:30:17 2011 JenneUpdateComputersDid someone just do something to fb??

Dataviewer couldn't connect to the framebuilder, so I checked the CDS status screen, and all the fb-related things on each model went white, then red, then computer-by-computer they came back green.  Now dataviewer works again.  Is someone secretly doing shit while not in the lab???  Not cool man!

  6112   Tue Dec 13 11:51:33 2011 JamieUpdateComputersDid someone just do something to fb??

Quote:

Dataviewer couldn't connect to the framebuilder, so I checked the CDS status screen, and all the fb-related things on each model went white, then red, then computer-by-computer they came back green.  Now dataviewer works again.  Is someone secretly doing shit while not in the lab???  Not cool man!

This happens on occasion, and I have reported it to the CDS guys.  Something apparently causes the framebuilder to crash, but I haven't figured out what it is yet.  I doubt this particular instance had anything to do with remote futzing.

  6117   Wed Dec 14 12:22:00 2011 VladimirHowToComputersligo_viewer installed on pianosa

I made a test installation of ligo_viewer in /users/volodya/ligo_viewer-0.5.0c . It runs on pianosa (the Ubuntu machine) and needs Tcl/Tk 8.5.

 

To try it out run the following command on pianosa:

cd /users/volodya/ligo_viewer-0.5.0c/

./ligo_viewer.no_install

 

Press "CONNECT" to connect to the NDS server and explore. There are slides describing ligo_viewer at http://volodya-project.sourceforge.net/Ligo_viewer.pdf

 

Installation notes:

Use /users/volodya/ligo_viewer-0.5.0c.tgz or later version - it has been updated to work with 64 bit machines.

Make sure Tcl and Tk development packages are installed. You can find required packages by running

apt-file search tclConfig.sh

apt-file search tkConfig.sh

If apt-file returns empty output run apt-file update

Unpack ligo_viewer-0.5.0c.tgz, change into the created directory.

Run the following command to configure:

export CFLAGS=-I/usr/include/tcl8.5
./configure --with-tcl=/usr/lib/tcl8.5/ --with-tk=/usr/lib/tk8.5/

This works on Ubuntu machines. --with-tcl and --with-tk should point to the directories containing tclConfig.sh and tkConfig.sh correspondingly.

Run "make".

You can test the compilation with ./ligo_viewer.no_install

If everything works install with make install

If Tcl/Tk 8.5 is unavailable it should work with Tcl/Tk 8.3 or 8.4

 

 

  6157   Tue Jan 3 15:45:04 2012 JenneUpdateComputersFB?

Is there a reason the framebuilder status light is red for all the front ends?

Also, I reenabled PRM watchdog.

  6159   Tue Jan 3 15:49:27 2012 JamieUpdateComputerspossible front-end timing issue

Quote:

Is there a reason the framebuilder status light is red for all the front ends?

Also, I reenabled PRM watchdog.

Apparently there is a bug in the timing cards having to do with the new year roll-over that is causing front-end problems.  From Rolf:

For systems using the Spectracom IRIG-B cards for timing information, the code did not properly roll over the time for
2012 (still thinks it is 2011 and get reports from DAQ of timing errors (0x4000)). I have made a temporary fix for this
in the controller.c code in branch-2.3, branch-2.4 and release 2.3.1. 

I was going to check to see if the 40m is suffering from this. I'll be over to see if that's the problem.

  6168   Wed Jan 4 09:06:50 2012 steveUpdateComputerspossible front-end timing issue

Quote:

Quote:

Is there a reason the framebuilder status light is red for all the front ends?

Also, I reenabled PRM watchdog.

Apparently there is a bug in the timing cards having to do with the new year roll-over that is causing front-end problems.  From Rolf:

For systems using the Spectracom IRIG-B cards for timing information, the code did not properly roll over the time for
2012 (still thinks it is 2011 and get reports from DAQ of timing errors (0x4000)). I have made a temporary fix for this
in the controller.c code in branch-2.3, branch-2.4 and release 2.3.1. 

I was going to check to see if the 40m is suffering from this. I'll be over to see if that's the problem.

 The problem is the same as yesterday.

  6171   Wed Jan 4 16:40:52 2012 JamieUpdateComputersfront-end fb communication restored

Communication between the front end models and the framebuilder has been restored.  I'm not sure exactly what the issue was, but rebuilding the framebuilder daqd executable and restarting seems to have fixed the issue.

I suspect that the problem might have had to do with how I left things after the last attempt to upgrade to RCG 2.4.  Maybe the daqd that was running was linked against some library that I accidentally moved after starting the daqd process.  It would have kept running fine as was, but if the process died and was attempted to be started again, it's broken linking might have kept it from running correctly.  I don't have any other explanation.

It turns out this was not (best I can tell) related to the new year time sync issues that wer seen at the sites.

  6243   Fri Feb 3 10:48:24 2012 DenUpdateComputersc1lsc kernel

This morning I killed again c1lsc kernel with the new realization of fxlms algorithm. It works fine with gcc compiler during the tests. However, smth forbidden for the kernel is going on. I'll spend some more time on investigatin it. Interesting thing is that I did not even pressed "On" at the OAF MEDM screen to make the code running. c1lsc suspended even before. May be there is some function-name mismatch.

After c1lsc suspention I recomiled back non-working code and rebooted c1lsc. c1sus is also bad after c1lsc reboot as they communicate. I killed x04, lsc, ass, oaf models on the c1lsc computer and sus, mcs, rfm, pem on the c1sus computer. Then I restarted x02 model and restored its burt snapshot from 08:07. After I started all models back and restored their burt snapshots from 08:07. Then I diag reset all started models.

Before starting new fxlms code I've shutted down all the optics so that possible c1lsc suspention would not make them crazy. After reboot I turned the coils back. Everything seems to work fine.

  6249   Fri Feb 3 17:29:28 2012 DenUpdateComputersc1lsc kernel

The reason I've killed the c1lsc kernel was the following - when the code starts to run, it initializes some parameters and this takes ~0.2 msec per dof. Now, the old code did nothing with a DOF if C1:OAF-ADAPT_???_ONOFF == OFF. My code still initialized the parameters but then does nothing because no witness channels are given. But it spends 8*0.2 = 1.6 msec for initializing all 8 dof. As the code is called with frequency 2k, this was the reason for crashing. Now I've corrected my code, it compiles, runs and does not kill c1lsc. However, the old code would also kill the kernel if all DOF are filtered. So, when we'll use all 8 DOF, we'll have to split variable initialization.

But this is not the biggest problem. C1OAF model must be corrected, because, as for now, all 8 DOF call the same ADAPT_XFCODE function. As this function uses static variables, they will be all messed up by different DOF signals.

  6314   Fri Feb 24 16:10:48 2012 mikeUpdateComputersPyNDS and a Plot

Power Spectral Density plot using PyNDS, comparing 5 fast data channels for ETMX.

**EDIT** Script here:

import nds
import numpy as np
import matplotlib.pyplot as plt
import time
daq=nds.daq('fb', 8088)
channels=daq.recv_channel_list()
e=0
start=int(time.time()-315964819)
rqst=['C1:SUS-ETMX_SENSOR_UR','C1:SUS-ETMX_SENSOR_UL','C1:SUS-ETMX_SENSOR_LL','C1:SUS-ETMX_SENSOR_LR','C1:SUS-ETMX_SENSOR_SIDE']    #Requested Channels
for c in channels:
    if c.name in rqst:
        daq=nds.daq('fb', 8088)
        data=daq.fetch(start-100, start, c.name)
        vars()['psddata'+str(e)], vars()['psdfreq'+str(e)]=plt.psd(data[0],NFFT=16384,Fs=c.rate)
        vars()['label'+str(e)]=c.name
        e+=1
plt.figure(1)
plt.clf()
plt.title('PSD Comparison')
plt.grid(True, which='majorminor')
plt.xlabel(r'Frequency $Hz$')
plt.ylabel(r'Decibels $\frac{dB}{Hz}$')       
for x in np.arange(0,e):
    plt.loglog(psdfreq0, 10*vars()['psddata'+str(x)], label=vars()['label'+str(x)])
plt.legend()
plt.show()

  6316   Fri Feb 24 18:59:04 2012 JenneUpdateComputersPyNDS and a Plot

Quote:

Power Spectral Density plot using PyNDS, comparing 5 fast data channels for ETMX.

 Is there any stuff to install, etc?  Y'know, for those of use who don't really know how to use computers and stuff....

  6341   Wed Feb 29 17:32:11 2012 MikeUpdateComputersPyNDS and a Plot

Quote:

Quote:

Power Spectral Density plot using PyNDS, comparing 5 fast data channels for ETMX.

 Is there any stuff to install, etc?  Y'know, for those of use who don't really know how to use computers and stuff....

 No new stuff for these computers.  Everything should be installed already.

  6431   Tue Mar 20 17:50:44 2012 SureshUpdateComputersBeam Scan machine fixed

There was something wrong with the Beam Scan PC.  The  mouse and screen were not responding and the PC was asking for drivers for any new hardware that we plugged in.  We called in the services of Junaid and co. since we do not have a Win98 Second Edition installation disk in the lab.   Junaid came with the disk, we changed the screen and the mouse and installed everything. 

We tried to get the network going on the PC so that we could update stuff easily over the net.  This didnt succeed. For now, we still have to depend on a Win98se CD to get drivers if any new hardware is connected to this machine. 

For future reference, some notes:

1)  We will get a copy of Win98SE for the lab from Junaid

2) We have to use a USB mouse from Dell. We have several spares of this. The drivers for these are present in the machine. 

 

 

The Beam Scan is working okay now.  We will proceed with the beam profile measurements.  

  6434   Wed Mar 21 19:12:27 2012 steveUpdateComputersAC power back on both ends

Quote:

Quote:

ETMY sus damping was disabled. Green locking laser and associated electronics turned off. Computers and power supplies turned off at rack 1Y4

The electricians picking up ac power from 1Y4 manual disconnect box and installing conduit line to ISCT-ETMY east end optical table.

There will be no more daisy chaining this way. 

 The power is back on at ETMY . c1iscey has not been restarted.

Now I'm turning ac power off at ETMX for the same job to be done.

 The power was turned back on at 4pm It took some time for Suresh to restart the computers. We have damping but things are not perfect yet. Auto BURTH did not work well.

  6463   Wed Mar 28 21:15:53 2012 ranaOmnistructureComputersWireless router for GC

I installed a NETGEAR Wireless Router (WPN824N) today on the 131 network. The admin password for it as well as the wireless access password are in the usual places.

The SSID is 40EARTH. I have set it to allow WPA as well as WPA2 access, so the speed is only 54 Mbps for now. In a year or so, we can turn off the WPA support and up the speed.

  6465   Thu Mar 29 13:23:05 2012 JenneOmnistructureComputersWireless router for GC

Quote:

I installed a NETGEAR Wireless Router (WPN824N) today on the 131 network. The admin password for it as well as the wireless access password are in the usual places.

The SSID is 40EARTH. I have set it to allow WPA as well as WPA2 access, so the speed is only 54 Mbps for now. In a year or so, we can turn off the WPA support and up the speed.

 This router was confiscated by the GC guys this morning around ~10am.  They barged in and said that someone at the 40m had connected a new router, and we had magically taken down half of the GC network.  The cable was plugged in to the wrong port on the back of the router. 

Junaid / Christian said that they would "secure" the router, and then reinstall it.  Apparently just having a password didn't satisfy them.  This was the compromise, versus them just taking the router and never bringing it back.

 

  6467   Thu Mar 29 19:13:56 2012 JamieOmnistructureComputersWireless router for GC

I retrieved the newly "secured" router from Junaid.  It had apparently been hooked up to the GC network via it's LAN port, but it's LAN services had no been shut off.  It was therefore offering a competing DHCP server, which will totally hose a network.  A definite NONO.

The new SSID is "40mWiFi", it's WPA2, and the password is pasted to the bottom of the unit (the unit is back in it's original spot on the office computer rack.

  6479   Tue Apr 3 12:42:19 2012 Mike J.UpdateComputersHysteresis Model

Here's my first hysteresis model in Simulink. It's based on the equation y=Amplitude*sin(frequency*t+phase)+(hysteresis/frequency2) as a solution to y''+frequency2*y+hysteresis=0. All values in the model are variables that should be manipulated through the model workspace or external code.

  6485   Wed Apr 4 21:43:16 2012 Mike J.UpdateComputersBetter Hysteresis Model

A better hysteresis model based on the simple harmonic oscillator equation. Useless variables have been removed and output can now be saved to workspace for plotting. The model is at "/users/mjenson/matlab/SHO_hyst.mdl".

  6487   Thu Apr 5 01:07:08 2012 Mike J.UpdateComputersHysteresis Plots

Here are the hysteresis plots from the most recent model, which uses a modified harmonic oscillator equation y''=-(Frequency)2*y-Hysteresis.  The hysteresis constant seems to change both the amplitude and equilibrium point of the pendulums, which is akin to changing the length of a pendulum without changing the frequency. This does not make sense. Perhaps the hysteresis value should be moved to the "spring" constant for the pendula and not restricted to a position-biasing value.

SHO_hyst_plot.png

  6494   Fri Apr 6 11:32:09 2012 JenneUpdateComputersRAID array is rebuilding....

Suresh reported to Den, who reported to me (although no elogs were made.....) that something was funny with the FB.  I went to look at it, and it's actually the RAID array rebuilding itself.  I have called in our guru, Jamie, to have a look-see.

  6495   Fri Apr 6 14:39:21 2012 JamieUpdateComputersRAID array is rebuilding....

The RAID (JetStor SATA 416S) is indeed resyncing itself after a disk failure.  There is a hot spare, so it's stable for the moment.  But we need a replacement disk:

    RAID disks:  1000.2GB Hitachi HDT721010SLA360

Do we have spares?  If not we should probably buy some, if we can.  We want to try to keep a stock of the same model number.

Other notes:

The RAID has a web interface, but it was for some reason not connected.  I connected it to the martian network at 192.168.113.119.

Viewing the RAID event log on the web interface silences the alarm.

I retrieved the manual from Alex, and placed it in the COMPUTER MANUALS drawer in the filing cabinet.

  6498   Fri Apr 6 16:35:37 2012 DenUpdateComputersc1ioo

c1ioo computer can not connect to the framebuilder and everything is red in the status for this machine, C1:FEC-33_CPU_METER is not moving.

EDIT by KI:

 We rebooted the c1ioo machine, but none of the ftont end model came back. It looked like they failed the burt process for some reasons according to dmesg.

Then we restarted each front end model one by one, and every time after immediately we restarted it we hit the 'BURT' button in the GDS screen.

Everyone came back to the normal operation.

  6502   Fri Apr 6 20:24:31 2012 Mike J.UpdateComputersSensoray

The Sensoray device is currently viewing Monitor 4 and plugged into Pianosa.  The user interface is run at /home/controls/Downloads/sdk_2253_1.2.2_linux/python demo.py. It can preview and capture the video stream, however the captured files are terrible. I believe it has something to do with the bitrate, since the captured video with lower bitrates are not as bad as the ones with higher bitrates, but  I am not certain.

  6503   Fri Apr 6 20:38:41 2012 Mike J.UpdateComputersSensoray

 Turns out that the "MPEG-4 VES" video format is just bad for captured video.  Everything except "MP4" and "MPEG-TS" works for streaming, and "MP4" and "MPEG-TS" seem to be the only captured formats that can be viewed properly.

  6505   Sat Apr 7 01:45:02 2012 Mike J.UpdateComputersEven Better Hysteresis Model and Plots

 The new hysteresis model is slightly based on the SHO equation, but with the force being out of phase with the position by an amount of hysteresis {x(t)=Amp*sin(freq*t), F(t)=Amp*sin(freq*t+Hyst)}. The new model can be found at /users/mjenson/matlab/hyst_v_3.mdl.  Pictures are: new hysteresis model, x(t) subsystem in new model[xh''(t) only lacks -1 multiplier and includes hysteresis variable], new plots.

 hyst_v_3.pnghyst_v_3-x(t).pnghyst_v3.png

  6507   Sat Apr 7 02:01:29 2012 Mike J.UpdateComputersProjector Cable Management

I replaced the projector video and power cables with longer ones, and zip-tied them to the ceiling and wall so they don't block the image.

projector_cables.jpg

ELOG V3.1.3-