40m QIL Cryo_Lab CTN SUS_Lab TCS_Lab OMC_Lab CRIME_Lab FEA ENG_Labs OptContFac Mariner WBEEShop
  40m Log, Page 253 of 344  Not logged in ELOG logo
ID Date Authorup Type Category Subject
  4129   Mon Jan 10 16:39:36 2011 josephb, alex, rolfUpdateCDSNew Time server for frame builder and 1PPS

Alex and Rolf came over today with a Tempus LX  GPS network timing server.  This has an IRIG-B output and a 1PPS output.  It can also be setup to act as an NTP server (although we did not set that up).

This was placed at waist height in the 1X7 rack.  We took the cable running to the presumably roof mounted antenna from the VME timing board and connected it to this new timing server.  We also moved the source of the 1PPS signal going to the master timer sequencer (big blue box in 1X7 with fibers going to all the front ends) to this new time server.  This system is currently working, although it took about 5 minutes to actually acquire a timing signal from the GPS satellites.  Alex says this system should be more stable, with no time jumps. 

I asked Rolf about the new timing system for the front ends, he had no idea when that hardware would be available to the 40m.

Currently, all the front ends and the frame builder agree on the time.  Front ends are running so the 1 PPS signal appears to be working as well.

  4130   Mon Jan 10 16:47:08 2011 josephb, alex, rolfUpdateCDSFixed c1lsc dolphin reflected memory

While Alex and Rolf were visiting, I pointed out that the Dolphin card was not sending any data, not even a time stamp, from the c1lsc machine.

After some poking around, we realized the IOP (input/output processor) was coming up before the Dolphin driver had even finished loading. 

We uncommented the line

#/etc/dolphin_wait

in the /diskless/root/etc/rc.local file on the frame builder.  This waits until the dolphin module is fully loaded, so it can hand off a correct pointer to the memory location that the Dolphin card reads and writes to.  Previously, the IOP had been receiving a bad pointer since the Dolphin driver had not finished loading.

So now the c1lsc machine can communicate with c1sus via Dolphin and from there the rest of the network via the traditional Ge Fanuc RFM.

  3036   Wed Jun 2 17:34:33 2010 josephb, alex, valeraUpdateCDSCDS updates

From what I understand, Alex rewrote portions of the framebuilder and testpoint codes and then recompiled them in order to get more than 1 testpoint per front end working.   I've tested up to 5 testpoints at once so far, and it worked.

We also have a new noise component added to the RCG code.  This piece of code uses the random number generator from chapter 7.1 of Numerical Recipies Third Edition to generate uniform numbers from 0 to 1.  By placing a filter bank after it should give us sufficient flexibility in generating the necessary noise types.  We did a coherence test between two instances of this noise piece, and they looked pretty incoherent.  Valera will add a picture of it when it finishe 1000 averages to this elog.

I'm in the process of propagating the old suspension control filters to the new RCG filter banks to give us a starting point.  Tomorrow Valera and I are planning to choose a subset of the plant filters  and put them in, and then work out some initial control filters to correspond to the plant.  I also need to think about adding the anti-aliasing filters and whitening/dewhitening filters.

 

  3251   Tue Jul 20 15:06:48 2010 josephb, bobOmnistructureGeneralNew speakers in ceiling in control room

After the crane training, Bob attached speakers to the ceiling right next to the projector, for use with presentations.

  4510   Mon Apr 11 14:17:22 2011 josephb, jamieUpdateCDSFrame wiper script installed

[Joe, Jamie, Alex]

Fixes:

I asked Alex which cron to use (dcron? frcron?).  He promptly did the following:

emerge dcron

rc-update add dcron default

Copied the wiper.pl script from LLO to /opt/rtcds/caltech/c1/target/fb/

At that point, I modified wiper.pl script to reduce to 95% instead of 99.7%.

I added controls to the cron group on fb:

sudo gpasswd -a controls cron

I then added the wiper.pl to the crontab as the following line using crontab -e.

0 6 * * *       /opt/rtcds/caltech/c1/target/fb/wiper.pl --delete &> /opt/rtcds/caltech/c1/target/fb/wiper.log

Notes:

Note, placing backups on the /frames raid array will break this script, because it compares the amount in the /frames/full/, /frames/trends/minutes, and /frames/trends/seconds to the total capacity. 

Apparently, we had backups from September 27th, 2010 and March 22nd, 2011.  These would have broken the script in any case. 

We are currently removing these backups, as they are redundant data, and we have rsync'd backups of the frames and trends.  We should now have approximately twice the lookback of full frames.

  4583   Thu Apr 28 16:12:19 2011 josephb, jamieUpdateCDSNew CDS model SVN

New SVN

We are now using the LIGO CDS SVN for storing our control models.

The SVN is at:

https://redoubt.ligo-wa.caltech.edu/websvn/

The models are under cds_user_apps, then trunk, then approriate subsystem (ISC for c1lsc for example), c1 (for caltech 40m), then models.

We have checked out /cds_user_apps to /opt/rtcds/.

So to find the c1lsc.mdl model, you would go to /opt/rtcds/cds_user_apps/trunk/ISC/c1/models/c1lsc.mdl

This SVN is shared by many people LIGO, so please follow good SVN practice.  Remember to update models ("svn update") before doing commits.  Also, after making changes please do an update to the SVN so we have a record of the changes.

New Practices

We are creating soft links in the /opt/rtcds/caltech/c1/core/advLigoRTS/src/epics/simLink/ to the models that you need to build.  So if you want to add a new model, please add it to the cds_users_apps SVN in the correct place and create a soft link to the simLink directory.

lrwxrwxrwx 1 controls controls      54 Apr 28 14:41 c1sus.mdl -> /opt/rtcds/cds_user_apps/trunk/SUS/c1/models/c1sus.mdl
lrwxrwxrwx 1 controls controls      54 Apr 28 14:41 c1sup.mdl -> /opt/rtcds/cds_user_apps/trunk/SUS/c1/models/c1sup.mdl
lrwxrwxrwx 1 controls controls      54 Apr 28 14:41 c1spy.mdl -> /opt/rtcds/cds_user_apps/trunk/SUS/c1/models/c1spy.mdl
lrwxrwxrwx 1 controls controls      54 Apr 28 14:41 c1spx.mdl -> /opt/rtcds/cds_user_apps/trunk/SUS/c1/models/c1spx.mdl
lrwxrwxrwx 1 controls controls      54 Apr 28 14:41 c1scy.mdl -> /opt/rtcds/cds_user_apps/trunk/SUS/c1/models/c1scy.mdl
lrwxrwxrwx 1 controls controls      54 Apr 28 14:41 c1scx.mdl -> /opt/rtcds/cds_user_apps/trunk/SUS/c1/models/c1scx.mdl
lrwxrwxrwx 1 controls controls      54 Apr 28 14:41 c1mcs.mdl -> /opt/rtcds/cds_user_apps/trunk/SUS/c1/models/c1mcs.mdl
lrwxrwxrwx 1 controls controls      54 Apr 28 14:41 c1x05.mdl -> /opt/rtcds/cds_user_apps/trunk/CDS/c1/models/c1x05.mdl
lrwxrwxrwx 1 controls controls      54 Apr 28 14:41 c1x04.mdl -> /opt/rtcds/cds_user_apps/trunk/CDS/c1/models/c1x04.mdl
lrwxrwxrwx 1 controls controls      54 Apr 28 14:41 c1x03.mdl -> /opt/rtcds/cds_user_apps/trunk/CDS/c1/models/c1x03.mdl
lrwxrwxrwx 1 controls controls      54 Apr 28 14:41 c1x02.mdl -> /opt/rtcds/cds_user_apps/trunk/CDS/c1/models/c1x02.mdl
lrwxrwxrwx 1 controls controls      54 Apr 28 14:41 c1x01.mdl -> /opt/rtcds/cds_user_apps/trunk/CDS/c1/models/c1x01.mdl
lrwxrwxrwx 1 controls controls      54 Apr 28 14:41 c1rfm.mdl -> /opt/rtcds/cds_user_apps/trunk/CDS/c1/models/c1rfm.mdl
lrwxrwxrwx 1 controls controls      55 Apr 28 14:41 c1dafi.mdl -> /opt/rtcds/cds_user_apps/trunk/CDS/c1/models/c1dafi.mdl
lrwxrwxrwx 1 controls controls      54 Apr 28 14:41 c1pem.mdl -> /opt/rtcds/cds_user_apps/trunk/ISC/c1/models/c1pem.mdl
lrwxrwxrwx 1 controls controls      54 Apr 28 14:41 c1mcp.mdl -> /opt/rtcds/cds_user_apps/trunk/ISC/c1/models/c1mcp.mdl
lrwxrwxrwx 1 controls controls      54 Apr 28 14:41 c1lsp.mdl -> /opt/rtcds/cds_user_apps/trunk/ISC/c1/models/c1lsp.mdl
lrwxrwxrwx 1 controls controls      54 Apr 28 14:41 c1lsc.mdl -> /opt/rtcds/cds_user_apps/trunk/ISC/c1/models/c1lsc.mdl
lrwxrwxrwx 1 controls controls      54 Apr 28 14:41 c1ioo.mdl -> /opt/rtcds/cds_user_apps/trunk/ISC/c1/models/c1ioo.mdl
lrwxrwxrwx 1 controls controls      54 Apr 28 14:41 c1gpv.mdl -> /opt/rtcds/cds_user_apps/trunk/ISC/c1/models/c1gpv.mdl
lrwxrwxrwx 1 controls controls      54 Apr 28 14:41 c1gfd.mdl -> /opt/rtcds/cds_user_apps/trunk/ISC/c1/models/c1gfd.mdl
lrwxrwxrwx 1 controls controls      54 Apr 28 14:41 c1gcv.mdl -> /opt/rtcds/cds_user_apps/trunk/ISC/c1/models/c1gcv.mdl
lrwxrwxrwx 1 controls controls      54 Apr 28 14:41 c1ass.mdl -> /opt/rtcds/cds_user_apps/trunk/ISC/c1/models/c1ass.mdl

  2067   Thu Oct 8 11:10:50 2009 josephb, jenneUpdateComputersEPICs Computer troubles

At around 9:45 the RFM/FB network alarm went off, and I found c1asc, c1lsc, and c1iovme not responding. 

I went out to hard restart them, and also c1susvme1 and c1susvme2 after Jenne suggested that.

c1lsc seemed to have a promising come back initially, but not really.  I was able to ssh in and run the start command.  The green light under c1asc on the RFMNETWORK status page lit, but the reset and CPU usage information is still white, as if its not connected.   If I try to load an LSC channel, say like PD5_DC monitor, as a testpoint in DTT it works fine, but the 16 Hz monitor version for EPICs is dead.  The fact that we were able to ssh into it means the network is working at least somewhat.

I had to reboot c1asc multiple times (3 times total), waiting a full minute on the last power cycle, before being able to telnet in.  Once I was able to get in, I restarted the startup.cmd, which did set the DAQ-STATUS to green for c1asc, but its having the same lack of communication as c1lsc with EPICs.

c1iovme was rebooted, was able to telnet in, and started the startup.cmd.  The status light went green, but still no epics updates.

The crate containing c1susvme1 and c1susvme2 was power cycled.  We were able to ssh into c1susvme1 and restart it, and it came back fully.  Status light, cpu load and channels working.  However I c1susvme2 was still having problems, so I power cycled the crate again.  This time c1susvme2 came back, status light lit green, and its channels started updating.

At this point, lacking any better ideas, I'm going to do a full reboot, cycling c1dcuepics and proceeding through the restart procedures.

  3156   Fri Jul 2 11:06:38 2010 josephb, kiwamuUpdateCDSCDS and Green locking thoughts

Kiwamu and I went through and looked at the spare channels available near the PSL table and at the ends.

First, I noticed I need another 4 DB37 ADC adapter box, since there's 3 Pentek ADCs there, which I don't think Jay realized.

PSL Green Locking

Anyways, in the IOO chassis that will put in, for the ADC we have a spare 8 channels which comes in the DB37 format.  So one option, is build a 8 BNC converter, that plugs into that box.

The other option, is build 4-pin Lemo connectors and go in through the Sander box which currently goes to the 110B ADC, which has some spare channels.

For DAC at the PSL, the IOO chassis will have 8 spare channel DAC channels since there's only 1 Pentek DAC.  This would be in a IDC40 cable format, since thats what the blue DAC adapter box takes.  A 8 channel DAC box to 40 pin IDC would need to be built.

 

End Green Locking

The ends have 8 spare DAC channels, again 40 pin IDC cable.   A box similar to the 8 channel DAC box for the PSL would need to be built.

The ends also have spare 4-pin Lemo capacity.  It looked like there were 10 channels or so still unused.  So lemo connections would need to be made.  There doesn't appear to be any spare 37 DB connectors on the adapter box available, so lemo via the Sander box is the only way.

 

Notes

Joe needs to provide Kiwamu with cabling pin outs.

If Kiwamu makes a couple spares of the 8 BNC to 37DB connector boards, there's a spare 37DB ADC input in the SUS machine we could use up, providing 8 more channels for test use.

  3230   Thu Jul 15 16:57:31 2010 josephb, kiwamuUpdateCDSThe temporarily named c1scx machine getting ready to connect to IO chassis

This is the machine which will be the new x end front end machine.  Its IP is 192.168.113.86. 

We changed the root and controls passwords to the usual.  We have modified the controls user group to be 1001, by using "usermod -u 1001 controls" (we had to use the non-rtl kernel to get that command to work).

We changed /etc/fstab to point to /cvs/cds on Linux rather than some downs machine.  We added a link to /cvs/cds/rtcds in the local /opt directory.

We modified the /etc/rc.d/rc.local file to no longer run /opt/open-mx/sbin/omx_init start, /cvs/cds/geo/target/fb/mx_stream -d scipe12:0 om1, and /cvs/cds/geo/target/fb/mx_stream -d scipe12:0 -e2 -r2 om2.  We modified the /usr/bin/setup_shmem.rtl to run only c1x00 c1scx and c1spx.

I also commented out a line0 "/bin/rm -f /rtl*"

 

  3257   Wed Jul 21 12:20:29 2010 josephb, kiwamuUpdateComputersMegatron temporarily disconnected, c1iscex firewalled, green FE test

We are moving towards a first test of getting Kiwamu's green locking signals into the new front end at the new X end, as well as sending signal out to the green laser temperature control.

Towards that end, we borrowed the router which we were using as a firewall for megatron.   At the moment, megatron is not connected to the network.  The router (a linksys N wire router), was moved to the new X end, and setup to act as a firewall for the c1iscex machine.

At this point, we need to figure which channels of the DAC correspond to which outputs of the anti-imaging board (D000186) and coil driver outputs.  Ideally, we'd like to simply take a spare output from that board and bring it to the laser temperature control.  The watchdogs will be disabled when testing to avoid any unfortunate mis-sent signals to the coils.  It looks like it should be something like channels 6,7,8 are free, although I'm not positive if thats the correct mapping or if there's a n*8 + 6,7,8 mapping.

The ADC should be much easier to determine,  since we only have a single 16 channel set coming from the lemo breakout box.  Once we've determined channels, we should be all set to do a test with the green system.

  3293   Mon Jul 26 14:24:46 2010 josephb, kiwamuUpdateCDSRFM test take 1

Kiwamu and I strung a temporary RFM fiber from the c1iscex machine (in the new 1X9 rack) to the c1sus machine (in the new 1X4 rack).  This was connected into the respective RFM cards.  Once we put the fiber in correctly, the status lights came on the RFM card, which is a good sign.  This did not go through the RFM bypass, and did not interfere with any other RFM connections.

We created a simple model to test the RFM card, which basically was 4 RFM memory locations passing back and forth between 2 filters on each machine.  These models were called c1rf0 (on c1sus) and c1rf1 (on c1iscex).  We added 4 entries to the /cvs/cds/caltech/chans/ipc/C1.ipc file corresponding to the 4 RFM memory locations, set their ipcType=RFM and set the ipcRate to 65536.  The ipcNum were set from 0 to 3. The models ran, however, the data we were trying to pass over the RFM card did not seem to be being passed.  Currently trying to contact Alex via e-mail to get debugging advice, and confirm the ipc file is setup correctly.

  3321   Thu Jul 29 15:35:21 2010 josephb, kiwamuUpdateCDSWorking out ADC/DAC/BO wiring

We are currently using the SUS wiring diagram found on Ben Abbott's page (link here) to determine the ADC/DAC/BO channel numbers for each individual optics inputs and outputs.  Basically it involves tracking the paths back from the Pentek's, XY220, and IC110Bs to a point where we can identify it as a Coil UL or a PD whitening filter control or whatever it might be.

Once done we will have a nice wiki page describing what the final wiring is going to be, along with which ADC effectively plugs into which analog board and so forth.

  3323   Thu Jul 29 17:12:48 2010 josephb, kiwamuUpdateCDSRe:Working out ADC/DAC/BO wiring

We have installed  4 BO boards, 3 DAC boards and 1 ADC board for new C1SUS.

They are on the 1Y5 rack. 

 

 DSC_2302.png

  3638   Fri Oct 1 18:19:24 2010 josephb, kiwamuUpdateCDSc1sus work

The c1sus model was split into 2, so that c1sus controls BS, PRM, SRM, ITMX, ITMY, while c1mcs controls MC1, MC2, MC3.  The c1mcs uses shared memory to tell c1sus what signals to the binary outputs (which control analog whitening/dewhitening filters), since two models can't control a binary output.

This split was done because the CPU time was running above 60 microseconds (the limit allowable since we're trying to run at 16kHz). Apparently the work Alex had done getting testpoints working had put a greater load on the cpu and pushed it over an acceptable maximum.    After removing the MC optics controls, the CPU time dropped to about 47 microseconds from about 67 microseconds.  The c1mcs is taking about 20 microseconds per cycle.

The new model is using the top_names functionality to still call the channels C1SUS-XXX_YYY.  However, the directory to find the actual medm filter modules is /opt/rtcds/caltech/c1/medm/c1mcs, and the gds testpoint screen for that model is called C1MCS-GDS_TP.adl.  I'm currently in the process of updating the medm screens to point to the correct location.

Also, while plugging in the cables from the coil dewhitening boards, we realized I (Joe) had made a mistake in the assignment of channels to the binary output boards.  I need to re-examine Jay's old drawings and fix the simulink model binary outputs.

  3639   Fri Oct 1 18:53:33 2010 josephb, kiwamuUpdateCDSThings needing to be done next week

We realized we cannot build code with the current RCG compiler on c1ioo or c1iscex, since these are not Gentoo machines.  We need either to get a backwards compatible code generator, or change the boot priority (removing the harddrives also probably works) for c1ioo and c1iscex so they do the diskless Gentoo thing.  This would involve adding some MAC address to the framebuilder dhcpd.conf file in /etc/dhcp along with the computer IPs, and then modifying the /diskless/root/etc/rtsystab with the right machine names and models to start.

I also need to bring some of the older, neglected models up to current build standards. I.e. use cdsIPCx_RFM instead of cdsIPCx and so forth. 

Need to fix the binary outputs for c1sus/c1mcs.  Need to actually get the RFM running, since Kiwamu was having some issues with his green RFM test model.  We have the latest checkout from Rolf, but we have no proof that it actually works.

  4012   Mon Dec 6 11:53:20 2010 josephb, kiwamuSummaryall down cond.power outage

The monitors for allegra and rossa's seemed to be in a weird state after the power outage.  I turned allegra and rossa on, but didn't see anything.  However, I was after awhile able to ssh in.  Power cycling the monitors did apparently got them talking with the computers again and displaying.

I had to power cycle the c1sus and c1iscex machines (they probably booted faster than linux1 and the fb machines, and thus didn't see their root and /cvs/cds directories).  All the front ends seem to be working normally and we have damped optics.

The slow crates look to be working, such as c1psl, c1iool0, c1auxex and so forth.

Kiwamu turned the main laser back on.

Quote:

Looks like there was a power outage.

 

  4027   Wed Dec 8 14:46:19 2010 josephb, kiwamuUpdateCDSWhy the ETMX daq channels were not recorded last night

When adding the ETMX DAQ channels using the daqconfig gui (located in /opt/rtcds/caltech/c1/scripts/) on C1SCX.ini, we forgot to set the acquire flag to 1 from 0.

So the frame builder was receiving the data, but not recording it.

We have since then added ETMX and the C1SCX.ini file to Yuta's useful "activateDAQ.py" script in /opt/rtcds/caltech/c1/chans/daq/, so that it now sets the sensor and SUSPOS like channels to be acquired at 2k when run.  You still need to restart the frame builder (telnet fb 8087 and then shutdown) for these changes to take effect.

The script now also properly handles files which already have had channels activated, but not acquired.

  4065   Thu Dec 16 15:10:18 2010 josephb, kiwamuUpdateCDSETMY working at the expense of ETMX

I acquired a second full pair of Host interface board cards (one for the computer and one for the chassis) from Rolf (again, 2nd generation - the bad kind).

However, they exhibited the same symptoms as the first one that I was given. 

Rolf gave a few more suggestions on getting it to work.  Pull the power plugs.  If its got slow flashing green lights, just soft cycle, don't power cycle.  Alex suggested turning the IO chassis on before the computer.

None of it seemed to help in getting the computer talking to the IO chassis.

 

I finally decided to simply take the ETMX IO chassis and place it at the Y end.  So for the moment, ETMY is working, while ETMX is temporarily out of commission. 

We also made the necessary cables (2x 37 d-sub female to 40 pin female and 40 pin female to 40 pin female) .  Kiwamu also did nice work on creating a DAC adapter box, since Jay had given me a spare board, but nothing to put it in.

  4134   Tue Jan 11 13:32:52 2011 josephb, kiwamuUpdateCDSUpdated some DAQ channel names

[Joe, Kiwamu]

We modified the activateDAQ.py script which lives in /opt/rtcds/caltech/c1/chans/daq/ and updates the C1SUS.ini, C1MCS.ini, C1RMS.ini, C1SCX.ini and C1SCY.ini files.  These files contain the DAQ channels for all the optics.

It has been modified so that channels like C1:SUS-ITMX_ULSEN_OUT_DAQ become C1:SUS-ITMX_SENSOR_UL.  Similarly the oplev signals go from C1:SUS-ITMX_OLPIT_OUT to C1:SUS-ITMX_OPLEV_PERROR.

After some debugging, we ran the script successfully and checked the output was correct.  We then restarted the frame builder (telnet fb 8088 and then shutdown) and also hit the DAQ reload button for all the front ends.

I tested in dataviewer that I could go back several years as well as going back just 1 hour in the history and see data for C1:SUS-ITMX_SENSOR_LL as well as C1:SUS-ITMX_OPLEV_YERROR.  I also tested realtime is also working for these channels.

 

The contents of the script are below.

 

inputfiles=["C1SUS.ini","C1RMS.ini","C1MCS.ini","C1SCX.ini","C1SCY.ini"]
prefix="[C1:SUS-"
optics=["BS_","ITMX_","ITMY_","PRM_","SRM_","MC1_","MC1_","MC2_","MC3_","ETMX_"]
#channels=["SUSPOS_IN1","SUSPIT_IN1","SUSYAW_IN1","SUSSIDE_IN1","ULSEN_OUT","URSEN_OUT","LRSEN_OUT","LLSEN_OUT","SDSEN_OUT","OL_SUM_IN1","OLPIT_IN1","OLYAW_IN1"]
channels_dict = {'SUSPOS_IN1':'SUSPOS_IN1_DAQ',
'SUSPIT_IN1':'SUSPIT_IN1_DAQ',
'SUSYAW_IN1':'SUSYAW_IN1_DAQ',
'SUSSIDE_IN1':'SUSSIDE_IN1_DAQ',
'ULSEN_OUT':'SENSOR_UL',
'URSEN_OUT':'SENSOR_UR',
'LRSEN_OUT':'SENSOR_LR',
'LLSEN_OUT':'SENSOR_LL',
'SDSEN_OUT':'SENSOR_SIDE',
'OLPIT_OUT':'OPLEV_PERROR',
'OLYAW_OUT':'OPLEV_YERROR',
'OL_SUM_OUT':'OPLEV_SUM'}

suffix="_DAQ]\n"

## set datarate
datarate=2048

## read the ini files
for inputfile in inputfiles:
    print inputfile
    outputfile=inputfile
    ifile = open(inputfile,'r')
    lines = ifile.readlines()
    ifile.close()

    for k in range(len(lines)):
        for op in optics:
            for ch in channels_dict:
                if (prefix+op+ch+suffix) in lines[k]:
                    lines[k]=prefix + op + channels_dict[ch] + "]\n"
                    lines[k+1]=lines[k+1].lstrip("#").rstrip(lines[k+1].split("=")[1])+"1\n"
                    lines[k+2]=lines[k+2].lstrip("#")
                    lines[k+3]=lines[k+3].lstrip("#").rstrip(lines[k+3].split("=")[1])+str(datarate)+"\n"
                    lines[k+4]=lines[k+4].lstrip("#")
    ofile = open(outputfile,'w')
    for k in range(len(lines)):
        ofile.write(lines[k])
        #print lines[k]
    ofile.close()

  3155   Thu Jul 1 17:05:36 2010 josephb, kiwamu, steveUpdateCDSNew SUS Chassis in rack

Thanks to Steve's work on some L brackets, and Kiwamu's lifting help, we now have a new SUS IO chassis in the new 1X4 rack (formerly the 1Y4 rack), just below the new SUS and LSC computers.  I have decided to call the sus machine, c1sus, and give it IP address 192.168.113.85.  We also put in a host interface adapter, OSS-HIB2-PE1x4-1x4 Re-driver HIB, which connects the computer to the IO chassis.

The IP was added to the linux1 name server.  However, the computer itself has not been configured yet.  I'm hoping to come in for an hour or two tomorrow and get the computer hooked up to a monitor and keyboard and get its network connection working, mount /cvs/cds and get some basic RCG code running.

We also ran ethernet cables for the SUS machine to the router in 1X6 (formerly 1Y6) as well as a cable for megatron from 1X3 (formerly 1Y3) to the router, in anticipation of that move next week.

During the day, I realized we needed 2 more ADCs, one of which I got from Jay immediately.  This is for two 110Bs and 4 Pentek ADCs.  However, there's a 3rd 110B connected to c0dcu1 which goes to a BNC patch panel.  Original Jay thought we would merge that into 4 pin lemo style into the 2nd 110B associated with the sus front ends.  We've decided to get a another ADC and adapter.  That will have to be ordered, and generally take 6-8 weeks.  However, it may be possible to "borrow" one from another project until that comes in to "replace" it.  This will leave us with our BNC patch panel and not force me to convert over 20 cables.

I also discovered we need one Contec DIO-1616L-PE Isolated Digital IO board for each Chassis, which I wasn't completely aware of.  This is used to control the ADCs/DACs adapter boards in the chassis.  It means we need still need to put a Binary Output board in the c1iscex chassis.  Hopefully the chassis as they come in come from Downs continue to come with the Contec DIO-1616L-PE boards (they have so far).

The current loadout of the SUS chassis is as follows:

Treton Board

Far left slot, when looking from the front has the OSS-MAX-EXP-ELB-C board, used to communicate with the c1sus computer.

Slot 1 ADC PMC66-16AI6455A-64-50M

Slot 2 DAC PMC66-16AO16-16-F0-OF 

Slot 3-6 BO Contec DIO-32L-PE Isolated Digital Output board

Slot 7 ADC PMC66-16AI6455A-64-50M

Slot 8-9 DAC PMC66-16AO16-16-F0-OF 

Slot 10-11 ADC PMC66-16AI6455A-64-50M

Slot 12 Contect DIO-1616L-PE Isolated Digital IO board

Back  Board

Slot 1 ADC adapter D0902006

Slot 2 DAC adapter D0902496-v1

Slot 7 ADC adapter D0902006

Slot 8-9 DAC adapter D0902496-v1

Slot 10-11 ADC adapter D0902006

 

  2267   Fri Nov 13 14:04:27 2009 josephb, kojiUpdateComputersUpdated wiki with RCG instructions/tips

I've placed some notes pertaining to what Koji and I have learned today about getting the RCG code working on the 40m wiki at:

http://lhocds.ligo-wa.caltech.edu:8000/40m/Notes_on_getting_the_CDS_Realtime_Code_Generator_working

We're still trying to fix the tst system, as the moment its reporting an invalid number of daq channels and during daq initialization it fails.  (This from the /cvs/cds/caltech/target/c1tst/log.txt file). Note: This problem is only on megatron and separated from the conventional DAQ system of the 40m.

cpu clock 2800014
Warning, could open `/rtl_mem_tst' read/write (errno=0)
configured to use 2 cards
Initializing PCI Modules
2 PCI cards found
***************************************************************************
1 ADC cards found
        ADC 0 is a GSC_16AI64SSA module
                Channels = 64
                Firmware Rev = 512

***************************************************************************
1 DAC cards found
        DAC 0 is a GSC_16AO16 module
                Channels = 16
                Filters = None
                Output Type = Differential
                Firmware Rev = 3

***************************************************************************
0 DIO cards found
***************************************************************************
0 IIRO-8 Isolated DIO cards found
***************************************************************************
0 IIRO-16 Isolated DIO cards found
***************************************************************************
0 Contec 32ch PCIe DO cards found
0 DO cards found
***************************************************************************
0 RFM cards found
***************************************************************************
Initializing space for daqLib buffers
Initializing Network
Found 1 frameBuilders on network
Waiting for EPICS BURT at 0.000000 and 0 ns 0x3c40c004
BURT Restore = 1
Waiting for Network connect to FB - 10
Reconn status = 0 1
Reconn Check = 0 1
Initialized servo control parameters.
DAQ Ex Min/Max = 1 32
DAQ Tp Min/Max = 10001 10094
DAQ XTp Min/Max = 10094 10144
DAQ buffer 0 is at 0x8819a020
DAQ buffer 1 is at 0x8839a020
daqLib DCU_ID = 10
DAQ DATA INFO is at 0x3e40f0a0
Invalid num daq chans = 0
DAQ init failed -- exiting

  2486   Fri Jan 8 10:38:35 2010 josephb, kojiUpdateComputersRFM and Megatron

Last night, we installed the VMI 5565 RFM card into Megatron.  After turning off the watchdogs for the ETMY optic, we disconnected the RFM fiber, and connected it to megatron, then powered it up. 

We modified the RCG code to have 3 rfmio blocks, which were reading 0x11a1c0 (ascPit), 0x11a1c4 (ascYaw), and 0x11a1c8 (lscPos).  These were connected to the approriate filter module inputs, and we also added grounds to the front of the rfmio blocks (we looked at the ass code which was setup that way, so we just did the same thing).  When we started it however, it didn't read properly.  If we turned off the input and set and offset, it calculated the output of the filter module correctly, (i.e. just the offset value), but as soon as we turned on the input, it was set to 0, no matter the offset value, which indicated it was reading something correctly.

After this test, the RFM fibers were reconnected to c1iscey, we rebooted c1iscey, and we confirmed that the system was working properly again.  We turned the watch dogs back on for ETMY.

 

 

  2539   Thu Jan 21 15:16:16 2010 josephb, kojiSummaryComputersMegatron used to lock Y arm

We succeeded in having a stable single arm (Y) lock using Megatron to replace c1iscey.

Now the lock with megatron is pretty easy. Really. It's very cool.

As we saw the oscillation of the YARM servo, we temporalily increased the gain of TRY filter by a factor of 2 (0.003->0.006). Also decreased the gain of YARM servo by the factor of  2 (1->0.5). This makes the servo gain reduced by a factor of 4 in total. This change seemed to come from the change of the ADC/DAC range.

We finally fixed the hi-gain pd transmission communications from Megatron to the c1lsc by tracking down the correct RFM memory location (which is unhelpfully labeled as a qpd channel in both losLinux and lsc40.m).  The memory location is 0x11a1e0, and is refered to as qpdData[3].

  3178   Thu Jul 8 15:19:27 2010 josephb, kojiConfigurationComputersAdded Zonet camera to IP table on linux1

We gave the Zonet camera the IP 192.168.113.26 and the name Zonet1.

We did this by modifying the /var/named/chroot/var/named/113.168.192.in-addr.arpa.zone and martian.zone files on linux1 as root.

  2255   Thu Nov 12 15:40:27 2009 josephb, koji, peterUpdateComputersETMY and Megatron test take 1

We connected megatron to the IO chassis which in turn was plugged into the rest of the ITMY setup.  We had manually turned the watchdogs off before we touched anything, to ensure we didn't accidently drive the optic.  The connections seem to go smoothly.

However, on reboot of megatron with the IO chassis powered up, we were unable to actually start the code.  (The subsystem has been renamed from SAS to TST, short for test).  While starttst claimed to start the IOC Server, we couldn't find the process running, nor did the medm screens associated with it work.

As a sanity test, we tried running mdp, Peter's plant model, but even that didn't actually run.  Although it also gave an odd error we hadn't seen before:

"epicsThreadOnceOsd epicsMutexLock failed."

Running startmdp a second time didn't give the error message, but still no running code.  The mdp medm screens remained white.

We turned the IO chassis off and rebooted megatron, but we're still having the same problem.

 

Things to try tomorrow:

1) Try disconnecting megatron completely from the IO chassis and get it to a state identical to that of last night, when the mdp and mdc did run.

2) Confirm the .mdl files are still valid, and try rebuilding them

  1508   Thu Apr 23 13:55:43 2009 josephb, peterUpdateComputersRCG example

We successfully compiled and installed the Real time Code Generator "Hello World" example (which is a skeleton for the ETMX suspension controller) on megatron.  In order to get it to compile, we had to add a flag indicating the computer is stand alone, and not using a myrinet card at the moment.  This was done by adding the shmem_daq = 1 flag to the cdsParameters module.  The symptom was it was unable to find gm.h (and there is no installed /opt/gm directory).

It is called "sam".  It was installed to /cvs/cds/caltech/target/sam, and produced medm screens in /cvs/cds/caltech/medm/c1/sam.  As nothing points to these, I figure it won't harm any of the current configuration, but lets us play around a bit.  If by some strange reason, these do cause problems, feel free to remove them.

  3221   Wed Jul 14 18:09:50 2010 josephb, razibUpdatePhase CameraSome cleanup behind 1Y2 rack of phasecamera electronics

We made an attempt at cleaning up the phase camera setup electronics.

We have moved a portion of the electronics onto the SP table (specifically the mixer, splitters, amplifiers, and associated power).  We put away a large number of cables which were unneeded, both BNC and power cables. The Innolight Mephisto power supply and one signal generator are still behind 1Y2 on top of a non-functioning VME crate.  The second VME crate was put along the south arm where two other VME crates already were.  We placed a fair number of BNC cables and power cords back on their cable racks or approriate storage space, so the rats nests of cables has been reduced.

We moved one power strip from plugging in beyind 1Y1, to the far side of the SP table (closer to the 1Y3 rack), and also found and plugged in another power strip (also on the far side of the SP table) and placed this underneath the SP table to be able to power the signal generator and Innolight Mephisto laser (its not plugged in currently, but we'd like to do so next week).

 

  2607   Tue Feb 16 14:10:06 2010 josephb, rob, kojiConfigurationLSCArms and PRC not locking

Quote:

Since last Friday either the arms or the PRC can't lock.

The montors show the beam flashing on the end mirrors, but the cavity can't get locked. The error signal looks fine. I suspect a computer problem.

Also PRC can't lock. SPOB is suspiciously stuck at about -95. Although that's not a fixed number, but covering the by hand the SPOB PD on the ITMY table doesn't change the number. I check the DC output of the photodetector and it is actually seen the beam.

Suspecting computer problems started after last Thursday's IP switch, I rebooted the frame builder, c1dcuepics, c1daqctrl and all the front ends. I then burtrestored to February 1st at 1:00 am.

Before I burtrestored c1iscepics, SPOB had gone back to more typical numbers around 0, as it usually read when PRC wasn't locked.

But burtrestoring c1iscepics, return it to the -95 of earlier.

Burterestoring to other times or dates didn't solve the problems.

 Koji and I started poking around, trying to understand what was going on.  At first, we thought it might be related to a computer error, as it seemed.

Fortunately, Rob stopped by and explained that the boost stage of the filter comes under c1lsc control, and will be turned on or off depending on the power in the arms.  Although if you turn it off, it will remain off, it just if its manually selected on, it may go on or off.

Similarly, the output from the Xarm filter bank to the ETMX  filter input will be turned on or off depending on the power in the arm.

Anyways, the locking trouble turns out to be due to no RF sidebands at 33 MHz.  The output of the Marconi was unplugged.  I don't know who, or why did it, but I've plugged it in for now, so we can lock the arms.  Let us know if you need in unplugged.  Thanks.

  606   Mon Jun 30 16:00:02 2008 josephb, samConfigurationComputers 
Sam and I setup Cat6 cable from Megatron to the 1Y6 Switch (131.215.113.252) and also connected the 1Y6 Hub to the control room switch.

While I was at it, I checked the configurations of the two switchs now connected (one in 1X4 and one in 1Y6) to the martian network. For some reason, the 1X4 had switched to DHCP enabled and was using 131.215.113.105 as an IP address. I had thought I had setup it correctly initially, so am not sure what caused the change.

The easiest way I know of to check the setup is use smartwizard discovery program from the Netgear install CD (in the equipment manual file cabinet of the control room) on a windows machine. The passwords have been set to the controls password.

Megatron should now see and be accessible through the martian network.
  4561   Fri Apr 22 12:07:38 2011 josephb, steveUpdateCDSRemoved hanging D-sub to SCSI in 1X2

Problem:

Way back, Jay had D-sub to SCSI adapters made to adapt our existing Sander box AA filters to the new SCSI based IO chassis.  However, these did not fit inside the box.

At the time, we simply left the cards outside hanging, which was a hack and needed to be replaced.

Solution:

Steve modified a black AA filter box so that it could fit the D-sub to SCSI adapter board on it, plus strain relief the SCSI cable, rather than let it hang.  The back of the box was cut, and an extending piece of metal attached to the bottom of the box.  The adapter board was screwed into the box, the SCSI plugged in, then the SCSI cable is clamped to the extending metal as well.

This modification will be propagated to the 3 remaining AA filter boards using the D-sub to SCSI adapter.

  4590   Fri Apr 29 14:36:36 2011 josephb, steveUpdateCDSRemoved hanging D-sub to SCSI in 1X5

Quote:

Problem:

Way back, Jay had D-sub to SCSI adapters made to adapt our existing Sander box AA filters to the new SCSI based IO chassis.  However, these did not fit inside the box.

At the time, we simply left the cards outside hanging, which was a hack and needed to be replaced.

Solution:

Steve modified a black AA filter box so that it could fit the D-sub to SCSI adapter board on it, plus strain relief the SCSI cable, rather than let it hang.  The back of the box was cut, and an extending piece of metal attached to the bottom of the box.  The adapter board was screwed into the box, the SCSI plugged in, then the SCSI cable is clamped to the extending metal as well.

This modification will be propagated to the 3 remaining AA filter boards using the D-sub to SCSI adapter.

 The same modification was carried out at 1X5 for PRM & SRM.

Note:  D68L8EX-850Hz  are removed  and bypassed in 7 channels.

Attachment 1: P1070621.JPG
P1070621.JPG
  2940   Mon May 17 17:17:49 2010 josephb, steve, alberto, kiwamuUpdateCDSNew CDS computers now in racks.

We placed 3 new computers in the racks.  One in 1X4 (machine running SCX) and 2 in 1Y4 (LSC and SUS).  These are 1U chassis, 4 core machines for the CDS upgrade.  I will be bringing over 2 IO chassis and their rails over tomorrow, one to be placed in 1Y4, and 1 in 1X4.

We still need some more 40 pin adapter cables and will send someone over this week to make them.  However, once we have those, we should be able to get two to three machines going, one end computer/chassis and the SUS computer/chassis.

After tomorrow we are still going to be owed 1 computer, another dolphin fiber, a couple of blue boxes, and the LSC, IO, and Y end IO chassis.  We also realized we need further fiber for the timing system.  We're going to need to get and then run fiber to both ends, as well as to 1X3, where the LSC IO chassis will be.

 

  3906   Fri Nov 12 10:49:34 2010 josephb, valeraUpdateCDSTest of ADC noise

Test:

Look at the effects of the ADC voltage range on the ADC noise floor.

ADC input was terminated with 50 ohms.  We then looked at the channel with DTT. This was at +/- 10 V range.  We used C1:SUS-PRM_SDSEN_IN1 as the test channel.

The map.c file (in /opt/rtcds/caltech/c1/core/advLigoRTS/src/fe/ ) then had two lines added at line 766.

//JCB temporary 2.5V test, remove me
  adcPtr[devNum]->BCR &= 0x84240;

This hard coded the 2.5 V range (we default to the 10 V range at the moment).

We then rebuilt the c1x02 model and reran the test.

Finally, we reverted the code change to map.c and rebuilt c1x02.

Results:
I've attached the DTT output of the two tests.

It appears the ADC is limited by 1.6 uV/rtHz.  Hence the increase in noise in counts by a factor of 4 when we drop to +/- 2.5 V from +/- 10 V.

Attachment 1: ADC_noise.pdf
ADC_noise.pdf
  4317   Thu Feb 17 22:51:04 2011 josephb, valeraSummary dither alignment model

We made a model for the dither angular stabilization system c1ass.mdl. The attached file shows the diagram.

The idea is to dither a combination of 6 optics (ETMs, ITMs, PZTs) at different frequencies and demodulate three PDs (TRX, TRY, REFL11I). Then form the DOFs from demodulted signals, filter, and send each DOF to a combination of optics.

This is enough to get started with arm cavities alignment (we may need to add the BS for the Y arm). More optics and PD can be added as they become available and/or needed.

The DAC for the fast PZT  are not connected and have to be commissioned.

Attachment 1: ass-model.png
ass-model.png
  4325   Fri Feb 18 17:52:25 2011 josephb, valeraUpdateCDSc1ass updated

We updated the c1ass model to include the BS.  We removed the dither excitation of the PZTs.  PZT control goes to epics. To do this, modified the /cvs/cds/caltech/target/c1iscaux/PZT_AI.db file.  We basically have it sum both the existing epics slider and our new output from c1ass.

More importantly we updated the color scheme.

We compiled and tested the Dolphin and RFM which work.

I should note we can't figure out why testpoints are not working properly with just this model.  Alex and Joe spent well over an hour trying to debug it to no success.  Current workaround is to add what channels you want from c1ass to the DAQ recording.  Other testpoints on other models appear to be working.

Attachment 1: c1ass_updated.png
c1ass_updated.png
  3652   Tue Oct 5 16:30:00 2010 josephb, yutaHowToCDSScreen settings and medm screens for new system

You can find the sitemap medm screen in

/opt/rtcds/caltech/c1/medm/master

The settings for the screens were last saved by burt in the original system on Sept 29, 2010 at 11:07.  So go to the

/cvs/cds/caltech/burt/autoburt/snapshots/2010/Sep/29/11:07

directory.  You can grep for the channels in the files in this directory.

You can also then use the autoBurt.req file in the /opt/rtcds/caltech/c1/target/sysname/sysnameepics (c1sus/c1susepics) to backup the settings entered.  Save to the /opt/rtcds/caltech/c1/target/snapshots directory for now.

 

 

  3653   Tue Oct 5 16:58:41 2010 josephb, yutaUpdateCDSc1sus front end status

We moved the filters for the mode cleaner optics over from the C1SUS.txt file in /opt/rtcds/caltech/c1/chans/ to the C1MCS.txt file, and placed SUS_ on the front of all the filter names.  This has let us load he filters for the mode cleaner optics.

At the moment, we cannot seem to get testpoints for the optics (i.e. dtt is not working, even the specially installed ones on rosalba). I've asked Yuta to enter in the correct matrix elements and turn the correct filters on, then save with a burt backup.

  3658   Wed Oct 6 11:03:51 2010 josephb, yutaHowToCDSHow to start diaggui for right now

I'm hoping to get a proper install this week done, but for now, this a stop gap.

To start diagnostic test tools, go to rosalba.  (Either sit at it, or ssh -X rosalba).

cd /opt/apps

type "bash", this starts a bash shell

source gds-env.bash

diaggui

 --------- Debugging section ------

If that throws up errors, try looking with "diag -i" and see if there's a line that starts with nds.  In the case last night, Alex had not setup a diagconf configuration file in the /etc/xinetd.d directory, which setups up the diagconf service under the xinit service.  To restart that service (if for example the nds line doesn't show up), go to /etc/init.d/ and type "sudo xinit start" (or restart).

Other problems can include awg and/or tpman not running for a particular model on the front end machine.  I.e. diag -i should show 3 results from 192.168.113.85 (c1x02, c1sus, c1mcs) at the moment , for both awg and tp.  If not, that means awg and tpman need to be restarted for those.

These can be started manually by going to the front end, to the /opt/rtcds/caltech/c1/target/gds/bin/ directory, and running awgtpman -s sysname (or in the case of IOP files [c1x02, c1x03, etc], awgtpman -s sysname -4.  Better is probably to run the start scripts which live /opt/rtcds/caltech/c1/scripts/ which kills and restarts all the process for you.

 

 

  3662   Wed Oct 6 16:16:48 2010 josephb, yutaUpdateCDSc1sus status

At the moment, c1sus and c1mcs on the c1sus machine seem to be dead in the water.  At this point, it is unclear to me why.

Apparently during the 40m meeting, Alex was able to get test points working for the c1mcs model.  He said he "had to slow down mx_stream startup on c1sus".   When we returned at 2pm, things were running fine. 

We began updating all the matrix values on the medm screens.  Somewhere towards the end of this the c1sus model seemed to have crashed, leaving only c1x02 and c1mcs running.  There were no obvious error messages I saw in dmesg and the target/c1sus/logs/log.txt file (although that seems to empty these days).  We quickly saved to burt snap shots, one of c1sus and one of c1mcs and saved them to /opt/rtcds/catlech/c1/target/snapshots directory temporarily.  We then ran the killc1sus script on c1sus, and then after confirming the code was removed, ran the startup script, startc1sus.  The code seemed to come back partly.  It was syncing up and finding the ADC/DAC boards, but not doing any real computations.  The cycle time was reporting reasonably, but the usr time (representing computation done for the model) was 0.  There were no updating monitor channels on the medm screens and filters would not turn on.

At this point I tried bringing down all 3 models, and restarting c1x02, then c1sus and c1mcs.  At this point, both c1sus and c1mcs came back partly, doing no real calculations.  c1x02 appears to be working normally (or at least the two filter banks in that model are showing changing channels from ADCs properly).  I then tried rebooting the c1sus machine.  It came back in the same state, working c1x02, non-calculating c1sus and c1mcs.

  3666   Thu Oct 7 10:48:41 2010 josephb, yutaUpdateCDSc1sus status

This problem has been resolved.

Apparently during one of Alex's debugging sessions, he had commented out the feCode function call on line 1532 of the controller.c file (located in /opt/rtcds/caltech/c1/core/advLigoRTS/src/fe/ directory).

This function is the one that actually calls all the front end specific code and without it, the code just doesn't do any computations.  We had to then rebuild the front end codes with this corrected file.

Quote:

At the moment, c1sus and c1mcs on the c1sus machine seem to be dead in the water.  At this point, it is unclear to me why.

Apparently during the 40m meeting, Alex was able to get test points working for the c1mcs model.  He said he "had to slow down mx_stream startup on c1sus".   When we returned at 2pm, things were running fine. 

We began updating all the matrix values on the medm screens.  Somewhere towards the end of this the c1sus model seemed to have crashed, leaving only c1x02 and c1mcs running.  There were no obvious error messages I saw in dmesg and the target/c1sus/logs/log.txt file (although that seems to empty these days).  We quickly saved to burt snap shots, one of c1sus and one of c1mcs and saved them to /opt/rtcds/catlech/c1/target/snapshots directory temporarily.  We then ran the killc1sus script on c1sus, and then after confirming the code was removed, ran the startup script, startc1sus.  The code seemed to come back partly.  It was syncing up and finding the ADC/DAC boards, but not doing any real computations.  The cycle time was reporting reasonably, but the usr time (representing computation done for the model) was 0.  There were no updating monitor channels on the medm screens and filters would not turn on.

At this point I tried bringing down all 3 models, and restarting c1x02, then c1sus and c1mcs.  At this point, both c1sus and c1mcs came back partly, doing no real calculations.  c1x02 appears to be working normally (or at least the two filter banks in that model are showing changing channels from ADCs properly).  I then tried rebooting the c1sus machine.  It came back in the same state, working c1x02, non-calculating c1sus and c1mcs.

 

  3668   Thu Oct 7 14:57:52 2010 josephb, yutaUpdateCDSc1sus status

Around noon, Yuta and I were trying to figure out why we were getting no signal out to the mode cleaner coils.  It turns out the mode cleaner optic control model was not talking to the IOP model. 

Alex and I were working under the incorrect assumption that you could use the same DAC piece in multiple models, and simply use a subset of the channels.  He finally went and asked Rolf, who said that the same DAC simulink piece in different models doesn't work.  You need to use shared memory locations to move the data to the model with the DAC card.  Rolf says there was a discussion (probably a long while back) where it was asked if we needed to support DAC cards in multiple models and the decision was that it was not needed.

Rolf and Alex have said they'd come over and discuss the issue.

In the meantime, I'm moving forward by adding shared memory locations for all the mode cleaner optics to talk to the DAC in the c1sus model.

 

Note by KA: Important fact that is worth remembering

  3706   Wed Oct 13 17:04:34 2010 josephb, yutaUpdateCDSburt restore now working with filter settings

Previously, burt restoring was not setting filters on and off, which was required us to constantly go through all the filter banks and turn them on.  We figured it out that the autoBurt.req file doesn't seem to be setup to restore those values, so that snapshots made with the default .req file don't restore either.

So I went to each of the suspension directories (/opt/rtcds/caltech/c1/target/c1SYS/c1SYSepics/   where SYS is sus, mcs, or rms) and modified teh autoBurt.req files found there with the following incantation:

sed -i 's/RO \(.*SW[12]R.*\)/\1/' autoBurt.req

This removes the RO at the beginning of the lines which have SW1R or SW2R in them.  These are the channels which correspond to filter bank switches.  As far as I can tell, the RO means to leave it alone.  Unfortunately, I didn't see an obvious autoBurt file specification in the DCC or on google in the 2 minutes I took to look for it.

We need to talk to Alex to figure out how that autoBurt.req file is generated and get it so it doesn't default to not restoring filter bank settings.

  3707   Wed Oct 13 17:12:33 2010 josephb, yutaUpdateCDSFilter name length problem found and fixed

The missing filter files for ULPOS, URPOS, and so forth for the mode cleaner optics was due to the length of the names of the filters. 

This was not a problem for the c1sus model because it was using its own name as the first 3 letters of its designation.  A filter for the sus model would be called something like BS_TO_COIL_MTRX_0_0, while for the mcs it would be called SUS_MC1_TO_COIL_MTRX_0_0, an extra 4 characters.

However, the c1mcs model used the "top_name" feature which uses a subsystem box within the simlink model to rename all the channels.  Apparently in the filter file, this means it has to add the top name to the front of everything, adding an additional 3 characters.  This pushed things over the length limit.

A hard cap of 18 characters has been added to the FiltMuxMatrix.pm file (located in /cvs/cds/caltech/c1/, so that it will prevent this type of problem in the future by stopping at compile time and presenting a helpful error message.

I also fixed a bug with too many spaces in the feCodeGen.pl file when dealing with top_names and the filtMuxMatrix.pm preventing some .adl files from being generated.

Also of interest, MC3 appears to never have had F2A filters.  For the moment we're running without them, but since they're just a fine tuning it shouldn't affect locking tonight.

 

Improbability factor of mode cleaner suspensions working tonight: ~20

 

  3797   Wed Oct 27 15:38:19 2010 josephb, yutaUpdateCDSIO chassis with bad timing was taken back to Downs

Problem:

The front end timing was not working properly for 2 of the IO chassis.  They were not being synced to the 1 PPS signal. 

This prevented the use of RFM for communication between front ends because time stamps on the transmitted data did not match the cycle on the receiving machine.

Action:

We took one of the incorrectly working chassis over to Downs.  Rolf said he would take a look at it tomorrow morning.

Joe will be going over tomorrow morning to talk with Rolf and see what needs to be done to fix it.

 

  3821   Fri Oct 29 11:25:15 2010 josephb, yutaSummaryCDS[EMERGENCY] accidentally deleted daqd

Problem:

Missing daqd file, i.e. the framebuilder executable.

Solution:

1) Go to /opt/rtcds/caltech/c1/core/advLigoRTS/

2) Look in the Makefile for a likely build suspect.  In this case it was build dc, which stands for data concentrator.

3) So we ran "make dc"

4) Go to the sub-directory build/dc/ and then copy the daqd file there to the /opt/rtcds/caltech/c1/target/fb directory

5) Test it to ensure we're getting channels (Yay!)

Future Safeguards:

Place the new target directory under SVN control.

 

  3845   Tue Nov 2 13:51:40 2010 josephb, yutaUpdateCDSRFM slowdown problem

Problem:

Each RFM memory location which needs to be read by a front end model slows the model significantly.

With no RFM memory locations to be read (replaced with grounds), the c1mcs model runs around 25 microseconds per cycle.

With 1 RFM memory location (MC_L), it runs around 29-33 microseconds.

With 3 RFM memory locations (MC_L, MC1_PIT, MC1_YAW), it runs around 45 microseconds.

With 7 RFM memory locations, the code generally doesn't run at all, going past the 62 microsecond maximum required to be able to keep up with the 16 kHz sample rate.

Last night Yuta somehow got it running with 7 RFM memory locations, but in that case, all the odd numbered RFM channels (1,3,5 as counted by the ipc file) did not work.  It was running at around 55 microseconds in that case.

The c1ioo code which is writing the data to the RFM card is experiencing no such slow down.

Current CDS status:

MC damp dataviewer diaggui AWG c1ioo c1sus c1iscex RFM Sim.Plant Frame builder  ...
                     ... 
  3861   Thu Nov 4 15:27:33 2010 josephb, yutaUpdateCDSc1ioo test points not working

Problem:

We can't access any of the c1ioo computer testpoints.  Dataviewer and DTT both fail to read any data from them.

According to diag -l (when run on Rosalba in /opt/apps), the testpoints are not being set.  Also at some point during the day when debugging, we also somehow messed up all the front end connections to the framebuilder.

Errors reported by dataviewer:

Server error 13: no data found
datasrv: DataWriteRealtime failed: daq_send: Illegal seek
Server error 12: no such net-writer
Server error 13: no data found
datasrv: DataWriteRealtime failed: daq_send: Illegal seek
Server error 12: no such net-writer
read(); errno=0
Server error 6532: invalid channel name
Server error 16080: unknown error
datasrv: DataWriteRealtime failed: daq_send: Illegal seek

Error reported by daqd.log:

[Thu Nov  4 13:29:35 2010] About to request `C1:IOO-MC1_PIT_IN1' 10022
on node 34
[Thu Nov  4 13:29:35 2010] Requesting 1 testpoints; tp[0]=10022; tp[1]=32531

[Thu Nov  4 13:29:35 2010] About to request `C1:SUS-MC2_SUSPOS_IN1'
10094 on node 36
[Thu Nov  4 13:29:35 2010] Requesting 1 testpoints; tp[0]=10094; tp[1]=32531

[Thu Nov  4 13:29:38 2010] ETIMEDOUT: test point `C1:IOO-MC1_PIT_IN1'
(tp_num=10022) was not set by the test point manager; request failed
[Thu Nov  4 13:29:38 2010] About to clear `C1:IOO-MC1_PIT_IN1' 10022 on node 34

Attempted Fixes:

Remove all daq related files: /opt/rtcds/caltech/c1/target/gds/param/tpchn_c1ioo.par and /opt/rtcds/caltech/c1/chans/daq/C1IOO.ini.  Rebuilt the front end code.

Double checked ethernet connection between c1ioo and the daq router and fb. 

Confirmed open mx was running on c1ioo. Confirmed awgtpman was running (although at one point I did find duplicate awgtpman running for c1x03, the IOP associated with c1ioo).

Rebooted the c1ioo machine. Confirmed all necessary codes came back.

Restarted the daqd process several times on the fb machine.

Current Status:

Framebuilder is running, and c1sus testpoints are available.  c1ioo test points are not.  Waiting to hear back from Alex on possible ideas.

  3880   Mon Nov 8 14:30:15 2010 josephb, yutaUpdateCDSAdded LIGONDSIP setting to cshrc.40m

We added the following line to the cshrc.40m file in the 64-bit linux and 32-bit linux sections:

setenv LIGONDSIP fb

This allows codes like tdsdmd to work properly on the linux machines (seemed to already work fine on the solaris op440m without this change).

  3946   Thu Nov 18 14:05:06 2010 josephb, yutaUpdateCDSc1sus is alive!

Problem:

We broke c1sus by moving ADC cards around.

Solution:

We pulled all the cards out, examined all contacts (which looked fine), found 1 poorly connected cable internally, going between an ADC and ADC timing interface card  (that probably happened last night), and one of the two RFM fiber cables pulled out of its RFM card.

We then placed all of the cards back in with a new ordering, tightened down everything, and triple checked all connections were on and well fit.

 

Gotcha!

Joe forgot that slot 1 and slot 2 of the timing interface boards have their last channels reserved for duotone signals.  Thus, they shouldn't be used for any ADCs or DACs that need their last channel (such as MC3_LR sensor input).  We saw a perfect timing signal come in through the MC3_LR sensor input, which prevented damping. 

We moved the ADC timing interface card out of the 1st slot  of the timing interface board and into slot 6 of the timing interface board, which resolved the problem.

Final Configuration:

 

 Timing Interface Board

Timing Interface Slot 1 (Duotone) 2 (Duotone) 3 4 5 6 7 8 9 10 11 12 13
Card None DAC interface (can't use last channel) ADC Interface ADC interface ADC interface

ADC

interface

None None None DAC interface DAC interface None None

 PCIe Chassis

Slot 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
PCIe Number Do Not Use 1 6 5 4 9 8 7 3 2 14 13 12 17 16 15 11 10
Card None ADC DAC ADC ADC ADC BO BO BO BO DAC DAC BIO RFM None None None None

Still having Issues with:

ITM West damps.  ITM South damps, but the coil gains are opposite to the other optics in order to damp properly.

We also need to look into switching the channel names for the watchdogs on ITMX/Y in addition to the front end code changes.

  3659   Wed Oct 6 12:00:23 2010 josephb, yuta, kiwamuUpdateCDSFound and fixed filter sampling rate problem with suspensions

While we looking using dtt and going over the basics of its operation, we discovered that the filter sample rates for the suspensions were still set to 2048 Hz, rather than 16384 Hz which is the new front end.  This caused the filters loaded into the front ends to not behave as expected.

After correcting the sample rate, the transfer functions obtained from dtt are now looking like the bode plots from foton.

We fixed the C1SUS.txt and C1MCS.txt files in the /opt/rtcds/caltech/c1/chans/ directory, by changing the SAMPLING lines to have 16384 rather than 2048.

ELOG V3.1.3-