40m QIL Cryo_Lab CTN SUS_Lab TCS_Lab OMC_Lab CRIME_Lab FEA ENG_Labs OptContFac Mariner WBEEShop
  40m Log, Page 282 of 341  Not logged in ELOG logo
ID Date Author Type Categorydown Subject
  3177   Thu Jul 8 14:32:42 2010 josephbUpdateCDSDaily Downs Update

After talking with Rolf, and clarifying exactly which machine I wanted, he gave me an 4600 Sun machine (similar to our current megatron).  I'm currently trying to find a good final place for it, but its at least here at the 40m. 

I also acquired 3 boards to plug our current VMIPMC 5565 RFM cards into, so they can be installed in the IO chassis.  These require +/- 5V power be connected to the top of the RFM board, which would be not possible in the 1U computers, so they have to go in the chassis.  These style boards prevent the top of the chassis from being put on (not that Rolf or Jay have given me tops for the chassis).  I'm planning on using the RFM cards from the East End FE, the LSC FE, and the ASC FE. 

I talked to Jay, and offered to upgrade the old megatron IO chassis myself if that would speed things up.  They have most of the parts, the only question being if Rolf has an extra timing board to put in it.  Todd is putting together a set of instructions on how to put the IO chassis together and he said he'd give me a copy tomorrow or Monday.  I'm currently planning on assembling it on Monday.  At that point I only need 1 more IO chassis from Rolf.

 

NEW CDS SETUP CHANGE:

When I asked about the dolphin IO chassis, he said we're not planning on using dolphin connections between the chassis and computer anymore.  Apparently there was some long distance telecon with the dolphin people and they said the Dolphin IO chassis connection and RFM doesn't  well together (or something like that - it wasn't very clear from Rolf's description).  Anyways, the other style apparently is now made in a fiber connected version (they weren't a year ago apparently), so he's ordered one.  When I asked why only 1 and what about the IOO computer and chassis, he said that would either require moving the computer/chassis closer or getting another fiber connection (not cheap).

So the current thought I hashed out with Rolf briefly was:

We use one of the thin 1U computers and place that in the 1Y2 rack, to become the IOO machine.  This lets us avoid needing a fiber.  Megatron becomes the LSC/OAF machine, either staying in 1Y3 or possibly moving to 1Y4 depending on the maximum length of dolphin connection because LSC and the SUS machine are still supposed to be connected via the Dolphin switch, to test that topology.

I'm currently working on an update to my CDS diagram with these changes and will attach it to this post later today.

  3230   Thu Jul 15 16:57:31 2010 josephb, kiwamuUpdateCDSThe temporarily named c1scx machine getting ready to connect to IO chassis

This is the machine which will be the new x end front end machine.  Its IP is 192.168.113.86. 

We changed the root and controls passwords to the usual.  We have modified the controls user group to be 1001, by using "usermod -u 1001 controls" (we had to use the non-rtl kernel to get that command to work).

We changed /etc/fstab to point to /cvs/cds on Linux rather than some downs machine.  We added a link to /cvs/cds/rtcds in the local /opt directory.

We modified the /etc/rc.d/rc.local file to no longer run /opt/open-mx/sbin/omx_init start, /cvs/cds/geo/target/fb/mx_stream -d scipe12:0 om1, and /cvs/cds/geo/target/fb/mx_stream -d scipe12:0 -e2 -r2 om2.  We modified the /usr/bin/setup_shmem.rtl to run only c1x00 c1scx and c1spx.

I also commented out a line0 "/bin/rm -f /rtl*"

 

  3243   Mon Jul 19 13:51:09 2010 kiwamuUpdateCDSTIming card at X end

[Joe, Kiwamu]

 

 The timing slave in the IO chassis on the new X end was not working with symptoms of no front "OK" green light, no "PPS" light, 3.3V testpoint not working and  ERROR testpoint bouncing between 5-6V.

We took out the timing slave from the X end IO chassis put in to the new Y end IO chassis .

It worked perfectly there. We took the working one from Y end put in the X end IO chassis.

We slowly added cables. First we added power , it worked fine and we saw green "OK" light. Then we added 1PPS signal by a fiber and it also worked.

We turned everything off and then we added 40pin IPC cable from the chassis and infiniband cable from the  computer.

When we turned ON it we didn't see the green light.

This means something in the computer configuration might be wrong not in the timing card, we now are trying to make contact with Alex.

We are comparing the setup of the C1SCX  machine and the working C1ISCEX machine.

  3273   Fri Jul 23 08:18:03 2010 josephbUpdateCDSNot enough room in IO chassis for RFM card - need to swap PMC to PCIe adapters

The End IO chassis have small trenton boards, which apparently only have 5 usuable PCI slots, even though there are 6 on the board.  This is because of the way the the host interface board is setup and its closeness to the 2nd to last PCI slot

The PMC to PCIe adapters I was handed by Jay for use with the RFM cards require a 4 pin power connection at the top, which are not available inside the thin 1U computers.

The only solution I can come up with is swap the PMC to PCIe adapters for the RFM cards with adapters for some of the already installed ADCs and DACs which do not require power directly from the power supply.  This should make it possible to mount the RFM card in the computer, at least for the ends.  Since the SUS and IOO chassis will have more slots available than needed, the RFM cards can be slotted into those.  The SUS has to fit in the chassis since the computer will have the Infiband host adapter and a dolphin connector for talking to the LSC machine.

There is still the problem of actually getting the RFM card into the computer, but that should be possible with a little bit of bending of the left side of the computer frame.

  3274   Fri Jul 23 10:16:44 2010 josephbUpdateCDSRerouted RFM around c1lsc, took RFM card out of c1lsc

 I re-routed around the c1lsc machine this morning.  I turned the crate off, and disconnected the transmission fiber from  c1lsc (which went to the receiver on c1asc).  I then took the receiving fiber from c1lsc and plugged it into the receiver on c1asc. 

I pulled out the c1lsc computer from the VME crate and pulled out the RFM card, which I needed for the CDS upgrade.  I then replaced the lsc card back in the crate and turned it back on.  Since there hasn't been a working version of the LSC code on linux1 since I overwrote it with the new CDS lsc code, this shouldn't have any significant impact on the interferometer.

I've confirmed that the RFM network seems to be in a good state (the only red lights on the RFM timing and status medm screen are LSC, ASC, and ETMX).  Fast channels can still be seen with dataviewer and fb40m appears to still be happy.

The RFM card has found its new home in the SUS IO Chassis.  The short fiber that used to go between c1asc and c1lsc is now on the top shelf of the new 1X3 rack.

  3275   Fri Jul 23 12:33:51 2010 josephbUpdateCDSmegatron, c1iscex, c1sus firewalled

Kiwamu and I strung an ethernet cable from the new 1X7 rack to the 1X3 rack. The cable is labeled c1iscex-daq on both ends.  This cable will eventually connect c1iscex's second ethernet port to the daq router.  However, for today, it plugged into the primary ethernet port and is going to a linksys router.  This is the same linksys router we used to firewall megatron.

The idea is to place megatron, c1sus, and c1iscex behind the firewall to prevent any problems with the currently running while doing RFM nework tests.

The way to get into the firewalled sub-network is to ssh into megatron.  The router will forward the ssh to megatron.  Inside the network, the computers will have the following IPs.  Router is 192.168.1.1, megatron is 192.168.1.2, c1sus is 192.168.1.3, and c1iscex is 192.168.1.4.

 

 

  3283   Fri Jul 23 21:35:48 2010 ranaUpdateCDSRerouted RFM around c1lsc, took RFM card out of c1lsc

 I just realized that an unfortunate casualty of this LSC work was the deletion of the slow controls for the LSC which we still use (some sort of AUX processor). For example, the modulation

depth slider for the MC is now in an unknown state.

  3289   Mon Jul 26 10:02:42 2010 josephbUpdateCDSRerouted RFM around c1lsc, took RFM card out of c1lsc

If you're refering to just the medm screen,  those can be restored from the SVN.  As we're moving to a new directory structure, starting with /opt/rtcds/caltech/c1/, the old LSC screens can all be put back in the /cvs/cds/caltech/medm/c1/lsc directory if desired.

The slow lsc aux crate, c1iscaux2, is still working, and those channels are still available.  I confirmed that one was still updating. As a quick test, I went to the SVN and pulled out the C1LSC_RFADJUST.adl file, renamed it to C1LSC_RFadjust.adl and placed it in /cvs/cds/caltech/medm/c1/lsc/, and checked it linked properly from the C1IOO_ModeCleaner.adl file.  I haven't touched the modulation depths, as I didn't want to mess with the mode cleaner, but if I get an OK, we can test that today and confirm that modulation depth control is still working.

Quote:

 I just realized that an unfortunate casualty of this LSC work was the deletion of the slow controls for the LSC which we still use (some sort of AUX processor). For example, the modulation

depth slider for the MC is now in an unknown state.

 

  3292   Mon Jul 26 12:31:36 2010 kiwamuUpdateCDSfront end machine for the X end

A brief report about the new front end machine "C1ISCEX" installed on the X end (old Y end).

Still the DAC is not working.

 

- Timing card

It's working correctly.

The 1PPS clock signal is supplied by the vertex clock distributer via a 40m long fiber.

 

- ADC

All 16 channels are working well.

We can see the signals in the medm screen while injecting some signals to the ADC by using a function generator.

 

-DAC

All 16 channels do NOT work.

We can not see any signals at the DAC SCSI cable while digitally injecting a signal on the medm screen.

  3293   Mon Jul 26 14:24:46 2010 josephb, kiwamuUpdateCDSRFM test take 1

Kiwamu and I strung a temporary RFM fiber from the c1iscex machine (in the new 1X9 rack) to the c1sus machine (in the new 1X4 rack).  This was connected into the respective RFM cards.  Once we put the fiber in correctly, the status lights came on the RFM card, which is a good sign.  This did not go through the RFM bypass, and did not interfere with any other RFM connections.

We created a simple model to test the RFM card, which basically was 4 RFM memory locations passing back and forth between 2 filters on each machine.  These models were called c1rf0 (on c1sus) and c1rf1 (on c1iscex).  We added 4 entries to the /cvs/cds/caltech/chans/ipc/C1.ipc file corresponding to the 4 RFM memory locations, set their ipcType=RFM and set the ipcRate to 65536.  The ipcNum were set from 0 to 3. The models ran, however, the data we were trying to pass over the RFM card did not seem to be being passed.  Currently trying to contact Alex via e-mail to get debugging advice, and confirm the ipc file is setup correctly.

  3316   Thu Jul 29 11:33:38 2010 kiwamuUpdateCDSPCI5565 driver for RFM

 Yesterday I installed a PCI-5565 driver on new C1SUS in order to test the RFM.

Since the RFM on the new CDS is not working, we had to test it by using some softwares.

I installed a driver for PCI-5565 on C1SUS and ran a test script wich is one of the packaged test scripts in the driver.

So far the RFM card on C1SUS looked correctly mounted, but I didn't check the memory location and the sending/ receiving functions.

This test will continue sometime on August because right now the RFM test is not higher priority.

 


Some notes:

Driver package

      Alex suggested to use a driver package for PIC-5565 called "RFM2g Linux 32/64-bit PCIE/PCI/PMC driver for x86 kernels R7.03" , which is available on  this web site.

And the package contains some useful test scripts which exactly we want to run for RFM test.

 

Installation and test script

      I downloaded the driver and put it on C1SUS.  

After doing usual "unzip", "tar" and "make" things, I ran one of the test script called "rfm2g_util".

Currently it lives under /home/controls/Desktop/162-RFM2G-DRV-LNX-R07_03-000/rfm2g/diags/rfm2g_util on C1SUS.

It invokes an interactive shell and firstly it asks the mount point of the RFM card.

I eventually found the card was mounted on #1 which means the card is correctly mounted.

 

Some detail procedures will be summarized on the wiki later.

  3317   Thu Jul 29 12:13:28 2010 kiwamuSummaryCDSnear term plan

 [Joe and Kiwamu]

current_setup_v2.png

  3319   Thu Jul 29 12:31:24 2010 josephbUpdateCDSWorking DAC, working IOP - next up SUS

Ok, after a few minutes of talking to Alex, I got the correct "GUI syntax" through my head, and we now have a simple working green end control which in fact puts signals out through the DAC.

Note to self, do not put any additional filters or controls in the IOP module.  Basically just change the master block with GDS numbers, DCU_ID numbers, etc.  When using a control model, copy the approriate ADC and ADC selector or DAC to the control model.  It will magically be connected to the IOP.

A correct example of a simple control model is attached.

Next in line is to get the adapter boxes for SUS into the new 1X5 rack and get started on SUS filter conversion and figuring out which ADC/DAC channels correspond to which inputs.

 

Attachment 1: Simple_Green_Control.png
Simple_Green_Control.png
  3321   Thu Jul 29 15:35:21 2010 josephb, kiwamuUpdateCDSWorking out ADC/DAC/BO wiring

We are currently using the SUS wiring diagram found on Ben Abbott's page (link here) to determine the ADC/DAC/BO channel numbers for each individual optics inputs and outputs.  Basically it involves tracking the paths back from the Pentek's, XY220, and IC110Bs to a point where we can identify it as a Coil UL or a PD whitening filter control or whatever it might be.

Once done we will have a nice wiki page describing what the final wiring is going to be, along with which ADC effectively plugs into which analog board and so forth.

  3323   Thu Jul 29 17:12:48 2010 josephb, kiwamuUpdateCDSRe:Working out ADC/DAC/BO wiring

We have installed  4 BO boards, 3 DAC boards and 1 ADC board for new C1SUS.

They are on the 1Y5 rack. 

 

 DSC_2302.png

  3342   Sat Jul 31 17:37:36 2010 josephbUpdateCDSCables needed for CDS test

Last Thursday, Kiwamu and I went through the cabling necessary for a full damping test of the vertex optics controled by the sus subsytem, i.e. BS, ITMX, ITMY, PRM, SRM.  The sus IO chassis is sitting in the middle of the 1X4 rack.  The c1sus computer is the top 1U computer in that rack.

ADCs:

The hardest part is placing the 2x D-sub connectors to scsi on the lemo break out boxes connected to the 110Bs.  The breakout boxes can be seen at the very top of the picture Kiwamu took here.  These will require a minor modification to the back panel to allow the scsi cable to get out.  There are two of these boxes in the new 1X5 rack.  These would be connected by scsi to the ADC adapters in the back of the sus IO chassis in 1X4.  The connectors are currently behind the new 1X5 rack (along with some spare ADCs/DACs/BOs.

There are 3 cables going from 40 IDC to 37 D-sub (the last 3 wires are not used and do not need to be connected, i.e. 38-40).  These plug into the blue and gold ADC adapter box, the top one shown here.  There is one spare connection which will remain unused for the moment.  The 40 IPC ends plug into the Optical Lever PD boxes in the upper right of the new 1X4 rack (as seen in the top picture here - the boards on the right). At the back of the blue and gold adapter box is a scsi adapter which goes to the back of the IO chassis and plugs into an ADC.

In the back of the IO chassis is a 4th ADC which can be left unconnected at this point.  It will eventually be plugged into the BNC breakout box for PEM signals over in the new 1X7 rack, but is unneeded for a sus test.

DACs:

There are 5 cables going from 3 SOS dewhite/anti-image boards and 2 LSC anti-image boards into 3 blue and gold DAC adapter boxes.  Currently they plug into the Pentek DACs at the bottom of the new 1X4 rack.  Ideally we should be able to simply unplug these from the Pentek DACs and plug them directly into the blue and gold adapter boxes.  However at the time we checked, it was unclear if they would reach.  So its possible new cables may need to be made (or 40 pin IDC extenders made). These boxes are then connected to the back of the IO chassis by SCSI cables.  One of the DAC outputs will be left unconnected for now.

Binary Output:

The Binary output adapter boxes are plugged into the IO chassis BO cards via D-sub 37 cables.  Note one has to go past the ADC/DAC adapter board in the back of IO chassis and plug directly into the Binary Output cards in the middle of the chassis.  The 50 pin IDC cables should be unplugged from XY220s and plugged into the BO adapter boxes.  It is unclear if these will reach.

Timing:

We have a short fiber cable (sitting on the top shelf of the new 1X3 rack) which we can plug into the master timing distribution (blue box located in the new 1X6 rack) and into the front of the SUS IO chassis.  It doesn't quite make it going through all the holes at the top of the racks and through the cabling trays, so I generally only plug it in for actual tests.

The IO chassis is already plugged into the c1sus chassis with an Infiniband cable.

 

So in Summary to plug everything in for a SUS test requires:

  • 6x SCSI cables (3 ADC, 3 DAC) (several near bottom of new 1X3 rack)
  • 4x 37 D-sub to 37 D-sub connector (end connectors can be found behind new 1X5/1X6 area with the IO chassis stuff - Need to be made) (4 BO)
  • 3x 40 IDC to 37 D-sub connectors (end connectors can be found behind new 1X5/1X6 area - Need to be made)(ADC)
  • 5x 64 pin ribbon to 40 IDC cable (already exist, unclear if they will reach) (DAC)
  • 8x 50 pin IDC ribbon (already exist, unclear if they will reach) (BO)
  • 1x Double fiber from timing master to timing card
  • 1x Infiniband cable (already plugged in)

Tomorrow, I will finish up a channel numbering plan I started with Kiwamu on Thursday and place it in the wiki and elog.  This is for knowing which ADC/DAC/BO channel numbers correspond to which signals.  Which ADCs/DACs/BOs the cables plug into matter for the actual control model, otherwise you'll be sending signals to the wrong destinations.

WARNING: The channel numbers on the front Binary Output blue and gold adapter boxes are labeled incorrectly.  Channels 1-16 are really in the middle, and 17-32 are on the left when looking at the front of the box.  The "To Binary IO Module" is correct.

  3353   Tue Aug 3 11:17:10 2010 kiwamuUpdateCDSDiagrams for Cables needed for CDS test

Current Wiring Setup for the Suspension Controls

sus_wiring.png

New Wiring Plan for the Suspension Controls  with the New CDS  

sus_new_wiring.png

Missing Stuff for the CDS test

Ideally we can reuse the existing cables, but some of them may not be long enough for the new wiring.

The diagram below shows extremely non-ideal case.

sus_missing_stuff.png

Some more information will be summarized on the wiki later.

 

  3399   Wed Aug 11 13:28:44 2010 josephbUpdateCDSNew cdsIPCx parts, old part breaks models

While working on a test stand at LLO, I've discovered that there's a new update to the Real Time Code Generator that breaks virtually all of our current models.

Previously, we've been using the cdsIPCx part as a flexible link between models, and could be set to RFM (reflected memory), SHMEM (shared memory on the local computer), or PCIE (pci express or dolphin I think) depending on what was written in the IPC file (found in /cvs/cds/caltech/chans/ipc/C1.ipc).

Recently, three new parts were added called cdsIPCx_SHMEM, cdsIPCx_RFM, cdsIPCx_PCIE, and the previous cdsIPCx broken.  The cdsIPCx_***** has to match the type written in the IPC file or else it errors out with a rather unhelpful error message which doesn't tell you which part/line is in conflict...

i.e.

***ERROR: IPCx type mis-match: I vs. ISHM
make: *** [g1lsc] Error 255

In anycase, when using a current checkout (updated or checked out after ~July 30th), all cdsIPCx blocks in the simulink models needs to be changed to the approriate type.  I'm trying to get a new CDS_PARTS.mdl file I updated into the svn, but for now you can just open up the model file directly in the /advLigoRTS/src/epics/simLink/lib directory and copy it in from there (i.e. cdsIPCx_SHMEM.mdl) to your model file.  I'm also trying to get the feCodeGen.pl file changed so the error message actually tells you what channel/part is mismatching so you don't have to guess.

An easy way to modify a file for all shared memory connections without doing a ton of cutting, pasting and renaming is:

sed -i 's/"cdsIPCx"/"cdsIPCx_SHMEM"/g' file_name_here.mdl

  3405   Thu Aug 12 11:19:04 2010 kiwamuUpdateCDScurrent status

Current status of the new CDS test (summaries can be found on the wiki)

- Cables 

  All the necessary cables are in our hands, such as SCSIs, 37 pin D-sub and 3 pin power cables.

  With a big help from Jenne and Koji, we made 37 pin F/M cables and 3 pin power cables on the last Tuesday.

  Now all the cables are connected to the machines except for the 3 pin power cables.

  To install the power cables I will switch off Sorensens at new 1X5  for a minute.

 

- Suspension model file 

  The model file named C1SUS was made and I succeeded in compiling and building it. So it is ready for the test.

  But some medm screens look like not correctly complied.

  For example the channel names which are listed on C1SUS_MONITOR_ADC0.adl aren't correctly assigned.

 

- ADCs 

 All the ADC channels are working. Also the channel assignments are correct.

 I checked all the ADC channels if it was working while I ran the front end realtime code C1SUS.

 By injecting a signal from a function generator directly to the SCSIs, I could clearly see the numbers hopping on medm screens.

 

- DACs 

  None of them are working although the computer can recognize that all three DAC cards are mounted.

  I think something in the IOP file and the model file may be wrong because this symptom looks similar to that of C1ISCEX we tested two weeks before ( see this entry ) 

  I have to fix it anyhow because the DACs are very necessary part for this damping test.

 

- Binary Outputs

  They didn't work at all. Even the computer didn't recognize the binary output cards.

  I think I should put some magical components on the model files.

 

- Conversion of the filter coefficients 

  The conversion of the filter coefficients has been doen yesterday.

  It looks running well because I can load and unload these filters on medm screens.

  I manually copied the filter coefficients from the current suspension filter file to that of new filter file.

 The current file can be found under /cvs/cds/caltech/chans/,  and the new one is under /cvs/cds/rtcds/caltech/c1/chans


Suspended works

   - Binary Outputs

   - simlink realtime model with the new CDS PARTS ( see the notice from Joe )

 

To be done

   - let DACs work

   - damping tests

 

  3407   Thu Aug 12 11:59:31 2010 kiwamuHowToCDSset up ntp daemom

(sad story)

When I was working on a new front end machine c1sus, I found that make command didn't run and gave the following message.

      "make:warning:clock skew detected.Your build may be incomplete"

This was caused by a clock difference between the nfs (nodus) and the terminal machine (c1sus).

I had to set up ntp daemon to synchronize them. Here is a procedure to set up it

 


(how to)

- log in to a front end machine

              ssh c1sus

- enable the ntp daemon

              sudo ntsysv

- configure the ntpd

              vi (emacs)  /etc/ntp.conf

- below is the contents I wrote on ntp.conf

             server 192.168.113.200  minpoll 4 maxpoll 4 iburst

      driftfile /var/lib/ntp/drift

 - let the daemon run
            sudo service ntpd start
- check it if it's running
             ntpq -p
  3408   Thu Aug 12 14:01:53 2010 josephbUpdateCDScurrent status

First, awesome progress.

Second, the Binary Output parts do in fact need to be added to the c1sus model.  They don't need to appear in the IOP though.  (They are somehow automatically taken care of by the IOP code). 

It looks like in the 2004 revision in SVN, the cdsCDO32 part (located in the CDS_PARTS.mdl/IO_PARTS) was broken.  I fixed it and updated the svn with it, so a svn update should pull the corrected version.

I'm not sure whats wrong with the DAQs.  I'll try to take a closer look at the model file tonight and see if I can make suggestions.  When you write outputs on DAC_0 or DAC_1 is the C1SUS GDS TP screen showing anything?

  3414   Thu Aug 12 18:30:08 2010 kiwamuUpdateCDSRe: current status

Yes.  

The medm screen C1SUS_GDS_TP.adl showed some numbers which correspond to that I wrote  on the outputs.

But I still could not get any physical voltage coming from the DACs.

Quote:

 When you write outputs on DAC_0 or DAC_1 is the C1SUS GDS TP screen showing anything? 

  3428   Tue Aug 17 02:07:11 2010 YoichiUpdateCDSDACs have correct number of channels

Quote:

- DACs 

  None of them are working although the computer can recognize that all three DAC cards are mounted.

  I think something in the IOP file and the model file may be wrong because this symptom looks similar to that of C1ISCEX we tested two weeks before ( see this entry ) 

  I have to fix it anyhow because the DACs are very necessary part for this damping test.

 Since we had a similar trouble with the DAC at CLIO, I checked if this problem comes from the same origin.

The origin of the problem we had was that although the board was sold as 16 channel DAC, the firmware was set to use only 8ch.

To see if this problem exist in the DAC boards of c1sus, I added the following code to the /cvs/cds/caltech/cds/advLigoRTS/src/fe/map.c

printk("DAC ASSC = 0x%x\n",dacPtr[devNum]->ASSC);

in the definition of mapDac() function.

This code prints the information of the ASSC (Assembly Configuration) register of the PMC66-16AO16 board as a kernel message at the start up time of the realtime code. You can check the message with dmesg command.

After the modification, I recompiled the c1sus and installed it.

make c1sus; make install-c1sus

After starting c1sus, I checked the dmesg. Then found that the ASSC register was set to 0x130006. To decipher this magic number, you have to consult with the manual of the DAC, which is available online.

http://www.generalstandards.com/view-products.php?product=pmc66-16ao16

The manual says the 17th and 18th bits of this number set the number of channels. Those bits are 11 for 0x130006. According to the manual, 11 means 16 channels are present. So it seems that the ASSC register is correctly set. In the case of CLIO, the ASSC register was 0x110003, meaning only 8 channels are present.

Unfortunately, the ASSC register was not the cause of the problem, but at least this possibility was checked.

  3432   Wed Aug 18 11:40:59 2010 josephb,kiwamu,yoichiUpdateCDSEnd station working with latest RCG code

We were able to get the latest SVN checkout (revision 2009), working with the c1x00 (IOP) and c1vgl models at the new X end on the c1iscex machine.

We were unable to get it working yesterday, and as far as we can tell, the only significant change was a reboot of the c1sus machine.  Before the reboot, it did not look like there were any conflicting models running on the c1sus machine, but apparently something was preventing the models on c1iscex from running properly.

Other simulated plant models still need to have their shared memory location blocks updated to the new type.

Now that we have proved we still can get the end model running, we are moving onto the c1sus machine.

  3434   Wed Aug 18 12:24:58 2010 josephb,kiwamuUpdateCDSshmem issue

Apparently its possible to have working models get into a bad state in regards to shared memory, which prevents the model from working after killing it and restarting it.  We found that by shutting all the models down, and then killing and restarting the setup_shmem process, it allowed models to function properly again.

The symptom was getting stuck at the burt restore step, according the log files (/opt/rtcds/caltech/c1/target/c1SYS/logs/log.txt).

  3458   Mon Aug 23 19:55:31 2010 kiwamuUpdateCDSsome progress

 {Joe, Kiwamu}

Today we did the following works in order to get ready for the new CDS test.

 - solved the DAC issue.

 - checked all the channel assignments of the ADC and the DAC.

 - preparation for modification of the AA filter chassis. 

 - checked DAC cable length.

 - connected the power cables of the BO boards to Sorensens.

 

Although we performed those works, we still couldn't do the actual damping tests.

To do the damping tests, we have to modify the AA chassis to let the SCSIs go in it. Now Joe and Steve are working for this issue.

 Also we found that we should make three more 37pin Dsub - 40pin IDC cables.  

But this is not a critical issue because the cables and the connectors are already in our hands. So we can make them any time later.

 

(Notes)


 (DAC issue)

  Now all the DAC channels are working correctly.  

There had been a combination of some issues.

  When I posted the elog entry last time, the DAC was not working at all (see here).

 But in the last week Joe found that the IO process didn't correctly run. He modified the IOP file named 'c1x02.mdl' and ran it after compiling and installing it.

 This made the situation better because we then were able to see the most of the signals coming out from the DACs.

     However we never saw any signals associated with SIDE_COILs.

 We checked the DAC cards, their slots and their timing signals. But they all were fine.

At that time we were bit confused and spent a couple of days because the DAC signals appeared at a different slot some time after we rebooted the computer. Actually this issue still remains unsolved...

Finally we found that SIDE_COILs had an input matrix which didn't show up in the medm screen.

We put 1 in the matrix and we successfully got the signal coming out from the DAC.

 

(channel assignments)

We checked all the channel assignments of the DACs and the ADCs.

All the channels are assigned correctly (i.e. pin number and channel name).

 

(AA chassis)

 We have been planning to put the SCSI cables into the AA chassis to get the ADC signals.

As Joe said in the past entry (see here) , we need a modification on the AA chassis to let the SCSIs go in it.

Joe and Steve will put an extension component so that we can make the chassis wider and eventually SCSI can go in.

 

 

(DAC cable length)

 In the default plan we are going to reuse some DAC cables which are connected to the existing systems.

To reuse them we had to make sure that the length of those cables are long enough for the new CDS.

After stopping the watchdogs, we disconnected those DAC cables and confirmed they were long enough.

Now those cables are connected to the original place where they have been before.

The same test will be performed for the binary outputs.

 

(power cables to Sorensens)

 Since the binary output boards need +/- 15V power, we hooked up the power cables to Sorensens sitting on the new 1X5 rack.

After cabling them, we turned on the power and successfully saw the green LEDs shining on the back panel of the boards.

  3473   Thu Aug 26 13:08:03 2010 josephbUpdateCDSWatch dogs for Vertex optics turned off

We are in the process of doing a damping test with the real time code and have turned off the vertex optics watchdogs temporarily, including BS, ITMs, SRM, PRM, MCs.

  3474   Thu Aug 26 17:10:26 2010 kiwamuUpdateCDSnew CDS test

[Joe, Kiwamu] 

Woooo Yeaaaah

With the new CDS we succeeded in damping of PRM and BS !!

  3479   Fri Aug 27 14:03:43 2010 kiwamuUpdateCDSWatch dogs for Vertex optics turned off

For a futher damping test, I again turned off the vertex optics watchdogs temporarily, including BS, ITMs, SRM, PRM, MCs.

  3480   Fri Aug 27 17:27:41 2010 kiwamuUpdateCDSnew CDS test

 { Joe, Kiwamu }

Yes !

We now are damping all of the vertex suspensions including PRM, BS, ITMs and MCs by the new CDS

( Note that we are not damping SRM because we don't have it in the chamber. )


 (things to be done)

- Make the binary outputs work.

- Make DTT work

  3481   Fri Aug 27 19:30:31 2010 ranaUpdateCDSSLOW controls

For the future SLOW controls our current plan is to keep using the VME based stuff and associated processors (Baja, Motorola, etc.). This is not

a sustainable plan since these are obsolete and eventually will die. One option is to use these boxes from Diamond Systems:

http://www.diamondsystems.com/products/octavio

 

  3490   Mon Aug 30 22:45:49 2010 kiwamuUpdateCDSbinary outputs for the new CDS

{ Joe and Kiwamu }

Today we made some efforts to get the binary outputs (BOs) working.

They still are not working but the situation is getting better.

    So far the BO cards were not recognized by any realtime codes when we ran the codes on the new front end machine C1SUS.

We put some printk commands in an initialization code like Yoichi did (see this entry) to confirm if the initialization of the BOs properly happens or not.

Then we found that we had to put the BO modules also in an IOP model file which controls all the ADCs and the DACs.

We put the BO modules in the IOP file and then BOs started being recognized by the IOP, however they still are not fully recognized by the realtime control process.

We continue this work...

 

[Some notes]


[front end code]

First of all we looked at the front end c-code c1sus.c living under /cvs/cds/calech/cds/advLigoRTS/src/fe/c1sus/.

It was okay because there was a proper BO statement like

CDS_CARDS cards_used[] = { {CON_32DO,0}, {CON_32DO,2}};

 

[initialization code]

 There is an initialization code called map.c living under /cvs/cds/calech/cds/advLigoRTS/src/fe.

This code is complied when we do the make commands as described on the wiki.

Eventually the initialization code is executed only when the IOP starts up. This happens when we type startc1x02 at /cvs/cds/rtcs/caltech/c1/script/.

 

[printk statments]

   We made a backup file named map_20100830.c.back for map.c. Then we added to map.c some pintk statements in a while loop which looks for available BOs.   

After running the make commands for the IOP file and startc1x02, we basically can check the results of those printk statements by using dmesg.

We found that map.c was running correctly because  map.c went in the while loop 4 times which is exactly the same number as the BOs we put in the model file.

However the code failed to install the BOs each time.

 

[BO modules in IOP file]

  Joe pointed out the failure in map.c was caused by lack of the BO modules in the IOP file c1x02.mdl.

Indeed putting the BO modules in the IOP fixed the problem. 

Another thing we found at this time is that there is a maximum number of BOs we can put in a model file.

The maximum number is 4, which is not enough for us because we need to put 5 of them including a 16bit BIO and four 32bit BOs.

Now Joe is asking to Alex about this issue.

Anyway now the IOP can recognize the BO cards, this fact can be found if you look at the log file /cvs/cds/rtcds/caltech/c1/target/c1x02/logs/log.txt.

The log file saids "3 Contec 32ch PCIe DO cards found", which is a good sign. 

 

[BO modules in realtime code] 

Although the IOP started seeing the BO cards, the realtime code c1sus still didn't fully recognize the BO cards.

If we look at the log file log.txt at /cvs/cds/rtcds/caltech/c1/target/c1sus/logs/, there is an evidence that the code found some cards.

The log file saids  

   Model 6 = 10

   Model 7 = 4

   Model 8 = 4

   Model 9 = 4

   Model 10 = 0.

 It looks like these corresponds to the BO cards.

So the code found some cards, but doesn't know what they are.

We need few more debugging for the BOs... 

  3492   Tue Aug 31 02:50:45 2010 kiwamuUpdateCDSvertex suspensions controlled by the new CDS

I plugged the new CDS to the vertex suspensions. 

Now PRM, SRM, ITMs, MCs and BS are under the control of the new CDS. 

From now on we will never go back to the old system.

 

Though,  the watchdogs are still running on the old system.

So if you need to turn on/off the watchdogs, you can simply enable/disable them from the usual medm screens.

  3494   Tue Aug 31 11:50:14 2010 kiwamuUpdateCDSmaximum number of BIOs

 { Joe and Kiwamu },

 Now we  are able to compile the model file with more than 4 binary input/outputs (BIOs).

 As I wrote in a past entry (see here), the number of the BIOs was limited to 4, which is not enough for us.

 

  We modified a header file called "cdsHardware.h"  in order to allow model files having more than four BIOs.

The header file lives under /cvs/cds/caltech/advLigoRTS/src/include/drv/.

There was a sentence defining the maximum number in the file:

 #define MAX_DIO_MODULES         4.

We changed this to

 #define MAX_DIO_MODULES         8.

  3498   Tue Aug 31 16:42:26 2010 josephbUpdateCDSTemporarily reverting to CDS revision 2005

Apparently updating to the latest revision of the RCG has some issues with diaggui and awgtpman.  Alex had to do some recompiling up at Hanford which apparently took him some time.  He'll be coming by tomorrow to try to bring those codes to the front end machines.

As a temporary fix, until Alex gets here tomorrow, we're reverting to the 2005 revision in the svn of the cds code.  I'm placing it in the location it is supposed to go in the new advLIGO scheme, which is /opt/rtcds/caltech/c1/core/, which is where Keith had it at LLO.  Once we get the new codes working, we will do an svn update on that location and migrate our work to that install location, at which point I'll remove the old /cvs/cds/caltech/cds/advLigoRTS/ location.

  3507   Wed Sep 1 12:24:47 2010 josephbUpdateCDSTrying to get up to date CDS code runnning

Alex, Joe:

We copied the latest x02 to c1x02 and modified to our the config block in it.

We removed gds_node_id.  We just have one number now, the dcuid, which is unique for each controller, simulated plant and IOP.  Set site to C1 and host to c1sus.

Alex made the latest awgtpman backwards compatible, and checked that into svn.

We installed the latest framecpp onto c1sus from www.ldas-sc.ligo.caltech.edu/packages/ using wget.

wget www.ldas-sc.ligo.caltech.edu/packages/framecpp-1.18 and then used make.

This let us compile diagd on c1sus, using the command make stand in the /advLigoRTS/build area.

We copied gds from the seiteststand over at handford and are trying to build that on megatron.  However, there's a bunch of packages we need for it to install properly.  Alex said he'd work on that later, possibly trying to make some portable binaries.

Checked out the latest dataviewer into /opt/rtcds/caltech/c1/core/daq, however its not quite working yet either.  This is another thing Alex said he'll work on later.

We are also going to test Alex and Rolf's kernel patch over on c1iscex on Centos base kernel (apparently they've been using Gentoo up at hanford for the test stands...) and see how that works.

  3515   Thu Sep 2 16:45:48 2010 josephbUpdateCDSNumbering scheme of the PCI bus

Rolf has recently written a document describing how one should fill out an IO chassis and how the numbering works out.  This can be found in the DCC at Rolf's PCIe numbering guide (T1000523).

Basically it works out that slot 1 corresponds to PCIe number 1, but slot 2 corresponds to PCIe number 6.  And so forth.  The following table gives a quick summary.

Slot 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
PCIe Number 1 6 5 4 9 8 7 3 2 14 13 12 17 16 15 11 10

 

  3516   Thu Sep 2 17:43:30 2010 josephbUpdateCDSOne working BO output module, others not so much

 Joe and Kiwamu:

We found one bug in the RCG code, where the second input for the CDO32 part (32 binary output) was simply a repeat of the first input, and totally ignored the second input.  This was fixed in the /advLigoRTS/src/epics/util/lib/CDO32.pm file by changing 

$calcExp .= $::fromExp[0];

to

$calcExp .= $::fromExp[1];

This fix has been added to the svn.  Unfortunately, while we have a single working binary output module, the 2nd and later modules do not seem to be responding at all.  We've done the usual swaping parts of the path in both software and hardware and can't find any bad pieces in our model files or the actual hardware.   That leaves me wondering about the c code, specifically if the CDO32Output[1], CDO32Output[2], and so forth array entries in the code are being handled properly.  I'll try to get some thoughts on it from Alex tomorrow.

  3527   Mon Sep 6 20:38:58 2010 KojiUpdateCDSSusension model reviewed

I have reviewed the suspension model of C1SUS and refined it.

It is comaptible to the current one but has minor additions.

Attachment 1: suspension_model.pdf
suspension_model.pdf
  3528   Mon Sep 6 21:08:44 2010 ranaUpdateCDSSusension model reviewed

We must remember that we are using the Rev.B SOS Coil Drivers and not the Rev. A. 

The main change from A->B was the addition of the extra path for the bias inputs. These inputs were previously handled by the slow EPICS system and not a part of the front end. So we used to have a separate bias screen for these than the bias which is in the front end. The slow bias is what was used for the alignment to avoid overloading the range of the main coil driver path.

  3534   Tue Sep 7 15:31:28 2010 josephb, alexUpdateCDSBinary Output working/ IO chassis fixed

As noted previously, we were having problems with getting multiple 32 channel binary output cards working.  Alex came by and we eventually tracked the problem down to an incorrect counter in the c code.  This has been fixed and checked into the CDS svn repository.  I tested the actual hardware and we are in fact able to turn our test LEDs on with multiple binary output boards.

 

Alex and I also looked at the non-functional IO chassis (the one which wouldn't sync with the 1PPS signal and wasn't turning on when the computer turned on.  We discovered one corner of the trenton board wasn't screwed down and was in fact slightly warped.  I screwed it down properly, straightening the board out in the process.  After this, the IO chassis worked with a host interface board to the computer and started properly.  We were able to see the boards attached as well with lspci.  So that chassis looks to be in working condition now.

Onwards to the RFM test.

  3545   Wed Sep 8 11:56:24 2010 kiwamuSummaryCDSSeptember CDS test plan

 Joe and Kiwamu

We discussed about our CDS plan for this September. The summary of the plan and "to do list" are now on the wiki page;

http://lhocds.ligo-wa.caltech.edu:8000/40m/Upgrade_09/CDS/September_CDS_plan

 

  Basically there are three major missions that we will do in this month;

1. complete damping of the vertex suspensions
2. Preparation for Green locking
3. Development of Simulated Plants
 

 We also try to keep updating the wiki page.

  3546   Wed Sep 8 14:44:37 2010 josephbUpdateCDSUpdated parse_mdl_to_ipc.py and feCodeGen.pl

I updated parse_mdl_to_ipc.py to correctly work with the 3 new cdsIPCx parts, namely cdsIPCx_PCIE (for dolphin connections), cdsIPCx_RFM (for our traditional reflected memory connections), and cdsIPCx_SHMEM (local shared memory on the computer).  These parts replaced cdsIPCx awhile back (see here ).

The code now correctly counts each type independantly with regards to ipcNum (I.e. you can have ipcNum = 0 for RFM and ipcNum = 0 for SHMEM for example).

 

I also went in and modified a few sections of the feCodeGen.pl (in /opt/rtcds/caltech/c1/core/advLigoRTS/src/epics/util/) so as to properly assign names to matrix adl files, matrix of filter bank adl files, and filter bank adl files.

  3553   Thu Sep 9 14:42:40 2010 josephbUpdateCDSUpdating suspensions model

I've been working on updating the suspensions model, incorporating Koji's refinements as well as trying to simplify the model and making it less cluttered. 

This had the side benefit of making an incorrect connection obvious.  I had incorrectly wired the ASCPIT input to be summed into the yaw path, and the ASCYAW input into the pitch path.  This has been corrected.

I've finished a single optic, and now I am in the process of propagating the changes to all the optics, as well as cleaning up the overall diagram using Rolf's new tags, which make things much less cluttered.  I've attached a screenshot of the PRM optic control model, and will be updating the Matlab web export once I've updated the full model.

Attachment 1: SingleSUS.png
SingleSUS.png
  3564   Mon Sep 13 10:22:48 2010 josephbUpdateCDSRCG bugs/feature request wiki page

I've started a wiki page under the Upgrade 09/New CDS section regarding known bugs and pending feature requests for the Real Time Code Generator.   It can be found at http://lhocds.ligo-wa.caltech.edu:8000/40m/Bugs_and_Pending_Feature_requests_for_the_RCG.  If you have any ideas to improve the RCG or encounter a bug in the code generation process (say a particular part doesn't work inside subsystems for example), please note them here.

Currently there are bugs with excitation points (they don't work inside of a subsystem block) and tags (they don't respect scope and only 1 "from" tag for each "goto" tag connected to the output of a subsystem block).

  3576   Wed Sep 15 14:34:57 2010 josephbSummaryCDSPlan for RFM switch over

Steps for RFM switch over:

1) Ensure the new frame builder code is working properly:

   A) Get Alex to finish compiling the frame builder and test on Megatron.

   B) Test the new frame builder code on fb40m (which is running Solaris) in a reversible way.  Change directory structure away from Data1, Data2, to use actual times.

   C) Confirm new frame builder code still records slow channels (c1dcuepics).

2) Ensure awg, tpman, and diagnostic codes (dtt) are working with the new front end code.

3) Physically move RFM cables from old front ends to the new front ends.  Remove excess connections from the network.

4) Merge the megatron/c1sus/c1iscex/c1ioo network with the main network.

   A) Update all the network settings on the machines as well as Linux1

   B) Remove the network switch separating the networks.

4) Start the new frame builder code on fb40m.

  3583   Fri Sep 17 12:11:42 2010 josephbUpdateCDSDowns update

In doing a re-inventory prior to the IOO chassis installation, I re-discovered we had a missing interface board that goes in an IO chassis.  This board connects the chassis to the computer and lets them talk to each other.  After going to Downs we remembered Alex had taken a possibly broken interface board back to downs for testing. 

Apparently the results of that testing was it was broken.  This was about 2.5 months ago and unfortunately it hadn't been sent back for repairs or a replacement ordered.  Its my fault for not following up on that sooner.

I asked Rolf what the plan for the broken one was.  His response was  they were planning on repairing it, and that he'd have it sent back for repairs today.  My guess the turn around time for that is on the order of 3-4 weeks (based on conversations with Gary), however it could be longer.  This will affect when the last IO chassis (LSC) can be made fully functional.  I did however pickup the 100 foot fiber cable for going between the LSC chassis and the LSC computer (which will be located in 1X3).

As a general piece of information, according to Gary the latest part number for these cards is OSS-SHB-ELB-x4/x8-2.0 and they cost 936 dollars (latest quote).

  3584   Fri Sep 17 14:55:01 2010 josephbUpdateCDSTook 5565 RFM card from IOVME to place in the new IOO chassis

I took the 5565 RFM card out of the IOVME machine to so I could put it in the new IO chassis that will be replacing it.  It is no longer on the RFM network.  This doesn't affect the slow channels associated with the auxilliary crate.

  3589   Mon Sep 20 11:39:45 2010 josephbUpdateCDSSwitch over

I talked to with Alex this morning, discussing what he needed to do to have a frame builder running that was compatible with the new front ends.

 

1) We need a heavy duty router as a separate network dedicated to data acquisition running between the front ends and the frame builder.  Alex says they have one over at downs, although a new one may need to be ordered to replace that one.

2) The frame builder is a linux machine (basically we stop using the Sun fb40m and start using the linux fb40m2 directly.).

3) He is currently working on the code today.  Depending on progress today, it might be installable tomorrow.

  3590   Mon Sep 20 16:59:26 2010 josephbUpdateCDSMegatron in 1X2 rack, to be come c1ioo

[Rana, Koji, Joe]

We pulled the phase shifters in the 1X2 rack out to make room for megatron.  Megatron will be converted into c1ioo, and the 8 core, 1U computer will be used as c1lsc.  A temporary ethernet cable was run from 1X2 to 1X3 to connect megatron to the same sub-network.

The c1lsc machine was worked on today, setting it up to run the real time code, along with the correct controls accounts, passwords, .cshrc files, etc.  It needs to be moved from 1X1 to 1X4 tomorrow.

ELOG V3.1.3-