40m QIL Cryo_Lab CTN SUS_Lab TCS_Lab OMC_Lab CRIME_Lab FEA ENG_Labs OptContFac Mariner WBEEShop
  40m Log, Page 249 of 341  Not logged in ELOG logo
ID Date Authorup Type Category Subject
  4396   Thu Mar 10 13:44:56 2011 josephbUpdateCDSAdded digitization noise to the c1spy model for simulated ADCs/DACs

To simulate digitization noise, the easiest way I found was to use the MathFunction block, found in the CDS_PARTS model, under simLinkParts. 

The MathFunction block supports square of input value, square root of input value, reciprocal of input value, and modulo of two input values.

The last is useful because it casts the input values as integers before taking the modulo.By placing this block after the saturation block (set to +/- 32768), adding 32768.5, choosing the 2nd input to be larger than 2 * 32768 (100,000 in this case), and then subtracting 32768, we wind up with a rounding function. 

The above method has been applied to the c1spy model in the CI and SO out sub-blocks.

  4404   Fri Mar 11 11:33:24 2011 josephbUpdateCDSFixed mistake in Matrix of Filter banks naming convention

While fixing up some medm screens and getting spectra of the simulated plant, I realized that the naming convention for the Matrices of Filter banks was backwards when compared to that of the normal matrices (and the rest of the world).  The naming was incorrectly column, row.

This has several ramifications:

1) I had to change the suspensions screens for the TO_COIL  output filters.

2) I had to change the filters for the suspension with regards to the TO_COIL output filters so they go in the correct filter banks.

3) Burt restores to times previous March 11th around noon, will put your TO_COIL output filters in a funny state that will need to be fixed.

4) The simplant RESPONSE filters had to be moved to the correct filter banks.

5) If you have some model I'm not aware of that uses the FiltMuxMatrix piece, it is going to correctly build now, but you're going to have to move filters you may have created with foton.

  4409   Sun Mar 13 16:46:48 2011 josephbUpdateCDSETMY Sim work

4. The blue Output Filters  section has been changed to agree with the new filter of matrices row, column labeling.  My fault for not testing it and realizing it was broken.  The change was made in /opt/rtcds/caltech/c1/medm/master/C1SUS_DEFAULTNAME.adl and then ,/generate_master_screens.py was run, updating all the screens.

5.  I have swapped the logic for the sensor filter banks (ULSEN, URSEN, etc).  It now sends a "1" to the Binary Output board controlling the OSEM analog whitening when the FM1 filter is ON.  This has been done for all the suspensions (BS, ITMX,ITMY, SRM, PRM, MC1, MC2,MC3, ITMX, ITMY).

I am also updating the first sensor filter banks for the BS, ITMX, ITMY, SRM, PRM,MC1,MC2,MC3, called "3:30", to match the Y and X ends.

8.  I can't find any documentation on how to get a momentary button press to toggle states.  I could stick a filter bank in and use the on/off feature of that part, but that feels like a silly hack.  I've decided for the moment to split the TM offset button into 2, one for ON, one for OFF.  I'll put in on the list of things to have added to the RCG code (either a method, or documentation if it already exists). 

EDIT:  TM offset still doesn't work.  Will worry about it next week.

9.  Fixed a connection in SPY/SPX models where  the side senor path that was missing a constant to a modulo block.

Quote:

I did some work on the ETMY real and Sim.

  1. Set SUS coil gains to have the same quadropole arrangement as the magnets do (-1, 1, 1, -1) so that POS = POS instead of pringle.
  2. Set the Sim Magnet polarities to match this. These are the ETMY_CI filter banks.
  3. Found that the Xycom cable from the ETMY slow controls was unplugged at the Xycom side. This was preventing enabling the ETMY coil driver and so there was no real damping of the suspension going on. I plugged it in and checked that the mirror could now be moved.
  4. The C1SUS_ETMY master screen's BLUE output filter area is now mis-labeled. If you trust the screen you would set it up to drive the suspension incorrectly. This MUST be fixed along with all of the other misleading features of the screen.
  5. ETMY SUSSIDE filter bank had a 2048 Hz sample rate and was making the damping not work correctly. Fixed to 16384 Hz.
  6. 12 Hz, 4th order Cheby low pass added and turned on for the local damping filtering. This is not optimum, but its just there to give us some filtering without introducing some instability via phase lag around 3 Hz.
  7. ETMY OL beam re-aligned on ISCT-EX.
  8. TM Offset buttons not working on the main overview screen.

It seems like there is still a problem with the input whitening filters. I believe the Xycom logic is set such that the analog whitening of the OSEM signals is turned ON only when the FM1 is turned OFF. Joe has got to fix this (and elog it) so that we can damp the suspension correctly. For now, the damping of the ETMY and the SETMY require different servo gains and signs, probably because of this.

 

  4410   Fri Mar 18 11:29:36 2011 josephbUpdateCDSMinute trend issues

[Joe, Alex]

Steve pointed out to me today he couldn't get trends for his PEM slow channels like C1:PEM-count_full. 

I experimented a bit and found for long time requests (over 20 days), it would produce minute trends up to the current time, but only if they started far enough back.  So the data was being written, but something was causing a problem for dataviewer/NDS to find it.

On further investigation it looks to be some incorrect time stamps at several points in the last few months are causing the problems.  Basically when Alex and I made mistakes in the GPS time stamp settings for the frame builder (daqd) code, the wrong time got written for hours to the raw minute trend data files.

So Alex is going to be running a script to go through the roughly 180 gigabytes of affected trend data to write new files with the correct time stamps.  Once it done, we'll move the files over.  We'll probably lose a few hours worth of recent trend data, depending on how quickly the scripts run, but after which minute trends should work as they are supposed to.

  4415   Fri Mar 18 17:25:21 2011 josephbUpdateCDSLockins in c1sus update, suspension screens updated

I updated our lockin simulink pieces to use the newer, more streamlined lockin piece that is currently in CDS_PARTS (with new documentation block!).  It means we are no longer passing clock signals through three levels of boxes.

In order to use the piece, you need to right click on it after copying from CDS_PARTS and go to Link Options->Disable Link.  This forces the .mdl to save all the relevant information about the block rather than just a pointer to the library.  I talked with Rolf and Alex today and we discussed setting up another model file, non-library format for putting generically useful user blocks into, rather than using the CDS_PARTS library .mdl.

The BS, ITMX, ITMY, PRM, SRM, ETMX, ETMY now have working lockins, with the input matrix to them having the 2nd input coming from LSC_IN, the 3rd from the oplev pitch, and the 4th from oplev yaw.

This necessitated a few name changes in the medm screens.  I also changed the lockin clock on/off switch to a direct amplitude entry, which turns green when a non-zero value is entered.

Currently, the Mode cleaner optic suspension screens have white lockins on them.  I started modifying a new set of screens just for them, and will modify the generate_master_screens.  Unfortunately, this requires modifying two sets of suspension screens going forward - the main interferometer optics and the MC optics.

  4431   Wed Mar 23 10:34:17 2011 josephbUpdateCDSTrend issue fixed

[Joe, Alex]

Yesterday during the day, Alex ran a script to fix the time stamps in the trends files we had messed up back during the daqd change overs around Feb 17th and 23rd.  See this elog for more information on the trend problem.

Due to how the script runs, basically taking all the data and making a new copy with the correct time stamps, the data collected while the script was running didn't get converted over.  So when he did the final copy of the corrected data, it created a several hour gap in the data from yesterday during the day time.

The original files still exist on the fb machine in /frames/trend/minute_raw_22mar2011 directory.

 

  4438   Thu Mar 24 13:56:05 2011 josephbUpdateelogelog restarted at 1:55pm

Restarted elog.

  4444   Fri Mar 25 11:16:19 2011 josephbFrogsGreen Lockingdigital frequency counting

I modified the c1gfd.mdl simulink model.  I made a backup as c1gfd_20110325.mdl. 

The first change was to use a top_names block to put everything in.  The block is labeled ALS.  So all the channels will now be C1:ALS-GFD_SOMETHING.  This means medm channel names will need to be updated.  Also, the filter modules need to be updated in foton because of this.

I then proceeded to add the suggested changes made by Matt.  To avoid a divide by zero case, I added a saturation part which saturates at 1e-9 (note this is positive) and 1e9.

Quote:

Today we tried the Schmitt trigger DFD, and while it works it does not improve the noise performance.  At least part of our problem is coming from the discrete nature of our DFD algorithm, so I would propose that an industrious day job person codes up a new DFD which avoids switching.  We can probably do this by mixing the input signal (after high-passing) with a time-delayed copy of itself... as we do now, but without the comparator.  This has the disadvantage of giving an amplitude dependent output, but since we are working in the digital land we can DIVIDE.  If we mix the signal with itself (without delay) to get a rectified version, and low-pass it a little, we can use this for normalization.  The net result should be something like:

output = LP2[ s(t) * s(t - dt) / LP1[ s(t) * s(t) ]],

where s(t) is the high-passed input and LP is a low-pass filter.  Remember not to divide by zero.

 

 

Attachment 1: C1GFD.png
C1GFD.png
  4445   Mon Mar 28 15:18:04 2011 josephbUpdateCDSCDS updates on Friday

Last Friday, we discovered a bug in the RCG where the delay part was not actually delaying.  We reported this to Alex who promptly put a fix in the same day.  This allowed Matt's newly proposed frequency discriminator to work properly.

It also required a checkout of the latest RCG code (revision 2328), and rebuild of the various codes.  We backed up all the kernel and executables first such as mbuf.ko and awgtpman.

We did the following:

1) Log into the fb machine.

2) Go to /opt/rtcds/caltech/c1/core/advLigoRTS/src/drv/mbuf and run make.  Copy the newly built mbuf.ko file to /diskless/root/modules/2.6.34.1/kernel/drivers/mbuf/mbuf.ko on the fb machine.

3) Use "sudo cp" to copy the newly built mbuf.ko file to /diskless/root/modules/2.6.34.1/kernel/drivers/mbuf/

4) Go to /cvs/cds/rtcds/caltech/c1/core/advLigoRTS/src/gds and run make.

5) Copy the newly built awgtpman executable to /opt/rtcds/caltech/c1/target/gds/bin/

6) Go to /opt/rtcds/caltech/c1/core/advLigoRTS/src/mx_stream/ and run make.

7) Copy the newly built mx_stream executable to /opt/rtcds/caltech/c1/target/fb/

  4446   Mon Mar 28 15:49:18 2011 josephbUpdateCDSLessons from LST

[Koji,Joe]

PART 1:

Koji was unable to build his c1lst model first thing this morning.  Turns out there was  a bug with RCG parser that was introduced on Friday when we did the RCG updates.  We talked Alex who did a quick comment fix.  The diff is as follows:

Index: Parser3.pm
==============================

=====================================
--- Parser3.pm  (revision 2328)
+++ Parser3.pm  (working copy)
@@ -1124,8 +1124,8 @@
  print "Flattening the model\n";
  flatten_nested_subsystems($
root);
  print "Finished flattening the model\n";
-  CDS::Tree::do_on_nodes($root, \&remove_tags, 0, $root);
-  print "Removed Tags\n";
+  #CDS::Tree::do_on_nodes($root, \&remove_tags, 0, $root);
+  #print "Removed Tags\n";
  #print "TREE\n";
  #CDS::Tree::print_tree($root);
  CDS::Tree::do_on_nodes($root, \&remove_busses, 0, $root);

This was some code to remove TAGs from the .mdl file for some reason which I do not understand at this time.  I will ask tommorrow in person so I can understand the full story.

PART 2:

Koji then rebuilt and started the c1lst process.  This is his new test version of the LSC code.  We descovered (again) that when you activate too many DAQ channels (simply uncommenting them, not even recording them with activate=1 in the .ini file) that the frame builder crashes.  In addition, the c1lsc machine, which the code was running on, also hard crashed.

When a channel gets added to the .ini file (or uncommented) it is sent to the framebuilder, irregardless of whether its recorded or not by the frame builder.  There is only about 2 megabytes per second bandwidth per computer.  In this case we were trying to do something like 200 channels * 16384 Hz * 4 bytes = 13 megabytes per second.

The maximium number of 16384 channels is roughly 30, with little to no room for anything else.  In addition, test points use the same allocated memory structure, so that if you use up all the capacity with channels, you won't be able to use testpoints to that computer (or thats what Alex has led me to believe).

The daqd process then core dumped and was causing all sorts of martian network slowdowns.  At the same time, the c1lsc computer crashed hard, and all of the front end processes except for the IOP on c1sus crashed.

We rebooted c1lsc, and restarted the c1sus processes using the startc1SYS scripts.  However, the c1susfe.ko apparently got stuck in a wierd state.  We were completely unable to damp the optics and were in general ringing them up severely.  We tried debugging, including several burt restores and single path checks.

Eventually we decided to reboot the c1sus machine after a bit of debugging.  After doing a burt restore after the reboot, everything started to damp and work happily.  My best guess is the kernel module crashed in a bad way and remained in memory when we simply did the restart scripts.

 

  4499   Thu Apr 7 13:14:23 2011 josephbUpdateCDSProposed plan for ITMX/ITMY control switch

Problem:

The controls (fast and slow both) think ITMX is ITMY and ITMY is ITMX.

Solution:

After some poking around today, I have convinced myself it is sufficient to simply swap all instances of ITMX for ITMY in the C1_SUS-AUX1_ITMX.db  file, and then rename it to C1_SUS-AUX1_ITMY.db (after having moved the original C1_SUS-AUX1_ITMY.db to a temporary holding file).

A similar process is then applied to the original C1_SUS-AUX1_ITMY.db file.  These files live in /cvs/cds/caltech/target/c1susaux.  This will fix all the slow controls.

To fix the fast controls, we'll modify the c1sus.mdl file located in /opt/rtcds/caltech/c1/core/advLigoRTS/src/epics/simLink/ so that the ITMX suspension name is changed to ITMY and vice versa.  We'll also need to clean up some of the labeling

At Kiwamu and Bryan's request, this will either be done tomorrow morning or on Monday.

So the steps in order are:

1) cd /cvs/cds/caltech/target/c1susaux

2) mv C1_SUS-AUX1_ITMX.db C1_SUS-AUX1_ITMX.db.20110408

3) mv C1_SUS-AUX1_ITMY.db C1_SUS-AUX1_ITMY.db.20110408

4) sed 's/ITMX/ITMY/g' C1_SUS-AUX1_ITMX.db.20110408 > C1_SUS-AUX1_ITMY.db

5) sed 's/ITMY/ITMX/g' C1_SUS-AUX1_ITMY.db.20110408 > C1_SUS-AUX1_ITMX.db

6) models

7) matlab

8) Modify c1sus model to swap ITMX and ITMY names while preserving wiring from ADCs/DACs/BO to and from those blocks.

9) code; make c1sus; make install-c1sus

10) Disable all watchdogs

11) Restart the c1susaux computer and the c1sus computer

 

  4515   Tue Apr 12 12:01:30 2011 josephbUpdateGeneralIFO controls, now with 10% less lying (ITMX/ITMY controls swapped)

The ITMX/ITMY control swap is complete.

The steps from this elog were followed.

In addition, I did a burt restore of c1sus, c1mcs.

I then swapped all the gain settings from ITMX to ITMY, and reenabled the watchdogs.

I did some basic kick tests (1000 counts into UL coil) and confirmed channels like C1:SUS-ITMX_ULPD_VAR (watchdogs mV readback) corresponded to the correct optic.  I also checked that the POS, PIT, YAW, SIDE produced reasonable damping when engaged.

  4516   Tue Apr 12 16:01:33 2011 josephbUpdateGeneralRFM errors

Problem:

Currently the c1scy, c1mcs, and c1rfm models are reporting an error with receiving some data sent over the GE Fanuc Reflected memory cards.

To be more exact, the C1:SUS-ETMY_ALS signal from the c1gcv FE code on the c1ioo computer going too the Y end is not being received However, the C1:SUS-ETMY_LSC signal is.  So the physical RFM card seems to be working.

Similarly, the TRY signal is being sent correctly from the Y end computer.  The X end is working fine and receiving both LSC and ALS signals.

The c1mcs and c1rfm models also receive data from the c1ioo computer and reporting receiving errors.

Theory:

Because the RFM cards are transmitting and receiving at least some channels, I'm guessing there was changes made to the C1.ipc file, which defines the memory locations of these various channels on the RFM network, and that when a model was rebuilt, a different one using the previous IPC file was not, and thus one of the computer is going to the wrong place to either read or write data.

Tomorrow, I'm planning on the  following:

1) Clean out the C1.ipc file (/opt/rtcds/caltech/c1/chans/ipc/)

2) Rebuild all models

3) Run activate_daq.py script

4) Restart models via script

If this doesn't clear up the problem, I'll continue  to bug hunt.

  4518   Wed Apr 13 11:34:07 2011 josephbUpdateCDSFixed IFO_ALIGN.adl

Problem:

I switched the ITMX and ITMY control channels yesterday, but forgot to update the IFO_ALIGN.adl file (/opt/rtcds/caltech/c1/medm/c1ifo/) which had the control labels swapped to make life easier.

Solution:

I swapped ITMX and ITMY control locations on the screen.

Question:

Are there any other screens involving ITMX and ITMY that had controls reversed to make life easier?

  4524   Thu Apr 14 12:57:15 2011 josephbUpdateCDSRFM network happy again

[Joe, Alex]

Problem Symptoms:

There were red lights on the status screen indicating RFM errors for the c1scy, c1mcs and c1rfm processes.

The c1iscey, c1sus machines were receiving data sent over the RFM network from the c1ioo computer with a bad time stamp, a few cycles too late.  The c1iscex computer was receiving data from c1ioo fine.

Problem:

The c1iscex RFM card had gotten into a bad state and was somehow slowing things down/corrupting data.  It didn't affect itself, but due to the loop topology was messing everyone else up.  Basically the only one who wasn't throwing an error was the culprit.

Solution:

Hard power cycling the c1iscex computer reset the RFM card and fixed the problem.

  4543   Tue Apr 19 15:48:43 2011 josephbUpdateCDSMEDM screens and Front Ends updated to new Matrices

Problem:

The original matrix naming conventions for the front ends was broken.  It used _11, _12,...,_1e, _1f, _110, _111 and so forth.  The code was changes to use _1_1, _1_2,...,_1_16,_1_17, and so on.

In addition the matrix of filter banks was modified to use the same naming convetion (instead of starting at zero, it now start with one).

Work Done:

I rebuilt all the models, and restarted them all.

I wrote a simple script to modify the burt restore files to have the correct names for all the stored matrix values.

I also modified all the suspension screens, by modifying the default screens in /opt/rtcds/caltech/c1/medm/master/

The C1SUS, C1SCX, C1SPX, C1SCY, C1SPY, and C1MCS models had their foton filter files modified to put filters into the newly changed named filters

  4545   Wed Apr 20 11:02:18 2011 josephbUpdateCDSMEDM screens and Front Ends updated to new Matrices
We simply didn't any matrices larger than 16x16. If we had, than that matrix would not have worked properly since the beginning.

Quote:

Just a curiosity:

I just wonder how you have distingushed the difference between _111 and _111.

They are equivalent alone themselves. Have you looked at the contexts of the lines?
Or you just did not have the larger matrix than 16x16, did you?

 

  4580   Thu Apr 28 10:53:50 2011 josephbUpdateCDSAdventures in Hyper-threading

What was done:

1) Turn off MC1, MC2, MC3, BS, ITMX, ITMY, PRM, SRM watchdogs.

2) Turn c1sus computer off (sudo shutdown now)

3) Go connect monitor and keyboard to c1sus.  Turn c1sus on.

4) Hit "del" key at the right time to go to setup (BIOS).

5) Go to BIOS advanced tab, CPU options, enable Multi-threading.

6) Hit F10 to save and let the computer continue booting.

What went wrong:

Once c1sus was up, I noticed several red lights and dead keep alives for the c1sus models.

Typing dmesg on c1sus revealed many messages like:

[  107.583420] c1x02: cycle 33737 time 20; adcWait 10; write1 0; write2 0; longest write2 0
[  107.583771] c1x02: cycle 33760 time 19; adcWait 11; write1 0; write2 0; longest write2 0

This indicates the Input/Output Processor (IOP) is not completing its duties within the 15 microseconds (1/64 kHz) that it has.  These lines indicate its take 20 or 19 microseconds.  (I saw messages ranging from 16 to 22 microseconds).

So this seems to agree with Rolf's observations that hyperthreading can cause a 5-10 microsecond increase in computation time.

So the next thing to do is modify which core the codes are running on, and try to get them paired up on the same physical core.

  4581   Thu Apr 28 12:25:11 2011 josephbUpdateCDSFurther adventures in Hyper-threading

First, I disabled front end starts on boot up, and brought c1sus up.  I rebuilt the models for the c1sus computer so they had a new specific_cpu numbers, making the assumption that 0-1 were one real core, 2-3 were another, etc.

Then I ran the startc1SYS scripts one by one to bring up the models.  Upon just loading the c1x02 on "core 2" (the IOP), I saw it fluctuate from about 5 to 12.  After bringing up c1sus on "core 3", I saw the IOP settle down to about 7 consistently.  Prior to hyper-threading it was generally 5. 

Unfortunately, the c1sus model was between 60 and 70 microseconds, and was producing error messages a few times a second

[ 1052.876368] c1sus: cycle 14432 time 65; adcWait 0; write1 0; write2 0; longest write2 0
[ 1052.936698] c1sus: cycle 15421 time 74; adcWait 0; write1 0; write2 0; longest write2 0

Bringing up the rest of the models (c1mcs on 4, c1rfm on 5, and c1pem on 6), saw c1mcs occasionally jumping above the 60 microsecond line, perhaps once a minute.   It was generally hovering around 45 microseconds.  Prior to hyper-threading it was around 25-28 microseconds.

c1rfm was rock solid at 38, which it was prior to hyper-threading.  This is most likely due to the fact it has almost no calculation and only RFM reads slowing it down.

c1pem continued to use negligible time, 3 microseconds out of its 480.

I tried moving c1sus to core 8 from core 3, which seemed to bring it to the 58 to 65 microsecond range, with long cycles every few seconds.

 

I built 5 dummy models (dua on 7, dub on 9, duc on 10, dud on 11, due on 1) to ensure that each virtual core had a model on it, to see if it helped with stabilizing things.  The models were basically copies of the c1pem model.

Interestingly, c1mcs seemed to get somewhat better and only taking to 30-32 microseconds, although still not as good as its pre-hyper-threading 25-28.  Over the course of several minutes it was no longer having a long cycle.

c1sus got worse again, and was running long cycles 4-5 times a second.

 

At this point, without surgery on which models are controlling which optics (i.e. splitting the c1sus model up) I am not able to have hyper-threading on and have things working.  I am proceeding to revert the control models and c1sus computer to the hyper-threading state.

 

 

  4608   Tue May 3 10:41:35 2011 josephbUpdateCDSMorning maintenance

1) Filled in the C1SUS_BS_OLMATRIX properly so as to make the BS oplev work for Steve.

2) Turned on the ITMX damping.  Apparently it had tripped this morning, possibly due to work in the lab area.

3) The ETMX FE controller (c1scx) had ADC timed out and died sometime around 8:30 am.  The c1x01 (the IOP on the ETMX computer) was also indicating a FB status error (mismatch in DAQ channels).

The reported error in dmesg on c1iscex was:

[1628690.250002] c1spx: ADC TIMEOUT 0 3541 21 3605
[1628690.250002] c1scx: ADC TIMEOUT 0 3541 21 3605

Just to be safe, I rebuilt the c1x01 and c1scx models, ran ./activateDAQ.py, and used the scripts killc1spx, killc1scx, and killc1x01.

I finally restarted the process with startc1x01, startc1scx, and startc1spx.  Everything is currently alive and indicating all green.

  4609   Tue May 3 10:59:31 2011 josephbUpdateCDS1Y2 binary output adapter board now powered

I temporarily turned off the power to the 1Y2 rack this morning while wiring in the binary output adapter board power (+/- 15V) into the cross connects.

The board is now powered and we can proceed to testing if can actually control the LSC whitening filters.

  4665   Mon May 9 13:14:48 2011 josephbUpdateLSCC1:LSC-TRIG_MTRX : wrong matrix size

[Joe, Kiwamu]

There is a feature/bug of the RCG code that you can only have 1 receiving tag for every sending tag.  There were 5 tags which were being received by two tags each, for two different matrices.  Only the first tag was receiving, the second was apparently ignored.

This has been fixed temporarily by putting in direct lines in place of these 5 tags.

Quote:

I found that C1:LSC-TRIG_MTRX has a wrong matrix size. It needs to be fixed.

It is designed to have a 11x8 matrix in the simlink model file, but it's been compiled as a 6x8 matrix.

 

  4666   Mon May 9 15:21:36 2011 josephbUpdatePSLFixed channel names for PSL QPDs, fixed saturation, changed signs

[Valera, Joe]

Software Changes:

First we changed all the C1:IOO-QPD_*_* channels to C1:PSL-QPD_*_* channels in the /cvs/cds/caltech/target/c1iool0/c1ioo.db file, as well as the /opt/rtcds/caltech/c1/chans/daq/C0EDCU.ini file.  We then rebooted the frame builder via "telnet fb 8087" and then "shutdown".

This change breaks continuity for these channels prior to today.

The C1:PSL-QPD_POS_HOR and C1:PSL-QPD_POS_VERT channels were found to be backwards as well.  So we modified the /cvs/cds/caltech/target/c1iool0/c1ioo.db file to switch them.

Lastly, we changed the ASLO and AOFF values for the C1:PSL-QPD_POS_SUM and the C1:PSL-QPD_ANG_SUM so as to provide positive numbers.  This was done by flipping the sign for each entry.

ASLO went from 0.004883 to -0.004883, and AOFF when from -10 to 10 for both channels.

Hardware Changes:

The C1:PSL-QPD_ANG_SUM channel had been saturated at -10V.  Valera reduced the power on the QPD to drop it to about 4V by placing an ND attenuator in the ANG QPD path.

  4677   Tue May 10 10:06:23 2011 josephbUpdatePSLFixed channel names for PSL QPDs, fixed saturation, changed signs

I added calculation entries to the /cvs/cds/caltech/target/c1iool0/c1ioo.db file which are named C1:IOO-QPD_*_*, as the channels were originally named.  These calculation channels have the identical data to the C1:PSL-QPD_*_* channels.  I then added the channels to the C0EDCU.ini file, so as to once again have continuity for the channels, in addition to having the newer, better named channels.

The c1iool0 machine ("telnet c1iool0", "reboot") and the framebuilder process ("telnet fb 8087", "shutdown") were both restarted after these changes.

These channels were brought up in dataviewer and compared.  The approriate channels were identical.

Quote:

[Valera, Joe]

Software Changes:

First we changed all the C1:IOO-QPD_*_* channels to C1:PSL-QPD_*_* channels in the /cvs/cds/caltech/target/c1iool0/c1ioo.db file, as well as the /opt/rtcds/caltech/c1/chans/daq/C0EDCU.ini file.  We then rebooted the frame builder via "telnet fb 8087" and then "shutdown".

This change breaks continuity for these channels prior to today.

 

  4679   Tue May 10 16:42:49 2011 josephbUpdateCDSNew upconversion model (c1uct)

[Ryan, Joe]

Ryan added the c1uct (upconversion tester) model to the c1ioo machine.   It is DCU_ID 32, CPU 6.  The relevant wiki page has been updated. It has been added to /diskless/root/etc/rtsystab on the fb machine and automatically comes up when the c1ioo computer is turned on. 

Joe still needs to add its status to the status screen.

It is soft linked from:

/opt/rtcds/caltech/c1/userapps/trunk/CDS/c1/models/c1uct.mdl

Ryan will expand upon its uses later.

  4680   Tue May 10 16:45:19 2011 josephbUpdateCDSc1ass now receiving AS55I from c1lsc

[Valera, Joe]

We added a cdsPCIx_SHMEM connection between the c1lsc and c1ass models.  This connection is called C1:LSC-ASS_AS55I, and sends the normalized AS55I data to Lockin 11 of the c1ass model.

In addition, in order to get the c1ass model to compile, we had to place all the non-IO parts inside a subsystem block, which we called ASS, and gave the top_names tag to.

The c1lsc and c1ass models were rebuilt, the frame builder restarted, and the models restarted.

  4748   Thu May 19 12:09:41 2011 josephbUpdateCDSAA filter box pulled from 1X5, optic suspensions currently off

[Steve, Joe]

Steve pulled the top AA filter box from 1X5 which handled some of the suspensions channels.  We turned off all the watchdogs before pulling it out, as well as recorded which cables were connected to which inputs.

The case  is undergoing a structural modification to have the ADC adapter card which previously was loosely connected via cables, securely attached to the case.

Steve still wants to do some cabling in the rack while the box is out, and will return it this afternoon once he has finished that.

  4770   Tue May 31 11:26:29 2011 josephbUpdateCDSCDS Maintenance

1) Checked in the changes I had made to the c1mcp.mdl model just before leaving for Elba.

2) The c1x01 and c1scx kernel modules had stopped running due to an ADC timeout. 

According to dmesg on c1iscex, they died at 3426838 seconds after starting (which corresponds to ~39 days).  "uptime" indicates c1iscex was up for 46 days, 23 hours. So my guess is about 8 days ago (last Monday or Tuesday),  they both died when the ADCs failed to respond quick enough for an unknown reason.

I used the kill scripts (in /opt/rtcds/caltech/c1/scripts/) to kill c1spx, c1scx, and c1x01.  I then used the start scripts to start c1x01, then c1scx, and then finally c1spx.  They all came up fine.

Status screen is now all green.  I renabled damping on ETMX and it seems to be happy. A small kick of the optic shows the approriately damped response.

  4776   Wed Jun 1 11:31:50 2011 josephbUpdateCDSMC1 LR digital reading close to zero, readback ~0.7 volts

There appears to be a bad cable connection somewhere on the LR sensor path for the MC1 optic.

The channel C1:SUS-MC1_LRPDMon is reading back 0.664 volts, but the digital sensor channel, C1:SUS-MC1_LRSEN_INMON, is reading about -16.  This should be closer to +1000 or so.

We've temporarily turned off the LRSEN filter module output while this is being looked into.

I briefly went out and checked the cables around the whitening and AA boards for the suspension sensors, but even after wiggling and making sure everything was plugged in solidly.  There was one semi-loose connection, but it wasn't on the MC1 board, but I pushed it all the way in anyways.  The monitor point on the AA board looks correct for the LR channels, although ITMX LR struck me as being very low at about -0.05 Volts.

According to data viewer, the MC1 LR sensor channel went bad roughly two weeks ago, around 00:40 on 5/18 UTC, or 17:40 on 5/17 PDT.

 

UPDATE:

It appears the AA board (or possibly the SCSI cable connected to it) is the problem in the chain.

  4800   Thu Jun 9 16:18:03 2011 josephbUpdateCDSSecond trends only go back 12 days

While answering a quick question by Kiwamu, I noticed we only had second trends going back to 99050000 GPS time, May 27th 2011. 

Trends (I thought) were intended to be kept forever, and certainly longer than full data, which currently goes back several months.

Jamie will need to look into this.

  4918   Thu Jun 30 06:54:07 2011 josephbUpdateCDSModified the automated scripts for producing model webviews

Dave Barker pointed out last week that the webview of our simulink model files, generated from the installed models (i.e. in /opt/rtcds/caltech/c1/target/<system name>/simLink/) was not handling libraries properly.  Essentially the web pages generated couldn't see inside library parts.

This was caused by 2 problems.  The first being the userapps not being in the matlab path when the slwebview call was done, so it couldn't even find the libraries.  The second problem is the slwebview code by default doesn't follow libraries and links, and needs a special command to be told to do so.

I added the following lines to the webview_simlink_update.m file:

addpath('/opt/rtcds/caltech/c1/core/trunk/src/epics/simLink/lib')
for sub = {'cds','isc','isi','sus','psl'}
 for spath = {'common/models','c1/models/lib'}
   addpath(['/opt/rtcds/caltech/c1/userapps/release/' sub{1} '/' spath{1}]);
 end
end

I also changed the following:

temp = slwebview(final_files{x},'viewFile',false);

became

temp = slwebview(final_files{x},'viewFile',false,'FollowLinks','on','FollowModelReference','on');

After confirming these changes worked, I have sent a corrected version to Dave and Keith.

The webview results can be found at: https://nodus.ligo.caltech.edu:30889/FE/

 

 

  6051   Wed Nov 30 11:04:26 2011 josephbUpdateCDSFiltering Noise issue tracked down ???

Quote:

For now, I suppose we can just change this number to 1e-40 or so. I don't know how to calculate what the right number should be. Not sure why this underflow is not an issue for the BiQuad, however.

According to the RCG SVN logs, the reason it was removed was a more general change done to the compiled code, not specific to just the biquad.  Basically, the ability to have an underflow number (subnormal) has been turned off completely by having any number that underflows set to zero. I'm not positive, but from a quick search looks that the smallest number before hitting is an underflow as a double is 2.2250738585072014e-308.

Alex's entry from the SVN log for 2663:

Added new fz_daz() function to turn on two bits in the FPU SSE control register.
Bits FZ (flush underflows to zero) and DOZ (denorms are zeros) are set to
avoid runaway code on float/double denorms (really small numbers).
Ref: http://software.intel.com/en-us/articles/how-to-avoid-performance-penalties-for-gradual-underflow-behavior/

SVN log 2664:

Removed +- 1e-20 limiting code, this is taken care of by setting FZ/DOZ bits
in the CPU SEE control register (see mathInline.h)

SVN log 2665:

Kill the underflows and roll down float denorms to zero,
see fz_doz() in mathInline.h.

  1892   Wed Aug 12 13:35:03 2009 josephb, AlexConfigurationComputersTested old Framebuilder 1.5 TB raid array on Linux1

Yesterday, Alex attached the old frame builder 1.5 TB raid array to linux1, and tested to make sure it would work on linux1.

This morning he tried to start a copy of the current /cvs/cds structure, however realized at the rate it was going it would take it roughly 5 hours, so he stopped.

Currently, it is planned to perform this copy on this coming Friday morning.

  3128   Mon Jun 28 13:40:53 2010 josephb, AlexUpdateCDSChanges added to CDS SVN, new checkout, new features, some changes made

Last week Alex merged in the changes I had made to the local 40m copy of the Real Time Code Generator.  These were to add a new part, called FiltMuxMatrix, which is a matrix of filter banks, as well as fixing the filter medm generation code so the filter banks actually have working time stamps.

I checked out a new version of the CDS SVN with these changes merged in.  Changes that will be added in the near future by Rolf and Alex include the addition of "tags".  These are pieces in simulink which act as a bridge between two points, so you can reduce the amount of wire clutter on diagrams.  Otherwise they have no real affect on the generated C code.  Also the ADC/DAC channel selector and in fact the ADC/DAC parts will be changing.  The MIT group has requested the channel selector be freed up for its original purpose in matlab, so Rolf is working on that.

The new checkout includes the new directory scheme Rolf is pushing.  So when you run the code generator and more specifically, install SYS, it places code in /opt/rtcds/caltech/c1/ type directories, like medm, chans, target, scripts.

For the time being, Alex has created a directory /rtcds on Linux1 under /home/cds.  He then created softlinks to that directory on megatron, c1iscex, and allegra in the /opt directory.  This was an easy way to have a shared path.

However, it does mean on each new FE  machine after setting up the mounting of /home/cds from Linux1, we also need to create the /opt/rtcds link to /cvs/cds/rtcds.

After checking out the CDS SVN, we discovered there some files missing that Alex had added to his version, but not the main branch.  Alex came over to the 40m and proceeded to get all those files checked in.  We then checked it out again.  Changes were made to the awg, framebuilder, and nds codes and needed to be rebuilt. 

There's a new naming scheme for models.  You need to include the site before the 3 letter system name.  So lsc.mdl become c1lsc.mdl.

Certain other file name conventions were also changed.  Instead of tpchn_c1.par, tpchn_c2.par, etc, its now tpchn_c1lsc.par, tpchn_c2lsp.par, etc.  The system name is included at the end of the filename, to help make it clearer what file goes with what.

This required an edit of the chnconf file, which has explicit calls to those file names.  Once we edited that file, we had to reload the xinetd service which its apparently a subpart of (this can be accomplished by /etc/init.d/xinetd stop, then /etc/init.d/xinetd start).

/etc/rc.d/rc.local also had to be edited for the new model names (c1lsc, c1lsp, etc).

The daqdrc file (for the framebuilder) now parses which dcu_rate to use from the tpchn_c1lsc.par type files, so the dcu_rate 20 = 16384 lines have been removed.  set gds_server has also been removed, and replaced with tpconfig "/opt/rtcds/caltech/c1/target/gds/param/testpoint.par";  from which it can get the hostname.  This information is now derived from the c1SYS.mdl file.

Hostname needs to be added to the .mdl files, in the cdsParameter block (i.e. host=megatron).

After that Alex informed me the IOP processor needs to be running for the other models to work properly, as well as for the Framebuilder to work.

The models and framebuilder now get their timing signal from the IOP (input/output processor).  This must be running in order for the other models or FB to run properly.  Its generally named c1x00 or c1x01 or similar.  The last two numbers ideally are unique to each FE computer.

Initially there was a problem running on Megatron, because the IOP gets its timing signal from the IO chassis, and there was none connected to megatron.  However, he has since modified the code so that if there's no IO chassis, the IOP processor just uses the system clock.  It has been tested and runs on megatron now.

 

  506   Fri May 30 12:03:08 2008 josephb, AndreyConfigurationCamerasHead to head comparison of cameras
Andrey and myself - Joseph B. - have examined the output of the GC650 (CCD) and GC750 (CMOS) prosilica cameras. We did several live motion tests (i.e. rotate the turning mirror, move and rotate the camera, etc) and also used a microscope slide to try to eliminate back reflections and interference.

Both the GC650 and GC750 produce dark lines in the images, some of which look parallel, while others are in much stranger shapes, such as circles and arcs.

Moving the GC750 camera physically, we have the spot moving around, with the dark lines appearing to be fixed to the camera itself, and remain in the same location on the detector. I.e. coming back to the same spot keeps showing a circle. In reasonably well behaved sections, these lines are about 10% dips in power, and could in principle be subtracted out. Its possible that the camera was damaged with too much light incident in the past, although going back to the pmc_trans images that were taken, similar lines are still visible.

Moving the GC650 camera physically seems to change the position of the lines (if one also rotates the turning mirror to get to the same spot on the CCD). It seems as if a slight change in angle has a large effect on these dark bands, which can either be thin, or very large, bordering on the size of the spot size. My guess is (as the vendor suggested) the light is interacting with the electronics behind the surface layer rather than a surface defect producing these lines. Using a microscope slide in between the turning mirror and the GC650, we were able to produce new fringes, but didn't affect the underlying ones.

Placing a microscope slide in between the last turning mirror and the GC750 does not affect the dark lines (although it does seem to add some), nor does turning the final turning mirror, so it seems unlikely to be caused by back reflection in this case.

So it seems the CMOS may be more consistent, although we need to determine if the current line problems are due to exposure to too much light at some point in the past (i.e. I broke it) or they come that way from the factory.

Attached are the results of image-processing of the images from the two our cameras using Andrey's new Matlab script.
Attachment 1: Waveform_Reconstruction_May30-2008.pdf
Waveform_Reconstruction_May30-2008.pdf Waveform_Reconstruction_May30-2008.pdf Waveform_Reconstruction_May30-2008.pdf Waveform_Reconstruction_May30-2008.pdf Waveform_Reconstruction_May30-2008.pdf Waveform_Reconstruction_May30-2008.pdf
  4406   Fri Mar 11 18:32:45 2011 josephb, Chris, JamieUpdateCDSDebugging simplant damping

The FM1 filter module change for XXSEN was propagated to the ETMX suspension.  This was a change from a 30,100:3 with a DC gain of 1 to a 3:30 which just compensates the hardware filter.

The good gains for the Sim damping were found to be:  100 for ETMX_SUSPOS, 0.1 ETMX_SUSPIT, and 0.1 ETMX_SUSYAW, ETMX_SUSSIDE is -70.  Gains much higher tended to saturate the simulated coils (i.e. hitting 10V limit) and then had to have the histories cleared for the RESPONSE matrix.

These seem to work to damp the real ETMX as well.

  558   Tue Jun 24 17:12:10 2008 josephb, EricConfigurationCamerasGC750 setup, 1X4 Hub connected, ETMX images
The GC750 camera has been setup to look at ETMX. In addition, the new 1X4 rack mounted switch (131.215.113.200) has been connected via new cat6 cable to the control room hub (131.215.113.1?), thanks to Eric. The camera is now plugged into 1X4 rack switch and now has a gigabit connection to the control room computers as well as Mafalda (131.215.113.23).

By using ssh -X mafalda or ssh -X 131.215.113.23, then typing:

target
cd Prosilica/bin-pc/x86/
./Sampleviewer

A viewer will be brought up. By clicking on the 3rd icon from the left (looks like an eye) will bring up a live view.

Closing the view, and then cd ../../40mCode, and then running ./Snap --help will tell you how to use a simple code for taking .tiff images as well as setting things such as exposure length and size of image (in pixels) to send.

When the interferometer was set to an X-arm only configuration, we took two series of 200 images each, with two different exposure lengths.

Attached are three pdf images. The first is just a black and white single image, the second is an average of 100 images, and the third is the standard deviation of the 100 images.
Attachment 1: GC750_ETMX_E30000_single.pdf
GC750_ETMX_E30000_single.pdf
Attachment 2: GC750_ETMX_E30000_avg.pdf
GC750_ETMX_E30000_avg.pdf
Attachment 3: GC750_ETMX_E30000_std.pdf
GC750_ETMX_E30000_std.pdf
  681   Wed Jul 16 15:59:04 2008 josephb, EricConfigurationCamerasPMC trans camera path
In order to reduce saturation, we placed a Y1 plate (spare from the SP table) in transmission just before the GC650 camera looking at the PMC transmision. The reflection (most of the light) was dumped to a convient razor blade dump. We also removed the 0.3 and 0.5 ND filters and placed them in the 24 hour loan ND filter box.

Good exposure values to view are now around 3000 for that camera.
  693   Fri Jul 18 12:24:15 2008 josephb, EricConfigurationCamerasChanged Lenses on GC750 at ETMX
We removed the giant TV zoom lens and replaced it with a much smaller fixed zoom lens. Currently it views the entire optic. We have another (also small) zoom lens which focuses much better on the spot itself. With how far back the camera is currently placed, neither of these fixed zoom lenses can touch or hit the view port or the chamber while still attached to camera and mount, even using all of the mount's motion range. So this should be less of a safety issue.

Ideally, we'd like to get some images of the full optic (including osems and so forth) with the X-arm locked, and then use the higher zoom lens while still locked, to get images we can use to calibrate the x and y length scales.
  767   Wed Jul 30 13:09:40 2008 josephb, EricConfigurationPSLPMC scan experiment
We turned the PSL power down by a factor of 4, blocked one half of the Mach Zehnder and scanned the PMC by applying a ramp signal to PMC PZT. Eric will adding plots later today of those results.

We returned the power to close to original level and removed the block on the Mach Zehnder, and then relocked the PMC.
  899   Fri Aug 29 12:41:26 2008 josephb, EricConfigurationComputersMore front ends moved to new network
Used Cat6 cables to finish moving all the front ends in 1Y4 and 1Y5 over to the new GigE network switches, specifically to the switch in 1Y6. This included the ones labeled c1susvme2, c1sosvme, and c1dscl1epics0.
  932   Fri Sep 5 09:56:14 2008 josephb, EricConfigurationComputersFunny channels, reboots, and ethernet connections
1) Apparently the I00-ICS type channels had gotten into a funny state last night, where they were showing just noise, exactly when Rana changed the accelerometer gains and did major reboots. A power cycle of the c1ioo crate and appropriate restarts fixed this.

2) c1asc looks like it was down all night. When I walked out to look at the terminal, it claimed to be unable to read the input file from the command line I had entered the previous night ( < /cvs/cds/caltech/target/c1asc/startup.cmd). In addition we were unable to telnet in, suggesting an ethernet breakdown and inability to mount the appropriate files. So we have temporarily run a new cat6 cable from the c1asc board to the ITMX prosafe switch (since there's a nice knee high cable tray right there). One last power cycle and we were able to telnet in and get it running.
  922   Thu Sep 4 11:33:25 2008 josephb, Eric, JenneConfigurationComputersAttempt to increase gain for C1:PSL-ISS_INMONPD_F via 110B
We were attempting to increase the gain on the channel C1:PSL-ISS_INMONPD_F in preparation to do a scan of the PMC at very low input power.

We started by adding a line to the C1:IOOF.ini file in /cvs/cds/caltech/chans/daq/ under that channel that said "gain=10.0". Before touching anything, the channel was outputting around 4000 counts.

We hit the reconfig button for c1iovme16k, then rebooted c1iovme (which turned out to do nothing) and then the framebuilder, in a method consistent with the wiki. This turned out to put the channel in an odd state, where it was showing very rapid, random spikes, virtually but still around 4000ish counts. We returned the file back to its original format, hit reconfig, and then rebooted the framebuilder. The channel however, was still behaving in the same broken way.

After poking around the PSL table, looking at some direct outputs, we came back and rebooted c1iovme and the framebuilder again, which fixed the channel, such that it was reading out correctly. Taking this as a sign that maybe we should reboot the framebuilder, then c1iovme to get the channel to load changes, we changed the file again to have "gain=10.0". Upon reboot of the framebuilder, the channel was still reading out fine, but at the same level. So we continued with the reboot of c1iovme. This still had no effect on the channel output.

The ini file has been set back at this point, however since Yoichi is working, I'm holding off doing a reconfig and reboot on the framebuilder until later.
  4509   Mon Apr 11 13:30:04 2011 josephb, JamieUpdateCDSNo Wiper script - Frames full over weekend

Problem:

The daqd process was dying every minute or so when it couldn't write frame.  This was slowing down the network by writing a 2.9G core dump over NFS every minute or so. (In /opt/rtcds/caltech/c1/target/fb/).

The problem was /frames/ was 100% full.

Apparently, when we switched the fb over to Gentoo, we forgot to install crontab and a wiper script.

Solution:

We will install crontab and get the wiper script installed.

  822   Mon Aug 11 11:36:11 2008 josephb, SteveConfigurationComputersc1susvme1 minor problems
Around 11 am c1susvme1 start having issues. Namely C1:SUS-PRM_FE_SYNC was railing at some large value like 16384 (2^14). I presume this means the computer was running catastophically late.

I turned off the BS and ITM watch dogs (the PRM was already off), tried hitting reset and sshing in, and running startup, but this didn't help. I then turned off the c1susvme2 associated watch dogs (MC1-3, SRM) and went out to do a hard reboot by switching the crate power off. c1susvme2 came back up fine, was restarted and associated watch dogs turned back on. However, c1susvme1 came back up without mounting /cvs/cds/.

As a test, I replaced the ethernet connection with a CAT6 cable to the Prosafe switch in 1Y6, and then ran reboot on c1susvme1. When it came back up, it had mounted properly, and I was able to run the ./startup.cmd file. At this point it seems to be happy. The new cable is in the trays coming in from the top of the 1Y4 and 1Y6 and approriately labeled.

Edit: Apparently ITMX and ITMY became excited after the reboot (perhaps I turned the watchdogs back on too early? Although that was after the DAQ light was listed as green for c1susvme). Steve noticed this when the alarms went off again (I had turned them off after the reboot seemed successful), and he damped them. Interestingly, the BS remained unexcited.
  1673   Mon Jun 15 15:17:33 2009 josephb, SteveConfigurationVACVacuum control and monitor screens

We updated the vacuum control and monitor screens  (C0VAC_MONITOR.adl and C0VAC_CONTROL.adl).  We also updated the /cvs/cds/caltech/target/c1vac1/Vac.db file.

1) We changed the C1:Vac-TP1_lev channel to C1:Vac-TP1_ala channel, since it now is an alarm readback on the new turbo pump rather than an indication of levitation.  The logic on printing the "X" was changed from X is printed on a 1 = ok status) to X is printed on a 0 = problem status.  All references within the Vac.db file to C1:Vac-TP1_lev were changed.  The medm screens also now are labeled Alarm, instead of Levitating.

2) We changed the text displayed by the CP1 channel (C1:Vac-CP1_mon in Vac.db) from "On" and "Off" to "Cold - On" and "Warm - OFF".

3) We restarted the c1vac1 front end as well as the framebuilder after these changes.

  1258   Thu Jan 29 16:50:53 2009 josephb, albertoConfigurationComputersMegatron fixed
The warning light on megatron and the fans at full speed were fixed by not just power cycling, but completely unplugging megatron from power, waiting for a minute, and then reconnecting the power cables.

Apparently, the Sunfire X4600s at Hanford have also had intermittent problems, which required unplugging the machines completely. In their case, they were unable to access the machine normally, nor could they access the the Intergrated Lights Out Manager (ILOM).

Here, we could interact normally with megatron, but were unable to connect to the ILOM. We were able to get BIOS, but unable to change any of the setting for the ILOM connection. Since the ILOM is a seperate processor and effectively always on, even when the power light is off, a normal shutdown won't reset it. Hence the need to completely disconnect the power supply.
  1555   Thu May 7 15:22:19 2009 josephb, albertoConfigurationComputersfb40m

Quote:

Having determined that Rana (the computer) was having to many issues with testing the new Raid array due to age of the system, we proceeded to test on fb40m.

 

We brought it down and up several times between 11 and noon.  We  eventually were able to daisy chain the old raid and the new raid so that fb40m sees both.  At this time, the RAID arrays are still daisy chained, but the computer is setup to run on just the original raid, while the full 14 TB array is initialized (16 drives, 1 hot spare, RAID level 5 means 14 TB out of the 16 TB are actually available).  We expect this to take a few hours, at which point we will copy the data from the old RAID to the new RAID (which I also expect to take several hours).  In the meantime, operations should not be affected.  If it is, contact one of us.

 

 

 

 

This afternoon the alignment script chrashed after returning sysntax errors. We found that the tpman wasn't running on the framebuilder becasue it had probably failed to get restarted in one of the several reboots executed in the morning by Alex and Jo.

Restarting the tpman was then sufficient for the alignment scripts to get back to work.

  1668   Thu Jun 11 14:54:18 2009 josephb, albertoUpdateComputersWireless network

After poking around for a few minutes several facts became clear:

1) At least one GPIB interface has a hard ethernet connection (and does not currently go through the wireless).

2) The wireless on the laptop works fine, since it can connect to the router.

3) The rest of the martian network cannot talk to the router.

This led to me replugging the ethernet cord back into the wireless router, which at some point in the past had been unplugged.  The computers now seem to be happy and can talk to each other.

 

  1554   Thu May 7 12:21:36 2009 josephb, alexConfigurationComputersfb40m

Having determined that Rana (the computer) was having to many issues with testing the new Raid array due to age of the system, we proceeded to test on fb40m.

 

We brought it down and up several times between 11 and noon.  We  eventually were able to daisy chain the old raid and the new raid so that fb40m sees both.  At this time, the RAID arrays are still daisy chained, but the computer is setup to run on just the original raid, while the full 14 TB array is initialized (16 drives, 1 hot spare, RAID level 5 means 14 TB out of the 16 TB are actually available).  We expect this to take a few hours, at which point we will copy the data from the old RAID to the new RAID (which I also expect to take several hours).  In the meantime, operations should not be affected.  If it is, contact one of us.

 

 

ELOG V3.1.3-