40m QIL Cryo_Lab CTN SUS_Lab TCS_Lab OMC_Lab CRIME_Lab FEA ENG_Labs OptContFac Mariner WBEEShop
  40m Log, Page 240 of 337  Not logged in ELOG logo
ID Date Authorup Type Category Subject
  501   Wed May 28 12:51:32 2008 josephbConfigurationComputersTwo more switches mounted
Two more Prosafe 24 port switches have been mounted in the racks, one in 1Y9 and one in 1Y6. (The first one was placed in 1X4).

The one in 1Y9 has been set to an IP address of, while the one in 1Y6 is set to, and these have been labeled as such.
  511   Mon Jun 2 12:20:35 2008 josephbBureaucracyCamerasBeam scan has moved
The beamscan has been moved from the Rana lab back over to the 40m, to be used to calibrate the Prosilica cameras.
  517   Wed Jun 4 13:46:42 2008 josephbConfigurationCamerasChanging incident angle images
Attached are images from the GC650 and GC750 when the incident angle was varied from 0 tilt (normal incidence) to 5,10, and 20 degrees. Each time the beam was realigned via the last turning mirror to be on roughly the same spot. This light was a pickoff of the PSL table light just before it leaves the table.

Images include the raw data, fit to the data, residual normalized by peak power "w(1)", and normalized by the individual bin power.

The first pdf includes 0 degrees (normal) and ~5 degrees of tilt for the GC650 (CCD) camera.

The second pdf includes ~10 and ~20 degrees of tilt images for the GC650 (CCD) camera.

The third pdf includes 0 and ~5 degrees of tilt for the GC750 (CMOS) camera.

The fourth pdf includes ~10 and ~20 degrees of tilt for the GC750 (CMOS) camera.

Things to note:
1) GC750 camera seems to have a structure on the camera itself, somewhat circular in nature. One possible explanation is the camera was damage at a previous juncture due to too much light. Need to check earlier images for this problem.
2) GC650 has "bands" which change direction and thickness with angle. Also at higher incidence angle, the sensitivity seems to drop (unlike the GC750 where overall power level seems to stay constant with increasing angle of incidence).
3) GC650 seems to have a higher noise floor,seen from the last plot of each pdf (where each pixel of the residual is normalized by the power in the corresponding pixel of the fit).
  519   Wed Jun 4 16:57:12 2008 josephbConfigurationCamerasDark images from cameras (electronics noise measurement)
The attached pdfs are 1 second and 1 millisecond long integrations from the GC650 and GC750 cameras with a cap in place - i.e. no light.

They include the mean and standard deviation values.

The single bright pixel in the 1 second long exposure image for the GC650 seems to be a real effect. Multiple images taken show the same bright pixel (although with slightly varying amplitudes).

The last pdf is a zoom in on the z-axis of the first pdf (i.e. GC650 /w 1 sec exposure time).

I'm not really sure what to make of the mean remaining virtually fixed for the different integration times for both cameras. I guess 0 is simply offset, but doesn't result in any runaway integrations in general. Although there are certainly some stronger pixels in the long exposures when compared to the short exposures.

Its interesting to note the standard deviation actually drops from the long exposure to the short exposure, possibly influenced by certain pixels which seem to grow with time.

The one with the least variation from its "zero" was the 1 millisecond GC750 dark image.
  520   Thu Jun 5 10:46:26 2008 josephbConfigurationCamerasApproximately uniform reflected white light
In an attempt to investigate the structures seen in previous images for the GC750, I aimed it at a relatively clean section of gray table top roughly a cm or two from the surface and took images (without a lens). As I was holding this with my hand, the angle wasn't completely even with the table, and thus there's a gradient of light in the pictures. However, one should in principle be able to pick out features (such as a circular spot with less sensitivity), but these do not show up.

In my mind, these images seem to indicate the electronics are fine, and suggest that the CMOS or CCD detectors themselves are undamaged (at least in regards to white light, as opposed to 1064nm). An issue with the plastic cap (protective piece) may be the culprit, or perhaps a tiny bit of dust, which the incoherent light from all angles goes around efficiently?

Will try blowing the cameras with clean nitrogen today and see if that removes or changes the circular structure we have seen.
  521   Thu Jun 5 13:35:23 2008 josephbConfigurationCamerasGC750 looking at 1064nm scattered light
I've taken 200 images of the GC750 (CMOS) camera while holding it by hand up to a beam card (also held by hand) in the path of ~5mW of beam power. I then averaged the images to produce the fourth attached plot.

Rob has pointed out the image looks a lot like PCB traces. So perhaps we're seeing the electronics behind the CMOS sensor?

I repeated the same experiment with HeNe laser light (again scattered off a card). These show none of the detailed structure (just what looks to be a large reflection from the card moving around depending on how steady my hand was). These are the first 3 attached plots. So only 1064nm light so far sees these features.

As a possible solution, I did a quick and dirty calibration by dividing a previous PSL output beam by the 1064 average scatter light values. These produce the last attached pdf (with multiple images). The original uncalibrated image is on top, while the very simply calibrated image is on the bottom of each plot.

It seems as the effect may be power dependent (which could still be calibrated properly, but would take a bit more effort than simply dividing), as determined by looking at the edges of the calibrated plot.
  525   Fri Jun 6 16:47:04 2008 josephbConfigurationCameras GC650 scatter images of 1064nm light
Took images similar to the scattered light images from earlier, except with the CCD GC650 camera. The first three attached plots are an average of all 200 images, an average of the first 100 and then an average of the last 100 images.

They show no definite structure. The big red blob which changes with time may be a brighter reflection, although it virtually the same type of setup as the GC750 images.

To do this properly, I should grab a short focal length lens and simply blow up the beam to a size greater than the detector area and simply fix both cameras looking into.

The last set of plots are mean and standard deviation plots from a previous set of runs on 5/29/08 with the GC750 and GC650 running at the same time. The GC650 was receiving approximately 33% of the total power and GC750 was receiving 66% (in otherwords a factor of 2 more).
  530   Wed Jun 11 15:30:55 2008 josephbConfigurationCamerasGC1280
The trial use GC1280 has arrived. This is a higher resolution CMOS camera (similar to the GC750). Other than higher resolution, it has a piece of glass covering and protecting the sensor as opposed to a plastic piece as used in the GC750. This may explain the reduced sensitivity to 1064nm light that the camera seems to exhibit. For example, the image averages presented here required a 60,000 microsecond exposure time, compared to 1000-3000 microseconds for similar images from the GC750. This is an inexact comparison, and the actual sensitivity difference will be determined once we have identical beams on both cameras.

The attached pdfs (same image, different angles of view) are from 200 averaged images looking at 1064nm laser light scattering from a piece of paper. The important thing to note is there doesn't seem to be any definite structure, as was seen in the GC750 scatter images.

One possibility is that too much power is reaching the CMOS detector, penetrating, and then reflecting back to the back side of the detector. Lower power and higher exposure times may avoid this problem, and the glass of the GC1280 is simply cutting down on the amount passing through.

This theory will be tested either this evening or tomorrow morning, by reducing the power on the GC750 to the point at which it needs to be exposed for 60,000 microseconds to get a decent image.

The other possibility is that the GC750 was damaged at some point by too much incident power, although its unclear what kind of failure mode would generate the images we have seen recently from the GC750.
  682   Wed Jul 16 16:28:14 2008 josephbConfigurationComputersFixed IP address on Switch
Realized today that the change I made back on June 30th to the switch was to the wrong switch. I had disabled the DHCP setting and mislabeled the switch in the control room (which seems to not have affected anything).

I've turned DHCP back on and labeled it correctly using the Netgear "Smartwizard discovery" program.
  777   Thu Jul 31 16:11:22 2008 josephbConfigurationComputersMatlab on Megatron
Matlab now works on megatron.

I did a few things:

1) Added to the PATH environment variable. Did this in .bash_profile in the /home/controls directory by adding the line

export PATH

This probably should be somewhere else up further up the line, but I was too lazy to figure it out.

2)Fixed a gateway mistake I had added earlier so the megatron could use the NAT router and see the outside world so yum worked.

3) Removed the i386 based libXp and openmotif packages.

4) Installed the x86_64 based libXp and openmotif packages.

Edit: Forgot that I also added the following line to the /etc/fstab file in order to mount the shared code. This was stolen directly from Rosalba's /etc/fstab file. This was so that it could see the matlab code.
linux1:/home/cds/ /cvs/cds nfs rw,bg,soft 0 0
  779   Fri Aug 1 10:45:46 2008 josephbConfigurationComputersMegatron now running tcsh
At Rana's request, I've remotely switched Megatron over to using tcsh. I had to ssh -X in order ot use the "/sbin/system-config-users" program which is a graphical UI for modifying users. I had to go to preferences and uncheck hide system users, which then allowed me to see the controls user (at the bottom of the list), and edit it.

I also created a .tcshrc file in the /home/controls directory and copied the information from the .bashrc file, and also moved the matlab path definition into the PATH environment variable.

Does anyone know if sourcing /cvs/cds/caltech/cshrc.40m would be usable on a 64 bit machine, or does a new one need to be made for Megatron and/or Rosalba?
  809   Thu Aug 7 11:54:26 2008 josephbConfigurationCamerasNew code + gstreamer allows for easy saving and compression of images
Modified the CamSnap code to output the image data stream to standard out. This can then be piped into a gstreamer plugin and then be used to save, encode, transmit, receive, slice, dice and or mangle video (or virtually any type of data stream).

The gstreamer webpage can be found at: http://www.gstreamer.net/

Under documentation you can find a list off all available plug-ins. Some good, some bad, some ugly.

Running the following command on Mafalda (via ssh -X mafalda) or Rosalba while in /cvs/cds/caltech/target/Prosilica/40mCode/SnapCode/

CamSnap -F 'Mono8' -c 44058 -E 15000 -X 0 -Y 0 -H 480 -W 752 -l 0 -m 1000 | gst-launch-0.10 fdsrc fd=0 blocksize=360960 ! video/x-raw-gray, height=480, width=752, bpp=8,depth=8,framerate=1/1 ! ffmpegcolorspace ! ximagesink

This command will create a window which displays what the camera with UID 44058 is looking at. It will display 1000 images, then quit. (You can switch the -m 100 to -i to just have it continue until the process is stopped).

You can also encode the data into compressed format and save it in a media file. The following command line will encode the images into an ogg media file (.ogm), which can be played with the totem viewer (available on Rosalba or almost any machine running Ubuntu or Centos) or any other viewer capable of handling ogm files. By switching the plugins you can generate other formats as well.

The compression is good, putting 300 images normally about 500K individually uncompressed to about 580K as a single file.

The following command line was used to generate the attached video file:

CamSnap -F 'Mono8' -c 44058 -E 5000 -X 0 -Y 0 -H 480 -W 752 -l 0 -m 300 | gst-launch-0.10 fdsrc fd=0 blocksize=360960 ! video/x-raw-gray, height=480, width=752, bpp=8,depth=8,framerate=30/1 ! ffmpegcolorspace ! theoraenc ! oggmux ! filesink location="./testVideo.ogm"

Currently looking into plugins which allow you to pull individual frames out of a video file and display or save them in a variety of formats. This would allow us to save long term images in compressed video format, and then pull out individual frames as needed.

Also need to look into how to "T" the streams, so one can be displaying while another encodes and saves.
  813   Fri Aug 8 10:58:05 2008 josephbConfigurationCamerasCameras and gstreamer
In regards to camera failure:
1) I forgot to reconnect that particular camera to the network (my fault) so thats why it was failing.

2) Even with the correct camera connected, I've realized at full frame rate, op440m is going to get a few frames and then fail, as I don't think it has a fast enough ethernet card. It will work on Rosalba, and will also work ssh-ing from Rosalba because it is using a new ethernet card. It also works on my laptop, which is where I originally tested the command. One way to get around this is to increase the time between pictures, by changing -l 0 to -l 1 (or higher), where the number after the "ell" is the number of seconds to wait between frame captures.

3) What I should do is figure out the UDP transmission plugins for gstreamers and compress first (using the theoraenc since it gets compression ratios of better than 100:1) and transmit that over the network.

I have since reconnected the camera, so it should work on Rosalba and any sufficiently well connected computer. For other machines like linux2 or op440, try the following line:

Running the following command on Mafalda (via ssh -X mafalda) or Rosalba while in /cvs/cds/caltech/target/Prosilica/40mCode/SnapCode/

CamSnap -F 'Mono8' -c 44058 -E 10000 -X 0 -Y 0 -H 480 -W 752 -l 1 -m 100 | gst-launch-0.10 fdsrc fd=0 blocksize=360960 ! video/x-raw-gray, height=480, width=752, bpp=8,depth=8,framerate=1/1 ! ffmpegcolorspace ! ximagesink

This will be at a much slower frame rate (1 per second) but should work on any of the machines. (Tested on linux2).
  815   Fri Aug 8 12:21:57 2008 josephbConfigurationComputersSwitched X end ethernet connections over to new switch
In 1X4, I've switched the ethernet connections from c1iscex and c1auxex over to the new Prosafe 24 port switches. They also use the new cat6 cables, and are labeled.

At the moment, everything seems to be working as normally as it was before. In addition:

I can telnet into c1auxex (and can do the same to c1auxey which I didn't touch).
I can't telnet into c1iscex (but I couldn't do that before, nor can I telnet into c1iscey either, and I think these are computers which once running don't let you in).
  823   Mon Aug 11 12:42:04 2008 josephbConfigurationComputersContinuing saga of c1susvme1
Coming back after lunch around 12:30pm, c1susvme1's status was again red. After switching off watchdogs, a reboot (ssh, su, reboot) and restarting startup.cmd, c1susvme1 is still reporting a max sync value (16384), occassionally dropping down to about 16377. The error light cycles between green and red as well.

At this point, I'm under the impression further reboots are not going to solve the problem.

Currently leaving the watchdogs associated with c1susvme1 off for the moment, at least until I get an idea of how to proceed.
  824   Mon Aug 11 13:59:23 2008 josephbConfigurationComputers 
While poking around the crate, I noticed an error light on one of the c1susvme2 related boards was lit, while the corresponding light on the c1susvme1 was not. This confuses me as the c1susvme1 is the one having problems.

As a quick sanity check, I unplugged the ethernet connection from the c1susvme1 labeled board, and confirmed I couldn't log into it, and then plugged it back in, restarted it, and re-ran the startup script. This time c1susvme1 seemed to come up fine. Re-enabling the watchdogs doesn't seem to kick anything, and in fact seems to be bringing everything into line properly.

Although the error light on the c1susvme2 clk drvr board is still on. So I'm not sure what thats trying to tell us. Open to suggestions.
  825   Mon Aug 11 15:07:49 2008 josephbConfigurationComputersProcyon aka fb40m switched to new switch
I've connected Procyon to the Prosafe 24 port switch with a new, labeled Cat6 cable. Quick tests with dataviewer shows that its working.
  828   Tue Aug 12 12:21:13 2008 josephbConfigurationCamerasVariation in fit over 140 images for GC650 and GC750
Used matlab to calculate Gaussian fits on 145 GC650 images and 142 GC750 images. These were individual images (no averaging) looking at the PSL output from May 29th 2008. The GC650 and GC750 were looking at a split, but had different exposure values, slightly different distances to the nominal waist of the beam, and were not centered on the beam identically. Mostly this is a test of the fluctuations in the fit from image to image.

Note the mm refer to the size or position on the CCD or CMOS detector itself.

Amplitude  X center       Y center       X waist  Y waist  Background offset from zero
           position (mm)  position (mm)  (mm)     (mm)
0.3743     1.7378         2.6220         0.7901   0.8650  0.0047

Standard Deviation
Amplitude  X center       Y center       X waist  Y waist  Background offset from zero
           position (mm)  position (mm)  (mm)     (mm)
0.0024     0.0006         0.0005         0.0005   0.0003  0.00001

Std/Mean x100 (percent)
Amplitude  X center       Y center       X waist  Y waist  Background offset from zero
           position (mm)  position (mm)  (mm)     (mm)
0.6%       0.03%          0.02%          0.06%    0.04%    0.29%


Amplitude  X center       Y center       X waist  Y waist  Background offset from zero
           position (mm)  position (mm)  (mm)     (mm)
0.2024     2.5967         1.4458         0.8245   0.9194  0.0418

Standard Deviation
Amplitude  X center       Y center       X waist  Y waist  Background offset from zero
           position (mm)  position (mm)  (mm)     (mm)
0.0011     0.0005         0.0005         0.0003   0.0005  0.00003

Std/Mean x100 (percent)
Amplitude  X center       Y center       X waist  Y waist  Background offset from zero
           position (mm)  position (mm)  (mm)     (mm)
0.6%       0.02%          0.04%          0.04%    0.05%    0.07%
  835   Thu Aug 14 15:51:35 2008 josephbSummaryCamerasFOUND! The Missing Standoff!
We used a zoom lens on the GC750 to take this picture of the standoff while inside a plastic rubber-glove bag. The standoff with bag is currently scotch-taped to the periodic table of the elements.
  839   Fri Aug 15 11:52:32 2008 josephbConfigurationCamerasMulti-computer display and recording of digital camera output
Through the magic of gstreamer, I've been able to live play on one machine, compress the image, send it to another machine via udp, and also display it there. The "tee" function also allows one to save at the same images at time as well.

The command line used on the "server", say Rosalba or Mafalda is:

CamServe -F 'Mono8' -c 44058 -E 20000 -X 0 -Y 0 -H 480 -W 752 -l 0 -m 100 | gst-launch-0.10 fdsrc fd=0 blocksize=360960 ! video/x-raw-gray, height=480, width=752, bpp=8,depth=8,framerate=60/1 ! tee name=t1 t1. ! video/x-raw-gray, height=480, width=752, bpp=8,depth=8,framerate=60/1 ! ffmpegcolorspace ! ximagesink t1. ! video/x-raw-gray, height=480, width=752, bpp=8,depth=8,framerate=60/1 ! ffmpegcolorspace ! queue ! smokeenc keyframe=8 qmax=40 ! udpsink host="" port=5000

This both displays the image and sends it to the host in this case.

I've written a primitive shell script that does most of this.

It requires at the minimum an IP address. You can also give it a number of images (the -m number) and also the exposure value (-E 20000).

Currently in /cvs/cds/caltech/target/Prosilica/40mCode/SnapCode/ there is a script called CameraServerScript.

Typing in "CameraServerScript" would send it to that IP address.

Typing in "CameraServerScript 500 40000" would run for 500 images at an exposure value of 40000.

To actually receive, you need gstreamer installed and run the following command:

gst-launch udpsrc port=5000 ! smokedec ! queue ! ffmpegcolorspace ! ximagesink sync=false

Make sure you have the right IP address to send to.

Still working on multicasting (basically a server is constantly sending out images, and the client subscribes to the multicast).
  847   Mon Aug 18 15:32:18 2008 josephbConfigurationCamerasHow to multicast with gstreamer and Gige Cameras
In order to get multicasting to work, one simply needs to understand the address scheme.

In general, the address range - are reserved for multicasting. Within in this address space, there are some base level operations in the 224.0.0.x range which shouldn't be interfered with.

For a single site, the address range between and is probably best.

Gstreamer and the current 40m network hubs are designed to handle this kind of communication already, so one merely needs to point them at the correct addresses.

While in /cvs/cds/caltech/target/Prosilica/40mCode/SnapCode type:

CamServe -F 'Mono8' -c 44058 -E 20000 -X 0 -Y 0 -H 480 -W 752 -l 0 -m 300 | gst-launch-0.10 fdsrc fd=0 blocksize=360960 ! video/x-raw-gray, height=480, width=752, bpp=8,depth=8,framerate=60/1 ! ffmpegcolorspace ! queue ! smokeenc keyframe=8 qmax=40 ! udpsink host= port=5000

This will multicast to the address, using port 5000.

On the machine you wish to subscribe type:

gst-launch udpsrc multicast-group= port=5000 ! smokedec ! ffmpegcolorspace ! ximagesink sync=false
  852   Tue Aug 19 13:34:58 2008 josephbConfigurationComputersSwitched c1pem1, c0daqawg, c0daqctrl over to new switches
Moved the Ethernet connections for c1pem1, c0daqawg, and c0daqctrl over to the Netgear Prosafe switch in 1Y6, using new cat6 cables.
  897   Fri Aug 29 11:01:49 2008 josephbConfigurationComputersAttempt to change a channel gain in ICS-110B
As noted earlier by Rana, I was playing around with the /cvs/cds/caltech/chans/daq/C1IOOF.ini file with help from Rob. I had made a backup before hand and saved it as C1IOOF.ini.Aug-28-2008. (I have since been informed that C1IOOF.ini.082808 would have been prefered as a name).

We had been trying to up the gain in the C1: PSL-ISS_INMONPD_F in order to do a very low power PMC sweep, in an attempt to get clean modes for fitting. Initially we pressed the reconfig button on the C0DAQ_DETAIL screen, but all that seemed to do was change the Config File CRC. We proceeded to reboot fb40m remotely. However, any change to the ini file (even an extra space at the end of the file) caused a 0x2000 status for C1IOVME16k on the C0DAQ_DETAIL screen. At the time I presumed it was comparing the CRC of the ini-file to something else.

Digging around on in Alex's webspace at http://www.ligo.caltech.edu/~aivanov/ , I found the NDS Access page, which indicated that 0x2000 was a conflict between the front-end and frame builder .ini files.

"There is also status bit 0x2000 which gets added when the DCU configuration is different in front-end and frame builder. That is you can change and .ini file an then reload DAQ configuration with Epics button, which reconfigures the front-end, but leaves frame builders with invalid old configuration. They will detect this change and set the status to 0x2000 to indicate this condition. You will have to restart frame builders to pick up new .ini file and set status back to zero for the affected DCU."

It was when I was going to try reseting the c1iovme via the C0DAQ_RFMNETOWRK medm screen that we realized the EPICS controls were not responding properly. The .ini file was returned to its original form, and mass reboots commenced.
  898   Fri Aug 29 11:05:11 2008 josephbSummaryComputersc1asc was down this morning
I had to manually reboot c1asc this morning, as for some unknown reason its status was red, and the fiber lights on the board were status:red, sig det:amber, own data: nothing. Shut the crate down, turned it back on, heard a beep, then followed wiki reboot instructions. Seems to be working now.
  900   Fri Aug 29 12:43:44 2008 josephbSummaryComputersc1susvme1 down
Around noon today, c1susvme was having problems. The C0DAQ_RFMNETWORK light was red. The status light was off, the sig det light was amber and the own data light was green. I could also ssh in, but could not not run startup. I switched off the watchdogs for c1susvme2 (the watchdogs for c1susvme1 had already been tripped), and manually power cycled the crate.

However, when c1susvme1 when it came back up it had not mounted the usual cvs/cds/ directories. c1susvme2 did however. c1susvme1 has been on the new network for awhile, while c1susvme2 was switch over today. So apparently switching networks doesn't help this particular problem.

I did a remote reboot of c1susvme1, and it came up with the correct files mounted. Both machines ran their approriate startup.cmd files and are currently green.
  941   Thu Sep 11 11:29:14 2008 josephbConfigurationComputersFinal netgear switch in place in 1Y2
I've placed the final (of 4) Netgear prosafe 24 port switch at the very top of 1Y2. At that location, there are no holes left to screw into, so it has 4 rubber feet and is sitting on the top most signal generator. It has been plugged in and connected to the control room hub with a labeled cat6 ethernet cable.

Its IP address has been set to, and has the usual controls password if using the "Smart Wizard Discovery Tool" which comes on the Netgear CD. The CD can be found in the Equipment manuals filing cabinet under Netgear. This program unfortunately only runs on a window PC.

To Do: Fix the C1:ASC ethernet connection which is currently coming straight out the front door and connected to the 1X4 switch (again through the front door).
  948   Mon Sep 15 14:00:52 2008 josephbConfigurationComputers1Y9 Hub and C1asc
The 1Y9 switch is now using a labeled Cat6 cable in cable trays to connect to the main switch in the offices. In addition, the c1asc cable which had been coming out the door was fixed last Friday, and is now labeled, going out the top and connects to the hub in 1Y2.

Note: Do not connect new ethernet cable from switch to switch without disconnecting the old cable to the rest of the network - this tends to make the Ethernet network unhappy with white flashing alarms.
  965   Thu Sep 18 14:36:54 2008 josephbConfigurationComputersName server and Epics
The problems Rob was experiencing last night was due to part of the setup (or rather testing of the setup) of the new nameserver running on linux1.

The name server was setup on linux1 by doing the following:

1) Installed xorg-x11-xauth via yum which was necessary to get remote x windows to work in linux1

2) Installed xorg-x11-fonts-Type1 in order to get the gui system-config-* programs to work

3) Ran system-config-bind, which created a default set of nameserver files. I unfortunately didn't understand the gui all that well, so I manually edited and added files to these base ones. The base files were generated in /var/named/chroot/etc/ and /var/named/chroot/var/named.

4) I added martian.zone and 113.215.131.in-addr.arpa.zone, named.conf.local, and edited named.conf so it loaded named.conf.local. The martian.zone file acts a forward look up (i.e. give it a name and it returns an IP number like The 113.215.131.in-addr.arpa.zone acts as a reverse look up (i.e. give it an IP number like and it tells you the name). The file named.conf.local merely points to these two files.

Note: One can add or change IP lookup by simply updating these two files. The format should be obvious from the files.

5) I specifically ssh'd in as root to linux1 (using su wasn't sufficient) and then typed "service named start" (without quotes). You can also use "restart" or "stop" instead of "start". This started the name server, giving an [Ok] message.

6) I edited the /etc/resolve.conf file on linux1 so that it pointed to itself first ("nameserver" at the top of the file). I also added the line "search martian", which allows one to simply use linux1 as opposed to linux1.martian.

I also edited the /etc/resolve/conf file on linux2, and it seems to resolve names fine.

7) And here is where I broke things. As a test, I moved /etc/hosts to /etc/hosts.bak, and then tested to see if names were being resolved correctly. By using the command host, I determined they were in fact working. I also tested with ssh.

However, something basic didn't like me moving the hosts file. Apparently when a front-end machine needed to reboot, it wouldn't come back up, without any ability to SSH or telnet into them.

With Yoichi and I did quite a bit of debugging this morning and determined the nameserver itself isn't conflicting, merely the lack of the host file was the source of the problem. One theory is that services don't know to go to DNS to resolve host names. I think by modifying the /etc/nsswitch.conf file to include dns as an option for services and other programs, it might work without the host file, however, I'm going to leave that to tomorrow morning which is less likely to interfere with current operations.

As it stands, things are working with the nameserver running and the host file in place.
  973   Fri Sep 19 11:21:45 2008 josephbConfigurationComputersNameserver and Rosalba
I tried modifying the nsswitch.conf file to include going to dns in addition to local files for everything (services, network, etc) and then moving the /etc/hosts file to /etc/hosts.bak. Unfortunately, this still didn't allow front-ends to reboot properly. So I'm not sure what is using the hosts file, but whatever it is, is apparently important. After the test I placed the hosts file back and reverted the nsswitch.conf file.

I also noticed that Rosalba was having problems connecting to the network. This apparently was because I had shut down the dhcp server on the NAT router, as had been discussed at the meeting on Wednesday.

To fix this, I modified the /etc/sysconfig/network-scripts/ifcfg-eth1 file to fix rosalba's ip as (which doesn't seem to be in use). I also updated rosalba's /etc/resolv.conf file to point at linux1's name server, and two additional name servers as well, and added the "search martian" line. I modified the /etc/sysconfig/network-scripts/ifcfg-eth0 file so the built in network card doesn't come up automatically, since its currently not plugged into anything. Lastly, I added rosalba and its IP to linux1's name server files.
  992   Thu Sep 25 14:03:08 2008 josephbConfigurationComputers 


We (Joe) need to buy a GigE card for linux1 and to also set up conlog and conlogger to run on Nodus.

A spare Intel Pro 1000/GT desktop adapter (gigabit ethernet card) has been added to Linux1 and is now using that card to connect to the network.

This was after a slight scare when I somehow reset the bios on Linux1 during the first reboot after adding the card.
After some debugging and discussion with Yoichi, the bios was fixed and the computer works again, with its new faster network connection.

Although we both noted that Linux1 is a rather old machine, with only half a gig of Ram and reaching about 80% capacity on its 58 gigabyte hard drive (raid). Might be worth upgrading in general.

Need to figure out how to install conlog/conlogger programs next...
  1006   Mon Sep 29 13:33:39 2008 josephbConfigurationComputersGigabit network finished and conlog available on Nodus
The last 100 Mb unmounted hub has been removed (or at least of the ones I could find). We should be on a fully gigabit network with Cat6 cables and lots and lots of labels.

In other news, the pearl script that runs the web interface on linux1 for the conlog has been copied to /cvs/cds/caltech/apache/cgi-bin/ and is now being pointed to by the apache server on Nodus.

  1033   Wed Oct 8 12:35:56 2008 josephbConfigurationComputersNew Network diagram for the 40m
Attached is a pdf of the new network diagram for the 40m after having removed all of the old hubs.
  1067   Wed Oct 22 12:37:47 2008 josephbUpdateComputersNetwork spreadsheet
Attached in open office format as well as excel format is spreadsheet containing all the devices with IP addresses at the 40m. Please contact me with any corrections.
  1098   Tue Oct 28 12:01:01 2008 josephbConfigurationComputerslinux2

I have removed linux2 and its cables from the control room and put it into 1Y3 along with op340m.

When Joe next comes in we can ask him to Cat6 it to the rest of the world, although it already
seems to me that the CDS hub/switch next Alberto's desk is too full and that we need to purchase
a 48 port device for there.

Note I still need to remove a fair bit of cabling no longer in use from the Martian network switch next to Alberto's desk. There's actually about 8-10 cables there which show no connectivity and are not being used. So there's really about 33% of the ports open in the control room hub, it just doesn't look like it.

As for linux2, I'll probably just connect it to the 1Y2 or 1Y6 Hubs when I get the chance.
  1175   Thu Dec 4 16:29:20 2008 josephbConfigurationComputersError message on Frame Builder Raid Array
The Fibrenetix FX-606-U4 RAID connected to the frame builder in 1Y7 is showing the following error message: IDE Channel #4 Error Reading
  1253   Mon Jan 26 14:51:54 2009 josephbConfigurationVAC 
We need a new RS-232 to Ethernet bridge in order to interface properly with the new RGA. The RGA has a fixed baud rate of 28.8k, and the current bridge (which used to work with the old RGA) doesn't have that baud rate as an option. Currently looking into purchasing a new bridge, and trying to make sure it can meet the communications requirements of the RGA.
  1294   Wed Feb 11 15:01:47 2009 josephbConfigurationComputersAllegra

So after having broke Allegra by updating the kernel, I was able to get it running again by copying the xorg.conf.backup file over xorg.conf in /etc/X11.  So at this point in time, Allegra is running with generic video drivers, as opposed to the ATI specific and proprietary drivers.

  1333   Mon Feb 23 16:42:08 2009 josephbConfigurationCamerasCamera Beta Testing

I've setup the GC650 camera (ID 32223) to look at the mode cleaner transmission.  I've also added an alias to the camera server and client for this camera.

To use, type: "pserv1 &"on the machine you want to run the server on and "pcam1 &" on the machine you want to actually view the video.  At the moment, this only works for the 64-bit Centos 5 machines, Rosalba, Allegra and Ottavia.

Note, you will generally want to start the client first (pcam1 or pcam2) to see if a server is already running somewhere.  The server will complain that it can't connect to a camera if it already is in use.

I've setup the GC750 camera (ID 44026) to look at the the right most analog quad TV.  This can be run by using "pserv2 &" and "pcam2 &". 

If the image stops playing you can try starting and stoping the server to see if will start back up. 

You can also try increasing or decreasing the exposure, to see if that helps.  The increase and decrease buttons change the exposure by a factor of 2 for each press.

  Lastly, the button "Read Epic Channel" reads in the current value from the channel: "C1:PEM-stacis_EEEX_geo" and uses it as the exposure value, in microseconds (in principle 10 to 1000000 should work).

For example, to exposre for 10000 microseconds, use "ezcawrite C1:PEM-stacis_EEEX_geo 10000" and press the "Read Epic Channel" button.

  1355   Wed Mar 4 17:20:04 2009 josephbUpdateCamerasCamera code upgrades

I've updated the digital camera python code as well as changed the network topology.

At the moment, both cameras are connected to a small gigabit switch which only talks to Ottavia.  This means all camera servers must be run on Ottavia, allow camera output is still UDP multicast so any machine capable of running gstreamer can pick up the images.

The server and client programs now have the ability to read a configuration file for the setup of the cameras.  They default to pcameraSettings.ini, but this can default can be changed with a -c or --config option

For example, "serverV3.py --config pcam1.ini" will run the server using the pcam1.ini settings file.  Similarly, "client.py --config pcam1.ini" will also take the IP settings from the config file so that it knows at which port and IP to listen.

These programs and .ini files have been placed in /cvs/cds/caltech/apps/linux64/python/pcamera/

I've updated the cshrc.40m aliases so that it uses the new configuration file options, so now pcam1 calls "client.py -c pcam1.ini" in the above directory.

So to start a client use pcam1 or pcam2 (for the 32223 camera in PSL looking at MC trans or 44026 looking at an analog moniter in the control room respectively).  These can be run on Allegra, Rosalba or Ottavia at the moment.

To start a server, use pserv1 or pserv2.  These *must* be run on Ottavia.

I've also added a -n or --no-gui option at Yoichi's request, one which just starts up and plays, with no graphical gui.

Lastly, I've made some changes to the base pcamerasrc.py file, which should make display more robust.  After a failed transmission of an image from the camera to Ottavia, it should re-attempt up to 10 times before giving up. I'm hoping this will make it more robust against packet loss.  The change in network topology has also helped this, allowing 640x480 to be transmitted on both cameras before tens of minutes before a packet loss causes a stop.

  1385   Wed Mar 11 11:30:15 2009 josephbConfigurationCameras 

I modified the Video.db file used by c1aux located in  /cvs/cds/caltech/target/c1aux.

I added the following channels to the db file, intended for either read in or read out by the digital camera scripts.











A better naming scheme can probably be devised, but these will do for now.

  1496   Sun Apr 19 11:34:33 2009 josephbHowToCamerasUSB Frame Grabber - How to

To use the Sensoray 2250 USB frame grabber:

Ensure you have the following packages installed: build-essential, libusb-dev

Download the Linux manual and linux SDK from the Sensoray website at:


Go to the Software and Manual tab near the bottom to find the links.  The software can also be found on the 40m computers at /cvs/cds/caltech/users/josephb/sensoray/

The files are Manual2250LinuxV120.pdf and s2250_v120.tar.gz

Run the following commands in the directory where you have the files.

tar -xvf s2250_v120.tar.gz

cd s2250_v120


cd ezloader


sudo make modules_install

cd ..

At this point plug in the 2250 frame grabber.

sudo modprobe s2250_ezloader

Now you can run the demo with

./sraydemo or ./sraydemo64

Options will show up on screen.  A simple set to start with is "encode 0", which sets the recording type, "recvid test.mpg", which starts the recording in file test.mpg, and "stop", which stops recording.  Note there is no on screen playback.  One needs an installed mpeg player to view the saved file, such as Totem (which can screen cap to .png format) or mplayer.

All these instructions are on the first few pages of the Manual2250LinuxV120 pdf.



  1497   Sun Apr 19 11:51:05 2009 josephbUpdateCamerasMafalda may need an update

I tried installing libusb-dev on mafalda in order to try getting the usb frame grabber to work on it, but could not as it could not download the package.

I then tried to do a sudo apt-get update, which failed completely, as the repository seems to have ceased existing.  Basically I had all 404 Not Found errors.

Turns out Mafalda is still running Ubuntu 7.04, whose support ended late 2008.  So there's a couple things that can be done:

1) Ignore it, and simply not update Mafalda anymore.  This also means some newer software and hardware simply won't work with it (like the usb frame grabber)

2) Try to find another, unofficial repository which still has all of the Ubuntu 7.04 packages.

3) Upgrade to a newer, still supported Ubuntu, such as 7.10, 8.04, or 8.10.

I'd personally lean towards the 3rd option, and go to the 8.04 long term support version.  If people agree with it, I could do the upgrade sometime Monday or Tuesday.



  1581   Wed May 13 12:41:14 2009 josephbUpdateCamerasTiming and stability tests of GigE Camera code

At the request of people down at LLO I've been trying to work on the reliability and speed of the GigE camera code.  In my testing, after several hours, the code would tend to lock up on the camera end.  It was also reported at LLO after several minutes the camera display would slow down, but I haven't been able to replicate that problem.

I've recently added some additional error checking and have updated to a more recent SDK which seems to help.  Attached are two plots of the frames per second of the code.  In this case, the frames per second  are measured as the time between calls to the C camera code for a new frame for gstreamer to encode and transmit.  The data points in the first graph are actually the averaged time for sets of 1000 frames.  The camera was sending 640x480 pixel frames, with an exposure time of 0.01 seconds.  Since the FPS was mostly between 45 and 55, it is taking the code roughly 0.01 second to process, encode, and transmit a frame.

During the test, the memory usage by the server code was roughly 1% (or 40 megabytes out of 4 gigabytes) and 50% of a CPU (out a total of  CPUs).

  1590   Fri May 15 16:47:44 2009 josephbUpdateCamerasImproved camera code

At Rob's request I've added the following features to the camera code.

The camera server, which can be started on Ottavia by just typing pserv1 (for camera 1) or pserv2 (for camera 2), now has the ability to save individual jpeg snap shots, as well as taking a jpeg image every X seconds, as defined by the user.

The first text box is for the file name (i.e. ./default.jpg will save the file to the local directory and call it default.jpg).  If the camera is running (i.e. you've pressed start), prsessing "Take Snapshot to" will take an image immediately and save it.  If the camera is not running, it will take an image as soon as you do start it.

If you press "Start image capture every X seconds", it will do exactly that.  The file name is the same as for the first button, but it appends a time stamp to the end of the file.

There is also a viedo recording client now.  This is access by typing "pcam1-mov" or "pcam2-mov".  The text box is for setting the file name.  It is currently using the open source Theora encoder and Ogg format (.ogm).  Totem is capable of reading this format (and I also believe vlc).  This can be run on any of the Linux machines.

The viewing client is still accessed by "pcam1" or "pcam2".

I'll try rolling out these updates to the sites on Monday.

The configuration files for camera 1 and camera 2 can be found by typing in camera (which is aliased to cd /cvs/cds/caltech/apps/linux64/python/pcamera) and are called pcam1.ini, pcam2.ini, etc.


  1901   Fri Aug 14 10:39:50 2009 josephbConfigurationComputersRaid update to Framebuilder (specs)

The RAID array servicing the Frame builder was finally switched over to JetStor Sata 16 Bay raid array. Each bay contains a 1 TB drive.  The raid is configured such that 13 TB is available, and the rest is used for fault protection.

The old Fibrenetix FX-606-U4, a 5 bay raid array which only had 1.5 TB space, has been moved over to linux1 and will be used to store /cvs/cds/.

This upgrade provides an increase in look up times from 3-4 days for all channels out to about 30 days.  Final copying of old data occured on August 5th, 2009, and was switched over on that date.

  1904   Fri Aug 14 15:20:42 2009 josephbSummaryComputersLinux1 now has 1.5 TB raid drive



nodus was rebooted by Alex at Fri Aug 14 13:53. I launched elogd.

cd /export/elog/elog-2.7.5/
./elogd -p 8080 -c /export/elog/elog-2.7.5/elogd.cfg -D

 It looks like Alex also rebooted all of the control room computers.  Or something.  The alarm handler and strip tool aren't running.....after I fix susvme2 (which was down when I got in earlier today), I'll figure out how to restart those.

 Alex switched the mount point for /cvs/cds on Linux1 to the 1.5 TB RAID array after he finished copying the data from old drives.  This required a reboot of linux1, with all the resulting /cvs/cds mount points on the other computers becoming stale.  Easiest way to fix that he found was to do a reboot of all the control room machines.  In addition, a reboot fest should probably happen in the near futuer for all the front end machines since they will also have stale mount points as well from linux1.

The 1.5 TB RAID array mount is now mounted on /home of linux1, which was the old mount point of the ~300 GB drive.  The old drive is now at /oldhome on linux1.


  1967   Fri Sep 4 16:09:26 2009 josephbSummaryVACRebooted RGA computer and reset RGA settings

Steve noticed the RGA was not working today.  It was powered on but no other lights were lit.

Turns out the c0rga machine had not been rebooted when the file system on linux1 was moved to the raid array, and thus no longer had a valid mount to /cvs/cds/.  Thus, the scripts that were run as a cron could not be called.

We rebooted c0rga, and then ran ./RGAset.py to reset all the RGA settings, which had been reset when the RGA had lost power (and thus was the reason for only the power light being lit).


Everything seems to be working now.  I'll be adding c0rga to the list of computers to reboot in the wiki.

  2068   Thu Oct 8 11:37:59 2009 josephbUpdateComputersReboot of dcuepics helped, c1susvme1 having problems

Power cycling c1dcuepics seems to have fixed the EPICs channel problems, and c1lsc, c1asc, and c1iovme are talking again.

I burt restored c1iscepics and c1Iosepics from the snapshot at 6 am this morning.

However, c1susvme1 never came back after the last power cycle of its crate that it shared with c1susvme2.  I connected a monitor and keyboard per the reboot instructions.  I hit ctrl-x, and it proceeded to boot, however, it displays that there's a media error, PXE-E61, suggests testing the cable, and only offers an option to reboot.  From a cursory inspection of the front, the cables seem to look okay.  Also, this machine had eventually come back after the first power cycle and I'm pretty sure no cables were moved in between.


  2183   Thu Nov 5 16:41:14 2009 josephbConfigurationComputersMegatron's personal network

In investigating why megatron wouldn't talk to the network, I re-discovered the fact that it had been placed on its own private network to avoid conflicts with the 40m's test point manager.  So I moved the linksys router (model WRT310N V2) down to 1Y9, plugged megatron into a normal network port, and connected its internet port to the rest of the gigabit network. 

Unfortunately, megatron still didn't see the rest of the network, and vice-versa.  I brought out my laptop and started looking at the settings.  It had been configured with the DMZ zone on for, which was Megatron's IP, so communications should flow through the router. Turns out it needs the dhcp server on the gateway router ( to be on for everyone to talk to each other.  However, this may not be the best practice.  It'd probably be better to set the router IP to be fixed, and turn off the dhcp server on the gateway.  I'll look into doing this tomorrow.

Also during this I found the DNS server running on linux1 had its IP to name and name to IP files in disagreement on what the IP of megatron should be.  The IP to name claimed while the name to IP claimed  I set it so both said  (These are in /var/named/chroot/var/ directory on linux1, the files are 113.215.131.in-addr.arpa.zone  and martian.zone - I modified the 113.215.131.in-addr.arpa.zone file).  This is the dhcp served IP address from the gateway, and in principle could change or be given to another machine while the dhcp server is on.

  2192   Fri Nov 6 10:35:56 2009 josephbUpdateComputersRFM reboot fest and re-enabled ITMY coil drivers

As noted by Steve, the RFM network was down this morning.  I noticed that c1susvme1 sync counter was pegged at 16384, so I decided to start with reboots in that viscinity.

After power cycling crates containing c1sosvme, c1susvme1, and c1susvme2 (since the reset buttons didn't work) only c1sosvme and c1susvme2 came back normally.  I hooked up a monitor and keyboard to c1susvme1, but saw nothing.  I power cycled the c1susvme crate again, and this time I watched it boot properly.  I'm not sure why it failed the first time.

The RFM network is now operating normally.  I have re-enabled the watchdogs again after having turned them off for the reboots.  Steve and I also re-enabled the ITMY coil drivers when I noticed them not damping once the watch dogs were re-enabled.  The manual switches had been set to disabled, so we re-enabled them.

ELOG V3.1.3-