Hmmm.  I'm no longer convinced that ETMY is healthy.  I think that when I gave it a kick, it's bouncing against something.  I can't fit the peaks to get the input matrix.  I guess step 1 is to try giving it a smaller kick for the free swinging spectra.  But if the owl shift folk feel like it, they might have a look-see.

I setup the usual mini-cleanroom setup around the ETMY chamber. Then I carried out the investigative plan outlined here.

Main finding: I saw a fiber of what looks like first contact on the bottom left (as viewed from HR side) of ETMY, connecting the optic to the cage. See Attachment #1. I don't know that this can explain the problem with the missing eigenmode, it's not a hard constraint.  Seems like something that should be addressed in any case. How do we want to remove this? Just use a tweezer and pull it off, or apply a larger FC patch and then pull it off? I'm pretty sure it's first contact and not a piece of PEEK mesh because I can see it is adhered to the HR side of the optic, but couldn't capture that detail in a photo.

There weren't any obvious problem with the magnet positioning inside the OSEM, or the suspension wire. All the EQ stop tips were >3mm away from the optic.

I also backed out the bottom EQ stops on the far (south side) of the optic by ~2 full turns of the screw. Taking another free-swinging dataset now to see if anything has changed. I will upload all the photos I took, with annotations, to the gPhotos later today eve. Light doors back on at ~1730.

Update 10pm: the photos have been uploaded. I've added a "description" to each photo which should convey the message of that particualr shot, it shows up in my browser on the bottom left of the photo but can also be accessed by clicking the "info" icon. Please have a look and comment if something sticks out as odd / requires correction.

Update 1045pm: I looked at the freeswinging data from earlier today. Still only 3 peaks around 1 Hz.

A pair of tweezer is OK as long as there is no magnets around. You need to (somewhat) constrain the mirror with the EQ stops so that you can pull the fiber without dragging the mirror.

The mechanical shutter on the Y end is now fully functional.

 It is newly named to 'C1AUX_GREEN_Y_Shutter' in the EPICS world.

It uses the same binary output channel which had been served for the POY shutter.

To change the EPICS name I edited a db file called ShutterInterLock.db, which resides in /cvs/cds/caltech/target/c1aux.

After editing the file I rebooted the c1aux machine, by telnet and the reboot command in order to make the change effective.

Also I added this new shutter on the ALS overview screen (see the attachment below)

The ETMY  shutter can be remotely switched from medm screen POY of mechanical shutters.

The new cable from ETMY controller goes to east vertex EV-ISCT where it is connected to POY shutter hook up BNC cable.

[Jenne, Jamie]

ETMY is now in its new nominal position, according to the rails that Kiwamu put in the other day.  OSEM voltages are all centered, and the magnets looked pretty well centered in the OSEM bores.  We're taking data for some free swinging spectra, to check the decoupling. 

Next up: Align Y-green to the arm, then move on to fixing the other optics that Jamie pointed out.

All ETMY optical table electronics- lasers-pds turned off, disconnected in order to remove enclosure.

 TODO:

1. We need to replace all of the floppy anodized Al dumps with clean razor blade dumps on stiff mounts. BOTH of the rejected ports of the 1064 FI need some kind of custom dump.
2. All of the leakthrough beams of the HR mirrors also need razor dumps. A good rule of thumb is that your first notion of how to implement the beam dump is NOT good enough.
3. The whole lens / modematching situation for the 1064 and 532 paths will have to be redone so as to put the beam waists inside the Faraday crystal (NOT outside). The beam waist in the 4.7 mm diameter Faraday should be ~0.3-0.4 mm.
4. The efficiency that we got for the doubling shows that we don't need the cylindrical lenses - they are nice, but not needed to get 90% of the max power.
5. The lenses between the 1064 FI and the doubler should be put onto a base that can be used to adjust the lens position for MM optimization. Nothing fancy, just something slideable.
6. For this iteration of the table, we can do as Annalisa has written so as to get the green MM lenses ordered ASAP. After next week we can come up with a new plan before dismantling the EX table.

According to c1scy.mdl, OL signals should be connected to adc_0_24 to adc_0_27 but they were connected to adc_0_16 to adc_0_19 which are assigned to QPD signals.

Actually cable connections were messed up. One ribbon cable was connected from QPD driver and ADC ports assigned for OL, and another ribbon cable was connected from the board combining the signals of oplev and QPD to ADC port assigned for QPD.

Now ETMY oplev is working well and aligned to center.

 Yesterday I swapped in new He/Ne laser with output power 3.5 mW  The return spot on qpd is large ~6mm in diameter and 20,500 counts

The spot size reduction require similar layout as ETMX oplev.

ETMY oplev is currently a work in progress.  The HeNe beam is hitting the photodiode, but the spot size there is pretty much the size of the entire QPD.  Thus, the ETMY oplev isn't really useful right now.  I'm re-figuring things out (note to self: close to the laser, you have to use Gaussian optics...regular ray tracing doesn't really work), and hopefully will have the oplev back under control by the time Alberto is finished realigning the IFO, so this doesn't keep anyone from doing any exciting locking work.

ETMY oplev is still out of order.  Hopefully I'll get it under control by tomorrow. 

 The typical sign of a dying gas laser is that it glows for a few minutes only. The power supplies are fine.

Two new  JDS - Uniphase 1103P lasers ( NT64-104 )  arriving on Monday, May 21

 ETMY oplev laser clearly showing a tail when it was projected up the sealing.

PS (10-4-2013): I checked the beam quality again as it was removed from the table: it had a good image at 3 meters

 We also redid the oplev path.

Someone had used the illegal tactic of using aluminum dogs to hold down mounts which already have screw holes in the bases. This is too flimsy to use.

We then adjusted the position of the second lens to get a ~1 mm spot on the QPD. There is a lot of stray red light from extra reflections, but so be it.

Its possible that we are now clipping the IPANG path, but we will need to lock the Y arm and verify the input pointing to be sure. At this point we are ready to measure the ETMY OL loop gain and tune, etc.

[Jenne, MikeJ]

The ETMY oplev quad sum was ~0, so we went to investigate.  The power supply was dead.  Keying it on and off didn't fix things, and it was certainly getting power since the front indicator light was flickering.  We replaced it with a different power supply, and the oplev is back.

Also, the Y-green AUX laser was off, presumably from the power outage a while back.  I turned it back on.

EDIT JD:  The first power supply we used didn't work....the oplev was on for a few minutes, then went off again.  We switched to one of the new (still JDSU) power supplies from Edmund Optics, and it's been happily working for at least an hour now.

And has the temp controller of the green been recovered too? (Push enable button)

 Just now, yes.

 I did relayed the oplev path with new  f 500 mm lens

[Jenne, Annalisa]

The ETMY Oplev servo didn't work properly, when it was activated the ETMY moved too much.

We measured the oplev TF for Pitch and Yaw and it turned out that the gain was too low by a factor 3, so we increased the gain from -.250 to -.750 on both.

We also locked the Y arm and we could see that the mirror's oscillations are actually suppressed.

 The oplev path is relayed and the spot size on the qpd is reduced. I still have to clean up and replace "Miki Mouse" lens holder.

There was no IP-ANG coming out of the chamber at this time!

 Flipped the sign on the ETMY oplev servo gain, since it was wrong.  (It was "-" for both, now it is "+" for both)

The whitening filters for the ETMY oplevs are back.

The whitening board had been in the rack but the ADC was connected directly to the oplev interface board without going through the whitening board.

In fact the interface board and the whitening board had been already connected. So the ADC was making a shortcut.

I disconnected the ADC from the interface board and plugged it to the output of the whitening board.

Here is an example of the new open-loop transfer function with the whitening filters.

 Note :

before the measurement I increased the control gain by an arbitrary number to obtain gain of more than 1 around 1 Hz.

 Late entry for Monday morning June 4, 2012

Cables were stress relieved. Cable entry - exit ports enlarged. Air gaps were minimized.

ETMY optical table top was grounded to the ETMY chamber through 1 Mohms this morning. I  also strain releifed relieved a few cables that were pulling on components directly.

Just for a record. This is the latest picture of the ETMY optical bench.

I will upload this picture on the wiki after the wiki gets up.

I didn't notice it the other day when I was working on putting in the trans QPD, but do we need to switch the mirror mount for the first turning mirror of the IR trans beam, which the green transmits through to go into the cavity?  It seems like we've set ourselves up for potential clipping.

I think there should be a scientifically based aveluation of the ETMY enclosure so we can make the ETMX better.

Meanwhile I'm counting pieces to move on with the south end table cover.

The ETMY enclosure feedthrough - north is installed. The sealing material is hard to work with.

The upper empty blocks will be replaced by something soft to make changing cables easy.

ETMY optical table enclosure feedthrough- south is in. Now it is time to see how air tightness increases performance.

I doubt we'll see any effect until we carefully seal the holes. If there's 1 hole in your boat it still sinks.

Summary:

Given that ETMX looks to be in good shape and the optic and suspension tower are ready for vacuum and air bakes respectively, I set about re-gluing the knocked off magnet of ETMY. In my previous elog, I had identified the knocked off magnet as the UL magnet. But in fact, it was the LR magnet that broke off. This is actually one of the magnets that was knocked off when Johannes was removing the optic from the vacuum chamber. I have edited the old elog accordingly. 

Step 1: Removing epoxy residue

• I used the teflon+glass rig Steve put together for this purpose
• After soaking for ~2 hours in acetone, I was able to remove approximately half of the ring residue by lightly pushing with a wipe.
• The other half wouldn't budge so I let it soak for another 4 hours
• After 6 hours of soaking, I was able to get all of the epoxy residue off - it doesn't simply dissolve in the acetone, I had to push a little with one of the cotton-tipped paddles in the cleanroom
• I gave the portion exposed to acetone a quick drag wipe with isopropanol. I didn't spend too much time trying to clean the AR side given that we will be using first contact anyways.
• I have not touched the HR side for now, even though a small portion of it was exposed to acetone. While cleaning the HR face with first contact, this portion can be inspected and cleaned if necessary

Step 2: Putting the optic in the magnet gluing jig

• I transferred the optic to the magnet gluing jig
• Given that we weren't touching any side magnets, I reasoned I did not have to go through the elaborate shimming routine to account for the wedge of the optic that we had to do in the recent past
• However, I did not think to put a thicker teflon spacer on the lower side of the wedge, and as a result, I knocked off the UR magnet as well as the jig did not have sufficient clearance
• Fortunately, the UR magnet came off cleanly, there was hardly any epoxy residue left on the optic. The UR magnet was NOT one of the magnets knocked off by Johannes while removing the optic from the vacuum chamber
• I gave the area formerly occupied by the UL magnet 3-4 wipes with acetone and then 1-2 wipes with isopropanol
• At this stage, I proceeded to re-insert the magnet-gluing jig. I used the two scribe lines on the outer side of the jig to fix the rotation of the jig, and used the remaining two attached face magnets to fix the overall position of the jig (by centering these magnets relative to the apertures on the jig). In order to center well, I had to unscrew the stuck silver plated screw on the jig by 1 turn
• Having arranged the jig satisfactorily, I proceeded to remove epoxy residue off the dumbbell of the recently knocked off UL magnet using first a razor blade, then sandpaper and finally made some new grooves with a razor blade. I then cleaned the surface of the dumbbell to be in contact with the optic with isopropanol. All of this was done for the LR magnet two weeks ago right after it was knocked off

Step 3: Gluing the magnets

• I prepared the magnets in the pickle pickers
• I discarded 1 full squeeze of the epoxy after it reached the tip of the mixing fixture, and then extracted another full squeeze of the gun for mixing and gluing the magnets
• I mixed the epoxy in an Al foil vessel for 3-4 minutes, and then placed a few drops on a piece of Al foil for a test bake at 200F for ~15 minutes
• The test bake went well, so I proceeded to apply glue to the dumbbells and re-glue the magnets to the optic
• The gluing was done around midnight, so we should be able to have a look at this post lunch tomorrow.

Provided the gluing goes well, the plan for tomorrow is:

1. Bring ETMY suspension tower from the vacuum chamber to the cleanroom along with its OSEMs
2. Suspend ETMY with a new length of wire (this should be much more straightforward than our ETMX exploits as both standoffs are already glued)
3. Insert OSEMs, check that all 4 face magnets are well centered w.r.t. their coils and also that at least one side magnet is well aligned relative to its coil and can be used 
4. If step 3 goes well, then ETMY is also ready for a vacuum bake. I guess we can also air bake the ETMY suspension tower, there's plenty of room in the oven

The pickle pickers came off nicely and both magnets seem to be glued on okay. The alignment of the face magnets look pretty good, but we will only really know once we suspend the mirror, check the pitch balance, and put in the OSEM coils.

I brought the ETMY suspension tower + OSEM coils out of the vacuum chamber into the cleanroom. Given that the old wire had a pretty sharp kink in it, I removed it with the intention of suspending the optic with a new length of wire. I noticed a few potential problems:

Attachment #1 - ETMY tower is different from ETMX tower: 

• The ETMY suspension seems to be of an older generation - it does not have the the two secondary wire clamps. 
• The top piece was attached to the body of the tower using non-silver-plated screws. Steve tells me this is the wrong type, and we can switch these out when we put it back together.
• The wire clamp itself doesn't have much of a groove from the wire. But the wires have made asymmetric grooves in the tower itself (the left groove is deeper than the right as seen in Attachment #1), that are clearly visible. Should we get these grooves removed before attempting re-suspension? How do we want to remove it? Steve thinks the best option is to send it to the shop for milling, as there is hardly any room to rub sandpaper along the piece because of the pins, and these pins don't come out. 
• Or do we just not care about these grooves for now, if we are planning to use new wire anyways after air-baking the towers? 
• Steve thinks we should have a few spares of these top blocks handy (the latest version, with the secondary clamps), he wants to know if we should place an order for these (we already have 10 spare wire clamp pieces available for if/when we need them)

Attachment #2 - the base of the tower is significantly rusty:

• A few wipes with an acetone soaked rag yielded quite a lot of rust
• Steve thinks this is because the wrong type of stainless steel was used
• Does this have to do with the cage being of an older variety? After a few vigorous wipes, no more rust came off, but the rusting process will presumably keep generating new rust? Is this a concern? Do we want to change this piece before putting the tower back in?

I am holding off on attempting to re-suspend the optic for now, until we decide if the old wire grooves need to be removed or not. If we are okay with re-using the same piece as is, or if we are okay with using sandpaper and not the machine shop to remove the grooves, I will resume the re-suspension process. 

Eric suggested another alternative, which is to use the old ETMX tower. I don't recall it being rusted, but this has to be checked again. The other problem of the wire-grooves would possibly still be an issue.

Regarding the vacuum bake of the ETMs, Bob tells us that the best case scenario we are looking at is September.

[lydia, gautam]

Rana felt it was alright to use the wire clamp and suspension cage in its existing condition for checking the ETMY magnet-OSEM coil alignment. So we set about trying to re-suspend ETMY. The summary of our attempts:

• Transferred optic from magnet gluing rig to the suspension cage
• Adjusted bottom EQ stops till the scribe lines on both sides were at 5.5" as verified with the microscope
• Looped cleaned length of wire around optic, attached free ends to winches, placed the wires under light tension by finger-pulling the slack out
• Lowered the bottom EQ stops
• Winched the optic to the right height
• Clamped the wire with the only wire clamp on this variant of the suspension cage. We used the same torque wrench at the same torque setting as was successful for ETMX. But after removing the winches, and releasing the face EQ stops, the optic seems to have sagged a lot - it now touches all the bottom EQ stops, and the more I lower it, the more it seems to come down. Perhaps it is the effect of the wire grooves in the cage, or that the wire-clamp itself is slightly different from the piece used on the ETMX cage, but 1.3Nm of torque doesn't seem to have tightened the wire clamp sufficiently
• We can still probably salvage the situation by re-attaching the winches to the top of the cage, setting the optic to the right height again, and clamping the wire clamp with more torque (as this is just a check to see that the reglued magnet configuration is compatible with the OSEM coil positions on the cage). Before air baking the cage, we will have the old wire grooves removed, and then suspend the optic with a fresh loop of wire after the bake
• We could not check the magnet-OSEM alignment because of the slipping of the wire through the clamp. We decided against pushing on tonight
• Optic is currently in the cage, resting on the bottom EQ stops and with all face EQ stops within 1mm of the optic. The OSEM coils have not been inserted into the holders

Regarding the vacuum bake of the optics: why do we want to do this again? Koji mentioned that the EP30-2 curing process does not require a bake, and there is also no mention of requiring a vacuum bake in the EP30-2 gluing guide. Is there any other reason for us to vacuum bake the optic?

Summary:

• ETMY has been re-suspended
• Reglued magnets (and also those that weren't knocked off) quite well with OSEM coils (see attachments)
• Pitch balance is off by ~2.8mrad (8mm over 1.5m lever arm) after inserting and centering OSEMs
• The same damping scheme used during the ETMX re-suspension process works reasonably well with ETMY as well

Details:

• I suspected that I had not tightened the wire clamp enough yesterday, and that the wire had slipped once the winches were removed
• Steve and I looked into the torque wrench situation today, and I realised that I had not been using the torque wrench correctly. What I thought were clicks indicating that the set torque has been reached was in fact just the sound the piece makes when going the opposite way relative to the direction set by the clip on the torque wrench. Anyways, the point is that while I thought I was tightening the screws with ~1.3Nm of torque, what was actually being applied was much less (although I don't have a good way to quantify how much less)
• So today I put the winches back on top of the tower, and winched the optic back up to the correct height using the ususal scribe line + microscope prescription
• I then tightened the wire clamp by hand. This is obviously not very repeatable, but it will have to do until we get a torque wrench with the correct range
• This seems to have done the trick - I did the tightening shortly after lunch, and after ~10 hours, there is no evidence of any wire sag
• I then proceeded to insert the OSEMs, first not all the way in to check the clearance available to the magnet, and once I was satisfied there was no danger of knocking anything off, went ahead and inserted the coils till the PD readouts were approximately half of the maximum (i.e. fully un-occluded) values. I used the OSEM coils originally on the ETMY tower, but all the other readout and drive electronics in the signal chain (satellite box included) belong to the ETMX setup (so as to avoid any cable routing over 80m from the Y end to the cleanroom). After some adjustment of the OSEM holding plates, I was able to center the magnets relative to the coils 
• The tower only allows for a side OSEM to be inserted on one side. The other side does not have a threaded hole for a set screw. So we are forced to use the reglued magnet and not the side magnet that was not knocked off. By eye, it looks like the magnet may never completely occlude the LED, but the Striptool trace I was using to monitor the output of the PD did not yield any conclusive evidence. The optic was moving around a lot and I did not perform this check after turning the damping on
• I was able to damp the optic as well as we were able to damp ETMX on the clean bench (with the HEPA turned OFF). I had to turn the YAW gain down from 100-->75 to avoid some oscillations 
• I then proceeded to check the pitch balance with the HeNe. The spot is low on an iris 1.43m away by ~8mm, which corresponds to a pitch misalignment of ~2.8mrad. I am not sure what to make of this - but perhaps its not unreasonable that we see this? Is there any record of what fine pitch balancing was achieved when the optic was put together back in 2010? This is also very sensitive to how far in/out the OSEM coils are, and though I've tried to center the coils as best as I can, I obviously have not done a perfect job...

What's next?

• Is the observed pitch imbalance a deal breaker? If so, I guess we need to re-glue a standoff? 
• Are we willing to accept the side OSEM situation? (Tomorrow, I need to do a check to see what, if any, dynamic range we lose, with the damping on)
• If both the above are not problems we need to worry about, then:
  • ETMY + ETMX -------> Vacuum bake on 22nd August (? - Bob also told me earlier today that he will try and put in some old turbo pump next week, and if that works, we could possibly get in the queue even before the 22nd)
  • ETMY tower -------> Steve for sanding and removing wire grooves -------> Air bake
  • ETMX tower -------> Air bake (provided the latest round of wire tightening has not left any grooves in the top piece of the tower, if it has, this needs to be cleaned up too)
  • Some lengths of SOS wire (for re-suspending optics after bake) -------> Air bake

Attachments:

Attachment #1: Striptool trace showing all OSEM coils have been pushed in till the PD readout is approximately half the fully open value

Attachment #2: Pitch balance is off by ~2.8mrad (the Iris center is 5.5" above the table)

Attachment #3: UR magnet

Attachment #4: UL magnet

Attachment #5: LR magnet

Attachment #6: LR magnet

Attachment #7: SD magnet

How much pitch bias do you need in order to correct this pitch misalignment?
That may give you the idea how bad this misalignment is.

I needed to move the pitch slider on the IFO align screen to -2.10 (V?) from 0 to get the HeNe spot to the center of the iris. The slider runs from -10V to 10V, so this is something like 10% of its range. I am not sure if it means anything, but the last saved backup value of this pitch slider was -3.70. Of course, application of the bias will affect all the coils, and when the optic is pitch balanced, the lower magnets are a little too far out and the upper magnets are a little too far in (see Attachment #1), as we expect for a downward pitch misalignment to be corrected. I suppose we can iteratively play with the coil positions and the bias such that the coils are centered and we are well balanced (maybe this explains the old value of -3.70). 

I also checked that the side magnet can completely occlude its PD. With the damping on, by pushing the coil all the way in, the output of the side PD went down to 0.

[Koji, Jenne]

I wish I could use a bigger font for this, but, the suspension work is totally done for the upgrade!!!

Now, nobody break any suspensions, or we're not going to be friends for a while. 

Koji and I put ETMY back in its tower, and made sure that both scribe lines are at the correct height.  We also confirmed that the balance is good (as Suresh mentioned in a previous elog, since we balanced using the AR surface, the HR surface is pointing downward a little bit, but it's well within the OSEMs ability to correct.

While we were in there, we also looked at Tip Tilt number 002.  As mentioned in elog 3645, the pitch pointing was off by a little bit.  Since the TTs don't have actuators, the pointing has to be pretty good.  We tweaked the balancing, and now the reflected beam goes completely back into the laser aperture, so it's as balanced as it's going to get.  This TT is now ready for installation onto the ITMY table as part of the SRC.

Kiwamu confirmed that he's going to install these optics tomorrow, since he's doing some other alignment work today.

Just for good measure, the Table:

Here are the results from this test. The data for 17 April is with the DC bias for ETMY set to the nominal values (which gives good Y arm cavity alignment), while on 18 April, I changed the bias values until all four shadow sensors reported values that were at least 100 cts different from 17 April. The times are indicated in the plot titles in case anyone wants to pull the data (I'll point to the directory where they are downloaded and stored later).

There are 3 visible peaks. There was negligible shift in position (<5 mHz)  / change in Q of any of these with the applied Bias voltage. I didn't attempt to do any fitting as it was not possible to determine which peak corresponds to which DoF by looking at the complex TFs between coils (at each peak, different combinations of 3 OSEMs have the same phase, while the fourth has ~180 deg phase lead/lag). FTR, the wiki leads me to expect the following locations for the various DoFs, and I've included the closest peak in the current measured data in parentheses:

However, this particular SOS was re-suspended in 2016, and this elog reports substantially different peak positions, in particular, for the YAW DoF (there were still 4). The Qs of the peaks from last week's measurements are in the range 250-350.

 ETMY sensors are glitching or getting kicked up.

Atm3, There is no seismic activity here.

  This is not really definitive. The 0.1-0.3 Hz band is not the right one to look for seismic transients - it should be the higher frequency ones.

The other test to do is to turn off the ETMY damping and then look for glitching in the sensors. And then, of course, check to see that no one has bumped the satellite box with a cart or a mop...

I noticed by eye that during one event when ETMY was getting kicked up, its CPU meter (C1:FEC-47_CPU_METER) went RED.

Thinking that this might be a clue I tried to trend this channel. Even though this channel is in the SCY EDCU file and the 'rtcds install' command claims to be 'installing C1EDCU_SCY', many of the channels named in the file are not actually showing up in ur dataviewer SLOW channels list.

I smell a cockroach in our RCG build process, but I can't find the log file for the make-install part of the build nor can I find the Makefile from which the make-install is born. Help us Jamie!

I have deleted a few filters from c1scy to see if that could reduce the CPU time and I have killed the c1tst process to see if that can cool down the entire computer. Next, we can try to open the rack doors and put a fan on there to see if we can shave a couple microseconds. I have a StripTool running on pianosa to see if we see some correlations between FEC47 and the ETMY SUS watchdog RMSs. Don't close it.

c1scy has been running slow (compared to c1scx, which does basically the exact same thing *) for many moons now.  We've looked at it but never been able to identify a reason why it should run slower.  I suspect there may be some bios setting that's problematic.

The RCG build process is totally convoluted, and really bad at reporting errors.  In fact, you need to be careful because the errors it does print are frequently totally misleading.  You have to look at the error logs for the full story.  The rtcds utility is ultimately just executing the "standard" build instructions.  The build directory is:

    /opt/rtcds/caltech/c1/rtbuild

The build/error logs are:

    <model>.log     <model>_error.log 

I'll add a command to rtcds to view the last logs.

(*) the phrase "basically the exact same thing" is LIGO code for "empirically not at all the same"

 While that would be good - it doesn't address the EDCU problem at hand. After some verbal emailing, Jamie and I find that the master file in target/fb/ actually doesn't point to any of the EDCU files created by any of the FE machines. It is only using the C0EDCU.ini as well as the *_SLOW.ini files that were last edited in 2011 !!!

So....we have not been adding SLOW channels via the RCG build process for a couple years. Tomorrow morning, Jamie will edit the master file and fix this unless I get to it tonight. There a bunch of old .ini files in the daq/ dir that can be deleted too.

We have now watched the ETMY computer situation for a little over 150 minutes, and have seen one 'event' where the CPU time of the scy model hit 62 microseconds, and a glitch in the ETMY OSEM sensors happened at the same time.  We also see no such glitches at any other time, which makes sense with our latest hypothesis, since this event was the only time that the CPU time reported being greater than 61 microseconds.  (1/16384 Hz = 61.1696 microseconds). 

I have now restarted the c1tst model, to see if that increases the rate of glitches (assuming that running another model heats up the whole computer a bit more, and that makes things run a little bit slower).

Wed Oct 23 21:05:28 2013 

RXA: It looks like there was a real effect. Its between -2.5 and 0 on the plot below.

I've stopped the process of c1tst again to make it get better. At 9:20, I also went and opened the front rack door (the back one was already open). One reason its hot may be that the exhaust vents on the top of c1iscey are blocked by one of the custom multi-pin adaptor boxes. In the morning, we should drop the computer down by 1 or 2 notches in the rack so that it can air cool itself better. Make sure to poweroff the computer from the terminal before moving it though.

I checked the cabling somewhat. The fat grey cable which comes out of the old Sander Liu AA chassis was connected to the blue adaptor box but the strain relief screws were not being used. I tightened them (we need to buy a set of small screwdrivers for the toolboxes at each end). While doing this, the Cat6 cable in the back labeled 'c1iscey' popped out and the screen went white. This cable has a broken latch on it so it doesn't stay put - needs to be replaced too during the computer move.

Atm1,  The strong glitches are back.

Atm2,  SUS-ETMX & Y_SENSOR_LL Damping OFF at (ref0 & ref1). damping ON (red & blue)

ETMY sus looks OK