After running into more problems with the upgrade, I eventually decided to abort todays upgrade attempt, and revert back to where we were this morning (RTS 2.1).  I'll try to follow this with a fuller report explaining what problems I encountered when attempting the upgrade.

However, when Alex and I were trying to figure out what was going wrong in the upgrade, it appears that the fb symmetricom card lost the ability to sync with the GPS receiver.  When the symmeticom module is loaded, dmesg shows the following:

[  285.591880] Symmetricom GPS card on bus 6; device 0
[  285.591887] PIC BASE 2 address = fc1ff800
[  285.591924] Remapped 0x17e2800
[  285.591932] Current time 947125171s 94264us 800ns 
[  285.591940] Current time 947125171s 94272us 600ns 
[  285.591947] Current time 947125171s 94280us 200ns 
[  285.591955] Current time 947125171s 94287us 700ns 
[  285.591963] Current time 947125171s 94295us 800ns 
[  285.591970] Current time 947125171s 94303us 300ns 
[  285.591978] Current time 947125171s 94310us 800ns 
[  285.591985] Current time 947125171s 94318us 300ns 
[  285.591993] Current time 947125171s 94325us 800ns 
[  285.592001] Current time 947125171s 94333us 900ns 
[  285.592005] Flywheeling, unlocked...

Because of this, the daqd doesn't get the proper timing signal, and consequently is out of sync with the timing from the models.

It's completely unclear what caused this to happen.  The card seemed to be working all day today, then Alex and I were trying to debug some other(maybe?) timing issues and the symmetricom card all of a sudden stopped syncing to the GPS.  We tried rebooting the frame builder and even tried pulling all the power to the machine, but it never came back up.  We checked the GPS signal itself and to the extend that we know what that signal is supposed to look like it looked ok.

I speculate that this is also the cause of the problems were were seeing earlier in the week.  Maybe the symmetricom card has just been acting flaky, and something we did pushed it over the edge.

Anyway, we will try to replace it tomorrow, but Alex is skeptical that we have a replacement of this same card.  There may be a newer Spectracom card we can use, but there may be problems using it on the old sun hardware that the fb is currently running on.  We'll see.

In the mean time, the daqd is running rogue, off of it's own timing.  Surprisingly all of the models are currently showing 0x0 status, which means no problems.  It doesn't seem to be recording any data, though.  Hopefully we'll get it all sorted out tomorrow.

RTS/RCG/DAQ UPGRADE TO COMMENCE

I will be attempting (again) to upgrade the RTS, including the RCG and the daqd, to version 2.4 today.  The RTS will be offline until further notice.

Dataviewer is recovered after heavy new year party.

Communication between the front end models and the framebuilder has been restored.  I'm not sure exactly what the issue was, but rebuilding the framebuilder daqd executable and restarting seems to have fixed the issue.

I suspect that the problem might have had to do with how I left things after the last attempt to upgrade to RCG 2.4.  Maybe the daqd that was running was linked against some library that I accidentally moved after starting the daqd process.  It would have kept running fine as was, but if the process died and was attempted to be started again, it's broken linking might have kept it from running correctly.  I don't have any other explanation.

It turns out this was not (best I can tell) related to the new year time sync issues that wer seen at the sites.

The dynamic power normalization system has been modified such that the normalization happen after the LSC input matrix.

Remember that we don't use /cvs/cds/rtcds anymore.  Everything should be in /opt/rtcds now.

For systems using the Spectracom IRIG-B cards for timing information, the code did not properly roll over the time for
2012 (still thinks it is 2011 and get reports from DAQ of timing errors (0x4000)). I have made a temporary fix for this
in the controller.c code in branch-2.3, branch-2.4 and release 2.3.1. 

 The problem is the same as yesterday.

       Locking plan            

Here is a plan in my mind and these are basically the details of the gantt chart (#6143):

• (1 day task) Measurement of the recycling gains of the RF sidebands with the PRMI and DRMI configuration, using POP22/110 RFPD.
  • I need to have confidence that I am really locking the DRMI with SRC resonating to 55 MHz.
  • Also those values will enable us to estimate losses and mode matching again (maybe ?).
• (3-4 days task) Measurement of the sensing matrix using the multiple-LOCKIN system.
  • Write a script to automatically measure the sensing matrix. This must be easy.
    • The results will enable us to diagonalize the input matrix and therefore it eventually gives more solid lock of the DRMI
    • Also it will give us the optical gains of 3f signals. So this is actually a step toward the 3f signal check.
• (3-4 days task) Noise budgeting on the 3f signals
  • This is a very important part of the DRMI characterization because the results will tell us whether we can hold the DRMI lock with a sufficient SNR or not.
  • If it turns out that they don't have good SNRs, we then have to come up with some ideas to improve the SNRs.
• (Extra fun task depending on schedule) 3f DRMI lock + Y arm ALS
  • If the beat-box electronics are not available by the time when the work above are completed, I will do this fun task.
  • Probably it is better to start preparing the common mode servo electronics because it will be needed anyway.

 Time to lower your expectations!

Do we really need 40 microns at 500 Hz? Or perhaps should there be a frequency dependent displacement requirement?

I don't know what exactly is going on, but MC became flaky and it's been frequently unlocked.

I have turned off the MC WFS servo to check if the WFSs are doing something bad. But it still tends to be unlocked without the WFS servo.

Right now it doesn't stay locked for more than 10 min.

 I checked on the two single-axis shakers that are present at the 40m that Steve pointed out:

• Brüel & Kjær type 4809, rated for 45 N peak, and
• Brüel & Kjær type 4810, rated for 10 N peak.

Neither of these meet the force requirement of 2.04 kN peak.

 I moved the following old and obsolete MEDM directories to an archive /cvs/cds/rtcds/caltech/c1/medm

• c1vga
• c1vgl
• c1gpt
• c1gpv
• c1nio
• c1spy
• c1spx

None of these models were present in /cvs/cds/rtcds/caltech/c1/target

I updated the design requirements for the Stewart platform.  I weighed the unloaded "dirty" SOS that was sitting on the workbench in the control room; its mass is 11 kg.  Steve suggested that the OSEMs (not installed on this model) would add another 0.5 kg.  From the specs in the final SOS design document, LIGO-T970135, I added 0.25 kg for the optic itself; I am therefore taking the total payload mass to be 11.75 kg.  (Now, the upper stage of the Stewart platform itself will likely add a nontrivial amount, but I am not worrying about this yet.)

I have e-mailed Janeen Romie to obtain the actual center of mass and principal moments of inertia of the platform.  I also cooked up a simple scheme to measure both quantities, should this information not be available.  It would involve rigidly mounting the dirty SOS to a rigid bar hung from a pivot.  By translating the mount point in two dimensions and measuring the period of the pendulum, I ought to be able to find the center of mass and moments of inertia by multilinear regression.  However, this elaborate scheme is not necessary to just compute some ballpark figures; it could wait until a later stage in the design.  For the time being, I just rescaled the moment of inertia proportional to the increase in mass, such that the torque-to-force ratio is unchanged.

As such, the design requirements are now 

• linear travel: 40 microns peak to peak (based on SOS design requirements in LIGO-T950011)
• angular travel: 3 mrad peak to peak (based on SOS design requirements in LIGO-T950011)
• payload mass: 11.75 kg (measured unloaded mass, plus educated guesses about combined mass of OSEMs and optic)
• payload moment of inertia: 0.0232 kg m^2 (wild guess)
• bandwidth: 500 Hz (suggestion of Rana and Koji: ~kHz)

From these assumptions, the revised actuator requirements and dimensions are:

• peak actuator force: 2.04 kN
• minimum radius of top platform: 15 cm
• minimum radius of bottom platform: 30 cm
• minimum height: 26 cm

See the attached PDF document.

It appears that the actuator that I had originally nominated, PI's model P-225.80, would very nearly meet the actuator force requirement.  Steve also pointed out the following single-axis shakers that are already in use in the 40m:

• Brüel & Kjær type 4809
• Brüel & Kjær type 4810

I want to find out if either of these would meet the present need, but I'm waiting on a response from the manufacturer to get access to the data sheets.

I found the PSL laser has been off for four hours. Nobody seemed to know why.

I just turned it on and it is now providing about 10% lower power compared with one before the shutdown.
Let's keep the eyes on the power if it can recover as the housing gets warm.

Apparently there is a bug in the timing cards having to do with the new year roll-over that is causing front-end problems.  From Rolf:

For systems using the Spectracom IRIG-B cards for timing information, the code did not properly roll over the time for
2012 (still thinks it is 2011 and get reports from DAQ of timing errors (0x4000)). I have made a temporary fix for this
in the controller.c code in branch-2.3, branch-2.4 and release 2.3.1. 

I was going to check to see if the 40m is suffering from this. I'll be over to see if that's the problem.

It turned out that the power normalization need a modification.

I will work on it tomorrow and it will take approximately 2 hours to finish the modification.

     Concept of Power Normalization         

1.  Static power normalization, which should be placed before the input matrix.
2.  Dynamic power normalization, which should be placed after the input matrix.

Is there a reason the framebuilder status light is red for all the front ends?

Also, I reenabled PRM watchdog.

Now a power normalization is doable for the LSC error signals.

It is working fine, but at some point we may want to have some kind of a saturation filter or limiter to avoid dividing a signal by a small number.

 (How to set the normalization)

•   Click a small matrix panel on the LSC OVERVIEW window (shown in the attached screen shot below).
  •     This will give you a pop-up-window, which shows a matrix to route the normalization signals

•   Choose a numerator channel, which you want to divide, and choose denominator channels, which you want to use as a power normalization factor.
•   Put some number in the corresponding matrix elements.
•   Once you put a non-zero element in the matrix, the corresponding numerator channel will be divided by the specified denominator channels.
  •     Otherwise the static normalization factors (e.g. C1:LSC-AS55_POW_NORM, etc.,) will be used for the denominator.

The high frequency noise, which has been a dominant noise above 30 Hz in the Y arm ALS (#6133), decreased by a factor of 5.

This reduction was done by increasing the modulation depth at the Y end PDH locking. Now the noise floor at 100 Hz went to 0.2 pm/sqrtHz.

However the noise source is not yet identified and hence it needs a further investigation.

 (Increasing the modulation depth)

I added an ALS feedback path on the MC2 suspension and this path will enable us to stabilise the MC length using the ALS scheme.

I have realigned the steering mirrors for PMC because the transmitted light had been at ~ 0.741

After the alignment it went back to ~ 0.850.

Some new screens have been made for the new multiple-LOCKIN system running on the LSC realtime controller.

The medm screens are not so pretty because I didn't spend so long time for it, but it is fine for doing some actual measurements with those new screens.

So the basic works for installing the multiple-LOCKIN are done.

 The attached figure is a screen shot of the LOCKIN overview window.

As usual most of the components shown in the screen are clickable and one can go to deeper levels by clicking them. 

      An example              

      How to set the thresholds         

1. Open the trigger matrix window, which is accessible from the C1LSC overview screen as usual.
2. Then type the desired upper and lower thresholds into the column.

The below is a screenshot of the trigger matrix screen. The thresholds column is pointed by a big white arrow.

 

Of course, DO NOT set the upper threshold value to be smaller than that of the lower threshold. Otherwise it won't correctly work.

Also if you want to have the usual trigger rather than the Schmitt trigger, simply put the upper and lower thresholds at the same values.

      Details         

(Some details)

The picture below is a screen shot of the LSC real time model, zoomed in the new LOCKIN part.

The LOCKIN module consists of three big components:

1. A Master oscillator
   • This shakes a desired DOF through the LSC output matrix and provides each demodulator with sine and cosine local oscillator signals.
   • This part is shown in the upper side of the screen shot.
   • The sine and cosine local oscillator signals appear as red and blue tags respectively in the screen shot.
2. An input matrix
   • To allow us to select the signals that we want to demodulate.
   • This is shown in the left hand side of the screen shot.
3. Demodulators
   • These demodulators demodulate the LSC sensor signals by the sine and cosine signals provided from the master oscillator.
   • With the input matrix fully diagonalized, one can demodulate all the LSC signals at once.
   • The number of demodulators is 27, which corresponds to that of available LSC error signals (e.g. AS55_I, AS55_Q, and etc.).
   • This part is shown in the middle of the screen shot.

I have edited c1scx.ini by hand in order to acquire some green locking related channels.

Somehow c1sus.ini, c1mcs.ini, c1scx.ini and c1scy.ini are not accessible via the daqconfig script.

As far as I remember it had been accessible via daqconfig a week ago when I edited c1scy.ini.

Anyway I had to edit it by hand. They need to be fixed at some point

Often people say "I don't use conlog because its real slow". Its a little like not driving because your car has no gas.

I looked into what's going on with conlog. No one has fixed its channel list in ~1 year so it didn't make much sense. Also since Oct 1 of this year, it expired the leap seconds epoch and has been waiting for someone to look at the log file and update the list of leap seconds.

Some issues:

• Don't use the phrases like OUTPUT, OUTMON, OUT16, or INMON as the usual part of a channel name. These are filter modules words which use to exclude channels from conlog. Please fix ALL of the LOCKIN screens to get rid of the OUTPUT filter banks.
• If you use an EPICS channel in a servo so that its getting changed 16 times a second, make sure to add it to the conlog exclude list.

There are a bunch of bad channels which are screwing up various tools (DV, DTT, etc.):

Examples: C1:LSC-Subsystem_NPRO_SW1, C1:-DOF2PD_MTRX_0_0_SW1, C1:BAD-BAR_CRAZY_2_RSET, C1:C1L-DOF2PD_MTRX_3_14_SW2, C1:DUB-SEIS_GUR2_Z_LIMIT, etc.

• There are a bunch of old, unused directories in c1/medm/. EVERYONE take a look in there and delete the OLD dead ones so that we don't keep recording those channels.

 To fix up some of these issues, I have deleted several MEDM directories which I thought were old (there are several extras left from Aidan's Green time). I also have put a bunch of exclude variables into the conlog 'scan_adls' script to prevent it from adding some of the new worthless channels. Finally, I have started this command

../bin/strip_out_channels '.*STAT.*','.*_ALIVE.*','C1:PEM.*','.*_Name.*','C1:UCT.*','C1:MCP.*','C1:SP.*','C1:DU.*','C1:RF.*','C1:NIO.*','C1:TST.*','C1:SUP.*','C1:X.*','C1:FEC.*','.*_LFSERVO.*','.*FSS_SLOWDC.*','C1:LSC-LA_MTRX_21','C1:LSC-PD.*OFFSET','C1:LSC-ETM.*OFFSET' conlog*.log

which should strip lots of the excess conlog data out of the conlog directory. The only downside is that its setting all of the timestamps of the .log files to today instead of the historical times but I don't think we'll care about this too much. Hopefully it will speed things up to have less than 450 GB of conlog files...

update: 12 hours later, its still running and has removed ~100 GB so far. It will probably take the rest of the weekend to finish.

After I did a fine alignment of the X green beam path on the PSL table, the X arm beat-note was also obtained.

Here is a picture of the latest setup. The blue lines represent S-polarizing green beams.

During I was working on the PSL table HEPA was at 80 %, and after the work I brought it to 20 %.

I set the limitters of the MC WFS servos not to inject the feedback more than 2000.

The residual of the WFS shakes the MC SUSs at every lock loss.

         Assumptions on the parameter estimations          

Here is the Gantt chart we discussed in the 40m meeting today.

Based on the discussions we had, I applied a little bit of corrections on the chart but the main stream remains the same.

While discussing the suspension hysteresis measurements, Koji, Kiwamu, and I realized that the suspension wire standoff is aluminum, whereas the standoff for the LIGO LOS are using quartz.

Using a soft aluminum standoff is bad. The movement of the suspension will slowly wear the groove and produce opportunities for mechanical upconversion and hysteresis.

In fact, the wire standoff as well as the clamping block on the top should be made of sapphire or ruby to prevent any such wearing issues. Steve is hot on the case.

I made the first trial of locking a Power-recycled single arm.

 This is NOT a work in the main stream,

but it gives us some prospects towards the full lock and perhaps some useful thoughts.

      Optical Configuration         

• Y arm and PRM aligned. They become a three-mirror coupled optical cavity
  • Power Recycling Cavity (PRC) is kept at anti-resonance for the carrier when the arm length is off from the resonance point
  • Hence bringing the arm length to the resonance point lets the carrier resonate in the coupled cavity
  • BS behaves as a loss term in PRC and hence results a low recycling gain
• Everything else are misaligned, including ITMX, ETMX, SRM and BS
  • Therefore there are neither Michelson, X arm nor Signal Recycling Cavity (SRC)

   Lock Acquisition Steps    

1. Misalign PRM such that there is only Y arm flashing at 1064 nm
2. Do ALS and bring the arm length to the resonance point
3. Record the beat-note frequency such that we can go back to this resonance point later
4. Displace the arm length by 13 nm, corresponding to a frequency shift of 200 kHz in the green beat note
5. Restore the alignment of PRM.
6. Lock PRC to the carrier anti-resonance condition using REFL33I. At this point the arm doesn't disturb the lock because it is off from the resonance anyway
7. Reduce the displacement in the arm and bring it back to the resonance

     Actual Time Series     

Below is a plot of the actual lock acquisition sequence in time series.

• The data starts from the time when the arm length was kept at the resonance point by the  ALS servo.
  • At this point PRM was still misaligned.
• At 120 sec, the arm length started to be displaced off from the resonance point.
• At 250 sec, the alignment of PRM was restored and the normalized DC reflection went to 1.
  • Error signals of PRC showed up in both REFL33 and POOY11
• At 260 sec, PRC was locked to the carrier anti-resonance point using the REFL33_I signal.
  • Both REFL33 and POY11 became quiet.
  • REFLDC started staying at 1, because the carrier doesn't enter to the cavities and directly goes back to the REFL port.
• At 300 sec, the arm length started to be brought to the resonance point.
• At 400 sec, the arm length got back to the resonance point.
  • The intracavity power went to 3.5 or so
  • REFLDC went down a bit because some part of the light started entering in the cavities
  • REFL33 became noisier possibly because the Y arm length error signal leaked to it.

Scripting of the single arm automated lock script is 80% done.

The remaining 20 % is not something immediately needed and I start decreasing the priority on the Y arm ALS.

(Remaining stuff)

• Automated optimization of I/Q phases at the frequency discriminator's signal.
  • this part will be done after we install Jamie's new beat box
• A routine function which checks if the beat note is within a reasonable bandwidth
  • This part can be done with the frequency-divided signal and the digital delay line frequency discriminator
  • Another approach is to install a frequency counter, which doesn't have to be so precise
• A state bit which tells us how far the script goes
• An exit handler.
  • This should run whenever the script is unexpectedly force-quite, to gently bring the ALS system down.
• A servo which brings the beat frequency to exactly a point where the infrared light is on a resonance point
  • Currently this part is partially human-aided. I put a little bit of correction in the frequency offset by looking at time series
  • To automate this part, we need another LOCKIN system to shake the arm length and demodulates the transmitted light

3/4 " thick colored acrylic material will be used in this air tight design. Surgical tubing for o-ring. We may have to put an o-ring into the bottom to have it really air tight.

Feedtrouhs: www.roxtec.com

The top drawing is not ready. It will have handle and industrial grade L-handle lock pin to hold cover down. There will be 2  one inch od post in the midle of the table to hold the cover and lock  the ball pin.

I'm waiting for your inputs, so I can send this preliminary design out for quote.

I have been looking at the swings in the RAMmon channels since the heater was reengaged, to compare them to the data from beforehand (with and without the foam box). With the large grains of salt that I will list after, it appears that the EOM temperature controller does in fact reduce the amplitude of the swings by a measurable factor.

Salt:

1. The reason I have not included any plots here is because the data suck. What we should ideally have is a continuous stream of RAMmon signals split into three chunks: 1) no foam, no heat, 2) foam, no heat, and 3) foam and heat. Instead, we have pieces of each kind of data on different days, before and after the MC has been realigned, some in old channels and some in new so that the calibration is different, etc. This piecemeal shit will not do.
2. I realized that the LF boost was not engaged on the heater when I turned it on most recently. For this reason, the EOM temperature has not been stabilized as well as it might have been on diurnal timescales, and so with the boost it could be that the noise reduction is greater. For posterity, the DC suppression level is ~20x without the boost.

It seems impractical to try and rope off essentially 3 straight days where nothign major can be done to the IFO just to take RAM data. Instead, I think we should figure out a way to mimic the diurnal temperature swings on ~hour timescales. The EOM can temperature follows PSL-FSS_RMTEMP almost exactly and with a very short delay, so we can probably even accomplish this by stepping the lab A/C temperature. If this won't work, we can use an incandescent lamp or something similar to heat up the area around the EOM by a noticeable amount.

I'll try to come up with a good way to do this so that we can get some reliable data...

Online filter diverges. I did offline simulations with current c-code. Offline filter also diverges, even in the simplest case 

witness = randn(1e6, 1); target = witness + 0.01*randn(1e6, 1);

I tried to create a new implementation of FXLMS algorithm as a c code. Then with this c code I did offline filtering with MCL and GUR signals and compared the error signals depending on the length of the filter.

One can see the code at the svn

adaptOnline - start here and choose algorithm

adaptive_filtering - Matlab implementation of AF

current_version.c - current version of the Filter (Matt's)

fxlms_filter.c - new version of the FXLMS filter

oaf.c - agent between Matlab and C (edited Matt's file)

Data samples can be found at nodus /users/den/wiener_filtering/data

Another hysteresis test has begun at 1:50 PT, Dec/19.

It will finish after 3 or 4 hours. During the measurement the PSL mechanical shutter will be kept closed.

I have recentered the oplev beams, including BS, ITMs and ETMs.

The measurement finished at ~ 21:50 PT.

Here are the latest plots that I have obtained from the Friday night:

    Time Series   

The residual motion in the arm displacements reached 70 pm in rms.

Koji has modified the script for the hysteresis measurement.

A new test started from 16:05 PT, Dec 18th and takes a couple of hours to finish the measurement.

Do not touch the suspensions until further notice.

The test was from: 2011-12-17 09:48 to 11:49 (UTC).
This corresponds to the period from 2011-12-17 01:48 to 3:49 (PST).

ZK: Thanks

Do you guys have timestamps for when you started/ended the test? I have been trying to take some long-term RAM data but keep getting foiled by stuff (this test, RTS upgrade, switching of RAMmon channels, etc.)

The hysteresis test has been aborted.

All of the suspensions have accumulated unexpectedly big DC biases of about 5 from their nominal points.

[Koji Kiwamu]

We wonder if the flakiness of MC2 comes from the suspension or not.

To test it, we are shaking all of the suspension biases +/-1.0 with a script.

The script is here:
/users/koji/111216/SUS_hysteresis.sh

For this test, we closed the mechanical shutter before the MC.

Also some amount of misalignment is anticipated.
Don't be surprised if you see nothing is working when you come to the lab in the weekend.

Quote from #6126

As shown in the noise budget below, the 60 Hz line noise currently dominates the arm displacement.

 The 60 Hz line noise has gone away.

As shown in the noise budget below, the 60 Hz line noise currently dominates the arm displacement.

       About Noise Budget       

Unfortunately, after working on it all day, I had to abort the upgrade and revert the system back to yesterday's state.

I think I got most of the upgrade working, but for some reason I could never get the new models to talk to the framebuilder.  Unfortunately, since the upgrade procedure isn't document anywhere, it was really a fly by the seat of my pants thing.  I got some help from Joe, which got me through one road block, but I ultimately got stumped.

I'll try to post a longer log later about what exactly I went through.

In any event, the system is back to the state is was in yesterday, and everything seems to be working.