Megatron's top fan, rear ps, and temperature front panel lights were all lit amber this morning.  I checked the service manual, found at :

http://docs.sun.com/app/docs/prod/sf.x4600m2?l=en&a=view

According to the manual, this means a front fan failed, a voltage event occured, and we hit a high temperature threshold.  However, there were no failure light on any of the individual front fans (which should have been the case given the front panel fan light).  The lights remained on after I shutdown megatron.  After unplugging, waiting 30 seconds, and replugging the power cords in, the lights went off and stayed off.  Megatron seems to come up fine.

I unplugged the IO chassis from megatron, rebooted, and tried to start Peter's plant model.  However, it still prints that its starting, but really doesn't.  One thing I forgot to mention in the previous elog on the matter, is that on the local monitor it prints "shm_open(): No such file or directory" every time we try to start one of these programs.

I called Alex this morning and explained the problems with megatron.

Turns out when he had been setting up megatron, he thought a startup script file, rc.local was missing in the /etc directory.  So he created it.  However, the rc.local file in the /etc directory is normally just a link to the /etc/rc.d/rc.local file.  So on startup (basically when we rebooted the machine yesterday), it was running an incorrect startup script file.  The real rc.local includes line:

/usr/bin/setup_shmem.rtl mdp mdc&

Hence the errors we were getting with shm_open().  We changed the file into a soft link, and resourced the rc.local script and mdp started right up.  So we're back to where we were 2 nights ago (although we do have an RFM card in hand).

Update:  The tst module wouldn't start, but after talking to Alex again, it seems that I need to add the module tst to the /usr/bin/setup_shmem.rtl mdp mdc& line in order for it to have a shared memory location setup for it.  I have edited the file (/etc/rc.d/rc.local), adding tst at the end of the line.  On reboot and running starttst, the code actually loads, although for the moment, I'm still getting blank white blocks on the medm screens.

I've placed some notes pertaining to what Koji and I have learned today about getting the RCG code working on the 40m wiki at:

http://lhocds.ligo-wa.caltech.edu:8000/40m/Notes_on_getting_the_CDS_Realtime_Code_Generator_working

We're still trying to fix the tst system, as the moment its reporting an invalid number of daq channels and during daq initialization it fails.  (This from the /cvs/cds/caltech/target/c1tst/log.txt file). Note: This problem is only on megatron and separated from the conventional DAQ system of the 40m.

cpu clock 2800014
Warning, could open `/rtl_mem_tst' read/write (errno=0)
configured to use 2 cards
Initializing PCI Modules
2 PCI cards found
***************************************************************************
1 ADC cards found
        ADC 0 is a GSC_16AI64SSA module
                Channels = 64
                Firmware Rev = 512

***************************************************************************
1 DAC cards found
        DAC 0 is a GSC_16AO16 module
                Channels = 16
                Filters = None
                Output Type = Differential
                Firmware Rev = 3

***************************************************************************
0 DIO cards found
***************************************************************************
0 IIRO-8 Isolated DIO cards found
***************************************************************************
0 IIRO-16 Isolated DIO cards found
***************************************************************************
0 Contec 32ch PCIe DO cards found
0 DO cards found
***************************************************************************
0 RFM cards found
***************************************************************************
Initializing space for daqLib buffers
Initializing Network
Found 1 frameBuilders on network
Waiting for EPICS BURT at 0.000000 and 0 ns 0x3c40c004
BURT Restore = 1
Waiting for Network connect to FB - 10
Reconn status = 0 1
Reconn Check = 0 1
Initialized servo control parameters.
DAQ Ex Min/Max = 1 32
DAQ Tp Min/Max = 10001 10094
DAQ XTp Min/Max = 10094 10144
DAQ buffer 0 is at 0x8819a020
DAQ buffer 1 is at 0x8839a020
daqLib DCU_ID = 10
DAQ DATA INFO is at 0x3e40f0a0
Invalid num daq chans = 0
DAQ init failed -- exiting

 Dmass tells me that you have to record at least one channel.  ie at least one channel in your .ini file must be set to acquire, otherwise the DAQ will flip out.  It seems to be unhappy when you're not asking it to do things.

I continued on the STAND ALONE debugging of the megatron codes.

- I succeeded to run c1aaa with ADC/DAC. (c1aaa is a play ground for debugging.)

  The trick was "copy DAC block from sam.mdl to aaa.mdl".
  I don't understand why this works. But it worked.
  I still have the problem of the matrices. Their medm screens are always blank. Needs more works.

- Also I don't understand why I can not run the build of c1tst when I copy the working aaa.mdl to tst.mdl.

- The problem Joe reported: "# of channels to be daqed" was solved by

make uninstall-daq-aaa
make install-daq-aaa



  This command is also useful.

daqconfig

- Now I am in the stable development loop with those commands

killaaa
make uninstall-daq-aaa
make aaa
make install-aaa
make install-daq-aaa
make install-screens-aaa
startaaa


  I have made "go_build" script under /home/controls/cds/advLigo

usage:
./go_build aaa

- Note for myself: frequently visited directories

/home/controls/cds/advLigo/src/epics/simLink (for model)
/home/controls/cds/advLigo (to build)
/cvs/cds/caltech/target/c1aaa (realtime code log)
/cvs/cds/caltech/target/c1aaaepics (ioc log)
/cvs/cds/caltech/medm/c1/aaa (medm screens)
/cvs/cds/caltech/chans (filter coeffs)
/cvs/cds/caltech/chans/daq (daq settings)

I am still working on the c1aaa code. Now it seems that C1AAA is working reasonably (...so far).

1) At a certain point I wanted clean up the system status. I have visited /etc/rc.local to add c1aaa for realtime to non-realtime task

before:
/usr/bin/setup_shmem.rtl mdp mdc tst&
after:
/usr/bin/setup_shmem.rtl mdp mdc tst aaa&


   I rebooted the system several times.

sudo /sbin/reboot

2) I found that gabage medm screens accumulated in ~/cds/advLigo/build/aaaepics/medm after many trials with several simulink models.
This directory is always copied to /cvs/cds/caltech/medm/c1/aaa at every make install-screens-aaa
This caused very confusing MEDM screens in the medm dir like C1AAA_ETMX_IN_MATRX.adl (NOT ETMY!)

I did

cd ~/cds/advLigo
make clean-aaa


to refresh aaaepics dir. The current development procedure is

killaaa
make clean-aaa
make uninstall-daq-aaa
make aaa
make install-aaa
make install-daq-aaa
make install-screens-aaa
startaaa

3) Sometimes startaaa does not start the task properly. If the task does not work, don't abandon.
Try restart the task. This may help. 

killaaa
(deep breathing several times)
startaaa

What to do next:

- MEDM works

* make more convenient custom MEDM screens so that we can easily access to the filters and switches
* retrofit the conventional SUS MEDM to the new system

- once again put/confirm the filter coeffs and the matrix elements

- configure DAQ setting so that we can observe suspension motion by dataviewer / dtt

- connect the suspension to megatron again

- test the control loop

This afternoon, I tried to lock the interferometer again after a few days.

After a couple of failed attempts, and relocks of the MZ, the interferometer stayed locked continuously for about 50 minutes, with arm power of about 92.

I just wanted to check the status of the interferometer so I didn't do any particular measurement in the meantime.

ATM1: Seismometers are saturating, suspensions are OK

Atm2 : activity next door,  diesel Backhoe and diesel concrete cutter are running

Atm3: CES exhaust fans output  is pick up by 40M-Annex-North AC unit  intake. The 40m office room has some diesel smell...........see # 2377

In order to get the gige camera code running robustly here at the 40m, I created a "Yoichi style" ezcaread, which is now the default, while the original ezcaread is located in ezcaread.bin.  This tries 5 times before failing out of a read attempt.

By including a differentiator from 10 Hz to 50 Hz, we increase the phase margin and the resulting

magnetic levitation system is stable even without the help of eddy-current damping.

The new block diagram for the system is the following:

Here the eddy-current damping component is removed and we add an additional differential

circuit with an operational amplifier OP27G.

In addition, we place the Hall sensor below the magnet to minimize the coupling between

the coil and the Hall sensor.

The resulting levitation system is shown by the figure below:

An interesting thing was happened in the OMC-LSC timing clock.

Right now the clock of the OMC and the LSC are completely synchronized.

 The trend data is shown below. At the first two measurements (Oct.27 and Nov.1),  LSC had constant retarded time of 3Ts (~92usec.).

The last measurement, on Nov.15, number of shifts goes to zero, this means there are no retarded time.

Also the variance between the two signal gets zero, so I conclude the OMC and the LSC are now completely synchronized.

The measurement on Nov.8 is somehow meaningless, I guess the measurement did not run correctly by an influence from megatron(?)

Dmass, Joe, Koji

A puzzle has been solved: Dmass gave us a great tip

"The RGC code does not work unless the name of the mdl file (simulink model) matches to the model name "

The model name is written in the second line. This is automatically modified if the mdl file is saved from simulink.
But we copied the model by using "cp" command. This prevent from the TST model working!

megatron:simLink>head tst.mdl
Model {
  Name                    "tst"
  Version                 7.3
  MdlSubVersion           0
...
...
...

This explained why the AAA model worked when the DAC block has been copied from the other model.
This was not because of the ADC block but the saving model fixed the model name mismatch!

Now our current working model is "C1TST". Most of the functionalities have been implemented now:

• The simulink model has been modified so that some of the functionalities can be accomodated, such as LSC/ASC PIT/ASC YAW.
• Some filter names are fixed so as to inherit the previous naming conventions.
• The SUS-ETMY epics screen was modified to fit to the new channel names, the filter topologies, and the matrices.
• The chans file was constructed so that the conventional filter coefficients are inherited.
• All of the gains, filter SWs, matrix elements have been set accordingly to the current ETMY settings.
• burt snapshot has been taken: /cvs/cds/caltech/target/c1tstepics/controls_1091117_024223_0.snap
burtrb -f /cvs/cds/caltech/target/c1tstepics/autoBurt.req -o controls_1091117_024223_0.snap -l /tmp/controls_1091117_024215_0.read.log -v

What to do next:

• Revisit Oplev model so that it accomodates a power normalization functionality.
• ETMY QPD model is also missing!
• Clean up mdl file using subsystem grouping

• Check consistency of the whitening/dewhitening switches.
• Connect ADC/DAC to megatron
• Test of the controllability
• BTW, what is happened to BIO?
• Implementation of the RFM card

Directories and the files:

• The .mdl file is backed up as
/home/controls/cds/advLigo/src/epics/simLink/tst.mdl.20091116_2100


• The default screens built by "make" is installed in
  /cvs/cds/caltech/medm/c1/tst/
They are continuously overridden by the further building of the models.


• The custom-built medm screens are stored in
/cvs/cds/caltech/medm/c1/tst/CustomAdls/

The backup is
/cvs/cds/caltech/medm/c1/tst/CustomAdls/CustomAdls.111609_2300/


• The custom-built chans file is
/cvs/cds/caltech/chans/C1TST.txt

The backup is
/cvs/cds/caltech/chans/C1TST.111609


• burt snap shot file
  /cvs/cds/caltech/target/c1tstepics/controls_1091117_024223_0.snap


The smell of diesel is particularly bad this morning.  Its concentrated enough to be causing me a headache.  I'm heading off to Millikan and will be working remotely on Megatron.

[Rana, Jenne]

We opened up the little Thorlabs battery operated PD to see what was inside.  Rana took some pictures, and I drew a schematic (attached).  It's just a diode, biased with a battery (albeit a crazy 22.5V battery).

---------------
Comment by KA: PD is Hamamatsu S1223-01 Si PIN diode.

What a crazy battery. The main point is that it looks like this can be used for reasonable purposes: uses a load resistor on the BNC connector and you can use some pre-amp (e.g. Busby box or SR560) to have a low noise PD readout. You can also use the SR560 in its A-B mode as an 'opamp'. Ground the A input and the use a pole at 1 Hz and make the Output go into the B input through some large series resistor. The BNC from the PD gets Teed into the B input as well.

Then this becomes a transimpedance circuit readout of the diode, using the current noise of the SR560 as the limit.

Had been disconnected for about two weeks.  I found a partially seated 4-pin LEMO cable coming from the OSEM PD interface board. 

The circuit design of multi-resonant EOM have proceeded.

By using numerical method, I found the some best choice of the parameters (capacitors and inductors).

In fact there are 6 parameters (Lp, L1, L2, Cp, C1, C2) in the circuit to be determined.

In general the less parameter gives the less calculation time with performing the numerical analysis. Of course it looks 6 parameters are little bit large number.

In order to reduce the arbitrary parameters, I put 4 boundary conditions.

Each boundary conditions fixed resonant peaks and valleys; first peak=11MHz, third peak=55MHz, first valley=19MHz, second valley=44MHz.

So now the remaining arbitrary parameters successfully get reduced to 2. Only we have to do is optimize the second peak as it to be 29.5MHz.

Then I take C1 and C2 as free parameters seeing how the second peak agree with 29.5MHz by changing the value of the C1 and C2.

the red color represents the good agreement with 29.5MHz, in contrast blue contour represents the bad.

 You can see some best choice along the yellow belt. Now what we should do is to examine some of that and to select one of those.

In designing the whole circuit it is better to know the characteristic of the EOM.

I made impedance measurement with the EOM (New Focus model 4064) and I found it has capacitance of 10pF.

This is good agreement with the data sheet which says "5-10pF".

The measured plot is attached below. For comparison there also plotted "open" and "10pF mica".

In the interested band( from 1MHz to 100MHz), EOM looks just a capacitor.

But indeed it has lead inductance of 12nH, resistance of 0.74[Ohm], and parasitic capacitance of 5.5pF.

In some case we have to take account of those parasites in designing.

How can I get those values from the figure?

I added ~500 cc of distilled water to the laser chiller yesterday.

8 days plot: Thurdsay, Friday, Sat and Sun without construction

This is a continuation from last night, where Peter, Koji, and I were trying to get test point channels working on megatron and with the TST module.

Things we noticed last night:

We could run starttst, and ./daqd -c daqdrc, which allowed us to get some channels in dataviewer.  The default 1k channel selection works, but none of the testpoints do. 

However, awgtpman -s tst does appear in the processes running list.

The error we get from dataviewer is:

Server error 861: unable to create thread
Server error 23328: unknown error
datasrv: DataWriteRealtime failed: daq_send: Illegal seek

Going to DTT, it starts with no errors in this configuration.  Initially it listed both MDC and TST channels.  However, as a test, I moved the tpchn_C4.par , tpchn_M4.par and tpchn_M5.par files to the directory backup, in /cvs/cds/caltech/target/gds/param.  This caused only the TST channels to show up (which is what we want when not running the mdc module.

We had changed the daqdrc file in /cvs/cds/caltech/target/fb, several times to get to this state.  According to the directions in the RCG manual written by Rolf, we're supposed to "set cit_40m=1" in the daqdrc file, but it was commented out.  However, when we uncommented it, it started causing errors on dtt startup, so we put it back.  We also tried adding lines:

set dcu_rate 13 = 16384;
set dcu_rate 14 = 16384;

But this didn't seem to help.  The reason we did this is we noticed dcuid = 13 and dcuid = 14 in the /cvs/cds/caltech/target/gds/param/tpchn_C1.par file.  We also edited the testpoint.par file so that it correctly corresponded to the tst module, and not the mdc and mdp modules.  We basically set:

[C-node1]
hostname=192.168.1.2
system=tst

in that file, and commented everything else out.

At this point, given all the things we've changed, I'm going to try a rebuild of the tst and daq and see if that solves things.

I did a full make clean and make uninstall-daq-tst, then rebuilt it.  I copied a good version of filters to C1TST.txt in /cvs/cds/caltech/chans/ as well as a good copy of screens to /cvs/cds/caltech/medm/c1/tst/.

Test points still appear to be broken.  Although for a single measurement in dtt, I was somehow able to start, although the output in the results page didn't seem to have any actual data in the plots, so I'm not sure what happened there - after that it just said unable to select test points.  It now says that when starting up as well.  The tst channels are the only ones showing up.  However, the 1k channels seem to have disappeared from Data Viewer, and now only 16k channels are selectable, but they don't actually work.  I'm not actually sure where the 1k channels were coming from earlier now that I think about it.  They were listed like C1:TST-ETMY-SENSOR_UL and so forth.

RA: Koji and I added the SENSOR channels by hand to the .ini file last night so that we could have data stored in the frames ala c1susvme1, etc.

I added an integrator to increase the gain at low frequencies (below 5 Hz). In addition, I increased

the band of the differentiator. The schematics for both integrator and differentiator are the following:

The magnetic is stably levitated.

I turned off the light to get rid of 60Hz noise on the photodiode. I tried to measured the

open-loop transfer function of this setup, but somehow the SR560 is always saturate

when I injected the signal from SR785, which produces some weird results at

low-frequencies.

In addition, I found out that when the light is turned on, the levitation

can be stable even when I inverted the sign of the control loop. The control signal

on the osciloscope is the following:

This oscillator is around 120 Hz, which should be  the harmonics of 60 Hz from light pollution.

I am not sure exactly why it is stable when the control-loop sign is flipped. This could

be similar to the Pauli trap in the iron trap, because the coil not only provides a force

but also provides the rigidity. The sign of such rigidity depends on the sign of the control

current. If such oscillating rigidity changes at a frequency much higher than the response

frequency of the magnet, it will stablize  the system simply by significantly increasing

the inertial of the magnet.More investigations are essential to completely understand it.

For information about Pauli trap, one can look at the wikipedia:

http://en.wikipedia.org/wiki/Quadrupole_ion_trap

[Sanjit, Jenne]

Sanjit and I are trying to put names to some signals which exist in SimuLink land, but which don't (yet) exist in EPICS land.  The deelio is that for each of the chosen SEIS signals in the ASS_TOP_PEM screen, the signal is split.  One part of the signal is used to decide how the adaptive filter should look, and the other part is actually used when doing the on-line subtraction.  Previously only the part of the signal which is used to decide on the Adaptive Filter could be seen on the screens, and had names. 

Before touching anything on the Simulink ASS.mdl, I did an svn check in, which put things at revision 36639. 

To try to make the desired signals exist, I put cdsFilt boxes (to create filter modules for each of these signals), and gave each of them a name (kind of like the Neverending Story....once they have a name, they'll exist).  My new names are C1:ASS-TOP_PEM_#_APPLY, which correspond to the previously-existing C1:ASS-TOP_PEM_#_ADPT (these are the ones that are along the top of the ASS_TOP_PEM matrix screen).  This version of the simulink model was checked in, and the svn is now at revision 36640.

We then did some "make clean", "make ass" and "make install-ass" action, and burt restored c1assepics, but nothing seems to be happening.  The screen doesn't have white boxes all over the place, and we didn't get any errors when we did the makes, and I'm sure we burt restored correctly (made sure the ASS GDS screen had a 1 in the lower left box etc), but all the values on the screen are still zero.  

When we ran the ass front end in terminal on the c1ass machine, we did see an error: "Invalid chan num found 2 = 30624" "DAQ init failed -- exiting".  I think this means that we need to have told some file somewhere that I was going to be adding 8 new channels. (maybe an .ini file?) Hopefully the Joe & Peter team can help us out with this, since they've been doing this kind of thing for the new system.

Moral of the story is, the new (non-working) simulink file has been svn checked in as revision 36640, and we're reverting to revision 36639, which was before I touched anything today.

This plot shows the RIN as measured by the ISS. Its ~2 x 10^-7, whereas its supposed to be more like 3 x 10^-8.

The ISS has DC coupled RIN channels (with a _F suffix) and AC coupled RIN channels (with a _FW suffix). By using a swept sine, Rob determined that the AC coupled channels have an AC coupling pole at ~80 Hz. The attached plot uses this and then has the overall gain adjusted to match with the _F channels below 10 Hz.

The _F channels can be converted directly into RIN by just dividing the spectra by the mean value of the time series. The dark offset of these channels is small and so this only introduces a ~5-10% calibration error.

Question #1: Why is the RIN so bad? According to the MEDM screen, the photocurrent on the MON/SENS PDs is 1.9/1.3 mA. That's sort of low, but should still allow us to get 5x10^-8 in RIN.

Question #2: Does it make an effect on the current DC Readout work? IF so, should we try to fix up the ISS in a temporary way? Since the in-loop and out-of-loop detectors are completely coherent, all of the noise is likely just unsuppressed noise from the laser. We are unable to increase the gain because of the high frequency noise from the NPRO.

Let's remember to replace this ISS with a new one that can drive an AOM. Need a volunteer to get us a new ISS.

 Here is a 7 years plot of  of the 40m temperature variations.

 Saturday's special event of braking up the large concrete pieces in CES bay was un event full.

As part of the fb40m restart procedure (Sanjit and I were restarting it to add some new channels so they can be read by the OAF model), I checked up on how the backup has been going.  Unfortunately the answer is: not well.

Alan imparted to me all the wisdom of frame builder backups on September 28th of this year.  Except for the first 2 days of something having gone wrong (which was fixed at that time), the backup script hasn't thrown any errors, and thus hasn't sent any whiny emails to me.  This is seen by opening up /caltech/scripts/backup/rsync.backup.cumlog , and noticing that  after October 1, 2009, all of the 'errorcodes' have been zero, i.e. no error (as opposed to 'errorcode 2' when the backup fails).  

However, when you ssh to the backup server to see what .gwf files exist, the last one is at gps time 941803200, which is Nov 9 2009, 11:59:45 UTC.  So, I'm not sure why no errors have been thrown, but also no backups have happened. Looking at the rsync.backup.log file, it says 'Host Key Verification Failed'.  This seems like something which isn't changing the errcode, but should be, so that it can send me an email when things aren't up to snuff.  On Nov 10th (the first day the backup didn't do any backing-up), there was a lot of Megatron action, and some adding of StochMon channels.  If the fb was restarted for either of these things, and the backup script wasn't started, then it should have had an error, and sent me an email.  Since any time the frame builder's backup script hasn't been started properly it should send an email, I'm going to go ahead and blame whoever wrote the scripts, rather than the Joe/Pete/Alberto team.

Since our new raid disk is ~28 days of local storage, we won't have lost anything on the backup server as long as the backup works tonight (or sometime in the next few days), because the backup is an rsync, so it copies anything which it hasn't already copied.  Since the fb got restarted just now, hopefully whatever funny business (maybe with the .agent files???) will be gone, and the backup will work properly. 

I'll check in with the frame builder again tomorrow, to make sure that it's all good.

[Jenne, Sanjit]

We would like several channels from the OAF/ASS screen to be saved to frames, so that we can use the channels for our OAF model.  In theory, this should involve uncommenting the desired channels in the .ini file (.../caltech/chans/daq/C1ASS.ini), and restart the frame builder.  Since this .ini file was generated a long time ago, and things have been changed since then, the chnnums in the .ini file and the corresponding .par file don't match up.  We need to go through the .par file (/cvs/cds/gds/param/tpchn_c3.par), and look up the chnnums for our channels, and copy those numbers into the .ini file.  Figuring out what was going on involved many fb40m restarts, but on the last one of the night, I restarted the backup script, so it should (hopefully) run tonight, and get all of the frames that we've been missing.

Notes to self: 

*  When adding channels to other front ends, the end of the process is to click the blue button on the C0DAQ_DETAIL screen next to your computer.  C1ASS isn't on that screen.  Instead, in the C1ASS_GDS screen, click DAQ Reload.

*  The channel names for the Test Points and the .ini files must be different.  That's why there's a '_2048' suffix at the end of every channel in our .ini file.

*  tpchn_C1 is all of the old-style system test points.  tpchn_C2 is the C1OMC, and tpchn_C3 is for the C1ASS testpoints.

*  When uncommenting channels in the C1ASS.ini file, make sure acquire is set to 1 for every channel we want saved.  The default in this .ini file is set to acquire = 0.

With Koji's help, I measured the length of the Mode Cleaner.

The new modulation frequencies (as quoted on the Marconi front panels) are: 

165.980580 MHz

 33.196116 MHz

132.784464 MHz

199.176696 MHz

The Frequency Counter readback is 165980584.101 Hz (a 4Hz difference).  All of the Marconi's front-panel frequencies read ###.##### MHz Ext, and the Frequency standard has it's "locked" light illuminated, and the 1pps input light blinking, so I think everything is still nicely locked to the frequency standard, and the frequency standard is locked to the GPS.

While changing the marconi's, I accidentally touched the MC's 29.5 MHz marconi.  It is set back to the nominal value (according to Kiwamu's rack photos) of 29.485MHz.  But the phase might be sketchy, although hopefully this doesn't matter since we don't do a double demodulation with it.

I also ran the scripts in the wiki page: How To/Diagonalize DRMI Length Control to set the DD Phases.

I aligned the beam goes to PMC. It increased the MC Trans from 8.25 to 8.30.

I also aligned the beam goes to RC.
When I touched the FSS box (wrong: this was the VCO driver) that was close to one of the steering mirror, suddenly the RC trans increased.
It is now 9.8. I am afraid that it gets saturated. I could not reproduce the phenomenon. This could be caused by a bad contact?
Note that I didn't see there is any loose optic.

I propose that from now on, we indicate in the elog what frequencies we're referring to. In this case, I guess its the front panel readback and not the frequency counter -- what is the frequency counter readback? And is everything still locked to the 10 MHz from the GPS locked Rubidium clock?

Plus, what FSS Box? The TTFSS servo box? Or the VCO driver? As far as I know, the RC trans PD doesn't go through the FSS boxes, and so its a real change. I guess that a bad contact in the FSS could have made a huge locking offset.

What I meant was the VCO driver, not the FSS box.

As for the frequency, all written numbers were the Marconi displays.
The number on the frequency counter was also recorded, and so will be added to the previous entry shortly... 

I'm working on the AP table. I also opened the auxiliary NPRO shutter. The auxiliary beam is on its path on the AP table and PSL table.

 All is well again in the world of backups.  We are now up to date as of ~midnight last night. 

 Closing the AP table and the NPRO shutter now.

Locking has gone sour.  The CARM to MCL handoff, which is fairly early in the full procedure and usally robust, is failing reliably. 

As soon as the SUS-MC2_MCL gain is reduced, lock is broken.  There appears to be an instability around 10Hz.  Not sure if it's related.

I'm working on the AP table. I also opened the auxiliary NPRO shutter. The auxiliary beam is on its path on the AP table and PSL table.

Last night something happened on the beat between the PSL beam and the auxiliary NPRO beam, that spoiled the quality of the beating I had before. As a result the PLL has become unable to lock the two lasers.

The amplitude of the beat at the spectrum analyzer has gone down to -40 dBm from -10 that it was earlier. The frequency has also become more unstable so that now it can be seen writhing within tens of KHz.

Meanwhile the power of the single beams at the PLL photodiode hasn't changed, suggesting that the alignment of the two beam didn't change much.

Changes in the efficiency of the beating between the two beams are not unusual. Although that typically affects only the amplitude of the beat and wouldn't explain why also its frequency has become unstable. Tuning the alignment of the PLL optics usually brings the amplitude back, but it was uneffective today.

It looks like something changed in either one of the two beams. In particular the frequency of one of the two lasers has become less stable.

Another strange thing that I've been observing is that the amplitude of the beat goes down (several dBm) as the beat frequency is pushed below 50 MHz. Under 10 MHz it even gets to about -60 dBm.

I noticed the change yesterday evening at about 6pm, while I was taking measurements of the PLL open loop tranfer function and everything was fine. I don't know whether it is just a coincidence or it is somehow related to this, but Jenne and Sanjit had then just rebooted the frame builder.

 I confirm what I said earlier. The amplitude of the beat is -10 dBm at 300MHz. It goes down at lower frequencies. In particular it gets to-60 dBm below 20 MHz. For some strange reason that I couldn't explain the beating efficiency has become poorer at low frequencies.

 Backup Fail.  At least this time however, it threw the appropriate error code, and sent me an email saying that it was unhappy.  Alan said he was going to check in with Stuart regarding the confusion with the ssh-agent.  (The other day, when I did a ps -ef | grep agent, there were ~5 ssh-agents running, which could have been then cause of the unsuccessful backups without telling me that they failed.  The main symptom is that when I first restart all of the ssh-agent stuff, according to the directions in the Restart fb40m Procedures, I can do a test ssh over to ldas-cit, to see what frames are there.  If I log out of the frame builder and log back in, then I can no longer ssh to ldas-cit without a password.  This shouldn't happen....the ssh-agent is supposed to authenticate the connection so no passwords are necessary.) 

I'm going to restart the backup script again, and we'll see how it goes over the long weekend. 

Just felt a big "kerplunk" type ground-shaking, presumably from all the antics next door.  MC2's watchdog tripped as a result.  The watchdog has been reenabled.

 Five day minute trend.  FAST_F doesn't appear to have gone crazy.

 Whatever the locking problem was, the power of magical thinking has forced it to retreat for now.  The IFO is currently locked, having completed the full up script.  One more thing for which to be thankful.

 NPRO shutter closed

but the increase in both the RCtrans and the RCrefl is consistent with my theory that the power going to the RC has increased ; its not just an increase in the visibility.

We should scan the AOM/VCO to make sure the frequency is matched to the resonance to within 0.5 dB.

I checked C1:PSL-FSS_VCODETPWR. The attached is the 4 months trend of the FSS RCTRANS / RFPDDC(=FSS REFL) / VCODETPWR / VCOMODLEVEL.

Although VCO modulation level setting was mostly constnt, VCODETPWR, which presumably represents the RF level, changes time by time.
It coincides with the recent reduction of the RCTRANS/RFPDDC. Actually, my touch restored the VCO to the previous more stable state.
One can see that this is not only a single occation, but it happened before too. (In the middle of Aug.)

This could be explained by the bad contact of some cable or connector.

Nevertheless we need more careful investigation:

1. Understand what VCODETPWR is exactly.
2. Investigate relationship between VCOMODLEVEL / VCODETPWR / AOM deflection efficiency / RCTRANSPD
3. Confirm the frequency matching between the VCO and AOM.

Problem found. Inspecting with Koji we found that there was a broken SMA-to-BNC connector in the BNC cable from the photodiode.

I measured the open loop gain of the PLL in the AbsL experiment.

I repeated the measurement twice: one with gain knob on the universal PDH box g=3.0; the second measurement with g=6.0

The UGF were 60 KHz and 100 KHz, respectively.

That means that one turn of the knob equals to about +10 dB.