Backup Fail.  At least this time however, it threw the appropriate error code, and sent me an email saying that it was unhappy.  Alan said he was going to check in with Stuart regarding the confusion with the ssh-agent.  (The other day, when I did a ps -ef | grep agent, there were ~5 ssh-agents running, which could have been then cause of the unsuccessful backups without telling me that they failed.  The main symptom is that when I first restart all of the ssh-agent stuff, according to the directions in the Restart fb40m Procedures, I can do a test ssh over to ldas-cit, to see what frames are there.  If I log out of the frame builder and log back in, then I can no longer ssh to ldas-cit without a password.  This shouldn't happen....the ssh-agent is supposed to authenticate the connection so no passwords are necessary.) 

I'm going to restart the backup script again, and we'll see how it goes over the long weekend. 

 Dan Kozak is rsync transferring /frames from NODUS over to the LDAS grid. He's doing this without a BW limit, but even so its going to take a couple weeks. If nodus seems pokey or the net connection to the outside world is too tight, then please let me and him know so that he can throttle the pipe a little.

The recently observed daqd flakiness looks related to this transfer. It appears to still be ongoing:

nodus:~>ps -ef | grep rsync
controls 29089   382  5 13:39:20 pts/1   13:55 rsync -a --inplace --delete --exclude lost+found --exclude .*.gwf /frames/trend
controls 29100   382  2 13:39:43 pts/1    9:15 rsync -a --delete --exclude lost+found --exclude .*.gwf /frames/full/10975 131.
controls 29109   382  3 13:39:43 pts/1    9:10 rsync -a --delete --exclude lost+found --exclude .*.gwf /frames/full/10978 131.
controls 29103   382  3 13:39:43 pts/1    9:14 rsync -a --delete --exclude lost+found --exclude .*.gwf /frames/full/10976 131.
controls 29112   382  3 13:39:43 pts/1    9:18 rsync -a --delete --exclude lost+found --exclude .*.gwf /frames/full/10979 131.
controls 29099   382  2 13:39:43 pts/1    9:14 rsync -a --delete --exclude lost+found --exclude .*.gwf /frames/full/10974 131.
controls 29106   382  3 13:39:43 pts/1    9:13 rsync -a --delete --exclude lost+found --exclude .*.gwf /frames/full/10977 131.
controls 29620 29603  0 20:40:48 pts/3    0:00 grep rsync

Diagnosing the problem:

I logged into fb and ran "top". It said that fb was waiting for disk I/O ~60% of the time (according to the "%wa" number in the header). There were 8 nfsd (network file server) processes running with several of them listed in status "D" (waiting for disk). The daqd logs were ending with errors like the following suggesting that it couldn't keep up with the flow of data:

[Wed Oct 22 18:58:35 2014] main profiler warning: 1 empty blocks in the buffer
[Wed Oct 22 18:58:36 2014] main profiler warning: 0 empty blocks in the buffer
GPS time jumped from 1098064730 to 1098064731

This all pointed to the possibility that the file transfer load was too heavy.

Reducing the load:

The following configuration changes were applied on fb.

Edited /etc/conf.d/nfs to reduce the number of nfsd processes from 8 to 1:

OPTS_RPC_NFSD="1"

(was "8")

Ran "ionice" to raise the priority of the framebuilder process (daqd):

controls@fb /opt/rtcds/rtscore/trunk/src/daqd 0$ sudo ionice -c 1 -p 10964

And to reduce the priority of the nfsd process:

controls@fb /opt/rtcds/rtscore/trunk/src/daqd 0$ sudo ionice -c 2 -p 11198

I also tried punishing nfsd with an even lower priority ("-c 3"), but that was causing the workstations to lag noticeably.

After these changes the %wa value went from ~60% to ~20%, and daqd seems to die less often, but some further throttling may still be in order.

Here's another useful link:

http://maps.google.com/maps?q=34.13928,-118.123756

 Santa Anna wind speed was locked around 60 kmph last night on campus. The strongest in 30 years.  The lab hold up well. We did not lose  AC power either.

Threes and windows were blown out and  over on campus.

We have 4 sliding glass windows without "heavy-laser proved" inside protection.

We should plan to upgrade ALL  sliding glass windows with metal protection from the inside.The strongest in 30 years.

The Kinemetrics dudes are going to visit us @ 1:45 tomorrow (Wednesday) to check out our stacks, seismos, etc.

I put these maps here on the elog since people are always getting lost trying to find the lab.

In the past year, pygwinc has expanded to support not just fundamental noise calculations (e.g., quantum, thermal) but also any number of user-defined noises. These custom noise definitions can do anything, from evaluating an empirical model (e.g., electronics, suspension) to loading real noise measurements (e.g., laser AM/PM noise). Here is an example of the framework applied to H1.

Starting with the BHD review-era noises, I have set up the 40m pygwinc fork with a working noise budget which we can easily expand. Specific actions:

• Updated the 40m fork to the latest pygwinc version (while preserving the commit history).
• Added a directory ./CIT40m containing the 40m-specific noise budget files (created by GV).
• Added an ipython notebook CIT40m.ipynb at the root level showing how to generate a noise budget.
• Integrated our DAC and seismic noise estimators into pygwinc.
• Marked the old 40m NB repo as obsolete (last commit > 2 yrs ago). Many of these noise estimates are probably stale, but I will work with GV to identify which ones can be migrated.

I set up our fork in this way to keep the 40m separate from the main pygwinc code (i.e., not added to as a built-in IFO type). With the 40m code all contained within one root-level directory (with a 40m-specific name), we should now always be able to upgrade to the latest pygwinc without creating intractable merge conflicts.

[Koji / Gautam (Remote)]

Lab status

• Gray Panel: The lab AC was off. Turned on all three (N/S, CTRL RM, E/W)
• The control room AC was running.

Work stations

• Control Room: All the control machines were running. We knew that nodus/chiara/fb were running
• 1X6/7:
  • JETSTOR was making beeping sound. “Power #1 failed””power #2 failed”
  • Optimus & megatron were off -> turned on -> up and running now
• 1X1/2:
  • ​Power cycled the netgear at the top of the IOO rack (maybe not necessary)
  • Turned on c1ioo -> up and running now
• 1X4/5: ​Rebooted c1sus / c1lsc -> up and running now
• 1X9: Rebooted c1iscex -> up and running now
• 1Y4: Rebooted c1iscex -> up and running now

Vacuum status

• Looked like everything was running as if it did not see the power glitch
• TP1 normal: Set speed 33.6k rpm / Actual speed 33.6k rpm 
• TP2 normal: 66k rpm / PTP2 16.0 mtorr
• TP3 normal: 31k rpm / PTP3 45.4mtorr
• P1 LOW / P2 1.7mtorr / CC2 1.1e-6 / P3 7.6e-2 / P4 LO
• Annuli: 2.7~3torr
• CC1 9.6e-6 / SUPER BEE 0.9mtorr

C1VAC recovery

• c1vac was alive, but was isolated from the martian network
• Checked the network I/F status with /sbin/ifconfig -a
  • eth0 had no IP
  • eth1 had the vac subnet IP (192.168.114.9)
• Ran sudo /sbin/ifdown eth0 then  sudo /sbin/ifup eth0
• The I/F eth0 started running and c1vac became visible from martian
• Later checked the vacuum screen: The pressure values and valve statuses looked normal.
  The interlock state was “running”. The system state was “unrecognized”.

End RTS recovery 

• The end slow machines (auxex and auxey) were already running
• Restarting end RT models:
  • c1iscey -> rtcds start --all
  • c1iscex -> rtcds start --all
• Confirmed that the models can dump the SUSs

Vertex RTS recovery

• We wanted to use the reboot script. (/opt/rtcds/caltech/c1/scripts/cds/rebootC1LSC.sh)
• c1susaux​​
  • To be safe, we wanted to bring c1susaux first.
  • c1susaux does not make the network I/Fs up automatically upon reboot.
    -> Connect an LCD display / keyboard / mouse to c1susaux
    -> Ran sudo /sbin/ifup eth0 and sudo /sbin/ifup eth1
  • Now c1susaux is visible from martian.
  • Login c1susaux and ran:  
    sudo systemctl start modbusIOC.service 
    -> c1susaux epics is up and running now
  • ...Meanwhile c1susaux lost its eth1 somehow. This made the slow values of 8 vertex sus all zero
    -> Ran sudo /sbin/ifdown eth1 and sudo /sbin/ifup eth1 again on c1susaux ->  this resolved the issue
• c1psl
  • Login c1psl and ran:  
    sudo systemctl start modbusIOC.service 
    -> c1psl epics is up and running now
• Prepared for the rebooting script
  • Ran /opt/rtcds/caltech/c1/scripts/cds/rebootC1LSC.sh
  • Rebooting was done successfully. All the suspensions looked free and healthy.
  • Burtrestored c1susaux (used Apr 12 21:19 snapshot)

Hardware

• PSL laser / Xend AUX laser / Yend AUX laser were off -> turned on
• The PMC was immediately automatically locked.
• The main marconi was off -> forgot to turn on
• The end temp controllers for the SHG crystals were on but not enabled -> now enabled

RTS recovery ~ part 2

• FB: FB status of all the RTS models were still red
• Timing: c1x01/2/3/5 were 1 sec behind of FB and c1x04 was 2 sec behind
• -> Remedy:  https://nodus.ligo.caltech.edu:8081/40m/14349
  • Software rebooting of FB
  • Manually start the open-mx and mx services using
  • sudo systemctl start open-mx.service 
  • sudo systemctl start mx.service
  • Check that the system time returned by gpstime matches the gpstime reported by internet sources. e.g. http://leapsecond.com/java/gpsclock.htm
  • Manually start the daqd processes using
    sudo systemctl start daqd_*
• This made all the FB(FE) indicators green!
• Ran the reboot script again -> All green!

IMC recovery

• The IMC status was checked
• No autolocker, but it could be manually locked. i.e. MC1/2/3 were not so much misalignment
• Autolocker/Slow FSS recovery along with https://nodus.ligo.caltech.edu:8081/40m/15121
  • sudo systemctl start MCautolocker.service
  • sudo systemctl start FSSSlow.service
• Both of them failed to run
• Note by Gautam: The problem with the systemctl commands failing was that the NFS mount points weren’t mounted. Which in turn was because of the familiar /etc/resolv.conf problem. I added chiara to the namespace in this file, and then manually mounted the NFS mount points. This fixed the problem.
  Now the IMC is locked and the autolocker is left running.

Burt restore

• Used Apr 12 21:19 snapshot
• c1psl
• c1alsepics/c1assepics/c1asxepics/c1asyepics
• c1aux/c1auxex/c1auxey/
• c1iscaux/c1susaux
• This made REFL and AS beams back to the CCDs. As has small fringes.
• Y arm has small IR flashes as well as green flashes.

JETSTOR recovery

• JETSTOR was beeping. 
• Shutdown megatron
• Followed the instruction https://nodus.ligo.caltech.edu:8081/40m/13107
• This stopped beeping. Waiting for JETSTOR to come up -> In a minute, JETSTOR display became normal and all disks showed green.
• Bring megatron back up again

N2 bottle

• The left N2 bottle was empty. The right one had 1500PSI.
• Replaced the left bottle with the spare one in the room.
• Now the left one 2680PSI and the right one 1400PSI.

Closing

• Closed PSL/AUX laser shutters
• Turned off the lights in the lab, CTRL room, and the office.

​Remaining Issues

• [done] MCAutoLocker / FSSSlow scripts are not running
• The PRM alignment slider has no effect (although the PRM is aligned…) -> SLOW DAQ frozen???
• JETSTOR is not mounted on megatron [gautam mounted Jetstor on megatron on 4/18 at 2pm]

Kiwamu, Alex and Zach are practicing mandatory IR-safety scan at the 40m-PSL

40m specific safety indoctrination were completed.

Josh Smith, Fabian Magana-Sandoval, Jackie Lee (Fullerton)

Thanks to Jamie and Jenne for the tour and the input on the pages.

We had a look at the GEO summary pages and thought about how best to make a 40m summary page that would eventually become and aligo summary page. Here's a rough plan:

- First we'll check that we can access the 40m NDS2 server to get data from the 40m lab in Fullerton.

- We'll make a first draft of a 40m summary page in python, using pynds, and base the layout on the current geo summary pages.

- When this takes shape we'll iterate with Jamie, Jenne, Rana to get more ideas for measurements, layout.

Other suggestions: Jenne is working on an automated noisebudget and suggests having a placeholder for it on the page. We can also incorporate some of the features of Aidan's 40m overview medm screen that's in progress, possibly with different plots corresponding to different parts of the drawing, etc. Jenne also will email us the link of once per hour medm screenshots.

40m summary webpages

 The aLIGO-style summary webpages are now running on 40m data! They are running on megatron so can be viewed from within the martian network at:

http://192.168.113.209/~controls/summary

At the moment I have configured the 5 seismic BLRMS bands, and a random set of PSL channels taken from a strip tool.

Technical notes

• The code is in python depending heavily on the LSCSoft PyLAL and GLUE modules.
  • /home/controls/public_html/summary/bin/summary_page.py
• The HTML is supported by a CSS script and a JS script which are held locally in the run directory, and JQuery linked from the google repo.
  • /home/controls/public_html/summary/summary_page.css
  • /home/controls/public_html/summary/pylaldq.js
• The configuration is controlled via a single INI format file
  • /home/controls/public_html/summary/share/c1_summary_page.ini

Getting frames

Since there are no segments or triggers for C1, the only data sources are GWF frames. These are mounted from the framebuilder under /frames on megatron. There is a python script that takes in a pair of GPS times and a frame type that will locate the frames for you. This is how you use it to find T type frames (second trends) for May 25 2012:

python /home/controls/public_html/summary/bin/framecache.py --ifo C1 --gps-start-time 1021939215 --gps-end-time 1022025615 --type T -o framecache.lcf

If you don't have GPS times, you can use the tconvert tool to generate them

$ tconvert May 25
1021939215

The available frame types, as far as I'm aware are R (raw), T (seconds trends), and M (minute trends).

Running the code

The code is designed to be fairly easy to use, with most of the options set in the ini file. The code has three modes - day, month, or GPS start-stop pair. The month mode is a little sketchy so don't expect too much from it. To run in day mode:

python /home/controls/public_html/summary/bin/summary_page.py --ifo C1 --config-file /home/controls/public_html/summary/share/c1_summary_page.ini --output-dir . --verbose --data-cache framecache.lcf -SRQDUTAZBVCXH --day 20120525

Please forgive the large apparently arbitrary collection of letters, since the 40m doesn't use segments or triggers, these options disable processing of these elements, and there are quite a few of them. They correspond to --skip-something options in long form. To see all the options, run

python /home/controls/public_html/summary/bin/summary_page.py --help

There is also a convenient shell script that will run over today's data in day mode, doing everything for you. This will run framecache.py to find the frames, then run summary_page.py to generate the results in the correct output directory. To use this, run

bash /home/controls/public_html/summary/bin/c1_summary_page.sh

Configuration

Different data tabs are disabled via command link --skip-this-tab style options, but the content of tabs is controlled via the ini file. I'll try to give an overview of how to use these. The only configuration required for the Seismic BLRMS 0.1-0.3 Hz tab is the following section:

[data-Seismic 0.1-0.3 Hz]
channels = C1:PEM-RMS_STS1X_0p1_0p3,C1:PEM-RMS_STS1Y_0p1_0p3,C1:PEM-RMS_STS1Z_0p1_0p3
labels = STS1X,STS1Y,STS1Z
frame-type = R
plot-dataplot1 =
plot-dataplot3 =
amplitude-log = True
amplitude-lim = 1,500
amplitude-label = BLRMS motion ($\mu$m/s)

The entries can be explained as follows:

1. '[data-Seismic 0.1-0.3 Hz] - This is the section heading. The 'data-' mark identifies this as data, and is a relic of how the code is written, the 'Seismic 0.1-0.3 Hz' part is the name of the tab to be displayed in the output.
2. 'channels = ...' - This is a comma-separated list of channels as they are named in the frames. These must be exact so the code knows how to find them.
3. 'labels = STS1X,STS1Y,STS1Z' - This is a comma-separated list of labels mapping channel names to something more readable for the plots, this is optional.
4. 'frame-type = R' - This tells the code what frame type the channels are, so it can determine from which frames to read them, this is not optional, I think.
5. 'plot-dataplotX' - This tells the code I want to run dataplotX for this tab. Each 'dataplot' is defined in it's own section, and if none of these options are given, the code tries to use all of them. In this configuration 'plot-dataplot1' tells the code I want to display the time-series of data for this tab.
6. 'amplitude-XXX = YYY' - This gives the plotter specific information about this tab that overrides the defaults defined in the dataplotX section. The options in this example tell the plotter that when plotting amplitude on any plot, that axis should be log-scale, with a limit of 1-500 and with a specific label. The possible plotting configurations for this style of option are: 'lim', 'log', 'label', I think.

Other compatible options not used in this example are:

• scale = X,Y,Z - a comma-separated list of scale factors to apply to the data. This can either be a single entry for all channels, or one per channel, nothing in between.
• offset = X,Y,Z - another comma-separate list of DC offsets to apply to the data (before scaling, by default). DAQ noise may mean a channel that should read zero during quick times is offset by some fixed amount, so you can correct that here. Again either one for all channels, or one per channel.
• transform = lambda x: f(x) - a python format lambda function. This is basically any mathematical function that can be applied to each data sample. By default the code constructs the function 'lambda d: scale * (d-offset)', i.e. it calibrates the data by removing the offset an applying the scale.
• band = fmin, fmax - a low,high pair of frequencies within which to bandpass the data. Sketchy at best...
• ripple_db = X - the ripple in the stopband of the bandpass filter
• width = X - the width in the passband of the bandpass filter
• rms_average = X - number of seconds in a single RMS average (combine with band to make BLRMS)
• spectrum-segment-length = X - the length of FFT to use when calculating the spectrum, as a number of samples
• spectrum-overlap = X - the overlap (samples) between neighbouring FFTs when calculating the spectrum
• spectrum-time-step = X - the length (seconds) of a single median-mean average for the spectrogram

At the moment a package version issue means the spectrogram doesn't work, but the spectrum should. At the time of writing, to use the spectrum simple add 'plot-dataplot2'.

You can view the configuration file within the webpage via the 'About' link off any page.

Please e-mail any suggestions/complaints/praise to duncan.macleod@ligo.org.

There is now a job in the crontab that will run the shell wrapper every hour, so the pages _should_ take care of themselves. If you make adjustments to the configuration file they will get picked up on the hour, or you can just run the script by hand at any time.

$ crontab -l
# m h  dom mon dow   command
0 */1 * * * bash /home/controls/public_html/summary/bin/c1_summary_page.sh > /dev/null 2>&1

I have moved the summary pages stuff that Duncan set up to a new directory that it accessible to the nodus web server and is therefore available from the outside world:

/users/public_html/40-summary

which is available at:

https://nodus.ligo.caltech.edu:30889/40m-summary/

I updated the scripts, configurations, and crontab appropriately:

/users/public_html/40m-summary/bin/c1_summary_page.sh
/users/public_html/40m-summary/share/c1_summary_page.ini

1. Busby box
2. SR554 transformer preamplifier

I update my old 40mUpgrade Optickle model, by adding the latest updates in the optical layout (mirror distances, main optics transmissivities, folding mirror transmissivities, etc). I also cleaned it from a lot of useless, Advanced LIGO features.

I calculated the expected power in the fields present at the main ports of the interferometer.

I repeated the calculations for both the arms-locked/arms-unlocked configurations. I used a new set of functions that I wrote which let me evaluate the field power and RF power anywhere in the IFO. (all in my SVN directory)

As in Koji's optical layout, I set the arm length to 38m and I found that at the SP port there was much more power that I woud expect at 44Mhz and 110 MHz.

It's not straightforward to identify unequivocally what is causing it (I have about 100 frequencies going around in the IFO), but presumably the measured power at 44MHz was from the beat between f1 an f2 (55-11=44MHz), and that at 110MHz was from the f2 first sidebands.

Here's what i found:

I found that When I set the arm length to 38.55m (the old 40m average arm length), the power at 44 and 110 MHz went significantly down. See here:

I checked the distances between all the frequencies circulating in the IFO from the closest arm resonance to them.

I found that the f2 and 2*f2 are two of the closest frequencies to the arm resonance (~80KHz). With a arm cavity finesse of 450, that shouldn't be a problem, though.

 I'll keep using the numbers I got to nail down the culprit.

Anyways, now the question is: what is the design length of the arms? Because if it is really 38m rather than 38.55m, then maybe we should change it back to the old values.

1. Give us the designed arm length. What is the criteria?

2. The arm lengths got shorter as the ITMs had to shift to the end. To make them longer is difficult. Try possible shorter length.

 If you have a working 40m Optickle model, put it in a common place in the SVN, not in your own folder.

I can't figure out why changing the arm length would effect the RF sidebands levels. If you are getting RF sidebands resonating in the arms, then some parameter is not set correctly.

As the RF sideband frequency gets closer to resonating in the arm, the CARM/DARM cross-coupling to the short DOFs probably gets bigger.

I uploaded the latest iscmodeling package to the SVN under /trunk. It includes my addition of the 40m Upgrade model: /trunk/iscmodeling/looptickle/config40m/opt40mUpgrade2010.m.

I don't know the causes of this supposed resonances yet. I'm working  to try to understand that. It would be interesting also to evaluate the results of absolute length measurements.

Here is what I also found:

It seems that 44, 66 and 110 are resonating.

If that is real, than 37.5m could be a better place. Although I don't have a definition of "better" yet.  All I can say is these resonances are smaller there.

Only the 40m cranes are running on 480VAC The electricians are rewiring this transformer on the mezzanine so it was shut down.

I tested all three cranes before the 480V power was turned off. The last thing to do with the cranes to wipe them down before use.

It will happen on next Tuesday morning.

CES Mezzanine is beeing rebuilt to accommodate our new neighbor: the 20ft high water slide...& .jacuzzi

All our ac power transformers are up there. Yesterday we labelled the power switch of 480VAC on the mezz

that we need to keep to run the 3 cranes in the lab.

I have tested the left 2ch of 4ch demod board.

The left most is for 11MHz, and the next one is for the 55MHz.

1) Linearity Test

LO input level was +10dBm. The LO freq was 11MHz and 55MHz for CH1 and CH2 respectively.
The IF frequency was fixed at 10kHz.

The amplitude of the RF input was swept from -50dBm to +15dBm.
Basically I and Q output of CH1 and CH2 was quite linear in this amplitude range.

2) Freqency Response

RF input was fixed at -20dBm and the IF frequency was swept from 1kHz to 1MHz.

The response was flat upto 100kHz, and have sensitivity upto 300kHz.

3) Output noise

Noise floor of the output is ~20nV/rtHz. All of the channels behave in the same way.
1/f start from 100Hz.

Looks like yesterday was particularly noisy.  It's unclear to me why diurnal variation much more visible in MC1_Y, and why the floor wanders.

The first plot shows 5 days.  The second plot shows 20 days.

Koji had the good idea of trying to measure the motion of the POP beam, and feeding that signal to PRM yaw to stabilize the motion.  To facilitate this, I have installed a 50% beam splitter before the POP 110/22 PD (so also before the camera). 

Before touching anything, I locked the PRM-ITMY half-cavity so that I had a constant beam at POP.  I measured the POP DC OUT to be 58.16 counts.  I then installed a 1" 50% BS, making sure (using the 'move card in front of optic while watching camera' technique) that I was not close to clipping on the new BS.  I then remeasured POP DC OUT, and found it to be 30.63.  I closed the PSL shutter to get the dark value, which was -0.30 .  This means that I now have a factor of 0.53 less light on the POP110/22 PD.  To compensate for this, I changed the values of the power normalization matrix from 0.01 (MICH) to 0.0189, and 100 (PRCL) to 189.

After doing this, I restored the ITMX and am able to get several tens of seconds of PRMI lock (using AS55Q and REFL33I). 

I found several QPDs in the PD cabinet down the Y arm, but no readout electronics.  The QPD I found is D990272.  I don't really want to spend any significant amount of time hacking something for this together, if Valera can provide a QPD with BNC outputs. For now, I have not installed any DC PD or razor blade (which can be a temporary proxy for a QPD, enough to get us yaw information).

Jessica will soon ELOG about some measurements suggesting that the conductive connector-ized ALS delay line enclosure is the way to go, when considering crosstalk between the delay lines. It is currently mounted and hooked up on the LSC rack, though I need to make a bunch of new SMA cables now that I think a semi-permanent arrangement has been reached. 

I did a rough re-calibration of the phase tracker output, since the increased cable delay changes the degree/Hz gain. This was done by fitting a line to a slow sawtooth FM of the SRS DS345's (1Hz rate, 10kHz deviation, 30MHz carrier). This resulted in the following calibration updates

• ALSX: 19230 -> 13051 Hz/count, 3.4dB more sensitive

• ALSY: 19425 -> 12585 Hz/count, 3.8db more sensitive

Again, this is a rough calibration. Nevertheless, it is not so surprising we don't get the 50m/30m = 4.4dB increase we would expect just from the lengths; the (I presume) increased cable loss matters. Also, the loss' frequency dependance is an additional reason that the phase tracker calibration is not constant over all frequencies. 

I took spectra with the arms in IR lock, but didn't see any real improvement beyond a possible dip in the floor from 100-200Hz. This doesn't surprise me too much, however, since I don't believe that we are currently dominated by electronic noises that this gain increase would help overcome. 

Last week, Koji mentioned the ALS phase noise added due to the post-cavity table motion the arm-transmitted green beams experience before hitting the beat PD. I should estimate the size of this effect for our situation. 

CP Stat 100  sheet-covers were replaced by clean ones on open chambers BS, ITMX, ITMY and ETMY this morning.

Try to fold the sheets such way that the clean side is facing each other, so they do not accumulate dust.

[Koji Den EricG]

We inspected the {REFL, AS, POP}55 demodulators.

Short in short, we did the following changes:

- The REFL55 PD RF signal is connected to the POP55 demodulator now.
Thus, the POP55 signals should be used at the input matrix of the LSC screens for PRMI tests.

- The POP55 PD RF signal is connected to the REFL55 demodulator now.

- We jiggled the whitening gains and the whitening triggers. Whitening gains for the AS, REFL, POP PDs are set to be 9, 21, 30dB as before.
However, the signal gain may be changed. The optimal gains should be checked through the locking with the interferometer.

- Test 1

Inject 55.3MHz signal to the demodulators. Check the amplitude in the demodulated signal with DTT.
The peak height in the spectrum was calibrated to counts (i.e. it is not counts/rtHz)
We check the amplitude at the input of the input filters (e.g. C1:LSC-REFL55_I_IN1). The whitening gains are set to 0dB.
And the whitening filters were turned off.

REFL55
f_inj = 55.32961MHz -10dBm
REFL55I @999Hz  22.14 [cnt]
REFL55Q @999Hz  26.21 [cnt]

f_inj = 55.33051MHz -10dBm
REFL55I @ 99Hz  20.26 [cnt]  ~200mVpk at the analog I monitor
REFL55Q @ 99Hz  24.03 [cnt]

f_inj = 55.33060MHz -10dBm
REFL55I @8.5Hz  22.14 [cnt]
REFL55Q @8.5Hz  26.21 [cnt]

----
f_inj = 55.33051MHz -10dBm
AS55I   @ 99Hz 585.4 [cnt]
AS55Q   @ 99Hz 590.5 [cnt]   ~600mVpk at the analog Q monitor

f_inj = 55.33051MHz -10dBm
POP55I  @ 99Hz 613.9 [cnt]   ~600mVpk at the analog I monitor
POP55Q  @ 99Hz 602.2 [cnt]

We wondered why the REFL55 has such a small response. The other demodulators seems to have some daughter board. (Sigg amp?)
This maybe causing this difference.

-----

- Test 2

We injected 1kHz 1Vpk AF signal into whitening board. The peak height at 1kHz was measured.
The whitening filters/gains were set to be the same condition above.

f_inj = 1kHz 1Vpk
REFL55I 2403 cnt
REFL55Q 2374 cnt
AS55I   2374 cnt
AS55Q   2396 cnt
POP55I  2365 cnt
POP55Q  2350 cnt

So, they look identical. => The difference between REFL55 and others are in the demodulator.

With the adjustable delay line box installed in the 55MHz modulation path, I've measured the PRMI sensing matrix as a function of delay / relative phase between the 11MHz and 55MHz modulations. The relative frequency difference of 44MHz tells us that this should be cyclical after ~23nsec of delay, but losses in the delay cable change this; see Koji's elogs about the modulation cancellation setup for details. 

TL;DR: Nothing really changes, other than REFL33 optical gain. MICH/PRCL angles remain degenerate.

The results aren't so surprising. The demod angles for the 55MHz diodes don't even change, since the same 55MHz signal is used for the modulator and demodulators, so delaying it before the split should go unnoticed. Most of these measurements were made during the same lock stretch, PRCL on REFL11 I and MICH on AS55Q.

The only signals we would expect to change much are ones that have significant contriubtions from field products influenced by both modulations. None of the 1F PDs are like this, nor is REFL165. REFL33 is the odd man out, where the +44MHz field produced as a -11MHz sideband on the +55MHz sideband beats with the +11MHz sideband (and the same with the signs flipped). I made a simulation for the 40m poster at the March 2015 LVC meeting, but I don't think it ever made it to the ELOG. 

So:

Here are the results for the 0ns and 4ns cases, as an illustration of what changes (REFL33), and what doesn't (everything else). Again, these are calibrated to Volts out of the analog demod boards per meter of DoF motion. 

So, since REFL33 is the only one really changing, let's just look at it by itself:

Qualitatively, the change in magnitude looks similar to the simulation result. The demod angles fall by some roughly linear amount. The angle difference is even more stationary than predicted there, though. 

The 7.5 in-lb of Wiha seems at the upper end of torque range for a 4-40 SS screw

Wiha 28502 ordered with range 5 -10 in-lb for silver plated 4-40 screws

Do not trust the Venzo torque wrench under 2 Nm ! It miss lead me.

Recommended torque values for silver-plated fasteners are here. For aLIGO we use the guidelines in T1100066-v6, This doc is posted in 40m wiki under Mechanics also.

So, we'll use 6 in-lbs  on silver plated 18-8 stainless steel socket head cap screw 4-40 x 3/8 into SS tower bridge.

Please replace these clamp screws every time if they were tightened without a torque wrench.

Much more success tonight.  I only started my tally after I got the CARM transition to work entirely by script, and I have 6 tally marks, so I probably made the CARM to RF-only transition 7 or maybe 8 times tonight in total.  Unfortunately, I only successfully made the DARM transition to AS55 once.    From the wall striptool, counting the number of times the transmitted power went high, I had about 40 lock trials total. 

The one RF-only lock ended around 1:27am.

I think 2 things were most important in their contributions to tonight's success.  I modified the bounceRoll filters in the CARM and DARM filter banks to eat less phase.  Also, using Q's recipe as inspiration, I started engaging the AO path partway through the CARM transition which makes it much less delicate. 

Bounce roll filter

Koji and I added a ~29Hz resonant gain in the bounce roll filter several months ago, to squish some noise that we were seeing in the CARM and DARM ALS error signals.  This does a lot of the phase-eating.  I'm assuming / hoping that that peak won't be present in the CARM and DARM RF error signals.  But even if it is, we can deal with it later.  For now, that peak is not causing so much motion that I require it.  So, it's gone. 

This allowed me to move the complex zero pair from 30 Hz down to 26 Hz.  Overall I think this gained me about 10 degrees of phase at 100Hz, and moved the low end of the phase bubble down by about 10Hz. 

Prep for REFL 11 I through the CM board and CM_SLOW

In order to use Q's recipe (elog 11138), I wanted to be able to lock CARM on REFL11 using the CM_SLOW filter bank. 

I did a few sweeps through CARM resonance while holding on ALS, and determined that the REFL1 input to the CM board needed a gain of -20dB in order to match the slope of CM_SLOW_OUT to CARM_IN (ALS), leaving all of Q's other settings alone.  Q had been using a REFL1 gain of 0dB for the PRY earlier today.

I needed to flip the sign in the input matrix relative to what Q had (he was using +1 in the CM_SLOW -> CARM_B, I used -1 there).  To match this in the fast path, I flipped the polarity of the CM board (Q was using minus polarity, I am using positive).

The CM_SLOW filter bank had a gain of 0.000189733.  I assume that Q did this so that the input matrix element could be unity.  I left this number alone.  It is of the same order as the plain REFL11I->CARM input matrix element of 1e-4 from Saturday night, so it seemed fine.

During my sweeps through CARM resonance, I also saw that I needed an offset to make CM_SLOW's average about 0.  With the crazy gain number, I needed an offest of -475 in the CM_SLOW filter bank.  As I type this though, it occurs to me that I should have put this in the CM board, since the fast path will have an offset that isn't handled.  Ooops. 

Trying Q's recipe for engaging AO path

I am able to get the MC2 AO gain slider up to -10dB (-7 is also okay).  If I increase the digital CARM gain too much, I see gain peaking at about 800Hz, so something good is happening.  (That was with a CARM_B gain of 2.0 and CARM_A gain of 0.  Don't go to 2.0)

I tried once without engaging his 300:80 1/f^2 filter in the CM_SLOW filter banks to start stepping up the CM REFL1 and MC AO gains together, but I only made it 2 steps of 1dB each before I lost lock. 

I tried once or twice turning on that 300:80 filter that Q said over the phone really helped his PRY locking, but it causes loop oscillations in CARM.  Also, I forgot to turn it off for ~45 minutes, and it caused several locklosses.  Ooops.  Anyhow, this isn't the right filter for this situation.

AS55 whitening problem

Twice I tried turning on the AS55 whitening.  Once, I was only partly transitioned from ALSdiff to AS55, the other time was the one time I made the full transition.  It caused the lockloss from the only RF-only lock I had tonight :(

Unfortunately I don't have the time series before the whitening filters (not _DQ-ed), but you can see a giant jump in the _ERR signals when I turn on the whitening, just before the arm power dies:

AS55whitening_lockloss_12March2015.pdf

The AS55 phase is -30, I has an offset of 28.2 and Q has an offset of 6.4.  Both have a gain of 1.  This should give us enough info to back out what the _IN1 signals looked like before I turned on the whitening if that's useful.

Other random notes

Ramp times for CARM_A, CARM_B, DARM_A and DARM_B are all 5 seconds.  This is set in the carm_cm_up script.

carm_cm_up script freezes the arm ASS before it starts the IR->ALS transition, to make it more convenient to run the ASS each lockloss.

carm_cm_up script no longer has a bunch of stuff at the bottom that we're not using.  It's all archived in the svn, but the remnants from things like variable finesse aren't actively  useful.

carm_cm_down script turns off the CM_SLOW whitening (which gets set in the up script)

carm_cm_down script clears the history of the ETM oplevs, in case they went bad (from some near divide-by-zero action?), but the watchdog isn't tripped. This clears away all the high freq crap and lets them do their job.

FSS Slow has been larger than 0.55 all night, larger than 0.6 most of the night, and larger than 0.7 for the last bit of the night.  MC seems happy.

both carm_cm_up and carm_cm_down are checked into the svn.  The up script is rev 45336 and the down script is 45337.

Some offset (maybe the fact that the fast AO path had an un-compensated offset?) is pulling the arm powers down as I make the transitions:

Recipe overview

• Lock PRMI with arms held on ALS at 3nm CARM offset.  Bring CARM offset to 0.
• Turn on CARM_B and DARM_B a little bit, then turn on their integrators
• Lower the PRCL and MICH gains a little.
• Increase the CARM_B gain a bit, then turn off FM1 for both CARM and DARM.
• Increase CARM_B gain, lowering CARM_A gain.
• Increase DARM_B gain, lowering DARM_A gain.  Now the power should definitely be stable (usually ends up around 80).
• Partly engage AO path.
  • CM board REFL1 gain = -20dB
  • CM board AO gain = 0dB
  • MC2 board AO gain starts at -32dB, stepped up to -20dB
• Increase CARM_B gain a bit
• More AO path:  MC2 board AO gain steps from -20dB to -10dB
• Increase CARM_B gain to 1.5, turn CARM_A gain to zero
• CM_SLOW whitening on

After that, I by-hand made the DARM transition on the 6th successful scripted CARM transition, and tried to script what I did, although I was never able to complete the DARM transition again.  So, starting where the recipe left off above,

• Turn off DARM's FM2 boost to win some more phase margin.
• Increase DARM_B gain to 0.5, lower DARM_A gain to 0.

Since DARM doesn't have an analog fast path, it is stuck in the delicate filter situation.  I think that I should probably start using the UGF servo once the arm power is stable so that DARM stays in the middle of its phase bubble.

Rather than typing out the details of the recipe, I am attaching the up script.

Since the DARM_OUT signal is only 500 counts_peak, I don't see why AS55 whitening needs to be switched on. Maybe in a couple weeks after the lock is robust. In any case, its much better to do the switching BEFORE you're using AS55, not after.

Summary of This Week's Activities:

6/16: LIGO Orientation; First Weekly Meeting; 40m tour with Jenne; Removed WFS Box Upper Panel, Inserted Cable, Reinstalled panel

6/17: Read Chapter 1 of Control Systems Book; LIGO Safety Meeting; Koji's Talk about PDH Techniques, Fabry-Perot Cavities, and Sensing/Control; Meeting w/ Nancy and Koji

6/18: LIGO Talk Part II; Glossed over "LASERS" book; Read Control Systems Book Chapter 2; Literary Discussion Circle

6/21: Modecleaner Matrix Discussion with Nancy; Suggested Strategy: construct row-by-row with perturbations to each d.f. --> Leads to some questions on how to experimentally do this.

6/22: Learned Simulink; Learned some Terminal from Joe and Jenne; LIGO Meeting; Rana's Talk; Christian's Talk; Simulink Intro Tutorial

6/23 (morning): Simulink Controls Tutorial; Successfully got a preliminary feedback loop working (hooray for small accomplishments!)

Outlook for the Upcoming Week:

Tutorials (in order of priority): Finish Simulink Tutorials, Work through COMSOL Tutorials

Reading (in order of priority): Jenne's SURF Paper, Controls Book, COMSOL documentation, Lasers by Siegman.

Work: Primarily COMSOL-related and pre-discussed with Rana

Summary of this Week's Activities:

6/23: LIGO Safety Tour; Simulink Controls Tutorial

6/24: Simulink Diagram for Feedback Loop; Constructed Pendulum Transfer Function; Discussion with Dr. Weinstein

6/25: Prepare for pump-down of vacuum chamber; crane broken due to locking failure; worked through COMSOL tutorials

6/28: Ran through Python Tutorials; Began learning about Terminal

6/29: Wrote Progress Report 1 First Draft

6/30: Began editing Progress Report 1

Summary of this Week's Activities:

6/30: 2nd and 3rd drafts of Progress Report

7/1: 4th draft and final drafts of Progress Report; submitted to SFP

7/5: Began working through busbar COMSOL example

7/6: LIGO meeting and lecture; meeting with Koji and Steve to find drawing of stacks; read through Giaime's thesis, Chapter 2 as well as two other relevant papers.

7/7: Continued working on busbar in COMSOL; should finish this as well as get good headway on stack design by the end of the day.

[JC, Yuta]

We have found that MC1 coils are causing 60 Hz noise.
Tripping watchdogs for MC1 coils reduced 60 Hz noise seen in YARM by a factor of 100.

Method:
 - Locked YARM with POY11 and measured YARM sensitivity to use it for 60 Hz frequency noise monitor0.187493**0.5
 - Tripped MC1, MC2, MC3 coil output watchdogs to see if they are causing this 60 Hz frequency noise. IMC WFS were turned off.

Result:
 - Attachment #1 is YARM sensitivity and MC_F in Hz with MC1,2,3 untripped (dotted) and MC1 tripped (solid).

YARM (PSL locked vs Yarm), MC1,2,3 untripped: 6.0e2 Hz/rtHz (2.6e2 Hz RMS)
MC_F (sum of noises in IMC loop), MC1,2,3 untripped: 4.8e4 Hz/rtHz (2.1e4 Hz RMS)
YARM (PSL locked vs Yarm), MC1 tripped: 6.6e0 Hz/rtHz (2.9e0 Hz RMS) -- reduced by a factor of 100
MC_F (sum of noises in IMC loop), MC1 tripped: 4.7e4 Hz/rtHz (2.0e4 Hz RMS)

 - We have also tried tripping MC2 and MC3 coils, but they didn't make much difference.
 - Untripping only one of MC1 face coils created 60 Hz frequency noise, so all the face coils seem to have the same level of 60 Hz noise.

Next:
 - Inspect around MC1 coil driver

[Anchal, Yuta]

We think we have narrowed down the source of 60 Hz noise to one fo the following possibilities:

• Ground loop present along the MC1 suspension damping loop
• 60 Hz DAC noise on inputs of MC1 coil driver
• 60 Hz noise injected at dewhitening board before the dewhitening filter

The second and third cases are unlikely because we see 60 Hz noise present only in MC1 coils, not MC3 coils while they both share the same connection from DAC to SOS dewhitening filter boards as they share the SOS dewhitening board D000316-A. So it is unlikely that only the MC1 channels have this noise while the MC3 channels do not.

This inference was made from following observations:

Note: Turning ON analog dewhitenign on MC1 coil is done by turning off FM9 switch which is the simulated digital dewhitening filter. Also note that theanalog dewhitening filter has an attenuation of 30 dB at 60 Hz.

MC1 has an unconvetional setup where the satellite amplifier is from the new generation while the coil driver and dewhitening boards are from the old generation. The new generation satellite amplifiers sen PD signal through differential ended signals but the old generation PD whitening interface expects single ended inputs, so we ahve been using PD monitor outputs from the satellite amplifier which connects the ground of the two boards to each other. Maybe this is the reason for the ground loop.

[Paco, Yehonathan, Yuta]

Since we have installed BH44, we are seeing side lobes of 60 Hz + harmonics in AS55, REFL55, BH55, BH44, preventing us from locking FPMI BHD (40m/17405).

BH55 RF amp removed:
  - We have noticed that the side lobes are there in BH55 (but not in BH44) when LO-ITMX single bounce is fringing (ETMs and ITMY mis-aligned).
  - Changing whitening gains and turning on/off whitening/unwhitening filters didn't help.
  - When LO-ITMX single bounce is locked with BH55, the side lobe in BH55 reduces.
  - Dithering LO1 at 11 Hz created 180 +/- 11 Hz signal, which confirms that this side lobes are from the up conversion of optic motion.
  - We thought it could be from RF saturation, so we have put a 55-67 MHz bandbass filter (SBP-60+) in between BH55 RFPD and RF amp (ZFL-1000LN+; 40m/17195). Didn't help.
  - We then removed the RF amp. This largely reduced the side lobes (but still some at 180 Hz). We could lock LO-ITMX single bounce without the RF amp, so we decided to remove it for now.

Side lobes only when one of the arms is locked:
  - When ETMs are mis-aligned, MICH fringing and BHD fringing, there are 60 Hz + harmonics, but the side lobes are not there.
  - But with Xarm is locked (ETMY, ITMY mis-aligned) or Yarm is locked (ETMX, ITMX mis-aligned), the side lobes appear in AS55, REFL55, BH55, BH44.
  - Changing whitening gains and turning on/off whitening/unwhitening filters didn't help.
  - As the error signals are normalized by TRX and TRY, we turned on/off the power normalization, but didn't help.
  - Switching 60 Hz comb in BS, ITMX, ITMY, ETMX, ETMY suspension damping didn't help.

POY11 Investigations:
  - When ETMs are mis-alined, POX11 had relatively large 60 Hz + harmonics, but almost none in POY11 (unlike other RFPDs; see Attchment #1).
  - However, when ETMY is aligned and Yarm is loked with POY11, the side lobe grows in POY11.
  - Changing the feedback point from ETMY to ITMY or MC2 didn't help.
  - We have unplugged the IQ demod board for BH44 from the eurorack (without removing the cables) and removed the fuse for the power supply of the RF amp for 44 MHz generation (40m/17401), but these also didn't help.
  - We have also tried locking Yarm with REFL55(= ~2 x POY11), BH55(= ~10 x POY11), ALSY(= ~2000 x POY11), but the side lobes were always there.
  
Next:
  - Disconnect cables in BH44 to open possible ground loops made during BH44 installation (especially 44 MHz generation part??).
  - Check if the noise was there before BH44 installation using past data.

Quote from #6126

As shown in the noise budget below, the 60 Hz line noise currently dominates the arm displacement.

 The 60 Hz line noise has gone away.

So far different measurements are consistent with the hypothesis that the 60 Hz noise is from PSL frequency noise (40m/17423).
We have done several measurements in REFL55 and POX/POY to show this hypothesis, and all are consistent with the frequency noise hypothesis.
However, 60 Hz noise in the IMC error point seems too small to explain the 60 Hz noise in DARM.
 

[Koji, Paco, Yuta]

REFL55 attenuation experiment:
 - To check if the 60 Hz is in the light or not, we have compared the spectrum of REFL55_I_IN1 with different ND fiters in front of REFL55 RF PD.
 - Attachment #1 shows the result. Spectrum was taken when both arms are locked indivitually using POX and POY, feeding back to ETMs, MICH freely swinging. Red curve is nominal, blue is with OD0.5, and green is with OD1.
 - Attenuation in 60 Hz noise and side lobes are consistent with OD filter attenuation, which suggets that the noise is from the light.
 - Note that having side lobes is natural, as MICH is fringing (sorry for confusing plot). However, if the side lobes come from the RF saturation, we expect side lobes to decrease more than OD filter attenuation, but this was not the case.

Phase measurements between POX and POY:
 - When both arms are locked indivitually using POX and POY, feeding back to ETMs, transfer function from POX to POY had gain of ~1 and the phase of -10 deg (Attachment #2).
 - With PSL shutter is closed, transfer function from POX to POY had gain of ~0.1 and the phase of -100 deg, with lower coherence (Attachment #3).
 - These also support that 60 Hz noise in POX and POY when the arms are locked are from common origin, such as frequency noise.

[Michael, Paco, Yuta]

IMC error point measurement:
 - Attachment #4 shows the IMC Servo Board configuration we used for the all three measurements below.

For this measurement we took TP1A (from MC Servo board) and buffered it with a battery powered SR560 (DC coupled, low noise, gain x1) before connecting it to the single ended A1 channel on a SR785. The noise level was set to -42 dBVpk, and three different noise spectra were acquired:

• In Blue, the IMC is locked
• In Orange, IMC is unlocked by closing PSL Shutter (dark)
• In green, IMC is unlocked by closing PSL Shutter and the Servo board IN1 is disabled.

The estimated (in loop) line noise (60 Hz) levels are 70 uV/rtHz, which using the calibration 13 kHz/Vrms (from 40m/14691) amounts to 0.9 Hz/rtHz of (supressed) frequency noise at IMC Error point.

This number (0.9 Hz/rtHz) in terms of displacement corresponds to 1.28e-15 m/rtHz. The measured DARM noise (2e-10 m/rtHz @ 60 Hz from 40m/17414) is not accounted for by this amount.

Next:
  - Check the IMC error signal calibration
  - Measure the calibrated out-of-loop frequency noise using various signals (POX, POY, REFL55, AS55 with single arm, ALSBEAT, PMC_CTRL)

Timeline (as far as written in the elog):
 - Dec 20: FPMI BHD locked using BH55 (40m/17367).
 - Dec 21: FPMI RF locked, but not BHD, DARM noise 1e-11 m/rtHz @ 60 Hz (40m/17369).
 - Jan 10-11: AS WFS boards testing at 1X2 (40m/17391, 40m/17393).
 - Jan 11: FPMI BHD locked using BH55, DARM noise 2e-11 m/rtHz @ 60 Hz (40m/17392).
 - Jan 13 2pm: FPMI BHD locked using BH55, DARM noise 2e-11 m/rtHz @ 60 Hz (40m/17399).
  - After measuring the sensing matrix etc., LO_PHASE locking became unstable and FPMI BHD could not be recovered (I thought something similar to Dec 21 is happening).
 - Jan 13 6pm: FPMI BHD locked using BH55 recovered. DARM noise 2e-11 m/rtHz @ 60 Hz. Discovered that the 60 Hz noise is higher when LO_PHASE locking is unstable (40m/17400).
 - Jan 17: BH44 hardware/software installed. Found IQ demod board needs tuning (40m/17401).
 - Jan 18: Tuned IQ demod board for BH44 installed (40m/17402).
 - Jan 19: BH44 RF PD placed and connected to IQ demod board. FPMI RF locked, LO_PHASE locked with BH44, but found 60 Hz noise everywhere (40m/17405).
 - Jan 24: FPMI RF locked (with CARM locked with POX11+POY11 instead of REFL55_I). DARM noise 2e-10 m/rtHz @ 60 Hz (40m/17414).
 - Jan 24: AS WFS boards mounted in 1X2 (40m/17416).
 - Jan 25: Isolating BH44 setup didn't help (40m/17423).
 - Jan 26: Fixed tripping of -5V supply in 1X1 (40m/17425).
 - Jan 27: IMC error point measurements at 1X2 (this elog).

[Paco, Yuta]

We estimated the frequency noise of IMC output beam at 60 Hz using different methods to see if they are consistent.
They are not inconsistent, but seems hard to explain by an easy single dominating noise source (multiple noise sources at similar noise level?).

IMC suspension damping:
 - We checked that 60 Hz comb filters are all on for all OSEM sensors of MC1, MC2, MC3 (Attachment #1), and they all have comb(60,30,-40,3), which is 60 Hz comb filter of Q=30, -40 dB, 3 harmonics.

Revisiting IMC error point calibrations:

 - We revisited this calibration in 40m/17431. First, 0.9 Hz/rtHz corresponds to 1.3e-13 m/rtHz, as L/nu = 40 m / 282 THz = 1.4e-13 m/Hz.
 - Also, we need to add a loop correction. MC servo board settings when we took this data was as follows:
   - +4 dB in IN1
   - 40 Hz pole, 4000 Hz zero filter was on
   - 0 boost
  assuming 1/f around UGF of 200 kHz (40m/17009), and 1/f^2 between 40-4000 Hz, openloop gain at 60 Hz will be (4e3/60)**2*(200e3/4e3)=2e5. So, the estimated frequency noise at the output of IMC in terms of arm length is 1.3e-13 m/rtHz * (1+G) = 2.6e-8 m/rtHz (or 1.8e-8 m RMS considering 0.5 Hz bandwith).
  - Noise measured with the same condition but PSL shutter closed was 7 uV/rtHz at 60 Hz (40m/17431). This correspond to 1.3e-14 m/rtHz (or 9.2e-15 m RMS), which is an estimated dark noise.

Measuring frequency noise using arms:
 - We then proceeded to measure frequency noise using arms locked with POX11 and POY11. Attachment #2 and #3 is calibrated XARM and YARM noise using the error signals and feedback signals. For both, it is 1e-10 m/rtHz at 60 Hz (or 4.3e-11 m RMS considering 0.187493 Hz bandwidth). And this is more than x10 higher than what we have measured in August 2022 (dotted lines).
 - MC_F calibrated using 1.4e-13 m/Hz reads 7.1e-9 m/rtHz at 60 Hz (or 3.1e-9 m RMS considering 0.187493 Hz bandwidth).
 - Noise measured at DARM in FPMI locked with RF (but CARM with POX11+POY11, as 60 Hz was too much to switch to REFL55_I) was 3e-10 m/rtHz at 60 Hz (or 1.8e-10 m RMS considering 0.374994 Hz bandwidth) (Attachment #5), which is roughly the same as past measurements (40m/17414).
 - To check if MC_F is calibrated correctly, we injected a line at 57 Hz with 3000 counts in amplitude into MC2. Using MC2 actuation efficiency -14.17e-9 /f^2 m/counts in arm length (40m/16978), this should give

14.17e-9/(60**2)*3000 = 1.2e-8 m -> 0.93e-8 m RMS

 in XARM length noise. RMS value of YARM calibrated spectra reads 1.1e-8 m (Attachment #4), which is consistent within ~20%, so MC_F calibration is OK. Note that MC_F at 60 Hz are at the same level in August 2022 (green curves).

Summary of frequency noise measurements at 60 Hz:
 - 1.8e-8 m RMS as measured at IMC error point TP1A
    This gives you total of IMC length noise, error point noise, PSL free run noise, feedback noise.
    Estimated dark noise at error point TP1A is 9.2e-15 m RMS, and is small.
    Calibration might be wrong, as this rely on IMC loop gain estimate and error signal calibration of 13kHz/V a while ago in 2018 (from 40m/14691, which is from 40m/13696), which might not be true for TP1A at 60 Hz (note that there is a 40 Hz/4000 Hz filter).
 - 3.1e-9 m RMS as measured at MC_F
    This gives you total of IMC length noise, error point noise, PSL free run noise, but the noise injected at feedback point before MC_F is suppressed by ~2e5.
    As estimated dark noise is much less, it is IMC length noise, PSL free run noise or noise injected after MC_F.
    Note that typical NPRO free run noise at 60 Hz is 1e4/60 Hz/rtHz * 1.4e-13 m/Hz = 2.3e-11 m/rtHz, and is small, but we might be having large NPRO noise.
 - 4.3e-11 m RMS as measured using XARM and YARM
     This gives you total of IMC length noise, error point noise, but PSL free run noise and feedback noise are suppresed by ~2e5.
     But this also includes noise injected in XARM and YARM loops.
     If this is mainly from PSL free run noise or feedback noise, we expect 3.1e-9 m RMS/2e5 = 1.6e-14 m RMS, so it doesn't explain 4.3e-11 m RMS.
     If this is mainly from IMC length noise, this should be equal to frequency noise measured at MC_F, but MC_F is higher by nearly two orders of magnitude.
     Noise in POX11 or POY11 are smaller by a factor of more than 100 when dark (see 40m/17431), so contribution from dark noise of POX11 and POY11 at 60 Hz to XARM/YARM noise is negligible.
     These mean that the noise might be from combination of IMC and PSL. (For example, if noise injected at error point is 9.2e-15 m RMS, IMC length noise is 4.3e-11 m RMS, PSL free run noise is 3.1e-9 m RMS, and noise injected at feedback point is 1.8e-8 m RMS, it explains all the measurements so far.)
 - 1.8e-10 m RMS as measured using FPMI DARM
     Frequency noise in DARM should be suppressed by common mode rejection, but it is actually x3 higher than what we see in XARM and YARM.
     There might be extra noise from FPMI loops (note that CARM is controlled by POX11+POY11 in this measurement).

Next:
  - Check IMC error point calibration (is 13 kHz/V correct?) by driving MC2 at around 60 Hz (but not at 60 Hz) by known amount
  - Measure frequency noise at IN1 of MC servo board to avoid 40 Hz/4000 Hz filter
  - Check what exactly are we measuring at MC_F. Are there possibility of additional noise for MC_F, which is not fed back to laser frequency?
  - Drive MC2 at around 60 Hz (but not at 60 Hz) to see if MC_F and X/YARM spectra matches
  - Estimate IMC length noise from MC OSEMs
  - Touch electronics around 1X2 to see if 60 Hz at IMC error point changes (monitor the live spectrum!)

For the loop calculation, don't you have to consider the IMC cavity pole? What about the analog filter on the output of the HV driver for the laser PZT?

[Anchal, Yuta]

We have measured OLTF of IMC loop, and revisited IMC error point calibration again.
Also, we have tried to break the ground loop between MC servo board and TTFSS, but didn't help.

IMC OLTF measurement:
 - IMC OLTF was measured using SR785 at TP1A and TP1B. MC servo board settings are the following.
   - +4 dB in IN1
   - 40 Hz pole, 4000 Hz zero filter was on
   - 0 boost
 - Eye-ball fit of OLTF gives zeros at [30e3,30e3] Hz, poles at [40,3e3,3e3] Hz (Attachment #1). 40 Hz pole is from 40:4000 Hz fiter in MC servo board and 4kHz zero is compensated by IMC cavity pole (~ 3.79 kHz). We are not sure where two 3k:30k are from.
 - Anyway, eye-ball fit gives OLTF gain of 1.7e5 at 60 Hz, which is accidentally roughly the same as previous estimate (40m/17446).

Revisiting IMC error point calibrations:
 - We realized that error signal calibration of 13kHz/V a while ago in 2018 (from 40m/14691, which is from 40m/13696) is a calibration for IN1.
 - So, 70 uV/rtHz at 60 Hz at TP1A corresponds to 70 uV/rtHz / 4dB / (4e3/60) * 13kHz/V = 0.009 Hz/rtHz, which corresponds to 1.2e-15 m/rtHz.
 - The estimated frequency noise at the output of IMC in terms of arm length is 1.2e-15 m/Hz * (1+G) = 2.0e-10 m/rtHz (or 1.4e-10 m RMS considering 0.5 Hz bandwith).
 - Noise measured with the same condition but PSL shutter closed was 7 uV/rtHz at 60 Hz (40m/17431). This correspond to 1.2e-16 m/rtHz (or 8.5e-17 m RMS), which is an estimated dark noise.

Summary of frequency noise measurements at 60 Hz:
 - 1.4e-10 m RMS (or 1.0e3 Hz RMS) as measured at TP1A (estimate of unsuppressed noise difference between IMC and PSL)
  - This being smaller than MC_F measurement is strange, as this should be an estimate of total unsuppressed noise (if 60 Hz noise is coherently cancelling each other, this can be explained).
 - 3.1e-9 m RMS (or 2.2e4 Hz RMS) as measured at MC_F
 - 4.3e-11 m RMS (or 3.1e2 Hz RMS)  as measured using XARM and YARM
 - 1.8e-10 m RMS as measured using FPMI DARM

Buffering MC servo board output to TTFSS:
 - We have inserted a battery-powered SR560 in between MC servo board output to TTFSS, trying to break the possible ground loop between 1X2 rack and PSL.
 - To do this, we had to lower IN1 gain to -6dB, to avoid saturation of SR560.
 - This didn't make any difference in MC_F or POY during YARM lock.

[Anchal, Yuta]

Yesterday, we have measured the frequency noise of PSL with IMC locked/unlocked using ALS BEATX/Y to narrow down where the 60 Hz is coming from.
All the measurements so far is consistent with a hypothesis that 60 Hz noise injected after MC_F is picked-off (it could be from MC_F DAQ readout or something in the IMC loop).

Method:
 - Measured YARM noise spectra when YARM is locked with POY11 to measure the frequency noise with respect to YARM, and compared with MC_F
 - Measured ALS BEATX and BEATY spectra when PSL is free running and when IMC is locked. Here, when "PSL is free running" is done with PSL shutter closed, but all the cables remained the same and FSS loop was in "down" state. Shutters at both ends were closed, and PZT inputs to AUX lasers were terminated to avoid noise injection from PDH locking with dark noise (this was necessary to reduce noise in BEATY).

Result:
 - Attachment #1 is YARM noise calibrated into Hz, and Attachment #2 is BEATX and BEATY spectra with PSL free running (solid lines) and IMC locked (dotted lines). Below are summary of noise level at 60 Hz (RMS is calculated using a bandwidth of 0.187493 Hz)

YARM (PSL locked vs Yarm): 6.5e2 Hz/rtHz (2.8e2 Hz RMS)
MC_F (sum of noises in IMC loop): 4.9e4 Hz/rtHz (2.2e4 Hz RMS)
BEATX free (PSL free vs Xend free): 3.3e3 Hz/rtHz (1.4e3 Hz RMS)
BEATX locked (PSL locked vs Xend free): 8.8e2 Hz/rtHz (3.8e2 Hz RMS)
BEATY free (PSL free vs Yend free): 1.6e4 Hz/rtHz (6.9e3 Hz RMS)
BEATY locked (PSL locked vs Yend free): 1.5e4 Hz/rtHz (6.5e3 Hz RMS)

Discussion:
 - "BEATX locked" measurement suggests that PSL locked to IMC (and Xend free) has noise less than 3.8e2 Hz RMS. This is roughly consistent with YARM measurement of frequency noise, and suggests that Yarm is stable enough to measure the PSL locked frequency noise.
 - "BEATX free" measurement suggests that PSL free run (with cables connected) has noise of 1.4e3 Hz RMS (note that Xend free is less than 3.8e2 Hz RMS).
 - MC_F measurement is the sum of noises in IMC loop, including IMC length noise + noise injected at error point (3.8e2 Hz RMS), PSL free run noise (1.4e3 Hz RMS), noise injected at feedback. Therefore, this suggests that 2.2e4 Hz RMS we see in MC_F is from noise injected after MC_F pickoff point (or in the MC_F DAQ readout).
 - BEATY having large 60 Hz noise probably comes from noise in the beat measurement.

Next:
 - Use BEATX to monitor 60 Hz noise.
 - Try terminating PZT input to see if 60 Hz noise reduces. Try different gains at different point of MC servo board and TTFSS when IMC is unlocked to see where exactly 60 Hz noise is coming from.

[Anchal, Yuta]

Measurements yesterday (40m/17458) suggested that 60 Hz noise is injected after MC_F is picked-off.
So, we terminated PSL PZT input at several points to see where 60 Hz noise is injected.
It seems like the 60 Hz frequncy noise we see in MC_F is from TTFSS box, but the 60 Hz noise we see in YARM is not limited by this.

The 60 Hz noise we see in YARM is probably limited by IMC length noise.

Method:
 - We terminated PZT input to the PSL laser at various points one by one and monitored 60 Hz frequency noise using BEATX. PSL shutter was closed and IMC was not locked.

Result:
 - Below is the result at 60 Hz (RMS is calculated using a bandwidth of 0.187493 Hz)

Reference from 40m/17458, YARM (PSL locked vs Yarm): 6.5e2 Hz/rtHz (2.8e2 Hz RMS)
Reference from 40m/17458, MC_F (sum of noises in IMC loop): 4.9e4 Hz/rtHz (2.2e4 Hz RMS)
MC_F when PSL shutter is closed but MC servo board configuration at IMC locked state: 2.7e2 Hz/rtHz (1.2e2 Hz RMS) -- this gives IMC loop gain enhanced sensing noise

BEATX free (PSL free vs Xend free): 3.5e3 Hz/rtHz (1.5e3 Hz RMS) -- consistent with previous measurements
With PZT input to NPRO terminated (Attachment #1): 8.1e2 Hz/rtHz (3.5e2 Hz RMS)
Connected a terminated small box (we see in Attachment #1) before NPRO PZT: 6.3e2 Hz/rtHz (2.7e2 Hz RMS)
Connected input terminated Thorlabs PZT driver (MDT694): 5.9e2 Hz/rtHz (2.6e2 Hz RMS)
Connected input terminated summing amp (Attachment#2): 4.4e2 Hz/rtHz (1.9e2 Hz RMS)
Connected input terminated TTFSS (C1:PSL-FSS_MGAIN=-10dB, C1:PSL-FSS_FASTGAIN=-10dB): 3.9e3 Hz/rtHz (1.7e3 Hz RMS) -- consistent with "BEATX free (PSL free vs Xend free)" measurement
Connected input terminated TTFSS (C1:PSL-FSS_MGAIN=-10dB, C1:PSL-FSS_FASTGAIN=+10dB): 7.6e3 Hz/rtHz (3.3e3 Hz RMS)
Connected input terminated TTFSS (C1:PSL-FSS_MGAIN=+4dB, C1:PSL-FSS_FASTGAIN=+19dB): 2.9e4 Hz/rtHz (1.3e4 Hz RMS) -- Nominal gains when IMC is locked; consistent with "MC_F" measurement 40m/17458
Connected input terminated TTFSS (C1:PSL-FSS_MGAIN=+19dB, C1:PSL-FSS_FASTGAIN=+4dB): 1.1e4 Hz/rtHz (4.8e3 Hz RMS)

Discussion:
 - Connecting TTFSS increased 60 Hz frequency noise, which suggests that TTFSS is creating this 60 Hz frequency noise.
 - Setting TTFSS gains to nominal gains to IMC locked, 60 Hz frequency noise matched with frequency noise measurement using MC_F. This quantitatively supports that TTFSS is creating this 60 Hz frequency noise.
 - Increasing C1:PSL-FSS_MGAIN and reducing C1:PSL-FSS_FASTGAIN reduced 60 Hz frequency nosie. This means that some portion of 60 Hz noise is from between these two gains.
 - Note that having 60 Hz noise in TTFSS does not necessarily mean that our YARM noise is limited by this, because IMC loop suppresses the TTFSS noise. Assuming all 1.3e4 Hz RMS is all from TTFSS noise, it is suppressed to less than 1.3e4 Hz RMS/2e5 = 6.5e-2 Hz RMS (where 2e5 is IMC loop gain without super boosts, but it is actually higher with them) as frequency noise we see in YARM. YARM noise is measured to be 6.5e2 Hz/rtHz (2.8e2 Hz RMS), so it is not limited by TTFSS noise.
 - Also dark noise measured at MC_F (1.2e2 Hz RMS) tells you that the dark noise is not limiting the frequency noise we see in YARM.

Touching various parts around TTFSS:
 - We moved on to touch various parts around TTFSS to see if 60 Hz noise reduces in MC_F. We removed unused cables around TTFSS interface, touched power cables into TTFSS (both at TTFSS interface in the rack and TTFSS box on PSL table), BNC cables into TTFSS, disconnected slow controls, tried to avoid grounding of cables going into EOM (there is a small box that sums FSS feedback signal and 33.5 MHz; Attachment #3), but 60 Hz noise we see in MC_F didn't change significantly.

Next:
 - Check grounding situation around TTFSS box.
 - Check IMC length noise and error point noise by monitoring BEATX.
 - Check coil drivers for MC1, MC2, MC3 by disconnecting drivers while IMC is locked.
 - Try feeding back IMC servo also to MC2 with 60 Hz resonant gain to cancel 60 Hz noise


Note added at 23:50 to clarify:
nIMC : IMC length noise in frequency
nPSL: PSL free run noise in frequency
ne: sensing noise in frequency
nf: feedback noise in frequency
G: IMC loop gain (estimated to be 2e5 at 60 Hz without boosts)

MC_F = G/(1+G) * (nIMC + nPSL + ne + nf) + [noises in MC_F DAQ]
 = 2.2e4 Hz RMS
 MC_F when dark, MC servo nominal gain = G * ne
 = 1.2e2 Hz RMS
PSL frequency noise after IMC lock = G/(1+G) * (nIMC + ne) + 1/(1+G) * (nPSL + nf)
YARM = [PSL frequency noise after IMC lock] + [noises from YARM loop]
 = 2.8e2 Hz RMS
BEATX when PSL is free run, TTFSS low gain connected = nPSL + [noises from Xend AUX and BEATX sensing]
 = 1.7e3 Hz RMS
BEATX when PSL is free run, TTFSS nominal gain connected = nPSL + nf + [noises from Xend AUX and BEATX sensing]
 = 2.9e4 Hz RMS
BEATX when IMC is locked = [PSL frequency noise after IMC lock] + [noises from Xend AUX and BEATX sensing]
 = 3.8e2 Hz RMS
 
So, our estimate is
ne ~ 1.2e2/G Hz RMS (small)
nPSL ~ 1.7e3 Hz RMS
nf ~ 2e4 Hz RMS (this dominates MC_F, but already suppressed enough in [PSL frequency noise after IMC lock])
[PSL frequency after IMC lock] ~ 3e2 Hz RMS (this dominates YARM and BEATX when IMC is locked)
nIMC ~ 3e2 Hz RMS (this dominates [PSL frequency noise after IMC lock])

There was a 60 Hz and 120 Hz oscillation on the green PDH photo diode output. After a long search, I could identify that

the source was a broken BNC cable which was connected to the photo diode. I exchanged that BNC cable and the 60 Hz

and 120 Hz are gone :-)

With the new cable the PD output was less noisy so that it was easier to achieve a better alignment of the light to the cavity.

The reflected power could be reduced from 40% to 30%. For perfect alignment the reflected power would be 20%.