40m QIL Cryo_Lab CTN SUS_Lab TCS_Lab OMC_Lab CRIME_Lab FEA ENG_Labs OptContFac Mariner WBEEShop
  40m Log, Page 211 of 341  Not logged in ELOG logo
ID Date Authorup Type Category Subject
  13332   Tue Sep 26 15:55:20 2017 gautamUpdateCDS40m files backup situation

Backups of the root filesystems of chiara and nodus are underway right now. I am backing them up to the 1 TB LaCie external hard drives we recently acquired.

I first initialized the drives by hooking them up to my computer and running the setup.app file. After this, plugging the drive into the respective machine and running lsblk, I was able to see the mount point of the external drive. To actually initialize the backup, I ran the following command from a tmux session called ddBackupLaCie:

sudo dd if=/dev/sda of=/dev/sdb bs=64K conv=noerror,sync

Here, /dev/sda is the disk with the root filesystem, and /dev/sdb is the external hard-drive. The installed version of dd is 8.13, and from version 8.21 onwards, there is a progress flag available, but I didn't want to go through the exercise of upgrading coreutils on multiple machines, so we just have to wait till the backup finishes.

We also wanted to do a backup of the root of FB1 - but I'm not sure if dd will work with the external hard drive, because I think it requires the backup disk size (for us, 1TB) to be >= origin disk size (which on FB1, according to df -h, is 2TB). Unsure why the root filesystem of FB is so big, I'm checking with Jamie what we expect it to be. Anyways we have also acquired 2TB HGST SATA drives, which I will use if the LaCie disks aren't an option.

 

  13333   Tue Sep 26 19:10:13 2017 gautamUpdateALSFiber ALS setup neatened

[steve, gautam]

The Fiber ALS box has been installed on the existing shelf on the PSL table. We had to re-arrange some existing cabling to make this possible, but the end result seems okay (to me). The box lid was also re-installed.

Some stuff that still needs to be fixed:

  1. Power supply to ZHL amplifiers - it is coming from a table-top DC supply currently, we should hook these up to the Sorensens.
  2. We should probably extend the corrugated fiber protection tubing for the three fibers all the way up to the shelf. 

Beat spectrum post changes to follow.

Quote:

Is it better to mount the box in the PSL under the existing shelf, or in a nearby PSL rack?

Quote:

 

Further characterization needs to be done, but the results of this test are encouraging. If we are able to get this kind of out of loop ALS noise with the IR beat, perhaps we can avoid having to frequently fine-tune the green beat alignment on the PSL table. It would also be ideal to mount this whole 1U setup in an electronics rack instead of leaving it on the PSL table

 

 

Attachment 1: IMG_7605.JPG
IMG_7605.JPG
  13335   Wed Sep 27 00:20:19 2017 gautamUpdateALSMore AM sweeps

Attachment #1: Result of AM sweeps with EX laser crystal at nominal operating temperature ~ 31.75 C.

Attachment #2: Tarball of data for Attachment #1.

Attachment #3: Result of AM sweeps with EX laser crystal at higher operating temperature ~ 40.95 C.

Attachment #4: Tarball of data for Attachment #2.


Remarks:

  • Confirmed that PDA 55 is in the "0dB" setting - the actual dial is unmarked, and has 5 states. I guessed that the left-most one is 0dB, and checked that if I twiddled the dial by one state to the right, the DC level on the scope increased by 10dB as advertized. Didn't check all the states.
  • DC level is ~2.3V on a high-impedance scope. So it will be ~1.15V to a 50ohm load, which is what the DC block is. The inverse of this value is used to calibrate the vertical axis of the TF measurement to RIN/V.
  • Input R (split RF source signal) attenuation: 20dB. Input A (PDA55 output) attenuation: 0dB.
  • Main problem is still network hangups when trying to do many sweeps.
  • Seems to persist even when I connect the GPIB box to one of the network switches - so don't think we can blame the WiFi.
  • Need to explore possibility of speedup - takes >2hours to run ~50scans!

To-do:

  • Overlay median and uncertainty plots for the two temp. settings. There is a visible diference in both the locations and depths/heights of various notches/peaks in the AM profile.
  • Repeat test with a fast focusing lens to focus the beam more tightly on the PD active area to confirm that the measured AM is indeed due to the PZT drive and not from beam-jitter (presently, spot diameter is ~0.5x active area diameter, to eye).
  • Get the PM data.
  • Depending on what the PM data looks like, do a more fine-grained scan around some promising AM notches / PM peaks.
Attachment 1: TFAG4395A_26-09-2017_202344_FourSquare.pdf
TFAG4395A_26-09-2017_202344_FourSquare.pdf
Attachment 2: lowTemp.tgz
Attachment 3: TFAG4395A_26-09-2017_231630_FourSquare.pdf
TFAG4395A_26-09-2017_231630_FourSquare.pdf
Attachment 4: highTemp.tgz
  13336   Wed Sep 27 22:25:21 2017 gautamUpdateLSCDAC noise measurement (again)

Attachment #1: Summary of results of measurements made on Friday. There is a lot in this plot, here is a breakdown:

  • I drove the excitation points of the coil output filter banks with raw time-series data downloaded during a DRMI lock with pyawg. Today during the meeting, Rana pointed out that we could just acquire median (as opposed to mean since the former is more immune to glitches during the averaging process) spectra during a lock, and then do the ifft in python to generate time series data for pyawg. Another advantage of doing it this way is that we don't need to store a large (~200MB in my case) file of 16k data for numerous channels. But since I already had this file, I decided against changing the methodology for this round of tests. Time series plots of the signals do not show any large glitches.
  • The SR785 was used in dual channel mode to acquire spectra from 2 coil driver outputs simultaneously, in the interest of saving time. Input range was set to -32dbVpk, AC coupled, which was the smallest value that worked for the given signal profile. Spectra were taken from DC-200Hz, with 801 points, 25 averages. The DB15 output of the coil driver board was connected to a DB15 breakout board, and then a BNC->Pomona mini-grabber adapter was used to connect to the SR785 input. The newly acquired linear power supplies for the GPIB box mean that spurious 60Hz harmonics were not present. 
  • Initially, I had planned to enable various bandstops from 20Hz-200Hz, to get a more complete profile of the noise. But in the end, I only used two elliptic bandstops (6th order, 60dB stopband attenuation): 60-90Hz, for which data is plotted in red and 90-200Hz, for which data is plotted in green
  • I've used the same noise model as I used here, plotted in dashed grey (summed with SR785 noise at the above input range, with input terminated via 50ohm terminator) - but had to tweak the parameters to get the curve to line up with the measurement. It looks like there is considerable variation between DAC channels, and certainly between the ITMX channels and the BS channels as groups.
  • I took the measurement in two conditions - with the coil de-whitening off (left column) and coil de-whitening on (right column). Note that the input to the excitation was acquired at the IN1 of the relevant filter bank, and since the de-whitening happens downstream of this, we don't have to do anything special.
  • In the right column, I have also plotted the LISO modelled noise, which was shown to match well with the measured curve, admittedly only for one channel (for the coil driver alone, so I am not taking into account the noise of the de-whitening board - I will fix this once I dig up that data).

Some remarks:

  1. According to this measurement, the de-whitening filters are the same on the ITMX channels and BS channels. So I don't understand the difference in the right column for BS and ITMX channels.
  2. While there is considerable variation between channels and also between ITMX and BS, there is certainly >6dB of reduction in the DAC noise when the de-whitening is engaged. However, no improvement was seen in the MICH error signal spectrum between 60-300Hz. So we have to continue to investigate other noises that can explain the noise in that band.
  3. Also, the realized improvement in DAC noise by turning on the coil de-whitening seems marginal - the low pass has gain of ~-80dB at 100Hz, but we seem to be hitting some sort of electronics noise in all channels at the level of ~100nV/rtHz (assuming the actual DAC noise doesn't degrade significantly when the digital simulated de-whitening filter is engaged).
  4. It remains to do the test for the ITMY channels.
  5. It would be useful to visualize the incoherent sum of all these channels - this is what should go into the MICH displacement NB. To be added.
  6. I'm currently loading pyawg from my user directory. Need to figure out a place to put this and add it to $PATH.

Data + code for this plot will be attached later.

Attachment 1: coilNoises.pdf
coilNoises.pdf
  13337   Wed Sep 27 23:44:45 2017 gautamUpdateALSProposed PM measurement setup

Attachment #1 is a sketch of the proposed setup to measure the PM response of the EX NPRO. Previously, this measurement was done via PLL. In this approach, we will need to calibrate the DFD output into units of phase, in order to calibrate the transfer function measurement into rad/V. The idea is to repeat the same measurement technique used for the AM - take ~50 1 average measurements with the AG4395, and look at the statistics. 

Some more notes:

  • Delay line box is passive, just contains a length of cable.
  • IQ Demodulation is done using an aLIGO 1U chassis unit, with the actual demod board electronics being D0902745
  • The RF beatnote amplitude out of the IR beat PD is ~ -8dBm.
  • The ZHL-3A amplifiers have gain of 24dB, so the amplified beat should be ~16dBm
  • At the LSC rack, the amplified beat is split into two - one path goes to the LO input of D0902745 (so at most 13dBm), the other goes through the delay line.
  • On the demod board, the LO signal is amplified with a AP1053, rated at 10dB gain, max output of 26dBm, so the signal levels should be fine for us, even though the schematic says the nominal LO level is 10dBm - moreover, I've ignored cable losses, insertion losses etc so we should be well within spec.
  • The mixer is PE4140. The datasheet quotes LO levels of 17dBm for all the "nominal" tests, we should be within a couple of dBm of this number.
  • There is no maximum value specified for the RF input signal level to the mixer on the datasheet, but I expect it to be <10dBm.
  • We should park the beatnote around 30MHz as this should be well within the operational ranges for the various components in the signal chain.
Attachment 1: IMG_7609.JPG
IMG_7609.JPG
  13339   Thu Sep 28 10:33:46 2017 gautamUpdateCDS40m files backup situation

After consulting with Jamie, we reached the conclusion that the reason why the root of FB1 is so huge is because of the way the RAID for /frames is setup. Based on my googling, I couldn't find a way to exclude the nfs stuff while doing a backup using dd, which isn't all that surprising because dd is supposed to make an exact replica of the disk being cloned, including any empty space. So we don't have that flexibility with dd. The advantage of using dd is that if it works, we have a plug-and-play clone of the boot disk and root filesystem which we can use in the event of a hard-disk failure.

  1. One option would be to stop all the daqd processes, unmount /frames, and then do a dd backup of the true boot disk and root filesystem.
  2. Another option would be to use rsync to do the backup - this way we can selectively copy the files we want and ignore the nfs stuff. I suspect this is what we will have to do for the second layer of backup we have planned, which will be run as a daily cron job. But I don't think this approach will give us a plug-and-play replacement disk in the event of a disk failure.
  3. Third option is to use one of the 2TB HGST drives, and just do a dd backup - some of this will be /frames, but that's okay I guess.

I am trying option 3 now. dd however does requrie that the destination drive size be >= source drive size - I'm not sure if this is true for the HGST drives. lsblk suggests that the drive size is 1.8TB, while the boot disk, /dev/sda, is 2TB. Let's see if it works.

Backup of chiara is done. I checked that I could mount the external drive at /mnt and access the files. We should still do a check of trying to boot from the LaCie backup disk, need another computer for that.

nodus backup is still not complete according to the console - there is no progress indicator so we just have to wait I guess.

Quote:

Backups of the root filesystems of chiara and nodus are underway right now. I am backing them up to the 1 TB LaCie external hard drives we recently acquired.

We also wanted to do a backup of the root of FB1 - but I'm not sure if dd will work with the external hard drive, because I think it requires the backup disk size (for us, 1TB) to be >= origin disk size (which on FB1, according to df -h, is 2TB). Unsure why the root filesystem of FB is so big, I'm checking with Jamie what we expect it to be. Anyways we have also acquired 2TB HGST SATA drives, which I will use if the LaCie disks aren't an option.

 

 

  13341   Thu Sep 28 23:32:38 2017 gautamHowToCDSpyawg

I've modified the __init.py__ file located at /ligo/apps/linux-x86_64/cdsutils-480/lib/python2.7/site-packages/cdsutils/__init__.py so that you can now simply import pyawg from cdsutils. On the control room workstations, iPython is set up such that cdsutils is automatically imported as "cds". Now this import also includes the pyawg stuff. So to use some pyawg function, you would just do (for example):

exc=cds.awg.ArbitraryLoop(excChan,excit,rate=fs)

One could also explicitly do the import if cdsutils isn't automatically imported:

from cdsutils import awg

pyawg-away!


Linking this useful instructional elog from Chris here: https://nodus.ligo.caltech.edu:8081/Cryo_Lab/1748

  13342   Thu Sep 28 23:47:38 2017 gautamUpdateCDS40m files backup situation

The nodus backup too is now complete - however, I am unable to mount the backup disk anywhere. I tried on a couple of different machines (optimus, chiara and pianosa), but always get the same error:

mount: unknown filesystem type 'LVM2_member'

The disk itself is being recognized, and I can see the partitions when I run lsblk, but I can't get the disk to actually mount.

Doing a web-search, I came across a few blog posts that look like the problem can be resolved using the vgchange utility - but I am not sure what exactly this does so I am holding off on trying.

To clarify, I performed the cloning by running

sudo dd if=/dev/sda of=/dev/sdb bs=64K conv=noerror,sync

in a tmux session on nodus (as I did for chiara and FB1, latter backup is still running). 

  13343   Thu Sep 28 23:50:04 2017 gautamUpdateLSCDAC noise measurement (again)

I am running some more measurements of the DAC noise, for which I've shut down the BS watchdog. Some of the cables on the coil driver side have been disconnected.

I will restore these tomorrow.


As Rana pointed out to me, one important fact to keep in mind w.r.t. DAC noise is that it can be non-linear. So the RMS of the DAC noise in a higher frequency band (say 60-100Hz) can be affected by the RMS of the requested DAC signal in some lower frequency band (say 10-20Hz). One test to see if this hypothesis can explain the difference @100Hz between the ITMX channels and BS channels I observed a couple of days ago is to see if the noise around 100Hz becomes lower when I enable a 20-40Hz bandstop in the digital signal chain.

  13345   Fri Sep 29 11:07:16 2017 gautamUpdateCDS40m files backup situation

The FB1 dd backup process seems to have finished too - but I got the following message:

dd: error writing ‘/dev/sdc’: No space left on device
30523666+0 records in
30523665+0 records out
2000398934016 bytes (2.0 TB) copied, 50865.1 s, 39.3 MB/s

Running lsblk shows the following:

controls@fb1:~ 32$ lsblk
NAME   MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sdb      8:16   0 23.5T  0 disk
└─sdb1   8:17   0 23.5T  0 part /frames
sda      8:0    0    2T  0 disk
├─sda1   8:1    0  476M  0 part /boot
├─sda2   8:2    0 18.6G  0 part /var
├─sda3   8:3    0  8.4G  0 part [SWAP]
└─sda4   8:4    0    2T  0 part /
sdc      8:32   0  1.8T  0 disk
├─sdc1   8:33   0  476M  0 part
├─sdc2   8:34   0 18.6G  0 part
├─sdc3   8:35   0  8.4G  0 part
└─sdc4   8:36   0  1.8T  0 part

While I am able to mount /dev/sdc1, I can't mount /dev/sdc4, for which I get the error message

controls@fb1:~ 0$ sudo mount /dev/sdc4 /mnt/HGSTbackup/
mount: wrong fs type, bad option, bad superblock on /dev/sdc4,
       missing codepage or helper program, or other error
       In some cases useful info is found in syslog - try
       dmesg | tail or so.

Looking at dmesg, it looks like this error is related to the fact that we are trying to clone a 2TB disk onto a 1.8TB disk - it complains about block size exceeding device size.

So if we either have to get a larger disk (4TB?) to do the dd backup, or do the backing up some other way (e.g. unmount /frames RAID, delete everything in /frames, and then do dd, as Jamie suggested). If I understand correctly, unmounting /frames RAID will require that we stop all the daqd processes for the duration of the dd backup

Quote:
 
sudo dd if=/dev/sda of=/dev/sdb bs=64K conv=noerror,sync

in a tmux session on nodus (as I did for chiara and FB1, latter backup is still running). 


Edit: unmounting /frames won't help, since dd makes a bit for bit copy of the drive being cloned. So we need a drive with size that is >= that of the drive we are trying to clone. On FB1, this is /dev/sda, which has a size of 2TB. The HGST drive we got has an advertised size of 2TB, but looks like actually only 1.8TB is available. So I think we need to order a 4TB drive.

  13347   Fri Sep 29 18:36:25 2017 gautamUpdateLSCDAC noise measurement (again)

BS connections and damping restored.

Quote:

I am running some more measurements of the DAC noise, for which I've shut down the BS watchdog. Some of the cables on the coil driver side have been disconnected.

I will restore these tomorrow.


 

  13349   Mon Oct 2 18:08:10 2017 gautamUpdateCDSc1ioo DC errors

I was trying to set up a DAC channel to interface with the AOM driver on the PSL table.

  • It would have been most convenient to use channels from c1ioo given proximity to the PSL table.
  • Looking at the 1X2 rack, it looked like there were indeed some spare DAC channels available.
  • So I thought I'd run a test by adding some TPs to the c1als model (because it seems to have the most head room in terms of CPU time used).
  • I added the DAC_0 block from CDS_PARTS library to c1als model (after confirming that the same part existed in the IOP model, c1x03).
  • Model recompiled fine (I ran rtcds make c1als and rtcds install c1als on c1ioo).
  • However, I got a bunch of errors when I tried to restart the model with rtcds restart c1als. The model itself never came up.
  • Looking at dmesg, I saw stuff like
    [4072817.132040] c1als: Failed to allocate DAC channel.
    [4072817.132040] c1als: DAC local 0 global 16 channel 4 is already allocated.
    [4072817.132040] c1als: Failed to allocate DAC channel.
    [4072817.132040] c1als: DAC local 0 global 16 channel 5 is already allocated.
    [4072817.132040] c1als: Failed to allocate DAC channel.
    [4072817.132040] c1als: DAC local 0 global 16 channel 6 is already allocated.
    [4072817.132040] c1als: Failed to allocate DAC channel.
    [4072817.132040] c1als: DAC local 0 global 16 channel 7 is already allocated.
    [4073325.317369] c1als: Setting stop_working_threads to 1
  • Looking more closely at the log messages, it seemed like rtcds could not find any DAC cards on c1ioo.
  • I went back to 1X2 and looked inside the expansion chassis. I could only find two ADC cards and 1 BIO card installed. The SCSI cable labelled ("DAC 0") running from the rear of the expansion chassis to the 1U SCSI->40pin IDE breakout chassis wasn't actually connected to anything inside the expansion chassis.
  • I then undid my changes (i.e. deleted all parts I added in the simulink diagram), and recompiled c1als.
  • This time the model came back up but I saw a "0x2000" error in the GDS overview MEDM screen.
  • Since there are no DACs installed in the c1ioo expansion chassis, I thought perhaps the problem had to do with the fact that there was a "DAC_0" block in the c1x03 simulink diagram - so I deleted this block, recompiled c1x03, and for good measure, restarted all (three) models on c1ioo.
  • Now, however, I get the same 0x2000 error on both the c1x03 and c1als GDS overview MEDM screens (see Attachment #1).
  • An elog search revealed that perhaps this error is related to DAQ channels being specified without recording rates (e.g. 16384, 2048 etc). There were a few DAQ channels inside c1als which didn't have recording rates specified, so I added the rates, and restarted the models, but the errors persist.
  • According to the RCG runtime diagnostics document, T1100625 (which admittedly is for RCG v 2.7 while we are running v3.4), this error has to do with a mismatch between the DAQ config files read by the RTS and the DAQD system, but I'm not sure how to debug this further.
  • I also suspect there is something wrong with the mx processes:
    controls@c1ioo:~ 130$ sudo systemctl status mx
    ● open-mx.service - LSB: starts Open-MX driver
       Loaded: loaded (/etc/init.d/open-mx)
       Active: failed (Result: exit-code) since Tue 2017-10-03 00:27:32 UTC; 34min ago
      Process: 29572 ExecStop=/etc/init.d/open-mx stop (code=exited, status=1/FAILURE)
      Process: 32507 ExecStart=/etc/init.d/open-mx start (code=exited, status=1/FAILURE)
    Oct 03 00:27:32 c1ioo systemd[1]: Starting LSB: starts Open-MX driver...
    Oct 03 00:27:32 c1ioo open-mx[32507]: Loading Open-MX driver (with  ifnames=eth1 )
    Oct 03 00:27:32 c1ioo open-mx[32507]: insmod: ERROR: could not insert module /opt/3.2.88-csp/open-mx-1.5.4/modules/3.2.88-csp/open-mx.ko: File exists
    Oct 03 00:27:32 c1ioo systemd[1]: open-mx.service: control process exited, code=exited status=1
    Oct 03 00:27:32 c1ioo systemd[1]: Failed to start LSB: starts Open-MX driver.
    Oct 03 00:27:32 c1ioo systemd[1]: Unit open-mx.service entered failed state.
  • Not sure if this is related to the DC error though.
Attachment 1: c1ioo_CDS_errors.png
c1ioo_CDS_errors.png
  13351   Mon Oct 2 19:03:49 2017 gautamUpdateCDS[Solved] c1ioo DC errors

This did the trick - I simply ran

sudo systemctl restart daqd_*

on FB1, and now all the CDS overview lights are green again.

I thought I had done this already, but I realize that I was supposed to restart the daqd processes on FB1 (which is where they are running) and not on c1ioo frown.

Thanks Jamie for the speedy resolution!

Quote:
Quote:

 

  • This time the model came back up but I saw a "0x2000" error in the GDS overview MEDM screen.
  • Since there are no DACs installed in the c1ioo expansion chassis, I thought perhaps the problem had to do with the fact that there was a "DAC_0" block in the c1x03 simulink diagram - so I deleted this block, recompiled c1x03, and for good measure, restarted all (three) models on c1ioo.
  • Now, however, I get the same 0x2000 error on both the c1x03 and c1als GDS overview MEDM screens (see Attachment #1).

From page 21 of T1100625, DAQ status "0x2000" means that the channel list is out of sync between the front end and the daqd.  This usually happens when you add channels to the model and don't restart the daqd processes, which sounds like it might be applicable here.

It looks like open-mx is loaded fine (via "rtcds lsmod"), even though the systemd unit is complaining.  I think this is because the open-mx service is old style and is not intended for module loading/unloading with the new style systemd stuff.

 

Attachment 1: CDSoverview.png
CDSoverview.png
  13352   Mon Oct 2 23:16:05 2017 gautamHowToCamerasCCD calibration

Going through some astronomy CCD calibration resources ([1]-[3]), I gather that there are in general 3 distinct types of correction that are applied:

  1. Dark frames --- this would be what we get with a "zero duration" capture, some documents further subdivide this into various categories like thermal noise in the CCD / readout electronics, poissonian offsets on individual pixels etc.
  2. Bias frames --- this effect is attributed to the charge applied to the CCD array prior to the readout.
  3. Flat-field calibration --- this effect accounts for the non-uniform responsivity of individual pixels on the CCDs. 

The flat-field calibration seems to be the most complicated - the idea is to use a source of known radiance, and capture an image of this known radiance with the CCD. Then assuming we know the source radiance well enough, we can use some math to back out what the actual response function of individual pixels are. Then, for an actual image, we would divide by this response-map to get the actual image. There are a number of assumptions that go into this, such as: 

  • We know the source radiance perfectly (I guess we are assuming that the white paper is a Lambertian scatterer so we know its BRDF, and hence the radiance, perfectly, although the work that Jigyas and Amani did this summer suggest that white paper isn't really a Lambertian scatterer). 
  • There is only one wavelength incident on the CCD.
  • We can neglect the effects of dust on the telescope/CCD array itself, which would obviously modify the responsivity of the CCD, and is presumably not stationary. Best we can do is try and keep the setup as clean as possible during installation.

I am not sure what error is incurred by ignoring 2 and 3 in the list at the beginning of this elog, perhaps this won't affect our ability to estimate the scattered power from the test-masses to within a factor of 2. But it may be worth it to do these additional calibration steps. 

I also wonder what the uncertainty in the 1.5V/A number for the photodiode is (i.e. how much do we trust the Ophir power meter at low power levels?). The datasheet for the PDA100A says the transimpedance gain at 60dB gain is 1.5 MV/A (into high impedance load), and the Si responsivity at 1064nm is ~0.25A/W, so naively I would expect 0.375 V/uW which is ~factor of 4 lower. Is there a reason to trust one method over the other?  

Also, are the calibration factor units correct? Jigyasa reported something like 0.5nW s / ct in her report.

Camera IP Calibration Factor CF
192.168.113.152 8.58 W*s
192.168.113.153 7.83 W*s

The incident power can be calculated as Pin =CF*Total(Counts-DarkCounts)/ExposureTime.

References:

[1] http://www.astrophoto.net/calibration.php

[2] https://www.eso.org/~ohainaut/ccd/

[3] http://www.astro.ufl.edu/~lee/ast325/handouts/ccd.pdf

  13353   Tue Oct 3 01:32:39 2017 gautamUpdateLSCLaser intensity noise coupling to MICH (simulated)

GV Oct 6: This coupling is probably not correct - Finesse outputs TF magnitude in units of W/W, and not W/RIN

Since I was foiled (by lack of DAC) in my attempt to measure the coupling of laser intensity noise to MICH in the DRMI (no arms) configuration, I decided to try understanding the effect with a simulation.

For this purpose, I used my DRMI Finesse model - this had mirror positions tuned for locking and photodiode demod phases tuned to give a sensing matrix model that wasn't too far from an actual measurement (within factor of a few). So the model seems okay for a first pass at estimating this coupling.

Measuring transfer functions in Finesse is straightforward - use the fsig command to modulate some quantity (in this case the input beam intensity), and use the pd2 detector to demodulate the effect of this modulation at the port of interest (in this case AS55_Q).

**Note that to apply a modulation to an input beam (i.e. Laser) in Finesse, the keyword for the "type" argument given to fsig is "amp" and not "amplitude" as the manual would had me believe. In fact, there seem to be quite a few such caveats. The best way to figure this out is to go to the pykat installation directory, find the file components.py, and look for the fsig_name for the component of interest. It is also indicated in the same file, via the canFsig argument, if that property of the component can be modulated for transfer function measurements.  

Attachment #1 shows the result of such a sweep.

To estimate what the actual contribution to the displacement noise is, I used the DQ-ed MC transmission (recorded at 1024Hz) from the DRMI lock, computed the ASD using scipy.signal.welch, divided by the nominal MC transmission of ~15,000 counts to convert to RIN/rtHz. The RIN was then multiplied by the above calculated coupling function, and divided by the sensing matrix element for AS55_Q (in units of W/m) to give the curve shown in Attachment #2. If we believe the simulation, then Laser Intensity Noise shouldn't be the limiting noise between 10Hz-1kHz. 

I will of course measure the actual coupling and see how it lines up with Attachment #1 - would be a nice additional validation of the Finesse model. I will also try using the Finesse model to estimate some other coupling transfer functions (e.g. Laser Frequency Noise, Oscillator Noise).

Quote:

The absence of evidence is not evidence of absence.

 

Attachment 1: MICH_intensityNoiseCoupling.pdf
MICH_intensityNoiseCoupling.pdf
Attachment 2: MICH_intensityNoiseASD.pdf
MICH_intensityNoiseASD.pdf
  13355   Tue Oct 3 19:39:10 2017 gautamUpdateCDSslow machine bootfest

Eurocrate key turning reboots for c1susaux, c1auxex,c1auxey, c1iscaux, c1iscaux2 and c1aux. Usual precautions were taken for ITMX. Did burtrestore for c1iscaux andc1iscaux2  in order to restore the LSC PD whitening gains.


Un-related to this work: input pointing into PMC was tweaked as the PMC_REFL spot was pretty bright.

  13356   Wed Oct 4 17:18:15 2017 gautamUpdateCDSFree DAC channels in c1lsc

There are at least 5 free DAC channels (4 if you discount the one channel from these that I am hijacking) available in the 1Y2 electronics rack.

Jamie's nice wiring diagram shows the topology - the actual DAC card sits in 1Y3 inside the c1lsc expansion chassis (while the c1lsc frontend itself is in 1X4). The output of the DAC goes via SCSI to an interface box (D080303) and then to some dewhitening/AI boards (D000316). There are a total of 16 DAC channels available, out of which 8 are used for the TTs, 2 are used for the DAFI model, and one is labeleld "From c1ioo 1X2" (I don't know what this one is for). So I'm going to use some of these channels for measuring the coupling of oscillator noise and intensity noise to MICH in the DRMI lock.

The de-whitening/AI board seems to be old - it has 2x 800Hz Butterworth LPFs and no notch for the clock frequency, but maybe this doesn't matter for the tests I have in mind. The AI board available on 1X2 is more modern but routing the DAC channels from 1Y2 to it is going to be some work.

I'm going to add my testpoint to c1daf given that it seems to be the least critical model on c1lsc.

EDIT: testpoints added to c1daf don't show up in the list of available channels - there was some issue with this model while we were getting the new RTCDS going. So I'm moving my temporary testpoint to c1cal instead.

  13357   Wed Oct 4 17:38:25 2017 gautamUpdateLSCFS725 for Marconi stabilization

I've located the Stanford Research FS725 Rb reference unit. The question is where to put it. This afternoon Steve and I put it inside the little electronics rack next to 1X3, but in hindsight, this probably isn't such a great place for a timing reference as there are a bunch of Sorensen power supplies in there (and presumably the accompanying harmonics from these switching supplies). 

The unit itself was repaired in 2015, and powering it on, it locked to the internal reference within a few minutes as prescribed in the manual. 

  13359   Thu Oct 5 02:14:51 2017 gautamUpdateLSCMore DRMI coupling measurements - setup

In the end I decided to access the available spare DAC channels via the C1ASS model - for this purpose, I added a namespace block "TEST" in the C1ASS simulink model, which is a SISO block. Inside is just a single CDS filter module. My idea is to use the EXC of this filter module to inject excitations for measuring various couplings. Rather than have a simple testpoint, we also have the option of adding in some filter shapes in the filter module which could possibly allow a more direct read-off of some coupling TF. Recompiling the model went smooth - there was a crash earlier in the day which required me to hard-reboot c1lsc (and also restart all models on c1sus and c1ioo but no reboots necessary for those machines).

Note that to get the newly added channels to show up in the channel lists in DTT/AWGGUI etc, you need to ssh into fb1 and restart the daqd processes via sudo systemctl restart daqd_*. If I remember right, it used to be enough to do telnet fb 8088 followed by shutdown. This is no longer sufficient.

It took me a while to get the DRMI locking going again. The model restarts earlier in the evening had changed a bunch of EPICS channel settings (and out config scripts don't catch all of these settings). In particular, I forgot to re-enable the x3 digital gain for the ITMs, BS and SRM (necessitated by removing an analog x3 gain on the de-whitening boards). I was hesitant to spend time re-adjusting all damping / oplev loop gains because if we change the series resistor on the coil driver board, we will have to do this again. I also didn't want this arbitrary FM to be enabled in the SDF safe.snap. But maybe it's worth doing it anyways - if nothing it'll be good practise.

Once I hunted down all the setting diffs and tweaked alignment, the DRMI locks were pretty robust.

I had hoped to make some of these TF measurements tonight. But I realized I needed to look up a bunch of stuff in manuals/datasheets, and think about these measurements a little. I wasn't sure if the DW/AI board could drive a signal over 40m of BNC cabling so I added an SR560 (DC coupled, gain=1, low noise mode, 50ohm output used) to buffer the output. The Marconi's external modulation input is high impedance (100k) but for the AOM driver we want 50ohm. For the Marconi, the external input accepts 1Vrms max, while for the AOM driver, we want to drive a signal between 0V and 1V at most.

The general measurement setup is schematically shown in Fig 1. Questions to address:

  • What happens if we apply a negative voltage to the input of the AOM driver? What is the damage threshold? Do we have to worry about SR560 offset level?
  • Is there a way to dynamically adjust the offset in DTT such that we can have different amplitude signals at different frequencies (usually done by specifying an envelope in DTT) but still satisfy the requirement that the entire signal lie between 0-1V?
  • For the Laser Intensity noise -> MICH coupling TF measurement, I guess we can use the AOM to inject an excitation, and measure the ratio of the response in MC_TRANS and in MICH_IN1. Then we multiply the in-loop MC_TRANS spectrum by the magnitude of this TF to get the Laser Intensity Noise contribution to MICH.
  • The Laser Frequency Noise coupling should be negligible in MICH - but the measurement principle should be the same. Drive the AO input of the Mode Cleaner Servo board from the DAC, look at ratio of response in MICH_IN1 and MC_F. Multiply the DRMI in-lock MC_F spectrum by this TF.
  • The oscillator noise seems more tricky to me (also Finesse modeling suggests this may be the most significant of the 3 couplings described in this elog, though I may just be computing the coupling in Finesse wrongly)
    • I don't understand all the External Modulation options specified in the manual.
    • DC? AC? FM? PM? AM? Need to figure out what is the right settings to use.
    • I'm not sure how independent the various modulations will be - i.e. if I select PM, how much AM is induced as a result of me driving the EXT MOD input?
    • What is the right level of excitation drive? I tried this a bunch of times tonight - set the PM range to 0.1rad (for the full scale 1Vrms sine wave input), but with an excitation of just a few counts, already saw non-lineaer coupling in MICH_IN1 which probably means I'm driving this too hard.
    • This measurement needs a bit more algebra. We have an estimate of the Marconi phase noise from Rana (is this the right one to use?). But the "Transfer Function" we'd measure is cts in MICH_IN1 in response to counts to Marconi via the signal chain in Attachment #1. So we'd need to know (and divide out) the AI/DW board TF, and the Marconi's TF, which the datasheet suggests has a lower 3dB frequency of 100Hz (assuming SR560 and cable can be treated as flat).
    • A simpler test may be to just hook up the Marconi to the Rb standard, and the Rb to 1pps from GPS, and look for a change in the MICH noise.

Am I missing something?

Attachment 1: CB4709D0-3FA7-43E3-BC25-3CF4164E6C6A.jpeg
CB4709D0-3FA7-43E3-BC25-3CF4164E6C6A.jpeg
  13360   Thu Oct 5 11:46:15 2017 gautamUpdateCDSslow machine bootfest

MC Autolocker was umnhappy because c1iool0 was unresponsive and hence it couldn't write to the "C1:IOO-MC_AUTOLOCK_BEAT" channel. I keyed the crate and IMC locked almost immediately. I'm moving this channel into the RTCDS model as we did for the IFO_STATE EPICS channel so that the autolocker isn't dependant on c1iool0 (which was the whole point of migrating the IFO-STATE variable anyways). I also commented out all of these channels in /cvs/cds/caltech/target/c1iool0/autolocker.db so that there aren't duplicate channels.

Quote:

Eurocrate key turning reboots for c1susaux, c1auxex,c1auxey, c1iscaux, c1iscaux2 and c1aux. Usual precautions were taken for ITMX. Did burtrestore for c1iscaux andc1iscaux2  in order to restore the LSC PD whitening gains.


Un-related to this work: input pointing into PMC was tweaked as the PMC_REFL spot was pretty bright.

 

  13361   Thu Oct 5 13:58:26 2017 gautamUpdateCDS40m files backup situation

The 4TB HGST drives have arrived. I've started the FB1 dd backup process. Should take a day or so.

Quote:
Edit: unmounting /frames won't help, since dd makes a bit for bit copy of the drive being cloned. So we need a drive with size that is >= that of the drive we are trying to clone. On FB1, this is /dev/sda, which has a size of 2TB. The HGST drive we got has an advertised size of 2TB, but looks like actually only 1.8TB is available. So I think we need to order a 4TB drive.

 

  13362   Thu Oct 5 18:40:27 2017 gautamUpdateLSCFS725 for Marconi stabilization

[steve, gautam]

  1. We installed the FS725 on the shelf inside the PSL enclosure - see Attachment #1.
  2. We ran a long BNC cable (labelled "GPS 1pps" on both ends) from 1X7 to the PSL enclosure - this was to pipe the 1PPS signal from the GPS timing unit (EndRun Technologies Tempus LX) rear panel (50 ohm output according to the datasheet) to the 1PPS input of the FS725 (high impedance). See Attachments #2. Note that the 1pps output was already tee'd on the rear panel. One port of the tee was unused (this now goes to the FS725) while the other was going to the 1PPS input of the Master Timing Sequencer (D050239), so I decided that there was no need to tee the 1pps input of the FS725 with a 50ohm terminator. In a few minutes, the Rb standard indicated that it was locked to its internal reference, and also to the external 1pps input (see Attachment #1). 
  3. We ran a long BNC cable (labelled "Rb 10MHz" on both ends) from the 10MHz output of the FS725 (50 ohm output impedance),  in the PSL enclosure to the rear BNC "FREQ_STD IN/OUT" BNC connector of the Marconi (1kohm input impedance). Changed the frequency reference setting on the Marconi to "External Direct". The FS725 datasheet recommends terminating the load with a 50ohm inline terminator, I have not yet done this (see Attachment #3). Is it appropriate to use a Balun (FTB-1-1) here? This would avoid ground loops between the Marconi and the FS725, and also make the load seen by the FS725 50ohms
  4. Found that there was an unused long cable from the PSL enclosure to the 1X2 electronics rack. We re-purposed this to drive the AOM driver via the DAC output in 1Y2. The cable is labelled "AOM driver" on both ends. This was to facilitate measurement of the coupling of laser intensity noise to AS55_Q in a DRMI lock.
  5. Removed 2 long cables between 1X7 and 1X2 that weren't connected to anything.
  6. Re-arranged the DC bench supply on the shelf in the PSL enclosure, whose only purpose seems to be to supply 12V to a fan attached to the rear of the PSL NPRO controller. Seems to be a waste of space! The fan was momentarily disconnected but has since been reconnected and is spinning again.
  7. Removed a couple of unused power cables from the mess on the shelf in the PSL enclosure. Also removed an unused Sony Video Squential Switcher YS-S6 from the PSL enclosure. 
Quote:

I've located the Stanford Research FS725 Rb reference unit. The question is where to put it. This afternoon Steve and I put it inside the little electronics rack next to 1X3, but in hindsight, this probably isn't such a great place for a timing reference as there are a bunch of Sorensen power supplies in there (and presumably the accompanying harmonics from these switching supplies). 

The unit itself was repaired in 2015, and powering it on, it locked to the internal reference within a few minutes as prescribed in the manual. 

 

Attachment 1: IMG_7617.JPG
IMG_7617.JPG
Attachment 2: IMG_7619.JPG
IMG_7619.JPG
Attachment 3: IMG_7618.JPG
IMG_7618.JPG
  13364   Fri Oct 6 12:46:17 2017 gautamUpdateCDS40m files backup situation

Looks to have worked this time around.

controls@fb1:~ 0$ sudo dd if=/dev/sda of=/dev/sdc bs=64K conv=noerror,sync
33554416+0 records in
33554416+0 records out
2199022206976 bytes (2.2 TB) copied, 55910.3 s, 39.3 MB/s
You have new mail in /var/mail/controls

I was able to mount all the partitions on the cloned disk. Will now try booting from this disk on the spare machine I am testing in the office area now. That'd be a "real" test of if this backup is useful in the event of a disk failure.

Quote:

The 4TB HGST drives have arrived. I've started the FB1 dd backup process. Should take a day or so.

 

  13365   Fri Oct 6 12:56:40 2017 gautamSummaryLSCRTCDS NN

[gabriele, gautam]

Gabriele had prepared a C code implementation of his NN for MICH/PRCL state estimation. He had been trying to get it going on some of the machines in WB, but was unsuccessful. The version of RCG he was trying to compile and run the code on was rather dated so we decided to give it a whirl on our new RCG3.4 here at the 40m. Just noting down stuff we tried here:

  • Code has been installed at /opt/rtcds/userapps/release/cds/c1/src/nn.This is to facilitate compilation by the RCG.
  • There is also a simulink block diagram (x3tst.mdl) in there which we copied and pasted into c1pem. Changed the appropriate paths in the C-Code block to point to the location in the previous bullet point.
  • c1pem was chosen for this test as it runs at 2k which is what the network is designed for.
  • Since we were running the test on c1sus and expected the machine to crash, I shutdown all the watchdogs for the test.
  • Code compiled without any problems (i.e. rtcds make c1pem and rtcds install c1pem executed successfully). There were some warning messages related to C-Code blocks but these are not new, they show up in all models in which we have custom C-code blocks. 
  • Unfortunately, the whole c1sus FE crashed when we tried rtcds restart c1pem.
  • We tried a couple of more iterations, and attempted to monitor dmesg during the restart process to see if there were any clues as to why this wasn't working, but got nothing useful.

All models have been reverted to their state prior to this test, and everything on the CDS_OVERVIEW MEDM screen is green now.

Attachment 1: post_NN_test.png
post_NN_test.png
  13367   Mon Oct 9 01:29:26 2017 gautamUpdateLSCDRMI Nosie Budget v3.0

Summary:

I spent this weekend doing a more careful investigation of the DRMI noise. I think I have some new information/insights. Attachment #1 is the noise budget (png attached because pdf takes forever to upload, probably some ImageMagick problem. The last attachment is a tarball of the PDF). Long elog, so here are the Highlights:

  1. Coil de-whitening does result in small improvement in noise in the 60-200Hz band.
  2. Above 200Hz, we seem to be limited by "Dark" noise. More on this below.
  3. The coupling from SRCL->MICH is the other limiting noise in the 60-200Hz band now.

Sensing Matrix Measurement:

  • I rotated the AS55 demod phase from -42 degrees to -82 degrees, the idea being to get more of the MICH error signal in AS55_Q.
  • Consequently, the MICH servo gain has been lowered from -0.035 to -0.021. Settings have been updated in the snap file used by the locking script.  
  • Seems to have worked.
  • Attachment #2 is the measured sensing elements.
  • One major source of uncertainty in these sensing element numbers is the actuator gains for PRM, SRM and BS. The coil driver electronics for the latter two have been modified recently, and for them, I am using numbers from this elog scaled by the expected factor as a result of removing the x3 gain in the de-whitening boards for SRM and BS.

MICH OLTF

  • Measurement was done in lock using the usual IN1/IN2 method.
  • Model made by loading the FOTON filters + assumed models for the BS pendulum and AA/AI filters in Matlab, and fitting to an overall gain + delay.
  • Attachment #3 shows the agreement between measurement and model.
  • The model was exported and used to invert in-loop signals to their out-of-loop counterparts in the noise budget.

DAC Noise

  • I had claimed that turning on the coil de-whitening did not improve the MICH noise.
  • This was not exactly true - I had only compared MICH noise with the BS de-whitening turned ON/OFF, while the ITM de-whitening was always on.
  • Turns out that there is in fact a small improvement - see Attachment #4 (DTT crashes everytime I try to print a pdf, so png screenshot will have to do for now).
  • I have also changed the way in which DAC noise is plotted in the Noise Budget code:
    • I used to directly convert the measured voltage noise (multiplied by appropriate scalar to account for quadrature sum of 4 coils each in 3 optics) to displacement noise using the sensing measurement cts/m values.
    • Now I convert the measured voltage noise first to current noise (knowing the series resistance), then to force noise (using the number 0.016 N/A per coil), then to displacement noise (assuming a mirror mas of 250g).
    • Quadrature sum is again taken for 4 coils on 3 optics.
  • I've also added the option to plot the DAC noise with the de-whitening filter TF applied (taking care that the maximum of filtered DAC noise / coil driver electronics noise is used at each frequency).
  • So the major source of uncertainty in the calculated DAC noise is the assumed actuator gain of 0.016 N/A.

The DAC noise is not limiting us anywhere when the coil de-whitening is switched on.


Dark Noise

I think this is the major find.

  • The dark noise spectrum is measured with:
    • the PSL shutter closed
    • the AS55 I and Q analog whitening filters (and corresponding digital de-whitening filters) engaged, to mimic the operating conditions under which the in-lock error signal is acquired.
  • Comparing the blue and black traces, it is clear that turning on the analog whitening is having some effect on the dark noise.
  • However, the analog whitening filters should suppress the ADC noise by ~30dB @ 100Hz - so assuming 1uV/rtHz, this would be ~30nV/rtHz @100Hz.
  • But the measured noise seems to be ~5x higher, with 4*10^-4 cts/rtHz translating to roughly 120nV/rtHz.
  • The photodiode dark noise is only 15nV/rtHz according to the wiki. Where is this measured?

So I don't understand the measured Dark Noise level, and it is limiting us at frequencies > 200Hz. Some busted electronics in the input signal chain? Or can the LSC demod daughter board gain of ~5 explain the observed noise?


Shot noise

  • The DC power on AS55 photodiode was measured to be ~13mW with the SRM misaligned.
  • This corresponds to ~100cts peak amplitude on the ASDC channel (derived from AS55 photodiode).
  • In the DRMI lock, the ASDC level is ~200cts.
  • I used these numbers, and equation 2.17 in Tobin's thesis, to calculate this curve.

Edit 1730 9 Oct: I had missed out the factor of 5 gain in the demod board in calculating the shot noise curve. Attachment #7 shows the corrected shot noise level. Explicitly:

n_{\mathrm{shot}} [m/\sqrt{\mathrm{Hz}}] = \alpha \sqrt{2 h \nu \bar{P} (\frac{1}{2} - \frac{1}{4}\mathrm{cos}2\theta)}, where \alpha [m/W] = (\mathcal{M}_{\mathrm{MICH}} [V/m] / 5 [V/V] / 420 [V/A] / 0.7 [A/W])^{-1}is to convert shot noise in W to displacement units.


AUX coupling

This is the other find.

  • While chatting with Gabriele, he suggested measuring the SRCL->MICH and PRCL->MICH cross couplings.
  • I injected a signal in SRCL servo EXC channel, and adjusted amplitude till coherence in MICH_IN1 was good.
  • The actual TF measured was MICH_IN1 / SRCL_IN1 (so units of cts/ct).
  • My multiplying the in-lock PRCL and SRCL IN1 signals by these coupling coefficients (assumed flat in frequency for now, note that measurement was only made between 100Hz and 1kHz), I get the trace labelled "AUX coupling" in Attachment #1 (this is the quadrature sum for SRCL and PRCL couplings).
  • Also repeated for PRCL -> MICH coupling in the same way.
  • Measurements of these TFs and coherence are shown in Attachment #5 (again png screenshot because of DTT).
  • However, there is no significant coherence in MICH/SRCL or MICH/PRCL in this frequency range.

This seems to be limiting us from saturating the dark noise once the coil de-whitening is engaged. But lack of coherence means the mechanism is not re-injection of SRCL/PRCL sensing noise? Need to think about what this means / how we can mitigate it.


OL A2L coupling

  • I didn't measure these
  • These couplings would have changed because I modified the Oplev loop shapes to allow engaging of coil de-whitening filters.
  • But anyways, their effect will only be below 100Hz because I made the roll-offs steeper.

Still to measure (but not likely to be limiting us anywhere in the current state):

  • Laser intensity noise -> MICH coupling (using AOM).
  • Laser frequency noise -> MICH coupling (using CM board IN2).
  • Oscillator noise (amplitude + phase) -> MICH coupling (using AM/FM input of Marconi).

I've also made several changes to the NB code - will push to git once I finish cleaning stuff up, but it is now much faster to make these plots and see what's what.

Attachment 1: DRMI_NB.png
DRMI_NB.png
Attachment 2: DRMI1f_Oct8.pdf
DRMI1f_Oct8.pdf
Attachment 3: MICH_OLTF_model.pdf
MICH_OLTF_model.pdf
Attachment 4: MICH_noises.png
MICH_noises.png
Attachment 5: AUX_couplings.png
AUX_couplings.png
Attachment 6: C1NB_disp_40m_MICH_NB_2017-10-08.pdf.tar.gz
Attachment 7: MICH_NB_corrected.png
MICH_NB_corrected.png
  13369   Mon Oct 9 22:18:34 2017 gautamUpdateLSCAS55Q Dark Noise

I measured the output voltage noise of the Q output of the AS55 Demod Board with the PSL shutter closed, using the SR785 (see Attachment #1). The measured noise is consistent with the expected number of ~120nV/rtHz around 100Hz. I had measured the gain of this board from RFPD input to Q output to be ~5.1: so if the PD dark noise is 16nV/rtHz, this would be amplified to ~80nV/rtHz. Still a discrepancy of ~50%. I didn't measure the noise with the PD input terminated. Added the noise of the demod board output with the RFPD input terminated. The level of ~100nV/rtHz seems consistent with the actual PD dark noise being ~80nV/rtHz, as their quadrature sum is around 130nV/rtHz. Need to dig up the schematics for the demod board + daughter board, and check against LISO, to see if this is consistent with what is expected.

Also - I think I was using the wrong value of the DC power on the AS55 photodiode for shot noise calculations - 13mW was for REFL55, not AS55. I did a crude measurement of the power by sticking the Ophir power meter (filter removed) in front of the AS55 PD with the Michelson flashing around, and noticed the maximum value registered was ~1.2mW. So in the DRMI lock, there would be ~2.4mW, which is 10x lower than the value I was assuming. I've made the correction in the NB code, for the next time the plot is generated. A more rigorous measurement would involve sticking the Ophir in front of the AS110 PD during a DRMI lock. The light from the AS port is split by a 50-50 BS to the AS55 and AS110 PDs (so measuring at AS110 is a reasonable proxy for power at AS55), and the AS110 signals are not used for triggering in the DRMI lock, so this is feasible.

 

Attachment 1: AS55Q_dark.pdf
AS55Q_dark.pdf
  13372   Wed Oct 11 14:42:03 2017 gautamUpdateLSCAS55Q Dark Noise

I keep adding traces to this plot, here is the most complete one I have now. Looks like the input noise to the D040179 (measured at "Q out" SMA jack of D990511 with RFPD input terminated) is ~10nV/rtHz. This supports the hypothesis that something is wonky on the daughter board, because the purple trace should only be the quad sum of the orange and green traces. I will pull it out and have a look.

Some other follow-ups on the questions raised at the meeting:

  1. Doesn't look like I've implemented thin film resistors on the input of the coil driver boards. De-whitening boards have the critical signal path resistors (judged as the ones with largest contribution as per LISO model) changed to thin film. Pictures are here.
  2. I think I didn't make a full elog of my demod board efficiency investigations, but from my notes and Attachment #4 of elog 12972, I calculated the gain in the signal path as the ratio of Vpp_out / Vpp_in.
Quote:

I measured the output voltage noise of the Q output of the AS55 Demod Board with the PSL shutter closed, using the SR785 (see Attachment #1). The measured noise is consistent with the expected number of ~120nV/rtHz around 100Hz. I had measured the gain of this board from RFPD input to Q output to be ~5.1: so if the PD dark noise is 16nV/rtHz, this would be amplified to ~80nV/rtHz. Still a discrepancy of ~50%. I didn't measure the noise with the PD input terminated. Added the noise of the demod board output with the RFPD input terminated. The level of ~100nV/rtHz seems consistent with the actual PD dark noise being ~80nV/rtHz, as their quadrature sum is around 130nV/rtHz. Need to dig up the schematics for the demod board + daughter board, and check against LISO, to see if this is consistent with what is expected.

Also - I think I was using the wrong value of the DC power on the AS55 photodiode for shot noise calculations - 13mW was for REFL55, not AS55. I did a crude measurement of the power by sticking the Ophir power meter (filter removed) in front of the AS55 PD with the Michelson flashing around, and noticed the maximum value registered was ~1.2mW. So in the DRMI lock, there would be ~2.4mW, which is 10x lower than the value I was assuming. I've made the correction in the NB code, for the next time the plot is generated. A more rigorous measurement would involve sticking the Ophir in front of the AS110 PD during a DRMI lock. The light from the AS port is split by a 50-50 BS to the AS55 and AS110 PDs (so measuring at AS110 is a reasonable proxy for power at AS55), and the AS110 signals are not used for triggering in the DRMI lock, so this is feasible.

 

 

Attachment 1: AS55Q_darkNoises.pdf
AS55Q_darkNoises.pdf
  13374   Wed Oct 11 19:31:32 2017 gautamUpdateLSCAS55Q Dark Noise

I tried replacing the AD797s on the daughter board with OP27s, and saw no significant improvement in the electronics noise of the demod board. Note that according to LISO, in this configuration, the voltage noise of the Op27 is expected to dominate the total noise of the daughter board. Measurement condition was that the RFPD input was terminated, but the LO input was still being driven (SR785 input range is -50dBVpk for all traces, and the input ranging was set to "UpOnly"). Need to do a more systematic investigation to figure out where this excess noise is coming from. I will upload photos of the board later.

Quote:

This supports the hypothesis that something is wonky on the daughter board, because the purple trace should only be the quad sum of the orange and green traces. I will pull it out and have a look.

 

Attachment 1: AS55Q_darkNoises.pdf
AS55Q_darkNoises.pdf
  13376   Thu Oct 12 01:50:11 2017 gautamUpdateLSCAS55Q Dark Noise

I worked on the daughter board a little more in the evening. I have somehow managed to make the dark noise ~25% worse [Attachment #1].

  • Earlier in the day, I had switched out both on-board AD797s for OP27. The latter has ~3x the input voltage noise, and LISO modeling suggests that this is the dominant contribution to the output voltage noise.
  • There are some differences in the actual components with which the board is stuffed, and the schematic. 
  • After updating the LISO model, I expect to get an output voltage noise of ~50nV/rtHz. But I measured ~2x this value (measured with LO input of demod board driven, RFPD input terminated).
  • While I had the board out, I replaced most of the installed thick-film resistors with thin film ones. For good measure, I also changed the AD829s.

After making all these changes, I re-installed the card in the eurocrate and repeated the measurement. The Q channel noise was close to the expected value (~50nV/rtHz), but the I channel is twice as noisy. I will continue this investigation tomorrow.

Attachment 1: AS55_dark.png
AS55_dark.png
  13378   Thu Oct 12 12:17:28 2017 gautamUpdateLSCAS55Q Dark Noise

Here is the marked up schematic with the board as it is stuffed. Annoyingly, there is a capacitor (C1) which according to the schematic is supposed to be open, but is stuffed in our board. I can't find any elog about this, and its a pain to measure the value of this capacitance. I will upload all of this + LISO + noise model/measurements to a 40m AS55 daughter board DCC page.

 

Attachment 1: D040179_AS55_40m.pdf
D040179_AS55_40m.pdf D040179_AS55_40m.pdf
  13379   Thu Oct 12 14:42:45 2017 gautamUpdateCDSslow machine bootfest

Steve reported problems getting the X arm locked. Alignment sliders were inaccessible. Eurocrate key turning reboots for c1susaux, c1auxex,c1auxey, c1iscaux and c1aux. Usual precautions were taken for ITMX.

This is becoming a once-a-week thing sad.

  13380   Fri Oct 13 12:26:12 2017 gautamUpdateLSCAS55Q Dark Noise

Attachment #1 - Measured / modelled noises for AS55 demod board. I've plotted quadrature sum of the LISO trace with the SR785 noise floor with input terminated to ground via 50ohm. Note that these measurements were made after all the changes in the marked up schematic in the previous elog were implemented.

Both channels should be identical - I don't understand why the I channels are noisier than their Q counterparts. This is almost certainly a problem on the daughter board, as the orange traces are pretty much identical for both channels.

The dark red curves were measured by shorting the inputs to D040179 to ground via 50ohms using some Pomona minigrabbers - I wanted to avoid ripping the daughter board out, but this probably explains the excess noise compared to the green trace at low frequencies. All other measurements were made with the board installed in the LSC rack eurocrate, with the LO input driven at the nominal level (I didn't measure this yesterday but a measurement from ~6months ago says that this level is 1.5dBm).

Attachment 1: AS55_DemodNoises.pdf
AS55_DemodNoises.pdf
  13381   Mon Oct 16 12:13:38 2017 gautamUpdateCDSMegatron maintenance

Wall StripTool traces showed that IMC has not been locked for at least 8 hours when I came in this morning. Going to the IMC autolocker log, it looks like the last timestamp was at ~6pm yesterday. Megatron was responding to ping, but I couldn't ssh into it. So I went over to the machine and did a hard-reboot via front panel power switch. The computer took ~10mins to come back online and respond to ping. Once it did, I was able to ssh into it. However, trying the usual commands to restart the IMC autolocker and FSS Slow loops didn't work. Specifically, monitoring the logfile with tail -f Autolocker.log, I would see that the autolocker seemed to get stuck after starting the "blinky" script. Trying to restart the process using sudo initctl restart MCautolocker, init would print to shell that the restart had worked, and reported the PID, but the logfile wouldn't update "live" as it should when tail is used with the -f option. All very strange frown.

Anyways, as a last resort, I kill -9'ed the PID for the init instance, and init automatically restarted the Autolocker - this did the trick, IMC is locked now and logfile seems to be getting updated normallyyes.

I also cleared a bunch of matlab crash dump files in the home directory.

  13382   Mon Oct 16 16:01:04 2017 gautamUpdateLSCAS55Q Dark Noise

Koji suggested looking at the output of the AS55 demod board on a fast oscilloscope to look for differences in the two channel outputs (if there is some high-frequency oscillations, for example, we could miss this information in the SR785 spectra). Besides, I was only looking at spectra out to a few kHz on the SR785. I grabbed this data with a 300MHz BW Tektronix oscilloscope (battery mode) today. Input impedance of both channels were set to 1Mohm, and the measurement was made with the RFPD input terminated, output of the daughter board is what is measured. The vertical scaling of the channels was set to the minimum allowed, 1mV/div.

Attachment #1 shows that there is indeed a visible difference between the two channels - the (noisier) I channel has a much larger DC offset of ~5mV compared to the Q channel (I tried switching channels on the O'scope and the larger DC offset remained on the I channel, so seems real). There is also some kind of oscillation going on in the I channel, although the frequency is pretty low, with the peaks spaced ~50us apart. Indeed, in the ASD of the acquired data, the excess power in the I channel at 20kHz and higher harmonics are evident (see Attachment #2). Anyway all of this points to something being anomalous on the daughter board I channel signal path - I will pull it out and monitor the outputs at various points along the signal path with the fast scope to see if I can narrow down what's going on where.

Quote:

Both channels should be identical - I don't understand why the I channels are noisier than their Q counterparts. This is almost certainly a problem on the daughter board, as the orange traces are pretty much identical for both channels.

 

Attachment 1: DemodBoardwOscope.pdf
DemodBoardwOscope.pdf
Attachment 2: DemodBoardwOscope_ASD.pdf
DemodBoardwOscope_ASD.pdf
  13384   Tue Oct 17 19:31:53 2017 gautamUpdateLSCAS55Q Dark Noise

[Koji, gautam]

We took a closer look at the AS55 demod board today. The procedure was to just be as thorough as possible, and check the behaviour of the circuit (both Transfer Function and Noise) stage by stage. Checking the transfer function was the key.

During this process, we found that the reason why the Q channels had lower noise than the I channels was because of the gain of the AD829 stage of the circuit was 0dB rather than 4dB (which is what it should be according to the component values used). Specifically, resistor R12, which is supposed to be 1.30kohm, was measured to be 1.03kohmfrown. Replacing this resistor, the transfer functions (see Attachment #1) and noise levels (see Attachment #2) match the expectations from LISO. Some notes:

  1. The daughter board essentially consists of 2 stages
    • OP27 stage, which has a design gain of 16dB ((=316ohm/50ohm) (flat at frequencies <100kHz).
    • AD829 stage, which has a design gain of 4dB (=1.3kohm/768ohm), and is a 2nd order Butterworth LPF with corner @ 1MHz.
    • So the overall gain of the daughter board is 20dB (i.e. x10) at audio frequencies.
  2. The output noise of D040179 is expected to be ~35nV/rtHz at 100Hz, and the measurement (made with inputs soldered together) is consistent with this value.
  3. The measured voltage noise at the input to D040179 (i.e. the output of the minicircuits mixer + SCLF-5 LPF) is ~9nV/rtHz.
  4. The output voltage noise of the demod board with RFPD input terminated then is expected to be the quadrature sum of the noise due to the D040179 electronics (i.e. 40nV/rtHz) and the input noise to the D040179 (i.e. 9nV/rtHz) multiplied by the gain of the daughter board (i.e. x10) == \sqrt{40^2 + 90^2} \approx 98nV/\sqrt{\mathrm{Hz}}.
  5. To calculate the "dark noise" contribution of AS55 to MICH displacement noise, we have to further add the photodiode dark noise contribution: this gets us up to \sqrt{98^2 + 80^2} \approx 130nV/\sqrt{\mathrm{Hz}}. This is consistent with the measurement (see Attachment #2).
  6. Assming the whitened ADC noise level is much below this (should only be ~10nV/rtHz), and given the measured sensing element of 6.2e8 V/m, this means that the dark noise sets a maximum achievable sensitivity of 2e-16m/rtHz.

To figure out what (if anything) is to be done next, we need to first figure out what is the goal. In the end, we care about DARM and not MICH. The optical gain for the former is ~300x the latter, so the dark noise contribution gets scaled by this factor (giving us a number of 7e-19 m/rtHz). There are certainly many noises above that level which have to be handled first. Indeed, looking at the DARM spectrum from DRFPMI lock back in March 2016, it looks like the current 1f DRMI (with coils de-whitened) Michelson sensitivity is within a factor of 2 of DARM in the full lock (albeit with vertex DoFs on 3f signals, and no coil de-whitening). Koji pointed out that we need to consider the photodiode resonant circuit itself too.

TODO: Upload all this onto the DCC

Attachment 1: D040179_TFs.pdf
D040179_TFs.pdf
Attachment 2: AS55_DemodNoises.pdf
AS55_DemodNoises.pdf
  13385   Tue Oct 17 23:07:52 2017 gautamUpdateCDSFEs unresponsive

While working on the IFO tonight, I noticed that the blinky status lights on c1iscex and c1iscey were frozen (but those on the other 3 FEs seemed fine). But all other lights on the CDS overview screen were green I couldn't access testpoints from these machines, and the EPICS readbacks for models on these FEs (e.g. Oplev servo inputs outputs etc) were frozen at some fixed value. This lasted for a good 5 minutes at least. But the blinky lights started blinking again without me doing anything. Not sure what to make of this. I am also not sure how to diagnose this problem, as trending the slow EPICS records of the CPU execution cycle time (for example) doesn't show any irregularity.

  13387   Wed Oct 18 02:09:32 2017 gautamUpdateCDSFEs unresponsive

I was looking at the ASDC channel on dataviewer, and toggling various settings like whitening gain. At some point, the signal just froze. So I quit dataviewer and tried restarting it, at which point it complained about not being able to connect to FB. This is when I brought up the CDS_OVERVIEW medm screen, and noticed the frozen 1pps indicator lights. There was certainly something going on with the end FEs, because I was able to ping the machine, but not ssh into it. Once the 1pps lights came back, I was able to ssh into c1iscex and c1iscey, no problems.

Could it be that some of the mx processes stalled, but the systemctl routine automatically restarted them after some time? 

Quote:

So this wasn't just an EPICS freeze?  I don't see how this had anything to do with any of the work I did earlier today.  I didn't modify any of the running front ends, didn't touch either of the end station machines or the DAQ, and didn't modify the network in any way.  I didn't leave anything running.

If you couldn't access test points then it sounds like it was more than just EPICS.  It sounds like maybe the end machines somehow fell of the network momentarily.  Was there anything else going on at the time?

 

  13392   Wed Oct 18 17:34:09 2017 gautamUpdateSUSASDC

Summary:

The signal path for the ASDC signal is AS55 PD --> D990543 (interface board) --> D990694 (whitening board) --> D000076 (AA board) --> ADC Ch 31. Everything in this signal chain should be able to handle signals in the range +/- 10V, which should correspond to the full range of our +/-10V, 16bit ADCs. But the ASDC signal seems to saturate at ~2000 counts (i.e. turning up the analog whitening gain doesn't make the signal get any bigger than this). I investigated this a little more today.

Details:

  • The ASDC signal is derived from the AS55 photodiode. According to the schematic, the Op27 that supplies this voltage is powered by +/- 15V, so the output should be able to swing between at least +/- 12V.
  • The DC signal goes from the DB15 connector on the side of the PD to the LSC electronics rack, 1Y2, where it is interfaced with an LSC PD Interface Card, D990543. Again, per the schematic, the Op27 driving this voltage is powered by +/- 15V, and so the available output voltage swing should be greater than +/-12V.
  • The D990543 output is to its backplane connector. There is an adaptor board hooked up to the backplane that makes these outputs available to a LEMO connector. A LEMO-SMA cable then pipes this output to a D990694.
    • I decided to test the functionality of this board.
    • Disconnected the SMA ASDC input signal (CH8 on the board).
    • Drove that channel with an SR function generator and gradually turned up the Vpp of the input signal (sine wave at 145Hz).
    • Monitored the ASDC channel on dataviewer while doing this.
    • Saw that the ASDC signal saturated at ~2000 counts. Turning up the signal amplitude did not have any effect.
  • From the whitening board, the signal goes through an anti-aliasing module (D000076). The final stage LT1125s on these boards should also be supplied with +/-15V.

So the problem lies somewhere downstream of the D990694. There are other anomalous behaviours of this channel - e.g. engaging the analog whitening filters changes the DC offset of the signal. I am going to pull out this board to check it out.

Why does this matter? I want to calibrate the ASDC level (and eventually the other PD DC signals as well) into Watts. This is useful for IFO diagnostics, noise budgeting the shot noise level etc.

According to the AS55 schematic, the DC transimpedance is 66.7 ohms. I claim that the DC power on the AS55 photodiode during a DRMI (no arms) lock is ~1mW. The C30642 photodiode (InGaAs) responsivity is ~0.8 A/W. So I'd expect ~50mV to be the signal level into the ADC (assuming gain of all the other electronics in the signal chain at the start of this elog is unity). This corresponds to ~163 counts (since the ADC conversion factor is 2^16 counts over 20volts). The DC signal level I observed is ~200 counts. So things seem roughly consistent.

*Note: Despite my above statement, I don't think it is true that the AS110 PD has more light on it - the BS splitting the light between

AS55 and AS110 PDs is a 50-50 BS, and using the crude method of putting an Ophir power meter in front of both PDs and

monitoring the power while the Michelson was swinging around freely showed roughly the same maximum value.

  13393   Wed Oct 18 19:17:42 2017 gautamUpdateGeneralPRC angular feedforward

Last night, I collected ~30mins of data for the vertex seismometer channels and the POP QPD PIT/YAW signals with the PRMI locked on carrier (angular FF OFF). The ITM Oplev loops weren't DC coupled, as they are in the full IFO locking sequence, but I feel like the angular FF filters can be improved - there are frequent sharp dives in the AS110 signal level which are correlated with large amplitude motion of the POP spot on the control room CCD monitor.

Repeating the frequency domain multicoherence analysis using BS_X and BS_Y seismometer channels as witnesses suggest that we can win significantly (see Attachment #1).

I've never really implemented feedforward filters - I was planning on using ericq's latest entry on this subject as a guide. From what I gather, the procedure is as follows:

  1. Pre-filter the target (POP QPD PIT or YAW) and witness (BS_X, BS_Y) channels
    • Downsample the 2k target data and 256Hz witness data to 32 Hz (how to choose this?)
    • Detrend (linear?)
    • Apply elliptic low pass filter (previously, a 3rd order Elliptic Low pass with 3dB ripple, 40dB stopband attenuation, corner at 5Hz was used).
  2. Filter the target signal (i.e. POP QPD PIT/YAW) by the inverse actuator TF.
    • This "actuator TF" is a measurement of how actuating on the angular DoFs of the PRM affects the POP QPD spot.
    • So by pre-filtering the target signal through the inverse actuator TF, we get a measure of how much the PRM angular motion is.
    • The reason we want to do this is to give the FIR filter that produces optic motion (output) given ground motion sensed by the seismometer (input) fewer poles/zeros to fit (?).
    • The actual actuator TF has to be measured using DTT, and fit - is there anything critical about this fitting? Seems like this should be just a simple pendulum transfer function so a pair of complex poles should be sufficient?
  3. The actual Wiener filter is calculated by the function miso_firlev.m. There are many versions of this floating around from what I can gather.
    • This function requires 3 input parameters.
      • Order of filter to be fit
      • Witness channels (can be multiple)
      • Target channel (has to be single, hence the "miso" in the function name).
    • Today, at the meeting, we talked about weighting the cost function that the optimal Wiener filter calculator minimizes.
    • The canonical wiener filter minimizes the mean squared error between the output of the filter and the desired signal profile (which for this particular problem is the angular motion of the PRM, calculated by dividing the target signal by the actuator TF, knowing which we can cancel it out).
    • But as seen in Attachment #1, the main reduction in RMS comes below f=5Hz.
    • So can we weight the cost function more heavily at lower frequencies? From what I can find in previous calculations, it looks like this weighting happens in the pre-filtering stage, which is not the same thing as including the frequency dependent weighting in the calculation of the Weiner filter? The PSD and acf are F.T. pairs per the Wiener-Khinchin theorem so intuitively I would think that weighting in the frequency domain corresponds to weighting on the lags at which the acf is calculated, but I need to think about this.
    • What kind of low-pass filter do we use to prevent noise injection at higher frequencies? Does the optimal filter calculation automatically roll-off the filter response at high frequencies?
  4. As I write this, seems like there is another level of optimization of "meta-parameters" possible in this whole process - e.g. what is the optimal order of filter to fit? what is the optimal pre-filtering of training data? Not sure how much we can gain from this though.

Some notes from Rana from some years ago: https://nodus.ligo.caltech.edu:8081/40m/11519

If anyone has pointers / other considerations I should take into account, please post here.

Attachment 1: pop_feedforward_potential.pdf
pop_feedforward_potential.pdf
  13394   Wed Oct 18 23:11:53 2017 gautamUpdateCDSFEs unresponsive

This happened again just now - it was roughly this time when this happened last night as well.

There was certainly an EPICS freeze of the kind we were used to seeing prior to replacing the martian wireless router sometime in late 2015 (or early 2016?). I was trying to run the dither alignment servos on the Y-arm at this time, and all the StripTool traces flatlined.

I took the opportunity to try accessing testpoints from the iscey ADCs - specifically C1:SUS-TRY_OUT, and it seemed to work just fine. However, I couldn't ssh into c1iscey.

Looking at the dmesg once I was able to ssh in eventually (~2 minutes deadtime tonight, I feel like it was longer yesterday but can't quantify), I see the following: not sure if there are any clues in here, or whether this is the correct log to check. But there are many instances of the nfs server related message in the log. Note that the system time-stamp corresponds to when this freeze happened.

[5461308.784018] nfs: server 192.168.113.201 not responding, still trying
[5461412.936284] nfs: server 192.168.113.201 OK
[5461412.937130] systemd[1]: Starting Journal Service...
[5461412.947947] systemd-journald[20281]: Received SIGTERM from PID 1 (systemd).
[5461412.996063] systemd[1]: Unit systemd-journald.service entered failed state.
[5461413.002627] systemd[1]: systemd-journald.service has no holdoff time, scheduling restart.
[5461413.008983] systemd[1]: Stopping Journal Service...
[5461413.014664] systemd[1]: Starting Journal Service...
[5461413.044262] systemd[1]: Started Journal Service.
[5461413.694838] systemd-journald[400]: Received request to flush runtime journal from PID 1

 

  13396   Fri Oct 20 16:30:17 2017 gautamUpdateCDSFB1 installed on shelves

[steve, jamie, gautam]

The machine that now serves as out Frame Builder, FB1, was sitting on top of megatron. I decided that this wasn't ideal, and asked Steve to get some alternative mounting solution. Today, he procured some shelves to put FB1 on. Jamie suggested looking for the slider-rail that came with the machine, and using that instead, as it will allow us to slide FB1 out of the rack as we do megatron and the old FB. But as luck would have it, the distance between the rack vertical posts is 26 inches, but the rail is 27 inches. So we had to accept using the less ideal solution of putting FB1 on two shelves, with no sliding option. Photo to be uploaded shortly.

For this work, I had to shutdown FB1 for about 1 hour between 3pm and 4pm. It seems to have come back up fine now.

  13398   Tue Oct 24 16:22:53 2017 gautamUpdateCDSToy DARM model setup in c1tst

[alex, gautam]

Alex is going to have an undergrad work on a calibration optimization project on the 40m RTCDS system. For this purpose, we wanted to setup a "Simulated DARM loop". Today, Alex and I set this up. I figured we can use the c1tst model for this purpose. We basically copied the topology from Figure 2 of the h(t) paper. Attached are screenshots of the MEDM screens of the system we setup, and the simulink block diagram - the main screen can be accessed from the "SIM PLANT" tab in the sitemp.

It remains to setup the appropriate filters in the filter banks, and an EPICS channel monitor for monitoring the single excitation testpoint in the model. We also did not set up any DQ channels for the time being, as it is not even clear to me what channels need to be DQ-ed.

Attachment 1: TOY_DARM.png
TOY_DARM.png
Attachment 2: TOY_DARM_SIMULINK.png
TOY_DARM_SIMULINK.png
  13404   Sat Oct 28 00:36:26 2017 gautamUpdateCDS40m files backup situation - ddrescue

None of the 3 dd backups I made were bootable - at boot, selecting the drive put me into grub rescue mode, which seemed to suggest that the /boot partition did not exist on the backed up disk, despite the fact that I was able to mount this partition on a booted computer. Perhaps for the same reason, but maybe not.

After going through various StackOverflow posts / blogs / other googling, I decided to try cloning the drives using ddrescue instead of dd.

This seems to have worked for nodus - I was able to boot to console on the machine called rosalba which was lying around under my desk. I deliberately did not have this machine connected to the martian network during the boot process for fear of some issues because of having multiple "nodus"-es on the network, so it complained a bit about starting the elog and other network related issues, but seems like we have a plug-and-play version of the nodus root filesystem now.

chiara and fb1 rootfs backups (made using ddrescue) are still not bootable - I'm working on it.

Nov 6 2017: I am now able to boot the chiara backup as well - although mysteriously, I cannot boot it from the machine called rosalba, but can boot it from ottavia. Anyways, seems like we have usable backups of the rootfs of nodus and chiara now. FB1 is still a no-go, working on it.

Quote:

Looks to have worked this time around.

controls@fb1:~ 0$ sudo dd if=/dev/sda of=/dev/sdc bs=64K conv=noerror,sync
33554416+0 records in
33554416+0 records out
2199022206976 bytes (2.2 TB) copied, 55910.3 s, 39.3 MB/s
You have new mail in /var/mail/controls

I was able to mount all the partitions on the cloned disk. Will now try booting from this disk on the spare machine I am testing in the office area now. That'd be a "real" test of if this backup is useful in the event of a disk failure.

 

 

Attachment 1: 415E2F09-3962-432C-B901-DBCB5CE1F6B6.jpeg
415E2F09-3962-432C-B901-DBCB5CE1F6B6.jpeg
Attachment 2: BFF8F8B5-1836-4188-BDF1-DDC0F5B45B41.jpeg
BFF8F8B5-1836-4188-BDF1-DDC0F5B45B41.jpeg
  13408   Mon Oct 30 11:15:02 2017 gautamUpdateCDSslow machine bootfest + vacuum snafu

Eurocrate key turning reboots today morning for c1psl and c1aux.c1auxex and c1auxey are also down but I didn't bother keying them for now. PSL FSS slow loop is now active again (its inactivity was what prompted me to check status of the slow machines).

Note that the EPCIS channels for PSL shutter are hosted on c1aux.But looks like the slow machine became unresponsive at some point during the weekend, so plotting the trend data for the PSL shutter channel would have you believe that the PSL shutter was open all the time. But the MC_REFL DC channel tells a different story - it suggests that the PSL shutter was closed at ~4AM on Sunday, presumably by the vacuum interlock system. I wonder:

  1. How does the vacuum interlock close the PSL shutter? Is there a non-EPICS channel path? Because if the slow machine happens to be unresponsive when the interlock wants to close the PSL shutter via EPICS commands, it will be unable to. The fact that the PSL shutter did close suggests that there is indeed another path.
  2. We should add some feature to the vacuum interlock (if it doesn't already exist) such that the PSL shutter isn't accidentally re-opened until any vacuum related issues are resolved. Steve was immediately able to identify that the problem was vacuum related, but I think I would have just re-opened the PSL shutter thinking that the issue was slow computer related.
  13410   Mon Nov 6 11:15:43 2017 gautamUpdateCDSslow machine bootfest + IFO re-alignment

Eurocrate key turning reboots today morning for and c1susaux, c1auxex and c1auxey. Usual precautions were taken to minimize risk of ITMX getting stuck.

The IFO hasn't been aligned in ~1week, so I recovered arm and PRM alignment by locking individual arms and also PRMI on carrier. I will try recovering DRMI locking in the evening.

As far as MC1 glitching is concerned, there hasn't been any major one (i.e. one in which MC1 is kicked by such a large amount that the autolocker can't lock the IMC) for the past 2 months - but the MC WFS offsets are an indication of when smaller glitches have taken place, and there were large DC offsets on the MC WFS servo outputs, which I offloaded to the DC MC suspension sliders using the MC WFS relief script.

I'd like for the save-restore routine that runs when the slow machines reboot to set the watchdog state default to OFF (currently, after a key-turning reboot, the watchdogs are enabled by default), but I'm not really sure how this whole system works. The relevant files seem to be in the directory /cvs/cds/caltech/target/c1susaux. There is a script in there called startup.cmd, which seems to be the initialization script that runs when the slow machine is rebooted. But looking at this file, it is not clear to me where the default values are loaded from? There are a few "saverestore" files in this directory as well:

  • saverestore.sav
  • saverestore.savB
  • saverestore.sav.bu
  • saverestore.req

Are the "default" channel values loaded from one of these?

  13412   Tue Nov 7 17:45:05 2017 gautamUpdateLSCDRMI Nosie Budget v3.1

Some days ago, I had tried to measure the SRCL->MICH and PRCL->MICH cross couplings using broadband noise injected between 120-180 Hz, a frequency band chosen arbitrarily, in hindsight, I could have done a more broadband test. I've spent some time including the infrastructure to calculate "White-Noise TFs" in the noise budgeting code, where a transfer function is estimated by injecting a "broadband" excitation into a channel of interest, and looking at the resulting response in MICH. I figured this would be useful to estimate other couplings as well, e.g. laser intensity nosie, oscillator noise etc.

I estimate the transfer function of the coupling using the relation (MICH is the median ASD of the MICH error signal in the below expression, and similarly for PRCL)

|H_{cpl}| = \sqrt{\frac{|\mathrm{MICH}^{2}_{\mathrm{exc}} - \mathrm{MICH}^{2}_{\mathrm{quiet}}|}{|\mathrm{PRCL}^{2}_{\mathrm{exc}} - \mathrm{PRCL}^{2}_{\mathrm{quiet}}|}}

Attachments #1 and #2 show the spectra of the MICH, PRCL and SRCL signals during 'quiet' times and during the injection, while Attachment #3 shows the calculated coupling TFs using the above relation. These are significantly different (more than 10dB lower) than the numbers I reported in elog 13367, where the measurement was made using swept sine. As can be seen in the attached plots, the injected broadband excitation is visible above the nominal noise level, and I calculated the white noise TFs using ~5mins of data which should be plenty, so I'm not sure atm what to make of the answers from swept-sine and broadband injections being so different.

Attachment #4 shows the noise budget from the October 8 DRMI lock with the updated SRCL->MICH and PRCL->MICH couplings (assumed flat, extrapolated from Attachment #2 in the 120-180Hz band). If these updated coupling numbers are to be believed, then there is still some unexplained noise around 100Hz before we hit the PD dark noise. To be investigated. But if Attachment #4 is to be believed, it is not surprising that there isn't significant coherence between SRCL/PRCL and MICH around 100Hz.

Nov 8 1600: Updating NB to inculde estimated Oplev A2L.

Quote:
 

AUX coupling

This is the other find.

  • While chatting with Gabriele, he suggested measuring the SRCL->MICH and PRCL->MICH cross couplings.
  • I injected a signal in SRCL servo EXC channel, and adjusted amplitude till coherence in MICH_IN1 was good.
  • The actual TF measured was MICH_IN1 / SRCL_IN1 (so units of cts/ct).
  • My multiplying the in-lock PRCL and SRCL IN1 signals by these coupling coefficients (assumed flat in frequency for now, note that measurement was only made between 100Hz and 1kHz), I get the trace labelled "AUX coupling" in Attachment #1 (this is the quadrature sum for SRCL and PRCL couplings).
  • Also repeated for PRCL -> MICH coupling in the same way.
  • Measurements of these TFs and coherence are shown in Attachment #5 (again png screenshot because of DTT).
  • However, there is no significant coherence in MICH/SRCL or MICH/PRCL in this frequency range.

This seems to be limiting us from saturating the dark noise once the coil de-whitening is engaged. But lack of coherence means the mechanism is not re-injection of SRCL/PRCL sensing noise? Need to think about what this means / how we can mitigate it.


Attachment 1: SRCL_MICH_whitenoise_tf.pdf
SRCL_MICH_whitenoise_tf.pdf
Attachment 2: PRCL_MICH_whitenoise_tf.pdf
PRCL_MICH_whitenoise_tf.pdf
Attachment 3: MICH_aux.pdf
MICH_aux.pdf
Attachment 4: C1NB_disp_40m_MICH_NB_2017-10-08.pdf
C1NB_disp_40m_MICH_NB_2017-10-08.pdf
  13413   Tue Nov 7 22:56:21 2017 gautamUpdateLSCDRMI locking recovered

I hadn't re-locked the DRMI after the work on the AS55 demod board. Tonight, I was able to recover the DRMI locking with the old settings.

The feature in the PRCL spectrum (uncalibrated, y-axis is cts/rtHz) at ~1.6kHz is mysterious, I wonder what that's about.

Wasted some time tonight futzing around with various settings because I couldn't catch a DRMI lock, thinking I may have to re-tune demod phases etc given that I've been mucking around the LSC rack a fair bit. But fortunately, the problem turned out to be that the correct feedforward filters were not enabled in the angular feedforward path (seems like these are not SDF monitored). Clue was that there was more angular motion of the POP spot on the CCD than I'm used to seeing, even in the PRMI carrier lock.

After fixing this, lock was acquired within seconds, and the locks are as robust as I remember them - I just broke one after ~20mins locked because I went into the lab. I've been putting off looking at this angular feedforward stuff and trying out some ideas rana suggested, seems like it can be really useful.

As part of the pre-lock work, I dither aligned arms, and then ran the PRCL/MICH dithers as well, following which I re-centered the ITM, PRM and BS Oplevspots onto their respective QPDs - they have not been centered for a couple of months now.

I'm now going to try and measure some other couplings like PSL RIN->MICH, Marconi phase noise->MICH etc.

 

Attachment 1: DRMI_7Nov20178.png
DRMI_7Nov20178.png
  13414   Wed Nov 8 00:28:16 2017 gautamUpdateLSCLaser intensity coupling measurement attempt

I tried measuring the coupling of PSL intensity noise by driving some broadband noise bandpassed between 80-300Hz using the spare DAC channel at 1Y3 that I had set up for this purpose a couple of weeks ago (via a battery powered SR560 buffer set to low-noise operation mode because I'm not sure if the DAC output can drive a ~20m long cable). I was monitoring the MC2 TRANS QPD Sum channel spectrum while driving this broadband noise - the "nominal" spectrum isn't very clean, there are a bunch of notches from a 60Hz comb and a forest of peaks over a broad hump from 300Hz-1kHz (see Attachment #1).

I was able to increase the drive to the AOM till the RIN in the band being driven increased by ~10x, and saw no change in the MICH error signal spectrum [see Attachment #1] - during this measurement, the RFPD whitening was turned on for REFL11, REFL55 and AS55, and the ITM coil drivers were de-whitened, so as to get a MICH spectrum that is about as "low-noise" as I've gotten it so far.

I tried increasing the drive further, but at this point, started seeing frequent MC locklosses - I'm not convinced this is entirely correlated to my AOM activities, so I will try some more, but at the very least, this places an upper bound on the coupling from intensity noise to MICH.

Attachment 1: PSL_RIN.pdf
PSL_RIN.pdf
  13416   Wed Nov 8 09:59:12 2017 gautamUpdateLSCDRMI Nosie Budget v3.1

The Oplev trace is missing for now, as I have not re-measured the A2L coupling since modifying the Oplev loop shape (specifically the low pass filter and overall gain) to allow engageing the coil de-whitening.

The averaging for the white noise TFs plotted is computed using median averaging - I have used a python transcription of Sujan's matlab code. I use scipy.signal.spectrogram to compute the fft bins (I've set some defaults like 8s fft length and a tukey window), and then take the median average using np.median(). I've also incorporated the ln(2) correction factor.

It seems like GwPy has some in-built capability to compute median (and indeed various other percentile) averages, but since we aren't using it, I just coded this up. 

Quote:

why no oplev trace in the NB ?

also, this method would work better if we had a median averaging python PSD instead of mean averaging as in Welch's method.

 

  13417   Wed Nov 8 12:19:55 2017 gautamUpdateSUScoil driver series resistance

We've been talking about increasing the series resistance for the coil driver path for the test masses. One consequence of this will be that we have reduced actuation range.

This may not be a big deal since for almost all of the LSC loops, we currently operate with a limiter on the output of the control filter bank. The value of the limit varies, but to get an idea of what sort of "threshold" velocities we are looking at, I calculated this for our Finesse 400 arm cavities. The calculation is rather simplistic (see Attachment #1), but I think we can still draw some useful conclusions from it:

  • In Attachment #1, I've indicated with dashed vertical lines some series resistances that are either currently in use, or are values we are considering.
  • The table below tabulates the fraction of passages through a resonance we will be able to catch, assuming velocities sampled from a Gaussian with width ~3um/s, which a recent ALS study suggests describes our SOS optic velocity distribution pretty well (with lcoal damping on).
  • I've assumed that the maximum DAC output voltage available for length control is 8V.
  • Presumably, this Gaussian velocity distribution will be modified because of the LSC actuation exerting impulses on the optic on failed attempts to catch lock. I don't have a good model right now for how this modification will look like, but I have some ideas.
  • It would be interesting to compare the computed success rates below with what is actually observed.
  • The implications of different series resistances on DAC noise are computed here (although the non-linear nature of the DAC noise has not been taken into account).
Series resistance [ohms] Predicted Success Rate [%] Optics with this resistance
100 >90 BS, PRM, SRM
400 62 ITMX, ITMY, ETMX, ETMY
1000 45 -
2000 30 -

So, from this rough calculation, it seems like we would lose ~25% efficiency in locking the arm cavity if we up the series resistance from 400ohm to 1kohm. Doesn't seem like a big deal, becuase currently, the single arm locking

Attachment 1: vthresh.pdf
vthresh.pdf
ELOG V3.1.3-