Sorry Jamie, I accidentally deleted your elog entry #11453

Possible explanations:

• The data transfers to LDAS had been shut off while we were doing the DAQ debugging. I don't know if they have been turned back on.  Unlikely this is the problem since you would probably see no data at all if this were the case.
• wiper script parameters might have been changed to store less of the trend data for some reason.
• Frame size is different and therefore wiper script parameters need to be adjusted.
• Steve deleted it all.
• ...

I was looking at the new seismometer data and plotted the coherence between the different arms of C1:PEM_GUR1 and C1:PEM_GUR2. There was not much coherence in the X arms, Y arms, or Z arms of each seismometer, but there were within the x and y arms of the seismometer.

I think the area we should focus on with filtering is lower ranges, between 0.01 and 0.1, because that it where coherence is most clearly high. It is higher in high frequencies but also incredibly noisy, meaning it probably wouldn't be good to try to filter there. 

Koji could pull out easily 11 of the wires  from their socket.

I tested both of the front panels (conductive and isolated SMAs) with the ALS Delay Line Box by driving extremely close frequencies through the cables. By doing this, we would expect that a spike would show up in the PSD if there was crosstalk between the cables. 

In the plots below, for the conductive panel, the frequencies used were

               X Arm:  22.329 MHz                        Y Arm: 22.3291 MHz        

For the isolated panel, the frequencies were

              X Arm: 22.294 MHz                         Y Arm: 22.2943 MHz    

This gives a difference of 100 Hz for the conductive panel and 300 Hz for the isolated panel. Focusing on these areas of the PSD, it can be seen that in the Y Arm cable there is a very clear spike within 30 Hz of these differences when frequencies are being driven through both cables as opposed to the signal being in only the Y Arm. In the X Arms, the noise in general is higher when both cables are on, but there is no distinct spike at the expected frequencies. This indicates that some sort of crosstalk is probably happening due to the strong spikes in the Y Arm cables.          

Previously, I had gotten the same results for the conductive and the isolated front panels. Today, I sanded off the anodized part on the back of the conductive front panel. I checked afterwards with a mulitmeter to ensure that it was indeed conductive through all the SMA connectors. 

I drove a frequency of 29.359 Hz through the X Arm cable and 29.3592 Hz through the Y Arm cable, giving a difference of 200 Hz. Previously, there would only be a spike in the Y Arm at the difference, while the X Arm did not change if the Y arm was on or off. Now that the panel is fully conductive, a spike can also be seen in the X arm, indicating that crosstalk may possibly be happening with this panel, now that the spike corresponds to both the X arm and Y arm. These results are only after one set of data. Tomorrow I'll take two more sets of data with this panel and do a more in depth comparison of these results to what had been previously seen.  

Koji had suggested that I sync up the two function generators to ensure that they have the same base frequency and so that crosstalk will actually appear at the expected frequency. After syncing up the two function generators, I drove the following frequencies through each cable:

Conductive SMAs:

     X: 29.537 MHz           Y: 29.5372 MHz

Isolated SMAs:

     X: 29.545 MHz           Y: 29.5452 MHz

Each time, the difference between the frequencies was 200 Hz, so if there was crosstalk, a spike should appear in the PSDs at 200 Hz when frequencies are being driven through both cables simulataneously, but not when just one is on. We very clearly see a spike at 200 Hz in both the X arm and the Y arm with the conductive SMAs, indicating crosstalk. For the front panel with isolated SMAs, we see a spike at 200 Hz when both frequencies are on, but it is much less pronounced than with the conductive SMAs. It seems as though there will be crosstalk using either panel, just less with the isolated SMAs. 

This continues to happen. I believe the network latency boogeyman is to blame. 

There was a long unlocked period because the enable switch for the MC servo fast path (FASTSW) was left off. Running the mcdown script fixed this, but included the error message:

Channel connect timed out: 'C1:IOO-MC_REFL_GAIN' not found.
CA Client Library: Ignored duplicate create channel response from CA server?

which means the IN1 gain didn't get touched. A second pass of the script produced no errors. 

I'm thinking of adding some logic that if the autolocker has failed to lock for some period (5 minutes?), it should rerun mcdown. 

After testing both the Conductive and Isolated front panels on the ALS delay line box using the actual beatbox and comparing this to the previous setup, I found that the conductive SMAs improved crosstalk the most. Also, as the old cables were 30m and the new ones are 50m, Eric gave me a conversion factor to apply to the new cables to normalize the comparison.

I used an amplitude of 1.41 Vpp and drove the following frequencies through each cable:
      X: 30.019 MHz          Y: 30.019203 MHz

which gave a difference of 203 Hz. 

In the first figure, it can be seen that, for the old setup with the 30m cables, in both cables there is a spike at 203 Hz with an amplitude of above 4 m/s^2/sqrt(Hz). When the 50m cables were measured in the box with the conductive front panel, the amplitude drops at 203 Hz by a factor of around 3. I also compared the isolated front panel with the old setup, and found that the isolated front panel worse by a factor of just over 2 than the old setup. Therefore, I think that using the conductive front panel for the ALS Delay Line box will reduce noise and crosstalk between the cables the most. 

I’m working on a code to determine the Gaussianity of a PSD.

Motivation:

It can be difficult to distinguish between GW events and non-Gaussian noise, especially in burst searches. By characterizing noise Gaussianity, we can better recognize noise patterns and distinguish between GW events and noise.

What I did:

I analyzed an hour of S5 L1 data. First, I plotted a timeseries, just to see what I was working with. Then, I produced a PSD (technically, an ASD) for the timeseries using Welch’s method in Python.

I split the data segment into smaller time-chunks and then produced a PSD for each chunk. All PSDs were superimposed in one plot. Here’s a plot for 201 time-chunks of equal length:

For a specific frequency, I can view the spread in PSD value through the production of a histogram.

Results:

I’ve made histograms displaying varying PSD values for the 201 PSD plot at 100 Hz, 500 Hz, and 1kHz.

For Gaussian noise, an exponential decay plot is expected. I will continue this analysis by following the statistical method in Ando et al. 2003 to calculate specific values indicative of the Gaussianity of various distributions. I’ll then look at different periods of time in the S5 L1 data to find periods of time suggesting non-Gaussian behavior.

I fixed a shaker that was claimed to be broken. I had to cut the rubber membrane to open the head.

Once it was opened, the cause of the trouble was obvious. The soldering joint could not put up with the motion of the head.

It is interesting to see that the spring has the damping layer between the metal sheets.

After the repair the DC resistance was measured. It was 1.9Ohm. The side of the shaker chassis said "3.5Ohm, Max 15VA". So it can take more than 4A (wow).

I gave 2A DC from the bench top supply and turn the current on and off. I could confirm the head was moving.

I'll claim the use of this shaker for the seismometer development.

I've continued to work on my Gaussianity tests for S5 L1 data. 

Following the statistical measure in Ando et al. (2003), I've calculated the Laguerre coefficient, c2, for all frequencies present in my S5 L1 PSD as a metric of Gaussianity. When c2 is zero, the distribution is Gaussian. A positive c2 corresponds to glitchy noise, while a negative c2 suggests stationary noise.

Below is a plot displaying variation in c2 for this PSD:

By observing the c2 value and histogram of distribution of various PSD values at a given frequency, we can elucidate statistical differences in the Gaussian nature of noise at that frequency which are unclear in the standard PSD.

I modified the perl script which does our hourly autoburt so that it can run on megatron instead of op340m (old Solaris machine). Nothing major, just some path stuff. That was the last function of op340m that I know of, so after a week of watching this we ought to be able to power it off and send it to e-waste.

Seems to work so far. It complains about some models that aren't running but mostly it reports successful snapshot taking based on the .req files.

Unfortunately, it seems that its only doing the new target directory, so its missing all of our old VME machines which still use the /cvs/cds/caltech/target area.

But I think Gautam and Jamie and Aidan have volunteered to start our slow controls upgrade by moving the EX slow controls to Acromag and into the new target area. We ought to modify the CRON to point at the old directory for now, but its a temporary fix hopefully.

The MC autolocker and FSSslow scripts weren't running on Megtron. These were started by running the following commands on megatron:

The new autoburt cronjob was failing because the .cron file was not executable (fixed by chmod +x burtnew.cron), and the new perl script didn't use the full path for ifconfig. Similarly, the simulink webview updating script was failing because the full path for matlab wasn't being given. Both of these fixes have been tested and commited to SVN. 

In general, cron scripts can be a real pain, since the cron process doesn't run our .bashrc, and so doesn't know about updates to $PATH, or other environment vairables that get updated through /ligo/cdscfg/workstationrc.sh, which is called by .bashrc. So something that manually works fine in the terminal may not play out as expected when run by cron.

Some things bouncing around my head that haven't made it to ELOG yet:

• Last week, Rana and I were investigating excess power line noise coming from the DFD demodulation. We put transformers on the green beat signals where they arrive at the LSC rack, to avoid connecting their signal ground from the PSL table to the LSC rack ground. This didn't help; it's unclear what the culprit is; maybe the demod board power board?
• Lately, when the interferometer loses lock, the Y arm will not lock on POY, or even flash its IR resonance. for a little while. The green beam can be locked, the X arm can be locked, and no excess angular noise is evident from glancing at the oplev XY plots. Mysterious.  
• Sometimes, when writing new values to the C1LSC SDF table, the c1lscepics process dies (though the write is successful). This is highly annoying. This may have been adressed in some slightly newer RCG code. 
• C1OAF is running with a big red NO SYNC message on its GDS screen. C1LSC has shown this too, but I think only when the SDF/epics crash happens. 
• C1OAF also doesn't seem to properly load the "safe" SDF table when starting up, and errantly puts ones in every element in the static FF matrix. Be careful when restarting OAF!

The new 2x2 fiber couplers arrived today so Eric gave me an overview on the changes to be made to the existing configuration of the FOL fiber box. I removed the box from the table after ensuring that the PDs were powered OFF and removing and capping all fiber leads on the front panel. Here is a summary of the changes made.

• On-Off positions for the rocker switches corrected - these switches for the power to the PDs were installed such that the "1" position was OFF. I flipped both the switches such that the "1" position now corresponds to ON (see Attachment #1).
• All the couplers/beam combiners/splitters were initially removed. 
• I then re-configured the layout as per the schematic (Attachment #2). I only needed to use one of the 4 new 2x2 couplers ordered. I think the 1x2 couplers are appropriate for mixing the PSL and AUX beams, as if we use a 2x2 coupler, half of the mixed light goes nowhere? Indeed, if we had one more such coupler, we could do away with the 2x2 coupler I am now using to divide the PSL light into two. 
• The spec-sheets on the inside of the top cover were updated to reflect the new hardware (Attachment #3).
• The old hardware from the box that was not used, along with their spec-sheets, are stored temporarily in a Thorlabs lab snacks box (all the fibers have been capped).
• The finished layout is shown in Attachment #4.

I then ran a quick check to see what the power levels were at the input to the PDs, using the fiber coupled power meter. However, I found that there was no light in the fiber marked "PSL light in" (the power meter read out "Sig. Low"). The X arm Aux light had an input power of 1.12 mW, which after the various coupling losses etc went down to 63 uW just before the PD. The corresponding figures for the Y arm are 200 uW and 2.2 uW. I am not too sure of how the AUX light is coupled into fibers so I am not trying to tweak the alignment to see if I get more power. 

Eric pointed out that the 1x2 couplers that were used in the previous arrangement and which I recycled, were in fact NOT appropriate - they are not 50-50 couplers but 90-10 couplers, which explains the measured power levels I quoted here.

I switched out these for a pair of the newly arrived 2x2 couplers, and have also replaced the datasheets on the inside of the top cover. I then redid the power level measurements, and got some sensible values this time (see Attachment #1 for revised layout and measured power levels, numbers in red are powers for PSL light, numbers in green are for AUX laser light, and all numbers are in mW). I did find that the 90-10 splitter in the PSL+Y path was not working (though the one in the PSL+X path seems to be working fine), and hence, have not quoted power levels at the output of these splitters. For now, I guess we can bypass the splitters and take the PSL+AUX light from the 2x2 couplers directly to the PDs.

Gautam and Steve,

The decommissioned server from LDAS is retired to the 40m   with 32 cores and 128GB of memory in rack 1X7   http://docs.oracle.com/cd/E19121-01/sf.x4600/

[gautam, ericq]

We had a look at the IR beat (PSL+Xarm) today using the new FOL fiber box, and compared it to the green beat signal for the same combination. We first switched out the green Y beat input into the RF amplifiers on the PSL table with the PSL+Xarm IR beat input (so in all the plots, the BEATY channels really correspond to the IR beat for PSL+X). The IR and green beat notes were found without much difficulty, and we compared the beat signal PSDs for the green and IR signals (see Attachment #1 - arms were locked to green and the X slow control was turned on). The pink trace (labeled REF1) corresponds to the green beat signal, and was in good agreement with an earlier reference trace Eric had saved for the same signal. The teal trace (labeled REF0) corresponds to the the IR beat signal monitored simultaneously. 

We then went back to the PSL table to check the amplitude of the signal from the broadband fiber PDs using the Agilent network analyzer. An initial measurement yielded a beat note (@~50MHz) at ~-22dbm (17mV rms). We figured that by bypassing the 90-10 splitter in this path, we could get a stronger signal. But after switching out the fiber connections we found that the signal amplitude had fallen to ~-27dbm (10mV rms). As per my earlier measurements here, we expect ~600uW of light on the PD, and a quick calculation suggested the signal should be more like 60mV, so we used the fiber power meter to check the power levels after each of the couplers again. We then found that the fiber connector on the front panel of the box for the PSL input wasnt ideal (the laser power after the first 50-50 coupler was only ~250 uW, though the input was ~1.2  mW). The power after the first coupler also fluctuated unpredictably (<100 uW to 350 uW) in response to slightly tightening/loosening the fiber connections on the front panel. I then switched the PSL input to one of the two unused fiber connectors on the front panel (meant for the 10% of the beat signal for the DC readout), and found that this input behaved much better, with ~450 uW of power available after the first 50-50 coupler. The power going into the beat PD was also measured to be ~550uW, closer to what was expected. The beat signal peak now was ~-14dbm (~30mV rms).

We then once again repeated the comparison between green and IR beat signals - but while in the control room, I noticed that the beat signal amplitude on the network analyzer in the control room was fluctuating by nearly 1.5 divisions on the vertical scale - not sure what the reason for this is. A look at the PSD of the IR beat with higher power incident on the PD was also not encouraging (see blue trace in Attachment #1), it seems to have gotten worse in the 10-30 Hz range. We also looked at the coherence between the beat spectrum and the beat note amplitude in order to look for any linear coupling between the two, but from Attachment #2, we cannot explain the disparity between the green and IR beat spectra. This warrants further investigation.

Everything on the PSL table has now been restored to the configurations before these investigations (i.e. the Y+PSL green beat cable has been reconnected to the RF amplifier, and both green beat PDs have been powered back ON. The fiber PDs are powered OFF) 

Eric needed a buffer to drive low input impedance (~130Ohm) of his pomona box, I quickly made a non-iverting buffer with G=+10. The DC power is obtained from the back of SR560. It uses 1.02K and 9.09K
to have the gain of ~10. The chip is OP27. In fact this limits the output swing to be +/-5V for the load resistance of 130Ohm. Eric thinks this is enough. If we need more, we need to swap the chip.

As SR560s tend to saturate too quickly, it would be very useful to have this kind of kit in all the labs
once it is packed in a box.

やったー！Yatta!

In the last few days, with Koji's help, I have recovered both the FS725 Rubidium references from W. Bridge, one from the ATF lab, and one from the CTN lab. Both are back at the 40m at the moment.

However, the one that was recovered from the ATF lab is no longer locking to the Rubidium reference frequency, although it was locked at the time we disconnected it from the ATF lab. I emailed the support staff at SRS, who seem to think that either the internal oscillator has drifted too far, or the Rb lamp is dead. Either ways, it needs to be repaired. They suggested that I run a check by issuing some serial commands to the unit to determine which of these is actually the problem, but I've been having some trouble setting up the serial link - I will try this again tomorrow. I'm also having trouble generating an RMA number that is needed to start the repair/maintenance process, but I've emailed SRS support again and hope to hear back from them soon. 

The other FS725, recovered from the CTN lab earlier today, seems to work fine and is locked to the Rb reference at the moment. I plan to redo the calibration of the phase tracker with an 'absolute' frequency reference with the help of the FS725 and out GPS timing unit tomorrow. Once that is done, the working unit can be returned to the CTN lab. 

In order to synchronise the FS725 Rb clock with our GPS timing signals, I laid out a longish cable running from 1X7 to the IOO rack via the overhead cable guide. There was a T-connector attached to the 1pps output of the GPS timing unit, with one of the outputs unused - I have connected one end of the cable I laid out to this output, with the other end going to the 1pps input of the FS725. I am now waiting for the FS725 to sync to the external reference, before running the calibration of the phase tracker once again using the same method detailed here, using the 10MHz output from the FS725 to serve as a reference for the Fluke RF signal generator...

A Caltech maintenance staff dropped by at around noon today, and told me that he had seen a small puddle of water on the other side of the door along the Y-arm that is kept locked (about 10m from the end-table, on the south side of the arm). He suspected a leak in the lab. Koji and I went down to the said door and observed that there was indeed a small puddle of water accumulated there. There isn't any obvious source of a leak on our side of the door, although the walls tiles in the area suggest that there could be a leak in one of the pipes running through the wall/under the floor. In any case, the leak doesn't seem too dramatic, and we have decided to consult Steve as to what is to be done about this once he is back on Wednesday.

The leak was found inside the wall. Fortunately the plumbers were able to access it from CES room 108

This has been leaking for sometimes. The damaged wall area is about 18 ft long and 1 ft high.

A new type of plot is now available for use in the summary pages, based on EricQ's 2D histogram plots (elog 11210). I have added an example of this to the SandBox tab (https://nodus.ligo.caltech.edu:30889/detcharsummary/day/20151119/sandbox/). The usage is straighforwad: the name to be used in config files is histogram2d; the first channel corresponds to the x-axis and the second one to the y-axis; the options accepted are the same as numpy.histogram2d and pyploy.pcolormesh (besides plot limits, titles, etc.). The default colormap is inferno_r and the shading is flat.

Dataviewer x axis end is not there.

On ( 2600 days) longer plots it is missing 8 moths and on (100 days) shorther plot it is missing 1 month.

SLOWDC servo was dead. I followed EricQ's instruction.

MC Autolocker got stack somewhere. I had to go to megatron and kill MC Autolocker.

init relaunched the autolocker automatically, and now it started properly.

[Eric Q, Gautam, Koji]

We went through the network connections to produce the mapping of the instruments.
Gautam summarized the notes into a spread sheet. See attachments.

We didn't find any irregular connections except for the connection of NETMGR port of c1ioo to Martian Network.
This cable was removed.

I have added a new cron job in pcdev1 at CIT using the 40m shared account. This will run the /home/40m/DetectorChar/bin/cleanarchive script one minute past midnight on the first of every month. The script removes GWsumm archive files older than 1 month old.

The Rubidium standard we had sent in for repair and recalibration has come back. I checked the following:

• Powered the unit on - it was locked to the internal rubidium reference within a few minutes as prescribed in the manual.
• After it had locked to the internal reference, I checked that it was able to lock to an external 1pps reference from our GPS timing unit- this too was achieved within a few minutes as prescribed in the manual. 

However, I am still having trouble setting up a serial communications link with the FS725 with a USB-serial adaptor - I've tried with a Raspberry Pi and my Mac (using screen to try and connect), and also using one of the old Windows laptops lying around on which I was able to install the native software supplied by SRS (still using the USB-serial adaptor to establish connection though). Could it be that the unit is incompatible with the USB-serial adaptor? I had specifically indicated in the repair request that this was also a problem. In any case, this doesn't seem to be crucial, though it would have been nice for diagnostics purposes in the future...

I've stored the repaired FS725 inside the electronics cabinet (marked "Eletronics Modules") for now (the other unit was returned to Antonio in W. Bridge some weeks ago). 

PMC locked manually and PRM sus damping restored.

Projector light bulb ordered

The new wifi router, a Netgear R6400, has been installed, next to the old one which is disconnected (but not yet removed). 

Same SSID, and I've added only the wireless MAC addresses of viviana, paoloa and asia, the three thinkpads inside.

Qualitatively, dataviewer at the X end seems pretty snappy. I'll do some more quantitative comparison of the two routers at some point soon. I will update the wiki, too.

I configured a new wifi bridge for a GPIB Instruments.

The some facts are described on https://wiki-40m.ligo.caltech.edu/Network

The setting up wasn't so straight forward. I added more details there as a linked page.
One thing I had to do with the martian wifi router was that I had to separate the name of SSIDs for 2GHz and 5GHz networks.

Now the data download from Agilent is super fast!
The first establishing the connection takes the most of the time, and the data transfer takes literary nothing.

controls@pianosa|netgpibdata > time ./netgpibdata -i 192.168.113.167 -d AG4395A -a 10 -f meas01
Connecting to host 192.168.113.167, GPIB 10...
done.
Data will be written into meas01.dat.
Parameters will be written into meas01.par.
Writing measurement data to file...
Writing to the parameter file.

real    0m4.056s
user    0m0.068s
sys    0m0.020s

I came into the 40m a few minutes ago, and noticed the following (approximately in this order):

• The striptool plots projected onto the wall were gone, even though the projector seemed to be working fine
• There was no light at all in the IFO 
• There was an incessnt beeping noise coming from inside the lab.

To investigate further, I checked today's summary pages, and whatever caused this, happened around 730am today morning (approx 5 hours ago). I also saw that all the watchdogs were tripped, except MC3, BS and SRM. 

I then tracked down the beeping - I believe that it is coming from Megatron.(in fact, it is coming from the Jetstor..) 

I also found that the PSL is OFF, and the Marconi, though ON, has the display parameters set to values that I normally see when it is first turned ON (i.e. the carrier frequency is 1200MHz, the output is -140dBm etc - this is what led me to suspect that somehow the power connection was interrupted? As far as the workstation computers are concerned, I don't think ROSSA was affected, but pianosa is frozen and donatella is at the login screen. The CDS overview MEDM screen refuses to load correctly (though some of the other MEDM screens are working fine). I'm not entirely sure how to go about fixing all of this, so for now, I'm leaving the PSL off and I've shutdown the remaining watchdogs.

It just occurred to me to check the status of the vacuum - the MEDM screen seems to suggest everything is fine (see Attachment #1). I went down to the X end to do a quick check on the status of the turbo pumps and everything looks normal there...

Looks like that's the case. LIGO GC also sent an e-mail that there was a popwer glitch.

Pasadena reported the Sunday night event as a power transient caused by the trip of a power transmission line. This affected the entire city. Once the loss was detected, the system automatically switches to an alternate source. It was about one second for the system to recover.

2W Innolight, both Lightwaves at the ends, PSL HEPA, Ref Cavity and 3 AC units  turned on.

The 40m vacuum did not trip. It is vacuum normal.

The Jetstore computer is indicating power failer.

Chiara reports an uptime of >195 days, so its UPS is working fine 

FB, megatron, optimus booted via front panel button.

Jetstor RAID array (where the frames live) was beeping, since its UPS failed as well. The beep was silenced by clicking on "View Events/Mute Beeper" at 192.168.113.119 in a browser on a martian computer. I've started a data consistency check via the web interface, as well. According to the log, this was last done in July 2015, and took ~19 hrs.

Frontends powered up; models don't start automatically at boot anymore, so I ran rtcds start all on each of them. 

All frontends except c1ioo had a very wrong datetime, so I ran sudo ntpdate -b -s -u pool.ntp.org on all of them, and restarted the models (just updating the time isn't enough). There is an /etc/ntp.conf in the frontend filesystem that points to nodus, which is set up as an NTP server, but I guess this isn't working.

PMC locking was hindered by sticky sliders. I burtrestored the c1psl.snap from Friday, and the PMC locked up fine. (One may be fooled by the unchanged HV mon when moving the offset slider into thinking the HV KEPCO power supplies need to be brought down and up again, but it's just the sliders)

Mode cleaner manually locked and somewhat aligned. Based on my memory of PMC camera/transmission, the pointing changed; the WFS need a round of MC alignment and WFS offset setting, but the current state is fine for operation without all that. 

 I was able to realign the arms, lock them, and have run the dither align to maximize IR transmission - looks like things are back to normal now. For the Y-end, I used the green beam initially to do some coarse alignment of the ITM and ETM, till I was able to see IR flashes in the control room monitors. I then tweaked the alignment of the tip-tilts till I saw TEM00 flashes, and then enabled LSC. Once the arm was locked, I ran the dither align. I then tweaked ITMX alignment till I saw IR flashes in the X arm as well, and was able to lock it with minimal tweaking of ETMX. The LSC actuation was set to ETMX when the models were restarted - I changed this to ITMX actuation, and now both arms are locked with nominal IR transmissions. I will center all the Oplev spots tomorrow before I start work on getting the X green back - I've left the ETM Oplev servos on for now.

While I was working, I noticed that frame builder was periodically crashing. I had to run mxstream restart a few times in order to get CDS back to the nominal state. I wonder if this is a persistent effect of the date/time issues we were seeing earlier today?

daqd has indeed continued to be unstable. I found system times had drifted apart again... I think something weird happened in the booting of the frontends. The monit processes were not running on any of the frontends. I ntpdate'd again, and manually started monit on each fronted via sudo /etc/init.d/monit start. 

10:15 power glitch today. ETMX Lightwave and air conditions turned back on

The CDS situation was not as catastrophic as the last time, it was sufficient for me to ssh into all the frontends and restart all the models. I also checked that monit was running on all the FEs and that there was no date/time issues like we saw last week. Everything looks to be back to normal now, except that the ntpd process being monitored on c1iscex says "execution failed". I tried restarting the process a couple of times, but each time it returns the same status after a few minutes.

Batteries replaced in control room UPS after 3 years from replaceUPSbattery.com

I configured three more mini wifi extender. They are ready to use.

We should add these to the host table (I forgot where it is)

192.168.113.233 NETGEAR_EX3700_1
192.168.113.234 NETGEAR_EX3700_2
192.168.113.235 NETGEAR_EX3700_3
192.168.113.236 NETGEAR_EX3700_4

Just a heads up that some equipment is hooked up at the PSL table for the repaired AUX laser PLL measurement, I plan to continue with it tonight.

I've taken a few spectra that, along with the PZT coefficient from the repair sheet, that suggest the noise level is ok (incoherent sum of AUX and PSL at about ~3e4 / f Hz/rtHz), but calibrated plots, etc. will follow in time.

The free running PSL+AUX beat frequency noise spectrum has been measured via PLL. AUX laser PZT PM and AM responses were measured too. 

Rough notes about these measurements:

Laser -> QWP -> HWP -> PBS -> 10% BS -> Beat
3.4Vpp out of PD, (40% contrast)
20dB Coupler, output to analyzer, coupled output to Mixer (-a few dBm, didn't check specifically)
Mixer: ZP-3+, BLP-5.1 at output
LO: OCXO @ 36MHz 13dBm->5dB Att-> +8dBm LO at Mixer

Got ~65mVpp out of Mixer

Mixer out -> SR560, LP 3Hz, G=500 -> Pomona Summing node -> Laser PZT
~30kHz UGF ~30 deg phase

Spectra, OLG via SR785 taken with free running PSL, anthropomorphic temperature servo. Data sheet calibration used for PZT. SR560 output noise dominates over analyzer, mixer, PD. Spectrum looks ok, I think.

PM measured with AG4395. High impedance probe used for laser PZT, otherwise couldn't lock. PM calibrated via mixer voltage span for fringe-to-fringe. 

PSL beam blocked, AUX power increased to read 8.0V, AM measured with AG4395.

AM/PM doesn't look to dissimilar to old measurements on wiki. ~230kHz looks like a fine modulation freq. 

Still to be done to AUX laser:
- joint PSL/AUX temperature sweeps
- Output power vs. diode current
- Beam profile

I've performed the temperature sweep of PSL vs Innolight 1W AUX laser.

• I followed the procedure in this elog - started by turning of FSS and FSS Slow servos, closed the PSL shutter, noted down the value of PSL temperature
• As noted in elog 3759, there are multiple temperatures at which a beat can be found. I recorded all that I could find. The IR beat frequency was < 20MHz at the temperatures recorded (and had an amplitude of a few dBm, but I used a 20dB coupler to look at the signal on the HP spectrum analyzer
• The PMC unlocked each time I changed the PSL temperature, but the PMC autolocker worked for me every time
• We should use curve 3 in attachment 1, it is the most reliable set of temperatures at which a beat can be found
• PSL diode current was 2.100A, AUX laser diode current was 2.001A
• Attachment 2 is the data

It remains to measure the output power vs diode current, and the beam profile. I will do the latter on the SP table where there is a little more space. Because we have 1W from this NPRO, the knife-edge method requires a power meter that has a large dynamic range and is sensitive enough to profile the beam accurately. After consulting the datasheets of the power meters we have available (Scientech, Ophir and Coherent) together with Koji, I have concluded that the Coherent calorimeter will be suitable. Its datasheet claims it can accurately measure incident powers of up to 100uW, although I think the threshold is more like 5-10mW, but this should still be plenty to get sufficient resolution for a Gaussian intensity profile with peak intensity of 1W. We also checked that the maximum likely power density we are likely to have during the waist measurement process (1W in a beam of diameter 160um) is within the 6kW/cm^2 quoted on the datasheet.

I have moved the 1W Innolight + controller from the PSL table to the SP table for beam profiling.