I just started a measuremtn that will be running for the next hour or so. Please be careful with the interferometer.

Done. IFO available

TL;DR: Have successfully measured the UGF of the AUX laser on my Red Pitaya! Attached is one of my data runs (pdf + txt file). 

---------------------------------------------------------------

• Figured out how to get a rudimentary coherence (use scipy.signal.coherence to get Cxy = abs(Pxy)**2/(Pxx*Pyy), then find what point is the closest to the frequency I'm inserting on that iteration of the swept sine and get the coherence closest to that). Not precisely the coherence at the frequency I'm inserting though, so not perfect... more of a lower bound of coherence.
• Figured out how to get the UGF from the data automatically (no error propagation yet... necessary next step)
• Put my red pitaya in the X-arm AUX laser control electronics (thank you to Anchal for help figuring out where to put it and locking the x-arm.) Counts dropped from 4500 to 1900 with the x-arm locked, so 58% mode matching. I lose lock at an amplitude >0.05 or so.
• Wrote a little script to take data and return a time-stamped text file with all the parameters saved and a time-stamped pdf of the TF magnitude, UGF, phase, and coherence, so should be easy to take more data next time!

----------------------------------------------------------------

• need to take more accurate coherence data
• need to propagate uncertainty on UGF (probably high...)
• take more data with higher coherence (the file attached doesn't have great coherence and even that was one of my better runs, will probably increase averaging since increasing amplitude was a problem)

We measured the TF of the X-arm laser PZT using the Moku so we can begin fitting to that data and hopefully creating a digital filter to cancel out PZT resonances. 

-------------------------------------------------------------

We calculated the DFD calibration (V/Hz) using:

Vrf = 0.158 mV (-6 dBm), Km = 1 (K_phi = Km*Vrf), cable length = 45m,  Tau = cable length/(0.67*3*10^8 m/s) ~ 220 ns. 

We've taken some preliminary data and can see the resonances around 200-300 kHz.

---------------------------------------------------------

Next steps are taking more data around the resonances specifically, calibrating the data using the DFD calibration we calculated, and adjusting parameters in our model so we can model the TF.

Yesterday, we set up the loop to measure the PZT of the transfer function - the MokuLab sends an excitation (note - a swept sine of 1.0 V) to the PZT. The cavity is locked to the PSL and the AUX is locked to the cavity. In order to measure the effect of our excitation, we take the beat note of the PSL and the AUX. This gives us a transfer function as seen in Attachment 1. The sampling rate of the MokuLab is set to 'ultrafast' (125kHz), so we can expect accurate performance upto 62.5kHz, however, in order to improve our readings beyond this frequency, modifications must be made to the script (MokuPhaseMeterTF) to avoid aliasing of the signal. A script should also be written to obtain and plot the coherence between the excitation and our output. 

Also attached are - Attachment 2 -  the circuit diagram of the setup, and Attachment 3 - the TF data calculated.

Edit - the SR560 as shown in the circuit diagram has since been replaced by a broadband splitter (Minicircuits ZFRSC-42-S+).

It was only a mediocre locking night.  I was foiled for a long time by 2 things, both of which are now taken care of in the scripts, so I don't waste so much time again.

• Somehow FM4 (LP700) in the CARM_A filter bank was turned off.  It took me a long time to figure this one out. 
  • At first, I had a 2056Hz oscillation, and then if I notched that, I would have problems at the edges of my notch.
  • I turned on the LP1000 in CARM_B (which we don't usually use) to fight the 2k resonances, but then I had violin mode problems. 
  • I couldn't turn off the violin mode filters on MC2 for the transition, so the edges of these notches were causing instability.
  • Anyhow, in the end, I realized that FM4 of CARM_B wasn't on, but it usually is.
  • It is now turned on in the carm_cm_up.sh script.
• After that fiasco, I had trouble turning on the ASC loops.
  • Turns out we had left the offsets in place from last night in the ASC loops, so that when I zeroed the outputs of the transmission QPDs, the offsets in the ASC loops didn't make any sense, and they pushed the IFO severely out of alignment.
  • Now the ASC down script (which is run by the carm down script) zeros the filter bank offset values

Scripts are checked into the svn.

I used Q's handy-dandy 2D histogram plotter (..../scripts/general/dataHist, which I have taken the liberty of adding to the svn) to set the PRCL offset when I was locked on REFL165.  Here is a version of the plot, when I had an offset of +10 in the PRCL filter bank. There was so much noise on the PRCL input that I quit bothering to try and put in an excitation or ramp the offset value.  Note that I have since moved this offset to PRCL_A's offset instead, so taking this plot again should have PRCL_IN1 centered around zero.

I had trouble doing something similar for PRCL when I was locked on REFL55.  At first, the offset was so poor that POP110 was only about half the value it was when locked on REFL165, and it had a huge amount of RIN.  I tried just doing a z avg of the PRCL_B_IN1 (REFL55I) while locked on REFL165, and that said that REFL55I had an offset of +33.8 counts, so I tried an offset of -33.8 counts to get to zero.  But, that was still terrible for POP110 power.  As I increased the offset, eventually up to +30 counts, POP110 kept getting better and better.  I lost lock at that point (while trying to get 10 sec of histogram data), so I'm not sure that +30 is the final value.  I want to also get equivalent histograms with POP22 and POPDC (and maybe arm transmissions?) as the X-axis on these plots.  There's no excitation, so all of these can be collected at once.

By babysitting the ITM alignment (looking at the rough DC values of the optical lever error siganls), and doing a little adjustment of the ASC differential offset, I was able to keep lock a few times for more than 2 or 3 minutes while all 1f.  Not a whole lot longer than that though, even if I wasn't "poking" the interferometer other than maintaining alignment.

Jamie and I have had a few back-and-forths on this, but I wanted to write down my thoughts on the parts of the SUS infrastructure that we need for the active tip tilts.

I think we want the ASC pitch and yaw filter banks.  I also think we want to change the channel names so that they are C1:SUS rather than C1:IOO, to make scripting easier.  A corollary to this is that we should make the DC bias sliders have the same naming structure as the regular suspensions (C1:SUS-TT#_{PIT/YAW}_COMM).  This makes scripts like the save/restore scripts easier.  If we keep the IOO naming, it would still be convenient to add the _COMM.

I am having trouble imagining what we might want the lockins for, so I propose we leave them out.

Do we want the output matrix (PIT/YAW -> coils) to be a filter bank matrix?  If we want f2a filters, we need to change this to a filter bank matrix.

I also think we want a master on/off switch for the AC actuation (ASC stuff).  We don't have sensors, so we won't have watchdog-ing, but it might be useful to have a 'panic' switch.  Perhaps though if we are careful to set limits on the ASC filter banks, we won't ever have a panic about actuating too hard.

I'm think I'm joining Jamie's side in the medm screen debate....I think we want a separate TT_SINGLE screen, laid out similar to the SUS_SINGLE, but without all of the irrelevant parts.

EDIT:  Yuta just pointed out to me that right now, the TT DC bias sliders are not recorded anywhere (_DQ, conlog,...).  We *must* fix this.

[Mirko, Den, Jenne]

We're modifying the c1oaf model, and we got to talking about the "Fx" part of "FxLMS".  In the past, we put in a guess for the filter.  What if we use the static wiener filter as the F, and send the wiener filtered witness channels to the adaptation algorithm? 

Are the wiener filter and the plant compensation filter ("Fx") the same?  Will this work?  Or is there some reason that they must be different?

Also:  Notes to self:  Put in filtered witness channels as well as regular into Adapt block.  Make sure that the order/number of inputs is correct.

 More meditations...

Mirko and I were talking about the tunability required for each DoF.  For right now, we're going to give each DoF its own mu and tau (adaptation gain and adaptation decay), but we're leaving all of the other things (number of taps, number of witnesses, delay, downsample rate) the same for each DoF's adaptation.  This may need to change later, but we'll get there when we get there. 

The one I'm most concerned about is the number of witnesses...  Mirko is suggesting that we give each adaptation algorithm all witnesses, and unused ones should converge to ~0.  I agree in principle, but I'm not sure that it's actually going to work that way.  I think we may need to be able to tell the algorithm which witnesses to look at for which DoF. 

Also, the Delay.... we may need to adjust it for each DoF.  In Matt's OAF "manual" in elog 395, he mentions the need to calculate the delay based on a plant TF.  Presumably since (except for the MC) all the witnesses come in together, and all the DoFs come in together, the delay should be about the same for all?  We'll have to see...

(Gautam, Pooja)

Aim: To develop medm screen for GigE.

Gautam helped me set up the medm screen through which we can interact with the GigE camera. The steps adopted are as follows:

(i) Copied CUST_CAMERA.adl file from the location /opt/rtcds/userapps/release/cds/common/medm/ to /opt/rtcds/caltech/c1/medm/MISC/.

(ii) Made the following changes by opening CUST_CAMERA.adl in text editor.  

• Changed the name of file to "/cvs/cds/rtcds/caltech/c1/medm/MISC/CUST_CAMERA.adl"
• Replaced all occurences of "/ligo/apps/linux-x86_64/camera/bin/" to "/opt/rtcds/caltech/c1/scripts/GigE/SnapPy_pypylon/" & "/ligo/cds/$(site)/$(ifo)/camera/" to "/opt/rtcds/caltech/c1/scripts/GigE/SnapPy_pypylon/"

(iii) Added this .adl file as drop-out menu 'GigE' to VIDEO/LIGHTS section of sitemap (circled in Attachment 1) i.e opened Resource Palette of VIDEO/LIGHTS, clicked on Label/Name/Args & defined macros as CAMERA=C1:CAM-ETMX,CONFIG=C1-CAM-ETMX in Arguments box of Related Display Data dialog box (circled in Attachment 2) that appears. In Related Display Data dialog box, Display label is given as GigE and Display File as ./MISC/CUST_CAMERA.adl

(iv) All the channel names can be found in Jigyasa's elog https://nodus.ligo.caltech.edu:8081/40m/13023

(v) Since the slider (circled in Attachment 3) for pixel sum was not moving, changed the high limit value to 10000000 in PV Limits dialog box. This value is set such that the slider reaches the other end on setting the exposure time to maximum.

(vii) Set the Snapshot channel C1:CAM-ETMX_SNAP to off (very important!). Otherwise we cannot interact with the camera.

(vii) GigE camera gstreamer client is run in tmux session.

Now we can change the exposure time and record a video by specifying the filename and its location using medm screen. However, while recording the video, gstream video laucher of GigE stops or is stuck.

I tried rebooting megatron, to see if that might help, but everything still acts the same. 

I tried using daqconfig and changed channels from deactiveated to activated.  I learned by activating them all, that the daq can't handle that, and eventually aborts from an assert checking a buffer size being too small.  I also tried activating 2 and looking at those channels, and it looks like the _DAQ versions of those channels work, or at least I get 0's out of C1:TST-ETMY_ASCPIT_OUT_DAQ (which is set in C1TST.ini file).

I've added the SENSOR channels back to the /csv/cds/caltech/chans/daq/C1TST.ini file, and those are again working in data viewer.

At this point, I'm leaving megatron roughly in the same state as last night, and am going to wait on a response from Alex.

Last night, I connected megatron's BO board to the analog dewhitening board.  However, I was unable to lock the y arm (although once I disconnected the cable and reconnected it back the xy220 the yarm did lock).

So either A) I've got the wrong cable, or B) I've got the wrong logic being sent to the analog dewhitening filters.

During testing, I ran into an odd continuous on/off cycle on one of the side filer modules (on megatron).  Turns out I had forgotten to use a ground input to the control filer bank (which allows you to both set switches as well as read them out), and it was reading a random variable.  Grounding it in the model fixed the problem (after re-making).

I've added the control logic for the outputs going to the Contec Digital Output board.  This includes outputs from the QPD filters (2 filters per quadrant, 8 in total), as well as outputs going to the coil input sensor whitening.

At this point, the ETMY controls should have everything the end station FE normally does.  I'm hoping to do some testing tomorrow afternoon with this with a fully locked IFO.

I was talking with Vladimir on Friday discussing the Binary Output board, we looked at the manual for it (Contec DO-32L-PE) and we realized its an opto-coupler isolated open-collector output.  He mentioned they had the right kind of 50 channel breakout board for testing in Riccardo's lab.

This morning I borrowed the 50 channel breakout board from Riccardo's lab, and along with some resistor loads, test the BO board.  It seems to be working, and I can control the output channel on/off state.

The tst model wasn't compiling this morning due to some incorrect lines in the RfmIOfloat.pm file located in /home/controls/cds/advLIgo/src/epics/util/lib file on megatron. 

The error was "Undefined subroutine &CDS::RfmIOfloat::partType called at lib/Parser2.pm line 354, <IN> line 3363."

I changed RfmIO to RfmIOfloat on lines 1 and 6.
Basically the first 6 lines are now

package CDS::RfmIOfloat;
use Exporter;
@ISA = ('Exporter');

sub partType {
        return RfmIOfloat;
}

The tst now compiles.  At the moment, I believe we should be able to switch megatron in for ETMY and attempt to lock the arm.  The whitening/dewhitening filters are still not automatically synced in software and hardware, but I don't think that should prevent locking.

I found megatron in a similar state to that which nodus was in yesterday. Clued by the fact that MCautolocker wasn't executing the mc scripts (as was evident from looking at the wall StripTool trace), I tried ssh-ing into megatron, but was unable to (despite it being responsive to ping requests). So I went into the VEA and plugged in a monitor to megatron - saw nothing on it. With no soft reboot options available, I power cycled the machine via the front panel button. It came back up smoothly. I manually restarted the autolocker, FSSslow and EX thermal control processes (the former two with initctl, while the latter runs in a tmux session). Everything seems alright for now. Not sure how long megatron has been dead for.

IMC was not locked for the past several hours. Turned out MC autolocker was stuck, and I could not ssh into megatron because it was in some unresponsive state. I had to hard-reboot megatron, and once it came back up, I restarted the MCautolocker, FSS slow servo and nds2 processes. IMC re-locked immediately.

I was pulling long stretches of OSEM data from the NDS2 server (megatron) last night, I wonder if this flakiness is connected. Megatron is still running Ubuntu12.

[koji, gautam]

In addition to c1psl needing a reboot, megatron was un-ssh-able (although it was responding to ping). Clue was that the NPRO PZT control voltage was drifting a lot on the StripTool trace. Koji hard-rebooted the machine. Now IMC is locked, and FSS slow servo is also running.

[Rana, Koji, Joe]

We pulled the phase shifters in the 1X2 rack out to make room for megatron.  Megatron will be converted into c1ioo, and the 8 core, 1U computer will be used as c1lsc.  A temporary ethernet cable was run from 1X2 to 1X3 to connect megatron to the same sub-network.

The c1lsc machine was worked on today, setting it up to run the real time code, along with the correct controls accounts, passwords, .cshrc files, etc.  It needs to be moved from 1X1 to 1X4 tomorrow.

 I started reverting Megatron into a standard Ubuntu workstation after Joe/Osamu's attempt to steal it for their real time mumbo jumbo.

First, I installed a hard drive that was sitting around on top of it. That whole area is still a mess; I'm not surprised that we have so many CDS problems in such a chaotic state. There's another drive sitting around there called 'RT Linux' which I didn't use yet.

Second, I removed the ethernet cables and installed a monitor/keyboard/mouse on it.

Then I popped in the Ubuntu 10.04 LTS DVD, wiped the existing CentOS install and started the standard graphical installation of Ubuntu.

Megatron's specs attached: 

I called Alex this morning and explained the problems with megatron.

Turns out when he had been setting up megatron, he thought a startup script file, rc.local was missing in the /etc directory.  So he created it.  However, the rc.local file in the /etc directory is normally just a link to the /etc/rc.d/rc.local file.  So on startup (basically when we rebooted the machine yesterday), it was running an incorrect startup script file.  The real rc.local includes line:

/usr/bin/setup_shmem.rtl mdp mdc&

Hence the errors we were getting with shm_open().  We changed the file into a soft link, and resourced the rc.local script and mdp started right up.  So we're back to where we were 2 nights ago (although we do have an RFM card in hand).

Update:  The tst module wouldn't start, but after talking to Alex again, it seems that I need to add the module tst to the /usr/bin/setup_shmem.rtl mdp mdc& line in order for it to have a shared memory location setup for it.  I have edited the file (/etc/rc.d/rc.local), adding tst at the end of the line.  On reboot and running starttst, the code actually loads, although for the moment, I'm still getting blank white blocks on the medm screens.

Wall StripTool traces showed that IMC has not been locked for at least 8 hours when I came in this morning. Going to the IMC autolocker log, it looks like the last timestamp was at ~6pm yesterday. Megatron was responding to ping, but I couldn't ssh into it. So I went over to the machine and did a hard-reboot via front panel power switch. The computer took ~10mins to come back online and respond to ping. Once it did, I was able to ssh into it. However, trying the usual commands to restart the IMC autolocker and FSS Slow loops didn't work. Specifically, monitoring the logfile with tail -f Autolocker.log, I would see that the autolocker seemed to get stuck after starting the "blinky" script. Trying to restart the process using sudo initctl restart MCautolocker, init would print to shell that the restart had worked, and reported the PID, but the logfile wouldn't update "live" as it should when tail is used with the -f option. All very strange .

Anyways, as a last resort, I kill -9'ed the PID for the init instance, and init automatically restarted the Autolocker - this did the trick, IMC is locked now and logfile seems to be getting updated normally.

I also cleared a bunch of matlab crash dump files in the home directory.

The MC autolocker and FSSslow scripts weren't running on Megtron. These were started by running the following commands on megatron:

The new autoburt cronjob was failing because the .cron file was not executable (fixed by chmod +x burtnew.cron), and the new perl script didn't use the full path for ifconfig. Similarly, the simulink webview updating script was failing because the full path for matlab wasn't being given. Both of these fixes have been tested and commited to SVN. 

In general, cron scripts can be a real pain, since the cron process doesn't run our .bashrc, and so doesn't know about updates to $PATH, or other environment vairables that get updated through /ligo/cdscfg/workstationrc.sh, which is called by .bashrc. So something that manually works fine in the terminal may not play out as expected when run by cron.

SLOWDC servo was dead. I followed EricQ's instruction.

MC Autolocker got stack somewhere. I had to go to megatron and kill MC Autolocker.

init relaunched the autolocker automatically, and now it started properly.

Last night around 5pm or so, Alex had remotely logged in and made some fixes to megatron.

First, he changed the local name from scipe11 to megatron.  There were no changes to the network, this was a purely local change.  The name server running on Linux1 is what provides the name to IP conversions.  Scipe11 and Megatron both resolve to distinct IPs. Given c1auxex wasn't reported to have any problems (and I didn't see any problems with it yesterday), this was not a source of conflict.  Its possible that Megatron could get confused while in that state, but it would not have affected anything outside its box.

Just to be extra secure, I've switched megatron's personal router over from a DMZ setup to only forwarding port 22.  I have also disabled the dhcp server on the gateway router (131.215.113.2).

Second, he turned the mdp and mdc codes on.  This should not have conflicted with c1omc.

This morning I came in and turned megatron back on around 9:30 and began trying to replicate the problems from last night between c1omc and megatron.  I called Alex and we rebooted c1omc while megatron was on, but not running any code, and without any changes to the setup (routers, etc).  We were able to burt restore.  Then we turned the mdp, mdc and framebuilder codes on, and again rebooted c1omc, which appeared to burt restore as well (I restored from 3 am this morning, which looks reasonable to me). 

Finally, I made the changes mentioned above to the router setups in the hope that this will prevent future problems but without being able to replicate the issue I'm not sure.

Megatron's top fan, rear ps, and temperature front panel lights were all lit amber this morning.  I checked the service manual, found at :

http://docs.sun.com/app/docs/prod/sf.x4600m2?l=en&a=view

According to the manual, this means a front fan failed, a voltage event occured, and we hit a high temperature threshold.  However, there were no failure light on any of the individual front fans (which should have been the case given the front panel fan light).  The lights remained on after I shutdown megatron.  After unplugging, waiting 30 seconds, and replugging the power cords in, the lights went off and stayed off.  Megatron seems to come up fine.

I unplugged the IO chassis from megatron, rebooted, and tried to start Peter's plant model.  However, it still prints that its starting, but really doesn't.  One thing I forgot to mention in the previous elog on the matter, is that on the local monitor it prints "shm_open(): No such file or directory" every time we try to start one of these programs.

I've changed megatron's controls account default shell to tcsh (like it was before).  It now sources cshrc.40m in /cvs/cds/caltech/ correctly at login, so all the usual aliases and programs work without doing any extra work.

We are moving towards a first test of getting Kiwamu's green locking signals into the new front end at the new X end, as well as sending signal out to the green laser temperature control.

Towards that end, we borrowed the router which we were using as a firewall for megatron.   At the moment, megatron is not connected to the network.  The router (a linksys N wire router), was moved to the new X end, and setup to act as a firewall for the c1iscex machine.

At this point, we need to figure which channels of the DAC correspond to which outputs of the anti-imaging board (D000186) and coil driver outputs.  Ideally, we'd like to simply take a spare output from that board and bring it to the laser temperature control.  The watchdogs will be disabled when testing to avoid any unfortunate mis-sent signals to the coils.  It looks like it should be something like channels 6,7,8 are free, although I'm not positive if thats the correct mapping or if there's a n*8 + 6,7,8 mapping.

The ADC should be much easier to determine,  since we only have a single 16 channel set coming from the lemo breakout box.  Once we've determined channels, we should be all set to do a test with the green system.

We changed the pointer on /cvs/cds/caltech/target/gds/bin/awgtpman from

/opt/gds/awgtpman    to

/cvs/cds/caltech/target/gds/bin/awgtpman.091215.

Then killed the megatron framebuilder and testpoint manager (daqd, awgtpman), restarted, hit the daq reload button from the GDS_TP screen. 

This did not fix everything. However, it did seem to fix the problem where it needed a rtl_epics under the root directory which did not exist.  Alex continued to poke around.  When next he spoke, he claimed to have found a problem in the daqdrc file.  Specifically, the cvs/cds/caltech/target/fb/ daqdrc file.

set gds_server = "megatron" "megatron" 10 "megatron" 11;

He said this need to be:

set gds_server = "megatron" "megatron"  11 "megatron" 12;

However, during this, I had looked file, and found dataviewer working, while still with the 10 and 11.  Doing a diff on a backup of daqdrc, shows that Alex also changed

set controller_dcu=10  to set controller_dcu=12, and commented the previous line. 

He also changed set debug=2 to set debug=0.

In a quick test, we changed the 11 and 12 back to 10 and 11, and everything seemed to work fine.  So I'm not sure what that line actually does.  However, the set controller_dcu line seems to be important, and probably needs  to be set to the dcu id of an actually running module (it probably doesn't matter which one, but at least one that is up).  Anyways, I set the gds_server line back to 11 and 12, just in case there's numerology going on.

I'll add this information to the wiki.

I did a full make clean and make uninstall-daq-tst, then rebuilt it.  I copied a good version of filters to C1TST.txt in /cvs/cds/caltech/chans/ as well as a good copy of screens to /cvs/cds/caltech/medm/c1/tst/.

Test points still appear to be broken.  Although for a single measurement in dtt, I was somehow able to start, although the output in the results page didn't seem to have any actual data in the plots, so I'm not sure what happened there - after that it just said unable to select test points.  It now says that when starting up as well.  The tst channels are the only ones showing up.  However, the 1k channels seem to have disappeared from Data Viewer, and now only 16k channels are selectable, but they don't actually work.  I'm not actually sure where the 1k channels were coming from earlier now that I think about it.  They were listed like C1:TST-ETMY-SENSOR_UL and so forth.

RA: Koji and I added the SENSOR channels by hand to the .ini file last night so that we could have data stored in the frames ala c1susvme1, etc.

Alex and I took a look at megatron this morning, and it was in the same state I left it on Friday, with file system errors.  We were able to copy the advLIGO directory Peter had been working in to Linux1, so it should be simple to restore the code.  We then tried just running fsck, and overwritting bad sectors, but after about 5 minutes it was clear it could potentially take a long time (5-10 seconds per unreadable block, with an unknown number of blocks, possibly tens or millions).  The decision was made to simply replace the hard drive.

Alex is of the opinion that the hard drive failure was a coincidence.  Or rather, he can't see how the RFM card could have caused this kind of failure.

Alex went to Bridge to grab a usb to sata adapter for a new hard drive, and was going to copy a duplicate install of the OS onto it, and we'll try replacing the current hard drive with it.

The "apt-get update" was failing on some machines because it couldn't find the 'Debian squeeze' repos, so I made some changes so that Megatron could be upgraded.

I think Jamie set this up for us a long time ago, but now the LSC has stopped supporting these versions of the software. We're running Ubuntu12 and 'squeeze' is meant to support Ubuntu10. Ubuntu12 (which is what LLO is running) corresponds to 'Debian-wheezy' and Ubuntu14 to 'Debian-Jessie' and Ubuntu16 to 'debian-stretch'.

We should consider upgrading a few of our workstations to Ubuntu 14 LTS to see how painful it is to run our scripts and DTT and DV. Better to upgrade a bit before we are forced to by circumstance.

I followed the instructions from software.ligo.org (https://wiki.ligo.org/DASWG/DebianWheezy) and put the recommended lines into the /etc/apt/sources.list.d/lsc-debian.list file.

but I still got 1 error (previously there were ~7 errors):

W: Failed to fetch http://software.ligo.org/lscsoft/debian/dists/wheezy/Release  Unable to find expected entry 'contrib/binary-i386/Packages' in Release file (Wrong sources.list entry or malformed file)

Restarting now to see if things work. If its OK, we ought to change our squeeze lines into wheezy for all workstations so that our LSC software can be upgraded.

I would recommend upgrading the workstations to one of the reference operating systems, either SL7 or Debian squeeze, since that's what the sites are moving towards.  If you do that you can just install all the control room software from the supported repos, and not worry about having to compile things from source anymore.

We succeeded in having a stable single arm (Y) lock using Megatron to replace c1iscey.

Now the lock with megatron is pretty easy. Really. It's very cool.

As we saw the oscillation of the YARM servo, we temporalily increased the gain of TRY filter by a factor of 2 (0.003->0.006). Also decreased the gain of YARM servo by the factor of  2 (1->0.5). This makes the servo gain reduced by a factor of 4 in total. This change seemed to come from the change of the ADC/DAC range.

We finally fixed the hi-gain pd transmission communications from Megatron to the c1lsc by tracking down the correct RFM memory location (which is unhelpfully labeled as a qpd channel in both losLinux and lsc40.m).  The memory location is 0x11a1e0, and is refered to as qpdData[3].

I have removed the RFM card from Megatron and left it (along with all the other cables and electronics) on the trolly in front of the 1Y9 rack.

Megatron proceeded to boot normally up until it started loading Centos 5.  During the linux boot process it checks the file systems.  At this point we have an error:

/dev/VolGroup00/LogVol00 contains a file system with errors, check forced

Error reading block 28901403 (Attempt to read block from filesystem resulted short read) while doing inode scan.

/dev/VolGroup00/LogVol00 Unexpected Inconsistency; RUN fsck MANUALLY

So I ran fsck manually, to see if I get some more information.  fsck reports back it can't read block 28901403 (due to a short read), and asks if you want to ignore(y)?.  I ignore (by hitting space), and unfortunately touch it an additional time.  The next question it asks is force rewrite(y)?  So I apparently forced a rewrite of that block.  On further ignores (but no forced rewrites) I continue seeing short read errors at 28901404, *40, *41,*71, *512, *513, etc.  So not totally continugous.  Each iteration takes about 5-10 seconds.  At this point I reboot, but the same problem happens again, although it starts 28901404 instead of 28901403.  So apparently the force re-write fixed something, but I don't know if this is the best way of going about this.  I just wondering if there's any other tricks I can try before I just start rewriting random blocks on the hard drive.  I also don't know how widespread this problem is and how long it might take to complete (if its a large swath of the hard drive and its take 10 seconds for each block that wrong, it might take a while).

So for the moment, megatron is not functional.  Hopefully I can get some advice from Alex on Monday (or from anyone else who wants to chime in).  It may wind up being easiest to just wipe the drive and re-install real time linux, but I'm no expert at that.

In investigating why megatron wouldn't talk to the network, I re-discovered the fact that it had been placed on its own private network to avoid conflicts with the 40m's test point manager.  So I moved the linksys router (model WRT310N V2) down to 1Y9, plugged megatron into a normal network port, and connected its internet port to the rest of the gigabit network. 

Unfortunately, megatron still didn't see the rest of the network, and vice-versa.  I brought out my laptop and started looking at the settings.  It had been configured with the DMZ zone on for 192.168.1.2, which was Megatron's IP, so communications should flow through the router. Turns out it needs the dhcp server on the gateway router (131.215.113.2) to be on for everyone to talk to each other.  However, this may not be the best practice.  It'd probably be better to set the router IP to be fixed, and turn off the dhcp server on the gateway.  I'll look into doing this tomorrow.

Also during this I found the DNS server running on linux1 had its IP to name and name to IP files in disagreement on what the IP of megatron should be.  The IP to name claimed 131.215.113.95 while the name to IP claimed 131.215.113.178.  I set it so both said 131.215.113.178.  (These are in /var/named/chroot/var/ directory on linux1, the files are 113.215.131.in-addr.arpa.zone  and martian.zone - I modified the 113.215.131.in-addr.arpa.zone file).  This is the dhcp served IP address from the gateway, and in principle could change or be given to another machine while the dhcp server is on.

I noticed recently that Megatron was running Ubuntu 12, so I've started its OS upgrade.

1. Unlocked the IMC + disabled the autolocker from the LockMC screen + closed the PSL shutter (IMC REFL shutter doesn't seem to do anythin)
2. Disabled the "FSS" slow servo on the FSS screen
3. did sudo apt-get update, sudo apt-get upgrade, and then sudo apt-get do-release-upgrade which starts the actual thing
4. According to the internet, the LTS upgrades will go in series rather than up to 18 in one shot, so its now doing 12 -> 14 (Trusty Tapir)

Megatron and IMC autolocking will be down for awhile, so we should use a different 'script' computer this week.

Mon Dec 9 14:52:58 2019

upgrade to Ubuntu 14 complete; now upgrading to 16

Megatron is now running Ubuntu 18.04 LTS.

We should probably be able to load all the LSC software on there by adding the appropriate Debian repos.

I have re-enabled the cron jobs in the crontab.

The MC Autolocker and the PSL NPRO Slow/Temperature control are run using 'initctl', so I'll leave that up to Shruti to run/test.

upgrade was done

cronjob testing wasn't one by one 😢 

burt snapshots were gone

i brought them back home 🏠 

The burt snapshotting is still not so reliable - for whatever reason, the number of snapshot files that actually get written looks random. For example, the 14:19 backup today got all the snaps, but 15:19 did not. There are no obvious red flags in either the cron job logs or the autoburt log files. I also don't see any clues when I run the script in a shell. It'll be good if someone can take a look at this.

[Jenne, Den]

Overall a "meh" night for locking I think.  The script to all-RF worked several times earlier in the evening, although it was delicate and failed at least 50% of the time.  Later in the evening, we couldn't get even ~10% of the lock attempts all the way to RF-only. 

Den looked into angular things tonight.  With the HEPA bench at the Xend on (which it was found to be), the ETMX oplevs were injecting almost a factor of 10 noise (around 10ish Hz?) into the cavity axis motion (as seen by the trans QPD) as compared to oplevs off.  Turning off the HEPA removed this noise injection.

Den retuned the QPD trans loops so that they only push on the ETMs, so that we can turn off the ETM oplevs, and leave the ITMs and their oplevs alone. 

We are worried again about REFL55.  There is much more light on REFL55 than there is on REFL11 (a 90/10 beam splitter divides the light between them), and we see this in the DC output of the PDs, but there seems to be very little actual signal in REFL55.  Den drove a line (in PRCL?) while we had the PRMI locked with the arms held off resonance, and REFL55 saw the line a factor of 1,000 less than REFL 11 or REFL165.  The analog whitening gain for REFL11 is +18dB, and for REFL55 is +21dB, so it's not that we have significantly less analog gain (that we think).  We need to look into this tomorrow.  As of now, we don't think there's much hope for transitioning PRMI to REFL55 without a health checkup. 

  I turned on the HEPA at the south end during the LSC. Sorry I ment to turn it off.

Put outside.