Guralp Vert1b and Guralp EW1b are plugged back in to PEM ADCU #10 and #12 respectively. Guralp NS1b remains plugged in. So,  PEM-SEIS_MC1_X,Y,Z should now corrsp to seismometer as before.

For the sake of writing it down: /cvs/cds/caltech/apps/linux64/rockNDS

I have done the following on allegra and rosalba:

[root@allegra caltech]# yum install glade2

On rosalba the matplotlib was out of date with respect to allegra.  I have no idea how the version 0.98 on allegra got there, but I left it.  However I updated rosalba to the epel version

  1 yum remove python-numpy
  2 yum install python-matplotlib numpy scipy --enablerepo=epel --disablerepo=rpmforge

This is all to support the LIGO data listener which now has a shortcut on rosalba and allegra's desktop.  It seems to work for (live mode) right now.
 

I added lubricating oil to roughing pumps RP1 and RP3 yesterday and this morning.  Also, I found a nearly full 5 gallon jug of grade 19 oil in the lab.  This should set us up for quite a while.  If you need to add oil the the roughing pumps, use the oil in the quart bottle in the flammables cabinet.  It is labeled as Leybold HE-175 Vacuum Pump Oil.  This bottle is small enough to fill the pumps in close quarters.

The periscope design for beam elevation of the green beams is posted. The design for the 90 deg steering version is also coming.

(2010-03-29: update drawings by daisuke)

90deg version: http://nodus.ligo.caltech.edu:8080/40m/2725

Here the design of the periscope for the 90 deg steering version is posted.

straight version http://nodus.ligo.caltech.edu:8080/40m/2709

I was asked by Koji to point out where a schematic of the triple resonant circuit is.
It seems that I had posted a schematic of what currently is installed (see elog 4562 from almost 4 yrs ago!).

(Some transfomer story)
Then I immediately noticed that it did not show two components which were wideband RF transformers. In order to get an effective turns ratio of 1:9.8 (as indicated in the schematic) from a CoilCrfat's transformer kit in the electronics table, I had put two more transformers in series to a PWB1040L which is shown in the schematic. If I am not mistaken, this PWB1040L must be followed by a PWB1015L and PWB-16-AL in the order from the input side to the EOM side. This gives an impedance ratio of 96 or an effective turns ratio of sqrt(96) = 9.8.

(An upgrade plan document)

Also, if one wants to review and/or upgrade the circuit, this document may be helpful:
https://wiki-40m.ligo.caltech.edu/Electronics/Multi_Resonant_EOM?action=AttachFile&do=get&target=design_EOM.pdf
This is a document that I wrote some time ago describing how I wanted to make the circuit better. Apparently I did not get a chance to do it.

 Very nice error signal. Still, I think we need to take into account the frequency shape of the transfer function TR -> CARM. 

Yend seismometer isolation kit (elog 8461) hosts Guralp seismometer. I made a cable for inside connection, assembled the kit and relocated the instrument from its previous position at the yend inside the kit.

Seismometer is connected to the readout box and running.

Eric, Den

We noticed that PRMI RIN is high when the cavity is locked on RELF33 I&Q signals. We compared the level of power fluctuations when PRMI was locked using REFL11, REFL55 and REFL 33. Attached plot "prmi_rin" shows the spectra.

Then we excited PRM and measured length to RIN coupling when PRMI was locked on REFL33 I&Q. DC offset of PRCL is only 3%. But MICH offset seems to be ~nm. When we gave offset of -15 cnts to the servo, power fluctuations improved by a factor of few.

Then we looked at DRMI. It seems that SRC macroscopic length is off but we still could lock it stably. To account for macroscopic length detuning we had to rotate REFL55 phase from 25 degrees to 50 degrees. Power at AS port increased by factor of ~2 compared to PRMI configuration. SPOP18 is decreased only by 30%. Attached plot "drmi_power" shows POP18, POP90, POPDC and ASDC in PRMI and DRMI configurations.

Plot "src_ol" shows srcl OL transfer function. UGF is 70 Hz. We have also centered SRM OL and copied the servo filters from PRM, gains are set to keep UGF at ~0.1 Hz and 7 Hz

This is a more detailed procedure:

1. Phase: REFL11: 19 degree, REFL55: 50 degrees (25 degrees for PRMI configuration)

2. Input matrix:

3. Servo parameters:

PRCL: gain = -0.02, FM4,5 + trigger FM2,3,6,9

MICH: gain = 0.8, FM4,5 + trigger FM2

SRCL: gain = 0.25, FM4,5 + trigger FM2,3

4. Triggering:

signal is SPOP22 , levels 40:25

 Xend internal cabling and external connector is ready. We are waiting for seismometer from Gyro lab. We still need to fix the pot with clamps after we put the instrument in.

We also need a long cable from Xend to the guralp readout box.

Tonight I have modified transition steps from als to pdh signals. I have added 1:20 filters to CARM_A and DARM_A filter banks to make them unconditionally stable. These filters made locking more robust -- duty cycle is was ~70% tonight. I have also modified slow/ao crossover to avoid ringing up of lines above 1kHz.

Once AO is engaged with high bandwidth, REFL55 signal looks good and I transition PRCL from 165I to 55I. Optical gain compared to PRMI reduced from 55I/165I = -330 down to 55I/165I = 30 in full lock.

I worked on alignment of ETMs. Looking on the cameras I could improve arm power up to 160 and ifo visibility was 80%. POP22 fluctuated by ~50% and every few minutes we loose lock because POP22 almost touches zero.

Tonight I measured seismic noise coupling to beam spot on PR2. There is coherence of 0.9 from X to PIT and Y to YAW around the stack resonances. TF was fited using vectfit and put into static matrix of oaf in the elements T240X -> PRM PIT, T240Y -> PRM YAW. I think we should actuate on the error point of the PRM OL but I decided not to go for a model change tonight. Data from seismometers and POP QPD was obtained during the UTC time 04:06:00 - 04:50:00 when PRMI was locked on sideband

Interferometer was locking rather robustly and every lock lasted on the everage of 3 minutes. During these lock periods I incresed bandwidth of optical lever servos of BS and test masses from 4Hz up to 10Hz and then closed transmission QPD loops. It seems from the camera that lock losses correspond to strong motion of the beam on pop camera. Scripts that change OPLEV bandwidth are in /users/den "increase_ol_bandwidth.sh" "decrease_ol_bandwidth.sh". Script "engage_qpd_servos" turns off ETM oplevs and turns on ETM -> trans QPD servos. These scripts can be copied to locking directrly if are useful.

Please, note that transition from 3f to 1f should still be tuned. Only PRCL was stably controlled using 1f so far

Allegra and Rossa wiped and updated to Ubuntu 12.04.5 by me and Ericq; the following procedure was followed:

1) create "controls" user with uid=1001, gid=1001
2) setup network configuration (IP, Mask, Gateway, DNS), .bashrc, /etc/resolv.conf
3) add synaptic package manager (Ubuntu Software Center used by default)
4) add package nfs-common (and possibly libnfs1) to mount nfs volumes; mount nfs volume adding the line "chiara:/home/cds/       /cvs/cds/       nfs     rw,bg   0    0" in /etc/fstab
5) add package firmware-linux-nonfree, needed for graphics hardware recognition (ATI Radeon HD 2400 Pro): due to kernel and xorg-server versions of 12.04.5, and because ATI dropped support for legacy cards in their proprietary driver fglrx the only solution is to keep Gallium drivers
6) add packages libmotif3 grace, needed by dataviewer
7) add repository from https://www.lsc-group.phys.uwm.edu/daswg/download/repositories.html (Debian Squeeze); install lcssoft-archive-keyring as first package or apt-get will complain
8) add package lscsoft-metaio libjpeg62, needed by diaggui/awggui (Ericq: used lalmetaio on rossa)
9) add packages python-numpy python-matplotlib python-scipy ipython
10) change ownership of /opt/ to controls:controls
11) add csh package
12) add t1-xfree86-nonfree ttf-xfree86-nonfree xfonts-75dpi xfonts-100dpi, needed by diaggui/awggui (needs reboot)
13) add openssh-server

Ubuntu creates the first user during installation with uid=1000 and gid=1000; if needed, they could be changed afterwards using a second user account and the following procedure (twice, if the second users gets the 1001 uid and gid):

sudo usermod -u <NEWUID> <LOGIN>   
sudo groupmod -g <NEWGID> <GROUP>
sudo find /home/ -user <OLDUID> -exec chown -h <NEWUID> {} \;
sudo find /home/ -group <OLDGID> -exec chgrp -h <NEWGID> {} \;
sudo usermod -g <NEWGID> <LOGIN>

Today I started looking into the WFS problem and improvement, after being briefed by Koji and Nicholas. I started taking some measurements of open loop transfer functions for both PIT and YAW for WFS1, WFS2 and MC2_TRANS. For both WFS1 and 2 there is a peak in close proximity of the region with gain>1, and the phase margin is not very high. Tomorrow I will make measurements of the local damping open loop transfer functions, then we'll think how to improve the sensors' behaviour.

[Diego,Koji]

Today we took some measurements of transfer functions and power spectra of suspensions of the MC* mirrors (open loop), for all the DOFs (PIT, POS, SIDE, YAW); the purpose is to evaluate the Q factor of the resonances and then improve the local damping system.

[Diego, Jenne]

The script is moving forward and we feel we are close, however we still have a couple of issues, which are:

1) some python misbehaviour between the system environment and the anaconda one; currently we call bash commands within the python script in order to avoid using the ezca library, which is the one complaining;

2) the fine scan is somewhat not so robust yet, need to investigate more; the main suspects are the wavelet parameters given to the algorithm, and the Offset and Ramp parameters used to perform the scan.

Here is an example of a best case scenario, with 20s ramp and 500 points:

 Tonight I tried some more tests on the script; it seems to work better, with both performance and robustness improved, although the Xarm behaved badly almost all the time. I did not perform all the tests I wanted because the ALS lock was pretty unstable tonight (not only because of the X arm), with more than a few lock losses; after the last lock loss, however, I couldn't restore the Xarm. I'll do some more tests as soon I can recover it, or post the result of the first batch of tests.

In addition, I encountered the following error multiple times, but I have no idea about what could it be:

Thu Nov 06 02:00:13 PST 2014
medmCAExceptionHandlerCb: Channel Access Exception:
Channel Name: Unavailable
Native Type: Unavailable
Native Count: 0
Access: Unavailable
IOC: Unavailable
Message: Virtual circuit disconnect
Context: fb.martian.113.168.192.in-addr.arpa:5064
Requested Type: TYPENOTCONN
Requested Count: 0
Source File: ../cac.cpp
Line number: 1214
 

EDIT on X arm: I found different settings in C1SUS_ITMX, with respect to ETMX, ITMY and ETMY (namely LSC/DAMP is OFF and LSC/BIAS is ON); I don't know if this is intended or for some reason ITMX was not recovered properly after the lock loss, so I didn't change anything, but it may be worth looking into that.

Still no luck in recovering the X arm, I am giving up for tonight; honestly I didn't try many things, as I don't know well the system and didn't want to mess things up.

Preliminary results so far:

I confirm that the best settings for the ramp of the ALS scan are 20s and 500 points; this causes however the script to be fairly slow (80s for the scan/data collection, 7s for the coarse peak finding, 17s for the fine peak finding, total ~2 min); in the best cases the TR*_OUT obtained is around 0.90, as shown in the first plot (early in the evening, all the following plots are in chronological order, if that can help finding the reason for the X arm misbehaviour...):

However, after a few minutes somehow the TR*_OUT went down a bit, without any kind of intervention; also, it is visible the instability of the X arm:

Even when X arm was somewhat stable, its performance and robustness were (far) worse than the Y arm ones:

The following plot shows (about the Y arm only) that there is still some margin, as the maximum value of TRY_OUT is not completely kept at the end of the procedure:

Finally the last plot I managed to obtain, before the X arm went completely crazy...

The next step, after obviously figuring out the X arm situation, is to try some averaging during the fine scan, I don' t know if this will improve the situation, however it shouldn't impact on the execution time. Tomorrow I'll post something more detailed on the script itself and the wavelet implementation.

[Diego, Koji]

X arm has been restored, after modifying the two parameters mentioned in http://nodus.ligo.caltech.edu:8080/40m/10676 (C1SUS_ITMX:  LSC/DAMP and LSC/BIAS); after that, a manual re-alignment of ETMX was necessary due to heavy PIT misalignment. I will check the ALS lock once work on the Y arm is done.

Yesterday I did some more tests with a modifies script; the main difference is that scipy's default wavelet implementation is quite rigid, and it allows only very few choices on the wavelet. The main issue is that our signal is a real, always positive symmetrical signal, while wavelets are defined as 0-integral functions, and can be both real or complex, depending on the wavelet; I found a different wavelet implementation, and I combined it with some modified code from the scipy source, in order to be able to select different wavelets. The result is the wavelet_custom.py module, which lives in the same ALS script directory and it is called by the script. In both the script and the module there the references I used while writing them. It is now possible to select almost any wavelet included in this custom module; "almost" means that the scipy code that calls the find_peaks_cwt routine is picky on the input parameters of the wavelet function, I may dig into that later. For the last tests, instead of using a Ricker wavelet (aka Mexican hat, or Derivative of Gaussian Order 2), I used a DOG(6), as it also has two lesser positive lobes, which can help in finding the resonance; the presence of negative lobes is, as I said, unavoidable. I attach an example of the wavelet forms that are possible, and in my opinion, excluding the asymmetric and/or complex ones, the DOG(6) seems the best choice, and it has provided slightly better results. There are other wavelet around, but they are not included in the module so I should implement them myself, I will first see if they seem fitting our case before starting writing them into the module. However, the problem of not finding the perfect working point (the "overshoot-like" plot in my previous elog) is not completely solved. Eric had a good idea about that: during the fine scan, the the PO*11_ERR_DQ signals should be in their linear range, so I could also use them and check their zero crossing to find the optimal working. I will be working on that.

The new nodus machine is being brought to life; until installation is finished and everything is fine, the old nodus will be unharmed. For future reference:

New Nodus hardware:

Case: SuperMicro SC825MTQ-R700U

M/B: SuperMicro X8DTU-F

CPU: 2x Intel Xeon X5650

RAM: 3x Kingston KVR1333D3S8R9S (2GB)

         3x Samsung M393B5673EH1-CH9 (2GB)

           Total 12 GB

HDD: Seagate ST3400832AS (400GB)

Current software situation and current issues :

1) Ubuntu Server 12.04.5 is installed and updated

2) The usual 'controls' user is present, with UID=1001 and GID=1001

3) Packages installed: nfs-common, rpcbind, ntp, dokuwiki, apache2, php5, openssh-server, elog-2.8.0-2 [from source], make, gcc, libssl-dev [dependencies for elog], subversion

4) Network: interface eth0 is set up (static IP and configuration in /etc/network/interfaces); eth1 is recognized and added, but not configured yet

5) DNS: configuration is in /etc/resolvconf/resolv.conf.d/base (since /etc/resolv.conf is overwritten by the resolvconf program using the 'base' database)

6) ntp is installed and (presumably) configured, but ntpd misbehaves (namely, all the servers are found, but a 

tail /var/log/syslog

shows that no actual synchronization is performed, and the daemon keeps

Listening on routing socket on fd #22 for interface updates

7) dokuwiki apache2 php subversion elog are installed but not configured yet (I need info about their current state, configuration and whereabouts)

8) I copied and merged the old nodus' .bashrc and .cshrc into new nodus' .bashrc, need to know if something has to be added

9) backup frames, backup user dirs and 40m public_html are not set yet, as in #7

Is there something missing?

If there is something missing from here (ligo/cds software, smartmontools/hddtemp and similar, or anything else) tell me and I'll set them up.

EDIT: some images look bad on the elog, and the notebook is parsed, which is is bad. Almost everything posted here is in the compressed file attachment.

As we've been discussing, we want to reduce the laser's jitter effect on the QPDs of the OpLevs, without losing sensitivity to angular motion of the mirror; the current setup is roughly described in this picture:

 The idea is to place an additional lens (or lenses) between the mirror and the QPD, as shown in the proposed setup in this picture:

 I did some ray tracing calculations to find out how the system would change with the addition of the lens. The step-by-step calculations are done at the several points shown in the pictures, but here I will just summarize. I chose to put the telescope at a variable relative distance x from the QPD, such that x=0 at the QPD, and x=1 at the mirror.

Here are the components that I used in the calculations:

Propagator

Tilted Mirror

Telescope

I used a 3x3 matrix formalism in order to have easier calculations and reduce everything to matrix multiplications; that because the tilted mirror has an annoying addictive term, which I could get rid of:

Therefore, n the results the third line is a dummy line and has no meaning.

For the first case (first schematic), we have, for the final r and Theta seen at the QPD:

In the second case, we have a quite heavy output, which depend also on x and f:

Now, some plots to help understand the situation.

What we want if to reduce the angular effect on the laser displacement, without sacrificing the sensitivity on the mirror signal. I defined two quantities:

Beta is the laser jitter we want to reduce, while Gamma is the mirror signal we don't want to lose. I plotted both of them as a function of the position x of the new lens, for a range of focal lengths f. I used d1 = d2 = 2m, which should be a realistic value for the 40m's OpLevs.

Plot of Beta

Plot of Gamma

Even if it is a bit cluttered, it is useful to see both of the same plot:

Plot of Beta & Gamma

 Apart from any kind of horrific mistakes that I may have done in my calculations, it seems that for converging lenses our signal Gamma is always reduced more than the jitter we want to suppress. For diverging lenses, the opposite happens, but we would have to put the lens very near to the mirror, which is somehow not what I would expect. Negative values of Beta and Gamma should mean that the final values at the QPD level are on the opposite side of the axis/center of symmetry of the QPD with respect to their initial position.

I will stare at the plots and calculations a bit more, and try to figure out if I missed something  obvious. The Mathematica notebook is attached.

I stared a bit longer at the plots and thanks to Eric's feedback I noticed I payed too much attention to the comparison between Beta and Gamma and not enough attention to the fact that Beta has some zero-crossings...

I made new plots, focusing on this fact and using some real values for the focal lengths; some of them are still a bit extreme, but I wanted to plot also the zero-crossings for high values of x, to see if they make sense.

Plot of Beta and Gamma

Plot of Beta and Gamma (zoom)

If we are not interested in the sign of our signals/noises (apart from knowing what it is), it is maybe more clear to see regions of interest by plotting Beta and Gamma in absolute value:

Plot of Beta and Gamma (Abs)

I don't know if putting the telescope far from the QPD and near the mirror has some disadvantage, but that is the region with the most benefit, according to these plots.

The plots shown so far only consider the coefficients of the various terms; this makes sense if we want to exploit the zero-crossing of Beta's coefficient and see how things work, but the real noise and signal values also depend on the Alpha and Theta themselves. Therefore I made another kind of plot, where I put the ratio r'(Alpha)/r'(Theta) and called it Tau. This may be, in a very rough way, an estimate of our "S/N" ratio, as Alpha is the tilt of the mirror and Theta is the laser jitter; in order to plot this quantity, I had to introduce the laser parameters r and Theta (taken from the Edmund Optics 1103P datasheet), and also estimate a mean value for Alpha; I used Alpha = 200 urad. In these plots, the contribute of r'(r) is not considered because it doesn't change adding the telescope, and it is overall small.

In these plots the dashed line is the No Telescope case (as there is no variable quantity), and after the general plot I made two zoomed subplots for positive and negative focal lengths.

Plot of Tau (may be an estimate of S/N)

Plot of Tau (positive f)

Plot of Tau (negative f)

If these plot can be trusted as meaningful, they show that for negative focal lengths our tentative "S/N" ratio is always decreasing which, given the plots shown before, it does little sense: although for these negative f Gamma never crosses zero, Beta surely does, so I would expect one singular value each.

I updated the medm C1ASS page for the Arm scripts:

ON : same as before

FREEZE OUTPUTS: calls new FREEZE_DITHER.py script, which sets Common Gain and LO Amplitudes to 0, therefore freezing the current output values

START FROM FROZEN  OUTPUTS: calls new UNFREEZE_DITHER.py script, which sets Commong Gain and LO Amplitudes as in the DITHER_ASS_ON.py script, but no burt restore is performed

OFFLOAD OFFSETS: it's the old "SAVE OFFSETS", calls the WRITE_ASS_OFFSET.py script

OFF: same as before

StripTool: same as before

Tonight I started testing a new method for the fine scan:

• the idea is to use the zero crossings of the PO*11_ERR_DQ signals after (or as an alternative of) the fine scan, but such signals are quite dirty, so I need to find some good way to smooth/filter them;
• I didn't manage to make many tests, because:
  • once arms were locked fine with ALS, the CARM & DARM lock wasn't very robust, in both acquiring and maintaining lock;
  • during the night, the slow OFSs of the arms misbehaved, and at least once per arm they raised their warning box (independently from each other, and it was hastily recovered), even for values that had been perfectly fine before; I am confused about this;
  • as a result, notwithstanding many tries, the beatnotes are gone;
• I have enough information to push the script a little further, but I'll do more testing soon;

 OUTDATED: see elog 10779

I started working on the scripts/FOL directory (I did a backup before tampering around!):

• I still need to make some serious polishing in the folder, and into the Raspberry Pi itself, in order to have a clean and understandable environment;
• as of now, I created an single armFC.c program, which takes as arguments the device (/dev/hidraw0 for the X arm, and /dev/hidraw1 for the Y arm) and the value to write into the frequency counter (0x3 for initialization and 0x2 for actual use); hence, no more need for recompilation!
• I improved the codetorun.py script (and gave the fellow a proper name, epics_channels.py) which handles the initialization AND the availability of the channels;
• On the Raspberry Pi, I created two init scripts, /etc/init.d/epics_server.sh and /etc/init.d/epics_channels.sh, which start at the end of the boot process with default runlevels; the former starts the softIOc process (epics itself), while the latter executes the constantly running epics_channels.py script; as they are services, they can be started/stopped with the usual sudo /etc/init.d/NAME start|stop|restart

As a result, as soon as the Raspberry Pi completes its boot process, the two beatnote channels are immediately available.

 OUTDATED: see elog 10779

 Update and corrections:

• I forgot to log that I added a udev rule in /etc/udev/rules.d/98-hidraw-permissions.rules in order to let the controls user access the devices without having to sudo all the time;
• I updated the ~/.bashrc and /opt/epics/epics-euser-env.sh files to fix syntax errors and add some aliases we usually use;
• since /etc/init.d/ doesn't support automatic respawn of processes, I purged the two scripts I did yesterday and added two lines to /etc/inittab. This works just as fine (I tried a couple of reboots to verify that) and the two processes now respawn automatically even if killed (and, I assume, if they die for any other reason)
• Another thing I forgot: for the time being, during the cleanup, the Raspberry Pi works on the network share script directory. Once cleaning is done and everything is fixed, everything will run locally on the RPi, and the scripts/FOL directory on chiara will be used as backup/repository.

[Diego, Steve]

We ran a Cat 6+ Ethernet cable from the 1X7 rack (where the new nodus is located) to the fast GC switch in the control room rack; now I will learn how to setup the 'outside world' network, iptables, and the like.

I remind that the current hardware/software status is posted in elog 10697 ; if additions or corrections are needed, let me know.

After I check a couple of things, we can use the new nodus (which is currently known in the martian network as rosalba) as a local test to see that everything is working. After that (and, mostly, after I'll have the network working), we will sync the data from the old nodus to the new one and make the switch.

 I finished the polishing in the scripts/FOL directory, this is the current status and this post replaces my two previous posts on the subject:

• the Raspberry Pi operates locally: everything is in its /opt/FOL directory, which is a mirror of /opt/rtcds/caltech/c1/scripts/FOL/ ; some backup/sync scripts should be set up, tell me what kind (sync direction, place to call the script from, etc..) is recommended and I'll set it up;

• the /opt/FOL directory contains:
  • ADC_interface                 : Akhil's ADC interface software and dependencies;
  • akhilTestCodes                : Akhil's work directory with his programs and data;
  • backup                             : two zip files with a full backup of the FOL stuff for both chiara and domenica at 2014/12/08, before my work on the directory;
  • fcreadoutApp                   : the EPICS app compiled on domenica. I didn't modify anything in particular here, as I don't know much about EPICS Apps; I'm not even sure if it is used by now, as I launch EPICS manually by just giving him a .db file (see below).
  • armFC*                        : it is the single program that constantly fetches data for the channels: it takes as arguments the RPi device (/dev/hidraw0 for the X arm, and /dev/hidraw1 for the Y arm) and the value to write into the frequency counter (0x3 for initialization and 0x2 for actual use); hence, there is no more need for recompilation!
  • epics_channels.py :  this is the new version of the old codetorun.py script; it handles the initialization and the availability of the two beatnote channels;
  • fcreadout / freqCountIOC : these are the binaries of the EPICS apps that I found on chiara/domenica; they are not used as of now, but could be useful;
  • fcreadout.db           : it is the database file that is loaded by EPICS to handle the channels;
  • FOLPID.pl              : the Perl PID controller; it is still the old version, we will work on this one later on (see Manasa's schedule at elog 10760 for info)

• Domenica's environment:
  • as I said, everything runs locally from /opt/FOL;
  • in particular, I added in /etc/inittab two lines that launch EPICS and the python script for the channels; respawn is supported so these processes should always be available. For this to happen, DO NOT MOVE armFC, epics_channels.py and fcreadout.db from the /opt/FOL directory on domenica!
  • I added a udev rule in /etc/udev/rules.d/98-hidraw-permissions.rules to let the controls user access the /dev/hidraw* devices without having to sudo all the time;
  • I updated the ~/.bashrc and /opt/epics/epics-user-env.sh files to fix syntax errors and add some aliases we usually use.

[Diego, EricQ]

Update: work is almost completed; the old nodus is still online, as I don't feel confident to make the switch and leave it on its own for the weekend. However, the new nodus is online with the IP address 131.215.114.87, so everyone can check that everything works. From my tests I can say that:

• ntp is working
• wikis are working:
  
  • AGWD wiki : http://131.215.114.87:30889/wiki/doku.php
  • AIC wiki     : http://131.215.114.87:30889/wiki/doku.php?id=aic_wiki
  • ATF wiki    : http://131.215.114.87:30889/ATFWiki/doku.php
  • Cryo wiki   :  http://131.215.114.87:30889/CryoWiki/doku.php
• public_html is working : http://131.215.114.87:30888/public_html/
• svn is working
• svn/apache ugly web interface is working : http://131.215.114.87:30889/svn/
• new fancy websvn web interface is working : http://131.215.114.87:30889/websvn/
• backup is not working: it will as soon the new nodus will replace the old one, so that I'm sure everything is writing stuff in the same place;
  
• SSL is not working (yet);

After everything will be in place, I will save every reasonably important configuration file of nodus into the svn.

I remind that every change made while accessing the 131.215.114.87 machine will be purged during the sync&switch

 [Diego, EricQ]

SSL, https and backups are now working too!

A backup of nodus's configuration (with some explaining) will be done soon.

[Rana, Diego]

We manually realigned the BS and PRM optical levers on the optical table.

 Nodus should be visible again from outside the Caltech Network; I added some basic configuration for postfix and smartmontools; configuration files and instructions for everything are in the svn in the nodus_config folder

 I found out that the Spectrum Analyzer gives bogus data... Since now is locking time, tomorrow I'll go and figure out what is not working

elog was not responding for unknown reasons, since the elogd process on nodus was alive; anyway, I restarted it. 

[EricQ]

EricQ suggested it may be some NFS related issue: if something, maybe some computer in the control room, is asking too much to chiara, then all the other machines accessing chiara will slow down, and this could escalate and lead to the Big Bad Freeze. As a matter of fact, chiara's dmesg pointed out its eth0 interface being brought up constantly, as if something is making it go down repeatedly. Anyhow, after the shutdown of all the computers in the control room, a  reboot of chiara, megatron and the fb was performed.

[Diego]

Then I rebooted pianosa, and most of the issues seem gone so far; I had to "mxstream restart" all the frontends from medm and everyone of them but c1scy seems to behave properly. I will now bring the other machines back to life and see what happens next.

[Diego, Jenne]

Everything seems reasonably back to normal:

Notes:

• the machines in the control room have been rebooted;
• the c1iscey frontend now behaves;
• I saw on nodus, which remained up and running the whole time, a bunch of   nfs: server chiara is not responding, timed out  messages, belonging to the freezing time; it may be that the sync option for the nfs share is too resource demanding, or some other network issue;
• the FSS was doing strange stuff and the MC couldn't recover the lock; the MCautolocker script wasn't running because of the lock loss of the MC and the lack of communication between the machines; so we did a sudo initctl start MCautolocker on megatron and recovered the MC too.

The error spectra I took so far are not that informative, I'm afraid. The first three posted here refer to Wed 17 in the afternoon, where things were quiet, the LSC control was off and the MC was reliably locked. The last two plots refer to Wed night, while Q and I were doing some locking work; in particular, these were taken just after one of the locklosses described in elog 10814. Sadly, they aren't much different from the "quiet" ones.

I can add some considerations though: Q and I saw some weird effects during that night, using a live reading of such spectra, which couldn't be saved though; such effects were quite fast both in appearance and disapperance, therefore difficult to save using the snapshot measurement, which is the only one that can save the data as of now; moreover, these effects were certainly seen during the locklosses, but sometimes also in normal circumstances. What we saw was a broad peak in the range 5e4-1e5 Hz with peak value ~1e-5 V/rtHz, just after the main peak shown in the attached spectra.

In the plot it is shown the behaviour of the PSL-FSS_SLOWDC signal during the last week; the blue rectangle marks an approximate estimate of the time when the scripts were moved to megatron. Apart from the bad things that happened on Friday during the big crash, and the work ongoing since yesterday, it seems that something is not working well. The scripts on megatron are actually running, but I'll try and have a look at it.

 [Jenne, Rana, EricQ, Diego]

Tonight we worked on getting the IFO back in a working status after the break, and then tried some locking.

• the MC is behaving better, it could stay in a stable condition for hours, even if a couple of times it lost lock, and one of them persisted for a little time;
• we managed to get to arm power of 20ish, before losing lock (this happened a couple of times);
• the main thing seems to be that we have only ~ 20 degrees of phase margin at UGF for DARM, which is evidently too little;
• one hypothesis is that DARM may change sign due to some weird length/angular interaction, and that this messes up the actuation causing the lockloss;
• one other possibility is that maybe, when arm power rises, there are some weird flashes that go back to the MC and then cause the locklosses, but this has to be verified;
• attached there is a plot of the last lockloss (and a zoom of it), which seems to point at DARM as the culprit;

We left the IFO uncontrolled and in a "flashy" state so that tomorrow we can look into the "back-flashing to the MC" hypothesis.

 [Jenne, Diego]

Today we began implementing the UGF Servos. Things we did:

• we updated the LSC model with both DARM and CARM servos, and moved them from after the control system to before it, at the level of the error signal;
• we updated the medm screens; new buttons are located in the main LSC screen;
• we started commissioning the DARM servo, at first using DARM for the lock of the single Y arm, then we moved on to the PRFPMI lock and the usual transition from ALS to Transmission;
• although we had several lock losses during the night, we managed to tweak the parameters of the DARM UGF servo (phases, excitation, gains), which now seems to work sufficiently fine;
• the filters added to the I and Q filter banks are a single lowpass in each, while the only filter in the main servio is a standard integrator;
• we don't have a step response at the moment, but we can say that the settling time of the servo is in the range of 10 seconds;
• we updated the ALSdown.py and ALSwatch.py scripts with a call to a new UGFdown.py script; this script, located in the scripts/PRFPMI folder, takes care of disabling the servos and putting the excitation to zero in case of a lock loss; re-enablement of such things must be done manually;