New flipping mirror installed on the AP table on the beam path to the REFL199 PD.

If you're missing the double demod signal, please check that it is actually down.

Mode cleaner length measured tonight.

33196198

132784792

165980990

199177188

[Tag by KA: modulation frequency, MC length]

[ Manasa, Ericq and Steve ]

 Vivitek D952HD with 186 hours installed.

 The east and south end of the 40m emergency exit doors received new safety signs.

foreach opt (ETMX ETMY ITMX ITMY MC1 MC2 MC3 BS PRM SRM)
  foreach dof (POS PIT YAW)
     ezcawrite C1:SUS-${opt}_SUS${dof}_INMON.PREC 0
  end
end

New right angle PVC, 2 x 2 x  1/4" installed at the AP table to strain relief the 1/4" spiral corrugated RF coaxes.

The new SOS sus wire finally is stored in a nitrogen filled dessicator. This was recommended by Ca. Fine Wire to minimize the aging - oxidation.

The dessicator was pumped down with " aux-drypump " to 1 Torr and than filled up with N2 to 760 Torr. This was repeated 2x and the dessicator was sealed off.

I just installed cdsutils r351 at /ligo/apps/linux-x86_64/cdsutils.  It should be available on all workstations.

It includes a bunch of bug fixes and feature improvements, including the step stuff that Rana was complaining about.

Cdsutils r361 installed, for the "avg" updates. aLOG  

 Hold on, did the arms get re-baptized?

Four nitrogen cylinders replaced the empties in the rack at the west entrance.  Additionally, Airgas will now deliver only once a week.  Let me know via email or text when the there are four empties in the rack and I'll order the next round.

This was the second week without wet mopping the floor and wet wipe down of the vacuum envelope.

We changed filters on the 40m "HEPA" vacuum pump. It still has very dirty exhaust. Do not use it in clean room!

AA filter box was removed and modified at 1Y7 today. The -5V power supply was current limited when I plugged it back in. It was removed for medical attention.

NO PEM channels available! because of this.

(Joe, Suresh, Yuta)

Currently, only 2 ADC cards work on c1sus machine.
No QPD inputs(e.g. MC2 trans), and no RFM.

Summary:
  We wanted to have PEM(physical environment montor) channels, so we moved a ADC card in c1sus machine.
  It ended up with destroying one of the 3 ADCs.

What we did:
  1. Moved ADC card at PCIe expansion board slot 0 to other empty slot.
     What we call PCI slot 0 was "DO NOT USE" in LIGO-T10005230-v1, so we moved it.

  2. Connected that ADC card to PEM channel box at 1X7 via SCSI cable.

  3. ADC card order is changed, so we checked ADC number assinging and re-labeled the cable.

  4. Found RFM is not working(c1sus and c1ioo not talking) and fb is in a weird state(Status: 0x4000 in GDS screens)

  5. Swapped the cabling so that ADC card 0 will be connected to timing interface card at slot1, but didn't help.
     More than that, we suffered ADC timeout.

  6. Tried ADC card swapping, slot position changing, taking out some of the ADC cards, etc.
     We found that ADC timeout doesn't happen with 2 ADC cards.
     But if we connect one of the ADC card to the timing interface card at slot 8, c1sus ADC timeouts with 2 ADC cards, too.
     So, I think that timing interface card is bad.

  7. Stopped rebooting c1sus again and again. We decided to investigate the problem tomorrow.
     We only need ADC card 0 and 1 for MC damping.(see this wiki page)
       ADC card 0: all UL/UR/LR/LL SENs
       ADC card 1: all SD SENs     
       ADC card 2: all QPDs

Result:
  We can damp optics and lock MC.
  We can't do A2L because RFM is not working.
  We can't see MC2 trans because we currently don't have ADC card 2.

CANCELLED, WILL BE RESCHEDULED*

The Nov.19 interruption was rescheduled to be on Nov.20,2011

          CALIFORNIA INSTITUTE OF TECHNOLOGY

                 FACILITIES MANAGEMENT

            UTILITY & SERVICE INTERRUPTION

**PLEASE POST**

Building:         CENTRAL ENGINEERING SERVICES, STEELE HOUSE, KECK LAB,

                  LIGO GRAVITATION PHYSICS BLDG, SAFETY STORAGE (BY CES)

Date:             Sunday, Nov. 20,2011

Time:             10:00 AM TO 11:00 AM

Interruption:     ELECTRICITY

Contact:          MIKE ANCHONDO X-4999 OR TOM BRENNAN X-4984

*THIS INTERRUPTION IS REQUIRED FOR MAINTENANCE OF HIGH VOLTAGE SWITCHGEAR

IN CAMPUS SUB-STATION.

For some reasons foton's deafault sample rate is NOT correct when it runs on the CentOS machines.

It tries to setup the sample rate to be 2048 Hz instead of 16384 Hz until you specify the frequency.

To avoid an accidental change of the sample rate,

running foton on CentOS is forbidden until any further notifications.

Run foton only on Pianosa.

Additionally I added an alias sentence in cshrc.40m such that people can not run foron on CentOS (csh and tcsh, technically speaking).

Below is an example of raw output when I typed foron on a CentOs machine.

    rossa:caltech>foton
    DO NOT use foton on CentOS


 Still I didn't see any beat note signals..

 With a help from Suresh, Yuta and Rana, I tried searching for the green beat note by changing the temperature of the X end NPRO.

The noise level after it goes through an RF amplifier (G=23dB) was about -70dBm at 50MHz.

This noise may cover the beat note signal.

I am going to post some details later.

Last night I tried searching for a beat note signal with two different PD trans impedance gains.

Although I didn't find a  beat note signal.

- (1. trans impedance gain = 2400)

 I started with a trans impedance resistance of R=2.4k, which is 10 times bigger resistance than the original.

The total PD gain should be about 960 [V/W] theoretically if we assume the responsibility of the PD is 0.4 [A/W].

Then I checked the bandwidth of the RFPD using Jenne laser.

The bandwidth was about 30MHz, which is 3 times narrower than the original. And it agrees with our expectation.

As Koji and I mentioned at the last weekly meeting, the cut off frequency of an RFPD follows inverse square root of the trans impedance resistance R.

where C is a capacitance of the photo diode. (See this)

I was expecting the signal level of  -50 dBm / rtHz with a 23dB RF amplifier, assuming the line width of the signal is 10kHz.

- (2. trans impedance gain = 240) 

 I also tried it with the original trans impedance gain (see this entry). 

 R = 240 [Ohm]

 G = 96 [V/W]

 BW = 100 [MHz]  (I didn't measure it in this time)

 expected signal level = -70 dBm/rtHz

Rana noticed that the sum on WFS2 was about 10 times smaller than that on WFS1. Though the beam appeared centered on the DC QPD screens it was not really true.  When I went and checked the actual beam position it was landing on the metal enclosure of the WFS2 sensor and scattering back on to the diode. 

I also checked the power levels of light landing on the sensors  It was about 0.25mW in both cases.  This needs further investigation since the power split at the beam spitter is like 0.25mW onto WFS1 and 0.45 towards WFS2. The lost 0,20 mW has to be traced and we have to be sure that it is not scattered around on the table.

It turns out there are no reasonable signal from the segment 1 on the IP_ANG QPD.

For right now I can still use it as a funny QPD, but I absolutely need somebody to check and fix it in a daytime.

[Valery, kiwamu, Jenne, Suresh]

    I first interchanged the two QPD's on the Y end table to see if the problem QPD related.  Exchanging the units did not make any difference.  The problem therefore had to be in the cables or the circuit boards in 1X4

    We traced the signals pertaining to the IP_ANG QPD ( "Initial Pointing Beam") using  Jay's wiring diagram (pages 2 and 5 of 7).  We noted that while the signals were available on all Segments till the Monitors (Lemo) on 1X4-2-2A card, two of the lines did not reach the output of the cross connect 1X4-B8.  We checked card to make sure that the signals were indeed reaching the back plane of the 1X4-2 chassis using a D990612 extension board.  The card was found to be okay.  We therefore suspected that the cable (CAB_1X4_?) going from the card to the cross connect 1X4-B8 was faulty.  Indeed visual inspection showed that the crimping of the connector was poor and weight of the cable had put further strain on the crimping.  

   I changed the 64-pin connector on the 1X3-2-2A side of the cable. 

When I connected everything back together the problems persisted. Namely the lines P1-1A  (Segment 1 high) and P1-2C (Segment 2 Low) were floating They were not reaching points 2T and 3T respectively on the output of the cross connect.

   I therefore replaced 1X4-B8 with a similar unit which I found in one of the shelves along the East (Y) arm. 

I then checked with the StripTool to make sure that all the quadrants are showing similar response to a flashlight on the QPD.   All Segments are working fine now. Currently the IR Initial Pointing beam reaches the QPD but is not centered on it. 

I did not attempt to center it since the beam appeared to be clipped and may anyway require repositioning.

JD: We need to meditate on where this beam could be getting clipped.  Suresh and I checked that it's not on the viewport on the beam's way out of the ETMY chamber by seeing that the beam is far away from the edges of the viewport, and also far away from the edges of the black beamtube between the viewport and the table.  Suresh mentioned that the clipping nature of the IP_ANG beam sometimes goes away.  I don't know if this is the same clipping that Kiwamu might be seeing with the main beam, or if this is separate clipping just with the IP beam, after it's been picked off.  I suspect it's the same as what Kiwamu is seeing....maybe when we move PZT1, we clip on one of the MMT mirrors or PZT2??  If this is true, it's a total pain since we might have to vent if we can't steer around it.

No signal is transmitted from C1:GCV-SCX_ETMX_ALS (on c1gcv) to C1:GCV-SCX_ETMX_ALS (on c1scx)

I can't find RFM definition for ALS channels in c1rfm. Where are they???

As you've correctly noted, the source of the C1:GCV-SCX_ETMX_ALS channels is in the c1gcv model. The first 3 letters of the channel name indicate this (GCV).

The destination of this channel is c1scx, the 2nd 3 letters indicate this (SCX). If it passed through the c1rfm model, it would be written like C1:GCV-RFM_ETMX_ALS.

This particular channel doesn't pass through the c1rfm model, because the computers these two run on (c1ioo and c1scx) are directly connected via our old VMIC 5565 RFM cards, and don't need to pass through the c1sus computer. This is in contrast to all communications going to or from the c1lsc machine, since that is only connected the c1sus machine by the Dolphin RFM. The c1rfm also handles a bunch of RFM reads from the mode cleaner WFS, since each eats up 3-4 microseconds and I didn't want to slow the c1mcs model by 24 microseconds (and ~50 microseconds before the c1sus/c1scx computer switch).

So basically c1rfm is only used for LSC communications and for some RFM reads for local suspensions on c1sus.

As for the reason we have no transmission, that looks to be a problem on c1ioo's end. I'm also noticing that MCL is not updating on the MC2 suspension screen as well as no changes to MC PIT and YAW channels, which suggests we're not transmitting properly.

I rebooted the c1ioo machine and then did a burt restore of the c1ioo and c1gcv models. These are now up and running, and I'm seeing both MCL and ALS data being transmitted now.

Its possible that when we were working on the c1gfd (green frequency divider model) on c1ioo machine we disturbed the RFM communication somehow. Although what exactly, I'm not sure.

The /boot partition was filling up with old kernels. Nodus has automatic security updates turned on, so new kernels roll in and the old ones don't get removed. 

I ran apt-get autoremove, which removed several old kernels. (apt is configured by default to keep two previous kernels around when autoremoving, so this isn't so risky)

Now: /dev/sda1                    236M   94M  130M  42% /boot

In principle, one should be able change a setting in /etc/apt/apt.conf.d/50unattended-upgrades that would do this cleanup automatically, but this mechanism has a bug whose fix hasn't propagated out yet (link). So, I've added a line to nodus' root crontab to autoremove once a week, Sunday morning. 

controls@nodus|~ > df -h
Filesystem                   Size  Used Avail Use% Mounted on
/dev/mapper/nodus2--vg-root  355G   69G  269G  21% /
udev                         5.9G  4.0K  5.9G   1% /dev
tmpfs                        1.2G  308K  1.2G   1% /run
none                         5.0M     0  5.0M   0% /run/lock
none                         5.9G     0  5.9G   0% /run/shm
/dev/sda1                    236M  210M   14M  94% /boot
chiara:/home/cds             2.0T  1.5T  459G  77% /cvs/cds
fb:/frames                    13T   11T  1.6T  88% /frames

Zach> Nodus seemed to be working fine again, and I was browsing the elog with no
Zach> problem. I tried making an entry, but when I started uploading a file it
Zach> became unresponsive. Tried SSHing, but I get no prompt after the welcome
Zach> blurb. ^C gives me some kind of tcsh prompt (">"), which only really
Zach> responds to ^D (logout). Don't know what else to do, but I assume someone
Zach> knows what's going on.

By gracefully rebooting nodus, the problem was solved.

It (">") actually was the tcsh prompt, but any commands with the shared or dynamic link libraries looked unfunctional.

I could use
    cd /.../...
and
    echo *
to browse the directory tree. The main mounted file systems like /, /usr, /var, /cvs/cds/caltech looked fine.
I was afraid that the important library files were damaged.

I tried
    umountall
and
    mountall
in order to flush the file systems.
These should run even without the libraries as mount must properly work even before /usr is mounted.

They indeed did something to the system. Once I re-launch a new login shell, the prompt was still ">"
but now I could use most of the commands.

I have rebooted by usual sudo-ing and now the services on nodus are back to the functional state again.

# nodus was working in the evening at around 9pm. I even made an e-log entry about that.
# So I like to assume this is not directly related to the linux1 incident. Something else could have happened.

SVN down

mafalda down

I am guessing that the NFS file system hangup may have caused some machines to get into an awkward state. We may be best off doing a controlled power cycle of everything...

svn is back after starting apache on nodus.

http://lhocds.ligo-wa.caltech.edu:8000/40m/ApacheOnNodus

mafalda is up now.

I found that the cable for mafalda (the sole red cable) had a broken latch.
The cable was about falling off from the switch. As a first-aid, I used this technique to put a new latch, and put it into the switch.

Now I can logged in it. I did not rebooted it.

The post OS migration admin for nodusa bout apache, elogd, svn, iptables, etc can be found in https://wiki-40m.ligo.caltech.edu/NodusUpgradeNov2017

Update: The svn dump from the old svn was done, and it was imported to the new svn repository structure. Now the svn command line and (simple) web interface is running. And "websvn" was also implemented.

I restored the nodus crontab (copied over from the Nov 17 backup of the same at /opt/rtcds/caltech/c1/scripts/crontab/crontab_nodus.20171117080001. There wasn't a crontab, so I made one using sudo crontab -e.

This crontab is supposed to execute some backup scripts, send pizza emails, check chiara disk usage, and backup the crontab itself.

I've commented out the backup of nodus' /etc and /export for now, while we get back to fully operational nodus (though we also have a backup of /cvs/cds/caltech/nodus_backup on the external LaCie drive), they can be re-enabled by un-commenting the appropriate lines in the crontab.

Confirmed that this crontab is running - the daily backup of the crontab seems to have successfully executed, and there is now a file crontab_nodus.ligo.caltech.edu.20171122080001 in the directory quoted below. The $HOSTNAME seems to be "nodus.ligo.caltech.edu" whereas it was just "nodus", so the file names are a bit longer now, but I guess that's fine...

We changed the password for controls on nodus this afternoon. We also zeroed out the authorized_keys file and then added back in the couple that we want in there for automatic backups / detchar.

Also did the recommended Ubuntu updates on there. Everything seems to be going OK so far. We think nothing on the interferometer side cares about the nodus password.

We also decided to dis-allow personal laptops on the new Martian router (to be installed soon).

nodus was rebooted by Alex at Fri Aug 14 13:53. I launched elogd.

cd /export/elog/elog-2.7.5/
./elogd -p 8080 -c /export/elog/elog-2.7.5/elogd.cfg -D

 It looks like Alex also rebooted all of the control room computers.  Or something.  The alarm handler and strip tool aren't running.....after I fix susvme2 (which was down when I got in earlier today), I'll figure out how to restart those.

Installed libmotif3 and libmotif4 on nodus so that we can run dataviewer on there.

Also, the lscsoft stuff wasn't installed for apt-get, so I did so following the instructions on the DASWG website:

https://www.lsc-group.phys.uwm.edu/daswg/download/repositories.html#debian

Then I installed libmetaio1, libfftw3-3. Now, rather than complain about missing librarries, diaggui just silently dies.

Then I noticed that the awggui error message tells us to use 'ssh -Y' instead of 'ssh -X'. Using that I could run DTT on nodus from my office.

I just did remote apt-get update, apt-get upgrade, and then reboot on nodus. ELOG started up by itself.

nodus was hanging because it was trying to mount the cit40m account from rigel and rigel was not responding.

Neither I nor Yoichi can recall what the cit40m account does for us on nodus anymore and so I commented it out of the nodus /etc/vfstab.

nodus may still need a boot to make it pay attention. I was unable to do a 'umount' to get rid of the rigel parasite. But mainly I don't want anything in

the 40m to depend on the LIGO GC system if at all possible.

None of the links here seem to work. I forgot what the story is with our special apache redirect 

https://wiki-40m.ligo.caltech.edu/Core_Optics

The story is: we currently don't expose the whole /users/public_html folder. Instead, we are symlinking the folders from public_html to /export/home/ on nodus, which is where apache looks for things

So, I fixed the links on the Core Optics page by running:

controls@nodus|~ > ln -sfn /users/public_html/40m_phasemap /export/home/

I tried to follow these instructions today to make the Simulink Webview accessible:

controls@nodus|public_html > ln -sfn /users/public_html/FE /export/home/

But...I got a "403 Forbidden" message. What is the secret handshake to get this to work? And why have we added this extra step of security?

This link works for me: https://nodus.ligo.caltech.edu:30889/FE/c1als_slwebview.html. The problem with just /FE/ is that there is no index.html, and we have turned off automatic directory listings.

IIRC, this arrangement was due to the fact that authentication of some of the folders (maybe the wikis) was broken during the nodus upgrade, so there was sensitive information being publicly displayed. This setup gives us discretion over what gets exposed.