40m QIL Cryo_Lab CTN SUS_Lab TCS_Lab OMC_Lab CRIME_Lab FEA ENG_Labs OptContFac Mariner WBEEShop
  40m Log, Page 341 of 344  Not logged in ELOG logo
New entries since:Wed Dec 31 16:00:00 1969
ID Date Author Typedown Category Subject
  15742   Mon Dec 21 09:28:50 2020 JamieConfigurationCDSUpdated CDS upgrade plan

Attached is the layout for the "intermediate" CDS upgrade option, as was discussed on Wednesday. Under this plan:

  • Existing FEs stay where they are (they are not moved to a single rack)

  • Dolphin IPC remains PCIe Gen 1

  • RFM network is entirely replaced with Dolphin IPC

Please send me any omissions or corrections to the layout.

I just want to point out that if you move all the FEs to the same rack they can all be connected to the Dolphin switch via copper, and you would only have to string a single fiber to every IO rack, rather than the multiple now (for network, dolphin, timing, etc.).

  15746   Wed Dec 23 23:06:45 2020 gautamConfigurationCDSUpdated CDS upgrade plan
  1. The diagram should clearly show the host machines and the expansion chassis and the interconnects between them.
  2. We no longer have any Gentoo bootserver or diskless FEs.
  3. The "c1lsc" host is in 1X4 not 1Y3.
  4. The connection between c1lsc and Dolphin switch is copper not fiber. I don't know how many Gbps it is. But if the switch is 10 Gbps, are they really selling interface cables that have lower speed? The datasheet says 10 Gbps.
  5. The control room workstations - Debian10 (rossa) is the way forward I believe. it is true pianosa remains SL7 (and we should continue to keep it so until all other machines have been upgraded and tested on Debian 10).
  6. There is no "IOO/OAF". The host is called "c1ioo".
  7. The interconnect between Dolphin switch and c1ioo host is via fiber not copper.
  8. It'd be good to have an accurate diagram of the current situation as well (with the RFM network).
  9. I'm not sure if the 1Y1 rack can accommodate 2 FEs and 2 expansion chassis. Maybe if we clear everything else there out...
  10. There are 2 "2GB/s" Copper traces. I think the legend should make clear what's going on - i.e. which cables are ethernet (Cat 6? Cat 5? What's the speed limitation? The cable? Or the switch?), which are PCIe cables etc etc. 

I don't have omnigraffle - what about uploading the source doc in a format that the excellent (and free) draw.io can handle? I think we can do a much better job of making this diagram reflect reality. There should also be a corresponding diagram for the Acromag system (but that doesn't have to be tied to this task). Megatron (scripts machine) and nodus should be added to that diagram as well.

Please send me any omissions or corrections to the layout.

  15771   Tue Jan 19 14:05:25 2021 JonConfigurationCDSUpdated CDS upgrade plan

I've produced updated diagrams of the CDS layout, taking the comments in 15476 into account. I've also converted the 40m's diagrams from Omnigraffle ($150/license) to the free, cloud-based platform draw.io. I had never heard of draw.io, but I found that it has most all the same functionality. It also integrates nicely with Google Drive.

Attachment 1: The planned CDS upgrade (2 new FEs, fully replace RFM network with Gen 1 Dolphin IPC)
Attachment 2: The current 40m CDS topology

The most up-to-date diagrams are hosted at the following links:

Please send me any further corrections or omissions. Anyone logged in with LIGO.ORG credentials can also directly edit the diagrams.

  15772   Tue Jan 19 15:43:24 2021 gautamConfigurationCDSUpdated CDS upgrade plan

Not sure if 1Y1 can accommodate both c1sus2 and c1bhd as well as the various electronics chassis that will have to be installed. There may need to be some distribution between 1Y1 and 1Y3. Does Koji's new wiring also specify which racks hold which chassis?

Some minor improvements to the diagram:

  1. The GPS receiver in 1X7 should be added. All the timing in the lab is synced to the 1pps from this.
  2. We should add hyperlinks to the various parts datasheets (e.g. Dolphin switch, RFM switch, etc etc) so that the diagram will be truly informative and self-contained.
  3. Megatron and nodus, but especially chiara (NFS server), should be added to the diagram. 
  15921   Mon Mar 15 20:40:01 2021 ranaConfigurationComputersinstalled QTgrace on donatella for dataviewer

I installed QTgrace using yum on donatella.angel Both Grace and XMgrace are broken due to some boring fight between the Fedora package maintainers and the (non existent) Grace support team. So I have symlinked it:

controls@donatella|bin> sudo mv xmgrace xmgrace_bak
controls@donatella|bin> sudo ln -s qtgrace xmgrace
controls@donatella|bin> pwd

I checked that dataviewer works now for realtime and playback.cool Although the middle click paste on the mouse doesn't work yet.angry

  15928   Wed Mar 17 09:05:01 2021 Paco, AnchalConfigurationComputers40m Control Room Changes
  • Switched positions of allegra and donatella.
  • While doing so, the hdmi cable previously used by donatella snapped. We replaced this cable by another unused cable we found connected only on one end to rossa. We should get more HDMI cables if that cable was in use for some other purpose.
  • Paco bought a bluetooth speaker/mic that is placed infront of allegra and it's usb adapter is connected to iMac's keyboard in the bottom. With the new camera installed, the 40m video call environment is now complete.
  • Again, we have placed allegra's monitor for place holder but it is not working and we need new monitors for it in future whenever it is going to be used.
  16027   Wed Apr 14 13:16:20 2021 AnchalConfigurationComputers40m Control Room Changes
  • I have confirmed that the old two monitors' backlighting is not working. One can see the impression of the display without any brightness on them. Both old monitors are on the shelf behind.
  • Today we got a monitor and mouse from Mike. I had to change /etc/default/grub GRUB_GFXMODE to 1920x1200@30 on allegra for it to work with the(any) monitor.
  • Allegra is Debian 10 with latest cds-workstation installed on it. It is a good test station to migrate our existing scripts to start using updated cds-workstation configuration.
  • Again, we have placed allegra's monitor for place holder but it is not working and we need new monitors for it in future whenever it is going to be used.


  16163   Wed May 26 11:45:57 2021 Anchal, PacoConfigurationIMCMC2 analog camera

[Anchal, Paco]

We went near the MC2 area and opened the lid to inspect the GigE and analog video monitors for MC2. Looked like whatever image is coming through the viewport is split into the GigE (for beam tracking) and the analog monitor. We hooked the monitor found on the floor nearby and tweaked the analog video camera around to get a feel for how the "ghost" image of the transmission moves around. It looks like in order to try and remove this "extra spots" we would need to tweak the beam tracking BS. We will consult the beam tracking authorities and return to this.

  16302   Thu Aug 26 10:30:14 2021 JamieConfigurationCDSfront end time synchronization fixed?

I've been looking at why the front end NTP time synchronization did not seem to be working.  I think it might not have been working because the NTP server the front ends were point to, fb1, was not actually responding to synchronization requests.

I cleaned up some things on fb1 and the front ends, which I think unstuck things.

On fb1:

  • stopped/disabled the default client (systemd-timesyncd), and properly installed the full NTP server (ntp)
  • the ntp server package for debian jessie is old-style sysVinit, not systemd.  In order to make it more integrated I copied the auto-generated service file to /etc/systemd/system/ntp.service, and added and "[install]" section that specifies that it should be available during the default "multi-user.target".
  • "enabled" the new service to auto-start at boot ("sudo systemctl enable ntp.service") 
  • made sure ntp was configured to serve the front end network ('broadcast') and then restarted the server ("sudo systemctl restart ntp.service")

For the front ends:

  • on fb1 I chroot'd into the front-end diskless root (/diskless/root) and manually specifed that systemd-timesyncd should start on boot by creating a symlink to the timesyncd service in the multi-user.target directory:
$ sudo chroot /diskless/root
$ cd /etc/systemd/system/multi-user.target.wants
$ ln -s /lib/systemd/system/systemd-timesyncd.service
  • on the front end itself (c1iscex as a test) I did a "systemctl daemon-reload" to force it to reload the systemd config, and then restarted the client ("systemctl restart systemd-timesyncd")
  • checked the NTP synchronization with timedatectl:
controls@c1iscex:~ 0$ timedatectl 
      Local time: Thu 2021-08-26 11:35:10 PDT
  Universal time: Thu 2021-08-26 18:35:10 UTC
        RTC time: Thu 2021-08-26 18:35:10
       Time zone: America/Los_Angeles (PDT, -0700)
     NTP enabled: yes
NTP synchronized: yes
 RTC in local TZ: no
      DST active: yes
 Last DST change: DST began at
                  Sun 2021-03-14 01:59:59 PST
                  Sun 2021-03-14 03:00:00 PDT
 Next DST change: DST ends (the clock jumps one hour backwards) at
                  Sun 2021-11-07 01:59:59 PDT
                  Sun 2021-11-07 01:00:00 PST
controls@c1iscex:~ 0$ 

Note that it is now reporting "NTP enabled: yes" (the service is enabled to start at boot) and "NTP synchronized: yes" (synchronization is happening), neither of which it was reporting previously.  I also note that the systemd-timesyncd client service is now loaded and enabled, is no longer reporting that it is in an "Idle" state and is in fact reporting that it synchronized to the proper server, and it is logging updates:

controls@c1iscex:~ 0$ sudo systemctl status systemd-timesyncd
â— systemd-timesyncd.service - Network Time Synchronization
   Loaded: loaded (/lib/systemd/system/systemd-timesyncd.service; enabled)
   Active: active (running) since Thu 2021-08-26 10:20:11 PDT; 1h 22min ago
     Docs: man:systemd-timesyncd.service(8)
 Main PID: 2918 (systemd-timesyn)
   Status: "Using Time Server (ntpserver)."
   CGroup: /system.slice/systemd-timesyncd.service
           â””─2918 /lib/systemd/systemd-timesyncd

Aug 26 10:20:11 c1iscex systemd[1]: Started Network Time Synchronization.
Aug 26 10:20:11 c1iscex systemd-timesyncd[2918]: Using NTP server (ntpserver).
Aug 26 10:20:11 c1iscex systemd-timesyncd[2918]: interval/delta/delay/jitter/drift 64s/+0.000s/0.000s/0.000s/+26ppm
Aug 26 10:21:15 c1iscex systemd-timesyncd[2918]: interval/delta/delay/jitter/drift 128s/-0.000s/0.000s/0.000s/+25ppm
Aug 26 10:23:23 c1iscex systemd-timesyncd[2918]: interval/delta/delay/jitter/drift 256s/+0.001s/0.000s/0.000s/+26ppm
Aug 26 10:27:40 c1iscex systemd-timesyncd[2918]: interval/delta/delay/jitter/drift 512s/+0.003s/0.000s/0.001s/+29ppm
Aug 26 10:36:12 c1iscex systemd-timesyncd[2918]: interval/delta/delay/jitter/drift 1024s/+0.008s/0.000s/0.003s/+33ppm
Aug 26 10:53:16 c1iscex systemd-timesyncd[2918]: interval/delta/delay/jitter/drift 2048s/-0.026s/0.000s/0.010s/+27ppm
Aug 26 11:27:24 c1iscex systemd-timesyncd[2918]: interval/delta/delay/jitter/drift 2048s/+0.009s/0.000s/0.011s/+29ppm
controls@c1iscex:~ 0$ 

So I think this means everything is working.

I then went ahead and reloaded and restarted the timesyncd services on the rest of the front ends.

We still need to confirm that everything comes up properly the next time we have an opportunity to reboot fb1 and the front ends (or the opportunity is forced upon us).

There was speculation that the NTP clients on the front ends (systemd-timesyncd) would not work on a read-only filesystem, but this doesn't seem to be true.  You can't trust everything you read on the internet.

  16614   Mon Jan 24 12:33:41 2022 ranaConfigurationWikiAIC Wiki: txz files allowed

I updated the mime.local.conf file for the AIC Wiki so as to allow attachments with the .txz format. THis should be persistent over upgrades, since its a local file.

  16874   Wed May 25 16:56:44 2022 PacoConfigurationBHDIFO recovery - IMC alignment

[Yuta, Paco]

We aligned IMC to recover the IFO progressively. First step was to center the MC REFL beamspot on the camera as well as the WFS DC. Then slide MC2 and MC3 together. Below are the alignment slider positions before/after.

  MC1 (before --> after) MC2 (before --> after) MC3 (before --> after)
PIT -0.3398 --> -0.4768 4.1217 --> 4.0737 -1.9808 --> -1.9308
YAW -0.8947 --> -0.7557 -1.2350 --> -1.3350 1.5598 --> 1.5638
  16875   Wed May 25 17:34:47 2022 yutaConfigurationBHDIFO recovery - IFO alignment

IFO aligned to maximize flashings, except for GRY and LO-AS.

What we did:
 0. After recovering IMC, C1:IOO-MC_TRANS_SUM was ~1300 with C1:IOO-MC_RFPD_DCMON of ~0.11 (~10% better than what we had during vent). Xarm and Yarm was already flashing and could see the beam at AS and POP cameras.
 1. Aligned ETMX and ITMX to green X input beam to maximize C1:ALS-TRX_OUT, to ~0.19.
 2. Aligned TT2-PR3 to get C1:SUS-ETMX_TRX_OUT flashing at 0.09 at max
 3. Aligned ITMY to have nice POP blinking of MICH at POP camera
 4. Aligned ETMY-PR3 to have C1:SUS-ETMX_TRX_OUT flashing at 0.06 at max
 5. Misaligned ITMY (with +2 in C1:SUS-ITMY_PIT_COMM), and aligned PRM to have PRX (PRM-ITMX cavity) flashing at C1:LSC-ASDC_IN1 at ~20 (offset -70) at max
 6. Misaligned PRM, and aligned SRM to have SRX (SRM-ITMX cavity) flashing at C1:LSC-ASDC_IN1 at ~20 (offset -70) at max
 7. Restored all the alignment. ITMY didn't quite come back, so I need to tweak the alignement to maximize TRY flashing.

Current alignment is as attached. IR beam at AS, REFL, MCR and green beam at GTRX cameras all seem slightly to the left from monitors, but looks as it was before the pump down.yes GTRY is still clipped, but green Y locks stably. Oplevs were not so useful to recover the alignment. ETMX/Y oplevs did not drifted too much probably because we don't have in-vac steering mirrors.

 - Tweak alignment of green Y input to follow Yarm
 - Do LO-AS alignment
 - REFL DC is not receiving beam. Re-alignment necessary
 - Oplev centering
 - BHD PDs need to be replaced to lower gain PDs and need to be connected to CDS

  16877   Thu May 26 19:55:43 2022 yutaConfigurationBHDOplevs centered, BHD DCPDs are now online

[Paco, Yuta]

We have aligned the IFO (except for LO-AS and GRY), and centered all the oplevs.
We have also restored Gautam's in-air BHD DCPD setup and placed it to ITMY table.
BHD DC PD signals are now online at C1:XO4-MADC1_EPICS_CH4 and CH5. 

 Aligned the IFO following the steps in elog 40m/16875.
 When we were woking on BHD DCPDs, we lost REFL beam on camera and both arms flashing. Alignment was restored mostly with TT2 pitch.
 We centered all the oplevs after the recovery (see attached).

 1. We removed a circuit box with M2 ISS photodetector readout board from AP table, in-air BHD photodiodes from optics graveyard. (see LIGO-E2000436 and elog 40m/15493 for wiring diagram)
 2. Taken out temporary two Thorlabs PDA100A used for aligning LO-AS during the vent from ITMY table, and placed the BHD setup in ITMY table (see attached and attached).
 3. DB9 cable (15ft+10ft) was connected from M2 ISS box to anti-aliasing chassis for ADC1 of C1X04 at 1Y2 rack (see attached).
 4. +/-18V power for M2 ISS box was supplied from 1Y1 rack.
 5. BHD DCPD signals are now available at C1:XO4-MADC1_EPICS_CH4 and CH5 (see attached).

 - Tweak alignment of green Y input to follow Yarm
 - Do LO-AS alignment
 - Centering of PDs everywhere with IFO aligned
 - Update RTS model for BHD

  16880   Fri May 27 17:45:53 2022 yutaConfigurationBHDBHD camera installed, GRY aligned

[JC, Paco, Yuta]

After the IFO recovery (elog 40m/16881), we installed an analog camera for BHD fringe using a BNC cable for old SRMF camera so that we can see it from the control room.
We also aligned AS-LO using LO1,LO2 and AS4.
We then aligned GRY injection to get maximum GTRY.

Maximum TEM00s right now are
 C1:ALS-TRX_OUT_DQ ~0.20
 C1:ALS-TRY_OUT_DQ ~0.18

  16886   Thu Jun 2 20:05:37 2022 yutaConfigurationPSLIMC input power recovered to 1W, some alignment works

[Paco, Yuta]

We have increased the output power from the PSL table to 951 mW (it was 96.7 mW).
IMC was recovered including WFS, and both arms are flashing nicely in IR.
We tweaked the alignment of GRX and GRY injection to align them with IR, but it was hard.
Right now IR beams are not centered on TMs. We should center them first.

What we did:
Power increase and IMC recovery
 - Replaced a beam splitter which splits the beam into IMC REFL RF PD path and WFS path from R=98% to R=10% one. Reflection goes to RF PD.
 - Put a R=98% beam splitter back into WFS path.
 - We also tried to put a window in front of IMC REFL camera to recover the arrangement in 40m wiki, but the beam reflected from the window was too weak for us to align. So, we decided not to place a window in front of the camera.
 - Attached photos are the IMC REFL path before and after the work.
 - Measured the PSL output power as Koji did in elog 40m/16672. It was measured to be 96.7+/- 0.5 mW.
 - Rotated the HWP using the Universal Motion Controller (it was not possible for us to do it from the MEDM screen). The position was changed from 73.99 deg to 36.99 deg. Output power was measured to be 951 +/- 1 mW
 - IMC locked without any other changes.
 - Changed C1:IOO-WFS_TRIGGER_THRESH_ON to 5000 (was 500). IMC WFS also worked.
 - After running MC WFS relief script, WFS DC offsets and RF offsets are adjusted following the steps in elog 40m/16835. Below are the results.

C1:IOO-WFS1_SEG1_DC.AOFF => -0.0008882080010759334
C1:IOO-WFS1_SEG2_DC.AOFF => -0.0006527877490346629
C1:IOO-WFS1_SEG3_DC.AOFF => -0.0005847311617496113
C1:IOO-WFS1_SEG4_DC.AOFF => -0.0010395992663688955
C1:IOO-WFS2_SEG1_DC.AOFF => -0.0025944841559976334
C1:IOO-WFS2_SEG2_DC.AOFF => -0.003191715502180159
C1:IOO-WFS2_SEG3_DC.AOFF => -0.0036688060499727726
C1:IOO-WFS2_SEG4_DC.AOFF => -0.004011172490815322

IOO-WFS1_I1         :  +1977.7 ->    +2250 (Significant change)
IOO-WFS1_I2         :  +3785.8 ->  +3973.2
IOO-WFS1_I3         :  +2014.2 ->  +2277.7 (Significant change)
IOO-WFS1_I4         :  -208.83 ->  +430.96 (Significant change)
IOO-WFS1_Q1         :  +2379.5 ->  +1517.4 (Significant change)
IOO-WFS1_Q2         :  +2260.4 ->  +2172.6
IOO-WFS1_Q3         :  +588.86 ->  +978.98 (Significant change)
IOO-WFS1_Q4         :  +1654.8 ->  +195.38 (Significant change)
IOO-WFS2_I1         :  -1619.9 ->  -534.25 (Significant change)
IOO-WFS2_I2         :  +1610.4 ->  +1619.8
IOO-WFS2_I3         :  +1919.6 ->  +2179.8 (Significant change)
IOO-WFS2_I4         :    +1557 ->  +1426.6
IOO-WFS2_Q1         :   -62.58 ->  +345.56 (Significant change)
IOO-WFS2_Q2         :  +777.01 ->  +805.41
IOO-WFS2_Q3         :  -6183.6 ->  -5365.8 (Significant change)
IOO-WFS2_Q4         :  +4457.2 ->  +4397.

IFO Alignment
 - Aligned both arms using IR. Both arm flashes at the following, which is consistent with the power increase.
 - With this, we tried to tweak GRX and GRY injection. The following is after the work. We could increase GTRX to 0.204 when the Xarm is aligned to green. This suggests that GRX injection is not aligned nicely yet. But the beams are also not centered on TMs. We should center them first.
 C1:ALS-TRX_OUT_DQ ~0.13
 C1:ALS-TRY_OUT_DQ ~0.07
 - GTRX and GTRY cameras are adjusted to have nicer images. In GRX path, the second and last lens before the PD and CCD was pulled ~ 1 cm behind its original position and both beams realigned. Then, on GRY path, the beam was re-centered on the first and only lens, the whole assembly pushed forward by ~ 2 cm and the beams re-centered.

 - Center the IR beam on TMs (first by our eyeballs; better to use A2L after arm locking is recovered and coils are balanced)
 - Tweak GRX and GRY injection (restore GRY PZTs?)
 - Install ETMXT camera (if it is easy)
 - Lock Xarm and Yarm (C1:LSC-TRX/Y_OUT needs to be fixed for triggering. Can we use other PDs for triggering?)
 - MICH locking (REFL and AS PDs might need to be re-aligned; they are not receiving much light)
 - RTS model for BHD needs to be updated

  16887   Fri Jun 3 12:13:58 2022 PacoConfigurationCDSFix RFM channels

[Paco, Yuta]

We tried fixing the issue of LSC_TRY and LSC_TRX channels not working. We first did some investigation, and just like previously reported by Chris, narrowed down the issue to the RFM channels coming from c1iscex/c1iscey.

First attempt : FAIL

In our first attempt, we

  1. Tripped ETMX/ETMY watchdogs, ssh to c1iscex/c1iscey and restart the rtcds models.
  2. Since the last step didn't fix things, we decided to do the same thing on c1lsc, c1sus, c1ioo.
  3. After hard rebooting c1ioo and c1lsc (because they died during the stopping of rtcds models), and not experiencing any timing issues (nice), we still don't fix the issue.

Second attempt: Success

A second attempt just followed Koji's previous fix explained here. Basic difference with our first attempt was a hard reboot of c1iscex/c1iscey in addition to the rtcds model restarting. RFM channels were then clear of errors and we recovered our IR transmission channels in the LSC model.

  16893   Mon Jun 6 16:09:23 2022 ranaConfigurationDetCharSummary Pages: seis BLRMS

I updated the config file c1pem.ini in /users/public_html/detcharsummary/ConfigFiles, and commited it so I hope it works, but I did not have git push permissions. Does anyone know what is the idea here? Should we do our own personal git clone and modify that way or shoudl we do it with the control account.

Wiki needs to clear out all the outdated information on this workflow.

The changes are to make the y-scales useful. Currently, all of the past seis BLRMS plots are not so useful because the scales have not been set based on the actual signal levels. Let's see if this works, and we  can re-evaluate after a few weeks.

  16924   Thu Jun 16 18:23:15 2022 PacoConfigurationBHDRecovering LO beam in BHD DCPDs

[Paco, Yuta]

We recovered the LO beam on the BHD port. To do this, we first tried reverting to a previously "good" alignment but couldn't see LO beam hit the sensor. Then we checked the ITMY table and couldn't see LO beam either, even though the AS beam was coming out fine. The misalignment is likely due to recent changes in both injection alignment on TT1, TT2, PR2, PR3, as well as ITMX, ITMY. We remembered that LO path is quite constrained in the YAW direction, so we started a random search by steering LO1 YAW around by ~ 1000 counts in the negative direction at which point we saw the beam come out of the ITMY chamber yes

We proceeded to walk the LO1-LO2 in PIT mostly to try and offload the huge alignment offset from LO2 to LO1 but this resulted in the LO beam disappearing or become dimmer (from some clipping somewhere). This is WiP and we shall continue this alignment offload task at least tomorrow, but if we can't offload significantly we will have to move forward with this alignment. Attachment #1 shows the end result of today's alignment.

  16932   Tue Jun 21 14:17:50 2022 yutaConfigurationBHDBHD DCPDs re-routed to c1sus2

After discussing with Anchal, we decided to route BHD related PD signals directly to ADC of c1sus2, which handles our new suspensions including LO1, LO2, AS1, AS4, so that we can control them directly.
BHD related PD signals will be sent to c1lsc for DARM control.

Re-cabling was done, and now they are online at C1:X07-MADC1_EPICS_CH16 (DC PD A) and CH17 (DC PD B) with 15ft DB9 cable.
Here, DC PD A is the transmission of BHD BS for AS beam, and DC PD B is the reflection of BHD BS for AS beam (see attached photo).

  17018   Tue Jul 19 16:00:34 2022 yutaConfigurationBHDFast channels for BHD DCPDs now available in c1lsc but not in c1hpc

[Paco, Anchal-remote-support, Yuta]

We added fast channels to BHD DC PDs.
C1:LSC-DCPD_(A|B)_IN1 are now available, but C1:HPC-DCPD_(A|B)_IN1 still gives us zero.

c1hpc situation -> not good
 - We can see the slow signal at C1:X07-MADC1_EPICS_CH16 (DC PD A) and CH17 (DC PD B)
 - C1:HPC-DCPD_(A|B)_IN1 is there, but zero.
 - We have modified c1hpc model to add DCPD_(A|B) filters in front of the input matrix (see Attachment #1).
 - After modifying the model, we run
ssh c1sus2
rtcds make c1hpc
rtcds install c1hpc
ssh fb1
sudo systemctl restart daqd_*

 - After this, we got 0x2000 error. So, we ran the following. This removed 0x2000 error, but DCPD signals are still zero. They are also not available in C1HPC-MONITOR_ADC1.adl screen (see Attachment #3).
ssh c1sus2
rtcds restart c1hpc

c1lsc situation -> good
 - We could see the slow signal at C1:X04-MADC1_EPICS_CH4 (DC PD A) and CH5 (DC PD B), and also C1:LSC-DCPD_(A|B)_NORM after making C1:LSC-DCPD_(A|B)_POW_NORM=1. The ADC channel and DCPD channel are exactly the same.
 - After confirming the above, we modified the c1lsc model to add DCPD_(A|B) filters in front of the input matrix (see Attachment #2).
 - After modifying the model, we run
ssh c1lsc
rtcds make c1lsc
rtcds install c1lsc
ssh fb1
sudo systemctl restart daqd_*

 - After this, we also got 0x2000 error. We also noticed that, for example, C1:X04-MADC0_EPICS_CH31 and C1:LSC-ASDC_INMON are different, which used to be the same (ASDC_INMON was largely attenuated).
 - In the end, we run the following to remove 0x2000 error, but it crashed c1lsc, as well as c1sus, c1ioo.
ssh c1lsc
rtcds restart c1lsc

 - So, we did rebootC1LSC.sh. This made c1lsc, c1ioo and c1sus as green as before, except for RFM issue in TRX/TRY, like we saw in June. We followed the steps in 40m/16887 to hard reboot c1iscex/c1iscey and ran rebootC1LSC.sh again. This made C1CDS_FE_STATUS.adl screen as green as before (see Attachment #3).

 - Fast channels C1:LSC-DCPD_(A|B)_IN1 are now available. They are also available in C1LSC-MONITOR_ADC1.adl screen (see Attachment #3).

  17025   Thu Jul 21 21:50:47 2022 TegaConfigurationBHDc1sus2 IPC update

IPC issue still unresolved.

Updated shared memory tag so that 'SUS' -> 'SU2' in c1hpc, c1bac and c1su2. Removed obsolete 'HPC/BAC-SUS' references from IPC file, C1.ipc. Restarted the FE models but the c1sus2 machine froze, so I did a manual reboot. This brought down the vertex machines---which I restarted using /opt/rtcds/caltech/c1/scripts/cds/rebootC1LSC.sh---and the end machines which I restarted manually. Everything but the BHD optics now have their previous values. So need to burtrestore these.

# IPC file:

# Model file locations:

# Log files:

SUS overview medm screen :

  • Reduced the entire screen width
  • Revert to old screen style watchdog layout
  17026   Fri Jul 22 15:05:26 2022 TegaConfigurationBHDc1sus2 shared memory and ADC fix

[Tega, Yuta]

We were able to fix the shared memory issue by updating the receiver model name from ''SUS' to 'SU2' and the ADC zero issue by including both ADC0 and ADC1 in the c1hpc and c1bac models as well as removing the grounding of the unused ADC channels (including chn#16 and chn#17 which are actually used in c1hpc) in c1su2. We also used shared memory to move the DCPD_A/B error signals (after signal conditioning and mixing A/B; now named A_ERR and B_ERR) from c1hpc to c1bac.
C1:HPC-DCPD_A_IN1 and C1:HPC-DCPD_B_IN1 are now availableangel (they are essentially the same as C1:LSC-DCPD_A_IN1 and C1:LSC-DCPD_B_IN1, except for they are ADC-ed with different ADC; see elog 40m/16954 and Attachment #1).
Dolphin IPC error in seding signal from c1hpc to c1lsc still remains.crying

  17028   Fri Jul 22 17:46:10 2022 yutaConfigurationBHDc1sus2 watchdog update and DCPD ERR channels

[Tega, Yuta]

We have added C1:HPC-DCPD_A_ERR and C1:HPC-DCPD_B_ERR testpoints, which can be used as A+B, A-B etc.
Restarting c1hpc crashed c1sus2, and also made c1lsc/ioo/sus models red.
We run /opt/rtcds/caltech/c1/Git/40m/scripts/cds/restartAllModels.sh to restart all the machines. It worked perfectly without manually pressing power buttons! Wow!heart

We have also edited /opt/rtcds/caltech/c1/medm/c1su2/C1SU2_WATCHDOGS.adl so that it will use new /opt/rtcds/caltech/c1/Git/40m/scripts/SUS/medm/resetFromWatchdogTrip.sh instead of old /opt/rtcds/caltech/c1/scripts/SUS/damprestore.py.

  17033   Mon Jul 25 17:58:10 2022 TegaConfigurationBHDc1sus2 IPC dolphin issue update

From the 40m wiki, I was able to use the instructions here to map out what to do to get the IPC issue resolved. Here is a summary of my findings.

I updated the /etc/dis/dishost.conf file on the frame builder machine to include the c1sus2 machine which runs the sender model, c1hpc, see below. After this, the file becomes available on c1sus2 machine, see attachment 1, and the c1sus2 node shows up in the dxadmin GUI, see attachment 2. However, the c1sus2 machine was not active. I noticed that the log file for the dis_nodemgr service, see attachment 3, which is responsible for setting things up, indicated that the dis_irm service may not be up, so I checked and confirmed that this was indeed the case, see attachment 4. I tried restarting this service but was unsuccessful. I restarted the machine but this did not help either. I have reached out to Jonathan Hanks for assistance.

  17034   Mon Jul 25 18:09:41 2022 TegaConfigurationBHDBHD Homodyne Phase control MEDM screen

[Paco, Tega, Yuta]

Today, we made a custom MEDM screen for the BHD Homodyne Phase Control, which is basically an overview of the c1hpc model. See Attachments 1 & 2 for details.

  17052   Mon Aug 1 18:42:39 2022 TegaConfigurationBHDc1sus2 IPC dolphin issue update

[Yuta, Tega]

We decided to give the dolphin debugging another go. Firstly, we noticed that c1sus2 was no longer recogonising the dolphin card, which can be checked using

lspci | grep Stargen

or looking at the status light on the dolphin card of c1sus2, which was orange for both ports A and B.

We decided to do a hard reboot of c1sus2 and turned off the DAQ chassis for a few minutes, then restared c1sus2. This solved the card recognition problem as well as the 'dis_irm' driver loading issue (I think the driver does not get loaded if the system does not recognise a valid card, as I also saw the missing dis_irm driver module on c1testand). 

Next, we confirmed the status of all dolphin cards on fb1, using

controls@fb1$ /opt/DIS/sbin/dxadmin

It looks like the dolphin card on c1sus2 has now been configured and is availabe to all other nodes. We then restated the all FE machines and models to see if we are in the clear. Unfortunately, we are not so lucky since the problem persisted.

Looking at the output of 'dmesg', we could only identity two notable difference between the operational dolphin cards on c1sus/c1ioo/c1lsc and c1sus2, namely: the card number being equal to zero and the memory addresses which are also zero, see image below.

Anyways, at least we can now eliminate driver issues and would move on to debugging the models next.

  17054   Tue Aug 2 17:25:18 2022 TegaConfigurationBHDc1sus2 dolphin IPC issue solved

[Yuta, Tega, Chris]

We did it!laugh

Following Chris's suggestion, we added "pciRfm=1" to the CDS parameter block in c1x07.mdl - the IOP model for c1sus2. Then restarted the FE machines and this solved the dolphin IPC problem on c1sus2. We no longer see the RT Netstat error for 'C1:HPC-LSC_DCPD_A' and 'C1:HPC-LSC_DCPD_B' on the LSC IPC status page, see attachement 1.

Attachment 2 shows the module dependencies before and after the change was made, which confirms that the IOP model was not using the dolphin driver before the change.

We encountered a burt restore problem with missing snapfiles from yesterday when we tried restoring the EPICS values after restarting the FE machines. Koji helped us debug the problem, but the summary is that restarting the FE models somehow fixed the issue.

Log files:
Request File list:
Snap files location:
Autoburt crontab on megatron:
19 * * * * /opt/rtcds/caltech/c1/scripts/autoburt/autoburt.cron > /opt/rtcds/caltech/c1/burt/burtcron.log 2>&1
  17126   Thu Sep 1 09:00:02 2022 JCConfigurationDaily ProgressLocked both arms and aligned Op Levs

Each morning now, I am going to try to align both arms and lock. Along with that, sometime at towards the end of each week, we should align the OpLevs. This is a good habit that should be practiced more often, not only by me. As for the Y Arm, Yehonathan and I had to adjust the gain to 0.15 in order to stabilize the lock.

  17135   Thu Sep 8 11:54:37 2022 JCConfigurationLab OrganizationLab Organization

The arms in the 40m laboratory have now been sectioned off. Each arm has been divided up into 15 sections. Along the Y arm, the section are labelled "Section Y1 - Section Y15". For the X arm, they are labelled "Section X1- Section X15". Anything changed or moved will now be updated into the elog with their appropriate section.


Below is an example of Section X6.

  17136   Thu Sep 8 12:01:02 2022 JCConfigurationLab OrganizationLab Organization

The floor cable cover has been changed out for a new one. This is in Section X11.

  17209   Tue Oct 25 09:57:34 2022 PacoConfigurationPEMAuto Z on trillium interface board

I pressed the Auto-Z(ero) button for ~ 3 seconds at ~9:55 local (pacific) time on the trillium interface on 1X5.

  17210   Tue Oct 25 13:55:37 2022 KojiConfigurationPEMAuto Z on trillium interface board

This nicely brought the sensing signal back to ~zero. See attachment

Some basic info:

  • BS Seismometer is T240 (Trillium)
  • The interface unit is at 1X5 Slot 26. D1002694
  • aLIGO Trillium 240 Interface Quick Start Guide T1000742
  17213   Tue Oct 25 22:01:53 2022 ranaConfigurationPEMAuto Z on trillium interface board

thanks, this seems to have recentered well.

It looks like it started to act funny at 0400 UTC on 10/24, so thats 9 PM on Sunday in the 40m. What was happening then?


  17264   Mon Nov 14 14:52:56 2022 PacoConfigurationSUSBHD SUS Coil output balance

[JC, Paco]

We installed a steering mirror intersecting the BHD beam path and put the AS beam on the ITMY Oplev QPD (see Attachment #1 for a photo of this temporary hack) . This is done to do coil balancing of AS1/AS4, LO1/LO2. QPD sees ~ 10000 counts when the beam is centered.

[Paco, Yuta]

We follow this procedure -- but with different sensors for all BHD suspension coil output balancing.


We dither BUTT first, lock the LO-AS fringe (DC lock), and look at the residual LO_PHASE spectrum to minimize POS coupling. We then unlock, misalign LO beam and look at the hijacked Oplev (ITMY) while dithering POS to minimize PIT and YAW couplings.


We dither BUTT first, lock thChangeset summarye LO-AS fringe (DC lock), and look at the residual LO_PHASE spectrum to minimize POS coupling. We then unlock, misalign AS beam and look at the hijacked Oplev (ITMY) PIT/YAW residual noise while dithering POS to minimize PIT/YAW coupling.

Changeset summary

The new coil output gains are summarized in the table below:

Optic / Coil UL UR LR LL
AS1 -0.939 1.040 -1.026 0.995
AS4 -0.9785 0.9775 -1.0695 0.9745
LO1 -0.939 1.003 -1.074 0.984
LO2 -1.051 1.342 -0.976 0.631

Finally, I reverted the hacked QPD setup to restore the ITMY OPLEV.

  17269   Tue Nov 15 17:58:00 2022 PacoConfigurationCamerasPOP camera realignment after IFO alignment

[Paco, Yuta]

I swapped the 1 inch BS and lenses along the POP beam to clear the apertures and avoid clipping this beam. The results are illustrated by the attached pictures; this was done right after Yuta had optimized IFO alignment so it's hopefully a good reference from now on. Yuta also tuned the alignment of BHDC path in ITMY table, which mostly improved the alignment to DCPD A (90-ish counts improved to 100-ish counts with ITMY single bounce).

  17275   Thu Nov 17 07:39:01 2022 JCConfigurationCamerasITMX Camera

Coming in this morning, I found ITMX Camera malfunctioning.

  17278   Thu Nov 17 12:24:48 2022 PacoConfigurationCamerasITMX Camera -- attempt at fix

I found that an old BNC cable for ITMXF video existed so I first tried swapping both ends of the cable, one on the ITMX viewport and the other one in the video MUX input in the rear. This didn't fix the issue.

I searched around in the CCD cabinet by XARM and found an identical analog camera so I swapped it and got the same image ...

I then searched for a AC/DC supply cable, but couldn't find one.


Coming in this morning, I found ITMX Camera malfunctioning.


  17280   Thu Nov 17 15:53:47 2022 JCConfigurationCamerasITMX Camera -- attempt at fix

The issue was the power supply.


I found that an old BNC cable for ITMXF video existed so I first tried swapping both ends of the cable, one on the ITMX viewport and the other one in the video MUX input in the rear. This didn't fix the issue.

I searched around in the CCD cabinet by XARM and found an identical analog camera so I swapped it and got the same image ...

I then searched for a AC/DC supply cable, but couldn't find one.


  17292   Mon Nov 21 14:55:58 2022 JCConfigurationGeneralLED Instead of Incandescent Lights

Electrical came by today to see the lights. The issue may be the switches, but they will come by tomorrow to solve the issue. A couple light bulbs were noticed to be out, but they no longer have incandescent lights. . . only LED. I figured this would be preffered because of the reduction on noise. I would prefer to go ahead and ask to change all the incandescent bulbs to LED bulbs. Are there any objections to this?

  50   Thu Nov 1 19:53:02 2007 Andrey RodionovBureaucracyPhotosTobin's picture
  51   Thu Nov 1 19:53:34 2007 Andrey RodionovBureaucracyPhotosRobert's photo
  52   Thu Nov 1 19:54:22 2007 Andrey RodionovBureaucracyPhotosRana's photo
  53   Thu Nov 1 19:55:03 2007 Andrey RodionovBureaucracyPhotosAndrey's photo
  54   Thu Nov 1 19:55:59 2007 Andrey RodionovBureaucracyPhotosAndrey, Tobin, Robert - photo
  55   Thu Nov 1 19:58:07 2007 Andrey RodionovBureaucracyPhotosSteve and Tobin's picture
  57   Fri Nov 2 08:59:30 2007 steveBureaucracySAFETYthe laser is ON
The psl laser is back on !
  75   Wed Nov 7 02:14:08 2007 AndreyBureaucracyIOOMore information about MC2 ringdown
As Tobin wrote two hours ago, we (Andrey, Tobin, Robert) made a series of ringdown measurements for MC2
in the spirit of the measurement described by Rana -> see
entry from Mon Oct 29 23:47:29 2007, rana, Other, IOO, MC Ringdowns.

I attach here some pictures that we saw on the screen of the scope, but I need to admit that I am not experienced enough to present a nice fit to these data, although I attach fits that I am able to do today.

I definitely learned a lot of new Matlab functions from Tobin - thanks to him!, but I need to learn two more things:

Firstly, I do not know how to delete "flat" region (regions before the ringdown starts) in Matlab ->
I needed to delete the entries for times before the ringdown ("negative times") by hand in the text-file, which is extremely non-elegant method;

Secondly, I tried to approximate the ringdown curve by a function ydata=a*exp(b*xdata) but I am not exactly sure if this equation of the fitting curve is a good fit or if a better equation can be used.

It seems, in this situation it is better for me to ask more experienced "comrades" on November 7th.

P.S. It seems I really like the type of message "Bureaucracy" - I put it for every message. As Alain noted, maybe that is because some things are very bureacratized in the former USSR / Russia. By the way, when I was young, November 7th was one of two most important holidays in the USSR - I liked that holiday because I really liked military parades on the red square. I attach a couple of pictures. November 7 is the anniversary of the Revolution of 1917.
  113   Fri Nov 16 18:46:49 2007 steveBureaucracyPSL MOPA was turned off & on
The "Mohana" boys scouts and their parents visited the 40m lab today.
The laser was turned off for their safety.
It is back on !
  115   Mon Nov 19 14:32:10 2007 steveBureaucracySAFETYgrad student safety training
John Miller and Alberto Stochino has received the 40m safety bible.
They still have to read the laser operation manual and sign off on it.
  130   Wed Nov 28 12:43:53 2007 AndreyBureaucracy Here was the PDF-file of my presentation

I was making a report with powerpoint presentation during that Wednesday's 40-m meeting.

Here was the pdf-file, but LATER IN THE EVENING I CREATED A WIKI-40M-page describing the algorithm, and now the pdf-file is ON THAT WIKI-40M PAGE.

NOTE ADDED AFTER THE PRESENTATION: I double checked, I am indeed taking the root-mean-square of a difference, as we discussed during my talk.

My slide #17 "Calculation of differential length" was wrong, but now I corrected it.
ELOG V3.1.3-