Electrical came by today to see the lights. The issue may be the switches, but they will come by tomorrow to solve the issue. A couple light bulbs were noticed to be out, but they no longer have incandescent lights. . . only LED. I figured this would be preffered because of the reduction on noise. I would prefer to go ahead and ask to change all the incandescent bulbs to LED bulbs. Are there any objections to this?

All incondescent lightsbulbs have been switched over to LED.

While recovering from the power outage, I took the opportunity to try having the timing system trigger on the rising edge of the 1 PPS signal from the GPS receiver (rather than the falling edge, as it had been doing since before the CDS upgrade). This was done in order to eliminate a timing offset between the clock used by the ADCs and the IRIG-B timestamps from the Spectracom boards. It was successful in that regard, and cleared the timing errors seen in the IOP statewords on most of the front ends.

The two exceptions were c1lsc and c1sus, where something is broken with the DuoTone readbacks. The attached plot shows the DuoTone signal in the last channel of ADC 0 on c1x01 (working normally) vs c1x02 and c1x04 (busted).

However, this change apparently also had some unexplained effect on the way the c1sbr model runs. There were frequent CPU time overruns and IPC errors. So on Sunday afternoon I reverted to the falling-edge trigger. This caused the timing errors to return, but allowed c1sbr to run normally.

Jamie reported that:

The logs (/var/log/daemon.log) on fb1 are filling with this line:

Jan 03 14:11:51 fb1 dhcpd[1152]: DHCPDISCOVER from 3c:ec:ef:c8:44:78 via enp3s0: network 10.0.113.0/24: no free leases

It seems that some machine on the network is trying to get an IP address but can't

• The MAC address 3c:ec:ef:xx:xx:xx indicates this is one of the supermicro units.
• The IP address indicates this is on the DAQ network which fb1 is spanning.
• There is a switch for this DAQ network. 
• There are 8 machines connected to the switch. fb1 is via optical, and the other 7 have yellow ethernet cables (Attachment 1).
• fb1 and other 6 RT machines already have 10.0.113.x assigned.
• The rest is c1shimmer. I can't ssh into it from martian. KVM on rack 1X7 shows a black screen assuming 1-7 is c1shimmer. I wonder what's the status of this machine, but left untouched so far.

The dhcpd error on the log file stopped when the yellow (DAQ) ethernet cable was removed. With Chris's permission I left it unconnected (Attachment 1).

Chris pinted that the IPMI on c1shimmer is supposed to be exposed to 192.168.113. net rather than DAQ net.

From dhcpd.conf on chiara:

host c1shimmer-ipmi {
  hardware ethernet 3c:ec:ef:c8:44:78;
  fixed-address 192.168.113.37;
}

So the ethernet connections of c1shimmer is still questionable. The next person to work with c1shimmer needs to check them.

This would also be related?
https://lanforge.wordpress.com/2015/11/10/turning-off-ipmi-dhcp/

The following changes were made to the WFS MASTER IMC Pitch and Yaw gains:

Gain values for the pitch and yaw on MC1, MC2, and MC3 filters on the SUS ASC inputs have been carried over to the WFS MASTER output filters.
This was done such that Tomohiro and I could take AC measurements at an oscillation freq of 77 Hz on the pitch and yaw mirrors, while being sure that the amplitude of the AC signal being applied to each mirror is the same. The filters on the WFS output will have gains changed from 1.0 to the previously mentioned calibration values described in ELOG 17481. 

The values calculated for each filter were inverses of the callibration constants. The filters at the SUS ASC inputs were modified to read gain values of 1.0 again.
See the table bellow for the values passed to each filter.
In summary:
originally IOO-MC1,2,3_PIT/YAW_GAIN = 1.0. Now:

MC1,2,3_ASCPIT/ASCYAW_GAIN >>  IOO-MC1,2,3_PIT/YAW_GAIN

IOO-MC1,2,3_PIT/YAW_GAIN >> 1.0

Locked FPMI, measured DARM and CARM OLTFs, locked YAUX and measured the analog loop TF at the cal line frequencies. Turned the cal lines on with the new filters Anchal added on MC2 (ResGain within and Notches outside the CARM bandwidth which is set to 200 Hz), and hope to get 3600 seconds of data this evening. Log and measurement data are saved under /opt/rtcds/caltech/c1/Git/40m/scripts/CAL/FPMI

On rossa in tmux session name FPMI_DARM_Cal, a script is running to take FPMI DARM calibration data at 1:00 am on March 9th. Please do not disturb the experiment untill 6 am. To stop the script do following on rossa:

The script will lock both arms, run ASS, then lock FPMI, then tune beatnote frequency with Y AUX laser to around 40 MHz, set phase tracked UGF to 2 kHz, clear phase history, take OLTF of DARM from 2 kHz to 10 Hz, take OLTF of CARM and AUX loop at calibration line frequencies, turn on the calibration lines, and wait for FPMI to unlock or 5 hours to pass, whatever happens first. At the end it will turn off the calibration lines.

Running this test again tonight. Will probably run it every night now.

this afternoon I did some swapping around of linear and RL for IMC WFS.

In the end, I left in in the 'both' state:

• The WFS1,2,MC_TRANS PIT loops are all on but with -40, -40, -20 dB of nominal gain
• the RL is on for PIT

this is so that we have good DC control with integrators and good HF performance too.

I had to realign the IMC today. When I came in, it was very bad, not much flashing at all, I had to do it from scratch. CH01 Camera on MON7 in the control room is completely white. Did the camera go out over the weekend? I will come back to poke around later, I have headed over to WB for a moment and will be back soon. sitemapsi

It took a while, but I was finally able to align IMC. It seems like WFS has been getting really whacky lately when we arent in the lab watching it . The picture attached has an arrow of where the beam spot was at this morning

[Radhika, Mayank]

Restarted the NDS2 script on Megatron following the instructions here

Steps  
1) SSH to megatron - "ssh megatron"

2) Switch to nds2mgr using "sudo su nds2mgr" 

3) Stop and restart the service using 

"sudo /etc/init.d/nds2 stop
sleep 20
sudo /etc/init.d/nds2 start" 

[Yuta, JC]

We are restoring the IFO alignment back to Nominal operating state


What we did:
  - Fixed the IMC and got WFS back to running state.
  - Returned all TMs to there original postion using the OPLEVs
  - Used BurtRestore to bring back the C1:HPC.adl and C1:BAC.adc
  - Align the OPLEVs.

Summary:

To start off the morning, I began to work on IMC which became misaligned ~6:00am. IMC WFS seemed to be the issue, so I turned these off and pushed worked to align IMC manually (This took a ton of time). After aligning, Yuta came in and showed me how to fix IMC WFS and we got it to work again. the main issue being that "Clear History" was stuck for a bit and the offsets were stuck to insane values. 

Next we moved forward with viewing the BHD PDs. Yuta knew there was an issue with the BurtRestore Yehonathan and I did yesterday because there were no values on the C1:HPC.adl page. To fix this, in the terminal we typed --> 

~> cd /opt/rtcds/caltech/c1/burt/autoburt/snapshots/2023/Jun/9

Now, you land in whichever date you chose, for us it was the 9th of June:

9> burtgooeyC1:HPC.adl
 
This will pull up BURT. from here go Restore-> Snapshot Files-> (Select your time) -> (Select which files you want to restore) -> Ok -> Cancel -> Restore. (Keywords: How to burtrestore )

This will restore the files of which you have chosen back to the date you selected.

Yuta and I restored C1:HPC.adl and C1:BAC.adl.

Following, now that all the BurtRestoring is done, we moved forward with aligning single arms. We had a very trashy mode on BHD and decided to align TT1 and TT2. 

I tried to align the Y arm to start off, but ETMy was acting very hectic. I was trying to align the OPLEVs to start off, but ETMY Oplev was VERY FAR OFF. It was very weird that I was moving YAW for a very good amount and yet the Red Beam would not move one bit in the YAW direction! I couldnt get it even close to hit the steering mirror to hit the ETMY_OPLEV_QPD. I asked Yuta for help and it turns out that the gain for the alignment offesets were 0! After Yuta found this, he restored the original offset values by using:

restoreAlignment -o ETMY -t 'now 14days'

(Keywords: How to restoreAlignment)

After this issue was fixed, I was able to align the OPLEV beam with ease and we started to get some flashing on the Y Arm. Yuta now has the Y Arm locked with flashing on the X Arm. We plan to align nicely and realign the OPLEVs before the end of the day.

The incompleteness of burtrestore is a critical issue. We need to track down what is the issue.

As the preparation of the flow sensor installation, I added two slow channels via c1psl (Acromag IOC).
C1:PEM-HEPA_FLOW_PSL1
C1:PEM-HEPA_FLOW_PSL2

This entry explains how this was done because there is a tricky hidden process that is scarcely explained elsewhere upon adding the slow channels
Tag: How to add slow channels / C0ECDU.ini / Acromag / Advanced LIGO RTS stand-alone EPICS data concentrator

1. Edited the db file

I learned that the spare ADC channels ADC4 CH5 and CH6 are available on c1psl
(See http://nodus.ligo.caltech.edu:8080/40m/[ELOG 17727] )

Created /cvs/cds/caltech/target/c1psl/psl_pem.db

c1psl>cat psl_pem.db
# Greated for Acromag PEM - Koji Arai 2023/8/3
#

grecord(ai,"C1:PEM-HEPA_FLOW_PSL1")
{
        field(DESC,"HEPA Air Flow Speed Monitor Unit 1")
        field(SCAN,".1 second")
    field(DTYP,"asynInt32")
    field(INP,"@asynMask(C1PSL_XT1221_ADC04 5 -16)MODBUS_DATA")
        field(PREC,"3")
        field(LINR,"LINEAR")
        field(EGUF,"10")
        field(EGUL,"-10")
        field(EGU,"Volts")
        field(HOPR,"10")
        field(LOPR,"-10")
        field(ASLO,"1.09225")
}

grecord(ai,"C1:PEM-HEPA_FLOW_PSL2")
{
        field(DESC,"HEPA Air Flow Speed Monitor Unit 2")
        field(SCAN,".1 second")
    field(DTYP,"asynInt32")
    field(INP,"@asynMask(C1PSL_XT1221_ADC04 6 -16)MODBUS_DATA")
        field(PREC,"3")
        field(LINR,"LINEAR")
        field(EGUF,"10")
        field(EGUL,"-10")
        field(EGU,"Volts")
        field(HOPR,"10")
        field(LOPR,"-10")
        field(ASLO,"1.09225")
}

2. Added the db file to cmd file.
Added the following line to /cvs/cds/caltech/target/c1psl/C1_PSL.cmd as

dbLoadDatabase("/cvs/cds/caltech/target/c1psl/psl_pem.db")

3. Restart modbusIOC at c1psl

c1psl>ssh c1psl
controls@c1psl's password:
...
controls@c1psl:~$ sudo systemctl restart modbusIOC.service

4. Added the channels to C0EDCU.ini

Add the following lines to /opt/rtcds/caltech/c1/chans/daq/C0EDCU.ini

Leave the first ~20 channels untouched, as they appears on dataviewer as the default channels.

#HEPA Air Flow Mon channels @c1psl

[C1:PEM-HEPA_FLOW_PSL1]
[C1:PEM-HEPA_FLOW_PSL2]

5. Restart fb

telnet fb 8087
Trying 192.168.113...
Connected to fb.martian.
Escape character is '^]'.
daqd> shutdown

6. Ssh into c1sus and run the following command:

This is a new process. Until this is done, the channels are not recorded.

controls@nodus|daq> ssh c1sus

controls@c1sus:~$ sudo systemctl restart rts-edc_c1sus

[C1:DMF-SEIS_1]
[C1:DMF-SEIS_2]
[C1:DMF-SEIS_3]
[C1:DMF-LINE_1]
[C1:DMF-LINE_2]
[C1:DMF-LINE_3]
[C1:DMF-MC_1]
[C1:DMF-MC_2]
[C1:DMF-MC_3]

The compiled version of seisBLRMS had been running ~2 weeks without crashing as of last night, when I killed it
so it wouldn't interfere with alignment scripts.  I added an EPICS channel C1:DMF-ENABLE, and updated the DMF
executables to check this channel while running.  So far it seems to work.  When you're running alignment acripts,
simply click the DISABLE button on the C1DMF_MASTER.adl screen, and then re-ENABLE when the scripts finish. 
It's not clear why this is necessary though.  Theories include the constant disk access is keeping the
framebuilder busy, reducing its ability to deal with ezcademod commands and DMF programs just flooding the
network with so much traffic that ezcademod-related packets run late and get ignored.

Also, for reasons of aesthetics, I changed the data delay from 6 minutes to 5 minutes.  We'll see if that's enough.

> 
> Also, for reasons of aesthetics, I changed the data delay from 6 minutes to 5 minutes.  We'll see if that's enough.

5 minutes didn't work.  

Joe and I have taken control of the EPICS channels C1:PEM-Stacis_EEEX_geo and C1:PEM-Stacis_EEEY_geo since we heard that they are no longer in use.  We are currently 
using them to test the ability for the Snap camera code to read and write from EPICS channels.  Thus, the information being written to these channels is completely unrelated
to their names or previous use.  This is only temporary; we'll create our own channels for the camera code shortly (probably within the next couple of days).

- Eric

 Kiwamu and I brought 2 SUPER MICRO PCs from Willson house into 40m.

Both PCs are hooked up into Martian network. One is named as bscteststand for BSC which has been set up by Cds people and another one is named kami1 for temporary use for CLIO which is a bland new, no operating installed PC. This bland new PC will be returned Cds or 40m once another new PC which we will order within several days arrives.

IP address for each machine is 131.215.113.83 and 131.215.113.84 respectively.

We have installed CentOS5.2 into the new PC.

This morning there was a confliction of tpman running on fb40m and kami1. Alex fixed it temporary but Rana suggested it was better to move both PCs outside martian. We moved both PCs physically to the control room and connected to general network with a local router. I believe it won't conflict anymore but if you guess these PC might have trouble please feel free to shutdown.

Today's work summary:

 *connected expansion chassis to bscteststand

 *obtained signals on dataviewer, dtt for both realtime and past data on bscteststand with 64kHz timing signal

Questions:

Excitation channels are not shown, only "other" is shown.

qts.mdl should run with 16kHz but 16kHz timing causes a slow speed on dataviewer and failing data aquisition on dtt. We are using 64kHz timing but is it really correct?