40m QIL Cryo_Lab CTN SUS_Lab TCS_Lab OMC_Lab CRIME_Lab FEA ENG_Labs OptContFac Mariner WBEEShop
 40m Log, Page 6 of 330 Not logged in
ID Date Author Type Category Subject
16372   Mon Oct 4 11:05:44 2021 AnchalSummaryCDSc1teststand problems summary

[Anchal, Paco]

We tried to fix the ntp synchronization in c1teststand today by repeating the steps listed in 40m/16302. Even though teh cloned fb1 now has the exact same package version, conf & service files, and status, the FE machines (c1bhd and c1sus2) fail to sync to the time. the timedatectl shows the same stauts 'Idle'. We also, dug bit deeper into the error messages of daq_dc on cloned fb1 and mx_stream on FE machines and have some error messages to report here.

### Attempt on fixing the ntp

• We copied the ntp package version 1:4.2.6 deb file from /var/cache/apt/archives/ntp_1%3a4.2.6.p5+dfsg-7+deb8u3_amd64.deb on the martian fb1 to the cloned fb1 and ran.
controls@fb1:~ 0$sudo dbpg -i ntp_1%3a4.2.6.p5+dfsg-7+deb8u3_amd64.deb • We got error messages about missing dependencies of libopts25 and libssl1.1. We downloaded oldoldstable jessie versions of these packages from here and here. We ensured that these versions are higher than the required versions for ntp. We installed them with: controls@fb1:~ 0$ sudo dbpg -i libopts25_5.18.12-3_amd64.deb
controls@fb1:~ 0$sudo dbpg -i libssl1.1_1.1.0l-1~deb9u4_amd64.deb • Then we installed the ntp package as described above. It asked us if we want to keep the configuration file, we pressed Y. • However, we decided to make the configuration and service files exactly same as martian fb1 to make it same in cloned fb1. We copied /etc/ntp.conf and /etc/systemd/system/ntp.service files from martian fb1 to cloned fb1 in the same positions. Then we enabled ntp, reloaded the daemon, and restarted ntp service: controls@fb1:~ 0$ sudo systemctl enable ntp
controls@fb1:~ 0$sudo systemctl daemon-reload controls@fb1:~ 0$ sudo systemctl restart ntp
• But ofcourse, since fb1 doesn't have internet access, we got some errors in status of the ntp.service:
controls@fb1:~ 0$sudo systemctl status ntp ● ntp.service - NTP daemon (custom service) Loaded: loaded (/etc/systemd/system/ntp.service; enabled) Active: active (running) since Mon 2021-10-04 17:12:58 UTC; 1h 15min ago Main PID: 26807 (code=exited, status=0/SUCCESS) CGroup: /system.slice/ntp.service ├─30408 /usr/sbin/ntpd -p /var/run/ntpd.pid -g -u 105:107 └─30525 /usr/sbin/ntpd -p /var/run/ntpd.pid -g -u 105:107 Oct 04 17:48:42 fb1 ntpd_intres[30525]: host name not found: 2.debian.pool.ntp.org Oct 04 17:48:52 fb1 ntpd_intres[30525]: host name not found: 3.debian.pool.ntp.org Oct 04 18:05:05 fb1 ntpd_intres[30525]: host name not found: 0.debian.pool.ntp.org Oct 04 18:05:15 fb1 ntpd_intres[30525]: host name not found: 1.debian.pool.ntp.org Oct 04 18:05:25 fb1 ntpd_intres[30525]: host name not found: 2.debian.pool.ntp.org Oct 04 18:05:35 fb1 ntpd_intres[30525]: host name not found: 3.debian.pool.ntp.org Oct 04 18:21:48 fb1 ntpd_intres[30525]: host name not found: 0.debian.pool.ntp.org Oct 04 18:21:58 fb1 ntpd_intres[30525]: host name not found: 1.debian.pool.ntp.org Oct 04 18:22:08 fb1 ntpd_intres[30525]: host name not found: 2.debian.pool.ntp.org Oct 04 18:22:18 fb1 ntpd_intres[30525]: host name not found: 3.debian.pool.ntp.org • But the ntpq command is giving the saem output as given by ntpq comman in martian fb1 (except for the source servers), that the broadcasting is happening in the same manner: controls@fb1:~ 0$ ntpq -p
remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
192.168.123.255 .BCST.          16 u    -   64    0    0.000    0.000   0.000

• On the FE machines side though, the systemd-timesyncd are still unable to read the time signal from fb1 and show the status as idle:
controls@c1bhd:~ 3$timedatectl Local time: Mon 2021-10-04 18:34:38 UTC Universal time: Mon 2021-10-04 18:34:38 UTC RTC time: Mon 2021-10-04 18:34:38 Time zone: Etc/UTC (UTC, +0000) NTP enabled: yes NTP synchronized: no RTC in local TZ: no DST active: n/a controls@c1bhd:~ 0$ systemctl status systemd-timesyncd -l
● systemd-timesyncd.service - Network Time Synchronization
Active: active (running) since Mon 2021-10-04 17:21:29 UTC; 1h 13min ago
Docs: man:systemd-timesyncd.service(8)
Main PID: 244 (systemd-timesyn)
Status: "Idle."
CGroup: /system.slice/systemd-timesyncd.service
└─244 /lib/systemd/systemd-timesyncd
• So the time synchronization is still not working. We expected the FE machined to just synchronize to fb1 even though it doesn't have any upstream ntp server to synchronize to. But that didn't happen.

### Digging into mx_stream/daqd_dc errors:

• We went and changed the Restart fileld in /etc/systemd/system/daqd_dc.service on cloned fb1 to 2. This allows the service to fail and stop restarting after two attempts. This allows us to see the real error message instead of the systemd error message that the service is restarting too often. We got following:
controls@fb1:~ 3$sudo systemctl status daqd_dc -l ● daqd_dc.service - Advanced LIGO RTS daqd data concentrator Loaded: loaded (/etc/systemd/system/daqd_dc.service; enabled) Active: failed (Result: exit-code) since Mon 2021-10-04 17:50:25 UTC; 22s ago Process: 715 ExecStart=/usr/bin/daqd_dc_mx -c /opt/rtcds/caltech/c1/target/daqd/daqdrc.dc (code=exited, status=1/FAILURE) Main PID: 715 (code=exited, status=1/FAILURE) Oct 04 17:50:24 fb1 systemd[1]: Started Advanced LIGO RTS daqd data concentrator. Oct 04 17:50:25 fb1 daqd_dc_mx[715]: [Mon Oct 4 17:50:25 2021] Unable to set to nice = -20 -error Unknown error -1 Oct 04 17:50:25 fb1 daqd_dc_mx[715]: Failed to do mx_get_info: MX not initialized. Oct 04 17:50:25 fb1 daqd_dc_mx[715]: 263596 Oct 04 17:50:25 fb1 systemd[1]: daqd_dc.service: main process exited, code=exited, status=1/FAILURE Oct 04 17:50:25 fb1 systemd[1]: Unit daqd_dc.service entered failed state.  • It seemed like the only thing daqd_dc process doesn't like is that mx_stream services are in failed state in teh FE computers. So we did the same process on FE machines to get the real error messages: controls@fb1:~ 0$ sudo chroot /diskless/root
fb1:/ 0#
fb1:/ 0# sudo nano /etc/systemd/system/mx_stream.service
fb1:/ 0#
fb1:/ 0# exit
• Then I ssh'ed into c1bhd to see the error message on mx_stream service properly.
controls@c1bhd:~ 0$sudo systemctl daemon-reload controls@c1bhd:~ 0$ sudo systemctl restart mx_stream
controls@c1bhd:~ 0sudo systemctl status mx_stream -l ● mx_stream.service - Advanced LIGO RTS front end mx stream Loaded: loaded (/etc/systemd/system/mx_stream.service; enabled) Active: failed (Result: exit-code) since Mon 2021-10-04 17:57:20 UTC; 24s ago Process: 11832 ExecStart=/etc/mx_stream_exec (code=exited, status=1/FAILURE) Main PID: 11832 (code=exited, status=1/FAILURE) Oct 04 17:57:20 c1bhd systemd[1]: Starting Advanced LIGO RTS front end mx stream... Oct 04 17:57:20 c1bhd systemd[1]: Started Advanced LIGO RTS front end mx stream. Oct 04 17:57:20 c1bhd mx_stream_exec[11832]: send len = 263596 Oct 04 17:57:20 c1bhd mx_stream_exec[11832]: OMX: Failed to find peer index of board 00:00:00:00:00:00 (Peer Not Found in the Table) Oct 04 17:57:20 c1bhd mx_stream_exec[11832]: mx_connect failed Nic ID not Found in Peer Table Oct 04 17:57:20 c1bhd mx_stream_exec[11832]: c1x06_daq mmapped address is 0x7f516a97a000 Oct 04 17:57:20 c1bhd mx_stream_exec[11832]: c1bhd_daq mmapped address is 0x7f516697a000 Oct 04 17:57:20 c1bhd systemd[1]: mx_stream.service: main process exited, code=exited, status=1/FAILURE Oct 04 17:57:20 c1bhd systemd[1]: Unit mx_stream.service entered failed state.  • c1sus2 shows the same error. I'm not sure I understand these errors at all. But they seem to have nothing to do with timing issues! As usual, some help would be helpful 16371 Fri Oct 1 14:25:27 2021 yehonathanSummarySUSPRM and BS Angular Actuation transfer function magnitude measurements {Paco, Yehonathan, Hang} We measured the sensing PRMI sensing matrix. Attachment 1 shows the results, the magnitude of the response is not calibrated. The orthogonality between PRCL and MICH is still bad (see previous measurement for reference). Hang suggested that since MICH actuation with BS and PRM is not trivial (0.5*BS - 0.34*PRM) and since PRCL is so sensitive to PRM movement there might be a leakage to PRCL when we are actuating on MICH. So there may be a room to tune the PRM coefficient in the MICH output matrix. Attachment 2 shows the sensing matrix after we changed the MICH->PRM coefficient in the OSC output matrix to -0.1. It seems like it made things a little bit better but not much and also there is a huge uncertainty in the MICH sensing. Attachment 1: MICH_PRM_-0.34.png Attachment 2: MICH_PRM_-0.1.png 16370 Fri Oct 1 12:12:54 2021 StephenUpdateBHDITMY (3002) CAD layout pushed to Box Koji requested current state of BHD 3D model. I pushed this to Box after adding the additional SOSs and creating an EASM representation (also posted, Attachment 1). I also post the PDF used to dimension this model (Attachment 2). This process raised some points that I'll jot down here: 1) Because the 40m CAD files are not 100% confirmed to be clean of any student license efforts, we cannot post these files to the PDM Vault or transmit them this way. When working on BHD layout efforts, these assemblies which integrate new design work therefore must be checked for most current revisions of vault-managed files - this Frankenstein approach is not ideal but can be managed for this effort. 2) Because the current files reflect the 40m as built state (as far as I can tell), I shared the files in a zip directory without increasing the revisions. It is unclear whether revision control is adequate to separate [current 40m state as reflected in CAD] from [planned 40m state after BHD upgrade]. Typically a CAD user would trust that we could find the version N assembly referenced in the drawing from year Y, so we wouldn't hesitate to create future design work in a version N+1 assembly file pending a current drawing. However, this form of revision control is not implemented. Perhaps we want to use configurations to separate design states (in other words, create a parallel model of every changed component, without creating paralle files - these configurations can be selected internal to the assembly without a need to replace files)? Or more simply (and perhaps more tenuously), we could snapshot the Box revisions and create a DCC page which notes the point of departure for BHD efforts? Anyway, the cold hard facts: - Box location: 40m/40m_cad_models/Solidworks_40m (LINK) - Filenames: 3002.zip and 3002 20211001 ITMY BHD for Koji presentation images.easm (healthy disregard for concerns about spaces in filenames) Attachment 1: 3002_20211001_ITMY_BHD_for_Koji_presentation_images.easm Attachment 2: 40m_upgrade_layout_20200611-ITMY_Beam_Dim.pdf 16369 Thu Sep 30 18:04:31 2021 PacoSummaryCalibrationXARM OLTF (calibration) with three lines [anchal, paco] We repeated the same procedure as before, but with 3 different lines at 55.511, 154.11, and 1071.11 Hz. We overlay the OLTF magnitudes and phases with our latest model (which we have updated with Koji's help) and include the rms uncertainties as errorbars in Attachment #1. We also plot the noise ASDs of calibrated OLTF magnitudes at the line frequencies in Attachment #2. These curves are created by calculating power spectral density of timeseries of OLTF values at the line frequencies generated by demodulated XARM_IN and ETMX_LSC_OUT signals. We have overlayed the TRX noise spectrum here as an attempt to see if we can budget the noise measured in values of G to the fluctuation in optical gain due to changing power in the arms. We multiplied the the transmission ASD with the value of OLTF at those frequencies as the transfger function from normalized optical gain to the total transfer function value. It is weird that the fluctuations in transmission power at 1 mHz always crosses the total noise in the OLTF value in all calibration lines. This could be an artificat of our data analysis though. Even if the contribution of the fluctuating power is correct, there is remaining excess noise in the OLTF to be budgeted. Attachment 1: XARM_OLTF_Model_and_Meas.pdf Attachment 2: Gmag_ASD_nb_withTRX.pdf 16368 Thu Sep 30 14:13:18 2021 AnchalUpdateLSCHV supply to Xend Green laser injection mirrors M1 and M2 PZT restored Late elog, original date Sep 15th We found that the power switch of HV supply that powers the PZT drivers for M1 and M2 on Xend green laser injection alignment was tripped off. We could not find any log of someone doing it, it is a physical switch. Our only explanation is that this supply might have a solenoid mechansm to shut off during power glitches and it probably did so on Aug 23 (see 40m/16287). We were able to align the green laser using PZT again, however, the maximum power at green transmission from X arm cavity is now about half of what it used to be before the glitch. Maybe the seed laser on the X end died a little. 16367 Thu Sep 30 14:09:37 2021 AnchalSummaryCDSNew way to ssh into c1teststand Late elog, original time Wed Sep 29 14:09:59 2021 We opened a new port (22220) in the router to the martian subnetwork which is forwarded to port 22 on c1teststand (192.168.113.245) allowing direct ssh access to c1teststand computer from the outside world using: Checkout this wiki page for unredadcted info. 16366 Thu Sep 30 11:46:33 2021 Ian MacMillanSummaryComputersQuantization Noise Calculation Summary First and foremost I have the updated bode plot with the mode moved to 10 Hz. See Attachment 1. Note that the comparison measurement is a % difference whereas in the previous bode plot it was just the difference. I also wrapped the phase so that jumps from -180 to 180 are moved down. This eliminates massive jumps in the % difference. Next, I have two comparison plots: 32 bit and 64bit. As mentioned above I moved the mode to 10 Hz and just excited both systems at 3.4283Hz with an amplitude of 1. As we can see on the plot the two models are practically the same when using 64bits. With the 32bit system, we can see that the noise in the IIR filter is much greater than in the State-space model at frequencies greater than our mode. Note about windowing and averaging: I used a Hanning window with averaging over 4 neighbor points. I came to this number after looking at the results with less averaging and more averaging. In the code, this can be seen as nperseg=num_samples/4 which is then fed into signal.welch Attachment 1: SS-IIR-Bode.pdf Attachment 2: PSD_32bit.pdf Attachment 3: PSD_64bit.pdf 16365 Wed Sep 29 17:10:09 2021 AnchalSummaryCDSc1teststand problems summary [anchal, ian] We went and collected some information for the overlords to fix the c1teststand DAQ network issue. • from c1teststand, c1bhd and c1sus2 computers were not accessible through ssh. (No route to host). So we restarted both the computers (the I/O chassis were ON). • After the computers restarted, we were able to ssh into c1bhd and c1sus, ad we ran rtcds start c1x06 and rtcds start c1x07. • The first page in attachment shows the screenshot of GDS_TP screens of the IOP models after this step. • Then we started teh user models by running rtcds start c1bhd and rtcds start c1su2. • The second page shows the screenshot of GDS_TP screens. You can notice that DAQ status is red in all the screens and the DC statuses are blank. • So we checked if daqd_ services are running in the fb computer. They were not. So we started them all by sudo systemctl start daqd_*. • Third page shows the status of all services after this step. the daqd_dc.service remained at failed state. • open-mx_stream.service was not even loaded in fb. We started it by running sudo systemctl start open-mx_stream.service. • The fourth page shows the status of this service. It started without any errors. • However, when we went to check the status of mx_stream.service in c1bhd and c1sus2, they were not loaded and we we tried to start them, they showed failed state and kept trying to start every 3 seconds without success. (See page 5 and 6). • Finally, we also took a screenshot of timedatectl command output on the three computers fb, c1bhd, and c1sus2 to show that their times were not synced at all. • The ntp service is running on fb but it probably does not have access to any of the servers it is following. • The timesyncd on c1bhd and c1sus2 (FE machines) is also running but showing status 'Idle' which suggested they are unable to find the ntp signal from fb. • I believe this issue is similar to what jamie ficed in the fb1 on martian network in 40m/16302. Since the fb on c1teststand network was cloned before this fix, it might have this dysfunctional ntp as well. We would try to get internet access to c1teststand soon. Meanwhile, someone with more experience and knowledge should look into this situation and try to fix it. We need to test the c1teststand within few weeks now. Attachment 1: c1teststand_issues_summary.pdf 16364 Wed Sep 29 09:36:26 2021 JordanUpdateSUS2" Adapter Ring Parts for SOS Arrived 9/28/21 The remaining machined parts for the SOS adapter ring have arrived. I will inspect these today and get them ready for C&B. Attachment 1: 20210929_092418.jpg 16363 Tue Sep 28 16:31:52 2021 PacoSummaryCalibrationXARM OLTF (calibration) at 55.511 Hz [anchal, paco] Here is a demonstration of the methods leading to the single (X)arm calibration with its budget uncertainty. The steps towards this measurement are the following: 1. We put a single line excitation through the C1:SUS-ETMX_LSC_EXC at 55.511 Hz, amp = 1 counts, gain = 300 (ramptime=10 s). 2. With the arm locked, we grab a long timeseries of the C1:LSC-XARM_IN1_DQ (error point) and C1:SUS-ETMX_LSC_OUT_DQ (control point) channels. 3. We assume the single arm loop to have the four blocks shown in Attachment #1, A (actuator + sus), plant (mainly the cavity pole), D (detection + electronics), and K (digital control). 1. At this point, Anchal made a model of the single arm loop including the appropriate filter coefficients and other parameters. See Attachments #2-3 for the split and total model TFs. 2. Our line would actually probe a TF from point b (error point) to point d (control point). We multiplied our measurement with open loop TF from b to d from model to get complete OLTF. 3. Our initial estimate from documents and elog made overall loop shape correct but it was off by an overall gain factor. This could be due to wrong assumption on RFPD transimpedance or analog gains of AA or whitening filters. We have corrected for this factor in the RFPD transimpedance, but this needs to be checked (if we really care). 4. We demodulate decimated timeseries (final sampling rate ~ 2.048 kHz) and I & Q for both the b and d signals. From this and our model for K, we estimate the OLTF. Attachment #4 shows timeseries for magnitude and phase. 5. Finally, we compute the ASD for the OLTF magnitude. We plot it in Attachment #5 together with the ASD of the XARM transmission (C1:LSC-TRX_OUT_DQ) times the OLTF to estimate the optical gain noise ASD (this last step was a quick attempt at budgeting the calibration noise). 1. For each ASD we used N = 24 averages, from which we estimate rms (statistical) uncertainties which are depicted by error bands ($\pm \sigma$) around the lines. ** Note: We ran the same procedure using dtt (diaggui) to validate our estimates at every point, as well as check our SNR in b and d before taking the ~3.5 hours of data. Attachment 1: OLTF_Calibration_Scheme.jpg Attachment 2: XARM_POX_Lock_Model_TF.pdf Attachment 3: XARM_OLTF_Total_Model.pdf Attachment 4: XARM_OLTF_55p511_Hz_timeseries.pdf Attachment 5: Gmag_55p511_Hz_ASD.pdf 16362 Mon Sep 27 17:04:43 2021 ranaSummaryComputersQuantization Noise Calculation Summary I suggest that you 1. change the corner frequency to 10 Hz as I suggested last week. This filter, as it is, is going to give you trouble. 2. Put in a sine wave at 3.4283 Hz with an amplitude of 1, rather than white noise. In this way, its not necessary to do any subtraction. Just make PSD of the output of each filter. 3. Be careful about window length and window function. If you don't do this carefully, your PSD will be polluted by window bleeding. 16361 Mon Sep 27 16:03:15 2021 Ian MacMillanSummaryComputersQuantization Noise Calculation Summary I have coded up the procedure in the previous post: The result does not look like what I would expect. As shown in Attachment1 I have the power spectrum of the 32-bit output and the 64-bit output as well as the power spectrum of the two subtracted time series as well as the subtracted power spectra of both. unfortunately, all of them follow the same general shape of the raw output of the filter. I would not expect quantization noise to follow the shape of the filter. I would instead expect it to be more uniform. If anything I would expect the quantization noise to increase with frequency. If a high-frequency signal is run through a filter that has high quantization noise then it will severely degrade: i.e. higher quantization noise. This is one reason why I am confused by what I am seeing here. In all cases including feeding the same and different white noise into both filters, I have found that the calculated quantization noise is proportional to the response of the filter. this seems wrong to me so I will continue to play around with it to see if I can gain any intuition about what might be happening. Attachment 1: DeltaNoiseSpectrum.pdf 16360 Mon Sep 27 12:12:15 2021 Ian MacMillanSummaryComputersQuantization Noise Calculation Summary I have not been able to figure out a way to make the system that Aaron and I talked about. I'm not even sure it is possible to pull the information out of the information I have in this way. Even the book uses a comparison to a high precision filter as a way to calculate the quantization noise: "Quantization noise in digital filters can be studied in simulation by comparing the behavior of the actual quantized digital filter with that of a refrence digital filter having the same structure but whose numerical calculations are done extremely accurately." -Quantization Noise by Bernard Widrow and Istvan Kollar (pg. 416) Thus I will use a technique closer to that used in Den Martynov's thesis (see appendix B starting on page 171). A summary of my understanding of his method is given here: A filter is given raw unfiltered gaussian data $f(t)$ then it is filtered and the result is the filtered data $x(t)$ thus we get the result: $f(t)\rightarrow x(t)=x_N(t)+x_q(t)$ where $x_N(t)$ is the raw noise filtered through an ideal filter and $x_q(t)$ is the difference which in this case is the quantization noise. Thus I will input about 100-1000 seconds of the same white noise into a 32-bit and a 64-bit filter. (hopefully, I can increase the more precise one to 128 bit in the future) then I record their outputs and subtract the from each other. this should give us the Quantization error $e(t)$: $e(t)=x_{32}(t)-x_{64}(t)=x_{N_{32}}(t)+x_{q_{32}}(t) - x_{N_{64}}(t)-x_{q_{64}}(t)$ and since $x_{N_{32}}(t)=x_{N_{64}}(t)$ because they are both running through ideal filters: $e(t)=x_{N}(t)+x_{q_{32}}(t) - x_{N}(t)-x_{q_{64}}(t)$ $e(t)=x_{q_{32}}(t) -x_{q_{64}}(t)$ and since in this case, we are assuming that the higher bit-rate process is essentially noiseless we get the Quantization noise $x_{q_{32}}(t)$. If we make some assumptions, then we can actually calculate a more precise version of the quantization noise: "Since aLIGO CDS system uses double precision format, quantization noise is extrapolated assuming that it scales with mantissa length" -Denis Martynov's Thesis (pg. 173) From this assumption, we can say that the noise difference between the 32-bit and 64-bit filter outputs: $x_{q_{32}}(t)-x_{q_{64}}(t)$ is proportional to the difference between their mantissa length. by averaging over many different bit lengths, we can estimate a better quantization noise number. I am building the code to do this in this file 16359 Thu Sep 23 18:18:07 2021 YehonathanUpdateBHDSOS assembly I have noticed that the dumbells coming back from C&B had glue residues on them. An example is shown in attachment 1: it can be seen that half of the dumbell's surface is covered with glue. Jordan gave me a P800 sandpaper to remove the glue. I picked the dumbells with the dirty face down and slid them over the sandpaper in 8 figures several times to try and keep the surface untilted. Attachment 2 shows the surface from attachment 1 after this process. Next, the dumbells will be sent to another C&B. Attachment 1: dumbell_before.png Attachment 2: dumbell_after.png 16358 Thu Sep 23 15:29:11 2021 PacoSummarySUSPRM and BS Angular Actuation transfer function magnitude measurements [Anchal, Paco] We had a second go at this with an increased number of averages (from 10 to 100) and higher excitation amplitudes (from 1000 to 10000). We did this to try to reduce the relative uncertainty a-la-Bendat-and-Pearsol $\delta G / G = \frac{1}{\gamma \sqrt{n_{\rm avg}}}$ where $\gamma, n_{\rm avg}$ are the coherence and number of averages respectively. Before, this estimate had given us a ~30% relative uncertainty and now it has been improved to ~ 10%. The re-measured TFs are in Attachment #1. We did 4 sweeps for each optic (BS, PRM) and removed the 1/f^2 slope for clarity. We note a factor of ~ 4 difference in the magnitude of the coil to angle TFs from BS to PRM (the actuation strength in BS is smaller). For future reference: With complex G, we get complex error in G using the formula above. To get uncertainity in magnitude and phase from real-imaginary uncertainties, we do following (assuming the noise in real and imaginary parts of the measured transfer function are incoherent with each other): $G = \alpha + i\beta$ $\delta G = \delta\alpha + i\delta \beta$ $\delta |G| = \frac{1}{|G|}\sqrt{\alpha^2 \delta\alpha^2 + \beta^2 \delta \beta^2}$ $\delta(\angle G) = \frac{1}{|G|^2}\sqrt{\alpha^2 \delta\alpha^2 + \beta^2 \delta\beta^2} = \frac{\delta |G|}{|G|}$ Attachment 1: BS_PRM_ANG_ACT_TF.pdf 16357 Thu Sep 23 14:17:44 2021 TegaUpdateElectronicsSat Amp modifications debugging update Debugging complete. All units now have the correct TP4 voltage reading needed to drive a nominal current of 35 mA through to OSEM LED. The next step is to go ahead and replace the components and test afterward that everything is OK. Unit Serial Number Issues Debugging Status S1200736 TP4 @ LED4 on chan 1-4 PCB reads 13V instead of 5V This issue was caused by a short between the Collector & Base legs of the Q1 transistor. Solution - Remove the short connection between the Collector & Base legs of the Q1 transistor legs DONE S1200737 NONE N/A DONE S1200739 NONE N/A DONE S1200746 TP4 @ LED3 on chan 5-8 PCB reads 0.765 V instead of 5V This issue was caused by a short between the Emitter & Base legs of the Q1 transistor. Solution - Remove the short connection between the Emitter & Base legs of the Q1 transistor legs Complications - I was extra careful this time because of the problem of loose cable from the last flip-over of the right PCB containing chan 5-8. Anyways, after I was done I noticed one of the pink wires (it carries the +14V to the left PCB) had come off on P1. At least this time I could also see that the corresponding front panel green LED turn off as a result. So I resecured the wire to the connector (using solder as my last attempt yesterday to reattach the via crimping didn't work after a long time trying. I hope this is not a problem.) and checked the front panel LED turns on when the unit is powered before closing the unit. These connectors are quite flimsy. DONE S1200747 TP4 @ LED2 on chan 1-4 PCB reads 13V instead of 5V This issue was caused by a short between the Collector & Base legs of the Q1 transistor. Solution - Remove the short connection between the Collector & Base legs of the Q1 transistor legs DONE 16356 Wed Sep 22 17:22:59 2021 TegaUpdateElectronicsSat Amp modifications [Koji, Tega] Decided to do a quick check of the remaining Sat Amp units before component replacement to identify any unit with defective LED circuits. Managed to examine 5 out of 10 units, so still have 5 units remaining. Also installed the photodiode bias voltage jumper (JP1) on all the units processed so far. Unit Serial Number Issues Debugging Status S1200738 TP4 @ LED3 on chan 1-4 PCB was ~0.7 V instead of 5V Koji checked the solder connections of the various components, then swapped out the IC OPAMP. Removed DB9 connections to the front panel to get access to the bottom of the board. Upon close inspection, it looked like an issue of a short connection between the Emitter & Base legs of the Q1 transistor. Solution - Remove the short connection between the Emitter & Base legs of the Q1 transistor legs. DONE S1200748 TP4 @ LED2 on chan 1-4 PCB was ~0.7 V instead of 5V This issue was caused by a short connection between the Emitter & Base legs of the Q1 transistor. Solution - Remove the short connection between the Emitter & Base legs of the Q1 transistor legs. DONE S1200749 NONE N/A DONE S1200750 NONE N/A DONE S1200751 NONE N/A DONE Defective unit with updated resistors and capacitors in the previous elog Unit Serial Number Issues Debugging Status S1200744 TP4 @ LED1,2 on PCB S2100568 is 13V instead of 5V TP4 @ LED4 on PCB S2100559 is 13V instead of 5V This issue was caused by a short between the Collector & Base legs of the Q1 transistor. Solution - Remove the short connection between the Collector & Base legs of the Q1 transistor legs Complications - During the process of flipping the board to get access to the bottom of the board, a connector holding the two middle black wires, on P1, came loose. I resecured the wires to the connector and checked all TP4s on the board afterwards to make sure things are as expected. DONE Quote: Running update of Sat Amp modification work, which involves the following procedure (x8) per unit: 1. Replace R20 & R24 with 4.99K ohms, R23 with 499 ohms, and remove C16. 2. (Testing) Connect LEDDrive output to GND and check that • TP4 is ~ 5V • TP5-8 ~ 0V. 3. Install 40m Satellite to Flange Adapter (D2100148-v1)  Unit Serial Number Issues Status S1200740 NONE DONE S1200742 NONE DONE S1200743 NONE DONE S1200744 TP4 @ LED1,2 on PCB S2100568 is 13V instead of 5V TP4 @ LED4 on PCB S2100559 is 13V instead of 5V DONE S1200752 NONE DONE 16355 Wed Sep 22 14:22:35 2021 Ian MacMillanSummaryComputersQuantization Noise Calculation Summary Now that we have a model of how the SS and IIR filters work we can get to the problem of how to measure the quantization noise in each of the systems. Den Martynov's thesis talks a little about this. from my understanding: He measured quantization noise by having two filters using two types of variables with different numbers of bits. He had one filter with many more bits than the second one. He fed the same input signal to both filters then recorded their outputs x_1 and x_2, where x_2 had the higher number of bits. He then took the difference x_1-x_2. Since the CDS system uses double format, he assumes that quantization noise scales with mantissa length. He can therefore extrapolate the quantization noise for any mantissa length. Here is the Code that follows the following procedure (as of today at least) This problem is a little harder than I had originally thought. I took Rana's advice and asked Aaron about how he had tackled a similar problem. We came up with a procedure explained below (though any mistakes are my own): 1. Feed different white noise data into three of the same filter this should yield the following equation: $\textbf{S}_i^2 =\textbf{S}_{ni}^2+\textbf{S}_x^2$, where $\textbf{S}_i^2$ is the power spectrum of the output for the ith filter, $\textbf{S}_{ni}^2$ is the noise filtered through an "ideal" filter with no quantization noise, and $\textbf{S}_x^2$ is the power spectrum of the quantization noise. Since we are feeding random noise into the input the power of the quantization noise should be the same for all three of our runs. 2. Next, we have our three outputs: $\textbf{S}_1^2$, $\textbf{S}_2^2$, and $\textbf{S}_3^2$ that follow the equations: $\textbf{S}_1^2 =\textbf{S}_{n1}^2+\textbf{S}_x^2$ $\textbf{S}_2^2 =\textbf{S}_{n2}^2+\textbf{S}_x^2$ $\textbf{S}_3^2 =\textbf{S}_{n3}^2+\textbf{S}_x^2$ From these three equations, we calculate the three quantities: $\textbf{S}_{12}^2$$\textbf{S}_{23}^2$, and $\textbf{S}_{13}^2$ which are calculated by: $\textbf{S}_{12}^2 =\textbf{S}_{1}^2-\textbf{S}_2^2\approx \textbf{S}_{n1}^2 -\textbf{S}_{n2}^2$ $\textbf{S}_{23}^2 =\textbf{S}_{2}^2-\textbf{S}_3^2\approx \textbf{S}_{n2}^2 -\textbf{S}_{n3}^2$ $\textbf{S}_{13}^2 =\textbf{S}_{1}^2-\textbf{S}_3^2\approx \textbf{S}_{n1}^2 -\textbf{S}_{n3}^2$ from these quantities, we can calculate three values: $\bar{\textbf{S}}_{n1}^2$$\bar{\textbf{S}}_{n2}^2$, and $\bar{\textbf{S}}_{n3}^2$ since these are just estimates we are using a bar on top. These are calculated using: $\bar{\textbf{S}}_{n1}^2\approx\frac{1}{2}(\textbf{S}_{12}^2+\textbf{S}_{13}^2+\textbf{S}_{23}^2)$ $\bar{\textbf{S}}_{n2}^2\approx\frac{1}{2}(-\textbf{S}_{12}^2+\textbf{S}_{13}^2+\textbf{S}_{23}^2)$ $\bar{\textbf{S}}_{n3}^2\approx\frac{1}{2}(\textbf{S}_{12}^2+\textbf{S}_{13}^2-\textbf{S}_{23}^2)$ using these estimates we can then estimate $\textbf{S}_{x}^2$ using the formula: $\textbf{S}_{x}^2 \approx \textbf{S}_{1}^2 - \bar{\textbf{S}}_{n1}^2 \approx \textbf{S}_{2}^2 - \bar{\textbf{S}}_{2}^2 \approx \textbf{S}_{3}^2 - \bar{\textbf{S}}_{n3}^2$ we can average the three estimates for $\textbf{S}_{x}^2$ to come up with one estimate. This procedure should be able to give us a good estimate of the quantization noise. However, in the graph shown in the attachments below show that the noise follows the transfer function of the model to begin with. I would not expect this to be true so I believe that there is an error in the above procedure or in my code that I am working on finding. I may have to rework this three-corner hat approach. I may have a mistake in my code that I will have to go through. I would expect the quantization noise to be flatter and not follow the shape of the transfer function of the model. Instead, we have what looks like just the result of random noise being filtered through the model. Next steps: The first real step is being able to quantify the quantization noise but after I fix the issues in my code I will be able to start liking at optimal model design for both the state-space model and the direct form II model. I have been looking through the book "Quantization noise" by Bernard Widrow and Istvan Kollar which offers some good insights on how to minimize quantization noise. Attachment 1: IIR64-bitnoisespectrum.pdf 16354 Wed Sep 22 12:40:04 2021 AnchalSummaryCDSXARM YARM UGF Servo and Oscillators shifted to OAF To reduce burden on c1lsc, I've shifted the added UGF block to to c1oaf model. c1lsc had to be modified to allow addition of an oscillator in the XARm and YARM control loops and take out test points before and after the addition to c1oaf through shared memory IPC to do realtime demodulation in c1oaf model. The new models built and installed successfully and I've been able to recover both single arm locks after restarting the computers. 16353 Wed Sep 22 11:43:04 2021 ranaSummaryCalibrationXARM calibration noise I would expect to see some lower frequency effects. i.e. we should look at the timeseries of the demod with the excitation on and off. I would guess tat the exc on should show us the variations in the optical gain below 3 Hz, whereas the exc off would not show it. Maybe you should do some low pass filtering on the time series you have to see the ~DC effects? Also, reconsider your AA filter design: how do you quantitatively choose the cutoff frequency and stopband depth? 16352 Tue Sep 21 11:13:01 2021 PacoSummaryCalibrationXARM calibration noise Here are some plots from analyzing the C1:LSC-XARM calibration. The experiment is done with the XARM (POX) locked, a single line is injected at C1:LSC-XARM_EXC at f0 with some amplitude determined empirically using diaggui and awggui tools. For the analysis detailed in this post, f0 = 19 Hz, amp = 1 count, and gain = 300 (anything larger in amplitude would break the lock, and anything lower in frequency would not show up because of loop supression). Clearly, from Attachment #3 below, the calibration line can be detected with SNR > 1. We read the test point right after the excitation C1:LSC-XARM_IN2 which, in a simplified loop will carry the excitation suppressed by 1 - OLTF, the open loop transfer function. The line is on for 5 minutes, and then we read for another 5 minutes but with the excitation off to have a reference. Both the calibration and reference signal time series are shown in Attachment #1 (decimated by 8). The corresponding ASDs are shown in Attachment #2. Then, we demodulate at 19 Hz and a 30 Hz, 4th-order butterworth LPF, and get an I and Q timeseries (shown in Attachment #3). Even though they look similar, the Q is centered about 0.2 counts, while the I is centered about 0.0. From this time series, we can of course show the noise ASDs in Attachment #3. The ASD uncertainty bands in the last plot are statistical estimates and depend on the number of segments used in estimating the PSD. A thing to note is that the noise features surrounding the signal ASD around f0 are translated into the ASD in the demodulated signals, but now around dc. I guess from Attachment #3 there is no difference in the noise spectra around the calibration line with and without the excitation. This is what I would have expected from a linear system. If there was a systematic contribution, I would expect it to show at very low frequencies. Attachment 1: XARM_signal_asd.pdf Attachment 2: XARM_demod_timeseries.pdf Attachment 3: XARM_demod_asds.pdf Attachment 4: XARM_cal_0921_timeseries.pdf 16351 Tue Sep 21 11:09:34 2021 AnchalSummaryCDSXARM YARM UGF Servo and Oscillators added I've updated the c1LSC simulink model to add the so-called UGF servos in the XARM and YARM single arm loops as well. These were earlier present in DARM, CARM, MICH and PRCL loops only. The UGF servo themselves serves a larger purpose but we won't be using that. What we have access to now is to add an oscillator in the single arm and get realtime demodulated signal before and after the addition of the oscillator. This would allow us to get the open loop transfer function and its uncertaintiy at particular frequencies (set by the oscillator) and would allow us to create a noise budget on the calibration error of these transfer functions. The new model has been committed locally in the 40m/RTCDSmodels git repo. I do not have rights to push to the remote in git.ligo. The model builds, installs and starts correctly. 16350 Mon Sep 20 21:56:07 2021 KojiUpdateComputersWifi internet fixed Ug, factory resets... Caltech IMSS announced that there was an intermittent network service due to maintenance between Sept 19 and 20. And there seemed some aftermath of it. Check out "Caltech IMSS" 16349 Mon Sep 20 20:43:38 2021 TegaUpdateElectronicsSat Amp modifications Running update of Sat Amp modification work, which involves the following procedure (x8) per unit: 1. Replace R20 & R24 with 4.99K ohms, R23 with 499 ohms, and remove C16. 2. (Testing) Connect LEDDrive output to GND and check that • TP4 is ~ 5V • TP5-8 ~ 0V. 3. Install 40m Satellite to Flange Adapter (D2100148-v1)  Unit Serial Number Issues Status S1200740 NONE DONE S1200742 NONE DONE S1200743 NONE DONE S1200744 TP4 @ LED1,2 on PCB S2100568 is 13V instead of 5V TP4 @ LED4 on PCB S2100559 is 13V instead of 5V DONE S1200752 NONE DONE Attachment 1: IMG_20210920_203456226.jpg 16348 Mon Sep 20 15:42:44 2021 Ian MacMillanSummaryComputersQuantization Code Summary This post serves as a summary and description of code to run to test the impacts of quantization noise on a state-space implementation of the suspension model. Purpose: We want to use a state-space model in our suspension plant code. Before we can do this we want to test to see if the state-space model is prone to problems with quantization noise. We will compare two models one for a standard direct-ii filter and one with a state-space model and then compare the noise from both. Signal Generation: First I built a basic signal generator that can produce a sine wave for a specified amount of time then can produce a zero signal for a specified amount of time. This will let the model ring up with the sine wave then decay away with the zero signal. This input signal is generated at a sample rate of 2^16 samples per second then stored in a numpy array. I later feed this back into both models and record their results. State-space Model: The code can be seen here The state-space model takes in the list of excitation values and feeds them through a loop that calculates the next value in the output. Given that the state-space model follows the form $\dot{x}(t)=\textbf{A}x(t)+ \textbf{B}u(t)$ and $y(t)=\textbf{C}x(t)+ \textbf{D}u(t)$ , the model has three parts the first equation, an integration, and the second equation. 1. The first equation takes the input x and the excitation u and generates the x dot vector shown on the left-hand side of the first state-space equation. 2. The second part must integrate x to obtain the x that is found in the next equation. This uses the velocity and acceleration to integrate to the next x that will be plugged into the second equation 3. The second equation in the state space representation takes the x vector we just calculated and then multiplies it with the sensing matrix C. we don't have a D matrix so this gives us the next output in our system This system is the coded form of the block diagram of the state space representation shown in attachment 1 Direct-II Model: The direct form 2 filter works in a much simpler way. because it involves no integration and follows the block diagram shown in Attachment 2, we can use a single difference equation to find the next output. However, the only complication that comes into play is that we also have to keep track of the w(n) seen in the middle of the block diagram. We use these two equations to calculate the output value $y[n]=b_0 \omega [n]+b_1 \omega [n-1] +b_2 \omega [n-2]$, where w[n] is $\omega[n]=x[n] - a_1 \omega [n-1] -a_2 \omega[n-2]$ Bit length Control: To control the bit length of each of the models I typecast all the inputs using np.float then the bit length that I want. This simulates the computer using only a specified bit length. I have to go through the code and force the code to use 128 bit by default. Currently, the default is 64 bit which so at the moment I am limited to 64 bit for highest bit length. I also need to go in and examine how numpy truncates floats to make sure it isn't doing anything unexpected. Bode Plot: The bode plot at the bottom shows the transfer function for both the IIR model and the state-space model. I generated about 100 seconds of white noise then computed the transfer function as $G(f) = \frac{P_{csd}(f)}{P_{psd}(f)}$ which is the cross-spectral density divided by the power spectral density. We can see that they match pretty closely at 64 bits. The IIR direct II model seems to have more noise on the surface but we are going to examine that in the next elog Attachment 1: 472px-Typical_State_Space_model.svg.png Attachment 2: Biquad_filter_DF-IIx.svg.png Attachment 3: SS-IIR-TF.pdf 16346 Mon Sep 20 15:23:08 2021 YehonathanUpdateComputersWifi internet fixed Over the weekend and today, the wifi was acting bad with frequent disconnections and no internet access. I tried to log into the web interface of the ASUS wifi but with no success. I pushed the reset button for several seconds to restore factory settings. After that, I was able to log in. I did the automatic setup and defined the wifi passwords to be what they used to be. Internet access was restored. I also unplugged and plugged back all the wifi extenders in the lab and moved the extender from the vertex inner wall to the outer wall of the lab close to the 1X3. Now, there seems to be wifi reception both in X and Y arms (according to my android phone). 16345 Mon Sep 20 14:22:00 2021 ranaSummarySUSPRM and BS Angular Actuation transfer function magnitude measurements I suggest plotting all the traces in the plot so we can see their differences. Also remove the 1/f^2 slope so that we can see small differences. Since the optlev servos all have low pass filters around 15-20 Hz, its not necessary to turn off the optlev servos for this measurement. I think that based on the coherence and the number of averages, you should also be able to use Bendat and Piersol so estimate the uncertainy as a function of frequency. And we want to see the comparison coil-by-coil, not in the DoF basis. 4 sweeps for BS and 4 sweeps for PRM. 16344 Mon Sep 20 14:11:40 2021 KojiUpdateBHDEnd DAC Adapter Unit D2100647 I've uploaded the schematic and PCB PDF for End DAC Adapter Unit D2100647. Please review the design. • CH1-8 SUS actuation channels. • 5CHs out of 8CHs are going to be used, but for future extensions, all the 8CHs are going to be filled. • It involves diff-SE conversion / dewhitening / SE-diff conversion. Does this make sense? • CH9-12 PZT actuation channels. It is designed to send out 4x SE channels for compatibility. The channels have the jumpers to convert it to pass through the diff signals. • CH13-16 are general purpose DIFF/SE channels. CH13 is going to be used for ALS Laser Slow control. The other 3CHs are spares. The internal assembly drawing & BOM are still coming. Attachment 1: D2100647_End_DAC_Adapter.pdf 16343 Mon Sep 20 12:20:31 2021 PacoSummarySUSPRM and BS Angular Actuation transfer function magnitude measurements [yehonathan, paco, anchal] We attempted to find any symptoms for actuation problems in the PRMI configuration when actuated through BS and PRM. Our logic was to check angular (PIT and YAW) actuation transfer function in the 30 to 200 Hz range by injecting appropriately (f^2) enveloped excitations in the SUS-ASC EXC points and reading back using the SUS_OL (oplev) channels. From the controls, we first restored the PRMI Carrier to bring the PRM and BS to their nominal alignment, then disabled the LSC output (we don't need PRMI to be locked), and then turned off the damping from the oplev control loops to avoid supressing the excitations. We used diaggui to measure the 4 transfer functions magnitudes PRM_PIT, PRM_YAW, BS_PIT, BS_YAW, as shown below in Attachments #1 through #4. We used the Oplev calibrations to plot the magnitude of the TFs in units of urad / counts, and verified the nominal 1/f^2 scaling for all of them. The coherence was made as close to 1 as possible by adjusting the amplitude to 1000 counts, and is also shown below. A dip at 120 Hz is probably due to line noise. We are also assuming that the oplev QPDs have a relatively flat response over the frequency range below. Attachment 1: PRM_PIT_ACT_TF.pdf Attachment 2: PRM_YAW_ACT_TF.pdf Attachment 3: BS_PIT_ACT_TF.pdf Attachment 4: BS_YAW_ACT_TF.pdf 16342 Fri Sep 17 20:22:55 2021 KojiUpdateSUSEQ M4.3 Long beach EQ M4.3 @longbeach 2021-09-18 02:58:34 (UTC) / 07:58:34 (PDT) https://earthquake.usgs.gov/earthquakes/eventpage/ci39812319/executive • All SUS Watchdogs tripped, but the SUSs looked OK except for the stuck ITMX. • Damped the SUSs (except ITMX) • IMC automatically locked • Turned off the damping of ITMX and shook it only with the pitch bias -> Easily unstuck -> damping recovered -> realignment of the ITMX probably necessary. • Done. 16341 Fri Sep 17 00:56:49 2021 KojiUpdateGeneralAwesome The Incredible Melting Man! 16340 Thu Sep 16 20:18:13 2021 AnchalUpdateGeneralReset Fridge brought back inside. Quote: Put outside.  Quote: It happened again. Defrosting required. Attachment 1: PXL_20210917_031633702.jpg 16339 Thu Sep 16 14:08:14 2021 Ian MacMillanFrogs Tour I gave some of the data analysts a look around because they asked and nothing was currently going on in the 40m. Nothing was changed. 16338 Thu Sep 16 12:06:17 2021 TegaUpdateComputer Scripts / ProgramsTemperature sensors added to the summary pages We can now view the minute trend of the temperature sensors under the PEM tab of the summary pages. See attachment 1 for an example of today's temperature readings. Attachment 1: TempPlot_2021-09-16_12.04.19PM.png 16337 Thu Sep 16 10:07:25 2021 AnchalUpdateGeneralMelting 2 Put outside.  Quote: It happened again. Defrosting required. Attachment 1: PXL_20210916_170602832.jpg 16336 Thu Sep 16 01:16:48 2021 KojiUpdateGeneralFrozen 2 It happened again. Defrosting required. Attachment 1: P_20210916_003406_1.jpg 16335 Thu Sep 16 00:00:20 2021 KojiUpdateGeneralRIO Planex 1064 Lasers in the south cabinet RIO Planex 1064 Lasers in the south cabinet Property Number C30684/C30685/C30686/C30687 Attachment 1: P_20210915_232426.jpg 16334 Wed Sep 15 23:53:54 2021 KojiSummaryGeneralTowards the end upgrade Ordered compoenents are in. - Made 36 more Sat Amp internal boards (Attachment 1). Now we can install the adapters to all the 19 sat amp units. - Gave Tega the components for the sat amp adapter units. (Attachment 2) - Gave Tega the componennts for the sat amp / coil driver modifications. - Made 5 PCBs for the 16bit DAC AI rear panel interface (Attachment 3) Attachment 1: P_20210915_231308.jpg Attachment 2: P_20210915_225039.jpg Attachment 3: P_20210915_224341.jpg 16333 Wed Sep 15 23:38:32 2021 KojiUpdateALSALS ASX PZT HV was off -> restored It was known that the Y end ALS PZTs are not working. But Anchal reported in the meeting that the X end PZTs are not working too. We went down to the X arm in the afternoon and checked the status. The HV (KEPCO) was off from the mechanical switch. I don't know this KEPCO has the function to shutdown the switch at the power glitch or not. But anyway the power switch was engaged. We also saw a large amount of misalignment of the X end green. The alignment was manually adjusted. Anchal was able to reach ~0.4 Green TRX, but no more. He claimed that it was ~0.8. We tried to tweak the SHG temp from 36.4. We found that the TRX had the (local) maximum of ~0.48 at 37.1 degC. This is the new setpoint right now. Attachment 1: P_20210915_151333.jpg 16332 Wed Sep 15 11:27:50 2021 YehonathanUpdateCDSc1auxey assembly {Yehonathan, Paco} We turned off the ETMX watchdogs and OpLevs. We went to the X end and shut down the Acromag chassi. We labeled the chassi feedthroughs and disconnected all the cables from it. We took it out and tied the common wire of the power supplies (the commons of the 20V and 15V power supplies were shorted so there is no difference which we connect) to the RTNs of the analog inputs. The chassi was put back in place. All the cables were reconnected. Power turn on. We rebooted c1auxex and the channels went back online. We turned on the watchdogs and watched the ETMX motion get damped. We turned on the OpLev. We waited until the beam position got centered on the ETMX. Attachment shows a comparison between the OSEM spectra before and after the grounding work. Seems like there is no change. We were able to lock the arms with no issues. Attachment 1: c1auxex_Grounding_OSEM_comparison1.pdf Attachment 2: c1auxex_Grounding_OSEM_comparison2.pdf 16331 Tue Sep 14 19:12:03 2021 KojiSummaryPEMExcess seismic noise in 0.1 - 0.3 Hz band Looks like this increase is correlated for BS/EX/EY. So it is likely to be real. Comparison between 9/15 (UTC) (Attachment 1) and 9/10 (UTC) (Attachment 2) Attachment 1: C1-ALL_393F21_SPECTRUM-1315699218-86400.png Attachment 2: C1-ALL_393F21_SPECTRUM-1315267218-86400.png 16330 Tue Sep 14 17:22:21 2021 AnchalUpdateCDSAdded temp sensor channels to DAQ list [Tega, Paco, Anchal] We attempted to reboot fb1 daqd today to get the new temperature sensor channels recording. However, the FE models got stuck, apparantely due to reasons explaine din 40m/16325. Jamie cleared the /var/logs in fb1 so that FE can reboot. We were able to reboot the FE machines after this work successfully and get the models running too. During the day, the FE machines were shut down manually and brought back on manually, a couple of times on the c1iscex machine. Only change in fb1 is in the /opt/rtcds/caltech/c1/chans/daq/C0EDCU.ini where the new channels were added, and some hacking was done by Jamie in gpstime module (See 40m/16327). 16329 Tue Sep 14 17:19:38 2021 PacoSummaryPEMExcess seismic noise in 0.1 - 0.3 Hz band For the past couple of days the 0.1 to 0.3 Hz RMS seismic noise along BS-X has increased. Attachment 1 shows the hour trend in the last ~ 10 days. We'll keep monitoring it, but one thing to note is how uncorrelated it seems to be from other frequency bands. The vertical axis in the plot is in um / s Attachment 1: SEIS_2021-09-14_17-33-12.png 16328 Tue Sep 14 17:14:46 2021 KojiUpdateSUSSOS Tower Hardware Yup this is OK. No problem. 16327 Tue Sep 14 16:44:54 2021 jamieFrogsCDSfb1 /var full after reboot, caused all sorts of problems Jonathan Hanks pointed me to this fix to the gpstime kernel module that was unfortunately put in after the 3.4 release that we're currently using: https://git.ligo.org/cds/advligorts/-/commit/6f6d6e2eb1d3355d0cbfe9fe31ea3b59af1e7348 I hacked the source in place (/usr/src/gpstime-3.4/drv/gpstime/gpstime.c) to get the fix, and then rebuilt the kernel module with dkms : sudo dkms uninstall gpstime/3.4 sudo dkms install gpstime/3.4 I then stopped daqd_dc, unloaded gpstime, reloaded it, restarted daqd_dc. The messages are no longer showing up in /var/log/messages, so I think we're ok for the moment. NOTE: the fix will be undone if we for some reason reinstall the advligorts-gpstime-dkms package. There shouldn't be a need to do that, but we should be aware. I'm discussing with Jonathan if we want to try to push out a new debian package to fix this issue... 16326 Tue Sep 14 16:12:03 2021 JordanUpdateSUSSOS Tower Hardware Yehonathan noticed today that the silver plated hardware on the assembled SOS towers had some pretty severe discoloration on it. See attached picture. These were all brand new screws from UC components, and have been sitting on the flow bench for a couple months now. I believe this is just oxidation and is not an issue, I spoke to Calum as well and showed him the attached picture and he agreed it was likely oxidation and should not be a problem once installed. He did mention if there is any concern from anyone, we could take an FTIR sample and send it to JPL for analysis, but this would cost a few hundred dollars. I don't believe this to be an issue, but it is odd that they oxidized so quickly. Just wanted to relay this to everyone else to see if there was any concern. Attachment 1: 20210914_160111.jpg 16325 Tue Sep 14 15:57:05 2021 jamieFrogsCDSfb1 /var full after reboot, caused all sorts of problems /var on fb1 filled up today, which caused all sorts of CDS issues. I found out about the problem by reading the logs of the services that were having trouble running, in which they complained about not being able to write to disk. I looked at the filesystem status with 'df' and noticed that /var was full, which is where applications write temporary data, and will always cause problems if it's full. I tracked the issue down to multiple multi-gigabyte log files: /var/log/messages and /var/log/messages.1. They were full of lines like this one: Aug 29 06:25:21 fb1 kernel: l called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl ca Seems like something related to the gpstime kernel module? Anyway, I deleted the log files for now, which cleared up the space on /var. Things should be back to normal now, until the logs fill up again... 16324 Mon Sep 13 18:19:25 2021 TegaUpdateComputer Scripts / ProgramsMoved modbus service from chiara to c1susaux [Tega, Anchal, Paco] After talking to Anchal, it was made clear that chiara is not the place to host the modbus service for the temperature sensors. The obvious machine is c1pem, but the startup cmd script loads c object files and it is not clear how easy it would integrate the modbus functionality since we can only login via telnet, so we decided to instead host the service on c1susaux. We also modified the /etc/motd file on c1susaucx which displays the welcome message during login to inform the user that this machine hosts the modbus service for the temperature sensor. Anchal plans to also document this information on the temperature sensor wiki at some point in the future when the page is updated to include what has been learnt so far. We might also consider updating the database file to a more modern way of reading the temperature sensor data using FLOAT32_LE which is available on EPICs version 3.14 and above, instead of the current method which works but leaves the reader bemused by the bitwise operations that convert the two 16 bits words (A and B) to IEEE-754 32-bit float, via field(CALC, "(A&D?(A&C?-1:1):0)*((G|A&E)*J+B)*2^((A&D)/G-F)")  where  field(INPA, "HiWord")
field(INPB, "$LoWord") field(INPC, "0x8000") # Hi word, sign bit field(INPD, "0x7F80") # Hi word, exponent mask field(INPE, "0x00FF") # Hi word, mantissa mask (incl hidden bit) field(INPF, "150") # Exponent offset plus 23-bit mantissa shift field(INPG, "0x0080") # Mantissa hidden bit field(INPJ, "65536") # Hi/Lo mantissa ratio field(CALC, "(A&D?(A&C?-1:1):0)*((G|A&E)*J+B)*2^((A&D)/G-F)") field(PREC, "4") as opposed to the more modern form field(INP,"@asyn($(PORT) \$(OFFSET))FLOAT32_LE")
16323   Mon Sep 13 17:05:04 2021 TegaSummaryPEMInfrasensing temperature sensor modbus configuration

Anchal mentioned it would be good to put more details about how I arrived at the values needed to configure the modbus drive for the temperature sensor, since this information is not in the manual and is hard to find on the internet, so here is a breakdown.

So the generic format is:

drvAsynIPPortConfigure("<TCP_PORT_NAME>","<UNIT_IP_ADDRESS>:502",0,0,1)
modbusInterposeConfig("<TCP_PORT_NAME>",0,5000,0)
drvModbusAsynConfigure("<PORT_NAME>","<TCP_PORT_NAME>",<slaveAddress>,<modbusFunction>,<modbusStartAddress>,<modbusLength>,<dataType>,<pollMsec>,<plcType>)

which in our case become:

drvAsynIPPortConfigure("c1pemxendtemp","192.168.113.240:502",0,0,1)
modbusInterposeConfig("c1pemxendtemp",0,5000,0)
drvModbusAsynConfigure("C1PEMXENDTEMP","c1pemxendtemp",0,4,199,2,0,1000,"ServerCheck")

As can be seen, the parameters of the first two functions "drvAsynIPPortConfigure" and "modbusInterposeConfig" are straight forward, so we restrict our discussion to the case of third function "drvModbusAsynConfigure". Well, after hours of trolling the internet, I was able to piece together a coherent picture of what needs doing and I have summarised them in the table below.

### drvModbusAsynConfigure

Once the asyn IP or serial port driver has been created, and the modbusInterpose driver has been configured, a modbus port driver is created with the following command:

drvModbusAsynConfigure(portName,                # used by channel definitions in .db file to reference this unit)
tcpPortName,             # reference to portName created with drvAsynIPPortConfigure command
modbusFunction,          #
modbusLength,            # length in dataType units
dataType,                #
pollMsec,                # how frequently to request a value in [ms]
plcType);                #


drvModbusAsynConfigure command
Parameter Data type Description
portName string Name of the modbus port to be created.

tcpPortName string Name of the asyn IP or serial port previously created.

tcpPortName = { 192.168.113.240:502192.168.113.241:502192.168.113.242:502 }

slaveAddress int The address of the Modbus slave. This must match the configuration of the Modbus slave (PLC) for RTU and ASCII. For TCP the slave address is used for the "unit identifier", the last field in the MBAP header. The "unit identifier" is ignored by most PLCs, but may be required by some.

ServersCheck API ignores this value, as confirmed with pymodbus query, so set to default value:

modbusFunction int

### modbus supports the following 8 Modbus function codes:

Modbus Function Codes
Access Function description Function code
Bit access Read Discrete Inputs 2
Bit access Write Single Coil 5
Bit access Write Multiple Coils 15
16-bit word access Read Input Registers 4
16-bit word access Read Holding Registers 3
16-bit word access Write Single Register 6
16-bit word access Write Multiple Registers 16
modbusStartAddress int Start address for the Modbus data segment to be accessed.
(0-65535 decimal, 0-0177777 octal).

Modbus addresses are specified by a 16-bit integer address. The location of inputs and outputs within the 16-bit address space is not defined by the Modbus protocol, it is vendor-specific. Note that 16-bit Modbus addresses are commonly specified with an offset of 400001 (or 300001). This offset is not used by the modbus driver, it uses only the 16-bit address, not the offset.

For ServersCheck, the offset is "30001", so that

modbusStartAddress = 30200 - 30001 = 199

modbusLength int The length of the Modbus data segment to be accessed.
This is specified in bits for Modbus functions 1, 2, 5 and 15.
It is specified in 16-bit words for Modbus functions 3, 4, 6 and 16.
Length limit is 2000 for functions 1 and 2, 1968 for functions 5 and 15,
125 for functions 3 and 4, and 123 for functions 6 and 16.

ServersCheck uses two's complement 32-bits word (with big-endian byte order & little-endian word order) format to store floating-point data, as confirmed with pymodbus query, so that:

modbusLength = 2

modbusDataType int The modbusDataType is used to tell the driver the format of the Modbus data. The driver uses this information to convert the number between EPICS and Modbus. Data is transferred to and from EPICS as epicsUInt32, epicsInt32, and epicsFloat64 numbers.

Modbus data type:
0 = binary, twos-complement format
1 = binary, sign and magnitude format
2 = BCD, unsigned
3 = BCD, signed

Some Modbus devices (including ServersCheck) use floating point numbers, typically by storing a 32-bit float in two consecutive 16-bit registers. This is not supported by the Modbus specification, which only supports 16-bit registers and single-bit data. The modbus driver does not directly support reading such values, because the word order and floating point format is not specified.

Note that if it is desired to transmit BCD numbers untranslated to EPICS over the asynInt32 interface, then data type 0 should be used, because no translation is done in this case.

For ServersCheck, we wish to transmit the untranslated data, so:

modbusDataType = 0

pollMsec int Polling delay time in msec for the polling thread for read functions.
For write functions, a non-zero value means that the Modbus data should
be read once when the port driver is first created.

ServersCheck recommends setting sensor polling interval between 1-5 seconds, so we can try:

pollMsec = 1000

plcType string Type of PLC (e.g. Koyo, Modicon, etc.).
This parameter is currently used only to print information in asynReport.
In the future it could be used to modify the driver behavior for a specific PLC.

plcType = "ServersCheck"

https://nodus.ligo.caltech.edu:8081/40m/16214

https://nodus.ligo.caltech.edu:8081/40m/16269

https://nodus.ligo.caltech.edu:8081/40m/16270

https://nodus.ligo.caltech.edu:8081/40m/16274

http://manuals.serverscheck.com/InfraSensing_Sensors_Platform.pdf

http://manuals.serverscheck.com/InfraSensing_Modbus_manualv5.pdf

https://community.serverscheck.com/discussion/comment/7419#Comment_7419

https://wiki-40m.ligo.caltech.edu/CDS/SlowControls

https://www.slac.stanford.edu/grp/ssrl/spear/epics/site/modbus/modbusDoc.html#Creating_a_modbus_port_driver

https://github.com/riptideio/pymodbus

https://en.wikipedia.org/wiki/Modbus

https://deltamotion.com/support/webhelp/rmctools/Communications/Ethernet/Supported_Protocols/Ethernet_Modbus_TCP.htm

16322   Mon Sep 13 15:14:36 2021 AnchalUpdateLSCXend Green laser injection mirrors M1 and M2 not responsive

I was showing some green laser locking to Tega, I noticed that changing the PZT sliders of M1/M2 angular position on Xend had no effect on locked TEM01 or TEM00 mode. This is odd as changing these sliders should increase or decrease the mode-matching of these modes. I suspect that the controls are not working correctly and the PZTs are either not powered up or not connected. We'll investigate this in near future as per priority.

ELOG V3.1.3-