40m QIL Cryo_Lab CTN SUS_Lab TCS_Lab OMC_Lab CRIME_Lab FEA ENG_Labs OptContFac Mariner WBEEShop
 40m Log, Page 243 of 344 Not logged in
ID Date Author Type Category Subject
2216   Mon Nov 9 15:08:29 2009 KojiOmnistructureEnvironmentTidying up BNC cables rack around the lab

Quote:

This would be a good trial once you put the label "BNC only" on the wall.

 Quote: We have thousands of miles of BNC cables in the lab but we still don't find one when we need it. I decided to solve the problem. This morning I tried to tidy up the several cable rack the we have in the lab. i tried to dedicate each rack to a speecific type of cables: special cables, hand made cables, BNCs, LEMOs, etc. In particular I tryed to concentrate BNC cable of several lengths on the rack near by the ITMX chamber. People are invited to preserve the organization.

Done! Check it out.

11392   Tue Jul 7 17:22:16 2015 JessicaSummary Time Delay in ALS Cables

I measured the transfer functions in the delay line cables, and then calculated the time delay from that.

The first cable had a time delay of 1272 ns and the second had a time delay of 1264 ns.

Below are the plots I created to calculate this. There does seem to be a pattern in the residual plots however, which was not expected.

The R-Square parameter was very close to 1 for both fits, indicating that the fit was good.

Attachment 1: cableA_fit.jpg
Attachment 2: cableA_resid.jpg
Attachment 3: cableB_fit.jpg
Attachment 4: cableB_resid.jpg
10265   Wed Jul 23 18:53:11 2014 NichinUpdateElectronicsTime delay in RG405 coaxial cables

A time delay can be modeled as the exponential transfer function :  e(-sTd)  as seen HERE . Therefore the slope of the phase gives us the time delay.

A RG405 coaxial cable, exactly 5.5 meters in length, was fit to an ideal delay function e(-sTd) , with Td = 150 ns.

The plots shows the actual data, fit data and data after correction using the ideal model stated above.

Conclusion:

Delay in RG405 cables is approximately 27.27 ns per meter. This value can be used to correct the phase in measurements of transimpedance for each PD by dividing out the ideal transfer function for time delay.

[EDIT: This looks like we have about 12 % the speed of light inside the RF cables. Too small to be true. I will check tomorrow if the Network analyzer itself has some delay and update this value.]

The varying attenuation of about 1dB due to the cable is not compensated by this. We need to separately include this.

Things to do:

1) Get the length of RF cables that is being used by each PD, so that the compensation can be made.

2) Calculate the attenuation and delay caused by RF multiplexer and Demodulator boards. Include these in the correction factor for transimpedance measurements.

Attachment 1: RFcable1.pdf
Attachment 2: RFcable2.pdf
10266   Wed Jul 23 19:30:34 2014 NichinUpdateElectronicsTime delay in the RF multiplexer (Rack 1Y1)

A time delay can be modeled as the exponential transfer function :  e(-sTd)  as seen HERE . Therefore the slope of the phase gives us the time delay.

The transfer function of RF multiplexer in rack 1Y1 (NI PXI-2547) was fit to an ideal delay function e(-sTd) , with Td = 59 ns.

The plots shows the actual data, fit data and data after correction using the ideal model stated above.

Conclusion:

Delay the RF Multiplexer is approximately 59 ns. This value can be used to correct the phase in measurements of transimpedance for each PD by dividing out the ideal transfer function for time delay.

Attachment 1: RFmux1.pdf
Attachment 2: RFmux2.pdf
1031   Tue Oct 7 12:17:57 2008 AlbertoConfigurationComputersTime reset on MEDM
Yoichi, Alberto

I noticed the MEDM screen time was about 7 minutes ahead of the right time. The time on MEDM is read on channel C0:TIM-PACIFIC_STRING which takes it from the C1VCU-EPICS computer. Yoichi found that that computer did not have the right time because one of the startup scripts, ntpd, which are contained in the directory /etc/init.d/ for some reason did not start. So restring it by typing ./ntpd start updated the time on that computer and fixed the problem.
16283   Thu Aug 19 03:23:00 2021 AnchalUpdateCDSTime synchornization not running

I tried to read a bit and understand the NTP synchronization implementation in FE computers. I'm quite sure that NTP synchronization should be 'yes' if timesyncd are running correctly in the output of timedatectl in these computers. As Koji reported in 15791, this is not the case. I logged into c1lsc, c1sus and c1ioo and saw that RTC has drifted from the software clocks too which does not happen if NTP synchronization was active. This would mean that almost certainly, if the computers are rebooted, the synchronization will be lost and the models will fail to come online.

My current findings are the following (this should be documented in wiki once we setup everything):

• nodus is running a NTP server using chronyd. One can check the configuration of this NTP serer in /etc/chornyd.conf
• fb1 is running an NTP server using ntpd that follows nodus and an IP address 131.215.239.14. This can be seen in /etc/ntp.conf.
• There are no comments to describe what this other server (131.215.239.14) is. Does the GC network have an NTP server too?
• c1lsc, c1sus and c1ioo all have systemd-timesyncd.service running with configuration file in /etc/systemd/timesyncd.conf.
• The configuration file set Servers=ntpserver but echo $ntpserver produces nothing (blank) on these computers and I've been unable to find anyplace where ntpserver is defined. • In chiara (our name server), the name server file /etc/hosts does not have any entry for ntpserver either. • I think the problem might be that these computers are unable to find the ntpserver as it is not defined anywhere. The solution to this issue could be as simple as just defining ntpserver in the name server list. But I'm not sure if my understanding of this issue is correct. Comments/suggestions are welcome for future steps. 16284 Thu Aug 19 14:14:49 2021 KojiUpdateCDSTime synchornization not running 131.215.239.14 looks like Caltech's NTP server (ntp-02.caltech.edu) https://webmagellan.com/explore/caltech.edu/28415b58-837f-4b46-a134-54f4b81bee53 I can't say it is correct or not as I did not make the survey at your level. I think you need a few tests of reconfiguring and restarting the NTP clients to see if time synchronization starts. Because the local time is not regulated right now anyway, this operation is safe I think. 16285 Fri Aug 20 00:28:55 2021 AnchalUpdateCDSTime synchornization not running I added ntpserver as a known host name for address 192.168.113.201 (fb1's address where ntp server is running) in the martian host list in the following files in Chiara: /var/lib/bind/martian.hosts /var/lib/bind/rev.113.168.192.in-addr.arpa Note: a host name called ntp was already defined at 192.168.113.11 but I don't know what computer this is. Then, I restarted the DNS on chiara by doing: sudo service bind9 restart Then I logged into c1lsc and c1ioo and ran following: controls@c1ioo:~ 0$ sudo systemctl restart systemd-timesyncd.service

controls@c1ioo:~ 0$sudo systemctl status systemd-timesyncd.service -l ● systemd-timesyncd.service - Network Time Synchronization Loaded: loaded (/lib/systemd/system/systemd-timesyncd.service; enabled) Active: active (running) since Fri 2021-08-20 07:24:03 UTC; 53s ago Docs: man:systemd-timesyncd.service(8) Main PID: 23965 (systemd-timesyn) Status: "Idle." CGroup: /system.slice/systemd-timesyncd.service └─23965 /lib/systemd/systemd-timesyncd Aug 20 07:24:03 c1ioo systemd[1]: Starting Network Time Synchronization... Aug 20 07:24:03 c1ioo systemd[1]: Started Network Time Synchronization. Aug 20 07:24:03 c1ioo systemd-timesyncd[23965]: Using NTP server 192.168.113.201:123 (ntpserver). Aug 20 07:24:35 c1ioo systemd-timesyncd[23965]: Using NTP server 192.168.113.201:123 (ntpserver). controls@c1ioo:~ 0$ timedatectl
Local time: Fri 2021-08-20 07:25:28 UTC
Universal time: Fri 2021-08-20 07:25:28 UTC
RTC time: Fri 2021-08-20 07:25:31
Time zone: Etc/UTC (UTC, +0000)
NTP enabled: yes
NTP synchronized: no
RTC in local TZ: no
DST active: n/a

The same output is shown in c1lsc too. The NTP synchronized flag in output of timedatectl command did not change to yes and the RTC is still 3 seconds ahead of the local clock.

Then I went to c1sus to see what was the status output before rstarting the timesyncd service. I got folloing output:

controls@c1sus:~ 0$sudo systemctl status systemd-timesyncd.service -l ● systemd-timesyncd.service - Network Time Synchronization Loaded: loaded (/lib/systemd/system/systemd-timesyncd.service; enabled) Active: active (running) since Tue 2021-08-17 04:38:03 UTC; 3 days ago Docs: man:systemd-timesyncd.service(8) Main PID: 243 (systemd-timesyn) Status: "Idle." CGroup: /system.slice/systemd-timesyncd.service └─243 /lib/systemd/systemd-timesyncd Aug 20 02:02:18 c1sus systemd-timesyncd[243]: Using NTP server 192.168.113.201:123 (ntpserver). Aug 20 02:36:27 c1sus systemd-timesyncd[243]: Using NTP server 192.168.113.201:123 (ntpserver). Aug 20 03:10:35 c1sus systemd-timesyncd[243]: Using NTP server 192.168.113.201:123 (ntpserver). Aug 20 03:44:43 c1sus systemd-timesyncd[243]: Using NTP server 192.168.113.201:123 (ntpserver). Aug 20 04:18:51 c1sus systemd-timesyncd[243]: Using NTP server 192.168.113.201:123 (ntpserver). Aug 20 04:53:00 c1sus systemd-timesyncd[243]: Using NTP server 192.168.113.201:123 (ntpserver). Aug 20 05:27:08 c1sus systemd-timesyncd[243]: Using NTP server 192.168.113.201:123 (ntpserver). Aug 20 06:01:16 c1sus systemd-timesyncd[243]: Using NTP server 192.168.113.201:123 (ntpserver). Aug 20 06:35:24 c1sus systemd-timesyncd[243]: Using NTP server 192.168.113.201:123 (ntpserver). Aug 20 07:09:33 c1sus systemd-timesyncd[243]: Using NTP server 192.168.113.201:123 (ntpserver). This actually shows that the service was able to find ntpserver correctly at 192.168.113.201 even before I changed the name server file in chiara. So I'm retracting the changes made to name server. They are probably not required. The configuration files for timesynd.conf are read only even with sudo. I tried changing permissions but that did not work either. Maybe these files are not correctly configured. The man page of timesyncd says to use field 'NTP' to give the ntp servers. Our files are using field 'Servers'. But since we are not getting any error message, I don't think this is the issue here. I'll look more into this problem. 16286 Fri Aug 20 06:24:18 2021 AnchalUpdateCDSTime synchornization not running I read on some stack exchange that 'NTP synchornized' indicator turns 'yes' in the output of command timedatectl only when RTC clock has been adjusted at some point. I also read that timesyncd does not do the change if the time difference is too much, roughly more than 3 seconds. So I logged into all FE machines and ran sudo hwclock -w to synchronize them all to the system clocks and then waited if the timesyncd does any correction on RTC. It did not. A few hours later, I found the RTC clocks drifitng again from the system clocks. So even if the timesynd service is running as it should, it si not performing time correction for whatever reason. Maybe we should try to use some other service?  Quote: The NTP synchronized flag in output of timedatectl command did not change to yes and the RTC is still 3 seconds ahead of the local clock. 16291 Mon Aug 23 22:51:44 2021 AnchalUpdateGeneralTime synchronization efforts Related elog thread: 16286 I didn't really achieve anything but I'm listing what I've tried. • I know now that the timesyncd isn't working because systemd-timesyncd is known to have issues when running on a read-only file system. In particular, the service does not have privileges to change the clock or drift settings at /run/systemd/clock or /etc/adjtime. • The workarounds to these problems are poorly rated/reviews in stack exchange and require me to change the /etc/systmd/timesyncd.conf file but I'm unable to edit this file. • I know that Paco was able to change these files earlier as the files are now changed and configured to follow a debian ntp pool server which won't work as the FEs do not have internet access. So the conf file needs to be restored to using ntpserver as the ntp server. • From system messages, the ntpserver is recognized by the service as shown in the second part of 16285. I really think the issue is in file permissions. the file /etc/adjtime has never been updated since 2017. • I got help from Paco on how to edit files for FE machines. The FE machines directories are exported from fb1:/diskless/root.jessie/ • I restored the /etc/systmd/timesyncd.conf file to how it as before with just servers=ntpserver line. Restarted timesyncd service on all FEs,I tried a few su the synchronization did not happen. • I tried a few suggestions from stackexchange but none of them worked. The only rated solution creates a tmpfs directory outside of read-only filesystem and uses that to run timesyncd. So, in my opinion, timesyncd would never work in our diskless read-only file system FE machines. • One issue in an archlinux discussion ended by the questioner resorting to use opennptd from openBSD distribution. The user claimed that opennptd is simple enough that it can run ntp synchornization on a read-only file system. • Somehwat painfully, I 'kind of' installed the openntpd tool in the fb1:/diskless/root.jessie directory following directions from here. I had to manually add user group and group for the FEs (which I might not have done correctly). I was not able to get the openntpd daemon to start properly after soe tries. • I restored everything back to how it was and restarted timesyncd in c1sus even though it would not do anything really.  Quote: This time no matter how we try to set the time, the IOPs do not run with "DC status" green. (We kept having 0x4000) 16293 Tue Aug 24 18:11:27 2021 PacoUpdateGeneralTime synchronization not really working tl;dr: NTP servers and clients were never synchronized, are not synchronizing even with ntp... nodus is synchronized but uses chronyd; should we use chronyd everywhere? Spent some time investigating the ntp synchronization. In the morning, after Anchal set up all the ntp servers / FE clients I tried restarting the rts IOPs with no success. Later, with Tega we tried the usual manual matching of the date between c1iscex and fb1 machines but we iterated over different n-second offsets from -10 to +10, also without success. This afternoon, I tried debugging the FE and fb1 timing differences. For this I inspected the ntp configuration file under /etc/ntp.conf in both the fb1 and /diskless/root.jessie/etc/ntp.conf (for the FE machines) and tried different combinations with and without nodus, with and without restrict lines, all while looking at the output of sudo journalctl -f on c1iscey. Everytime I changed the ntp config file, I restarted the service using sudo systemctl restart ntp.service . Looking through some online forums, people suggested basic pinging to see if the ntp servers were up (and broadcasting their times over the local network) but this failed to run (read-only filesystem) so I went into fb1, and ran sudo chroot /diskless/root.jessie/ /bin/bash to allow me to change file permissions. The test was first done with /bin/ping which couldn't even open a socket (root access needed) by running chmod 4755 /bin/ping then ssh-ing into c1iscey and pinging the fb1 machine successfully. After this, I ran chmod 4755 /usr/sbin/ntpd so that the ntp daemon would have no problem in reaching the server in case this was blocking the synchronization. I exited the chroot shell and the ntp daemon in c1iscey; but the ntpstat still showed unsynchronised status. I also learned that when running an ntp query with ntpq -p if a client has succeeded in synchronizing its time to the server time, an asterisk should be appended at the end. This was not the case in any FE machine... and looking at fb1, this was also not true. Although the fb1 peers are correctly listed as nodus, the caltech ntp server, and a broadcast (.BCST.) server from local time (meant to serve the FE machines), none appears to have synchronized... Going one level further, in nodus I checked the time synchronization servers by running chronyc sources the output shows controls@nodus|~> chronyc sources 210 Number of sources = 4 MS Name/IP address Stratum Poll Reach LastRx Last sample =============================================================================== ^* testntp1.superonline.net 1 10 377 280 +1511us[+1403us] +/- 92ms ^+ 38.229.59.9 2 10 377 206 +8219us[+8219us] +/- 117ms ^+ tms04.deltatelesystems.ru 2 10 377 23m -17ms[ -17ms] +/- 183ms ^+ ntp.gnc.am 3 10 377 914 -8294us[-8401us] +/- 168ms I then ran chronyc clients to find if fb1 was listed (as I would have expected) but the output shows this -- Hostname Client Peer CmdAuth CmdNorm CmdBad LstN LstC ========================= ====== ====== ====== ====== ====== ==== ==== 501 Not authorised So clearly chronyd succeeded in synchronizing nodus' time to whatever server it was pointed at but downstream from there, neither the fb1 or any FE machines seem to be synchronizing properly. It may be as simple as figuring out the correct ntp configuration file, or switching to chronyd for all machines (for the sake of homogeneity?) 16295 Tue Aug 24 22:37:40 2021 AnchalUpdateGeneralTime synchronization not really working I attempted to install chrony and run it on one of the FE machines. It didn't work and in doing so, I lost the working NTP client service on the FE computers as well. Following are some details: • I added the following two mirrors in the apt source list of root.jessie at /etc/apt/sources.list deb http://ftp.us.debian.org/debian/ jessie main contrib non-free deb-src http://ftp.us.debian.org/debian/ jessie main contrib non-free • Then I installed chrony in the root.jessie using sudo apt-get install chrony • I was getting an error E: Can not write log (Is /dev/pts mounted?) - posix_openpt (2: No such file or directory) . To fix this, I had to run: sudo mount -t devpts none "$rootpath/dev/pts" -o ptmxmode=0666,newinstance
sudo ln -fs "pts/ptmx" "$rootpath/dev/ptmx" • Then, I had another error to resolve. Failed to read /proc/cmdline. Ignoring: No such file or directory start-stop-daemon: nothing in /proc - not mounted? To fix this, I had to exit to fb1 and run: sudo mount --bind /proc /diskless/root.jessie/proc • With these steps, chrony was finally installed, but I immediately saw an error message saying: Starting /usr/sbin/chronyd... Could not open NTP sockets • I figured this must be due to ntp running in the FE machines. I logged into c1iscex and stopped and disabled the ntp service: sudo systemctl stop ntp sudo systemctl disable ntp • I saw some error messages from the above coomand as FEs are read only file systems: Synchronizing state for ntp.service with sysvinit using update-rc.d... Executing /usr/sbin/update-rc.d ntp defaults insserv: fopen(.depend.stop): Read-only file system Executing /usr/sbin/update-rc.d ntp disable update-rc.d: error: Read-only file system • So I went back to chroot in fb1 and ran the two command sabove that failed: /usr/sbin/update-rc.d ntp defaults /usr/sbin/update-rc.d ntp disable • The last line gave the output: insserv: warning: current start runlevel(s) (empty) of script ntp' overrides LSB defaults (2 3 4 5). insserv: warning: current stop runlevel(s) (2 3 4 5) of script ntp' overrides LSB defaults (empty). • I igored this and moved forward. • I copied the chronyd.service from nodus to the chroot in fb1 and configured it to use nodus as the server. The I started the chronyd.service sudo systemctl status chronyd.service but got the saem issue of NTP sockets. â—Â chronyd.service - NTP client/server Loaded: loaded (/usr/lib/systemd/system/chronyd.service; disabled) Active: failed (Result: exit-code) since Tue 2021-08-24 21:52:30 PDT; 5s ago Process: 790 ExecStart=/usr/sbin/chronyd$OPTIONS (code=exited, status=1/FAILURE)

Aug 24 21:52:29 c1iscex systemd[1]: Starting NTP client/server...
Aug 24 21:52:30 c1iscex chronyd[790]: Could not open NTP sockets
Aug 24 21:52:30 c1iscex systemd[1]: chronyd.service: control process exited, code=exited status=1
Aug 24 21:52:30 c1iscex systemd[1]: Failed to start NTP client/server.
Aug 24 21:52:30 c1iscex systemd[1]: Unit chronyd.service entered failed state.

• I tried a few things to resolve this, but couldn't get it to work. So I gave up on using chrony and decided to go back to ntp service atleast.

• I stopped, disabled and checked status of chrony:
sudo systemctl stop chronyd
sudo systemctl disable chronyd
sudo systemctl status chronyd
This gave the output:

â—Â chronyd.service - NTP client/server
Active: failed (Result: exit-code) since Tue 2021-08-24 22:09:07 PDT; 25s ago

Aug 24 22:09:07 c1iscex systemd[1]: Starting NTP client/server...
Aug 24 22:09:07 c1iscex chronyd[2490]: Could not open NTP sockets
Aug 24 22:09:07 c1iscex systemd[1]: chronyd.service: control process exited, code=exited status=1
Aug 24 22:09:07 c1iscex systemd[1]: Failed to start NTP client/server.
Aug 24 22:09:07 c1iscex systemd[1]: Unit chronyd.service entered failed state.
Aug 24 22:09:15 c1iscex systemd[1]: Stopped NTP client/server.

• I went back to fb1 chroot and removed chrony package and deleted the configuration files and systemd service files:
sudo apt-get remove chrony

• But when I started ntp daemon service back in c1iscex, it gave error:
sudo systemctl restart ntp
Job for ntp.service failed. See 'systemctl status ntp.service' and 'journalctl -xn' for details.

• Status shows:

sudo systemctl status ntp
â—Â ntp.service - LSB: Start NTP daemon
Active: failed (Result: exit-code) since Tue 2021-08-24 22:09:56 PDT; 9s ago
Process: 2597 ExecStart=/etc/init.d/ntp start (code=exited, status=5)

Aug 24 22:09:55 c1iscex systemd[1]: Starting LSB: Start NTP daemon...
Aug 24 22:09:56 c1iscex systemd[1]: ntp.service: control process exited, code=exited status=5
Aug 24 22:09:56 c1iscex systemd[1]: Failed to start LSB: Start NTP daemon.
Aug 24 22:09:56 c1iscex systemd[1]: Unit ntp.service entered failed state.

• I tried to enable back the ntp service by sudo systemctl enable ntp. I got similar error messages of read only filesystem as earlier.
Synchronizing state for ntp.service with sysvinit using update-rc.d...
Executing /usr/sbin/update-rc.d ntp defaults
insserv: warning: current start runlevel(s) (empty) of script ntp' overrides LSB defaults (2 3 4 5).
insserv: warning: current stop runlevel(s) (2 3 4 5) of script ntp' overrides LSB defaults (empty).
Executing /usr/sbin/update-rc.d ntp enable

• I went back to chroot in fb1 and ran:
/usr/sbin/update-rc.d ntp defaults
insserv: warning: current start runlevel(s) (empty) of script ntp' overrides LSB defaults (2 3 4 5).
insserv: warning: current stop runlevel(s) (2 3 4 5) of script ntp' overrides LSB defaults (empty).
and
/usr/sbin/update-rc.d ntp enable

• I came back to c1iscex and tried restarting the ntp service but got same error messages as above with exit code 5.

• I checked c1sus, the ntp was running there. I tested the configuration by restarting the ntp service, and then it failed with same error message. So the remaining three FEs, c1lsc, c1ioo and c1iscey have running ntp service, but they won't be able to restart.

• As a last try, I rebooted c1iscex to see if ntp comes back online nicely, but it doesn't.

Bottom line, I went to try chrony in the FEs, and I ended up breaking the ntp client services on the computers as well. We have no NTP synchronization in any of the FEs.

Even though Paco and I are learning about the ntp and cds stuff, I think it's time we get help from someone with real experience. The lab is not in a good state for far too long.

 Quote: tl;dr: NTP servers and clients were never synchronized, are not synchronizing even with ntp... nodus is synchronized but uses chronyd; should we use chronyd everywhere?

16292   Tue Aug 24 09:22:48 2021 AnchalUpdateGeneralTime synchronization working now

Jamie told me to use chroot to log in into the chroot jail of debian os that are exported for the FEs and install ntp there. I took following steps at the end of which, all FEs have NTP synchronized now.

• I logged into fb1 through nodus.
• chroot /diskless/root.jessie /bin/bash took me to the bash terminal for debian os that is exported to all FEs.
• Here, I ran sudo apt-get install ntp which ran without any errors.
• I then edited the file in /etc/ntp.conf , i removed the default servers and added following lines for servers (fb1 and nodus ip addresses):
server 192.113.168.201
server 192.113.168.201
• I logged into each FE machine and ran following commands:
sudo systemctl stop systemd-timesyncd.service; sudo systemctl status systemd-timesyncd.service;
timedatectl; sleep 2;sudo systemctl daemon-reload;  sudo systemctl start ntp; sleep 2; sudo systemctl status ntp; timedatectl
sudo hwclock -s
• The first line ensures that systemd-timesyncd.service is not running anymore. I did not uninstall timesyncd and left its configuration file as it is.
• The second line first shows the times of local and RTC clocks. Then reloads the daemon services to get ntp registered. Then starts ntp.service and shows it's status. Finally, the timedatectl command shows the synchronized clocks and that NTP synchronization has occured.
• The last line sets the local clock same as RTC clock. Even though this wasn't required as I saw that the clocks were already same to seconds, I just wanted a point where all the local clocks are synchronized to the ntp server.
• Hopefully, this would resolve our issue of restarting the models anytime some glitch happens or when we need ot update something in one of them.

Edit Tue Aug 24 10:19:11 2021:

I also disabled timesyncd on all FEs using sudo systemctl disable systemd-timesyncd.service

I've added this wiki page for summarizing the NTP synchronization knowledge.

8098   Mon Feb 18 11:54:15 2013 Max HortonUpdateSummary PagesTiming Issues and Calendars

Crontab: The bug of data only plotting until 5PM is being investigated.  The crontab's final call to the summary page generator was at 5PM.  This means that the data plots were not being generated after 5PM, so clearly they never contained data from after 5PM.  In fact, the original crontab reads:

0 11,5,11,17 * * * /users/public_html/40m-summary/bin/c1_summary_page.sh 2>&1

I'm not exactly sure what inspired these entries.  The 11,5,11,17 entries are supposed to be the hours at which the program is run.  Why is it run twice at 11?  I assume it was just a typo or something.

The final call time was changed to 11:59PM in an attempt to plot the entire day's data, but this method didn't appear to work because the program would still be running past midnight, which was apparently inhibiting its functionality (most likely, the day change was affecting how the data is fetched).  The best solution is probably to just wait until the next day, then call the summary page generator on the previous day's data.  This will be implemented soon.

Calendars: Although the calendar tabs on the left side of the page were fixed, the calendars displayed at: https://nodus.ligo.caltech.edu:30889/40m-summary/calendar/ appear to still have squished together text.  The calendar is being fetched from https://nodus.ligo.caltech.edu:30889/40m-summary/calendar/calendar.html and displayed in the page.  This error is peculiar because the URL from which the calendar is being fetched does NOT have squished together text, but the resulting calendar at 40m-summary/calendar/ will not display spaces between the text.  This issue is still being investigated.

10193   Mon Jul 14 13:03:23 2014 AkhilSummaryElectronicsTiming Issues of Mini Circuits UFC-6000: Solved

Main Problem:

The frequency counter (FC) takes in an analog RF input(signal) and outputs the frequency of the signal(Ranging from 1 MHz- 6000 MHz) in the digital domain (into a processor). The FC samples the data with a given sample rate( user defined) which ranges from 0.1 s to 1 s(faced problems in fixing this initially).  For data acquisition, we have been using a Raspberry Pi(as a processor) which is connected to the martian network and can communicate with the computers inside the 40m.  The ultimate challenge which I faced( and been knocking my head off from past two-three weeks) is the synchronization of clocks between the Raspberry Pi and the FC i.e the clock which the FC uses to sample and dump data( every 'x' s) and the clock inside the raspberry pi( used  in the loop to wait for a particular amount of time the frequency counter takes to dump successive data).

Steps Taken:

• To address this problem, first I added an external clock circuit which monitors the Raspberry Pi and the FC to dump and read data at a particular rate(which is equal to the sampling rate of the FC)In detail: http://nodus.ligo.caltech.edu:8080/40m/10129.
• While doing so, at first the level trigger algorithm was used which means that the external clock frequency was half as that of  the reciprocal of the sampling rate and a trigger was seen every time the level shifts from +DC to -DC(of the external square wave).
• But this did not completely mitigate the issues and there were still few issues on how quickly the ADC reads the signal and R Pi processes it.
• To minimize these issues completely, an edge trigger algorithm which detects a pos edge(rising)  of the clock was used. The clock  frequency is now equal to the reciprocal of the sampling rate. This algorithm showed better results and greatly minimized the drift of the sampling time.

Psuedo Code(code attached):

open device : FC via USB-HID;

open device : ADC via I2C;

always(for t= recording time):

if pos edge detected:

read data from FC and store it in a register;

end

write data stored in the register to a file( can be an Epics channel or a text file);

Results:

The attached are the plots showing the time between samples for a large number of samples taken for different sampling times of the FC. The percentage error is the percentage of standard error in the timing between two samples for the data for the entire measurement. It can be inferred that this error has been cut down to the order of ms.

To do next:

• I have started taking phase measurements( analysis and plots will follow this elog) and also the PSD plots with the improved timing characteristics.

Attachment 1: 0.2timinganalysis.png
Attachment 2: 0.3timinganalysis.png
Attachment 3: 0.5timinganalysis.png
Attachment 4: 1stiminganalysis.png
Attachment 5: pdf.zip
10194   Mon Jul 14 14:28:27 2014 KojiSummaryElectronicsTiming Issues of Mini Circuits UFC-6000: Solved

Looks good. Now you have the internal timer to verify the external clock.
If you can realize the constant rate sampling without employing the external clock, that's quite handy.

10199   Tue Jul 15 01:31:13 2014 AkhilSummaryElectronicsTiming Issues of Mini Circuits UFC-6000: Solved

The attached are the PSD plots with improved FC timing(with the same code as in http://nodus.ligo.caltech.edu:8080/40m/10151). More plots(Phase and PSD) to follow.

Attachment 1: qnoise.png
Attachment 2: qnoise.pdf
1581   Wed May 13 12:41:14 2009 josephbUpdateCamerasTiming and stability tests of GigE Camera code

At the request of people down at LLO I've been trying to work on the reliability and speed of the GigE camera code.  In my testing, after several hours, the code would tend to lock up on the camera end.  It was also reported at LLO after several minutes the camera display would slow down, but I haven't been able to replicate that problem.

I've recently added some additional error checking and have updated to a more recent SDK which seems to help.  Attached are two plots of the frames per second of the code.  In this case, the frames per second  are measured as the time between calls to the C camera code for a new frame for gstreamer to encode and transmit.  The data points in the first graph are actually the averaged time for sets of 1000 frames.  The camera was sending 640x480 pixel frames, with an exposure time of 0.01 seconds.  Since the FPS was mostly between 45 and 55, it is taking the code roughly 0.01 second to process, encode, and transmit a frame.

During the test, the memory usage by the server code was roughly 1% (or 40 megabytes out of 4 gigabytes) and 50% of a CPU (out a total of  CPUs).

Attachment 1: newCodeFPS.png
Attachment 2: newCodeFPS_hist.png
15515   Wed Aug 12 17:36:42 2020 gautamUpdateCDSTiming distribution slot availability

See Attachment #1. J8 was connected to a "LASTI timing slave" sitting in the rack that Chiara lives in - we don't use this for anything and I confirmed that there was no effect on the RTCDS when I pulled that fiber out. The LASTI timing slave also had a blinky that was blinking when the fiber was plugged in - which I take to believe that the slot works.

Can we get away with just using these two available slots, J8 and J13? Do we really need three new expansion chassis?

Attachment 1: IMG_8706.JPG
15518   Wed Aug 12 20:14:06 2020 KojiUpdateCDSTiming distribution slot availability

I believe we will use two new chassis at most. We'll replace c1ioo from Sun to Supermicro, but we recycle the existing timing system.

15521   Thu Aug 13 11:30:19 2020 gautamUpdateCDSTiming distribution slot availability

That's great. I wonder if we can also get away with not adding new Dolphin infrastructure. I'd really like to avoid changing any IPC drivers.

 Quote: I believe we will use two new chassis at most. We'll replace c1ioo from Sun to Supermicro, but we recycle the existing timing system.
15522   Thu Aug 13 13:35:13 2020 KojiUpdateCDSTiming distribution slot availability

The new dolphin eventually helps us. But the installation is an invasive change to the existing system and should be done at the installation stage of the 40m BHD.

15525   Fri Aug 14 10:03:37 2020 JonUpdateCDSTiming distribution slot availability

That's great news we won't have to worry about a new timing fanout for the two new machines, c1bhd and c1sus2. And there's no plan to change Dolphin IPC drivers. The plan is only to install the same (older) version of the driver on the two new machines and plug into free slots in the existing switch.

 Quote: The new dolphin eventually helps us. But the installation is an invasive change to the existing system and should be done at the installation stage of the 40m BHD.
4004   Wed Dec 1 13:41:21 2010 josephb, alex, rolfUpdateCDSTiming is back

## Problem:

We had timing problems across the front ends.

## Solution:

Noticing that the 1PPS reference was not blinking on the Master Timing Distribution box.  It was supposed to be getting a signal from the c0dcu1 VME crate computer, but this was not happening.

We disconnected the timing signal going into c0dcu1, coming from c0daqctrl, and connected the 1PPS directly from c0daqctrl to the Ref In for the Master Timing distribution box (blue box with lots of fibers coming out of it in 1X5).

We now have agreement in timing between front ends.

After several reboots we now have working RFM again, along with computers who agree on the current GPS time along with the frame builder.

## Status:

RFM is back and testpoints should be happy.

We still don't have a working binary output for the X end.  I may need to get a replacement backplane with more than 4 slots if the 1st slot of this board has the same problem as the large boards.

I have burt restored the c1ioo, c1mcs, c1rms, c1sus, and c1scx processes, and optics look to be damped.

14386   Fri Jan 4 17:43:24 2019 gautamUpdateCDSTiming issues

[J Hanks (remote), koji, gautam]

Summary:

The problem stems from the way GPS timing signals are handled by the FEs and FB. We effected a partial fix:

• Now, old frame data is no longer being overwritten
• For the channels that are indeed being recorded now, the correct time stamp is being applied so they can be found in /frames by looking for the appropriate gpstime.

Details:

• The usual FE/FB power cycling did not fix the problem.
• The gps time used by FB and associated RT processes may be found by using  cat /proc/gps (i.e. this is different from the system time found by using date, or gpstime).
• This was off by 2 years.
• The way this worked up till now was by adding a fixed offset to this time.
• This offset can be found as a line saying set symm_gps_offset=31622382 in daqdrc.fw (for example)
• There were similar lines in daqdrc.rcv and daqdrc.dc - however, they were not all the same offset! We couldn't figure out why.
• All these files live in /opt/rtcds/caltech/c1/target/daqd/.

Changes effected:

1. First, we tried changing the offset in the daqdrc.fw file only.
• Incremented it by 24*60*60*365 = number of seconds in a year with no leap seconds/days.
• This did not fix the problem.
2. So J Hanks decided to rebuild the Spectracom driver - these commands may not be comprehensive, but I think I got everything).
• The relevant file is spectracomGPS.c (made a copy of /usr/src/symmetricom-3.3~rc1, called symmetricom-3.3~rc1-patched, this file is in /usr/src/symmetricom-3.3~rc1-patched/include/drv)
/* 2018 had no leap seconds or leap days, so adjust for that */
pHardware->gpsOffset += 31536000;
• re-built and installed the modified symmetricom driver.
• Checked that cat /proc/gps now yields the correct time.
• Reset the gps time offsets in daqdrc.fw, daqdrc.rcv and daqdrc.dc to 0
• With these steps, the frames were being written to /frames with the correct timestamp.
3. Next, we checked the timing on the FEs
• Basically, J Hanks rebuilt the version of the symmetricom driver that is used by the rtcds models to mimic the changes made for FB.
• This did the trick for c1lsc and c1ioo - cat /proc/gps now returns the correct time on those FEs.
• However, c1sus remains problematic (it initially reported a GPS time from 2 years ago, and even after the re-installed driver, is 4 days behind) - he suspects that this is because c1sus is the only FE with a Symmetricom/Spectracom card installed in the I/O chassis. So c1sus reports a gpstime that is ~4 days behind the "correct" gpstime.
• He is going to check with Rolf/other CDS experts to figure out if it's okay for us to simply remove the card and run the models, or if we need to take other steps.
• As part of this work, the c1x02 IOP model was recompiled, re-installed and re-started.

The realtime models were not restarted (although all the vertex FEs are running) - we can regroup next week and decide what is the correct course of action.

 Quote: Attachment #2 shows the minute trend of the pressure gauges for a 12 day period - it looks like there is some issue with the frame builder clock, perhaps this issue resurfaced? But checking the system time on FB doesn't suggest anything is wrong.. I double checked with dataviewer as well that the trends don't exist... But checking the status of the individual daqd processes indeed showed that the dates were off by 1 year, so I just restarted all of them and now the time seems correct. How can we fix this problem more permanently? Also, the P1b readout looks suspicious - why are there periods where it seems like we are reading values better than the LSB of the device?
14392   Wed Jan 9 11:33:35 2019 gautamUpdateCDSTiming issues still persist

Summary:

The gps time mismatch between /proc/gps and gpstime seems to be resolved. However, the 0x4000 DC errors still persist. It is not clear to me why.

Details:

On the phone with J Hanks on Friday, he reminded me that c1sus seems to be the only machine with an IRIG-B timing card installed. I can't find the elog but I remembered that Jamie, ericq and I had done this work in 2016 (?), and I also remembered Jamie saying it wasn't working exactly as expected. Since the DAQ was working fine before this card was installed, and since there are no problems with the recording of channels from the other four FE machines without this card installed, I decided to simply pull out the card from the expansion chassis. The card has been stored in the CDS/FE cabinet along the Y arm for now. There was also a cable that interfaces to the card which brings over the 1pps from the GPS unit, which has also been stored in the CDS/FE cabinet.

This seems to have resolved the mismatch between the gpstime reported by cat /proc/gps and the gpstime commands - Attachment #1 (the <1 second mismatch is presumably due to the deadtime between commands). However, the 0x4000 DC errors still persist. I'll try the full power cycle of FEs and FB which has fixed this kind of error in the past, but apart from that, I'm out of ideas.

Update 1215:

Following the instructions in this elog did not fix the problem. The problem seems to be with the daqd_fw service, which reports the following:

controls@fb1:~ 0\$ sudo systemctl status daqd_fw.service  ● daqd_fw.service - Advanced LIGO RTS daqd frame writer    Loaded: loaded (/etc/systemd/system/daqd_fw.service; enabled)    Active: failed (Result: start-limit) since Wed 2019-01-09 12:17:12 PST; 2min 0s ago   Process: 2120 ExecStart=/usr/bin/daqd_fw -c /opt/rtcds/caltech/c1/target/daqd/daqdrc.fw (code=killed, signal=ABRT)  Main PID: 2120 (code=killed, signal=ABRT)

Jan 09 12:17:12 fb1 systemd[1]: Unit daqd_fw.service entered failed state. Jan 09 12:17:12 fb1 systemd[1]: daqd_fw.service holdoff time over, scheduling restart. Jan 09 12:17:12 fb1 systemd[1]: Stopping Advanced LIGO RTS daqd frame writer... Jan 09 12:17:12 fb1 systemd[1]: Starting Advanced LIGO RTS daqd frame writer... Jan 09 12:17:12 fb1 systemd[1]: daqd_fw.service start request repeated too quickly, refusing to start. Jan 09 12:17:12 fb1 systemd[1]: Failed to start Advanced LIGO RTS daqd frame writer. Jan 09 12:17:12 fb1 systemd[1]: Unit daqd_fw.service entered failed state.

Update 1530:

The frame-writer error was tracked down to a C0EDCU issue. Jon told me that the Hornet CC1 pressure gauge channel was renamed to . C1:Vac-CC1_pressure, and I made the change in the C0EDCU file. However, it returns a value of 9990000000.0, which the frame writer is not happy about... Keeping the old channel name makes the frame-writer run again (although the actual data is bunk).

Update 1755:

J Hanks suggested adding a 1 second offset to the daqdrc config files. This has now fixed the 0x4000 errors, and we are back to the "nominal" RTCDS status screen now - Attachment #2.

Attachment 1: gpstimeSync.png
Attachment 2: Screenshot_from_2019-01-09_17-56-58.png
17193   Mon Oct 17 11:57:27 2022 ChrisOmnistructureCDSTiming system monitoring

I started a GPS timing monitor script, /opt/rtcds/caltech/c1/Git/40m/scripts/cds/tempusTimingMon/tempusTimingMon.py, which runs inside a docker container on optimus. It accesses the GPS receiver (EndRun Tempus LX) status via pysnmp, and creates the following epics channels with pcaspy:

• C1:DAQ-GPS_TFOM “The Time Figure of Merit (TFOM) value ranges from 3 to 9 and indicates the current estimate of the worst case time error. It is a logarithmic scale, with each increment indicating a tenfold increase in the worst case time error boundaries.” (nominally 3, corresponding to 100 nsec)
• C1:DAQ-GPS_STATE “Current GPS signal processor state. One of 0, 1 or 2, with 0 being the acquisition state and 2 the fully locked on state.”
• C1:DAQ-GPS_SATS “Current number of GPS satellites being tracked.”
• C1:DAQ-GPS_DAC “Current 16 bit, Voltage Controlled TCXO DAC value. Typical range is 20000 to 40000, where more positive numbers have the effect of raising the TCXO frequency.”
• C1:DAQ-GPS_SNR “The current average carrier to noise ratio of all tracked satellites, in units of dB. Values less than 35 indicate weak signal conditions.”

Attached is a 1-day trend of the above channels, along with the IRIG-B timing offsets from the IOPs. No big timing excursions or slippages were seen yet, although the c1sus IOP (FEC-20) timing seems to be hopping around by one IOP sample time (15 µsec).

Attachment 1: timing.png
5731   Mon Oct 24 20:00:21 2011 MirkoUpdateCDSTiny little scripts

Located in /opt/rtcds/caltech/c1/userapps/release/cds/common/scripts

Script 1: Diagreset.sh
Hits the diag reset buttons on the models on c1lsc and c1sus computers.

Script 2: Burtrest.sh
Restores the burt files from "today" 4am. Use a text editor if you want to change the times.

3162   Tue Jul 6 17:38:27 2010 JenneUpdateSUSTip Tilt Progress!

[Jenne, Kyung Ha]

We successfully suspended the 4 eddy current dampers for the first Tip Tilt.  We had some lessons learned, including how to carefully get an allen wrench in between the dampers to do up some of the screws, and how to be careful not to bend the wire while tightening the screws.  More tomorrow...

7601   Tue Oct 23 18:12:18 2012 JenneUpdateAlignmentTip tilt wires - the truth

 Quote: At this point I cried foul.  This is not an acceptable situation.  Very little stimulation to the tip-tilts can repoint the beam inside the PR cavity. Steve says that the TT weights, which will attach to the base of the TT mirror mounts and should help keep the mirrors vertical and not hysteretic, are being baked now and should be available tomorrow.

We also wrote down the serial numbers (top center of each TT, inscribed by hand) for what tip tilt is installed where.  I then went through the elog to determine which TT was suspended with what kind of wire (thick or thin).  Summary: all installed tip tilts have thick wire, 0.0036" diameter.

As noted in elog 3295, we had found that there was similar hysteresis whether we used the thick or the thin wire, so we had decided not to go back and re-suspend every optic.

Also, since we will redo the pitch balance tomorrow with the new hardware tomorrow, I think we should put in the new LaserOptik mirrors at the same time.  We have not yet gotten phase maps of them, but we might as well do this rebalancing once, rather than twice.

 Serial number Installed as Wire thickness Notes, elog reference 001 SR 3 0.0036" See elog 3437 002 SR 2 0.0036" See elog 3295 003 PR 2 0.0036" No elog, but inferred since there were 4 with thick wire, and #004 is the thin wire one.  Elog 3437 has notes on the 4 thick, 1 thin situation. 004 spare, dirty originally 0.0017", but looks redone with thicker wire See elog 3295 005 PR 3 0.0036" Was supposed to be spare according to elog 3437, but was installed.  See elog 3437

7625   Thu Oct 25 20:44:11 2012 JenneUpdateSUSTip tilts in progress

Jamie and I spent some time with tip tilt SN001 this afternoon.  This was installed as SR3, so I was going to put a new LaserOptik mirror in there.  I accidentally snapped one of the wires (I forgot how strong the magnets are - one zipped from the mirror holder and captured the wire).  Jamie and I put the new LaserOptik mirror in, with the wedge correct, but we need to re-resuspend it with the 0.0036" wire tomorrow.  We'll also keep working on re-pitch aligning the other optics.

PR2 needs to be put back as a G&H, and we need to put a LaserOptik mirror into PR3.

7630   Fri Oct 26 10:44:25 2012 JenneUpdateSUSTip tilts in progress

 Quote: Jamie and I spent some time with tip tilt SN001 this afternoon.  This was installed as SR3, so I was going to put a new LaserOptik mirror in there.  I accidentally snapped one of the wires (I forgot how strong the magnets are - one zipped from the mirror holder and captured the wire).  Jamie and I put the new LaserOptik mirror in, with the wedge correct, but we need to re-resuspend it with the 0.0036" wire tomorrow.  We'll also keep working on re-pitch aligning the other optics. PR2 needs to be put back as a G&H, and we need to put a LaserOptik mirror into PR3.

We resuspended SN001 this morning with 0.0036" wire.  We did as Koji suggested, and flipped the wire clamp so the suspension point is a little higher, so we'll see if that helps.  We put LaserOptik mirror SN1 into this TT001.

We put the G&H mirror back into TT004, which is PR2.  We also put a LaserOptik mirror (SN5) into TT005, which is SR3.

Jamie is working on re-pitch aligning TT004 and TT005 (we already did 001), then we can re-install them in the vacuum system later this afternoon.

7631   Fri Oct 26 13:08:14 2012 JenneUpdateSUSTip tilts in progress

Quote:

 Quote:

Jamie is working on re-pitch aligning TT004 and TT005 (we already did 001), then we can re-install them in the vacuum system later this afternoon.

The tip tilts have all been pitch-adjusted now, and they have all been put back onto the tables, with the same serial numbers in the same places as we took them out.  Jamie also re-leveled the BS table.

Raji and I will align things after I finish measuring the MC spot positions.

6756   Tue Jun 5 20:42:59 2012 SureshSummaryIOOTip-Tilt Cabling

I have made a preliminary sketch of the cabling involved in connecting the Tip-tilt coil drivers.   This is a preliminary document.

 Required parts Quantity Solution 1) DAC Card inserted into C1IOO machine 1 buy or borrow from Cymacs 2) SCSI cable from DAC to D080303 box 1 buy or find at the 40m 3) D080303 box (SCSI to IDC breakout box) 1 Jay may have had spare, if not we have to make one 4) 40 pin IDC cables from D080303 to AntiImaging filter 2 Jay may have kept some stock if not make them 5) 10 pin IDC cables from Anti Imaging filters to Whitening filters 2 make 6) sma to lemo cables from Whitening to coil drivers 4x4=16 buy 7) 15pin IDC to 25pin DSub cables from drivers to feedthroughs on the chambers 4  (length?) make 8) 25pin DSub feedthrough on OMC chamber 1 check in 40m stock else buy 9) 25pin DSub  Kapton strip cable from OMC wall to table top 1 check if any spare are available in aLIGO stock 10) 25pin DSub Kapton strip cable from post to tip-tilt 4 aLIGO team said they have a few to spare if not buy 10) Quadrapus cables on the tip-tilts 4 Jamie is negotiating with aLIGO cable team

6759   Tue Jun 5 22:39:06 2012 JenneSummaryIOOTip-Tilt Cabling

2 questions (so far) regarding your diagram / doc:

We are using 3 of the feed-throughs on the BS chamber, and 1 on the OMC chamber, even though we have 2 TTs on the BS table, 1 on the OMC table, and 1 on the IMC table? Just wanted to check.

Does your list / table at the bottom include all of the cables we already have, as well as the ones we need?  (Or maybe we just have nothing so far, so this is a moot question).

6762   Wed Jun 6 01:23:32 2012 SureshSummaryIOOTip-Tilt Cabling

 Quote: 2 questions (so far) regarding your diagram / doc: We are using 3 of the feed-throughs on the BS chamber, and 1 on the OMC chamber, even though we have 2 TTs on the BS table, 1 on the OMC table, and 1 on the IMC table? Just wanted to check. Does your list / table at the bottom include all of the cables we already have, as well as the ones we need?  (Or maybe we just have nothing so far, so this is a moot question).

The scheme currently is to run a 25pin Kapton strip cable from BS to IMC table inside the chamber.  However we cannot do the same for the OMC table since it will cross the bellows which we often remove.  So we need to supply the one tip-tilt on the OMC table from outside.  I could not spot a spare unused feedthough on the OMC chamber.  We will have to swap one of the blank flanges for one with a few feed throughs.

We do not have any of the cables.   So everything listed has to be arranged for.   The pics are from the existing coil driver system on the SUS machine.

14742   Wed Jul 10 10:04:09 2019 gautamUpdateSUSTip-Tilt moved from South clean cabinet to bake lab cleanroom

Arnaud and I moved one of the two spare TT suspensions from the south clean cabinet to the bake lab clean room. The main purpose was to inspect the contents of the packaging. According to the label, this suspension was cleaned to Class A standards, so we tried to be clean while handling it (frocks, gloves, masks etc). We found that the foil wrapping contained one suspension cage, with what looked like all the parts in a semi-assembled state. There were no OSEMs or electronics together with the suspension cage. Pictures were taken and uploaded to gPhoto. Arnaud is going to plan his tests, so in the meantime, this unit has been stored in Cabinet #6 in the bake lab cleanroom.

6770   Wed Jun 6 19:46:46 2012 SureshSummaryIOOTip-tilt assembly: current status and work remaining

Recent History

The lower blades which I had given to the Physics Workshop for making a vacuum relief hole (using a sinker-EDM process) came back about ten days ago.   Merih Eken <meken@caltech.edu>,  the supervisor at the Physics Dept workshop, handled this matter for us.  The blades were sent to a local EDM machineshop and returned in about three working days ( a weekend intervened).

Bob cleaned and handed them over to me yesterday evening.

Current status

Today I have reassembled the four tip-tilts.  I have repacked them in clean bags (double bagged) shifted them to Clean Optics Cabinet (near the ETMX chamber).  The four tip-tilts are in the bottom-most shelf in the cabinet.  There are also some tip-tilt spares in a separate envelope.

Note:  The mirror holder is now held tightly by the eddy current dampers.  This was done for safety of the wires during transportation from LHO.  The eddy current damper in the front of the mirror has to be retracted to allow the mirror holder to swing free.  It has be to about 1mm away from the suspended mirror holder

Work Remaining

1) We need to install the quadrapus cables.  The connector placement on the BOSEM side will have some issues.  It is best to loosen the BOSEM seating as well as the connector seating screws and then push the cable connector into place.  Caution:  when the connector seating screws on the BOSEM are loosened the flexible ckt could be damaged by the loose connector.

2) Insert the mirrors into the mirror holders and balance the suspension such that the mirror's hang vertical and do not have a large yaw offset.

3) Adjust the wire suspension point height so that the flags are in the center of the BOSEM aperture.  Else they will strike against the

4) We need to adjust the position of the BOSEMs such that the shadow sensor signals are at 50%.  This ensures that all the magnets hang at an appropriate distance from their respective coils.

5) To do (3) we need to set up a shadow sensor read-out set-up for one tip-tilt (four sensors)

Attachment 2: IMG_0687.JPG
14047   Mon Jul 9 17:29:28 2018 Udit KhandelwalSummaryTip-TIltTipTilt mirror holder final changes

Final Summary of changes to mirror holder in Tip-Tilt holder.

Determining minimum range for Side Clamp:

1. The initial distance b/w wire-release point and mirror assembly COM = 0.265 mm

2. But this distance is assuming that wire-release point is at mid-point of clamp. So I'm settling on a range of +/- 1mm. The screenshots below confirm range of ~1mm between (1) side screw & protrusion and (2) clamp screw and clamp.

Determining length of tilt-weight assembly rod at the bottom to get $\pm$ 20mRad range

The tilt-weight assembly is made from following Mcmaster parts:
Rod   - 95412A864 18-8 SS  #2-56 Threaded Rod
Nuts  - 91855A103 18-8 SS #2-56 Acorn Cap Nut

Since the weights are fixed, only rod length can be changed to get the angle range.

$tan \theta =\frac{d}{h}$

$d= h \times tan\theta = 34.25\text{mm} \times tan(20 \text{mRad}) = 0.69 \text{mm}$
So a range of 1 mm between nut's inner face and mirror-holder face should be enough. Since holder is 12 mm thick, rod length = 12mm + 2 x 1mm + 2 x (nut length) = 12 + 2 + 9.6 = 23.6 mm = 0.93 inch. So a 1" rod from Mcmaster will be fine.

Attachment 4: 2-1.png
4199   Tue Jan 25 06:48:55 2011 kiwamuUpdateGreen LockingTo do list

Here are some tasks that I want someone to work on during my absence.

# 1. Y-arm alignment for IR

Basically we gradually have to move onto the Y-arm locking at some point.

Prior to it we need to align the Y arm for IR. Probably we have to touch PZT1 and PZT2.

It would be very nice if the X-arm alignment also gets improved together with this work.

# 2. Temperature feedback with a digital control for X end PDH lock

Need a temperature feedback not with an analog way but with a digital way because we want to put an offset and the feedback signal at the same time (#4198).

Right now the temperature control input of the laser is connected to a slow DAC (#3850).

Probably we should plug the feedback signal from the PDH box to the fast ADC (i.e. c1iscex), and then connect a fast DAC to the laser temperature.

# 3. Calibration of optical gain for IR arm locking

In order to evaluate the performance of the green locking, one of the key points is the IR PDH signal.

Because it tells us how much the motion of the X arm is suppressed at IR when the green lock is engaged.

To get this information in m/sqrtHz, we need to know the optical gain.

# 4. MC servo characterization and PSL frequency noise measurement

SInce the green beat note tells us the frequency difference between the MC and the arm in the current configuration, we should know how the MC servo is.

Along with this work, I need someone to measure the PSL frequency noise, when it is locked to the MC over the frequency range from 0.01Hz to 1kHz.

# 5. PLL characterization

Solve this issue (#4195) and make it reliable.

9667   Mon Feb 24 23:43:10 2014 ranaSummaryGeneralToDo

1) Fixup REFL165: remove ND filters, get box for PD, dump diode reflections, put less light on diode, change DC transimpedance (?), max power dissipation on BBPD < 0.5 W w/ 25 V bias. Perhaps replace OP27 with TLE2027.

2) Make plan for fixing fiber layout up and down the arms. Need tubing for the whole run. Don't make it cheesy. Two fibers per arm.

3) Fix LSC model to allow user switching of whitening. Get back to working on AutoLock scripts (not Guardian).

4) Manasa, Q, Jenne, tune Oplev servos Tuesday morning/afternoon.

5) Reconnect the other seismometers (Steve, Jenne). For real.

6) Balance PRMI coils at high frequency.

50   Thu Nov 1 19:53:02 2007 Andrey RodionovBureaucracyPhotosTobin's picture
Attachment 1: DSC_0053.JPG
4282   Mon Feb 14 01:19:18 2011 ranaUpdateCDSToday's CDS problems

This is just a listing of  CDS problems I still notice today:

1. MC2-MCL button was left ON due to BURT failure. This, of course, screws up our Green locking investigations because of the unintended feedback. Please fix the BURT/button issue.

2. The GCV - FB0 status is RED. I guess this means there's something wrong? Its really a bad idea to have a bunch of whited out or falsely red indicators. No one will ever use these or trust these in the future.
3. MC1/2/3 Lockins are all white. Also, the MODE switches for the dewhitening are all white.
4. Is the MC SIDE coil dewhitening filter synced with anything? It doesn't seem to switch anything. Maybe the dewhite indicators at the top right of the SUS screens can be made to show the state of the binary output instead of just the digital filter
5. MC WFS is all still broken. We need a volunteer to take this on - align beams, replace diodes, fix code/screens.
4285   Mon Feb 14 07:58:43 2011 SureshUpdateCDSToday's CDS problems

I am concentrating on the RF system just now and will be attending to the RF PDs one by one.  Also plan to work on some of the simpler CDS problems when I overlap with Joe.  Will be available for helping out with the beam alignment.

4309   Wed Feb 16 23:54:34 2011 ranaUpdateCDSToday's CDS problems
1. ETMX cannot load his filter coefficients. Even if I change the file, the load button doesn't work. I tried changing the lockin filters but they won't load.
2. The ETMX filter modules appear to have 2048 for some of the modules and 16384 for some of the others. How come the make script doesn't make them all 16384?
3. There are a bunch of kill/start scripts in the scripts/ directory instead of in the scripts/FE/ directory. Did this get reverted after a new code exchange?
4. I tried restarting the code using c1startscx, but that doesn't work correctly. It cannot find the burtrb and burtwb files even though they are in the normal path.
5. Kiwamu was using a bunch of cockamamie filters I found.
6. I can't get any minute trend data. I tried on rossa, rosalba, and op440m.
 MC damp dataviewer diaggui AWG c1lsc c1ioo c1sus c1iscex c1iscey RFM The Dolphins Sim.Plant Frame builder TDS Cabling
5214   Fri Aug 12 17:27:49 2011 YoichiSummaryCDSToggle button for RCG
Bottom line: I made an RCG block to realize a toggle button easily.

Read on if you need such a button, or if you want to know how to
write a new RCG block with C.

-----------------
When I was making MEDM screens for FFC, I wanted to have a toggle
button to enable/disable the FFC path.
I wanted to have something like the ON/OFF buttons of the filter bank
screens, the one changes its state every time I click on it.
However, I could not find an easy way to realize that.

From MEDM, I can send a value to an EPICS channel using a "Message Button".
This value is always the same, say 1.
In the RCG model, I used a cdsEpicsMomentary block so that whenever the channel
gets 1, it stays to be 1 for a while and turns back to 0 in a second or so.
This generates a pulse of 1 when I click on a message button on a MEDM screen.
Then I needed a block to keep its internal state (0 or 1), and flips its state
whenever it receives a pulse of 1.
Since I couldn't find such a block in the current RCG library, I implemented one
using the cdsFunctionCall block. It allows you to implement a block with C code.

There is a good explanation of how to use this block in the CDS_PARTS library.
Here is basically what I did.

(1) Drag and drop thee cdsFunctionCall block to my model.

(2) In the "Block Properties", I put the following line in the Description field.
inline cdsToggle /opt/rtcds/caltech/c1/userapps/release/cds/common/src/cdsToggle.c

This means to call a function cdsToggle(), whose code is in the file indicated above.

(3) The contents of the source code is very simple.
void cdsToggle(double *in, int inSize, double *out, int outSize){
static double x = 0;
static double y = 0;

if (*in != y){
y = *in;
if (y == 1){
x = (x == 1) ? 0 : 1;
*out = x;
}
}
}

The function prototype is always the same. *in and *out are the pointers to the arrays of doubles
for input and output signals of the block. In simuLink, the signals have to be
multiplexed so that the RCG can know how many signals are handed to or returned from the function.
In order to keep the internal state of my custom block, I used "static" keyword in the
declaration of the variables. The rest of the code should be obvious.

(4) Just compile the model as usual. The RCG will automatically include the source code and put
a call to the function in the proper place.

I made the block a library so that people can use it.
/opt/rtcds/caltech/c1/userapps/trunk/cds/common/models/cdsToggle.mdl
is the one.
For the usage of it, please have a look at
/opt/rtcds/caltech/c1/userapps/trunk/isc/c1/models/c1lsc
16485   Wed Nov 24 17:13:31 2021 YehonathanMetaphysicsGeneralToilet tank broken

The toilet tank in the big bathroom stopped refilling. I contacted PPService@caltech.edu and put up an "Out of Order sign".

16487   Tue Nov 30 11:03:44 2021 YehonathanMetaphysicsGeneralToilet tank broken

a plumber came in yesterday and fixed the issue.

 Quote: The toilet tank in the big bathroom stopped refilling. I contacted PPService@caltech.edu and put up an "Out of Order sign".

483   Fri May 16 17:27:55 2008 AndreyOmnistructureGeneralToilets are broken, do not use them !!!

Both toilets in 40-meter were constantly flushing, the leaking water was on the floor inside of the restrooms, so

BOTH RESTROOMS ARE CLOSED TILL MONDAY

I have heard the constant loud sound of flushing water, opened the door, and was unpleasantly surprised because all the floor was under the layer of water and the toilets were constantly flushing. I called security at X5000, a plumber came in and told that a team of plumbers needs to repair the flushing system after the weekend. The plumber today just shut off the flushing water, wiped off the floor and told not to use the restrooms in the weekend. We should expect a team of plumbers on Monday.

Sinks are working, so you can wash your hands.
5076   Sun Jul 31 17:28:34 2011 kiwamuSummaryLSCTolerance of Arm length = 2 cm

Required arm length = 37.7974 +/- 0.02 [m]

This is a preliminary result of the estimation of the Arm length tolerance.

This number was obtained from a simulation based on Optickle.
Note that the simulation was done by considering misplacements in only the arm lengths while keeping PRCL, SRCL and MICH at the ideal lengths.
Therefore the tolerance will be somewhat tighter if misplacements in the central part are taken into account.

Next : check 3f signals, and include misplacements in PRCL, SRCL and MICH.

(Background)
We will re-position the ETMY/Y suspensions to adjust the arm lenghts during the coming vent.
To get a reasonable sensing matrix for LSC, the arm length must be adjisted within a certain precision.
So we need to know the tolerance of the arm lengths.

(How to estimate)
Optickle, a frequency domiain interferomtere simulator, is used to model the response of the 40m interferometer.
I buit a 40m model in Optickle, and in this model every optical distance is adjusted to the perfect length.
Then some offsets are added on the macroscopic position of ETMs to see what will happen in the LSC sensing matrix.
When putting the offsets, the amount of offsets are randomly assigned with a Gaussian distribution (see Figure.1).
Therefore the calculation is a Monte-Calro style, but this doesn't have to be a Monte-Calro
because the parameter space is only 2-dimensions (i.e. X-arm and Y-arm length) and it can be done by simply scanning the 2-dimentional parameter space.
The reason why the Monte-Calro style was chosen is because I wanted to expand this simulation
to a more general simulation which can handle PRCL, SRCL and MICH misplacements as well.
This time I ran the Monte-Calro 1000 times.

Figure.1 History of random walk in X-Y arm lengths parameter space.
The position of ETMY and ETMX are randomly chosen with a Gaussian distribution function in every simulation.
This example was generated when \sigma_x = \sigma_y = 2 cm, where \sigma is the standard deviation of
the Gaussian function. The number of simulation is 1000 times.

(Criteria)
I made two criteria for the acceptable sensing matrix as follows :
(1) The decrease in the optical gain of the important signals (diagonal signals) must be within a factor of 3 (factor of ~ 0.5 in log scale).
(2) MICH and SRCL signals are separated within a range of 60 - 120 deg in their demodulation phases on POP55.

(Results1 : sensing matrix)
Figure.2 shows the resultant sensing matrix with histograms when \sigma_x = \sigma_y = 2,
where \sigma_x, \sigma_y are the given standard deviation in the position of ETMX and ETMY.
The diagonal signals (in red-rectangular window) shows variation in their optical gain within a factor of 0.5 in log scale (factor of 3 in linear scale).
This satisfies my requirement (1) mentioned in the last section.

Figure.2  A sensing matrix of the 40-m DRFPMI while changing the position of ETMX/Y by \sigma = 2 cm.
For convenience,  only REFL11, AS55, POP11 and POP55 are shown. They are the designed signal ports that
mentioned in the aLIGO LSC document (T1000298). In all the histograms, x-axis represents the optical gain in log scale in units of [W/m].
The y-axis is the number of events. The diagonal ports are surrounded by red rectangular window.

(Results2 : demodulation phase of MICH and SRCL on POP55)
Now a special attention should be payed on the MICH and SRCL signals on POP55.
Since MICH and SRCL are designed to be taken from POP55, they must be nicely separated in their demodulation phases.
Therefore the demodulation phase of MICH and SRCL has to be carefully examined.
The plot in Figure.3 is the resultant phase difference between MICH and SRCL on POP55 when \sigma_x = \sigma_y = 2 cm.
As shown in the plot the phase are always within a range of 60 - 120 deg, which satisfies my requirement (2) mentioned in the last section.

Figure.3 Difference in the demodulation phase of MICH and SRCL on POP55.
x-axis is the difference in the demodulation phase of MICH and SRCL, and y-axis the number of events.

(Notes on the Optickle model)