| ID |
Date |
Author |
Type |
Category |
Subject |
Text |
 |
|
9513
|
Thu Jan 2 10:15:20 2014 |
Jamie | Summary | General | linux1 RAID crash & recovery | Well done Koji! I'm very impressed
with the sysadmin skillz. |
|
|
9536
|
Tue Jan 7 23:53:35 2014 |
Jamie | Update | CDS | daqd can't connect to c1vac1, c1vac2 | dadq is logging the following error messages
to it's log related to the fact that it can't
connect to c1vac1 and c1vac2: |
|
|
9574
|
Fri Jan 24 13:10:12 2014 |
Jamie | HowTo | LSC | Procedure to measure PRC length |
|
|
|
9966
|
Fri May 16 20:55:18 2014 |
Jamie | Frogs | lore | un-full-screening Ubuntu windows with F11 | Last week Rana and I struggled to figure
out how to un-full-screen windows on the
Ubuntu workstations that appeared to be stuck |
|
|
10018
|
Tue Jun 10 09:25:29 2014 |
Jamie | Update | CDS | Computer status: should not be changing names | I really think it's a bad |
|
|
10033
|
Thu Jun 12 15:31:47 2014 |
Jamie | Update | CDS | Note on cables for talking to slow computers |
|
|
|
10040
|
Sun Jun 15 14:26:30 2014 |
Jamie | Omnistructure | CDS | cdsutils re-installed |
|
|
|
10041
|
Sun Jun 15 14:41:08 2014 |
Jamie | Omnistructure | CDS | cdsutils re-installed |
|
|
|
10190
|
Sun Jul 13 11:37:36 2014 |
Jamie | Update | Electronics | New Prologix GPIB-Ethernet controller |
|
|
|
10276
|
Sat Jul 26 13:38:34 2014 |
Jamie | Update | General | Data Acquisition from FC into EPICS Channels |
|
|
|
11384
|
Tue Jun 30 11:33:00 2015 |
Jamie | Summary | CDS | prepping for CDS upgrade | This is going to be a big one. We're
at version 2.5 and we're going to go
to 2.9.3. |
|
|
11390
|
Wed Jul 1 19:16:21 2015 |
Jamie | Summary | CDS | CDS upgrade in progress | The CDS upgrade is now underway
Here's
what's happened so far:
Installed and linked in |
|
|
11393
|
Tue Jul 7 18:27:54 2015 |
Jamie | Summary | CDS | CDS upgrade: progress! | After a couple of days of struggle, I made
some progress on the CDS upgrade today:
|
|
|
11396
|
Wed Jul 8 20:37:02 2015 |
Jamie | Summary | CDS | CDS upgrade: one step forward, two steps back | After determining yesterday that all the
daqd issues were coming from the frame writing,
I started to dig into it more today. |
|
|
11397
|
Wed Jul 8 21:02:02 2015 |
Jamie | Summary | CDS | CDS upgrade: another step forward, so we're back to where we started (plus a bit?) | Koji did a bit of googling to determine
that 'Wrong Network' status message
could be explained by the fb myrinet |
|
|
11398
|
Thu Jul 9 13:26:47 2015 |
Jamie | Summary | CDS | CDS upgrade: new mx 1.2.16 installed | I rebuilt/installed mx 1.2.16 to use "ether-mode",
instead of the default MX-10G:
controls@fb |
|
|
11400
|
Thu Jul 9 16:50:13 2015 |
Jamie | Summary | CDS | CDS upgrade: if all else fails try throwing metal at the problem | I roped Rolf into coming over and adding
his eyes to the problem. After much
discussion we couldn't come up with any |
|
|
11402
|
Mon Jul 13 01:11:14 2015 |
Jamie | Summary | CDS | CDS upgrade: current assessment | daqd is still behaving unstably.
It's still unclear what the issue is.
The current failures look like |
|
|
11404
|
Mon Jul 13 18:12:50 2015 |
Jamie | Summary | CDS | CDS upgrade: left running in semi-stable configuration | I have been watching daqd all day and I
don't feel particularly closer to understanding
what the issues are. However, things |
|
|
11406
|
Tue Jul 14 09:08:37 2015 |
Jamie | Summary | CDS | CDS upgrade: left running in semi-stable configuration | Overnight daqd restarted itself only about
twice an hour, which is an improvement:
controls@fb /opt/rtcds/caltech/c1/target/fb |
|
|
11412
|
Tue Jul 14 16:51:01 2015 |
Jamie | Summary | CDS | CDS upgrade: problem is not disk access | I think I have now determined once and
for all that the daqd problems are NOT due
to disk IO contention. |
|
|
11415
|
Wed Jul 15 13:19:14 2015 |
Jamie | Summary | CDS | CDS upgrade: reducing mx end-points as last ditch effort | I tried one last thing, suggested by Keith
and Gerrit. I tried reducing the number
of mx end-points on fb to zero, which should |
|
|
11417
|
Wed Jul 15 18:19:12 2015 |
Jamie | Summary | CDS | CDS upgrade: tentative stabilty? | Keith Thorne provided his eyes on the situation
today and had some suggestions that might
have helped things |
|
|
11427
|
Sat Jul 18 15:37:19 2015 |
Jamie | Summary | CDS | CDS upgrade: current status | So it appears we have found a semi-stable
configuration for the DAQ system post upgrade:
|
|
|
11455
|
Tue Jul 28 17:07:45 2015 |
Jamie | Update | General | Data missing |
|
|
|
13125
|
Wed Jul 19 08:37:21 2017 |
Jamie | Update | CDS | Update on front-end/DAQ rebuild | After the catastrophic fb disk failure
last week we lost essentially the entire
front end system (not any of the userapp |
|
|
13127
|
Wed Jul 19 14:26:50 2017 |
Jamie | Update | CDS | Update on front-end/DAQ rebuild |
|
|
|
13130
|
Fri Jul 21 18:03:17 2017 |
Jamie | Update | CDS | Update on front-end/DAQ rebuild | Update:
front ends booting with
the new Debian jessie diskless root image |
|
|
13132
|
Sun Jul 23 15:00:28 2017 |
Jamie | Omnistructure | VAC | strange sound around X arm vacuum pumps | While walking down to the X end to reset
c1iscex I heard what I would call a "rythmic
squnching" sound coming from under the |
|
|
13136
|
Mon Jul 24 10:59:08 2017 |
Jamie | Update | CDS | c1iscex models died |
|
|
|
13138
|
Mon Jul 24 19:28:55 2017 |
Jamie | Update | CDS | front end MX stream network working, glitches in c1ioo fixed | MX/OpenMX
network running
Today I
got the mx/open-mx networking working for |
|
|
13145
|
Wed Jul 26 19:13:07 2017 |
Jamie | Update | CDS | daqd showing same instability as before | I recompiled daqd on the updated fb1, similar
to how I had before, and we're seeing
the same instability: process crashes when |
|
|
13149
|
Fri Jul 28 20:22:41 2017 |
Jamie | Update | CDS | possible stable daqd configuration with separate DC and FW | This week Jonathan Hanks and I have been
trying to diagnose why the daqd has been
unstable in the configuration used by the |
|
|
13153
|
Mon Jul 31 18:44:40 2017 |
Jamie | Update | CDS | CDS system essentially fully recovered | The CDS system is mostly fully recovered
at this point. The mx_streams are all
flowing from all front ends, and from all |
|
|
13164
|
Thu Aug 3 19:46:27 2017 |
Jamie | Update | CDS | new daqd restart procedure | This is the daqd restart procedure:
$ ssh |
|
|
13165
|
Thu Aug 3 20:15:11 2017 |
Jamie | Update | CDS | dataviewer can not raise test points | For some reason dataviewer is not able
to raise test points with the new daqd setup,
even though dtt can. If you raise a |
|
|
13198
|
Fri Aug 11 19:34:49 2017 |
Jamie | Update | CDS | CDS final bits status update | So it appears we now have full frames and
second, minute, and minute_raw trends.
We are still not able to raise |
|
|
13205
|
Mon Aug 14 19:41:46 2017 |
Jamie | Update | CDS | front-end/DAQ network down for kernel upgrade, and timing errors | I'm upgrading the linux kernel for
all the front ends to one that is supposedly
more stable and won't freeze when we |
|
|
13215
|
Wed Aug 16 17:05:53 2017 |
Jamie | Update | CDS | front-end/DAQ network down for kernel upgrade, and timing errors | The CDS system has now been up moved to
a supposedly more stable real-time-patched
linux kernel (3.2.88-csp) and RCG r4447 (roughly |
|
|
13217
|
Wed Aug 16 18:01:28 2017 |
Jamie | Update | CDS | front-end/DAQ network down for kernel upgrade, and timing errors |
|
|
|
13219
|
Wed Aug 16 18:50:58 2017 |
Jamie | Update | CDS | front-end/DAQ network down for kernel upgrade, and timing errors |
|
|
|
13258
|
Mon Aug 28 08:47:32 2017 |
Jamie | Summary | LSC | First cavity length reconstruction with a neural network |
|
|
|
15742
|
Mon Dec 21 09:28:50 2020 |
Jamie | Configuration | CDS | Updated CDS upgrade plan |
Quote:
|
|
|
16299
|
Wed Aug 25 18:20:21 2021 |
Jamie | Update | CDS | GPS time on fb1 fixed, dadq writing correct frames again | I have no idea what happened to the GPS
timing on fb1, but it seems like the issue
was coincident with the power |
|
|
16302
|
Thu Aug 26 10:30:14 2021 |
Jamie | Configuration | CDS | front end time synchronization fixed? | I've been looking at why the front
end NTP time synchronization did not seem
to be working. I think it might not |
|
|
17109
|
Sun Aug 28 23:14:22 2022 |
Jamie | Update | Computers | rack reshuffle proposal for CDS upgrade | @tega This looks great, thank you for putting
this together. The rack drawing in
particular is great. Two notes: |
|
|
17379
|
Tue Jan 3 12:10:46 2023 |
Jamie | Update | CDS | Yearly DAQD fix 2023, did not work :( | This
whole procedure is no longer needed after
the CDS upgrade. We don't do things |
|
|
7173
|
Tue Aug 14 11:33:14 2012 |
Jamie Alex Den | Update | CDS | AI and AA filters | When signals are transmitted between the
models running at different rates, no AI
or AA filters are automatically applied. |
|
|
10188
|
Fri Jul 11 22:02:52 2014 |
Jamie, Chris | Omnistructure | CDS | cdsutils: multifarious upgrades | To make the latest cdsutils available in
the control room, we've done the following:
Upgrade pianosa to Ubuntu 12 (cdsutils |
|
|
13207
|
Mon Aug 14 20:12:09 2017 |
Jamie, Gautum | Update | CDS | Weird problem with GPS receiver | Today we saw a weird issue with the GPS
receiver (EndRun Technologies Tempus LX).
GPS timing on fb1 (which is handled via IRIG-B |
|