40m QIL Cryo_Lab CTN SUS_Lab TCS_Lab OMC_Lab CRIME_Lab FEA ENG_Labs OptContFac Mariner WBEEShop
  40m Log, Page 317 of 344  Not logged in ELOG logo
ID Date Author Typedown Category Subject
  10961   Fri Jan 30 11:37:20 2015 manasaFrogsTreasureSP table madness ends

SP table has been a mess because Q and I had let our SURF leave without cleaning up.

I cleaned up the SP table, put things back where they belong and did some sorting. I will put back the optomechanics where they belong sometime later.

For now, check out the SP table next time you are looking for a Y1  or lens or BS.

 

 

 

  11078   Thu Feb 26 16:37:21 2015 manasaFrogsLSCRIP illegal power supply

No more illegal power supply at the LSC rack yes

The amplifiers are now being powered by the rack power supply through fuse blocks.

To make new connections, I shutdown the +/-15 V low noise power supplies. They were turned back ON after the work.

Quote:

 Is this your illegally installed HP bench power supply?

20131023_222351.jpg

If so, or if not but you care about the signal that passes through these amplifiers, I suggest you remove this temporary power supply and wire the power from the rack power supplies through the fuse blocks and possibly use a voltage regulator.

In 24 hours, that power supply will be disconnected and the wires snipped if they are still there.

 

  11124   Mon Mar 9 16:50:35 2015 Champagne DuckFrogsTreasureCelebrating Lock

  11129   Tue Mar 10 19:59:13 2015 KojiFrogsCamerasMessage from the IFO

  11241   Thu Apr 23 23:07:23 2015 DugoliniFrogsALARMlaptops warning

Please!

Don't put laptops on the ISC Tables!

  11289   Wed May 13 10:07:36 2015 ranaFrogsPEMGuralp breakout paddle

Reward being offered for the safe return of this thing:

  11290   Wed May 13 13:33:34 2015 SteveFrogsPEMGuralp breakout box recovered

COD_Sugar napolion is due to Steve:  Item delivered, model CMG-SCU-0013, sn G9536

Quote:

Reward being offered for the safe return of this thing:

 

  11521   Thu Aug 20 18:08:28 2015 IgnacioFrogs40m upgradingFatality. Something broke.

So I made coffee at 1547 and was astonished to find the above. Its a sad, very sad day.

At first I thought that something (a gravity wave?) or someone, accidentally hit the thing and it fell and broke. But Koji told me that the janitor was cleaning around the thing and it did indeed fell accidentally.

  11618   Fri Sep 18 09:06:26 2015 ranaFrogsComputer Scripts / Programsremote data access: volume 1, Inferno

Trying to download some data using matlab today, I found that my ole mDV stuff doesn't work because its MEX files were built for AMD64...

Tried to rebuild the NDS1 MEX according to 7 year old instructions didn't work; our GCC is 'too' new.

From the Remote Data Access wiki (https://wiki.ligo.org/RemoteAccess/MatlabTools) I got the new 'get_data.m' and 'GWdata.m'. These didn't run, so I updated the nds2-client and matlab-nds2-client on Donatella.

Still doesn't run to get 40m data. It recognizes that we're C1, but throws some java exception error. Maybe it doesn't work on the NDS1 protocol of our framebuilder?

So then I noticed that our NDS2 server on megatron is no longer running...thought it was supposed to run via init.d. Found that the nds2 binary doesn't run because it can't find libframecpp.so.5; maybe this was blown away in some recent upgrade? We do have versions 3, 4, 6, 7, & 8 of this library installed.

So now, after an hour or two, I'm upgrading the nds2 server on megatron (plus a hundred dependencies) as well as getting a newer version of matlab to see if there's some kind of java version issue there.

Of course python still works to get data, but doesn't have any of the wiener filter calculating code that matlab has...

  11623   Fri Sep 18 19:19:49 2015 ranaFrogsComputer Scripts / Programsremote data access: volume 1, Inferno

NDS2 restarted after hours long upgrade process; testing has begun. Let's try to get some long stretches of MC locked with MCL FF ON this weekend so's I can test out the angular FF idea.

  11725   Mon Nov 2 17:39:01 2015 KojiFrogsGeneralDRFPMI celebration

やったー!Yatta!

  12207   Tue Jun 21 11:26:42 2016 varunFrogsCDSmedm command not working

"medm: command not found" error when run through command line both in pianosa and rossa in both editing and execution modes. It however gets executed and edited through the sitemap button. Don't know the source of the problem. Gautam did check the .bashrc file. aliases for SITEMAP and m40m are intact in the .bashrc file.

  12208   Tue Jun 21 11:49:29 2016 ericqFrogsCDSmedm command not working

The workstations' .bashrc is a symbolic link to /users/controls/.bashrc

In it, someone commented out the critical line:

#source /ligo/cdscfg/workstationrc.sh

I uncommented it. medm (and all of the other things like cdsutils) work again.

I blame jamie.

  12214   Sun Jun 26 15:27:28 2016 ranaFrogsIOOPMC /MC lopced

Found PMC unlocked for many hours so I relocked it. IMC relocked by itself, but the input switch seems to be flickering to fast. Also the Keep Alive bit is not flashing. no

  12646   Tue Nov 29 17:46:18 2016 rana, gautamFrogsComputer Scripts / Programsgateway PWD change

We found that someone had violated all rules of computer security decency and was storing our nodus password as a plain text file in their bash_profile.

After the flogging we have changed the pwd and put the new one in the usual secret place.

  12694   Fri Jan 6 17:00:26 2017 ranaFrogsTreasureVideo of Lab Tour

In this video: https://youtu.be/iphcyNWFD10, the comments focus on the orange crocs, my wrinkled shirt, and the first aid kit.

  13037   Sun Jun 4 14:19:33 2017 ranaFrogsComputersNetwork slowdown: Martians are behind a waterwall

A few weeks ago we did some internet speed tests and found a dramatic difference between our general network and our internal Martian network in terms of access speed to the outside world.

As you can see, the speed from nodus is consistent with a Gigabit connection. But the speeds from any machine on the inside is ~100x slower. We need to take a look at our router / NAT setup to see if its an old hardware problem or just something in the software firewall. By comparison, my home internet download speed test is ~48 Mbit/s; ~6x faster than our CDS computers.


controls@megatron|~> speedtest
/usr/local/bin/speedtest:5: UserWarning: Module dap was already imported from None, but /usr/lib/python2.7/dist-packages is being added to sys.path
  from pkg_resources import load_entry_point
Retrieving speedtest.net configuration...
Testing from Caltech (131.215.115.189)...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by Race Communications (Los Angeles, CA) [29.63 km]: 6.52 ms
Testing download speed................................................................................
Download: 6.35 Mbit/s
Testing upload speed................................................................................................
Upload: 5.10 Mbit/s
controls@megatron|~> exit
logout
Connection to megatron closed.
controls@nodus|~ > speedtest
Retrieving speedtest.net configuration...
Testing from Caltech (131.215.115.52)...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by Phyber Communications (Los Angeles, CA) [29.63 km]: 2.196 ms
Testing download speed................................................................................
Download: 721.92 Mbit/s
Testing upload speed................................................................................................
Upload: 251.38 Mbit/s

  13316   Mon Sep 18 15:00:15 2017 rana, gautamFrogsComputer Scripts / Programsgateway PWD change

We implemented the post-SURF-season nodus password change today.

New password can be found at the usual location.

  13470   Fri Dec 8 23:31:31 2017 johannesFrogsASSc1ass slow channel offloading scripts with small

While staring at epics records all day I noticed something about the PIT/YAW offset sliders and ASS offset offloading to slow channels scripts that I'm not sure others are aware off, so I'll briefly discuss it in this post.

The PIT and YAW sliders directly control soft channels that are hosted on the slow machine. Secondary epics records disentangle them for the individual coils:

  • UL = PIT+YAW
  • LL = -PIT+YAW
  • UR = PIT-YAW
  • LR = -PIT-YAW

These channels are the direct input for the physical output channels that generate the control voltage.

The fast channels for PIT and YAW have a numerical correction factor built in that accounts for differences between the OSEMs, but the slow channels don't. This means that the slow PIT/YAW controls are not entirely orthogonal but have crosstalk on the order of 10 percent. This in itself is not that dramatic, however the offload offsets scripts for the dither alignment use the fast PIT/YAW values as inputs, which represent the necessary adjustments to the OSEMs only after the individual correction factors have been applied. The offloading to slow knows nothing of this calibration difference between the OSEMs. The result is that there is a ~10 percent of the offset correction error on the mirror alignment AFTER offloading. This will of course converge after a few iterations, but in any case it is recommendable to run the dither alignment again after offloading and not offload the new offsets to the fast channels.

  13476   Thu Dec 14 19:33:20 2017 gautamFrogsASSc1ass slow channel offloading scripts with small

I don't think this is really a problem - we offload to the fast channels and not to the slow (although we really should offload to the slow channels). I think the best approach is to use the ezcaservo utility to offload the DC part of the ASS control signals to the slow channels, so as to not waste fast channel DAC counts on DC offsets. In principle, this approach should be somewhat immune to the slow channel calibration not being perfect.

Quote:

While staring at epics records all day I noticed something about the PIT/YAW offset sliders and ASS offset offloading to slow channels scripts that I'm not sure others are aware off, so I'll briefly discuss it in this post.

The PIT and YAW sliders directly control soft channels that are hosted on the slow machine. Secondary epics records disentangle them for the individual coils:

  • UL = PIT+YAW
  • LL = -PIT+YAW
  • UR = PIT-YAW
  • LR = -PIT-YAW

These channels are the direct input for the physical output channels that generate the control voltage.

The fast channels for PIT and YAW have a numerical correction factor built in that accounts for differences between the OSEMs, but the slow channels don't. This means that the slow PIT/YAW controls are not entirely orthogonal but have crosstalk on the order of 10 percent. This in itself is not that dramatic, however the offload offsets scripts for the dither alignment use the fast PIT/YAW values as inputs, which represent the necessary adjustments to the OSEMs only after the individual correction factors have been applied. The offloading to slow knows nothing of this calibration difference between the OSEMs. The result is that there is a ~10 percent of the offset correction error on the mirror alignment AFTER offloading. This will of course converge after a few iterations, but in any case it is recommendable to run the dither alignment again after offloading and not offload the new offsets to the fast channels.

 

  13910   Fri Jun 1 21:47:23 2018 KojiFrogsGeneralTouch screen manipulation of the IFO

[Koji Gautam]

We talked about touch interface of medm. We realized that android (and iOS) has vnc clients. I just installed VNC viewer on my phone and connected to my mac. Typing is tricky but I managed to get into pianosa, then launched sitemap. We could unlock/lock the IMC by screen touch!

Basically we can connect to one of the laptops (or control machines) from a tablet (either android or ipad). It'd be better to put both in a same network. It'd be great if we have a tablet case with a keyboard so that we can type without blocking the screen.

  14186   Tue Aug 28 15:29:19 2018 SteveFrogsPEMRat is cut

The rat is cut by mechanical trap and it was removed from ITMX south west location.

A nagy kover patkanyt a fogo elkapta es megolte.

  14336   Fri Dec 7 19:42:47 2018 ranaFrogselogcan't upgrade DokuWiki because of PHP / SL7

All of our wikis (except the 40m one which unfortunately got turned into ligo.org mess) use DokuWiki. This now has an auto-upgrade feature through the Admin web interface.

I tried this recently and it fails with this message:

DokuWiki 2018-04-22a "Greebo" is available for download.
 You're currently running DokuWiki Release 2017-02-19e "Frusterick Manners".
! New DokuWiki releases need at least PHP 5.6, but you're running 5.4.16. You should upgrade your PHP version before upgrading!

So we'll have to wait until SL7 (which is what NODUS is running).

I DID do a 'yum upgrade' which updated all the packages. I also installed yum-cron so that the RPM listings get updated daily. But sadly, SL7 only has PHP 5.4.16 (which is a June 2013 release):

> Package php-5.4.16-43.el7_4.1.x86_64 already installed and latest version

  14545   Mon Apr 15 22:55:34 2019 gautamFrogsThermal CompensationLab thermostat adjusted

It is feeling cold in the office area. According to the digital wall clock near the coffee machine, it is 19C. Rana bumped the thermostat setpoint up by 2F (from 75F to 77F). We need to setup long-term monitoring.

  15081   Fri Dec 6 15:22:01 2019 gautamFrogsLSCDAFI system revived

[Jordan, gautam]

We did the following:

  • Route the fiber from the control room to 1Y2.
  • Plug fiber in to FiBox at either end, turned FiBoxes ON.
  • Tested the optical connection by driving a 1Vpp 440 Hz sine wave from a function generator - Yehonathan hears it loud and clear in the control room.
  • Tested that both CH1 and CH2 work - only CH1 is connected to the speakers in the control room at the moment.
  • There is some cross-coupling between the channels - not sure if this is happening in the multi-mode fiber or in the electroncis, but I estimate the isolation to be >30dB.
  • Connected CH8 and CH9 of DAC0 in the c1lsc expansion chassis to CH1 and CH2 respectively of the FiBox in 1Y2. 
  • Restarted the c1daf model on c1lsc, came up smooth.
  • Routed the POY11 error signal through the various matrices in c1daf, and we could 👂 the Y-arm cavity 🔐 😎 
  • Channels are muted for now - I'll give this a whirl while doing the PRFPMI locking.
  15162   Tue Jan 28 08:26:53 2020 ranaFrogsPEMshaking

https://breakthrough.caltech.edu/magazine/2019-aug/#article-Listening-with-Light

  15882   Mon Mar 8 20:11:51 2021 ranaFrogsComputer Scripts / Programsactivate_matlab out of control on Megatron

there were a zillion processes trying to activate (this is the initial activation after the initial installation) matlab 2015b on megatron, so I killed them all. Was someone logged in to megatron and trying to run matlab sometime in 2020? If so, speak now, or I will send the out-of-control process brute squad after you!

  16325   Tue Sep 14 15:57:05 2021 jamieFrogsCDSfb1 /var full after reboot, caused all sorts of problems

/var on fb1 filled up today, which caused all sorts of CDS issues.  I found out about the problem by reading the logs of the services that were having trouble running, in which they complained about not being able to write to disk.  I looked at the filesystem status with 'df' and noticed that /var was full, which is where applications write temporary data, and will always cause problems if it's full.

I tracked the issue down to multiple multi-gigabyte log files: /var/log/messages and /var/log/messages.1.  They were full of lines like this one:

Aug 29 06:25:21 fb1 kernel: l called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl called cmd = 1gpstime iotcl ca

Seems like something related to the gpstime kernel module?

Anyway, I deleted the log files for now, which cleared up the space on /var.  Things should be back to normal now, until the logs fill up again...

  16327   Tue Sep 14 16:44:54 2021 jamieFrogsCDSfb1 /var full after reboot, caused all sorts of problems

Jonathan Hanks pointed me to this fix to the gpstime kernel module that was unfortunately put in after the 3.4 release that we're currently using:

https://git.ligo.org/cds/advligorts/-/commit/6f6d6e2eb1d3355d0cbfe9fe31ea3b59af1e7348

I hacked the source in place (/usr/src/gpstime-3.4/drv/gpstime/gpstime.c) to get the fix, and then rebuilt the kernel module with dkms :

sudo dkms uninstall gpstime/3.4
sudo dkms install gpstime/3.4

I then stopped daqd_dc, unloaded gpstime, reloaded it, restarted daqd_dc.  The messages are no longer showing up in /var/log/messages, so I think we're ok for the moment.

NOTE: the fix will be undone if we for some reason reinstall the advligorts-gpstime-dkms package.  There shouldn't be a need to do that, but we should be aware.  I'm discussing with Jonathan if we want to try to push out a new debian package to fix this issue...

  16339   Thu Sep 16 14:08:14 2021 Ian MacMillanFrogs Tour

I gave some of the data analysts a look around because they asked and nothing was currently going on in the 40m. Nothing was changed.

  17261   Fri Nov 11 20:01:56 2022 ranaFrogsComputer Scripts / ProgramsFSS SLOW servo not running

I was trying to debug why the NPRO PZT is all over the place, and it turns out that the new FSS SLOW script is not actually running.

The BLINKY is blinking, but the script is not running. I wasn't able to figure out how to kill the broken Docker thing, but if the code reports that its running but actually does not, we should probably just put back the old perl or python script that ran before. I don't know how to debug this current issue, but the IMC locks will be limited in length due to this servo being broken. Whoever knows about this, please stop that Docker PID and we can just run the old python script on megatron.

I also tried to post a trend plot, but the minute trends don't yet reach the current date (!!!). They seem to have stopped recording a few days ago, so I guess the Framebuilder still needs some help or its tough to figure out things like when exactly the new SLOW servo stopped working.

  17262   Fri Nov 11 20:59:13 2022 ChrisFrogsComputer Scripts / ProgramsFSS SLOW servo not running

The problem with trends was due to the epics data collection process (standalone_edc) that runs on c1sus. When all the FEs were rebooted earlier this week, this process was started automatically, but for some reason it hasn’t been doing its thing and sending epics data to the framebuilder. I restarted it just now, and it’s working again. Until this problem is sorted out, we need to remember to check on this process after rebooting c1sus.

Quote:

I also tried to post a trend plot, but the minute trends don't yet reach the current date (!!!). They seem to have stopped recording a few days ago, so I guess the Framebuilder still needs some help or its tough to figure out things like when exactly the new SLOW servo stopped working.

  17263   Sat Nov 12 21:59:24 2022 AnchalFrogsComputer Scripts / ProgramsFSS SLOW servo not running

I stopped the Docker PID script and started the old python script on megatron. Instructions on how to do this are here.

On optimus I ran:

sudo docker stop scripts_PID_FSS_Slow_1

On megatron I ran:

sudo systemctl enable FSSSlow
sudo systemctl restart FSSSlow

However, the daemon service keeps failing and restarting. So currently the FSSSlow is not running. I do not know how to debug this script.


On a side note, I tested the docker service by restarting it, and it was working. From the logs, it seems like it got stuck because it could not find C1:IOO-MC_LOCK channel which occurs when c1psl epics servers fail or get stuck. The blinker on this script runs when the script is running but it does not stop if the script gets stuck somewhere. If someone decides to use this script in the future, they would need to correct error catching so that no reply from caget looks like an error and the script restarts rather than keep trying to get the channel value. Or the blinker implementation should change in the script so that it displays a stuck state.

Quote:
 

 Whoever knows about this, please stop that Docker PID and we can just run the old python script on megatron.

 

  17281   Thu Nov 17 16:48:07 2022 AnchalFrogsComputer Scripts / ProgramsFSS SLOW servo running Now

I've moved the FSS Slow PID script running to megatron through systemd daemons. The script is working as expected right now. I've updated megatron motd and the always running scripts page here.

  94   Mon Nov 12 14:09:19 2007 robDAQGeneraltpman dead on fb40m
The testpoint manager was dead on fb40m. I know I re-started it sometime after the power outage, so something must have killed it. If you get an error from DTT like
"diagnostic kernel does not support: testpoints", then log into fb40m as root, check for the tpman with a ps -ef | grep tpman. If it's not there, then run /usr/controls/tpman & and close the terminal window.
  152   Fri Nov 30 21:27:24 2007 ranaDAQPEMweather / stacis / c1pem1
I was trying to add some Seis BLRMS channels to the c1pem1 processor so that we could have DMT trends.

Then I found that none of the Weather channels have been working for a year or so. I could also not
telnet into it. I tried resetting it but no luck. There was no entry in the Wiki for it so I added
a place holder.

Have the weather channels ever worked? Do we have those sensors? I think I've never actually looked
for this. Seems like a fine ugrad job.
  157   Mon Dec 3 00:10:42 2007 ranaDAQComputer Scripts / Programslinemon
I've started up one of our first Matlab based DMT processes as a test.

There's a matlab script running on Mafalda which is measuring the height of the 60 Hz peak
in the MC1 UL SENSOR and writing it to an unused EPICS channel (PZT1_PIT_OFFSET).

The purpose of this is just to see if such a thing is stable over long periods of time. Its
open on a terminal on linux3 so it can be killed at any time if it runs amok.

Right now the code just demods the channel and tracks the absolute value of the peak. The
next upgrade will have it track the actual frequency once per minute and then report that
as well. We also have to figure out how to make it a binary and then make a single script
that launches all of the binaries.

For now you can watch its progress on the StripTool on op540m; its cheap and easy DMT viewer.
  160   Mon Dec 3 19:06:49 2007 ranaDAQComputer Scripts / Programslinemon
I turned up my nose at Matlab's special tools. I modified the linetracker to use the
relationship phase = 2*pi*f*t to estimate the frequency each minute. The
code uses 'polyfit' to get the mean and trend of the unwrapped phase and then determines
how far the initial frequency estimate was off. It then uses the updated number as the
initial guess for the next minute.

I looked at a couple hours of data before letting it run. It looks like the phase of the
'60 Hz' peak varies at 20 second time scales but not much faster or rather anything faster
would be a glitch and not a monotonic frequency drift.

From the attached snapshot you can see that the amplitude (PZT1_PIT) varies by ~10 %
and the frequency by ~40 mHz in a couple hour span.
  170   Wed Dec 5 19:25:07 2007 ranaDAQCDSDMF
I made a database file on C1AUX called dmf.db. It has 9 DMF EPICS channels which are also trended
so that one can now write data to those channels from a DMF Monitor and the data will be records.

New channels:
[C1:DMF-SEIS_1]
[C1:DMF-SEIS_2]
[C1:DMF-SEIS_3]
[C1:DMF-LINE_1]
[C1:DMF-LINE_2]
[C1:DMF-LINE_3]
[C1:DMF-MC_1]
[C1:DMF-MC_2]
[C1:DMF-MC_3]

I added these to C1AUX because it doesn't do much and can be booted without having much effect.
(it controls Mech Shutters, Video, and Illuminators. It used to also do the EO Shutter but I
removed that from its startup.cmd and it will no longer load those records).
  204   Wed Dec 19 20:28:27 2007 AndreyDAQPEMNames for all 6 accelerometers have been changed

I eventually changed the names for all 6 accelerometers (see my ELOG entry # 172 from Dec. 05 about my intent to do that).

I removed the word "BS" from their names,
and I changed the word combination "ACC_BS_EAST" in the old name for "ACC_ITMX" in the new name;
as well "ACC_BS_WEST" is now replaced by "ACC_ETMX".
(the reasoning behind such a change should become clear from my ELOG entry #172).

New accelerometer names are:
(note: there are no spaces (nowhere!) in the names of accelerometers, but ELOG replaces ": P" written without a space by a strange symbol Tongue)

C1 : PEM - ACC _ ETMX _ X ;
C1 : PEM - ACC _ ETMX _ Y ;
C1 : PEM - ACC _ ETMX _ Z ;
C1 : PEM - ACC _ ITMX _ X ;
C1 : PEM - ACC _ ITMX _ Y ;
C1 : PEM - ACC _ ITMX _ Z .

One can find them in "C1 : PEM - ACC" in Dataviewer.

  312   Tue Feb 12 16:34:07 2008 robDAQDMFseisBLRMS 1.1
The compiled version of seisBLRMS had been running ~2 weeks without crashing as of last night, when I killed it
so it wouldn't interfere with alignment scripts.  I added an EPICS channel C1:DMF-ENABLE, and updated the DMF
executables to check this channel while running.  So far it seems to work.  When you're running alignment acripts,
simply click the DISABLE button on the C1DMF_MASTER.adl screen, and then re-ENABLE when the scripts finish. 
It's not clear why this is necessary though.  Theories include the constant disk access is keeping the
framebuilder busy, reducing its ability to deal with ezcademod commands and DMF programs just flooding the
network with so much traffic that ezcademod-related packets run late and get ignored.

Also, for reasons of aesthetics, I changed the data delay from 6 minutes to 5 minutes.  We'll see if that's enough.
  316   Thu Feb 14 15:04:53 2008 robDAQDMFDMF delay

Sometime ago I edited seisBLRMS to keep of track of how long it was taking to write RMS data (that is, the delay between the accelerometer data and the write of the EPICS rms data). Here's a plot of that info, showing how the delay increases over time. I think this indicates a logical flaw in the timing of the seisBLRMS program, which sort of relies on everything running well consistently; this should not be difficult to fix. I'll maybe try increasing the delay to ~10 minutes, and making it relatively inflexible.
  317   Thu Feb 14 15:05:18 2008 robDAQDMFseisBLRMS 1.1
> 
> Also, for reasons of aesthetics, I changed the data delay from 6 minutes to 5 minutes.  We'll see if that's enough.

5 minutes didn't work.  
  358   Tue Mar 4 23:22:32 2008 robDAQComputersc1susvme1&2 rebooted

I found that some channels from c1susvme1 and c1susvme2 were not being recording by the DAQ (and were not showing up in DV). I rebooted these processors, which fix the problem. If you see other cases of this (signal exactly zero, but not a testpoint problem), just reboot the corresponding processor.
  440   Wed Apr 23 22:39:54 2008 AndreyDAQComputer Scripts / ProgramsProblem with "get_data" and slow PEM channels

It turns out that I cannot read minute trends for the slow weather channels for more than 1000 seconds back (roughly more than 15 minutes ago) using "get_data" script.

For comparison, I tried MC1 slow channels, and similar problem did not arise there. Probably, something is wrong with the memory of slow weather channels. At the same time, I can see minute-trends in Dataviewer as long ago as I want.

In response to
>>get_data('C1: PEM-weather_outsideTemp', 'minute', gps('now') - 3690, 3600);
I get the error message:
"Warning: Missong C1: PEM-weather_outsideTemp M data at 893045156".
  456   Sun Apr 27 18:11:58 2008 robDAQComputersbr40m?

The testpoint manager (which runs on fb40m) crashed this afternoon. Upon re-starting it, I found there was a rogue dtt process on op440m and also a daqd daemon running on br40m. One or both of these caused the tpman to crash. br40m is the frame broadcaster, which is never used here as we don't run DMT. I killed the daqd process there.

The way to find if there is a rogue process is to watch the output to the console from the tpman when you start it:

Allocate new TP handle 56 by 131.215.113.203
Allocate new TP handle 57 by 131.215.113.203
Allocate new TP handle 58 by 131.215.113.203
Allocate new TP handle 59 by 131.215.113.203
Allocate new TP handle 60 by 131.215.113.203
Allocate new TP handle 61 by 131.215.113.203
Allocate new TP handle 62 by 131.215.113.203
Allocate new TP handle 63 by 131.215.113.203
Allocate new TP handle 64 by 131.215.113.203
Allocate new TP handle 65 by 131.215.113.203
Allocate new TP handle 66 by 131.215.113.203
Allocate new TP handle 67 by 131.215.113.203
Allocate new TP handle 68 by 131.215.113.203


If you see something like this, with a new TP handle being allocated every few seconds, you need to log in to the corresponding host and kill whatever process has run away.
  457   Sun Apr 27 22:57:15 2008 ajwDAQComputersbr40m?

Quote:

The testpoint manager (which runs on fb40m) crashed this afternoon. Upon re-starting it, I found there was a rogue dtt process on op440m and also a daqd daemon running on br40m. One or both of these caused the tpman to crash. br40m is the frame broadcaster, which is never used here as we don't run DMT. I killed the daqd process there.

The way to find if there is a rogue process is to watch the output to the console from the tpman when you start it:

Allocate new TP handle 56 by 131.215.113.203
Allocate new TP handle 57 by 131.215.113.203
Allocate new TP handle 58 by 131.215.113.203
Allocate new TP handle 59 by 131.215.113.203
Allocate new TP handle 60 by 131.215.113.203
Allocate new TP handle 61 by 131.215.113.203
Allocate new TP handle 62 by 131.215.113.203
Allocate new TP handle 63 by 131.215.113.203
Allocate new TP handle 64 by 131.215.113.203
Allocate new TP handle 65 by 131.215.113.203
Allocate new TP handle 66 by 131.215.113.203
Allocate new TP handle 67 by 131.215.113.203
Allocate new TP handle 68 by 131.215.113.203


If you see something like this, with a new TP handle being allocated every few seconds, you need to log in to the corresponding host and kill whatever process has run away.


I *think* Alex is responsible for the daqd daemon running on br40m (he set up some new stuff recently, a data concentrator and broadcaster); I'll make sure he sees this post.
  459   Tue Apr 29 21:09:12 2008 ranaDAQCDSFE Filters
These are new FE filters for downsampling and upsampling. We will be going from native hardware sampling rates of 64k down to 32k, 16k, and 2k.

The attached plot shows these filters. They are 3dB ripple, 40 dB stopband, 4th order elliptic filters in which I have moved the zeros around
into good places (e.g. to the Nyquist frequency).

I'm also attaching the .txt file containg the filter coefficients and the design strings. The filters are called x2, x4, and x32, for the
D2, D4, and D32 downsampling, respectively.
  583   Fri Jun 27 15:20:52 2008 robDAQLSC.ini file change

I removed C1:LSC-XARM_CTRL from the frames and added C1:LSC-CARM_ERR
  640   Mon Jul 7 13:58:37 2008 Eric, josephbDAQPEMUsing unused PEM channels to test camera code
Joe and I have taken control of the EPICS channels C1:PEM-Stacis_EEEX_geo and C1:PEM-Stacis_EEEY_geo since we heard that they are no longer in use.  We are currently 
using them to test the ability for the Snap camera code to read and write from EPICS channels.  Thus, the information being written to these channels is completely unrelated
to their names or previous use.  This is only temporary; we'll create our own channels for the camera code shortly (probably within the next couple of days).

- Eric
ELOG V3.1.3-