As a test, I did a remote reboot of both Megatron and c1iscex, to make sure there was no code running that might interfere with the dataviewer.  Megatron is behind a firewall, so I don't see how it could be interfering with the frame builder.  c1iscex was only running a test module from earlier today when I was testing the multi-filter matrix part.  No daqd or similar processes were running on this machine either, but it is not behind a firewall at the moment. 

Neither of these seemed to affect the lack of past data.  I note the error message from dataviewer was "read(); errno=9".

Going to the frame builder machine, I ran dmesg.  I get some disturbing messages from May 26th and June 7th. There are 6-7 of these pairs of lines for each of these days, spread over the course of about 30 minutes.

Jun 7 14:05:09 fb ufs: [ID 213553 kern.notice] NOTICE: realloccg /: file system full

Jun 7 14:11:14 fb last message repeated 19 times

There's also one:

Jun 7 13:35:14 fb syslogd: /usr/controls/main_daqd.log: No space left on device

I went to /usr/controls/ and looked at the file.  I couldn't read it with less, it errored with Value too large for defined data type.  Turns out the file was 2.3 G.  And had not been updated since June 7th.  There were also a bunch of core dump files from May 25th, and a few more recent.  However the ones from May 25th were somewhat large, half a gig each or so.  I decided to delete the main_daqd.log file as well as the core files.

This seems to have fixed the data history for the moment (at least with one 16k channel I tested quickly). However, I'm now investigating why that log file seems to have filled up, and see if we can prevent this in the future.