Something funny is going on with the framebuilder's communication with the LSC machine.
This is a different failure mode / error than I have seen before. It's not the type of problem that is solved by restarting the mxstreams (that is indicated by also the 2 blocks on top of one another, that are green on the lsc machine right now, being red), although I did try that, before I looked closer and realized that that wasn't the problem.
ssh-ing to c1lsc, and doing a "rtcds restart all" seems to be fixing the problem. Both c1oaf and c1cal needed another round of restarting, because they needed their BURT buttons pressed manually. All of the models on the lsc machine are running fine now, though.
Here's a screenshot of the CDS overview screen, with the error lights:
This definitely looks like a timing problem on the c1lsc front end computer. The red lights on the left mean that the timing synchronization is lost at the user model. I'm perplexed why it looks like the IOP is not seeing the same error, though, since it should originate at the ADC. The red lights to the right just mean the timing synchronization is lost with the DAQ, which is too be expected given a timing loss at the front end.
We'll have to take a closer look when this happens again.