It looks as though we may have two IO chassis with bad timing cards.
Symptoms are as follows:
We can get our front end models writing data and timestamps out on the RFM network.
However, they get rejected on the receiving end because the timestamps don't match up with the receiving front end's timestamp. Once started, the system is consistently off by the same amount. Stopping the front end module on c1ioo and restarting it, generated a new consistent offset. Say off by 29,000 cycles in the first case and on restart we might be 11,000 cycles off. Essentially, on start up, the IOP isn't using the 1PPS signal to determine when to start counting.
We tried swapping the spare IO chassis (intended for the LSC) in ....
# Joe will finish this in 3 days.
# Basically, in conclusion, in a word, we found that c1ioo IO chassis is wrong.