40m QIL Cryo_Lab CTN SUS_Lab TCS_Lab OMC_Lab CRIME_Lab FEA ENG_Labs OptContFac Mariner WBEEShop
  40m Log  Not logged in ELOG logo
Message ID: 8431     Entry time: Tue Apr 9 14:55:13 2013
Author: Jamie 
Type: Update 
Category: CDS 
Subject: overbooked test points cause of DAQ problems 

Folks were complaining that they were getting zeros whenever they tried to open fast channels in DTT or Dataviewer.  It turned out that the problem was that all available test points were in use in the c1lsc model:

lsc-gds.png

There is a limit to how many test points can be open to a single model (in point of fact I think the limit is on the data rate from the model to the frame builder, not the actual number of open test points).  In any event, they was all used up.  The grid at the bottom right of the C1LSC GDS screen was all full of non-zeros, and the FE TRATE number was red, indicating that the data rate from this model had surpassed threshold.

The result of this overbooking is that any new test points just get zeros.  This is a pretty dumb failure mode (ideally one would not be able to request the TP at all with an appropriate error message), but it is what it is.  This usually means that there are too many dtt/dataviewers left with open connections.

We tried killing all the open processes that we could find that might be holding open test points, but that didn't seem to clear them up.  Stuck open test points is another known problem.  Referencing the solution in #6968 I opened the diag shell and killed all test points everywhere:

controls@pianosa:~ 0$ diag -l -z
Set new test FFT
NDS version = 12
supported capabilities: testing  testpoints  awg  
diag> tp clear * *
test point cleared
diag> quit
EXIT KERNEL
controls@pianosa:~ 0$
ELOG V3.1.3-