40m QIL Cryo_Lab CTN SUS_Lab TCS_Lab OMC_Lab CRIME_Lab FEA ENG_Labs OptContFac Mariner WBEEShop
  40m Log  Not logged in ELOG logo
Message ID: 6046     Entry time: Tue Nov 29 22:54:49 2011
Author: Den 
Type: Update 
Category: digital noise 
Subject: single precision 

 In the previous elog we've proved that signals C1:SUS-MC3_LSC_EXCMON and  C1:SUS-MC3_LSC_OUTMON are in single precision. Now we try to understand if the precision is lost when the signals enter dtt tools or in the online code. For this measurement we just switch on one of the filters between the signals. By this we add mathematical operations inside the online code. If double precision is used there, then we'll see the same error as before, because the real error is still very small (~10-15) and dtt tools add this 10-7 error. But if the digital error will increase then no matter what kind of precision use dtt, online code uses single precision. At least at some points.

Test 1.   cheby1("LowPass",6,1,12) 

filter_coherence.jpg

 

Test 2.  cheby1("Lowpass",2,0.1,3)

chby1_coherence.jpg

 Test 3. butter("LowPass",8,10)

butter8_coherence.jpg

Test 4.  butter("LowPass",2,10)

butter2_coherence.jpg

We can see that coherence become worse. And longer filter - more digital error. This means that single precision is used in calculations.

 

ELOG V3.1.3-