If the problem is the precision in DTT, then why would the noise change when the corner frequency of the filter is changed?
And how about checking for filter noise in the situation we saw online? 4th order low pass at 1 Hz or 8 Hz.
This is because when we plot signals with sampling frequencies 2k and 16k with the same BW, we actually create psd/coherence using different numbers of points in FFT calculations as NFFT ~ fs/bw, fs-sampling frequency. So we secretly used 8 times more fft points for 16k signal then for 2k signal. Following plots demonstrate this effect. The first plot shows transfer function and coherence for filtering of 16k signal with butter('LowPass',4,8) and 2k signal with butter('LowPass',4,1) when BW=0.1. There is a disturbance in coherence for 2k signal below 2 Hz. Now let's take BW=0.8 and plot transfer function and coherence for 16k signal. We can see the same effect of coherence disturbance.
The similar effect takes place when we change the cut-off frequency. The following plots show transfer function and coherence of two pairs of 2kHz signals. 4 order butterworth low-pass filter was used. For the first pair cut-off frequency was 1 Hz, for the second 10 Hz. On the first plot BW=0.1 and there is a disturbance in coherence below 1 Hz. However on the second plot when BW=0.01, this effect is lost.
I guess my goal is to figure out when these effects come from fft calculations and when from digital filter noise.