I tried to figure out why offline LMS filter subtract seismic noise much better from MC_F then the Wiener filter. I did the calculations twice - with my codes and with Matlab in-build functions, the results are the same. So this is not a code error.
The coherence between GUR 1, 2 and MC_F is still poor. Wiener filter is linear and its performance is confined to the frequency ranges where we see coherence. Lms filter is non-linear and it may be possible to subtract the noise even if non-linear effects are present in the system.
I've checked seismometer readout box again. I've soldered 50 Ohms to plus and minus inputs to VERT 1,2 N/S 1,2, E/W 1,2 - GUR 1 and 2 use these channels. Then I put the box back and connected it to the ADC.
The plot shows that the readout box noise is below the ADC noise. It is possible that amplifiers introduce non-linear effects. To check this I plotted the coherence between OSEM sensors and GUR1X signal:
The coherence between OSEM sensors and GUR1X is pretty good, so may be witness path is not responsible for low coherence at 0.1 - 0.5 Hz between MC_F and GUR 1,2. IT seems that MC_F is bad at low frequencies. I terminated the input to the Channel 1 of the Pentek Generic board, where MC_F is plugged in.
ADC is also good. Something else is wrong.