It turns out that I had to do some clean-up of my NN code:
1) The SNRs were wrong. The problem is that after summing all kinds of seismic waves and modes, the total field should have a certain spectral density, which is specified by the user. Now the code works and the seismic field has the correct spectral density no matter how you construct it.
2) I started with a pretty unrealistic scenario. The noise on the test mass, and by this I mean everything but the NN, was too strong. Since this is a simulation of NN subtraction, we should rather assume that NN is much stronger than anything else.
3) I filtered out the wrong kind of NN. I am now projecting NN onto the direction of the arm, and then I let the filter try to subtract it. It turns out, and it is fairly easy to prove this with paper and pencil, that a single seismometer CANNOT never ever be used to subtract NN. This is because of a phase-offset between the seismic displacement at the origin and NN at the origin. It is easy to show that the single-seismometer method only works for the vertical NN or underground for body waves.
This plot is just the prove for the phase-offset between horizontal NN and gnd displacement at origin. The offset is depends on the wave content of the seismic field:
The S0 points in the following plot are now obsolete. As you can see, the Wiener filter performs excellently now because of the high NN/rest ratio of TM dispalcement. The numbers in the titel now tell you how much NN power is subtracted. So a '1' is pretty nice...
One question is why the filter performance varies from simulation to simulation. Can't we guarantee that the filter always works? Yes we can. One just needs to understand that the plot shows the subtraction efficiency. Now it can happen that a seismic field does not produce much NN, and then we don't need to subtract much. Let's check if the filter performance somehow correlates with NN amplitude:
As you can see, it seems like most of the performance variation can be explained by a changing amplitude of the NN itself. The filter cannot subtract much only in cases when you don't really need to subtract. And it subtracts nicely when NN is strong.