D = 2 d Tan(\phi) Cos(\theta), where \phi = ArcSin(Sin(\theta) * n)
\theta is the angle of incidence. For a small \theta, D is propotional to \theta.
n1Sin(\theta1) = n2 Sin(\theta2)
So it should be
\phi = ArcSin(Sin(\theta) / n
I did check the reflected images for larger angles of incidence, about 20 deg and visibly (on the IR card) I did not see much change in the separation. But I will check it with the camera again to confirm on that.