I built a Simulink model of the magnetic levitation system and try to explain the dip in the open-loop transfer function that was observed.

One can download the model in the svn. The corresponding block diagram is shown by the figure below.

Here "Magnet" is equal to inverse of the magnet mass. Integrator "1/s" gives the velocity of the magnet. A further integrator gives the displacement of the magnet.

Different from the free-mass response, the response of the magnet is modified due to the existence of the Eddy-current damping  and negative spring in the vertical

direction, as indicated by the feedback loops after two integrals respectively. The motion of the magnet will change the magnetic field strength which in turn will pick

up by the Hall-effect sensor. Unlike the usual case, here the Hall sensor also picks up the magnetic field created by the coil as indicated by the loop below the mechanical

part. This is actually the origin of the dip in the open-loop transfer function. In the figure below, we show the open-loop transfer function and its phase contributed by both

the mechanical motion of the magnet and the Hall sensor with the black curve "Total". The contribution from the mechanical motion alone is shown by the magenta curve

"Mech" which is obtained by disconnecting the Hall sensor loop (I rescale the total gain to fit the measurement data due to uncertainties in those gains indicated in the figure). 

The contribution from the Hall sensor alone is indicated by the blue curve "Hall" which  is obtained by disconnecting the mechanical motion loop. Those two contributions

have the different sign as shown by the phase plot, and they destructively interfere with each other and create the dip in the open-loop transfer function.

In the following figure, we show the close-loop response function of the mechanical motion of the magnet.

As we can see, even though the entire close loop of the circuit is stable, the mechanical motion is unstable around 10 Hz. This simply comes from the fact that

around this frequency, the Hall sensor almost has no response to the mechanical motion due to destructive interference as mentioned.

In the future, we will replace the Hall sensor with an optical one to get rid of this undesired destructive interference.