Summary:
I've been trying to put together the costfunction that will be used to optimize the Oplev loop shape. Here is what I have so far.
Details:
All of the terms that we want to include in the cost function can be derived from:
 A measurement of the openloop error signal [using DTT, calibrated to urad/rtHz]. We may want a breakdown of this in terms of "sensing noises" and "disturbances" (see the previous elog in this thread), but just a spectrum will suffice for the optimal controller given the current noises.
 A model of the optical plant, P(s) [validated with a DTT sweptsine measurement].
 A model of the controller, C(s). Some/all of the poles and zeros of this transfer function is what the optimization algorithm will tune to satisfy the design objectives.
From these, we can derive, for a given controller, C(s):
 Closedloop stability (i.e. all poles should be in the lefthalf of the complex plane), and exactly 2 UGFs. We can use MATLAB's allmargin function for this. An unstable controller can be rejected by assigning it an extremely high cost.
 RMS rrror signal suppression in the frequency band (0.5Hz  2Hz). We can require this to be >= 15dB (say).
 Minimize gain peaking and noise injection  this information will be in the sensitivity function, . We can require this to be <= 10dB (say).
 RMS of the control signal between 10 Hz and 200 Hz, multiplied by the digital suspension whitening filter, should be <10% of the DAC range (so that we don't have problems engaging the coil dewhitening).
 Smallest gain margin (there will be multiple because of the various notches we have) should be > 10dB (say). Phase margin at both UGFs should be >30 degrees.
 Terms 15 should not change by more than 10% for perturbations in the plant model parameters (f0 and Q of the pendulum) at the 10% (?) level.
We can add more terms to the cost function if necessary, but I want to get some minimal set working first. All the "requirements" I've quoted above are just numbers out of my head at the moment, I will refine them once I get some feeling for how feasible a solution is for these requirements.
Quote: 
An elog with a first pass at a mathematical formulation of the costfunction for controller optimization to follow shortly.

For a start, I attempted to model the current Oplev loop. The modeling of the plant and openloop error signal spectrum have been described in the previous elogs in this thread.
I am, however, confused by the controller  the MEDM screen (see Attachment #2) would have me believe that the digital transfer function is FM2*FM5*FM7*FM8*gain(10). However, I get much better agreement between the measured and modelled inloop error signal if I exclude the overall gain of 10 (see Attachments #1 for the models and #3 for measurements).
What am I missing? Getting this right will be important in specifying Term #4 in the cost function...
GV Edit 2 Aug 0030: As another sanity check, I computed the whitened Oplev control signal given the current loop shape (with suboptimal highfrequency rolloff). In Attachment #4, I converted the yaxis from urad/rtHz to cts/rtHz using the approximate calibration of 240urad/ct (and the fact that the Oplev error signal is normalized by the QPD sum of ~13000 cts), and divided by 4 to account for the fact that the control signal is sent to 4 coils. It is clear that attempting to whiten the coil driver signals with the present Oplev loop shapes causes DAC saturation. I'm going to use this formulation for Term #4 in the cost function, and to solve a simpler optimization problem first  given the existing loop shape, what is the optimal elliptic lowpass filter to implement such that the cost function is minimized?
There is also the question of how to go about doing the optimization, given that our cost function is a vector rather than a scalar. In the coating optimization code, we converted the vector cost function to a scalar one by taking a weighted sum of the individual components. This worked adequately well.
But there are techniques for vector costfunction optimization as well, which may work better. Specifically, the question is if we can find the (infinite) solution set for which no one term in the error function can be made better without making another worse (the socalled Pareto front). Then we still have to make a choice as to which point along this curve we want to operate at. 