Summary:

From the measurements I have, the Y arm loss is estimated to be 58 +/- 12 ppm. The quoted values are the median (50th percentile) and the distance to the 25th and 75th quantiles. This is significantly worse than the ~25 ppm number Johannes had determined. The data quality is questionable, so I would want to get some better data and run it through this machinery and see what number that yields. I'll try and systematically fix the ASS tomorrow and give it another shot.

Model and analysis framework:

Johannes and I have cleaned up the equations used for this calculation - while we may make more edits, the v1 of the document lives here. The crux of it is that we would like to measure the quantity , where  is the power reflected from the resonant cavity (just the ITM). This quantity can then be used to back out the round-trip loss in the resonant cavity, with further model parameters which are:

1. ITM and ETM power transmissivities
2. Modulation depths and mode-matching efficiency into the cavity
3. The statistical uncertainty on the measurement of the quantity , call it 

If we ignore the 3rd for a start, we can calculate the "expected" value of  as a function of the round-trip loss, for some assumed uncertainties on the above-mentioned model parameters. This is shown in the top plot in Attachment #1, and while this was generated using emcee, is consistent with the first order uncertainty propagation based result I posted in my previous elog on this subject. The actual samples of the model parameters used to generate these curves are shown in the bottom. What this is telling us is that even if we have no measurement uncertainty on , the systematic uncertainties are of the order of 5 ppm, for the assumed variation in model parameters.

The same machinery can be run backwards - assuming we have multiple measurements of , we then also have a sample variance, . The uncertainty on the sample variance estimator is also known, and serves to quantify the prior distribution on the parameter  for our Monte-Carlo sampling. The parameter  itself is required to quantify the likelihood of a given set of model parameters, given our measurement. For the measurements I did this week, my best estimate of . Plugging this in, and assuming uncorrelated gaussian uncertainties on the model parameters, I can back out the posterior distributions.

For convenience, I separate the parameters into two groups - (i) All the model parameters excluding the RT loss, and (ii) the RT loss. Attachment #2 and Attachment #3 show the priors (orange) and posteriors (black) of these quantities. 

Interpretations:

1. This particular technique only gives us information about the RT loss - much less so about the other model parameters. This can be seen by the fact that the posteriors for the loss is significantly different from the prior for the loss, but not for the other parameters. Potentially, the power of the technique is improved if we throw other measurements at it, like ringdowns.
2. If we want to reach the 5 ppm uncertainty target, we need to do better both on the measurement of the DC reflection signals, and also narrow down the uncertainties on the other model parameters.

Some assumptions:

So that the experts on MC analysis can correct me wheere I'm wrong.

1. The prior distributions are truncated independent Gaussians - truncated to avoid sampling from unphysical regions (e.g. negative ITM transmission). I've not enforced the truncation analytically - i.e. I just assume a -infinity probability to samples drawn from the unphysical parts, but to be completely sure, the actual cavity equations enforce physicality independently (i.e. the MC generates a set of parameters which is input to another function, which checks for the feasibility before making an evaluation). One could argue that the priors on some of these should be different - e.g. uniform PDF for loss between some bounds? Jeffrey's prior for ?
2. How reasonable is it to assume the model parameter uncertainties are uncorrelated? For exaple,  are all determined from the ALS-controlled cavity scan.