I've calculated the predicted cavity power transmission and reflection coefficients using the T values that I measured the other day. I used an estimated scatter loss of 150ppm per mirror, which is reasonable according to Frank.

I measured the T of each cavity mirror using both S and P light, because I was concerned that there was some funny polarization rotation business going on inside our cavity. Just looking at the result for S, though, I'm pretty sure we can explain the extra light output in an easier way:

When we first calculated the power that should be coming out of the curved Y1S mirrors, we used the nominal Y1 transmission "spec" of 0.5% to calculate the circulating power in the cavity (using the power we measured at the output). We then multiplied this circulating power by the 50ppm spec for the Y1S. In talking to Frank, I learned that CVI gives the 99.5% R spec for the Y1 to cover their asses in case scatter loss is ridiculously high, and that its T is usually right around the 50-100ppm that I measured the other day. This means that the circulating power in the cavity was much higher than we thought, and so the light escaping the cavity through the curved mirrors was reasonable.

This result can be seen graphically above, as the transmission through the curved mirror(s) is of the same order as that through the output coupler. Needless to say, this is a bad impedance matching solution. What we want is to have most of our light escaping through the output (of course!), and a cavity that isn't dominated by losses. For example, if the transmission through the output coupler were truly 0.5%, we would have something that looks like this:

This shows even more clearly why we thought we had too much light coming out of the curved mirrors.

Side note: I think we may have been using an incorrect formula to calculate finesses in the past. From now on, I will use the proper definition

where rho is the fraction of the power remaining in the cavity after one round trip (with no pumping field), i.e., rho = 1 - total round-trip loss. The approximation is good for high finesse.