As discussed during the teleconference, a series of experiments have been conducted which attempt to measure the thermally induced defocus in the Hartmann sensor measurement. However, there was a limiting source of noise which caused a very large displacement of the centroids between images, making the images much too noisy to properly analyze.
The general setup of this series of experiments is as follows: the fiber output from the SLED was mounted about one meter away from the Hartmann sensor. No other optics were placed in the optical path. Everything except for the Hartmann sensor was enclosed (a box was constructed out of wall segments and posterboard, with a hole cut in the end which allowed the beam to propagate into the sensor. The sensor was a short distance from the end of the box, less than a centimeter. There was no obvious difference in test images taken with the lights on and the lights off, which previously suggested to me that ambient light would not have a large effect). Temperature variations in the sensor were induced by changing the set temperature of the lab with the thermostat. A python script was used to take cumulative sums of 200 images (taken at 11Hz) every ~5 minutes.
This overly large centroid displacement appeared only in certain areas of the images. However, changing the orientation of the plate appeared to change the regions which were noisy. That is, if the orientation of the Hartmann plate was not changed between measurements, the noise would appear in the same regions in consecutive experiments (even in experiments conducted on different days). However, if the orientation of the Hartmann plate was changed between measurements, the noise would appear in a different region in the next experiment. This suggests that the noise is perhaps due to a physical phenomenon which would change with the orientation of the plate.
There were a few hypotheses which attempted to explain this noise but were shown to not be the likely cause. I hypothesized that the large thermal expansion coefficient of the aluminum camera housing could be inducing a stress on the invar frontplate, causing the Hartmann plate to warp. This hypothesis was tested by loosening the screws which attach the front and back portion of the frontplate (such that the Hartmann plate was not strongly mechanically coupled with the rest of the frontplate) and running another iteration of the experiment. The noisy regions were seen to still appear, indicating that thermally induced stress was not the cause of the distortion. Furthermore, experiments done while the sensor was in relative thermal equilibrium over long periods still showed noisy regions, and there was no apparent correlation between noise magnitude and sensor temperature for any experiment, indicating that thermal effects in general were not responsible.
Another suspected cause was the increased noise at intensity levels of 128 (as discussed in a previous eLog). However, it was observed that there was no apparent difference in the prevalence of 128-count pixels between the noisy regions and the cleaner regions, indicating that this was not the cause either.
A video was made which shows vector plots of centroid displacements for each summed image relative to the first image taken in an experiment, and was posted as an unlisted youtube video at: http://www.youtube.com/watch?v=HUH1tHRr98I
The length of each vector in the video is proportional to the magnitude of the displacement. The localization of the noise can be seen. Notice also the sudden appearance and disappearance of the noise at images 19 and 33, indicating that the cause of the noise is relatively sudden and does not vary smoothly.
Another video showing a logarithmic plot of the absolute value of the difference of each image from the first image (for the same experiment as previous) can be seen here: http://www.youtube.com/watch?v=_CiaMpw9Ig0
Notice there are jumps in the background level which appear to correspond with the disappearance and appearance of the noisy regions in the centroids (at images 18 and 32) (I forgot to manually set the framerate on these last three .avi's, so they go by a little too quickly, but it's still all there). The one-image delay between the intensity shift and centroid noise shift is perhaps related to the fact that the analysis uses the previous image centroids as the reference to find the new image centroid locations.
A video showing histograms of the intensity of each pixel in an image (within the intensity range of 50 and 140 in the averaged summed-image) for this same experiment can be seen at: http://www.youtube.com/watch?v=MogPd-vaWn4
Notice that the peak of the distribution corresponding to the background appears to shift by ~5 counts at images 18 and 32.
An experiment was then done which had the exact same procedure except that it was done at a stabilized lab temperature and with the SLED turned off, such that only the background appears in each image. A logarithmic plot of the absolute value of the difference in intensity at each pixel for each image can be seen at: http://www.youtube.com/watch?v=Y66wL5usN18
Other work was being done in the lab throughout the day, so the lights were on for every image but one. I made a point of turning off the lights while the 38th image was being taken. The framerate of the linked video is unfortunately a little too fast to really see what goes on (I adjusted the framerate while viewing it in MATLAB but forgot to do so for the AVI), but you can clearly see a major change in the image during the 38th image, and during that image only (it looks like a red 'flash' at the 38th frame, near the very end). The only thing that was changed while taking this specific image was the ambient light level, so this major difference must be due to ambient light. A plot of the difference between images 38 and 1 is shown below:
Note that the maximum difference between the images is 1107 levels, which for the 200 images in each summed image corresponds to an average shift of ~5.5 levels. This is of a very similar magnitude to the shift that can be seen in the histogram of the previous experiment. This suggests that changes in ambient light levels are perhaps somehow responsible for the noisy regions of the image. Note also the non-uniformity of the ambient light; such a non-uniform change could certainly shift the centroid positions.
One question is how, exactly, this change might have propagated into the analysis. The shape of the background level change appears to be very different from the shape of the noisy regions seen for this plate configuration. This is something which I need to examine further; this, combined with the fact that the changes in the noise appear to occur one image after the actual change in intensities, suggests to me that there could perhaps be some subtle things going on with my data analysis procedures which I don't currently fully understand.
Still, I highly suspect that ambient light is the root cause of the noisy regions. It would be a remarkable coincidence if the centroid displacement shift was not ultimately due to the observed intensity shift, or if the intensity shift was not due to a change in ambient light (since the intensity shift in the histogram analysis and ambient light change in the background analysis are observed to correspond to roughly the same magnitude of intensity change). I had initially suspected that effects from ambient light would be negligible since, while taking test images while setting up each experiment, the image did not appear to change based upon whether I had the lights on or off. I checked this a few times, but did not examine the images closely enough to be able to detect such a small non-uniform change in the intensity of each image.
If ambient light was responsible, this could also perhaps explain why the location of the noise appeared to depend on the orientation of the plate. The Hartmann plate would be in the optical path of any ambient light leaking in, so a change in the orientation of the plate could perhaps change the way that the ambient light was propagating onto the sensor (especially since the Hartmann plates are slightly warped and not perfectly planar). That's all purely speculation at this point, but it's something that I intend to investigate further.
I tried analyzing some previous data by subtracting part of the background, but was unsuccessful at reducing the noise in the results. I attempted to reduce the background in previous data by setting all values below a certain threshold equal to zero (before inputting the image into the centroiding function). However, the maximum threshold which I could use before getting an error message was ~130. If I set the threshold to, say, 135, I received an error from the centroiding function that the image was 'too dissimilar to the hex grid'. I did analysis of the images with a threshold of 130, but this still left random patches of background spaced between the spots in each image. The presence of only patches of background as opposed to the complete background actually increased the level of noise in the results by about a factor of 3. I would need to come up with a better method of subtracting the background level if I wanted to actually reduce the noise in this data.
The next step in this work, I think, will perhaps be to better enclose the system from ambient light to where I'm confident that it could have little or no effect. If noisy regions were not seen to appear after this was done, that would more or less confirm that ambient light was the cause of all this trouble. Hopefully, if ambient light is indeed the cause of the noise, reducing it will enable an accurate and reliable measurement of thermally induced defocus within the Hartmann sensor.