Image correlation across multiple image modalities and local spatial distortions

Abstract

This thesis presents two techniques for correlating the pixels in two images of the same or similar subjects under various change conditions. The first technique is a new modality invariant keypoint descriptor that attempts to combine the best elements of two successful keypoint types used in image registration and object recognition algorithms, the Scale Invariant Feature Transform (SIFT) descriptor at Laplacian of Gaussian (LoG) peak locations and the relative structural distances used by Shape Contexts, into a single keypoint descriptor method that is capable of matching between images that differ in sensor modality or lighting conditions. This is meant to be a replacement for the SIFT+LoG keypoints used in the initialization phase of the Generalized Dual-Bootstrap ICP (GDB-ICP) algorithm, since while the currently used SIFT descriptor does locally normalize intensities to be mostly invariant to lighting, it was not designed to be invariant to strongly different image modalities such as between various types of medical and satellite/aerial images that are captured with different sensors that acquire different kinds of data . This approach has a statistically significant effect on the registerability of images overall within the context of single point initialization algorithms such as GDB-ICP since it improves initialization scores on pairs considered to be difficult and maintains sufficiently good performance on pair types already considered to be trivial. The second technique is an extension of the GDB-ICP algorithm to produce a piecewise pseudo-registration between two images where a single global transformation model may not be appropriate. One presented application of this is to the Mutual Information (MI) approach to stereo and optical flow by energy minimization. The proposed technique uses many locally constrained GDB-ICP registrations to produce a semi-dense correspondence map as an initialization point for MI stereo and optical flow on images where a global registration model would fail and the naive initialization approach presented by the papers using the technique so far is insufficient. The published approach to MI stereo2 initializes a progressively refined joint probability density function (PDF) for an image by setting initial disparity estimates to be random within the range of known disparities. This appears to work in testing primarily because the common practice for stereo evaluation tends to be images with known and relatively limited disparity ranges. Furthermore, performance testing between stereo algorithms is typically tuned by allowing the maximum search distance to be set manually rather than algorithmically, a practice which would no longer be necessary with the proposed technique. Many subjects of research do not lend themselves well to these restrictions. Aerial ground photography, as one example, may not be well aligned initially and tends to have local spatial deformations, whether by physical projection of the subject or by artifact of the acquisition method. Therefore, an initialization approach that is insensitive to initial alignment and local spatial deformation is needed for the MI stereo process to be fully capable of handling these types of image sets. The proposed technique handles both issues simultaneously.