| Feature Correspondence Image Fusion Systems Research cs@imgfsr.com ______________________________________________________________ |
The main purpose of selecting features in images is to find correspondence between points in the
images. Knowing a set of corresponding points in the images, a transformation function can be
determined to change the geometry of the one image to that of the other. Below, methods for
determining the correspondence between features in two images are described. Correspondence will
be determined between points, lines, and templates.
Point Pattern Matching
The most desired features in images are points as their coordinates can be directly used to determine
the transformation function that registers the images. Point features are either directly determined or
determined from the intersection of lines, centroid of regions, or locally peak curvature points on region
boundaries. We assume a set of points from each image is given and want to find the correspondence
between them. Due to noise and other factors, it is possible that some points appear in only one of the
two sets. Such points are called outliers. It is also possible that some points are slightly shifted from
their true positions due to quantization and/or signal noise. We would like to identify the outliers and
determine the correspondence between the remaining points in the provided sets. We assume that the
only information carried by the points is their positions. There are three main methods for point-pattern
matching.
Matching Using Coherence Constraint: Because of scene coherence, if correspondence between
three non-colinear points in two images are established, by aligning the images at the three points, the
remaining points in the images will align also. This, of course, requires that the three points accurately
correspond to each other and the images relate by a linear transformation. If the points contain noise,
and so only approximately correspond to each other, by aligning the images at the three points, the
remaining points in the images will approximately align also. To determine the likelihood that the three
points selected in the images correspond to each other, first from the three corresponding points, the
parameters of a linear transformation are determined. The obtained transformation is then used to map
points in set 2 to points in set 1. Next, a measure of match between points in set 1 and points in
transformed set 2 is determined, and if the match-rating is sufficiently high, the three point pair used in
this process are considered correct correspondences.
Matching Using Clustering: In clustering, the transformation parameters are estimated through
a voting process. For the linear imaging model, six 1-D accumulator arrays are used to estimate
parameters a - f of the transformation. The accumulator arrays are initialized to 0. Then, for each
combination of three point pairs in the images, parameters a - f are determined and corresponding
entries in arrays a through f are incremented by 1. After testing a sufficiently large number of
combinations, entries showing the peak count in the arrays will show the transformation parameters
that match the most points in the images. The obtained parameters are then used in the linear relation
to map points from set 2 to corresponding points in set 1. Points falling within the required distance of
each other are then considered corresponding points.
Matching Using Invariants: Knowing the positions of three non-colinear points P1, P2, P3 in an
image, the positions of other points in the image can be described in terms of the three points by
P = P1 + a(P2 - P1) + b(P3 - P1)
If another image of the same scene is available and the relation between the two images can be written
by an affine transformation, the same relation holds between the corresponding points in the new
image. That is,
Q = Q1 + a(Q2 - Q1) + b(Q3 - Q1)
Using this affine invariance property, combinations of three points can be selected from the images
and the relation between remaining points in the images with respect to the three points can be
determined and compared. If the three point pairs selected in the images truly correspond to each
other, relations of other points in the images with respect to them will be similar. This similarity of
relations can be determined and used as a measure to verify the correctness of the three
correspondences. The relations can be represented by an image. For all points in the first image,
parameters a and b can be determined and used as indexes to a 2-D accumulator array. The obtained
array entry is then incremented by 1. After processing all the points an accumulator array will be
obtained that describes the relation between all points in the image with respect to the three selected
points.
Line Matching
Line segments have position, orientation, and length. Usually, the orientation of a line is least influenced
by noise and its length is most influenced by noise. The position of a line can be considered the
coordinates of the midpoint of the line. Positions of line segments can be more accurately determined
than the lengths of line segments because a midpoint is obtained from the average of line end points
and inaccuracies in line end points that are the result of zero-mean noise will cancel each other out.
Nevertheless, line positions contain some inaccuracies also.
If the two images from which the lines are extracted have translation, rotation, and scaling differences,
the rotational difference between the images can be determined first by resampling the sensed image
lines to have the same orientation as the corresponding reference image lines and then determine the
translational and scaling differences between the images knowing the line correspondences.
To determine the rotational difference between the images, two lines are selected, one from each set
and hypothesized as corresponding lines. The correctness of the hypothesis is then tested. If the
selected lines truly correspond to each other, by rotating the lines in one set by the rotational difference
between the selected lines, all corresponding lines in the two sets will have the same or very close
orientations. This can be determined by an algorithm. Once the rotational difference between the two
sets is determined, one set is rotated with respect to the other set so all corresponding lines have the
same or very close orientations. The line midpoints are then used in a point-matching algorithm to
determine the correspondence between the lines in the two sets.
Template Matching
Template matching is the process of locating a subimage or template from one image in another. By searching
templates from the reference image in the target image, correspondence is established between reference and
target image points. The template-matching process involves shifting a template over the target image and at
each shift position determining the similarity between the template and the corresponding window in the target,
and determining the shift position where maximum similarity is obtained between the template and the window. It
is assumed that the images do not have scaling, rotation, and nonlinear geometric differences.
Various similarity measures have been used in template matching. They are, the sum of absolute differences,
the cross-correlation coefficient, the Fourier transform coefficients, the invariant moments, and more recently
the mutual information.
images. Knowing a set of corresponding points in the images, a transformation function can be
determined to change the geometry of the one image to that of the other. Below, methods for
determining the correspondence between features in two images are described. Correspondence will
be determined between points, lines, and templates.
Point Pattern Matching
The most desired features in images are points as their coordinates can be directly used to determine
the transformation function that registers the images. Point features are either directly determined or
determined from the intersection of lines, centroid of regions, or locally peak curvature points on region
boundaries. We assume a set of points from each image is given and want to find the correspondence
between them. Due to noise and other factors, it is possible that some points appear in only one of the
two sets. Such points are called outliers. It is also possible that some points are slightly shifted from
their true positions due to quantization and/or signal noise. We would like to identify the outliers and
determine the correspondence between the remaining points in the provided sets. We assume that the
only information carried by the points is their positions. There are three main methods for point-pattern
matching.
Matching Using Coherence Constraint: Because of scene coherence, if correspondence between
three non-colinear points in two images are established, by aligning the images at the three points, the
remaining points in the images will align also. This, of course, requires that the three points accurately
correspond to each other and the images relate by a linear transformation. If the points contain noise,
and so only approximately correspond to each other, by aligning the images at the three points, the
remaining points in the images will approximately align also. To determine the likelihood that the three
points selected in the images correspond to each other, first from the three corresponding points, the
parameters of a linear transformation are determined. The obtained transformation is then used to map
points in set 2 to points in set 1. Next, a measure of match between points in set 1 and points in
transformed set 2 is determined, and if the match-rating is sufficiently high, the three point pair used in
this process are considered correct correspondences.
Matching Using Clustering: In clustering, the transformation parameters are estimated through
a voting process. For the linear imaging model, six 1-D accumulator arrays are used to estimate
parameters a - f of the transformation. The accumulator arrays are initialized to 0. Then, for each
combination of three point pairs in the images, parameters a - f are determined and corresponding
entries in arrays a through f are incremented by 1. After testing a sufficiently large number of
combinations, entries showing the peak count in the arrays will show the transformation parameters
that match the most points in the images. The obtained parameters are then used in the linear relation
to map points from set 2 to corresponding points in set 1. Points falling within the required distance of
each other are then considered corresponding points.
Matching Using Invariants: Knowing the positions of three non-colinear points P1, P2, P3 in an
image, the positions of other points in the image can be described in terms of the three points by
P = P1 + a(P2 - P1) + b(P3 - P1)
If another image of the same scene is available and the relation between the two images can be written
by an affine transformation, the same relation holds between the corresponding points in the new
image. That is,
Q = Q1 + a(Q2 - Q1) + b(Q3 - Q1)
Using this affine invariance property, combinations of three points can be selected from the images
and the relation between remaining points in the images with respect to the three points can be
determined and compared. If the three point pairs selected in the images truly correspond to each
other, relations of other points in the images with respect to them will be similar. This similarity of
relations can be determined and used as a measure to verify the correctness of the three
correspondences. The relations can be represented by an image. For all points in the first image,
parameters a and b can be determined and used as indexes to a 2-D accumulator array. The obtained
array entry is then incremented by 1. After processing all the points an accumulator array will be
obtained that describes the relation between all points in the image with respect to the three selected
points.
Line Matching
Line segments have position, orientation, and length. Usually, the orientation of a line is least influenced
by noise and its length is most influenced by noise. The position of a line can be considered the
coordinates of the midpoint of the line. Positions of line segments can be more accurately determined
than the lengths of line segments because a midpoint is obtained from the average of line end points
and inaccuracies in line end points that are the result of zero-mean noise will cancel each other out.
Nevertheless, line positions contain some inaccuracies also.
If the two images from which the lines are extracted have translation, rotation, and scaling differences,
the rotational difference between the images can be determined first by resampling the sensed image
lines to have the same orientation as the corresponding reference image lines and then determine the
translational and scaling differences between the images knowing the line correspondences.
To determine the rotational difference between the images, two lines are selected, one from each set
and hypothesized as corresponding lines. The correctness of the hypothesis is then tested. If the
selected lines truly correspond to each other, by rotating the lines in one set by the rotational difference
between the selected lines, all corresponding lines in the two sets will have the same or very close
orientations. This can be determined by an algorithm. Once the rotational difference between the two
sets is determined, one set is rotated with respect to the other set so all corresponding lines have the
same or very close orientations. The line midpoints are then used in a point-matching algorithm to
determine the correspondence between the lines in the two sets.
Template Matching
Template matching is the process of locating a subimage or template from one image in another. By searching
templates from the reference image in the target image, correspondence is established between reference and
target image points. The template-matching process involves shifting a template over the target image and at
each shift position determining the similarity between the template and the corresponding window in the target,
and determining the shift position where maximum similarity is obtained between the template and the window. It
is assumed that the images do not have scaling, rotation, and nonlinear geometric differences.
Various similarity measures have been used in template matching. They are, the sum of absolute differences,
the cross-correlation coefficient, the Fourier transform coefficients, the invariant moments, and more recently
the mutual information.
______________________________________________________________________________
 | |  |