Search
In this task, we will learn how to register two images that differ by translation and rotation. The registration can be split into two parts:
We will employ phase correlation [1] and polar transform [2] of the Fourier amplitude to estimate the transformation between the two images, as will be further described. To transform an image, we will implement bilinear interpolation [3].
In the second part, we will estimate the transformation between two images. We will learn how to register two images which differ by translation and rotation. Below, you can see an example. The two images were obtained by taking two photos of a scene with a mobile phone camera at two different times and using different camera orientations.
1 First, implement the phase correlation function phase_corr.m. This function estimates translation between two images by the method described in [1]. The method is very similar to deconvolution, which you have used before. You can test the implementation in the first part of hw_img_reg_2.m.
phase_corr.m
hw_img_reg_2.m
Shift in image domain can be achieved by convolution with Dirac delta function, which is equivalent to multiplication with complex exponential in frequency domain: $$b[x, y] ~=~ a[(x - dx), (y - dy)] ~=~ a[x,y] * \delta(dx,dy)$$ $$A = \text{FFT}(a)\quad B = \text{FFT}(b)\quad e^{2 \pi i (u~dx + v~dy)} = \text{FFT}(\delta(dx,dy))$$ $$B[u,v] ~=~ A[u,v] e^{2 \pi i (u~dx + v~dy)}$$ In phase correlation we first compute the complex exponential, which is equivalent to the normalized cross-power spectrum of the two image spectrums and then get the Dirac delta function using inverse Fourier transform.
You should also implement the function register_translation.m. First, use the phase correlation to estimate the relative shift and then translate the second image to look the same as the first image. You should have the implementation of translation ready from the previous part of this homework so you can reuse it. You can test your implementation of register_translation in hw_img_reg_2.m on different inputs.
register_translation.m
register_translation
Our implementation of phase correlation just returns the pixel coordinates of the Dirac pulse, so it is able to precisely estimate only translations by integer pixel values. You would have to implement more complex method for estimation with sub-pixel precision.
2 To estimate rotation between two images, we will employ the fact that the rotation of an image also rotates its magnitude spectra around its center by the same angle:
We can see that the spectrum of the second image is rotated by the same angle as the image itself is rotated relative to the first image. Note that the spectra are displayed as log(10+abs(FFTIM)) where FFTIM is the FFT of an image. This monadic transformation is also further used for comparing the two spectra and estimating the rotation between them (you may experiment with other monadic transformations of the FFT magnitudes.) Also, note that the provided function ffcenter may be used for getting the center of the spectrum after applying fftshift to it.
log(10+abs(FFTIM))
ffcenter
fftshift
2a Implement the function polar_transform.m, which converts any input image to polar coordinates as described in [2]. In rotation estimation, the polar transform is not used on the images themselves but on their magnitude spectra. The function allows us to estimate rotation between the spectra as translation in polar coordinates.
polar_transform.m
2b Implement function register_rotation.m. Use the function phase_corr.m to estimate translation between the two spectra in polar coordinates. From the translation in y-coordinate, compute the corresponding angle for image2 and rotate it so that it matches the orientation of image1:
register_rotation.m
Note that the estimation of rotation angle will be ambiguous - both value X and value X+pi (180 degrees) will be equally good. Use the one that is closer to 0 degrees (i.e., the one for which the rotation of the image is smaller).
The phase correlation on the spectra in polar coordinates also returns a shift along the X-axis (radius), which can be used for relative image scale estimation. We omit scale in this homework for simplicity (all the testing image pairs have identical scales).
If the rotation registration is working for A and A_rt image pair, but not for more complex images, you can:
bilinear.m
image(y,x)
r = linspace(0, H, M)
phi = linspace(0, 2*pi, N)
3 Implement register_full.m by using your implementations of register_rotation.m and register_translation.m. This will provide you with the solution for registering image2 to image1 from the first image on this page.
register_full.m
[1] slide 7 https://dcgi.fel.cvut.cz/home/sykorad/dzo/slides/dzo-l08.pdf
[2] slide 2 https://dcgi.fel.cvut.cz/home/sykorad/dzo/slides/dzo-l09.pdf
[3] slides 10, 11 https://dcgi.fel.cvut.cz/home/sykorad/dzo/slides/dzo-l07.pdf