A12 – Correcting Geometric Distortions

There are two types of geometric distortion.
1. pincushion - the image seems pinched in the middle and expanded in the boundaries.
2. barrel - the image seems bloated in the middle and pinched at the sides.

For this activity, we will use bilinear interpolation. "the graylevel v at an arbitrary location (x , y)
in the distorted image can be found using v (x , y )=ax+by+c xy+d.

We used a distored capiz window shown below,
I cropped the image
and using bilinear interpolation, the result is

Acknowledgements:

Activity 12 manual

Grade:
9/10 - because I think that even though I find an undistorted image. It is far from what is expected.

A11 – Camera Calibration

An image is a projection of brightness values from surfaces existing in 3D world space to 2D sensor space. The aim of this activity is to be able to model the physical processes involved in the geometric aspects of image formation.


Procedure:
1.Mount a 3D calibration checkerboard (to be provided) onto a surface and steady your camera with a tripod. Take an image of the checkerboard.
2. Pick an origin and a right-handed coordinate system in the grid and measure coordinates of around 20 to 25 corners of the squares of the checkerboard pattern.

I picked 23 corners.
3.From the image of the same points, find their [xi,yi] coordinates in image space. Use locate to get the coordinates in the image figure window using a mouse.

we have two coodinates since the image is flat. We will have two equations.
The o subscript corresponds to the object and i subscript corresponds to image.
finding the coordinates of selected points.

4. From Lecture 2, rederive Equation 13 this time with the z or Z axis as the camera axis, and x,y as the planar coordinates. Using all the points you've selected, set up the left and right-hand side matrices in Equation 13. Compute the elements of the matrix A using Equation 15.

Equation 13.
Thus,
5. To verify if you got the calibration correctly, predict the image coordinates of some cornerpoints of the checkerboard which were not used in the calibration. Overlay the predicted cornerpoints on the image. Comment on the accuracy of your prediction.
I found out that the average error for yi is 0.016708 and for zi is 0.0045 . And I think it this is accurate.

Using points that are not used in finding a
average error:
yi = 0.013684
zi = 0.010258

I think this is still acccurate.

--------------------------------------------------------------------
codes

stacksize(20000000);
img = imread('rix.jpg');
imshow(img);
x = locate(23);

yi = [326.66667 357.77778 424.66667 300.22222b 356.22222 393.55556 497.77778 508.66667 491.55556 150.88889 217.77778 113.55556 183.55556 118.22222 185.11111 88.666667 301.77778 241.11111 32.666667 28 147.77778 336 326.66667];
zi = [340.44444 184.88889 251.77778 161.55556 111.77778 500.66667 328 550.44444 158.44444 331.11111 178.66667 461.77778 373.11111 194.22222 505.33333 58.888889 419.77778 295.33333 366.88889 564.44444 552 534.88889 265.77778];
xo = [0 0 0 0 0 0 0 0 0 5 3 6 4 6 4 7 0 2 8 8 5 0 0];
yo = [1 2 4 1 2 3 6 6 6 0 0 0 0 0 0 0 0 0 0 0 0 1 1];
zo = [7 3 5 2 1 11 7 12 3 7 3 10 8 4 11 1 9 6 8 12 12 12 5];

//finding a
for i = 1:length(xo)
O((2*i)+1) = [xo(i) yo(i) zo(i) 1 0 0 0 0 -(yi(i)*xo(i)) -(yi(i)*yo(i)) -(yi(i)*zo(i))];
O((2*i)+2) = [0 0 0 0 xo(i) yo(i) zo(i) 1 -(zi(i)*xo(i)) -(zi(i)*yo(i)) -(zi(i)*zo(i))];
I((2*i)+1) = yi(i);
I((2*i)+2) = zi(i);
end

a = inv(O'*O)*O'*I;

//prediction
for j = 1:length(xo)
yinew(j) = ((a(1,1)*xo(j))+(a(1,2)*yo(j))+(a(1,3)*zo(j))+a(1,4))/((a(3,1)*xo(j))+(a(3,2)*yo(j))+(a(3,3)*zo(j))+1);
zinew(j) = ((a(2,1)*xo(j))+(a(2,2)*yo(j))+(a(2,3)*zo(j))+a(2,4))/((a(3,1)*xo(j))+(a(3,2)*yo(j))+(a(3,3)*zo(j))+1);
end

Acknowledgements:
Activity 11 manual.

Beth, Aiyin, Lei
- for answering my questions

Grade :
10/10 because I think that my data is accurate and I did what the activity asked. :)

A10 – Preprocessing Handwritten Text

In this activity, we will try to extract handwritten text from an imaged document with lines. In handwriting recognition, individual examples of letters must be extracted.

Procedure.
1. Download one of the images DSC00469.JPG or Untitled_0001.jpg.
2. Crop a portion of the image with handwritten text within lines in the form.

image 1

I cropped image 1 to

image 2 - cropped image

3. Remove the lines using image processing techniques.
To remove the horizontal lines we will use the filter below.

image 3 - filter

This filter is based on the fft of the cropped image which is shown in figure 4.

image 4 - fft of cropped image

Then we will filter the image by convolution,

image 5 - filtered image

4.Binarize the handwriting, clean the image and process such that each handwritten letter is one pixel thick.

image 6 - binarized image

image 7 - image after using opening operator

5. Use bwlabel to label each letter. Comment on the quality of your preprocessing.

image 8 - labeled image

code used:

im=imread('image.jpg');
img=imread('filter.jpg');
im = im2gray(im);
imf = im2gray(img);
ff1=fft2(im);
ff2=fftshift(imf);
imft=ff1.*ff2;
imshow(ff1);


//filtering the image
a=real(fft2(imft));
scf(1)
imshow(a,[]);

//binarize the image
b=im2bw(a, 127/255);//threshold from GIMP
c=1-b;
scf(2);
imshow(c,[]);

// structuring element
SE = ones(2,2)

//opening operation
d=erode(c,SE);
e=dilate(d,SE)
scf(3);
imshow(e,[]);

//label the image
f=bwlabel(e);
scf(4);
imshow(f,[]);

From the labeled image, the processed image is not that clear. :(

Acknowledgements:
Ed
- for the help

Grade:
10/10
- because I did my best and I was able to use what I learned in the activities before.

A9 – Binary Operations

In image-based measurements, it is often desired that the region of interest (ROI) is made well segmented from the background either by edge detection or by specifying the ROI as a blob.

Binarizing an image simplifies the separation of background from ROI. However, the optimum threshold must be found by examing the histogram.

Procedure:
1. Cut the image in 256 x 256 subimages.


In my case, I have 12 subimages.

2. Find the threshold of the image.

3. Binarize the image

bw = im2bw(im, 0.84);
subplot(142)
imshow(bw, [])
xtitle('binary')

4. Closing operator - dilate then erode
//closing operator
SE = ones (4,4);
imdilatec = dilate(bw, SE);
imerodec = erode(imdilatec, SE);
subplot(143)
imshow(imerodec, [])
xtitle('closing')
Opening operator - erode then dilate
//opening operator
SE = ones (4, 4);
imerodeo = erode(bw, SE);
imdilateo = dilate(imerodeo, SE);
subplot(144)
imshow(imdilateo, [])
xtitle('opening')

* we will not use both operators, we will just choose which one will show a good image where the blobs are defined.

5.Use bwlabel to label each contiguous blob on the binarized image.
// Label every connected component with a unique number.
[L, n] = bwlabel(imdilateo);
// Shows each component with a different color
scf(1)
subplot(121)
imshow(imdilateo, [])
subplot(122)
imshow(L+1,rand(n+1,3));
xtitle('label')

6. Find the area
for i=1:n
[r,c] = find(L==i);
a=size(r);
area(i) = a(2)
end

One of the subimages is shown below:

subimage1

After finding the area by pixel counting, it will show this histogram(using Excel)
From the graph, it shows that the area of the blob is near 525-500. I only use the areas of the circle from 400 to 700 because I think that below 400, the circle is truncated and more than 700 there is overlapping of circles.

Verifying our assumption:

1. I find the diameter of the blob, using GIMP, the diameter is around 26 pixels. Hence, the radius is 13 pixels. Using the formula for circle, which is pi*(r^2), the area should be 531 pixels which is in the range.

2. Another method is to pixel count a blob, it is found that the area is around 537 pixels, which is still in the range.

3. We will get the mean and the standard deviation.
Mean: 527 pixels
Standard Deviation: 22 pixels
which is still in the range of the peak from the histogram.

Acknowledgements:
Lei
Beth
Aiyin
Ed
- for the help. :)

Grade:
10/10
because I did what is needed to do.

A8 – Morphological Operations

Morphology refers to shape or structure. In image processing, classical morphological operations are treatments done on binary images, particularly aggregates of 1's that form a particular shape, to improve the image for further processing or to extract information from it. All morphological operations affect the shape of the image in some way, for example, the shapes may be expanded, thinned, internal holes could be closed, disconnected blobs can be joined. In a binary image, all pixels which are “OFF” or have value equal to zero are considered background and all pixels which are “ON” or 1 are foreground.

We will deal with two types of morphological operation, dilation and erosion.

and

The erosion operator is defined as

For this activity we will use 5 binary images.
1. square ( 50 x 50)
2.triangle (base = 50 , height = 30)
3. circle (radius 25)
4. hollow square (60×60, edges are 4 pixels thick)
5. plus sign (8 pixels thick and 50 pixels long for each line)

and we will use structuring element for dilation and erosion
1. 4×4 ones
2. 2×4 ones
3. 4×2 ones
4. cross, 5 pixels long, one pixel thick.

The algorithm for dilation is:

img = imread('filename.bmp');
SE = [1 1 1 1; 1 1 1 1; 1 1 1 1; 1 1 1 1];
SE2 = [ 1 1 1 1; 1 1 1 1];
SE3 = [ 1 1; 1 1; 1 1; 1 1];
SE4=[0 0 1 0 0; 0 0 1 0 0; 1 1 1 1 1; 0 0 1 0 0; 0 0 1 0 0];
im = dilate(img,SE);
im2 = dilate(img, SE2);
im3 = dilate(img, SE3);
im4 = dilate(img, SE4);
imwrite(im, 'filename1.bmp');
imwrite(im2, 'filename2.bmp');
imwrite(im3, 'filename3.bmp');
imwrite(im4, 'filename4.bmp');

Dilation:

The algorithm for erosion is

img = imread('filename.bmp');
SE = [1 1 1 1; 1 1 1 1; 1 1 1 1; 1 1 1 1];
SE2 = [ 1 1 1 1; 1 1 1 1];
SE3 = [ 1 1; 1 1; 1 1; 1 1];
SE4=[0 0 1 0 0; 0 0 1 0 0; 1 1 1 1 1; 0 0 1 0 0; 0 0 1 0 0];
im = erode(img,SE);
im2 = erode(img, SE2);
im3 = erode(img, SE3);
im4 = erode(img, SE4);
imwrite(im, 'filename1.bmp');
imwrite(im2, 'filename2.bmp');
imwrite(im3, 'filename3.bmp');
imwrite(im4, 'filename4.bmp');

Erosion:




Other morphological operations

thin - thinning by border deletion
use thin in the original square image.

skel - skeletonization, thinning, Medial Axis Transform

Acknowledgements:
Activity 8 Manual

BETH
JULIE
AIYIN
ANGEL
APRIL
- for helping me.

Grade:
10/10 - because sone of my predictions are right with the simulated image.

A7 – Enhancement in the Frequency Domain

The Fourier Transform (FT) of a signal gives the spatial frequency distribution of
that signal. Essentially, the Fourier Theorem states that any image or signal can be
expressed as a superposition of sinusoids. Unique to the 2D FT (as compared to
the 1D FT) is the fact that rotation of the sinusoids result to the rotation of their
FT's.

For this activity we have three tasks:
A. Anamorphic Property of Fourier Transform
B. Ridge Enhancement
C. Line Removal

7A. A Anamorphic property of the Fourier Transform

Create a 2D sinusoid using Scilab in the X direction.

nx = 100; ny = 100;
x = linspace(-1,1,nx);
y = linspace(-1,1,ny);
[X,Y] = ndgrid(x,y);
f = 4;
z = sin(2*%pi*f*X);
imshow(z,[]);

The image is like a corrugated roof. The image is a sine function.

Then we will get the fourier transform of the image.

I = imread('roof.bmp');
Igray = im2gray(I);
FIgray = fft2(Igray);
imshow(fftshift(abs(FIgray)),[]);
xset("colormap", hotcolormap(255));

The image shows two dots at the center of the image.The dots represent the frequency of the sine function. There are no pixels in the x-direction, because the image is the same everywhere in that direction.


When we increase the frequency, we will see that

f = 10

We can see that the number of lines increases and the dots are near the edges of the image.

As the frequency increases, the dots go farther from each other.

When we create a 2-D sinusoid in the X- direction,the dots are in Y-direction.

Then we will rotate the sinusoid,

f = 4;
theta = 30;
z = sin(2*%pi*f*(Y*sin(theta) + X*cos(theta)));

In the FT image, the dots are in the X- direction and the second dot is tilted, at the same degree as the sinusoid is rotated.

Using different values of theta


We will then create a pattern. I used the pattern given in the manual
z = sin(2*%pi*4*X).*sin(2*%pi*4*Y); this is a product of two corrugated roofs, one running in the X-direction, the other in Y.

f = 4;
z = sin(2*%pi*4*X).*sin(2*%pi*4*Y);

We can see that the resulting image is a checkered black and white image. The image 2D cosine with both horizontal and vertical components. They have 4 cycles horizontally and vertically. And the FT image has for dots forming a square near the center of the image.

fx = 4;
fy = 8
z = sin(2*%pi*fx*X).*sin(2*%pi*fy*Y);

The dots in the FT image correspond to the frequency of the stripes in the original image.

Activity 7.B Fingerprints : Ridge Enhancement

For this activity, we will use fingerprint.

img = imread('finger.jpg');
scf(1)
imshow(img)
img = im2gray(img);
image =img-mean(img);
scf(2);
imshow(image);

im=fft2(image);

ff=real((im).*conj(im));
scf(3); imshow(fftshift(log(ff+1)),[]);
xset('colormap',jetcolormap(64));

//Transfer function
filter=mkfftfilter(image,'exp',30);
scf(4);
imshow(filter);

//High-pass filter
IM=im.*fftshift(1-filter);
IMAGE=real(fft2(IM));
scf(5); imshow(IMAGE);

scf(6);
imshow(abs(fftshift(fft2(IMAGE))), []);
xset('colormap', jetcolormap(64));

Using the image in the Activity 7 manual

Using my own image(c/0 Julie D.)


Activity 7.C Lunar Landing Scanned Pictures : Line removal
Using the same commands we used in the previous activity for getting the fft and using filters.

// getting the FT of original image
im = imread('moon.jpg');
imgray = im2gray(im); // convert to grayscale
Fimgray = fft2(imgray);// Fourier Transform
scf(1);
imshow(fftshift(abs(Fimgray)),[]);
xset("colormap", hotcolormap(255));// using colormap

// getting the enhanced image by filtering
b = imread('filter.jpg');
a= imread('moon.jpg')
bgray = im2gray(b);
agray = im2gray(a);
Fb = fftshift(bgray);
Fa = fft2(agray);
FRA = Fb.*(Fa);
IRA = fft2(FRA); //inverse FFT
FImage = abs(IRA);
imshow(FImage, [ ]);



The lines in the filter corresponds to the line in the original image which we want to remove.

Acknowledgements:
http://www.cs.unm.edu/~brayer/vision/fourier.html
http://student.kuleuven.be/~m0216922/CG/fourier.html

Julie D. - for the scanned fingerprint
Aiyin and Beth - for answering my questions

Grade:
10/10
- because I did what the activity asks to do.

A6 - Fourier Transform Model of Image Formation

Activity 6.A – Familiarization with discrete FFT
First we made a 128 x 128 image of a circle using paint
Using scilab, we will convert the image to grayscale and apply Fourier transform.

im = imread('circle.bmp');
imgray = im2gray(im); // convert to grayscale
Fimgray = fft2(imgray);// Fourier Transform
scf(1);
imshow(abs(Fimgray),[]);
xset("colormap", hotcolormap(255)); // using colormap

applying fftshift

scf(2);
imshow(fftshift(abs(Fimgray)),[]);
xset("colormap", hotcolormap(255));// using colormap

The result is an airy disc.

applying Fourier transform again
scf(3);
imshow(abs(fft2(Fimgray)),[]);


The resulting image is the original image but the circle is shifted from the center to the left.

The next task is to apply the coomands to a letter A.

im = imread('A.bmp');
imgray = im2gray(im); // convert to grayscale
Fimgray = fft2(imgray);// Fourier Transform
scf(1);
imshow(abs(Fimgray),[]);
xset("colormap", hotcolormap(255)); // using colormap

applying fftshift

scf(2);
imshow(fftshift(abs(Fimgray)),[]);
xset("colormap", hotcolormap(255));// using colormap


applying Fourier transform again

scf(3);
imshow(abs(fft2(Fimgray)),[]);

The resulting image is an inverted letter A.
Activity 6.B Simulation of an imaging device.

I created an image "VIP" using paint.
This will be the "object".

Using the commands

b = imread('circle.bmp');
a= imread('vip.bmp')
bgray = im2gray(b);
agray = im2gray(a);
Fb = fftshift(rgray);
Fa = fft2(agray);
FRA = Fb.*(Fa);
IRA = fft2(FRA); //inverse FFT
FImage = abs(IRA);
imshow(FImage, [ ]);
xset("colormap", hotcolormap(255));

We will use different radii of circles. The circles will be the "lens".

radius 1:
The resulting image will be
radius 2:
The resulting image will be:

radius3:
The resulting image will be:
A smaller lens aperture will results in a blurred image. Comparing this with the bigger aperture, the image is sharper and most closely resembles the original.

Activity 6.C Template Matching using correlation

Template Matching is a pattern recognition technique suitable for finding exactly identical patterns in a scene such as in the case of finding a certain word in a document.

We made two images.
First, with the words "THE RAIN IN SPAIN STAYS MAINLY IN THE PLAIN."
Second, the letter A.

Using the commands
im = imread('rain.bmp');
img = imread('A2.bmp');
imgray = im2gray(im);
imggray = im2gray (img);
Fim = fft2(imgray);
Fimg = fft2(imggray);
FRA = Fimg.*conj(Fim);
IRA = fft2(FRA);
FImage = abs(IRA);
imshow(fftshift(FImage), [ ]);
xset("colormap", hotcolormap(255));

will produce the image below:

In the image, there are 5 dots which corresponds to the 5 A's in the sentence.
The A's are from words "rain", "spain", "stays", "mainly", "plain".

Activity 6.D Edge detection using the convolution integral
Edge detection can be seen as template matching of an edge pattern with an image

Using the commands,

hor = [-1 -1 -1; 2 2 2; -1 -1 -1]; //horizontal pattern
ver = [-1 2 -1; -1 2 -1; -1 2 -1]; //vertical pattern
spot = [-1 -1 -1; -1 8 -1; -1 -1 -1]; //spot pattern
pattern = spot;
vip = gray_imread('vip.png');
im = imcorrcoef(vip, pattern);
imshow(im);

The image that will appear in horizontal pattern is

For vertical pattern:
And for spot pattern:
The edges are more prominent depending on what kind of pattern the image was convolved with.

Acknowledgements:
Mark Leo
Aiyin
Beth
- for answering my questions

Grade: 10/10
because I know I did well and I did what the activity needed.