Project 3: Face Morphing

-Boyu Zhu

Part 1. Defining Correspondences

For this part, I used the online labeling tool given in the project spec to define the correspondences on the two images I’d like to morph. I manully add four corner points for better morphing.

me

George

Then I used these corrspondences points to calculate Delaunay triangulations of both images

Delaunay triangulation

Part 2. Computing the "Mid-way Face"

For this part, I first implemented the function to compute the affine matrix based on the coordinates of the three points in the simplice from the triangulation. Using the affine matrix, I can now map the all the points in the simplice back to the source image and destination image. Then, I chose to use the Bilinear Interpolation to calculate the color of the halfway image. And finally I averaged the color of the two midway images to obtain the final halfway image.

Part 3. The Morph Sequence

Combining the mid-way face of every step into a gif, we can get this morph sequence.

Part 4. The "Mean face" of a population

In this part, I used the dataset of FEI face database. The FEI face database is a Brazilian face database that contains a set of face images taken between June 2005 and March 2006 at the Artificial Intelligence Laboratory of FEI in São Bernardo do Campo, São Paulo, Brazil.

The dataset has two subsets: neutral faces and smiling faces.

The dataset give an annotation of each image with 46 correspondences. I used these correspondences and four corner points I manully add to compute the average simplices and average shape.

After computing the average shape, I warped every image into the average shape. Below are some comparison between original image and warped image.

Aftering comuting all of the warped image, I averaged all of the images in the respective subset to compute the average faces.

average image of neural faces

average image of smiling faces

I then warped my face to the average image of neural faces, and warped the average image of neural faces to my face

my face warped to the average image of neural faces

the average image of neural faces warped to my face

Part 5. Caricatures: Extrapolating from the mean

To create caricatures, I change the value of α outside the normal range of [0, 1] to exaggerate features of my face to the average face's features or exaggerate the average face's features to my face.

α = 2

α = 1.5

α = -0.5

α = -1

Bells and Whistles

1 Finding Correspondences Automatically

To comlete bells and whistles, the first thing I tried to do is to look for a way of labeling automatically, but I found it exhausting to label manually. Therefore, I used the dlib to automatically compute the 68 important feature points on the face of the image. I don’t need to manually click on the points and can explore more interesting experiments.

2 Change gender

I changed my gender using the average Chinese famale searched online.

I tried with three kinds of morphing:

1. shape only: I only morphed my face to the shape of average Chinese famale.

2. appearance only: only morphing the appearance of the average Chinese famale to my face shape.

3. shape & appearance: I morphed my face to the shape of average Chinese famale and then combine the morphed face with the appearance of average Chinese famale.

shape & appearance morphing achieved the best result.

me without glasses

average Chinese famale

shape only

appearance only

shape & appearance

3 PCA basis

I computed the PCA basis on neutral and smiling datasets. Here are top 25 Eigenfaces for each of the subset.

Then I used the Eigenfaces to reconstruct the original faces. Here are two examples.

From the result, we can see that the reconstructed image is blurrier and less detailed compared to the original one. But it still retains the overall structure and facial features.

4 morphing music video

I created a music video of my friend Sunny at different ages. The output is shown below.