Face analysis

Face registration

3D reconstruction

Disparity map

Calibration

Zero crossings for correlating between stereo pairs

Zero crossing has proved useful in creating an image mask to cut the face from the stereo images, then we can retify this mask and have a calibrated stereo pair containing only the face we are reconstructing. Combining Gabor and SSD matching techniques seems to work very well. Each point was matched successfully in this test (including the matches in areas where lighting conditions are very poor). At the moment the algorithm is very slow but could easily be optimized but it certainly appears that the combination of the two matching techniques is superior to either on its own, as even in the example shown a number of the points would have been matched incorrectly had a single matching algorithm been used. For example, the console window shows the positions and strengths of some of the matched points and it can been seen that in the case of the last point SSD produced an incorrect answer where as Gabor produced a correct match - a combination of the two resulted in a correct match. There were other cases where SSD yielded the correct result where Gabor failed and also situations where both algorithms failed and the combination provided the correct result.

The second screenshot shows the results of calculating the zero crossing from a single camera's input - its pretty fast and works fine in real time, and it is this processing that allows the creation of the mask for the head extraction in the first screenshot. The blue lines accross the stereo face pairs are simple to prove that the calibration / rectification is correct.

Super resolution demo

The display on the left is the projection of the face cloud onto the xy plane with the grid shown on top and the display on the right shows the 3D projection of the model with the front and back faces of the grid cells drawn in blue and green respectivly. For each front plane the z co-ordinate is set to the mean of all the points in the cell plus 2X the standard deviation. The back plane z co-ordinate is set to the mean of all the points in the cell minus 2X the standard deviation. Points with a z co-ordinate between these two planes are valid (and shown in white), outliers are drawn in red. Obviously it's a bit hard to tell whats going on without being able to rotate about the model. The second screenshot shows the same model with the same filter parameters but without the local front and back threshold planes shown. For the final screenshot a third party application is used to create a surface and fill holes in filtered point cloud. The only problem with this algorithm at the moment is that the normal to the face must be rotated to align with the world z axis before it can be used.