[ pdf | chapters | single document ]

Acquisition

Algorithm

A number of algorithms were considered for recovery of depth from images. For the purposes of this project, the ideal algorithm would:

• allow any type of pattern on the cloth, including the planned grid pattern. Algorithms such as shape-from-shading require unpatterned surfaces to work.
• allow stereo video sequences. A passive technique is hence preferable to an active technique such as shape-from-lighting-variation. Likewise, any algorithm that requires a series of viewpoints is not feasible. Multiple cameras could alleviate this restriction, but were not considered in this project.
• allow non-rigid deformations. Unlike rigid structures, cloth deforms in many ways: stretching, shearing and bending. Furthermore, deformation may be at a fine scale, involving small wrinkles or folds. Algorithms such as shape-from-texture may be difficult to adapt to these requirements.
• provide precise depth data.
• be easy to implement or have readily available implementations.
• work with any type of cloth material.
• not distort the motion or shape of the cloth. Any changes made to the cloth must not affect the weight or stiffness of the cloth.

Of the available shape recovery algorithms, stereo correspondence comes the closest to meeting these requirements. Correspondence algorithms require textured cloth--not necessarily large-scale patterns, but certainly fine-scale texture, in the form of visible strands or fluff. This limits the range of materials that can be captured; in my tests, cotton T-shirts were often unacceptable, but fluffy teatowels worked quite well. Fine texture could be added in other acceptable ways, however, such as a light spray-paint layer. Stereo correspondence is a passive technique, allows capture of any surface type, and is easy to implement. Its precision is limited, since depth samples are quantised by the capture process.

Method

Capture was performed using a Digiclops camera, manufactured by Point Grey Research. Due to limited dynamic range, lighting had to be carefully controlled. Incandescent light produced good results, but indirect natural light seemed did an even better job at depth recovery. White cloth was found to be easier to capture than darker shades. The cloth was suspended by one edge from a stand, and placed against a dark background for easy segmentation. The cloth used was a white cotton teatowel with printed blue and yellow stripes. The vertical stripes were solid, while the horizontal stripes were a mixture of colour and white stitching.

The Digiclops provided three images (right, top and left), each at 1024x768. Stereo processing was performed on greyscale images using the Triclops SDK. Images were rectified, low-pass filtered, and edge-filtered. Stereo correspondence was performed on the images, and points were validated using a median filter. The full list of parameters used is shown in Table 1. After processing with the Triclops SDK, the final output of the capture process was a colour image, a depthmap, and a mask indicating the valid pixels. All outputs are shown from the right camera's view in Figure 1.

Table 1: Digiclops Parameters

Parameter	Value
Gain	525
Shutter	500
Depth Range	$[0.1 \ldots 1]$
Resolution	1024x768
Colour buffer	24 bits
Depth buffer	16 bits
Low-Pass Filter	on
Rectification	on
Edge Correlation	on
Edge Mask Size	7
Stereo Mask Size	11
Disparity Range	$[0 \ldots 64]$
Disparity Mapping	off
Texture Validation	on
Texture Validation Threshold	1.1
Unique Validation	on
Unique Validation Threshold	0.8
Subpixel Interpolation	on

Figure 1: Outputs of acquisition stage. Top row: colour image, depth image. Bottom row: mask, masked depth image with scaled intensities.

Subpixel interpolation was used, but had some definite disadvantages. Stereo correspondence algorithms are limited to a maximum precision of one pixel. This leads to quantisation of the depth data, yielding a coarse, jagged appearance. Subpixel interpolation attempts to solve this problem by interpolating depth values to produce a smoother surface. Unfortunately, high contrast edges of the cloth gave rise to errors in the interpolated depth data, as seen in Figure 2. These ridges are not present in the actual geometry of the cloth. A flat sheet of paper with a checkerboard pattern produced similar artefacts.

Figure 2: Left: colour image of region of cloth. Right: depth image of same region. Vertical lines in depth data are artefacts, caused by high-contrast vertical stripes. Lower-contrast horizontal stripes do not cause artefacts.

For the purposes of this paper, these ridges are accepted as a source of error. For more accurate results, they should be filtered out, or some other technique should be used to deal with quantisation errors. It remains uncertain whether the errors are caused by subpixel interpolation, or if they were already present in the data and were simply more obvious after interpolation.