David Pritchard and
Wolfgang Heidrich
Computer Graphics Forum (Eurographics 2003), 22(3):263-271, September 2003.
This paper was presented at the 2003 conference of the European Association for Computer Graphics (Eurographics 2003), and was published as part of Computer Graphics Forum. A short sketch was also presented at SIGGRAPH 2003.
Some more recent results related to this work are reported in my M.Sc. thesis
Recent years have seen an increased interest in motion capture systems. Current systems, however, are limited to only a few degrees of freedom, so that effectively only the motion of linked rigid bodies can be acquired. We present a system for the capture of deformable surfaces, most notably moving cloth, including both geometry and parameterisation. We recover geometry using stereo correspondence, and use the Scale Invariant Feature Transform (SIFT) to identify an arbitrary pattern printed on the cloth, even in the presence of fast motion. We describe a novel seed-and-grow approach to adapt the SIFT algorithm to deformable geometry. Finally, we interpolate feature points to parameterise the complete geometry.
Motion capture systems do not have enough degrees of freedom to capture moving cloth. Our system recovers the geometry and parameterisation of a sheet of moving cloth by using stereo correspondence and an extension of the scale-invariant feature transform (SIFT) introduced by [Lowe 1999]. Previous work on cloth capture [Guskov 2002] required a checkerboard pattern to be printed on the cloth, and recovered much fewer features.
A stereo vision system is used to capture a video sequence of moving cloth, and produce a rectified disparity map for each frame. Using a multiscale algorithm, we fill holes and interpolate to generate a smoothly-varying floating-point disparity map.
To parameterise the surface, we use SIFT to detect scale and rotation-invariant features in one of the captured stereo images. Features are also detected in a scanned reference image of the flattened cloth. We use a novel seed-and-grow algorithm to match features in the stereo image to features in the reference image.
We first find a seed feature in the captured image, and find the best matching reference feature. Each SIFT feature has an associated vector, representing a coarse sampling of the local image gradient. The Euclidean distance between SIFT feature vectors provides a reasonable metric for evaluating the quality of a match. From the matched seed feature, we grow outwards, finding matches for adjacent features. We repeat this with multiple seeds until all features have been either matched or rejected as unmatchable. Cloth strongly resists stretching and compression, and our algorithm uses this as a constraint on the matching process, limiting stretch to a maximum of 10\%. To do this, we consider pairs of matched features, and examine both the distance between reference features and the 3D geodesic distance between the matching captured features.
Following matching, we use a voting scheme to verify the correctness of matches. This helps to eliminate both bad seeds, and incorrect matches of individual features.
The matching process allows us to establish the parametric coordinates of every feature. Features are sparse, so parameters must be interpolated if a dense parameterisation is desired. We perform an interpolation to yield a dense, regularly sampled parametric map corresponding directly to the disparity map. To reduce distortion, we ensure constant spacing of isoparametric curves without requiring perpendicularity of the curves. This fits with the nature of cloth, which resists stretch but allows some shearing.
In Figure 1 we show some results. In this 1024×768 captured image, 2532 features were successfully matched. Over the two second sequence of twenty images, an average of 2103 features were matched per frame.
 |
Our approach requires a distinct, non-repeating pattern to be printed on the cloth. SIFT's multiresolution nature allows features to be detected in the presence of motion blur, allowing even fast movements to be captured. Our method will extend to calibrated camera systems with any number of cameras.
Guskov, I. 2002. Efficient tracking of regular patterns on non-rigid geometry. In Proceedings of IEEE International Conference on Pattern Recognition (ICPR 2002), vol. 2, pages 1057-1060. IEEE Computer Society, 2002.
Lowe, D. 1999. Object recognition from local scale-invariant features. In Proceedings of IEEE International Conference on Computer Vision (ICCV 1999), pages 1150-1157. IEEE Computer Society, 1999.
Last updated 2003.10.01.