[ HTML | PDF (web) | PDF (print) ]

6. Results

In this chapter, the results produced by our cloth capture system are described in detail. A 63×67cm cloth was selected for capture, with line art images printed on it in a distinct, non-repeating pattern. The SIFT system detects features using edges, and line art provided a natural way of obtaining a high density of edges. The system was tested with several cloth motions. The principal test consisted of drawing one corner of the cloth along a string, over the course of 20 frames. The numbers cited here refer to this dataset.

Figure 6.1: The Digiclops camera used for triocular video acquisition.

Input data was acquired using a triocular Digiclops camera from Point Grey Research, shown in Figure 6.1. Images were captured at a resolution of 1024×768 and a rate of 10 Hz. The Triclops SDK was used to create a disparity map using a Sum of Absolute Differences (SAD) correlation method, and conservative settings yielded a sparse but reliable disparity map. The stereo mask was kept to a small 7×7 window to limit foreground fattening. A mask image of the cloth was constructed by thresholding and combining the intensity and disparity images. The reference image was acquired using a flatbed scanner and image stitching tools, and was scaled down to a resolution of 992×1024.

The feature detector found 21000 features in the reference image, and an additional 43000 features in the oblique views of the reference image. The captured images yielded 4200-6400 features, with the number of features typically directly proportional to the visible cloth area. Feature vectors of 128 dimensions were used, but smaller sizes would also likely be suitable.

The seed-and-grow algorithm accepted matches for 50-60% of the captured features. Stretch and compression of up to 10% was permitted. This margin allowed for error in our approximation of geodesic distance, $\tilde{g}(c_s,c_n)$, and permitted some diagonal stretch (i.e., shear) in the cloth, but was still sufficient to perform quality matching.

In the main dataset, the first ten seeds were typically sufficient to classify over 50% of F_c, and the first 80% of F_c was usually classified using the first thirty seeds. This process was fairly quick and efficient, and yielded a good dense map of features in the flat regions of the cloth.

Classification of the final 20% of F_c, however, was much slower. These features were typically near folds or poorly illuminated regions of the cloth, and little growth was possible. Consequently, many of these features had to be matched with a slow brute force algorithm, and many were later rejected by the verification algorithm. Nevertheless, a few good matches were made, justifying the continued search.

We found that the oblique reference views for the SIFT algorithm were definitely valuable for the matching process. Of the matched captured features, over half were matched with reference features from oblique views. Some extremely oblique views were also attempted, scaling the reference image by a factor of four. These views gave very small improvements, usually amounting to less than 5% of all matches, and we therefore chose not to use them.

The verification algorithm was fairly conservative in its acceptance of features, rejecting over 40% of the matched features. Table 6.1 shows the number of accepted features after feature detection, matching, and verification. As can be seen, only 46% of the detected features were accepted. Despite using a conservative verification, it was still possible to track roughly an order of magnitude more features than would be feasible with traditional motion capture or using Guskov's method. [44,46,45]

Table 6.1: Number of features found, matched, and verified for selected frames.

Frame	Visible	Initial	Matched	Verified
	area	features	features	features
1	271k	6464	4978	2980
6	271k	6458	4966	2948
11	241k	5710	4349	2481
16	207k	4731	3558	2064
20	190k	4249	3233	1861
Average	236k	5567	4408	2578

The performance of the system is shown in Table 6.2. Matching was clearly a bottleneck in the system, and the seeding process was the slowest part of matching. The speed of matching on each frame was highly dependent on the initial success of the growth algorithm.

Table 6.2: Performance of our system in selected frames, measured in seconds on a Pentium IV 1.8GHz system.

Frame	Hole filling,	Feature	Matching	Verification &
	smoothing	detection		parameterisation
1	3:15	0:14	2:15	0:36
6	2:53	0:15	2:18	0:42
11	2:34	0:14	2:03	0:33
16	2:47	0:14	1:41	0:28
20	2:17	0:16	1:27	0:25
Average	2:46	0:15	1:57	0:34

Our final results after parameterisation are shown in Figure 6.2. A checkered texture is used to illustrate the parameterisation of the surface, but clearly any texture could be applied.

Figure 6.2: Top row: input images, frames 6,11,16. Middle row: parameterised geometry with checkered texture. Bottom row: comparison of matched and verified feature density in $\mathcal{R}$

Capture of fast-moving cloth was practical using this system. Figure 6.3 demonstrates one example, where the top left corner of the cloth fell and pivoted about the fixed corner in the top right. This image was taken at the start of the fall, where the left side of the cloth is moving quickly while the right side stays still. Motion blur is evident in the fast-moving left side. As can be seen, capture and parameterisation were successful in both the slow-moving and fast-moving sections of the cloth. SIFT features are scale-invariant, and consequently large features could still be found in the presence of motion blur. We are unaware of any other tracking technology that could achieve similar results.

Figure 6.3: Left: captured image of fast moving cloth. Right: parameterised geometry. Left inset is moving quickly while right inset is still.