Filtering MoCap Data for Real-time Applications

We have implemented an alternative filter design method based on multi-objective optimization to target this challenge. With this method, we have thoroughly compared different design methods and designed nearly optimal filters with arbitrary specifications that compare favorably to existing filters in the literature. Additionally, to create filters specific to our applications, we have experimented to determine the frequency specifications of MoCap data. Finally, based on these tools and results, we have proposed a range of filters suitable for real-time MoCap applications.

Introduction

Motion capture (MoCap) and sensor technologies are seen used in real-time interactive applications, such as computer game controllers (e.g., Wii Remote, PlayStation Move, Kinect) and other gestural controllers (e.g., new musical interfaces, mobile phones, novel interfaces for desktop computer). The increased availability of new and improved MoCap technologies, together with algorithms that interpret user motion as control data, make it increasingly affordable and feasible to use for musical interaction. However, most of the utilized MoCap technologies are known to possess noise properties that may be problematic [1, 2]. Therefore, applying filters to alter these noise problems is often necessary. And, as one might expect, there will always be a corresponding delay penalty when employing a digital filter. It is, therefore, relevant to consider filters with low group delay for applications that demand good real-time performance [3].

Two main digital filter types exist finite impulse response (FIR) filters and infinite impulse response (IIR) filters. Essentially, IIR filters offer an effective way of achieving a long impulse response without having to use long FIR filters. Therefore, if the goal is to minimize the time delay, the use of IIR filters seems reasonable since they can have a dramatically lower order than FIR filters with similar performance [4]. Our results in [a] support this claim as well. However, no established method exists for designing IIR filters with minimal group delay. Additionally, there is little literature concerning best practice methods for filtering MoCap data for real-time applications. The goal of the current project has therefore been to target these challenges.

The outcome of the project

• Implemented an alternative filter design method
  We proposed and implemented an alternative filter design method based on multi-objective optimization. With this method, we can find nearly optimal IIR filters with arbitrary specifications [a][b].
• Explored low delay filter design methods (low group delay)
  Based on the above method, we have scoured the potential of designing filters with low group delay [a]. In short, it was shown that filter delay is a trade-off with noise attenuation. Furthermore, our method found more optimal low delay filters than the currently available design methods can produce [a].
• Designed IIR low-pass differentiators (of degrees 1 and 2)
  With the above design method, we have been able to design low-pass differentiators that compare favorably with existing differentiators in literature [b]. Low-pass differentiators are especially relevant for us since they avoid the undesirable amplification of noise in the higher frequency band, which is characteristic of MoCap data [b]. We have also been able to custom design low-pass differentiators of degree two (non-cascaded), which is not found presented earlier in the literature. (Using one differentiator of degree 2 is equivalent to using two differentiators of degree 1 in cascade. However, the former will generally give a more optimal solution.)
• Implemented a frequency content analyzer method
  We have implemented a method for analyzing the frequency content of MoCap data and proposed an approach for interpreting these data to determine reasonable frequency cutoffs, i.e., which frequencies a filter should filter out. This method is also relevant when designing filters for post-processing of MoCap data, i.e. when real-time features are insignificant [c].
• Found generic frequency properties of hand motion
  We have further experimented to find the generic frequency properties of free hand motion. Here we recorded the hand motion of 20 subjects, which were then analyzed with the above analysis method. Based on this experiment, we have proposed three cutoff frequencies for different scenarios and filtering needs: 5, 10, and 15 Hz, which relate to heavy, medium, and light filtering, respectively [c]. The 5 Hz and partly the 10 Hz cutoff will attenuate some high-frequency parts of rapid hand motion. However, this may be necessary to get the needed noise suppression [c].
• Proposed a set of filters for filtering MoCap data for real-time applications
  Finally, we have proposed a set of appropriate filters for real-time MoCap applications for filtering hand motion data, both low-pass filters and low-pass differentiators of degrees 1 and 2, based on the frequency properties found above and our proposed filter design method [c]. These filters are presented below.

Available tools/software

Max/MSP IIR MoCap filter patch

The above-mentioned IIR filters are implemented in the following MAX / MSP patch. See the guidelines in [c] to choose a suitable cut-off frequency or use the following analysis method. (The specification of these filters are given at the end of this page.)

• IIR MoCap Filters

Frequency analysis of MoCap data with the residual analysis

The residual analysis was the best method for determining the frequency content of MoCap data and can be used to determine a reasonable cut-off frequency [c]. The technique consists of low-pass filtering the data with different cutoff frequencies and calculating the residual, i.e., what is left over when we subtract the filtered data from the raw data. As long as the filter only attenuates noise, the residual should be relatively small. However, the residual will become more prominent when the filter attenuates the desired signal. We get an overall picture of their impact by performing this analysis for several cutoff frequencies and plotting the resulting residuals. This plot can then serve as a basis for determining a reasonable cutoff frequency [2]. See the included help file or [c] for more information about this method. The function is written in Matlab and should work in most Matlab versions. 

• Residual Analysis

The proposed real-time IIR filters

Below we have given the transfer functions of the proposed IIR filters. These are the same filters embedded in the above Max/MSP patch. All filters have a group delay of 2 samples or less and have better low-delay performance than what currently established filter design methods can create [b] (between 5 and 16 dB noise attenuation gain compared to comparable FIR filters). The transfer functions can be used with the filter method in Matlab. For C++ implementation, see [5] or similar programming literature. Notice that it is more optimal to use one low-pass differentiator of degree 2 instead of two subsequent low-pass differentiators in cascade.

Publications

[a] Skogstad, Ståle A. and Holm, Sverre and Høvin, Mats, Digital IIR Filters With Minimal Group Delay for Real-Time Applications, The International Conference on Engineering and Technology (2012) 

[b] Skogstad, Ståle A. and Holm, Sverre and Høvin, Mats, Designing Digital IIR Low-Pass Differentiators With Multi-Objective Optimization, IEEE Computer Society (2012)

[c] Skogstad, Ståle A. and Nymoen, Kristian and Holm, Sverre and Høvin, Mats and Jensenius, Alexander Refsum. Filtering Motion Capture Data for Real-Time Applications, In Proceedings of NIME (2013)

References

1. S. A. Skogstad, K. Nymoen, and M. Hovin, “Comparing inertial and optical mocap technologies for synthesis control,” in Proc. Sound and Music Computing, 2011.
2. D. Winter, Biomechanics and Motor Control of Human Movement. John Wiley & Sons, 2009.
3. D. Wessel and M. Wright, “Problems and prospects for intimate musical control of computers,” in Proc. NIME, (Seattle, USA), 2001.
4. S. Mitra, Digital signal processing: a computer based approach. McGraw-Hill H. E., 2005.
5. W. Press, Numerical Recipes 3rd Edition: The Art of Scientific Computing. Numerical Recipes: The Art of Scientific Computing, Cambridge University Press, 2007.
6. H. Woltring, “On optimal smoothing and derivative estimation from noisy displacement data in biomechanics,” Human Movement Science, no. 3, 1985.
7. G. Wood, “Data smoothing and differentiation procedures in biomechanics,” Exercise and sport sciences reviews, vol. 10, no. 1, p. 308, 1982.
8. D. Robertson, Research Methods in Biomechanics. Human Kinetics, 2004.