2018-2019: Research Seminar 3
Synthesizing multi-character interactions using data-driven approaches.
Character animation production pipeline still requires a lot of manual work from artists nowadays.
Researchers in the Computer Graphics (CG) community have been trying to develop new approaches
for the professionals in the entertainment industry such that they can focus their effort on creativity
rather than tedious manual, repetitive tasks. With the advancement of motion capture technology
and machine learning techniques, it is possible to develop data-driven approaches to assist artists in
daily routine work. My research interests focus on the areas of CG and Computer Vision (CV). In
particular, I am interested in applying state-of-the-art machine learning techniques demonstrated in
the CV community in solving research problems in CG. In this talk, I will present two recent CG projects.
In the first project, an intuitive way for controlling characters in crowd simulation using multi-
touch devices is proposed. We propose a data-driven gesture-based crowd control system, in which
the control scheme is learned from example gestures provided by different users. In particular, we
build a database with pairwise samples of gestures and crowd motions. To effectively generalize the
gesture style of different users, we propose a set of gesture features and crowd motion features that
are extracted from a Gaussian mixture model. Given a run-time gesture, our system extracts the K
nearest gestures from the database and interpolates the corresponding crowd motions in order to
generate the run-time control. Our system is accurate and efficient, making it suitable for real-time
applications such as real-time strategy games and interactive animation controls.
In the second project, we introduce a data-driven method to generate a large number of
plausible, closely interacting 3D human pose-pairs, for a given motion category, e.g., wrestling or salsa
dance. With much difficulty in acquiring close interactions using 3D sensors, our approach utilizes
abundant existing video data which cover many human activities. Instead of treating the data
generation problem as one of reconstruction, either through 3D acquisition or direct 2D-to-3D data
lifting from video annotations, we present a solution based on Markov Chain Monte Carlo (MCMC)
sampling. With a focus on efficient sampling over the space of close interactions, rather than pose
spaces, we develop a novel representation called interaction coordinates (IC) to encode both poses
and their interactions in an integrated manner. Plausibility of a 3D pose-pair is then defined based on
the ICs and with respect to the annotated 2D pose-pairs from video. We show that our sampling-based
approach is able to efficiently synthesize a large volume of plausible, closely interacting 3D pose-pairs
which provide a good coverage of the input 2D pose-pairs.
Speaker: Edmond Shu-lim Ho from Northumbria University, UK.