related.tex

\chapter{Related Works}

\section{Control of Highly Dynamic Motions}

Designing controllers for physically simulated biped characters
is a challenging problem due to its nonlinear dynamics and
under-actuated control.
Hodgins \etal \cite{Hodgins:1995:AHA,Wooten:1998:Phd} showed that 
carefully crafted control algorithms can simulate highly athletic 
motions, including diving, tumbling, vaulting, and leaping. 
Faloutsos \etal \cite{Faloutsos:2001:CCF} composed primitive 
controllers to simulate more complex motor skills, 
such as a kip-up move or a dive down stairs. 
Liu \etal \cite{Liu:2010:SCM} successfully tracked
contact-rich mocap sequences using a sampling-based approach. 
They showed that vigorous motions with complex contacts, such as a
dive-roll or a kip-up move, can be dynamically simulated, provided
full body mocap sequences as desired trajectories. 
Other techniques directly edit ballistic motion sequences
under the constraints imposed by conservation of momentum
\cite{Majkowska:2007:FPM,Sok:2010:EDH}, or apply a hybrid method for
synthesizing dynamic response to perturbation in the environment
\cite{Shapiro:2003:HCI}.  If the contact positions and timing are
known, spacetime optimization techniques can also generate compelling
dynamic motions
\cite{Liu:2002:SCD,Fang:2003:ESP,Safonova:2004:SPR,Sulejmanpavic:2004:APB}.
In this work, we take the approach of physical simulation, but we seek
for a more general and robust control algorithm such that the
controller can operate under a wide range of initial conditions and
allow for runtime perturbations. Furthermore, our controller does not
depend on any pre-scripted or captured motion trajectories.

\section{Control of Safe Falling Motions}

Safe falling and landing for bipeds is a topic that
receives broad attention in many disciplines. Robotic researchers are
interested in safe falling from standing height for the purpose of
reducing damages on robots due to accidental falls. Previous work has
applied machine learning techniques to predict falling
\cite{Kalyanakrishnan:2011:LPH}, as well as using an abstract model to
control a safe fall
\cite{Fujiwara:2002:FMC,Fujiwara:2007:OPF,Yun:2009:SFH}. In contrast
to the related work in robotics, our work focuses on falls from higher
places. In those cases, control strategies during long airborne phase
become critical for safe landing. We draw inspiration from
kinesiology literature and sport practitioners. In particular, the
techniques developed in freerunning and parkour community are of
paramount importance for designing landing control algorithms capable
of handling arbitrary scenarios
\cite{Edwardes:2009:TPF,HLJ:2011:URL}. 

\section{Sampling-based Optimization}
In character animation, a sampling-based method, 
Covariance Matrix Adaption Evolution Strategy (CMA-ES) 
\cite{Hansen:2004:CMA}, has been
frequently applied to discontinuous control problems, such as biped
locomotion \cite{Wang:2009:OWC,Wang:2010:OWC,Wang:2012:OLC},
parkour-style stunts\cite{Liu:2012:TRC,Ha:2014:ITD}, or swimming
\cite{Tan:2011:ASC}.  To compensate the expensive cost of
sampling-based algorithm, different approaches have been proposed,
including exploiting the domain knowledge
\cite{Wang:2009:OWC,Wang:2010:OWC,Wang:2012:OLC} or shortening the
problem horizons \cite{Sok:2007:SBB}. Based on the previous success
of CMA-ES, we developed a new sampling-based algorithm resembles the
evolution process of distribution

\section{Human-in-the-loop Optimization}

Without human guidance, fully automated optimization algorithms
sometimes produce undesired solutions due to unmodelled factors or
unexpected situations. To fill the gap, researchers have developed
semi-automatic systems which involve a human in the process to provide
prior knowledge and guidance to the
optimization.\cite{Scott:2002:IHC}. 
This type of optimization systems, called human-in-the-loop (HITL)
optimization, have proven effective for various problems, such as
space shuttle scheduling \cite{Chien:1999:APS}, vehicle planning
\cite{Waters:1984:IVR}, and interface optimization
\cite{Quiroz:2007:IEX}.  The level of user interaction
varies from simply selecting of the generated solutions
\cite{Sims:1991:AEC} to directly editing the search parameters and
constraints \cite{Sreevalsan-Nair:2007:HGE}. Unlike
most previous work which primarily focused on developing user
interaction and visualization techniques for HTIL optimization
systems, we develop new optimization algorithms that exploit the nature of
HITL computation paradigm.


\section{Learning from Demonstration}

Learning from demonstration (LfD), also known as programming by demonstration,
has been an attractive paradigm for training motor skills to robots.
In this paradigm, a set of examples are provided by human teachers,
and an optimal policy is generated from such examples.
Since the early work of Kuniyoshi \etal \cite{kuniyoshi:1989:TBS},
it has been proven to be effective for training motor skills in
numerous task domains, including object manipulation 
\cite{Atkeson:1997:RLD,Calinon:2007:LRG,Ueda:2010:MNH},
navigation \cite{Konidaris:2011:RLD}, 
full-body motion generation \cite{Kulic:2011:ILF}, and so on.
To increase the robustness, the learned motor skills are further 
generalized using machine learning techniques,
such as gausian mixture model \cite{Calinon:2007:LRG} or
motion primitives \cite{Pastor:2009:LGM}.
However, dynamic motor skills of humanoids have
not been fully examined yet, except the only few works on the
locomotion \cite{Nakanishi:2004:LDA}, which is our target domain
in this proposal.