Human-Machine Interaction

Artificial intelligence techniques, such as deep learning and deep reinforcement learning, are deployed to address the nonlinear and model-infeasible problems involved in human-robot interaction. Ongoing research topics include human pose estimation and dual arm manipulator control. To accurately predict human poses, multiple sensor modalities are employed to enrich the input feature space. A deep neural network, making use of trainable feature extraction and data fusing mechanisms, learns to detect human key points accurately and then a greedy policy is used to assemble those points based on a human skeletal model. A multiple camera sensing module was developed to obtain rich information about the human pose and a neural network is trained to optimize the fusion process for better prediction performance. Dual arm manipulator control research involves sampling efficiency in sparse reward environments; transfer learning in reinforcement learning between different sources, target tasks, and environments; and cooperated manipulation with constrains among others.