Abstract

Understanding human intent is fundamental to robots that can collaborate naturally and effectively. Intent prediction involves forecasting time-series data – such as human motion trajectories, gaze patterns, and interaction data – to enable machines to anticipate human actions, respond appropriately, and learn from interaction. This capability paves the way for safer, faster, and intuitive human-robot collaboration.

This work presents a framework that combines Imitation Learning techniques with Foundation Models to advance intent-aware robot learning. The approach is demonstrated across diverse tasks, including target prediction in extended reality, human-robot handovers, and multi-robot coordination. By leveraging multimodal cues—such as hand motion, gaze, and interaction history – the system enhances prediction accuracy. Additionally, large language and vision models enable the interpretation of high-level human instructions for task planning and robot navigation. Together, these contributions move toward the goal of Physical AI, where robots can learn from humans and understand and act on their intent in real-world environments.

Bio: Mukund Mitra holds a B.Tech in Mechanical Engineering from NIT Raipur and is currently affiliated with the Robert Bosch Centre for Cyber-Physical Systems at the Indian Institute of Science (IISc) Bangalore. He was awarded the Prime Minister’s Research Fellowship (PMRF), conferred to the top 1% of researchers in India. He has published in top venues including ICRA , ACM IUI , and ACM Transactions on Human-Robot Interaction (THRI).

His research focuses on developing predictive models for Physical Artificial Intelligence, with emphasis on imitation learning and generative models. His work contributes to time-series data prediction, with applications spanning user interface design, eXtended Reality (XR), and motion planning for autonomous systems.