直播时间：2024年11月26日（周二）20:00-22:00

 

直播平台：

 

 

科学网APP

 

https://weibo.com/l/wblive/p/show/1022:2321325104963931209758

 

（科学网微博直播间链接）

 

 

科学网微博

 

 

科学网视频号

 

【直播简介】

 

北京时间11月26日晚八点，iCANX Youth Talks第八十三期邀请到了上海交通大学副教授马道林、新加波国立大学助理教授邵林、麻省理工学院杨歌博士主讲，北京大学助理教授仉尚航担任研讨嘉宾，上海交通大学副教授马道林担任主持人，期待你一起加入这场知识盛宴。

 

【嘉宾介绍】

 

 

马道林

 

上海交通大学

 

面向机器人灵巧操作的触觉传感与感知

 

【Abstract】

 

Tactile perception is a critical sensory modality for intelligent robots, and the hardware foundation of tactile perception—tactile sensors—is a key core component for humanoid robots, listed among the 35 "bottleneck" technologies. Whether in intelligent assembly in factories, home assistance scenarios, or in-orbit services in space, and special operations in the deep sea, tactile perception is the key basis for intelligent robots to deal with complex physical contact behaviors in tasks, thus endowing them with sensitive tactile perception capabilities has profound practical significance and academic value. There is a significant gap between the research on traditional array tactile sensors based on principles such as piezoelectricity and pressure sensing and the actual needs of robotic dexterous manipulation. Therefore, it is necessary to develop new tactile perception principles to create tactile sensors for robots that can perceive like human hands and to build the key theoretical and methodological foundation for the integration of tactile sensing and robotic manipulation. The aforementioned theories and methods are inseparable from a deep understanding of contact mechanics and dynamics, as well as the construction and solution of tactile perception inverse problems. This report will introduce the latest research progress in this field by the presenter.

 

触觉感知是智能机器人的关键感知模态，触觉感知的硬件基础--触觉传感器是人形机器人的关键核心基础，位列35项“卡脖子”技术之一。无论是对工厂智能装配、家具生活辅助场景，还是空间在轨服务、深海特种作业场景，触觉感知都是智能机器人应对任务中包含的复杂物理接触行为的关键基础，因此赋予其灵敏的触觉感知能力具有深刻的现实意义和学术价值。传统的基于压电、压感等原理的阵列式触觉传感器研究与机器人灵巧操作的实际需求之间有巨大的鸿沟，因此需要通过新型触觉感知原理的研发，为机器人研发像人手一样感知能力的触觉传感器，并为触觉传感与机器人操作的结合构筑关键的理论和方法基础。上述理论和方法都离不开对接触力学及动力学的深刻理解以及对其触觉感知反问题的构建和求解，本报告将介绍报告人在此领域的最新研究进展。

 

【BIOGRAPHY】

 

Ma Daolin, Associate Professor and Ph.D. supervisor at the School of Naval Architecture, Ocean and Civil Engineering, Shanghai Jiao Tong University, "Bo Le Plan" mentor. He completed his undergraduate, masters, and doctoral studies in the Mechanics Department at Peking University from 2005 to 2015. From 2016 to 2021, he conducted postdoctoral research in the Department of Mechanical Engineering at the Massachusetts Institute of Technology. In 2021, he joined the School of Naval Architecture, Ocean and Civil Engineering at Shanghai Jiao Tong University and established the "Manipulation Perception and Intelligence Lab (MPI Lab)." The lab is dedicated to developing advanced tactile sensing, perception, and control technologies, endowing robots with the intelligence to interact autonomously with the physical environment, enabling them to independently complete operations rich in contact behaviors in engineering applications such as advanced manufacturing and deep space and deep-sea exploration.

 

He has achieved innovative results in the fields of contact modeling, tactile sensing, and contact perception, receiving the best conference paper award at ICRA, the most influential international conference in the field of robotics (1/4056), winning the Amazon Robotics Challenge Stow Task championship, and the Amazon Robotics Best Systems Paper Award, among others. He has been selected for the JKW Talent Program, the "Young Talent Reservoir" project of the Chinese Society of Theoretical and Applied Mechanics, and the Shanghai Overseas High-Level Talents program. He leads the National Natural Science Foundation of Chinas general and youth projects and presides over horizontal projects on robot intelligence with CATL and Siemens, guiding students to win the national gold award in the 2023 "China International College Students Innovation Competition" (formerly known as the "Internet+" competition).

 

马道林，上海交通大学船建学院副教授，博士生导师，“伯乐计划”导师。本硕博于2005-2015年在北京大学力学专业就读。2016-2021年于麻省理工学院机械系开展博士后工作，2021年加入上海交通大学船建学院，建立了“机器人操作感知与智能实验室”（Manipulation Perception and Intelligence Lab, MPI Lab）。实验室致力于开发先进的触觉传感、感知和控制技术，赋予机器人与物理环境自主交互的智能，使其在先进制造、深空深海探测等工程应用中自主完成富含接触行为的操作任务。在接触建模、触觉传感与接触感知等领域取得创新成果，获得机器人领域最有影响力的国际会议ICRA最佳会议论文奖（1/4056），获亚马逊机器人挑战赛Stow Task冠军、亚马逊机器人最佳系统论文奖等。入选JKW人才项目、中国力学学会“青年人才蓄水池”项目、上海市海外高层次人才等，主持国家自然科学基金面上项目、青年项目，主持宁德时代、西门子的机器人智能化横向项目，指导学生获得2023年“中国国际大学生创新大赛”（原“互联网+”大赛）国赛金奖。

 

 

邵林

 

新加坡国立大学

 

跨多样化对象、机器人和任务的生成性规划与接触合成：迈向通用机器人操控的基础模型

 

【ABSTRACT】

 

To substantially advance robot intelligence, there is a pressing need to develop a foundation model that enables robots to perform a wide range of manipulation tasks. Developing such a model requires integrating high-level planning with low-level action generation across diverse objects, robots, and tasks, allowing robots to operate effectively in unstructured environments.In this talk, I will introduce two recent approaches that address high-level generation planning and low-level contact synthesis. First, I will present FLIP, a flow-centric generative planning framework. FLIP supports general-purpose task planning by synthesizing long-horizon plans from an initial image and language instruction. By representing actions through flows, FLIP not only facilitates planning across diverse objects, robots, and tasks but also provides rich guidance for long-horizon video generation. Next, I will discuss our contact synthesis model, which tackles the extensive variability in objects, robots, and tasks by framing manipulation as a contact synthesis problem. The model takes as input object and robot manipulator point clouds, object physical attributes, target motions, and manipulation region masks. It outputs contact points on the object and associated contact forces or post-contact motions for robots to achieve the desired manipulation task. Our results demonstrate that this model can guide a variety of robots to manipulate rigid, articulated, and 1D/2D/3D deformable objects across diverse tasks, achieving an average success rate of 90%. I will conclude this talk by discussing my future research directions.

 

为了大幅提高机器人智能，迫切需要开发一个基础模型，使机器人能够执行广泛的操控任务。开发这样一个模型需要将高级规划与低级动作生成整合在一起，跨越多样化的对象、机器人和任务，使机器人能够在非结构化环境中有效操作。在这次演讲中，我将介绍两种最近的方法，它们分别解决高级生成规划和低级接触合成问题。首先，我将介绍FLIP，这是一个以流程为中心的生成性规划框架。FLIP通过从初始图像和语言指令合成长期规划来支持通用任务规划。通过用流程表示动作，FLIP不仅促进了跨多样化对象、机器人和任务的规划，还为长期视频生成提供了丰富的指导。接下来，我将讨论我们的接触合成模型，该模型通过将操控问题框架化为接触合成问题，解决了对象、机器人和任务的广泛变异性。模型以对象和机器人操作器的点云、对象物理属性、目标运动和操控区域掩码作为输入。它输出对象上的接触点以及相关的接触力或接触后运动，以实现机器人的期望操控任务。我们的结果表明，这个模型可以引导各种机器人操作刚性、铰接和1D/2D/3D可变形对象，跨越多样化的任务，平均成功率达到90%。我将通过讨论我的未来研究方向来结束这次演讲。

 

【BIOGRAPHY】

 

Lin Shao is an Assistant Professor in the Department of Computer Science at the School of Computing, National University of Singapore (NUS). His research interests lie at the intersection of Robotics and Artificial Intelligence. His long-term goal is to build general-purpose robotic systems that intelligently perform a diverse range of tasks in a large variety of environments in the physical world. His lab focuses on developing algorithms and systems that equip robots with robust perception and manipulation capabilities. He serves as the co-chair of the Technical Committee on Robot Learning in the IEEE Robotics and Automation Society and an Associate Editor for IEEE Robotics and Automation Letters and for ICRA in 2024 and 2025. His work received the Best System Paper Award finalist at RSS 2023. Previously, he received his PhD at Stanford University, advised by Jeannette Bohg.

 

邵林是新加坡国立大学（NUS）计算机学院计算学部的助理教授。他的研究兴趣位于机器人学和人工智能的交叉领域。他的长期目标是构建通用的机器人系统，这些系统能够在物理世界的各种环境中智能地执行多样化的任务。他的实验室专注于开发算法和系统，使机器人具备强大的感知和操控能力。他担任IEEE机器人与自动化学会机器人学习技术委员会的联合主席，以及IEEE机器人与自动化信函和2024年及2025年ICRA的副编辑。他的工作获得了2023年RSS最佳系统论文奖的决赛入围者。此前，他在斯坦福大学获得博士学位，导师是Jeannette Bohg。

 

 

杨歌

 

麻省理工学院

 

从合成数据中进行机器人学习的可扩展方案

 

【ABSTRACT】

 

AI breakthroughs in the past decade have been driven mainly by increasing the quantity and quality of data. Unlike images and text that are abundantly available from online sources, robotic datasets are magnitudes more scarce. While many believe that collecting more real-world demonstrations will eventually yield a generalist robot, my research focuses instead on developing the systems and techniques to enable closed-loop training on synthetic data. In this talk, I will show you how to make synthetic data significantly more abundant and systematically varied than real-world datasets with the help of generative AI. I will present LucidSim, our new milestone towards solving key problems in both robotics and general intelligence.

 

过去十年中，人工智能的突破主要得益于数据量和质量的增加。与可以从在线来源大量获取的图像和文本不同，机器人数据集要稀缺得多。尽管许多人认为收集更多的现实世界演示最终将产生一个通用型机器人，我的研究重点却在开发系统和技术，以实现基于合成数据的闭环训练。在这次演讲中，我将展示如何在生成性人工智能的帮助下，使合成数据比现实世界数据集更加丰富和系统性地多样化。我将介绍LucidSim，这是我们在解决机器人学和通用智能中的关键问题上迈出的新里程碑。

 

【BIOGRAPHY】

 

Ge Yang studies robot learning and intelligence. He believes simulators powered by generative AI might offer a better route toward generalist robots than real-world data alone. He is also building a data foundry, vuer.ai, and developing closed-loop evaluation using high-fidelity digital twins via the Neverwhere project. Ge is currently a postdoc with Phillip Isola at MIT CSAIL and works closely with Xiaolong Wang at UCSD. He is a recipient of the NSF IAIFI Postdoc Fellowship and the Best Paper Award at The Conference of Robot Learning (CoRL) in 2023. Ge graduated from UChicago with a Ph.D. in Physics advised by David I. Schuster (now at Stanford), and Yale University with a B.S. in Mathematics and Physics. 

 

杨歌博士研究机器人学习和智能。他认为，由生成性人工智能驱动的模拟器可能比单独使用现实世界数据提供了一条更好的通往通用型机器人的途径。他还正在建立一个数据铸造厂vuer.ai，并通过网络未知项目开发使用高保真数字孪生的闭环评估。目前在麻省理工学院CSAIL与Phillip Isola合作进行博士后研究，并与加州大学圣地亚哥分校的王晓龙密切合作。杨歌是NSF IAIFI博士后奖学金和2023年机器人学习会议（CoRL）最佳论文奖的获得者。杨歌在芝加哥大学获得物理学博士学位，导师是David I. Schuster（现任斯坦福大学），并在耶鲁大学获得数学和物理学学士学位。

 

【主持人介绍】

 

 

马道林

 

上海交通大学

 

【研讨嘉宾介绍】

 

 

仉尚航

 

北京大学