MEMMO is a collaborative project in Advanced robot capabilities research and take-up. It is supported by the European Union within the H2020 Program, under Grant Agreement No. 780684. The project starts in January 2018, for 4 years. One year and half after project started, we did an interview with the Scientific Coordinator, Dr Nicolas MANSARD from LAAS-CNRS, to talk about the overall objectives, the work performed from the beginning and the expected results and potential impacts.

Hello, Dr Mansard. The MEMMO Project started one year and half ago. Can you remind us overall objectives of the project?

MEMMO has the ambition to create such a motion-generation technology that will revolutionize the motion capabilities of robots and unlock a large range of industrial and service applications. Based on optimal-control theory, we develop a unified yet tractable approach to motion generation for complex robots with arms and legs. The approach relies on three innovative components:

1) A massive amount of pre-computed optimal motions are generated offline and compressed into a “memory of motion”.

2) These trajectories are recovered during execution and adapted to new situations with real-time model predictive control. This allows generalization to dynamically changing environments.

3) Available sensor modalities (vision, inertial, haptic) are exploited for feedback control which goes beyond the basic robot state with a focus on robust and adaptive behaviour.

In order to demonstrate the generality of the approach, MEMMO is organized around three relevant industrial applications, where MEMMO technologies have a huge innovation potential.

For each application, we will demonstrate the proposed technology in relevant industrial or medical environments, following specifications designed by the end-users partners of the project.

• A high-performance humanoid robot will perform advanced locomotion and industrial tooling tasks in a 1:1 scale demonstrator of a real aircraft assembly (cf. picture 1).
• An advanced exoskeleton paired with a paraplegic patient will demonstrate dynamic walking on flat floor, slopes and stairs, in a rehabilitation centre under medical surveillance (cf. picture 2).
• A challenging inspection task in a real construction site will be performed with a quadruped robot. While challenging, these demonstrators are feasible, as assessed by preliminary results obtained by MEMMO partners that are all experts or stakeholders of their domain (cf. picture 3)

                 Picture 1                                                       Picture 2                                                                            Picture 3 

MEMMO is a collaborative project that seeks a breakthrough in the use of complex robots in open industrial and medical scenarios. Based on the definition of a new concept, the "memory of motion", it will lead to a new technology for robot control that has the potential to impact several market domains. The variety of impacts are emphasized by demonstrating the technology in three relevant environments, defined by our partner stakeholders:

• PAL ROBOTICS in Barcelona, Spain, is targeting the market of mobile robots for end-users like AIRBUS;
• WANDERCRAFT, based in Paris, France, has designed of the most advanced exoskeleton for paraplegics, targeting, for the first time, rehabilitation centres like those handle by the Centre for Physical Medicine and Rehabilitation of APAJH, based in Pionsat, France;
• COSTAIN, a civil-engineering stakeholder, is seeking new solutions for inspection in its constructions sites.

The objectives of the project require an active collaboration joining expertise in motion planning (brought by LAAS-CNRS, Toulouse, France), robot learning (brought by IDIAP, Switzerland), computer vision (brought by University of Oxford, UK), force control (brought by Max-Planck Institute, Tubingen, Germany and University of Trento, Italy), optimal control (brought by University of Edinburgh, UK) and capabilities to set up realistic pilot experiments in robotics.

What activities and tasks did you perform in those first months? What are the main results achieved so far?

The first year has been devoted to setting up the structure and the basis of the project. While the main collaborations inside the consortium were bootstrapped, we built a first prototype of a complete memory of motion for simple robots (inverted pendulum, quadcopters without or with swinging loads). This prototype, formulated as the IREPA algorithm, uses off-the-shelf components (the ACADO trajectory optimizer, a probabilistic roadmap for storing the optimal trajectories, a neural network to approximate and interpolate them) and demonstrated a quick and safe convergence up to dimension 15. The goal of the project is now clear: to scale this algorithm prototype up to dimension 100, the typical dimension of humanoid and other legged robots.

The consortium also released early versions of the final components that should compose the memory of motion. In the first workpackage (dedicated to the generation of the basis of movements using motion planning), we released a locomotion planner and run it on a cluster to produce to locomotion database, totalling around 60 GB of motion data. In the second workpackage (dedicated to the encoding of the motion database into a memory of motion using machine learning), we have explored several possible formulation to learn this mass of data, and decided which strategy we will apply and implement during the second year. In Workpackage 3 (dedicated to optimal control), we have implemented a dedicated optimal control solver for legged robot, and scale it expected performance to 10ms of computation for 1 second of previewed motion of the humanoid robot: these performances go far beyond the state of the art and beat our optimistic expectations. In workpackage 4 (dedicated to sensor-based feedback), we have proposed a model to capture the sensori-motor behaviour of a robot in contact based on measured data, that should later be exploited to close the loop with the optimal control solver. In workpackage 5 (dedicated to embodiment on the robotic platforms), we have design a new version of our exoskeleton, implemented an excellent torque controller on the TALOS robot and prepared dense sensing algorithms that have been tried on the quadruped robots of the project. Finally, the other workpackages have set up the experimental platforms and the benchmark criteria of the project, and prepare the scenarios and the demonstrators corresponding to the three industrial case studies: humanoid co-worker in the factory, exoskeleton for rehabilitation and quadruped for inspection.

What will be the expected results of the project and potential impacts of the project?

In the beginning of the second year, the consortium focuses on releasing a first complete memory of motion encoding the motion database that we already produced, and an efficient implementation of the optimal control solver. Based on these two technical achievements, we should demonstrate a first complete implementation of real-time optimal control on the real robots of the project by the end of the year.

The main idea is that we have basic prototypes of the "memory of motion" concept, running for simplistic systems; and the standalone implementations of all the part of the scale-up version of the memory that we want to implement on our robots:

• We have a full-scale motion databased produced by CNRS with the help of Univ. Oxford, Univ. Edinburg and IDIAP;
• We have an implementation of the machine learning algorithms that we want to use for encoding the memory, produced by IDIAP in collaboration with Max-Planck Institute and Univ. Edinburgh;
• We also have an optimal control solver with sufficient computation efficiency (developed by CNRS with the help of Max-Planck Institute and Univ. Edinburgh) and the sensori-motor feedback models to use it on the robot (developed by Max-Planck Institute with the help of IDIAP);
• Finally, we have the three robots ready and a clear definition of the scenarios exemplifying our practical case studies: humanoid robot in an aerospace factory of the future, exoskeleton for rehabilitation in a medical centre, and quadruped in a civil-engineering environment for inspection.

The work of the second year is to put all these scale-up components in a common architecture, and to demonstrate the validity of the concepts in laboratory. Based on such a success, we would then bring our robots in the real industrial environments in the second part of the project.

Thank you Dr Mansard - A last word?

See you in December 2019 for the first tests of the memory of motion on our three robots.

The interview was done at LAAS-CNRS on the 10^th July 2019 and published on the 11^th July 2019.