The Work Program follows the following structure. Work packages WP1 to WP4 comprise the core scientific research, namely the memory of motion and its use for sensor-based control. WP5 connects the scientific developments to realistic scenarios so as to achieve embodiment on the robot platforms. WP6 and WP7 focus on testing the memory of motion in laboratory for benchmarking (WP6) and then in three targeted industrial applications (WP7). Assuming that a particular motion problem is given (e.g. stabilize the exoskeleton, navigate to a particular location), our objective is to define a generic methodology to convert this motion problem into an effective controller.

WP1 (exploration of motion capabilities) deals with the problem of automatically solving this motion problem offline, where computation time is not an issue. The objective is rather to use the cloud to solve this problem in many ways. In each iteration, the initial conditions (robot state, position of obstacles and objects of interest, disturbances, etc) are re-sampled. This results in a distribution capturing the robot's motion capabilities. The main scientific problem is to build a motion solver able to find a movement in any realistic initial condition. Additionally, we will study how to efficiently sample the initial conditions, how to massively parallelize the computation, and how to sample incrementally, by capitalizing on previously found solutions. The output of WP1 output a dataset sampling the motion capabilities of the robot.
As mentioned previously, the size of this dataset (>500Tbytes) is too large to allow naive storage.

WP2 (representation and encoding of movement) studies how to compress the movements stored in the dataset. A side benefit will be that this will also reduce the time to access a specific sample. The core problem is to create an efficient representation of movements and skills, by considering multiple representation spaces, geometries, metrics or frames of reference. The problem will be addressed using a probabilistic framework: by extracting invariant patterns from the dataset generated in WP1. We will rely on encoding approaches inspired by the field of Learning from Demonstration. Instead, we will exploiting a different way of acquiring data through offline motion generation (experiences collected by the robot in simulation). The major output of WP2 is a compressed memory of motion, efficiently storing the motion capabilities of the robot to answer the initial motion problem. The role of WP2 is to provide an initial guess to a whole-body MPC problem that will be refined in WP3.

WP3 (whole-body MPC) tackles the problem of generating, in real time on the physical robot, a proper control policy to adopt in the specific situation. It will solve the same initial motion problem as discuss in WP1. However, the objective here is to solve it very quickly (less than 100ms) to cope with the dynamics imposed by the real robot. We will use ``simple but fast'' MPC solvers that are able to converge quickly but with a low radius of convergence. The key to using these approaches will be to draw a good approximation of the soon to be executed movement from the memory of motion. WP3 considers a state-feedback approach: (i) from the latest sensor measurements, the robot state is estimated; (ii) an approximate solution is extracted from the memory of motion; (iii) the approximate movement is quickly refined by the predictive controller and sent to the robot motors. The core scientific problems are in the computation efficiency of MPC sovler and in the connection with the memory of motion on one side and the robot sensors on the other side. The main output of WP3 is a state-feedback MPC exploiting the memory of motion, that can be used as a first solution to control the robot.

WP4 (data-driven control) explores how to go beyond state-feedback MPC by exploiting multi-modal sensing from the robot and coupling them to the MPC scheme. Instead of fusing sensor modalities (IMUs, position and force sensors, etc) in an observer to solely reconstruct the state of the robot, this work package aims to use this information to directly perform feedback control in sensor space. Indeed, MPC is intrinsically a model-based control scheme, which is expected to behave properly on perfectly-modeled robots evolving in a known environment. The objective is to enable the MPC to base its decisions on a prediction of the behavior of the robot in the sensor space. By doing that, it will enable the system to capture important information about the environment that cannot be captured by a state space model (e.g. interaction with an unknown environment). The core scientific problem is to learn sensor models enabling real prediction capabilities. Our research relies on a data-driven control methodology where sensori-motor models capturing the invariant features of a task are learned. These models will then be used to (i) create feedback controllers directly in sensor space that can run in-between each MPC update and (ii) compute MPC controllers in sensor space that complement the controllers from WP3. The main output of WP4 is a sensor-based control methodology which fully exploits robot sensing capabilities.

WP5 (embodiment on the real robots) is a development work package, aimed at implementing several state-of-the-art methods (torque control, model id, visual-haptic perception) that are necessary to deploy the robots in realistic scenarios. We are not expecting to significantly contribute to these problems but will push the state of the art where necessary. The output of WP5 is a corpus of perception and low-level control methods to enable deployment of the memory of motion. This will result in a complete method for high-performance robot control which will be demonstrated in realistic scenarios in WP6.

WP6 (benchmarking in lab trials) corresponds to the experimental validation of the memory of motion in the laboratory using the robots of CNRS, UEDIN and MPI. We target well-identified challenging tasks where rigorous performance criteria are easy to implement: biped locomotion on difficult terrain, multi-contact locomotion and manipulation. We will make use of the benchmarking criteria established in previous EU projects.
The output of WP6 is a systematic validation of the methods based on demonstrators which clearly push the state of the art.

WP7 (case studies in aerospace industry, rehabilitation and civil inspection) takes care of the implementation on real industrial setups supported by the industrial partners. AIRBUS will provide an airplane prototype used to validate the deployment of new production methods. WAN will provide a medical test bed to validate its rehabilitation exoskeleton to support dynamic walking by paralyzed patients. COSTAIN will provide a mock up test environment and UEDIN will coordinate with them to create a duplicate in their laboratory. The output of WP7 is composed of three industrial demonstrators upon which the plans for the exploitation of the results will be based.

WP8 (dissemination, communication and exploitation) will take care of the dissemination of the results of the project, in particular  exploitation plans, while WP9 (project coordination and management) takes care of the organization of the project. WP10 (ethics) take care of the respect of ethical issues during the recruitment of paraplegic patients and for the storage of the data needed for the experiments of Case-Study 2 (exoskeleton).