RESEARCH CENTRE

Inria Centre
at Université Côte d’Azur

2023
ACTIVITY REPORT

Project-Team

ACENTAURI

Artificial intelligence and efficient
algorithms for autonomus robotics

DOMAIN

Perception, Cognition and Interaction

THEME

Robotics and Smart environments

Contents

Project-Team ACENTAURI

1 Team members, visitors, external collaborators

2 Overall objectives

3 Research program

3.1 Axis A: Augmented spatio-temporal perception of complex environments . . . . . . . . . .
3.2 Axis B: Situation awareness for decision and planning . . . . . . . . . . . . . . . . . . . . . .
3.3 Axis C: Advanced multi-sensor control of autonomous multi-robot systems . . . . . . . . .

4 Application domains

4.1 Environment monitoring with a collaborative robotic system . . . . . . . . . . . . . . . . . .
4.2 Transportation of people and goods with autonomous connected vehicles . . . . . . . . . .

5 Social and environmental responsibility

5.1 Footprint of research activities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.2 Impact of research results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1

3

4

4
5
6
6

6
7
7

9
9
9

6 Highlights of the year

10
6.1 Team progression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
6.2 Awards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

7 New software, platforms, open data

10
7.1 New software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
7.1.1 OPENROX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
7.2 New platforms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

8 New results

14

9 Bilateral contracts and grants with industry

19
9.1 Bilateral contracts with industry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

10 Partnerships and cooperations

20
10.1 International initiatives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
10.1.1 Associate Teams in the framework of an Inria International Lab or in the framework

10.2 European initiatives

of an Inria International Program . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
10.2.1 Digital Europe . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
10.2.2 Other european programs/initiatives . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
10.3 National initiatives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
10.3.1 ANR project MOBIDEEP (2017-2023)
. . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
10.3.2 ANR project ANNAPOLIS (2022-2025) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
10.3.3 ANR project SAMURAI (2022-2026)
10.3.4 ANR project TIRREX (2021-2029) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
10.3.5 PEPR project NINSAR (2023-2026) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
10.3.6 Defi Inria-Cerema ROAD-AI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
10.3.7 ROBFORISK (2023-2024) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

11 Dissemination

23
11.1 Promoting scientific activities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
11.1.1 Scientific events: organisation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
11.1.2 Scientific events: selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
11.1.3 Journal
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
11.1.4 Leadership within the scientific community . . . . . . . . . . . . . . . . . . . . . . . . 24
11.1.5 Scientific expertise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

11.1.6 Research administration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
11.2 Teaching - Supervision - Juries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
11.2.1 Teaching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
11.2.2 Supervision . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
11.2.3 Juries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
11.3 Popularization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
11.3.1 Articles and contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
11.3.2 Interventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

12 Scientific production

26
12.1 Major publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
12.2 Publications of the year

Project ACENTAURI

1

Project-Team ACENTAURI

Creation of the Project-Team: 2021 May 01

Keywords

Computer sciences and digital sciences

A3.4.1. – Supervised learning

A3.4.3. – Reinforcement learning

A3.4.4. – Optimization and learning

A3.4.5. – Bayesian methods

A3.4.6. – Neural networks

A3.4.8. – Deep learning

A5.4.4. – 3D and spatio-temporal reconstruction

A5.4.5. – Object tracking and motion analysis

A5.4.7. – Visual servoing

A5.10.2. – Perception

A5.10.3. – Planning

A5.10.4. – Robot control

A5.10.5. – Robot interaction (with the environment, humans, other robots)

A5.10.6. – Swarm robotics

A5.10.7. – Learning

A6.2.3. – Probabilistic methods

A6.2.4. – Statistical methods

A6.2.5. – Numerical Linear Algebra

A6.2.6. – Optimization

A6.4.2. – Stochastic control

A6.4.3. – Observability and Controlability

A6.4.4. – Stability and Stabilization

A6.4.6. – Optimal control

A7.1.4. – Quantum algorithms

A8.2. – Optimization

A8.3. – Geometry, Topology

A8.11. – Game Theory

A9.2. – Machine learning

A9.5. – Robotics

A9.6. – Decision support

A9.10. – Hybrid approaches for AI

2

Inria Annual Report 2023

Other research topics and application domains

B5.1. – Factory of the future

B5.6. – Robotic systems

B7.2. – Smart travel

B7.2.1. – Smart vehicles

B7.2.2. – Smart road

B8.2. – Connected city

Project ACENTAURI

3

1 Team members, visitors, external collaborators

Research Scientists

• Ezio Malis [Team leader, INRIA, Senior Researcher, HDR]

• Philippe Martinet [INRIA, Senior Researcher, HDR]

• Patrick Rives [INRIA, Emeritus, HDR]

Post-Doctoral Fellows

• Minh Quan Dao [INRIA, Post-Doctoral Fellow, from Dec 2023]

• Hasan Yilmaz [INRIA, Post-Doctoral Fellow, from Jun 2023]

PhD Students

• Emmanuel Alao [UTC Compiegne, co-supervision]

• Matteo Azzini [UNIV COTE D’AZUR]

• Kaushik Bhowmik [Inria CHROMA, co-supervision]

• Enrico Fiasche [UNIV COTE D’AZUR]

• Monica Fossati [UNIV COTE D’AZUR, from Oct 2023]

• Stefan Larsen [INRIA]

• Fabien Lionti [INRIA]

• Ziming Liu [INRIA]

• Diego Navarro Tellez [CEREMA]

• Mathilde Theunissen [LS2N Nantes, co-supervision]

Technical Staff

• Erwan Amraoui [INRIA, Engineer]

• Marie Aspro [INRIA, Engineer, from Nov 2023]

• Matthias Curet [INRIA, Engineer, from May 2023]

• Pierre Joyet [INRIA, Engineer]

• Pardeep Kumar [INRIA, Engineer, from Aug 2023]

• Quentin Louvel [INRIA, Engineer]

• Louis Verduci [INRIA, Engineer]

Interns and Apprentices

• Thomas Campagnolo [INRIA, Intern, from Sep 2023]

• Monica Fossati [INRIA, Intern, from Feb 2023 until Jul 2023]

Administrative Assistant

• Patricia Riveill [INRIA]

4

Inria Annual Report 2023

2 Overall objectives

The goal of ACENTAURI is to study and to develop intelligent, autonomous and mobile robots that
collaborate between them to achieve challenging tasks in dynamic environments. The team focuses on
perception, decision and control problems for multi-robot collaboration by proposing an original hybrid
model-driven / data driven approach to artificial intelligence and by studying efficient algorithms. The
team focuses on robotic applications like environment monitoring and transportation of people and
goods. In these applications, several robots will share multi-sensor information eventually coming from
infrastructure. The team will demonstrate the effectiveness of the proposed approaches on real robotic
systems like Autonomous Ground Vehicles (AGVs) and Unmanned Aerial Vehicles (UAVs) together with
industrial partners.

The scientific objectives that we want to achieve are to develop:

• robots that are able to perceive in real-time through their sensors unstructured and changing envir-
onments (in space and time) and are able to build large scale semantic representations taking into
account the uncertainty of interpretation and the incompleteness of perception.The main scientific
bottlenecks are (i) how to exceed purely geometric maps to have semantic understanding of the
scene and (ii) how to share these representations between robots having different sensomotoric
capabilities so that they can possibly collaborate together to perform a common task.

• autonomous robots in the sense that they must be able to accomplish complex tasks by taking high-
level cognitive-based decisions without human intervention. The robots evolve in an environment
possibly populated by humans, possibly in collaboration with other robots or communicating with
infrastructure (collaborative perception). The main scientific bottlenecks are (i) how to anticipate
unexpected situations created by unpredictable human behavior using the collaborative perception
of robots and infrastructure and (ii) how to design robust sensor-based control law to ensure robot
integrity and human safety.

• intelligent robots in the sense that they must (i) decide their actions in real-time on the basis of the
semantic interpretation of the state of the environment and their own state (situation awareness),
(ii) manage uncertainty both on sensor, control and dynamic environment (iii) predict in real-time
the future states of the environment taking into account their security and human safety, (iv)
acquire new capacities and skills, or refine existing skills through learning mechanisms.

• efficient algorithms able to process large amount of data and solve hard problems both in robotic
perception, learning, decision and control. The main scientific bottlenecks are (i) how to design
new efficient algorithms to reduce the processing time with ordinary computers and (ii) how to
design new quantum algorithms to reduce the computational complexity in order to solve problems
that are not possible in reasonable time with ordinary computers.

3 Research program

The research program of ACENTAURI will focus on intelligent autonomous systems, which require to
be able to sense, analyze, interpret, know and decide what to do in the presence of dynamic and living
environment. Defining a robotic task in a living and dynamic environment requires to setup a framework
where interactions between the robot or the multi-robots system, the infrastructure and the environ-
ment can be described from a semantic level to a canonical space at different levels of abstraction. This
description will be dynamic and based on the use of sensory memory and short/long term memory
mechanism. This will require to expand and develop (i) the knowledge on the interaction between robots
and the environment (both using model-driven or data-driven approaches), (ii) the knowledge on how to
perceive and control these interactions, (iii) situation awareness, (iv) hybrid architectures (both using
model-driven or data-driven approaches), for monitoring the global process during the execution of the
task.

Figure 1 illustrates an overview of the global systems highlighting the core topics. For the sake of
simplicity, we will decompose our research program in three axes related to Perception, Decision and

Project ACENTAURI

5

Control. However, it must be noticed that these axes are highly interconnected (e.g. there is a duality
between perception and control) and all problems should be addressed in a holistic approach. Moreover,
Machine Learning is in fact transversal to all the robot’s capacities. Our objective is the design and the
development of a parameterizable architecture for Deep Learning (DL) networks incorporating a priori
model-driven knowledge. We plan to do this by choosing specialized architectures depending on the task
assigned to the robot and depending on the input (from standard to future sensor modalities). These DL
networks must be able to encode spatio-temporal representations of the robot’s environment. Indeed, the
task we are interested in considers evolution in time of the environment since the data coming from the
sensors may vary in time even for static elements of the environment. We are also interested to develop
a novel network for situation awareness applications (mainly in the field of autonomous driving, and
proactive navigation).

Figure 1: Intelligent autonomous mobile robot system overview highlighting core axis of research and
methodologies, Machine Learning and Efficient Algorithms.

Another transversal issue concerns the efficiency of the algorithms involved. Either we must process a
large amount of data (for example using a standard full HD camera (1920x1080 pixels) the data size to pro-
cess is around 5 Terabits/hour) or the problem is hard to solve even when the underlying graph is planar.
For example, path optimization problems for multiple robots are all non-deterministic polynomial-time
complete (NP-complete). A particular emphasis will be given to efficient numerical analysis algorithms
(in particular for optimization) that are omnipresent in all research axes. We will also explore a completely
different and radically new methodology with quantum algorithms. Several quantum basic linear algebra
subroutines (BLAS) (Fourier transforms, finding eigenvectors and eigenvalues, solving linear equations)
exhibit exponential quantum speedups over their best known classical counterparts. This quantum
BLAS (qBLAS) translates into quantum speedups for a variety of algorithms including linear algebra,
least-squares fitting, gradient descent, Newton’s method. The quantum methodology is completely new
to the team, therefore the practical interest of pursuing such research direction should be validated in the
long-term.

The research program of ACENTAURI will be decomposed in the following three research axes:

3.1 Axis A: Augmented spatio-temporal perception of complex environments

The long-term objective of this research axis is to build accurate and composite models of large-scale
environments that mix metric, topological and semantic information. Ensuring the consistency of these
various representations during the robot exploration and merging/sharing observations acquired from

6

Inria Annual Report 2023

different viewpoints by several collaborative robots or sensors attached to the infrastructure, are very
difficult problems. This is particularly true when different sensing modalities are involved and when the
environments are time-varying. A recent trend in Simultaneous Localization And Mapping is to augment
low-level maps with semantic interpretation of their content. Indeed, the semantic level of abstraction is
the key element that will allow us to build the robot’s environmental awareness (see Axis B). For example,
the so-called semantic maps have already been used in mobile robot navigation, to improve path planning
methods, mainly by providing the robot with the ability to deal with human-understandable targets. New
studies to derive efficient algorithms for manipulating the hybrid representations (merging, sharing,
updating, filtering) while preserving their consistency are needed for long-term navigation.

3.2 Axis B: Situation awareness for decision and planning

The long-term objective of this research axis is to design and develop a decision-making module that is
able to (i) plan the mission of the robots (global planning), (ii) generate the sub-tasks (local objectives)
necessary to accomplish the mission based on Situation Awareness and (iii) plan the robot paths and/or
sets of actions to accomplish each subtask (local planning). Since we have to face uncertainties, the
decision module must be able to react efficiently in real-time based on the available sensor information
(on-board or attached to an IoT infrastructure) in order to guarantee the safety of humans and things. For
some tasks, it is necessary to coordinate a multi-robots system (centralized strategy), while for other each
robot evolves independently with its own decentralized strategy. In this context, Situation Awareness is at
the heart of an autonomous system in order to feed the decision-making process, but also can be seen as
a way to evaluate the performance of the global process of perception and interpretation in order to build
a safe autonomous system. Situation Awareness is generally divided into three parts: perception of the
elements in the environment (see Axis A), comprehension of the situation, and projection of future states
(prediction and planning). When planning the mission of the robot, the decision-making module will
first assume that the configuration of the multi-robot system is known in advance, for example one robot
on the ground and two robots on the air. However, in our long-term objectives, the number of robots and
their configurations may evolve according to the application objectives to be achieved, particularly in
terms of performance, but also to take into account the dynamic evolution of the environment.

3.3 Axis C: Advanced multi-sensor control of autonomous multi-robot systems

The long-term objective of this research axis is to design multi-sensor (on-board or attached to an
IoT infrastructure) based control of potentially multi-robots systems for tasks where the robots must
navigate into a complex dynamic environment including the presence of humans. This implies that
the controller design must explicitly deal not only with uncertainties and inaccuracies in the models of
the environment and of the sensors, but also to consider constraints to deal with unexpected human
behavior. To deal with uncertainties and inaccuracies in the model, two strategies will be investigated.
The first strategy is to use Stochastic Control techniques that assume known probability distribution
on the uncertainties. The second strategy is to use system identification and reinforcement learning
techniques to deal with differences between the models and the real systems. To deal with unexpected
human behavior, we will investigate Stochastic Model Predictive Control (MPC) techniques and Model
Predictive Path Integral (MPPI) control techniques in order to anticipate future events and take optimal
control actions accordingly. A particular emphasis will be given to the theoretical analysis (observability,
controllability, stability and robustness) of the control laws.

4 Application domains

ACENTAURI focus on two main applications in order to validate our researches using the robotics
platforms described in section 7.2. We are aware that ethical questions may arise when addressing such
applications. ACENTAURI follows the recommendations of the Inria ethical committee like for example
confidentiality issues when processing data (RGPD).

Project ACENTAURI

7

4.1 Environment monitoring with a collaborative robotic system

The first application that we will consider concerns monitoring the environment using an autonomous
multi-robots system composed by ground robots and aerial robots (see Figure 2). The ground robots
will patrol following a planned trajectory and will collaborate with the aerial drones to perform tasks
in structured (e.g. industrial sites), semi-structured (e.g. presence of bridges, dams, buildings) or un-
structured environments (e.g. agricultural space, forest space, destroyed space). In order to provide a
deported perception to the ground robots, an aerial drone will be in operation while the second one
will be recharging its batteries on the ground vehicle. Coordinated and safe autonomous take-off and
landing of the aerial drones will be a key factor to ensure the continuity of service for a long period
of time. Such a multi-robot system can be used to localize survivors in case of disaster or rescue, to
localize and track people or animals (for surveillance purpose), to follow the evolution of vegetation
(or even invasion of insects or parasites), to follow evolution of structures (bridges, dams, buildings,
electrical cables) and to control actions in the environment like for example in agriculture (fertilization,
pollination, harvesting, ...), in forest (rescue), in land (planning firefighting). To successfully achieve such
an application will require to build a representation of the environment and localize the robots in the
map (see Axis A in section 3.1), to re-plan the tasks of each robot when unpredictable events occurs (see
Axis B in section 3.2) and to control each robot to execute the tasks (see Axis C in section 3.3). Depending
on the application field, the scale and the difficulty of the problems to be solved will be increasing. In
the Smart Factories field, we have a relatively small size environment, mostly structured, with highly
instrumented (sensors) and with the possibility to communicate. In the Smart Territories field, we have
large semi-structured or unstructured environments that are not instrumented. To set up demonstrations
of this application, we intend to collaborate with industrial partners and local institutions. For example,
we plan to set up a collaboration with the Parc Naturel Régional des Prealpes d’Azur to monitor the
evolution of fir trees infested by bark beetles.

4.2 Transportation of people and goods with autonomous connected vehicles

The second application that we will consider, concerns the transportation of people and goods with
autonomous connected vehicles (see Figure 3). ACENTAURI will contribute to the development of
Autonomous Connected Vehicles (e.g. Learning, Mapping, Localization, Navigation) and the associated
services (e.g. towing, platooning, taxi). We will develop efficient algorithms to select on-line connected
sensors coming from the infrastructure in order to extend and enhance the embedded perception
of a connected autonomous vehicle. In cities, there exists situations where visibility is very bad for
historical reason or simply occasionally because of traffic congestion, service delivery (trucks, buses)
or roadworks. It exists also situation where danger are more important and where a connected system
or intelligent infrastructure can help to enhance perception and then reduce the risk of accident (see
Axis A in section 3.1). In ACENTAURI, we will also contribute to the development of assistance and
service robotics by re-using the same technologies required in autonomous vehicles. By adding the
social level in the representation of the environment, and using techniques of proactive and social
navigation, we will offer the possibility of the robot to adapt its behavior in presence of humans (see
Axis B in section 3.2). ACENTAURI will study sensing technology on SDVs (Self-Driving Vehicles) used
for material handling to improve efficiency and safety as products are moved around Smart Factories.
These types of robots have the ability to sense and avoid people, as well as unexpected obstructions in
the course of doing its work (see Axis C in section 3.3). The ability to automatically avoid these common
disruptions is a powerful advantage that keeps production running optimally. To set up demonstrations
of this application, we will continue the collaboration with industrial partners (Renault) and with the
Communauté d’Agglomération Sophia Antipolis (CASA). Experiments with 2 autonomous Renault Zoe
cars will be carried out in a dedicated space lend by CASA. Moreover, we propose, with the help of the
Inria Service d’Expérimentation et de Développement (SED), to set up a demonstration of an autonomous
shuttle to transport people in the future extended Inria/UCA site.

8

Inria Annual Report 2023

Figure 2: Environment monitoring with a collaborative robotic system composed by aerial and ground
robots.

Project ACENTAURI

9

Figure 3: Transportation of people and goods with autonomous connected vehicles in human populated
environments.

5 Social and environmental responsibility

ACENTAURI is concerned with the reduction of its environmental footprint activities and it is involved in
several research projects related to the environmental challenges.

5.1 Footprint of research activities

The main footprint of our research activities comes from travels and power consumption (computers
and computer cluster). Concerning travels, after the limitation due to the COVID-19 pandemic, they
have increased again but we make our best efforts to privilegiate visioconferencing. Concerning power
consumption, besides classical actions to reduce the waste of energy, our research focus on efficient op-
timization algorithms to minimize the computation time of computers onboard of our robotic platforms.

5.2 Impact of research results

We have planned to propose several projects related to the environmental challenges. We give below two
examples of the most advanced projects that have been recently started.

The first two concerns the monitoring of forest, one in collaboration with the Parc Naturel Regional
des Préalpes d’Azur, ONF and DRAAF (EPISUD Project) and one in collaboration with INRAE UR629
Ecologie des Forêts Méditerranéennes (see ROBFORISK project in section 10.3.7) .

The third concerns the autonomous vehicles in agricultural application in collaboration with INRAE
Clermont-Ferrand in the context of the PEPR "Agrologie et numérique". We aim to develop robotic
approaches for the realization of new cultural practices, capable of acting as a lever for agroecological
practices (see NINSAR Project in section 10.3.5).

10

Inria Annual Report 2023

6 Highlights of the year

6.1 Team progression

This year that has proven not only promising but also dedicated to the continuous growth of our team. In
particular we have:

• welcomed six new members to our team, each bringing unique skills, experiences, and perspectives.

• kick off three new collaborative projects (Robforrisk, Ninsar, Agrifood-TEF).

• started the AISENSE associated team with the AVELAB of KAIST in Korea.

We organised a two days team seminar in Saint Raphael to foster scientific discussion and collabora-

tions between team members (see Figure 4).

Figure 4: The ACENTAURI team at the seminar in Saint Raphael

We have also initiated a biweekly robotic seminar involving both Inria (HEPHAISTOS and ACENTAURI)
and I3S (OSCAR, ROBOTVISION, ...) robotic teams in order to disseminate information about the latest
advancements, trends, and research in robotics.

6.2 Awards

Mathilde Theunissen won the Best Poster Award at JJCR (Journée des Jeunes Chercheurs en Robotique),
first day of the JNRR (Journées Nationales de la Recherche en Robotique) for her work titled "Multi-robots
localization and navigation for infrastructure monitoring".

7 New software, platforms, open data

7.1 New software

7.1.1 OPENROX

Keywords: Robotics, Library, Localization, Pose estimation, Homography, Mathematical Optimization,

Computer vision, Image processing, Geometry Processing, Real time

Functional Description: Cross-platform C library for real-time robotics:

- sensors calibration - visual identification and tracking - visual odometry - lidar registration and
odometry

Project ACENTAURI

11

Authors: Ezio Malis, Ezio Malis

Contact: Ezio Malis

Partner: Robocortex

7.2 New platforms

Participants:

Ezio Malis, Philippe Martinet, Nicolas Chleq, Emmanuel Alao,
Quentin Louvel.

ICAV platform ICAV platform has been funded by PUV@SOPHIA project (CASA, PACA Region and state),
self funding, Digital Reference Center from UCA, and Academy 1 from UCA. We have now two autonomous
vehicles, one instrumented vehicle, many sensors (RTK GPS, Lidars, Cameras), Communications devices
(C-V2X, IEEE 802.11p), and one standalone localization and mapping system.

ICAV platform is composed of (see Figure 5)

• ICAV1 is an old generation of ZOE. It has been bought fully robotized and intrumented. It is equiped
with Velodyne Lidar VLP16, low cost IMU and GPS, three cameras and one embedded computer.

• ICAV2 is a new generation of ZOE which has been instrumented and robotized in 2021. It is equiped
with Velodyne Lidar VLP16, low cost IMU and GPS, three cameras, two solidstate Lidars RS-M1,
one embedded computer and one NVIDIA Jetson AGX Xavier.

• ICAV3 will be instrumented with different LIDARS and multi cameras system (LADYBUG5+)

• A ground truth RTK system. An RTK GPS base station has been installed and a local server con-
figured inside the Inria Center. Each vehicle is equiped with an RTK GPS receiver and connected to
a local server in order to compute a centimeter localization accuracy.

• A standalone localization and mapping system. This system is composed of a Velodyne Lidar VLP16,

low cost IMU and GPS, and one NVIDIA Jetson AGX Xavier.

• A communication system V2X based on the technology C-V2X and IEEE 802.11p.

• Different lidar sensors (Ouster OS2-128, RS-LIDAR16, RS-LIDAR32, RS-Ruby), and one multi-

cameras system (LADYBUG5+)

The main applications of this platform are:

• datasets acquisition

• localization, Mapping, Depth estimation, Semantization

• autonomous navigation (path following, parking, platooning, ...), proactive navigation in shared

space

• situation awareness and decision making

• V2X communication

• autonomous landing of UAVs on the roof

ICAV2 has been used by Maria Kabtoul in order to demonstrate the effectiveness of autonomous

navigation of a car in a crowd.

12

Inria Annual Report 2023

Figure 5: Overview of ICAV platform (ICAV1, ICAV2-1, ICAV3)

Project ACENTAURI

13

Indoor autonomous mobile platform The mobile robot platform has been funded by the MOBIDEEP
project in order to demonstrate autonomous navigation capabilities in encumbered and crowded envir-
onment. This platform is composed of (see Figure 6):

• one omnidirectional mobile robot (SCOOT MINI with mecanum wheels from AGIL-X)

• one NVIDIA Jetson AGX Xavier for deep learning algorithm implementation

• one general labtop

• one Robosense RS-LIDAR16

• one Ricoh Z1 360° camera

• one Sony RGB-D D455 camera

Figure 6: Overview of MOBIDEEP platform

The main applications of this platform are:

• indoor datasets acquisition

• localization, Mapping, depth estimation, Semantization

• proactive navigation in shared space

• pedestrian detection and tracking

This platform is used in MOBI-DEEP project for integration of different work from the consortium. It is
used to demonstrate new results on social navigation.

14

Inria Annual Report 2023

E-Wheeled platform E-WHEELED is an AMDT Inria project (2019-22) coordinated by Philippe Martinet.
The aim is to provide mobility to things by implementing connectivity techniques. It makes available an
Inria expert engineer (Nicolas Chleq) in ACENTAURI in order to demonstrate the Proof of Concept using
a small size demonstrator (see Figure 7). Due to the COVID19, the project has been delayed.

Figure 7: Overview of E-wheeled platform

8 New results

Navigation of Autonomous Vehicles Around Pedestrians

Participants: Maria Kabtoul, Manon Prédhumeau (LIG Grenoble), Anne Spalanzani,

Julie Dugdale (LIG Grenoble), Philippe Martinet.

The navigation of autonomous vehicles around pedestrians is a key challenge when driving in urban
environments. It is essential to test the proposed navigation system using simulation before moving
to real-life implementation and testing. Evaluating the performance of the system requires the design
of a diverse set of tests which spans the targeted working scenarios and conditions. These tests can
then undergo a process of evaluation using a set of adapted performance metrics. The work presented
in [3] addresses the problem of performance evaluation for an autonomous vehicle in a shared space
with pedestrians. The methodology for designing the test simulations is discussed. Moreover, a group
of performance metrics is proposed to evaluate the different aspects of the navigation: the motion
safety, the quality of the generated trajectory and the comfort of the pedestrians surrounding the vehicle.
Furthermore, the success/fail criterion for each metric is discussed. The implementation of the proposed
evaluation method is illustrated by evaluating the performance of a pre-designed proactive navigation
system using a shared space crowd simulator under Robot Operating System (ROS).

Project ACENTAURI

15

Representation of the environment in autonomous driving applications

Participants:

Ziming Liu, Ezio Malis, Philippe Martinet.

Visual odometry is an important task for the localization of autonomous robots and several ap-
proaches have been proposed in the literature. We consider visual odometry approaches that use stereo
images since the depth of the observed scene can be correctly estimated. These approaches do not
suffer from the scale factor estimation problem of monocular visual odometry approaches that can only
estimate the translation up to a scale factor. Traditional model-based visual odometry approaches are
generally divided into two steps. Firstly, the depths of the observed scene are estimated from the disparity
obtained by matching left and right images. Then, the depths are used to obtain the camera pose. The
depths can be computed for selected features like in sparse methods or for all possible pixels in the image
like in dense direct methods [1].

Recently, more and more end-to-end deep learning visual odometry approaches have been proposed,
including supervised and unsupervised models. However, it has been shown that hybrid visual odometry
approaches can achieve better results by combining a neural network for depth estimation with a
model-based visual odometry approach. Moreover, recent works have shown that deep learning based
approaches can perform much better than the traditional stereo matching approaches and provide more
accurate depth estimation.

State-of-the-art supervised stereo depth estimation networks can be trained with the ground truth
disparity (depth) maps. However, ground truth depth maps are extremely difficult (almost impossible
for dense depth maps) to be obtained for all pixels in real and variable environments. Therefore, some
works explore to train a model on simulated environments and reduce its gap with the real world data.
However, there are only few works focusing on unsupervised stereo matching. Besides using the ground
truth depth maps or unsupervised training, we can also build temporal images reconstruction loss with
ground truth camera pose which is easy to be obtained.

The main contributions of our work published in [2] are (i) a novel pose-supervised network (that
is named PDENet) for dense stereo depth estimation, that can achieve State Of The Art (SOTA) results
without ground truth depth maps and (ii) a dense hybrid stereo visual odometry system, which combines
the deep stereo depth estimation network with a robust model-based Dense Pose Estimation module
(that is named DPE).

In [7] we address the problem of increasing the precision of the previous work. Indeed, machine
learning methods generate hallucinated depths even in areas where it is impossible to estimate the
depth due to several reasons, like occlusions, homogeneous areas, etc. Generally, this produces wrong
depth estimation that leads to errors in odometry estimation. To avoid this problem, we propose a
new approach to generate multiple masks that will be combined to discard wrong pixels and therefore
increase the accuracy of visual odometry. Our key contribution is to use the multiple masks not only in the
odometry computation but also to improve the learning of the neural network for depth map generation.
Experiments on several datasets show that masked dense direct stereo visual odometry provides much
more accurate results than previous approaches in the literature.

Decision making for autonomous driving in urban environment

Participants: Monica Fossati, Philippe Martinet, Ezio Malis.

Autonomous vehicles operating in urban environments face a complex decision-making problem
due to the uncertain and dynamic nature of their surroundings. Successful progress has been made
in assisted and autonomous driving in certain contexts, but fully self-driving vehicles that can operate
safely in a human environment and protect the safety of passengers and those around the vehicle
have yet to be developed. This work focuses on two main aspects of the problem: the ability to learn

16

Inria Annual Report 2023

from experience how to navigate in the environment, thus being able to handle new situations, and
the partial observability of the navigation environment due to uncertainties in perception. To face
the first challenge, reinforcement learning principles were applied, addressing navigation problems
with progressively complex algorithms, ultimately solving a navigation problem on a multi-lane road
using a Deep Q-Network (DQN). Additionally, the partial observability issue is tackled by modeling a
multi-lane peripheral road driving problem as a Partially Observable Markov Decision Process (POMDP)
and employing Partially Observable Monte Carlo Planner (POMCP) and Partially Observable Upper
Confidence Trees (POUTC) for solving it. Performance analysis considers different model precision levels.
The research contributes to enhancing safe and efficient autonomous navigation systems, enabling
vehicles to learn navigation strategies from experience and handle uncertainties in real-world urban
environments.

Autonomous navigation in human populated environment

Participants:

Enrico Fiasché, Philippe Martinet, Ezio Malis.

Recently, robots are widely used around humans, in homes and in public, due to their advantages. In
order to allow the robot and human to share the same space under natural interaction, the ability for
robot to generate a socially acceptable trajectory is the key step. Autonomous robots find difficult to
navigate in a crowded environment due to high level of uncertainty caused by dynamic agents present
in the environment. This uncertainty leads to the so called Freezing Robot Problem (FRP). Many of the
state-of-the-art techniques used to solve this problem, involving collaboration between humans and
robots, are studied.

The work presented in [5] focuses on the control aspects of the autonomous navigation in human
populated environment, using models that are able to predict the future outputs over a certain predic-
tion time horizon, considering the social model for the pedestrian. Rather than letting the robot move
passively to avoid people, the aim of the master thesis is to ensure safe and secure navigation in which
the robot has a proactive role agents and AV collaboration. In particular, the master thesis studied and
analyzed the Model Predictive Control (MPC) technique, focusing on the performances. Different types of
Control Parameterization were used and compared in order to meet real-time feasibility. The Social Force
Model (SFM) is used to predict the pedestrian motions. Simulations and experiments were carried out
and evaluated in different types of social scenarios, demonstrating that the proactive navigation provided
by the model is able to generate secure and safe trajectories avoiding the Freezing Robot Problem.

Multi-Spectral Visual Servoing

Participants:

Enrico Fiasché, Philippe Martinet, Ezio Malis.

The objective of this work is to propose a novel approach for Visual Servoing (VS) using a multispectral
camera, where the number of data are more than three times that of a standard color camera. To meet
real-time feasibility, the multispectral data captured by the camera is processed using a dimensionality
reduction technique. Unlike conventional methods that select a subset of bands, our technique leverages
the best information coming from all the bands of the multispectral camera. This fusion process sacrifices
spectral resolution but keeping spatial resolution, crucial for precise robotic control in forests. The
proposed approach is validated through simulations, demonstrating its efficacy in enabling timely and
accurate VS in natural settings. By leveraging the full spectral information of the camera and preserving
spatial details, our method offers a promising solution for VS in natural environment, surpassing the
limitations of traditional band selection approaches.

Project ACENTAURI

17

Efficient and accurate closed-form solution to pose estimation from 3D correspondences

Participants:

Ezio Malis.

Computing the pose from 3D data acquired in two different frames is of high importance for several
robotic tasks like odometry, SLAM and place recognition. The pose is generally obtained by solving
a least-squares problem given points-to-points, points-to-planes or points to lines correspondences.
The non-linear least-squares problem can be solved by iterative optimization or, more efficiently, in
closed-form by using solvers of polynomial systems.

In [4], a complete and accurate closed-form solution for a weighted least-squares problem has been
studied. Adding weights for each correspondence allow to increase robustness to outliers. Contrary
to existing methods, the proposed approach is complete since it is able to solve the problem in any
non-degenerate case and it is accurate since it is guaranteed to find the global optimal estimate of the
weighted least-squares problem. The key contribution of this work is the combination of a robust u-
resultant solver with the quaternion parametrisation of the rotation, that was not considered in previous
works. Simulations and experiments on real data demonstrate the superior accuracy and robustness of
the proposed algorithm compared to previous approaches.

During this year, we aimed to address the limitations and unanswered questions that emerged from
earlier study and offer novel perspectives and methodologies to advance the field. Notably, one of the
main contributing factors to the limited efficiency observed in the u-resultant approach was the failure to
leverage the two-fold symmetry inherent in the quaternion parametrization, a missed opportunity that
could have led to a more efficient solver. For these reasons, a new approach has been studied that shifts
from utilizing the u-resultant method to adopting the h-resultant approach. The h-resultant approach
consists in hiding one variable considering it as a parameter. Such a hidden variable approach has been
used in the past to solve different problems in computer vision. Therefore, the key contribution of the
current work is the design of a novel efficient h-resultant solver that exploits the two-fold symmetry of the
quaternion parametrisation, that was not considered in previous work. This works has been accepeted
for publication in RA-L 2024.

Reliable Risk Assessment and Management in autonomous driving

Participants:

Emmanuel Alao, Lounis Adouane (UTC Compiegne), Philippe Mar-
tinet.

Autonomous driving in urban scenarios has become more challenging due to the increase in powered
personal mobility platforms (PPMPs). PPMPs correspond mostly to electric devices such as gyropods and
scooters. They exhibit varying velocity profiles as a result of their high acceleration capacity. Multiple
hypotheses about their possible motion makes autonomous driving very difficult, leading to the highly
conservative behavior of most control algorithms. This work proposes to solve this problem by perform-
ing continuous risk assessment using a Probabilistic Fusion of Predictive Inter-Distance Profile (PF-PIDP).
It extends the traditional PIDP to account for uncertainties in the future trajectories of a PPMP. The
proposed method takes advantage of the spatio-temporal characteristic of the PIDP and the robustness of
multi-modal motion prediction methods. A combination of the proposed PF-PIDP and SMPC algorithm
is then used for risk management to mitigate possible collisions and ensure a smooth motion of the
autonomous vehicle for the comfort of passengers and other traffic participants. The advantages of the
proposed approach are demonstrated through simulations of multiple scenarios.

18

Inria Annual Report 2023

Parameter estimation of the lateral vehicle dynamics

Participants:

Fabien Lionti, Nicolas Gutowski (LERIA Angers), Sébastien Au-
bin (DGA), Philippe Martinet.

Estimating the parameters of a nonlinear dynamic system represents a major challenge in many
areas of research and application. In [6] we present a new two-step method to estimate the parameters
that govern the lateral dynamics of a vehicle, taking into account the scarcity and noise of the data.
The method combines spline smoothing of observations of the system with a Bayesian approach to
estimate the parameters. The first step of this method consists of applying a spline smoothing to the
observations of the state variables of the system. This step filters out the noise present in the data and
obtain accurate estimates of the derivatives of system state variables. Thus, the proposed method allows
to estimate the parameters directly from the equations differentials describing the dynamics of the
system, without resorting to to time-consuming integration methods. The second step is based on the
estimation of the parameters from the residues of the differential equations, using a Bayesian approach
called likelihood-free ABC-SMC. This Bayesian approach has several advantages. First of all, it makes it
possible to compensate for the scarcity of data by incorporating a priori knowledge about the physical
characteristics of the vehicle. In addition, it offers a high level of interpretability by providing a posteriori
distribution on the parameters likely to have generated the observed data. Use of this new method has
the advantage of allowing a robust estimation of the parameters of the lateral dynamics of the vehicle,
even when the data are limited and noisy.

A Novel 3D Model Update Framework for Long-Term Autonomy

Participants:

Stefan Larsen, Ezio Malis, El Mustafa Mouaddib, Patrick Rives.

Research on autonomous systems shows increased demands and capabilities within fields such as
surveillance, agriculture and autonomous vehicles. However, the ability to perform long-term opera-
tions without human interventions persists as a bottleneck for real world applications. For long-term
localization and monitoring in dynamic scenes, vision-based robotic systems can benefit from accurate
maps of changes to improve their efficiency. Our work proposes a novel framework for long-term model
update using image-based 3D change localization and segmentation. Specifically, images are used to
detect and locate significant geometric change areas in a pre-made 3D mesh model. New and missing
objects are detected and segmented, to provide model updates for robotic tasks such as localization. The
method for geometric change localization is robust to seasonal, viewpoint, and illumination differences
that may occur between operations. From testing with both our own and publically available datasets,
the framework improves on relevant state-of-the-art methods.

Multi-robots localization and navigation for infrastructure monitoring

Participants: Mathilde Theunissen, Isabelle Fantoni, Ezio Malis, Philippe Martinet.

In the context of the ANR SAMURAI project we studied the interest in leveraging the robot formation
control to enforce the localization precision. This involves optimizing the shape of the formation based
on localizability indices that depend on the sensors used by the robots and the environment. The first
focus is on the definition and optimization of a localizability index applied to collaborative localization
using UWB antennas. This work has been presented at the JJCR (Journée des Jeunes Chercheurs en

Project ACENTAURI

Robotique).

19

9 Bilateral contracts and grants with industry

9.1 Bilateral contracts with industry

ACENTAURI is responsible of two research contracts with Naval Group (2022-2024).

Usine du Futur

Participants:

Ezio Malis, Philippe Martinet, Erwan Emraoui, Marie Aspro, Pierre Al-
liez (TITANE).

The context is that of the factory of the future for Naval Group in Lorient, for submarines and surface
vessels. As input, we have a digital model (for example of a frigate), the equipment assembly schedule
and measurement data (images or Lidar). Most of the components to be mounted are supplied by
subcontractors. At the output, we want to monitor the assembly site to compare the "as-designed" with
the "as-built". The challenge of the contract is a need for coordination on the construction sites for
the planning decision. It is necessary to be able to follow the progress of a real project and check its
conformity using a digital twin. Currently, as you have to see on board to check, inspection rounds are
required to validate the progress as well as the mountability of the equipment: for example, the cabin
and the fasteners must be in place, with holes for the screws, etc. These rounds are time-consuming and
accident-prone, not to mention the constraints of the site, for example the temporary lack of electricity or
the numerous temporary assembly and safety equipment.

La Fontaine

Participants:

Ezio Malis, Philippe Martinet, Hasan Yilmaz, Pierre Joyet.

The context is that of decision support for a collaborative autonomous multi-agent system with
a common objective and who try to get around ”obstacles” which, in turn, try to prevent them from
reaching their goals. As part of a collaboration with NAVAL GROUP, we wish to study a certain number of
issues related to the optimal planning and control of cooperative multi-agent systems. The objective of
this contract is therefore to identify and test methods for generating trajectories responding to a set of
constraints, dictated by the interests, the modes of perception, and the behavior of these actors. The first
problem to study is that of the strategy to adopt during the game. The strategy consists in defining “the
set of coordinated actions, skillful operations, maneuvers with a view to achieving a specific objective”. In
this framework, the main scientific issues are (i) how to formalize the problem (often as optimization of a
cost function) and (ii) how to be able to define several possible strategies while keeping the same tools for
implementation (tactics).

The second problem to study is that of the tactics to be followed during the game in order to implement
the chosen strategy. The tactic consists in defining the tools to execute the strategy. In this context, we
study the use of techniques such as MPC (Model Predictive Control) and MPPI (Model Predictive Path)
Integral) which make it possible to predict the evolution of the system over a given horizon and therefore
to take the best action decision based on knowledge at time t.

The third problem is that of combining the proposed approaches with those based on AI and in
particular the machine learning. Machine Learning can intervene both in the choice of the strategy and in
the development of tactics. The possibility of simulating a large number of parts could allow the learning
of a neural network whose architecture remains to be designed.

20

Inria Annual Report 2023

10 Partnerships and cooperations

10.1 International initiatives

10.1.1 Associate Teams in the framework of an Inria International Lab or in the framework of an

Inria International Program

AISENSE associated team with the AVELAB of Kaist

Participants:

Ezio Malis, Philippe Martinet, Matteo Azzini, Ziming Liu, Minh
Quan Dao.

The main scientific objective of the AISENSE associated team is to study how to build a long-term
perception system in order to acquire situation awareness for safe navigation of autonomous vehicles.
The perception system will perform the fusion of different sensor data (lidar and vision) in order to
localize a vehicle in a dynamic peri-urban environment, to identify and estimate the state (position,
orientatation, velocity, ...) of all possible moving agents (cars, pedestrians, . . . ), and to get high level
semantic information. To achieve such objective, we will compare different methodologies. From
one hand, we will study model-based techniques, for which the rules are pre-defined accordingly to a
given model, that need few data to be setup. On the other hand, we will study end-to-end data-based
techniques, a single neural network for aforementioned multi-tasks (e.g., detection, localization, and
tracking) to be trained with data. We think that the deep analysis and comparison of these techniques will
help us to study how to combine them in a hybrid AI system where model-based knowledge is injected in
neural networks and where neural networks can provide better results when the model is too complex to
be explicitly handled. This problem is hard to solve since it is not clear which is the best way to combine
these two radically different approaches. Finally, the perception information will be used to acquire
situation awareness for safe decision making.

10.2 European initiatives

10.2.1 Digital Europe

Agrifood-TEF (2023-2027)

Participants:

Philippe Martinet, Ezio Malis, Nicolas Chleq, Pardeep Kumar, Mat-
thias Curet.

AGRIFOOD-TEF project is a co-funded project by the Eurpean Union and the different countries
involved. It is organized in three national nodes (Italy, Germany, France) and 5 satellite nodes (Poland,
Belgium, Sweden, Austria and Spain), offers its services to companies and developers from all over Europe
who want to validate their robotics and artificial intelligence solutions for agribusiness under real-life
conditions of use, speeding their transition to the market.

The main objectives are:

• to foster sustainable and efficient food production, AgrifoodTEF empowers innovators with valida-
tion tools needed to bridge the gap between their brightest ideas and successful market products.

• to provides services that help assess and validate third party AI and Robotics solutions in real-world

conditions aiming to maximise impact from digitalisation of the agricultural sector.

Five impact sectors propose tailor-made services for the testing and validation of AI-based and robotic

solutions in the agri-food sector

• Arable farming: testing and validation of robotic, selective weeding and geofencing technologies to
enhance autonomous driving vehicle performances and therefore decrease farmers’ reliance on
traditional agricultural inputs.

Project ACENTAURI

21

• Tree crop: testing and validation of AI solutions supporting optimization of natural resources and
inputs (fertilisers, pesticides, water) for Mediterranean crops (Vineyards, Fruit orchards, Olive
groves).

• Horticulture: testing and validation of AI-based solutions helping to strike the right balance of

nutrients while ensuring the crop and yield quality.

• Livestock farming: testing and validation of AI-based livestock management applications and

organic feed production improving the sustainability of cows, pigs and poultry farming.

• Food processing: testing and validation of standardized data models and self-sovereign data

exchange technologies, providing enhanced traceability in the production and supply chains

Inria will put in place a Moving Living Lab going to the field in order to provide two kind of services:
data collection with mobile ground robot or aerial robot, and mobility algorithms evaluation with mobile
ground robot or aerial robot.

10.2.2 Other european programs/initiatives

The team is part of the euROBIN, the Network of Excellence on AI and robotics that was launched in 2021.

10.3 National initiatives

10.3.1 ANR project MOBIDEEP (2017-2023)

Participants:

Philippe Martinet, Patrick Rives.

Technology-aided MOBIlity by semantic Deep learning: INRIA (ACENTAURI), GREYC, INJA, SAFRAN
(Group, Electronics & Defense). We are involved in Personal assistance for blind people, Proactive
navigation of a robot in human populated environment, and Deep learning in depth estimation and
semantic learning.

10.3.2 ANR project ANNAPOLIS (2022-2025)

Participants:

Philippe Martinet, Ezio Malis, Emmanuel Alao, Kaushik Bhowmik,
Minh Quan Dao, Nicolas Chleq, Quentin Louvel.

AutoNomous Navigation Among Personal mObiLity devIceS: INRIA (ACENTAURI, CHROMA), LS2N,
HEUDIASYC. This project has been accepted in 2021. We are involved in Augmented Perception using
Road Side Unit PPMP detection and tracking, Attention map prediction, and Autonomous navigation in
presence of PPMP.

10.3.3 ANR project SAMURAI (2022-2026)

Participants:

Ezio Malis, Philippe Martinet, Patrick Rives, Nicolas Chleq,
Stefan Larsen, Mathilde Theunissen.

22

Inria Annual Report 2023

ShAreable Mapping using heterogeneoUs sensoRs for collAborative robotIcs: INRIA (ACENTAURI),
LS2N, MIS. This project has been accepted in 2021. We are involved in building Shareable maps of a
dynamic environment using heterogeneous sensors, Collaborative task of heterogeneous robots, and
Update the shareable maps.

10.3.4 ANR project TIRREX (2021-2029)

Participants:

Philippe Martinet, Ezio Malis.

TIRREX is an EQUIPEX+ project funded by ANR and coordinated by N. Marchand. It is composed of
six thematic axis (XXL axis, Humanoid axis, Aerial axis, Autonomous Land axis, Medical axis, Micro-Nano
axis) and three transverse axis (Prototyping & Design, Manipulation, and Open infrastructure). The
kick-off has been done in december 2021. Acentauri is involved in:

• Autonomous Land axis (ROB@t) is coordinated by P. Bonnifait and R. Lenain is covering Autonom-

ous Vehicles and Agricultural robots.

• Aerial Axis is coordinated by I. Fantoni and F. Ruffier.

10.3.5 PEPR project NINSAR (2023-2026)

Participants:

Philippe Martinet, Ezio Malis.

Agroecology and digital. In the framework on this PEPR, ACENTAURI is leading the coordination (R.
Lenain (INRAE), P. Martinet (INRIA), Yann Perrot (CEA)) of a proposal called NINSAR (New ItiNerarieS for
Agroecology using cooperative Robots) accepted in 2022. It gathers 17 research teams from INRIA (3),
INRAE(4), CNRS(7), CEA, UniLasalle, UEVE.

10.3.6 Defi Inria-Cerema ROAD-AI

Participants:

Ezio Malis, Philippe Martinet, Diego Navarro (Cerema), Pierre Joyet.

The aim of the Inria-Cerema ROAD-AI (2021-2024) defi is to invent the asset maintenance of infrastruc-
tures that could be operated in the coming years. This is to offer a significant qualitative leap compared to
traditional methods. Data collection is at the heart of the integrated management of road infrastructure
and engineering structures and could be simplified by deploying fleets of autonomous robots. Indeed,
robots are becoming an essential tool in a wide range of applications. Among these applications, data
acquisition has attracted increasing interest due to the emergence of a new category of robotic vehicles
capable of performing demanding tasks in harsh environments without human supervision.

10.3.7 ROBFORISK (2023-2024)

Project ACENTAURI

23

Participants:

Philippe Martinet, Ezio Malis, Enrico Fiasche, Louis Verduci,
Thomas Boivin (INRAE Avignon), P i l ar
(INRAE
Avignon).

F er nand ez

ROBFORISK (Robot assistance for characterizing entomological risk in forests under climate con-

straints) is a common project between Inria-ACENTAURI and INRAE (UR629 Ecologie des Forêts Méditerranéennes
Three main objectives have been defined : to develop an innovative approach to insect hazard and tree
vulnerability in forests, to tackle the challenge of drone visual servoing using multispectral camera in-
formation in natural environments, and to develop innovative research on pre-symptomatic stress tests
of interest to forest managers and stakeholders.

∗

).

11 Dissemination

Participants:

Ezio Malis, Philippe Martinet.

11.1 Promoting scientific activities

11.1.1 Scientific events: organisation

Member of the organizing committees

• Philippe Martinet has been Editor of IROS23 (102 papers, 17 Associated Editors).

• Philippe Martinet has been Associated Editor of ITSC23 (8 papers).

• Ezio Malis has been Associated Editor of IROS23 (6 papers).

• Ezio Malis has been part of the Scientific Committee of the 3rd Inria-DFKI European Summer

School on AI (IDESSAI 2023).

11.1.2 Scientific events: selection

Member of the conference program committees

• Philippe Martinet is member of the International Conference on Autonomic and Autonomous

Systems (ICAS) 2023 Program Committee.

• Philippe Martinet is member of the International Conference on Models and Technologies for

Intelligent Transportation Systems (MT-ITS) 2023 Program Committee.

• Philippe Martinet is member of the International Conference on Advances in Vehicular Systems,

Technologies and Applications (VEHICULAR) 2023 Program Committee.

• Ezio Malis is member of the International Conference on Robotics,Computer Vision and Intelligent

Sytems (ROBOTVIS) Program Committee.

Reviewer

• Ezio Malis has been reviewer for European Control Conference 2024 (1 paper).

11.1.3 Journal

Member of the editorial boards

• Ezio Malis is Associated Editor of Robotics and Automation Letters in the area “Vision and Sensor-

Based Control”, 6 papers.

24

Inria Annual Report 2023

11.1.4 Leadership within the scientific community

• Philippe Martinet is chair of the Robotics for Unstructured Environment RAS Cluster.

11.1.5 Scientific expertise

• Ezio Malis is expert reviewer for the scientific selection committee ANR CES33 (1 proposal).

• Philippe Martinet is expert for the scientific selection committee ANR CES22 (9 paragraphre-

proposals, 4 proposals).

• Philippe Martinet is expert for the scientific selection committee ANR TSIA (5 proposals).

• Philippe Martinet is expert for the scientific selection committee ANR MOBILEX (2 proposals).

• Philippe Martinet is expert reviewer for the European project BACCHUS.

• Philippe Martinet is expert reviewer for the European project ROBUTCHER.

• Philippe Martinet is expert reviewer for the Call for aid to fund projects for research into climate

change mitigation and adaptation (CLIMA 2023) in Catalonia (3 research applications).

11.1.6 Research administration

• Philippe Martinet is the coordinator of the PEPR project NINSAR 10.3.5.

• Philippe Martinet is the coordinator of the ANR project ANNAPOLIS 10.3.2.

• Philippe Martinet is the coordinator of the ANR project MOBIDEEP 10.3.1.

• Philippe Martinet is the co-coordinator of the INRIA/INRAE project ROBFORISK 10.3.7.

• Philippe Martinet is the local coordinator of the European project AGRIFOOD-TEF 10.2.1.

• Ezio Malis is the coordinator of the ANR project SAMURAI 10.3.3.

• Ezio Malis is the local coordinator of the Defi Inria-CEREMA ROAD-AI 10.3.6.

• Philippe Martinet is member of DS4H COSP

• Ezio Malis is a member of BECP (Bureau des comité de projets) at Centre Inria d’Université Côte

d’Azur

• Ezio Malis is the scientific leader of the Inria - Naval Group partnerships

• Diego Navarro has been in charge of the organisation the Biweekly Robotic Seminar (14 seminars).

• Matteo Azzini has been the social media manager for the Linkedin account of the team.

• Enrico Fiasche has been the manager for the Website and the Youtube channel of the team.

11.2 Teaching - Supervision - Juries

11.2.1 Teaching

• Ezio Malishas taught 10 hours on Signal Processing at Polytech Nice.

Project ACENTAURI

25

11.2.2 Supervision

The team has received 2 master 2 students:

• Monica Fossati (01/02/2023 - 31/11/2023). Subject: "Decision-making for autonomous driving in

urban environment". Supervisors: Philippe Martinet and Ezio Malis.

• Thomas Campagnolo (01/09/2023 - 29/02/2024). Subject: "Detection and tracking of moving
objects with an event based camera on an autonomous car". Supervisors: Philippe Martinet and
Ezio Malis.

The permanent team members supervised the following PhD students:

• Ziming Liu (in progress): Representation of the environment in autonomous driving applications,

(1/12/2020-...) Phd supervisors: Philippe Martinet and Ezio Malis.

• Diego Navarro (in progress): Precise localisation and control of an autonomous multi robot system
for long-term infrastructure inspection, Defi Inria-Cerema ROAD-AI, (1/12/2021-...) Phd super-
visors: Ezio Malis and C. Fauchard, co-supervisors: N.Mitton, P.Martinet, R. Antoine (CEREMA)

• Matteo Azzini (started on 1/10/2022) "Lidar-vision fusion for robust robot localization and map-

ping", Phd supervisors: Ezio Malis (director) and P.Martinet (co-director).

• Enrico Fiasché (started on 1/10/2022) "Modeling and control of a heterogeneous and autonomous

multi-robot system", Phd supervisors: P.Martinet (director) and Ezio Malis (co-director).

• Stefan Larsen (started on 1/10/2022) "Detection of changes and update of environment represent-
ation using sensor data acquired by multiple collaborative robots", Phd supervisors: Ezio Malis
(director) and El Mustapha Mouaddib (MIS Amiens), co-director, P. Rives, co-supervisors.

• Mathilde Theunissen (started on 1/11/2022) "Multi-robot localization and navigation for infrastruc-
ture monitoring", Phd supervisor Isabelle Fantoni, co-supervisors: Ezio Malis and, Philippe Mar-
tinet.

• Fabien Lionti (started on 1/10/2022) "Dynamic behavior evaluation by artificial intelligence: Ap-
plication to the analysis of the safety of the dynamic behavior of a vehicle", Phd supervisors:
Philippe Martinet; co-supervisors: N. Gutoswski (LERIA, Angers), S. Aubin (DGA-TT, Angers).

• Emmanuel Alao (started on 1/10/2022) "Probabilistic risk assessment and management architec-
ture for safe autonomous navigation", Phd supervisors: L. Adouane (Heudiasyc, Compiègne) and
Philippe Martinet.

• Kaushik Bhowmik (started on 1/05/2023) "Modeling and prediction of pedestrian behavior on bikes,
scooters or hoverboards", Phd supervisor Anne Spalanzani, co-supervisors: Philippe Martinet.

• Monica Fossati (started on 1/10/2023) "Navigation sûre en environnement urbain", Phd supervisor:

Philippe Martinet (director) and Ezio Malis (co-director).

11.2.3 Juries

• Philippe Martinet has been reviewer and member of the jury of the PhD of Agapius Bou Ghosn

(CAOR, Paris)

• Philippe Martinet has been reviewer and member of the jury of the PhD of Jorge Garcia Fontan

(LS2N, Nantes)

• Philippe Martinet has been reviewer and member of the jury of the PhD of Alessandro Colotti

(LS2N, Nantes)

• Philippe Martinet has been reviewer and member of the jury of the HdR of Eric Lucet (CEA-LIST,

Paris)

26

Inria Annual Report 2023

• Philippe Martinet has been reviewer and member of the jury of the HdR of Sergio Rodriguez (SATIE,

Paris)

• Ezio Malis has been reviewer of the PhD of Quentin Picard (CEA LIST, Paris)

• Ezio Malis has been member of the jury the PhD of Minh Quan Dao (LS2N, Nantes)

• Ezio Malis has been member of the jury for the Best Thesis Award of the French National Robotic

Network - GdR Robotique

11.3 Popularization

11.3.1 Articles and contents

• Ezio Malis gave an interview for the SOPHIA MAG on IA for autonomous vehicles.

11.3.2 Interventions

• Ezio Malis gave a talk on the future of robotics for celebrating the 40th year anniversary of Inria

Sophia-Antipolis at Palais des Festivals in Cannes

12 Scientific production

12.1 Major publications

[1] A. I. Comport, E. Malis and P. Rives. ‘Real-time Quadrifocal Visual Odometry’. In: The International
Journal of Robotics Research 29.2 (2010), pp. 245–266. DOI: 10.1177/0278364909356601. URL:
https://hal.inria.fr/hal-00766839.

[2] Z. Liu, E. Malis and P. Martinet. ‘A New Dense Hybrid Stereo Visual Odometry Approach’. In: IROS
2022 - 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems. KYOTO, Japan:
IEEE, 23rd Oct. 2022, pp. 6998–7003. DOI: 10.1109/IROS47612.2022.9981814. URL: https://in
ria.hal.science/hal-03935158.

12.2 Publications of the year

International journals

[3] M. Kabtoul, M. Prédhumeau, A. Spalanzani, J. Dugdale and P. Martinet. ‘How To Evaluate the
Navigation of Autonomous Vehicles Around Pedestrians?’ In: IEEE Transactions on Intelligent Trans-
portation Systems (23rd Oct. 2023), pp. 1–11. DOI: 10.1109/TITS.2023.3323662. URL: https://i
nria.hal.science/hal-04255479.

[4] E. Malis. ‘Complete closed-form and accurate solution to pose estimation from 3D correspondences’.
In: IEEE Robotics and Automation Letters 8.3 (Mar. 2023), pp. 1786–1793. DOI: 10.1109/LRA.2023
.3240941. URL: https://hal.science/hal-03957104.

International peer-reviewed conferences

[5] E. Fiasché, P. Martinet and E. Malis. ‘Towards autonomous robot navigation in human populated
environments using an Universal SFM and parametrized MPC’. In: IROS 2023 - IEEE/RSJ Inter-
national Conference on Intelligent Robots and Systems. Detroit (MI), United States, 2023. URL:
https://inria.hal.science/hal-04210032.

[6] F. Lionti, N. Gutowski, S. Aubin and P. Martinet. ‘Towards simulation of radio-frequency component
with physics informed neural networks’. In: CAID 2023 - 5e Conference on Artificial Intelligence for
Defense. Actes de la conférence CAID 2023. Rennes, France, 22nd Nov. 2023. URL: https://hal.sc
ience/hal-04328447.

Project ACENTAURI

27

[7] Z. Liu, E. Malis and P. Martinet. ‘Multi-masks Generation for Increasing Robustness of Dense Direct
Methods’. In: ITSC 2023 - 26th IEEE International Conference on Intelligent Transportation Systems.
Bilbao, Spain, 24th Sept. 2023. URL: https://inria.hal.science/hal-04227583.
<scratchpad>
- Main topic: Research Centre at Université Côte d’Azur on Artificial Intelligence and Efficient Algorithms for Autonomous Robotics
- Key points:
  - Project-Team ACENTAURI focuses on perception, cognition, and interaction for robotics and smart environments
  - Research program includes three axes: Perception, Decision, and Control
  - Application domains: Environment monitoring and transportation of people and goods with autonomous connected vehicles
  - Social and environmental responsibility: Reduction of environmental footprint and impact of research activities
- Structure: The document is divided into sections, including Team members, Project-Team ACENTAURI, Activity Report, Contents, etc.
- Potential artifacts to ignore: Web-related elements such as links, HTML tags, and unnecessary text
</scratchpad>

<final_summary>
The Research Centre at Université Côte d’Azur is focused on developing intelligent, autonomous, and mobile robots that collaborate to achieve complex tasks in dynamic environments. The Project-Team ACENTAURI is a key initiative in this area, with a research program that includes three axes: Perception, Decision, and Control. The team's application domains include environment monitoring and transportation of people and goods with autonomous connected vehicles. The centre also prioritizes social and environmental responsibility, aiming to reduce its environmental footprint and minimize the impact of research activities. Overall, the centre's research aims to advance the field of autonomous robotics and its applications in various sectors.
The Research Centre at Université Côte d’Azur is focused on developing intelligent, autonomous, and mobile robots that collaborate to achieve complex tasks in dynamic environments. The Project-Team ACENTAURI is a key initiative in this area, with a research program that includes three axes: Perception, Decision, and Control. The team's application domains include environment monitoring and transportation of people and goods with autonomous connected vehicles. The centre also prioritizes social and environmental responsibility, aiming to reduce its environmental footprint and minimize the impact of research activities. Overall, the centre's research aims to advance the field of autonomous robotics and its applications in various sectors.