RESEARCH CENTRE

Inria Centre
at Université de Lorraine

IN PARTNERSHIP WITH:

CNRS, Université de Strasbourg

2023
ACTIVITY REPORT

Project-Team

MIMESIS

Computational Anatomy and Simulation
for Medicine

IN COLLABORATION WITH: ICube

DOMAIN

Digital Health, Biology and Earth

THEME

Computational Neuroscience and
Medicine

Contents

Project-Team MIMESIS

1 Team members, visitors, external collaborators

2 Overall objectives

2.1 Team Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.2 Challenges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

3 Research program

3.1 Interactive computational models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.1.1 Immersed Boundary Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.1.2 Scientific Machine Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.1.3 Non-Smooth Mechanics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.1.4 Order in random systems by stochastic perturbations . . . . . . . . . . . . . . . . . .
3.2 Optimization and control for Computer-Aided-Interventions . . . . . . . . . . . . . . . . . .
3.2.1 Optimal control and differentiable simulation . . . . . . . . . . . . . . . . . . . . . . .
3.2.2 Data Assimilation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.2.3 Closed loop control for neurostimulation . . . . . . . . . . . . . . . . . . . . . . . . . .
3.2.4 Control in medical robotics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

4 Application domains

4.1 Patient-specific stent design through shape optimization . . . . . . . . . . . . . . . . . . . .
4.2 Computer-based surgical training . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.3 Pre-operative planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.4 Intra-operative guidance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.5 Feedback control of neural activity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.6 Open source software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

5 Highlights of the year

6 New software, platforms, open data

6.1 New software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.1.1 DeepPhysX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.1.2 SSD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1

2

3
3
3

4
4
4
4
5
5
5
5
5
6
6

6
6
7
7
7
7
7

8

8
8
8
9

7 New results

9
7.1 Digital twins of organs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
9
7.2 Accurate and real-time digital twin of the liver . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
7.3 Physics-informed neural networks for organ shape prediction from fluoroscopic images . . 10
7.4 Physically inspired regularization for deformable image registration . . . . . . . . . . . . . . 10
7.5 Patient-specific stent design through shape optimization . . . . . . . . . . . . . . . . . . . . 11
7.6 Recovering surgically induced deformations of the liver . . . . . . . . . . . . . . . . . . . . . 11
7.7 Non-smooth mechanics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
7.8 Order in random systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
7.9 Dynamics in systems under neurostimulation . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
7.10 Binaural beats affect the attention in healthy and attention-deficit human subjects . . . . . 12
7.11 Optimal control to limit epidemia . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
7.12 Error control in biomechanics
7.13 Optimal control of computer-aided interventions . . . . . . . . . . . . . . . . . . . . . . . . . 13

8 Bilateral contracts and grants with industry

14
8.1 Bilateral contracts with industry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

9 Partnerships and cooperations

9.1 International research visitors

14
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
9.1.1 Visits of international scientists . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
9.2 National initiatives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
9.3 Regional initiatives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

10 Dissemination

18
10.1 Promoting scientific activities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
10.1.1 Scientific events: organisation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
10.1.2 Scientific events: selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
10.1.3 Journal
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
10.1.4 Invited talks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
10.1.5 Scientific expertise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
10.1.6 Research administration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
10.2 Teaching - Supervision - Juries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
10.2.1 Teaching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
10.2.2 Supervision . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
10.2.3 Juries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
10.3 Popularization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
10.3.1 Interventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

11 Scientific production

21
11.1 Major publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
11.2 Publications of the year

Project MIMESIS

1

Project-Team MIMESIS

Creation of the Project-Team: 2021 May 01

Keywords

Computer sciences and digital sciences

A2.5. – Software engineering

A3.1.1. – Modeling, representation

A3.1.4. – Uncertain data

A3.2.2. – Knowledge extraction, cleaning

A5.1. – Human-Computer Interaction

A5.3.4. – Registration

A5.4.4. – 3D and spatio-temporal reconstruction

A5.4.5. – Object tracking and motion analysis

A5.6. – Virtual reality, augmented reality

A6.1.1. – Continuous Modeling (PDE, ODE)

A6.1.2. – Stochastic Modeling

A6.1.5. – Multiphysics modeling

A6.2.3. – Probabilistic methods

A6.2.4. – Statistical methods

A6.2.5. – Numerical Linear Algebra

A6.2.8. – Computational geometry and meshes

A6.3.1. – Inverse problems

A6.3.2. – Data assimilation

A6.3.3. – Data processing

A6.3.4. – Model reduction

A9.2. – Machine learning

A9.10. – Hybrid approaches for AI

Other research topics and application domains

B1.2. – Neuroscience and cognitive science

B2.2.6. – Neurodegenerative diseases

B2.4. – Therapies

B2.4.3. – Surgery

B2.6. – Biological and medical imaging

B2.7. – Medical devices

B2.7.1. – Surgical devices

2

Inria Annual Report 2023

1 Team members, visitors, external collaborators

Research Scientists

• Stephane Cotin [Team leader, INRIA, Senior Researcher, HDR]

• Hadrien Courtecuisse [CNRS, Researcher, HDR]

• Michel Duprez [INRIA, Researcher]

• Axel Hutt [INRIA, Senior Researcher, HDR]

Faculty Member

• Vanessa Lleras [UNIV MONTPELLIER, Associate Professor Delegation, until Aug 2023]

Post-Doctoral Fellow

• Pablo Arturo Alvarez Corrales [INRIA, Post-Doctoral Fellow]

PhD Students

• Sidaty El Hadramy [INRIA]

• Thuc Long Ha [UNIV STRASBOURG]

• François Lecomte [INRIA]

• Frederique Lecourtier [INRIA, from Oct 2023]

• Claire Martin [INRIA]

• Alban Odot [INRIA, until Apr 2023]

• Josephine Riedinger [INRIA]

• Valentina Scarponi [UNIV STRASBOURG]

• Killian Vuillemot [UNIV MONTPELLIER]

• Thomas Wahl [INRIA]

• Nicola Zotto [UNIV STRASBOURG]

Technical Staff

• Paul Baksic [UNIV STRASBOURG, Engineer, until Jun 2023]

• Robin Enjalbert [INRIA, Engineer]

• Vincent Italiano [INRIA, Engineer]

• Antoine Oberle [CNRS, Engineer, from Sep 2023]

Interns and Apprentices

• Gabriel Alves Castro [UNIV STRASBOURG, from Mar 2023 until Aug 2023]

• Chiara Cignolini [UNIV STRASBOURG, Intern, until Sep 2023]

• Francesco Dettori [University of Strasbourg, from Oct 2023]

• Frederique Lecourtier [UNIV STRASBOURG, from Feb 2023 until Jul 2023]

• Valentin Rieth [UNIV STRASBOURG, from Feb 2023 until Aug 2023]

Project MIMESIS

3

Administrative Assistant

• Ouiza Herbi [INRIA]

Visiting Scientist

• Alexei Lozinski [UNIV FRANCHE-COMTE, until Jan 2023, HDR]

External Collaborators

• Nazim Haouchine [HARVARD MEDICAL SCHOOL]

• Alexei Lozinski [UNIV FRANCHE-COMTE, until Jan 2023, HDR]

• Didier Pinault [INSERM, until Nov 2023, HDR]

• Juan Verde [IHU STRASBOURG, from Aug 2023]

2 Overall objectives

2.1 Team Overview

The MIMESIS team develops numerical methods for computer-based training, surgical planning and
computer-assisted interventions. Our aim is to facilitate the development of digital twins and improve
their predictive capabilities through novel numerical methods, data assimilation techniques and machine
learning. To pursue these directions we have assembled a team with a multidisciplinary background,
and have established close collaborations with academic and clinical partners. We contribute to the
development of the SOFA framework as a means to disseminate our results to the community.

2.2 Challenges

In a first research axis, our research aims at developing more advanced simulations, with stronger
mathematical and biomechanical foundations, and their adaptation to a specific patient. We want, in
particular, to develop more stable numerical methods that would, at the same time, be suited for the
automatic generation of digital twins of organs. We chose to investigate immersed boundary methods
that do not require an exact discretization of the domain. It led to the development of Φ-FEM, a finite
element method on domains defined by level-sets that provides better convergence while significantly
simplifying the mesh generation process. To achieve real-time computation using complex models
based on nonlinear PDEs, we investigate the use of deep learning techniques for accelerating physics-
based simulations. Given that our main field of application is computer-guided surgery, we also study
the numerical aspects of contact problems. Such problems are very hard to solve in real time, as they
involve non-smooth dynamics. In addition, we investigate data assimilation methods to estimate patient-
specific model parameters from sparse observations, as they typically occur during surgical interventions.
Random processes play an important role in biological systems and need specific mathematical analysis.
In neuroscience, stochastic fluctuations have been shown to control oscillatory brain states that may
be induced by neurostimulation in clinical practice. To improve our understanding of nonlinear neural
dynamics during neurostimulation and thus improve experimental neurostimulation protocols in single
patients, we have extended the mathematical analysis of additive noise-induced system evolution.

Our second research axis is derived from our application context and essentially consists of developing
optimization and control methods for computer-assisted interventions. At the core of our activity is
the hypothesis that data-driven simulation has the potential to bridge the gap between medical data
(most often images) and clinical routine by updating pre-operative knowledge with the information
available at the time of the procedure. We have been pioneers in the use of biomechanical models
and real-time finite element simulation to perform augmented reality on deformable organs. In the
field of non-rigid registration, we have demonstrated the benefit of our physics-based approaches as
they bring not only an implicit regularization of the solution but also a plausible explanation of the

4

Inria Annual Report 2023

deformation. We have continued this work with an emphasis on robustness to uncertainty and outliers in
the information extracted in real-time from image data, and in real-time parameter estimation. The work,
initially done using Bayesian methods, has been recently extended through optimal control methods.
Optimal control may be achieved in a closed feedback loop and we investigate its implementation in a
neurostimulation setup. The objective of this loop is to control the spectral power density of a subject’s
brain activity. In a Kalman observer feedback model, brain observations are fed into a Kalman Filter
which estimates instantaneous stimulation input to the brain in real-time. Another approach treats the
brain as a linear time-invariant system and optimizes the brain stimulation input by direct feedback
control based on observed brain signals. Finally, in the field of robotic control of flexible structures,
we are developing methods for autonomous insertion of needles during percutaneous procedures and
autonomous navigation of catheters and guidewires during endovascular interventions. This research
combines our expertise in real-time simulation of deformable structures and our results in contact
modeling, while the control is either performed via differentiable simulations or using reinforcement
learning methods.

3 Research program

3.1 Interactive computational models

3.1.1 Immersed Boundary Methods

For several years, we have investigated finite element methods that fall under the class of unfitted (also
known as immersed boundary) methods. Because such methods do not require a discretization that
strictly conforms to the domain boundary, they are particularly suited for the development of digital twins,
as they facilitate the automatic generation of patient-specific simulations on complex geometries. Our
current results have focused on the development, the numerical analysis and mathematical foundations
of a new method called Φ-FEM. The main advantage of our method is that it uses the classical Finite
Element tools on unfitted meshes. We have also highlighted that the method significantly improves
the convergence when compared to a similar, fitted, discretization of the domain. The benefit and
applicability of the Φ-FEM method have already been demonstrated on different problems: heat transfer,
crack, interface between two material, and fluid structure interaction. We are now working on extending
the method to 3D (non-linear) elasticity with both Neumann and Dirichlet boundary conditions.

3.1.2 Scientific Machine Learning

Various dimensionality reduction techniques have been proposed to speed up the simulation of non-
linear solids. These techniques remain essentially linear transformations that may, in some cases, be
insufficient to capture correctly the nonlinearity that can be found in biological tissues. In 2018, we
proposed to address these limitations by developing physics-based deep neural networks, a supervised
version of the Physically Informed Neural Networks (PINNs) approach. As preliminary work, we developed
a neural network that learns the relationship between an input force and the resulting displacement field
via a U-Net architecture [4]. The choice of this U-net architecture is not random, as it matches the model
reduction process through a neural network (with the latent space representing the reduced model). The
predictions were about 100 times faster than our reference FEM computation while providing sufficient
accuracy for our applications. This work was followed by several new ideas and publications. We proposed
to solve elastic registration problems for augmented surgery using the same network architecture but
trained to learn a mapping between a partial, noisy point cloud and the deformed state of an elastic body.
We added a physics-based loss to a fully connected network to improve its prediction accuracy and to
make it “physics-aware”. We also enforced the robustness of the solution by integrating the prediction of
the network within a classical Newton Raphson algorithm. Our next steps concern the extension of our
methods to the case of dynamic simulation, and also the development of neural network architectures
that provide more genericity in the solution space (by being somewhat invariant to the meshing of the
domain for instance).

Project MIMESIS

5

3.1.3 Non-Smooth Mechanics

Many clinical interventions rely on mechanical interactions between a device and the anatomy. Surgery
and interventional radiology are the two main areas where these interactions are essential and generally
complex. During this evaluation period we have continued our activity on constraint modeling and
simulation of contacts with friction. The complexity of the problem often leads to unstable numerical
approaches, inaccuracy of the solution and slow computation times. Our latest works address the issue of
computational performance through an asynchronous Cholesky decomposition and by exploiting matrix
sparsity and time-consistency.

3.1.4 Order in random systems by stochastic perturbations

The brain is a complex system with several spatial and temporal scales. The microscopic scales are rather
unstructured in space and activity observations show random fluctuations, whereas upper hierarchical
levels at the mesoscopic and macroscopic scales exhibit more regular dynamics. Some years ago, we
found in theoretical studies that additive random input on the microscopic scale to random networks
tunes the systems stability and may induce stability change. Such so-called bifurcations induce ordered
structures, being in space or time or in both. For instance, additive noise may induce coherence resonance
in Erd˝os-Rényi neural networks which exhibit an oscillatory spatially coherent state with frequency of
40 Hz. Since this oscillatory state evolves in the gamma frequency range in which visual information
is processed, this theoretical result indicates that the brain may tune internal random fluctuations to
induce coherence resonance and hence process visual information.

3.2 Optimization and control for Computer-Aided-Interventions

3.2.1 Optimal control and differentiable simulation

We recently started to investigate optimal control as a solution to several challenges in the field of
computer-aided intervention. The first problem we addressed was the elastic intraoperative registration
of a preoperative liver model and, as an extension, force estimation during robotic surgery. Both are
formulated as optimal control problems where the unknown is the surface force distribution applied onto
the organ. The deformation is computed using an hyperelastic model, and the optimization problem
is solved using an adjoint method. This approach provides greater control over the set of admissible
forces and leads to physically-consistent displacement fields. As a byproduct, it permits the recovery of
the forces that led to the deformation, and can provide visual haptic feedback in medical robotics. We
also rely on optimal control in other problems addressed in the team, such as autonomous endovascular
navigation, where we have a similar control challenge. In this case the deformable structure is the catheter
or guidewire rather than an organ. Finally, to address the relatively slow computation times that are
typical of such optimization methods, we have started the development of differentiable algorithms. They
exploit both GPU acceleration and the automatic differentiation tools from frameworks such as PyTorch.
This significantly speeds up the computation of the adjoint. This approach is combined with the use of
deep neural networks to predict the solution of the forward elastic problem in just a few milliseconds.

3.2.2 Data Assimilation

An essential step in the development of digital twins is the parametrization of the underlying model.
A possibility is to rely on Bayesian inference. Model parameters are estimated using a reduced-order
unscented Kalman filter based on observations taken, for instance, from laparoscopic image streams. As
an example, estimating boundary conditions rather than using generic data from the literature improves
the accuracy of the simulation by 75% when compared to methods integrating a priori knowledge as
boundary conditions. Our approach significantly improves the accuracy during registration problems
involving sparse and noisy observations. We also demonstrated that our data assimilation method can be
used for the reconstruction of the spatiotemporal profile of soft tissue temperature evolution during laser
irradiation.

6

Inria Annual Report 2023

3.2.3 Closed loop control for neurostimulation

Today typical clinical neurostimulation, such as transcranial electric or magnetic stimulation, applies
stimulation protocols with pre-defined parameters, such as stimulation intensity, duration and inter-
stimulus interval. However, such patient-unspecific open-loop stimulation protocols may not be effective
for all patients. We propose to develop a closed-loop neurostimulation protocol that estimates the optimal
stimulation intensity in real-time adaptively from observed brain activity. The new stimulation protocol
aims to tune the brain’s spectral power distribution in a pre-defined way which has been determined
in previous experimental open-loop neurostimulation studies. Two PhD projects address different
approaches. In one project, observed brain activity is fed into a Kalman observer model (implemented as
an Ensemble Kalman Filter (ETKF) which estimates the optimal stimulation and is applied to the brain
instantaneously. The nonlinear Kalman observer model defines the target spectral power distribution. In
a different project, the closed-loop technique assumes linear brain dynamics and applies conventional
feedback control techniques taking into account feedback delay and model errors. In contrast to the
first project, which is rather heuristic, this second project applies mathematical analysis techniques to
develop the optimal stimulation.

3.2.4 Control in medical robotics

Standard robotic solutions address the deformation problem by extracting a set of features from live
images (visual servoing) and adjusting the pose/motion of the robot locally to compensate for the
deformations. Nevertheless, visual servoing raises several limitations, particularly for needle insertion
applications: 1) intraoperative images usually offer poor visibility of anatomical structures and extracting
meaningful information in real-time is challenging; 2) when large deformations occur, the control
law of the robot can be significantly modified, which is extremely difficult to relate with image-based
displacements; 3) traditional controllers do not exploit biomechanical models capable of predicting the
deformation of organs in real-time, which is essential for some control. To address this problem, we
proposed a numerical solution to solve inverse Finite Elements simulations (iFE) and derive robotic
commands to steer a needle in soft tissues. The method combines a non-rigid registration process to keep
the model aligned with the actual organ. Then, we derive robotic commands by computing, from the FE
simulation, the Jacobian linking the displacement of the base of the needle (controlled by the robot) and
the corresponding displacement of the needle tip inside the tissue. We showed that the Jacobian helps
account for complex nonlinear and discontinuous interactions (such as friction and contacts) and allows
adapting the robot’s behavior faster than waiting for image-based corrections. We recently extended the
method with a shared control strategy to increase the safety, stability, and accuracy and the developed
solution’s acceptance. The primary motivation is to leave potentially dangerous decisions and actions to
the practitioner. Other controls of the needle, e.g. to compensate for breathing motion, are performed
automatically.

4 Application domains

4.1 Patient-specific stent design through shape optimization

Cardiovascular disease is the world’s leading cause of death. Many pathologies can be treated using
image-guided catheter procedures such as balloon angioplasty and stenting. However, the risk of post-
operative complications and follow-up surgery due to pathological tissue reactions, such as restenosis, is
relatively high. This is what motivates computational methods as a tool for improving understanding
of the underlying causes and for the computer-aided design of new endovascular devices to prevent
postoperative complications in the future. Due in part to their complex geometry and the significant
deformations they undergo during insertion, effective simulation of the structure of endoprostheses and
their interaction with their environment is necessary and remains a challenge. In this project, we aim at
optimizing the stent design to better adapt to the geometry and mechanical characteristics of the artery,
and therefore reduce the risk of post-operative complications.

Project MIMESIS

7

4.2 Computer-based surgical training

Virtual training helps medical students to get familiar with surgical procedures before manipulation
of real patients. The development of simulation used for medical training usually requires important
computational power, since realistic behaviors are key to deliver a high-fidelity experience to the trainee.
Further, the quality of interaction with the simulator (usually via visual and haptic rendering) is also of
paramount importance. All these constraints make the development of training systems time-consuming,
thus limiting the deployment of virtual simulators in standard medical curriculum.

Our activity in this area is twofold: we continue the development of fast and accurate numerical
solutions for computing the interactions between a medical device and anatomical structures, and we
investigate means of assessing trainee’s performance and ways of providing valuable feedback in the form
of a virtual coach.

4.3 Pre-operative planning

Beyond training, clinicians ask for innovative tools that can assist them in the pre-operative planning of
an intervention. Using the patient information acquired before the operation, physics-based simulations
allow to simulate the effect of therapy with no risk to the patient. The clinicians can thus virtually assess
different strategies and select the optimal procedure. Compared to a training simulation, a planning
system requires a high accuracy to ensure reliability. Constrained by the time elapsed between the
preoperative acquisition and the intervention, the computation must also be efficient.

4.4 Intra-operative guidance

Besides the surgery training and planning, another major need from clinicians is surgical guidance. While
the clinician is performing the operation, a guidance system provides enriched visual feedback. This is
especially useful with the emergence of minimally invasive surgery (MIS) where the visual information is
often strongly limited. It can be used for example to avoid critical areas such as vessels or to highlight
the position of a tumor during its resection. In the MIS technique, the clinician does not interact with
organs directly as in the open surgery, but manipulates instruments inserted through trocars placed in
small incisions in the wall of the abdominal cavity. The surgeon can observe these instruments on a
display showing a video stream captured by an endoscopic camera inserted through the navel. The main
advantage of the method resides in reducing pain and time recovery, in addition to reducing bleeding
and risks of infection. However, from a surgical standpoint, the procedure is quite complex since the field
of view is considerably reduced and the direct manipulation of organs is not possible.

4.5 Feedback control of neural activity

In the medical treatment of mental disorders, pharmacological medication plays an important role.
However, it is well-known that patients develop resistencies against the medication after some time
and hence alternative treatment is mandatory. Besides psycho-cognitive treatment, neurostimulation
plays a more and more important role. Today typical clinical neurostimulation, such as transcranial
electric or magnetic stimulation, applies stimulation protocols with pre-defined parameters, such as
stimulation intensity, duration and inter-stimulus interval. However, such patient-unspecific open-loop
stimulation protocols may not be effective for all patients. Our research aims to develop a closed-loop
neurostimulation protocol that estimates the optimal stimulation intensity in real-time adaptively based
on observed brain activity. In this context, we work together with the laboratory INSERM 1114 and the
ICube team IMIS.

4.6 Open source software

For many years, members of the team have considered essential to disseminate our research results (and
the algorithms to produce them) in an open manner. The objective was to also develop a framework that
could be used internally as a means to integrate our various contributions and facilitate validation and
technology transfer. Many of our research results have since been released to the community as open
source code, either through improvements of SOFA or as plugins of the framework.

8

Inria Annual Report 2023

SOFA is an efficient and accurate simulation framework written in C++, developed by our team
and researchers from a couple of other Inria teams. It is one of the few open source frameworks for
the simulation of heterogeneous mechanical systems. It is particularly aimed at real-time applications
involving contacts. It is also often aimed at medical applications. Thanks to its high level of modularity,
SOFA allows users to rapidly create complex simulations based on the large set of algorithms ranging
from collision detection methods to volume rendering to finite element methods. It is structured around
an open source core and many plugins, often open source as well. More than 30 researchers, students or
engineers have contributed at various degrees to SOFA, for a total of about 1,500,000 lines of code. Today,
the project has gone far beyond French borders.

5 Highlights of the year

Living systems are part of complex environments, which may perturb the systems functions. Nevertheless,
most living systems, such as humans, exhibit resilience towards such environmental perturbations.
Biophysical diversity enriches neural systems’ dynamical repertoire while it remains challenging to
reconcile with the robustness and persistence of brain function over time (resilience). We analyzed
both analytically and numerically a complex biology-inspired neural network model and show that
homogeneous networks demonstrate enhanced instability in response to a slowly-varying modulatory
fluctuation [14]. Conversely, excitability heterogeneity was found to implement a homeostatic control
mechanism enhancing network resilience. This points out the importance of heterogeneity in neural
systems to retain the system’s resilience.

Two startups have been created this year:

• EVE – EVolving Education: the objective of this startup is to develop a disruptive model of education
and training in surgery, with a low-cost, portable, easy and fun-to-use simulator, mixing physics-
based simulation and gaming. More here.

• Twinical: the goal of this startup is to develop a new paradigm for surgical planning and navigation
in deformable anatomies. Our technology combines artificial intelligence, physics-based simu-
lation, computer vision and augmented reality for digitally-enhanced visual guidance during a
surgery. More here.

6 New software, platforms, open data

6.1 New software

6.1.1 DeepPhysX

Name: DeepPhysX: interfacing AI with multi-physics simulation

Keywords: Numerical simulations, Deep learning, Neural networks, Python

Functional Description: The purpose of DeepPhysX framework is to provide an interface between deep
learning algorithms and numerical simulations. It is a full Python project with two main pipelines,
allowing both to train artificial neural networks with simulated data and to use trained neural
networks as components of numerical simulations. DeepPhysX manages not only the production
of synthetic data with multiple numerical simulations in multiprocessing but also the storage of the
produced dataset. Additional tools are provided to visualize numerical simulations and to follow
the evolution of training sessions.

URL: https://gitlab.inria.fr/mimesis/DeepPhysX

Authors: Robin Enjalbert, Alban Odot, Stephane Cotin

Contact: Robin Enjalbert

Project MIMESIS

9

6.1.2 SSD

Name: SimulationSimpleDatabase

Keywords: Numerical simulations, SQL database, 3D rendering, Python

Scientific Description: The SSD project provides Python3 tools allowing users to easily develop data
storage and visualization strategies for their numerical simulations with a minimal lines of code.

This project has two main objectives: * Easy storage management system for any data from a
numerical simulation, * Easy storage & rendering management systems for visual data from a
numerical simulation.

The SSD project is mainly using the Peewee Python3 library and was mostly designed to fit the
DeepPhysX and SOFA frameworks.

Functional Description: The open source project DeepPhysX, an interface between deep learning and
numerical simulations, has many features that could be useful for the numerical simulation com-
munity that does not use deep learning. Some of these features are implemented for the whole
numerical simulation community in this external project - which is now a dependency of Deep-
PhysX.

A compatibility package is also implemented to take full advantage of this project for the SOFA
framework.

Release Contributions: The last version of the SSD project includes changes made to SOFA’s Python
bindings API. Read and write acces has been optimized on databases with several tables: parallel-
ization has been implemented to speed up interactions with the database, which was previously
sequential. This was particularly useful for managing visual data. Finally, a new rendering library is
available as an option: it is now possible to visualize with either Vedo or Open3D.

URL: https://github.com/RobinEnjalbert/SimulationSimpleDatabase.

Contact: Robin Enjalbert

7 New results

7.1 Digital twins of organs

Participants: Michel Duprez, Vanessa Lleras, Vincent Italiano, Frederique Lecourtier,

Stéphane Cotin.

For several years, we have investigated finite element methods that falls under the class of unfitted
(also known as immersed boundary) methods. Because such methods do not require a discretization
that strictly conforms to the domain boundary, they are particularly suited for the development of digital
twins, as they facilitate the automatic generation of patient-specific simulations on complex geometries.
We use a patient-specific bio-mechanical model to reproduce real-time deformations of an organ
by combining finite-element simulations and deep neural networks in order to satisfy the rapidity and
accuracy requirements of medical applications [7]. We worked in particular on hyperparameter tuning,
implementation improvements, and validation of the method.

Moreover, our current results have focused on the development, the numerical analysis and mathem-
atical foundations of a new method called Φ-FEM. The main advantage of our method is that it uses the
classical Finite Element tools on unfitted meshes. We have also highlighted that the method significantly
improves the convergence when compared to a similar, fitted, discretization of the domain. The benefit
and applicability of the Φ-FEM method have already been demonstrated on different problems: heat
transfer, crack, interface between two materials, and fluid structure interaction [10, 11]. Our current activ-
ity on Φ-FEM consists in an Open Source implementation in a SOFA plugin. This is done in relationship
with another development activity that integrates automatic differentiation tools from the project FEniCS

10

Inria Annual Report 2023

(www.fenicsproject.org) to quickly and efficiently add new constitutive models in our code base, through
simple Python scripting. We are convinced of the impact of this work for both our research activity and
the field in general. It also ties nicely with our work on deep learning for physics, as it offers an elegant
alternative to graph networks to perform convolutions on unstructured meshes.

7.2 Accurate and real-time digital twin of the liver

Participants:

Sidaty El Hadramy, Stéphane Cotin.

Due to the highly deformable nature of the liver, hepatic surgery presents unique challenges in
retrieving the location of relevant landmarks and securing tumor resection while avoiding undesired
damage to vascular structures. Recent developments in real-time registration methods have made it
possible to bring Augmented Reality (AR) into the operating room [6]. In this work, we propose to use
untracked IntraVascular Ultrasound (IVUS) as an intraoperative modality for an accurate and real-time
digital twin of the organ. To this end, we introduce a novel method for trackerless ultrasound volume
reconstruction, particularly in the realm of minimally invasive surgery. The proposed approach employs
a Siamese architecture, featuring a recurrent neural network that utilizes ultrasound image features
and optical flow to estimate the relative position of frames. The method operates without the need for
additional sensors and was evaluated using ex vivo porcine data [20]. The vascular tree is segmented from
the IVUS volume where the centerlines are computed and used to update a biomechanical model [17]
built with the preoperative Computer Tomography (CT)data of the patient. In addition, we have proposed
an alternative approach where we aim to augment intra-operative Non-contrasted CT (NCCT) images as
a means to improve needle CT-guided techniques while reducing the need for contrast agent injection
during tumor ablation procedures. This is done by using a neural network that learns local vessel tree
image features in an NCCT by leveraging the known vessel tree geometry and topology extracted from
a matching Contrasted CT. The augmented CT is generated by fusing the predicted vessel tree with the
NCCT [21].

7.3 Physics-informed neural networks for organ shape prediction from fluoroscopic

images

Participants:

Francois Lecomte, Stéphane Cotin.

In minimally invasive image-guided surgery, several sources of visual information are used. In our
context, pre-operative CT-Scan enables the practitioner to segment anatomical structures, while intra-
operative fluoroscopic imaging is used for intra-operative navigation. During surgery, deformation of
the anatomy leads to a significant mismatch between the fluoroscopic image and the preoperative CT
scan. The aim of this work is to develop a non-rigid registration method to correct the mismatch between
3D preoperative imaging and 2D intra-operative imaging. The 2D nature of intra-operative imaging
implies a loss of information, making it difficult to reconstruct 3D anatomy from a 2D fluoroscopic image.
Moreover, there are no 3D/2D image pairs in sufficient quantities to directly train a neural network to
solve this problem. Consequently, a synthetic data generation method has been developed to generate
training data for learning the 2D/3D registration task. A neural network architecture has also been
introduced to address this ill-posed problem. Our first results [22] were presented during the Hamlyn
Symposium (London, June 2023) and received a lot of attention.

7.4 Physically inspired regularization for deformable image registration

Participants:

Pablo Alvarez, Stéphane Cotin.

Project MIMESIS

11

Numerous regularization methods for deformable image registration aim at enforcing smooth trans-
formations, but are difficult to tune-in a priori and lack a clear physical basis. To cope with this problem,
physically motivated regularization has been introduced. These methods consider the organs in medical
images as hyperelastic bodies, and exploit the theory of continuum mechanics for the design of regu-
larization strategies. In addition to their strong physical basis, they also inherently strengthen desirable
properties such as transformation smoothness and invertibility. However, to date, they still require
complex domain discretization and resolution schemes. In our work, we have extended on these ideas
to propose a physically-derived regularization strategy that can be computed at arbitrary points in the
domain, therefore not requiring any discretization. Consequently, our strategy is compatible with existing
deformable image registration frameworks and optimizers, while maintaining the desirable properties of
physically motivated regularization. We have implemented our regularization scheme using deep neural
networks [25], and have obtained promising results in a 4DCT lung database widely used in the literature
for evaluation of deformable image registration methods.

We plan on using this regularization approach in more challenging deformable registration context
such as 2D-3D registration, as well as in the regularization of deep learning methods for digital twins,
where observations are scarce.

7.5 Patient-specific stent design through shape optimization

Participants: Michel Duprez, Stéphane Cotin.

Cardiovascular disease, which disrupts blood flow to the heart, brain, arms and legs, is the world’s
leading cause of death. This high number of deaths motivates the design of minimally invasive procedures
and explains the success of image-guided catheter treatment procedures such as balloon angioplasty and
stent insertion. However, the risk of post-operative complications and follow-up surgery due to patholo-
gical tissue reactions, such as restenosis, is relatively high. This is what motivates computational methods
as a tool for improving understanding of the underlying causes and for the computer-aided design of new
endovascular devices to prevent postoperative complications in the future. Due in part to their complex
geometry and the significant deformations they undergo during insertion, effective simulation of the
structure of endoprostheses and their interaction with their environment is still necessary and remains a
challenge. Atherosclerosis is characterized by the deposition of an essentially lipid-based plaque (known
as atheroma) on artery walls. This plaque is not deposited uniformly throughout the artery. Stent design
that takes into account this non-uniformity, as well as the geometry of the artery, can help reduce the risk
of complications following stent placement. This is the aim of this project.

7.6 Recovering surgically induced deformations of the liver

Participants:

Francois Lecomte, Juan Verde, Stéphane Cotin.

We have performed an extensive validation on recovering surgically induced deformations of the liver
from synthetic X-Ray images [29, 22] The next steps are, other than methodological improvements, to
validate the accuracy of the method on real X-Ray images implying experiments in collaboration with
IHU Strasbourg.

7.7 Non-smooth mechanics

Participants:

Ziqui Zeng, Hadrien Courtecuisse, Stéphane Cotin.

12

Inria Annual Report 2023

Non-smooth mechanics: many clinical interventions rely on mechanical interactions between a
device and the anatomy. Surgery and interventional radiology are the two main areas where these interac-
tions are essential and generally complex. During this evaluation period we have continued our activity
on constraint modeling and simulation of contacts with friction. The complexity of the problem often
leads to unstable numerical approaches, inaccuracy of the solution and slow computation times. Our
latest work relies on the idea of solving the constrained problem as a Linear Complementarity Problem
(LCP) through an iterative solver (Gauss Seidel in our case) and addresses the issue of computational
performance. We use a preconditioner-based contact method relying on an asynchronous Cholesky
decomposition. By exploiting the sparsity in assembled matrices, we propose a reduced and parallel
computation scheme to address the expensive computation of the Schur complement. An additional
numerical scheme that exploits the time consistency of contacts across multiple time steps makes it
possible to obtain real-time simulations in the presence of hundreds of contacts. Our results have direct
applications in CAI problems such as robotic control of needle insertion or endovascular navigation, but
also in surgical training. They are also very relevant in Computer Graphics.

7.8 Order in random systems

Participants:

Axel Hutt.

The brain is a complex system with several spatial and temporal scales. The microscopic scales
are rather unstructured in space and activity observations show random fluctuations, whereas upper
hierarchical levels at the mesoscopic and macroscopic scales exhibit more regular dynamics. In previous
studies, we found that additive random input on the microscopic scale to random networks tunes the
systems stability and may induce stability changes. Such so-called bifurcations induce ordered structures,
being in space or time or in both. This additive noise-induced system evolution (ANISE) has been shown
to describe successfully synchronization and desynchronization observed in electroencephalographic
data [16].

Moreover, if neural networks exhibit heterogeneity in the nodes properties, then the systems response
to external input is changed characteristically dependent on the degree of diversity. This has been shown
in spiking networks [18] and rate-coding networks [14, 15].

7.9 Dynamics in systems under neurostimulation

Participants:

Thomas Wahl, Axel Hutt.

Brain stimulation is a modern therapy in clinical practice. The various types of stimulation affect
the brain’s internal structure and functioning, which results to learning processes and, in case of mental
disorders, to improvement of the patient’s health situation. To understand better how neurostimulation
modulates the brain dynamics, a data analysis study of previously recorded cortical Local Field Potential
in rats has been performed. It shows that neurostimulation in the brainstem affects strongly the network
synchronisation in the gamma-frequency range (30Hz-50Hz) while retaining connectivity in the beta-
frequency range (0.5Hz-3Hz) [13].
These experimentally induced cortical modulations in certain frequency bands can also be achiveved
by a closed-loop feedback control method [19]. We developed a new method which affects the spectral
power in pre-specified frequency bands in real-time.

7.10 Binaural beats affect the attention in healthy and attention-deficit human

subjects

Project MIMESIS

13

Participants: Gabriel Alves Castro, Axel Hutt.

Brain stimulation is a modern therapy in clinical practice. Non-invasive stimulation has the advantage
against invasive stimulation that it does not rely on surgery and general anesthesia and thus is less costly
for the patient’s health. Most non-invasive neurostimulation techniques apply an electromagnetic
stimulation, e.g. electric currents or magnetic fields applied to the subject’s scalp. This procedure requires
a scientific institution or laboratory to apply the stimulation. Today, electric neurostimulation devices
can be purchased on the free market but are costly for the individual and rarely reimbursed by health
assurances. To provide an alternative to such devices, non-electromagnetic stimulation techniques, such
as auditory stimulation, may be preferable. We have developed a psycho-physical experimental setup to
test subject’s visual attention subjected to auditory tones (noise, monaural beats and binaural beats) [26,
28]. Synchronous recording of EEG in healthy subjects and subjects showing attention deficits have
revealed that monaural beats may improve visual attention compared to the impact of noise stimulation,
while binaural beats do not have a beneficial effect.

7.11 Optimal control to limit epidemia

Participants: Michel Duprez.

There exists no efficient vaccin against arboviruses like Malaria, Dengue, Lymphatic Filariasis, Zika,
Chikungunya, Yellow fever, and Japanese encephalitis. One way to limit the propagation of such diseases
is to fight their vector which is the mosquito. The sterile insect technique (SIT) is a biological control tech-
nique that can be used either to eliminate or decay a wild mosquito population under a given threshold
to reduce the nuisance or the epidemiological risk. We have studied the mathematical properties of the
optimal release strategies in order to maximize the efficiency of this technique. In [24, 8], we have studied
the impact of the migration on the control strategy. Some theoretical results of controllability have been
obtained in [9].

7.12 Error control in biomechanics

Participants: Michel Duprez.

The Finite Element Method (FEM) is a well-established procedure for computing approximate solu-
tions to deterministic engineering problems described by partial differential equations. FEM produces
discrete approximations of the solution with a discretisation error that can be an be quantified with a pos-
teriori error estimates. The practical relevance of error estimates for biomechanics problems, especially
for soft tissue where the response is governed by large strains, is rarely addressed. In [27], we propose an
implementation of a posteriori error estimates targeting a user-defined quantity of interest, using the
Dual Weighted Residual (DWR) technique tailored to biomechanics. The proposed method considers a
general setting that encompasses three-dimensional geometries and model non-linearities, which appear
in hyperelastic soft tissues. We take advantage of the automatic differentiation capabilities embedded in
modern finite element software, which allows the error estimates to be computed generically for a large
class of models and constitutive laws. First we validate our methodology using experimental measure-
ments from silicone samples, and then we illustrate its applicability for patient-specific computations of
pressure ulcers on a human heel.

7.13 Optimal control of computer-aided interventions

14

Inria Annual Report 2023

Participants:

Stéphane Cotin, Alban Odot.

We recently started to investigate optimal control as a solution to several challenges in the field of
computer-aided intervention. The first problem we addressed was the elastic intraoperative registration
of a preoperative liver model and, as an extension, force estimation during robotic surgery. Both are
formulated as optimal control problems where the unknown is the surface force distribution applied onto
the organ. The deformation is computed using an hyperelastic model, and the optimization problem
is solved using an adjoint method. This approach provides greater control over the set of admissible
forces and leads to physically-consistent displacement fields. As a byproduct, it permits the recovery of
the forces that led to the deformation, and can provide visual haptic feedback in medical robotics. We
also rely on optimal control in other problems addressed in the team, such as autonomous endovascular
navigation, where we have a similar control challenge. In this case the deformable structure is the catheter
or guidewire rather than an organ. Finally, to address the relatively slow computation times that are
typical of such optimization methods, we have started the development of differentiable algorithms. They
exploit both GPU acceleration and the automatic differentiation tools from frameworks such as PyTorch.
This significantly speeds up the computation of the adjoint. This approach is combined with the use of
deep neural networks to predict the solution of the forward elastic problem in just a few milliseconds.

8 Bilateral contracts and grants with industry

Participants:

Stephane Cotin, Vincent Italiano, Sidaty El Hadramy, Michel Duprez,
Robin Enjalbert.

8.1 Bilateral contracts with industry

In July 2023, team members have been involved in the foundation of the start-up EVE: EVolving Education.
Moreover, team members are involved in the new start-up Twinical founded in November. In the context
of their foundation, research code has been transferred to both start-ups.

9 Partnerships and cooperations

Participants: Michel Duprez, Axel Hutt, Stephane Cotin.

9.1 International research visitors

9.1.1 Visits of international scientists

Other international visits to the team

Christian Uhl

Status Prof. Dr. rer. nat.

Institution of origin: University of Applied Sciences Ansbach

Country: Germany

Dates: 04/10/2023 - 06/10/2023

Context of the visit: Kick-off meeting of a future collaboration

Project MIMESIS

15

Mobility program/type of mobility: Visiting Grant from BayFrance / Germany

9.2 National initiatives

ADT - AI for Surgical Vision

Title: AI for Surgical Vision

Duration: 2021 - 2024

Coordinator: Stéphane Cotin

Partners:

• BOPA innovation chair

• Paul Brousse Hospital

Inria contact: stephane.cotin@inria.fr

Summary: The objective of the project is to develop and integrate computer vision algorithms capable
of processing images from the operating room in real time into a clinically usable augmented
reality prototype. This project reinforces the work carried out in partnership with the "Augmented
Operating Room" (BOPA) innovation chair at Hospital Paul Brousse in Paris. The project is part
of the Blok-Viz axis whose goal is to process video feeds of the operating field to extract relevant
information on which AI algorithms dedicated to attention analysis, registration and augmented
reality can be based.

ANR - SPECULAR

Title: Simulation of needle insertion with virtual reality and haptics.

Duration: 2021 - 2025

Coordinator: Stephane Cotin (MIMESIS)

Partners:

• Inria Antenna in Strasbourg, MIMESIS team (France)

• Inria Research Center at Université de Lille, DEFROST team (France)

• InfinityTech 3D (France)

Inria contact: stephane.cotin@inria.fr

Summary: The objective of this project is to develop a complete virtual reality training system for radi-
ofrequency ablation. The research program includes the real-time simulation of the needle-organ
interactions, realistic and immersive rendering of the operating room, medical image generation
and haptic feedback [23]. The results of this project will accelerate the training of these procedures
and could change the standard of care which remains a surgery in many cases.

ANR - VATSOP

Title: Images and models for computer guidance during Video Assisted Thoracic Surgery (VATS).

Duration: 2021 - 2025

Coordinator: Jean-Louis Dillenseger (LTIS Rennes)

Partners:

• TIMC-IMAG Laboratory, Grenoble (France)

16

Inria Annual Report 2023

• Inria Antenna in Strasbourg, MIMESIS team (France)

• LTSI Laboratoire Traitement du Signal et de l’Image (Rennes, France)

Inria contact: stephane.cotin@inria.fr

Summary: In minimally invasive image-guided surgery, several sources of visual information are used.
In our context, pre-operative CT scans enable the practitioner to segment anatomical structures,
while intra-operative fluoroscopic imaging is used for intra-operative navigation. The 2D nature
of intra-operative imaging implies a loss of information, making it difficult to reconstruct 3D
anatomy from a single 2D fluoroscopic image. However, the pre-operative 3D anatomy is known
from the pre-operative CT scan. To augment intra-operative fluoroscopic imaging with accurate
3D anatomical information, it is necessary to recover surgery-related deformations, an operation
known as non-rigid registration. Our work aims to develop a non-rigid registration method to
correct the mismatch between 3D pre-operative imaging and 2D intra-operative imaging. Our
approach uses a fully convolutional network architecture to solve the associated inverse problem.
Because there exists no dataset of 3D/2D image pairs suited to train a neural network to solve
this problem, a synthetic data generation method is developed to generate a training dataset.
Supervised learning is performed on this synthetic training data, with Digitally Reconstructed
Radiographs as input and displacement fields as output. Contrary to other 2D-3D registration
methods, this novel data generation approach does not rely on a statistical motion model. This
enables the network to accurately predict deformations beyond predetermined motion patterns. As
an example of clinical application, we show that our model can accurately handle surgery-induced
deformations in the liver.

ANR - TRECOS

Title: New Trends in Control and Stabilization: Constraints and a Non-local Terms.

Duration: 2020 - 2024

Coordinator: Sylvain Ervedoza (Institut de Mathématiques de Bordeaux, Laboratoire Traitement du

Signal et de l’Image)

Partners:

• IMT Institut de Mathématiques de Toulouse (France)

• Inria Antenna in Strasbourg, MIMESIS team (France)

• IMB Institut de mathématiques de Bordeaux (France)

• LJLL Laboratoire Jacques-Louis Lions, Paris (France)

Inria contact: michel.duprez@inria.fr

Summary: The goal of this project is to develop new solutions in control theory for partial differential
equations, motivated by models arising in ecology and biology. The project focuses on two aspects.
The first one is related to the constraints required on the controls or on the controlled trajectories,
for instance positivity constraints, which appears naturally when the state models a temperat-
ure. The second aspect concerns the questions of controllability and stabilization d of problems
involving non-local operators, such as integral operators in space, in order to take into account
phenomena depending on the total mass of the population for instance, or delay and memory
terms, such as in visco-elastic fluids, often used to a model blood flows, or more generally models
described by systems coupling hyperbolic and parabolic effects.

ANR - S-KELOID

Title: Understanding Keloid Disorders: A multi-scale in vitro/in vivo/in silico approach towards digital

twins of skin organoids on the chip.

17

Project MIMESIS

Duration: 2021 - 2025

Coordinator: Raluca EFTIMIE and Stéphane BORDAS (Univ. Luxembourg)

Partners:

• Laboratoire de Mathématiques de Besançon (France)

• CHU de Besançon (France)

• FEMTO-ST, Besançon (France)

• Institut Mathématiques de Bourgogne, Dijon (France)

• Université du Luxembourg (Luxembourg)

• Inria Antenna in Strasbourg, MIMESIS team (France)

Inria contact: michel.duprez@inria.fr

Summary: Mathematical and numerical modeling approaches allow us to integrate pathological pro-
cesses that occur across different scales: single cell, cell assembly and tissue. The S-Keloid project
aims to investigate the role of mechanical and inflammatory environmental factors on cells associ-
ated with keloid disorders, which is formation of a type of scar. From applying experimental tests
at tissue-scale and using a multiscale approach, the mechanical stress fields will be integrated into
the 3D mathematical model. Parameter identification, optimization and their use across multiple
scales will ensure the realism of the models and the quantitative and qualitative predictions of the
keloid disorder.

ANR - PhiFEM

Title: φ-FEM : development of a Finite Element Method for the design of real-time digital twins in surgery

Duration: 2022 - 2026

Coordinator: Michel Duprez (Inria)

Partners:

• MIMESIS team

• Laboratoire de Mathématiques de Besançon (France)

• Institut de Mathématiques Alexander Grothendieck, Montpellier (France)

• Institut de Recherche en Mathématiques Appliquées, Strasbourg (France)

• Univertsité du Luxembourg (Luxembourg)

Inria contact: michel.duprez@inria.fr

Summary: φ-FEM is a recently proposed finite element method for the efficient numerical solution of
partial differential equations posed in domains of complex shapes, using simple regular meshes.
The main goal of this project is to further develop φ-FEM turning it into a tool for efficient, patient-
specific and real-time simulations of human organs. To reach this objective, we shall adapt φ-FEM
to the equations appropriate to biomechanics, provide an efficient implementation for it allowing
for the use of actual organ geometries, and finally combine it with convolution neural networks
to make it real time after training. The ultimate, long-term, goal is thus to contribute to the con-
struction of digital twins of organs able to guide the surgical act in real time using information
acquired before the operation and to reduce the costs of a medical doctors’ training by working on
visual organs. The innovation of φ-FEM lies in its ability to combine the ease of implementation
of classical immersed boundary methods with the accuracy of more recent CutFEM/XFEM ap-
proaches. It incorporates, by its very construction, the popular description of geometry by Level Set
functions, which can represent the real geometry with whatever accuracy desired which makes this
approach numerically less expensive than classical finite element methods. The φ-FEM paradigm
will also be used to develop efficient registration algorithms. Our results will be integrated into the
open-source SOFA platform developed in the MIMESIS team to facilitate dissemination.

18

Inria Annual Report 2023

National Exploratory Action - A/D Drugs

Title: A/D Drugs

Duration: 2020 - 2023

Coordinator: Axel Hutt

Partners: Didier Pinault (INSERM 1114)

Participants: Axel Hutt, Joséphine Riedinger, Didier Pinault

Summary: When it comes to treating mental disorders, the emergence of resistance to medication is a
major problem. Replacing chemical medicine with digital medicine (neural stimulation) could be
one way of getting around the problem. This is the concept behind "A/D Drugs" (PI : Axel Hutt)
which will deploy a process of data assimilation and control in order to adapt stimulation to each
patient. But this research is not without its risks – very little is known about the links between the
effect of the chemical molecules and neurostimulation (www.inria.fr/en/ad-drugs-exploratory-
action-aimed-optimising-neurostimulation).

9.3 Regional initiatives

ITI HEALTHTECH Strasbourg - Master

Title: Optimal auditory neurostimulation to alleviate visual attention deficits in Attention-Deficit/Hyperactivity

Disorder (ADHD)

Duration: 2022 - 2023

Coordinator: Axel Hutt

Partners: Anne Bonnefond (INSERM 1114)

Participants: Gabriel ALVES CASTRO

Summary: Attention-deficit/hyperactivity disorder (ADHD) is a neuropsychiatric disorder and patients
suffer from inattention, hyperactivity and impulsivity. Since pharmacological treatment may yield
adverse cognitive side effects, non-pharmacological neurostimulation is suggested as an alternative.
The project investigates the impact of auditory beat stimulation on visual attention in healthy
subjects showing ADHD symptoms. The student will perform psychophysical experiments in
the presence of binaural beats or isochronic tones and will identify stimulation parameters that
improve best visual attention in subjects.

10 Dissemination

Participants: Michel Duprez, Stephane Cotin, Axel Hutt, Pablo Arturo Alvarez Cor-

rales, Josephine Riedinger.

10.1 Promoting scientific activities

10.1.1 Scientific events: organisation

General chair, scientific chair

• Axel Hutt has organized the mini-symposium Theoretical and computational approaches in neuros-

cience: Non-invasive neurostimulation at the conference NeuroFrance23, Lyon, May 25, 2023

Project MIMESIS

19

Member of the organizing committees

• Axel Hutt has been member of the organization committee of the Summer School Advanced tools

for data analysis in Neuroscience on Aug 31 - Sept 9, 2023.

10.1.2 Scientific events: selection

Chair of conference program committees Axel Hutt has been Co-Chair of the Programme Commit-
tee of the Computational Neuroscience Organization (CNS), 2023-2024. The CNS organizes the yearly
Computational Neuroscience Conference.

Reviewer Pablo Alvarez has been reviewer for the 26th International Conference on Medical Image
Computing and Computer Assisted Intervention (MICCAI 2023), Vancouver, October 8-12, 2023

10.1.3 Journal

Member of the editorial boards

• Axel Hutt is Chief Section Editor of the section Dynamical Systems in Frontiers in Applied Mathem-

atics and Statistics

• Axel Hutt is Associate Editor for the Journal of Clinical Medicine

Reviewer - reviewing activities

• Axel Hutt has been reviewer for journals European Physical Journal, Qeios, Sensors, Behavioral

Sciences, Cells, PLoS One

• Michel Duprez has been reviewer for Optimal Control, Applications and Methods, Journal de Math-
ématiques Pures et Appliquées, Computational and Applied Mathematics and Applied Mathematics
and Computation.

• Pablo Alvarez has been reviewer for IEEE Transactions of Biomedical Engineering.

10.1.4 Invited talks

• Axel Hutt has been invited to

– Nonlinear Data Analysis and Modeling: Advances, Applications, Perspectives, Potsdam, March

17

– Neural fields equations, from Wilson-Cowan to neural engineering, Paris, June 20

– Canadian Computational Neuroscience Spotlight, online, October 5

• Joséphine Riedinger has been invited to the mini-symposium Theoretical and computational
approaches in neuroscience: Non-invasive neurostimulation at the conference NeuroFrance23,
Lyon, May 25.

• Michel Duprez has been invited to

– SMAI 2023, May

– CFC 2023, May

– the LJK laboratory (Grenoble), June 2023

– the conference ENUMATH 2023, September

• Stéphane Cotin has been invited to

– Essilor-Luxottica as part of their seminar series (Charenton-Le-Pont), April

– the 17th U. S. National Congress of Computational Mechanics (Albuquerque, USA), July

– the J.FIG conference 2023 (Montpellier), November

20

Inria Annual Report 2023

10.1.5 Scientific expertise

• Axel Hutt has been reviewer for the Research Agency National Research and Development Agency

(ANID) of the Ministry of Science, Technology, Knowledge and Innovation of Chile

• Axel Hutt has been a grant reviewer for The Fields Institute for Research in Mathematical Sciences

Toronto / Canada.

10.1.6 Research administration

• Axel Hutt has been member of the jury for the Chaire Professeur Junior on Computational Neuros-

cience in Strasbourg 2023

• Stéphane Cotin has been member of the committee in charge of selecting the head of the new AI

department at IMT Atlantique (October 2023)

10.2 Teaching - Supervision - Juries

10.2.1 Teaching

• Lecture: Valentina Scarponi, Mecanobiologie du vivant, 4h, University of Strasbourg

• License:

– Michel Duprez, Numerical analysis techniques, 15h, L3, University of Strasbourg.

– Sidathy El Hadramy, MPA Algebra and Analysis, 20h, L1 and L2, University of Strasbourg

• Master:

– Valentina Scarponi, Introduction To Scientific Reporting, M2, ITI HealthTech Master, University

of Strasbourg

– Valentina Scarponi, Continuum Mechanics, 8h, ITI HealthTech Master, University of Stras-

bourg

– Michel Duprez, Optimal control, 28h, M2, University of Strasbourg.

– Michel Duprez, Incertitude quantification, 14h, M2, University of Strasbourg.

– Michel Duprez, Calcul Scientifique, 10h, M2, University of Strasbourg

– Hadrien Courtecuisse, Real-time simulation, 80h, M2, University of Strasbourg

– Axel Hutt, Spectral Analysis, 6h, M2, University of Strasbourg, February

• Stéphane Cotin, Numerical simulation and machine learning, ITI HealthTech Strasbourg, May

• Stéphane Cotin, Medical imaging, simulation, and AI, ITI IRMIA Strasbourg, November

• Axel Hutt, Tutorial on Spectral Analysis for participants of the Computational Neuroscience Confer-

ence 2023, Leipzig, July 15

10.2.2 Supervision

• Axel Hutt supervises Joséphine Riedinger (2020-2023). Title: Closed-loop transcranial electric

stimulation of neural networks in a rat psychosis model.

• Axel Hutt supervises Thomas Wahl (2021-2024). Title: Model-based closed-loop neurostimulation

with application to schizophrenia.

• Hadrien Courtecuisse has supervised Ziqiu Zeng (2019-2023). Title: Real-Time FE Simulation for
Large-Scale Problems Using Precondition-Based Contact Resolution and Isolated DOFs Constraints.

Project MIMESIS

21

• Michel Duprez supervises Killian Vuillemot (October 2022-September 2025) together with Vanessa
Lleras and Mohammadi Bijan. Title: Unfitted finite element method for the development of organ
digital twins.

• Michel Duprez supervises Valentina Scarponi (2021-2024) together with Florent Nageotte and

Stéphane Cotin. Title: Autonomous Catheter Navigation.

• Michel Duprez supervises Frédérique Lecourtier (2023-2026) together with Emmanuel Franck and

Vanessa Lleras. Title: Finite element methods and neural network for the augmented surgery.

• Stephane Cotin supervises Sidaty El Hadramy (2022-2025) together with Benoit Gallix (ICube). Title:

AI-enabled Intraoperative planning for liver surgery using ultrasound imaging.

• Stephane Cotin has supervised Alban Odot (2020-2023). Title: Data-driven computational biomech-

anics using deep neural networks. The thesis was defended in November.

• Stephane Cotin supervises Nicola Zotto (2022-2025) together with Benoit Gallix (ICube). Title:

Combining AI and biomechanics for computer-assisted interventions.

• Stephane Cotin supervises Francois Lecomte (2021-2024) together with Jean-Louis Dillenseger
(Univ Rennes). Title: Physics informed neural networks for organ shape prediction from interven-
tional images. Application to Augmented Lung Surgery.

10.2.3 Juries

• Stéphane Cotin has been member of the PhD committee (and reviewer) of PhD-thesis of Nora

Hagmeyer, Universität der Bundeswehr München, June 2023

• Stéphane Cotin has been member of the PhD committee (and reviewer) of the thesis of Saurabh

Deshpande, Luxembourg University, October

• Stéphane Cotin has been member of thePhD committee (and supervisor) of the thesis of Alban

Odot, Strasbourg University, November

10.3 Popularization

10.3.1 Interventions

Joséphine Riedinger has made a two-hour presentation in front of terminal high school students at the
high school Le Gymnase in Strasbourg.

11 Scientific production

11.1 Major publications

[1] N. Golse. ‘Anatomical, Hemodynamic and Physiological Modeling in Liver Surgery’. Université
Paris-Saclay, 20th Sept. 2021. URL: https://tel.archives-ouvertes.fr/tel-03355990.

[2] A. Hutt, S. Rich, T. Valiante and J. Lefebvre. ‘Intrinsic neural diversity quenches the dynamic
volatility of neural networks’. In: Proceedings of the National Academy of Sciences of the United
States of America 120.28 (2023), e2218841120. DOI: 10.1073/pnas.2218841120. URL: https://i
nria.hal.science/hal-04177196.

[3]

J. Lefebvre and A. Hutt. ‘Induced synchronisation by endogenous noise modulation in finite-size
random neural networks: a stochastic mean-field study’. In: Chaos: An Interdisciplinary Journal of
Nonlinear Science (14th July 2023). DOI: 10.1063/5.0167771. URL: https://inria.hal.scien
ce/hal-04162581.

[4] A. Mendizabal, P. Márquez-Neila and S. Cotin. ‘Simulation of hyperelastic materials in real-time
using deep learning’. In: Medical Image Analysis 59 (2019), p. 101569. DOI: 10.1016/j.media.201
9.101569. URL: https://hal.inria.fr/hal-02097119.

22

Inria Annual Report 2023

[5] Z. Zeng, S. Cotin and H. Courtecuisse. ‘Real-Time FE Simulation for Large-Scale Problems Using
Precondition-Based Contact Resolution and Isolated DOFs Constraints’. In: Computer Graphics
Forum 41.6 (6th June 2022), pp. 418–434. DOI: 10.1111/cgf.14563. URL: https://hal.science
/hal-03694167.

11.2 Publications of the year

International journals

[6] B. Acidi, M. Ghallab, S. Cotin, E. Vibert and N. Golse. ‘Augmented reality in liver surgery’. In: Journal
of Visceral Surgery 160.2 (Apr. 2023), pp. 118–126. DOI: 10.1016/j.jviscsurg.2023.01.008.
URL: https://inria.hal.science/hal-04414274.

[7] S. Cotin, G. Mestdagh and Y. Privat. ‘Organ registration from partial surface data in augmented
surgery from an optimal control perspective’. In: Proceedings of the Royal Society A: Mathematical,
Physical and Engineering Sciences 480 (22nd Mar. 2023), p. 20230197. DOI: 10.1098/rspa.2023.0
197. URL: https://hal.science/hal-04043695.

[8] Y. Dumont and M. Duprez. ‘Modeling the impact of rainfall and temperature on sterile insect
control strategies in a Tropical environment’. In: Journal of Biological Systems (5th Jan. 2024). DOI:
10.1142/S0218339024500128. URL: https://hal.science/hal-03811327.

[9] M. Duprez, M. González-Burgos and D. Souza. ‘Remarks on the controllability of parabolic systems
with non-diagonalizable diffusion matrix’. In: Discrete and Continuous Dynamical Systems - Series
S 16.6 (2023), pp. 1346–1382. DOI: 10.3934/dcdss.2023086. URL: https://hal.science/hal-
03772372.

[10] M. Duprez, V. Lleras and A. Lozinski. ‘φ-FEM: an optimally convergent and easily implementable
immersed boundary method for particulate flows and Stokes equations’. In: ESAIM: Mathematical
Modelling and Numerical Analysis 57 (May 2023), pp. 1111–1142. DOI: 10.1051/m2an/2023010.
URL: https://hal.science/hal-03588715.

[11] M. Duprez, V. Lleras, A. Lozinski and K. Vuillemot. ‘phi-FEM for the heat equation: optimal conver-
gence on unfitted meshes in space’. In: Comptes Rendus. Mathématique 361.G11 (21st Dec. 2023),
pp. 1699–1710. DOI: 10.5802/crmath.497. URL: https://hal.science/hal-03685445.

[12] A. Hutt. ‘Editorial on Insights in Dynamical Systems 2022’. In: Frontiers in Applied Mathematics
and Statistics 9 (2023). DOI: 10.3389/fams.2023.1254646. URL: https://inria.hal.science
/hal-04387591.

[13] A. Hutt and A. G. Hudetz. ‘Arousal system stimulation and anesthetic state alter visuoparietal
connectivity’. In: Frontiers in Systems Neuroscience 17 (17th Apr. 2023). DOI: 10.3389/fnsys.2023
.1157488. URL: https://inria.hal.science/hal-04071044.

[14] A. Hutt, S. Rich, T. Valiante and J. Lefebvre. ‘Intrinsic neural diversity quenches the dynamic
volatility of neural networks’. In: Proceedings of the National Academy of Sciences of the United
States of America 120.28 (2023), e2218841120. DOI: 10.1073/pnas.2218841120. URL: https://i
nria.hal.science/hal-04177196.

[15] A. Hutt, D. Trotter, A. Pariz, T. A. Valiante and J. Lefebvre. ‘Diversity-induced trivialization and
resilience of neural dynamics’. In: Chaos: An Interdisciplinary Journal of Nonlinear Science (2024).
URL: https://inria.hal.science/hal-04162586.

[16]

J. Lefebvre and A. Hutt. ‘Induced synchronisation by endogenous noise modulation in finite-size
random neural networks: a stochastic mean-field study’. In: Chaos: An Interdisciplinary Journal of
Nonlinear Science (14th July 2023). DOI: 10.1063/5.0167771. URL: https://inria.hal.scien
ce/hal-04162581.

[17] A. Mazier, S. El Hadramy, J.-N. Brunet, J. S. Hale, S. Cotin and S. P. A. Bordas. ‘SOniCS: Develop intu-
ition on biomechanical systems through interactive error controlled simulations’. In: Engineering
with Computers (12th July 2023). URL: https://inria.hal.science/hal-04167398.

Project MIMESIS

23

[18] A. Pariz, D. Trotter, A. Hutt and J. Lefebvre. ‘Selective control of synaptic plasticity in heterogeneous
networks through transcranial alternating current stimulation (tACS)’. In: PLoS Computational
Biology (2023). DOI: 10.1371/journal.pcbi.1010736. URL: https://inria.hal.science/ha
l-04071053.

[19] T. Wahl, J. Riedinger, M. Duprez and A. Hutt. ‘Delayed closed-loop neurostimulation for the treat-
ment of pathological brain rhythms in mental disorders: a computational study’. In: Frontiers in
Neuroscience (23rd Mar. 2023). DOI: 10.3389/fnins.2023.1183670. URL: https://hal.scienc
e/hal-04037276.

International peer-reviewed conferences

[20]

[21]

S. El Hadramy, J. Verde, K.-P. Beaudet, N. Padoy and S. Cotin. ‘Trackerless Volume Reconstruction
from Intraoperative Ultrasound Images’. In: MICCAI 2023. Vancouver, Canada, 8th Oct. 2023. URL:
https://inria.hal.science/hal-04108248.

S. El Hadramy, J. Verde, N. Padoy and S. Cotin. ‘Intraoperative CT augmentation for needle-based
liver interventions’. In: MICCAI 2023. Vancouver, Canada, 8th Oct. 2023. URL: https://inria.ha
l.science/hal-04167447.

[22] F. Lecomte, V. Scarponi, P. A. Alvarez, J. M. Verde, J.-L. Dillenseger, E. Vibert and S. Cotin. ‘Enhan-
cing Fluoroscopy-Guided Interventions: a Neural Network to Predict Vessel Deformation without
Contrast Agents’. In: Hamlyn Symposium on Medical Robotics. The Hamlyn Symposium on Medical
Robotics. London, United Kingdom: The Hamlyn Centre, Imperial College London London, UK,
26th June 2023, pp. 75–76. DOI: 10.31256/HSMR2023.38. URL: https://hal.science/hal-043
56336.

[23] C. Martin, Z. Zeng and H. Courtecuisse. ‘Efficient Needle Insertion Simulation using Hybrid Con-
straint Solver and Isolated DOFs’. In: Eurographics 2023 - Short Papers. Saarbrücken, Germany:
The Eurographics Association, 2023. DOI: 10.2312/egs.20231003. URL: https://hal.science
/hal-04181906.

Conferences without proceedings

[24] Y. Dumont, M. Duprez and Y. Privat. ‘About sterile insect control strategies in a two patches
system’. In: BIOMATH 2023 - International Conference on Mathematical Methods and Models in
Biosciences. Pomorie, Bulgaria, 18th June 2023. URL: https://hal.science/hal-04162767.

Reports & preprints

[25] P. Alvarez and S. Cotin. Deformable Image Registration with Stochastically Regularized Biomechan-

ical Equilibrium. 21st Dec. 2023. URL: https://hal.science/hal-04356505.

[26] G. Alves Castro, A. Hutt and A. Bonnefond. Optimal auditory neurostimulation to alleviate visual
attention deficits in Attention-Deficit/Hyperactivity Disorder (ADHD). 2023. URL: https://inria
.hal.science/hal-04225013.

[27] M. Duprez, F. Chouly, P.-Y. Rohan, S. P. A. Bordas, M. Bucki, H. P. Bui and A. Lejeune. Automatic

mesh refinement for soft tissue. 15th Sept. 2023. URL: https://hal.science/hal-04208610.

[28] A. Hutt. Conversion of power estimation in Audacity to real Sound Pressure Level in ear phones.
INRIA Nancy Grand Est, 4th Aug. 2023. URL: https://inria.hal.science/hal-04177151.

[29] F. Lecomte, J.-L. Dillenseger and S. Cotin. CNN-based real-time 2D-3D deformable registration from
a single X-ray projection. 27th Mar. 2023. DOI: 10.48550/arXiv.2212.07692. URL: https://inr
ia.hal.science/hal-04387845.

[30] A. Odot, R. Haferssas and S. Cotin. DeepPhysics: a physics aware deep learning framework for real-
time simulation. 14th Mar. 2023. DOI: 10.1002/nme.6943. URL: https://hal.science/hal-03
327818.

[31] Z. Zeng and H. Courtecuisse. An efficient implicit constraint resolution scheme for interactive FE

simulations. 11th June 2023. URL: https://hal.science/hal-04125042.

24

Inria Annual Report 2023

[32] Z. Zeng and H. Courtecuisse. Efficient parallelization strategy for real-time FE simulations. 8th June

2023. URL: https://hal.science/hal-04122582.
<scratchpad>
- Main topic: Research Centre at Inria, focusing on computational anatomy and simulation for medicine
- Key points:
  - Development of numerical methods for computer-based training, surgical planning, and computer-assisted interventions
  - Investigation of immersed boundary methods, scientific machine learning, and non-smooth mechanics
  - Application domains include patient-specific stent design, computer-based surgical training, and feedback control of neural activity
  - Partnerships with industry, academia, and clinical partners
- Structure: The document is divided into sections on Project-Team, Activity Report, Contents, Highlights of the Year, New Software, New Results, Bilateral Contracts and Grants with Industry, Partnerships and Cooperations, Dissemination, Scientific Production, and Annual Report.
- Potential artifacts to ignore: Links, HTML tags, and web-related elements.
</scratchpad>

<final_summary>
The Inria Research Centre at Université de Lorraine focuses on computational anatomy and simulation for medicine. The team develops numerical methods for computer-based training, surgical planning, and computer-assisted interventions, with a strong emphasis on immersed boundary methods, scientific machine learning, and non-smooth mechanics. The research has various application domains, including patient-specific stent design, computer-based surgical training, and feedback control of neural activity. The team collaborates with industry, academia, and clinical partners to advance the field of computational anatomy and simulation for medicine.
The Inria Research Centre at Université de Lorraine focuses on computational anatomy and simulation for medicine. The team develops numerical methods for computer-based training, surgical planning, and computer-assisted interventions, with a strong emphasis on immersed boundary methods, scientific machine learning, and non-smooth mechanics. The research has various application domains, including patient-specific stent design, computer-based surgical training, and feedback control of neural activity. The team collaborates with industry, academia, and clinical partners to advance the field of computational anatomy and simulation for medicine.