RESEARCH CENTRE

Inria Centre
at Université de Lorraine

IN PARTNERSHIP WITH:

Université de Lorraine, CNRS

2023
ACTIVITY REPORT

Project-Team

TANGRAM

Visual Registration with Physically
Coherent Models

IN COLLABORATION WITH: Laboratoire lorrain de recherche en
informatique et ses applications (LORIA)

DOMAIN

Perception, Cognition and Interaction

THEME

Vision, perception and multimedia
interpretation

1

2

3

3
3
3
4
4
4

4

5
5

5
5
5
6
6

7
7
7
8
8

Contents

Project-Team TANGRAM

1 Team members, visitors, external collaborators

2 Overall objectives

3 Research program

. . . . . . . . . . . . . . . . .
3.1 Localization and geometric reasoning with high level features
3.2 Building dedicated models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.3 Estimation and inverse problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.3.1 Optimization, variational calculus and numerical schemes . . . . . . . . . . . . . . .
3.3.2 Machine learning for physical problems . . . . . . . . . . . . . . . . . . . . . . . . . .

4 Application domains

5 Highlights of the year

5.1 Awards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

6 New software, platforms, open data

6.1 New software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.1.1 OA-SLAM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.1.2 P1E . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.1.3 DeepAnePose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

7 New results

7.1 Visual localization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7.2 Vanishing point computation and applications . . . . . . . . . . . . . . . . . . . . . . . . . .
7.3 Handling non rigid deformation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7.3.1 Individual mitral valve modeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7.3.2 Image-based biomechanical simulation of the diaphragm during mechanical ventil-
8
ation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
8
7.4 Interventional radiology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
8
7.4.1 Detection of brain aneurysms using deep learning . . . . . . . . . . . . . . . . . . . .
9
7.4.2 Predictive simulation of catheter navigation . . . . . . . . . . . . . . . . . . . . . . . .
9
7.5 Image and signal processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
9
7.5.1 Computational photomechanics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
9
7.5.2 Variational methods for image processing . . . . . . . . . . . . . . . . . . . . . . . . .
7.6 Application of deep learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
7.6.1 Extensive searches for complex intermetallic catalysts . . . . . . . . . . . . . . . . . . 10
7.6.2 Multivariate analysis of geochemical, physical and mineralogical signatures of uranium

deposits in the Athabasca Basin (Saskatchewan, Canada)

. . . . . . . . . . . . . . . . 10

8 Partnerships and cooperations

10
8.1 International initiatives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
8.1.1 Associate Teams in the framework of an Inria International Lab or in the framework

8.2 International research visitors

of an Inria International Program . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
8.2.1 Visits of international scientists . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
8.2.2 Visits to international teams
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
8.3.1 Other european programs/initiatives . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
8.4 National initiatives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

8.3 European initiatives

9 Dissemination

13
9.1 Promoting scientific activities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
9.1.1 Scientific events: organisation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
9.1.2 Scientific events: selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
9.1.3 Journal
9.1.4 Invited talks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
9.1.5 Scientific expertise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
9.1.6 Research administration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
9.2 Teaching - Supervision - Juries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
9.2.1 Teaching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
9.2.2 Supervision . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
9.2.3 Juries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
9.3 Popularization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
9.3.1 Internal or external Inria responsibilities . . . . . . . . . . . . . . . . . . . . . . . . . . 17
9.3.2 Articles and contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
9.3.3 Education . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
9.3.4 Interventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

10 Scientific production

18
10.1 Major publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
10.2 Publications of the year
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
10.3 Cited publications

Project TANGRAM

1

Project-Team TANGRAM

Creation of the Project-Team: 2020 December 01

Keywords

Computer sciences and digital sciences

A5.3. – Image processing and analysis

A5.4. – Computer vision

A5.4.1. – Object recognition

A5.4.5. – Object tracking and motion analysis

A5.4.6. – Object localization

A5.6. – Virtual reality, augmented reality

A5.10.2. – Perception

Other research topics and application domains

B2.6. – Biological and medical imaging

B5.9. – Industrial maintenance

B9.5.3. – Physics

2

Inria Annual Report 2023

1 Team members, visitors, external collaborators

Research Scientists

• Marie-Odile Berger [Team leader, INRIA, Senior Researcher, HDR]

• Erwan Kerrien [INRIA, Researcher, HDR]

Faculty Members

• Fabien Pierre [UL, Associate Professor]

• Gilles Simon [UL, Associate Professor, HDR]

• Frédéric Sur [UL, Professor, HDR]

• Pierre-Fredéric Villard [UL, Associate Professor]

• Brigitte Wrobel-Dautcourt [UL, Associate Professor]

PhD Students

• Youssef Assis [UL, until Nov 2023]

• Nathan Boulangeot [UL]

• Abdelkarim Elassam [INRIA]

• Radhouane Jilani [INRIA]

• Nariman Khaledian [INRIA]

• Hugo Leblond [UL, from Oct 2023]

• Liang Liao [CHRU NANCY]

• Nicolas Maignan [UL]

Technical Staff

• Youssef Assis [INRIA, Engineer, from Nov 2023]

• Romain Boisseau [INRIA, Engineer, until Aug 2023]

Interns and Apprentices

• Gaetano Agazzotti [UL, until Mar 2023]

Administrative Assistant

• Véronique Constant [INRIA]

External Collaborators

• Cédric Demonceaux [UNIV BOURGOGNE, from Mar 2023, HDR]

• Renato Martins [UNIV BOURGOGNE, from Mar 2023]

Project TANGRAM

3

2 Overall objectives

Visual registration is a research topic with a rich history in computer vision. Though a plethora of methods
have been developed and can be used for general situations, there are still many open problems which
originate in the nature of the scene (poorly textured or specular environments), in the type of motion
undergone by the object (tiny motions which hardly emerge from the noise floor, or in contrast, highly
deformable objects) and in dissimilarities which may occur in the scene between the time the modeling
stage occurs and the application time.

Registration is in practice tightly linked to the choice of the model which represents the scene and
the desirable physical properties of the objects. Handling complex —possibly dynamic— scenes thus
requires a tradeoff between physical realism of the model, convergence issues and robustness of the
registration or tracking tasks.

Recent years have seen a surge in research at the intersection of image and deep learning which
has impacted many topics of computer vision. Besides our continued exploration of modeling and
registration with traditional approaches derived from signal processing, geometry, and robust estimation,
one of the team’s aims is to integrate machine learning methods, either as end-to-end methods or as
components, into these 2D or 3D geometric tasks.

Targeted trans-disciplinary applications are mixed and augmented reality, computational photomech-

anics and minimally invasive medical interventions.

3 Research program

3.1 Localization and geometric reasoning with high level features

Our goal is to push forward vision-based scene understanding and localization through the joint use
of learning-based methods with geometrical reasoning. Our hypothesis is that the use of intermediate
representations instead or in addition to the classical point feature will lead to increased capacity in terms
of scale and robustness to changing conditions. These intermediate representations can be concrete
objects which are recognized and used directly in the global pose computation, in the continuity of our
works on ellipsoid modeling of objects, or conceptual objects such as vanishing points (VP) or horizon
lines that are of specific interest both for localization and modeling of urban or industrial scenes.

A first goal is to improve our method for localization from sets of ellipse/ellipsoid correspondences
[3]. Besides the need to have more accurate prediction of ellipses, another objective is to elaborate robust
strategies and associated numerical schemes for refining the initial pose from a set of objects. This
requires us to develop appropriate metrics for characterizing good reprojection of 3D objects onto 2D
ones and study their impact on minimization issues in localization. Another goal is to define strategies to
integrate into the localization procedure various features such as points, objects and VPs, which each
bring information at different levels. We especially want to investigate how predictive uncertainty and
explainability mechanisms can be used to select and weight these various features in the estimation
process.

3.2 Building dedicated models

In this line of research, our goal is to build physically coherent models with a good accuracy vs. efficiency
compromise despite the interactive time constraint set in some targeted applications. Though general
purpose solutions exist for building models, such techniques are still greatly challenged in more complex
cases when specific constraints on the shape or its deformation must be met. This is especially the case
in medical imaging of thin deformable organs, such as the diaphragm, the mitral valve or blood vessels,
but also for classical scene modeling where constraints, such as ellipsoidal abstraction of objects, must
be introduced. The use of mechanical models has become increasingly important in the team’s activities
in medical imaging, especially for handling organs with large deformations. We want to push forward the
development of such models with image-guided procedures or predictive simulation in view.

Facing difficulties of meshing complex geometries, especially thin ones, we want to promote mesh
free methods such as implicit models. In the continuity of past works [4], automatic adaptation of
node locations and sizes to the image will be investigated to improve compactness, and computational

4

Inria Annual Report 2023

efficiency of implicit models. As the fidelity of a mechanical model is often impaired by approximations
required to solve its dynamical system equations at interactive frame rates, a second objective is to take
advantage of our implicit models to improve contact and deformation resolution.

Another topic of interest is the investigation of shape-aware methods either for shape segmentation
or shape recognition, in order to be able to enforce global shape constraints or geometric shape priors
in the output of CNNs. This topic is still addressed in the team in the context of localization from 3D
ellipsoidal abstraction of objects [10]. Two applications are especially targeted: (i) We aim at improving
the detection of pathologies (e.g. brain aneurysms that are mostly located at vessel bifurcations), through
adapted and guided sampling of input data during training, as well as through mechanisms inspired
by visual attention modules. (ii) In the context of fluid structure simulation methods for patient-based
mitral valve simulation, it often appears that the geometric segmented model leads to divergence of the
numerical scheme. Our intention is to identify geometric conditions under which simulation works well
and to incorporate them in the segmentation process.

3.3 Estimation and inverse problems

Most aforementioned tasks lead to image-based inverse, possibly ill-posed, problems. While some of
them can be solved with well-established estimation techniques, others necessitate the design of new
strategies. In this perspective, we consider in this research axis several fundamental aspects of estimation,
common to our problems, such as sampling methods, traditional optimization methods, or end-to-end
learning methods for pose estimation.

3.3.1 Optimization, variational calculus and numerical schemes

We are interested in non-convex optimization problems, especially those raised by variational calculus.
While the convergence of numerical schemes is well established for convex problems, this is not always
the case for non-convex functionals. Our aim is to continue the work already carried out in the biconvex
framework [8], and extend it to primal-dual algorithms. We especially want to address energy minimiza-
tion problems where the energy is convex with respect to each variable, but non-convex with respect to
the pair of variables.

Another research topic is to investigate new neural architectures adapted to non-Euclidean data, and
also to plug variational methods into deep learning approaches to regularize the results. The obtained
theoretical results will be applied to image colorization, with the idea to reduce artefacts caused both
by a lack of regularization and by the non-Euclidean structure of color information as perceived by the
human visual system.

3.3.2 Machine learning for physical problems

We aim at continuing our efforts towards supervised and unsupervised learning for estimation problems.
Concerning supervised learning, we intend to investigate further the opportunities offered by neural
network estimation of displacement and strain fields in experimental mechanics that we have recently
introduced with colleagues in mechanics and signal processing [1]. Besides, we also aim at developing
unsupervised learning in problems where a quantity has to be estimated over a spatio-temporal domain,
which is a recent trend in several application domains. Neural networks are indeed universal approximat-
ors whose derivative can be exactly computed with the backpropagation algorithm, which is supposed to
make them robust to acquisition noise.

4 Application domains

Applications on which our program is expected to have an impact are mixed reality, computational pho-
tomechanics and minimally invasive medical interventions. These fields correspond to areas where we
have established trans-disciplinary collaborations with academic or industrial experts of the applicative
fields. Common to these applications are the need for finely characterizing the acquisition context of
vision-based applications and the need for accurate registration procedures. Another common point is
the availability of a limited amount of data for characterizing the variability of the observed phenomena.

Project TANGRAM

5

Mixed reality Being able to perform reliable and accurate registration under large viewpoint variations,
seasonal or lighting changes opens the way towards challenging mixed reality applications. Urban AR
and industrial maintenance in large and cluttered environments are examples of application fields that
would successfully capitalize on more robust localization solutions. Improved robustness of camera
localization is especially expected for poorly textured, specular environments and in the presence of
repeated patterns that are common in industrial contexts

Photomechanics Photomechanics is the field of experimental mechanics which is dedicated to mech-
anical measurement from images. In particular, we are interested in contactless image-based methods
for extensometry, that is the estimation of displacement and strain fields on the surface of materials
subjected to different types of mechanical loads. Full-field extensometry is a challenging task since strains
often have tiny values and result in gray level changes at the limit of the sensor noise floor. The economic
stakes are high and concern for example the automotive and aeronautics industries, or civil engineering.
In order for these methods to be adopted by industry, it is, however, necessary to quantify their metrolo-
gical performance, which is limited by the registration process or by the image acquisition chain, and
especially by sensor noise. This topic is the subject of a long-term trans-disciplinary collaboration with
Institut Pascal (Clermont-Ferrand Université).

Minimally invasive medical interventions The trend towards the design and performance of minimally
invasive procedures will increase in the near future. But the benefit for the patient is at the expense
of the surgeon who can only sense the surgical scene through intra-operative imaging. Commercial
solutions now exist to teach this increasingly difficult surgical gesture with interactive simulation techno-
logies. However, challenges remain to fill the gap between the learning environment, where qualitative
correctness of the setup is sufficient, and the surgical theater, where accuracy and predictability are
required. In this context, we aim at addressing the key problem of modeling the geometry and dynamics
of deformable organs and surgical devices, in order to make progress towards a faithful 3D rendition of
the surgical scene. To circumscribe practical and experimental difficulties, three specific applications
will be addressed with our clinical partners: intra-operative guidance in interventional neuroradiology,
augmented reality for laparoscopic liver surgery, and simulation of the mitral valve behaviour.

5 Highlights of the year

5.1 Awards

• The TANGRAM team received the best paper presentation award at FIMH 2023 for the paper
"Influence of Anisotropy on Fluid-Structure Interaction Simulations of Image-Based and Generic
Mitral Valves" written by Nariman Khaledian, Pierre-Frédéric Villard and Marie-Odile Berger [20].

• Youssef Assis received a STAR (STudent-Author Registration) award at MICCAI 2023 conference
for his paper "Aneurysm Pose Estimation with Deep Learning" written with co-authors Liang Liao,
Fabien Pierre, René Anxionnat and Erwan Kerrien [16].

6 New software, platforms, open data

6.1 New software

6.1.1 OA-SLAM

Name: Object -aided SLAM

Keywords: Localization, 3D reconstruction, Object detection

Scientific Description: Details on the method can be found in the paper published in ISMAR 2022 [9].

6

Inria Annual Report 2023

Functional Description: OA-SLAM uses objects as landmarks to improve the relocalization capabilities
of SLAM systems. OA-SLAM builds on the point-based ORB-SLAM2. It allows online reconstruction
of 3D objects modeled as ellipsoids from their detections in 2D images. OA-SLAM dramatically
improves the relocalization capabilities of SLAM.

News of the Year: The code was consolidated through its demonstration at CVPR 2023.

URL: https://gitlab.inria.fr/tangram/oa-slam

Publication: hal-03837883

Contact: Gilles Simon

Participants: Matthieu Zins, Gilles Simon, Marie-Odile Berger

6.1.2 P1E

Name: Perspective-1-Ellipsoid

Keywords: Visual localization, Object detection

Functional Description: Pose computation from one correpondance ellipse/ellipsoid. This Matlab code

implements the solution described in [12].

URL: https://gitlab.inria.fr/vgaudill/p1e/

Publication: hal-04132261

Contact: Gilles Simon

Participants: Vincent Gaudilliere, Gilles Simon, Marie-Odile Berger

6.1.3 DeepAnePose

Name: Pose estimation of brain aneurysms using deep learning

Keywords: Deep learning, Anomaly detection, Medical imaging, Brain MRI, Brain aneurysm, Pose

estimation

Functional Description: DeepAnePose is a deep convolution network for the detection and pose estim-
ation of intracranial aneurysms from 3D TOF-MRI images. It is a YOLOv3-inspired anchor-free
detection model in 3D, extended with a pose estimation head, coupled with an original strategy for
small patch generation that combines data augmentation and data synthesis.

News of the Year: The code was finalized, consolidated, and issued through our publication at MICCAI

2023.

URL: https://gitlab.INRIA.fr/yassis/DeepAnePose

Publications: hal-04207337, hal-03391884, hal-03897642

Contact: Erwan Kerrien

Participants: Youssef Assis, Erwan Kerrien

Partner: Loria

Project TANGRAM

7

7 New results

7.1 Visual localization

Participants: Marie-Odile Berger, Romain Boisseau, Hugo Leblond, Gilles Simon.

For several years now, and following the track of research inited in [3], the team has been investigating
the use of objects as landmarks for pose computation. Objects are here approximated by ellipsoids from
their 2D detection in images as ellipses. Using generic object detectors allows handling any environment
without accurate knowledge of its geometry.

This year we have investigated the fundamental problem of camera pose estimation from one ellipse-
ellipsoid correspondence, referred to as the P1E problem. On the theoretical side, and except in the case
of a spherical object, we have demonstrated in [12] that the solutions are a variety of dimension 1. We
have also provided an effective way to compute the camera locus (i.e. set of solutions). This problem was
addressed by Wokes and Palmer in 2010 [23] in the particular case of a spheroid (specific ellipsoid having
an axis of revolution) but was never considered for general ellipsoids. To the best of our knowledge, we
are the first to propose a constructive solution to the P1E problem without resorting to any additional
approximation nor prior knowledge. In this study, we also consider two particular cases of important
practical interest: (i) computing the camera position when the orientation is known (ii) computing
the orientation when the position is known. Besides the theoretical aspects, solving the P1E problem
opens the way towards automatic positioning solutions in texture-less or low-textured environments, for
instance leveraging several ellipse-ellipsoid correspondences. The accompagnying code of the method is
available here.

Still in the context of object-based positionning, our objet aided SLAM method OA-SLAM [9] was

selected in the demo session at CVPR 2023.

7.2 Vanishing point computation and applications

Participants: Marie-Odile Berger, Romain Boisseau, Abdelkarim Elassam, Gilles Si-

mon.

For several years now, the Tangram team has been working on the automatic detection of vanishing
points in an image, a preliminary step in the rectification of planar structures present in a picture, which is
also useful for determining the intrinsic and orientation parameters of a camera. Traditionally considered
as a problem of grouping line segments by meeting points, this task has been reconsidered in recent years
in the light of deep learning techniques. The main difficulties lie in the inability to predict the number of
vanishing points expected and the possibility of obtaining such points outside and sometimes far from
the image. A first solution was to model the extraction of vanishing points as a problem of classifying their
positions among intervals of coordinates along the horizon line. An a-contrario method made it possible
to detect the peak scores obtained at the output of a CNN, each peak corresponding to a horizontal
vanishing point.

In a paper published at ICIP 2023, we demonstrated that the performance of this method can be
improved by simultaneously detecting the vertical structures associated with vanishing points, in the
form of a map of discretized normal orientations, using the same classification as for vanishing points
[18]. The method makes it possible to detect additional vanishing points, corresponding to score peaks in
the output of the previous CNN, which were however considered to be meaningless by the a-contrario
decision scheme. Considered as a by-product of the method, the vertical regions associated with vanishing
points are currently being investigated to match vanishing points between distant images, something
that was difficult to envisage until now, vanishing points being abstract primitives not directly linked
to photometry. This paves the way for robust vanishing point matching and relative pose computation
between distant images.

In parallel with these activities, a collaboration with the historian Ludovic Balavoine (Université de
Bordeaux Montaigne) was undertaken to analyse the vanishing points, using the probabilistic method

8

Inria Annual Report 2023

presented in [6], of around a hundred paintings of Flemish primitive art. This study led to a revision
of twentieth-century historiography, which predicted a late introduction of perspective in Flanders
compared with Italy. A book resulting from this collaboration has recently been accepted for publication
by the international publisher Brepols.

7.3 Handling non rigid deformation

Participants: Marie-Odile Berger, Nariman Khaledian, Pierre-Frédéric Villard.

7.3.1 Individual mitral valve modeling

Our previous work on the mitral valve includes segmenting the valve anatomy and simulating its closure
using fluid-structure interaction (FSI) with a generic valve. This work allows us to improve the detection of
blood leaks by creating a map of contact [5]. This year we have focused on moving towards patient-based
modelling and incorporating anisotropy into our model to get closer to real valve behaviour [20]. Four
valves extracted from medical images were tested to show that our biomechanical model can be scaled to
work with real data. We also investigated the importance of the anisotropic directions by testing various
fiber map strategies: random, parallel to the annulus and based on physical computation. Although
the simulation result is affected by this choice, there is no strong evidence that it leads to better sealing
properties. Our next step will be to collect experimental data, which can provide the missing ground
truth, and to run simulations under the same conditions.

7.3.2 Image-based biomechanical simulation of the diaphragm during mechanical ventilation

This ultimate goal of this project is to perform high-fidelity, real-time simulations of the respiratory
function of a critical care patient. The focus is on the respiratory muscles, where the diaphragm is the
main muscle. This involves as the first step a realistic tissue model, which allows us to model the passive
deformation of the muscle. This year, we have been working on a model for active muscle contraction,
which includes at least an approximate model for the muscle fiber bundles and their orientation. From a
numerical perspective this corresponds to solving non-linear PDEs in thin structures. The methods used
are localized radial basis function (RBF) methods. Last year, we implemented a linear elasticity solver
using the RBF with the partition of unity method (PUM) method, which was used to deform a shell bunny
model, a webbed cube and the diaphragm. We have done a FEM implementation for comparison. We
are also continuing to work on RBF-PUM applied to elasticity problems. For next year, we aim to also
consider models that are not linear. We will try different types of models to see how they perform. We
would also like to make the first tests on muscle activation.

7.4 Interventional radiology

Participants:

Youssef Assis, Radhouane Jilani, Erwan Kerrien, Liang Liao, Fa-
bien Pierre, Pierre-Frédéric Villard.

7.4.1 Detection of brain aneurysms using deep learning

The YOLO-inspired anchor free deep neural network designed by Youssef Assis during his PhD thesis was
further validated this year using two clinical experts to assess each aneurysm detection by our model,
nnDetection [22] and nnUnet, on a database of 270 patients. The performance assessed using the induced
new clinical ground truth (double blind annotation with consensus) was even improved, exceeding the
experts’ sensitivity. This model was extended this year to tackle the innovative task of aneurysm pose
estimation [16]. Median errors of 0.5 mm for the position and 12° for the orientation were deemed
accurate enough for the display of a reformatted cut plane through the aneurysm that is clinically useful.
This work received a STAR award (STudent-Author Registration) at the MICCAI 2023 conference. The

Project TANGRAM

9

corresponding code and ground truth annotations were made available in the DeepAnePose software.
In the context of his PhD thesis, Liang Liao is leading an extensive data collection campaign to have 8
medical doctors annotate the database. The aim is to 1) further characterize the strengths and weaknesses
of our model; 2) better assess the impact of our model to improve the physician’s performance, depending
on their expertise; and 3) study how multiple annotations can compensate for the lack of patient data, in
order to tighten the statistical significance of the model comparison.

7.4.2 Predictive simulation of catheter navigation

Our main contribution to the PreSPIN ANR project consists in achieving smooth, interactive and pre-
dictive simulation of a catheter navigating in the brain vasculature. In the course of his PhD thesis work,
Radhouane Jilani proposed a new collocation method to solve the boundary value problem resulting from
the Cosserat formulation. This method relies on the approximation of the internal forces and moments
of a 1D rod on a finite Chebyshev basis. This method was demonstrated to be more stable than previous
methods, though a bit slower than the shooting method in the static case. Our method was extended to
the dynamic case and experimented in various standard tests used in the robotics community [19]. A
further extension to manage contact with implicit surfaces is currently under development.

7.5

Image and signal processing

Participants: Gaetano Agazzotti, Nicolas Maignan, Fabien Pierre, Frédéric Sur.

7.5.1 Computational photomechanics

The work of this year concerns two aspects of displacement and strain field estimation over a material
submitted to a mechanical load. Displacement estimation shows similarities with optical flow estimation.
In order to process the surface of the material, either a contrasted marking is projected on it, giving a
speckle pattern, or a periodic pattern (a checkerboard) is marked on the surface. We have customized a
state-of-the-art convolutional neural network dedicated to optical flow estimation (for natural images)
to reach better performance when processing speckle images. We have designed a lightweight CNN by
reducing as much as possible the number of filters while keeping equivalent metrological performance
to the original version (and almost equivalent to state-of-the-art standard digital image correlation
estimation), in order to accelerate image processing on a power-efficient compact Graphics Processing
Unit (GPU) [11]. The ultimate goal is to develop smart-cameras dedicated to displacement and strain
measurement. Another contribution of this year concerns the estimation of displacement and strain
fields from a deformed checkerboard pattern, with windowed Fourier analysis. In this case, we have
proved that the estimated displacement and strain fields are merely the convolution of the true fields
with the analysis window. We have proposed and fine-tuned a deconvolution process to enhance the
small and sharp details in displacement and strain maps [13].

7.5.2 Variational methods for image processing

During the internship of Gaetano Agazzotti, the combination of deep image priors and variational
methods has been explored for single image colorization. The deep image prior model is able to produce
realistic images while suffering from overfitting of the data that results in noisy images and blurring of
the colorization results (the so-called halo effect). With a regularization of the result with coupled total
variation, these effects can be avoided and controled [15].

The colorization of videos is mostly based on color transfer between frames. The state-of-the-art
methods suffer from occlusions and dis-occlusions during the matching phase. Since such problems
are generally solved through 3D knowledge on the scene, the PhD thesis of Nicolas Maignan aims, in
particular, at incorporating the 3D information from a monocular video sequence in the colorization
process.

10

Inria Annual Report 2023

7.6 Application of deep learning

Participants: Nathan Boulangeot, Mehdi Serdoun, Frédéric Sur.

7.6.1 Extensive searches for complex intermetallic catalysts

Material science is at the heart of many technological revolutions. The search for new materials is crucial
to meet the challenges posed by climate change, or the growing global demand for energy and consumer
goods. In this context, the discovery of new catalytic materials is a major challenge to improve the
performance of industrial processes, for example in the context of the hydrogen economy, in order to
provide energy sources with limited environmental impact. Although some theoretical models exist,
catalyst research is still largely empirical and based on a trial-and-error approach. This is probably
because the application of theoretical models requires particularly expensive calculations based on
density functional theory (DFT). Recently, the materials community has been seizing on statistical
learning tools to limit the cost of calculations and to accelerate the discovery of new catalysts. Together
with Emilie Gaudry at Institut Jean-Lamour (IJL) at Nancy, we have developed a collaboration on this
subject since September 2021 through Nathan Boulangeot’s PhD thesis. The work of this year concerns
models for infering the energy of the system and the forces applied to the atoms in the context of
predicting the adsorption properties of Al 13C o4 based on Gaussian process regression. In particular,
we train a model to efficiently select adsorption configurations and capture the symmetry of the true
adsorption with a minimal numer of DFT calculations.

7.6.2 Multivariate analysis of geochemical, physical and mineralogical signatures of uranium depos-

its in the Athabasca Basin (Saskatchewan, Canada)

We are engaged in the co-supervision (together with Julien Mercadier, GéoRessources) of the PhD thesis
of Mehdi Serdoun, which is part of the GeoMin3D project funded by ANR and Orano Mining, since
January 2022. The goal is to develop statistical learning models to analyze the large amount of data of
diverse nature provided during the exploratory drillings in Athabasca basin, the largest known source of
uranium. The ultimate goal is to develop new analysis tools to accelerate exploration and reduce its cost,
in cooperation with the industrial actors.

8 Partnerships and cooperations

8.1 International initiatives

8.1.1 Associate Teams in the framework of an Inria International Lab or in the framework of an Inria

International Program

CURATIVE

Title: CompUteR-based simulAtion Tool for mItral Valve rEpair

Duration: 2021-2024

Coordinator: Pierre-Frédéric Villard

Partners:

• Harvard University (États-Unis)

Participants: Marie-Odile Berger, Nariman Khaledian, Pierre-Fredéric Villard

Summary: The mitral valve of the heart ensures one-way flow of oxygenated blood from the left atrium
to the left ventricle. However, many pathologies damage the valve anatomy producing undesired
backflow, or regurgitation, decreasing cardiac efficiency and potentially leading to heart failure if

Project TANGRAM

11

left untreated. Such cases could be treated by surgical repair of the valve. However, it is technically
difficult and outcomes are highly dependent upon the experience of the surgeon. One of the main
difficulties of valve repair is that valve tissues must be surgically altered during open heart surgery
such that the valve opens and closes effectively after the heart is closed and blood flow is restored.
In order to do this successfully, the surgeon must essentially mentally predict the displacement and
deformation of anatomically and biomechanically complex valve leaflets and supporting structures
[11]. Even if patient-based mitral valve models have been recently used for scientific understanding
of its complex physiology, the patient geometry is manually segmented on medical images. This
task is long and cumbersome except if the valve has been artificially isolated in-vitro. There is a
lack in the literature about the variety of metrics in both anatomy and biomechanics of the valve.
In order to study mitral valve behavior or to prepare models for planning, it is necessary to have
methods to extract the valve components i) on real clinical data ii) with minor user input and ii)
that are mechanically valid.

8.2 International research visitors

8.2.1 Visits of international scientists

Other international visits to the team

Peter Hammer

Status researcher

Institution of origin: Harvard Medical School

Country: USA

Dates: 5/06/2023 - 09/06/2023

Context of the visit: Visit within the context of the associated team CURATIVE. In order to perform
mitral valve modeling with real patient’s data, our discussions focussed on (i) future experiments in
Boston to determine anisotropic constitutive law on porcine data (ii) the analysis of our first results
with real patient’s data.

Mobility program/type of mobility: research stay

Hao Gao

Status senior lecturer

Institution of origin: University of Glasgow

Country: Great Britain

Dates: 2/12/2023 - 21/12/2023

Context of the visit: After meeting at the FIMH conference in June, Hao Gao agreed to visit our team.
He is a well-known expert in cardiac modelling. We worked on sharing our knowledge of mitral
valve modelling. He gave a seminar to the team. He helped us to install the Immersed Boundary
Framework that he uses for his research and we started to use it with our own dataset (1 generic
and 4 image-based valves). The idea is to continue working on the comparison of our two methods
in 2024.

Mobility program/type of mobility: research stay funded by Inria

8.2.2 Visits to international teams

Research stays abroad

12

Inria Annual Report 2023

Pierre-Frédéric Villard

Visited institution: Uppsala University

Country: Sweden

Dates: 01/04/2023 - 31/04/2023

Context of the visit: Visit within the context of the project INVIVE. The aim was first to continue the
supervision of a medical student who started working last year on segmenting the diaphragm. One
new dataset was segmented and added to our database. The aim was also to supervise the PhD
student (Andreas Michael) on the mechanical modeling of the diaphragm.

Mobility program/type of mobility: research stay

Pierre-Frédéric Villard

Visited institution: Boston Robotics Lab

Country: USA

Dates: 01/11/2023 - 30/11/2023

Context of the visit: Visit within the context of the associated team CURATIVE. The aim was mainly to
perform experiments on a phantom valve closing in a fully controlled environment. The phantom
valve was already scanned last year to have the 3D geometry. Ultrasound and stereoscopic images
were acquired with various conditions as well as pressure and flowrate data. Pierre-Frédéric Villard
participated in the Harvard Robotics lab life by attending and giving seminars.

Mobility program/type of mobility: research stay

8.3 European initiatives

8.3.1 Other european programs/initiatives

Title: Towards robust spatial scene understanding in dynamic environments using intermediate repres-

entations

Partners: DFKI Kaiserslautern

Coordinator: Marie-Odile Berger

Participants: M.-O. Berger, R. Boisseau, A. Elassam, G. Simon

Duration: 2020-2023

Additionnal info/keywords: The aim of the MOVEON project is to push forward the state of the art
in vision-based, spatio-temporal scene understanding by merging novel machine-learning ap-
proaches with geometrical reasoning. Deep-learning-based recognition and understanding of
high-level concepts such as vanishing points or large object classes will serve as unitary building
blocks for a spatio-temporal localization and environment reconstruction that will use geometric
reasoning as underlying support. This research will lead to a novel generation of visual positioning
systems that go beyond classical localization and mapping, which focuses currently only on point
cloud reconstruction. In contrast, our aim is to allow for 6DoF positioning and global scene under-
standing in wild and dynamic environments (e.g. crowded streets) that scales up nicely with the
size of the environment, and that can be used persistently over time by reusing consistent maps.

8.4 National initiatives

PEPR ICCARE

Gilles Simon is coordinator of the "Museum and Cultural Heritage" sector of the PEPR ICCARE (Cultural

and Creative Industries: Action, Research, Experimentation).

Project TANGRAM

ANR Arcé

Title :Colorisation automatique de vidéos

Coordinator: Fabien Pierre

Participants: F. Pierre, N. Maignan, F. Sur

Duration: 2022-2026

13

Additionnal info/keywords: The Arcé project aims at proposing new methods for automatic, fast and
perceptually satisfying video colorization. Image colorization methods based on deep learning
based have encountered a great success in recent years. These techniques are fully automatic and
very fast, but they have not been adopted by colorization industry. The reason is that they do not
ensure the temporal coherence of the colorization, which is particularly disturbing for the viewer.
The ultimate goal is the use of our work in audiovisual production studios.

ANR PRC PreSPIN

Title: Predictive Simulation for Planning Interventional Neuroradiology procedures

Partners: CReSTIC (Reims), Creatis (Lyon) and CIC-IT/CHRU Nancy

Coordinateur: Erwan Kerrien

Participants: Y. Assis, R. Jilani, E. Kerrien, P.-F Villard.

Duration: 2020-2024

Additionnal info/keywords: This 4-year project is coordinated by E. Kerrien. It aims at improving the
planning phase in the therapeutic management of cerebral ischemic strokes thanks to predictive
simulation of both the therapeutic interventional gesture and post-interventional perfusion images.
The consortium is set to address the challenges of geometrical and topological modeling of the full
brain vasculature; physics-based simulation of interventional devices; simulation of MRI perfusion
images; and clinical validation.

9 Dissemination

9.1 Promoting scientific activities

9.1.1 Scientific events: organisation

Member of the organizing committees

• Erwan Kerrien was a member of the organizing committee for the Digital and Health Sciences day
that brought together 100 numerical scientists and clinicians at the University Hospital in Nancy.

9.1.2 Scientific events: selection

Member of the conference program committees

• Frédéric Sur was a program committee member for the International Conference on Computer

Vision Theory and Applications (VISAPP 2023).

14

Reviewer

Inria Annual Report 2023

• Marie-Odile Berger was a reviewer for IPCAI (International Conference on Information Processing
in Computer-Assisted Interventions), IROS (International Conference on Intelligent Robots and Sys-
tems), ICRA (International Conference on Robotics and Automation) and for the French conference
ORASIS.

• Erwan Kerrien was a reviewer for MIDL (Medical Imaging with Deep Learning), MICCAI (Interna-
tional Conference on Medical Image Computing and Computer Assisted Interventions), IROS, and
the French conference ORASIS.

• Gilles Simon was a reviewer for IROS and the French conference ORASIS.

• Pierre-Frédéric Villard was a reviewer for the Eurographics Workshop on Visual Computing for
Biology and Medicine 2023 and the International Conference on Computer Graphics, Visualization,
Computer Vision And Image Processing 2023.

9.1.3 Journal

Reviewer - reviewing activities

• Erwan Kerrien was a reviewer for Medical Image Analysis and Medical Physics.

• Fabien Pierre was a reviewer for SIAM imaging sciences

• Gilles Simon was a reviewer for the International Journal of Computer Vision (IJCV).

• Frédéric Sur was a reviewer for Experimental Mechanics, Optics and Lasers in Engineering, Meas-

urement, and IEEE Transactions on Instrumentation and Measurement.

9.1.4 Invited talks

• Abdelkadir Elassam gave a seminar in September 2023 in the one day workshop "Calcul visuel
de pose en robotique et réalité-X" organized by GDR ISIS. Title: "Advancing Wide Baseline Scene
Registration with Structural Vanishing Points".

• Fabien Pierre gave a seminar at INSA Rouen in May 2023. Title "Couplage de méthodes variation-

nelles et CNN pour la colorisation d’images".

• Gilles Simon gave a seminar to the Translitteræ research school at ENS Paris-Saclay. Title: “Les

dispositifs optiques en peinture chez les maîtres anciens" on June 14th 2023.

• Pierre-Frédéric Villard did a presentation to the Harvard Biorobotics Lab within the CURATIVE

collaboration. Title: "New contributions in mitral valve simulation" on November 9th 2023.

• Pierre-Frédéric Villard gave a seminar at the department of information technology of Uppsala

University. Title: "Simulation of Mitral Closing with anisotropy" on May 17th 2023.

• Pierre-Frédéric Villard gave an invited talk at the workshop "Simulation and Imaging for Mitral
Regurgitation" on June 22nd 2023 in Lyon. Title: "Mitral Valve Modeling: From Medical Image
Analysis to Fluid Structure Interaction"

• Pierre-Frédéric Villard gave a keynote talk at the national conference " journée de la simulation
numérique en mécanique" on July 12th in Metz. Title: "Modélisation basée patient de la fermeture
de la valve mitrale avec de l’Interaction Fluide-Structure"

9.1.5 Scientific expertise

• Marie-Odile Berger was a member of the committee that awards the annual AFRIF thesis prize. She

was an expert for the Israel Science Foundation and for ANRT.

• Erwan Kerrien was an expert for the ANR and the FC3R scientific interest group.

Project TANGRAM

15

9.1.6 Research administration

• Marie-Odile Berger is head of the INRIA COMIPERS PhD and postdoctoral recruitment committee.
She was a member of the recruitment committee for an assistant professor position position at
Université Savoie Mont-Blanc.

• Gilles Simon was an elected member of the CNU (Conseil National des Universités)

• Frédéric Sur was a member of the recruitment committee for two Assistant Professor positions at
Université de Lorraine, and one recruitment committee for a Professor position at Université de
Strasbourg.

• Pierre-Frédéric Villard is an elected member of the Scientific Council of the Université de Lorraine

9.2 Teaching - Supervision - Juries

9.2.1 Teaching

The assistant professors of the TANGRAM team actively teach at Université de Lorraine with an annual
number of around 200 teaching hours in computer sciences, some of them being accomplished in the
field of image processing. INRIA researchers have punctual teaching activities in computer vision and
shape recognition mainly in the computer science Master of Nancy and in several Engineering Schools
near Nancy (ENSMN Nancy, SUPELEC Metz, ENSG). Our goal is to attract Master students with good
skills in applied mathematics towards the field of computer vision.
The list of courses given by staff members is detailed below:

M.-O. Berger

Master : Shape recognition, 24 h, Université de Lorraine.

Master : Introduction to image processing, 12 h, ENSMN Nancy.

Master : Image processing for Geosciences, 12h, ENSG.

E. Kerrien

Master : Introduction to image processing, 15 h, ENSMN Nancy.

Licence : Initiation au développement, 80h, IUT St Dié-des-Vosges.

Fabien Pierre

Master: Introduction à l’apprentissage automatique, 14h, Mines Nancy.

Master: Vision artificielle et traitement des images, 12h, Polytech Nancy.

Licence: Introduction au traitement d’image, 30h, IUT Saint-Dié des Vosges.

Licence: Algorithmique et programmation, 87h, IUT Saint-Dié des Vosges

Licence: Culture scientifique et traitement de l’information, 69h, IUT Saint-Dié des Vosges

Licence: Programmation objet et évènementielle, 35h, IUT Saint-Dié des Vosges

Licence: Initiation à l’intelligence artificielle, 18h, IUT Saint-Dié des Vosges

G. Simon

Master: Augmented reality, 9 h, Télécom-Nancy.

Master: Augmented reality, 24h, M2 Informatique FST

Master: Visual data modeling, 12h, M1 Informatique FST

Master: Computer Vision, 12h, M1 Informatique FST

Licence pro: 3D modeling and augmented reality, 50h FST - CESS d’Epinal

Licence: Programming methodology, L1 informatique, 48h FST

16

F. Sur

Inria Annual Report 2023

Master: Introduction to machine learning, 40 h, Université de Lorraine (Mines Nancy).

Licence: Javascript programming, 100h, IUT Charlemagne

P.-F Villard

Master : Augmented and Virtual Reality, 16h, M2 Cognitive Sciences and Applications, Institut des

Sciences du Digital, Université de Lorraine

Licence: Computer Graphics with webGL, 30h, IUT Saint-Dié des Vosges.

Licence: Virtual and Augmented Reality in Industrial Maintenance, 2h, Faculty of Science and

Technology, Université de Lorraine

Licence: Web programming, 20h, IUT Saint-Dié des Vosges.

Licence: Graphical user interface programming, 30h, IUT Saint-Dié des Vosges.

Licence: Security and life privacy with internet, 2h, IUT Saint-Dié des Vosges.

Licence: Parallel programming, 18h, IUT Saint-Dié des Vosges.

Licence: Initiation to machine learning, 24h, IUT Saint-Dié des Vosges.

Licence: Initiation to cryptography, 12h, IUT Saint-Dié des Vosges.

B. Wrobel-Dautcourt

Licence: interfaces graphiques, 22h, FST

9.2.2 Supervision

• PhD in progress: Abdelkarim Elassam, Robust visual localization using high level features, October

2020, Marie-Odile Berger, Gilles Simon.

• PhD in progress: Nariman Khaledian, Toward a Functional Model of the Mitral Valve, October 2020,

Marie-Odile Berger, Pierre-Frédéric Villard.

• PhD in progress: Youssef Assis, Deep learning for the automated detection of brain aneurysms,

November 2020, Erwan Kerrien, René Anxionnat (CHRU Nancy).

• PhD in progress: Radhouane Jilani, Predictive simulation for interventional neuroradiology, October

2021, Erwan Kerrien, Pierre-Frédéric Villard.

• PhD in progress: Liang Liao, Detection of cerebral aneurysms from MRI images using deep learning:
deep neural network creation and its clinical evaluation, November 2022, René Anxionnat (CHRU
Nancy) and Erwan Kerrien.

• PhD in progress: Nathan Boulangeot, Extensive searches for complex intermetallic catalysts, Octo-

ber 2021, Émilie Gaudry (Institut Jean-Lamour), Frédéric Sur.

• PhD in progress: Mehdi Serdoun, Multivariate analysis of mineralogical, geochemical and physical

signatures, January 2022, Julien Mercadier (GéoRessources), Frédéric Sur.

• PhD in progress: Nicolas Maignan, Image and video colorization, October 2022, Fabien Pierre,

Frédéric Sur

• PhD in progress: Hugo Leblond, November 2023, Analyse de scènes dynamiques à partir d’une
représentation neuronale implicite (NeRF) basée sur des données LiDAR-caméra, Gilles Simon,
Renato Martins

Project TANGRAM

9.2.3 Juries

17

• Marie-Odile Berger was president of the PhD committee of Prerak Srivastava (Université de Lor-
raine), Nuwan Herath Mudiyanselage (Université de Lorraine) and Yumin Du (ENPC). She was
external reviewer for the PhD thesis of Clémentin Boittiaux (Université de Toulon). She was exam-
iner of M. Tammazousti’s HDR committee (CEA List).

• Gilles Simon was a reviewer for the PhD thesis of Andrea Macario Barros (CEA List, Université de

Paris Saclay).

• Frédéric Sur was president of the PhD committee of Sandipana Dowerah (Université de Lorraine).

• Erwan Kerrien was an external reviewer for the PhD theses of Jonas Lamy (Université de Lyon 2)

and Aurélien de Turenne (Université de Rennes 1).

9.3 Popularization

9.3.1 Internal or external Inria responsibilities

• Erwan Kerrien is Chargé de Mission for scientific mediation at Inria Nancy-Grand Est, and thereby is
part of the Inria scientific mediation network. As such, he is a member of the steering committee of
"la Maison pour la Science de Lorraine", and member of the IREM Lorraine (Institut de Recherche
sur l’Enseignement des Mathématiques - Research Institute for Teaching Mathematics) steering
council. He is also the local scientific referent for the "Chiche!" initiative.

• Erwan Kerrien shares the local coordination of MATh.en.JEANS in the Lorraine area with Samuel

Tapie from the IECL lab (mathematics).

9.3.2 Articles and contents

• Frédéric Sur wrote a paper about his experience in setting up a new course on machine learning at

École des Mines de Nancy [14].

9.3.3 Education

• Pierre-Frédéric Villard was a mentor for the secondary school of Champigneulles (France) as a
"Collège Pilote" of "La Main à la pâte" foundation. He gave a seminar on augmented and virtual
realities to the pupils, he helped the teacher with preparing some activities with augmented and
virtual reality technologies. Eventually, he is supervising master students to produce teaching
applications with augmented reality technologies that will be used in secondary school classes.

• Pierre-Frédéric Villard presented a workshop on automatic character recognition using deep

learning technique at the "Fête de la Science" in St-Dié-des-Vosges.

• Pierre-Frédéric Villard presented a workshop for high school teachers in computer science at Loria

on April 5th on how to program a person recognition application.

• Pierre-Frédéric Villard did two science cafés: "Ecodesign and responsible digital practices" Thursday
on March 9 2023, at the amphitheater of IUT Saint-Dié (Saint Dié des Vosges) and "The other side
of streaming", on April 13rd 2023, at Inria Grand Est (Nancy)

• Pierre-Frédéric Villard and Erwan Kerrien did presentations to high school students in the context

of the "Chiche!" initiative.

• Erwan Kerrien was an associate researcher to a MATh.en.JEANS workshop within Henri Loritz high
school in Nancy, Alfred Cytère secondary school in Rambervillers and Jean d’Arc technical school
in Commercy.

• Erwan Kerrien animated a workshop during the APMEP day (Maths teachers association in public

education).

18

Inria Annual Report 2023

9.3.4 Interventions

• Gilles Simon gave a talk as part of the CNRS "Visites insolites" outreach intiative.

10 Scientific production

10.1 Major publications

[1] S. Boukhtache, K. Abdelouahab, F. Berry, B. Blaysat, M. Grediac and F. Sur. ‘When Deep Learning
Meets Digital Image Correlation’. In: Optics and Lasers in Engineering 136 (Jan. 2021), p. 106308.
DOI: 10.1016/j.optlaseng.2020.106308. URL: https://hal.archives-ouvertes.fr/hal-
02933431.

[2] A. Fond, M.-O. Berger and G. Simon. ‘Model-image registration of a building’s facade based on
dense semantic segmentation’. In: Computer Vision and Image Understanding 206 (May 2021),
p. 103185. DOI: 10.1016/j.cviu.2021.103185. URL: https://hal.inria.fr/hal-03204477.

[3] V. Gaudillière, G. Simon and M.-O. Berger. ‘Camera Relocalization with Ellipsoidal Abstraction
of Objects’. In: ISMAR 2019 - 18th IEEE International Symposium on Mixed and Augmented
Reality. Beijing, China: IEEE, 14th Oct. 2019, pp. 19–29. DOI: 10.1109/ISMAR.2019.00017. URL:
https://hal.archives-ouvertes.fr/hal-02170784.

[4] E. Kerrien, A. Yureidini, J. Dequidt, C. Duriez, R. Anxionnat and S. Cotin. ‘Blood vessel modeling for
interactive simulation of interventional neuroradiology procedures’. In: Medical Image Analysis 35
(Jan. 2017), pp. 685–698. DOI: 10.1016/j.media.2016.10.003. URL: https://hal.inria.fr/h
al-01390923.

[5] N. Khaledian, P.-F. Villard and M.-O. Berger. ‘Capturing Contact in Mitral Valve Dynamic Closure
with Fluid-Structure Interaction Simulation’. In: International Journal of Computer Assisted Radi-
ology and Surgery (2022). DOI: 10.1007/s11548-022-02674-4. URL: https://inria.hal.sci
ence/hal-03708218.

[6] G. Simon. ‘Jan Van Eyck’s Perspectival System Elucidated Through Computer Vision’. In: Proceedings
of the ACM on Computer Graphics and Interactive Techniques 4.2 (July 2021). DOI: 10.1145/34656
23. URL: https://hal.univ-lorraine.fr/hal-03287031.

[7] G. Simon, A. Fond and M.-O. Berger. ‘A-Contrario Horizon-First Vanishing Point Detection Using
Second-Order Grouping Laws’. In: ECCV 2018 - European Conference on Computer Vision. Munich,
Germany, 8th Sept. 2018, pp. 323–338. URL: https://hal.inria.fr/hal-01865251.

[8] P. Tan, F. Pierre and M. Nikolova. ‘Inertial Alternating Generalized Forward-Backward Splitting for
Image Colorization’. In: Journal of Mathematical Imaging and Vision 61.5 (Feb. 2019), pp. 672–690.
DOI: 10.1007/s10851-019-00877-0. URL: https://hal.archives-ouvertes.fr/hal-0179
2432.

[9] M. Zins, G. Simon and M.-O. Berger. ‘OA-SLAM: Leveraging Objects for Camera Relocalization
in Visual SLAM’. In: ISMAR 2022 - 21st IEEE International Symposium on Mixed and Augmented
Reality. Singapour, Singapore, 17th Oct. 2022. URL: https://hal.science/hal-03837883.

[10] M. Zins, G. Simon and M.-O. Berger. ‘Object-Based Visual Camera Pose Estimation From Ellipsoidal
Model and 3D-Aware Ellipse Prediction’. In: International Journal of Computer Vision 130 (7th Mar.
2022), pp. 1107–1126. DOI: 10.1007/s11263-022-01585-w. URL: https://hal.science/hal-
03602394.

10.2 Publications of the year

International journals

[11]

S. Boukhtache, K. Abdelouahab, A. Bahou, F. Berry, B. Blaysat, M. Grédiac and F. Sur. ‘A lightweight
convolutional neural network as an alternative to DIC to measure in-plane displacement fields’. In:
Optics and Lasers in Engineering 161 (Feb. 2023), p. 107367. DOI: 10.1016/j.optlaseng.2022.1
07367. URL: https://hal.science/hal-03897689.

Project TANGRAM

19

[12] V. Gaudillière, G. Simon and M.-O. Berger. ‘Perspective-1-Ellipsoid: Formulation, Analysis and
Solutions of the Camera Pose Estimation Problem from One Ellipse-Ellipsoid Correspondence’. In:
International Journal of Computer Vision (9th June 2023), p. 24. DOI: 10.1007/s11263-023-0179
4-x. URL: https://inria.hal.science/hal-04132261.

[13] M. Grédiac, X. Balandraud, B. Blaysat, T. Jailin, R. Langlois, F. Sur and A. Vinel. ‘Fine-Tuning
a Deconvolution Algorithm to Restore Displacement and Strain Maps Obtained with LSA’. In:
Experimental Mechanics 63.9 (13th Oct. 2023), pp. 1509–1537. DOI: 10.1007/s11340-023-00997-
0. URL: https://hal.science/hal-04250310.

[14] F. Sur. ‘Enseignement de l’apprentissage automatique à destination d’ingénieur·e·s généralistes’.
In: Academic Journal of Civil Engineering 41.2 (2023), pp. 100–105. DOI: 10.26168/ajce.41.2.10.
URL: https://hal.science/hal-04213203.

International peer-reviewed conferences

[15] G. Agazzotti, F. Pierre and F. Sur. ‘Deep image prior regularized by coupled total variation for image
colorization’. In: Lecture Notes in Computer Science. SSVM2023 9th International Conference on
Scale Space and Variational Methods in Computer Vision. Vol. 14009. Sardinia, Italy: Springer,
24th May 2023, pp. 301–313. DOI: 10.1007/978-3-031-31975-4_23. URL: https://hal.scien
ce/hal-04035467.

[16] Y. Assis, L. Liao, F. Pierre, R. Anxionnat and E. Kerrien. ‘Aneurysm Pose Estimation with Deep
Learning’. In: MICCAI, Lecture Notes in Computer Science book series (LNCS). Medical Image
Computing and Computer Assisted Intervention (MICCAI). Vol. 14221. Vancouver, Canada, 1st Oct.
2023. DOI: 10.1007/978-3-031-43895-0_51. URL: https://hal.univ-lorraine.fr/hal-0
4207337.

[17] L. Calvet, N. Maignan, B. Brument, J. Mélou, S. Tozza, J.-D. Durou and Y. Quéau. ‘Multi-View Normal
Estimation - Application to Slanted Plane-sweeping’. In: SSMV 2023. 9th International Conference
on Scale Space and Variational Methods in Computer Vision (SSMV 2023). Vol. 14009. Lecture Notes
in Computer Science. Santa Margherita di Pula, Sardinia, Italy: Springer International Publishing,
10th May 2023, pp. 704–716. DOI: 10.1007/978-3-031-31975-4_54. URL: https://hal.scien
ce/hal-04038245.

[18] A. Elassam, G. Simon and M.-O. Berger. ‘COLLAB-VP: Structure-enhanced VP Detector’. In: IEEE
International Conference on Image Processing (ICIP 2023). kuala Lumpur, Malaysia, 8th Oct. 2023.
DOI: 10.1109/ICIPC59416.2023.10328343. URL: https://hal.science/hal-04192288.

[19] R. Jilani, P.-F. Villard and E. Kerrien. ‘An Orthogonal Collocation Method for Static and Dynamic
Cosserat Rods’. In: International Conference on Intelligent Robots and Systems (IROS). Detroit,
United States, 1st Oct. 2023. DOI: 10.1109/IROS55552.2023.10341631. URL: https://hal.sci
ence/hal-04246775.

[20] N. Khaledian, P.-F. Villard, P. E. Hammer, D. P. Perrin and M.-O. Berger. ‘Influence of Anisotropy
on Fluid-Structure Interaction Simulations of Image-Based and Generic Mitral Valves’. In: LNCS
FIMH 2023: Functional Imaging and Modeling of the Heart. The 12th International Conference On
Functional Imaging And Modeling Of The Heart. Vol. 13958. Lyon, France: Springer, June 2023,
pp. 455–464. DOI: 10.1007/978-3-031-35302-4_47. URL: https://hal.science/hal-04117
508.

Other scientific publications

[21] Y. Assis, L. Liao, F. Pierre, R. Anxionnat and E. Kerrien. ‘Intracranial Aneurysm Detection using
Spherical Representations’. In: Colloque Français d’Intelligence Artificielle en Imagerie Biomédicale.
Paris, France, 30th Mar. 2023. URL: https://hal.univ-lorraine.fr/hal-04206825.

20

Inria Annual Report 2023

10.3 Cited publications

[22] M. Baumgartner, P. F. Jäger, F. Isensee and K. H. Maier-Hein. ‘nnDetection: a self-configuring

method for medical object detection’. In: Medical Image Computing and Computer Assisted Intervention–
MICCAI 2021: 24th International Conference, Strasbourg, France, September 27–October 1, 2021,
Proceedings, Part V 24. Springer. 2021, pp. 530–539.

[23] D. S. Wokes and P. L. Palmer. ‘Perspective Reconstruction of a Spheroid from an Image Plane Ellipse’.

In: International Journal of Computer Vision 90.3 (2010), pp. 369–379.
<scratchpad>
- Main topic: Research Centre TANGRAM, focusing on visual registration, perception, cognition, and interaction.
- Key points: Visual registration with physically coherent models, machine learning methods for estimation problems, applications in mixed reality, photomechanics, and minimally invasive medical interventions.
- Structure: The document is divided into sections, including activity report, project-team, domain, theme, contents, new software, new results, and dissemination.
- Potential artifacts to ignore: Links, HTML tags, and web-related elements.
</scratchpad>

<final_summary>
The TANGRAM research centre focuses on visual registration, perception, cognition, and interaction, with a specific emphasis on visual registration with physically coherent models. The team aims to integrate machine learning methods into traditional approaches to improve estimation problems in various application domains, including mixed reality, photomechanics, and minimally invasive medical interventions. Key research areas include visual localization, vanishing point computation, and handling non-rigid deformation, with a strong focus on developing physically coherent models and machine learning methods for estimation problems. The team's work has led to the development of new software, platforms, and open data, and has been recognized through various awards and publications.
The TANGRAM research centre focuses on visual registration, perception, cognition, and interaction, with a specific emphasis on visual registration with physically coherent models. The team aims to integrate machine learning methods into traditional approaches to improve estimation problems in various application domains, including mixed reality, photomechanics, and minimally invasive medical interventions. Key research areas include visual localization, vanishing point computation, and handling non-rigid deformation, with a strong focus on developing physically coherent models and machine learning methods for estimation problems. The team's work has led to the development of new software, platforms, and open data, and has been recognized through various awards and publications.