- A3.4. Machine learning and statistics
- A3.4.1. Supervised learning
- A3.4.2. Unsupervised learning
- A3.4.3. Reinforcement learning
- A3.4.4. Optimization and learning
- A3.4.5. Bayesian methods
- A3.4.6. Neural networks
- A3.4.8. Deep learning
- A6.2. Scientific computing, Numerical Analysis & Optimization
- A6.2.4. Statistical methods
- A6.2.6. Optimization
- A8.2. Optimization
- A8.7. Graph theory
- A9.2. Machine learning
- A9.3. Signal analysis
- A9.7. AI algorithmics
- B1. Life sciences
- B1.1. Biology
- B1.2. Neuroscience and cognitive science
- B2.6. Biological and medical imaging
1 Team members, visitors, external collaborators
- Emilie Chouzenoux [Inria, Researcher, HDR]
- Jean-Christophe Pesquet [Team leader, CentraleSupélec, Professor, HDR]
- Marc Castella [IMT Atlantique, Associate Professor]
- Fragkiskos Malliaros [Centrale-Supélec, Associate Professor]
- Nora Ouzir [Centrale-Supélec, Associate Professor, from Oct 2020]
- Hugues Talbot [Centrale-Supélec, Professor, HDR]
- Maria Vakalopoulou [Centrale-Supélec, Associate Professor]
- Andres Antonio Contreras Tavarez [Inria, from Jul 2020]
- Theodore Aouad [Centrale-Supélec, from Oct 2020]
- Enzo Battistella [Centrale-Supélec]
- Younes Belkouchi [Centrale-Supélec, from Oct 2020]
- Abdulkadir Celikkanat [Centrale-Supélec]
- Tasnim Dardouri [Univ Paris-Nord]
- Theo Estienne [Centrale-Supélec]
- Jean Baptiste Fest [Inria, from Jun 2020]
- Mouna Gharbi [Centrale-Supélec, from Sep 2020]
- Thomas Guilmeau [Inria, from Dec 2020]
- Kavya Gupta [Thales, CIFRE]
- Yunshi Huang [Univ Paris-Val de Marne]
- Surabhi Jagtap [IFPEN]
- Othmane Laousy [Centrale-Supélec, from Mar 2020]
- Marvin Lerousseau [Centrale-Supélec]
- Arthur Marmin [Centrale-Supélec, until Nov 2020]
- Segolene Martin [Centrale-Supélec]
- Leo Milecki [Centrale-Supélec, from Mar 2020]
- Ana Neacsu [Centrale-Supélec]
- Maria Papadomanolaki [Centrale-Supélec]
- Mihir Sahasrabudhe [CentraleSupélec, until Jan 2020]
- Marion Savanier [Centrale-Supélec]
- Gabriele Scrivanti [Centrale-Supélec, from Nov 2020]
- Maissa Sghaier [Centrale-Supélec]
- Roger Sun [CentraleSupélec, until Sep 2020]
- Matthieu Terris [Université Heriot-Watt Edimbourg - Ecosse]
- Sagar Verma [Centrale-Supélec]
- Mario Viti [Centrale-Supélec, CIFRE]
- Nguyen Vu [Inria, Engineer, From Dec 2019 to Jan 2021]
- Peng Wang [Centrale-Supélec, Engineer, from Oct 2020 to Dec 2020]
- Paul Zheng [Inria, Engineer, from Jul 2020 until Dec 2020]
Interns and Apprentices
- Victor Bennini [Centrale-Supélec, from Nov 2020]
- Remi Castera [Centrale-Supélec, from May 2020 until Aug 2020]
- Mathieu Chalvidal [Centrale-Supélec, until Mar 2020]
- Benoit Charmettant [Inria, until Feb 2020]
- Alexandre Duval [Centrale-Supélec, from Jun 2020 until Nov 2020]
- Sidi Mohamed El Kharchi [Centrale-Supélec, from Nov 2020]
- Elie Leroy [Centrale-Supélec, from Apr 2020 until Oct 2020]
- Mengda Li [Inria, from Apr 2020 until Jun 2020]
- Rebeca Predescu [Centrale-Supélec, from Feb 2020 until Mar 2020]
- Jana Dutrey [Centrale-Supélec]
- Irina Lahaye [Inria]
2 Overall objectives
Mathematical optimization is the key to solving many problems in Science, based on the observation that physical systems obey a general principle of least action. While some problems can be solved analytically, many more can only be solved via numerical algorithms. Research in this domain has been steadily ongoing for decades.
In addition, many fields such as medecine continue to benefit from considerable improvements in data acquisition technology, based on sophisticated tools from optics and physics (e.g., new laser sources in microscopy, multi-coil systems in MRI, novel X-ray schemes in mammography, etc). This evolution is expected to yield significant improvements in terms of data resolution, making the interpretation and analysis of the results easier and more accurate for the practitioner. The large amounts of generated data must be analyzed by sophisticated optimization tools so that, in recent years, optimization has become a main driving force fostering significant advances in data processing. Previously hidden or hard to extract information can be pried from massive datasets by modern recovery and data mining methods. At the same time, automated decision and computer-aided diagnoses are made possible through optimal learning approaches.
However, major bottlenecks still exist. Recent advances in instrumentation techniques come with the need to minimize functions involving increasingly large number of variables (at least one billion variables in 3D digital tomography modality), and with increasingly complex mathematical structure. The computational load for solving these problems may be too high for even state-of-the-art algorithms. New algorithms must be designed with computational scalability, robustness, and versatility in mind. In particular, the following severe requirements must be fulfilled: i) ability to tackle high-dimensional problems in a reasonable computation time; ii) low-requirements in terms of memory usage; iii) robustness to incomplete or unreliable information; iv) adaptivity to statistically varying environments; v) resilience to latency issues arising in architectures involving multiple computing units.
These difficulties are compounded in the medical and biomedical areas. In these contexts, datasets are not easily available due to patient confidentiality and/or instrument limitations. Moreover, high-level expertise is necessary to interpret the data which can be of very high dimension. Finally, the developed analysis methods must be reliable and interpretable by the medical/biomedical community.
The objective of the OPIS project is to design advanced optimization methods for the analysis and processing of large and complex data. Applications to inverse problems and machine learning tasks in biomedical imaging will be major outcomes of this research project. We will seek optimization methods able to tackle data with both a large sample-size (“big " e.g., ) and/or many measurements (“big " e.g., ). The methodologies to be explored will be grounded on nonsmooth functional analysis, fixed point theory, parallel/distributed strategies, and neural networks. The new optimization tools that will be developed will be set in the general framework of graph signal processing, encompassing both regular graphs (e.g., images) and non-regular graphs (e.g., gene regulatory networks).
More specifically, three main research avenues will be explored, namely:
- proposing novel algorithms able to encompass high-dimensional continuous optimization problems, with established convergence guarantees, and that are well-suited to parallel implementation;
- designing efficient optimization approaches for the resolution of graph signal processing and graph mining problems;
- developing a new generation of deep learning strategies, characterized by robustness guarantees, fast training and suitable account for prior information.
Our research program is detailed in the next subsections. We also indicate applications in the medical and biomedical areas on which our program is expected to have a significant impact.
3 Research program
3.1 Accelerated algorithms for solving high-dimensional continuous optimization problems
Variational problems requiring the estimation of a huge number of variables have now to be tackled, especially in the field of 3D reconstruction/restoration (e.g. variables in 3D imaging). In addition to the curse of dimensionality, another difficulty to overcome is that the cost function usually reads as the sum of several loss/regularization terms, possibly composed with large-size linear operators. These terms can be nonsmooth and/or nonconvex, as they may serve to promote the sparsity of the sought solution in some suitable representation (e.g. a frame) or to fufill some physical constraints. In such a challenging context, there is a strong need for developing fast parallelized optimization agorithms for which sound theoretical guarantees of convergence can be established. We explore deterministic and stochastic approaches based on proximal tools, MM (Majorization-Minimization) strategies, and trust region methods. Because of the versatility of the methods that will be proposed, a wide range of applications in image recovery are considered: parallel MRI, breast tomosynthesis, 3D ultrasound imaging, and two-photon microscopy. For example, in breast tomosynthesis (collaboration with GE Healthcare), 3D breast images have to be reconstructed from a small number of X-ray projections with limited view angles. Our objective is to facilitate the clinical task by developing advanced reconstruction methods allowing micro-calcifications to be highlighted. In two-photon microscopy (collaboration with XLIM), our objective is to provide effective numerical solutions to improve the 3D resolution of the microscope, especially when cheap laser sources are used, with applications to muscle disease screening.
3.2 Optimization over graphs
Graphs and hypergraphs are rich data structures for capturing complex, possibly irregular, dependencies in multidimensional data. Coupled with Markov models, they constitute the backbones of many techniques used in computer vision. Optimization is omnipresent in graph processing. Firstly, it allows the structure of the underlying graph to be inferred from the observed data, when the former is hidden. Second, it permits to develop graphical models based on the prior definition of a meaningful cost function. This leads to powerful nonlinear estimates of variables corresponding to unknown weights on the vertices and/or the edges of the graph. Tasks such as partitioning the graph into subgraphs corresponding to different clusters (e.g., communities in social networks) or graph matching, can effectively be performed within this framework. Finally, graphs by themselves offer flexible structures for formulating and solving optimization problems in an efficient distributed manner. On all these topics, our group has acquired a long-term expertise 3 that we plan to further strengthen. In terms of applications, novel graph mining methods are proposed for gene regulatory and brain network analysis. For example, we plan to develop sophisticated methods for better understanding the gene regulatory network of various microscopic fungi, in order to improve the efficiency of the production of bio-fuels (collaboration with IFP Energies Nouvelles).
3.3 Toward more understandable deep learning
Nowadays, deep learning techniques efficiently solve supervised tasks in classification or regression by utilizing large amounts of labeled data and the powerful high level features that they learn by using the input data. Their good performance has caught the attention of the optimization community since currently these methods offer virtually no guarantee of convergence, stability or generalization. Deep neural networks are optimized through a computationally intensive engineering process via methods based on stochastic gradient descent. These methods are slow and they may not lead to relevant local minima. Thus, more efforts must be dedicated in order to improve the training of deep neural networks by proposing better optimization algorithms applicable to large-scale datasets. Beyond optimization, incorporating some structure in deep neural networks permits more advanced regularization than the current methods. This should reduce their complexity, as well as allow us to derive some bounds regarding generalization. For example, many signal processing models (e.g. those based on multiscale decompositions) exhibit some strong correspondence with deep learning architectures, yet they do not require as many parameters. One can thus think of introducing some supervision into these models in order to improve their performance on standard benchmarks. A better mathematical understanding of these methods permits to improve them, but also to propose some new models and representations for high-dimensional data. This is particularly interesting in settings such as the diagnosis or prevention of diseases from medical images, because they correspond to critical applications where the made decision is crucial and needs to be interpretable. One of the main applications of this work is to propose robust models for the prediction of the outcome of cancer immunotherapy treatments from multiple and complementary sources of information: images, gene expression data, patient profile, etc (collaboration with Institut Gustave Roussy).
4 Application domains
4.1 Artificial intelligence fighting against COVID-19
Participants: Guillaume Chassagnon, Emilie Chouzenoux, Kavya Gupta, Yingping Li, Ana Neascu, Jean-Christophe Pesquet, Hugues Talbot, Matthieu Terris, Maria Vakalopoulou, Sagar Verma (Collaboration: Institut Gustave Roussy ; Kremlin-Bicetre APHP ; Owkin France ; Hopital Cochin Broca Hotel Dieu ; Therapanacea ; IIIT Delhi)
Coronavirus disease 2019 (COVID-19) emerged in 2019 and disseminated around the world rapidly. Computed tomography (CT) imaging has been proven to be an important tool for screening, disease quantification and staging. The latter is of extreme importance for organizational anticipation (availability of intensive care unit beds, patient management planning) as well as to accelerate drug development through rapid, reproducible and quantified assessment of treatment response. Even if currently there are no specific guidelines for the staging of the patients, CT together with some clinical and biological biomarkers are used. Through two collaborative projects with medical partners and French start-ups, we aim in 10, 22 to investigate and propose artificial intelligence tools that could help on the clinical practice.
Our work in 77, 76, in collaboration with IIIT Delhi, builds computational models for drug re-positioning for COVID-19 treatment. The aim is to assist clinicians with a tool for selecting prospective antiviral treatments. The main contributions are (i) a manually curated database publicly shared, comprising of existing associations between viruses and their corresponding antivirals along with a set of state-of-the-art computational drug re-positioning tools based on matrix completion, (ii) a novel multiple graph regularized matrix completion approach with assessed performance on antiviral repositioning.
4.2 Design of robust neural networks in safety critical industrial domains
Participants: Jean-Christophe Pesquet (Collaboration: P.L. Combettes, North Carolina State University; Audrey Repetti, Yves Wiaux, Heriot Watt University)
One of the main challenges faced today by companies like Thales or Schneider Electric designing advanced industrial systems, is to ensure the safety of new generations of products based on the use of neural networks. Since 2013, neural networks have been shown to be sensitive to adversarial perturbations. Deep neural networks can thus be fooled, in an intentional (security issue) or in undeliberate manner (safety issue), which raises a major robustness concern for safety-critical systems which need to be certified by an independent certification authority prior to any entry into production/operation. Techniques based on mathematical proofs of robustness are generally preferred by industrial safety experts since they enable a safe-by-design approach that is more efficient than a robustness verification activity done a posteriori with a necessarily bounded effort. Among the possible mathematical approaches, we focus on those relying upon the analysis of the Lipschitz properties of neural networks 78, 2, 62. Such properties play a fundamental role in the understanding of the internal mechanisms governing these complex nonlinear systems. Besides, they make few assumptions on the type of non-linearities used and are thus valid for a wide range of networks.
4.3 Radiology, Hyper-Progressive Disease and immunotherapy
In recent years, novel treatments for a variety of cancers have emerged, as a “fourth way”, beyond the well-established chemotherapy, surgery and radiotherapy. These treatments have been in development for decades under the heading of “immunotherapy” but have finally met with considerable success.
One defining feature of cancer cells is that they are considered as host cells by the patient's immune system, even when they are malignant, and so they are allowed to grow unchecked, whereas normally abnormal mutant cells are recognized and destroyed by the host.
Successful immunotherapy has focused on programmed cell death, also known as apoptosis. These approaches against some forms of cancer aim to block the ability of tumor cells to resist patient immune response by acting on the checkpoints of immune cells. These drugs are able to either block the PD-1 receptor present on the surface of the lymphocytes, or the PD-L1 and PD-L2 ligands expressed by the cancer cells. This can prevent the binding of both, blocking the immunomodulatory signal and allowing the killer T cells continue to be active against the tumor.
However, no therapy is yet universal. One aspect of these approaches is that some patient will respond (and their tumor will therefore not “progress”) and some not. “Progressor” patient, in oncology, are those who continue to see their tumor grow. It is important to use all available data as early as possible in the treatment to decide quickly if the patient is responding positively. One of our projects in this sections aims at doing this.
4.3.1 Imaging radiomics and pathomics to assess response to treatment
Roger Sun, Théo Estienne, Enzo Batistella, Marvin Lerousseau, Maria Vakalopoulou, Mihir Sahasrabudhe, Hugues Talbot (Collaboration: E. Deutsch and Nikos Paragios, Institut Gustave Roussy, Therapanacea)
The response of patients with cancer to immunotherapy can vary considerably (see section 4.3.2), innovative predictors of response to treatment are needed to improve treatment outcomes. We aimed to develop and independently validate radiomics-based biomarkers of tumour-infiltrating cells in patients included in trials of the two most common, recent immunotherapy treatments: anti-programmed cell death protein (PD)-1 or anti-programmed cell death ligand 1 (PD-L1) monotherapy. We also aimed to evaluate the association between the biomarker, and tumour immune phenotype and clinical outcomes of these patients.
However, sometimes, not only do patient respond poorly, but immunotherapy seems to make things worse. Some patients see they tumoral load increase significantly faster after immunotherapy is started. These patients are called “hyper-progressors”. One of our project has been to clearly define and detect this class of patients. This is so novel that the very notion of hyperprogressive patient was still controversial when our work was published, but is accepted now.
4.3.2 Hyperprogressive Disease and immunotherapy
Baptiste Kas, Hugues Talbot (Collaboration: Institut Gustave Roussy)
Hyperprogressive disease (HPD) is an aggressive pattern of progression reported for patients treated with programmed cell death 1 (PD-1) / programmed cell death 1 ligand (PD-L1) inhibitors as a single agent in several studies. However, the use of different definitions of HPD introduces the risk of describing different tumoral behaviors. In this paper 21, we assess the accuracy of each HPD definition to identify the frequency of HPD and the association with poorer outcomes of immune-checkpoint inhibitor (ICI) treatment in patients with advanced non-small cell lung cancer (NSCLC) and to provide an optimized and homogenized definition based on all previous criteria for identifying HPD. The findings of this retrospective cohort study of patients with NSCLC suggest that the previous five definitions of HPD were not associated with the same tumor behavior. A new definition, based on a more physical model of tumor growth rate, appeared to be associated with the characteristics expected with HPD (increase of the tumor kinetics and poor survival).
More generally, it is being increasingly recognized that cancer is such a diverse and complex disease that it can only be tackled successfully by recognizing the specificity of every patient. A new institute, PRISM, has been launched in collaboration between IGR and CS and involving among others Inria and Inserm to deal with the complexity of each patient genetic and epigenetic makeup using machine learning, medical and biomedical imaging.
4.3.3 Vision, machine learning and precision medicine
Younes Belkouchi, Hugues Talbot (Collaboration: Institut Gustave Roussy)
In March 2020, the PRISM institute of Gustave-Roussy was launched. The aim of this project, funded for 5 years, is to develop targeted treatments that are more likely to work on specific patients.
The mission of this “second-generation” precision medicine centre will be to model cancer on an individual scale by creating numerical avatars of tumours. The aim is to identify patients with the most aggressive cancers very early in the disease, without waiting for relapses, in order to offer them the most appropriate treatment from the start of treatment, using the huge volume of clinical, biological and molecular data and their analysis by artificial intelligence. PRISM will conduct large-scale clinical studies and develop molecular analysis technologies and data analysis methods.
Coordinated by Professor Fabrice André, Research Director of Gustave Roussy, Inserm Research Director and Professor at Paris-Saclay University, Prism aims to revolutionize the understanding of the molecular and biological mechanisms of cancer development and progression through artificial intelligence. Based on increasingly rich data of various types (clinical, genomic, microbiological, imaging, etc.), learning algorithms make it possible to develop finer diagnostic and prognostic tools, and thus to propose therapies that are personalised according to the characteristics of the individual.
Funded by the French National Research Agency, PRISM received the IHU label in 2018, followed by the National Center for Precision Medicine label.
4.4 Sparse signal processing in chemistry
Participants: Marc Castella, Emilie Chouzenoux, Arthur Marmin, Jean-Christophe Pesquet (Collaboration: Laurent Duval, IFPEN, Rueil Malmaison)
Mass Spectrometry (MS) is a powerful tool used for robust, accurate, and sensitive detection and quantification of molecules of interest. Thanks to its sensibility and selectivity, MS is widely used in proteomics such anti-doping, metabolomics, medicine or structural biology. In particular, it has applications in clinical research, personalized medicine, diagnosis process and tumours profiling and pharmaceutical quality control. In an MS experiment, the raw signal arising from the molecule ionization in an ion beam is measured as a function of time via Fourier Transform-based measures such as Ion Cyclotron Resonance (FT-ICR) and Orbitrap. A spectral analysis step is then performed to improve the quality of data. The goal is then to determine from this observed pattern distribution the most probable chemical composition of the sample, through the determination of the monoisotopic mass, charge state and abundance of each present molecule. This amounts to solve a large scale signal estimation problem under specific sparsity constraints 13. Collaboration with Dr. L. Duval, Research Engineer at IFP Energies Nouvelles, France is on-going in this applicative context.
4.5 Image restoration for two-photon microscopy
Participants: Emilie Chouzenoux, Jean-Christophe Pesquet, Mathieu Chalvidal (Collaboration: Claire Lefort, XLIM, CNRS, Limoges)
Through an ongoing collaboration with physicists from XLIM laboratory (CNRS, Limoges, France), we propose advanced mathematical and computational solutions for multiphoton microscopy (MPM) 3D image restoration. This modality enjoys many benefits such as a decrease in phototoxicity and increase in penetration depth. However, blur and noise issues can be more severe than with standard confocal images. Our objective is to drastically improve the quality of the generated images and their resolution by improving the characterization of the PSF of the system 45 and compensating its effect. We consider the application of the improved MPM imaging tool to the microscopic analysis of muscle ultrastructure and composition, with the aim to help diagnosing muscle disorders including rare and orphan muscle pathologies.
4.6 Representation learning for biological networks
Participants: Fragkiskos Malliaros, Abdulkadir Çelikkanat (Collaboration: Duong Nguyen, UC San Diego)
Networks (or graphs) are ubiquitous in the domain of biology, as many biological systems can naturally be mapped to graph structures. Characteristic examples include protein-protein interaction and gene regulatory networks. To this extend, machine learning on graphs is an important task with many practical applications in network biology. For example, in the case on protein-protein interaction networks, predicting the function of a protein is a key task that assigns biochemical roles to proteins. The main challenge here is to find appropriate representations of the graph structure, in order to be easily exploited by machine learning models. The traditional approach to the problem was relying on the extraction of “hand-crafted" discriminating features that encode information about the graph, based on user-defined heuristics. Nevertheless, this approach has demonstrated severe limitations, as the learning process heavily depends on the manually extracted features. To this end, feature (or representation) learning techniques can be used to automatically learn to encode the graph structure into low-dimensional feature vectors – which can later be used in learning tasks. Our goal here is to develop a systematic framework for large-scale representation learning on biological graphs. Our approach takes advantage of the clustering structure of these networks, to further enhance the ability of the learned features to capture intrinsic structural properties.
4.7 Reconstruction approaches in medical imaging
Participants: Emilie Chouzenoux, Jean-Christophe Pesquet, Marion Savanier, Maissa Sghaier (collaboration S. Muller, C. Riddel, Y. Trousset, GE Healthcare)
The objective of these works, in collaboration with researchers from GE Healthcare, is to develop high quality reconstruction methodologies, for two modalities of medical imaging, namely digital breast tomosynthesis (DBT) and CT for interventional surgery.
The reconstruction of a volumetric image from DBT measurements is an ill-posed inverse problem, for which existing iterative regularized approaches can provide a good solution. However, the clinical task is somehow omitted in the derivation of those techniques, although it plays a primary role in the radiologist diagnosis. In this work, we address this issue by introducing a novel variational formulation for DBT reconstruction, tailored for a specific clinical task, namely the detection of microcalcifications. Our methods 80, 60 aim at simultaneously enhancing the detectability performance and enabling a high-quality restoration of the background breast tissues.
Discretizing and implementing tomographic forward and backward operations is a crucial step in the design of model-based iterative reconstruction algorithms in CT. The mathematical constraint of symmetry on the projector/backprojector pair prevents linear interpolation, which is a standard method in analytical reconstruction, to be used. Consequently, it often happens that these operators are approximated numerically, so that the adjoint property is no longer fulfilled. In the works 15, 59, we focus on the proximal gradient algorithm stability properties when such an adjoint mismatch arises. By making use of tools from convex analysis and fixed point theory, we establish conditions under which the algorithm can still converge to a fixed point. We provide bounds on the error between this point and the solution to the minimization problem. We illustrate the applicability of our theoretical results through numerical examples in the context of CT.
4.8 Inference of gene regulatory networks
Participants: Surabhi Jagtap, Fragkiskos Malliaros, Jean-Christophe Pesquet (collaboration A. Pirayre and L. Duval, IFPEN)
The discovery of novel gene regulatory processes improves the understanding of cell phenotypic responses to external stimuli for many biological applications, such as medicine, environment or biotechnologies. To this purpose, transcriptomic data are generated and analyzed from DNA microarrays or more recently RNAseq experiments. They consist in genetic expression level sequences obtained for all genes of a studied organism placed in dierent living conditions. From these data, gene regulation mechanisms can be recovered by revealing topological links encoded in graphs. In regulatory graphs, nodes correspond to genes. A link between two nodes is identified if a regulation relationship exists between the two corresponding genes. In our work, we propose to address this network inference problem with recently developed techniques pertaining to graph optimization. Given all the pairwise gene regulation information available, we propose to determine the presence of edges in the considered GRN by adopting an energy optimization formulation integrating additional constraints. Either biological (information about gene interactions) or structural (information about node connectivity) a priori are considered to restrict the space of possible solutions. Different priors lead to different properties of the global cost function, for which various optimization strategies, either discrete and continuous, can be applied.
4.9 Imaging biomarkers and characterization for chronic lung diseases
Participants: Guillaume Chassagnon, Maria Vakalopoulou (in collaboration with Marie-Pierre Revel and Nikos Paragios: AP-HP - Hopital Cochin Broca Hotel Dieu; Therapanacea)
Diagnosis and staging of chronic lung diseases is a major challenge for both patient care and approval of new treatments. Among imaging techniques, computed tomography (CT) is the gold standard for in vivo morphological assessment of lung parenchyma currently offering the highest spatial resolution in chronic lung diseases. Although CT is widely used its optimal use in clinical practice and as an endpoint in clinical trials remains controversial. Our goal is to develop quantitative imaging biomarkers that allow (i)severity assessment (based on the correlation to functional and clinical data) and (ii) monitoring the disease progression. In the current analysis we focus on scleroderma and cystic fibrosis as models for restrictive and obstructive lung disease, respectively. Two different approaches are investigated: disease assessment by deep convolutional neural networks and assessment of the regional lung elasticity through deformable registration. This work is in collaboration with the Department of Radiology, Cochin Hospital, Paris.
4.10 A generative model for heart left ventricle perfusion analysis
Participant: Hugues Talbot (Collaboration: L. Najman ESIEE Paris ; I. Vignon-Clementel, REO Team leader, Inria ; C. Taylor, Heartflow Inc.)
Cardio-vascular diseases continue to be the leading cause of mortality in the world. Understanding these diseases is a current, challenging and essential research project. The leading cause of heart malfunction are stenoses causing ischemia in the coronary vessels. Current CT and MRI technology can assess coronary diseases but are typically invasive, requiring risky catheterization and renal-toxic contrast agents injection. In collaboration with the REO team headed by Irène Vignon-Clementel, and Heartflow, a US based company, we have in the past contributed to Heartflow's major product, that replaces these physical exams with image-based exams only, limiting the use of contrast agents and in the cases that do not require a stent insertion, eliminating catheterisation. Heartflow is current the market leader in non-invasive coronary exams and the owner of most of the relevant IP in this domain.
Unfortunately, current imaging technology is unable to assess coronary disease along the full length of coronary vessels. CT is limited to a resolution of about 1mm, whereas coronary vessels can be much smaller, down to about 10 micrometers in diameter. To assess blood vessel health down to the smallest sizes, blood perfusion imaging technique throughout the heart muscle must be used instead. Perfusion imaging with PET or a Gamma camera, the current gold standard, is an invasive technology requiring the use of radioactive tracers. To avoid using these, a lower quality estimate of perfusion can be achieved using some ToF or injected gated MRI modalities.
We have investigated patient-specific vessel generation models together with porous model simulations in order to propose a direct model of perfusion imaging, based on the known patient data, computer flow dynamic simulations as well as experimental data consistent with known vessel and heart muscle physiology. The objective of this work is to both to provide a useful, complex forward model of perfusion image generation, and to solve the inverse problem of locating and assessing coronary diseases given a perfusion exam, even though the affected vessels may be too small to be imaged directly.
Continuing on our work from the period 2015-2019, this year we proposed a functional myocardial perfusion model consisting of the CT-derived segmented coronary vessels, a simulated vessel tree consisting of several thousands of terminal vessels, filling the myocardium in a patient-specific way, consistent with physiology data, physics-based and empirically-observed vessel growth rules, and a porous medium. We produced and validated a CFD code capable of simulating blood flow in all three coupled compartments, which allows us to simulate perfusion realistically.
This work was published in 33.
5 Social and environmental responsibility
5.1 Footprint of research activities
The research carried out in OPIS aims at developing advanced techniques in the domain of data science for precision medicine. One of the main features of this research is to ensure that the proposed methods are not only efficient, but also grounded on sound mathematical foundations inherited from the areas of optimization and fixed point algorithms. In the biomedical domain, it appears indeed mandatory to guarantee the reliability and the explainability of the proposed approaches in their use by medical doctors or producers of medical imaging devices.
5.2 Impact of research results
OPIS participates in the design of innovative products developed by big companies working in the domain of medical imaging (GE Healthcare and Essilor) and several startups. Various application fields are targeted (breast cancer detection, surgical radiology, interventional surgery, coronary disease monitoring, vision correction, ...). In addition, OPIS has active collaborations with several hospitals, particularly Institut Gustave Roussy and hospitals from APHP in Paris. The purpose of these collaborations is to develop artificial intelligence tools aiding medical doctors in their practice. A large part of this research activity is oriented toward fighting against cancer using different kinds of data (CT scans, MRI, genomic data, microscopy images,...) OPIS is also currently involved in several projects for helping to better diagnose and cure COVID-19 infection.
6 Highlights of the year
- J.-C Pesquet was awarded a Teaching Research Chair in Artificial Intelligence by the Agence Nationale de Recherche.
- The article on “Influence Maximization using Influence and Susceptibility Embeddings” co-authored by G. Panagopoulos, F. Malliaros, and M. Vazirgiannis, received the best paper honorable mention award at the AAAI International Conference on Web and Social Media (ICWSM), 2020. The acceptance rate was .
- T. Estienne took the 3rd place on the Learn2Reg competition 2020 of MICCAI conference.
- F. Malliaros received the Young Researcher Grant (JCJC) by the French National Research Agency (ANR).
- Our M.Sc. program in Data Sciences and Business Analytics (with ESSEC Business School) was ranked 3rd worldwide in the QS World University Rankings.
7 New software and platforms
7.1 New platforms
7.1.1 The Proximity Operator Repository
Participants: Emilie Chouzenoux and Jean-Christophe Pesquet (in collaboration with Giovanni Chierchia, Univ. Paris Est, and Patrick Combettes, North Carolina State University).
Proximity operators have become increasingly important tools as basic building blocks of proximal splitting algorithms, a class of algorithms that decompose complex composite convex optimization methods into simple steps involving one of the functions present in the model. This website provides formulas for efficiently computing the proximity operator of various functions, along with the associated codes.
Participants: Emilie Chouzenoux and Jean-Christophe Pesquet (in collaboration with Claire Lefort, XLIM, Limoges).
The plugin FIGARO, for ImageJ software, helps tracking resolution of microscope systems by extracting dimensions and orientation of standardized microbeads images, acquired from test samples. In the development of optical microscopes for biomedical imaging, the evaluation of resolution is a fundamental parameter achieved by Point Spread Function (PSF) measurements. Sometimes, PSF measurement procedure is not easy or impossible in case of microspheres images presenting a high noise level. The current method proposed into the plugin FIGARO is based on a variational approach for PSF modeling through multivariate Gaussian fitting, adapted to images acquired in a high noise context 45.
7.1.3 BiasedWalk: Learning latent node features with biased random walks
Participants: Fragkiskos Malliaros and Duong Nguyen (UC San Diego)
The BiasedWalk network representation learning algorithm, computes latent node features on graphs based on biased random walks. The framework has been implemented in Python and has been built upon widely used modules, including networkx, numpy, gensim and scikit-learn.
7.1.4 DiffuGreedy: Influence maximization in complex networks based on diffusion cascades
Participants: Fragkiskos Malliaros, Georgios Panagopoulos and Michalis Vazirgiannis (École Polytechnique)
The DiffuGreedy is an algorithm for influence maximization in complex networks, that is based on diffusion cascades. It has been implemented in Python and has been built upon widely used modules, including networkx, igraph, numpy and pandas.
7.1.5 Graph-based text categorization
Participants: Fragkiskos Malliaros, Konstantinos Skianis and Michalis Vazirgiannis (École Polytechnique)
Graph-based TC is a framework for text categorization that relies on a graph representation of documents. The framework uses various graph centrality criteria to determine the importance of a term within a document. It also makes use of word embeddings to further boost the performance of graph-based methods. It has been implemented in Python and has been built upon widely used modules, including networkx, igraph, numpy and scikit-learn.
7.1.6 KernelNE - Topical Node Embeddings
Participants: Abdulkadir Çelikkanat, Fragkiskos Malliaros
KernelNE learns node representations on graphs based on a weighted matrix factorization model which encodes random walk-based information about the nodes.
7.1.7 EFGE - Exponential Family Graph Embeddings
Participants: Abdulkadir Çelikkanat, Fragkiskos Malliaros
EFGE learns node embeddings generalizing random walk-based network representation learning techniques to exponential family conditional distributions.
7.1.8 Semi-supervised Fake News Detection
Participants: Adrien Benamira, Benjamin Devillers, Etienne Lesot, Ayush K. Ray, Manal Saadi, and Fragkiskos Malliaros
We propose a graph-based semi-supervised fake news detection method, based on graph neural networks.
7.1.9 IMINFECTOR: Multi-task Learning for Influence Estimation and Maximization
Fragkiskos Malliaros, George Panagopoulos, and Michalis Vazirgiannis
IMINFECTOR (Influence Maximization with INFluencer vECTORs) is a unified approach that leverages representations learned from diffusion cascades to perform model-independent influence maximization that scales in real-world networks 32, 57.
7.1.10 The PINK image library
Participant: Hugues Talbot
The PINK image library is a general-purpose, open-source, portable image processing library specializing in discrete geometry and mathematical morphology. It is the result of several decades of research in these domains and features state-of-the art algorithmic implementation of both classical and leading edge DG and MM operators. These include nD parallel thinning and skeletonization methods and efficient hierarchical morphological transforms.
7.1.11 The Vivabrain AngioTK toolkit
Participant: Hugues Talbot
AngioTK is a toolkit supported by Kitware (the authors of VTK) for the filtering, segmentation, generation and simulation of blood vessels. It was started in the context of the Vivabrain ANR project in 2012, but continues with the same as well as new partners. Applications are numerous, from the simulation and understanding of perfusion (see associated theme) to the simulation of realistic blood flow MRI images with associated ground truth, via the generation of blood vessel atlases.
7.1.12 The PET/CT FIJI Viewer
Participant: Hugues Talbot
Quantitive Positron Emission Tomography is a new area for which software is not readily availble. With our partners from Toulouse and Beth Israel hospitals, we have proposed a free open-source plugin allowing clinicians to view, interact and perform automated and interactive lesion segmentation in the context of PET/CT.
8 New results
8.1 Severity Prediction of Hospitalized COVID-19 Patients
Participants: Emilie Chouzenoux, Kavya Gupta, Yingping Li, Ana Neascu, Jean-Christophe Pesquet, Hugues Talbot, Matthieu Terris, Sagar Verma (Collaboration: Institut Gustave Roussy ; Kremlin-Bicetre APHP ; Owkin France)
The SARS-COV-2 pandemic has put pressure on intensive care units, so that identifying predictors of disease severity is a priority that we address in the work 22. We collect 58 clinical and biological variables, and chest CT scan data, from 1003 coronavirus-infected patients from two French hospitals. We train a deep learning model based on CT scans to predict severity. We then construct the multimodal AI-severity score that includes 5 clinical and biological variables (age, sex, oxygenation, urea, platelet) in addition to the deep learning model. We show that neural network analysis of CT-scans brings unique prognosis information, although it is correlated with other markers of severity (oxygenation, LDH, and CRP) explaining the measurable but limited 0.03 increase of AUC obtained when adding CT-scan information to clinical variables. Here, we show that when comparing AI-severity with 11 existing severity scores, we find significantly improved prognosis performance; AI-severity can therefore rapidly become a reference scoring approach.
8.2 Influence of the MRI field strength on radiomic features
Participants: Emilie Chouzenoux, Hugues Talbot (Collaboration: Institut Gustave Roussy)
The development and clinical adoption of quantitative imaging biomarkers (radiomics) has established the need for the identification of parameters altering radiomics reproducibility. The aim of the study 5 was to assess the impact of magnetic field strength on magnetic resonance imaging (MRI) radiomics features in neuroradiology clinical practice.
T1 3D SPGR sequence was acquired on two phantoms and 10 healthy volunteers with two clinical MR devices from the same manufacturer using two different magnetic fields (1.5 and 3T). Phantoms varied in terms of gadolinium concentrations and textural heterogeneity. 27 regions of interest were segmented (phantom: 21, volunteers: 6) using the LIFEX software. 34 features were analyzed.
In the phantom dataset, 10 (67%) out of 15 radiomics features were significantly different when measured at 1.5T or 3T (student's t-test, p < 0.05). Gray levels resampling, and pixel size also influence part of texture features. These findings were validated in healthy volunteers. Conclusions: According to daily used protocols for clinical examinations, radiomic features extracted on 1.5T should not be used interchangeably with 3T when evaluating texture features. Such confounding factor should be adjusted when adapting the results of a study to a different platform, or when designing a multicentric trial.
8.3 Artificial Intelligence Data Challenges Based on CT and MRI
Participants: Emilie Chouzenoux (Collaboration: Institut Gustave Roussy)
The second edition of the artificial intelligence (AI) data challenge was organized by the French Society of Radiology with the aim to: (i), work on relevant public health issues; (ii), build large, mul-ticentre, high quality databases; and (iii), include three-dimensional (3D) information and prognostic questions 23. Relevant clinical questions were proposed by French subspecialty colleges of radiology. Their feasibility was assessed by experts in the field of AI. A dedicated platform was set up for inclusion centers to safely upload their anonymized examinations in compliance with general data protection regulation. The quality of the database was checked by experts weekly with annotations performed by radiologists. Multidisciplinary teams competed between September 11 th and October 13 th 2019. Three questions were selected using different imaging and evaluation modalities, including: pulmonary nodule detection and classification from 3D computed tomography (CT), prediction of expanded disability status scale in multiple sclerosis using 3D magnetic resonance imaging (MRI) and segmentation of muscular surface for sarcopenia estimation from two-dimensional CT. A total of 4347 examinations were gathered of which only 6% were excluded. Three independent databases from 24 individual centers were created. A total of 143 participants were split into 20 multidisciplinary teams. Conclusion: Three data challenges with over 1200 general data protection regulation compliant CT or MRI examinations each were organized. Future challenges should be made with more complex situations combining histopathological or genetic information to resemble real life situations faced by radiologists in routine practice.
8.4 SPOQ lp over lq Regularization for Sparse Signal Recovery applied to Mass Spectrometry
Participants: Emilie Chouzenoux, Jean-Christophe Pesquet (Collaboration: Afef Cherni, Université Aix Marseille ; Laurent Duval, IFPEN)
Underdetermined or ill-posed inverse problems require additional information for sound solutions with tractable optimization algorithms. Sparsity yields consequent heuristics to that matter, with numerous applications in signal restoration, image recovery, or machine learning. Since the l0 count measure is barely tractable, many statistical or learning approaches have invested in computable proxies, such as the l1 norm. However, the latter does not exhibit the desirable property of scale invari-ance for sparse data. Generalizing the SOOT Euclidean/Taxicab l1/ l2 norm-ratio initially introduced for blind deconvolution, we propose in 13 SPOQ, a family of smoothed scale-invariant penalty functions. It consists of a Lipschitz-differentiable surrogate for p-over-q quasi-norm/norm ratios with and . This surrogate is embedded into a novel majorize-minimize trust-region approach, generalizing the variable metric forward-backward algorithm. For naturally sparse mass-spectrometry signals, we show that SPOQ significantly outperforms l0, l1, Cauchy, Welsch, and CEL0 penalties on several performance measures. Guidelines on SPOQ hyperparameters tuning are also provided, suggesting simple data-driven choices.
8.5 Deep Convolutional Transform-based Fusion Framework for Supervised and Unsupervised Data Processing
Participants: Emilie Chouzenoux (Collaboration: Giovanni Chierchia, ESIEE Paris ; Pooja Gupta, Jyoti Maggu, Angshul Majumdar, IIIT Delhi, India)
The work 19 proposes an unsupervised fusion framework based on deep convolutional transform learning. The great learning ability of convolutional filters for data analysis is well acknowledged. The success of convolutive features owes to the convolutional neural network (CNN). However, CNN cannot perform learning tasks in an unsupervised fashion. In a recent work, we show that such shortcoming can be addressed by adopting a convolutional transform learning (CTL) approach, where convolutional filters are learnt in an unsupervised fashion. The present paper aims at (i) proposing a deep version of CTL, (ii) proposing an unsupervised fusion formulation taking advantage of the proposed deep CTL representation , and (iii) developing a mathematically sounded optimization strategy for performing the learning task. We apply the proposed technique, named DeConFuse, on the problem of stock forecasting and trading. A comparison with state-of-the-art methods (based on CNN and long short-term memory network) shows the superiority of our method for performing a reliable feature extraction.
In the work 20, we propose a supervised multi-channel time-series learning framework for financial stock trading. Although many deep learning models have recently been proposed in this domain, most of them treat the stock trading time-series data as 2-D image data, whereas its true nature is 1-D time-series data. Since the stock trading systems are multi-channel data, many existing techniques treating them as 1-D time-series data are not suggestive of any technique to effectively fusion the information carried by the multiple channels. To contribute towards both of these shortcomings, we propose an end-to-end supervised learning framework inspired by the previously established (unsupervised) convolution transform learning framework. Our approach consists of processing the data channels through separate 1-D convolution layers, then fusing the outputs with a series of fully-connected layers, and finally applying a softmax classification layer. The peculiarity of our framework, that we call SuperDeConFuse (SDCF), is that we remove the nonlinear activation located between the multi-channel convolution layers and the fully-connected layers, as well as the one located between the latter and the output layer. We compensate for this removal by introducing a suitable regularization on the aforementioned layer outputs and filters during the training phase. Specifically, we apply a logarithm determinant regularization on the layer filters to break symmetry and force diversity in the learnt transforms, whereas we enforce the non-negativity constraint on the layer outputs to mitigate the issue of dead neurons. This results in the effective learning of a richer set of features and filters with respect to a standard convolutional neural network. Numerical experiments confirm that the proposed model yields considerably better results than state-of-the-art deep learning techniques for the real-world problem of stock trading.
8.6 Transformed Subspace Clustering
Participants: Emilie Chouzenoux (Collaboration: Giovanni Chierchia, ESIEE Paris ; Jyoti Maggu, Angshul Majumdar, IIIT Delhi, India)
Subspace clustering assumes that the data is separable into separate subspaces. Such a simple assumption, does not always hold. We assume that, even if the raw data is not separable into subspaces, one can learn a representation (transform coefficients) such that the learnt representation is separable into subspaces. To achieve the intended goal, we embed in the work 27, subspace clustering techniques (locally linear manifold clustering, sparse subspace clustering and low rank representation) into transform learning. The entire formulation is jointly learnt; giving rise to a new class of methods called transformed subspace clustering (TSC). In order to account for non-linearity, kernelized extensions of TSC are also proposed. To test the performance of the proposed techniques, benchmarking is performed on image clustering and document clustering datasets. Comparison with state-of-the-art clustering techniques shows that our formulation improves upon them. We then extend the approach in 26 by embedding a deep transform learning approach. The entire formulation is jointly learnt; giving rise to a new class of methods called deeply transformed subspace clustering (DTSC) with assessed performance on image clustering problems.
8.7 Deep Latent Factor Model for Collaborative Filtering
Participants: Emilie Chouzenoux (Collaboration: Aanchal Mongia, Neha Jhamb, Angshul Majumdar, IIIT Delhi, India)
Latent factor models have been used widely in collaborative filtering based recommender systems. In recent years, deep learning has been successful in solving a wide variety of machine learning problems. Motivated by the success of deep learning, we propose in 31 a deeper version of latent factor model. Experiments on benchmark datasets shows that our proposed technique significantly outperforms all state-of-the-art collaborative filtering techniques.
8.8 Convergence of Proximal Gradient Algorithm in the Presence of Adjoint Mismatch
Participants: Emilie Chouzenoux, Jean-Christophe Pesquet, Marion Savanier (Collaboration: Cyril Riddell, Yves Trousset, GE Healthcare)
We consider in 15 the proximal gradient algorithm for solving penalized least-squares minimization problems arising in data science. This first-order algorithm is attractive due to its flexibility and minimal memory requirements allowing to tackle large-scale minimization problems involving non-smooth penalties. However, for problems such as X-ray computed tomography, the applicability of the algorithm is dominated by the cost of applying the forward linear operator and its adjoint at each iteration. In practice, the adjoint operator is thus often replaced by an alternative operator with the aim to reduce the overall computation burden and potentially improve conditioning issues. In this paper, we propose to analyze the effect of such an adjoint mismatch on the convergence of the proximal gradient algorithm in an infinite-dimensional setting, thus generalizing the existing results on PGA. We derive conditions on the step-size and on the gradient of the smooth part of the objective function under which convergence of the algorithm to a fixed point is guaranteed. We also derive bounds on the error between this point and the solution to the original minimization problem. We illustrate our theoretical findings with two image reconstruction tasks in computed tomography.
8.9 Blind Kalman filtering for time series modeling and inference
Participants: Emilie Chouzenoux (Collaboration: Shalini Sharma, Angshul Majumdar, IIIT Delhi, India; Victor Elvira, University of Edinburgh)
Modeling and inference with multivariate sequences is central in a number of signal processing applications such as acoustics, social network analysis, biomedical, and finance, to name a few. The linear-Gaussian state-space model is a common way to describe a time series through the evolution of a hidden state, with the advantage of presenting a simple inference procedure due to the celebrated Kalman filter.
A fundamental question when analyzing multivariate sequences is the search for relationships between their entries (or the modeled hidden states), especially when the inherent structure is a non-fully connected graph. In such context, graphical modeling combined with parsimony constraints allows to limit the proliferation of parameters and enables a compact data representation which is easier to interpret by the experts. In the work 42, we propose a novel expectation-minimization algorithm for estimating the linear matrix operator in the state equation of a linear-Gaussian state-space model. Lasso regularization is included in the M-step, that we solved using a proximal splitting Douglas-Rachford algorithm. Numerical experiments illustrate the benefits of the proposed model and inference technique, named GraphEM, over competitors relying on Granger causality.
In the work 37, we address the problem of short-term load forecasting. We propose a blind Kalman filter algorithm associated to mini-batch processing strategy to allow on-the-fly forecasting. The experimental results show that the proposed method outperforms the state-of-the-art techniques, both on load profile estimation and peak load forecast problems.
8.10 Majorize-Minimize Adapted Metropolis-Hastings Algorithm
Participants: Emilie Chouzenoux, Jean-Christophe Pesquet (Collaboration: Yosra Marnissi, SAFRAN TECH ; Amel Benazza, Supcom Tunis)
The dimension and the complexity of inference problems have dramatically increased in statistical signal processing. It thus becomes mandatory to design improved proposal schemes in Metropolis-Hastings algorithms, providing large proposal transitions that are accepted with high probability. The proposal density should ideally provide an accurate approximation of the target density with a low computational cost. In the paper 30, the authors derive a novel Metropolis-Hastings proposal, inspired from Langevin dynamics, where the drift term is preconditioned by an adaptive matrix constructed through a Majorization-Minimization strategy. We propose several variants of low-complexity curvature metrics applicable to large scale problems. We demonstrate the geometric ergodicity of the resulting chain for the class of super-exponential distributions. The proposed method is shown to exhibit a good performance in two signal recovery examples.
8.11 A Proximal Interior Point Algorithm with Applications to Image Processing
Participants: Emilie Chouzenoux, Marie-Caroline Corbineau, Jean-Christophe Pesquet
In the work 14, we introduce a new proximal interior point algorithm (PIPA). This algorithm is able to handle convex optimization problems involving various constraints where the objective function is the sum of a Lipschitz differentiable term and a possibly nonsmooth one. Each iteration of PIPA involves the minimization of a merit function evaluated for decaying values of a logarithmic barrier parameter. This inner minimization is performed thanks to a finite number of subiterations of a variable metric forward-backward method employing a line search strategy. The convergence of this latter step as well as the convergence the global method itself are analyzed. The numerical efficiency of the proposed approach is demonstrated in two image processing applications.
8.12 Deep Unfolding of a Proximal Interior Point Method for Image Restoration
Participants: Emilie Chouzenoux, Marie-Caroline Corbineau, Jean-Christophe Pesquet (Collaboration: Carla Bertocchi, Marco Prato, Universita di Modena, Italy)
Variational methods are widely applied to ill-posed inverse problems for they have the ability to embed prior knowledge about the solution. However, the level of performance of these methods significantly depends on a set of parameters, which can be estimated through computationally expensive and time consuming methods. In contrast, deep learning offers very generic and efficient architectures, at the expense of explainability, since it is often used as a black-box, without any fine control over its output. Deep unfolding provides a convenient approach to combine variational-based and deep learning approaches. Starting from a variational formulation for image restoration, we developed in 1, iRestNet, a neural network architecture obtained by unfolding a proximal interior point algorithm. Hard constraints, encoding desirable properties for the restored image, are incorporated into the network thanks to a logarithmic barrier, while the barrier parameter, the stepsize, and the penalization weight are learned by the network. We derive explicit expressions for the gradient of the proximity operator for various choices of constraints, which allows training iRestNet with gradient descent and backpropagation. In addition, we provide theoretical results regarding the stability of the network for a common inverse problem example. Numerical experiments on image deblurring problems show that the proposed approach compares favorably with both state-of-the-art variational and machine learning methods in terms of image quality.
8.13 Proximal approaches for matrix optimization problems: Application to robust precision matrix estimation
Participants: Emilie Chouzenoux, Jean-Christophe Pesquet (Collaboration: Alessandro Benfenati, Universita di Milano)
In recent years, there has been a growing interest in mathematical models leading to the minimization, in a symmetric matrix space, of a Bregman divergence coupled with a regularization term. We address problems of this type within a general framework where the regularization term is split into two parts, one being a spectral function while the other is arbitrary. A Douglas–Rachford approach is proposed to address such problems, and a list of proximity operators is provided allowing us to consider various choices for the fit–to–data functional and for the regularization term. Based on our theoretical results, two novel approaches are proposed for the noisy graphical lasso problem, where a covariance or precision matrix has to be statistically estimated in presence of noise. The Douglas–Rachford approach directly applies to the estimation of the covariance matrix. When the precision matrix is sought, we solve a nonconvex optimization problem. More precisely, we propose a majorization–minimization approach building a sequence of convex surrogates and solving the inner optimization subproblems via the aforementioned Douglas–Rachford procedure. We establish conditions for the convergence of this iterative scheme. We illustrate the good numerical performance of the proposed approaches with respect to state–of–the–art approaches on synthetic and real-world datasets 8.
8.14 Graph Representation Learning with Exponential Family Distributions
Participants: Fragkiskos Malliaros, Abdulkadir Çelikkanat
Graph representation learning (GRL) methods aim to map each vertex into a low dimensional space by preserving both local and global structure of a given network. A widely applied graph representation learning paradigm is based on the combination of random walks for sampling context nodes and the traditional Skip-Gram model to capture center-context node relationships. We have emphasized on exponential family distributions to capture rich interaction patterns between nodes in random walk sequences. We have introduced the generic exponential family graph embedding model, that generalizes random walk-based network representation learning techniques to exponential family conditional distributions 40. We have studied three particular instances of this model, analyzing their properties and showing their relationship to existing unsupervised learning models. The experimental evaluation on real-world datasets has demonstrated that the proposed techniques outperform well-known baseline methods in two downstream machine learning tasks.
8.15 Influence Estimation and Maximization with Graph Representation Learning
Participants: Fragkiskos Malliaros (Collaboration: George Panagopoulos, Michalis Vazirgiannis, École Polytechnique)
Finding a set of users that can maximize the spread of information in a complex network is an important problem in social media analysis—being a critical part of several real-world applications such as viral marketing, political advertising, and epidemiology. Although influence maximization has been studied extensively in the past, the majority of works focus on the algorithmic aspect of the problem, overlooking several practical improvements that can be derived by data-driven observations or the inclusion of machine learning. Motivated by the recent criticism on diffusion models and the galloping advancements in influence learning, we have proposed IMINFECTOR (Influence Maximization with INFluencer vECTORs) 32, a method that uses representations learned from diffusion cascades to perform model-independent influence maximization (this is an extended version of the CELFIE model 57). The first part of the methodology is a multi-task neural network that learns embeddings of nodes that initiate cascades (influencer vectors) and embeddings of nodes that participate in them (susceptible vectors). The norm of an influencer vector captures a node's aptitude to initiate lengthy cascades and is used to reduce the number of candidate seeds. The combination of influencer and susceptible vectors form the diffusion probabilities between nodes. These are used to reformulate the computation of the influence spread and propose a greedy solution to influence maximization that retains the theoretical guarantees. We have applied our method in three sizable datasets and evaluate it using cascades from future time steps. IMINFECTOR's scalability and accuracy outperform various competitive algorithms and metrics from the diverse landscape of influence maximization.
8.16 Tools for Document Similarity via the Word Mover's Distance
Participants: Fragkiskos Malliaros (Collaboration: Konstantinos Skianis, Michalis Vazirgiannis, École Polytechnique; Nikolaos Tziortziotis, Jellyfish)
Measuring distance between documents has always been a key component in many natural language processing tasks, such as biomedical document classification, machine translation, question answering, and text generation. Nevertheless, the task poses various challenges, making it not trivial; whether two documents are similar or not, is not always clear and may vary from application to application. Word embeddings have opened a new path in creating novel approaches for addressing traditional problems in the natural language processing (NLP) domain. Nevertheless, using word embeddings to compare text documents remains a relatively unexplored topic—with Word Mover's Distance (WMD) being the prominent tool used so far. In the work 61, we present a variety of tools that can further improve the computation of distances between documents based on WMD. We have demonstrateed that, alternative stopwords, cross document-topic comparison, deep contextualized word vectors and convex metric learning, constitute powerful tools that can boost WMD.
8.17 Dynamic Monitoring of Medical Imaging Software Use with Recurrent Neural Networks
Participants: Fragkiskos Malliaros (Collaboration: Chloé Adam, Paul-Henry Cournède, MICS, CentraleSupélec; Antoine Aliotti, GE Healthcare)
User interaction with a software may be formalized as a sequence of actions. We have proposed two methods – based on different representations of input actions – to address two distinct industrial issues: next action prediction and software crash risk detection 4. Both methods take advantage of the recurrent structure of Long Short Term Memory neural networks to capture dependencies among our sequential data as well as their capacity to potentially handle different types of input representations for the same data. Given the history of user actions in the interface, our first method aims at predicting the next action. The proposed recurrent neural network outperforms state-of-the-art proactive user interface algorithms with standard one-hot vectors as inputs. Using the same data set, the second method aims at crash risk detection. To address this task, we have proposed to use feature vectors composed of actions with above average crash probabilities as inputs of the LSTM—with the idea to take advantage of its ability to learn relevant past information to detect crash patterns. We have observed that the method outperforms state-of-the-art sequence classification methods. Our approaches have been demonstrated on medical imaging software logs from ten different hospitals worldwide, though they might be applied to various user interfaces in a wide range of applications.
8.18 Robustness of Neural Networks
Participants: Ana Neacsu, Kavya Gupta, Fragkiskos Malliaros, Jean-Christophe Pesquet (Collaboration: Patrick Combettes, North Caroline State University; Corneliu Burileanu, Polithenica University of Bucharest)
Obtaining sharp Lipschitz constants for feed-forward neural networks is essential to assess their robustness in the face of perturbations of their inputs. In 16, we derive such constants in the context of a general layered network model involving compositions of nonexpansive averaged operators and affine operators. By exploiting this architecture, our analysis finely captures the interactions between the layers, yielding tighter Lipschitz constants than those resulting from the product of individual bounds for groups of layers. The proposed framework is shown to cover in particular many practical instances encountered in feed-forward neural networks. Our Lipschitz constant estimates are further improved in the case of structures employing scalar nonlinear functions, which include standard convolutional networks as special cases.
In [hal-03140282], we show how to practically build positive neural networks with a given Lipschitz constant bound. An application to automatic gesture recognition based on EMG signals is demonstrated. In 53, this robust design method is extended to a new class of neural networks which constitute an intermediate solution between fully connected networks and convolutive ones. An application to audio denoising is presented.
8.19 Prediction of Time Series with Neural Networks
Participants: Marc Castella, Jean-Christophe Pesquet, Sagar Verma
Neural networks offer new perspectives in terms of prediction of time series. We are especially interested in signals related to electrical motors, which are the most important source of mechanical energy in the industrial world. Their modeling traditionally relies on a physics-based approach, which aims at taking their complex internal dynamics into account. In 65, we explore the feasibility of modeling the dynamics of an electrical motor by following a data-driven approach, which uses only its inputs and outputs and does not make any assumption on its internal behaviour. We propose a novel encoder-decoder architecture which benefits from recurrent skip connections. We also propose a novel loss function that takes into account the complexity of electrical motor quantities and helps in avoiding model bias. We show that the proposed architecture can achieve a good learning performance on our high-frequency high-variance datasets. Two datasets are considered: the first one is generated using a simulator based on the physics of an induction motor and the second one is recorded from an industrial electrical motor. We benchmark our solution using variants of traditional neural networks like feedforward, convolutional, and recurrent networks. We evaluate various design choices of our architecture and compare it to the baselines. We show the domain adaptation capability of our model to learn dynamics just from simulated data by testing it on the raw sensor data. We finally show the effect of signal complexity on the proposed method ability to model temporal dynamics.
The work in 64 focuses on the quantitative analysis of deep neural networks used in data-driven modeling of induction motor dynamics. With the availability of a large amount of data generated by industrial sensor networks, it is now possible to train deep neural networks. Recently researchers have started exploring the usage of such networks for physics modeling, online control, monitoring, and fault prediction in induction motor operations. We consider the problem of estimating speed and torque from currents and voltages of an induction motor. Neural networks provide quite good performance for this task when analysed from a machine learning perspective using standard metrics. We show, however, that there are some caveats in using machine learning metrics to analyze a neural network model when applied to induction motor problems. Given the mission-critical nature of induction motor operations, the performance of neural networks has to be validated from an electrical engineering point of view. To this end, we evaluate several traditional neural network architectures and recent state of the art architectures on dynamic and quasi-static benchmarks using electrical engineering metrics.
8.20 Learning Maximally Monotone Operators for Image Recovery
Participant: Jean-Christophe Pesquet (Collaboration: Audrey Repetti, Matthieu Terris, Yves Wiaux, Heriot Watt University, UK)
In the work 78, the authors introduce a new paradigm for solving regularized variational problems. These are typically formulated to address ill-posed inverse problems encountered in signal and image processing. The objective function is traditionally defined by adding a regularization function to a data fit term, which is subsequently minimized by using iterative optimization algorithms. Recently, several works have proposed to replace the operator related to the regularization by a more sophisticated denoiser. These approaches, known as plug-and-play (PnP) methods, have shown excellent performance. Although it has been noticed that, under nonexpansiveness assumptions on the denoisers, the convergence of the resulting algorithm is guaranteed, little is known about characterizing the asymptotically delivered solution. In the current article, we propose to address this limitation. More specifically, instead of employing a functional regularization, we perform an operator regularization, where a maximally monotone operator (MMO) is learned in a supervised manner. This formulation is flexible as it allows the solution to be characterized through a broad range of variational inequalities, and it includes convex regularizations as special cases. From an algorithmic standpoint, the proposed approach consists in replacing the resolvent of the MMO by a neural network (NN). We provide a universal approximation theorem proving that nonexpansive NNs provide suitable models for the resolvent of a wide class of MMOs. The proposed approach thus provides a sound theoretical framework for analyzing the asymptotic behavior of first-order PnP algorithms. In addition, we propose a numerical strategy to train NNs corresponding to resolvents of MMOs. We apply our approach to image restoration problems and demonstrate its validity in terms of both convergence and quality.
8.21 Sparse signal reconstruction for nonlinear models via piecewise rational optimization
Participants: Marc Castella, Arthur Marmin, Jean-Christophe Pesquet (Collaboration: Laurent Duval, IFPEN)
In the work 28, the authors propose a method to reconstruct sparse signals degraded by a nonlinear distortion and acquired at a limited sampling rate. Our method formulates the reconstruction problem as a nonconvex minimization of the sum of a data fitting term and a penalization term. In contrast with most previous works which settle for approximated local solutions, we seek for a global solution to the obtained challenging nonconvex problem. Our global approach relies on the so-called Lasserre relaxation of polynomial optimization. We here specifically include in our approach the case of piecewise rational functions, which makes it possible to address a wide class of nonconvex exact and continuous relaxations of the l0 penalization function. Additionally, we study the complexity of the optimization problem. It is shown how to use the structure of the problem to lighten the computational burden efficiently. Finally, numerical simulations illustrate the benefits of our method in terms of both global optimality and signal reconstruction.
8.22 Deep Neural Network Structures Solving Variational Inequalities
Participant: Jean-Christophe Pesquet (Collaboration: Patrick Louis Combettes, North Carolina University, USA)
Motivated by structures that appear in deep neural networks, we investigate nonlinear composite models alternating proximity and affine operators defined on different spaces. We first show that a wide range of activation operators used in neural networks are actually proximity operators. We then establish conditions for the averagedness of the proposed composite constructs and investigate their asymptotic properties. It is shown that the limit of the resulting process solves a variational inequality which, in general, does not derive from a minimization problem 2.
8.23 Global Optimization for Recovery of Clipped Signals Corrupted with Poisson-Gaussian Noise
Participant: Marc Castella, Arthur Marmin, Jean-Christophe Pesquet (Collaboration: Anna Jezierska, Polish Academy of Sciences, Warsaw, Poland)
In the work 29, the authors study a variational formulation for reconstructing nonlinearly distorted signals corrupted with a Poisson-Gaussian noise. In this situation, the data fidelity term consists of a sum of a weighted least squares term and a logarithmic one. Both of them are precomposed by a nonlinearity, modelling a clipping effect, which is assumed to be rational. A regularization term, being a piecewise rational approximation of the 0 function provides a suitable sparsity measure with respect to a preset linear operator. We propose a global optimization approach for such a problem. More specifically, it is first transformed into a generalized moment problem by introducing some auxiliary variables. Then, a hierarchy of semidefinite programming relaxations is built. Numerical examples show the good performance of the proposed approach.
8.24 Coupled Feature Learning for Multimodal Medical Image Fusion
Participant: Nora Ouzir (Collaboration: F. Veshki, S. Vorobyov, and E. Ollila)
Multimodal image fusion aims to combine relevant information from images acquired with different sensors. In medical imaging, fused images play an essential role in both standard and automated diagnosis. In 83, the authors propose a novel image fusion method based on coupled dictionary learning. Unlike many current medical fusion methods, the proposed approach does not suffer from intensity attenuation nor loss of critical information. The images are decomposed into coupled and independent components obtained using sparse representations with identical supports and a Pearson correlation constraint. An alternating minimization algorithm is designed to address the resulting optimization problem. Experiments are conducted using various multimodal inputs, such as in vivo MR-CT and MR-PET images.
8.25 Robust Registration of Multi-modal Medical Images Using Huber's Criterion
Participant: Nora Ouzir (Collaboration: E. Ollila and S. Vorobyov)
Registration of multi-modal medical images is an essential pre-processing step, for example, for fusion or image guided-interventions. However, the alignment process is prone to high variability in tissue appearance between modalities, in addition to local intensity variations and artefacts. The work in 56 introduces a robust approach that mitigates the undesirable effect of such variability. We propose a novel method using Huber's criterion, which enables a jointly convex estimation of the motions and the associated scale parameters. The problem is formulated as a complex 2D transformation estimation and a novel dictionary learning-based data fidelity term is proposed.
8.26 Shaping for PET Image Analysis
Participant: Hugues Talbot (Collaboration: Eloïse Grossiord, Laurent Najman, ESIEE Paris ; Benoit Naegel, iCube, Strasbourg ; Nicolas Passat, CRESTIC, Reims)
Component-trees constitute an efficient data structure for hierarchical image modeling. In particular they are relevant for processing and analyzing images where the structures of interest correspond either to local maxima or local minima of intensity. This is indeed the case of functional data in medical imaging. This motivates the use of component-tree-based approaches for analyzing Positron Emission Tomography (PET) images in the context of oncology. In this article 18, we present a simple, yet efficient, methodological framework for PET image analysis based on component-trees. More precisely, we show that the second-order paradigm of shaping, that broadly consists of computing the component-tree of a component-tree, provides a relevant way of generalizing the threshold-based strategies classically used by medical practitioners for handling PET images. In addition, it also allows to embed relevant priors regarding the sought cancer lesions.
8.27 Myocardial Perfusion Simulation for Coronary Artery Disease
Participant: Hugues Talbot (Collaboration: L. Najman ESIEE Paris ; I. Vignon-Clementel, REO Team leader, Inria ; C. Taylor, Heartflow Inc.)
Patient-specific models of blood flow are being used clinically to diagnose and plan treatment for coronary artery disease. A remaining challenge is bridging scales from flow in arteries to the micro-circulation supplying the myocardium. Previously proposed models are descriptive rather than predictive and have not been applied to human data. In this article 33, we develop a multiscale patient-specific model enabling blood flow simulation from large coronary arteries to myocardial tissue. Patient vasculatures are segmented from coronary computed tomography angiography data and extended from the image-based model down to the arteriole level using a space-filling forest of synthetic trees. Blood flow is modeled by coupling a 1D model of the coronary arteries to a single-compartment Darcy myocardium model. Simulated results on five patients with non-obstructive coronary artery disease compare overall well to PET exam data for both resting and hyperemic conditions. Results on a patient with severe obstructive disease link coronary artery narrowing with impaired myocardial blood flow, demonstrating the model's ability to predict myocardial regions with perfusion deficit. This is the first report of a computational model for simulating blood flow from the epicardial coronary arteries to the left ventricle myocardium applied to and validated on human data.
8.28 In-vivo Quantification of Skin Ageing
Participant: Hugues Talbot (Collaboration: Laboratoires Clarins)
Reflectance confocal microscopy (RCM) is a powerful tool to visualize the skin layers at cellular resolution up to a depth of 200m. A semi-quantitative score of skin aging from RCM images has been previously published, requiring visual assessment of the images by experienced dermatologists. In this article 34, we propose new computer-based methods to automatically quantify the skin aging process on RCM image.
8.29 Hyperprogressive Disease During Immunotherapy
Participant: Baptiste Kas, Hugues Talbot (Collaboration: Institut Gustave Roussy)
Hyperprogressive disease (HPD) is an aggressive pattern of progression reported for patients treated with programmed cell death 1 (PD-1)/programmed cell death 1 ligand (PD-L1) inhibitors as a single agent in several studies. However, the use of different definitions of HPD introduces the risk of describing different tumoral behaviors. In this paper 21, we assess the accuracy of each HPD definition to identify the frequency of HPD and the association with poorer outcomes of immune-checkpoint inhibitor (ICI) treatment in patients with advanced non-small cell lung cancer (NSCLC) and to provide an optimized and homogenized definition based on all previous criteria for identifying HPD. The findings of this retrospective cohort study of patients with NSCLC suggest that the previous five definitions of HPD were not associated with the same tumor behavior. A new definition, based on a more physical model of tumor growth rate, appeared to be associated with the characteristics expected with HPD (increase of the tumor kinetics and poor survival).
8.30 Bio-particle Sizing
Participant: Hugues Talbot (Collaboration: Nanyang Technological University, Singapore)
High accuracy measurement of size is essential in physical and biomedical sciences. Various sizing techniques have been widely used in sorting colloidal materials, analyzing bioparticles and monitoring the qualities of food and atmosphere. Most imaging-free methods such as light scattering measure the averaged size of particles and have difficulties in determining non-spherical particles. Imaging acquisition using camera is capable of observing individual nanoparticles in real time, but the accuracy is compromised by the image defocusing and instrumental calibration. In this work, a machine learning-based pipeline is developed to facilitate a high accuracy imaging-based particle sizing. In this article 25, we propose a pipeline consisting of an image segmentation module for cell identification and a machine learning model for accurate pixel-to-size conversion. The results manifest a significantly improved accuracy, showing great potential for a wide range of applications in environmental sensing, biomedical diagnostical, and material characterization.
8.31 Position Estimation of 3D Fluorescent Spherical Beads
Participant: Hugues Talbot (Collaboration: Alessandro Benfenati, Universita di Milano, Francesco Bonacci, Laboratoire Navier, Tarik Bourouina, ESIEE Paris)
Particle estimation is a classical problem arising in many science fields, such as biophysics, fluid mechanics and biomedical imaging. Many interesting applications in these areas involve 3D imaging data: In this work 7, we present a technique to estimate the 3D coordinates of the center of spherical particles. This procedure has its core in the processing of the images of the scanned volume: It firstly applies denoising techniques to each frame of the scanned volume and then provides an estimation of both the center and the profile of the 2D intersections of the particles with the frames, by coupling the usage of Total Variation functional and of a regularized weighted Least Squares fit. Then, the 2D information is used to retrieve the 3D coordinates using geometrical properties. The experiments provide evidence that image denoising has a large impact on the performance of the particle tracking procedures, since they strongly depend on the quality of the initial acquisition. This work shows that the choice of tailored image denoising technique for Poisson noise leads to a better estimation of the particle positions
8.32 An Elastica-driven Digital Curve Evolution Model for Image Segmentation
Participant: Hugues Talbot (Collaboration: University Grenoble-Alpes Savoie Mont-Blanc)
Geometric priors have been shown to be useful in image segmentation to regularize the results. For example, the classical Mumford-Shah functional uses region perimeter as prior. This has inspired much research in the last few decades, with classical approaches like the Rudin-Osher-Fatemi and most graph-cut formulations, which all use a weighted or binary perimeter prior. It has been observed that this prior is not suitable in many applications, for example for segmenting thin objects or some textures, which may have high perimeter/surface ratio.Mumford observed that an interesting prior for natural objects is the Euler elastical model, which involves the squared curvature. In other areas of science, researchers have noticed that some physical binarization processes, like emulsion unmixing, can be well-approximated by curvature-related flow like the Willmore flow. However, curvature-related flows are not easy to compute because curvature is difficult to estimate accurately, and the underlying optimization processes are not convex. In this article 6, we propose to formulate a digital flow that approximates an Elastica-related flow using a multigrid-convergent curvature estimator, within a discrete variational framework. We also present an application of this model as a post-processing step to a segmentation framework
8.33 Segmentation of White Matter in Thick 3d Confocal Images of Eebrafish
Participant: Hugues Talbot (Collaboration: Paris-Saclay TEFOR institute, CNRS)
Tissue clearing methods have boosted the microscopic observations of thick samples such as wholemount mouse or zebrafish. However, even with the best tissue clearing methods, specimens are not completely transparent and light attenuation increases with depth, reducing signal output and signal-to-noise ratio. In addition, since tissue clearing and microscopic acquisition techniques have become faster, automated image analysis is now an issue. In this context, mounting specimens at large scale often leads to imperfectly aligned or oriented samples, which makes relying on predefined, sample-independent parameters to correct signal attenuation impossible. In this article 24, we propose a sample-dependent method for contrast correction. It relies on segmenting the sample, and estimating sample depth isosurfaces that serve as reference for the correction. We segment the brain white matter of zebrafish larvae. We show that this correction allows a better stitching of opposite sides of each larva, in order to image the entire larva with a high signal-to-noise ratio throughout. We also show that our proposed contrast correction method makes it possible to better recognize the deep structures of the brain by comparing manual vs. automated segmentations. This is expected to improve image observations and analyses in high-content methods where signal loss in the samples is significant.
8.34 Multiband Astronomical Source Detection
Participant: Hugues Talbot (ERC SUNDIAL ITN Grant, collaboration with 7 other institutes in Europe)
Component-graphs provide powerful and complex structures for multi-band image processing. In this article 55, we propose a multiband astronomical source detection framework with the component-graphs relying on a new set of component attributes. We propose two modules to differentiate nodes belong to distinct objects and to detect partial object nodes. Experiments demonstrate an improved capacity at detecting faint objects on a multi-band astronomical dataset.
8.35 Superresolution and OCR
Participant: Hugues Talbot (Collaboration: Artefact)
Recently, many deep learning methods have been used to handle single image super-resolution (SISR) tasks and often achieve state-of-the-art performance. From a visual point of view, the results look convincing. Yet, does it mean that those techniques are reliable and robust enough to be implemented in real business cases to enhance the performance of other computer vision tasks? In this article 58, we investigate the use of SISR to construct higher-resolution images of real receipt photos sent by the customers of a company and evaluate its impact on the performance of an OCR task (receipt information retrieval). Using built-in task-based performance evaluation methods, we show that the use of SISR can significantly improve OCR performance in the case where recognition was poor in low-resolution, but can also deteriorate the performance for receipts that were already successfully recognized. As a conclusion, we provide recommendations on how to best use SISR in a production environment.
8.36 Artificial Intelligence for COVID-19 Disease Quantification
Participants: Maria Vakalopoulou (Collaboration: AP-HP - Hopital Cochin Broca Hotel Dieu; Therapanacea)
Coronavirus disease 2019 (COVID-19) emerged in 2019 and disseminated around the world rapidly. Computed tomography (CT) imaging has been proven to be an important tool for screening, disease quantification and staging. The latter is of extreme importance for organizational anticipation (availability of intensive care unit beds, patient management planning) as well as to accelerate drug development through rapid, reproducible and quantified assessment of treatment response. Even if currently there are no specific guidelines for the staging of the patients, CT together with some clinical and biological biomarkers are used. In 10, we collected a multi-center cohort and we investigated the use of medical imaging and artificial intelligence for disease quantification, staging and outcome prediction. Our approach relies on automatic deep learning-based disease quantification using an ensemble of architectures, and a data-driven consensus for the staging and outcome prediction of the patients fusing imaging biomarkers with clinical and biological attributes. Highly promising results on multiple external/independent evaluation cohorts as well as comparisons with expert human readers demonstrate the potentials of our approach.
8.37 Artificial Intelligence Applications for Thoracic imaging
Participants: Maria Vakalopoulou (Collaboration: AP-HP - Hopital Cochin Broca Hotel Dieu; Therapanacea)
Artificial intelligence is a hot topic in medical imaging. The development of deep learning methods and in particular the use of convolutional neural networks (CNNs), have led to substantial performance gain over the classic machine learning techniques. Multiple usages are currently being evaluated, especially for thoracic imaging, such as such as lung nodule evaluation, tuberculosis or pneumonia detection or quantification of diffuse lung diseases. Chest radiography is a near perfect domain for the development of deep learning algorithms for automatic interpretation,requiring large annotated datasets, in view of the high number of procedures and increasing data availability. Current algorithms are able to detect up to 14 common anomalies, when present as isolated findings. Chest computed tomography is another major field of application for artificial intelligence, especially in the perspective of large scale lung cancer screening. It is important for radiologists to apprehend, contribute actively and lead this new era of radiology powered by artificial intelligence. Such a perspective requires understanding new terms and concepts associated with machine learning. The objective of this study 11 is to provide useful definitions for understanding the methods used and their possibilities, and report current and future developments for thoracic imaging.Prospective validation of AI tools will be required before reaching routine clinical implementation.
Finally, in 74, we develop a deep learning algorithm for the automatic assessment of the extent of systemic sclerosis (SSc)-related interstitial lung disease (ILD) on chest CT images. In particular, a multicomponent deep neural network (AtlasNet) was trained on 6888 fully annotated CT images ( for training and for validation) from 17 patients with no, mild, or severe lung disease. The model was tested on a dataset of 400 images from another 20 patients, independently partially annotated by three radiologist readers. The ILD contours from the three readers and the deep learning neural network were compared by using the Dice similarity coefficient (DSC). The correlation between disease extent obtained from the deep learning algorithm and that obtained by using pulmonary function tests (PFTs) was then evaluated in the remaining 171 patients and in an external validation dataset of 31 patients based on the analysis of all slices of the chest CT scan. The Spearman rank correlation coefficient () was calculated to evaluate the correlation between disease extent and PFT results.
8.38 Learning-based Registration of Medical Imaging
Participants: Théo Estienne, Enzo Battistella, Marvin Lerousseau, Roger Sun, Hugue Talbot, Maria Vakalopoulou (Collaboration: Institut Gustave Roussy; Therapanacea)
Image registration and segmentation are the two most studied problems in medical image analysis. Deep learning algorithms have recently gained a lot of attention due to their success and state-of-the-art results in variety of problems and communities. In 17, we propose a novel, efficient, and multi-task algorithm that addresses the problems of image registration and brain tumor segmentation jointly. Our method exploits the dependencies between these tasks through a natural coupling of their interdependencies during inference. In particular, the similarity constraints are relaxed within the tumor regions using an efficient and relatively simple formulation. We evaluated the performance of our formulation both quantitatively and qualitatively for registration and segmentation problems on two publicly available datasets (BraTS 2018 and OASIS 3), reporting competitive results with other recent state-of-the-art methods. Moreover, our proposed framework reports significant amelioration (p ) for the registration performance inside the tumor locations, providing a generic method that does not need any predefined conditions (e.g. absence of abnormalities) about the volumes to be registered.
Moreover, in 74, we summarize our work presented on Learn2Reg challenge 2020. The main contributions of our work rely on (i) a symmetric formulation, predicting the transformations from source to target and from target to source simultaneously, enforcing the trained representations to be similar and (ii) integration of variety of publicly available datasets used both for pretraining and for augmenting segmentation labels. Our method reports a mean dice of for task 3 and for task 4 on the test sets, taking third place on the challenge.
8.39 Radiomics and Pathomics using Advanced Machine and Deep Learning Approaches
Participants: Théo Estienne, Enzo Battistella, Marvin Lerousseau, Roger Sun, Hugues Talbot, Maria Vakalopoulou (Collaboration: Institut Gustave Roussy; University of Lyon; Therapanacea)
Combining radiotherapy (RT) with immuno-oncology (IO) therapy (IORT) may enhance IO-induced antitumor response. Quantitative imaging biomarkers can be used to provide prognosis, predict tumor response in a non-invasive fashion and improve patient selection for IORT. A biologically inspired CD8 T-cells-associated radiomics signature has been developed on previous cohorts. In 38 we evaluated whether this CD8 radiomic signature is associated with lesion response, whether it may help to assess disease spatial heterogeneity for predicting outcomes of patients treated with IORT. We also evaluated differences between irradiated and non-irradiated lesions.
Histopathological image segmentation is a challenging and important topic in medical imaging with tremendous potential impact in clinical practice. State of the art methods rely on hand-crafted annotations which hinder clinical translation since histology suffers from significant variations between cancer phenotypes. In 46, we propose a weakly supervised framework for whole slide imaging segmentation that relies on standard clinical annotations, available in most medical systems. In particular, we exploit a multiple instance learning scheme for training models. The proposed framework has been evaluated on multi-locations and multi-centric public data from The Cancer Genome Atlas and the PatchCamelyon dataset. Promising results when compared with experts' annotations demonstrate the potentials of the presented approach.
Segmentation and accurate localization of nuclei in histopathological images is a very challenging problem, with most existing approaches adopting a supervised strategy. These methods usually rely on manual annotations that require a lot of time and effort from medical experts. In 68, we present a self-supervised approach for segmentation of nuclei for whole slide histopathology images. Our method works on the assumption that the size and texture of nuclei can determine the magnification at which a patch is extracted. We show that the identification of the magnification level for tiles can generate a preliminary self-supervision signal to locate nuclei. We further show that by appropriately constraining our model it is possible to retrieve meaningful segmentation maps as an auxiliary output to the primary magnification identification task. Our experiments show that with standard post-processing, our method can outperform other unsupervised nuclei segmentation approaches and report similar performance with supervised ones on the publicly available MoNuSeg dataset.
Finally, a novel approach to combine medical imaging with additional clinical information is presented in 79. In particular, we investigate the use of recent advances in deep learning and propose an end-to-end trainable multi-instance convolutional neural network within a mixture-of-experts formulation that combines information from two types of data-images and clinical attributes-for the diagnosis of lymphocytosis. The convolutional network learns to extract meaningful features from images of blood cells using an embedding level approach and aggregates them. Moreover, the mixture-of-experts model combines information from these images as well as clinical attributes to form an end-to-end trainable pipeline for diagnosis of lymphocytosis. Our results demonstrate that even the convolutional network by itself is able to discover meaningful associations between the images and the diagnosis, indicating the presence of important unexploited information in the images. The mixture-of-experts formulation is shown to be more robust while maintaining performance via. a repeatability study to assess the effect of variability in data acquisition on the predictions. The proposed methods are compared with different methods from literature based both on conventional handcrafted features and machine learning, and on recent deep learning models based on attention mechanisms. Our method reports a balanced accuracy of and outperforms the handcrafted feature-based and attention-based approaches as well that of biologists which scored , and respectively. These results give insights on the potentials of the applicability of the proposed method in clinical practice.
9 Bilateral contracts and grants with industry
9.1 Bilateral contracts with industry
- PhD Contract with IFP Energies Nouvelles
- Project title: Polynomial optimization for sparse signal recovery
- Duration: 2018-2020
- Leaders: M. Castella and J.-C. Pesquet
- PhD Contract with IFP Energies Nouvelles
- Project title: Seismic signal analysis by using neural networks
- Duration: 2019-2022
- Leaders: A. Fraysse and J.-C. Pesquet
- PhD Contract with Thales Group
- Project title: Neural network solutions for safety of complex systems
- Duration: 2019-2022
- Responsible: J.-C. Pesquet and F. Malliaros
- PhD Contract with General Electric Healthcare
- Project title: Minimally invasive assessement of coronary disease
- Duration: 2018-2021
- Leader: Hugues Talbot
- PhD Contract with General Electric Healthcare
- Project title: Optimization methods for breast tomosynthesis
- Duration: 2017-2020
- Leader: J.-C. Pesquet and E. Chouzenoux
- PhD Contract with General Electric Healthcare
- Project title: Reconstruction 3D interventionnelle
- Duration: 2019-2022
- Leader: J.-C. Pesquet and E. Chouzenoux
- PhD Contract with IFP Energies nouvelles
- Project title: Graph-based learning from integrated multi-omics and multi-species data
- Duration: 2019-2022
- Leader: F. Malliaros and J.-C. Pesquet
- Contract with Schneider Electric
- Project title: Neural network modeling of electrical motors
- Duration: 2019
- Leader: J.-C. Pesquet
- Contract with SNCF
- Project title: SNCF Platipus: Examining the potential of machine learning algorithms in the analysis of scouring reports of aquatic foundations.
- Duration: 2019-2020
- Leader: F. Malliaros, M. Vakalopoulou.
- Contract with ESSILOR
- Project title: Software development for photo-refraction analysis
- Duration: 2020-2021
- Leader: E. Chouzenoux, J.C. Pesquet
10 Partnerships and cooperations
10.1 International initiatives
10.1.1 Inria international partners
Informal international partners
- Sup'Com Tunis - Prof. Amel Benazza-Benhayia. Collaboration Topic: Multispectral imaging and image compression.
- North Carolina State University - Prof. Patrick Louis Combettes. Collaboration Topic: Fixed point theory.
- Heriot-Watt University, UK - Prof. Audrey Repetti and Prof. Yves Wiaux. Collaboration Topic: Large-scale image restoration.
- University of Edinburgh, UK - Prof. Victor Elvira. Collaboration Topic: Bayesian signal processing.
- Indraprastha Information Institute Technology, Delhi, India - Prof. Angshul Majumdar. Collaboration Topic: Dictionary learning.
- Universidad Técnica Federico Santa María, Valparaíso, Chile - Prof. Luis M. Briceño-Arias. Collaboration Topic: Stochastic optimization.
- Aristotle University of Thessaloniki, Greece - Prof. Apostolos N. Papadopoulos. Collaboration Topic: Large-scale graph embeddings.
- Aalto University, Finland - Profs. Sergiy Vorobyov and Esa Ollila. Collaboration topic: Robust Dictionary Learning for Medical Imaging.
10.2 International research visitors
10.2.1 Visits to international teams
Research stays abroad
A. Marmin, 1 month internship (January 2020), Department of Biomedical Engineering, Faculty of Electronics, Telecommunications and Informatics, Gdansk University of Technology, Poland (collaboration with Anna Jezierska)
10.3 European initiatives
10.3.1 FP7 & H2020 Projects
- Program: H2020 ITN Marie Sklodowska-Curie
- Project acronym: Trade-Opt
- Project title: Training Data-Drive Experts in Optimization
- Duration: 2020-2024
- Coordinator: S. Villa (Univ. Genova, Italy), Local: J.C. Pesquet
- Program: H2020 ITN Marie Sklodowska-Curie
- Projet acronym: SUNDIAL
- Project Title: SUrvey Network for Deep Imaging Analysis and Learning
- Duration: 2017-2021
- Coordinator: Reynier Peletier (U. Groningen, NL), local: Hugues Talbot
- Program: ERC StG
- Project acronym: MAJORIS
- Project title: Majoration-Minimization algorithms for image processing
- Duration: 2020-2024
- Coordinator: E. Chouzenoux
10.4 National initiatives
- Program: Chaire IA - ANR
- Project acronym: Bridgeable
- Project title: BRIDinG thE gAp Between iterative proximaL methods and nEural networks
- Duration: 2020-2024
- Coordinator: J.C. Pesquet
- Program: ANR PRC
- Project acronym: CoMeDIC
- Project title: Convergent Metrics for DIscrete Calculus
- Duration: 2016-2021
- Coordinator: J.-O. Lachaud (Univ. Rhones Alpes Savoie Mont-Blanc), Local: H. Talbot
- Program: ANR PRCE
- Project acronym: R-Vessel-X
- Project title: Extraction et interprétation robustes des réseaux vasculaires dans les images biomédicales hépatiques
- Duration: 2018-2022
- Coordinator: A. Vacavant (Univ. Clermont Auvergne), local: H. Talbot
- Program: ANR JCJC
- Project acronym: GraphIA
- Project title: Scalable and robust representation learning on graphs
- Duration: 2021-2025
- Coordinator: H. Talbot
- Program: ANR JCJC
- Project acronym: MajIC
- Project title: Majorization-Minimization Algorithms for Image Computing
- Duration: 2017-2021
- Coordinator: E. Chouzenoux
- Program: ANR JCJC
- Project acronym: AVENUE
- Project title: A Visual memory network for scene understanding
- Duration: 2018-2022
- Coordinator: Dr. Karteek Alahari (Inria Grenoble - Rhône-Alpes). Local: F. Malliaros.
- Program: PHC - Campus France
- Projet acronym: POLONIUM
- Project title: When Poisson and Gauss meet in imaging
- Duration: 2018-2020
- Coordinator: J.C. Pesquet
10.5 Regional initiatives
- DATAIA UltraBioLearn (2019-2022). The project aims to research machine learning approaches for medical applications, in particular by leveraging semi-supervised learning using generative, graphbased and certifiable networks, in the context of predicting patient response to cancer treatments. Responsible: H. Talbot, F. Malliaros (N. Lassau, Institut Gustave Roussy).
11.1 Promoting scientific activities
11.1.1 Scientific events: organisation
Member of the organizing committees
- F. Malliaros. Member of the organizing committee of the 33rd European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML PKDD), Grenoble, France, 2022
- F. Malliaros. Member of the organizing committee of the 15th International Workshop on Graph-Based Natural Language Processing (TextGraphs), Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), Mexico City, Mexico, 2021
- F. Malliaros. Member of the organizing committee of the 14th International Workshop on Graph-Based Natural Language Processing (TextGraphs), International Conference on Computational Linguistics (COLING), Barcelona, 2020 (held online)
- H. Talbot. Member of the organizing committee for ICCV 2023 in Paris.
11.1.2 Scientific events: selection
Chair of conference program committees
- E. Chouzenoux. Elected member of the EURASIP Signal and Data Analytics for Machine Learning technical committee (since 2018), and of the IEEE Signal Processing Theory and Methods technical committee (since 2017).
Member of the conference program committees
- E. Chouzenoux. Organization of a special session at the 45th International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2020.
- H. Talbot. Member of the board and program committee for the conference Discrete Geometry and Mathematical Morphology, Uppsala, Sweden, April 2021.
- M. Vakalopoulou. Program Committee (PC) member on the Medical Imaging with Deep Learning (MIDL) 2020.
The members of the team reviewed numerous papers for several international conferences, such as for the annual conferences on Computer Vision and Pattern Recognition (CVPR), Medical Image Computing and Computer Assisted Intervention (MICCAI), Neural Information Processing Systems (NIPS) and International Conference on Learning Representations (ICLR), IEEE International Conference and Acoustics Speech and Signal Processing (ICASSP), IEEE International Conference on Image Processing (ICIP), IEEE Statistical Signal Processing workshop (SSP), European Signal Processing Conference (EUSIPCO), AAAI Conference on Artificial Intelligence (AAAI), The Web Conference (WWW), Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), International Conference on Web and Social Media (ICWSM), International Conference on Machine Learning (ICML), Conference on Neural Information Processing Systems (NeurIPS), International Conference on Complex Networks and Their Applications (Complex Networks), International Workshop on Graph-Based Natural Language Processing (TextGraphs).
Member of the editorial boards
- Emilie Chouzenoux: Associate Editor at IEEE Transactions on Signal Processing
- Emilie Chouzenoux: Associate Editor of the SIAM Journal on Mathematics of Data Sciences
- Jean-Christophe Pesquet: Associate Editor of the SIAM Journal on Imaging Sciences
- Hugues Talbot: Editor in Chief, Mathematical Morphology, Theory and Applications (De Gruyter)
- Hugues Talbot: Review Editor for Frontiers in Computer Vision, Nature group
- Maria Vakalopoulou: Editor at Computer Vision and Image Understanding (CVIU) Journal
Reviewer - reviewing activities
- E. Chouzenoux: IEEE Trans. on Image Processing, IEEE Trans. Signal Processing, SIAM Journal on Imaging Science, Journal of Optimization Theory and Applications, Journal of Global Optimization, Journal of Mathematical Imaging and Vision, Inverse Problems.
- J.-C. Pesquet: IEEE Trans. on Signal Processing, IEEE Trans. on Image Processing, IEEE Trans. on Information Theory (IEEE-TI), Signal Processing, SIAM Journal on Optimization, SIAM Journal on Imaging Sciences, Journal of Mathematical Imaging and Vision, Journal of Optimization Theory and Applications, Mathematical Programming.
- F. Malliaros: IEEE Transactions on Knowledge and Data Engineering (TKDE), ACM Transactions on Knowledge Discovery from Data (TKDD), Applied Network Science, International Journal on Artificial Intelligence Tools (IJAIT).
- H. Talbot: IEEE Pattern Analysis and Machine Intelligence, IEEE Transaction on Image Processing, Computer Vision and Image Understanding, J. on Mathematical Imaging and Vision, Signal Processing Letters, Transactions on Signal Processing. Reviewer for the conferences ACCV, BMVC, CVPR, ICIP, ICML, ISBI, and NeurIPS (ex-NIPS).
- M. Vakalopoulou: International Journal of Computer Assisted Radiology and Surgery (IJCARS), IEEE Trans. on Geoscience and Remote Sensing (TGRS), Journal of Selected Topics in Applied Earth Observations and Remote Sensing (JSTARS), ISPRS Journal of Photogrammetry and Remote Sensing, Computer Methods and Programs in Biomedicine (CMPB), Pattern Recognition Letters, Computer Vision and Image Understanding (CVIU), SPIE Optical Engineering (OE), Remote Sensing MDPI.
11.1.4 Invited talks
Séminaire Français d'Optimisation, GDR-MOA Meeting, September 2020 (Online)
Webinar Women in Data Science event, Univ. Paris Saclay, September 2020 (Online)
Séminaire Français d'Optimisation, GDR MOA - PGMO - SMAI-MODE, October 2020 (Online)
Unithé ou Café, Inria Saclay, December 2020 (Online)
Seminar GdR MIA, IRIT Toulouse, October 2020 (Online)
Seminar Inria Parietal, Saclay, November 2020 (Online)
IFP Energies nouvelles, France, March 2020
Booking.com, The Netherlands, May 2020 (Online)
Rakuten Institute of Technology, Rakuten Inc., France, November 2020 (Online)
International Conference on Image Processing Tools and Applications, Paris, November 2020 (online)
Gustave-Roussy seminar, October 2020
Université Paris-Est Créteil seminar, October 2020
IFPEN seminar, November 2020
Société de Pneumologie de Langue Francaise, October 2020 (Online)
Table ronde CentraleSupelec, Vive la Recherche, September 2020 (Online)
Department of Computer Science, Stony Brook University, July 2020 (Online)
MIDL 2020, July 2020 (Online)
MICS seminar, CentraleSupélec, May 2020 (Online)
GDR-ISIS Meeting, May 2020 (Online)
Masterclass IA imagerie médical, February 2020 (Online)
11.1.5 Leadership within the scientific community
J.-C. Pesquet is senior member of the Institut Universitaire de France and a Fellow of IEEE.
E. Chouzenoux is a senior member of the IEEE.
11.1.6 Scientific expertise
The members of the team participated to numerous PhD Thesis Committees, HDR Committees, recruiting Committees, and served as Grant Reviewers.
11.2 Teaching - Supervision - Juries
- Master : E. Chouzenoux. Foundations of Distributed and Large Scale Computing, 26h, M.Sc. in Data Sciences and Business Analytics, 3rd year CentraleSupélec, M.Sc. MVA, ENS Cachan, and ESSEC Business School, FR
- Master: E. Chouzenoux. Advanced Machine Learning, 18h, 3rd year CentraleSupélec, FR
- Master: E. Chouzenoux. Large Scale Optimization for Inverse Problems, 3h, M.Sc. ATSI, Univ. Paris Saclay, FR
- Master: F. Malliaros. Foundations of Machine Learning, 27h, M.Sc. in Data Sciences and Business Analytics, CentraleSupélec and ESSEC Business School, FR
- Master: F. Malliaros. Network Science Analytics, 27h, M.Sc. in Data Sciences and Business Analytics, CentraleSupélec and ESSEC Business School, M.Sc. in Artificial Intelligence, CentraleSupélec, 3rd year data science mention, CentraleSupélec, FR
- Master: F. Malliaros. 2EL1730: Machine Learning, 33h, 2nd year course at CentraleSupélec, FR
- Master: F. Malliaros. ST2: Mathematical Modeling of Propagation Phenomena – Propagation on Graphs, 15h, 1st year course at CentraleSupélec, FR
- Master : N. Ouzir. Advanced Machine Learning, 24h, M.Sc. in Data Sciences and Business Analytics, EN
- Master : J.C. Pesquet. Advanced course on Optimization, 33h, 2nd year course, CentraleSupélec, FR
- Master: J.C. Pesquet. Introduction to Optimization, 6h, M.Sc. MVA, ENS Cachan, FR
- Master: J.C. Pesquet. Convex Optimization Algorithms, 15h, M.Sc. in Optimization, Univ. Paris-Saclay , FR
- Master: H. Talbot. High-performance computing, 2nd year course, CentraleSupelec, 30h, EN
- Master: H. Talbot. Advanced optimization, M.Sc in Data Science and Business Analytics, CentraleSupélec and ESSEC Business School, 30h, EN
- Master: H. Talbot. Optimisation for AI, M.Sc in AI, CentraleSupelec and ESSEC Business School, 30h, EN
- Master: M. Vakalopoulou. Introduction to Visual Computing, 25h, 3rd year course, CentraleSupélec, FR
- Master: M. Vakalopoulou. Deep Learning, 25h, 3rd year course, CentraleSupélec, FR
- Master: M. Vakalopoulou. Introduction to Machine Learning, 33h, 2nd year course, CentraleSupélec, FR
- PhD (defended): Maïssa Sghaier, clinical Task-Based Reconstruction in tomosynthesis, 2017-2020, supervised by J.-C. Pesquet and S. Muller (GE Healthcare).
- PhD (defended): Arthur Marmin, Rational models optimized exactly for chemical processes improvement, 2017-2020, supervised by M. Castella (Telecom Paristech) and J.-C. Pesquet.
- PhD (defended): Mihir Sahasrabudhe, Unsupervised ans Weakly Supervised Deep Learning Methods for Computer Vision and Medical Imaging, 2016-2020, supervised by N. Paragios.
- PhD (defended): Sylvain Lempereur. Analyse quantitative de la morphologie de poissons aux stades larvaire et juvénile. 2017-2020. Supervised by Hugues Talbot and Jean-Stéphane Joly (CNRS).
- PhD (defended): Daniel Antunes: Contraintes géométriques et approches variationnelles pour l'analyse d'image. 2016-2020. Supervised by Hugues Talbot and Jacques-Olivier Lachaud (U. Savoie-Mont Blanc).
- PhD (in progress) : Jean-Baptiste Fest, Stochastic Majorization-Minimization algorithms, 2020-2023, supervised by E. Chouzenoux.
- PhD (in progress) : Gabriele Scrivanti, New algorithms for large scale problems exploiting geometric problem structure and convex relaxation, 2020-2023, supervised by J.C. Pesquet.
- PhD (in progress) : Segolène Martin, Majorization-Minimization methods for constrained optimization, 2020-2023, supervised by J.C. Pesquet and Ismail Ben Ayed (ETS Montréal, Canada).
- PhD (in progress) : Mouna Gharbi, Unfolded Majorization-Minimization algorithms, 2020-2023, supervised by E. Chouzenoux and L. Duval (IFPEN).
- PhD (in progress) : Yingping Li, Deep learning for medical imaging, 2018-2021, supervised by E. Chouzenoux and N. Lassau (Institut Gustave Roussy).
- PhD (in progress) : Abdulkadir Çelikkanat, Representation learning methods on graphs, 2017-2020, supervised by F. Malliaros and N. Paragios.
- PhD (in progress): Yunshi Huang, Majorization-Minimization approaches for large scale problems in image processing, 2018-2021, supervised by E. Chouzenoux and V. Elvira (Univ. Edinburgh).
- PhD (in progress) : Samy Ammari, Imagerie médicale computationnelle en neuro oncologie, 2019-2022, supervised by C. Balleyguier (Institut Gustave Roussy) and E. Chouzenoux.
- PhD (in progress) : Georgios Panagopoulos, Influence maximization in social networks, 2018-2021, supervised by F. Malliaros and M. Vazirgiannis (École Polytechnique).
- PhD (in progress): Surabhi Jagtap, Graph-based learning from multi-omics data, 2019-2022, supervised by F. Malliaros, J.-C. Pesquet, and L. Duval (IFP Energies Nouvelles).
- PhD (in progress): Kavya Gupta, Neural network solutions for safety of complex systems, 2019-2022, supervised by F. Malliaros, J.-C. Pesquet and F. Kaakai (Thales Group).
- PhD (in progress): Maria Papadomanolaki, Change Detection from Multitemporal High Resolution Data with Deep Learning, 2017-2021, supervised by M. Vakalopoulou and with K. Karantzalos.
- PhD (in progress): Théo Estienne, Improving anticancer therapies efficacy through Machine Learning on Medical Imaging & Genomic Data, 2017-2020, supervised by M. Vakalopoulou and N. Paragios.
- PhD (in progress): Enzo Battistella, Development of novel imaging approaches for tumour phenotype assessment by noninvasive imaging 2017-2020, supervised by M. Vakalopoulou and N. Paragios.
- PhD (in progress): Roger Sun, Deep learning and computer vision approaches on medical imaging and genomic data to improve the prediction of anticancer therapies' efficacy, 2017-2020, supervised by M. Vakalopoulou and N. Paragios.
- PhD (in progress): Ana Neacsu, Méthodes d'apprentissage profond inspirées d'algorithmes de traitement du signal, 2019-2022, supervised by J.-C. Pesquet and C. Burileanu (Politehnica Bucarest).
- PhD (in progress): Sagar Verma, Modélisation, contrôle et supervision de moteurs électriques par réseaux de neurones profonds, 2019-2022, supervised by M. Castella and J.-C. Pesquet.
- PhD (in progress): Marion Savanier, Reconstruction 3D interventionnelle, 2019-2022, supervised by E. Chouzenoux and C. Riddell (GE Healthcare).
- PhD (in progress): Marie-Charlotte Poilpre: Méthode de comparaison faciale morphologique, adaptée aux expertise judiciaires, basée sur la modélisation 3D. 2017-2020. Supervised by Hugues Talbot and Vincent Nozick (U. Paris-Est).
- PhD (in progress): Thank Xuan Nguyen. Détection et étude morphologique des sources extra-galactiques par analyse variationnelle. 2018-2021. Supervised by Hugues Talbot and Laurent Najman (ESIEE).
- PhD (in progress): Marvin Lerousseau. Apprentissage statistique en imagerie médicale et en génomique pour prédire l'efficacité des thérapies anti-tumorales. 2018-2021. Supervised by Nikos Paragios (Therapanacea), Eric Deutch (IGR) and Hugues Talbot.
- PhD (in progress): Mario Viti. Low-dose assessment of coronal vessel health on CT. 2019-2022. Supervised by Hugues Talbot.
- PhD (in progress): Théodore Aouad: Geometric Semi-Supervised Machine-Learning methods in medical imaging. 2020-2023. Supervised by Hugues Talbot.
- PhD (in progress): Younes Belkouchi. Graph Neural Network Generative methods in oncology. 2020-2023. Supervised by Nathalie Lassau, Fragkiskos Malliaros, and Hugues Talbot.
The faculty members of the team serve regularly as a jury Member to Final Engineering Internship and the Research Innovation Project for students of CentraleSupélec, FR.
11.3.1 Articles and contents
J.C. Pesquet and H. Talbot published the popularization article “Des problèmes inverses aux trous noirs”, in the CentraleSupélec website: https://
Vision talk, in the event “Women in Data Science” organized by Univ. Paris Saclay in september 2020.
Member of the mentoring session on MIDL 2020 conference;
Participation to the “Research week” for the 1st year students of CentraleSupelec.
12 Scientific production
12.1 Major publications
- 1 article 'Deep Unfolding of a Proximal Interior Point Method for Image Restoration'. Inverse Problems 2020
- 2 article 'Deep Neural Network Structures Solving Variational Inequalities *'. Set-Valued and Variational Analysis February 2020
- 3 article 'The Core Decomposition of Networks: Theory, Algorithms and Applications'. The VLDB Journal 2019
12.2 Publications of the year
International peer-reviewed conferences
Conferences without proceedings
Edition (books, proceedings, special issue of a journal)
Doctoral dissertations and habilitation theses
Reports & preprints