## Section: New Results

### Biological and Computational Vision

#### Shape manifolds and applications to shape segmentation

Participants : Patrick Etyngier, Renaud Keriven, Florent Segonne [ CERTIS ] .

"Non-linear shape prior are introduced for the deformable model framework that are learnt from a set of shape samples using recent manifold learning techniques. A category of shapes is modeled as a finite dimensional manifold approximated using Diffusion maps. The method computes a Delaunay triangulation of the reduced space, considered as Euclidean, and uses the resulting space partition to identify the closest neighbors of any given shape based on its Nystrom extension. A non-linear shape prior term is designed to attract a shape towards the shape prior manifold at given constant embedding. Results on shapes of ventricle nuclei demonstrate the potential of the method for segmentation tasks.

This work was presented in [Oops!] , [Oops!] , [Oops!] , [Oops!] .

#### New algorithm for solving variational problems: Application to image restoration

Keywords : variational problems, optimization, Sobolev spaces, space of bounded variations, numerical approximation, image restauration, infinite Lapalcian.

Participants : Gilles Aubert [ Laboratoire J.A. Dieudonné, Université de Nice-Sophia Antipolis ] , Pierre Kornprobst.

We proposed in
[Oops!] a new unifying method for solving variational problems defined on the Sobolev spaces
W^{1,
p}(
) or on the space of functions of bounded variations
BV(
) . The method is based on a recent new characterization of these spaces by Bourgain, Brezis and Mironescu (2001), where norms can be approximated by a sequence of
integral operators involving a differential quotient and a suitable sequence of radial mollifiers. We use this characterization to define a variational formulation, for which existence, uniqueness and convergence of the solution is proved. The proposed approximation is valid for any
p and does not depend on the attach term. Implementation details are given and we show examples on the image restoration problem.

#### Efficient Segmentation of Piecewise Smooth Images

Participants : Jérome Piovano, Mikaël Rousson [ Siemens SCR ] , Théo Papadopoulo.

[Oops!] proposes a fast and robust segmentation model for piecewise smooth images. Rather than modeling each region with global statistics, we introduce local statistics in an energy formulation. The shape gradient of this new functional gives a contour evolution controlled by local averaging of image intensities inside and outside the contour. To avoid the computational burden of a direct estimation, we express these terms as the result of convolutions. This makes an efficient implementation via recursive filters possible, and gives a complexity of the same order as methods based on global statistics. This approach leads to results similar to the general Mumford-Shah model but in a faster way, without solving a Poisson partial differential equation at each iteration. The method is applied to synthetic and real data, and results are compared with the piecewise smooth and piecewise constant Mumford-Shah models.

#### Symmetrical Dense Optical Flow Estimation with Occlusions Detection

Participants : Luis Alvarez [ Universidad de Las Palmas de Gran Canaria ] , Rachid Deriche, Théo Papadopoulo, Javier Sànchez [ Universidad de Las Palmas de Gran Canaria ] .

Traditional techniques of dense optical flow estimation do not generally yield symmetrical solutions: the results will differ if they are applied between images I1 and I2 or between images I2 and I1 . [Oops!] presents a method to recover a dense optical flow field map from two images, while explicitely taking into account the symmetry across the images as well as possible occlusions in the flow field. The idea is to consider both displacements vectors from I1 to I2 and I2 to I1 and to minimise an energy functional that explicitely encodes all those properties. This variational problem is then solved using the gradient flow defined by the Euler–Lagrange equations associated to the energy. To prove the importance of the concepts of symmetry and occlusions for optical flow computation, we have extended a classical approach to handle those. Experiments clearly show the added value of these properties to improve the accuracy of the computed flows.

#### Towards bridging the Gap between Biological and Computational Image Segmentation

Keywords : Computer Vision, Biological Vision, Neural Networks, Boundary Contour System, Feature Contour System, Line Process, Surface Process, Variational Techniques for Computer Vision, Edge Grouping, Perceptual Grouping, Mean Field Theory, Boltzmann Machines Learning.

Participants : Rachid Deriche, Olivier Faugeras, Theo Papadopoulo, Iasonas Kokkinos [ Odyssée/National Technical University of Athens Computer Vision, Speech Communication and Signal Processing Group ] , Petros Maragos [ National Technical University of Athens Computer Vision, Speech Communication and Signal Processing Group ] .

This work has been initiated some years ago during the visit of I. Kokkinos to our research project for 4 months internship but it's only this year that this report has been published [Oops!] .

This work was partially supported by the EC IP project FP6-015879, FACETS and the Fondation d'Entreprise EADS

- Computational Vision
The branch of computer science and applied mathematics that studies way of emulating visual performances with computers.

This work [Oops!] presents a joint study of biological and computational vision. First we briefly review the most common models of neurons and neural networks and the function of cells in the V1/V2 areas of the visual cortex. Subsequently, we present the biologically plausible models for image segmentation that have been proposed by Stephen Grossberg and his collaborators during the previous two decades in a series of papers. We have implemented the B.C.S. (Boundary Contour System) and F.C.S. (Feature Contour System) models that form the basic building blocks of this model of biological vision, known as FACADE (Form And Colour and DEpth) theory. During their implementation, we faced several problems, like a large number of parameters and instability with respect to these; this was not traded off with a higher performance when compared to classical computer vision algorithms. This has led us to propose a simplified version of the B.C.S./F.C.S. system, and to explore the merits of using nonlinear recurrent dynamics. The biologically plausible model we propose is paralleled with classical computational vision techniques, while a link with the variational approach to computer vision is established. By interpreting the network function in a probabilistic manner we derive an algorithm for learning the network weights using manually determined segmentations excerpted from the Berkeley database. This facilitates learning the terms involved in the variational criterion that quantifies edge map quality from ground truth data. Using the learned weights our network outperforms classical edge detection algorithms, when evaluated on the Berkeley segmentation benchmark.

#### From variational to spiking network image segmentation techniques

Keywords : Segmentation, Synchonicity, Spiking neurons, Phase representation.

Participants : Léonard Gérard, Pierre Kornprobst, Gabriel Montero, Thierry Viéville.

This work was partially supported by the EC IP project FP6-015879, FACETS and the Fondation d'Entreprise EADS

Many traditional image segmentation techniques are based on variational approaches seen as Mumford Shah's approach variants. A step further, it has been argued, e.g. by Sarti et al, that such mechanism could provide an abstract view of brain's segmentation. Following this track, we "implement" segmentation using a retinotopical neuron network [Oops!] .

In our approach, the first step is to consider a discrete approximation of the Mumford Shah functional, as proposed by Chambolle, yielding a dynamical system grid.

We explore then different possibilities to link it to a grid of neurons, the processed value being directly the phase, the membrane voltage or a more complex neuron state evaluation, all this depending on the concidered neuron model (from integrate and fire to Hodgkin-Huxley) and encoding (with it's phase, membrane voltage or spiking rate..).

From this theoretical study and the related numerical experiment, we are able to compare these alternatives, while an original biologically inspired segmentation network emerges from our study.

#### How do high-level specifications of the brain relate to variational approaches?

Keywords : cortical maps, variational approaches, neural networks, Hopfield.

Participants : Thierry Viéville, Sandrine Chemla, Pierre Kornprobst.

This work was partially supported by the EC IP project FP6-015879, FACETS and the Fondation d'Entreprise EADS

High-level specification of how the brain represents and categorizes the causes of its sensory input allows to link "what is to be done" (perceptual task) with "how to do it" (neural network calculation). In [Oops!] , we described how the variational framework, which encountered a large success in modeling computer vision tasks, has some interesting relationships, at a mesoscopic scale, with computational neuroscience. We focus on cortical map computations such that "what is to be done" can be represented as a variational approach, i.e., an optimization problem defined over a continuous functional space. In particular, generalizing some existing results, we show how a general variational approach can be solved by an analog neural network with a given architecture and conversely. Numerical experiments are provided as an illustration of this general framework, which is a promising framework for modeling macro-behaviors in computational neuroscience.

We are now considering how to extend this formalism to the case of many connected cortical maps, i.e., coupled variational approaches.

#### Area MT and motion classification

Keywords : Action recognition, Biological motion recognition, Spiking networks, Motion analysis, V1, MT.

Participants : Maria-Jose Escobar, Pierre Kornprobst, Guillaume Masson [ Centre de Recherche en Neurosciences Cognitives, CNRS, FRE2098, 13402 Marseille, France ] , Thierry Viéville.

- biological motion
motion of a living character

We propose a bio-inspired MT model working in a fully spiking mode: Our MT layer receives spiking inputs coming from a previous spiking V1 layer. MT layer integrates this information to produce spikes as output. Interestingly, this spike to spike model allows us the study and modelling of some dynamics existing in V1 and MT, and due the causality of our cells representation it is also possible to integrate some top-down feedbacks. This model differs from existing ones such as e.g. [104] and [103] , whose generally have analogue entry and consider motion stimuli in a continuous regime (as plaids or gratings) discarding dynamic behaviors.

The first layer of the model is formed as an array of direction-selective V1 complex cells tuned for different speeds and directions of motion. Each V1 complex cell is modelled with a motion energy detector following
[98] . The second layer of the model corresponds to a spiking MT cell array. Each MT cell has as input the
spike trains of the V1 complex cells inside its receptive field. From the spike trains of MT cells a motion map of velocity distribution is built representing a sequence. In order to show the efficiency of these models, the motion maps here obtained are used in the biological motion
recognition task. We ran the experiments using two databases Giese and Weizmann, containing two (
*march, walk* ) and ten (
*e.g., march, jump, run* ) different classes, respectively. The results revealed that the motion map here proposed could be used as a reliable motion representation.

This work has been presented at [Oops!] , [Oops!] , and more details are available in [Oops!]

#### Virtual Retina: Large-scale retina simulator

Keywords : Retina, large-scale simulator, contrast gain control, spikes.

Participants : Adrien Wohrer, Pierre Kornprobst, Thierry Viéville.

A detailed retina model is proposed, that transforms a video sequence into a set of spike trains, as those emitted by retinal ganglion cells. It includes a linear model of filtering in the Outer Plexiform Layer (OPL), a contrast gain control mechanism modeling a non-linear feedback loop on bipolar cells, and a spike generation process modeling ganglion cells. A strength of the model is that each of its features can be associated to a precise physiological signification and location. The resulting retina model can simulate physiological recordings on mammalian retinas, including such non-linearities as cat Y cells, or contrast gain control.

This work has been concretized in a large-scale simulator software under CeCILL C licence,
*Virtual Retina* , that can emulate the spikes of up to 100,000 neurons. More recently, a mathematical study of the gain control loop present in the retina model has been undertaken. This simulator is described in details in
[Oops!] (see also the software section to learn more).

The dynamical system of contrast gain control has been also studied mathematically in [Oops!] .

#### Biological motion integration based on form cues

Keywords : Motion analysis, V1, MT.

Participants : émilien Tlapale, Guillaume Masson [ Institut de Neurosciences Cognitives de la Méditerranée, UMR 6193, CNRS, Marseille, France ] , Thierry Viéville, Pierre Kornprobst.

We present a model of motion integration and segmentation controlled by form cues in the primate visual cortex. Motion is diffused by a V1/MT like recurrent system biologically implemented by V1 pattern cells. Motion segmentation and asymetric center-surround effects came from the form information coming from the ventral pathway. The model is able to give results conforming to our percept for complex stimuli involving extrinsic junctions such as the Chopstick illusion.

This work has been presented at [Oops!] , and more details are available in [Oops!]