## Section: New Results

### Effective 3D global visibility, theory and applications

Participants : Guillaume Batog, Hazel Everett, Xavier Goaoc, Sylvain Lazard, Sylvain Petitjean, Linqiao Zhang.

In recent years, our activity in the area of 3D visibility focused on
three main directions: (i) the computation and complexity analysis of
the *3D visibility complex,* (ii) the computation and complexity
analysis of the boundary of shadows cast by area light sources, and
(iii) the study of some fundamental questions in geometric transversal
theory.

The 3D visibility complex is a partition of the space of rays according to
visibility. The questions we consider are two-fold. On one hand,
we study its size, both experimentally and theoretically. This size
is reflected in the number of unobstructed line segments tangent to
four objects of the scene; while this number can be (n^{4}) in
unstructured scenes, its behavior for more *realistic* input is
still not understood. On the other hand, we work on the computation
of the *3D Visibility skeleton,* a substructure of the
visibility complex. A few years ago we presented a worst-case near-optimal sweep-plane algorithm for
computing this structure in a restrictive setting [3] . Over the last
years, we have been working on an efficient and reliable
implementation of that algorithm, and on the development of an algorithm in a more general setting.

Shadows play a central role in human perception and a wide variety of approaches have been considered for simulating and rendering them. Unfortunately, computing realistic shadows efficiently is a very difficult problem, particularly in the case of non-point light sources, due to the complicated internal structure that such shadows may have. The only current solution for that problem goes through a discretization of area light sources by many point light sources; shadows cannot be certified with this approach and their structure is lost. As a first approximation, one can compute the main shadow boundaries, that is the boundary between the regions in full-light, penumbra, and umbra. Specifically, a point is in the umbra if it does not see any part of any light source; it is in full light if it sees entirely all the light sources; otherwise, it is in the penumbra. These boundaries are traced out by one-parameter families of segments that belong to the 3D Visibility skeleton. On one hand, we try to estimate the complexity of these shadow boundaries, both in theory and in practice. On the other hand, we work on applying our methods for computing the visibility skeleton to this particular problem.

The study of fundamental properties of sets of lines, that is,
*line geometry,* underlies many algorithmic questions including
3D visibility computations. A few years ago, we generalized Helly's
classical theorem to sets of line transversals to disjoint unit balls
in [5] . This
theorem builds on two types of results: a local Helly-type theorem,
also called a *pinning theorem,* and a bound on the topological
complexity of the space of line transversals, obtained by counting
so-called *geometric permutations* . Our activity focuses on
extending both types of results.

#### Visibility complex and skeleton

This year, our PhD student Linqiao Zhang,
co-supervised with S. Whitesides at McGill University, defended her Ph.D. thesis entitled
*On the three-dimensional visibility skeleton: implementation and
analysis* [11] . She developed a first
implementation of our sweep-plane algorithm that computes robustly the
vertices of the visibility skeleton of a set of convex polyhedra in
generic position. This implementation is a key element of a prototype we developed for
computing limits of umbra [24] (see
below); it also allowed us to study experimentally the size of the 3D
visibility skeleton
in a random setting and, in particular, to show that the
constant involved in the asymptotic complexity is small
[11] [66] .
Related to this work, we studied various predicates, arising in
three-dimensional visibility, concerning line transversals to lines
and segments in 3D. In particular, we computed the degrees of
standard methods of evaluating these predicates. We showed that the
degrees of some of these methods are surprisingly high (up to 168),
which may explain why computing line transversals with
finite-precision floating-point arithmetic is prone to error
[11] ; this work also published this year
in the journal *Computational Geometry, Theory and
Applications* [16] .

We also obtained a new result on the complexity of the set free lines
and free lines segments tangent to balls in 3D. In particular, we
proved that, in the presence of n disjoint unit balls, the set of
free line segment, that is the so-called visibility complex, has
complexity (n^{4}) .
This result is very surprising because it was natural to conjecture that the
visibility complex of fat objects of similar size had a lower worst-case complexity than that for thin
triangles. Our result settles negatively this conjecture, and shows exactly the opposite, that is,
that fatness and similarity, alone, do not reduce the worst-case complexity of that structure.
This result was submitted to the *Symposium on Computational
Geometry* in December 2009 [35] .

In the context of the PhD thesis of Guillaume Batog, we are working on an invariant-based method for designing efficient evaluation strategies for predicates. In a nutshell, the idea is to find a group action on the input space of a predicate with two properties: it has few orbits, and any two inputs in the same orbit are equivalent for the predicate. The invariants of that action are known, under certain conditions, to distinguish the various orbits, and can then be used to decide the predicate. This year, we unfolded this approach for predicates on line systems that arise in visibility computations, for instance deciding the number of lines intersecting four given lines. A communication is in preparation.

#### Shadow computations

We published this year two results on the umbra and penumbra cast by non-trivial light sources.

First, we published the first non-trivial bounds on the size
of the umbra region. These result show that the umbra can be
surprisingly complicated, even in the
presence of disjoint fat obstacles; for instance, we proved that the umbra cast
on a plane by a segment light source in the presence of two fat
polytopes of size n , can have up to (n) connected
components in the worst case. These results were published in the
journal *Computational Geometry, Theory and
Applications* [13] .

We also developed, over the last years, a new method for the exact
computation of shadow boundaries. The associated prototype software
is, up to our knowledge, the first implementation that computes
exactly such shadow boundaries of polygonal light sources in the
presence of polyhedral obstacles for nontrivial scenes (*e.g.,*
50 polytopes with 1 500 vertices). Using this implementation, we
observed experimentally a significant gap between the size of the
visibility skeleton and the, much smaller, complexity of the shadow
boundaries. We presented these results at the *European Workshop
on Computational Geometry* [24] .

#### Pinning and line transversals

In the last few years, we strived to understand the combinatorial
properties of so-called *isolated line transversals,* also
called *pinnings* : these lines meet every member in a family of
objects, but they lose this property when subject to any arbitrarily
small perturbation. We obtained a *basis theorem* when the
objects are disjoint balls in in
2006 [5] : any pinning of a line
by disjoint balls in contains a pinning of size at
most 2d-1 . We strengthened this result in three ways. First, we
proved that this constant, 2d-1, is best possible for all
dimension, a result obtained in 2008. In 2009, we presented this
result at the *Eurocomb*
conference [21] and submitted a
complete version for journal publication. Then, we extended this
result to pinnings of a line by possibly intersecting smooth convex
sets [37] .
Last, in 2008 we extended this
result to pinnings by polyhedra in . This year, we
added to this extension a complete description of minimal pinning
configurations; these results were submitted to the *Symposium on Computational
Geometry* in December 2009 [29] .

An order in which a line can intersect a collection of geometric
objects is called a *geometric permutation* of that
collection. When the objects are disjoint balls, the geometric
permutations count the connected components of the space of line
transversals ; more generally, it gives a lower bound on its
topological complexity. Substantial effort has been devoted to
understanding how the geometry of the objects constrains the number
of geometric permutations; the asymptotic behavior of the maximum
number of geometric permutations of n disjoint convex sets in
has remained an open problem for two decades. We have
been working on a new approach to this question. The idea is to
analyze how the knowledge of a bound for constant-size problems (say, knowing
that the maximum number of geometric permutations for 10 objects is
at most 24) helps bounding the asymptotic behavior for n
objects. This year, we made a first step in this direction: we
delineated the main growth rates (constant, polynomial, exponential)
through a generalization of Sauer's Lemma for *shatter
functions* of set systems.