Biological Vision : A Source of Insights for Computer Science

New! Pdf of the presentation

A half day course presented by Simon Thorpe

October 12th, 2008, afternoon

European Conference on Computer Vision 2008, Marseille, France

Course Presentation

Despite David Marr's attempts to fuse biological and computer vision into a single discipline, researchers in the two areas have tended to work fairly independently. However, the basic problems faced by natural selection and computer vision scientists are very similar – both are looking for fast and reliable recognition and categorization of objects in complex cluttered scenes. For biology, the hardware constraints are fairly different, since neurons are intrinsically quite slow with firing rates that rarely exceed 100 pulses (spikes) per second, and conduction velocities within the brain are often only 1-2 m/s. Despite this, the primate brain is capable of determining whether a flashed image contains an animal or a human in as little as 100 ms. How is this possible?

In recent years, considerable progress has been made in understanding how this sort of very fast processing can be achieved, and much of this work has direct relevance to workers in computer vision. In particular, work on the neurobiological mechanisms underlying high-level vision has been progressing rapidly, and one of the aims of this course will be to bring researchers in computer vision up to date with the latest findings from biology.

Two particular ideas will be developed. One is the idea that the nervous system does not need to code information as absolute values, but rather by coding the relative values. One way to achieve this is to encode information by using the order in which neurons fire, rather than their firing rates. A second insight is the realization that fast image processing leaves little or no time for full image segmentation and that decisions about scene gist and the presence of certain key elements must be possible on the basis of a feed-forward pass.

There appear to be increasingly strong parallels between the way that the primate visual system performs this sort of analysis and the latest state of the art methods in scene categorization and object detection. One aim of this course will be to try and make these links explicit, with the aim of helping researchers in computer vision draw inspiration from strategies developed during the evolution of the visual system.

Course Structure

Introduction : Biological and Computer Vision

Performance of Biological Vision systems

• Temporal constraints on scene and object processing

• Parallelism in visual processing

• Feedforward vs Feedback based processing

Processing in the visual system

• Retina

• Primary visual cortex

• Processing in the ventral stream

Computational aspects of higher order vision

• Face processing

• Object processing

• Scene processing

Coding strategies in Biological Vision systems

• Rate coding

• Temporal coding

• Rank Order coding

Learning in Biological Vision systems

• Development of "feature-detectors"

• Spike-Time Dependent Plasticity

• Unsupervised learning vs supervised learning

Implications for Computer Vision

• Bioinspired Vision

• Bioinspired Hardware

Simon Thorpe

Centre de Recherche Cerveau & Cognition

Université Paul Sabatier

Toulouse, France

simon.thorpe@cerco.ups-tlse.fr

Simon Thorpe was born near London in 1956 and studied Physiology and Psychology at the University of Oxford where he obtained a first class honours degree in 1977. He stayed on in the Department of Experimental Psychology to study with Prof Edmund Rolls and obtained his doctorate in 1981. After a year as a post-doc in Max Cynader's lab at Dalhousie University in Canada, he moved to France to join Michel Imbert's group in Paris. He was recruited as a full-time research scientist by the CNRS in 1983 and in 1993 he moved to Toulouse as one of the founding members of the Brain and Cognition Research Centre (www.cerco.ups-tlse.fr). Over the years, his research has combined a wide range of experimental and theoretical approaches including single unit recording in monkeys and cats, functional imaging in humans and computer modelling. For more than 15 years his research interests have concentrated on trying to understand the remarkable speed with which the brain can process complex natural images. He has proposed a number of original ideas concerning the way that information is encoded by populations of spiking neurons, ideas that have been tested using SpikeNet, a bio-inspired image processing system that uses the order of firing as a code. This work led to the creation of a high-tech spin-off company, SpikeNet Technology, in 1999 (see www.spikenet-technology.com). Currently a Research Director with the CNRS, he leads the team working on Object and Scene Perception at the lab.

Some Selected Publications (see http://www.cerco.ups-tlse.fr/fr_vers/annuaire/simon_thorpe.htm)

Thorpe S, Fize D, Marlot C. 1996. Speed of processing in the human visual system. Nature 381:520-2

Thorpe S, Delorme A, VanRullen R. 2001. Spike-based strategies for rapid processing. Neural Netw 14:715-25.

Thorpe SJ, Gegenfurtner KR, Fabre-Thorpe M, Bulthoff HH. 2001. Detection of animals in natural images using far peripheral vision. Eur J Neurosci 14:869-76.

VanRullen R, Thorpe SJ. 2001. The time course of visual processing: from early perception to decision- making. J Cogn Neurosci 13:454-61.

VanRullen R, Thorpe SJ. 2001. Rate coding versus temporal order coding: what the retinal ganglion cells tell the visual cortex. Neural Comput 13:1255-83.

Thorpe SJ. 2002. Ultra-Rapid Scene Categorization with a Wave of Spikes. Biologically Motivated Computer Vision: 2nd International Workshop, BMCV 2002, pp. 1-15. Tübingen, Germany: Springer-Verlag

VanRullen R, Thorpe SJ. 2002. Surfing a spike wave down the ventral stream. Vision Res 42:2593-615.

Guyonneau R, Vanrullen R, Thorpe SJ. 2004. Temporal codes and sparse representations: A key to understanding rapid processing in the visual system. J Physiol Paris 98:487-97

Rousselet GA, Thorpe SJ, Fabre-Thorpe M. 2004. How parallel is visual processing in the ventral pathway? Trends Cogn Sci 8:363-70

Thorpe SJ, Guyonneau R, Guilbaud N, Allegraud JM, Vanrullen R. 2004. SpikeNet: Real-time visual processing with one spike per neuron. Neurocomputing 58-60:857-64

Bacon-Mace N, Mace MJ, Fabre-Thorpe M, Thorpe SJ. 2005. The time course of visual processing: Backward masking and natural scene categorisation. Vision Res 45:1459-69

Guyonneau R, Vanrullen R, Thorpe SJ. 2005. Neurons Tune to the Earliest Spikes Through STDP. Neural Comput 17:859-79

Guyonneau R, Kirchner H, Thorpe SJ. 2006. Animals roll around the clock: the rotation invariance of ultrarapid visual processing. J Vis 6:1008-17

Kirchner H, Thorpe SJ. 2006. Ultra-rapid object detection with saccadic eye movements: Visual processing speed revisited. Vision Res 46:1762-76

Masquelier T, Thorpe SJ. 2007. Unsupervised Learning of Visual Features through Spike Timing Dependent Plasticity. PLoS Comput Biol 3:e31

Masquelier T, Guyonneau R, Thorpe SJ. 2008. Spike timing dependent plasticity finds the start of repeating patterns in continuous spike trains. PLoS ONE 3:e1377