. -->
S u b t h e m e 
P N A 4 . 1
Content-Based Image Retrieval and Image Understanding
 


 

The research described below is carried out by the subtheme  Content-Based Image Retrieval  within the CWI research theme "Signals and Images"

Subtheme coordinator:  Eric Pauwels

There is a pressing need for efficient information management and mining of the huge quantities of image data that are routinely being amassed. These data are potentially an extremely valuable source of information, but their value is limited unless they can be effectively explored and retrieved, and it is becoming increasingly clear that in order to be efficient, data mining needs to be based on semantics. However, the extraction of semantically rich meta-data from the computationally accessible "low-level" features poses tremendous scientific challenges.

Indeed, retrieval of images based exclusively on a number of fixed and global features is often too crude to produce satisfactory results. What is required is some form of adaptive (i.e. data-driven) description that captures whatever is salient in each individual image. Furthermore, the user should be able to provide the system with relevance feedback to expedite the navigation-process through a large database. This means that the search-engine should be equipped with an inference-engine that is able to observe the user-interaction and learn from it.

Given the above, we feel that there is both a need and an opportunity to systematically incorporate machine learning into an integrated approach to image data mining. Enriching image databases with additional layers of automatically generated semantic meta-data as well as with artificial intelligence to reason about these (meta)data, seems the only conceivable way forward.

 
 
International Research Projects 

Our research group is involved in the following European research projects:  
 
FOUNDIT   (Coordinator) Content-based image retrieval (FP5)
 
MUSCLE   (Coordinator) Multimedia Understanding through Semantics, Computation and Learning (FP6 Network of Excellence)
 
BioVision Roadmap for the future of biometrics in Europe through to 2010 (FP5)
 
BioSecure Biometrics for secure Authentication (FP6 Network of Excellence)
 
Europhlukes Computer-assisted Photo-identification of Ceteceans (FP5)
 

 
 
Research Interests 

 

Clustering for Image Segmentation

Eric Pauwels,
Joint work with Greet Frederix, Katholieke Universiteit Leuven, Belgium.

When characterising the visual content of natural images, global descriptions are often to coarse to be really useful.  In order to obtain a more flexible description one needs to locate regions of interest that can be characterised separately.  For this reason, we are working on robust segmentation algorithms that are based on non-parametric clustering.

 

Multimodal integration

Eric Pauwels, Mark Huiskes

To achieve robust performance in multimedia applications one is faced with the problem of how to integrate various sources of data resulting from different perceptual modalities. These include general-purpose modalities such as vision, audio/speech and text, and a wide variety of special-purpose sensors (motion detectors, pressure, IR etc). We are investigating statistical approaches to the integration problem, particularly aimed at exploiting both redundancy and non-accidentality; additionally we are working on bio-inspired systems using layered and agent-based approaches. Applications are in sensor networks, multimedia understanding and multimodal interface design.

 

Learning, visualisation and simulation in CBIR

Eric Pauwels, Mark Huiskes
Joint work with Geert Caenen, Katholieke Universiteit Leuven, Belgium.

As image interpretation is user- and task-dependent, relevance feedback is required for reaching an adaptive and interactive understanding of a user's wishes. Based on a detailed analysis of the special structure of the relevance feedback learning problem, taking into account for instance partial relevance and feature saliency, we are designing new methods for statistical inference to interpret the user feedback data. In addition we are analyzing the performance of relevance feedback inference methods by combining a new approach to feature simulation with realistic search scenarios.

 

Image segmentation and perceptual grouping

Eric Pauwels, Mark Huiskes

To reach an understanding of images in terms of their constituent parts, low-level segmentation based on simple homogeneity and connectedness is generally not sufficient. Additional mechanisms such as the famous Gestalt principles (e.g. similarity and goodness-of-curve) and rules of saliency and figure-ground organization play an important intermediary role in going from signal to meaning. To get a handle on the problem of integrating the diverse grouping principles, we are currently investigating the feasibility of various agent-based solutions.

 

Interactive feature annotation and design

Eric Pauwels, Mark Huiskes, Ben Schouten

For meaningful searching and browsing in image and video collections, accurate representations of image content in terms of a wide range of features are crucial. Automatic generation of features shows great promise, but perfection yet remains a distant dream. Taking the results of automatic methods as a starting point, we are designing interfaces for both interactive feature annotation and interactive feature design. Feature annotation is aimed at conveniently enhancing the automatically obtained features using for instance automatic speech recognition technology and drag-and-drop interfaces. Interactive feature design uses various mining and learning approaches as a means to explore feature spaces for adaptive feature creation.

 

Numerical Methods for Image Processing

Paul de Zeeuw

An image processing method is devised which involves the concepts of image transforms, partial differential equations and multiresolution all in one. The resulting method has implications for image fusion, segmentation and denoising.

 

Computer-Aided Photo-Identification of Cetaceans

Eric Pauwels, Elena Ranguelova, Adri Steenbeek

Individual identification of animals is an important issue in marine biology and zoology in general. The method of photo-identification relies on the uniqueness of the physical characteristics of each individual and is a tedious task given the large size of the photographic catalogues. The aim is to develop algorithms for segmentation of anatomical structures, extraction of shape features and design of feature matching algorithms invariant under illumination changes and geometric distortions. Semi-automatic segmentation method based on watershed transformation combined with regions of interest extraction and feature description are being proposed for photo-identification of humpback whales.
 
More information...

Updated on  13 April 2004