Working Notes for the CLEF 2003 Workshop
21-22 August,
Trondheim, Norway


edited by Carol Peters



         Carol Peters

PART I: CLEF 2003 Core Tracks

Cross-Language and More

Report on CLEF-2003 Multilingual Tracks

         Jacques Savoy


Multilingual Experiments of UTA at CLEF 2003: The Impact of Different Merging Strategies and Word Normalizing Tools

         Eija Airio, Heikki Keskustalo, Turid Hedlund and Ari Pirkola


JHU/APL Experiments in Tokenization and Non-Word Translation

         Paul McNamee and James Mayfield


Report on CLEF-2003 Experiments: Two Ways of Extracting Multilingual Resources from Corpora

         Nicola Cancedda, Hervé Déjean, Éric Gaussier, Jean-Michel Renders and Alexei Vinokourov


Combining Query Translation and Document Translation in Cross-Language Retrieval

         Aitao Chen and Fredric C. Gey


Cross-Language Experiments with IR-n System

         Fernando Llopis and Rafael Muñoz


CONTROL: CLEF-2003 with Open, Transparent Resources Off-Line

         Monica Rogati and Yiming Yang


ITC-irst at CLEF 2003: Monolingual, Bilingual and Multilingual Information Retrieval

         Nicola Bertoldi and Marcello Federico


The University of Amsterdam at CLEF 2003

         Jaap Kamps, Christof Monz, Maarten de Rijke and Börkur Sigurbjörnsson


Multilingual Retrieval Experiments with MIMOR at the University of Hildesheim

René Hackl, Ralph Kölle, Thomas Mandl, Alexandra Ploedt, Jan-Hendrik Scheufen and Christa Womser-Hacker


The LIC2M’s CLEF 2003 System

Romaric Besançon, Gaël de Chalendar, Olivier Ferret, Christian Fluhr, Olivier Mesnard and Hubert Naets


Report on CLEF 2003 Experiments at UB

         Miguel E. Ruiz


SINAI at CLEF 2003: Decompounding and Merging

Fernando Martínez-Santiago, Arturo Montejo-Ráez, L.Alfonso Ureña-López, Manuel Carlos Díaz-Galiano


Merging Results by Using Predicted Retrieval Effectiveness

         Wen-Cheng Lin and Hsin-Hsi Chen


Evaluation of MIRACLE Approach Results for CLEF 2003

José Luis Martínez, Julio Villena Román, Jorge Fombella, Ana G. Serrano, Alberto Ruiz, Paloma Martínez, José M. Goñi and José C. González


Clairvoyance CLEF-2003 Experiments

         Yan Qu, Greg Grefenstette and David A. Evans
Two Stages Refinement of Query Translation for Pivot Language Approach to Cross Lingual Information Retrieval: A Trial at CLEF 2003
         Kazuaki Kishida and Noriko Kando


Regular Sound Changes for Cross-language Information Retrieval

Michael P Oakes and Souvik Banerjee


EXETER at CLEF 2003: Experiments with Machine Translation for Monolingual, Bilingual and Multilingual Retrieval

Adenike M. Lam-Adesina and Gareth J. F.Jones


Multilingual Retrieval in Twente

         Franciska de Jong, Dennis Reidsma and Djoerd Hiemstra

Monolingual Experiments

Lexical and Algorithmic Stemming Compared for 9 European Languages with Hummingbird SearchServerTM  at CLEF 2003

         Stephen Tomlinson


Océ at CLEF 2003

         Roel Brand, Marvin Brünner, Samuel Driessen, Pascha Iljin and Jakob Klok


Comparing Weighting Models for Monolingual Information Retrieval

         Gianni Amati, Claudio Carpineto and Giovanni Romano


Report on CLEF-2003 Monolingual Tracks: Fusion of Probabilistic Models for Effective Monolingual Retrieval

         Jacques Savoy


Pruning Texts with NLP and Expanding Queries with an Ontology: TagSearch

         Gil Francopoulo


Selective Compound Splitting of Swedish Queries for Boolean Combinations of Truncated Terms

         Rickard Cöster, Magnus Sahlgren and Jussi Karlgren


COLE Experiments at CLEF 2003: Spanish Monolingual Track

         Jesús Villares, Miguel A. Alonso and Francisco J. Ribadas


Ricoh at CLEF 2003

         Yuichi Kojima, Hideo Itoh, Hiroko Mano and Yasushi Ogawa


The University of Padova at CLEF 2003: Experiments to Evaluate Probabilistic Models for Automatic Stemmer Generation and Query Word Translation

         Giorgio Maria Di Nunzio, Nicola Ferro, Massimo Melucci and Nicola Orio


Clustering and Retrieval of Spanish News Documents Using Self Organizing Maps

Javier Fernández, Ricardo Mones, Irene Díaz, José Ranilla and Elías F. Combarro


MediaLab at CLEF-2003: Using Keyword Disambiguation

Peter van der Weerd

Mainly Domain-Specific Information Retrieval

UC Berkeley at CLEF 2003 – Russian Language Experiments and Domain-Specific Cross-Language Retrieval

         Vivine Petras, Natalia Perelman and Fredric Gey


Data Compression Approach to Monolingual GIRT Task: an Agnostic Point of View

         Daniela Alderuccio, Luciana Bordoni and Vittorio Loreto


University of Hagen at CLEF 2003: Natural Language Access to the GIRT4 Data

         Johannes Leveling

PART II: CLEF 2003 Additional Tracks

Interactive Track

The CLEF 2003 Interactive Track

Douglas W. Oard and Julio Gonzalo,


iCLEF 2003 at Maryland: Translation Selection and Document Selection

Bonnie J. Dorr, Daqing He, Jun Luo, Douglas W. Oard, Richard Schwartz, Jianqiang Wang and David Zajic


UNED at iCLEF 2003: Searching Cross-Language Summaries

         Fernando López-Ostenero, Julio Gonzalo and Felisa Verdejo

Comparing Syntactic Semantic Patterns and Passages in Interactive Cross Language Information Access (iCLEF at University of Alicante)

         Borja Navarro, Fernando Llopis and Miguel Ángel Varó

Question Answering at CLEF

The Multiple Language Question Answering Track at CLEF 2003

Bernardo Magnini, Simone Romagnoli, Alessandro Vallin, Jesús Herrera, Anselmo Peñas, Vìctor Peinado, Felisa Verdejo and Maarten de Rijke


Creating the DISEQuA Corpus: a Test Set for Multilingual Question Answering

Bernardo Magnini, Simone Romagnoli, Alessandro Vallin, Jesús Herrera, Anselmo Peñas, Vìctor Peinado, Felisa Verdejo and Maarten de Rijke


Bridging Languages for Question Answering: DIOGENE at CLEF 2003

         Matteo Negri, Hristo Tanev and Bernardo Magnini


Cross-Language Question Answering at the USC Information Sciences Institute

Abdessamad Echihabi, Douglas W. Oard, Daniel Marcu and Ulf Hermjakob


The University of Amsterdam at QA@CLEF2003

         Valentin Jijkoun, Gilad Mishne and Maarten de Rijke


Cross Lingual QA: A Modular Baseline

         Lucian Vlad Lita, Monica Rogati, Jaime Carbonell


Question Answering in Spanish

         José L. Vicedo, Ruben Izquierdo, Fernando Llopis and Rafael Muñoz


Quantum, a French/English Cross-Language Question Answering System

         Luc Plamondon and George Foster


A Cross-Language Question/Answering-System for German and English

         Günter Neumann and Bogdan Sacaleanu


Cross-Language French-English Question Answering using the DLT System at CLEF 2003

Richard F. E. Sutcliffe, Igal Gabbay and Aoife O'Gorman


The CLEF 2003 Cross Language Image Retrieval Task

         Paul Clough and Mark Sanderson


Sheffield at ImageCLEF 2003

         Paul Clough and Mark Sanderson


Foreign Name Backward Transliteration in Chinese-English Cross-Language Image Retrieval

         Wen-Cheng Lin, Changhua Yang and Hsin-Hsi Chen


MIRACLE Evaluation of Results for ImageCLEF 2003

Julio Villena Román, José Luis Martínez, Jorge Fombella, Ana G. Serrano, Alberto Ruiz, Paloma Martínez, José M. Goñi and José C. González


Scene of Crime Information System: Playing at St. Andrews

         Bogdan Vrusias, Mariam Tariq and Lee Gillam

Cross-Language Spoken Document Retrieval

The CLEF 2003 Cross-Language Spoken Document Retrieval Track

         Marcello Federico and Gareth Jones


Spoken Document Retrieval Experiments with IR-n System

         Fernando Llopis and Patricio Martínez-Barco


EXETER at CLEF 2003: Cross-Language Spoken Document Retrieval Experiments

         Adenike Lam-Adesina and Gareth Jones

PART III: Evaluation Initiatives and Issues

Proper Names in the Multilingual CLEF Topic Set

         Thomas Mandl and Christa Womser-Hacker


Evaluation of Information Access Technologies at NTCIR Workshop

    Noriko Kando

Appendix A: Question Answering Run Statistics

Monolingual results

Bilingual results

Appendix B: Results for Core Tracks


List of run characteristics

Overview graphs





Bilingual to German (DE) target language

Bilingual to English (EN) target language

Bilingual to Spanish (ES) target language

Bilingual to Italian (IT) target language

Bilingual to Dutch (NL) target language

Bilingual to Russian (RU) target language


Monolingual German (DE)

Monolingual English (EN)

Monolingual Spanish (ES)

Monolingual Finnish (FI)

Monolingual French (FR)

Monolingual Italian (IT)

Monolingual Dutch (NL)

Monolingual Russian (RU)

Monolingual Swedish (SV)


GIRT to German (DE) target language

GIRT to English (EN) target language