CLEF AGENDA for 2000

Task Description

There were three main evaluation tracks in CLEF 2000. There was also a special sub-task for domain-specific cross-language evaluation.

  1. Multilingual Information Retrieval

    The main task of CLEF 2000 required searching a multilingual document collection, containing English, German, French, and Italian texts, for relevant documents. Using a selected topic language, the goal was to retrieve documents for all languages in the collection, listing the results in a merged, ranked list.

    The official topic languages for CLEF 2000 were Dutch, English, French, German, Italian, Spanish, Swedish, Finnish. 

  2. Bilingual Information Retrieval

    A cross-language task was provided in which the query language could be Dutch, French, German, Italian, Spanish, Swedish or Finnish and the target document collection was English. 

  3. Monolingual (non-English) Information Retrieval

    Monolingual system testing was offered for French, German and Italian in CLEF 2000.

  4. Special task GIRT

    In addition to three main tasks, there was a special task for CLEF-2000, consisting of a data collection from a vertical domain (social sciences); this collection contains now nearly 80,000 documents in a structured database. This task will have 25 questions, created in German but translated to all languages.
    Groups could run these questions: 1. either as monolingual task against the 80,000 documents of this database (GIRT) or against both GIRT and the German newspaper articles; 2. or as multilingual task using the translated elements of GIRT. 


The CLEF test collection for 2000 consisted of SGML formatted newspaper documents for English, French, German and Italian, from the same time period. 


English: NIST, Gaithersburg MD, USA (Ellen Voorhees)
French: University of Zurich, Switzerland (Michael Hess)
German: IZ Sozialwissenschaften (Social Science Information Centre), Bonn, in co-operation with University of Hildesheim, Germany (Michael Kluck, Jürgen Krause, Christa Womser-Hacker)
Italian: IEI-CNR, Pisa, Italy (Carol Peters)
Spanish: IEEC-UNED, Madrid, Spain (Felisa Verdejo, Julio Gonzalo)
Repositories (Test Collections and Resources): Eurospider, Switzerland (Peter Schäuble, Martin Braschler)

Important Dates

Data Release - 1 April 2000
Topic Release - 8 May 2000
Receipt of results from participants - 1 July 2000
Release of relevance assessments and individual results - 15 August 2000
Submission of paper for Working Notes - 5 September 2000
Workshop - 21-22 September 2000