CLEF 2008 | How to Participate
order to participate in CLEF 2008 and have access to the test collections, a
registration form and (for most tracks) a data release form must be first compiled,
signed, and sent to Carol Peters at the address below . On receipt of the forms, you will be sent information on how
to download the data. Some
information is repeated on the different forms. The reason for this is that
they will be kept on different sites. All forms must be signed by a person authorised
by your organisation for such signatures (e.g. Department or Administrative
Head or similar).
Please note that previous CLEF participants must resubmit End-User Agreements each year in order to renew their authorisation to access the data covered by the agreement.
For copyright reasons, for some tracks, you will have to sign seperate data release forms. In these cases, these forms and data access is provided directly by the track coordinators. This is indicated on the track website.
Please compile the forms carefully, inserting all relevant information. However, first, read the task descriptions in the Agenda for CLEF 2008 to see which data collections you should request - this depends on the tasks you will be performing. Remember that access to the data will only be provided when we have received signed, original copies of the forms (not electronic or fax versions).
The Registration Form is your statement of intention to participate in CLEF 2008. All participating groups must submit this form. Please fill it in carefully, providing full details of the tasks in which you intend to participate and the languages you will be using. Registration remains open until 1 May 2008
The End-User Agreement must be submitted in two original signed copies. Even if you have already participated in a CLEF campaign and have access to some of the data, you must have this form (in two copies) compiled and signed by the appropriate person in your organisation in order to be authorised to continue to use the data for CLEF 2008. You should indicate on this form only the data sets you need, depending on the tasks that you intend to perform. On receipt of the two copies of this form, they will both be signed on behalf of CLEF, and one will be kept in our archives and the other will be returned to you.
The following collections are made available through the End User Agreement. Not all will be used in CLEF2008.
English: Data provided by The European Library; Copyright British Library (BL)
French: Data provided by The European Library; Copyright Bibliothèque Nationale de France (BnF)
German: Data provided by The European Library; Copyright Austrian National Library (ONB)
The CLEF document collection consists of SGML or XML-formatted documents from national newspapers, journals and news agencies from two distinct time periods (1994/1995 and 2002).
Dutch: NRC Handelsblad - 1994, 1995; Algemeen Dagblad - 1994, 1995
English: Los Angeles Times - 1994; The Herald -1995 (also in WSD versions)
Finnish: Aamulehti - 1995 (+small amount of 94 data);
French: Le Monde - 1994, 1995; SDA French Swiss news agency data - 1994, 1995
German: Frankfurter Rundschau - 1994, 1995; Der Spiegel - 1994; SDA German Swiss news agency data - 1994, 1995
Italian: La Stampa - 1994; SDA Italian Swiss news agency data - 1994, 1995
Portuguese: Público - 1994, 1995 (Portuguese newspaper); Folha de São Paulo (FSP) - 1994; 1995 (Brazilian newspaper)
Russian: Izvestia - 1995
Spanish: Agencia EFE S.A. (Spanish news agency data) - 1994, 1995
Swedish: Tidningarnas Telegrambyrå -1994, 1995
Persian: Hamshahri newspaper corpus - 1996-2002
2000, 2001, 2002
Basque: Egunkaria, 2000, 2001, 2002
Bulgarian: Sega - 2002; Standart - 2002; Novinar - 2002
Czech: Mlada fronta DNES - 2002; Lidové Noviny - 2002
English: Los Angeles Times - 2002 Hungarian: Magyar Hirlap - 2002
Cambridge Sociological Abstracts (CSA): Bibliographical data from the Sociological Abstracts Database GIRT4: German Indexing and Retrieval Test Database
RSSC: Russian Social Science Corpus
ISISS: Russian Economic and Social science bibliographic data
ImageCLEFmed Radiological Medical Database
IRMA Database of Medical Images
The following document collections are used in CLEF but access rights are provided via the relevant track website
Multilingual query-biased collections of web documents
A multilingual crawl of Wikipedia text data, used in QA@CLEF
A collection of photos and graphics from Wikipedia is used in ImageCLEF
Video See the VideoCLEF track website for the video data
When you have submitted the Registration and Data Release forms you will be given the necessary passwords to access the data and participation guidelines in the Workspace for Registered Participants.
The topic/question sets will be made accessible via the track websites from 1 April on. Topics will be prepared in a number of languages depending on the task. Please refer to track/task descriptions. Other languages may be added depending on demand.
We gratefully acknowledge the support of all the data providers and copyright holders, and in particular:
The Los Angeles Times, for the American-English news data collections;
SMG Newspapers (The Herald) for the British-English news data collection
Le Monde S.A. and ELDA: Evaluations and Language resources Distribution Agency, for the French news data.
Frankfurter Rundschau, Druck und Verlagshaus Frankfurt am Main; Der Spiegel, Spiegel Verlag, Hamburg, for the German newspaper collections.
InformationsZentrum Sozialwissen-schaften, Bonn, for the GIRT social science database
SocioNet system for the Russian Social Science Corpora
Institute of Scientific Information for Social Sciences of the Russian Academy of Science (ISISS RAS) for the ISISS database
Hypersystems Srl, Torino and La Stampa, for the Italian news data.
Agencia EFE S.A. for the Spanish news data.
NRC Handelsblad, Algemeen Dagblad and PCM Landelijke dagbladen/Het Parool for the Dutch newspaper data.
Aamulehti Oyj and Sanoma Osakeyhtiö for the Finnish newspaper data
Russika-Izvestia for the Russian newspaper data
Hamshahri newspaper and DBRG, Univ. Tehran, for the Persian newspaper data
Público, Portugal, and Linguateca for the Portuguese (PT) newspaper collection
Folha de São Paulo, Brazil, and Linguateca for the Portuguese (BR) newspaper collection
Tidningarnas Telegrambyrå (TT) SE-105 12 Stockholm, Sweden for the Swedish newspaper data
Schweizerische Depeschenagentur, Switzerland, for the French, German and Italian Swiss news agency data
Ringier Kiadoi Rt. [Ringier Publishing Inc.].and the Research Institute for Linguistics, Hungarian Acad. Sci. for the Hungarian newspaper documents
Sega AD, Sofia; Standart Nyuz AD, Sofia, Novinar OD, Sofia, and the BulTreeBank Project, Linguistic Modelling Laboratory, IPP, Bulgarian Acad. Sci, for the Bulgarian newspaper documents
Mafra a.s. and Lidové Noviny a.s. for the Czech newspaper data
The British Library, Bibliothèque Nationale de France and Deutsche Nationalbibliothek for the library catalog records forming part of The European Library (TEL)
University and University Hospitals, Geneva, Switzerland and Oregon Health and Science University for the ImageCLEFmed Radiological Medical Database
Aachen University of Technology (RWTH), Germany for the IRMA database of annotated medical images.
The Radiology Dept. of the University Hospitals of Geneva for the Casimage database and the PEIR (Pathology Education Image Resource) for the images and the HEAL (Health Education Assets Library) for the Annotation of the Peir dataset.
Mallinkrodt Institue of Radiology for permission to use their nuclear medicine teaching file
University of Basel's Pathopic project for their Pathology teaching file
Michael Grubinger, administrator of the IAPR Image Benchmark, Clement Leung who initiated and supervised the IAPR Image Benchmark Project, and André Kiwitz, the Managing Director of Viventura for granting access to the image database and the raw image annotations of the tour guides.
USC Shoah Foundation Institute, and IBM (English) and The Johns Hopkins University Center for Language and Speech Processing (Czech) for the speech transcriptions
AFP Agence France Presse for the English, French and Arabic newswire data
used in the INFILE track
Without their help, this evaluation activity would be impossible.
Please mail all forms by express or priority post to the address below. Access to the data will be provided only on receipt of the relevant forms.
Carol Peters - CLEF Coordinator
Area della Ricerca CNR
Via Moruzzi, 1 - 56124 Pisa, Italy
Fax: +39 050 315 3464/2810 Tel: +39 050 315 2897