Resources & Links

This page contains pointers to helpful resources and references. Please help us extend and maintain this list by sending your feedback.

Evaluation Forums


Cross-Language Evaluation Forum


Text Retrieval Conference


NII-NACSIS Test Collection for IR Systems


INitiative for the Evaluation of XML Retrieval

FIRE Forum for Information Retrieval Evaluation

Research Programmes


European Network of Excellence in Human Languages Technologies


Translingual Information Detection, Extraction and Summarization (DARPA)



Evaluations and Language Resources Distribution Association


Linguistic Data Consortium


United Nations Official Documents Online - consists of 800,000 searchable parallel documents in Arabic, Chinese, English, French, Russian and Spanish, primarily in PDF and / or MS Word format.  Resource Shelf  summarizes the main features and other UN resources.


Free retrieval system from NIST, complete with (simplistic) German and French stemmers

SDA data
German/French/Italian 1988-90
Please consult the README.txt file.

Training collection used for the TREC6-8 CLIR tracks (password-protected. Please consult the CLEF administration.)

Web of Online-Dictionaries

online dictionaries for 100s of languages

Altavista Babelfish

(Systran-powered) online machine translation from Altavista


On-line translation tools


On-line translation tools


On-line translation tools

Reverso Online

On-line translation tools

Selected References on Evaluation
(for CLEF papers, see our Website under Publications)

Harman, D. K. (1993): Overview of the First Text REtrieval Conference (TREC-1). In The First Text REtrieval Conference (TREC-1), NIST Special Publication 500-207.

Harman, D. K. (1995): The TREC Conferences. In Sparck Jones, K. and Willett, P. (Eds.): Readings in Information Retrieval, Morgan Kaufmann Publishers.

Kando, N., Kuriyama, K., Nozue, T., Eguchi, K., Kato, H., Hidaka, S., and Adachi, J. (1999): The NTCIR Workshop: The First Evaluation Workshop on Japanese Text Retrieval and Cross-Language Information Retrieval. In Proceedings of the 4th International Workshop on Information Retrieval with Asian Languages.

Schäuble, P., and Sheridan, P. (1998): Cross-Language Information Retrieval (CLIR) Track Overview. In Proceedings of the Sixth Text Retrieval Conference (TREC6). NIST Special Publication 500-240.

Sparck Jones K., and Van Rijsbergen C. (1975): Report on the Need for and Provision of an "Ideal" Information Retrieval Test Collection. In British Library Research and Development Report 5266, Computer Laboratory, University of Cambridge.

Voorhees, E. (1998): Variations in Relevance Judgments and the Measurement of Retrieval Effectiveness. In Proceedings of the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval.

Additions and comments welcome!