|
CLEF 2000: WORKSHOP ON CROSS-LANGUAGE
INFORMATION RETRIEVAL AND EVALUATION
21 - 22
September, Lisbon, Portugal
(immediately
following
ECDL 2000)
Description
The workshop was held over two days.
The first day was open to all those active or interested in the area of
Cross-Language Information Retrieval (CLIR). The morning was dedicated to the
presentation of invited papers and discussion sessions on CLIR research related
issues. The afternoon addressed the topic of CLIR system evaluation: the current
situation and future developments. The objective was to promote a discussion on
"What is needed to improve CLIR system performance"
and "How can evaluation campaigns assist in this". The goal
was to identify the actual contribution of evaluation to system
development and to determine what could or should be done in the future to stimulate progress.
The second day was restricted to participants in the CLEF 2000 evaluation campaign. The results of
this activity were presented and discussed.
Proceedings
The Proceedings of the two-day Workshop will be published by Springer in their
Lecture Notes for Computer Science series. The volume will include a record of the talks given on
Day 1 and of the experiments and results of the CLEF evaluation campaign presented in Day 2. All papers will be
revised and extended with respect to the preliminary draft version
as a result of the discussions during the Workshop.
Day 1 - 21 September 2000
|
9.15 - 9.30 |
Opening Session |
| |
Welcome on behalf of the
Local Coordinators, Nuno Mamede, IST-CSTC, Lisbon,
Portugal
Introduction to the Workshop, Carol Peters,
IEI-CNR, Pisa, Italy |
| |
|
|
9.30 - 11.00 |
Session 1: CLIR - What's
happening now? |
| |
Chair:
Felisa Verdejo Maillo, UNED, Madrid, Spain
|
|
9.30 - 9.50 |
An Overview of the Current
State-of-the-Art in Multilingual Information Access
Peter Schäuble, Eurospider, Switzerland (slides)
|
|
9.50 - 10.10 |
New Challenges for Cross-Language Information
Retrieval: multimedia data and the user experience
Gareth Jones, University of Exeter, UK
(slides) |
|
10.10 - 10.30 |
Multilingual Information
Access: The user's perspective
Douglas W. Oard, University of Maryland, USA (slides)
|
|
10.30 - 11.00 |
"What is now needed in CLIR research to
improve system performance?" |
| |
Position Papers:
Fredric C. Gey, UC Berkeley, USA
Tim Gollins, University of Sheffield, UK (slides)
and Open Discussion.
|
| |
|
|
11.00 - 11.30 |
Break |
| |
|
|
11.30 - 12.40 |
Session 2: Resources for
CLIR |
| |
Chair:
Michael Kluck, IZ Sozialwissenschaften, Bonn, Germany
|
|
11.30 - 11.50 |
Tools for Multilingual Information Access
Martin Braschler, Eurospider, Switzerland (slides)
|
|
11.50 - 12.10 |
May the Best Team Win: Language Resources in CLIR
Anne Diekema, Syracuse University, USA (slides) |
|
12.10 - 12.40 |
"What resources are really needed and how
should they be made available to the R&D
community?" |
| |
Position Paper:
Julio Gonzalo Arroyo, UNED, Madrid, Spain (slides)
and Open Discussion.
|
| |
|
|
12.40 - 14.00 |
Lunch |
| |
|
|
14.00 - 15.30 |
Session 3: Issues in CLIR
System Evaluation |
| |
Chair:
Martin Braschler, Eurospider, Switzerland
|
|
14.00 - 14.20 |
CLIR Evaluation at TREC
Donna Harman, National Institute for Standards
and Technology (NIST), USA (slides) |
|
14.20 - 14.40 |
CLIR Evaluation at NTCIR
Noriko Kando, National Institute of Informatics
(NII), Japan (slides)
|
|
14.40 - 15.00 |
CLEF: Cross-language System
Evaluation for European Languages
Carol Peters, National Research Council, Pisa,
Italy (slides) |
| |
|
|
15.20 - 15.50 |
Break |
| |
|
|
15.50 - 17.00 |
Panel on "CLIR
Evaluation in the Future" |
| |
Moderator:
Peter Schäuble, Eurospider, Switzerland
|
| |
Panelists:
Donna Harman, National Institute for Standards and Technology (NIST), USA
Noriko Kando, National Institute of Informatics
(NII), Japan (slides)
Jussi Karlgren, SICS, Sweden (slides)
Wessel Kraaij, TNO-TPD, The Netherlands
William Ogden, Computing Research Lab, New Mexico State University, USA
(slides)
|
| |
|
|
Day 2 - |
22 September 2000 |
| |
|
|
9.00 - 9.30 |
CLEF 2000: Overview of the Results |
| |
Martin Braschler, Eurospider, Switzerland (slides)
|
| |
|
|
9.30 - 10.15 |
Monolingual (non-English) Retrieval |
| |
Chair:
Julio Gonzalo, UNED, Madrid, Spain
|
| |
Presentations:
Isabelle Moulinier, West Group, USA;
Arjen P. de Vries, CWI, The Netherlands; Marcello Federico,
ITC-irst, Italy (slides),
and Discussion. |
| |
|
|
10.15 - 10.45 |
Break |
| |
|
|
10.45 - 12.45 |
Cross-Language Retrieval |
| |
Chair:
Donna Harman, NIST, USA
|
| |
Presentations:
Djoerd Hiemstra, University of Twente, The Netherlands;
Paul McNamee, Johns Hopkins University Applied Physics Lab, USA;
Turid Hedlund, Tampere University, Finland;
Bärbel Ripplinger, Univ. Saarlaandes, Germany;
N. Nassr, IRIT/SIG, Toulose, France;
Carlos G. Figuerola, Univ. Salamanca, Spain,
and Discussion. |
| |
|
|
12.45 - 14.00 |
Lunch |
| |
|
|
10.45 - 12.45 |
Domain-Specific Retrieval |
| |
Chair:
Michael Kluck, IZ-Bonn, Germany
|
| |
Presentations:
Norbert Goevert, Univ. Dortmund, Germany;
Fred Gey, UC Berkeley, USA,
and Discussion. |
| |
|
|
15.00 - 16.30 |
CLEF 2001: What should we keep? What
should we change? What should we add?
|
| |
Moderator:
Peter Schäuble, Eurospider, Switzerland
|
The draft papers outlining the approaches used by the participants in the CLEF 2000
evaluation campaign were printed in the
Working Notes
and distributed at the Workshop.
Workshop
Steering Committee
Martin Braschler, Eurospider, Switzerland
Julio Gonzalo Arroyo, UNED, Madrid, Spain
Donna Harman, National Institute for Standards and Technology (NIST), USA
Michael Hess, University of Zurich, Switzerland
Michael Kluck, IZ Sozialwissenschaften, Bonn, Germany
Carol Peters, IEI-CNR, Pisa, Italy
Peter Schäuble, Eurospider, Switzerland
Local
Coordinators
Caroline Hagège, CSTC, Lisbon, Portugal
Nuno Mamede, IST-CSTC, Lisbon, Portugal
Cross-Language
Evaluation Forum
The Workshop was
organised by the Cross-Language Evaluation Forum (CLEF).
To find out more about the activities of CLEF, please
click
here.
Contact
Carol
Peters - IEI-CNR
Area della Ricerca di San Cataldo, 56100 PISA (Italy)
Tel: +39 050 315 2897 - Fax: +39 050 315 2810
E-mail: carol@iei.pi.cnr.it
|