|
CLEF
2000: WORKSHOP ON CROSS-LANGUAGE INFORMATION RETRIEVAL AND
EVALUATION
21 - 22 September,
Lisbon, Portugal
(immediately following ECDL 2000)
Description
The workshop was held over two days.
The first day was open to all those active or interested in the area of
Cross-Language Information Retrieval (CLIR). The morning was dedicated to
the presentation of invited papers and discussion sessions on CLIR
research related issues. The afternoon addressed the topic of CLIR system
evaluation: the current situation and future developments. The objective
was to promote a discussion on "What is needed to improve CLIR system
performance" and "How can evaluation campaigns assist in this". The goal
was to identify the actual contribution of evaluation to system
development and to determine what could or should be done in the future to
stimulate progress.
The second day was restricted to participants in the CLEF 2000
evaluation campaign. The results of this activity were presented and
discussed.
Proceedings
The
Proceedings of the two-day
Workshop have been published by Springer in their Lecture Notes for Computer
Science series. The volume includes a record of the talks given on Day
1 and of the experiments and results of the CLEF evaluation campaign
presented in Day 2. All papers have been revised and extended with respect
to the preliminary draft version as a result of the discussions during the
Workshop.
Day 1 - 21 September
2000
|
9.15 -
9.30 |
Opening
Session |
|
|
Welcome on behalf
of the Local Coordinators, Nuno Mamede, IST-CSTC, Lisbon, Portugal
Introduction to the Workshop, Carol Peters, IEI-CNR, Pisa,
Italy |
|
|
|
|
9.30 -
11.00 |
Session 1: CLIR
- What's happening now? |
|
|
Chair:
Felisa Verdejo Maillo, UNED, Madrid,
Spain |
|
9.30 -
9.50 |
An Overview of the
Current State-of-the-Art in Multilingual Information Access
Peter
Schäuble, Eurospider, Switzerland (slides)
|
|
9.50 -
10.10 |
New Challenges for
Cross-Language Information Retrieval: multimedia data and the user
experience
Gareth Jones, University of Exeter, UK (slides) |
|
10.10 -
10.30 |
Multilingual
Information Access: The user's perspective
Douglas W. Oard,
University of Maryland, USA (slides)
|
|
10.30 -
11.00 |
"What is now needed
in CLIR research to improve system performance?" |
|
|
Position
Papers: Fredric C. Gey, UC
Berkeley, USA
Tim Gollins, University of Sheffield, UK (slides)
and Open Discussion. |
|
|
|
|
11.00 -
11.30 |
Break
|
|
|
|
|
11.30 -
12.40 |
Session 2:
Resources for CLIR |
|
|
Chair:
Michael Kluck, IZ Sozialwissenschaften,
Bonn, Germany |
|
11.30 -
11.50 |
Tools for
Multilingual Information Access
Martin Braschler, Eurospider,
Switzerland (slides)
|
|
11.50 -
12.10 |
May the Best Team
Win: Language Resources in CLIR
Anne Diekema, Syracuse
University, USA (slides)
|
|
12.10 -
12.40 |
"What resources are
really needed and how should they be made available to the R&D
community?" |
|
|
Position
Paper: Julio Gonzalo Arroyo,
UNED, Madrid, Spain (slides)
and Open Discussion. |
|
|
|
|
12.40 -
14.00 |
Lunch
|
|
|
|
|
14.00 -
15.30 |
Session 3:
Issues in CLIR System Evaluation |
|
|
Chair:
Martin Braschler, Eurospider,
Switzerland |
|
14.00 -
14.20 |
CLIR Evaluation at
TREC
Donna Harman, National Institute for Standards and
Technology (NIST), USA (slides) |
|
14.20 -
14.40 |
CLIR Evaluation at
NTCIR
Noriko Kando, National Institute of Informatics (NII),
Japan (slides)
|
|
14.40 -
15.00 |
CLEF:
Cross-language System Evaluation for European Languages
Carol
Peters, National Research Council, Pisa, Italy (slides) |
|
|
|
|
15.20 -
15.50 |
Break
|
|
|
|
|
15.50 -
17.00 |
Panel on "CLIR
Evaluation in the Future" |
|
|
Moderator:
Peter Schäuble, Eurospider,
Switzerland |
|
|
Panelists:
Donna Harman, National Institute
for Standards and Technology (NIST), USA
Noriko Kando, National
Institute of Informatics (NII), Japan (slides)
Jussi Karlgren, SICS, Sweden (slides)
Wessel Kraaij, TNO-TPD, The Netherlands
William Ogden,
Computing Research Lab, New Mexico State University, USA (slides)
|
|
|
|
|
Day 2 - |
22
September 2000 |
|
|
|
|
9.00 -
9.30 |
CLEF 2000:
Overview of the Results |
|
|
Martin Braschler,
Eurospider, Switzerland (slides)
|
|
|
|
|
9.30 -
10.15 |
Monolingual
(non-English) Retrieval |
|
|
Chair:
Julio Gonzalo, UNED, Madrid, Spain
|
|
|
Presentations: Isabelle Moulinier, West Group, USA;
Arjen P. de Vries, CWI, The Netherlands; Marcello Federico,
ITC-irst, Italy (slides),
and Discussion. |
|
|
|
|
10.15 -
10.45 |
Break
|
|
|
|
|
10.45 -
12.45 |
Cross-Language
Retrieval |
|
|
Chair:
Donna Harman, NIST, USA
|
|
|
Presentations: Djoerd Hiemstra, University of Twente,
The Netherlands; Paul McNamee, Johns Hopkins University Applied
Physics Lab, USA; Turid Hedlund, Tampere University, Finland; Bärbel
Ripplinger, Univ. Saarlaandes, Germany; N. Nassr, IRIT/SIG, Toulose,
France; Carlos G. Figuerola, Univ. Salamanca, Spain,
and
Discussion. |
|
|
|
|
12.45 -
14.00 |
Lunch
|
|
|
|
|
10.45 -
12.45 |
Domain-Specific
Retrieval |
|
|
Chair:
Michael Kluck, IZ-Bonn, Germany
|
|
|
Presentations: Norbert Goevert, Univ. Dortmund,
Germany; Fred Gey, UC Berkeley, USA,
and
Discussion. |
|
|
|
|
15.00 -
16.30 |
CLEF 2001: What
should we keep? What should we change? What should we add?
|
|
|
Moderator:
Peter Schäuble, Eurospider,
Switzerland |
The draft papers outlining the approaches used by the participants in
the CLEF 2000 evaluation campaign were printed in the Working Notes
and distributed at the Workshop.
Workshop Steering
Committee
Martin Braschler, Eurospider,
Switzerland
Julio Gonzalo Arroyo, UNED, Madrid, Spain
Donna
Harman, National Institute for Standards and Technology (NIST), USA
Michael Hess, University of Zurich, Switzerland
Michael Kluck, IZ
Sozialwissenschaften, Bonn, Germany
Carol Peters, IEI-CNR, Pisa, Italy
Peter Schäuble, Eurospider, Switzerland
Local
Coordinators
Caroline Hagège, CSTC, Lisbon, Portugal
Nuno Mamede, IST-CSTC,
Lisbon, Portugal
Cross-Language Evaluation Forum
The Workshop was organised by the
Cross-Language Evaluation Forum (CLEF). To find out more about the
activities of CLEF, please click here.
Contact
Carol Peters -
IEI-CNR
Area della Ricerca di San Cataldo, 56100 PISA (Italy)
Tel:
+39 050 315 2897 - Fax: +39 050 315 2810
E-mail: carol@iei.pi.cnr.it |