L I M S I - C N R S

LIMSI Spoken Language Processing Group (TLP)

Laboratory
CNRS home page
LIMSI home page
LIMSI TLP Group

Speech & Language Technology Evaluation


NIS
DFKI
ITC-IRTS
LIMSI

Paris time: 06h24

Workshop on Evaluation for Language and Dialogue Systems
ACL/EACL 2001, 
Toulouse, France, July 6-7, 2001

Morning Day 1 : the Evaluation of Dialogue Systems and Models

Time Title Speaker
08.45 - 09.15  - Welcome
09.15 - 09.45 Invited Talk: Views on Multi-Modal and Multi-Party Contexts Anton Nijholt (U. Twente)
09.45 - 10.00 Questions -
10.00 - 10.30  "Empirical Methods for Evaluating Dialog Systems" Tim Paek (Microsoft Research)
10.30 - 10.45  questions -
10.45 - 11.15 Coffee Break,  -
11.15 - 11.45 "Usability Evaluation in Spoken Language Dialogue Systems", Laila Dybkjær (NIS Lab) and
Niels Ole Bernsen (NIS Lab)
11.45 - 12.00 Questions -
12.00 - 12.45 Panel discussion -
12.45 - 14.00 Lunch Break -

Afternoon Day 1: Evaluation in Language Engineering

Time Title Speaker
14.00 - 14.30 "Translating Treebank Annotation for Evaluation" Stephen Watkinson (U. of York) and
Suresh Manandhar (U. of York)
14.30 - 14.45 Questions -
14.45 - 15.15 "Two Levels of valuation in a Complex NL System" Jean-Baptiste Berthelin (Limsi-CNRS),
Brigitte Grau (Limsi-CNRS), and
Martine Hurault-Plantet (Limsi-CNRS),
15.15 - 15.30 Questions -
15.30 - 16.00 Coffee Break -
16.00 - 16.30  "Verification and validation of language processing systems: Is it evaluation?" Valerie Barr (Hofstra University) and
Judith Klavans (Columbia University).
16.30 - 16.45 Questions -
16.45 - 17.15 "The ARC A3 Project: Terminology Acquisition Tools: Evaluation Method and Task" Widad Mustafa El Hadi (U. Lille 3),
Ismail Timimi (U. Lille 3),
Annette Beguin (U. Lille 3),
Marcilio de Brito (U. Lille 3).
17.15 - 17.30 Questions -

Morning Day 2: Evaluation in Language Engineering

Time Speaker Title
08.45 - 09.15 Welcome -
09.15 - 09.45 Invited Talk: "The Importance of Focused Evaluations: a Case Study of TREC and DUC" Donna Harman (NIST)
09.45 - 10.00 Questions -
10.00 - 10.30  "Using the Distribution of Performance for Studying Statistical NLP Systems and Corpora" Yuval Krymolowski (Bar-Ilan U.)
10.30 - 10.45 Questions -
10.45 - 11.15  Coffee Break -
11.15 - 11.45  "A Cross-Comparison of Two Clustering Methods" Michèle Jardino (Limsi-CNRS),
Brigitte Grau (Limsi-CNRS),
Olivier Ferret (CEA / Limsi-CNRS)
11.45 - 12.00 Questions -
12.00 - 12.45 Panel discussion -
12.45 - 14.00 Lunch Break -

Afternoon Day 2: Work session

Time Title Speaker
14.00 - 15.30 "Planning for the Deployment of Evaluation in Language Engineering" General Work Session

[CLASS main site home page]
[CLASS Evaluation subsite home page]
[LIMSI home page]
[Last updated: Mon Jan 29th 2001]