[Adapt] Introduction to Current Evaluation Approaches for Dialogue Systems

[杨若澜]

Hi Adapters,


Evaluation is a long-standing issue in developing conversational dialogue systems. Unlike those task-oriented dialogue systems whose success can be measured in relation to an a priori defined ground truth, chatbots do not solve a clearly-defined task, it has an open-ended nature. The major challenge with the evaluation of open-domain dialog systems comes from the one-to-many relationship between the user’s input and plausible responses.


In tomorrow's talk, I'll first give you a brief introduction about existing evaluation approaches for open-domain dialogue systems. Next, I'll introduce the work of the paper "Spot The Bot: A Robust and Efficient Framework for the Evaluation of Conversational Dialogue Systems." in detail. Finally some discussions about what we can do to improve their work. Due to the limitation of time, i'll introduce my own work about this topic next time. 


Hope you enjoy it!


Time: March 17 at 4pm
Venue: SEIEE 3-414


Best,
Ruolan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://cs.sjtu.edu.cn/pipermail/adapt/attachments/20210316/723f50b8/attachment.html>