[Adapt] Simulating User Satisfaction for the Evaluation of Task-oriented Dialogue Systems

[李子通]

Hi Adapters, 

Evaluation is crucial in the development process of task-oriented dialogue systems. As an evaluation method, user simulation allows us to tackle issues such as scalability and cost-efficiency, making it a viable choice for large-scale automatic evaluation. I will talk about a paper "Simulating User Satisfaction for the Evaluation of Task-oriented Dialogue Systems". To help build a human-like user simulator that can measure the quality of a dialogue, the authors propose the following task: simulating user satisfaction for the evaluation of task-oriented dialogue systems. The purpose of the task is to increase the evaluation power of user simulations and to make the simulation more human-like. To overcome a lack of annotated data, they also propose a user satisfaction annotation dataset, User Satisfaction Simulation (USS).

I hope my talk makes you feel clear, interesting and helpful.

Time: Wed 4:00pm
Venue: SEIEE 3-414
Best wishes.
Zitong