[Adapt] Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedback

[李子通]

Hi Adapters,

Large language models (LLMs), such as ChatGPT, are able to generate human-like, fluent responses for many downstream tasks, e.g., task-oriented dialog and question answering. However, applying LLMs to realworld, mission-critical applications remains challenging mainly due to their tendency to generate hallucinations and their inability to use external knowledge. The paper "Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedback" proposes a LLM-AUGMENTER system, which augments a black-box LLM with a set of plug-and-play modules. Their system makes the LLM generate responses grounded in external knowledge, e.g., stored in task-specific databases. It also iteratively revises LLM prompts to improve model responses using feedback generated by utility functions, e.g., the factuality score of a LLM-generated response. The effectiveness of LLM-AUGMENTER is empirically validated on two types of scenarios, taskoriented dialog and open-domain question answering. LLM-AUGMENTER significantly reduces ChatGPT’s hallucinations without sacrificing the fluency.

Wish you would like it!

Time: Wed 4:00 pm

Venue: SEIEE 3-404

Best wishes,

Zitong