[Adapt] [ADAPT Seminar] Holistic Evaluation of Language Models

[forest_zch]

Hi ADAPTers, 

Language models (LMs) are becoming the foundation for almost all major language technologies. More and more LLMs are surging nowadays,  and it is useful for users and researchers to find out their difference in performance. Today my presentation is about a work for the evaluation of LMs which helps to improve the transparency of language models. This work taxonomizes the vast design space of language model evaluation into scenarios and metrics, covers most of the scenarios and conducts the experiments on 30 language models. More details of this work can be explored on https://crfm.stanford.edu/helm/v0.1.0/?.


Hope you enjoy it.
Chunhao
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://cs.sjtu.edu.cn/pipermail/adapt/attachments/20230517/e206bf3b/attachment.htm>