[Adapt] [Seminar] What does BERT learn about the structure of language?

张盛瑶 sophie_zhang at sjtu.edu.cn
Tue Oct 15 22:04:54 CST 2019

Hi Adapters,

Tomorrow I will introduce the paper “What does BERT learn about the structure of language?” which is accepted by ACL 2019. This paper talks about the interpretability of BERT. The author performs series of  experiments to investigate the linguistic structure implicitly learned by BERT’s representations. 

The results show that BERT’s intermediate layers encode a rich hierarchy of linguistic information, with surface features at the bottom, syntactic features in the middle and semantic features at the top.  It also tells that BERT representations capture linguistic information in a compositional way which mimics classical, tree-like structures. 

Time: 16:30 October 15
Venue: SEIEE 3-414

Thanks & Best Regards !

More information about the Adapt mailing list