[Adapt] [Seminar] What does BERT learn about the structure of language?

[张盛瑶]

Hi Adapters,

Tomorrow I will introduce the paper “What does BERT learn about the structure of language?” which is accepted by ACL 2019. This paper talks about the interpretability of BERT. The author performs series of  experiments to investigate the linguistic structure implicitly learned by BERT’s representations. 

The results show that BERT’s intermediate layers encode a rich hierarchy of linguistic information, with surface features at the bottom, syntactic features in the middle and semantic features at the top.  It also tells that BERT representations capture linguistic information in a compositional way which mimics classical, tree-like structures. 

Time: 16:30 October 15
Venue: SEIEE 3-414


Thanks & Best Regards !
Sophie