[Adapt] [Seminar][Attention Mechanism in Neural Networks]

[Luo Kangqi]

Hi Adapters,

This time, I'm going to introduce "attention mechanism", which is widely used in neural network models.


Recent years, the neural network becomes an effective model in many NLP tasks, such as machine translation, textual entailment and entity disambiguation. The common idea is to map a natural language text into a hidden vector space, and handle different classification problems based on these latent features. Both CNN and RNN are able to build the hidden representation of the whole sentence from the vector of each word in the sentence, while attention mechanism helps dynamically measure the importance of each word, showing very promising improvements on these NLP tasks.


In this seminar, I will show you two real example of attention-based models, one is for RNN, and the other is for CNN. Please check out the following links if you are not so familiar with these NN models. Don't worry, just need to have some basic intuition, and I will introduce more during my presentation.


CNN: http://www.36dsj.com/archives/24006
RNN: http://blog.csdn.net/heyongluoyao8/article/details/48636251


The seminar will take place at 4:30 pm tomorrow, in SEIEE 3-404. Hope you can enjoy, see you then !

Best,
Kangqi

------------------
Kangqi Luo, PhD Candidate
ADAPT Lab, SEIEE 3-341
Shanghai Jiao Tong University
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://cs.sjtu.edu.cn/pipermail/adapt/attachments/20170307/89019d09/attachment.html>