Monthly Archives: April 2016

Deep Learning for Text Summarization

A few years back I was involved in a project trying to do realtime/low latency text summarization (without deep learning) using a Nvidia Tesla C1060 GPU (roughly state-of-the-art GPU back then), the motivation for doing realtime summarization was driven by an idea of improving search results in general. The issue with search is that it typically returns disjunct results (with somewhat arbitrary relationship between each search result), instead of providing a coherent answer to the query in a more cross-result and summarized way (it has query driven summaries – snippets – on individual result level though, but not cross result). The project unfortunately never materialized in the form of great results, so it is great to see that Deep Learning based summarization is thriving (with much more powerful GPUs this time). See below for some recent research papers on Deep Learning based summarization:

  1. AttSum: Joint Learning of Focusing and Summarization with Neural Attention – authors: Z Cao, W Li, S Li, F Wei
  2. A Convolutional Attention Network for Extreme Summarization of Source Code – authors: M Allamanis, H Peng, C Sutton
  3. Sequence-to-Sequence RNNs for Text Summarization – authors: R Nallapati, B Xiang, B Zhou
  4. Learning Summary Statistic for Approximate Bayesian Computation via Deep Neural Network – authors: B Jiang, T Wu, C Zheng, WH Wong
  5. LCSTS: A Large Scale Chinese Short Text Summarization Dataset – authors: B Hu, Q Chen, F Zhu
  6. Deep Dependency Substructure-Based Learning for Multidocument Summarization – authors: S Yan, X Wan
  7. Ranking with Recursive Neural Networks and Its Application to Multi-document Summarization – authors: Z Cao, F Wei, L Dong, S Li, M Zhou
  8. Query-oriented Unsupervised Multi-document Summarization via Deep Learning – authors: S Zhong, Y Liu, B Li
  9. Abstractive Multi-Document Summarization via Phrase Selection  – authors: L Bing, P Li, Y Liao, W Lam, W Guo, RJ Passonneau
  10. Modelling ‚Visualising and Summarising Documents with a Single Convolutional Neural Network – authors: N de Freitas
  11. SRRank: Leveraging Semantic Roles for Extractive Multi-Document Summarization – authors: S Yan, X Wan

Best regards,

Amund Tveit ()

btw: if you want to work (with me) as a Data Scientist on Deep Learning, check out this position

Deep Learning for Named Entity Recognition

About a year ago I wrote a blog post about recent research in Deep Learning for Natural Language Processing covering several subareas. One of the areas I didn’t cover was Deep Learning for Named Entity Recognition – so here are some interesting recent (2015-2016) papers related to that:

  1. Capturing Semantic Similarity for Entity Linking with Convolutional Neural Networks – authors: M Francis

  2. Entity Attribute Extraction from Unstructured Text with Deep Belief Network – authors: B Zhong, L Kong, J Liu

  3. Learning Word Segmentation Representations to Improve Named Entity Recognition for Chinese Social Media – authors: N Peng, M Dredze

  4. Biomedical Named Entity Recognition based on Deep Neutral Network – authors: L Yao, H Liu, Y Liu, X Li, MW Anwar

  5. Shared tasks of the 2015 workshop on noisy user-generated text: Twitter lexical normalization and named entity recognition – authors: T Baldwin, MC de Marneffe, B Han, YB Kim, A Ritter…

  6. Semi-Supervised Approach to Named Entity Recognition in Spanish Applied to a Real-World Conversational System – authors: SS Bojórquez, VM González

  7. Boosting Named Entity Recognition with Neural Character Embeddings – authors: C dos Santos, V Guimaraes, RJ Niterói, R de Janeiro

  8. Exploring Recurrent Neural Networks to Detect Named Entities from Biomedical Text – authors: L Li, L Jin, D Huang

  9. Entity-centric search: querying by entities and for entities – authors: M Zhou

  10. Automatic Entity Recognition and Typing from Massive Text Corpora: A Phrase and Network Mining Approach – authors: X Ren, A El

  11. Boosting Named Entity Recognition with Neural Character Embeddings – authors: CN Santos, V Guimarães

  12. Named Entity Recognition in Chinese Clinical Text Using Deep Neural Network. – authors: Y Wu, M Jiang, J Lei, H Xu

  13. Context-aware Entity Morph Decoding – authors: B Zhang, H Huang, X Pan, S Li, CY Lin, H Ji, K Knight…

  14. Training word embeddings for deep learning in biomedical text mining tasks – authors: Z Jiang, L Li, D Huang, L Jin

  15. Entity Attribute Extraction from Unstructured Text with Deep Belief Network – authors: B Zhong, L Kong, J Liu

  16. Building Text-mining Framework for Gene-Phenotype Relation Extraction using Deep Leaning – authors: D Jang, J Lee, K Kim, D Lee

  17. Text Mining in Social Media for Security Threats – authors: D Inkpen

  18. Text Understanding from Scratch – authors: X Zhang, Y LeCun

  19. Syntax-based Deep Matching of Short Texts – authors: M Wang, Z Lu, H Li, Q Liu

  20. PTE: Predictive Text Embedding through Large-scale Heterogeneous Text Networks – authors: J Tang, M Qu, Q Mei

  21. Automatic Entity Recognition and Typing from Massive Text Corpora: A Phrase and Network Mining Approach – authors: X Ren, A El

  22. Domain-Specific Semantic Relatedness from Wikipedia Structure: A Case Study in Biomedical Text – authors: A Sajadi, EE Milios, V Kešelj, JCM Janssen

  23. Deep Unordered Composition Rivals Syntactic Methods for Text Classification – authors: M Iyyer, V Manjunatha, J Boyd

  24. Representing Text for Joint Embedding of Text and Knowledge Bases – authors: K Toutanova, D Chen, P Pantel, H Poon, P Choudhury…

  25. In Defense of Word Embedding for Generic Text Representation – authors: G Lev, B Klein, L Wolf

Best regards,

Amund Tveit ()

btw: if you want to work (with me) as a Data Scientist on Deep Learning, check out this position