Deep Learning for Text Summarization

• deeplearningkit
• 8 months ago
• Categories: Deep Learning, Natural Language Processing
• Tags: natural language processing, summarization, text summarization

A few years back I was involved in a project trying to do realtime/low latency text summarization (without deep learning) using a Nvidia Tesla C1060 GPU (roughly state-of-the-art GPU back then), the motivation for doing realtime summarization was driven by an idea of improving search results in general. The issue with search is that it typically returns disjunct results (with somewhat arbitrary relationship between each search result), instead of providing a coherent answer to the query in a more cross-result and summarized way (it has query driven summaries – snippets – on individual result level though, but not cross result). The project unfortunately never materialized in the form of great results, so it is great to see that Deep Learning based summarization is thriving (with much more powerful GPUs this time). See below for some recent research papers on Deep Learning based summarization:

1. AttSum: Joint Learning of Focusing and Summarization with Neural Attention – authors: Z Cao, W Li, S Li, F Wei
2. A Convolutional Attention Network for Extreme Summarization of Source Code – authors: M Allamanis, H Peng, C Sutton
3. Sequence-to-Sequence RNNs for Text Summarization – authors: R Nallapati, B Xiang, B Zhou
4. Learning Summary Statistic for Approximate Bayesian Computation via Deep Neural Network – authors: B Jiang, T Wu, C Zheng, WH Wong
5. LCSTS: A Large Scale Chinese Short Text Summarization Dataset – authors: B Hu, Q Chen, F Zhu
6. Deep Dependency Substructure-Based Learning for Multidocument Summarization – authors: S Yan, X Wan
7. Ranking with Recursive Neural Networks and Its Application to Multi-document Summarization – authors: Z Cao, F Wei, L Dong, S Li, M Zhou
8. Query-oriented Unsupervised Multi-document Summarization via Deep Learning – authors: S Zhong, Y Liu, B Li
9. Abstractive Multi-Document Summarization via Phrase Selection  – authors: L Bing, P Li, Y Liao, W Lam, W Guo, RJ Passonneau
10. Modelling ‚Visualising and Summarising Documents with a Single Convolutional Neural Network – authors: N de Freitas
11. SRRank: Leveraging Semantic Roles for Extractive Multi-Document Summarization – authors: S Yan, X Wan

Best regards,

Amund Tveit ()

btw: if you want to work (with me) as a Data Scientist on Deep Learning, check out this position