Dhaka University Journal of Applied Science & Engineering

• Nasid Habib Barna
  Department of Computer Science and Engineering, University of Dhaka, Dhaka, Bangladesh
• Hasnain Heickal
  Department of Computer Science and Engineering, University of Dhaka, Dhaka, Bangladesh

Abstractive text summarization is one of the most interesting problems in the research field of Natural Language Processing. Recent advances in sequence to sequence model have made it possible to apply new approaches for abstractive text summarization and perform significantly. But most of the existing systems suffer from some drawbacks like word repetition, producing inaccurate or irrelevant information etc. In this work we propose a novel architecture incorporating advanced word embedding layer and topical feature with a pointer generator network to generate more topic oriented summaries in a logically sequenced way. Adding a word embedding layer with the model can capture semantic features of words in the input sequence more accurately. Also our proposed system with incorporated topical features ensures that the summaries focus on the most important parts of the source document. We applied our model to the CNN/Daily Mail dataset and outperformed the baseline model by all the ROUGE scores.

1. H. P. Luhn, “The automatic creation of literature abstracts,” IBM Journal of Research and Development, vol. 2, no. 2, pp. 159–165, Apr 1958
2. Z. Cao, F. Wei, W. Li, and S. Li, “Faithful to the original: Fact aware neural abstractive summarization,” in Thirty-Second AAAI Conference on Artificial Intelligence, 2018
3. R. Nallapati, F. Zhai, and B. Zhou, “Summarunner: A recurrent neural network based sequence model for extractive summarization of documents,” in Thirty-First AAAI Conference on Artificial Intelligence, 2017
4. S. Narayan, N. Papasarantopoulos, S. B. Cohen, and M. Lapata, “Neural extractive summarization with side information,” arXiv preprint arXiv:1704.04530, 2017.
5. S. Alshaina, A. John, and A. G. Nath, “Multi-document abstractive summarization based on predicate argument structure,” in 2017 IEEE International Conference on Signal Processing, Informatics, Communication and Energy Systems (SPICES), pp. 1–6, 2017.
6. I. Sutskever, O. Vinyals, and Q. V. Le, “Sequence to sequence learning with neural networks,” CoRR, vol. abs/1409.3215, 2014. [Online]. Available: http://arxiv.org/ abs/1409.3215
7. A. M. Rush, S. Chopra, and J. Weston, “A neural attention model for abstractive sentence summarization,” in Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Lisbon, Portugal: Association for Computational Linguistics, Sep. 2015, pp. 379–389. [Online]. Available: https://www.aclweb.org/ anthology/D15-1044
8. S. Chopra, M. Auli, and A. M. Rush, “Abstractive sentence summarization with attentive recurrent neural networks,” in Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 93–98, 2016.
9. A. See, P. J. Liu, and C. D. Manning, “Get to the point: Summarization with pointer-generator networks,” CoRR, vol. abs/1704.04368, 2017. [Online]. Available: http://arxiv.org/ abs/1704.04368
10. R. Nallapati, B. Zhou, C. Gulcehre, B. Xiang et al., “Abstractive text summarization using sequence-to-sequence rnns and beyond,” arXiv preprint arXiv:1602.06023, 2016.
11. S. Song, H. Huang, and T. Ruan, “Abstractive text summarization using lstm-cnn based deep learning,” Multimedia Tools and Applications, vol. 78, no. 1, pp. 857–875, 2019. [Online]. Available: https://doi.org/10.1007/ s11042-018-5749-3
12. J. R. Firth, “A synopsis of linguistic theory 1930-55.” vol. 1952-59, pp. 1–32, 1957.
13. J. Pennington, R. Socher, and C. D. Manning, “Glove: Global vectors for word representation,” in Empirical Methods in Natural Language Processing (EMNLP), pp. 1532–1543, 2014. [Online]. Available: http://www.aclweb.org/ anthology/D14-1162
14. D. Bahdanau, K. Cho, and Y. Bengio, “Neural machine translation by jointly learning to align and translate,” 2014.
15. C.-Y. Lin, “ROUGE: A package for automatic evaluation of summaries,” in Text Summarization Branches Out. Barcelona, Spain: Association for Computational Linguistics, pp. 74–81, 2004. [Online]. Available: https://www.aclweb.org/anthology/W04-1013
16. [Online]. Available: https://www.dailymail.co.uk/sport/ football/article3029323/Emmanuel-Adebayor-vows-contract-Tottenham-striker-rejectsexit-talk.html