Improved Sequence-to-Sequence Models for Abstractive Text Summarization Using Meta Heuristic Approaches

Full Length Article

•

Improved Sequence-to-Sequence Models for Abstractive Text Summarization Using Meta Heuristic Approaches

DOI

format_quote Cite this article

View PDF open_in_new

Abstract

As human society transitions into the information age, reduction in our attention span is a contingency, and people who spend time reading lengthy news articles are decreasing rapidly and the need for succinct information is higher than ever before. Therefore, it is essential to provide a quick overview of important news by concisely summarizing the top news article and the most intuitive headline. When humans try to make summaries, they extract the essential information from the source and add useful phrases and grammatical annotations from the original extract. Humans have a unique ability to create abstractions. However, automatic summarization is a complicated problem to solve. The use of sequence-to-sequence (seq2seq) models for neural abstractive text summarization has been ascending as far as prevalence. Numerous innovative strategies have been proposed to develop the current seq2seq models further, permitting them to handle different issues like saliency, familiarity, and human lucidness and create excellent synopses. In this article, we aimed toward enhancing the present architecture and models for abstractive text summarization. The modifications have been aimed at fine-tuning hyper-parameters, attempting specific encoder-decoder combinations. We examined many experiments on an extensively used CNN/DailyMail dataset to check the effectiveness of various models.

Keywords

Natural Language Processing (NLP) Machine Learning (ML) Sequence to Sequence Encoder-Decoder LSTM Transformer Attention Sequence

References

1.

[1] Li, Ping, et al. "HG-News: News Headline Generation Based on a Generative Pre-Training Model." IEEE Access 9 (2021): 110039-110046.L

[2] Ao, X., Wang, X., Luo, L., Qiao, Y., He, Q., & Xie, X. PENS: A Dataset and Generic Framework for Personalized News Headline Generation.

[3] Singh, R. K., Khetarpaul, S., Gorantla, R., & Allada, S. G. (2021). SHEG: summarization and headline generation of news articles using deep learning. Neural Computing and Applications, 33(8), 3251-3265.

[4] Hayashi Y., Yanagimoto H. (2018) Headline Generation with Recurrent Neural Network. In: Matsuo T., Mine T., Hirokawa S. (eds) New Trends in E-service and Smart Computing. Studies in Computational Intelligence, vol 74.

[5] Shavrina, T., & Shliazhko, O. Using Generative Pretrained Transformer-3 Models for Russian News Clustering and Title Generation tasks.

[6] Singh B., Marathe A., Rizvi A.A., Joshi A.R. (2021) Retaining Named Entities for Headline Generation. In: Smys S., Balas V.E., Kamel K.A., Lafata P. (eds) Inventive Computation, and Information Technologies. Lecture Notes in Networks and Systems, vol 173. Springer, Singapore

[7] Y. Thu and W. P. Pa, "Myanmar News Headline Generation with Sequence-to-Sequence model," 2020 23rd Conference of the Oriental COCOSDA International Committee (O-COCOSDA), 2020

[8] Bukhtiyarov A., Gusev I. (2020) Advances of Transformer-Based Models for News Headline Generation. In: Filchenkov A., Kauttonen J., Pivovarova L. (eds) Artificial Intelligence and Natural Language. AINL 2020.

[9] Gavrilov D., Kalaidin P., Malykh V. (2019) Self-attentive Model for Headline Generation. In: Azzopardi L., Stein B., Fuhr N., Mayr P., Hauff C., Hiemstra D. (eds) Advances in Information Retrieval. ECIR 2019. Lecture Notes in Computer Science, vol 11438.

[10] Zhang, R., Guo, J., Fan, Y., Lan, Y., & Cheng, X. (2020). Structure Learning for Headline Generation. Proceedings of the AAAI Conference on Artificial Intelligence, 34(05), 9555-9562

[11] Vaswani, Ashish & Shazeer, Noam & Parmar, Niki & Uszkoreit, Jakob & Jones, Llion & Gomez, Aidan & Kaiser, Lukasz & Polosukhin, Illia. (2017). Attention Is All You Need.

[12] Lopyrev, Konstantin. "Generating news headlines with recurrent neural networks." arXiv preprint arXiv:1512.01712(2015).

[13] M. Mauro, L. Canini, S. Benini, N. Adami, A. Signoroni, and R. Leonardi, “A freeWeb API for single and multi-document summarization,” ACM Int. Conf. Proceeding Ser., vol. Part F1301, 2017, doi: 10.1145/3095713.3095738.

[14] G. Silva, R. Ferreira, S. J. Simske, L. Rafael Lins, M. Riss, and H. O. Cabral, “Automatic text document summarization based on machine learning,” DocEng 2015 - Proc. 2015 ACM Symp. Doc. Eng., pp. 191–194, 2015, doi: 10.1145/2682571.2797099.

[15] El-Haj, M., Kruschwitz, U., & Fox, C. (2010), 'Using Mechanical Turk to create a corpus of Arabic summaries', In Proceedings of the Seventh conference on International Language Resources and Evaluation, Valletta, Malta, 19-21 ,05,2010.

[16] Al-Omour, M. M., & Al-Taani, A. T. (2014), 'An Extractive Graph-based Arabic Text Summarization Approach', The International Arab Conference On Information Technology (Acit2014), University of Nizwa, Oman, 158-163.

[17] Ibrahim, A., Elghazaly, T., & Gheith, M. (2013), "A Novel Arabic Text Summarization Model Based on Rhetorical Structure Theory and Vector Space Model", International Journal of Computational Linguistics and Natural Language Processing, 2(8), 480-485.

[18] R. Z. Al-Abdallah and A. T. Al-Taani, "Arabic Text Summarization using Firefly Algorithm," 2019 Amity International Conference on Artificial Intelligence (AICAI), 2019, pp. 61-65, doi: 10.1109/AICAI.2019.8701245.

[19] R. Rautray, R.C. Balabantaray, Comparative study of DE and PSO over document summarization, in: Intelligent Computing, Communication and Devices, Springer, India, 2015, pp. 371e377.

[20] R. Rautray, R.C. Balabantaray, A. Bhardwaj, Document summarization using sentence features, Int. J. Inf. Retr. Res. (IJIRR) 5 (1) (2015) 36e47.

[21] H. Asgari, B. Masoumi, O.S. Sheijani, Automatic text summarization based on multi-agent particle swarm optimization, in: ICIS Iranian Conference on Intelligent Systems, IEEE, 2014, pp. 1e5.

[22] M. Modiri-Delshad, S.H.A. Kaboli, E. Taslimi-Renani, N.A. Rahim, Backtracking search algorithm for solving economic dispatch problems with valve-point effects and multiple fuel options, Energy 116 (2016) 637e649.

[23] S. Hr Aghay Kaboli, J. Selvaraj, N.A. Rahim, Long-term electric energy consumption forecasting via artificial cooperative search algorithm, Energy 115 (2016) 857e871.

[24] A.R. Rafieerad, A.R. Bushroa, B. Nasiri-Tabrizi, S.H.A. Kaboli, S. Khanahmadi, A. Amiri, J. Vadivelu, F. Yusof, W.J. Basirun, K. Wasa, Toward improved mechanical, tribological, corrosion and in-vitro bioactivity properties of mixed oxide nanotubes on Tie6Ale7Nb implant using multi-objective PSO, J. Mech. Behav. Biomed. Mater. 69 (2017) 1e18.

[25] S. Hr Aghay Kaboli, J. Selvaraj, N.A. Rahim, Rain-fall optimization algorithm: a population-based algorithm for solving constrained optimization problems, J. Comput. Sci. 19 (2017)31e42.

[26] S.S. Sebtahmadi, Azad H. Borhan, S. Hr Aghay Kaboli, M.D. Islam, S. Mekhilef, A PSO-DQ current control scheme for performance enhancement of Z-source matrix converter to drive IM fed by abnormal voltage, IEEE Trans. Power Electron. 99 (2017) 1e15.

[27] S.H.A. Kaboli, A. Fallahpour, J. Selvaraj, N.A. Rahim, Longterm electrical energy consumption formulating and forecasting via optimized gene expression programming, Energy 126 (2017) 144e164.

[28] Nicolas, Z., Skubacz, S., 2007. Content extraction from news pages using particle swarm optimization on linguistic and structural features. In: Proceedings of International Conference on Web Intelligence, pp. 242–249.

[29] Mesleh, A.M., Kanaan, G., 2008. Support vector machine text classification system: Using Ant Colony Optimization based feature subset selection. In: Image and Signal Processing for Remote Sensing X. International Conference on Computer Engineering and Systems (ICCES 2008), vol. 5573, pp. 143–148

[30] Harrag, A. Harrag, N., 2014. Firefly Feature subset selection application to Arabic speaker recognition system

[31] Gupta, S., Sharaff, A. & Nagwani, N.K. Control Stochastic Selection-Based Biomedical Text Summarization Using Sim-TLBO. Arab J Sci Eng 49, 3593–3607 (2024). https://doi.org/10.1007/s13369-023-08056-5

Cite This Article

Choose your preferred format

format_quote

. "Improved Sequence-to-Sequence Models for Abstractive Text Summarization Using Meta Heuristic Approaches." Journal of Artificial Intelligence and Metaheuristics, vol. , no. , , pp. . DOI:

(). Improved Sequence-to-Sequence Models for Abstractive Text Summarization Using Meta Heuristic Approaches. Journal of Artificial Intelligence and Metaheuristics, (), . DOI:

. "Improved Sequence-to-Sequence Models for Abstractive Text Summarization Using Meta Heuristic Approaches." Journal of Artificial Intelligence and Metaheuristics , no. (): . DOI:

() 'Improved Sequence-to-Sequence Models for Abstractive Text Summarization Using Meta Heuristic Approaches', Journal of Artificial Intelligence and Metaheuristics, (), pp. . DOI:

. Improved Sequence-to-Sequence Models for Abstractive Text Summarization Using Meta Heuristic Approaches. Journal of Artificial Intelligence and Metaheuristics. ;():. DOI:

, "Improved Sequence-to-Sequence Models for Abstractive Text Summarization Using Meta Heuristic Approaches," Journal of Artificial Intelligence and Metaheuristics, vol. , no. , pp. , . DOI:

Digital Archive Ready