Volume 5 • Issue 2 • PP: 70-76 • 2021
Robust Neural Language Translation Model Formulation using Seq2seq approach
View PDF Abstract
In this work, the approach used is to sequence powerful models that have achieved excellent performance on language translation encoding-decoding tasks. A language transformer model is used in this work based on the sequence-to-sequence approach, which uses a Long Short-Term Memory (LSTM) to map the input sequence to a vector of fixed dimensionality. Then another deep LSTM decodes the target sequence from the vector. Evaluated the model efficiency through BLEU score and LSTM's BLEU score was penalized on out-of-vocabulary words. Additionally, the LSTM did not have difficulty with long-short of sentences. This work performed the deep LSTM setup English-Japanese translation accuracy at an order of magnitude faster speed, both on GPU and CPU. The variety of the data is introduced into it to evaluate the robustness using the BLEU score. Finally, a better result is achieved by merging the two different types of datasets and getting the highest BLEU score of 40.1 at the end.
Keywords
References
[1] Sutskever, Ilya, Martens, James, Dahl, George E., and Hinton, Geoffrey E. On the importance of initialization and momentum in deep learning, ICML, 2013.
[2] Bahdanau, Dzmitry, Cho, Kyunghyun, and Bengio, Yoshua, Neural machine translation by jointly learning to align and translate, arXiv preprint at Xiv:1409.0473, 2014.
[3] Waibel, Alex, Hanazawa, Toshiyuki, Hinton, Geoffrey, Shikano, Kiyohiro, and Lang, Kevin J. Phoneme Recognition using Time-delay Neural Networks. IEEE transactions on acoustics, speech, and signal processing, 37(3):328–339, 1989.
[4] Jonas Gehring 1 Michael Auli 1 David Grangier 1 Denis Yarats 1 Yann N. Dauphin 1, Sequence to Sequence Learning with Neural Networks, neurips, 2014.
[5] Mikolov, M. Karafiat, L. Burget, J. Cernock ´ y, and S. Khudanpur. Recurrent neural network based language model. In INTERSPEECH, pages 1045–1048, 2010.
[6] Devlin, R. Zbib, Z. Huang, T. Lamar, R. Schwartz, and J. Makhoul. Fast and robust neural network joint models for statistical machine translation, ACL, 2014.
[7] Cho, B. Merrienboer, C. Gulcehre, F. Bougares, H. Schwenk, and Y. Bengio. Learning phrase representations using RNN encoder-decoder for statistical machine translation, arxiv preprint arXiv:1406.1078, 2014.
[8] Pouget-Abadie, D. Bahdanau, B. van Merrienboer, K. Cho, and Y. Bengio. Overcoming the curse of sentence length for neural machine translation using automatic segmentation. arXiv preprint, arXiv: 1409.1257, 2014.
[9] Nadir Durrani, Barry Haddow, Philipp Koehn, and Kenneth Heafield. Edinburgh’s phrase-based machine translation systems for wmt-14, WMT, 2014.
[10] K. Yu, M. Arifuzzaman, Z. Wen, D. Zhang and T. Sato, “A Key Management Scheme for Secure Communications of Information Centric Advanced Metering Infrastructure in Smart Grid,” IEEE Transactions on Instrumentation and Measurement, vol. 64, no. 8, pp. 2072-2085, August 2015.
[11] Z. Guo, L. Tang, T. Guo, K. Yu, M. Alazab, A. Shalaginov, “Deep Graph Neural Network-based Spammer Detection Under the Perspective of Heterogeneous Cyberspace”, Future Generation Computer Systems, https://doi.org/10.1016/j.future.2020.11.028.
[12] L. Tan, H. Xiao, K. Yu, M. Aloqaily, Y. Jararweh, “A Blockchain-empowered Crowdsourcing System for 5G-enabled Smart Cities”, Computer Standards & Interfaces, https://doi.org/10.1016/j.csi.2021.103517
[13] N. Shi, L. Tan, W. Li, X. Qi, K. Yu, “A Blockchain-Empowered AAA Scheme in the Large-Scale HetNet”, Digital Communications and Networks, https://doi.org/10.1016/j.dcan.2020.10.002.
Cite This Article
Choose your preferred format