1
Department of Computer Science, College of Computing and Information Technology, Shaqra University, Shaqra 11961, Saudi Arabia;Department of Computer Science, Faculty of Computer and Information Sciences, Ain Shams University, Cairo 11566, Egypt
(abdelaziz@su.edu.sa)
Abstract :
The severe circumstances caused by COVID-19 make online education the best replacement for regular face-to-face education for continuing the education process. One year ago, and till now most schools adopted online learning during this pandemic shutdown, which indicates the applicability of this teaching methodology. However, the efficiency of this method needs to be improved to guarantee its effectiveness. Although face-to-face teaching has many advantages over online education, there is a chance to promote online learning by utilizing the recent techniques of artificial intelligence. From this perspective, we propose a framework to detect and recognize emotions in the speech of students during virtual classes to keep instructors updated with the feelings of students so and can behave accordingly. The approach of detecting emotions from the speech is much more helpful for cases when turning on the cameras at the student's side could be embarrassing. This case is very common, especially for schools in Middle East countries. The proposed framework can also be applied to other similar scenarios such as online meetings.
Keywords :
Speech emotions; Online learning; Machine Learning.
References :
[1] C. Darwin and P. Prodger, 'e Expression of the Emotions in Man and Animals, Oxford
University Press, Oxford, MA, USA, 1998.
[2] Y.-I. Tian, T. Kanade, and J. F. Cohn, “Recognizing action units for facial expression analysis,”
IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 23, no. 2, pp. 97–115,
2001.
[3] R. Donoso, C. San Mart´ın, and G. Hermosilla, “Reduced isothermal feature set for long wave
infrared (LWIR) face recognition,” Infrared Physics&Technology, vol. 83, pp. 114 –123, 2017.
[4] T. Liu, H. Liu, Z. Chen et al., “FBRDLR: fast blind reconstruction approach with dictionary
learning regularization for infrared microscopy spectra,” Infrared Physics & Technology, vol.
90, pp. 101–109, 2018.
[5] Z. Huang, H. Fang, Q. Li et al., “Optical remote sensing image enhancement with weak structure
preservation via spatially adaptive gamma correction,” Infrared Physics & Technology, vol. 94,
pp. 38–47, 2018.
[6] Y. Bi, M. Lv, Y. Wei, N. Guan, and W. Yi, “Multi-feature fusion for thermal face recognition,”
Infrared Physics & Technology, vol. 77, pp. 366–374, 2016.
[7] H. Liu, Z. Zhang, S. Liu, J. Shu, T. Liu, and T. Zhang, “Blind spectrum reconstruction algorithm
with L0-sparse representation,”Measurement Science and Technology, vol. 26, no. 8, pp.
085501–085507, 2015.
[8] H. Wu, Y. Liu, L. Qiu, and Y. Liu, “Online judge system and its applications in C language
teaching,” in Proceedings of the International Symposium on Educational Technology (ISET),
[9] pp. 57–60, Beijing, China, July 2016.
[10] T. Liu, Z. Chen, H. Liu, Z. Zhang, and Y. Chen, “Multi-modal hand gesture designing in multiscreen
touchable teaching system for human-computer interaction,” in Proceedings of the
Second International Conference on Advances in Image Processing, pp. 100–109, Chengdu
China, June 2018.
[11] P. Ekman and W. V. Friesen, “Constants across cultures in the face and emotion,” Journal of
Personality and Social Psychology, vol. 17, no. 2, pp. 124–129, 1971.
[12] P. Ekman, “Strong evidence for universals in facial expressions: a reply to Russell‟s mistaken
critique,” Psychological Bulletin, vol. 115, no. 2, pp. 268–287, 1994.
[13] Samarth Tripathi and Sarthak Tripathi and Homayoon Beigi, " ulti-Modal Emotion recognition
on IEMOCAP Dataset using Deep Learning," arXiv, 1804.05788, 2019.
[14] Wagner, J., Lingenfelser, F., & Andre, E.. The Social Signal Interpretation Framework (SSI) for
Real Time Signal Processing and Recognitions, In Proceedings of INTERSPEECH, Florence,
Italy, 2011.
[15] Wagner, J., Lingenfelser, F., Baur, T., Damian, I., Kistler, F., & Andre, E. The Social Signal
Interpretation (SSI) Framework: Multimodal Signal Processing and Recognition in Real-time.
Proceedings of the 21st ACM International Conference on Multimedia, MM „13. Barcelona,
Spain. 831–834, 2013.
[16] Jo Jianhua, T., Tieniu, T., & RosalindW, P. Affective computing: a review. Affective computing
and intelligent interaction. Springer Berlin Heidelberg, 3784, 981–995, 2005.
[17] Jones, C., & Sutherland, J. Acoustic emotion recognition for affective computer gaming. In C.
Peter & R. Beale (Eds.), Affect and Emotion in Human-Computer Interaction. LNCS. 4868.
Heidelberg: Springer, 2008.
[18] Bahreini, K., Nadolski, R. & Westera, W. Towards real-time speech emotion recognition for
affective e-learning. Educ Inf Technol 21, 1367–1386 (2016).
[19] Beale, R., & Creed, C. Affective interaction: How emotional agents affect users. International
Journal of Human-Computer Studies, 67(9), 755–776, 2009.
[20] Feidakis, M., Daradoumis, T., & Caballe, S. Emotion Measurement in Intelligent Tutoring
Systems: What, When and How to Measure. Third International Conference on Intelligent
Networking and Collaborative Systems, 807–812. 2011.
[21] Huhnel, I., Fölster, M., Werheid, K., & Hess, U. Empathic reactions of younger and older adults:
no age related decline in affective responding. Journal of Experimental Social Psychology, 50,
136–143, 2014.
[22] Nwe, T., Foo, S., & De Silva, L. Speech emotion recognition using hidden Markov models.
Speech Communication, 41, 603–623, 2003.
[23] Pfister, T., & Robinson, P. Real-time recognition of affective states from nonverbal features of
speech and its application for public speaking skill analysis. IEEE Transactions on Affective
Computing, 2(2), 66–78, 2011.
[24] Chen, L., Mao, X., Xue, Y., & Cheng, L. L. Speech emotion recognition: features and
classification models. Digital Signal Processing, 22(6), 1154–1160, 2012.
[25] Bahreini, K., Nadolski, R., & Westera, W. FLITWAM and Voice Emotion Recognition. Games
and Learning Alliance (GaLA) Conference. Paris, France, 23–25, 2013.
[26] Beale, R., & Creed, C. Affective interaction: How emotional agents affect users. International
Journal f Human-Computer Studies, 67(9), 755–776, 2009.
[27] Ben Ammar, M., Neji, M., Alimi, A. M., & Gouardères, G. The affective tutoring system. Expert
Systems with Applications, 37(4), 3013–3023, 2010.
[28] Chen, L., Mao, X., Xue, Y., & Cheng, L. L. Speech emotion recognition: features and
classification models. Digital Signal Processing, 22(6), 1154–1160, 2012.
[29] Happy, S. L. Dasgupta, A., Patnaik, P., Routray, A. Automated Alertness and Emotion Detection
for Empathic Feedback during e-Learning. IEEE Fifth International Conference on Technology
for Education (T4E). 47–50, 2013.
[30] López-Cózar, R., Silovsky, J., & Kroul, M. Enhancement of emotion detection in spoken
dialogue systems by combining several information sources. Speech Communication, 53(9–10),
1210–1228, 2011.
[31] Pekrun, R.. The impact of emotions on learning and achievement: towards a theory of cognitive/
motivational mediators. Journal of Applied Psychology, 41, 359–376, 1992..
[32] Hamzah A. Alsayadi, Abdelaziz A. Abdelhamid, Islam Hegazy, and Zaki T. Fayed: Arabic
speech recognition using end-to-end deep learning. IET Signal Process. 15( 8), 521– 534 (2021).
[33] A. Abdelhamid, E.-S. M. El-kenawy, B. Alotaibi, M. Abdelkader, A. Ibrahim et al., Robust
speech emotion recognition using CNN+LSTM based on stochastic fractal search optimization
algorithm, IEEE Access, vol. 10, pp. 49265-49284, 2022.
[34] A. Abdelhamid, W. Abdulla, B. Macdonald. WFST-Based Large Vocabulary Continuous
Speech Decoder for Service Robots. ACTA Press, 2012. www.actapress.com,
https://doi.org/10.2316/P.2012.771-009
| Style | # |
|---|---|
| MLA | Abdelaziz A. Abdelhamid. "Speech Emotions Recognition for Online Education." Fusion: Practice and Applications, Vol. 10, No. 1, 2023 ,PP. 78-87 (Doi : https://doi.org/10.54216/FPA.100104) |
| APA | Abdelaziz A. Abdelhamid. (2023). Speech Emotions Recognition for Online Education. Journal of Fusion: Practice and Applications, 10 ( 1 ), 78-87 (Doi : https://doi.org/10.54216/FPA.100104) |
| Chicago | Abdelaziz A. Abdelhamid. "Speech Emotions Recognition for Online Education." Journal of Fusion: Practice and Applications, 10 no. 1 (2023): 78-87 (Doi : https://doi.org/10.54216/FPA.100104) |
| Harvard | Abdelaziz A. Abdelhamid. (2023). Speech Emotions Recognition for Online Education. Journal of Fusion: Practice and Applications, 10 ( 1 ), 78-87 (Doi : https://doi.org/10.54216/FPA.100104) |
| Vancouver | Abdelaziz A. Abdelhamid. Speech Emotions Recognition for Online Education. Journal of Fusion: Practice and Applications, (2023); 10 ( 1 ): 78-87 (Doi : https://doi.org/10.54216/FPA.100104) |
| IEEE | Abdelaziz A. Abdelhamid, Speech Emotions Recognition for Online Education, Journal of Fusion: Practice and Applications, Vol. 10 , No. 1 , (2023) : 78-87 (Doi : https://doi.org/10.54216/FPA.100104) |