Implementation of Facial Emotion Recognition System using CNN and IoT

Full Length Article

•

Implementation of Facial Emotion Recognition System using CNN and IoT

DOI

format_quote Cite this article

View PDF open_in_new

Abstract

A lot of effort has been paid to emotion modelling and recognition by fields including psychology, cognitive science, and, more recently, engineering. While behavioral modalities have been the subject of extensive investigation, physiological signals have received less attention. Electrocardiograph (ECG) signals can vary depending on the emotion, and different emotions can be identified by different changes in ECG signals. The goal of this study is to use ECG signals to recognize emotions. Four different emotions are represented by the data: happy, thrilling, tranquil, and tense. A finite impulse filter is then used to de-noise the raw data. To improve the accuracy of emotion recognition, we utilize the Discrete Cosine Transform (DCT) to extract characteristics from the collected data. Electrocardiograms (ECGs) and GSR are used in this project's emotion recognition research as both a unimodal and multimodal approach to emotion recognition systems. There are critical observations made of the following processes: pre-processing, validation, dimensionality reduction, feature extraction, feature selection, and data collecting. Additionally, this project showcases architectures with accuracy levels greater than 90%. Also evaluated are the existing ECG and GSR inclusive emotional databases, and a popularity analysis is provided. This review also covers the advantages of emotion recognition technologies for healthcare systems. We conclude with a full discussion of the topic and recommendations for future work based on the evaluated literature. The results offered here are helpful for aspiring researchers looking to review the overview of earlier studies on ECG and GSR -based emotion recognition systems, identify knowledge gaps, and develop and design future applications of emotion recognition systems, particularly for enhancing healthcare.

Keywords

ECG GSR CNN Normalization Facial Expressions IoT.

References

[1] R.BA (ed) (1995).” Substance Abuse and Mental Health Statistics Sourcebook” (DHHS Publication No. SMA 95-3064). Washington, DC, U.S. Government Printing Office

[2] B.Ahmed, “Depression and anxiety: a snapshot of the situation in Pakistan,” International Journal of Neuroscience and Behavioral Science, February 2016

[3] N.Bach,”World Mental Health Day 2017: Illness in the Workplace Is More Common Than You Might Think”,[Online].Available: https://fortune.com/2017/10/10/world-mental-health-day-2017- workplace- depression-anxiety/

[4] A.Greco, “Affective computing in virtual reality:emotion recognition from brain and heartbeat dynamics using wearable sensors,” SCIENTIFIC REPORTS ,September 2018.

[5] E.Andre, “Emotion Recognition Based on Physiological Changes in Music Listening,” IEEE Transactions on Pattern Analysis and Machine Intelligence,January 2009

[6] P.Tzirakis, G.Trigeorgis, M.A.Nicolaou, B. W. Schuller, and S. Zafeiriou,”End-to-end multimodal emotion recognition using deep neural networks”, IEEE Journal of Selected Topics in Signal Processing, 11(8):1301–1309, 2017

[7] H.Ferdinando, T.Seppanen, and E.Alasaarela,” Enhancing emotion recognition from ecg signals using supervised dimensionality reduction”, In ICPRAM, p.112–118, 2017

[8] Y.Xu and G.Yuan Liu,”A method of emotion recognition based on ecg signal. In Computational Intelligence and Natural Computing”, 2009.CINC'09, International Conference on, vol. 1, p. 202-205, IEEE,2009

[9] S.Katsigiannis and N.Ramzan,”Dreamer: a database for emotion recognition through EEG and ECG signals from wireless low-cost o_- the-shelf devices”, IEEE journal of biomedical and health informatics, 2018.

[10] Busso, C.; Deng, Z.; Yildirim, S.; Bulut, M.; Lee, C.M.; Kazemzadeh, A.; Lee, S.; Neumann, U.; Narayanan, S. Analysis of Emotion Recognition using Facial Expressions, Speech and Multimodal Information. In Proceedings of the 6th International Conference on Multimodal Interfaces, Sorrento, Italy, 25–29 November 2012; pp. 205–211

[11] Tivatansakul, S.; Ohkura, M. Improvement of emotional healthcare system with stress detection from ECG signal. In Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS, Milan, Italy, 25–29 August 2015.

[12] Minhad, K.N.; Ali, S.H.M.D.; Reaz, M.B.I. A design framework for human emotion recognition using electrocardiogram and skin conductance response signals. J. Eng. Sci. Technol. 2017, 12, 3102–3119

[13] Alemi, O.; Li, W.; Pasquier, P. Affect-expressive movement generation with factored conditional Restricted Boltzmann Machines. In Proceedings of the 2015 International Conference on Affective Computing and Intelligent Interaction, ACII, Xi’an, China, 21–24 September 2015

[14] Lang, P.J.; Bradley, M.M.; Cuthbert, B.N. International Affective Picture System (IAPS): Instruction Manual and Affective Ratings; University of Florida: Gainesville, FL, USA, 2005.

[15] Mahmoodabadi, S.Z.; Ahmadian, A.; Abolhasani, M.D.; Eslami, M.; Bidgoli, J.H. ECG feature extraction based on multiresolution wavelet transform. In Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology, Shanghai, China, 17–18 January 2006; pp. 3902–3905.

Cite This Article

Choose your preferred format

format_quote

. "Implementation of Facial Emotion Recognition System using CNN and IoT." Journal of Intelligent Systems and Internet of Things, vol. , no. , , pp. . DOI:

(). Implementation of Facial Emotion Recognition System using CNN and IoT. Journal of Intelligent Systems and Internet of Things, (), . DOI:

. "Implementation of Facial Emotion Recognition System using CNN and IoT." Journal of Intelligent Systems and Internet of Things , no. (): . DOI:

() 'Implementation of Facial Emotion Recognition System using CNN and IoT', Journal of Intelligent Systems and Internet of Things, (), pp. . DOI:

. Implementation of Facial Emotion Recognition System using CNN and IoT. Journal of Intelligent Systems and Internet of Things. ;():. DOI:

, "Implementation of Facial Emotion Recognition System using CNN and IoT," Journal of Intelligent Systems and Internet of Things, vol. , no. , pp. , . DOI:

Digital Archive Ready