215 106
Full Length Article
Journal of Artificial Intelligence and Metaheuristics
Volume 7 , Issue 2, PP: 08-17 , 2024 | Cite this article as | XML | Html |PDF

Title

Exploring Optimization Algorithms: A Review of Methods and Applications

  Abdulrahman Abdullah Farag 1 * ,   Ziad Mohammed Ali 2 ,   Ahmed Mohamed Zaki 3 ,   Faris H.Rizk 4 ,   Marwa M. Eid 5 ,   EL-Sayed M. EL-Kenawy 6

1  Department of Communications and Electronics, Delta Higher Institute of Engineering and Technology (DHIET), Mansoura 35111, Egypt
    (CH2100197@dhiet.edu.eg)

2  Department of Communications and Electronics, Delta Higher Institute of Engineering and Technology (DHIET), Mansoura 35111, Egypt
    (CH2100056@dhiet.edu.eg)

3  Computer Science and Intelligent Systems Research Center, Blacksburg 24060, Virginia, USA
    (azaki@jcsis.org)

4  Computer Science and Intelligent Systems Research Center, Blacksburg 24060, Virginia, USA
    (farisrizk@jcsis.org)

5  Faculty of Artificial Intelligence, Delta University for Science and Technology, Mansoura 35111, Egypt
    (mmm@ieee.org)

6  Department of Communications and Electronics, Delta Higher Institute of Engineering and Technology (DHIET), Mansoura 35111, Egypt
    (skenawy@ieee.org)

Full Text Download


Doi   :   https://doi.org/10.54216/JAIM.070201

Received: May 13, 2023 Revised: September 12, 2023 Accepted: February 10, 2024

Abstract :

This article review focuses on feature selection as the main parameter that plays a major role in tuning machine learning models. Several algorithms of optimization such as MFO (Moth-Flame Optimization), the GA-GSA algorithm’s hybrid type, SOA (Seagull Optimization Algorithm), WOA (Whale Optimization Algorithm), GOA (Grasshopper Optimization Algorithm), HGSO (Henry Gas Solubility Optimization), and SafeOpt are widely used in engineering design, power systems scheduling, The paper stresses the importance of optimization in improving efficiency, lessening mistakes and increasing understandability of machine learning models. The literature addresses the widest directions in the usage of optimization for the following fields of science such as structural engineering, additive manufacturing, and landslide susceptibility mapping. A comprehensive summary table is generated, which shows an overview of each study, algorithm, focus, and methodology and has a stoke of key findings. The conclusions reveal the adaptiveness, competitiveness and compossibility of the optimization algorithms applied to a wide range of domains. The summary shows how optimization has the potential to change decision-making processes and activities by being a decisive factor that determines the future of branches of various industries. The main objective of this work is to direct researchers and practitioners by providing them with some innovative ideas and approaches and offering insights on the existing cutting-edge approaches while laying the groundwork for future innovations in optimization.

Keywords :

Optimization Algorithms; Metaheuristic Optimization Methods; Deep Learning; Optimization Applications; Machine learning models.

References :

[1]      Van Geit, W., De Schutter, E., & Achard, P. (2008). Automated neuron model optimization techniques: A review. Biological Cybernetics, 99(4), 241–251. https://doi.org/10.1007/s00422-008-0257-6

[2]      Van Horenbeek, A., Pintelon, L., & Muchiri, P. (2010). Maintenance optimization models and criteria. International Journal of System Assurance Engineering and Management, 1(3), 189–200. https://doi.org/10.1007/s13198-011-0045-x

[3]      Alexandrov, N. M., & Lewis, R. M. (2001). An Overview of First-Order Model Management for Engineering Optimization. Optimization and Engineering, 2(4), 413–430. https://doi.org/10.1023/A:1016042505922

[4]      Theng, D., & Bhoyar, K. K. (2024). Feature selection techniques for machine learning: A survey of more than two decades of research. Knowledge and Information Systems, 66(3), 1575–1637. https://doi.org/10.1007/s10115-023-02010-5

[5]      Bengio, Y., Lodi, A., & Prouvost, A. (2021). Machine learning for combinatorial optimization: A methodological tour d’horizon. European Journal of Operational Research, 290(2), 405–421. https://doi.org/10.1016/j.ejor.2020.07.063

[6]      Hamdia, K. M., Zhuang, X., & Rabczuk, T. (2021). An efficient optimization approach for designing machine learning models based on genetic algorithm. Neural Computing and Applications, 33(6), 1923–1933. https://doi.org/10.1007/s00521-020-05035-x

[7]      On hyperparameter optimization of machine learning algorithms: Theory and practice—ScienceDirect. (n.d.). Retrieved February 28, 2024, from https://www.sciencedirect.com/science/article/abs/pii/S0925231220311693

[8]      Kaur, K., Singh, U., & Salgotra, R. (2020). An enhanced moth flame optimization. Neural Computing and Applications, 32(7), 2315–2349. https://doi.org/10.1007/s00521-018-3821-6

[9]      Sahoo, S. K., Saha, A. K., Ezugwu, A. E., Agushaka, J. O., Abuhaija, B., Alsoud, A. R., & Abualigah, L. (2023). Moth Flame Optimization: Theory, Modifications, Hybridizations, and Applications. Archives of Computational Methods in Engineering, 30(1), 391–426. https://doi.org/10.1007/s11831-022-09801-z

[10]   Ebrahim, M. A., Becherif, M., & Abdelaziz, A. Y. (2018). Dynamic performance enhancement for wind energy conversion system using Moth-Flame Optimization based blade pitch controller. Sustainable Energy Technologies and Assessments, 27, 206–212. https://doi.org/10.1016/j.seta.2018.04.012

[11]   Zamani, H., Nadimi-Shahraki, M. H., Mirjalili, S., Soleimanian Gharehchopogh, F., & Oliva, D. (2024). A Critical Review of Moth-Flame Optimization Algorithm and Its Variants: Structural Reviewing, Performance Evaluation, and Statistical Analysis. Archives of Computational Methods in Engineering. https://doi.org/10.1007/s11831-023-10037-8

[12]   Khadanga, R. K., & Satapathy, J. K. (2015). A new hybrid GA–GSA algorithm for tuning damping controller parameters for a unified power flow controller. International Journal of Electrical Power & Energy Systems, 73, 1060–1069. https://doi.org/10.1016/j.ijepes.2015.07.016

[13]   Dasgupta, K., Mandal, B., Dutta, P., Mandal, J. K., & Dam, S. (2013). A Genetic Algorithm (GA) based Load Balancing Strategy for Cloud Computing. Procedia Technology, 10, 340–347. https://doi.org/10.1016/j.protcy.2013.12.369

[14]   Elsayed, S. M., Sarker, R. A., & Essam, D. L. (2014). A new genetic algorithm for solving optimization problems. Engineering Applications of Artificial Intelligence, 27, 57–69. https://doi.org/10.1016/j.engappai.2013.09.013

[15]   Hwang, S.-F., & He, R.-S. (2006). A hybrid real-parameter genetic algorithm for function optimization. Advanced Engineering Informatics, 20(1), 7–21. https://doi.org/10.1016/j.aei.2005.09.001

[16]   Hatamlou, A., Abdullah, S., & Nezamabadi-pour, H. (2012). A combined approach for clustering based on K-means and gravitational search algorithms. Swarm and Evolutionary Computation, 6, 47–52. https://doi.org/10.1016/j.swevo.2012.02.003

[17]   Naserbegi, A., Aghaie, M., Minuchehr, A., & Alahyarizadeh, G. (2018). A novel exergy optimization of Bushehr nuclear power plant by gravitational search algorithm (GSA). Energy, 148, 373–385. https://doi.org/10.1016/j.energy.2018.01.119

[18]   Ewees, A. A., Mostafa, R. R., Ghoniem, R. M., & Gaheen, M. A. (2022). Improved seagull optimization algorithm using Lévy flight and mutation operator for feature selection. Neural Computing and Applications, 34(10), 7437–7472. https://doi.org/10.1007/s00521-021-06751-8

[19]   Dhiman, G., Singh, K. K., Slowik, A., Chang, V., Yildiz, A. R., Kaur, A., & Garg, M. (2021). EMoSOA: A new evolutionary multi-objective seagull optimization algorithm for global optimization. International Journal of Machine Learning and Cybernetics, 12(2), 571–596. https://doi.org/10.1007/s13042-020-01189-1

[20]   Long, W., Jiao, J., Liang, X., Xu, M., Tang, M., & Cai, S. (2022). Parameters estimation of photovoltaic models using a novel hybrid seagull optimization algorithm. Energy, 249, 123760. https://doi.org/10.1016/j.energy.2022.123760

[21]   Che, Y., & He, D. (2022). An enhanced seagull optimization algorithm for solving engineering optimization problems. Applied Intelligence, 52(11), 13043–13081. https://doi.org/10.1007/s10489-021-03155-y

[22]   Nadimi-Shahraki, M. H., Zamani, H., Asghari Varzaneh, Z., & Mirjalili, S. (2023). A Systematic Review of the Whale Optimization Algorithm: Theoretical Foundation, Improvements, and Hybridizations. Archives of Computational Methods in Engineering, 30(7), 4113–4159. https://doi.org/10.1007/s11831-023-09928-7

[23]   Salgotra, R., Singh, U., & Saha, S. (2019). On Some Improved Versions of Whale Optimization Algorithm. Arabian Journal for Science and Engineering, 44(11), 9653–9691. https://doi.org/10.1007/s13369-019-04016-0

[24]   Chakraborty, S., Kumar Saha, A., Sharma, S., Mirjalili, S., & Chakraborty, R. (2021). A novel enhanced whale optimization algorithm for global optimization. Computers & Industrial Engineering, 153, 107086. https://doi.org/10.1016/j.cie.2020.107086

[25]   Yildiz, B. S., Pholdee, N., Bureerat, S., Yildiz, A. R., & Sait, S. M. (2022). Enhanced grasshopper optimization algorithm using elite opposition-based learning for solving real-world engineering problems. Engineering with Computers, 38(5), 4207–4219. https://doi.org/10.1007/s00366-021-01368-w

[26]   Veza, I., Deniz Karaoglan, A., Ileri, E., Afzal, A., Tuan Hoang, A., Tamaldin, N., Gazali Herawan, S., Abbas, M. M., & Farid Muhamad Said, M. (2022). Multi-objective optimization of diesel engine performance and emission using grasshopper optimization algorithm. Fuel, 323, 124303. https://doi.org/10.1016/j.fuel.2022.124303

[27]   Sukhija, B., Turchetta, M., Lindner, D., Krause, A., Trimpe, S., & Baumann, D. (2023). GoSafeOpt: Scalable safe exploration for global optimization of dynamical systems. Artificial Intelligence, 320, 103922. https://doi.org/10.1016/j.artint.2023.103922

[28]   Shehab, M., Abualigah, L., Al Hamad, H., Alabool, H., Alshinwan, M., & Khasawneh, A. M. (2020). Moth–flame optimization algorithm: Variants and applications. Neural Computing and Applications, 32(14), 9859–9884. https://doi.org/10.1007/s00521-019-04570-6

[29]   Garg, H. (2019). A hybrid GSA-GA algorithm for constrained optimization problems. Information Sciences, 478, 499–523. https://doi.org/10.1016/j.ins.2018.11.041

[30]   Dhiman, G., & Kumar, V. (2019). Seagull optimization algorithm: Theory and its applications for large-scale industrial engineering problems. Knowledge-Based Systems, 165, 169–196. https://doi.org/10.1016/j.knosys.2018.11.024

[31]   Gharehchopogh, F. S., & Gholizadeh, H. (2019). A comprehensive survey: Whale Optimization Algorithm and its applications. Swarm and Evolutionary Computation, 48, 1–24. https://doi.org/10.1016/j.swevo.2019.03.004

[32]   Katebi, J., Shoaei-parchin, M., Shariati, M., Trung, N. T., & Khorami, M. (2020). Developed comparative analysis of metaheuristic optimization algorithms for optimal active control of structures. Engineering with Computers, 36(4), 1539–1558. https://doi.org/10.1007/s00366-019-00780-7

[33]   Abualigah, L., & Diabat, A. (2020). A comprehensive survey of the Grasshopper optimization algorithm: Results, variants, and applications. Neural Computing and Applications, 32(19), 15533–15556. https://doi.org/10.1007/s00521-020-04789-8

[34]   Hashim, F. A., Houssein, E. H., Mabrouk, M. S., Al-Atabany, W., & Mirjalili, S. (2019). Henry gas solubility optimization: A novel physics-based algorithm. Future Generation Computer Systems, 101, 646–667. https://doi.org/10.1016/j.future.2019.07.015

[35]   Nazir, A., Abate, K. M., Kumar, A., & Jeng, J.-Y. (2019). A state-of-the-art review on types, design, optimization, and additive manufacturing of cellular structures. The International Journal of Advanced Manufacturing Technology, 104(9), 3489–3510. https://doi.org/10.1007/s00170-019-04085-3

[36]   Berkenkamp, F., Krause, A., & Schoellig, A. P. (2023). Bayesian optimization with safety constraints: Safe and automatic parameter tuning in robotics. Machine Learning, 112(10), 3713–3747. https://doi.org/10.1007/s10994-021-06019-1

[37]   Moayedi, H., Mehrabi, M., Mosallanezhad, M., Rashid, A. S. A., & Pradhan, B. (2019). Modification of landslide susceptibility mapping using optimized PSO-ANN technique. Engineering with Computers, 35(3), 967–984. https://doi.org/10.1007/s00366-018-0644-0

[38]   Nguyen, H., Moayedi, H., Foong, L. K., Al Najjar, H. A. H., Jusoh, W. A. W., Rashid, A. S. A., & Jamali, J. (2020). Optimizing ANN models with PSO for predicting short building seismic response. Engineering with Computers, 36(3), 823–837. https://doi.org/10.1007/s00366-019-00733-0

[39]   Khari, M., Jahed Armaghani, D., & Dehghanbanadaki, A. (2020). Prediction of Lateral Deflection of Small-Scale Piles Using Hybrid PSO–ANN Model. Arabian Journal for Science and Engineering, 45(5), 3499–3509. https://doi.org/10.1007/s13369-019-04134-9

[40]   Lv, L., Chen, T., Dou, J., & Plaza, A. (2022). A hybrid ensemble-based deep-learning framework for landslide susceptibility mapping. International Journal of Applied Earth Observation and Geoinformation, 108, 102713. https://doi.org/10.1016/j.jag.2022.102713

Cite this Article as :
Style #
MLA Abdulrahman Abdullah Farag, Ziad Mohammed Ali , Ahmed Mohamed Zaki, Faris H.Rizk, Marwa M. Eid, EL-Sayed M. EL-Kenawy. "Exploring Optimization Algorithms: A Review of Methods and Applications." Journal of Artificial Intelligence and Metaheuristics, Vol. 7, No. 2, 2024 ,PP. 08-17 (Doi   :  https://doi.org/10.54216/JAIM.070201)
APA Abdulrahman Abdullah Farag, Ziad Mohammed Ali , Ahmed Mohamed Zaki, Faris H.Rizk, Marwa M. Eid, EL-Sayed M. EL-Kenawy. (2024). Exploring Optimization Algorithms: A Review of Methods and Applications. Journal of Journal of Artificial Intelligence and Metaheuristics, 7 ( 2 ), 08-17 (Doi   :  https://doi.org/10.54216/JAIM.070201)
Chicago Abdulrahman Abdullah Farag, Ziad Mohammed Ali , Ahmed Mohamed Zaki, Faris H.Rizk, Marwa M. Eid, EL-Sayed M. EL-Kenawy. "Exploring Optimization Algorithms: A Review of Methods and Applications." Journal of Journal of Artificial Intelligence and Metaheuristics, 7 no. 2 (2024): 08-17 (Doi   :  https://doi.org/10.54216/JAIM.070201)
Harvard Abdulrahman Abdullah Farag, Ziad Mohammed Ali , Ahmed Mohamed Zaki, Faris H.Rizk, Marwa M. Eid, EL-Sayed M. EL-Kenawy. (2024). Exploring Optimization Algorithms: A Review of Methods and Applications. Journal of Journal of Artificial Intelligence and Metaheuristics, 7 ( 2 ), 08-17 (Doi   :  https://doi.org/10.54216/JAIM.070201)
Vancouver Abdulrahman Abdullah Farag, Ziad Mohammed Ali , Ahmed Mohamed Zaki, Faris H.Rizk, Marwa M. Eid, EL-Sayed M. EL-Kenawy. Exploring Optimization Algorithms: A Review of Methods and Applications. Journal of Journal of Artificial Intelligence and Metaheuristics, (2024); 7 ( 2 ): 08-17 (Doi   :  https://doi.org/10.54216/JAIM.070201)
IEEE Abdulrahman Abdullah Farag, Ziad Mohammed Ali, Ahmed Mohamed Zaki, Faris H.Rizk, Marwa M. Eid, EL-Sayed M. EL-Kenawy, Exploring Optimization Algorithms: A Review of Methods and Applications, Journal of Journal of Artificial Intelligence and Metaheuristics, Vol. 7 , No. 2 , (2024) : 08-17 (Doi   :  https://doi.org/10.54216/JAIM.070201)
↑