Fusion: Practice and Applications FPA 2692-4048 2770-0070 10.54216/FPA https://www.americaspg.com/journals/show/4193 2018 2018 Department of Information and Communication Technology, School of Technology, Pandit Deendayal Energy University, Gandhinagar, India Astha Astha Department of Information and Communication Technology, School of Technology, Pandit Deendayal Energy University, Gandhinagar, India Astha Soni School of Computer Science and Engineering, VIT-AP University, Amaravati, 522237, Andhra Pradesh, India Gokul Yenduri Department of Computer Science and Engineering, School of Technology, Pandit Deendayal Energy University, Gandhinagar, India; Department of Applied Information Systems School of Consumer Intelligence and Information Systems, College of Business & Economics, University of Johannesburg, Johannesburg, South Africa Rutvij H. Jhaveri Department of Applied Information Systems School of Consumer Intelligence and Information Systems, College of Business & Economics, University of Johannesburg, Johannesburg, South Africa Stella Bvuma The ever-worsening mortality rates due to various diseases such as heart disease, breast cancer, and kidney disease are of great concern. Early diagnosis of the disease can be of great help. This process can be automated with the help of Artificial intelligence (AI). But, the main worry of using AI in healthcare is its black-box behaviour. The majority of the models characterized by high accuracy are often black-box in nature. This can be overcome by the use of eXplainable Artificial Intelligence (XAI), which is capable of explaining the predictions made by these black box models. We have exploited 3 different XAI frameworks: SHAP, LIME, and DALEX, to understand the working and the facilities provided by the three frameworks and compare them. We have used 5 disease datasets (3 heart disease, 1 cancer and 1 kidney disease) to carry out our work. Each dataset was trained with 3 machine learning models, namely Support Vector Machine (SVM), Logistic regression (LR), and K-Nearest neighbours (KNN), and the best model was used to feed to the XAI framework. LR performed best for one of the heart disease datasets with 72.31%accuracy, while SVM outperformed in all the other datasets, thus proving the efficacy of such approaches for early disease prediction. 2025 2025 171 199 10.54216/FPA.200213 https://www.americaspg.com/articleinfo/3/show/4193