In this paper, we propose a framework that integrates XAI methods in higher education applications. The framework aims to provide clear, transparent, trustworthy AI-based systems. The framework we developed addresses different challenges that hinder the integration of XAI in education and utilising Design Science Research (DSR) to ensure the research rigour through the lens of affordance theory to examine the interaction between the AI-based application and the educators as end-users.