The recent years' rapid development of sophisticated machine learning (ML) models has increased the interest in explainable AI (XAI), fueled by the risk that systems acting like black boxes could contribute to unjustifiable or illegitimate decisions and outputs. Despite the growing interest in XAI, there is ambiguity in the conceptual understanding of it. This might challenge the development of a cumulative research front enabling researchers to critically reflect on how the XAI-field is evolving. Also, looking at recent regulations pointing towards the importance of explainability, having a common understanding of XAI is of practical importance. This paper aims to explore how XAI could be conceptualized within the IS field and discuss the implications that follow. To do so, a three-step conceptual analysis involving both theoretical and empirical steps is undertaken resulting in a conceptual framework for XAI, framing it as a multidisciplinary, layered, and dynamic concept.