The ongoing debate between "rigor” and “relevance” has been central in Information Systems (IS) research. Scholars argue that while rigorous scientific methods are essential, relevance is equally crucial to advancing the discipline. As an applied science, IS ought to be relevant beyond academia. However, ambiguity and differing opinions persist on what constitutes relevant research in IS - questions arise such as: How can IS researchers achieve relevant contributions that advance the field? and What criteria can be used to evaluate the relevance of IS research? This paper explores these questions by compiling and selecting assessment criteria from existing literature. Through an empirical application of this framework to top IS journals, we explore the field’s practical relevance. Results reveal limited engagement with practitioner concerns, a lack of directly applicable outputs, and minimal emphasis on practical implications. The study encourages IS research to position itself as practically impactful, without compromising its scientific robustness.