From metrics to meaning in research evaluation

Abstract

The evaluation of research impact in higher education has long been dominated by citation-based metrics and their derivatives, such as impact factors and h-indices. While these measures offer the appeal of standardisation and comparability, they capture only a narrow dimension of scholarly influence and often reinforce global, disciplinary and linguistic inequities. The rise of altmetrics has promised a broader, more immediate view of research engagement, yet these too remain vulnerable to bias, manipulation and structural asymmetries that privilege well-resourced institutions and English-language scholarship. We argue that the problem lies not only in the tools of measurement but in the broader metrification of higher education itself, where numerical indicators have supplanted expert judgement and contextual understanding. Drawing on the South African context, we argue for a recalibrated approach to research impact assessment. We outline the principles that should inform the development of a flexible, context-sensitive framework that integrates quantitative and qualitative evidence. Such an approach aligns with global movements for responsible research assessment, including the San Francisco Declaration on Research Assessment (DORA) and the Leiden Manifesto, while foregrounding local priorities such as transformation, equity and societal relevance. Ultimately, we call for a shift from counting outputs to interpreting contributions, towards an evaluative culture grounded in judgement, context and inclusion.

Significance:

This paper advances the international movement for responsible research assessment by arguing for a context-sensitive framework that integrates quantitative and qualitative evidence. It situates global critiques of metrification within the South African higher education context, where transformation, equity and societal relevance are central concerns. By revealing how citation and altmetric indicators perpetuate structural inequities and distort academic priorities, the paper lays the groundwork for a holistic, inclusive, contextually grounded framework for evaluating research impact aligned with South Africa’s developmental and social justice imperatives.