METRICS AND INDICATORS

New methods for evaluating research excellence are shifting away from purely bibliometric indicators (like journal impact factors) toward more holistic, qualitative, and impact-oriented approaches. Key developments focus on "responsible assessment," recognizing a wider range of research outputs, and evaluating research culture alongside, or instead of, individual outputs. 

Key Trends and New Frameworks

• Responsible Metrics (The Leiden Manifesto): Emphasizes that quantitative evaluation should support, not replace, qualitative expert assessment. This approach favors assessing performance against the specific goals of the researcher or institution rather than a universal metric.

• Narrative CVs and Contextualized Assessment: Moving away from checklists of publications to "narrative-style" CVs (e.g., Resume for Researchers) that allow researchers to explain their personal contributions, leadership, and diverse contributions to research.

• Research Excellence Framework (REF) 2028/2029 (UK): The upcoming UK framework proposes a shift from evaluating individuals to assessing institutions. The new weighting is expected to be 50% for contribution to knowledge (outputs), 25% for engagement and impact, and 25% for people, culture, and environment.

• CoARA and the Agreement on Reforming Research Assessment: A European-led initiative where over 580 organizations have pledged to update evaluation criteria within 5 years, prioritizing quality and impact over quantity, and promoting open science practices.

• The "Hidden REF" Movement: Aims to recognize all research outputs, not just journal articles, including datasets, software, code, and public engagement activities. 

Methods for Evaluating Impact and Quality

• Impact Case Studies: Structured narratives describing the "reach and significance" of research on the economy, society, or culture.

• Altmetrics and Open Peer Review: Utilizing data from social media, policy documents, and open peer-review platforms to track the broader societal and academic influence of research.

• Theory-Based Evaluation: Methods like Contribution Analysis or Process Tracing are used to understand how and why research led to a particular impact, rather than just measuring that it did.

• Participatory Approaches: Techniques like "Most Significant Change" are used to identify unexpected or qualitative impacts of research projects, often in international development contexts. 

Key Principles of New Approaches

• Broadening "Excellence": Recognizing that excellence can exist in applied research, teaching, and knowledge exchange, not just in basic research publications.

• Focus on Culture: Assessing whether research environments are inclusive, collaborative, and sustainable, rather than just productive.

• Diversity in Output: Valuing research outputs in varied forms, including digital products, creative works, and policy advice.