Promoting Open Science and Reforming Research Assessment from a European to a Local Context

On 23 February 2023, several researchers, policymakers, students, and administrators gathered at the PRBB auditorium for an ‘Open Science and Research Assessment’ workshop. The event was organised by ISGlobal in the context of the projects REDES and Severo Ochoa, with the collaboration of the MELIS-UPF and the Group of Good Scientific Practice at PRBB.

Though the idea of research has significantly evolved in the past decades, systems that evaluate research have not. Researchers and institutions are still trapped in conventional assessment practices marked by metrics such as publishing in journals with high impact factors. The ‘publish or perish’ culture fails to adequately recognise and reward inputs and research activities outside the realm of publishing. But there are winds of change. The workshop highlighted some of the ongoing efforts to create holistic and fairer research assessment systems.

An EU Initiative to Reform Research Assessment

On 20 July 2022, the Agreement on Reforming Research Assessment was presented by the Coalition for Advancing Research Assessment (CoARA), so far signed by 359 organisations from 40 countries, including ISGlobal. In his keynote address, Javier Lopez Albacete ( European Commission Policy Officer for Open Science) discussed the journey of reforming research assessment at the EU level and presented the new agreement which aims to put quality and impact at the centre of the assessment process, by recognising the diversity of contributions and research careers, abandoning the inappropriate use of publication-based metrics, and basing research assessment primarily on qualitative evaluation. “We have to make it normative, easy, and possible to increase the uptake of open science and make it the new normal,” concluded Javier.

What are others doing – the example of Utrecht University

The second keynote speaker of the workshop was Professor Paul Boselie from Utrecht University’s School of Governance. He demonstrated how universities could steer away from the use of impact factors while making hiring and promoting decisions. “The world does not benefit from scientists being one trick ponies,” argued Boselie. By reforming the rewards and recognition system, researchers are encouraged to collaborate and dedicate more time to other crucial activities like teaching and translation. Utrecht University is currently pioneering the TRIPLE model for rewards and recognition based on the principles of Team, Research, Impact, Professional Performance, Leadership, and Education.

Watch the keynote addresses by Javier Lopez Albacete and Professor Paul Boselie in this video.

What is happening in the local context?

The final section of the workshop was a roundtable discussion with Ignasi López (Director of the Department of Research and Innovation from “la Caixa” Foundation), Gonzalo Arévalo, (Director General of Planning, Monitoring and Evaluation at the Spanish Ministry of Science and Innovation), Lluis Rovira (Director of CERCA), and Emilià Pola (Executive Director of ICREA), moderated by Michela Bertero (Director of Strategy at IDIBAPS). 

From left to right: Emilià Pola, Lluis Rovira, Michela Bertero, Gonzalo Arévalo, Ignasi López

The representatives of the funding agencies highlighted how their selection process is transitioning to consider metrics and outcomes beyond publications when it comes to funding new projects. Gonzalo Arévalo reiterated, “Change is going to come, and we need to be ready to adapt to them.” 

In Catalunya, institutions like CERCA and ICREA are already implementing alternate assessment systems that take into account outcomes such as collaborative endeavours, knowledge transfer and outreach activities, and efforts to achieve gender balance.

The question-answer session of the workshop addressed related issues including some of the flaws of the peer review system and how funding agencies ensure a robust and diverse panel of evaluators.

Watch the round table discussion in this video.

ISGlobal’s journey

ISGlobal’s commitment to the principles of open science and fair recognition of research dates back to 2018 when the organisation signed the Declaration on Research Assessment and committed to the Leiden Manifesto, as reflected in the ‘Tenure Track’ researchers’ evaluation and promotion policy. Eva Casamitjana, one of the organisers of the workshop explained, “We are proud to be one of the earliest Spain-based signatories of the Agreement on Research Assessment Reform. This workshop aimed at fostering debate and networking among our local and national research community for the transition to an improved research assessment system.”