Experts Call for an Overhaul of the Researcher Evaluation System to Promote Open Science

International experts invited to speak at B·Debate—a joint initiative of Biocat and the ”la Caixa” Foundation—agree that the researcher evaluation system must be revised to promote open science, a movement devoted to making science effective, reproducible, transparent and more accessible to everyone.

The evaluation process that dictates the trajectory of a research career remains heavily focused on the number of publications and the impact factor of the journals where the articles appear. Various international movements—including the group behind the San Francisco Declaration on Research Assessment—have argued that this system needs to be revamped to improve the assessment of research quality and impact. Proponents of such a change argue that research evaluations should consider not only quantity but also quality.

At the B·Debate in Barcelona, René von Schomberg, the European Commission expert who coined the term responsible research and innovation (RRI), noted that the current evaluation system is effective at the individual level but not at the systemic level. This means, for example, that although a lot of research is published in Europe, it does not translate into similar levels of innovation. The European Commission therefore plans to invest in multi-stakeholder coalitions that will work cooperatively to find better solutions to complex, persistent problems.

On 4^th and 5^th October, Dr von Schomberg was joined at the B·Debate by representatives of various universities, research centres, publishing houses and funding bodies at CosmoCaixa in Barcelona. The speakers reflected on the research process, research evaluation and various ethical, legal and social factors, discussed the challenges and opportunities associated with the transition to an open-science model, and made proposals to promote this model in Catalonia, Spain and the rest of Europe. The B·Debate was co-organised by five institutions with experience in this realm—the Centre for Genomic Regulation (CRG), the University of Barcelona (UB), the Open University of Catalonia (UOC), the IrsiCaixa AIDS Research Institute and the Barcelona Institute for Global Health (ISGlobal)—all of which have been involved in European RRI and open-science projects.

The B·Debate participants recognised that the long, difficult journey ahead must begin with the development of a new evaluation model. The experts argued that this new model should be more flexible, that it should take the variety of research fields into account, that the content of scientific articles (as opposed to just their metrics) should be considered, and that the impact of research results should be viewed in context. These and other concerns expressed by the scientific community will be included in a recommendations document, to be drafted in the coming weeks, that will help local, regional, national and European institutions in their efforts to promote open science.

The governments of Spain and Catalonia both recently expressed their willingness to promote open science through the implementation of national plans. At the European level, last month the European Commission and 12 national R&D funding bodies announced the launch of Plan S. Under this plan, researchers who receive funds from the participating funders will be required to publish their results in open-access journals starting in 2020.