A day of exchanges on the assessment of scientists

"We're working to make sure the scientific assessment of researchers is no longer left to journal editors, however prestigious they might be", explains Sylvie Rousset, the director of CNRS's Open Research Data Department (DDOR) that was set up in 2020. This is why the CNRS's third annual conference on open science held on November 15th will be devoted to research assessment practices.

As every year, this event is a day for internal exchanges aimed primarily at the CNRS's institutes and more specifically their directors, deputy scientific directors and members of their scientific councils. The CNRS Scientific Council and the directors of the organisation's functional departments have also been invited along with the presidents of the sections of the National Committee for Scientific Research (CoNRS)¹. To facilitate discussion and debate, this 'in-person' conference will be held in Paris's Cordeliers campus for this restricted audience although the recorded presentations will be made available subsequently along with the conference proceedings.

"Assessment is one of the four fundamental pillars of the CNRS's Roadmap for Open Science", as Sylvie Rousset reminds us. This is why the CNRS is working to integrate "more qualitative and less quantitative" assessment criteria for researchers' work. Previously, assessment and promotion files featured endless lists of publications with an emphasis on the best-rated journals which are often more expensive for open access publishing or simply do not offer the possibility. Such lists are a thing of the past with the CNRS now intending to introduce four guiding principles for individual assessments of researchers.

The first is the assessment of research results themselves rather than whether these were published in a prestigious journal or not. Now, researchers will also be asked to present a small selection of significant results with an explanation of each result's scope and impact and their personal contribution to obtaining them. This is the second principle: "Communicating on scientific advances isn't best served by fuelling the mad race to achieve publication, the volume of which increases each year to the delight of publishers. It's much better to focus on a limited number of reliable, exhaustive and important results", explains the DDOR's director.

The third principle stipulates that research productions can also include pre-publications², conferences, research data, software and so forth. Such reports will also take into account other activities than the production of scientific results alone like training work, innovation, management or investment in open science. Finally, all research productions listed in application files must be available³ in open archives like HAL.

Sylvie Rousset considers these modifications to represent "a profound cultural shift" and believes the 'Open Science & Research Assessment' Day will give all stakeholders the opportunity to find out more about this subject and exchange views. Similarly, the CNRS intends to suggest that the recently elected presidents of the CoNRS's sections invited to attend "integrate" the four new main principles. "We are in phase with French universities which have made statements along the same lines as our Roadmap for Open Science and Research Data Plan".

This approach is also part of an "international movement", as Sylvie Rousset explains. The DORA⁴ declaration was signed by the CNRS in July 2018 along with over 20,000 assessment stakeholders in 148 countries including research organisations and individual scientists. These signatories agreed to make 18 main commitments. For example, signatory institutions must now explicitly display their recruitment, tenure and promotion criteria and their assessors have committed to taking decisions in these areas based on scientific content rather than on publication indicators or a given journal's brand image.

This subject is clearly very much on the rise. "The European Commission is also taking strong action on the assessment of scientists in Europe", adds Sylvie Rousset. In fact the CNRS has submitted a project with several European partners in response to a call for European projects on research assessment. Pilot experiments could therefore be organised at four European research organisations to find out which changes to research assessment would be the best received and the most effective on the European scale before making recommendations thereon. The Open Science European Conference OSEC 2022 will take place in Paris on February 4th and 5th 2022 and this issue of assessment is among the subject on the agenda. "France can and owes it to itself to make progress on this issue during the French Presidency of the European Union", concludes Sylvie Rousset.