New flexible research framework can improve reproducibility and transparency
Woohoo – you’ve finally submitted the article you’ve worked on for years to your favorite peer-reviewed journal. Three months later, you receive a rejection letter, not for the content, but because you have not provided the required guarantees. But wait, you don’t need to provide this to the last journal you submitted work to, or the one before … so why does this one need it?
Since there is no standard publication framework, different journals have different guidelines. There are existing guidelines, but these are often specialized standards like ARRIVE, which covers animal research, and CONSORT, which is associated with clinical trial notification. This has led to a fragmented scientific publishing landscape that has increased the burden on authors and publishers.
Working with six renowned journals, a team from the University of Edinburgh and the Center for Open Science has developed a new framework that they believe will harmonize the recording of results and improve reproducibility, replication and transparency in results. life sciences. The new MDAR (Research Materials, Design, Analysis and Reporting) framework is presented in a new publication in PNAS.
“Improving research is a challenge; it requires continuous effort, adapting to the changing demands and circumstances of the times, ”said Malcolm Macleod, study author and academic leader for research improvement and research integrity. research at the University of Edinburgh. sufficient, but we hope that the MDAR framework can contribute to the range of initiatives [that] support improvement. “
The heart of the framework is its flexibility and ease of adoption. In its current structure, MDAR has three separate outputs. The framework itself sets out minimum requirements and best practice recommendations in the four key areas that make up its name. The optional checklist helps operationalize the framework by serving as an implementation tool to help authors comply with journal policies and editors and reviewers assess reporting and adherence to the standard. Finally, the development document is a user guide that provides context for the framework.
“In many ways, the MDAR framework is a joint iteration of our collective previous experience with guidelines, checklists and editorial requirements towards a harmonized and practical guideline for minimum reporting requirements,” explain the researchers in their article.
To gain valuable feedback on the MDAR checklist and ensure its practicality, the team piloted the framework in 13 journals and 289 manuscripts. According to the article, 80% of the 211 matching authors who responded to the team’s survey found the checklist “very” or “somewhat” helpful. Similarly, 84% of the 33 writers who participated in the pilot also found the checklist “very” or “somewhat” useful.
An important takeaway from the pilot program was that only 15 of the 42 checklist items were considered relevant for more than half of the 289 manuscripts. While study participants suggested organizing the checklist in a nested fashion to allow users to easily skip irrelevant sections, the authors concluded on a different approach given their emphasis on flexibility. The team said they have decided that “the best organization could be journal specific” and will leave the option as an implementation decision for journals that choose to use the checklist.
Also in the name of flexibility, the research team opted for three increasingly stringent implementation levels for journals – the recommended term, the limited term, and the full term.
While the ultimate goal is to report transparently in journals to improve research, the team envisions the MDAR framework being used by other stakeholders in the life science community. They hypothesize that it could be used by researchers to design, conduct, analyze and report studies; by institutions to teach best research practices; and by funders and others involved in research evaluation to help assess rigor and reporting.
Finally, the tool has the potential to be particularly impactful in the ever-growing field of scientific pre-impressions.
“With the growth of pre-prints in recent years, pre-print servers are ideally placed to collect (and publish) an MDAR checklist completed by the author, who could then travel with the manuscript when submitted. to a journal, ”write the authors.
The full set of MDAR resources is available in a collection on the Open science framework. The authors say it will be maintained and updated as a community resource, evolving further over time and implementation rates increase.
“We have identified a basic minimum requirement as well as an ambitious set of best practices that provide guidance guidance for the future evolution of requirements. Over time, we expect that items currently identified as “best practices” will instead be identified as a “baseline minimum requirement” as reporting performance improves, “the authors conclude.