Updated ARRIVE guidelines: reproducibility and animal research

Why we need to ARRIVE at reproducible experiments

Amy Beale

14 July 2020 saw the publication of ARRIVE 2.0 (Animal Research: Reporting of In Vivo Experiments) – a revised checklist of recommendations to improve the reporting of research involving animals. The guidelines aim to maximise the quality and reliability of published research, and enable others to better scrutinise, evaluate and reproduce it.

FRAME Scientific Liaison Officer Amy Beale discusses the issue and consequences of poor reproducibility in animal research and why reproducibility needs to become a priority for research teams.

Reproducible experiments

I spent over 15 years in the classroom teaching science to GCSE students. As well as the subjects of chemistry, biology and physics, the science curriculum has always included a section on ‘working scientifically.’ This section includes requirements for specifications to develop scientific thinking, experimental skills, and opportunities to carry out analysis and evaluation through theory and practical work in all the sciences. When I studied science and began teaching it, we used to use the term ‘reliable’ to explain why we repeated a measurement or experiment, to check the results were correct. Over the past ten years, ‘reliable’ has been replaced with ‘repeatable’ and ‘reproducible’, so that it aligns with the terminology used within academia and industry.

The concept of collecting data that is reproducible is so important that we teach it to 11-year olds, yet, sadly, it seems it is not rigorously applied by all researchers or mandated thoroughly by all journals. When research uses animals, this becomes more critical as irreproducible research is effectively useless and should be regarded as a waste of animal lives. Whilst it is heartening to know that this problem is being acknowledged and addressed within the scientific community, it feels like more can be done.

ARRIVE Guidelines 2.0

Last week the revamped ARRIVE (Animal Research: Reporting of In Vivo Experiments) guidelines 2.0 were published. NC3Rs developed the ARRIVE guidelines in 2010 to try and improve the quality of published animal research by providing a list of the minimum information necessary to include when reporting in vivo (animal) experiments. Despite widespread acknowledgement that the guidelines could be highly impactful, there is still a lack of transparency in animal research. The ARRIVE guidelines are an underutilised tool which could help to reduce animal use in research by preventing unethical and wasted animal research projects.

The reproducibility issue

Poor reproducibility is a widely accepted problem in animal research, particularly when it impedes preclinical testing. A study or experiment is reproducible if it gives the same results when replicated using the same method, under the same conditions. This might be by the same scientist or an independent researcher. Much published research has been shown to have poor reproducibility.

Why should research be reproducible?

If experiments and data are reproducible it means the findings are more likely to be correct. It means other people can check the research and data, science can move on and future studies can build on the research. In Isaac Newton’s words, we can see further by ‘standing on the shoulders of giants.’ If research is not reproducible, scientific progress is hampered.

What causes experiments to have poor reproducibility?

There can be many reasons. When using live animals, there will always be some level of variation between individuals that cannot be accounted for and therefore may produce measurements that cannot be replicated. However, shockingly, in some published research, the lack of reproducibility arises from avoidable errors. This can be from poor study design, insufficient sample size, low power, unconscious bias, poor data analysis, or poor reporting of the study. Basically, poor science, or poor reporting of that science can provide data that cannot be replicated. These are issues that can be addressed and improved.

Alongside this, there are other issues which can impact experimental design or reporting, such as financial constraints leading to the paring down of study design or the pressure from funders, journals and institutes to produce interesting outcomes and have papers published.

What are the consequences of irreproducibility?

Irreproducibility leads to experiments providing inconclusive outcomes, or in the worst-case scenario, completely useless data. This is still happening in animal research published in scientific journals. This leads to wasted lives where laboratory animals have essentially been used for no good reason. This is obviously unethical, particularly when it arises from avoidable errors. It also sadly means that nothing is learnt from that study and no progress is made, so others may repeat it in different ways to try and answer the same question, often using more animals. This is a concern when so much work is done to support the implementation of the 3Rs, reduce animal use and report on the numbers of animals used each year in scientific research. If we can improve reproducibility in animal experiments, we can improve the value of the research, reduce future animal use in that area, and encourage a move towards the use of non-animal methods.

There are also financial advantages to be had – if outcomes of experiments are reproducible, then funding has been well spent. This is particularly important in drug discovery and preclinical research where one does not want to go back to the drawing board due to unreliable or inconclusive data.

Researchers are also responsible for maintaining the trust of the public. As the current COVID-19 pandemic demonstrates, science and public understanding has to be built on facts. There is a level of trust in the scientific community which is at risk of erosion if a lack of reproducibility renders scientific claims worthless.

What can we do?

The onus is on everyone involved in research to make reproducibility a priority. Educators must ensure young scientists learn about robust experimental design, researchers must maintain good research integrity, and funders, regulators and journals must check the detail in applications and submissions. The key elements that will help ensure good reproducibility must be present in the studies and the details should be shared in published papers for the rest of the scientific community to read. In the interest of transparent, robust scientific progress, studies where groups have tried and failed to replicate previous work, or those that provide ‘null’ results, should be shared, and published.

Journals play a major role in shaping changes and it is heartening to see some progress. For example, the British Journal of Pharmacology has updated its editorial policy in response to ARRIVE 2.0. It is good to see they are also asking authors who use animals in their research to justify why and how they are a valid ‘model’ of a human condition. Small changes like this will start to have a greater impact on what gets published and therefore the choices researchers make in the planning stages of a project.

Where animals are used, there are always going to be potential issues with reproducibility, however if these can be avoided through using good, robust science, there really is no excuse not to do better. Researchers and organisations must embrace ARRIVE 2.0 and its guidelines, to help us tackle the issue of poor in vivo research reporting.

To find out more about the ARRIVE guidelines, visit the website here.