Why is experimental design education so important?

As FRAME publishes a report reviewing experimental design training provision for biosciences PhD students, FRAME Education and Outreach Manager Amy Beale reflects on her personal experiences of experimental design.

Amy explains why FRAME decided to look into the availability of this training at universities and considers how gaps in training might be addressed in the future.

I was a secondary school science teacher for over 15 years and the requirement to carry out an investigation and write it up was a fundamental part of the coursework for GCSE science subjects for many years; later evolving into “controlled assessments” and today, a set of ‘required practical activities’ that must be carried out with students during the course. These activities are there to consolidate scientific concepts; develop investigative skills such as controlling variables or interpreting data; and master practical skills, such as taking measurements or carrying out risk assessments.

Science teachers and the students in their lessons therefore understand the importance of being able to create a hypothesis, plan an experiment, collect, analyse, and present data and evaluate and review your own, and other people’s investigations. Terms such as reproducibility, experimental controls, and validity will be familiar to anyone who has studied GCSE science. Within the curriculum, there are also elements relating to bias, for example in GCSE biology when learning about ‘double-blind’ trials which are clinical trials where neither the patient or researcher know whether they are getting the drug being tested or the placebo, therefore eliminating potential bias when feeding back information or recording data.

Aside from doing my degree and researching my dissertation, my experience of scientific paper publication, research systems and academic culture were limited before working for FRAME, however I understood the importance of quality science and the basic elements required to achieve this.

I was slightly surprised, then, to learn of the scale of poor reproducibility in published research and the cost of in terms of hindering scientific progress but also the potential cost to laboratory animals when animal research is irreproducible. I wrote a blog about it which you can read here. Experiments using animals are much harder to control due to the natural variation shown between individual organisms, even when for example, strains of mice with the same genetics, age and sex are used. Understanding how to plan and carry out animal studies, and how to report them is therefore even more critical.

Even more concerningly, as I read about the issue, I realised that some research is not reproducible due to avoidable mistakes in experimental design that leave studies open to problems such as bias. In a 2016 online survey of 1,576 researchers by the journal Nature, when asked to choose from 11 approaches to improve reproducibility, nearly 90% chose ‘more robust experimental design,’ ‘better statistics,’ and ‘better mentorship.’

It is not a surprise than that our annual FRAME Training School in experimental design and statistics for researchers working with animals continues to be popular and well attended. Feedback from several years indicated that some Training School attendees had not received specific experimental design training during their degrees or PhD. We currently run the Training School in collaboration with the University of Nottingham. Three years ago, FRAME was approached to see if we could support the BBSRC (Biotechnology and Biological Sciences Research Council) funded Doctoral Training Partnership (DTP) with its induction training, in particular around experimental design. Director of the FAL Dr Andrew Bennett stepped up, taking time out from his other academic work to run the sessions. I supported him – although I was learning from him too and ended up asking more questions in those first sessions than helping!

We have been supporting this training every year since and experimental design is now a mandatory session in this induction programme.

This got us thinking about what other institutions and doctoral programmes provide and whether PhD funders guide the provision for experimental design training. The national curriculum sometimes falls under scrutiny, but it does provide a framework of content and skills that must be taught in all GCSE and A-Level subjects, including science. There is no such guidance once you get to university level, so content and quality of training across science subjects varies, and sometimes distils down to the skills, passion or diligence of the individual providing the training or supervising a student.

Provision of training, mentoring, or teaching in experimental design is surely key to improving the quality of science, and indeed scientific publications in the future. We know it is important, we know training courses for early career, and established career researchers, are popular, but we don’t fully understand what the picture is at university level or what tools/resources might support experimental design training for PhD students.

This became a project idea and when Margarita Kalamara, a PhD student from the University of Dundee joined us for a three-month Professional Internship Placement, it is one she investigated. Today, we are pleased to share her report about the current provision of experimental design training for Biosciences PhD students. The report highlights large variety in mandatory and optional training provision between universities and institutions. Whilst there is widespread acknowledgement that experimental design training is vital for PhD researchers, there is room to do more. We hope that in situations where there is currently no robust framework for experimental design training, institutions, funders and external organisations such as FRAME can collaborate to help put in place systems, tools and resources to change this.

Read ‘A review of experimental design training provision for biosciences PhD students at UK universities’ report here.