Me

• Lecturer at Newcastle University
• Teach how to:
  • use R
  • do statistics (using R)

 

• Local network co-lead of UK Reproducibility Network (UKRN)
• Organise activities on reproducibility & open research

Teaching statistics

• First module for MSc Data Science
• Exploratory analysis of a simple data set in R
• Write report using R Markdown
• Use the Rmd to reproduce their report:
  • Can it be generated with another machine?
  • Can it be generated with new data?
  • Are the results still valid with new data?

 

Lesson

• Lots need to be taught
  • … alongside the “core” materials
  • even absolute vs relative paths
  • let alone packages e.g. ProjectTemplate, reprex
• It takes effort to reproduce
  • binder (https://mybinder.org) very useful

First contact

Revisiting

• Can the results still be reproduced?
• Yes! Not perfect though
  • didn’t use packrat or renv
  • dplyr::summarise_all() is deprecated
• More importantly …

Rookie mistake

• My actual code

## prelim
print(Sys.time())
remove(list = ls())
library(hybridProcess)
source("gots7_prelim.R")
source("initial_values_20170716_single.R")

## run
run_mcmc("20170716", 2468L, me_etas, N.gots7, 
         t.gots7, b.gots7, f.gots7, a0.gots7,
         b0.gots7, p0.gots7, p01.gots7,
         p10.gots7, write = FALSE, l0.gots7,
         beta.gots7, kappa.gots7, lambda.gots7,
         phi.gots7, psi.gots7, tau.gots7,
         theta.gots7, s_beta.gots7, s_kappa.gots7,
         s_lambda.gots7, s_psi.gots7, s_theta.gots7,
         s_e_init.gots7, rebuild = TRUE)

## exit
remove(list = ls())
print(Sys.time())

 

Lesson

• Nothing lasts forever
  • Keep up-to-date what’s new
  • Let package management tools take care of the past
• Future-proof your work
  • It helps others to reproduce your work …
  • … including your future self
  • README helps
• Adopt early
  • (Ask your PhD student / postdoc to) do it now
  • Doesn’t need to be perfect the first time
  • It will save you time in the future

So far

• A lot skipped
  • Version control, documentation, package development, testing
  • Container briefly mentioned: binder, docker, etc.
• Computational reproducibility is only the beginning
  • Social sciences / medical statistics care about related issues
  • Reproducibility? Replicability?
• Do new data / experiments lead to the same conclusion?
  • Not identical \(p\)-values to what you’ve got (unlikely)
  • Same hypothesis supported / not-supported
• Does the methodology make sense?
  • Verified results aren’t necessarily good or useful
• Has the method been changed to fit the data?
  • Garden of the forking paths

Open Research & UKRN

• A national consortium promoting best practice in Open Research
  • https://www.ukrn.org
• Institutions form their own local network
  • Colleagues & research students within Newcastle University
• ReproducibiliTea journal club
  • Sharing good practices from papers or their own work
• Drop-in sessions
  • Support colleagues with open research practices
• Join/create your local network if you have not already

 

Thank you!

• Website: https://clement-lee.github.io
• Slides: https://bit.ly/rss-2024