Skip to content
LumenaryAcademy
Professional 70 min

Project: a complete analysis, from raw data to report

One question answered end to end in base R, with checks, tested functions, a sensitivity analysis, second opinions, a figure and a generated report, followed by where to go next.

Code tested · 4AI-draftedAwaiting review by a computer scientist
MotivationH

From a question to a report

Every lesson so far has taught one part of the work: vectors and data frames, functions, cleaning, models, plots, packages and reports. Real analysis is all of them at once, in an order that matters, with decisions at every step that someone else should be able to see and question. This lesson builds one complete analysis, the way an experienced R analyst would, and then points you to what comes after this course.

The question is a classic of applied statistics. In an experiment on early growth, 50 chicks were each given one of four experimental diets and weighed from hatching, every second day, for three weeks. Do the diets lead to different growth, and by how much? The data comes with R as ChickWeight (from Crowder and Hand's 1990 book on repeated measurements), so you can run everything here yourself.

It looks like a simple comparison of four groups, and that is exactly what makes it a good project. The data has the problems real data has: chicks that were not weighed to the end, growth that is not a straight line, spreads that differ between groups, and many measurements of each animal that are tempting to treat as independent. Handling each of these honestly is the difference between an analysis that is merely run and one that can be trusted.

ExplanationH

Plan first: the question, the outcome and the folder

Before writing code, experienced analysts write down three things. The question, in a sentence. The main outcome, the single number per experimental unit that will answer it, chosen before looking at the results: here, each chick's weight on day 21, the end of the experiment. And what else will be checked and how, so that later choices are not made to suit the answer. Clinical trials make this formal by registering an analysis plan in advance; a short plan at the top of a README does the same job for everyday work.

Notice the phrase per experimental unit. The diets were given to chicks, so the chick is the unit: 50 of them, not 578 weighings. Most of the serious mistakes in this kind of analysis come from losing sight of that, as the second-opinions step shows.

The project lives in its own folder, in version control, laid out so that anyone can see what came from outside and what the code produced:

text
chick-growth/
├── README.md            the question, the plan, how to run it
├── analysis.R           raw data to results: Rscript --vanilla analysis.R
├── report.qmd           the report, rendered with quarto render
├── R/
│   └── functions.R      small functions used by the analysis
├── tests/testthat/
│   └── test-functions.R tests of those functions
├── data/raw/
│   └── chickweight.csv  the data as received, never edited
├── output/              figures and tables, all regenerated by the code
└── renv.lock            package versions, from renv::snapshot()

Everything in output/ can be deleted and rebuilt by running the code, so it is not edited by hand and usually not committed. Only base R is needed for the analysis below. The tests use testthat and the report uses Quarto with knitr; renv records the R packages, and the README notes the Quarto version.

ComputationC

Step 1: look before you analyse

The first program does no statistics at all. It gives the data short, consistent column names, then asks the questions that decide whether the planned analysis makes sense: what is the design, did the groups start alike, and is anything missing?

r
# ChickWeight comes with R: 578 weighings of 50 chicks on four diets, from
# hatching (day 0) to day 21. Copy it into a plain data frame with short,
# consistent column names.
chicks <- data.frame(chick  = as.character(ChickWeight$Chick),
                     diet   = factor(ChickWeight$Diet),
                     day    = ChickWeight$Time,
                     weight = ChickWeight$weight)
str(chicks)

# The design: chicks per diet, and the days on which they were weighed.
first <- chicks[!duplicated(chicks$chick), ]
table(first$diet)
sort(unique(chicks$day))

# Did every diet start from the same place?
day0 <- subset(chicks, day == 0)
tapply(day0$weight, day0$diet, mean)

# Completeness: twelve weighings were planned for every chick.
n_weighed <- table(chicks$chick)
n_weighed[n_weighed < 12]
last_day <- tapply(chicks$day, chicks$chick, max)
finished <- names(last_day)[last_day == 21]
table(diet = first$diet, weighed_on_day_21 = first$chick %in% finished)

# What the rest of the analysis assumes, checked in code.
stopifnot(
  !anyNA(chicks),
  !anyDuplicated(chicks[, c("chick", "day")]),
  all(chicks$weight > 0),
  nlevels(chicks$diet) == 4
)
cat("Checks passed:", nrow(chicks), "rows,", length(n_weighed), "chicks\n")
text
'data.frame':	578 obs. of  4 variables:
 $ chick : chr  "1" "1" "1" "1" ...
 $ diet  : Factor w/ 4 levels "1","2","3","4": 1 1 1 1 1 1 1 1 1 1 ...
 $ day   : num  0 2 4 6 8 10 12 14 16 18 ...
 $ weight: num  42 51 59 64 76 93 106 125 149 171 ...

 1  2  3  4
20 10 10 10
 [1]  0  2  4  6  8 10 12 14 16 18 20 21
   1    2    3    4
41.4 40.7 40.8 41.0

15 16 18 44  8
 8  7  2 10 11
    weighed_on_day_21
diet FALSE TRUE
   1     4   16
   2     0   10
   3     0   10
   4     1    9
Checks passed: 578 rows, 50 chicks

Read the output as a checklist. The design is unbalanced: diet 1 had twenty chicks and the others ten each, so diet 1's mean will be more precise. Chicks were weighed every second day plus day 21, twelve weighings in all. The four groups started at almost the same weight, between 40.7 and 41.4 grams, which is what you would hope to see if chicks were assigned to diets fairly, so differences at the end are not inherited from the start.

The completeness check matters most. Five chicks have fewer than twelve weighings; chick 18 was weighed only twice. The two-way table shows that five chicks have no weight on day 21, and that four of the five were on diet 1. The data does not say why they stopped being weighed. In growth experiments a common reason is that an animal died or was removed because it was unwell, and such chicks are probably weaker than those that stayed. Keep that in mind: it will limit what the main outcome can say about diet 1.

The last part turns assumptions into code. stopifnot() stops the script if there is a missing value, a chick weighed twice on the same day, an impossible weight or an unexpected number of diets. Next month, if someone hands you a corrected file, these checks run again without anyone having to remember them.

Worked examplesH

Step 2: building blocks you can test

The analysis needs two calculations more than once: a chick's average daily weight gain, and a per-group summary with a confidence interval. Each becomes a small function, in the project's R/functions.R, with tests. In a real project the tests would use testthat, as the previous lesson described:

r
# tests/testthat/test-functions.R
test_that("daily_gain() is the least-squares slope", {
  expect_equal(daily_gain(c(0, 2, 4), c(40, 50, 60)), 5)
  expect_true(is.na(daily_gain(c(0, 2), c(40, 44))))
})

Here, so that the program runs with nothing but R, a three-line check() function plays the part of testthat:

r
# Average daily gain: the least-squares slope of weight against day, in grams
# per day. With fewer than three weighings there is too little to go on.
daily_gain <- function(day, weight) {
  stopifnot(is.numeric(day), is.numeric(weight), length(day) == length(weight))
  if (length(day) < 3) return(NA_real_)
  cov(day, weight) / var(day)
}

# Per-group summary: n, mean, standard deviation and a 95% confidence
# interval for the mean, one row per group, ignoring missing values.
summarise_by <- function(x, group, digits = 1) {
  one_group <- function(v) {
    v <- v[!is.na(v)]
    n <- length(v)
    half_width <- qt(0.975, df = n - 1) * sd(v) / sqrt(n)
    c(n = n, mean = mean(v), sd = sd(v),
      lower = mean(v) - half_width, upper = mean(v) + half_width)
  }
  rows <- do.call(rbind, lapply(split(x, group), one_group))
  data.frame(group = rownames(rows), round(rows, digits), row.names = NULL)
}

# Tests: each is a name and a condition that must be TRUE.
check <- function(name, condition) {
  ok <- isTRUE(condition)
  cat(if (ok) "ok:  " else "FAIL:", name, "\n")
  ok
}
s <- summarise_by(c(4, 6, 8, NA, 10, 30), c("a", "a", "a", "a", "b", "b"))
results <- c(
  check("a straight line's gain is its slope",
        all.equal(daily_gain(c(0, 2, 4), c(40, 50, 60)), 5)),
  check("the gain agrees with lm()",
        all.equal(daily_gain(cars$speed, cars$dist),
                  unname(coef(lm(dist ~ speed, data = cars))["speed"]))),
  check("two weighings are too few", is.na(daily_gain(c(0, 2), c(40, 44)))),
  check("groups are counted without their NAs", identical(s$n, c(3, 2))),
  check("means are per group", all.equal(s$mean, c(6, 20))),
  check("each interval surrounds its mean", all(s$lower < s$mean & s$mean < s$upper))
)
cat(sum(results), "of", length(results), "tests passed\n")
s
text
ok:   a straight line's gain is its slope
ok:   the gain agrees with lm()
ok:   two weighings are too few
ok:   groups are counted without their NAs
ok:   means are per group
ok:   each interval surrounds its mean
6 of 6 tests passed
  group n mean   sd  lower upper
1     a 3    6  2.0    1.0  11.0
2     b 2   20 14.1 -107.1 147.1

daily_gain() computes the slope of the least-squares line directly as the covariance of day and weight divided by the variance of day, which is the same number lm() would give, as the second test confirms on R's cars data, but much cheaper to compute for fifty chicks. It refuses to estimate a slope from fewer than three weighings, returning NA rather than a number that would look as trustworthy as the rest.

summarise_by() returns one row per group with the count, mean, standard deviation and a 95% confidence interval for the mean, computed from Student's t distribution: the mean plus or minus qt(0.975, n - 1) standard errors. It drops missing values but counts only what it used, so n is always honest. The small example shows why the count must be reported: group b has two values, and its interval runs from −107 to 147, which tells you that two observations say almost nothing about a mean.

Notice what the tests cover: a case where the answer is known exactly, agreement with a trusted implementation, the edge case of too little data, and the handling of missing values. Those four kinds of test catch most mistakes in analysis code.

ComputationC

Step 3: the analysis script

Here is analysis.R, the whole analysis from raw file to report. It runs top to bottom with Rscript --vanilla analysis.R, so it depends on nothing left over from an interactive session. In the project the two functions would be loaded with source("R/functions.R"); they are repeated here so that the script runs on its own.

r
# analysis.R: do chicks on four diets grow at different rates?
# Data: R's ChickWeight, from Crowder and Hand (1990). Run: Rscript --vanilla analysis.R

## 1. Read and check -----------------------------------------------------------
# In the project this reads data/raw/chickweight.csv. So that it runs anywhere,
# it first writes R's built-in copy of the data to a temporary file of that form.
raw_file <- tempfile(fileext = ".csv")
write.csv(data.frame(chick = ChickWeight$Chick, diet = ChickWeight$Diet,
                     day = ChickWeight$Time, weight = ChickWeight$weight),
          raw_file, row.names = FALSE)
chicks <- read.csv(raw_file, colClasses = c("character", "factor", "numeric", "numeric"))
stopifnot(identical(names(chicks), c("chick", "diet", "day", "weight")),
          !anyNA(chicks), !anyDuplicated(chicks[, c("chick", "day")]))
diet_of <- tapply(as.character(chicks$diet), chicks$chick, function(d) d[1])

## 2. Functions (in the project: R/functions.R, with tests in tests/) -----------
daily_gain <- function(day, weight) {
  if (length(day) < 3) return(NA_real_)
  cov(day, weight) / var(day)
}
summarise_by <- function(x, group, digits = 1) {
  one_group <- function(v) {
    v <- v[!is.na(v)]
    n <- length(v)
    half_width <- qt(0.975, df = n - 1) * sd(v) / sqrt(n)
    c(n = n, mean = mean(v), sd = sd(v),
      lower = mean(v) - half_width, upper = mean(v) + half_width)
  }
  rows <- do.call(rbind, lapply(split(x, group), one_group))
  data.frame(group = rownames(rows), round(rows, digits), row.names = NULL)
}

## 3. Main outcome: weight on day 21 --------------------------------------------
final <- subset(chicks, day == 21)
by_diet <- summarise_by(final$weight, final$diet, digits = 0)
by_diet
model <- aov(weight ~ diet, data = final)
p_anova <- summary(model)[[1]][["Pr(>F)"]][1]
pairs <- TukeyHSD(model)$diet
clear <- rownames(pairs)[pairs[, "p adj"] < 0.05]
round(pairs[clear, , drop = FALSE], 3)

## 4. Sensitivity: daily gain over days 0 to 12, which 49 of 50 chicks reached --
early <- subset(chicks, day <= 12)
gains <- vapply(split(early, early$chick),
                function(d) daily_gain(d$day, d$weight), numeric(1))
gain_by_diet <- summarise_by(gains, diet_of[names(gains)])
gain_by_diet

## 5. Figure -------------------------------------------------------------------
figure <- file.path(tempdir(), "chick-growth.pdf")
pdf(figure, width = 9, height = 4.5)
par(mfrow = c(1, 2))
plot(weight ~ day, data = chicks, type = "n",
     xlab = "Day", ylab = "Weight (g)", main = "Growth of each chick")
for (one in split(chicks, chicks$chick)) {
  lines(one$day, one$weight, col = as.integer(one$diet[1]))
}
legend("topleft", legend = paste("Diet", 1:4), col = 1:4, lty = 1, bty = "n")
boxplot(weight ~ diet, data = final, xlab = "Diet", ylab = "Weight on day 21 (g)",
        main = "Weight on day 21")
invisible(dev.off())

## 6. Report -------------------------------------------------------------------
not_weighed <- setdiff(names(diet_of), final$chick)
top <- by_diet[which.max(by_diet$mean), ]
fast <- gain_by_diet[which.max(gain_by_diet$mean), ]
report <- c(
  "# Chick growth on four diets",
  "",
  sprintf("- %d of %d chicks were weighed on day 21. The %d others were on diets %s.",
          nrow(final), length(diet_of), length(not_weighed),
          paste(sort(diet_of[not_weighed]), collapse = ", ")),
  sprintf("- Mean weight on day 21 was highest on diet %s: %.0f g (95%% CI %.0f to %.0f), against %.0f g on diet 1.",
          top$group, top$mean, top$lower, top$upper, by_diet$mean[by_diet$group == "1"]),
  sprintf("- The diets differ overall (one-way ANOVA, p = %.3f). Pairs that differ clearly after Tukey's adjustment: %s.",
          p_anova, paste0(clear, " (p = ", sprintf("%.3f", pairs[clear, "p adj"]), ")", collapse = ", ")),
  sprintf("- Over days 0 to 12, chicks on diet %s gained weight fastest: %.1f g a day (95%% CI %.1f to %.1f).",
          fast$group, fast$mean, fast$lower, fast$upper),
  sprintf("- Figure: %s (%s).", basename(figure), if (file.exists(figure)) "written" else "missing")
)
writeLines(report, file.path(tempdir(), "report.md"))
writeLines(report)
text
  group  n mean sd lower upper
1     1 16  178 59   146   209
2     2 10  215 78   159   271
3     3 10  270 72   219   322
4     4  9  239 43   205   272
     diff    lwr     upr p adj
3-1 92.55 23.489 161.611 0.005
  group  n mean  sd lower upper
1     1 19  5.6 2.7   4.3   6.9
2     2 10  7.5 3.2   5.3   9.8
3     3 10  8.6 2.3   6.9  10.2
4     4 10  9.3 1.3   8.4  10.2
# Chick growth on four diets

- 45 of 50 chicks were weighed on day 21. The 5 others were on diets 1, 1, 1, 1, 4.
- Mean weight on day 21 was highest on diet 3: 270 g (95% CI 219 to 322), against 178 g on diet 1.
- The diets differ overall (one-way ANOVA, p = 0.007). Pairs that differ clearly after Tukey's adjustment: 3-1 (p = 0.005).
- Over days 0 to 12, chicks on diet 4 gained weight fastest: 9.3 g a day (95% CI 8.4 to 10.2).
- Figure: chick-growth.pdf (written).

Section by section, here is what it does and why.

Read and check. read.csv() is given the column types with colClasses instead of guessing them, so chick identifiers stay text and diet is a factor from the start. The checks from step 1 are repeated in short form, because this is the script that will be re-run on new data.

Main outcome. The table answers the question as planned. Chicks on diet 3 averaged 270 grams on day 21, against 178 on diet 1, with diets 2 and 4 in between. The confidence intervals are wide and overlap, as they must be with nine to sixteen chicks per group. A one-way analysis of variance says the four means differ by more than chance would easily produce (p = 0.007), and Tukey's honest significant difference method, which compares all six pairs while keeping the overall chance of a false alarm at 5%, finds that only diets 3 and 1 differ clearly: by about 93 grams, with a 95% interval from 23 to 162. The other pairs may well differ too; this experiment is too small to tell.

Sensitivity analysis. Day-21 weights exist only for chicks that reached day 21, and diet 1 lost four chicks. So the script also measures something almost every chick has: average daily gain over days 0 to 12, which 49 of the 50 reached (chick 18, with only two weighings, is left out by daily_gain() itself). The picture is broadly the same, with diet 1 slowest, but the order of the leaders changes: over the first twelve days, diet 4 gained fastest, 9.3 grams a day with remarkably little spread, while diet 3 overtook it later. A conclusion that holds under both measures (diet 1 is the slowest) is much stronger than one that depends on the measure (whether diet 3 or 4 is best).

Figure. Base graphics write a two-panel PDF: one line per chick, coloured by diet, and box plots of the day-21 weights. The line plot shows what the summary tables cannot: growth curves upward, accelerating with age, and the chicks within each diet fan out widely. That curvature is also why the daily gain over days 0 to 12 is smaller than a gain measured over all 21 days would be, and why gains must be compared over the same period.

Report. Every number in the report comes from a variable, formatted with sprintf(), never typed by hand. If the data changes, the report changes with it. Even the list of diets of the chicks without a final weight is computed, so the caveat about diet 1 can never silently go out of date.

Worked examplesH

Step 4: second opinions, and a tempting mistake

The analysis of variance assumes that the four groups have similar spreads and roughly normal distributions. The table shows standard deviations from 43 to 78 grams, so it is worth asking whether the conclusion depends on those assumptions. Experienced analysts answer such doubts by trying methods that do not make them, and seeing whether the answer changes.

r
# Second opinions on the day-21 comparison, and one tempting mistake.
chicks <- data.frame(chick = as.character(ChickWeight$Chick), diet = ChickWeight$Diet,
                     day = ChickWeight$Time, weight = ChickWeight$weight)
final <- subset(chicks, day == 21)

# 1. Welch's test compares the means without assuming equal spreads.
round(oneway.test(weight ~ diet, data = final)$p.value, 4)

# 2. The Kruskal-Wallis test compares ranks, so it does not assume normal data.
round(kruskal.test(weight ~ diet, data = final)$p.value, 4)

# 3. A bootstrap interval for "diet 3 minus diet 1": resample the chicks of each
#    diet with replacement, many times, and see how much the difference moves.
w1 <- final$weight[final$diet == "1"]
w3 <- final$weight[final$diet == "3"]
set.seed(2026)
boot <- replicate(10000, mean(sample(w3, replace = TRUE)) - mean(sample(w1, replace = TRUE)))
round(mean(w3) - mean(w1), 1)
round(quantile(boot, c(0.025, 0.975)), 1)
round(t.test(w3, w1)$conf.int[1:2], 1)

# The mistake: treating repeated weighings of one chick as independent chicks.
late <- subset(chicks, day >= 16)
nrow(late)
signif(summary(aov(weight ~ diet, data = late))[[1]][["Pr(>F)"]][1], 2)

# The fix: one number per chick (its mean late weight), then compare.
per_chick <- aggregate(weight ~ chick + diet, data = late, FUN = mean)
nrow(per_chick)
signif(summary(aov(weight ~ diet, data = per_chick))[[1]][["Pr(>F)"]][1], 2)
text
[1] 0.0122
[1] 0.0142
[1] 92.6
 2.5% 97.5%
 41.2 141.6
[1]  35.5 149.6
[1] 185
[1] 1.1e-09
[1] 47
[1] 0.0064

All three second opinions agree with the first. Welch's test (oneway.test()), which allows each diet its own spread, gives p = 0.012. The Kruskal–Wallis test, which uses only the ranks of the weights and so assumes nothing about their distribution, gives p = 0.014. And a bootstrap, which resamples the chicks of each diet ten thousand times and recomputes the difference each time, puts the difference between diets 3 and 1 between about 41 and 142 grams, close to Welch's t interval of 36 to 150. set.seed() makes the bootstrap give the same numbers on every run, which a report needs.

The last lines show the most common serious mistake in data like this. Using every weighing from day 16 onwards gives 185 rows and a p-value of about one in a billion. But those 185 rows come from only 47 chicks, weighed two to four times each, and a chick that is heavy on day 16 is still heavy on day 18. The analysis of variance treats them as 185 independent animals, so it is far too confident. This error is called pseudo-replication. The fix is to reduce the data to one number per chick, here its mean late weight, before comparing: 47 rows, and p = 0.006, in line with everything else. The rule is simple and general: analyse at the level of the unit that received the treatment.

ExplanationH

From script to shared report

A script and a text report are enough for yourself. For colleagues, the same code goes into a Quarto document, which mixes explanation with the code's results and renders to HTML, PDF or Word:

markdown
---
title: "Chick growth on four diets"
format: html
---

```{r}
#| include: false
source("R/functions.R")
chicks <- read.csv("data/raw/chickweight.csv",
                   colClasses = c("character", "factor", "numeric", "numeric"))
final <- subset(chicks, day == 21)
by_diet <- summarise_by(final$weight, final$diet, digits = 0)
```

Of `r length(unique(chicks$chick))` chicks, `r nrow(final)` were weighed on day 21.

```{r}
#| echo: false
knitr::kable(by_diet, caption = "Weight on day 21 (grams), by diet")
```

quarto render report.qmd runs every chunk in a fresh session, so the document cannot drift away from the data. The include: false chunk loads the functions and data without showing anything, the inline r expressions put computed numbers into sentences, and knitr::kable() formats a data frame as a table. A final chunk calling sessionInfo() records the R and package versions that produced the report.

Teams that use the tidyverse would write the summary step with dplyr, as you saw in earlier lessons. It is shown here for reading, since dplyr is not installed where this course's code is checked:

r
library(dplyr)

chicks |>
  filter(day == 21) |>
  group_by(diet) |>
  summarise(n = n(), mean_g = mean(weight), sd_g = sd(weight),
            lower = mean_g - qt(0.975, n - 1) * sd_g / sqrt(n),
            upper = mean_g + qt(0.975, n - 1) * sd_g / sqrt(n))

A statistician reviewing this project would suggest one more step. Rather than reducing each chick to one number, a mixed-effects model uses every weighing while respecting that weighings of the same chick are related: each chick gets its own intercept and growth rate, drawn from a distribution for its diet. The nlme package, which comes with R, and the popular lme4 package fit such models, and growth this curved would also call for a curved term in day or a transformed weight:

r
nlme::lme(weight ~ day * diet, random = ~ day | chick, data = chicks)
lme4::lmer(weight ~ day * diet + (day | chick), data = chicks)

Learning when and how to use such models is the natural next step after this course's statistics. As projects grow, analysts also adopt pipeline tools such as the targets package, which reruns only the steps whose inputs changed.

Common misconceptionsH

Mistakes that change conclusions

Misconception

"Missing values only cost precision." That is true only when the reason for missingness has nothing to do with the outcome. Here the chicks that stopped being weighed were probably the weakest, and four of five were on diet 1, so diet 1's day-21 mean describes its survivors and may flatter it. Count missing data by group, think about why it is missing, and check the conclusion with a measure that more units have, as the sensitivity analysis did.

Misconception

"More rows means more evidence." Only independent rows do. Repeated measurements of the same chick, patient, school or country are not new units, and treating them as if they were produces p-values that are far too small. Find the unit that received the treatment, and analyse at that level or with a model, such as a mixed-effects model, that knows the structure.

Misconception

"I can choose the outcome after seeing which one gives the clearest result." Every choice made after looking (which outcome, which chicks to exclude, which test) is a chance to find a pattern that is not there. Choose the main outcome in advance, report the others as sensitivity analyses, and say which is which.

Misconception

"Diets 2 and 4 did not differ significantly from diet 1, so they work no better." Failing to detect a difference is not showing that there is none. The intervals for those comparisons are wide, from slightly negative to well over 100 grams, so this experiment cannot tell. Report the interval, not just the verdict.

Further reading

Where to go next

You can now take data from a raw file to a report in R: vectors and data frames, functions and tests, cleaning, models, plots and reproducible documents. Here are good next steps, depending on where you want to go.

  • Know the language deeply. The official R manuals are the reference: An Introduction to R, R Data Import/Export, The R Language Definition and Writing R Extensions. Advanced R explains environments, functional and object-oriented programming (S3, S4 and R6) and performance.
  • Work the tidyverse way. R for Data Science teaches the whole cycle of import, tidying, transformation, visualisation and communication with the tidyverse, and is free online.
  • Share your code as a package. R Packages shows how to turn functions like daily_gain() into a documented, tested package, and CRAN or R-universe can publish it.
  • Learn the methods for your field. The CRAN Task Views list the packages for each area, from clinical trials and econometrics to spatial data and machine learning; Bioconductor covers genomics and molecular biology. Lumenary Math's statistics topics explain the theory behind the methods.
  • Join the community. The R Project's website lists the useR! conference, mailing lists and ways to contribute; The R Journal publishes peer-reviewed articles on R packages and methods; rOpenSci peer-reviews packages for open science; and local R user groups meet in cities all over the world.
  • Add another language. Data work often mixes tools: the SQL, Python and Julia courses on this site pair naturally with R, and Lumenary CS covers the computer science underneath.
Optional Christian reflection · A just weight

A reflection from the Christian tradition that inspires Lumenary Academy. It is not part of the mathematics and you are welcome to skip it.

The book of Proverbs says that a false balance is an abomination to the Lord, but a just weight is his delight. It was written about merchants' scales, but it fits data analysis remarkably well. An analyst holds the scales whenever they decide which outcome to report, which chicks to leave out or which test to run, and it is easy to lean on them, often without noticing. The habits in this lesson (deciding in advance, reporting what is missing, showing intervals rather than verdicts, letting the code produce every number) are ways of keeping the balance true, so that people who rely on the result, and who cannot check it themselves, are not misled.

Exercises

Exercises

1. Project: a complete analysis, from raw data to report

Warm-upHuman

In the chick experiment, each diet was given to whole chicks, and each chick was weighed up to twelve times. What is the experimental unit, and so how many independent observations does the comparison of diets have at most?

Choose one

2. Project: a complete analysis, from raw data to report

CoreComputational

With the lesson's daily_gain(), what does this print?

daily_gain(c(0, 2, 4, 6), c(40, 44, 52, 64))

3. Project: a complete analysis, from raw data to report

CoreComputational

With the lesson's summarise_by(), what does this print? Write it with single spaces.

summarise_by(c(2, 4, 6, 9, 11), c("x", "x", "x", "y", "y"), digits = 0)$mean

4. Project: a complete analysis, from raw data to report

StretchComputational

Using the lesson's chicks data frame and the completeness output of step 1 (chick 18 was last weighed on day 2, chick 16 on day 12, chick 15 on day 14, chick 44 on day 18 and chick 8 on day 20), how many rows does this have?

nrow(aggregate(weight ~ chick + diet, data = subset(chicks, day >= 20), FUN = mean))

5. Project: a complete analysis, from raw data to report

CoreComputational

What does this print? Write the text without the [1] and without the quotes.

sprintf("%.0f g (95%% CI %.0f to %.0f)", 269.6, 218.6, 321.6)

6. Project: a complete analysis, from raw data to report

CoreHuman

Four of the five chicks without a day-21 weight were on diet 1. If, as is likely, they were weaker than average, what does that do to the comparison based on day-21 weights?

Choose one

7. Project: a complete analysis, from raw data to report

CoreHuman

Which of these files from the project folder belong in version control (Git)? List their letters.

  • A: analysis.R
  • B: R/functions.R
  • C: renv.lock
  • D: tests/testthat/test-functions.R
  • E: output/chick-growth.pdf, written by analysis.R
  • F: .RData, a saved workspace
  • G: renv/library/, the installed packages

Write elements separated by commas. Order and repeats don’t matter — just like in a set!

Further reading and resources

  • The R manuals · R Core TeamAn Introduction to R, R Data Import/Export, The R Language Definition, Writing R Extensions and the rest, for the current release.
  • CRAN Task ViewsCurated guides to the packages for each field, from Bayesian inference to time series.
  • R for Data Science (2e) · Hadley Wickham, Mine Çetinkaya-Rundel and Garrett GrolemundA free book that teaches the whole analysis cycle with the tidyverse.
  • Advanced R (2e) · Hadley WickhamA free book on how R itself works, for programmers who want to go deeper.
  • QuartoThe publishing system used to turn analyses into reports, websites and books.
  • The R JournalThe R Project's open-access journal, with peer-reviewed articles about R packages and methods.
  • rOpenSciA community that develops and peer-reviews R packages for open science.