Total core time: about 100 minutes.
| Section | Time | Type |
|---|---|---|
| When Do You Use This? + Learning Objectives | 5 min | Reading |
| Background: what a Findings row is | 20 min | Reading |
| Example 1: Building LB, results, units, and a duplicate | 25 min | Worked example |
| Example 2: Reference ranges, a real outlier, and baseline | 25 min | Worked example |
| The finished SDTM, saved | 10 min | Reading |
| What Can Go Wrong | 10 min | Reading |
| Exercises + Comprehension Check | 15 min | Practice / Self-test |
The sibling of Events and the last domain-building session in Part 2. It resolves three deliberate defects (a duplicate record, a missing unit, a real out-of-range value) and completes the persisted SDTM the ADaM sessions will read. Split after Example 1 for self-paced study.
Every code chunk below is written so you can copy it into your R console (or an R script) and run it yourself, in the order shown. Where you see a “Run It Yourself” box, stop and run the code above it before reading the explanation that follows. Run R from the project root, as in earlier sessions.
Controlled-terminology checks use the sdtm.terminology package, as in SDTM Concepts.
A lab transfer lands and you have to make it submission-ready: it has a record the vendor sent twice, a result whose unit field is blank, and a value so high it looks like a typo, except it isn’t. Then the analysis team asks for a finished SDTM they can build ADaM from.
All of it is the same task: representing measurements. This session asks: once you know a dataset is about results rather than occurrences, how do you build it, and how does the standard help you tell a data error from a real extreme?
Builds on SDTM Concepts (the result quartet; controlled terminology), The DM Domain (RFSTDTC and the study-day rule), and Events (its sibling). After this session the SDTM is complete.
After completing this session you will be able to:
Estimated time: ~20 minutes (reading)
A Findings row records what a measurement was (its test and its result, kept twice as collected and standardised) and the domain’s natural key is what makes a duplicate detectable and a real outlier keepable.
Laboratory Test Results (LB) is a Findings-class domain, and its structure is one record per lab test per time point per visit per subject (SDTMIG v3.4, §6.3.5.6). That sentence is also the domain’s natural key: the combination of subject, visit, and test that should identify a row uniquely. Hold onto it. It is what turns “the vendor sent a duplicate” from a vague worry into a checkable fact.
The topic is LBTESTCD (the test), and the result is not one column but a small family, all Expected (SDTMIG v3.4, §6.3.5.6; §4.5.1.1): LBORRES (the result as collected) with LBORRESU (its unit), and the standardised pair LBSTRESC (character) / LBSTRESN (numeric) with LBSTRESU. You met this in SDTM Concepts: the original is always kept beside the standardised version. Units are controlled terminology - LBORRESU and LBSTRESU draw on the UNIT codelist, and a unit the vendor uses that is not a submission value must be mapped to an equivalent or added in the Define-XML (SDTMIG v3.4, §6.3.5.6, assumption 7).
Findings also carry a reference range and an indicator of where the result falls in it (SDTMIG v3.4, §6.3.5.6): LBORNRLO/LBORNRHI (the range in original units), LBSTNRLO/LBSTNRHI (in standard units), and LBNRIND: the “Reference Range Indicator”, which “indicates where the value falls with respect to reference range defined by LBORNRLO and LBORNRHI” (SDTMIG v3.4, §6.3.5.6), taking values from the NRIND codelist (NORMAL, LOW, HIGH, ABNORMAL).
LBNRIND is a mechanical comparison of a number against a range. It is not a clinical-significance flag and not a judgement about whether the value is a data error (SDTMIG v3.4, §6.3.5.6, assumption 3; §4.5.5). Whether a HIGH value is a genuine extreme or a transcription mistake is a separate question, answered by data review, not by the flag. Example 2 turns on exactly this distinction.
One more, because it recurs: a value produced by a central lab following its own procedures is considered collected, not derived, so its LBDRVFL (derived flag) is null (SDTMIG v3.4, §6.3.5.6, assumption 6). The same fact is recorded a second time, in a different place: in the Define-XML origin metadata, where each variable’s origin is declared as collected, derived or assigned, a central-lab result’s origin is Collected (SDTMIG v3.4, §4.1.8.1). Note the division of labour - LBDRVFL is a variable in the dataset, while origin is metadata about the variable in define.xml; they agree here, but they are not the same artefact.
That is why a byte-perfect duplicate from the vendor is a transmission artefact to remove, not a real second measurement to keep. Nothing derived it, so nothing could legitimately have produced it twice.
Estimated time: ~25 minutes (worked example)
Read the raw lab file and test the natural key directly, is any (subject, visit, test) combination present more than once?
lb_raw <- read_csv("data/raw/lb_raw.csv", show_col_types = FALSE,
col_types = cols(.default = col_character()))
nrow(lb_raw)[1] 12389lb_raw |> count(subjid, visit, test) |> filter(n > 1)# A tibble: 1 × 4
subjid visit test n
<chr> <chr> <chr> <int>
1 101-010 SCREENING HBA1C 2One key value occurs twice: subject 101-010’s screening HbA1c. Look at the two rows:
lb_raw |>
filter(subjid == "101-010", visit == "SCREENING", test == "HBA1C")# A tibble: 2 × 9
subjid sex visit colldt test result unit ref_lo ref_hi
<chr> <chr> <chr> <chr> <chr> <chr> <chr> <chr> <chr>
1 101-010 M SCREENING 2024-03-12 HBA1C 8.2 % 4 5.6
2 101-010 M SCREENING 2024-03-12 HBA1C 8.2 % 4 5.6 The two rows are byte-for-byte identical, same result, same date, same everything. Because a central-lab value is collected once (SDTMIG v3.4, §6.3.5.6, assumption 6), two identical rows for one key cannot both be real measurements; the second is a transmission artefact. This is the one case where removing a row is correct, and the natural key is what licenses it. Contrast this with the out-of-range value in Example 2, which is also unusual but must be kept: the difference is that the duplicate violates the key and the outlier does not.
Remove it with distinct(), which drops only fully identical rows:
lb <- distinct(lb_raw)
c(before = nrow(lb_raw), after = nrow(lb), removed = nrow(lb_raw) - nrow(lb)) before after removed
12389 12388 1 Exactly one row goes. Had the two rows differed in any field: a different result, a different date - distinct() would have kept both, and you would have a genuine discrepancy to investigate rather than a duplicate to drop. De-duplication is safe only because the rows are identical.
Now map the results. The collected value and unit become LBORRES/LBORRESU; the standardised versions become LBSTRESC/LBSTRESN/LBSTRESU. These analytes need no unit conversion, so standardising is a copy, but first, defect D7: one glucose result arrived with a blank unit.
lb |>
filter(subjid == "101-037", visit == "WEEK 8", test == "GLUC") |>
select(subjid, test, result, unit)# A tibble: 1 × 4
subjid test result unit
<chr> <chr> <chr> <chr>
1 101-037 GLUC 9.6 <NA> # a blank is not a valid unit value
is_term("", clst_code = "C71620")[1] FALSEThe unit is missing, and a blank is not a legal UNIT value. You met this row in SDTM Concepts; here you resolve it. The resolution is not a guess: the vendor’s documented panel measures glucose in mmol/L, and every other glucose result in the file confirms it, so the unit is recovered from the documented source and the gap is logged as a query to the vendor: the same discipline as the sex conflict in The DM Domain:
panel_unit <- c(HBA1C = "%", GLUC = "mmol/L", ALT = "U/L", CREAT = "umol/L")
lb <- lb |>
mutate(unit = if_else(is.na(unit) | unit == "", unname(panel_unit[test]), unit))
# the recovered unit is valid CT
is_term("mmol/L", clst_code = "C71620")[1] TRUERun the chunks. The blank returns FALSE (not a unit), and mmol/L returns TRUE. Notice the shape of the fix: the value was recovered from a documented source and logged, not invented to fill the column. If the vendor’s panel had not pinned the unit, the honest step would be to leave it null and query, never to assume.
Estimated time: ~25 minutes (worked example)
The raw file carries the vendor’s reference range per result (ref_lo/ref_hi). Map them to LBORNRLO/LBORNRHI, and derive LBNRIND by comparing the result to the range, never hard-code it:
lb <- lb |>
mutate(
LBORRES = result,
LBSTRESN = as.numeric(result),
LBORNRLO = ref_lo,
LBORNRHI = ref_hi,
LBNRIND = case_when(
as.numeric(result) > as.numeric(ref_hi) ~ "HIGH",
as.numeric(result) < as.numeric(ref_lo) ~ "LOW",
TRUE ~ "NORMAL"
)
)
lb |> filter(test == "HBA1C") |> count(LBNRIND)# A tibble: 2 × 2
LBNRIND n
<chr> <int>
1 HIGH 3080
2 NORMAL 17Almost every HbA1c reads HIGH. That is correct and expected: the reference range (4.0–5.6%) is the non-diabetic normal, and this is a trial of people with type 2 diabetes, so nearly all of them sit above it: exactly as they would on a real lab report. LBNRIND is doing precisely its job: a mechanical comparison against the range. It is not telling you anything is wrong. Whether HbA1c should have flagged against a diabetic-appropriate range is an analysis decision (ADaM); SDTM records the range the lab supplied.
Now defect D5. One screening HbA1c is 13.9%. Against the range it is HIGH, like every other HbA1c, so the flag alone does not single it out. What singles it out is its magnitude:
screening <- lb |>
filter(test == "HBA1C", visit == "SCREENING") |>
mutate(result = as.numeric(result))
c(median = median(screening$result),
max = max(screening$result),
n_at_or_above_13.9 = sum(screening$result >= 13.9)) median max n_at_or_above_13.9
8.4 13.9 1.0 13.9% is the single highest screening HbA1c in the trial: one value of 400, against a median of about 8.4. That is the kind of extreme an edit check flags automatically: it looks like it could be a transcription error (a slipped digit, an 8.9 mistyped). The flag routes the value into data review: a process, not a hunch, which checks it against the source record. Here the source confirms it: a genuinely, severely uncontrolled patient. So the value is kept.
This is the whole lesson of D5, and it is a process, not a person’s opinion: an automated check flags an implausible-looking value → data review investigates it against source → the value is confirmed real → it is retained, unchanged. A range flag (LBNRIND = HIGH) and a plausibility flag are different signals; neither is permission to delete or “correct” a value. Silently changing 13.9 to something tidier would be fabricating data: the opposite of what the review exists to prevent.
Finally, LBLOBXFL: the flag SDTM uses for a consistent pre-treatment reference value, defined as the last non-missing value prior to RFXSTDTC (SDTMIG v3.4, §4.5.9). It needs each subject’s first-exposure date, which lives in the DM you built last session:
dm <- read_csv("data/sdtm/dm.csv", show_col_types = FALSE,
col_types = cols(.default = col_character()))
lb <- lb |>
left_join(dm |> select(subjid = SUBJID, RFXSTDTC), by = "subjid") |>
group_by(subjid, test) |>
mutate(
is_pre = as.Date(colldt) < as.Date(RFXSTDTC),
last_pre = suppressWarnings(max(as.Date(colldt)[is_pre])),
LBLOBXFL = if_else(is_pre & is.finite(last_pre) & as.Date(colldt) == last_pre,
"Y", NA_character_)
) |>
ungroup()
lb |>
filter(subjid == "101-010", test == "HBA1C") |>
select(visit, colldt, LBLOBXFL)# A tibble: 8 × 3
visit colldt LBLOBXFL
<chr> <chr> <chr>
1 SCREENING 2024-03-12 Y
2 BASELINE 2024-03-28 <NA>
3 WEEK 4 2024-04-26 <NA>
4 WEEK 8 2024-05-25 <NA>
5 WEEK 12 2024-06-21 <NA>
6 WEEK 16 2024-07-20 <NA>
7 WEEK 20 2024-08-18 <NA>
8 WEEK 26 2024-09-24 <NA> Exactly one row per subject per test is flagged Y. Here the screening draw, the last HbA1c taken before this subject’s first dose. Everything after treatment starts is null. LBLOBXFL gives every study a consistent pre-treatment reference, computed the same way everywhere. But note the standard’s own caution (SDTMIG v3.4, §4.5.9): this is an operational flag, and the authoritative baseline for analysis (which visit counts as baseline, how it is chosen) is defined later, in ADaM. You will set it in ADSL. SDTM marks a consistent reference; ADaM decides what baseline means for the analysis.
Estimated time: ~10 minutes (reading)
You have now built all four of GLPX-1’s domains across three sessions - DM, AE, and here LB and VS. Rebuilding them by hand every time a later session needs them would invite drift: the DM logic in one session subtly disagreeing with another. So the course reifies exactly these mappings in one script, R/build_sdtm.R, which reads data/raw/ and writes the finished domains to data/sdtm/:
data/raw/*.csv -- simulate_trial.R (the EDC + vendor extracts)
|
| R/build_sdtm.R (the mappings these sessions taught)
v
data/sdtm/{dm,ae,lb,vs}.csv -- conformant, committed, reproducibleTwo properties make this more than a convenience, and both are the subject matter of the course, not incidental to it:
simulate_trial.R is fixed-seed and build_sdtm.R has no randomness, so data/sdtm/ is fully determined by those two scripts. Anyone who clones the repository regenerates byte-identical datasets. Reproducibility is not a nicety in regulated work; it is a requirement, and here the pipeline demonstrates it.data/sdtm/lb.csv traces back through build_sdtm.R to a row in data/raw/lb_raw.csv: the provenance a reviewer needs to follow a result from a table back to its source. You will meet traceability again, formally, in define.xml and The Analysis Data Reviewer’s Guide.From here on, the ADaM sessions read data/sdtm/ rather than rebuilding from raw. The committed CSVs are the hand-off point between “what happened in the trial” (SDTM) and “what the trial shows” (ADaM).
built <- read_csv("data/sdtm/lb.csv", show_col_types = FALSE,
col_types = cols(.default = col_character()))
c(rows = nrow(built), subjects = n_distinct(built$USUBJID)) rows subjects
12388 400 Estimated time: ~10 minutes (reading)
Deleting the out-of-range value as an error
The 13.9% HbA1c is the most tempting mistake in the session. It looks wrong; it is real. A range flag and a plausibility flag both send it to review, and review (checking source) keeps it. Deleting or “correcting” a value because it is extreme is fabricating data.
De-duplicating rows that are not duplicates
distinct() is safe only for byte-perfect duplicates. Two rows for one key that differ in any field are a discrepancy to investigate, not a duplicate to drop, collapsing them silently discards a real, conflicting measurement.
Guessing a missing unit
The blank glucose unit was recovered from the vendor’s documented panel and logged. Typing mmol/L because it “must be” that is a different act: an assumption dressed as data. Recover from a source, or leave null and query.
Reading LBNRIND as a quality or significance flag
LBNRIND = HIGH on nearly every HbA1c does not mean the data is bad or the finding clinically significant. It is a mechanical comparison against the supplied range (SDTMIG v3.4, §6.3.5.6, assumption 3). Clinical significance and analysis-appropriate ranges are separate, later, and elsewhere.
Treating LBLOBXFL as the analysis baseline
LBLOBXFL is an operational SDTM flag; the baseline that drives change-from-baseline analysis is defined in ADaM (SDTMIG v3.4, §4.5.9). Using the SDTM flag as if it were the analysis baseline skips the decision ADaM exists to record.
“An out-of-range value is a data error” LBNRIND marks where a value sits relative to a range; it says nothing about whether the value is correct. The 13.9% is out of range and real. Range position and data quality are different questions.
“Every HIGH flag needs action” In a diabetes trial almost every HbA1c is HIGH against the non-diabetic range. A flag that fires for nearly everyone is describing the population, not signalling individual problems.
“The vendor file is clean because it is machine-generated” This file had a duplicate transmission and a blank unit. Consistent formatting is not the same as correct data: the same lesson as Why Standards.
“data/sdtm/ is just cached output I can edit” It is generated. Editing a CSV by hand breaks the traceability the whole pipeline exists to provide; the way to change it is to change build_sdtm.R and regenerate.
The Findings exercise builds the VS domain (the structural twin of LB, with one deliberate difference to catch: Exercise 3) SDTM: Findings (Labs & Vital Signs). Solutions are in the paid tier.
Estimated time: ~10 minutes
LBORRESU, and what makes that a resolution rather than a guess?LBNRIND = HIGH. Is that a data problem? What is LBNRIND actually telling you?LBLOBXFL marks one pre-treatment value per subject per test. Why is it not the baseline used for the analysis?The natural key is subject + visit + test - “one record per lab test per time point per visit per subject” (SDTMIG v3.4, §6.3.5.6). One key value (subject 101-010, screening, HbA1c) appeared twice, and the two rows were byte-for-byte identical, so one was a transmission artefact of a collected-once value (assumption 6) and distinct() removed exactly it. Removal was safe only because the rows were identical.
mmol/L, recovered from the vendor’s documented panel, which measures glucose in mmol/L, as every other glucose result confirms; the gap was logged as a query. It is a resolution rather than a guess because the value came from a documented source, not from assuming what the unit “should” be. Had no source pinned it, the honest step is null plus a query.
Not a data problem. LBNRIND is a mechanical comparison of the result against the supplied reference range (SDTMIG v3.4, §6.3.5.6, assumption 3). The range is the non-diabetic normal, so a diabetic cohort reads HIGH almost everywhere: the flag is describing the population relative to that range, not signalling errors or clinical significance.
An automated edit check flagged 13.9% as an implausible extreme (the highest of 400 screening values, against a median near 8.4); that routed it to data review; review checked it against the source record and confirmed a genuinely uncontrolled patient; so it was retained unchanged. Deleting or “correcting” it would fabricate data: the value is real, and its being extreme is not evidence that it is wrong.
LBLOBXFL is an operational flag: the last non-missing value before first exposure, computed identically across studies (SDTMIG v3.4, §4.5.9). The authoritative baseline for analysis (which visit counts, how it is chosen, how missing baselines are handled) is a decision recorded in ADaM, not SDTM. You set it in ADSL.