Getting started with trialdiff

Why trialdiff?

Clinical trial datasets are regenerated many times during a study. Between two data cuts, observations are added, removed or corrected, derivations change, and metadata evolves. Generic tools such as diffdf and waldo can tell you that two datasets differ. trialdiff answers the clinical-programming question that comes next:

What changed, what does the change represent, and which downstream analyses or outputs might be affected?

trialdiff is organised into five layers, each usable on its own:

  1. Compare - [compare_cut()]
  2. Classify - [classify_changes()]
  3. Trace - [define_lineage()], [trace_dependencies()]
  4. Assess - [assess_impact()]
  5. Report - [report_diff()]

All five layers are deterministic and rule-based. Nothing is inferred by an opaque model, and no statistical significance is ever claimed.

The example data

The package ships synthetic ADSL and ADLB data cuts. No proprietary data is used.

library(trialdiff)
str(adsl_cut1, max.level = 1)
#> tibble [304 × 55] (S3: tbl_df/tbl/data.frame)
#>  - attr(*, "label")= chr "Subject Level Analysis"
#>  - attr(*, "_xportr.df_arg_")= chr "ADSL"

1. Compare two data cuts

diff <- compare_cut(
  old = adsl_cut1,
  new = adsl_cut2,
  by = "USUBJID",
  dataset = "ADSL"
)
diff
#> 
#> ── trialdiff comparison ────────────────────────────────────────────────────────
#> ADSL: old → new
#> Keys: "USUBJID"
#> Observations: 304 → 306 (304 matched)
#> Added: 2 Removed: 0 Modified cells: 2
#> Schema changes: 0

The result is a tdiff object with added, removed, modified and schema tables. For example, the treatment-assignment change:

diff$modified[, c("USUBJID", "variable", "old_value", "new_value", "change")]
#> # A tibble: 2 × 5
#>   USUBJID     variable old_value new_value           change
#>   <chr>       <chr>    <chr>     <chr>               <chr> 
#> 1 01-701-1015 TRT01P   Placebo   Xanomeline Low Dose value 
#> 2 01-701-1015 TRT01A   Placebo   Xanomeline Low Dose value

2. Classify the changes

classified <- classify_changes(diff)
table(classified$register$category_label)
#> 
#>                 New subject Treatment-assignment change 
#>                           2                           2

Every classification carries a plain-language explanation:

classified$register$reason[classified$register$category == "treatment_assignment_change"]
#> [1] "Treatment variable 'TRT01P' changed from 'Placebo' to 'Xanomeline Low Dose' for subject '01-701-1015'."
#> [2] "Treatment variable 'TRT01A' changed from 'Placebo' to 'Xanomeline Low Dose' for subject '01-701-1015'."

3. Define lineage

Lineage is explicit metadata. Nodes are datasets (ADSL), variables (ADSL.TRT01P), analyses (MMRM) or outputs (Table_14_2_1).

lineage <- define_lineage(
  lineage_edge("ADSL.TRT01P", "ADLB.TRT01P", relationship = "groups_by"),
  lineage_edge("ADLB.AVAL", "MMRM", relationship = "models"),
  lineage_edge("MMRM", "Table_14_2_1", relationship = "reports")
)
trace_dependencies(lineage, from = "ADSL.TRT01P")
#> # A tibble: 1 × 5
#>   node        node_type depth path                       edge_relationship
#>   <chr>       <chr>     <int> <chr>                      <chr>            
#> 1 ADLB.TRT01P variable      1 ADSL.TRT01P -> ADLB.TRT01P groups_by

4. Assess impact

impact <- assess_impact(classified, adsl_adlb_lineage)
impact$impacts[, c("node", "node_type", "level", "requires_rerun")]
#> # A tibble: 8 × 4
#>   node                     node_type level                requires_rerun
#>   <chr>                    <chr>     <chr>                <lgl>         
#> 1 ADLB.TRT01P              variable  definitely_affected  FALSE         
#> 2 ADSL.TRT01A              variable  definitely_affected  FALSE         
#> 3 Efficacy_Set             analysis  potentially_affected TRUE          
#> 4 Safety_Set               analysis  potentially_affected TRUE          
#> 5 Lab_Summary_By_Treatment analysis  potentially_affected TRUE          
#> 6 MMRM                     analysis  potentially_affected TRUE          
#> 7 Table_14_2_1             output    potentially_affected TRUE          
#> 8 Table_14_2_2             output    potentially_affected TRUE

Impact is graded as definitely_affected, potentially_affected or unlikely. Analyses and outputs are flagged as requiring review/rerun; the package never claims statistical impact.

5. Report

report <- report_diff(diff, impact = impact, output = "list")
report$data$review_items
#> # A tibble: 7 × 3
#>   item           detail                                                    owner
#>   <chr>          <chr>                                                     <chr>
#> 1 Rerun required Efficacy_Set (potentially_affected) - Potentially affect… Stat…
#> 2 Rerun required Safety_Set (potentially_affected) - Potentially affected… Stat…
#> 3 Rerun required Lab_Summary_By_Treatment (potentially_affected) - Potent… Stat…
#> 4 Rerun required MMRM (potentially_affected) - Potentially affected: MMRM… Stat…
#> 5 Rerun required Table_14_2_1 (potentially_affected) - Potentially affect… Stat…
#> 6 Rerun required Table_14_2_2 (potentially_affected) - Potentially affect… Stat…
#> 7 Lineage gap    1 changed node(s) have no declared lineage: ADSL.         Prog…

Use output = "report.html" to write a self-contained HTML report, or as_json() for machine-readable output suitable for automated QC pipelines.

Next steps