This release corrects six defects that affected results, and changes
two defaults. The three headline defects are listed first; the weighted,
Mondrian and Jackknife+/CV+ fixes further down also change what you get
back. Anyone using conformal_aci(),
conformal_aps(), conformal_raps(), or passing
a formula or fitted model as model should re-run their
analysis.
conformal_aci() applied the online update with the
operands reversed: alpha_t + gamma * (err_t - alpha)
instead of alpha_t + gamma * (alpha - err_t) (Gibbs and
Candes 2021, Eq. 2). A miscoverage event therefore narrowed the next
interval instead of widening it, turning the intended negative feedback
into positive feedback, and alpha_t ran away to a clip
boundary. Under a variance shift this drove empirical coverage to 0.605
against a 0.90 target.
The model argument was discarded whenever it was a
formula or a fitted model object, and a plain lm(y ~ .) was
fitted in its place. model = y ~ a silently fitted every
column of x; lm(y ~ poly(v1, 3) + v2) was
refitted as y ~ v1 + v2; ranger
hyperparameters were dropped. Formulas and fitted models are now
honoured, refitted on each conformal split. Objects that cannot be
refitted raise an error naming make_model() instead of
silently substituting a default.
conformal_aps() and conformal_raps()
could return the full label set for every observation. The set builder
included the class that crossed the threshold rather than
inverting the calibrated score, and randomize = TRUE
randomised the calibration scores but never the set construction. With
oracle probabilities on a four-class problem, APS returned a mean set
size of 3.90 out of 4 at 99.9% coverage; it now returns 2.69 at
88.6%.
conformal_jackknife(), conformal_cv(),
and predict() clamped the interval bounds to the smallest
and largest order statistics when the quantile index fell outside
1..n. Barber et al. (2021) define those bounds as infinite,
which occurs for fewer than 9 observations at alpha = 0.10.
They now return -Inf / Inf, matching
conformal_split().
conformal_weighted() substituted the mean
calibration weight for the test-point weight, so every test point
received the same quantile. The new weights_new argument
gives the exact procedure of Tibshirani et al. (2019), in which each
test point receives its own quantile. Omitting it with non-uniform
weights now warns.
A fitted glm passed to any classification method
failed with “missing columns for class levels”, because the internal
probability matrix was unnamed.
coverage_by_bin() failed with “‘breaks’ are not
unique” whenever predictions contained ties. Duplicate breaks are now
collapsed, with a warning reporting the number of bins actually
used.
Conformal functions called set.seed() on the global
random stream and left it altered. The seed argument now
applies for the duration of the call only, and the user’s
.Random.seed is restored on exit. Seeded calls remain
reproducible.
plot() failed with “need finite ‘ylim’ values” on
unbounded intervals. These are now drawn to the plot edge with a
message.
conformal_mondrian() and
conformal_mondrian_class() silently substituted the pooled
quantile for groups with fewer than three calibration points, voiding
the group-conditional guarantee for exactly the groups that needed it.
Such groups now receive an unbounded interval, with a warning naming the
number of calibration points required.
Data frames containing non-numeric columns produced the
misleading error “must not contain NaN or Inf values”. They now name the
offending columns and suggest
stats::model.matrix().
conformal_aps() and conformal_raps()
now default to randomize = TRUE, the method as published.
Deterministic scoring (randomize = FALSE) remains available
and now warns when the conformal quantile saturates at 1. Pass
seed for reproducible randomised sets.
Prediction sets are now the exact inversion of the calibrated
score. New allow_empty argument on
conformal_lac(), conformal_aps(),
conformal_raps(), conformal_class_split(), and
conformal_mondrian_class(); the default FALSE
keeps the previous behaviour of replacing an empty set with the most
probable class.
conformal_cqr()’s quantiles argument
previously had no effect. It is now validated, recorded on the returned
object, and checked for consistency with alpha.
conformal_cv() and
conformal_jackknife() now validate that x_new
has the same number of columns as x.
Jackknife+ prediction makes one call to predict_fun
per leave-one-out model rather than one per (model, test point)
pair.
conformal_cv(x_new = NULL) records
train_approximation = TRUE, and print() states
that those intervals do not carry the CV+ guarantee.
inst/CITATION now reports the installed version
rather than a hardcoded one.
paper/, Makefile, and
llms*.txt are excluded from the source tarball, which
removes an R CMD check WARNING about GNU extensions in
paper/slides/Makefile and cuts the tarball from 4.2
MB.
README and the accompanying paper: corrected the CRAN status,
dependency count, and competitor versions, and softened the Mondrian and
ACI exclusivity claims (see conformalForecast and
AdaptiveConformal).
predict() for LAC and Mondrian classification
objects: randomize field now defaults to FALSE
when not set by the fitting method.graphics and grDevices to
DESCRIPTION Importsconformal_mondrian() and
conformal_mondrian_class() for group-conditional (Mondrian)
conformal predictionconformal_weighted() for weighted conformal prediction
under covariate shiftconformal_aci() for adaptive conformal inference
(sequential prediction)conformal_pvalue() for conformal p-valuesconformal_compare() for benchmarking multiple methods
side-by-sidecoverage_by_group() and coverage_by_bin()
for conditional coverage diagnosticsverbose = TRUE for
conformal_jackknife() and conformal_cv()