Three Approaches: Supervised, Unsupervised, and Causal

PsyML is organized around three approaches to machine learning. Each one answers a different research question and is run by a single function:

Approach	Research question	Main function
Supervised learning	How well can we predict an outcome we already observe?	`supervised_ml()`
Unsupervised learning	Are there groups or unusual cases when there is no outcome?	`unsupervised_ml()`
Causal inference	What is the effect of a treatment or exposure on an outcome?	`causal_ml()`

The three functions return objects with print(), summary(), plot() and predict() methods. The examples below use small simulated data and fast settings so that they run in a few seconds; in real studies the defaults compare more models and use more resampling.

1. Supervised learning: predicting a known outcome

The outcome y is observed for every participant. supervised_ml() splits the data, trains the models with cross-validation, evaluates them on the test set and reports the best one.

set.seed(1)
n <- 200
df <- data.frame(x1 = rnorm(n), x2 = rnorm(n), x3 = rnorm(n))
df$y <- factor(ifelse(df$x1 + 0.5 * df$x2 + rnorm(n) > 0, "yes", "no"))

sup <- supervised_ml(df, target = "y", models = "glm", cv_folds = 3,
                     tune_best = FALSE, run_shap = FALSE, n_cores = 1,
                     verbose = FALSE)
sup
#> 
#> ============================================================ 
#>   supervised_ml - Resultados del Analisis
#> ============================================================ 
#> 
#> CONFIGURACION:
#>   Tarea: classification 
#>   Variable objetivo: y 
#>   Observaciones (train/test): 160 / 40 
#>   Variables: 3 
#>   Modelos evaluados: glm 
#> 
#> MEJOR MODELO: Regresion Logistica/Lineal 
#> 
#> METRICAS EN TEST:
#>    roc_auc : 0.7626 
#>    accuracy : 0.7 
#>    sensitivity : 0.5556 
#>    specificity : 0.8182 
#>    f_meas : 0.625 
#>    mcc : 0.3898 
#>    kap : 0.3814 
#>    f2_meas : 0.5814 
#> 
#> TOP 5 VARIABLES IMPORTANTES:
#>   1. x1 (5.31)
#>   2. x2 (3.564)
#>   3. x3 (0.639)
#> 
#> GRAFICOS GENERADOS: 3 
#>   Disponibles: plots, figures_catalog, total_figures 
#>   Usar: plot(resultado) para panel combinado
#>         plot(resultado, type = 'nombre') para grafico individual
#>         save_all_plots(resultado, path = '.') para guardar todos
#> 
#>  ------------------------------------------------------------ 
#> Tiempo de ejecucion: 5.9 segundos
#> ============================================================

Each step of the pipeline is also available as a separate function, for example preprocess_data(), train_models(), evaluate_model(), optimize_threshold(), analyze_fairness() and interpret_model().

2. Unsupervised learning: finding groups without an outcome

There is no outcome here: unsupervised_ml() compares clustering algorithms, chooses the number of clusters, reduces the dimensionality of the data and flags anomalous cases.

set.seed(1)
X <- data.frame(a = c(rnorm(40), rnorm(40, 3)),
                b = c(rnorm(40), rnorm(40, 3)),
                c = rnorm(80))

uns <- unsupervised_ml(X,
                       clustering_algorithms = c("kmeans", "hierarchical"),
                       k_range = 2:4, reduction_methods = "pca",
                       anomaly_methods = "mahalanobis",
                       run_eda = FALSE, seed = 1, verbose = FALSE)
uns$evaluation$comparison
#>      algorithm silhouette calinski_harabasz davies_bouldin   dunn
#> 1       kmeans     0.4313             70.61         0.9336 0.8669
#> 2 hierarchical     0.4296             70.05         0.9380 0.8645

cluster_profile() describes each cluster in terms of the original variables.

3. Causal inference: estimating the effect of a treatment

Here x1 affects both who receives the treatment and the outcome, so a simple comparison of treated and untreated participants would be biased. causal_ml() adjusts for the covariates and estimates the treatment effect (the true effect in this simulation is 1.5).

set.seed(1)
n <- 300
dc <- data.frame(x1 = rnorm(n), x2 = rnorm(n), x3 = rnorm(n))
dc$treat <- rbinom(n, 1, plogis(dc$x1))
dc$y <- 1.5 * dc$treat + dc$x1 + 0.5 * dc$x2 + rnorm(n)

cau <- causal_ml(dc, treatment = "treat", outcome = "y",
                 methods = c("propensity", "meta_learners"),
                 ps_methods = c("ipw", "doubly_robust"),
                 meta_learners = "T", run_eda = FALSE,
                 seed = 1, verbose = FALSE)
cau$summary_table
#>                 method    ate     se ci_lower ci_upper  p_value
#> 1                  IPW 1.5017 0.7143   0.1016   2.9018 0.035531
#> 2 AIPW (Doubly Robust) 1.4469 0.1515   1.1500   1.7439 0.000000
#> 3            T-Learner 1.6107 0.0462   1.5202   1.7012 0.000000

Double machine learning, causal forests and targeted maximum likelihood estimation are available through the methods and run_tmle arguments when their optional packages are installed.

Tools shared by the three approaches

ml_sample_size() estimates the sample size needed to reach a target performance (see vignette("sample-size-estimation", package = "PsyML")).
block_split() and diversity_split() build train and test sets.
The eda_*() functions describe the data before modelling.
export_verbose_txt() and export_verbose_json() save the results.

Three Approaches: Supervised, Unsupervised, and Causal

Jose Ventura-Leon

1. Supervised learning: predicting a known outcome

2. Unsupervised learning: finding groups without an outcome

3. Causal inference: estimating the effect of a treatment

Tools shared by the three approaches