Package {autotune}

autotune: Lasso with data-driven tuning for linear models

Description

This package fits lasso path for high-dimensional regression using coordinate descent while automatically tuning it's regularization parameter. autotune Lasso is 10-50 times than the standard glmnet implementation of lasso, and over 100 times faster than scaled lasso. Additionally, it gives reliable noise level estimate for the regression problem.

Details

Very simple to use. Accepts x,y data for linear regression model, and produces the regularization path which automatically estimates an optimal tuning parameter lambda.

Functions

Only 1 function, more autotune algorithms for VAR, random forests, coming soon!

• autotune_lasso

Author(s)

Tathagata Sadhukhan Ines Wilms Stephan Smeekes and Sumanta Basu
Maintainer: Tathagata Sadhukhan(ts767@cornell.edu)

Examples

set.seed(10)
n <- 80
p <- 500
s <- 5
type <- 1
snr <- 3
betatrue <- c(rep(1,s), rep(0, p - s))
x <- scale(matrix(rnorm(n * p), ncol = p))
error.sd <- sqrt((betatrue %*% betatrue)/snr)
err <- rnorm(n, sd = error.sd)
y <- x %*% betatrue + err
y <- y - mean(y)
ans <- autotune_lasso(x, y, trace_it = TRUE)
b <- betatrue
# The Predictors which are actually significant:
which(b != 0)
# The Predictors which had nonzero estmated coefficients:
which(ans$beta != 0)
# Top 10 predictors X_i's in the ranking of X_i's given by autotune:
ans$sorted_predictors[1:10] + 1
# No of significant predictors in each CD iteration when sigma_hat is allowed to vary:
ans$count_sig_beta
# Sigma estimates in each CD iteration:
ans$sigma2_seq
# Empirical noise variance:
var(err)

Data-adaptive automatic and fast tuning of LM with Lasso regularization

Description

Fits a linear model via alternative optimization of penalized gaussian maximum likelihood which is a biconvex function of regression coefficients \beta and noise variance \sigma^2. The regularization path of autotune_lasso quickly picks out a good lambda for Lasso and then returns the corresponding linear fit along with various attributes related to the fit.

Usage

autotune_lasso(
  x,
  y,
  alpha = 0.01,
  standardize = TRUE,
  standardize_response = TRUE,
  intercept = TRUE,
  active = FALSE,
  trace_it = FALSE,
  tolerance = 1e-04,
  beta_tolerance = 0.001,
  iter_max = 30,
  beta_iter_max = 40,
  active_iter_max = 5,
  PR_norm_l2 = FALSE,
  ...
)

Arguments

Value

A list with various intermediate and final outputs produced by autotune Lasso in its regularization path.

Examples

library(autotune)
set.seed(10)
n <- 80
p <- 400
s <- 5
snr <- 4
betatrue <- c(rep(1,s), rep(0, p - s))
x <- matrix(rnorm(n * p), ncol = p)
error.sd <- sqrt((betatrue %*% betatrue)/snr)
err <- rnorm(n, sd = error.sd)
y <- x %*% betatrue + err
ans <- autotune_lasso(x, y, trace_it = TRUE)
b <- betatrue
# The Predictors which are actually significant:
which(b != 0)
# The Predictors which had nonzero estmated coefficients:
which(ans$beta != 0)
# Top 10 predictors X_i's in the ranking of X_i's given by autotune:
ans$CD.path.details$sorted_predictors[1:10]
# Cardinality of autotune's estimated support set in each CD iteration before lambda converged:
ans$CD.path.details$count_sig_beta
# Sigma estimates in each CD iteration:
ans$CD.path.details$sigma_sq_seq
# Empirical noise variance:
var(err)

Extract coefficients for autotune_lasso objects

Description

This function extracts the regression coefficients estimated by an autotune lasso model, using the stored "autotune_lasso" object

Usage

## S3 method for class 'autotune_lasso'
coef(object, intercept = TRUE, path = FALSE, sparse = TRUE, ...)

## S3 method for class 'autotune_lasso_path'
coef(object, intercept = TRUE, a0 = NULL, ...)

Arguments

Value

• coef.autotune_lasso(path = FALSE, sparse = TRUE) returns a sparse dgCMatrix of dimension (nvars + 1) \times 1 (or nvars \times 1 if intercept = FALSE).

• coef.autotune_lasso(path = FALSE, sparse = FALSE) returns a named numeric vector.

• coef.autotune_lasso(path = TRUE) calls coef.autotune_lasso_path(), which returns a sparse dgCMatrix of dimension (nvars + 1) \times (k + 1) (or nvars \times (k + 1) if intercept = FALSE), with columns named lambda_1, ..., lambda_k, final_lambda.

Examples

set.seed(10)
n = 300
p = 500
s = 10
beta = c(rep(1, s), rep(0, p - s))
x = matrix(rnorm(n * p), ncol = p)
# Maunal sigma allocation
# y = x %*% beta + rnorm(n, sd = 1)
# Dynamic sigma allocation with snr specified
snr = 2
y = x %*% beta + rnorm(n, sd = sqrt(var(x%*%beta)/snr))
fit <- autotune_lasso(x, y)
coef(fit)                    # final solution
coef(fit, path = TRUE)        # coefficient path (sparse)

Plots for autotune_lasso objects

Description

Plots for autotune_lasso objects

Usage

## S3 method for class 'autotune_lasso'
plot(x, max_preds = NULL, cumulative = TRUE, ...)

Arguments

Value

Invisibly returns a data frame with columns:

n_predictors

Number of predictors included in the nested linear model.

predictor_index

Index of the predictor added at each step.

has_nonzero_beta

Logical value indicating whether the predictor added at each step has a non-zero coefficient.

r_squared

Cumulative R-squared value.

adj_r_squared

Adjusted R-squared value.

See Also

autotune_lasso

Examples

# Fit autotune lasso
set.seed(10)
n = 300
p = 500
s = 10
beta = c(rep(1, s), rep(0, p - s))
x = matrix(rnorm(n * p), ncol = p)
# Maunal sigma allocation
# y = x %*% beta + rnorm(n, sd = 1)
# Dynamic sigma allocation with snr specified
snr = 2
y = x %*% beta + rnorm(n, sd = sqrt(var(x%*%beta)/snr))
fit <- autotune_lasso(x, y)

# Basic diagnostic plot
plot(fit)

# Plot adjusted R-squared for first 15 predictors
plot(fit, max_preds = 15, cumulative = FALSE)

Predict from a autotune_lasso fit

Description

Predict from a autotune_lasso fit

Usage

## S3 method for class 'autotune_lasso'
predict(
  object,
  newx = NULL,
  type = c("response", "coefficients"),
  path = NULL,
  ...
)

Arguments

Value

If type = "response", returns the prediction at the x provided for prediction.

If type = "coefficients", returns the fitted coefficients.

Examples

set.seed(10)
n = 300
p = 500
s = 10
beta = c(rep(1, s), rep(0, p - s))
x = matrix(rnorm(n * p), ncol = p)
# Maunal sigma allocation
# y = x %*% beta + rnorm(n, sd = 1)
# Dynamic sigma allocation with snr specified
snr = 2
y = x %*% beta + rnorm(n, sd = sqrt(var(x%*%beta)/snr))
fit <- autotune_lasso(x, y)
predict(fit, newx = x[1:5, , drop = FALSE])
predict(fit, type = "coefficients")

predict(fit, type = "coefficients", path = TRUE)

S&P 500 Stock Data

Description

The sp500 data file contains a year's worth of close-of-day data for most of the Standard and Poor's 500 stocks. The data are in reverse chronological order, with the first row corresponding to December 31, 2008.

Usage

data("sp500")

Format

A list containing the following two matrices:

sp500.percent

A 252 by 494 matrix of daily percentage changes. The first column is the DJIA index, the second is the S&P 500 index, and the remaining columns are labeled individual stocks.

sp500.2008

A 253 by 494 matrix of raw close-of-day data. The first column is the DJIA index, the second is the S&P 500 index, and the remaining columns are labeled individual stocks.

Source

Redistributed from version 1.0.1 of the GPL-2-licensed R package scalreg.

References

This database was used in the R package plus.

Examples

data("sp500")
names(sp500)
dim(sp500$sp500.percent)

attach(sp500)
head(sp500.percent[, 1:5])
detach(sp500)