Reading and Cleaning Data with {mnirs}

Introduction

With modern wearable muscle near-infrared spectroscopy (mNIRS) devices it is growing ever easier to collect local muscle oxygenation data during dynamic activities.

The real challenge comes with deciding how to clean, filter, process, and eventually interpret those data.

The {mnirs} package aims to provide standardised, reproducible methods for reading, processing, and analysing mNIRS data. The goal is to help practitioners detect meaningful signal from noise, and improve our confidence in interpreting and applying information to the clients we work with.

In this vignette we will demonstrate how to:

mnirs is currently Lifecycle: experimental. Functionality may change! Stay updated on development and follow releases at github.com/jemarnold/mnirs.

mnirs is designed to process NIRS data, but it can be used to read, clean, and process other time series data which require many of the same processing steps. Enjoy!

📂 Read data from file

We will read an example data file with two NIRS channels from an incremental ramp cycling assessment recorded with Moxy muscle oxygen monitor.

First, install and load the mnirs package and other required libraries.

mnirs can be installed with install.packages("mnirs").

# pak::pak("jemarnold/mnirs") ## install development version
library(ggplot2) ## load for plotting
library(mnirs)

The first function called will often be read_mnirs(). This is used to read data from .csv or .xls(x) files exported from common wearable NIRS devices.

Exported data files will often have multiple rows of file header metadata before the data table with NIRS recordings begins. read_mnirs can extract and return this data table along with the file metadata for further processing and analysis.

See read_mnirs() for more details.

read_mnirs()

Example data files

A few example data files are included in the mnirs package. File paths can be accessed with example_mnirs()

nirs_channels = c(renamed1 = "original_name1", 
                  renamed2 = "original_name2")
## {mnirs} includes sample files from a few NIRS devices
example_mnirs()
#> [1] "artinis_intervals.xlsx"  "moxy_intervals.csv"     
#> [3] "moxy_ramp.xlsx"          "portamon-oxcap.xlsx"    
#> [5] "train.red_intervals.csv"

## partial matching will error if matches multiple
try(example_mnirs("moxy"))
#> Error in example_mnirs("moxy") : ✖ Multiple files match "moxy":
#> ℹ Matching files: "moxy_intervals.csv" and "moxy_ramp.xlsx"

data_raw <- read_mnirs(
    file_path = example_mnirs("moxy_ramp"), ## call an example data file
    nirs_channels = c(
        smo2_left = "SmO2 Live",            ## identify and rename channels
        smo2_right = "SmO2 Live(2)"
    ),
    time_channel = c(time = "hh:mm:ss"),    ## date-time format will be converted to numeric
    event_channel = NULL,                   ## leave blank if unused
    sample_rate = NULL,                     ## if blank, will be estimated from time_channel
    add_timestamp = FALSE,                  ## omit a date-time timestamp column
    zero_time = TRUE,                       ## recalculate time values from zero
    keep_all = FALSE,                       ## return only the specified data channels
    verbose = TRUE                          ## show warnings & messages
)
#> ! Estimated `sample_rate` = 2 Hz.
#> ℹ Define `sample_rate` explicitly to override.
#> Warning: ! Duplicate or irregular `time_channel` samples detected.
#> ℹ Investigate at `time` = 211.99 and 1184.
#> ℹ Re-sample with `mnirs::resample_mnirs()`.

## Note the above info message that sample_rate was estimated correctly at 2 Hz ☝
## ignore the warnings about irregular sampling for now, we will resample later

data_raw
#> # A tibble: 2,203 × 3
#>     time smo2_left smo2_right
#>    <dbl>     <dbl>      <dbl>
#>  1 0            54         68
#>  2 0.400        54         68
#>  3 0.960        54         68
#>  4 1.51         54         66
#>  5 2.06         54         66
#>  6 2.61         54         66
#>  7 3.16         54         66
#>  8 3.71         57         67
#>  9 4.26         57         67
#> 10 4.81         57         67
#> # ℹ 2,193 more rows

📊 Plot mnirs data

mnirs data can be easily viewed by calling plot(). This generic plot function uses {ggplot2} and will work on data frames generated or read by mnirs functions where the metadata contain class = *"mnirs"*.

plot.mnirs

## note the `time_labels` plot argument to display time values as `h:mm:ss`
plot(
    data_raw,
    time_labels = TRUE,
    n.breaks = 5,
    na.omit = FALSE
)

mnirs includes a custom *ggplot2* theme and colour palette available with theme_mnirs() and palette_mnirs(). See those documentation references for more details.

🔍 Metadata stored in mnirs data frames

Data frames generated or read by mnirs functions will return class = *"mnirs"* and contain metadata, which can be retrieved with attributes(data).

Instead of re-defining values like our channel names or sample rate, certain mnirs functions can automatically retrieve them from metadata. They can always be overwritten manually in subsequent functions, or by using a helper function create_mnirs_data().

See create_mnirs_data() for more details about metadata.

## view metadata (omitting item two, a list of row numbers)
attributes(data_raw)[-2]
#> $class
#> [1] "mnirs"      "tbl_df"     "tbl"        "data.frame"
#> 
#> $names
#> [1] "time"       "smo2_left"  "smo2_right"
#> 
#> $nirs_device
#> [1] "Moxy"
#> 
#> $nirs_channels
#> [1] "smo2_left"  "smo2_right"
#> 
#> $time_channel
#> [1] "time"
#> 
#> $sample_rate
#> [1] 2
#> 
#> $start_timestamp
#> [1] "2026-03-24 00:29:00 PDT"

🧹 Replace local outliers, invalid values, and missing values

We can see some data issues in the plot above, so let’s clean those with a single wrangling function replace_mnirs(), to prepare our data for digital filtering and smoothing.

mnirs tries to include basic functions which work on vector data, and convenience wrappers which combine functionality and can be used on multiple channels in a data frame at once.

Let’s explain the functionality of this data-wide function, and for more details about the vector-specific functions see replace_mnirs().

replace_mnirs()

data_cleaned <- replace_mnirs(
    data_raw,           ## blank channels will be retrieved from metadata
    invalid_values = 0, ## known invalid values in the data
    invalid_above = 90, ## remove data spikes above 90
    outlier_cutoff = 3, ## recommended default value
    width = 10,         ## window to detect and replace outliers/missing values
    method = "linear"   ## linear interpolation over `NA`s
)

plot(data_cleaned, time_labels = TRUE)

That cleaned up all the obvious data issues.

⏱️ Resample data

Say we have NIRS data recorded at 25 Hz, but we are only interested in exercise responses over a time span of 5-minutes, and our other outcome measure heart rate data are only recorded at 1 Hz anyway. It may be easier and faster to work with our NIRS data down-sampled from 25 to 1 Hz.

Alternatively, if we have something like high-frequency EMG data, we may want to up-sample our NIRS data to match samples for easier synchronisation and analysis (although, we should be cautious with up-sampling as this can artificially inflate statistical confidence with subsequent analysis or modelling methods).

resample_mnirs()

data_resampled <- resample_mnirs(
    data_cleaned,      ## blank channels will be retrieved from metadata
    resample_rate = 2, ## blank by default will resample to `sample_rate`
    method = "linear"  ## linear interpolation across resampled indices
)
#> ℹ Output is resampled at 2 Hz.

## note the altered "time" values from the original data frame 👇
data_resampled
#> # A tibble: 2,419 × 3
#>     time smo2_left smo2_right
#>    <dbl>     <dbl>      <dbl>
#>  1   0        54         68  
#>  2   0.5      54         68  
#>  3   1        54         67.9
#>  4   1.5      54         66.0
#>  5   2        54         66  
#>  6   2.5      54         66  
#>  7   3        54         66  
#>  8   3.5      55.9       66.6
#>  9   4        57         67  
#> 10   4.5      57         67  
#> # ℹ 2,409 more rows

Use resample_mnirs() to smooth over irregular or skipped samples

If we see a warning from read_mnirs() about duplicated or irregular samples like we saw above, we can use resample_mnirs() to restore time_channel to a regular sample rate and interpolate across skipped samples.

Note: if we perform both of these steps in a piped function call, we will still see the warning appear about irregular sampling at the end of the pipe. This warning is returned by read_mnirs(). But viewing the data frame can confirm that resample_mnirs() has resolved the issue.

📈 Digital filtering

To improve the signal-to-noise ratio in our dataset without losing information, we should apply digital filtering to smooth the data.

Choosing a digital filter

There are a few digital filtering methods available in mnirs. Which option is best for you will depend in large part on the sample rate of your data and the frequency of the response signal/phenomena you are interested in observing.

Choosing filter parameters is an important processing step to improve signal-to-noise ratio and enhance our subsequent interpretations. Over-filtering the data can introduce data artefacts which can negatively influence signal analysis and interpretations, just as much as trying to analyse overly-noisy raw data.

It is perfectly valid to choose a digital filter by iteratively testing filter parameters until the signal or response of interest appears to be visually optimised with minimal data artefacts, to your satisfaction.

We will discuss the process of choosing a digital filter more in depth in another article coming soon.

filter_mnirs()

Smoothing-spline

Butterworth digital filter

For filtering vector data and more details about Butterworth filter parameters, see filter_butter().

Moving average

For filtering vector data and more details about the moving average filter, see filter_ma()

Apply the filter

Let’s try a Butterworth low-pass filter, and we’ll specify some empirically chosen filter parameters for these data. See filter_mnirs() for further details on each of these filtering methods and their respective parameters.

data_filtered <- filter_mnirs(
    data_resampled,         ## blank channels will be retrieved from metadata
    method = "butterworth", ## Butterworth digital filter is a common choice
    order = 2,              ## filter order number
    W = 0.02,               ## filter fractional critical frequency `[0, 1]`
    type = "low",           ## specify a "low-pass" filter
    na.rm = TRUE            ## explicitly ignore NAs
)

## we will add the non-filtered data back to the plot to compare
plot(data_filtered, time_labels = TRUE) +
    geom_line(
        data = data_cleaned, 
        aes(y = smo2_left, colour = "smo2_left"), alpha = 0.4
    ) +
    geom_line(
        data = data_cleaned, 
        aes(y = smo2_right, colour = "smo2_right"), alpha = 0.4
    )

That did a nice job reducing the high-frequency noise in our data, while preserving the sharp edges, such as the rapid reoxygenation after maximal exercise.

⚖️ Shift and rescale data

NIRS values are not measured on an absolute scale (arguably not even percent (%) saturation/SmO2). Therefore, we may need to adjust or calibrate our data to normalise NIRS signal values between muscle sites, individuals, trials, etc. depending on our intended comparison.

For example, we may want to set our mean baseline value to zero for all NIRS signals at the start of a recording. Or we may want to compare signal kinetics (the rate of change or time course of a response) after rescaling signal amplitudes to a common dynamic range.

These functions allow us to either shift NIRS values up or down while preserving the dynamic range (the absolute amplitude from minimum to maximum values) of our NIRS channels, or rescale the data to a new dynamic range with larger or smaller amplitude.

We can group NIRS channels together to preserve the absolute and relative scaling among certain channels, and modify that scaling between other groups of channels.

shift_mnirs()

For this data set, we want to shift each NIRS channel so that the mean of the 2-minute baseline is equal to zero, which would then give us a change in deoxygenation from baseline during the incremental cycling protocol.

{tidyverse}-style channel name specification

This is a good time to note that here and in most mnirs functions, data channels can be specified using {tidyverse}-style naming; Data frame column names can be specified either with quotes as a character string ("smo2"), or as a direct symbol (smo2).

{tidyselect} support functions such as starts_with(), matches() can also be used.

data_shifted <- shift_mnirs(
    data_filtered,     ## un-grouped nirs channels to shift separately 
    nirs_channels = list(smo2_left, smo2_right), 
    to = 0,            ## NIRS values will be shifted to zero
    span = 120,        ## shift the *first* 120 sec of data to zero
    position = "first"
)

plot(data_shifted, time_labels = TRUE) +
    geom_hline(yintercept = 0, linetype = "dotted")

Before shifting, the minimum (end of exercise) values for smo2_left and smo2_right were similar, but the starting baseline values were different.

Whereas when we shift both baseline values to zero, we can see that the smo2_left signal has a smaller deoxygenation amplitude compared to the smo2_right signal, and a (slightly) greater hyperaemic reoxygenation peak.

We have to consider how our assumptions and processing decisions will influence our interpretations; by shifting both starting values, we are normalising for, and implicitly assuming that the baseline represents the same starting condition for the tissues in both legs.

This may or may not be an appropriate assumption for your research question; for example, this may be appropriate when we are more interested in the relative change (delta) in each leg during an intervention or exposure (often referred to as "∇SmO2"), but not if we were interested in asymmetries that could influence SmO2 at rest.

rescale_mnirs()

We may also want to rescale our data to a new dynamic range, changing the units to a new amplitude.

data_rescaled <- rescale_mnirs(
    data_filtered,    ## un-grouped nirs channels to rescale separately 
    nirs_channels = list(smo2_left, smo2_right), 
    range = c(0, 100) ## rescale to a 0-100% functional exercise range
)

plot(data_rescaled, time_labels = TRUE) +
    geom_hline(yintercept = c(0, 100), linetype = "dotted")

Here, our assumption is that during a maximal exercise task, the minimum and maximum values represent the functional capacity of each tissue volume being observed. So we rescale the functional dynamic range in each leg.

By normalising this way, we might lose meaningful differences captured by the different amplitudes between smo2_left and smo2_right, but we might be more interested in the trend or time course of each response.

Our interpretation may be that smo2_right appears to start at a slightly higher percent of its functional range, deoxygenates faster toward a minimum, and reaches a quasi-plateau near maximal exercise. While smo2_left deoxygenates slightly slower and continues to deoxygenate until maximal task tolerance.

Additionally, the left leg reoxygenates slightly faster than right during recovery. This might, for example, indicate exercise capacity differences between the limbs (although these differences are marginal and only discussed as representative for influence on interpretations).

🔀 Pipe-friendly functions

Most mnirs functions can be piped together using Base R 4.1+ (|>) or {magrittr} (%>%). The entire pre-processing stage can easily be performed in a sequential pipe.

To demonstrate this, we’ll read a different example file recorded with Train.Red FYER muscle oxygen sensor and pipe it through each processing stage straight to a plot.

This is also a good time to demonstrate how to use the global mnirs.verbose argument to silence all warning & information messages, for example when dealing with a familiar dataset. We recommend leaving verbose = TRUE by default whenever reading and exploring a new file.

options(mnirs.verbose = FALSE)

nirs_data <- read_mnirs(
    example_mnirs("train.red"),
    nirs_channels = c(
        smo2_left = "SmO2 unfiltered",
        smo2_right = "SmO2 unfiltered"
    ),
    time_channel = c(time = "Timestamp (seconds passed)"),
    zero_time = TRUE
) |>
    resample_mnirs() |> ## default settings will resample to the same `sample_rate`
    replace_mnirs(
        invalid_above = 73,
        outlier_cutoff = 3,
        span = 7
    ) |>
    filter_mnirs(
        method = "butterworth",
        order = 2,
        W = 0.02,
        na.rm = TRUE
    ) |>
    shift_mnirs(
        nirs_channels = list(smo2_left, smo2_right), ## 👈 channels grouped separately
        to = 0,
        span = 60,
        position = "first"
    ) |>
    rescale_mnirs(
        nirs_channels = list(c(smo2_left, smo2_right)), ## 👈 channels grouped together
        range = c(0, 100)
    )

plot(nirs_data, time_labels = TRUE)

We have two exercise intervals in this data set. Let’s demonstrate some of the common analysis methods currently available with mnirs.

🧮 Interval extraction

After the NIRS signal has been cleaned and filtered, it should be ready for further processing and analysis.

mnirs is under development to include functionality for processing discrete intervals and events, e.g. reoxygenation kinetics, slope calculations for post-occlusion microvascular responsiveness, and critical oxygenation breakpoints.

extract_intervals()

Often we will need to locate and extract smaller intervals from a NIRS recording, for further processing and analysis. For example, if we want to extract the last three minutes from repeated exercise trials.

We may have marked event labels or have incremental lap numbers in the specified event_channel, or we may simply have to manually specify the event markers by time_channel value or row number in the data frame. See extract_intervals() for more details.

## return each interval independently with `event_groups = "distinct"`
distinct_list <- extract_intervals(
    nirs_data,                  ## channels blank for "distinct" grouping
    start = by_time(177, 904),  ## manually identified interval start times
    end = by_time(357, 1084),   ## interval end time (start + 180 sec)
    event_groups = "distinct",  ## return a list of data frames for each (2) event
    span = c(0, 0),             ## no modification to the 3-min intervals
    zero_time = FALSE           ## return original time values
)

## use `{patchwork}` package to plot intervals side by side
library(patchwork)

plot(distinct_list[[1L]]) + plot(distinct_list[[2L]])

## ensemble average both intervals with `event_groups = "ensemble"`
ensemble_list <- extract_intervals(
    nirs_data,                  ## channels recycled to all intervals by default
    nirs_channels = c(smo2_left, smo2_right),
    start = by_time(177, 904),  ## alternatively specify start times + 180 sec
    event_groups = "ensemble",  ## ensemble-average across two intervals
    span = c(0, 180),           ## span recycled to all intervals by default
    zero_time = TRUE            ## re-calculate common time to start from `0`
)

plot(ensemble_list[[1L]])

The ensemble-averaged kinetics capture a more representative ‘average’ response across the two exercise intervals. Note the transient spikes in each interval are somewhat smoothed over.

Ensemble-averaging can help mitigate data dropouts and other quality issues from measurement error or biological variability. For example, ensemble-averaging is common for evaluating systemic VO2 kinetics, where responses can be highly variable trial to trial.

Conclusion

This vignette walks through the core functionality of mnirs to read, clean, and pre-process data in preparation for analysis.

Future development and articles will cover standardised analysis methods, including deoxygenation & reoxygenation kinetics and critical oxygenation breakpoint analysis.