---
title: "aidR: Glooko export format"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{aidR: Glooko export format}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r, include = FALSE}
knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>"
)
```

```{r setup, message = FALSE}
library(aidR)
```

## Overview

```{r, include=FALSE}
path <- system.file("extdata", "glooko_example.zip", package = "aidR")
tmpdir <- file.path(tempdir(), "glooko_example")
unzip(path, exdir = tmpdir)
files <- list.files(tmpdir, recursive = TRUE)
```

A Glooko export is a folder (possibly zipped) containing CSV files for several data types:

```{r, comment = "", echo = FALSE}
sub("^[^/]+/", "", files) # removes glooko_example/ prefix
```

A single data type may be split across two or more files, which are then numbered `cgm_data_1.csv`, `cgm_data_2.csv`, and so on; `aidR` reads and concatenates all files matching a given type.

Every file begins with a one-line header, such as `Name:Jane Doe,Date Range:2024-01-01 - 2024-03-01`, followed by the column headers. Both the one-line header and the column headers are language-specific and vary with the language of the user's Glooko account (example in German):

```{r, comment = "", echo = FALSE}
writeLines(readLines(file.path(tmpdir, files[4]), n = 10))
```

## How aidR detects this format

Because the headers in a Glooko export are language-specific, `aidR` cannot identify a file from its header names. It relies instead on the *structure* of the first line (`something:something,something:date - date`), and then distinguishes between data types by the expected number of columns, which columns are numeric, and, where available, a language-independent keyword such as `"(g)"` for carbohydrates or `"Insulin"` for the bolus, basal, and total-insulin files.

## Files and the data types they produce

To inspect the files, we parse the export with `clean = FALSE`, which returns the full information provided by Glooko:

```{r}
data <- parse_data(id = "1", paths = path, clean = FALSE)
```

Each entry below describes one raw file and shows an example.

- **`alarms_data_*.csv`**: pump alarms.

```{r, comment=""}
head(data$alarm)
```

- **`bg_data_*.csv`**: self-monitored blood glucose (SMBG).

```{r, comment=""}
head(data$SMBG)
```

- **`carbs_data_*.csv`** / **`cgm_carbs_data_*.csv`**: carbohydrate/meal events recorded by or imported from a CGM system. These are different from carbohydrate amounts entered into a pump's bolus calculator. In practice, these files are almost always empty in exports of AID users. They are therefore ignored when building the clean `carbs` output, which instead uses carbohydrate amounts recorded in the bolus files (see below).

```{r, comment=""}
head(data$carbs_manual)
```

- **`cgm_data_*.csv`**: CGM data.
```{r, comment=""}
head(data$cgm)
```

- **`Insulin data/basal_data_*.csv`**: basal rates. For Omnipod users these are often very sparse or missing. For these users, the few entries that do appear correspond to basal rates delivered while in manual mode; basal delivered in automated mode is unknown and can only be roughly approximated from the daily aggregated insulin.
```{r, comment=""}
head(data$basal)
```

- **`Insulin data/bolus_data_*.csv`**: boluses. Also records whether a bolus was normal, square-wave, or dual-wave through the amounts listed under initial and extended delivery (although the duration is unknown).
```{r, comment=""}
head(data$bolus)
```

- **`Insulin data/insulin_data_*.csv`**: daily aggregated bolus, basal, and total insulin.
```{r, comment=""}
head(data$total_insulin)
```

- **`Manual data/` files** (`exercise_data_*`, `food_data_*`, `manual_insulin_data_*`, `medication_data_*`, `notes_data_*`): self-reported entries made by the user in the Glooko app - exercise, food intake, manual insulin deliveries, medication, and free-text notes, respectively. These are often very sparse or missing entirely. If present, they are available through `data$exercise`, `data$food`, `data$manual_insulin`, `data$medication` and `data$notes`, respectively.

## Cleaning files

When `parse_data` is run with `clean = TRUE`, the cleaned data types are built from the following raw files and attributes:

| Cleaned data type | Used files                      | Used attributes of `clean = FALSE` |
|-------------------|---------------------------------|-----------------|
| `cgm`             | `cgm_data_*.csv`                | `data$cgm`      |
| `basal`           | `Insulin data/basal_data_*.csv` | `data$basal`    |
| `bolus`           | `Insulin data/bolus_data_*.csv` | `data$bolus`    |
| `carbs`           | `Insulin data/bolus_data_*.csv` | `data$bolus`    |
| `SMBG`            | `bg_data_*.csv`                 | `data$SMBG`     |
| `total_basal`     | `Insulin data/insulin_data_*.csv` | `data$total_insulin` |
| `total_bolus`     | `Insulin data/insulin_data_*.csv` | `data$total_insulin` |

## Notes

- CGM and SMBG values are reported in mg/dL or mmol/L depending on account settings; `aidR` converts to mg/dL automatically.
- `timezone_offset` is always `NA`, since Glooko exports report only local (wall-clock) time.
- Serial numbers are reported for some data types (e.g. basal, bolus) but are not mapped to a device/pump name.
- Glooko is the only supported platform that reports the daily aggregated insulin itself, so `total_basal` and `total_bolus` both have `source = "reported"`. This matters for Omnipod users, whose basal rates cover manual mode only: their `basal` is sparse, while `total_basal` still holds the full amount delivered.
- `sensor_name` for CGM is always `NA`, since Glooko exports do not report it.
- Each row of the bolus file holds a bolus together with the carbohydrates entered for it, so `bolus` and `carbs` are linked by `link_id`. Boluses without carbohydrates have `link_id = NA`.
- Glooko exports from CamAPS FX users may contain slowly absorbed carbohydrate entries, which are encoded as a series of equal-sized entries, one every 30 minutes. Such entries are automatically detected and merged into a single entry by aidR, (see [Slowly absorbed carbohydrates in Glooko](aidR.html#slowly-absorbed-carbohydrates-in-glooko))).