Revamped Matrix Profile Workflow in R: Enhanced Efficiency and Usability

A New Era for Matrix Profile in R

The tsmp package, once considered static, has undergone significant transformation. The latest release introduces a clear distinction between the computational backbone and the user-friendly data-mining interface, enhancing both performance and usability.

With version 0.5.1, tsmp now functions solely as a pure-R analytical layer. Its computationally demanding tasks have been offloaded to the new matrixprofiler package, built upon an optimized C++17 engine. This change doesn't replace tsmp; rather, it strengthens the infrastructure that supports it.

Two Packages, Specific Roles

Historically, the single tsmp package combined algorithms and analysis tools, which made it an easy gateway to explore time-series data. However, as datasets expanded and complexities increased, the computational workload often bottlenecked productivity.

The restructured architecture delineates responsibilities:

• matrixprofiler serves as the low-level engine, offering optimized numerical primitives and multithreaded operations via RcppThread.
• tsmp remains focused on user interactivity, providing high-level functions for motifs, chains, segmentation, and visualization.

This separation allows computational processes to run within the faster C++ engine, while users maintain ease of access. Importantly, tsmp no longer requires compilation from source, since it leverages the computational capabilities of matrixprofiler.

Analysts can continue working with familiar commands:

library(tsmp)
profile <- tsmp(series, window_size = 80)
motifs <- find_motif(profile, n_motifs = 3)
plot(motifs)

The enhancement lies in the underlying functionality, allowing heavy computations to harness the full power of the backend without requiring analysts to engage with its complexities directly.

Evolving from Contrast Score to Actionable Insights

A notable feature in the new engine is the native implementation of the Contrast Profile. Its primary goal is to identify subsequences that are recurrent in relevant time series while absent in others.

This technique analyzes positive and negative sequences, contrasting each positive subsequence against its closest match within the negative data. In theoretical terms:

\[ C_m = \frac{MP_m^{(+,-)} - MP_m^{(+,+)}}{\sqrt{2m}}, \]

Here, \(m\) represents the window size. A high contrast score indicates a pattern well-preserved in the positive dataset yet not found in the negative one. The leading candidate subsequence identified is termed the Plato, signifying a signature behavior of interest.

In practical R usage, the Contrast Profile can be computed using:

cp <- matrixprofiler::contrast(negative_data = negative_series, positive_data = positive_series,
window_size = 80L, distance = "pearson", n_workers = 4L)
cp$plato
cp$plato_idx

Though not yet implemented in the current package, the original paper discusses the Top-K Platos, highlighting distinct subsequences representative of varied target behaviors. This process requires users to manually orchestrate subsequent contrasts once an initial Plato is identified. Computationally intensive calculations remain within matrixprofiler, leaving the Top-K analysis to the user.

Introducing RFCP: Focusing on Frequency

The new Relative Frequency Contrast Profile (RFCP) expands on the contrastive concept. Instead of merely pinpointing nearest matches, it evaluates patterns based on ordered ranks of non-overlapping neighbors. This shift allows insights into how pattern recurrence varies between datasets.

The function matrixprofiler::rfcp() returns the contrast profile along with the top Plato. For comprehensive discovery, matrixprofiler::rfcp_top_k() iteratively identifies unique patterns by suppressing previously discovered ones and their trivial neighbors.

top_patterns <- matrixprofiler::rfcp_top_k(positive_data = positive_series, negative_data = negative_series,
window_size = 80L, k_platos = 3L, max_freq = c(4L, 3L, 2L), adaptive_max_freq = TRUE, n_workers = 4L)
top_patterns$platos

This method focuses on identifying a series of distinct patterns rather than simply the highest overlapping scores, providing richer insights especially when classes exhibit multiple representations.

Optional CUDA Acceleration

The standard installation of matrixprofiler functions as a multithreaded CPU engine suitable for various algorithms. However, for those with NVIDIA GPUs, a manual CUDA-enabled installation is available, enhancing computational performance for algorithms like MPX and RFCP.

This setup requires the CUDA Toolkit and compatible NVIDIA hardware, following the build script:

git clone https://github.com/matrix-profile-foundation/matrixprofiler.git
cd matrixprofiler
./inst/cuda/build_cuda_rcpp.sh R CMD INSTALL .

After building, users can verify CUDA support with:

matrixprofiler::has_cuda()

In case CUDA is not adapted during the initial build, installing the standard CRAN package will not enable this feature automatically.

Getting Started Made Simple

For current tsmp users, the new migration guide is essential, particularly for those utilizing doSNOW clusters or relying on specific index conventions. The updated parallel interface now employs n_workers, with missing indices defaulting to R’s NA_integer_ format.

For newcomers, installing both essential packages is straightforward:

install.packages(c("tsmp", "matrixprofiler"))

tsmp continues to offer a user-friendly interface for time-series data mining, while matrixprofiler enhances performance and provides the groundwork necessary for advanced algorithmic methods. Together, they revitalize the R ecosystem, supporting robust development without complicating daily tasks.

For more details, visit tsmp on CRAN, check matrixprofiler on CRAN, or explore the matrixprofiler documentation. To delve deeper into the research behind the Contrast Profile, refer to Mercer et al.’s work, “Matrix Profile XXIII: Contrast Profile: A Novel Time Series Primitive that Allows Real World Classification”.