otsad

Online Time Series Anomaly Detectors

This package provides anomaly detectors in the context of online time series and their evaluation with the Numenta score.

Installation

Dependencies

CAD-OSE algorithm is implemented in Python. It uses bencode library in the hashing step. This dependency can be installed with the Python package manager pip.

$ sudo pip install bencode-python3

otsad package

You can install the released version of otsad from CRAN with:

# Get the released version from CRAN
install.packages("otsad")

# Get the latest development version from GitHub
devtools::install_github("alaineiturria/otsad")

Most useful functions

Detectors

• PEWMA
  • Offline: CpPewma
  • Online: IpPewma
• SD-EWMA
  • Offline: CpSdEwma
  • Online: IpSdEwma
• TSSD-EWMA
  • Offline: CpTsSdEwma
  • Online: IpTsSdEwma
• KNN-ICAD
  • Offline: CpKnnCad(ncm.type = "ICAD")
  • Online: IpKnnCad(ncm.type = "ICAD")
• KNN-LDCD
  • Offline CpKnnCad(ncm.type = "LDCD")
  • Online: IpKnnCad(ncm.type = "LDCD")
• CAD-OSE
  • Offline and Online: ContextualAnomalyDetector
• EORELM-AD
  • Offline and Online: EorelmAD

NAB score

• Get score: GetDetectorScore
• Normalize score: NormalizeScore + GetNullAndPerfectScores

False Positve Reduction

• Offline and Online: ReduceAnomalies

Static or interactive visualizations

• Offline: PlotDetections

From prediction to anomaly detection framework

It is developed a framework that eases the adoption of any online time series prediction algorithm into an anomaly detection algorithm.

The framework is composed of two main components, one for online data normalization and the other for streaming anomaly scoring based on prediction error. The procedure to adapt an online prediction model into anomaly detection using this framework is shown in the following Figure. First, if the prediction model requires it, the current data point is normalized incrementally. Then, the normalized data point is used to train and predict the expected value using the chosen prediction model. After that, to compute the outlierness, the prediction error is calculated and passed to the outlier scoring function.

Online normalization

• Dynamic normalization
  • Online: DinamycNormalizer
• Window normalization
  • Online: WindowNormalizer
• Adaptive normalization
  • Online: AdaptiveNormalizer
• Adaptive normalization2
  • Online: AdaptiveNormalizer2

Outlier scoring

• Anomaly likelihood
  • Online: AnomalyLikelihoodScorer
• Dynamic threshold
  • Online: DynamicThresholdScorer
• Sigma scoring
  • Online: SigmaScorer
• Dynamic sigma scoring
  • Online: DynamicSigmaScorer

NOTE: As usual in R, the documentation pages for each function can be loaded from the command line with the commands ? or help:

?CpSdEwma
help(CpSdEwma)

Example

This is a basic example of the use of otsad package:

library(otsad)

## basic example code

# Generate data
set.seed(100)
n <- 500
x <- sample(1:100, n, replace = TRUE)
x[70:90] <- sample(110:115, 21, replace = TRUE) # distributional shift
x[25] <- 200 # abrupt transient anomaly
x[320] <- 170 # abrupt transient anomaly
df <- data.frame(timestamp = 1:n, value = x)

# Apply classic processing SD-EWMA detector
result <- CpSdEwma(data = df$value, n.train = 5, threshold = 0.01, l = 3)

res <- cbind(df, result)
PlotDetections(res, title = "SD-EWMA ANOMALY DETECTOR", return.ggplot = TRUE)

See plotly interactive graph

For more details, see otsad documentation and vignettes.