Learn R Programming

otsad

Online Time Series Anomaly Detectors

This package provides anomaly detectors in the context of online time series and their evaluation with the Numenta score.

Installation

Dependencies

CAD-OSE algorithm is implemented in Python. It uses bencode library in the hashing step. This dependency can be installed with the Python package manager pip.

$ sudo pip install bencode-python3

otsad package

You can install the released version of otsad from CRAN with:

# Get the released version from CRAN
install.packages("otsad")

# Get the latest development version from GitHub
devtools::install_github("alaineiturria/otsad")

Most useful functions

Detectors

  • PEWMA
    • Offline: CpPewma
    • Online: IpPewma
  • SD-EWMA
    • Offline: CpSdEwma
    • Online: IpSdEwma
  • TSSD-EWMA
    • Offline: CpTsSdEwma
    • Online: IpTsSdEwma
  • KNN-ICAD
    • Offline: CpKnnCad(ncm.type = "ICAD")
    • Online: IpKnnCad(ncm.type = "ICAD")
  • KNN-LDCD
    • Offline CpKnnCad(ncm.type = "LDCD")
    • Online: IpKnnCad(ncm.type = "LDCD")
  • CAD-OSE
    • Offline and Online: ContextualAnomalyDetector

NAB score

  • Get score: GetDetectorScore
  • Normalize score: NormalizeScore + GetNullAndPerfectScores

False Positve Reduction

  • Offline and Online: ReduceAnomalies

Static or interactive visualizations

  • Offline: PlotDetections

NOTE: As usual in R, the documentation pages for each function can be loaded from the command line with the commands ? or help:

?CpSdEwma
help(CpSdEwma)

Example

This is a basic example of the use of otsad package:

library(otsad)

## basic example code

# Generate data
set.seed(100)
n <- 500
x <- sample(1:100, n, replace = TRUE)
x[70:90] <- sample(110:115, 21, replace = TRUE) # distributional shift
x[25] <- 200 # abrupt transient anomaly
x[320] <- 170 # abrupt transient anomaly
df <- data.frame(timestamp = 1:n, value = x)

# Apply classic processing SD-EWMA detector
result <- CpSdEwma(data = df$value, n.train = 5, threshold = 0.01, l = 3)
res <- cbind(df, result)
PlotDetections(res, title = "SD-EWMA ANOMALY DETECTOR", return.ggplot = TRUE)

See plotly interactive graph

For more details, see otsad documentation and vignettes.

Copy Link

Version

Install

install.packages('otsad')

Monthly Downloads

30

Version

0.2.0

License

AGPL (>= 3)

Issues

Pull Requests

Stars

Forks

Maintainer

Alai<c3><b1>e Iturria

Last Published

September 6th, 2019

Functions in otsad (0.2.0)

ContextualAnomalyDetector

Contextual Anomaly Detector - Open Source (CAD)
GetNumTrainingValues

Get Number of Training Values
CpPewma

Classic Processing Probabilistic-EWMA (PEWMA).
GetLabels

Get Lables
CpKnnCad

Classic processing KNN based Conformal Anomaly Detector (KNN-CAD)
CpTsSdEwma

Classic Processing Two-Stage Shift-Detection based on EWMA
GetWindowLength

Get Window Length
CpSdEwma

Classic Processing Shift-Detection based on EWMA (SD-EWMA).
GetNullAndPerfectScores

Get Null And Perfect Scores
GetDetectorScore

Get detector score
IpTsSdEwma

Incremental Processing Two-Stage Shift-Detection based on EWMA
NormalizeScore

Normalize Score using Max and Min normalization
OcpTsSdEwma

Optimized Classic Processing Two-Stage Shift-Detection based on EWMA
OipPewma

Optimized Incremental Processing Probabilistic-EWMA (PEWMA).
IpPewma

Incremental Processing Probabilistic-EWMA (PEWMA).
IpSdEwma

Incremental Processing Shift-Detection based on EWMA (SD-EWMA).
IpKnnCad

Incremental processing KNN based Conformal Anomaly Detector (KNN-CAD).
GetWindowsLimits

Get windows limits
OcpPewma

Optimized Classic Processing Probabilistic-EWMA (PEWMA).
OcpSdEwma

Optimized Classic Processing Shift-Detection based on EWMA (SD-EWMA).
Twitter_volume_AMZN

Twitter_volume_AMZN.
Twitter_volume_AAPL

Twitter_volume_AAPL.
Twitter_volume_PFE

Twitter_volume_PFE.
Twitter_volume_CRM

Twitter_volume_CRM.
Twitter_volume_CVS

Twitter_volume_CVS.
OipSdEwma

Optimized Incremental Processing Shift-Detection based on EWMA (SD-EWMA).
PlotDetections

PLOT DETECTIONS
OipTsSdEwma

Optimized Incremental Processing Two-Stage Shift-Detection based on EWMA
art_daily_jumpsdown

art_daily_jumpsdown
ReduceAnomalies

Reduce Anomalies
art_daily_jumpsup

art_daily_jumpsup
ec2_cpu_utilization_77c1ca

ec2_cpu_utilization_77c1ca.
ec2_cpu_utilization_5f5533

ec2_cpu_utilization_5f5533.
Twitter_volume_UPS

Twitter_volume_UPS.
ec2_cpu_utilization_825cc2

ec2_cpu_utilization_825cc2.
ec2_cpu_utilization_ac20cd

ec2_cpu_utilization_ac20cd.
art_load_balancer_spikes

art_load_balancer_spikes.
cpu_utilization_asg_misconfiguration

cpu_utilization_asg_misconfiguration.
Twitter_volume_IBM

Twitter_volume_IBM.
art_daily_nojump

art_daily_nojump
art_increase_spike_density

art_increase_spike_density
ec2_disk_write_bytes_c0d644

ec2_disk_write_bytes_c0d644.
Twitter_volume_KO

Twitter_volume_KO.
exchange_3_cpm_results

exchange_3_cpm_results.
exchange_4_cpc_results

exchange_4_cpc_results.
Twitter_volume_FB

Twitter_volume_FB.
ec2_network_in_257a54

ec2_network_in_257a54.
Twitter_volume_GOOG

Twitter_volume_GOOG.
exchange_2_cpm_results

exchange_2_cpm_results.
TravelTime_387

TravelTime_387.
ec2_request_latency_system_failure

ec2_request_latency_system_failure.
ec2_network_in_5abac7

ec2_network_in_5abac7.
occupancy_t4013

occupancy_t4013.
TravelTime_451

TravelTime_451.
speed_6005

speed_6005.
rogue_agent_key_updown

rogue_agent_key_updown.
rds_cpu_utilization_cc0c53

rds_cpu_utilization_cc0c53.
speed_t4013

speed_t4013.
speed_7578

speed_7578.
ec2_cpu_utilization_53ea38

ec2_cpu_utilization_53ea38.
ec2_cpu_utilization_24ae8d

ec2_cpu_utilization_24ae8d.
ec2_cpu_utilization_fe7f93

ec2_cpu_utilization_fe7f93.
ec2_disk_write_bytes_1ef3de

ec2_disk_write_bytes_1ef3de.
ambient_temperature_system_failure

ambient_temperature_system_failure.
exchange_3_cpc_results

exchange_3_cpc_results.
nyc_taxi

nyc_taxi.
iio_us_east1_i_a2eb1cd9_NetworkIn

iio_us_east1_i_a2eb1cd9_NetworkIn.
rds_cpu_utilization_e47b3b

rds_cpu_utilization_e47b3b.
rogue_agent_key_hold

rogue_agent_key_hold.
machine_temperature_system_failure

machine_temperature_system_failure.
occupancy_6005

occupancy_6005.
elb_request_count_8c0756

elb_request_count_8c0756.
exchange_2_cpc_results

exchange_2_cpc_results.
art_daily_flatmiddle

art_daily_flatmiddle.
exchange_4_cpm_results

exchange_4_cpm_results.
grok_asg_anomaly

grok_asg_anomaly.