-
Notifications
You must be signed in to change notification settings - Fork 61
/
README.Rmd
163 lines (111 loc) · 6.58 KB
/
README.Rmd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
---
output: github_document
---
# Anomalize is being Superceded by Timetk:
# anomalize <img src="man/figures/anomalize-logo.png" width="147" height="170" align="right" />
<!-- badges: start -->
[](https://github.com/business-science/anomalize/actions/workflows/R-CMD-check.yaml)
[](https://lifecycle.r-lib.org/articles/stages.html)
[](https://app.codecov.io/github/business-science/anomalize?branch=master)
[](https://cran.r-project.org/package=anomalize)
<!-- badges: end -->
<!-- README.md is generated from README.Rmd. Please edit that file -->
```{r setup, include = FALSE}
knitr::opts_chunk$set(
collapse = TRUE,
comment = "#>",
fig.path = "man/figures/README-",
out.width = "100%",
dpi = 200,
message = F,
warning = F
)
library(anomalize)
library(dplyr) # for pipe
```
The `anomalize` package functionality has been superceded by `timetk`. We suggest you begin to use the `timetk::anomalize()` to benefit from enhanced functionality to get improvements going forward. [Learn more about Anomaly Detection with `timetk` here.](https://business-science.github.io/timetk/articles/TK08_Automatic_Anomaly_Detection.html)
The original `anomalize` package functionality will be maintained for previous code bases that use the legacy functionality.
To prevent the new `timetk` functionality from conflicting with old `anomalize` code, use these lines:
``` r
library(anomalize)
anomalize <- anomalize::anomalize
plot_anomalies <- anomalize::plot_anomalies
```
<!-- # anomalize -->
> Tidy anomaly detection
`anomalize` enables a tidy workflow for detecting anomalies in data. The main functions are `time_decompose()`, `anomalize()`, and `time_recompose()`. When combined, it's quite simple to decompose time series, detect anomalies, and create bands separating the "normal" data from the anomalous data.
## Anomalize In 2 Minutes (YouTube)
<a href="https://www.youtube.com/watch?v=Gk_HwjhlQJs" target="_blank"><img src="http://img.youtube.com/vi/Gk_HwjhlQJs/0.jpg"
alt="Anomalize" width="100%" height="350"/></a>
Check out our entire [Software Intro Series](https://www.youtube.com/watch?v=Gk_HwjhlQJs&list=PLo32uKohmrXsYNhpdwr15W143rX6uMAze) on YouTube!
## Installation
You can install the development version with `devtools` or the most recent CRAN version with `install.packages()`:
``` r
# devtools::install_github("business-science/anomalize")
install.packages("anomalize")
```
## How It Works
`anomalize` has three main functions:
- `time_decompose()`: Separates the time series into seasonal, trend, and remainder components
- `anomalize()`: Applies anomaly detection methods to the remainder component.
- `time_recompose()`: Calculates limits that separate the "normal" data from the anomalies!
## Getting Started
Load the `anomalize` package. Usually, you will also load the tidyverse as well!
```{r, eval = F}
library(anomalize)
library(tidyverse)
# NOTE: timetk now has anomaly detection built in, which
# will get the new functionality going forward.
# Use this script to prevent overwriting legacy anomalize:
anomalize <- anomalize::anomalize
plot_anomalies <- anomalize::plot_anomalies
```
Next, let's get some data. `anomalize` ships with a data set called `tidyverse_cran_downloads` that contains the daily CRAN download counts for 15 "tidy" packages from 2017-01-01 to 2018-03-01.
Suppose we want to determine which daily download "counts" are anomalous. It's as easy as using the three main functions (`time_decompose()`, `anomalize()`, and `time_recompose()`) along with a visualization function, `plot_anomalies()`.
```{r tidyverse_anoms_1, fig.height=8}
tidyverse_cran_downloads %>%
# Data Manipulation / Anomaly Detection
time_decompose(count, method = "stl") %>%
anomalize(remainder, method = "iqr") %>%
time_recompose() %>%
# Anomaly Visualization
plot_anomalies(time_recomposed = TRUE, ncol = 3, alpha_dots = 0.25) +
ggplot2::labs(title = "Tidyverse Anomalies", subtitle = "STL + IQR Methods")
```
Check out the [`anomalize` Quick Start Guide](https://business-science.github.io/anomalize/articles/anomalize_quick_start_guide.html).
## Reducing Forecast Error by 32%
Yes! Anomalize has a new function, `clean_anomalies()`, that can be used to repair time series prior to forecasting. We have a [brand new vignette - Reduce Forecast Error (by 32%) with Cleaned Anomalies](https://business-science.github.io/anomalize/articles/forecasting_with_cleaned_anomalies.html).
```{r}
tidyverse_cran_downloads %>%
dplyr::filter(package == "lubridate") %>%
dplyr::ungroup() %>%
time_decompose(count) %>%
anomalize(remainder) %>%
# New function that cleans & repairs anomalies!
clean_anomalies() %>%
dplyr::select(date, anomaly, observed, observed_cleaned) %>%
dplyr::filter(anomaly == "Yes")
```
## But Wait, There's More!
There are a several extra capabilities:
- `plot_anomaly_decomposition()` for visualizing the inner workings of how algorithm detects anomalies in the "remainder".
```{r, fig.height=7}
tidyverse_cran_downloads %>%
dplyr::filter(package == "lubridate") %>%
dplyr::ungroup() %>%
time_decompose(count) %>%
anomalize(remainder) %>%
plot_anomaly_decomposition() +
ggplot2::labs(title = "Decomposition of Anomalized Lubridate Downloads")
```
For more information on the `anomalize` methods and the inner workings, please see ["Anomalize Methods" Vignette](https://business-science.github.io/anomalize/articles/anomalize_methods.html).
## References
Several other packages were instrumental in developing anomaly detection methods used in `anomalize`:
- Twitter's `AnomalyDetection`, which implements decomposition using median spans and the Generalized Extreme Studentized Deviation (GESD) test for anomalies.
- `forecast::tsoutliers()` function, which implements the IQR method.
# Interested in Learning Anomaly Detection?
Business Science offers two 1-hour courses on Anomaly Detection:
- [Learning Lab 18](https://university.business-science.io/p/learning-labs-pro) - Time Series Anomaly Detection with `anomalize`
- [Learning Lab 17](https://university.business-science.io/p/learning-labs-pro) - Anomaly Detection with `H2O` Machine Learning