Anomaly Detection

Anomaly Detection#

Anomaly detection answers “is this observation, sequence, or collection unusual?” The operator runs it as a building block in fraud detection, intrusion detection, telemetry monitoring, defect surfacing, and target enumeration on captured data. The output is a score or a binary label.

Regimes#

  • Supervised, labeled anomalies and normal examples. Use a classifier (Supervised) tuned for class imbalance.

  • Semi-supervised, labels for normal only. Fit a model of normal; flag deviations.

  • Unsupervised, no labels. The common case. Assume anomalies are rare and far from the bulk of data; rank by an anomaly score; threshold or alert on the top.

Statistical methods#

Method

Detail

z-score

Distance from the mean in standard deviations. Assumes data is roughly Gaussian. Cheap baseline.

Modified z-score (MAD)

Median absolute deviation. Robust to outliers in the training set.

Tukey’s fences (IQR)

Below Q1 - 1.5*IQR or above Q3 + 1.5*IQR is an outlier. Box-plot logic.

Grubbs’ test

Single-outlier hypothesis test under normality.

Generalized ESD

Multi-outlier extension of Grubbs.

Empirical CDF / percentile

Flag anything above the 99.9th percentile. Distribution-free.

Distance and density methods#

Method

Detail

k-NN distance

Distance to the k-th nearest neighbor as the anomaly score. Simple and effective in low dimensions.

LOF (Local Outlier Factor)

Ratio of local density vs neighbors’ local density. Catches anomalies in clusters with different densities.

DBSCAN noise label

Points DBSCAN labels as noise are anomalies under that density model.

One-class SVM

Fit a tight boundary around the bulk of data; outside is anomaly. Sensitive to scale and kernel choice.

Mahalanobis distance

Distance accounting for covariance. Detects multivariate outliers correlated structure misses.

Model-based methods#

Method

Detail

Isolation Forest

Random tree splits; anomalies isolate faster. Cheap, scales well, the operator’s default for tabular anomaly.

GMM likelihood

Fit a Gaussian mixture; low likelihood => anomaly.

Autoencoder reconstruction

Train an autoencoder on normal data; large reconstruction error => anomaly. Strong on images, sequences, high-dimensional structured data.

VAE / GAN-based

Generative models; anomalies score poorly under the learned distribution.

Self-supervised methods

Train a pretext task on unlabelled data; anomalies score poorly. State-of-the-art on images.

Time-series#

Time-series anomalies come in three types.

Type

Detection

Point

One observation deviates. Z-score on residuals after smoothing; isolation forest on lag features.

Contextual

Value is unusual given context (time of day, day of week). Residual from a seasonal model.

Collective

Subsequence is unusual. Matrix profile, autoencoder over windows, sliding-window two-sample tests.

Change-point detection#

A different question. When did the data-generating process shift?

  • CUSUM, cumulative sum of deviations from a target.

  • Bayesian Online Change-Point Detection (Adams & MacKay, 2007), posterior probability over run length.

  • PELT (ruptures library), exact penalised search over segmentation cost.

  • Two-sample tests over windows, slide two windows, test whether their distributions differ.

Operator practice#

Anomaly score over time with a fixed threshold. Excursions above the line are what the detector pages on; the operator tunes the threshold from the score distribution, not by eye.

        xychart-beta
    title "Anomaly score vs threshold (illustrative)"
    x-axis [t1, t2, t3, t4, t5, t6, t7, t8, t9, t10, t11, t12]
    y-axis "score" 0 --> 10
    line [1.2, 0.9, 1.5, 1.1, 2.0, 1.4, 8.6, 1.8, 1.3, 7.2, 1.6, 1.0]
    line [5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5]

  • Define “normal” first. Anomaly detection is only as good as the model of normal. Bake in seasonality, business hours, expected ranges before chasing residuals.

  • Threshold on tail probabilities, not scores. Anomaly scores are not comparable across deployments; fitted percentiles generalise better.

  • Combine signals. A multi-method ensemble (statistical + density + model) gives better precision than any one method.

  • Triaging matters. Anomaly detectors page often; the operator needs runbooks and the ability to acknowledge false positives.

  • Drift kills. Periodic retraining on a sliding window is mandatory.

Implementations#

Tool

Detail

PyOD

Most outlier methods in one Python library.

scikit-learn

Isolation Forest, LOF, One-class SVM, Elliptic Envelope.

alibi-detect

Drift and anomaly detection, including deep methods.

Prophet residuals, statsmodels SARIMA residuals

Time-series residual-based detection.

stumpy

Matrix Profile in Python. Strong for time-series anomalies and motif discovery.

Bro / Zeek / Suricata

Network-side anomaly detection for security telemetry.

Pitfalls#

  • Anomalies in the training set bias the model of normal. Use robust methods (Isolation Forest, LOF) or pre-filter extremes.

  • Class imbalance is structural. Don’t use accuracy; use precision-at-k or precision-recall.

  • Drift vs anomaly. A whole shift in distribution is not the same as a single anomalous point. Use change-point detection for the former.

  • Concept of “anomaly” is task-specific. Statistical outliers are not always operationally interesting.

References#

Contents