Probabilistic

Probabilistic#

Probabilistic models describe systems with hidden state and noisy observations. The operator picks them when the data is sequential, when the underlying process is partly unobserved, or when the right output is a posterior distribution rather than a point estimate. The catalog runs from simple Markov chains through HMMs, Kalman / particle filters, Bayesian networks, and MCMC inference.

Markov chains#

A Markov chain is a sequence of states where the next state depends only on the current one (the Markov property).

  • State space, discrete (e.g. user-journey stage, device health label).

  • Transition matrix P, P[i, j] is the probability of moving from i to j in one step.

  • Stationary distribution, π = π P, the long-run proportion of time spent in each state. The basis for PageRank.

Operator uses: model click-stream funnels, MTBF-style state machines, generative text (n-gram language models), user-cohort flows.

import numpy as np

P = np.array([[0.7, 0.2, 0.1],
              [0.1, 0.6, 0.3],
              [0.2, 0.3, 0.5]])
# stationary distribution via eigenvector
eigvals, eigvecs = np.linalg.eig(P.T)
pi = eigvecs[:, np.argmax(np.real(eigvals))]
pi = np.real(pi / pi.sum())

Hidden Markov Models#

A Hidden Markov Model has unobserved (hidden) states evolving as a Markov chain, with observations emitted from each state. The operator only sees the observations; the hidden states are inferred.

  • Parameters: state-transition matrix, emission distribution per state, initial-state distribution.

  • Three problems (Rabiner’s tutorial layout):

    • Evaluation, probability of an observation sequence given the model. Solved by the forward algorithm.

    • Decoding, most likely state sequence given an observation sequence. Solved by the Viterbi algorithm.

    • Learning, fit parameters from observations. Solved by Baum-Welch (EM for HMMs) or by gradient methods.

Classic operator applications: speech recognition, POS tagging, gene-finding, network-traffic profiling, fraud detection (state = customer behavior class; observations = transactions).

from hmmlearn import hmm

# 3 hidden states, Gaussian observations
model = hmm.GaussianHMM(n_components=3, covariance_type="full", n_iter=100)
model.fit(X)                           # X is an (n_samples, n_features) sequence
states = model.predict(X)              # Viterbi decode

Conditional Random Fields#

CRFs are HMMs’ discriminative cousins. Same sequential output structure, but the model conditions on the entire input sequence without the conditional-independence assumptions HMMs make about the observations. The default before transformers for sequence labeling (POS, NER, chunking).

  • Linear-chain CRF, the common case.

  • Higher-order / general CRFs, used in vision and structured prediction.

Tools: sklearn-crfsuite, pytorch-crf.

Kalman filter#

A Kalman filter is the optimal recursive estimator for a linear Gaussian state-space model. The operator sees noisy observations of an underlying process and wants to estimate the hidden state in real time.

  • State equation, x(t+1) = A x(t) + B u(t) + w, w ~ N(0, Q).

  • Observation equation, z(t) = H x(t) + v, v ~ N(0, R).

  • Two-step recursion: predict (forward the state and its covariance) then update (correct using the new measurement).

Operator uses: sensor fusion, GPS / IMU position tracking, target tracking, online estimation of unobserved metrics (true demand from noisy clicks).

Variants:

  • Extended Kalman Filter (EKF), linearise around the current estimate. Handles smooth non-linear systems.

  • Unscented Kalman Filter (UKF), sigma-point sampling. Better than EKF on strongly non-linear systems.

  • Information filter, Kalman in the information form. Better numerics in some sensor-fusion problems.

Particle filter#

When the model is non-linear or non-Gaussian, the Kalman machinery breaks down. A particle filter (Sequential Monte Carlo) represents the posterior as a weighted set of samples (particles) and propagates them through the dynamics.

  • Propagate, sample each particle through the state transition.

  • Weight, weight each particle by the observation likelihood.

  • Resample, draw a new set proportional to weights.

Tools: particles (Chopin), pyparticleest, custom implementations are common.

Operator uses: robot localisation (SLAM), target tracking with non-Gaussian noise, regime-switching financial models.

Bayesian networks#

A Bayesian network is a directed acyclic graph where each node is a random variable and each edge encodes a conditional dependency. The joint distribution factorises into a product of local conditionals.

  • Structure, the graph.

  • CPD (Conditional Probability Distribution), the local probability at each node given its parents.

  • Inference, compute marginals or conditionals (variable elimination, belief propagation, MCMC).

Operator uses: diagnostic / decision systems, root-cause analysis, expert systems, fault trees. pgmpy is the standard Python library.

Topic models#

  • LDA (Latent Dirichlet Allocation), a probabilistic topic model. Each document is a Dirichlet mixture of topics; each topic is a Dirichlet-multinomial over words. Fit by collapsed Gibbs sampling or variational inference.

  • HDP (Hierarchical Dirichlet Process), non-parametric variant. Number of topics inferred from data.

For text-specific application see Text.

Markov Chain Monte Carlo (MCMC)#

When the posterior of a model has no closed form, MCMC samples from it. The operator runs an MCMC sampler whenever Bayesian inference is the right answer and the model is not analytically tractable.

Sampler

Detail

Metropolis-Hastings

The classical sampler. Propose, accept or reject. Easy to implement; slow to mix on hard posteriors.

Gibbs sampling

Sample each variable in turn from its conditional. Strong when the conditionals are easy.

Hamiltonian Monte Carlo (HMC)

Uses gradient information to propose long steps. Mixes much faster on continuous posteriors.

NUTS (No-U-Turn Sampler)

Auto-tuned HMC; what PyMC, Stan, NumPyro use by default.

Slice sampling

Adaptive without proposal tuning. Workable middle ground.

PyMC example (Bayesian linear regression):

import pymc as pm

with pm.Model() as model:
    a = pm.Normal("a", 0, 10)
    b = pm.Normal("b", 0, 10)
    sigma = pm.HalfNormal("sigma", 5)
    mu = a + b * x
    y_obs = pm.Normal("y", mu=mu, sigma=sigma, observed=y)
    idata = pm.sample(2000, tune=1000, chains=4)

Variational inference#

The alternative to MCMC. Posit a tractable family q(θ; φ) and fit φ to minimize KL divergence to the true posterior. Faster than MCMC, biased by the choice of family. The default for large-scale Bayesian deep learning.

Implementations#

Tool

Detail

hmmlearn

HMMs in scikit-learn style.

pomegranate

HMMs, Bayes nets, mixtures, naive Bayes; cython-fast.

pgmpy

Bayesian networks, dynamic Bayes nets, inference.

PyMC

Bayesian modeling with NUTS / variational inference. Python-native.

Stan / cmdstanpy

Reference Bayesian inference engine. Probabilistic-programming language compiled to C++.

NumPyro

JAX-backed; very fast on GPU.

filterpy

Reference Kalman / EKF / UKF / particle filter implementations.

Pitfalls#

  • Identifiability, two different parameter sets explain the data equally well. The posterior has multiple modes; chains get stuck.

  • Stationarity / convergence checks, R-hat < 1.01 on every parameter; effective sample size > 400; trace plots without trends. MCMC without diagnostics is just decoration.

  • Initialisation, EKF and HMM EM are non-convex; bad init finds bad local optima. Try multiple restarts.

  • Drift, learned transitions / emissions go stale. Plan for retraining cadence.

References#

Contents