As an avid reader of this blog (right?), you’ll have heard the terms “Markov model” and “hidden Markov model” thrown around. They might sound intimidating, but they are really just a way of describing things that change over time, where the next step depends on where you are right now.

The three locations: a plain Markov chain

Imagine I track where I am during a typical day. For simplicity, let us say I am always in one of three places: Home, Work, or Cafe. That is my state space – the complete list of states I can be in.

Now, the interesting question: if I am at Work right now, where am I likely to be next? Probably still at Work. Maybe Home. Less likely Cafe. And if I am at Home, I am more likely to go to Work than to the Cafe. The key idea of a Markov chain is that the probability of where I go next depends only on where I am now – not on where I was yesterday, not on the whole history of my day. That assumption is called the Markov property (or memorylessness): the future depends on the present, and the present alone.

We can write these probabilities down as a transition matrix. Each row is the state I am in now, each column is the state I go to next, and each cell is the probability of that move. Every row has to add up to 1, because from any state I have to go somewhere (including staying put).

          Home   Work   Cafe
Home      0.6    0.3    0.1
Work      0.2    0.7    0.1
Cafe      0.4    0.2    0.4

Reading that: if I am at Home, there is a 60% chance I stay home, 30% I go to Work, and 10% I go to the Cafe. If I am at Work, 70% I stay at Work. And so on.

Once you have a transition matrix, you can do useful things. You can simulate days forward. You can ask “if I start at Home, what is the probability I am at the Cafe after three steps?” You can even find the steady state – the long-run proportion of time I spend in each place, no matter where I started. That last one is genuinely useful: it is how you answer “on average, how much of my life is spent at work?” without tracking yourself for a year.

Markov chains were invented by the Russian mathematician Andrey Markov in 1906, originally to analyse the sequence of vowels and consonants in Pushkin’s poem Eugene Onegin. He wanted to show that the next letter depended on the previous one, rather than being purely random. The tool he built for that little literary experiment turned out to be one of the most widely used ideas in all of applied mathematics.

The umbrella: a hidden Markov model

The plain Markov chain assumes you can see the state. But in real life, the thing you care about is often hidden, and all you get is a noisy observation of it. That is where the hidden Markov model (HMM) comes in.

Here is the example I commonly use to explain the concept. There is a colleague (Gary) in the office who seems to check the weather report every single morning. We’ll pretend that I never actually see the weather from my desk – no windows, fluorescent lights, the works (where I work is actually quite nice, but that doesn’t help our story). But I do see whether Gary walks in carrying an umbrella.

The thing I actually care about – whether it is raining outside – is the hidden state. The thing I can observe – umbrella or no umbrella – is the observation. The model has two layers:

  1. Transition probabilities between hidden states: if it rained yesterday, it is more likely to rain today. (This is the Markov chain part, on the hidden layer. And remember, in my hypothetical corporate cave à la Plato, I have no way of directly seeing this.)
  2. Emission probabilities from each hidden state to each observation: if it is raining, my colleague is very likely to carry an umbrella; if it is sunny, they almost certainly will not.

So the structure is: hidden states (rain / no rain) evolve like a Markov chain, and each hidden state “emits” an observable signal (umbrella / no umbrella) with some probability. I never see the rain directly, but by watching the umbrella over a sequence of days, I can infer the most likely weather – and, crucially, I can predict the probability it is raining today, or will rain tomorrow.

This is the classic “Umbrella World” example from Stuart Russell and Peter Norvig’s Artificial Intelligence: A Modern Approach, and it is the standard first HMM for a reason: it separates the hidden dynamics from the observable evidence cleanly, and it is small enough to compute by hand.

The clever bit is the inference. Given a sequence of observations (umbrella, no umbrella, umbrella, umbrella…), an HMM can answer three questions:

  • Filtering: what is the probability of the hidden state right now, given everything observed so far?
  • Smoothing: what was the hidden state at some point in the past, given everything observed up to the present?
  • Prediction: what is the probability of the hidden state in the future?

Those three operations are the workhorses of everything from speech recognition to GPS positioning to, as it happens, monitoring servers.

HMMs in the server room

You do not need a weather-obsessed colleague to make this useful. The same approach applies beautifully to infrastructure monitoring, and this is where I find myself considering it in practice.

Consider a web server. There are hidden “regimes” it moves between: healthy, degraded, and failing. You cannot observe the regime directly – it is a hidden state only known if you are looking directly at the server and all its metrics in detail (with a cluster in an enterprise environment, this becomes nearly impossible). But you can observe noisy signals that depend on the regime/state that the server is in, at that moment:

  • Server ping / latency: healthy servers respond in a few milliseconds; a degraded one starts to creep higher; a failing one times out.
  • CPU utilisation: healthy is a normal sawtooth as jobs come in and are responded to; degraded is sustained high; failing is pegged at 100% or spiking erratically.
  • HTTP 500-type errors: healthy servers throw the occasional one; degraded servers throw more; failing servers throw them like they’re going out of fashion and it’s a closing down sale (only the thing closing down is your site).

That maps perfectly onto an HMM. The hidden states are the health regimes, the observations are your metrics, the transition matrix captures “a healthy server occasionally degrades, a degraded server sometimes recovers and sometimes fails”, and the emission probabilities capture “when the server is failing, you see lots of 500s”.

Train the model on historical data and you get something genuinely useful: a system that watches the sequence of observations and tells you the probability the server is currently in each regime – and, more importantly, flags when the probability of “failing” starts climbing before the pager goes off. A single 500 error is noise. A 500 error, followed by rising latency, followed by CPU pegging, is a story – and an HMM is exactly the tool that reads that story from the sequence rather than from any single data point.

This is not a hypothetical. Hidden Markov models have been used for anomaly and intrusion detection in network traffic for years – there is a solid body of research applying HMMs to TCP traffic classification and workload anomaly detection, and the approach keeps showing up because it fits the problem shape so well: hidden system state, observable noisy metrics, sequential dependencies.

Using the HMM approach might not give you 100% accuracy in knowing that there absolutely is a technical incident occurring, but it’s more likely to be able to flag those weird incidents that catch you off guard because Gary forgot his umbrella, there’s a grey cloud appearing, and he nails some half tested code into the CI/CD pipeline at 4.30pm on a Friday. Dammit Gary.

More data, more states: the double-edged sword

The natural instinct, once you have an HMM working, is to make it bigger. More data. More states. More observation signals. Sometimes that helps. Sometimes it genuinely hurts, and it is worth understanding why.

More data usually helps – up to a point. An HMM’s parameters (transition and emission probabilities) have to be estimated from data, usually with the Baum-Welch algorithm (the Baum that worked with Jim Simons at Renaissance Technologies and made oodles of money using HMMs in trading markets). With too little data, those estimates are noisy and the model overfits: it memorises the training sequence instead of learning the underlying dynamics. More data smooths that out. But the data has to be representative. If you train on a month of calm traffic and the server then hits a real incident, the model has never seen the “failing” regime and will flail. More data of the same kind is not the same as more data of all kinds.

More states is a genuine trap. Here is the thing that surprises people: adding hidden states does not just add complexity, it adds it quadratically. A transition matrix for N states has N x N cells. Go from 3 states to 6 and you have gone from 9 parameters to 36. Go to 10 and you have 100. Every extra state needs enough data to estimate its transitions and emissions reliably, or the model starts inventing states that exist only in the training noise (like with Glenn and his early forays into trading markets and not making oodles of money with the cunning idea of using a 30 state HMM matrix).

There is a degenerate case that makes the problem vivid: give the model as many states as you have distinct observations, and it can “cheat” – each state glues itself to one observation, the likelihood looks fantastic on the training data, and the model is completely useless for prediction. It has memorised, not learned. People hit this in practice all the time: fit an HMM with more and more states and watch the validation score climb, peak, and then fall as the model starts overfitting. There is even a body of research on “overfitted” HMMs – models with more states than the data genuinely supports – and on methods (like the Bayesian Information Criterion) for picking the number of states that actually earns its keep.

The practical rule I now use: start small, add states only when the data demands them, and always validate on data the model has never seen. A 3-state model that generalises beats a 10-state model that memorises, every single time.

Wrapping up

Markov models and hidden Markov models are not exotic. A Markov chain is just “the next step depends on where I am now”, written down as a table of probabilities. An HMM is that same idea with the twist that you cannot see the state – you only see noisy evidence of it, and you have to infer the rest. The Home-Work-Cafe chain and the umbrella-carrying Gary are the same mathematics as the server that is quietly heading towards a 500-error storm (also Gary); you just swap the states and the observations.

And the lesson that carries across all of it: the model is only as good as the data it was built from, and a bigger model is not automatically a better one. Sometimes the most valuable thing you can do is resist the urge to add one more state.

Sources (for your further reading Markov Safari)