Posterior Visualizer

September 4, 2026

The posterior is crucial in a variety of machine learning methods, such as variational auto-encoders (VAEs). Here, I seek to build up and demonstrate a visual understanding of what the posterior actually is, and the steps involved in computing it.

The posterior is defined in Bayes' rule, which relates it to the likelihood, the prior, and the evidence/marginal probability.

$$p(\theta \mid X) = \frac{p(X \mid \theta)\, p(\theta)}{p(X)}$$

Let's try and see if we can better illustrate what Bayes' rule is actually relating. We consider a motivating example of a number of coin tosses of a (potentially) biased coin. We assume that the actual bias is unknown or unobservable to us, but we can estimate it using a parameter $\theta$.

$\theta$

In this example, we have a single parameter $\theta$ which represents the coin's bias, with $\theta = 0.5$ representing a fair coin, $\theta = 1.0$ representing a coin which will only ever come up heads, and $\theta = 0.0$ representing a coin which will only ever come up tails.

$X$

$X$ is our dataset: our actual observed set of coin flips. In the real world, you don't usually get to choose this, but here you can build your own $X$ below. Tap on a coin face to flip it.

$h$ = 3 heads, $t$ = 1 tail

Likelihood: $p(X \mid \theta)$

Now that we have our observed dataset $X$ and our parameter $\theta$, one of the first things we might want to know is: how good is our guess? Intuitively, if you rolled heads 10 times in a row, you would think it's highly unlikely that the coin was fair, and you would think it's even more unlikely that the coin is tails-biased.

Take a second and consider exactly why that is. If you do, you'll probably back up your claim using some calculation assuming the probability of heads is 0.5, and then showing how unlikely it is to get 10 heads in a row. And then if we assume the probability of heads is 0.1 (i.e the coin has a 90% change of tails), it's even less likely.

This notion is precisely what likelihood quantifies. It quantifies "given our guess of $\theta$, how likely is our dataset $X$?" Below you can see the math worked out for your current guess of $\theta = 0.7$.

Once we have the likelihood for one value of $\theta$, the natural next step is to take all possible values of $\theta$ and plot the likelihood function.

• likelihood

This lets us see which values of $\theta$ best explain the dataset. Look at your value of $\theta$ plotted above using the vertical line. Is it the most likely value? Which other values may explain $X$ better?

Prior: $p(\theta)$

So we have our parameter $\theta$ which we can vary and see which values best fit the data. But what if, just before the coin tosses, a mysterious stranger pulled you aside and said "don't ask me how I know, but the coin is biased 70% heads." This would make you much likely to at least start your guess off at 0.7. Conversely, you may have no clue about the coin, but generally believe like most of us that most coins are fair coins, in which case perhaps you would start your guess at 0.5, and be much more doubtful as the coin got more biased either way.

We want some ability to nudge our algorithm to take into account this prior information we may have about the world. Fittingly, this is called the prior in probability, and written $p(\theta)$.

Below, you can experiment with different kinds of distributions for your prior using a Beta distribution. You don't really need to know what the means. Just know that it's bound from 0 to 1, which makes it a natural fit for $\theta$ here. It also allows us to easily express both a uniform distribution ($\alpha = \beta = 1$) and a bell-shaped distribution if you increase $\alpha$ and $\beta$ simultaneously. Increasing $\alpha$ pushes the distribution towards heads at a higher certainty, while increasing $\beta$ pushes the distribution towards tails.

Evaluating $p(\theta)$ at $\theta$ gives us the density 1.000

Evidence: $p(X)$

Whereas the likelihood tells us how well each $\theta$ explains $X$, the evidence tells us how well our model explains $X$, by averaging the likelihood weighted by the prior.

$$p(X) = \int p(X \mid \theta)\, p(\theta)\, d \theta$$

We plot the joint $p(X \mid \theta)\, p(\theta)$ below. The area under this curve represents how well our model explains $X$, which is $p(X)$.

$p(X) =$ 0.0500

Posterior: $p(\theta \mid X)$

Now we have finally arrived: the posterior. The posterior is the flipside of the prior. Whereas the prior asks "how likely is $\theta$ before seeing $X$," the posterior asks "how likely is $\theta$ after seeing $X$?" It is an update of the prior, weighted by the likelihood.

$$p(\theta \mid X) = \frac{p(X \mid \theta) p(\theta)}{p(X)}$$

Notice that this is literally just Bayes' rule. Below are all parameter sliders so you can directly see the effects of different $\theta$ and the prior $p(\theta)$ on the posterior.

• prior
• likelihood (rescaled)
• posterior

$p(\theta \mid X)$ = 0.1029 × 1.0000.0500 = 2.058

Notice a few consequences. First, if we use the uniform distribution, we are essentially saying that we don't know anything about $\theta$. Consequently, we are pulled 100% to likelihood, because there is no prior information to update. Looking at the math, you can see that $p(\theta) = 1$ in the case of uniform, and thus

$$p(\theta \mid X) = \frac{p(X \mid \theta) \cdot 1}{p(X)} = \frac{p(X \mid \theta)}{p(X)}$$

just a scaled version of the likelihood.

Also, notice that the more you increase $\alpha$ or $\beta$, we are expressing more certainty over our prior, and our posterior is pulled closer to the prior. Conversely, decreasing $\alpha$ or $\beta$ expresses less certainty, and the posterior is pulled closer to the likelihood.