9 · PMF, PDF & CDF
Discrete RVs get a PMF; continuous RVs get a PDF (density, not probability); both have a CDF. Master all three.
Discrete RVs get a PMF (probability OF each value); continuous RVs get a PDF (density, not probability); both have a CDF (cumulative).
Without this:
Without these three functions, you can't compute P(X ≤ x), draw a histogram against a fitted distribution, or interpret a model's output.
A random variable (RV) is a function that maps outcomes of a random experiment to numbers. There are two flavors:
-
Discrete RV: takes countable values (0, 1, 2, …). Described by a Probability Mass Function (PMF): p(x) = P(X = x). Each value has a genuine probability mass ≥ 0 and the masses sum to 1: Σ p(x) = 1.
-
Continuous RV: takes values on a continuous range (any real number in an interval). Described by a Probability Density Function (PDF): f(x), where probabilities are areas, not heights — P(a ≤ X ≤ b) = ∫_a^b f(x) dx. Critically: f(x) is not a probability. It is a density. It can exceed 1 without violating anything, but the integral over any interval is always in [0, 1].
Both types of RVs share one universal description:
Cumulative Distribution Function (CDF): F(x) = P(X ≤ x). It is always:
- Non-decreasing: F(a) ≤ F(b) whenever a ≤ b
- Right-continuous
- F(x) → 0 as x → −∞
- F(x) → 1 as x → +∞
The CDF connects everything: for discrete RVs, F(x) = Σ_{k ≤ x} p(k). For continuous RVs, F(x) = ∫_{-∞}^x f(t) dt, so f(x) = F'(x).
Python (in browser)
Each bar in a PMF bar chart is a genuine probability. All bars sum to exactly 1. Open /tmp/binom_pmf.png to view. The Binomial(10, 0.4) peaks at k=4 (the expected value n·p = 4).
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Python (in browser)
The PDF curve height at x=0 is 0.399, NOT a probability. The blue shaded area under the curve between −1 and +1 IS a probability (≈ 0.683). Open /tmp/norm_pdf.png. Probabilities are always areas, never heights, for continuous distributions.
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Python (in browser)
Left: the discrete Binomial CDF is a staircase — it jumps at each integer. Right: the continuous Normal CDF is a smooth S-curve. Both start near 0 and end at 1. Open /tmp/cdfs.png to compare side-by-side.
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
The CDF is the bridge between PMF/PDF and probability intervals. For discrete: F(k) = Σ p(j); PMF = differences. For continuous: F(x) = ∫ f(t)dt; PDF = derivative. Probability over an interval is always CDF(b) − CDF(a).
X is a continuous random variable. What is P(X = 3.14)?
Discrete and Continuous Probabilities — the MML chapter that defines PMFs and PDFs rigorously, introduces the expectation operator E[X], and connects them to the CDF with formal proofs.
- **PMF** p(x) = P(X = x): only for discrete RVs. Each value has a true probability mass; all masses sum to 1.
- **PDF** f(x): only for continuous RVs. It is a *density*, not a probability. P(a ≤ X ≤ b) = ∫ f(x)dx = CDF(b) − CDF(a).
- **CDF** F(x) = P(X ≤ x): defined for all RVs. Non-decreasing, from 0 to 1. The universal tool for computing probability over any interval.
A regression model's output is a PDF parameter (mean of a Gaussian). A classifier's softmax is a PMF over classes. AUC is the area under the CDF of true positives vs false positives.
If you remove it: You can't interpret a loss like negative log-likelihood without knowing what 'likelihood' means in PMF/PDF terms.