Polya's Urn and Self-Reinforcing Draws

The problem. An urn starts with 1 red ball and 1 blue ball. You draw a ball at random, note its color, put it back, and add one more ball of that same color to the urn. Repeat many times. Intuitively, whichever color gets an early lead should dominate more and more — a rich-get-richer effect. Does the fraction of red balls settle down to a predictable number, like 50%, as the number of draws grows? Or does it stay genuinely random forever?

Why this differs from ordinary sampling

In ordinary sampling with replacement, each draw is independent and the long-run fraction converges to the fixed true probability (Law of Large Numbers, settling near 50% here). Polya's urn breaks that independence on purpose: every draw changes the composition of the urn in the direction of what was just drawn, which is exactly the "success breeds success" mechanic behind phenomena like technology adoption, citation counts, or a strategy attracting more capital after early gains. The urn actively reinforces its own history.

The surprising two-part answer

Part 1 — it does converge, but to a random limit. As the number of draws grows, the fraction of red balls in the urn converges to some limiting value $p$ — but $p$ itself is a random variable, uniformly distributed on $[0,1]$, determined by the accidents of the early draws. Run the process many times and you'll get different limiting fractions each time — some runs settle near 90% red, others near 10% red, and every value in between is equally likely as a limit. Early luck effectively locks in the long-run outcome.

Part 2 — despite that, each individual draw is still exactly a coin flip. At step $n$, before you know how many balls of each color are in the urn, the unconditional probability the next draw is red is always exactly $\frac12$, by a symmetry argument (any labeling of "which ball is red" is exchangeable). The process is path-dependent in its long-run limit, yet perfectly fair draw by draw.

Polya's urn is, underneath, a beta-distribution process much like the explorer above: each draw is like a data point updating a running belief about the red fraction. Watch how the "posterior" shifts more decisively after early observations and settles down as more data accumulates — the same mechanism that lets an early lucky streak of red draws lock in a high long-run red fraction here.

Worked example: tracking one path by hand

Start: 1 red, 1 blue (2 total). Draw 1: suppose red is drawn (prob $\frac12$) — urn becomes 2 red, 1 blue (3 total). Draw 2: probability of red is now $\frac23$; suppose red is drawn again — urn becomes 3 red, 1 blue (4 total). Draw 3: probability of red is now $\frac34$; suppose blue is drawn this time — urn becomes 3 red, 2 blue (5 total), red fraction $=60\%$. Notice how quickly one early red streak pushed the red probability up to $75\%$ before the first blue even appeared — a concrete illustration of the reinforcement, even though this specific path could have gone very differently with different early draws.

The general technique: exchangeability

The clean way to see that each draw is unconditionally fair (Part 2) is exchangeability: the probability of any particular sequence of red/blue outcomes depends only on how many reds and blues appear, not on their order — swapping the order of two draws in a sequence doesn't change its probability. That symmetry is what lets you conclude each marginal draw is $\frac12$ red even though the conditional probability, given the current urn state, is constantly shifting.