Six online learning algorithms (four bandits, two with full information) decide each period which assets to hold. They start knowing nothing and learn only from the returns they observe. The page reloads the latest market data on every visit and replays the whole history, so you can see how each learner would have done up to today and how far it trails the best asset in hindsight.
| Learner | Feedback | $1 became | CAGR | Regret | Holding next period |
|---|
Each period (a day, week or month) every learner commits to a distribution pt over the K assets. The market then reveals gross returns xt,i = Pi(end)/Pi(start). Prices are total-return (dividends reinvested), fetched from Yahoo Finance by a scheduled GitHub Action after every US trading day.
The bandit algorithms need rewards in [0,1], so the log-return g = log x is mapped to r = ½ + g / (2c), clipped, with c = 5% daily, 10% weekly and 20% monthly. This is an affine map, so maximising reward is the same as maximising log-wealth, except on the rare periods that get clipped.
UCB1 holds the asset with the highest optimistic index, mean + √(2 ln t / ni). It assumes each asset's rewards are i.i.d. KL-UCB builds a tighter upper confidence bound from the Bernoulli KL divergence: the largest q with ni·kl(mean, q) ≤ ln t. EXP3 makes no statistical assumptions. It keeps importance-weighted reward estimates and samples an asset from exponential weights, which guarantees O(√(KT log K)) regret even against an adversarial market. EXP3++ (Seldin & Slivkins, 2014) adds exploration that shrinks with the empirical gaps between assets. It keeps EXP3's adversarial guarantee and gets polylogarithmic regret when the market behaves stochastically.
In a real market you see every price, not only the one you hold, so bandit feedback discards information you would have for free. The two full-information learners show what that information is worth. Hedge (exponential weights) uses the same rewards as the bandits but updates on all K of them each period. EG (Exponentiated Gradient, Helmbold et al., 1998) is a true portfolio algorithm. It holds a diversified, rebalanced mix and takes multiplicative steps along the gradient of log-wealth. Diversification lets a portfolio beat every single asset, so EG's regret can go negative.
Regret is measured in log-wealth against the best single asset in hindsight, re-chosen at every date. When the leader changes (bonds in 2010–11, Apple in the mid-2010s, NVIDIA since), the curves can jump or fall. Over periods this short the gaps between assets' mean returns are tiny next to their volatility, so the stochastic bandits explore for a long time and EXP3 stays close to uniform. That is the theory working as stated, not a bug: √T regret only becomes small relative to T after many rounds.
Caveats: no transaction costs, taxes or slippage, and the named stocks were picked with hindsight (survivorship bias), which flatters every strategy that finds them. This is an illustration of learning algorithms, not investment advice.