The experiment
Victor Haghani and Richard Dewey ran this experiment on 61 “college age students in economics and finance and young professionals at finance firms.” Their very readable working paper is available here. Despite the financial sophistication of the subjects, they didn’t do very well with the betting.
The experimenters capped winnings at $250, but only 13 of the subjects got to $200. In fact, 17 of them ended up with less than $2. The most surprising finding to me is that 41 subjects bet on tails at least once, 29 bet on tails more than 5 times, and 13 subjects bet on tails more than one-quarter of the time!
Given how poorly the subjects did in this fairly simple game, it’s hard to be optimistic about people making sensible choices in the much more complex world of investing.
Strategy
After reading the game’s rules, my first thought on the correct way to play was to repeatedly bet on heads with a fixed fraction of the current bankroll. How to choose the fraction of bankroll to bet goes all the way back to Daniel Bernoulli in 1738 who suggested that we should treat money as having logarithmic utility. This may sound complex, but it’s not too difficult. It just means that each doubling of your net worth is equally valuable, and that each new dollar you get is worth a little less than the last dollar.
The standard way to derive the correct fraction to bet isn’t very accessible to many readers, but an equivalent way to look at it is easier. Let’s call the betting fraction x. So, if we always bet 15% of the current bankroll, then x=0.15. Your money grows by a factor of 1+x if you win, and 1-x if you lose. After 5 tosses, the median outcome is 3 heads and 2 tails. If you bet on heads each time, this median outcome grows your money by a factor of (1+x)3(1-x)2. If you try different values of x, you’ll find that this expression is largest when x=0.2. This means betting 20% of your money on each coin flip.
So, the first bet would be $5, leaving you with either $20 or $30. If you lost, the next bet would be only $4. If you won, the next bet would be $6. After the first bet, because your odds of winning are 60%, your expected bankroll is $26. So, on average, your bankroll grows 4% on each bet.
This method of determining the fraction of your money to bet is known as the Kelly Criterion. The two researchers suggest this approach as a good way to play their game. Indeed, it is a good way to play the game, but not optimal. Getting back to the experiment, “average bet size across all subjects was 15% of the bankroll,” which is actually more conservative than the Kelly Criterion. However, betting on tails and erratic bet sizing sunk several subjects.
A slightly modified game
Suppose the subjects knew in advance that the cap on winnings was $250 and knew they’d get exactly 300 bets. Further, let’s assume that $250 is small enough in each subject’s life that we can treat the money as having linear utility (this means each extra dollar won is worth the same as the last dollar won). This doesn’t seem like a huge difference in the game. However, the optimal strategy in this case is very different from what you get with the Kelly Criterion. In fact, the optimal strategy doesn’t even depend on the coin’s bias!
The experimenters’ goal was to create an experiment that mimics the properties of investing as closely as possible without costing them too much money. Even this simple experiment put them at risk of losing $250 to 61 subjects, for a total of $15,250. Fortunately, poor play reduced their costs significantly.
Unfortunately, the $250 cap on winnings makes optimal play quite different from optimal investing strategies. This means that chastising subjects for betting patterns that differ from the Kelly Criterion can be misguided. No doubt many subjects made bets that can’t be justified, but as I’ll show below, optimal betting doesn’t look much like the Kelly Criterion.
The authors’ simulations of strategies based on the Kelly Criterion gave the expected value of the game as “just under $240.” However, when the $250 cap is known, the optimal strategy has an expected outcome of $246.60.
I found this result with a method that gave exact results rather than using Monte Carlo simulations. I also used computation methods with large integer values to avoid floating point rounding errors. The general method was to start at the end and work back.
After the last bet, the value of each bankroll amount ($0.00, $0.01, $0.02, ..., $250.00) is just the bankroll amount. (To take into account utility, you could replace these values with their utilities, but that isn’t what I did.) Backing up one bet, we look at every bankroll amount and every possible bet size and choose the bet size that maximizes the expected outcome. Then the value of each bankroll amount before the last roll is the expected value after the optimal last bet.
As an example, if you have $125 before the last bet, your best strategy is to bet it all. This has an expected outcome of $150 because of the coin’s bias. Working through all possible amounts you could have before the last bet, we can find the expected final bankroll given the best bet size. We can then go back to the second to last bet. Continuing this way, we can back up a total of 300 bets. Starting with $25, the expected outcome works out to $246.60.
A continuous version of the game
It turns out that this game is actually easier to analyze if we eliminate the restriction that bets be multiples of a penny. The outcome doesn’t change much by removing this restriction. When bets are multiples of a penny, the expected outcome is $246.6063. When fractions of a penny are permitted, the expected outcome is $246.6066, just 3 hundredths of a penny more.
To analyze the unrestricted game, let’s start with the simple relationship between bankroll size and final winnings after the last bet. They are equal:
Now, let’s back up one bet. If you have less than $125, your best move is to bet it all with an expected increase of 20%. If you have more than $125, your best move is to bet enough that you’d end up with $250 if the coin comes up heads. This gives the following relationship between your bankroll before the last best and your expected winnings:
It looks like we nailed down the ends of the blue line, grabbed it in the middle, and stretched it up 20% to make the red line. If we then analyze the best bet size on the second to last bet, it turns out that there are often many bet sizes that are equally good. A simple rule that gives one of the optimum bet sizes is to bet the difference between your bankroll and the closest one of $0, $125, or $250.
With this strategy, the new line can be calculated from the red line in a similar way to the way we made the red line. Start by nailing down the red line at the ends and in the middle where the slope changes. Then grab each half of the red line in the middle and stretch it up to increase the slope of its first half by 20%. This gives us the relationship between the bankroll with 2 bets left and the expected final winnings:
Notice that the middle two segments of the green line have the same slope, so we only have slope changes at $62.50 and $187.50. With 3 bets left, tied for best strategy is to bet the difference between your bankroll and the closest one of $0, $62.50, $187.50, or $250.
To get to the line showing the expected winnings with 3 bets left, we nail the green line down at the ends and the two points where the slope changes (not the middle). Then drag each line segment up as before:
It’s getting a little harder to see what’s going on, but the purple line has 4 different slopes. The first 1/8 of the way it has slope 1.2*1.2*1.2. The next 3/8 of the way, its slope is 1.2*1.2*0.8. The next 3/8 has slope 1.2*0.8*0.8, and the final 1/8 has slope 0.8*0.8*0.8. Some readers may recognize that the bankroll gets divided into segments that look suspiciously like Pascal’s triangle, and the expected winnings follow the Binomial distribution.
Suppose your bankroll is at at one of the 3 points where the purple line changes slope (bankroll of $0, $31.25, $125, $218.75, or $250). Call these the nailed-down points. Then the optimal strategy is to bet enough to get to one of the nearest nailed down points of the green line. Which nailed down point you get to on the green line depends on whether you win or lose the bet (except if you’re already at $0 or $250). The optimal strategy on the second to last bet will take you to a nearest nailed down point on the red line.
This rule works all the way back to the start when you have 300 bets left. If you are ever on a nailed-down point, you just keep jumping to other nailed-down points until finally you end up with nothing or the full $250. The following chart shows the expected winnings for optimal play with 300 bets left. This is the same chart we would have produced if I had continued the above sequence of charts for 300 steps (except that below I only show the last line and not all that came before it).
The blue curve is actually made up of 301 line segments. It shows that optimal play gives the maximum payout of $250 with high probability for all but the smallest starting bankrolls. When you start with $25, the odds are 98.64% that you’ll get the full $250. For the first wager, bet amounts from $1.99 to $3.16 are equally good.
However, if we play this game for 10,000 bets, the best starting bet drops to between $0.37 and $0.51. And if we are given only 3 bets, the optimum strategy is to just bet it all 3 times hoping to end up with $200. So, we see that optimum bet sizing can be very different from the Kelly Criterion.
As promised earlier, optimal bet sizes don’t even depend on the coin’s bias. In the procedures where we nailed down lines grabbed them in the middle and stretched them up, the coin’s bias affected how high we stretched the lines up, but not the bet sizes.
If heads comes up only 55% of the time, the expected winnings drop from $246.60 to $168.56. If heads comes up 70% of the time, the expected final bankroll is less than one-millionth of a cent under $250. But the bet sizes don’t change, even though the Kelly Criterion would have you making different bet sizes.
Not knowing the winnings cap
Returning to the original games where subjects don’t know the winnings cap, the optimal strategy depends on your opinion of the probability distribution of the winnings cap. However, for any reasonable assumed distribution that would not destroy the experimenters financially, optimal strategies won’t look much like the Kelly Criterion.
This doesn’t mean that the Kelly Criterion is wrong in other contexts. It just means that the winnings cap changes this game significantly. We can’t say that subjects are crazy for making bets that don’t agree with the Kelly Criterion. While Kelly-sized bets work reasonably well, there are many other strategies that work well, and some that work better for this game.
Probably the biggest challenge for the experimenters is that the amount of money at stake is fairly small compared to the subjects’ other assets, including human capital. This shows up most at the end of the game where is makes sense to make large bets. When you’re handling your actual portfolio, situations where it makes sense to bet it all are highly contrived.
Conclusion
These experimenters have done some fascinating research that digs into how people understand investment risk. Despite the fact that the game they designed has limitations that are hard to fix, their results were illuminating, mainly because the subjects performed so poorly. Another conclusion readers can draw is that I enjoy analyzing games and dug further into this one than was necessary to explain the game’s differences from actual investing.
Now that was really interesting. I just played it without reading your analysis or thinking about it too much. Still haven't read your analysis. In the first 20 flips I got 6 tails in a row (which took me down less than $5) followed by 8 heads. So I'm suspicious that this is actually modeling a 60% biased coin flip. Seems like there are some pretty long tails in there like in the markets.
ReplyDeleteEventually I clawed my way up to $120, realizing with 5 minutes left that I could set up the next bet while waiting for the last. I only got in about 140 bets, I think I could have hit the limit if I had got in 300 bets. I bet to much at the beginning, intuitively the lesson I learned from playing is don't bet what you can't afford to lose.
I'd be interested to hear your and other's experiences playing this.
Now, to read your analysis :).
@Greg: Glad you found it interesting. If the flips aren't as advertised, I'm guessing it's a technical problem rather than being deliberate. But most likely it works fine and you just got a slightly unusual sequence of flips. I got to around $150 easily enough by betting 20% of my bankroll each time. Then I just bet $10 at a time until I got to $250. But as I found when I analyzed the game, there are better strategies.
DeleteInteresting simply-posed but fairly complex example, MJ!
ReplyDeleteThink it highlights a couple of interesting points.
1) Most people, most of the time, are not in a position (capability, time...) to devise and follow optimal strategies in the face of uncertainty. This is particularly true since little twists may well significantly change the optimal answer (e.g. Monty Hall problem). In investing (your blog) or business decisions (my consulting area), it's therefore often much more important to avoid obvious decision pathologies, e.g. betting on tails, and choose reasonable heuristics, e.g. Kelly criterion even if not fully optimal.
2) I think it's a good example of a distinction in risk management that's actually pretty important. There's aleatoric uncertainty, which is uncertainty from randomness (how will the coin tosses go). And there is epistemic uncertainty, lack of knowledge about the problem (what is the cap). Quite often, the best solution to aleatoric uncertainty is optimization, while to epistemic uncertainty it is finding the right heuristics.
3) Without having read all your (interesting) analysis in detail, my immediate reaction to your description of the problem was that the best on-the-spot strategy would depend on my subjective judgment on how "remote" the cap as well as the time limit are going to be as likely constraints. The Kelly criterion applies for uncapped, long-run returns. If I have no time limit but strong cap, I should inch up with minuscule bets (to minimize chance of ruin) to discover the cap. If I have limited time but no cap, there's some sort of binomial distribution (more or less?) optimum. If I had just pencil and paper and 5 mins to figure out a reasonable approach, it would be based on my guesstimate whether the experimental design constraints are likely to be so that the (unknown) cap is likely to be more restrictive or the (known) time limit is.
@Martin: While I enjoyed finding optimal strategies and found it very interesting that optimal strategies differ greatly from the Kelly Criterion, what's actually important is the number of subjects who could not help themselves from betting too much and sometimes betting on tails. Almost any reasonable strategy gives great results in this game, but subjects did terribly anyway. I can't see why subjects bet on tails, but I can understand over-sized bets. When I played, I never bet more than 20% of my bankroll, but I did feel some frustration when my progress was thwarted by tails outcomes. It was tempting to bet more and make back the loss quickly. Fortunately, the math of why this is a bad strategy is front of mind for me and I didn't give in. If you bet more than about 39% of your bankroll, odds are that it will shrink over time, even with the 60% bias for heads.
DeleteI think anybody who bets on tails simply doesn't understand that past flips have no effect on future flips.
Delete@Greg: That's one explanation, but I think that seeing patterns in randomness is baked into humans. It takes mental effort to convince yourself that the pattern you see is just randomness. Playing craps in Las Vegas, it's very easy to believe in a "hot table". Tables do feel hot at times, even though I know it makes no sense. And it's almost impossible to convince most players that table hotness doesn't exist. In the same way, players of this game who think they see a pattern may well choose to bet on tails, even though we know it's a bad move.
DeleteI fell into ruin, ending up with $0.18. Perhaps 120 bets? I bet half my bankroll each time, I guess too aggressive. Seemed intuitive that the odds were in my favour and I should bet big. If I had won, I would have felt justified, but now I see that a run of bad luck can ruin good odds.
ReplyDeleteI could tell there was some sort of trick to optimizing outcomes, but I got it wrong. I suppose my error was impatience. If one has fewer chances, maybe more aggressive being is better?
@Gene: Unfortunately, your outcome is not unusual using this strategy. One way to understand this is to imagine you start with $32 and bet half your bankroll 5 times. You expect to win 3 times and lose twice. The order of wins and losses doesn't change the final outcome, so let's apply the losses first. Your starting $32 goes to $16, then to $8. Next you win 3 times. Your $8 goes to $12, then $18, and finally $27. So, this typical outcome loses you $5. This means that the median outcome is to lose 5/32 of your bankroll every 5 flips. Starting with $25 and flipping 120 (=5x24) times, your median final bankroll is 25x(27/32)^24 = $0.42. So, you were a little unlucky, but not much. A much better strategy is to be less aggressive.
DeleteI played the game before I read your analysis and reached $250 in 27 flips betting 1/3 of my bankroll for each flip. I got three tails in a row on flip 16 and dropped to $18.91, but then I got a string of good luck. When I won over $200 I was told the cap was $250, so I only bet enough to reach the cap.
DeleteI considered betting 1/2 to 1/4 of my bankroll per flip, but settled on 1/3 when I considered that the odds of getting three tails was about 6% (40%^3). I figured if I bet 1/2 with each flip, I might drop too low if I got a bad sequence so I settled on 1/3 as a rough approach. The investment return and time horizon also played a roll in my strategy. I considered that the investment return was good at about 10% return per flip (60% - 40%)/2, and the investment time horizon was long (I figured I could make at least 60 flips in 30 minutes). I assumed I didn't know the cap when I started so I wanted to be aggressive enough to get into the thousands if the cap allowed ($25 x 1.1^60 = ~7k.
What did I learn about investing from this? I wish my time horizon and possible investment return was as good as this game. The real world isn't this easy.
Or maybe the expected return was 25/3 x 1.1^60 = $2.5k. You're the math expert - feel free to correct all this.
DeleteI tried the game again and it took 100 flips to get to $250, dropping as low as $1.23. The game reinforced something else about investing - sometimes we just get lucky and sometimes we don't. It was starting to get painful when my bankroll dropped so low.
Delete@Blitzer68: If you bet it all on each flip, your expected return is 20%/flip. But betting 1/3 each time, your expected outcome is 60%(4/3) + 40%(2/3) - 1 = 6.7%. However, expected returns are misleading in this game. Much of the expected return after many flips comes from enormous payoffs with minuscule probabilities. It's better to look at the median outcome (3 heads out of 5). Betting 1/3 each time would leave you with ((4/3)^3)(2/3)^2 = 256/243 times your original bankroll after each 5 flips. After 60 flips, you expect to grow your bankroll by about a factor of 1.9. After 100 flips, the expected growth factor is 2.8. So, it appears you got somewhat lucky. The optimum bet size to grow the bankroll fastest is 20%.
Delete