To understand these problems, we need to look at the fairly simple way these simulators work. To generate a possible outcome for your portfolio, many of these simulators begin with some actual historical investment returns from a range of years. They build your simulated results one year at a time by choosing one of the historical years randomly and applying that year’s returns to your portfolio.
Before getting into why this method has some issues, let’s go to a couple of charts. I grabbed some annual TSX investment returns for the past 47 years. Then I made a chart of portfolio growth for all 28 overlapping 20-year periods. Here are the 28 results for a starting portfolio of $20,000:
It’s hard to learn too much from this mess, but bear with me. What matters is the range of outcomes. The next thing I did was to put the 47 years of TSX returns into a random order and draw the same chart again. I used all the same return percentages, but in a different order. Here is the result:
The main thing to notice is how much more spread out the 20-year returns were in the random order case. Both charts have exactly the same log-scale. The portfolios for the TSX returns in actual order had an ending portfolio value range of $82k to $251k. For the randomized return order, the range was $46k to $339k. This is a substantial difference. Once the return order is randomized, the portfolio outcomes become much wilder.
What’s the explanation for the change? It turns out that the stock market has a tendency to revert to the mean faster than random chance would suggest. A string of years with poor returns are somewhat more likely to be followed by good returns than more poor returns, and vice-versa. When we randomize the order of the annual returns, we eliminate this effect.
So, many retirement simulators give results that are wilder than you can really expect in real life. If you choose a spending level and run the simulations, the simulator might overstate the likelihood of running out of money. On the other hand, if average annual stock returns are lower in the future than they were in the past, the simulator may be understating the likelihood of running out of money. We might hope that these effects would cancel, but it’s likely that one will dominate the other, and it’s hard to say which.
This doesn’t mean that using Monte Carlo simulations to get a sense of the range of possible outcomes is a bad idea. It’s just important not to be too impressed with the science and math. These simulations have two important limitations: they remove return correlations across time, and they assume that future average returns will match past average returns. Buyer beware.
So is there a way of generating more realistic random returns that includes this effect?
ReplyDelete@Anonymous: I haven't worked out a way I'm satisfied with. One possible approach would be to separately model growth in earnings and market P/E ratio.
DeleteAnother approach would be to use actual returns over longer time periods. For buy-and-hold investors, most dollars remain invested for many years. Instead of trying to apply many one-year returns, we could use historical return information to choose a few decade-long returns.
All types of models have their limitations, but it's possible to do better than most existing simulators. In the end, though, there is no certainty in life.
Use Jim Otars calculator that uses actual returns going back over 100 years. Shows what would have happened to if you retired just before 1929 or 2008 etc.
ReplyDelete@Thomas: Otar's calculator can give useful results, but the future could easily have outcomes that have never been seen in the past. The simulators I described in this article may give outcomes that are too wild, but Otar's calculator is likely to give outcomes that aren't wild enough.
DeleteDan Hallett had a comment on this post, but something went wrong with the comment interface:
ReplyDeleteMichael, good post on using Monte Carlo for retirement. What you describe - i.e. using actual market returns - is exactly what Jim Otar has been doing for some time. It's an appealing approach because, as you point out, market returns aren't random. One big piece of empirical evidence of this is the momentum effect which has been observed in all kinds of financial asset prices all around the world.
If you haven't yet have a look at Otar's work. If I remember his statements correctly, he says that the factor that dominates all others in determining the ultimate success/failure of a withdrawal strategy is: luck.
Moshe Milevsky has done some interesting work on this problem too. He came up with something he calls the probability of ruin. And he devised a formula - a much simpler alternative to Monte Carlo - that he claims is at least as accurate. He offered up a spreadsheet with his 'calculator' for some time. A quick search resulted in finding the short calculator but I have both the short and more detailed versions (downloaded years ago).
Finally, when I was digging into this topic just about 15 years ago I happened upon a good paper on the appropriate use of MC models and the many weaknesses of using MC in financial planning and related decisions.
http://www56.homepage.villanova.edu/david.nawrocki/Finance%20and%20Monte%20Carlo%20-%20Nawrocki.pdf
@Dan: Otar doesn't reorder past returns (as far as I can tell), so his calculator won't have the problem of destroying correlations in time. But his calculator will miss return patterns that have never happened in the past but are possible in the future.
DeleteThe paper you pointed to explains the problems of doing Monte Carlo simulations based on the normal distribution (rather than actual historical returns).