Questionable Studies

We hear about studies all the time. If I’m to believe what I read in the newspaper, I should cap off each meal with a couple of glasses of red wine, a beer, some chocolate, and a couple of cups of coffee. There must be a study out there somewhere showing that heroin is good for me as well.

When I’m sceptical of a study’s results as described by a reporter, I sometimes read the technical paper by the study’s authors. Reporters sometimes leave out crucial details. For example, a study might show that coffee improves concentration. You might view this result differently if you knew that the subjects were denied caffeine for two days before the tests were performed.

It’s not always the reporters who get it wrong, though. Sometimes the authors of the study mess things up. This is the case with a study by Schleef and Eisinger called  Hitting or missing the retirement target: comparing contribution and asset allocation schemes of simulated portfolios (pdf).

These researchers used past returns (1926-2005) on US stocks and bonds to investigate whether you should shift money from stocks to bonds as you age. One thing they got right was that putting 100% of the money in stocks gave the best odds of reaching a target portfolio amount after 30 years.

Surprising Results

The crazy part comes next. They compared three investment approaches:

1. Decreasing stock percentage as you age. (e.g., 90% ... 50%)
2. Constant stock percentage every year. (e.g., 70%)
3. Increasing stock percentage as you age. (e.g., 50% ... 90%)

As long as the average percentages in stocks and bonds over the 30 years is the same in each case (70% in the examples above), you’d think that all three approaches would give the same results when averaged over many simulations. That’s not what these researchers found.

They conclude that the best option is to increase the stock percentage over time (the opposite of the usual advice). Unfortunately, this conclusion is just a consequence of the strange way that they did their simulations.

Where They Went Wrong

To do each simulation, the researchers chose a starting year at random between 1926 and 2005. Then they used the returns from the 30 years starting at that point in time. If they reached 2005 before the 30 years were up, they continued from a new random starting point.

This approach may seem reasonable enough, but it means that by the time you get to the end of the 30-year simulation, you are very unlikely to be using returns from the early years of the 1926-2005 range. This creates a bias.

From 1926 to 1945, US stocks didn’t outperform bonds by very much. The difference was greater in later years. This means that in the 30-year simulations, the later years tended to have a greater advantage of stocks over bonds.

To boost returns in a given simulation, it was more important to have a high allocation to stocks at the end of the 30 years than at the beginning. This explains why approach #3 above where the stock allocation was high at the end gave the best results. But, this doesn’t reflect any real-world truth. It’s just a result of a poor choice in how to run the simulations.

Reality

In reality, there is no reason to believe that any one of the three approaches will produce higher returns than the other two.

The purpose of shifting money from stocks to fixed income as you near retirement is not to maximize your wealth; it is to create more certainty in how much money you’ll have to live on. If the stock market suddenly goes down, at least you’ll have a few years to adapt your lifestyle to the new reality.

Based on historical data, I choose to invest 100% of my money that I won’t need for about 3 years in stocks. I use short-term bonds and cash for money that I’ll need to spend within the next 3 years.