Investing Skill or Luck?

Investing contests are often won by someone who puts all of his hypothetical money into a few risky stocks whose share prices double or better over a short time. Is this skill or luck? The truth is that we can’t tell. If the same person wins several contests in a row, then we may be forced to believe that the good results come from skill.

Any time you have thousands of people picking stocks, it’s inevitable that a few will have outstanding results. How can we tell if the winners were skillful or lucky? In his book, The Intelligent Portfolio, Christopher L. Jones discusses methods of distinguishing skill from luck.

Jones chose the example of the Legg Mason Value Trust fund as an example. This fund beat the S&P 500 each and every year from 1991-2005 by an average of 7%! To get a handle on whether this is an unusual result, Jones uses simulations.

He simulated 10,000 funds with similar investing styles, but making random stock selections, and checked how they performed compared to Legg Mason. It turned out that about 1 out of 30 simulations beat Legg Mason. Based on this, Jones concluded that Legg Mason’s results were not unusual and could easily have simply been luck.

If you think you smell something fishy here, it’s because something is fishy. If 1 out of 30 simulated funds beat Legg Mason, then why didn’t any real funds beat Legg Mason? This seems like a very unusual result, and it made me suspicious of Jones’ analysis.

Unfortunately, Jones didn’t give much detail in how he did the simulations. One correct way to do the simulations is to have funds select stocks at random according to the same distribution as Legg Mason, and assign returns equal to the actual returns of those stocks in the relevant time period.

An incorrect way to do the simulations would be to randomly generate entirely new stock market histories. This would be answering the question “what are the odds that some 15-year period will produce better returns than Legg Mason produced?” This is entirely different from the question “what are the odds that a random stock picker could have done better than Legg Mason from 1991-2005?”

While I agree with Jones that strong returns could easily be just luck, I’d like to know more about how he did the Legg Mason analysis. If he got this wrong, then it casts doubt on the correctness of other analyses in his book.

I found it curious that Jones didn’t tackle the best long-term track record available: Berkshire Hathaway. Berkshire’s results have been so strong for so long that it seems inconceivable that it is just luck. But, I haven’t attempted to analyze this case.