Investing Skill or Luck?
Investing contests are often won by someone who puts all of his hypothetical money into a few risky stocks whose share prices double or better over a short time. Is this skill or luck? The truth is that we can’t tell. If the same person wins several contests in a row, then we may be forced to believe that the good results come from skill.
Any time you have thousands of people picking stocks, it’s inevitable that a few will have outstanding results. How can we tell if the winners were skillful or lucky? In his book, The Intelligent Portfolio, Christopher L. Jones discusses methods of distinguishing skill from luck.
Jones chose the example of the Legg Mason Value Trust fund as an example. This fund beat the S&P 500 each and every year from 1991-2005 by an average of 7%! To get a handle on whether this is an unusual result, Jones uses simulations.
He simulated 10,000 funds with similar investing styles, but making random stock selections, and checked how they performed compared to Legg Mason. It turned out that about 1 out of 30 simulations beat Legg Mason. Based on this, Jones concluded that Legg Mason’s results were not unusual and could easily have simply been luck.
If you think you smell something fishy here, it’s because something is fishy. If 1 out of 30 simulated funds beat Legg Mason, then why didn’t any real funds beat Legg Mason? This seems like a very unusual result, and it made me suspicious of Jones’ analysis.
Unfortunately, Jones didn’t give much detail in how he did the simulations. One correct way to do the simulations is to have funds select stocks at random according to the same distribution as Legg Mason, and assign returns equal to the actual returns of those stocks in the relevant time period.
An incorrect way to do the simulations would be to randomly generate entirely new stock market histories. This would be answering the question “what are the odds that some 15-year period will produce better returns than Legg Mason produced?” This is entirely different from the question “what are the odds that a random stock picker could have done better than Legg Mason from 1991-2005?”
While I agree with Jones that strong returns could easily be just luck, I’d like to know more about how he did the Legg Mason analysis. If he got this wrong, then it casts doubt on the correctness of other analyses in his book.
I found it curious that Jones didn’t tackle the best long-term track record available: Berkshire Hathaway. Berkshire’s results have been so strong for so long that it seems inconceivable that it is just luck. But, I haven’t attempted to analyze this case.
Any time you have thousands of people picking stocks, it’s inevitable that a few will have outstanding results. How can we tell if the winners were skillful or lucky? In his book, The Intelligent Portfolio, Christopher L. Jones discusses methods of distinguishing skill from luck.
Jones chose the example of the Legg Mason Value Trust fund as an example. This fund beat the S&P 500 each and every year from 1991-2005 by an average of 7%! To get a handle on whether this is an unusual result, Jones uses simulations.
He simulated 10,000 funds with similar investing styles, but making random stock selections, and checked how they performed compared to Legg Mason. It turned out that about 1 out of 30 simulations beat Legg Mason. Based on this, Jones concluded that Legg Mason’s results were not unusual and could easily have simply been luck.
If you think you smell something fishy here, it’s because something is fishy. If 1 out of 30 simulated funds beat Legg Mason, then why didn’t any real funds beat Legg Mason? This seems like a very unusual result, and it made me suspicious of Jones’ analysis.
Unfortunately, Jones didn’t give much detail in how he did the simulations. One correct way to do the simulations is to have funds select stocks at random according to the same distribution as Legg Mason, and assign returns equal to the actual returns of those stocks in the relevant time period.
An incorrect way to do the simulations would be to randomly generate entirely new stock market histories. This would be answering the question “what are the odds that some 15-year period will produce better returns than Legg Mason produced?” This is entirely different from the question “what are the odds that a random stock picker could have done better than Legg Mason from 1991-2005?”
While I agree with Jones that strong returns could easily be just luck, I’d like to know more about how he did the Legg Mason analysis. If he got this wrong, then it casts doubt on the correctness of other analyses in his book.
I found it curious that Jones didn’t tackle the best long-term track record available: Berkshire Hathaway. Berkshire’s results have been so strong for so long that it seems inconceivable that it is just luck. But, I haven’t attempted to analyze this case.
1 in 30 sounds pretty high for that level of performance (if you're talking about completely random picking, wouldn't you expect the number who just beat the index to be cut in half each year?). But maybe the reason no real funds beat that record is because the managers are "smarter" than a random stock picker :)
ReplyDeleteIt would seem that doing that well for 14 years is a very long run of luck.
Richard: I didn't make it clear in the post, but the 1 in 30 applied to the overall return over the 15 years rather than the odds of beating the S&P 500 in every year. So, some of the simulations that beat Legg Mason over the 15 years may have underperformed the S&P 500 in one or more years.
ReplyDeleteYour intuition is right anyway, though. 1 in 30 is just too high for such outperformance. The Legg Mason results may yet be just luck, but I don't think Jones' analysis shed any light on this question.
7% is a huge outperformance - if 1 in 30 random funds can do that well, then there must be quite a few that can do at least 2% (which is still very good over that time period). Maybe this just supports the value of index funds rather than random stockpickers :)
ReplyDeleteRichard: You're right that 7% is a huge outperformance. But, I don't believe that 1 in 30 random funds can do better. I think Jones messed up his analysis.
ReplyDelete