Misleading Stock-Picker Performance Statistics
Lately I’ve seen several ads and web sites touting the records of different stock-pickers and their systems. One thing the proud claims had in common was that they didn’t count all of the picks for reporting performance figures. This can give misleading results as I’ll show with an example.
Two stock pickers, Amy and Bill, made one pick each day for the past 10 years. They held their picks for the day and moved the resulting money to another pick the next trading day. After examining their records, we find the following results:
Amy: 99% of her picks resulted in a cumulative outperformance of the S&P TSX index by 327%!
Bill: 99% of his picks resulted in a cumulative underperformance of the S&P TSX index by 70%.
It seems apparent that Amy is a great stock picker and Bill should find another job. Amy’s record appears so strong over 10 years that she could easily sell access to her picks through a newsletter.
If you’re suspecting some sort of catch, you’re right. Both Amy and Bill just bought the S&P TSX index and left their money there for the full 10 years. The way that I reported their results was highly misleading, but accurate.
After eliminating the worst 25 days, the returns really are 327% higher. Of course, Amy just matched the index, but her best 99% of days led to huge outperformance. Similarly, Bill matched the index too, but after eliminating his best 25 days, his results look very poor.
To be fair, none of the ads and web sites I saw abused the statistics this badly. If they had, then the reported results would have seemed too incredible. Instead they abused the statistics just enough to make the results seem very good, but still believable.
Two stock pickers, Amy and Bill, made one pick each day for the past 10 years. They held their picks for the day and moved the resulting money to another pick the next trading day. After examining their records, we find the following results:
Amy: 99% of her picks resulted in a cumulative outperformance of the S&P TSX index by 327%!
Bill: 99% of his picks resulted in a cumulative underperformance of the S&P TSX index by 70%.
It seems apparent that Amy is a great stock picker and Bill should find another job. Amy’s record appears so strong over 10 years that she could easily sell access to her picks through a newsletter.
If you’re suspecting some sort of catch, you’re right. Both Amy and Bill just bought the S&P TSX index and left their money there for the full 10 years. The way that I reported their results was highly misleading, but accurate.
After eliminating the worst 25 days, the returns really are 327% higher. Of course, Amy just matched the index, but her best 99% of days led to huge outperformance. Similarly, Bill matched the index too, but after eliminating his best 25 days, his results look very poor.
To be fair, none of the ads and web sites I saw abused the statistics this badly. If they had, then the reported results would have seemed too incredible. Instead they abused the statistics just enough to make the results seem very good, but still believable.
I have seen similarly distorted stats, but I didn't realize how easy it was to get such disparate return calculations.
ReplyDeleteA friend told me he was looking at his mutual fund statement and could see the return the fund calculated far outstripped his results. Surprised, he called the fund company for an explanation. He said the explanation was so ridiculous, it disgusted him. I wish I knew what the methodology was.
Here's an interesting article addressing similar issues. Might be worth a read, or at least a skim:
http://www.canadianfundwatch.com/modules.php?name=News&file=article&sid=41
Gene: I like the point made by Bogle at the link you cited about mutual fund reporting dollar-weighted returns. This would take care of return distortion from fund incubation.
ReplyDeleteI don't believe in stock pickers. I see plenty of advertising but ignore them all. I believe over the long term the best way to go is to track the index. It's ETF's and and index linkers whenever possible for me ensuring I keep fees to a minimum.
ReplyDeleteRetirement Investing Today: I think your approach makes a lot of sense.
ReplyDelete