By Jack M. Forehand (@practicalquant) —
The Wall Street Journal article last week that called into question the predictive value of Morningstar star ratings has caused a lot of debate in the investment industry. Many investors rely on Morningstar ratings to select funds and believe that selecting five star funds gives them the best chance of future outperformance. The Journal article shows that relying on those ratings to predict future performance is probably a bad idea. That really is not anything new to anyone who has studied fund performance because past performance has never really been predictive of future results, but it does provide an opportunity to discuss the issues surrounding using past performance in investment decisions.
The Morningstar rating rates funds based on three-year, five-year and ten-year risk-adjusted performance. If all three periods are not available, then the longest periods that are available for any given fund are used. The top 10% of funds are awarded the five star rating, the bottom 10% are awarded the one star rating, and the middle 80% are awarded ratings between two and four stars.
The first thing to understand about Morningstar ratings is that they are often used for two distinct purposes, and those purposes are sometimes in opposition to each other.
The first is to evaluate funds’ past performance and highlight the best performing funds. This is an important function and allows investors to see funds that have done well both recently and over time. Morningstar ratings work better for this purpose.
The second purpose is the more difficult one. This goal is to identify the funds that are likely to outperform going forward.
To Morningstar’s credit, they have always said that their star ratings have only moderate predictive power regarding future returns. Data shows that predicting funds that will outperform in the future using past returns is incredibly difficult, if not impossible. So creating a rating using past data for this purpose is a big challenge.
But if you were going to try to use past returns to predict the future, the way Morningstar is doing it is certainly not optimal in my opinion.
To illustrate this, let’s take a look at the three periods they are using.
Ten year performance can have some predictive power, since in many cases managers who perform well over ten years have performed well in different market environments. The true test of a manager is the ability to do well when markets are up and markets are down, and when different investing styles go in and out of favor. Many ten year periods include varying market environments like that, and those that do can be a fair way to evaluate a manager.
Three and five year performance, on the other hand, are a different story entirely. I have seen no studies that show that five year performance has any predictive power at all. The data I have seen regarding three year performance shows that if anything it is inversely correlated to future performance, meaning that the best performing funds over three years tend to underperform in the future and vice versa.
In 2013, Michael Mauboussin, managing director and head of Global Financial Strategies at Credit Suisse, published a paper titled, “The Success Equation: Untangling Skill and Luck in Business, Sports, and Investing. The image below, which is taken from Mauboussin, paper shows “the correlations between three-year alphas for mutual funds from 2005 to 2007 and three-year alphas for mutual funds from 2008 to 2010. The result is that there is a very low correlation from one period to the next, which is why it is so difficult to identify investment skill on a forward-looking basis (i.e., ex ante) using past (i.e., ex post) results. This approach is neither persistent nor predictive.” The very last sentence is the key, which is why I’ve emphasized it.
Source: http://www.cfapubs.org/doi/full/10.2469/cp.v30.n3.1
At Validea, we do a lot of performance evaluation for the factor-based models we run, and often look at whether past performance for these models can help predict future returns. I thought it might be interesting to take a look at the results of that research and see how we might change the Morningstar rating system if its only goal was to predict future performance.
Here are a few changes I think might help.
[1] Stop Rating Funds After Just Three Years of Performance
One of the big issues with the Morningstar rating in terms of predicting future performance in my opinion is the fact the many funds’ ratings are calculated with only three year performance. Having managed some mutual funds and ETFs, I can say that the date a fund reaches its three year track record and is awarded a Morningstar rating can be a very big day. If that rating is five stars, it can mean a significant boost in assets. The problem, of course, is that three year performance tends to be mean reverting, so awarding a five star rating to a fund based on three years of good performance is the exact opposite of what you would do if you wanted to select funds with the best chance to outperform in the future. Three year performance also doesn’t do a good job of performing the first function of the star rating (evaluating manager performance) because evaluating anyone’s performance after three years doesn’t really tell you anything about their skill as a money manager.
The problem with my suggestion, however, is that if you wait longer to award a star rating, then less funds will have one, which means it can’t be used at all to evaluate those funds. The chart below shows the number of new funds launched, as well as those that are merged or liquidated. Since 2012, there have been over 3,000 fund launches and 2,362 closures or mergers. If say, you waited five or even seven years to assign a rating, potentially thousands of funds would go unrated. On balance, I think that is better than providing a rating using a time period that is typically mean reverting, though.
Source: https://www.ici.org/pdf/2017_factbook.pdf
[2] Use Longer Term Periods
One of the things we have found when evaluating the performance of quantitative strategies is that the longer the period you use, the better (at least up to a point). The key is to evaluate a strategy over a period long enough to include a full market cycle. So you want the period you are looking at to include both bull and bear markets, and periods where value and growth were in favor, and periods where small and large-caps outperformed. As of today, if I was looking for a fair period to evaluate a strategy, I would probably start in 2000. The period from 2000 until now includes two bear markets, both of which were totally different, and two bull market periods. It also includes a period from 2000 to 2007 where value was in favor and a period from 2007 on where growth was dominant. There are also significant periods in that range where both large- and small-caps performed well. So by using that seventeen year period, you are getting a true feel for whether a strategy truly adds value, and not just whether it performs well in a certain type of market.
Morningstar handles this issue to an extent with their current star ratings by comparing funds to other funds in the same style box (i.e rating small-cap value funds against other small-cap value funds), but many funds move between boxes, and there are still significant differences within each style box in terms of the strategies used, so using the longest period you can for performance evaluation is probably better than using three, five and ten year performance.
The other issue with using three, five and ten year periods is two-fold. One, those periods put too much weight on recent years. This is because the three most recent years are in all three of them, so they count three times as much as the other years. And years four and five are in two of them, so they count twice as much. Second, when funds are in between periods, some years don’t count at all. For example, if a fund has a nine year track record, its star rating is based on its three and five year performance since it hasn’t made it to ten years. Years six through nine aren’t counted at all. And then when it reaches its ten year track record, those years are added back, which can make a major impact on the star rating. So the fund’s performance may not have changed much from year nine to year ten, but its star rating can since four other years are added back in.
The chart below shows the impact that each year has on the Morningstar rating. Since the first three years are counted in all three periods, they end up counting for more than 50% of the rating.
If you wanted the Morningstar rating to be as predictive as possible regarding future returns, I would get rid of the standard periods and evaluate each fund against their peers using the longest period possible.
The one caveat to this is that once the evaluation period gets too long, you run the risk that something fundamentally has changed in the market, or the fund has lost its edge. For example, if you evaluate Warren Buffett’s performance history and split it in two, the annualized performance of the first half is significantly higher than the second half. The second half is still very good, but the first half is off the charts good. When evaluating that track record going forward, it would probably be fairer to expect long-term relative returns similar to the second half rather than the first. So for funds, it would probably be best to cap the evaluation period at something like 20 years.
[3] Incorporate Mean Reversion
To the extent we have been able to use historical performance to predict future results with our models, there have typically been two conditions present.
- The strategy has good long-term performance
- The strategy has struggled in recent years.
When those two things come together, we have found that adding exposure to the strategy boosts future performance in most cases. The reason is that all strategies have bad periods. The longer those bad periods are, the more likely it gets that you will get mean reversion and the strategy will begin to outperform again. So in our system, when we have a strategy that has a long-term record of beating the market, but has struggled over the three year period, we will typically add exposure to it.
Studies have shown that mutual fund returns also mean revert like that. If you were developing a star rating to predict future performance, it may make sense to have it be inversely affected by three year performance. When a strategy has a great long-term record, but has been out of favor in recent years, you would increase the star rating, rather than reducing it.
This would be counterintuitive to what most people believe, and fund managers would never accept having their star rating fall during periods of hot performance, so this would obviously never happen. But if you wanted to use past performance data to try to predict future results, I would argue that it should.
Can Past Performance Predict Future Results?
On balance, past performance is not a good predictor of future results, and even with these changes, the Morningstar rating would still be subject to that rule. But by relying on the core principles above, if your only goal with the Morningstar rating was to predict future results, I think it would become more predictive than it is today.
At the end of the day, none of these changes will ever happen, but I think the discussion is interesting to have regardless of that because it gets at the major issues that exist with using past performance to predict the future. If this discussion helps investors to understand those issues, the whole debate will have been a very useful one, even if nothing changes with the Morningstar rating.
—–
Jack Forehand is Co-Founder and President at Validea Capital. He is also a partner at Validea.com and co-authored “The Guru Investor: How to Beat the Market Using History’s Best Investment Strategies”. Jack holds the Chartered Financial Analyst designation from the CFA Institute. Follow him on Twitter at @practicalquant.
