In a recent paper published by the firm’s quantitative research team, the Morningstar Rating, also known as the “star rating”, is summarized and evaluated with respect to accuracy. Morningstar now publishes star ratings on more than 217,000 managed investments across 72 countries.

The rating system was introduced in 1985 to help investors and advisors better understand and assess the “crowded mutual fund landscape.” It evaluates the performance of a variety of funds (on a risk-adjusted basis and accounting for fees charged) for the period between January 2003 and December 2015 and assigns a rating of between 1 and 5 (lowest to highest performer).

The paper explains the evolution of the star rating system as follows:

“In 1985, investors had very little information available to assess a fund, so most could draw conclusions from only short-term performance, which was heavily marketed by fund companies. Information on sales loads was ignored, risk was not part of the conversation, and there was little focus on long-term returns. These are all facets that were embedded in the star rating’s formulas since day one.”

While the rating system was originally designed for a “simple analysis of a fund,” Morningstar’s paper explores whether the rating system could in fact serve as a predictor of future fund returns through analysis of; (1) Fama-MacBeth regression, and (2) an event study. In the first approach, the team analyzed whether higher-rated funds illustrated superior returns after controlling for expenses and other risk factors. The second approach involved evaluating what the “typical investor could expect to experience over a variety of holding periods (exclusive of fees and risk factors).

Using both frameworks, the Morningstar team found “relatively strong results that suggest the star rating had predictive power during the sample period.” Among equity funds, the study shows that the average three-year cumulative return was 27.74% for 5-star funds and 25.78% for 1-star funds.

While the paper advises against drawing “strong conclusions” from study results, it concludes that over the long term, “the event study indicates that the star rating has appeared to do more good than harm to investors during the period studied.”