What Benchmarking Tells Investors
Benchmarking is a popular exercise for evaluating fund performance. But is this exercise helpful to investors? Does it provide the performance information investors need to assess their investments?
What is benchmarking?
Benchmarking is the popular exercise of comparing the returns of a fund, ETF, or separate account manager to those earned by an index, or a weighted basket of indexes, over a specific reporting period. Investors, both institutional and retail, are obsessed with doing so. But this obsession often results in confusion around what benchmarking does and doesn’t tell us about fund manager performance.
What’s an index?
An index is little more than a theoretical construct, a “paper portfolio” designed to provide insight into how specific types of stocks or bonds have performed over a given time horizon (either by the minute or over many decades). They can utilize any number of portfolio construction methodologies. For example, they can be either capitalization-weighted, equal-weighed, or weighted using some other esoteric methodology. There’s nothing special about any particular index that makes it necessarily a “better investment” than another. In fact, investors cannot invest directly in an index. This reality perhaps questions their ultimate utility as benchmarks in the first place since, unlike an index, funds must contend with real-world investing challenges like trading, liquidity, capacity constraints, fees, rebalancing, and cash flows. Indexes deal with none of these, all of which can weigh heavily on fund returns.
What does benchmarking tell us?
Benchmarking tells us several things. To overstate the obvious, benchmarking tells us whether or not a specific fund has outperformed a given index. More precisely, benchmarking tells us the degree to which a fund’s portfolio looked like the index, in terms of holdings and weights, over a specific reporting period. That’s about it.
What doesn’t benchmarking tell us?
It’s equally important to understand what benchmarking doesn’t tell us. Contrary to popular belief, it doesn’t tell us whether a fund is doing a “good” job. It also doesn’t tell us whether a fund has more or less risk than the benchmark. Further, it doesn’t tell us whether the index we’re using for our benchmarking is an appropriate comparison of the fund in question. Finally, it certainly doesn’t tell us whether we should buy, hold, or sell a given fund; nor does it tell us whether a past decision to buy, hold, or sell a fund was a wise one. Benchmarking doesn’t provide investors direction, just data; useful data, yes, but only if we know how to use it. To derive meaning from that data, investors need to investigate why a fund’s returns differed—either positively or negatively—from those of the index. Benchmarking provides investors results for a given reporting period; it says nothing about why we have the results that we do.
What it means to look different
Many fund managers build portfolios that purposely look different from popular indexes. They typically do so to manage risk or pursue the prospect of higher returns. For example, a value fund has a mandate to invest in undervalued stocks based on the empirical observation that, over time, such stocks have been shown to outperform both the broad market and more expensive growth stocks. Subsequently, we should fully expect a value fund’s portfolio to look materially different than, say, that of the S&P 500 Index. If value stocks as an asset class underperform the S&P 500 Index, we should fully expect all value funds to do likewise. In fact, it would be highly problematic if they didn’t for it would mean the fund manager did something other than what they were hired to do—invest in value stocks.
Value: What’s in a name?
So why not compare a value fund to a value index? Comparing a value fund to a value index is better than comparing a value fund to, say, the S&P 500 Index. However, rarely do two indexes—including value indexes—utilize the same approach to portfolio construction. For example, consider three value indexes: the Russell 1000 Value Index, the MSCI USA Enhanced Value Index, and the S&P 500 Value Index. All three use different portfolio construction methodologies that provide widely varying results. The Russell 1000 Value is constructed using book value and lower expected earnings growth rates; the S&P 500 Value Index is constructed using book value, earnings, and sales-to-price. The MSCI USA Enhanced Value Index is constructed using book value, forward earnings, and enterprise value-to-cash flow from operations. None of them even begin with the same universe of investable companies. The S&P 500 Value Index, for example, begins its construction process with a universe of companies half the size of Russell’s, resulting in a heavy large cap bias. Despite their differences, all three indexes are similar enough to be considered “value indexes”, yet different enough to deliver very different returns. For example, in 2019 the spread between the S&P 500 Value Index and Russell 1000 Value Index was a staggering 5.39%.
Exhibit 1: Returns of select value indexes, 2000-2019. Source: FactSet, Inc.
The point is that comparing a value fund to any specific value index tells us the degree to which the fund’s portfolio looked like that of the index during a given reporting period. It doesn’t tell us why the fund’s portfolio might’ve differed from that of the index. Naturally, we wouldn’t expect much differentiation if the fund in question was an index fund tasked with mirroring a specific index. But not all value funds are index funds and not all index funds track the same indexes. Many value funds, for example DFA’s US Large Cap Value III fund (DFUVX), measure value a bit differently from popular value indexes. DFA, for example, uses book value exclusively to measure value, which is different from how most value indexes measure value. Understanding why and how DFA arrived at their decision—and whether you think their decision is a wise one or not—is material to evaluating whether one should buy, hold, or sell the fund.
When it comes to benchmarking, the most important question isn’t whether a fund outperformed a given index. The more important question is why a fund’s portfolio differed from that of a comparable index during the reporting period and, subsequently, evaluating the quality of the manager’s decision-making processes.