Click here to view this study and its figures as an Adobe Acrobat PDF.

The assets managed by hedge funds have grown dramatically in recent years. At the end of 2005, more than $1 trillion U.S. dollars were invested in hedge funds around the globe.[1] They have been marketed as excellent diversifiers for all-equity portfolios because they provide generous returns in all stock market environments. These funds employ substantial leverage, so they play a far more important role in global securities markets than the size of their net assets indicates. Market makers on the floor of the New York Stock Exchange have estimated that during 2004, trades by hedge funds often accounted for more than half of the total daily number of shares changing hands. Moreover, investments in hedge funds have become an important part of the asset mix of institutions and even wealthy individual investors.

Despite their large global presence, hedge funds remain relatively free from many regulatory requirements of the Securities and Exchange Commission and other regulatory bodies. Unlike mutual funds, they are not required to report performance data. In fact, the availability of hedge fund performance data from data-gathering services such as TASS?[2] depends on the funds’ willingness to supply them. In our research, we examined the reasonably comprehensive TASS database of hedge fund returns and estimated the magnitude of two substantial biases that can influence measures of hedge fund performance in the data series. We found that these biases may be far greater than has been estimated in previous studies.

In addition, we examined funds that reported their performance to the TASS database at some point in time and then chose to cease reporting. We explored the funds’ likely reasons for ceasing to report, and we contrasted the characteristics of these funds with those that continued to report their performance. Our analysis shows that the non-reporting funds experience poor performance in the months leading up to the point when they cease to report. This suggests that funds cease to report their performance to data-gathering services primarily because they fail. We also observed that better-performing funds, larger funds, and funds that outperformed their peers were less likely to fail.

The putative argument for investing in hedge funds is straightforward: With higher rates of return and lower risk, they appear to be more attractive than the general stock market. Supposedly, it is the lack of correlation between returns from hedge funds and stocks in general that makes hedge funds good diversifiers. But we are not convinced. The empirical evidence in our research suggests that hedge funds are far riskier and provide lower returns than is commonly supposed. Although some hedge funds have provided generous returns, investors face the risk of buying poorly performing funds or, worse, failing ones.

Biases in Reported Hedge Fund Returns

Several biases arise in the published data of hedge fund returns. Here we describe and provide measures of the two most significant ones.

Backfill Bias. Managers often establish hedge funds with seed capital and begin reporting their results at some later date and only if the initial results are favorable. Moreover, the most favorable of the early results are then backfilled into the database along with reports of contemporaneous results. This bias is often called incubation bias. Our measure of backfill bias includes a second source of bias: if a fund previously reported to another database and then began reporting to TASS, it may not have reported all the previous data that were contemporaneously included in another database.[3] Instead, the fund may have reported to TASS only the data it wanted potential investors to see. This second type of bias is not incubation bias, as the term is commonly used, but is a type of backfill bias that can influence all databases. Fortunately, TASS indicates when the hedge fund began reporting.

We examined backfilled hedge fund returns and compared them with those reported contemporaneously. Panel A of Table 1 shows that in the early years, especially 1994 and 1995, the vast majority of the reported returns were backfilled. Only in 2002 and later did the number of non-backfilled returns exceed the number of backfilled returns. Table 1 shows that backfilled returns tended to be substantially higher than the contemporaneously reported ones, particularly in the early years.4 On average, the backfilled returns were more than 700 basis points higher than the contemporaneously reported returns. Simply put, the use of backfilled returns data to judge the performance of hedge fund management significantly biases the results upward.

Survivorship Bias. Another important bias in the published hedge fund returns data is survivorship bias. Databases available at any point in time tend to reflect the returns earned by then-existing hedge funds. They do not include the returns from hedge funds that no longer exist or which exist but no longer report their results. As we show below, unsuccessful hedge funds do not tend to survive, leaving mostly successful funds in the database.

In order to examine this phenomenon, we obtained from TASS all the past records of funds that are dead (or for any other reason have stopped reporting) as of April 2004. We refer to these as defunct funds. We classify funds that continued to report in 2004 as live funds. We compared the returns from live and defunct funds and performed the analysis without any backfilled data, which we have shown is substantially upwardly biased.

Panel B of Table 1 shows a substantial difference each year between the returns of live and defunct hedge funds.[5] It is reasonable to assume that the performance of all hedge funds (live and defunct) is the best reflection of the performance of the hedge fund industry as a whole. We see from Panel B that the average return of the surviving funds was 13.74 percent over the 1996-2003 period. The average return for all funds was only 9.32 percent, for a difference of 442 basis points.[6]

We also estimated survivorship bias by hedge fund category. We found substantial differences between live and defunct funds in all categories. Of particular note is that we found substantial survivorship bias in the fund-of-funds category. This contradicts the claim of Lamm (2003) that survivorship bias in this category is relatively small.

Our estimates of survivorship bias are considerably larger than those found by other investigators.[7] We believe there are several reasons for this. First, other investigators have used different databases from those we employed. Moreover, we estimated survivorship bias using only contemporaneously reported data rather than both contemporaneous and backfilled data. In addition, we used a more recent period than other investigators did, and our sample size was substantially larger. Finally, because data on defunct funds were not easily available from the data-gathering services, we were particularly careful to obtain from TASS the data on all hedge funds that stopped reporting during the period studied.[8]

In short, the attrition that characterizes the hedge fund universe yields considerable survivorship bias in the returns of indexes composed of only existing funds. Investors should be aware that substantial differences exist between the average returns of existing funds and of defunct funds. To judge the true performance of a hedge fund category, one must look at the past average returns of all funds, not just of the existing ones in that category.

Persistence in Hedge Fund Returns

Financial consultants usually examine the past investment returns for hedge fund managers in the belief that past investment success is a good predictor of future success. We tested this hypothesis by asking if winners tend to repeat their success in the subsequent year. We call a winner a hedge fund that realizes a return larger than the median hedge fund return of all funds for that year. A loser has a return below the median. We found, for example, about 61 percent of the winners from 1995 were winners in 1996. But in 2003, this repeat figure was only 40 percent. Over the entire 1996-2003 period, only 52 percent of funds were repeat winners. So over the course of time that we studied, the probability of observing repeat hedge fund winners was roughly 50-50.

The high attrition rates for hedge funds and lack of persistence in their returns underscore an aspect of risk not often examined. The top-performing hedge funds may yield very great rewards, but the risk of selecting a poorly performing or failing one is similarly great.

Why Do Hedge Funds Stop Reporting Their Performance?

The question of why hedge funds fail or cease reporting to a database is an entire area of inquiry on its own. As noted, available hedge fund performance data from data-gathering services such as TASS depend on hedge funds’ willingness to supply them. Therefore, comparing the characteristics of funds that choose to report and those that do not has the potential to yield insights. However, an empirical analysis is not possible because of the obvious absence of data on funds that choose not to report. Instead, in our study, we examined the funds that reported their performance to the TASS database at some point in time and then ceased reporting. We explored the funds’ likely reasons for ceasing to report and contrasted the characteristics of these funds with those of funds that continued to report their performance.

The financial literature presents two competing explanations for why some hedge funds stop reporting their performance data. Some researchers have contended that funds stop reporting when they underperform other funds,[9] asserting that funds stop reporting because they fail. However, cessation of performance reporting can also result from reasons unrelated to performance, including mergers and name changes. Other researchers have offered a competing explanation: well-performing funds do not have an incentive to continue reporting their performance because they do not need to attract additional capital.[10]

Examining the degree of survivorship bias in average hedge fund-performance estimates lends insight into which of these hypotheses is true. If it were true that the performance of funds that stop reporting is comparable to that of the funds that continue to report, then the average returns of the two groups would be very similar, and, as a result, there would be no evidence of survivorship bias. The fact that survivorship bias estimates are positive suggests, albeit indirectly, that funds that cease to report are generally worse performers than those that continue to report.

We compared the performance of exiting funds in the final three, six, and nine months before they exited the TASS database with their prior performance (see Table 2). Our findings indicated a markedly worse performance in the period immediately before the funds stopped reporting. For example, in the final six months, the exiting funds’ average monthly return was -0.56 percent, compared to an average monthly return of 0.65 percent during their reporting lives (excluding the final six months). In the last three months, the average returns fell to -0.61 percent, compared to an average monthly return of 0.49 percent during their reporting lives prior to the last three months.

The unambiguous pattern of declining performance during funds’ final months of reporting suggests that, on average, it is poor performance, and very likely failure, that explains hedge funds’ cessation of reporting to the TASS database. The hypothesis that successful funds stop reporting because they do not want to attract additional capital is not supported by the data.

Hedge Funds’ Time to Failure

Using modern quantitative techniques, such as survival analysis,[11] we found the probability of a fund’s failure increased rapidly during the first three years of its existence and reached its maximum at the fourth year. After the fourth year, the probability of failure declined, but only very gradually. Even after a fund has existed for 10 years, its likelihood of failure falls minimally. Indeed, the likelihood of failure of a 10-year-old hedge fund is not significantly different from the likelihood of failure of a four-year-old fund. This suggests that the risk of failure for hedge funds remains relatively high, even after a fairly long existence.

One reason the risk of failure remains high is the existence of so-called high-water marks. Suppose a hedge fund has enjoyed a strong long-run performance but then suffers a sharp loss in net asset value in a single year. The fund manager will not only fail to earn an incentive fee (usually about 20 percent of any profits) during that poor year, but will also be less likely to earn an incentive in the following years. This is so because the incentive will be earned only if the net asset value exceeds the previous high net asset value. Thus, the manager may prefer to close the fund and open a new fund that is not burdened by a high-water mark that will limit his incentive compensation. This will be especially true if assets under management are relatively small.

We also examined the factors that affect hedge funds’ failure rates. In particular, we analyzed how hedge funds’ failure rates related to fund characteristics such as size, performance, and investment style. We found that better-performing funds, larger funds, and funds that outperformed their peers have a lower likelihood of failure.

Conclusion

The findings described here have important implications for investors. We show that the practice of voluntary reporting (and backfilling only favorable past results) can cause returns calculated from hedge fund databases to be biased upward. Moreover, the substantial attrition that characterizes the hedge fund industry results in substantial survivorship bias in the returns of indexes composed of only existing funds. Correcting for such biases, we found that hedge funds have lower returns than is commonly supposed.

Although some hedge funds provide generous returns, we show that investors face a high risk of buying a poorly performing or failing fund. Moreover, because failure rates remain high, even for longstanding funds, this risk cannot be mitigated by restricting one’s purchases to funds with a long record of past success.

The policy implications of our research findings are clear. We believe much of the problem of backfill bias and survivorship bias would disappear if hedge funds were required to report their performance, contemporaneously and accurately, to a single data-gathering service. This would allow individuals and institutions to make better-informed investment decisions based on comparative analysis of performance data that is free from biases. Accurate information about hedge fund performance would not only benefit potential investors, but would also allow the better-performing funds to distinguish themselves from the rest of the pack.

Atanu Saha is a managing principal in the New York office of Analysis Group. He specializes in applying economics and finance to complex business litigation. He has served as an expert witness in a wide range of areas and has provided expert testimony in matters involving valuation of securities, analysis of risk-return characteristics of investment portfolios, and economic damages. Mr. Saha has published numerous peer-reviewed articles in economics and finance journals. Mr. Saha was formerly on the faculty at Texas A&M University.

References

Ackermann, Carl, McEnally, Richard and Ravenscraft, David. “The Performance of Hedge Funds: Risk, Return and Incentives,” Journal of Finance, June 1999, 54:3, 833-874.

Brown, Stephen, J., Goetzmann, William and Ibbotson, Roger, G. “Offshore Hedge Funds: Survival and Performance, 1989-1995,” Journal of Business, 1999, 72:1, 91-117.

Brown, Stephen, J., Goetzmann, William and Park, James. “Careers and Survival: Competition and Risk in the Hedge Fund and CTA Industry,” Journal of Finance, 2001, 56:5.

Fung, William and Hsieh, David. “A Primer on Hedge Funds,” Journal of Empirical Finance, 1999, 6, 309-331.

Fung, William and Hsieh, David. “The Risk in Hedge Fund Strategies: Theory and Evidence from Trend Followers,” Review of Financial Studies, Summer 2001, 14:2, 313-341.

Grecu, Alex, Malkiel, Burton G., and Saha, Atanu. “Why Do Hedge Funds Stop Reporting Their Performance?” Working Paper.

Kiefer, Nicholas M. “Economic Duration Data and Hazard Functions,” Journal of Economic Literature, Vol. XXVI, June 1988, 646-679.

Lamm Jr., R. McFall. “Why Not 100% Hedge Funds? Still a Viable Approach After a Half Decade,” Deutsche Bank, November 10, 2003.

Lancaster, Tony. The Econometric Analysis of Transition Data, Cambridge University Press, 1990.

Liang, Bing. “Hedge Funds: The Living and the Dead,” Journal of Financial and Quantitative Analysis, September 2000, 35, 309-326.

Liang, Bing. “Hedge Fund Performance: 1990-1999,” Financial Analysts Journal, January/February 2001, 57, 11-18.

Malkiel, Burton G., and Saha, Atanu, “Caveat Emptor,” Wall Street Journal, July 26, 2005.

Malkiel, Burton G., and Saha, Atanu, “Hedge Funds: Risk and Return,” Financial Analysts Journal, Vol. 61, No. 6, 2005, 80-88.

Notes

1. According to Van Hedge Fund Advisors International.
2. TASS, a unit of Tremont Capital Management, covers between one third and one half of the total number of hedge funds in existence. It is, of course, possible that the funds not included in the TASS database have characteristics different from those reported in TASS.
3. The TASS service was purchased by Tremont in March 1999. At that time, Tremont persuaded many of those hedge funds that reported to Tremont and other database services to begin reporting to TASS. As a result, during the early 2000s, TASS became one of the most comprehensive services to cover all varieties of funds, and we find it to be broadly representative of the hedge fund universe.
4. When only partial-year data were available, we filled in the missing partial years by assuming that the fund earned the monthly average of all reporting hedge funds during the missing month. Thus, if we had data available from March through December, we used the average hedge fund return from January and February to calculate an annual return for that fund.
5. The analysis excludes data for 1994 and 1995 because almost all of these data were backfilled rather than contemporaneously reported.
6. The averages above are calculated by weighting equally the returns of all funds. Data on asset size were not available for all funds. We also found a consistent relationship between a fund’s decision to report assets and fund performance; in particular, non-asset reporting funds tended to be underperforming.
7. Measuring the bias as the difference between the returns of all hedge funds and of surviving funds only, we found a bias averaging 442 basis points. Estimates of survivor bias by Brown, Goetzmann, and Ibbotson (1999); Brown, Goetzmann, and Park (2001); Liang (2000, 2001); and Fung and Hsieh (1999, 2001) ranged from 60 basis points to 360 basis points per year for various hedge fund types. In a study covering a period prior to ours, Ackermann, McEnally, and Ravenscraft (1999) found small and insignificant estimates of survivorship bias.
8. A third bias can be called end-of-life bias. Hedge funds generally stop reporting poor results during the last months of their existence. Data are not available to estimate this bias, so even our adjusted return data may be biased upward.
9. See, for example, Malkiel and Saha, 2005.
10. For example, Ackermann, McEnally, and Ravenscraft (1999) contend, “Excluding disappearing funds has virtually no impact on our assessment on overall performance. This self-selection bias has two interesting implications for hedge funds research. First, some hedge funds may not actively seek new money because there may be diminishing returns to their arbitrage strategies. Second, some of the best hedge fund managers may be opting out of the databases.”
11. Survival analysis has been used to study duration phenomena in many fields. Kiefer (1988) and Lancaster (1990) provide excellent reviews and numerous other examples.

 Click here to view this study and its figures as an Adobe Acrobat PDF.