Analysts at the U.S. Government Accountability Office are criticizing the stochastic modeling methods the Pension Benefit Guaranty Corp. has used to justify its new investment strategy.

Since 2004, the PBGC, the federal pension guaranty agency, has been acting on the advice of a consultant to decrease the share of its $68 billion in assets invested in U.S. bonds and debt securities to 42%, from 75%, and, for the first time, to put 38% of the portfolio in investments such as international stocks, emerging market stocks, real estate, private equity holdings and emergency market bonds.

The consultant was unable to provide the exact data it gave to the PBGC to analyze the sensitivity of the portfolio to changes in the underlying economic assumptions, and, when the PBGC conducted sensitivity analyses of its own in response to GAO criticism, the PBGC did not give a detailed description of the assumptions it used in the analyses, Barbara Bovbjerg and Thomas McCool, GAO directors, write in a letter about their views to the leaders of the Senate Finance Committee and the leaders of the Senate Health, Education, Labor and Pensions Committee.

“Our report emphasized that the quality of PBGC’s forecasts, which used stochastic modeling, depends on the technique used to model uncertain returns and on the assumed values of key parameters, including the distribution of returns, means, standard deviations, and correlations between assets,” Bovbjerg and McCool write. “Therefore, reasonable variation of these assumptions is needed to better inform the degree of uncertainty in the results.”

When GAO analysts came up with estimates of the assumptions, the PBGC consultant had used and tried substituting 6 different sets of assumptions. The GAO analysts found a higher level of portfolio risk using 5 of the 6 alternative assumption sets, the GAO officials write.

“Our analysis shows that the consultant’s measures of risk associated with fixed income are particularly sensitive to changes,” the officials write. “For example, the consultant set the assumption of market risk (as measured by the standard deviation) for long Treasury bonds at 11.2%. In contrast, other sources use a lower risk assumption, such as 7.62% in JP Morgan’s Capital Market assumptions or roughly 9.3% based on Ibbotson historical data. When the risk on high-quality corporate bonds and long Treasury bonds is lowered by just 2%, the new allocation becomes riskier than the previous allocation.”

The PBGC should consider whether the consultant and other advisors are providing enough variation in the economic assumptions to test how the new portfolio really would do under a wide range of conditions, Bovbjerg and McCool write.

Bovbjerg and McCool are recommending that the PBGC board improve the way it monitors investment policy and conduct more formal analyses of PBGC investment policy.

PBGC Director Charles Millard has written in a response that its sensitivity analyses show that a better diversified portfolio should be less risky and produce higher returns, in the long run, than a portfolio consisting solely of U.S. government bonds, U.S. corporate bonds and U.S. stocks.

The PBGC conducted a sensitivity analysis that included 5,000 economic scenarios; Millard writes.

“All calculations have indicated that the Sharpe ratio [risk/return indicator) of the new policy is supperior to that of the prior policy,” Millard writes.

Bovbjerg and McCool are emphasizing the limitations of the Sharpe ratio.

“The Sharpe ratios are based on assumptions about future asset returns and the volatility of those returns – unknown items given the uncertainty of future events,” Bovbjerg and McCool write. “An analysis of quantitative uncertainty does not fully account for real world uncertainty.”

In addition, a higher Sharpe ratio merely shows that an investment policy expected to provide a better risk-adjusted performance, Bovbjerg and McCool write.

“Better risk-adjusted performance should not be interpreted to imply that the new strategy is less risky than the previous policy,” Bovbjerg and McCool write.

HYPERLINK “http://www.gao.gov/new.items/d08667.pdf”A copy of the GAO report, which includes the responses from the PBGC and the PBGC’s board, is available here.