As a crew rearranges Nobel Memorial Prize-winning economist Bill Sharpe’s living room to film a recent interview, Sharpe motions me over to his iMac to show what he’s been working on. He’s trying to solve the retirement income problem at his home in Carmel, California.

In the 1960s, Sharpe rocked the world of finance by bringing order and logic to a business that continues to be driven by emotion and stories. After being invited to sit on a local charitable foundation board, Sharpe listened to a pitch by an investment manager describing the various complex investment strategies the (unfortunate) manager employed in creating the portfolio.

Sharpe asked him if his sophisticated portfolio had outperformed a risk-adjusted market index. The charity switched to index funds.

At 84, Sharpe still comes up with simple truths that can rock even a veteran’s understanding of retirement planning. Consider the popular period-certain option with an income annuity.

(Related: Finke: Why Annuities (and Free Steak Dinners) Deserve a Second Chance)

According to Sharpe, choosing a 20-year period-certain option is just like having a bond ladder with a deferred annuity tacked on the end. If building a bond ladder for 20 years is cheaper net of fees, just create a bond ladder and buy a deferred annuity. Why didn’t I think of that?

As we sit in front of Sharpe’s computer, he shows me a new program he’s been working on to simulate variable-annuity income streams. Using a random return generator and programming based on the annuity contract features, he simulates what happens to income when market returns rise and fall during retirement. The computer program spits out a series of squiggly lines that represent possible income paths.

The lines are all over the place. Some spike early in retirement and result in a high income that decreases gradually over time with inflation. Some fall flat early in retirement, presumably leaving the simulated retirees with a disappointing lifestyle with fewer vacations and fancy dinners. Each line is a hand that the retiree is dealt when they accept investment risk within the rules of a financial product.

The squiggly lines are Sharpe’s way of visualizing the trade-offs that all retirees face when turning their nest egg into a lifestyle in retirement. All of us want a high and stable income in retirement. But we can’t have both.

We can either have an income that’s low and stable, or an income that’s risky, unstable and possibly higher. Sharpe wants us to understand how to choose the right income path, and which mix of investments and products provides the most optimal results for the amount of risk taken.

Balancing Income and Risk

To Sharpe, the role of a financial advisor is to understand the range of options all clients face in order to build an income that efficiently balances risk and return. The advisor needs to understand these tradeoffs, and he or she needs to be able to explain them to a client. He’s even written a book that’s available on his Stanford website to help advisors “focus more on communicating possible outcomes and helping the clients understand the options.” (To read “Retirement Income Analysis with Scenario Matrices,” go to http://web.stanford.edu/~wfsharpe.)

Many advisors view retirement income through what Sharpe calls the “constant spending” lens of stable spending, using a portfolio of stocks and bonds. How should an advisor judge the efficiency of the conventional “4% rule” approach? The same way we judge all income strategies. How high and how squiggly is the income line?

(Related: What’s the Point of Investing?)

Sharpe’s critique of the 4% rule is that the methodology is supposed to safely provide a straight line of after-inflation income using a portfolio of random returns over an unknown lifetime. But when you don’t know what asset returns will look like and how long you’ll live, a retiree has two lifestyle choices: (1) live well and risk running out of money, or (2) be conservative and risk giving too much to your heirs.

If the retiree doesn’t have a strong bequest motive then most of the nest egg should be used to fund spending. A strategy isn’t very efficient if a big chunk of the money ends up unspent.

Running hundreds of simulated retirements with random longevity and asset returns allows Sharpe to calculate the present value of where the money goes. Using a 3% withdrawal rule means that more than 30% of the initial portfolio doesn’t get spent. A 30% portfolio efficiency loss means that a retiree will need to save significantly more money to create a lifestyle. Even a 4% strategy leaves one-sixth of the nest egg unspent.

Why not increase spending to 5%? Raising the income line from 4% to 5% of initial wealth means that a significantly larger percentage of simulations will lead to the client running out of money before they die. Unlike the variable-annuity simulation in which the worst-case scenario is a low lifetime income and real purchasing power that decreases with inflation, an unlucky retiree using a fixed withdrawal rate will see their income line starting high, remaining flat in real terms and ending in complete ruin.

The tradeoff that an advisor needs to help a client understand when deciding on a fixed withdrawal strategy is between running out of money too early and leaving too much on that table. The sudden loss of lifestyle if markets don’t cooperate will lead many to choose a lower withdrawal rate, but the consequence is a loss of efficiency.

Fees and Flexibility

Sharpe doesn’t shy away from the controversial topic of investment management fees. A DIY investor who pays only 10 basis points on index funds will pay the present value equivalent of just 1.3% of their nest egg in fees. At a still modest 100 basis points of total investment expenses, the present value share of a retiree’s nest egg consumed by fees is a whopping 11.3%.

Asset fees of 1% may not sound like much, but it ultimately eats up one-tenth of a retiree’s expected lifestyle. This is an important point to make when comparing the expenses of a fixed withdrawal rate to product strategies.

The good news for the investments-only retirement income approach is that being more flexible about adjusting lifestyle over time in response to market risk can significantly improve efficiency. This makes far more sense than creating a strategy at the beginning of retirement with a 90% probability of success, and then failing to adjust lifestyle downward when the probability of success falls after a bear market.

A flexible strategy assumes that the retiree keeps an eye on performance and expected longevity and adjusts accordingly. Nonetheless, in the absence of sharing mortality risk with other retirees, at least one-fifth of the portfolio will remain unspent.

Income Annuities

Like most economists, Sharpe is surprisingly sanguine about the efficiency of income annuities. If you want to minimize the “waste” of unspent retirement funds while protecting against the risk of running out of income, annuities are the best option.

(Related: Time for RIAs to Rethink Insurance)

What may be surprising to some is Sharpe’s interest in the complex mechanics of variable annuities with guaranteed lifetime withdrawal benefits (GLWB). He devotes an entire chapter of his latest book to analyses of these complex and oft-maligned retirement products.

Why should a retiree annuitize? Sharpe provides a simple example using Sue, a healthy 65-year-old retiree. Sue wants to set aside about $10,000 in bonds today to fund $20,000 of spending in 20 years. There’s a 70% chance that Sue will be alive in 20 years.

Why not get together with a large group of other 65-year-old female retirees and pool savings to fund the spending of the 70% who will still be alive in 20 years? Instead of saving $10,000, Sue can save $7,000.

The math gets even more compelling when Sue wants to fund spending in 30 years. Instead of setting aside $7,000 today, she can pool $2,400 with other retirees who collectively have a 34% chance of being alive. It’s a lot cheaper to fund the same income by pooling savings with other retirees and doling it out to whoever is still around in old age.

Despite their theoretical efficiency, Sharpe doesn’t let income annuities off the hook as an ideal source of safe retirement income. While they protect against longevity risk, annuities that provide a fixed nominal income are particularly susceptible to erosion in purchasing power late in retirement. The risk of annuities that are not inflation-protected is the drop in real spending from living too long and/or experiencing an inflationary period such as the late 1970s.

Sharpe notes that annuities that provide inflation protection are rare (outside of Social Security), and retirees may balk at the reduction in initial lifestyle required when an annuity pays a rising nominal income. Another caveat is that this decline in purchasing power may not be such a big deal if, as consumer spending data suggest, retiree spending needs decline in retirement.

Remember the 11% in expenses that are eaten up by a 1% AUM fee using the 4% rule? Sharpe tries to back out the expenses on simple annuity products by using mortality tables and current bond yields but finds no evidence that insurance companies are earning anything on the sale of these products.

This may sound impossible, but my co-researchers David Blanchett, Wade Pfau and I found the same thing when we tried to back out expense loads from annuity quotes using today’s bond rates. Sharpe’s explanation is that insurance companies typically invest a small percentage of reserves in higher-risk, higher-yield assets to earn a profit.

Either way, it’s hard to make the argument that simple income annuities are expensive. They may in fact be too cheap.

Of course, a downside of income annuities is that they invest in bonds resulting in a safer, but lower, expected income. What if an investor was willing to take a little more risk to increase the possibility of higher income while still avoiding the risk of running out of income late in life?

Variable Products

This the promise of the variable annuity with a somewhat unusual structure that provides a unique range of income paths in retirement. But how efficient is a variable annuity with a guaranteed lifetime withdrawal benefit rider? Sharpe uses a variable product available through Vanguard in his simulations, but the general results translate to other similar products.

One of the quirks of GLWB riders on VA policies is that they are what Sharpe terms “convenient but rather crude” where “simplicity seems to have outweighed actuarial imperatives.” The product being studied provides the same guaranteed withdrawal percentage within a wide range of ages.

This benefit is more valuable if you buy the annuity when you’re at the lower end of that range. For example, the product being studied provides a 4.5% guaranteed payout on a joint life GLWB for a couple who buys the product between ages 65 and 79.

For example, a VA policy with a GLWB rider will have fees roughly equal to the total present value of fees on a managed investment portfolio (about 11%) if a couple buys the product at age 79. However, if the same couple bought the product at 65, the youngest age of the 4.5% rate guarantee, the net present value of the rider fees would be negative 3%.

This is a head scratcher. Sharpe figures that the insurance company expects enough VA owners to withdraw more than the 4.5% minimum to reduce the value of the guarantee, but it also highlights the importance of understanding how insurance contract features affect value.

Running a handful of scenarios provides a good example of how these products work. The couple invests $100,000 at age 65 with a 4.5% joint GLWB.

In the worst scenario, markets fall early in retirement, and the couple simply lives on the $4,500 per year. In another scenario, income rises from $4,500 to $5,000 at year 8 and remains at this amount. In yet another scenario, markets rise slightly early in retirement, providing a ratchet up to about $6,000 in year 3 and never rises again. In one lucky scenario, the couple experiences a bull market early in retirement, and the income rises to $13,000 per year after 14 years.

Of course, the 4.5% income guarantee is not like the 4% rule, since it does not provide a stable after-inflation income throughout retirement. As an example, one simulated retiree got somewhat lucky as their income grew to just over $5,000 by year 5, but they also lived long enough to see the real value of their after-inflation income fall below $3,000. GLWB riders provide some random upside to lucky retirees and guaranteed income for life, but they don’t take away purchasing power risk for a long-lived retiree.

Sharpe’s conclusion is that the peculiar income ratcheting characteristic of GLWBs means that they are not any more efficient than the fixed withdrawal strategy. Many simulations result in low incomes that decline in purchasing power. But this isn’t because of greedy insurance companies.

On average, the present value of fees on GLWB VAs is only 3.3% of the initial value invested. And only 5.5% goes unspent to the couple’s estate, meaning that more of the initial nest egg ends up being spent. Unfortunately, the downside of the product structure is that the range of income paths might not provide the highest expected welfare to the couple who aren’t willing to accept the wide range of lifestyles offered by the GLWB.

Maximizing Retirement Efficiency

How does a retiree choose the right balance of lifestyle and risk? What combination of investments and insurance products provides the best solution? How does a retiree eliminate the risk of ruin that comes with an investments-only fixed withdrawal approach, protect against inflation, and take an appropriate amount of investment risk?

One approach is to place the majority of retirement savings in an income floor. The floor should optimally be constructed by purchasing an inflation-protected annuity (a rare beast) or through a ladder of Treasury inflation-protected securities, known as TIPS, in which a portion of value at maturity can be spent and the remainder can be used to purchase a ladder of annuitized nominal income. For example, a retiree can invest in 20-year TIPS that will grow by 1% above inflation at today’s rates that at maturity are used to purchase an annuity whose value will not have been eroded by inflation over time.

The remainder could be invested in a leveraged equity ETF. If the leveraged equity portfolio rises beyond the initial allocation to risky assets (say 15%), then the surplus can be used to add income to the riskless floor. If the leveraged strategy fails, the retiree will simply live on the income floor. An unleveraged portfolio would allocate a higher percentage to equities but follow the same approach of gradually building the guaranteed income floor over time.

This strategy may be complex for retirees and even advisors to implement. Ideally, innovation in financial products will allow advisors to build a retirement income investment plan using more efficient products that provide the greatest income security with the right amount of upside potential.

It took decades for Sharpe’s vision of a low-cost internationally diversified market portfolio to gain wide acceptance. Hopefully, it won’t take decades for the industry to build a more efficient and simple retirement income solution.

Michael Finke is Chief Academic Officer of The American College of Financial Services and a regular contributor to Investment Advisor.

Bill Sharpe: Courtesy of The American College of Financial Services