Many seniors face potential liabilities from the taxman when it comes to their Social Security benefits. An average middle-class couple can expect between $22,000 and $30,000 in combined annual Social Security payments if they wait to collect until full retirement age (66 and 67). The Social Security Administration estimates that benefits only pay about 40 percent of projected retirement expenses, meaning that the couple would need another $45,000 per year to pay for everything from health care to a round of golf. In today’s retirement reality with the majority of seniors lacking savings and pensions, much, if not all, of the extra income would result from work.

Enter the taxman. The federal taxes for our couple would be $10,800 to $12,000 per year in the 15 percent tax bracket, and $25,500 of the $30,000 of Social Security benefits would be taxed. The retiree faces the double whammy of paying federal income taxes on the maximum taxable amount of 85 percent of Social Security benefits while continuing to pay employment taxes into the system. The couple also has a net after-tax cash flow of $63,000, leaving a shortfall in the 60 percent of extra income they need.

How did this happen? Our couple’s modified adjusted gross income was high enough to exceed the current threshold of $44,000 for a married couple filing jointly, subjecting 85 percent of their Social Security to federal taxes. The threshold is $34,000 for an individual. If our couple’s modified gross adjusted income had only exceeded $32,000, only half their Social Security benefits would be taxable. The rate for an individual is $25,000. If the modified adjusted gross income is less than the minimum thresholds, then no federal taxes apply on Social Security.

The above numbers are important for retirees to keep in mind when contemplating life decisions during retirement. Is taking a job worth exposing 50 percent or more of Social Security benefits to federal taxes? Keep in mind that 50 percent of Social Security income, non-taxable interest on municipal bonds, wages, interest, dividends, capital gains, alimony and distributions from a 401(k) or traditional IRA all count toward figuring out modified adjusted gross income that could nudge Social Security into the taxable arena. Some forms of income don’t count, but most do.

The good news is unrealized capital gains in taxable accounts, asset growth in tax-deferred retirement accounts, and deferrals in annuities don’t figure into modified adjusted gross income calculations. More good news is that in the 10 to 15 percent tax brackets capital gains and non-qualified dividends are not taxed, so investments delivering that sort of income are preferred above interest on a certificate of deposit or money market that is taxed as ordinary income. Returns on capital gains and non-qualified dividends would still figure into the couple’s modified adjusted gross income to determine whether or not the minimum or maximum threshold for Social Security taxes was reached. But personal exemptions and standard deductions would substantially reduce or eliminate any resulting federal taxes on Social Security benefits if most of the additional income was beyond the taxman’s reach.

The crux, then, is where a senior makes up that 60 percent of cash flow they need above Social Security. Qualifying dividends, like those from dividend-paying stocks, are taxed as long-term capital gains, or at a lower rate than interest from bonds or CDs. Making informed decisions is vital. It pays to be in the know.