Mass-affluent millennials — those with $50,000–$250,000 in investable assets — are in a funk over their student loans. They believe that they haven’t saved enough toward retirement because of that mountain of student debt hanging over their heads.

Among mass affluent respondents in the U.S., according to a survey conducted by Braun Research Inc. on behalf of Bank of America’s latest Merrill Edge report, the feeling in general is guilt — for not investing enough in 2014 and for other financial decisions they’ve made during the year.

While a full third of respondents said they feel guilty for not investing enough during the year and lesser numbers feel guilty about food and drink decisions (18 percent) and lack of time they spent with their loved ones (17 percent), millennials stood out as chalking up their lack of retirement savings to student debt.

More than half of respondents to the survey (51 percent) said they saved nothing at all for retirement in 2014. And only a third (35 percent) of those nonsavers said they intended to change their ways in 2015.

However, on the positive side, 83 percent of those who did sock money away for their golden years during 2014 intend to do the same next year.

But then there’s that pesky student debt. Of those respondents who have outstanding student loans, 65 percent are saying they have to pay that debt off completely before they can save for retirement. And of those with student debt, 45 percent say they feel guilty for not investing enough in 2014, while only 27 percent of those who aren’t laboring under that particular sword of Damocles feel guilty for not investing enough.

And as far as 2015 goes, 59 percent of mass affluents overall intend to make retirement saving a goal. Among those, 78 percent of Gen Xers are focused on investing and saving for retirement in 2015, compared with boomers (68 percent) or millennials (56 percent).

See also: 

Whole life insurance: The best kept college savings plan secret