You might think that amid a much-touted “recovery” that American workers would be feeling better about their finances in general, and about retirement in particular, but you’d be wrong.

Confidence is falling for the second year in a row, according to a new study from Hearts & Wallets, which found that a skimpy 13 percent of households say they’re “confident, comfortable and secure” about their finances, and only one out of every three working Americans say their retirement savings are on track.

In 2013, people were more optimistic, with just one out of four households laying claim to moderate to high financial anxiety; now that’s risen to one in every three.

While the study said that the increase is driven by those who have less than $100,000 in investable assets, it added that the worry level is up across all income levels.

Retirement worries dominate the worry list, with the future of Social Security and setting aside sufficient funds for retirement claiming the top and third places on the list. (The future of health care in the U.S. snagged the second-highest spot—not surprising, since it too is a massive worry during retirement.)

Social Security fears were in third place in 2014, but have jumped two places to head the 2015 list—also not surprising, considering how much political chatter revolves around cutting or eliminating benefits for various segments of the population.

As a corollary, the number of people confident that Social Security would be a reliable source of retirement income fell by 2 percentage points.

There was an even bigger drop in the number of people who believe they’re saving enough for retirement. In 2014, 36 percent said they were on track with retirement savings; this year that’s fallen to just 31 percent.

With all these worries about the future on their mind, it’s probably to be expected that the top goal on the financial list was to build up an emergency fund (42 percent).

And people still hope to be able to work less, although the goal of completely retiring has receded; 34 percent hope to work less or spend their time as they’d like as they age.