The Real Value of Risk Questionnaires

Perhaps the best financial planning lesson I’ve ever seen was back in 1984 when entrepreneur Ross Perot (who later became an Independent Party presidential candidate) sold the company he founded in 1962, Electronic Data Systems, to General Motors for $2.5 billion in cash. Perot subsequently revealed that he invested his entire $2 billion-plus after-tax gain in a portfolio comprised entirely of triple-tax-free muni bonds, saying: “I don’t like filing tax returns, and besides, how much money do you need to live on?”

I think about Perot’s portfolio whenever the subject of risk tolerance tests comes up. I’ve never been a big fan of these tests for two reasons. For one thing, it seems unlikely that clients sitting in a nice office on a sunny day, happy with their new financial advisor and full of optimism about their financial plan, will be able to accurately assess what they’ll be feeling after a 30% drop in their portfolio. (In fact, a couple of years ago, research by John Grable and Sonya Brill at Kansas State University showed that men’s risk tolerance mysteriously soars when the financial planner across the desk is an attractive woman. Go figure.)

More importantly, it’s always seemed to me there is only one rational answer to the question, “How much risk do you want to take in your retirement portfolio?” Which is: “As little as possible to still meet my financial obligations and reach my goals.” As it turns out, we old dogs really can learn a new trick or two (if you can get us to stop sticking our heads out of the car window). Case in point: I recently had two enlightening conversations that have brought me around on risk tolerance tests, if you keep in mind that they’re not really about testing risk tolerance.

The first conversation to grab my attention was when Geoff Davey, cofounder and director of FinaMetrica Ltd. in Sydney, said: “The standard built around suitability in the United States is the worst in world. Well, maybe behind Canada. A group of academics recently studied the roots of advice that had been deemed unsuitable: More than half had resulted from the use of a risk tolerance test.”

Strange talk from a guy who creates risk tolerance tests for a living. He explained: “In the ‘70s and early ‘80s, advice was given on the basis of what story brokers had to tell about the product they were trying to push. Most clients ended up with eclectic portfolios. Then MPT came along with optimized and efficient portfolios, and created a new question: How did brokers sell each fund? Risk questionnaires were invented to create a story, with no validity at all.”

Validity is something Davey takes very seriously. Typically, to create a FinaMetrica risk questionnaire with 25 questions, he starts with about 150 questions and then tests them: His most recent risk test took two years to compile. “First, we trial the questions against a population for understandability and answerability,” he explained. “Anything with a technical term, even ‘bonds,’ people didn’t like. They didn’t get it when we quoted potential range of returns in numbers, either. Once we get a set of questions with high usability, we do a statistical analysis to see if questions actually differentiate people: whether they elicit a range of answers or do they tend to fall into one or two groups. Our test sorts clients into seven risk groups, but it’s not an even distribution: It’s like clothing, with more in the middle.”

While the methodology is interesting to a nerd like me, so is how Davey intends the results to be used by advisors. “Our system turns the whole business from a compliance risk management chore to a relationship-building opportunity,” he said. “The report provides trigger points for discussions with clients. For instance, we left in an insurance question, even though we think it depends as much on knowledge as risk tolerance. So don’t score it, but advisors can use it to start a conversation.” According to Davey, his reports help advisors have better conversations with new clients, so the clients become better “connected” to the advice, and trade-off decisions are better documented. (I knew he’d throw the compliance angle in there somewhere.)

However, he also said risk tolerance really isn’t the point: “The advice process might not even use the tolerance that comes out,” he said. “If the client says any drop in portfolio value will make them uncomfortable, the advisor will have a conversation and come to a compromise solution, or just lower the risk group.” The final question he asks clients is to predict their risk score: If there’s a big difference, that leads to another conversation.