Recommendations Do Boost Robo Rep Power: Researchers

Robo reps that give concrete product purchase recommendations may generate noticeably higher sales than robo reps that simply give consumers personalized product analyses.

Three researchers have provided evidence for that possibility in a “working paper,” or research paper draft.

(Related: T3 Event Shows How Advisors Can Keep Robots at Bay)

The researchers — Kate Bundorf of Stanford, Maria Polyakova of Stanford, and Ming Tai-Seale of the University of California San Diego — studied the power of robo rep recommendations by creating web-based decision support tool aimed at people who were  shopping for Medicare Part D prescription drug coverage.

The Study

The researchers recruited the 1,185 study participants by offering $50 gift certificates to 29,451 people ages 66 to 85 who were getting medical care from providers at the Palo Alto Medical Foundation.

The researchers kept patients who were using Medicaid or Medicare Advantage coverage off the invitation list, to maximize the number of study participants who would be eligible by sign up for stand-alone Medicare Part D drug coverage.

The researchers designed the decision support tool to give each available plan an “expert score,” based on variables such as the shopper’s estimated total spending and the plan’s quality rating.

The researchers divided the study participants into three groups:

• Control group: These participants logged on to a website and saw a list of plan information sources.
• Information group: These participants received personalized estimates of what their total spending would be in each available plan, along with the plan’s premium amount and star rating.
• Information + Expert group: These participants received the same cost and quality information as the information group participants, along with a list of three plans labeled “recommended for you.” The system used the expert score to determine which plans were labeled “recommended for you.”

The average age of the participants was 74. That implies that most of the study participants were considering whether to keep or replace Medicare Part D coverage that was already in force.

The people who actually participated seemed to have a higher income and socioeconomic status than other patients invited to participate, and they seemed to be much more interested in switching plans than the typical Medicare Part D plan enrollee, the researchers report.

Fewer than 10% of all Medicare Part D policyholders switch policies, but 28% of the people in the study control group, who received no extra decision support tool information about the available Part D policies, switched plans, the researchers say.

‘Treatment Effects’

About 29% of the participants in the Information group switched plans.

The difference between their switching rate and the control group’s switching rate was so small that it might have been due to chance, the researchers say.

For participants in the Information + Expert group, the switching rate was 36%.

The number of plan switchers in the Information + Expert group was 28% higher than in the control group, the researchers say.

The researchers also found that the average expected savings level was $112 for the control group participants and $206 for participants in the Information + Expert group.

Non-Participants

The researchers suggest that people who sit out of expert system studies might be the people who would be most likely to use the system’s advice.

The researchers developed a forecast of what the non-participants would have done, if they had participated, by applying trend information from the Information + Expert group to people with the demographics of the non-participants.

Given the demographics of the non-participants, and the demographics of the Information + Expert system users who did switch plans, it looks as if many of the non-participants would have taken the expert system advice and switched plans, if they had seen the recommendations, the researchers say.

Robo Recommendations Matter

The evidence that decision support tools can actually change people’s decisions “raises concerns over the possibility that algorithms may influence decision-making in ways that have poorly understood or unintended consequence for consumers,” the researchers write. “Algorithms may generate biases in decision making, either strategic or inadvertent, that have important downstream consequences.”

Resources

A copy of the Bundorf algorithm paper is available here, behind a paywall, on the NBER website.

— Read 5 New Things Finance Professors Are Saying About Your Prospects, on ThinkAdvisor.

— Connect with ThinkAdvisor Life/Health on Facebook, LinkedIn and Twitter.