When you find out that something that was meant to help you has been quietly working against you, you feel uneasy. That feeling came up a lot this year when people talked about persuasive technology research, a field that started out with good intentions but has since become so complicated that even the people who work in it feel uncomfortable.
The academic language is so technical that it’s almost cute. “Gamified persuasive systems” are what researchers call digital tools that take the reward loops from video games and add them to real-life apps. Money monitors. Apps for grocery shopping. Shopping helpers with cute icons and bars that show how far you’ve come. Years of written work have said that the goal is to help people make better choices, like eating better, sticking to their budgets, and not buying the third kitchen gadget this month on a whim. Up to a point, honorable.
But with different incentive structures, the same architecture that can push someone to choose a smart food choice can also push them to spend more. The mechanics are the same. What changes is who gains from the result.
Researchers talked about the results of a study with 333 people who used a prototype shopping app called MyShoppingBuddy. The app was made to help people make three types of decisions at the same time: smart shopping, budgeting, and time management. The participants thought it was convincing. They were able to use it. They did what it told them to do well. Another thing that the study brought up, albeit in a more subtle way, is how much the usefulness of these systems depends on the personality type of the person using them. People who scored high on agreeableness were much more likely to be persuaded by the app’s features. These people were more likely to trust, cooperate, and go along with what was said.
Take a moment to think about that. The system works best on people who are likely to do what they’re told without questioning it. That is not a flaw in the design. That’s the result of design.
It’s not hard to see what kinds of arguments are being used here. A lot of research has been done on tools like leaderboards, reward points, progress bars, personalized suggestions, simulated shopping scenarios, and achievement badges that are used in fitness games, health apps, and language learning platforms like Duolingo. They get people more involved. They make people do things. And they work amazingly consistently when they are made to fit the person. It has been known for a long time that a persuasive system that is tailored to your personality works better than a general one. Which brings up a clear question that academic papers often avoid: what goals are they good at achieving?
Some people think that the retail industry has been paying a lot more attention to this research than the conferences themselves usually admit. Researchers are using behavioral science to help people stop spending too much, and companies can use the same science to help people spend more. Alarms that go off. product suggestions based on your needs. For reward coins that run out. Flash deals are aimed at users who, according to data, are most sensitive to signals of scarcity. One system is meant to take away people’s savings, and the other is meant to keep them safe. The difference is not a technical one. It’s a moral one.
This is an interesting (or, depending on your mood, somewhat scary) time because of how far the infrastructure for personalization has come. The research that looked at how different personality types react to being persuaded used the Big Five personality model, which isn’t just used in academia. It has been a part of marketing data pipelines for many years. In a business setting, knowing that users who are highly agreeable respond better to social proof or users who are highly conscientious respond better to goal-setting features is not neutral information.
As I watch these sessions and published results, I can’t help but feel that the field is at one of those quiet turning points where the tools have moved faster than the talk about what they should be used for. Researchers who are making apps to help people save money are really doing useful work. They give good advice on how to design things. But the same conference circuit that shows this work also exists in a world where the biggest online stores have their own behavioral science teams. These teams work without the ethical constraints that academic publishing puts on them and with a lot more data than any university study could ever collect.
It’s still not clear whether the answer lies in rules, requirements for transparency, or just consumers getting better at spotting these patterns when they see them. It’s clear that the algorithms that are meant to take away people’s savings aren’t coming. They’re already here, and a lot of the research that was used to make them was also used to make them safer.
