One of the hardest lessons to learn in UX is that researchers can’t always trust what test participants tell them. People don’t always behave rationally, and this means that research participants are sometimes unable to give accurate answers to even the simplest questions. Though it sounds counter-intuitive, it’s backed by decades of research on the difference between reported behavior (what we say we do) and actual behavior (what we actually do).
I encountered the gap recently, while researching how to improve smartphone banking apps. I asked two sets of users whether they wanted the app log-in process to be more convenient or more secure. Depending on whether I asked them while using their app or removed from the process, the question garnered strikingly different responses. When I observed people using their banking apps, they told me that the security requirements for logging in were too rigorous. They’d prefer a less secure, more convenient log-in process. Yet, when we sent a survey to other banking app users, their responses were the exact opposite. These users said they’d like more security, even at the expense of convenience.
