User Testing Insights: Keeping Advanced Features Simple

by on Aug 28, 2013

This article originally appeared in the Summer Edition of Consulting Insights.

According to a recent International Data Corporation report, tablet sales will outpace personal computers by the end of 2013. While tablet usage haUser Insight Mobiles increased dramatically in the past three years, users have not abandoned their desktops or laptops en masse. Rather, many users have added these devices to their technology toolbox, employing a particular device – computer, tablet or smartphone – based on factors such as their location or the task at hand.

With this in mind, Corporate Insight designed a usability study that focused on the mobile and desktop offerings of five leading banks – Bank of America, Chase, Citizens Bank, U.S. Bank and Wells Fargo. We invited participants to the usability lab in our Midtown Manhattan offices in order to understand how mobile applications match up against traditional website interfaces both in terms of functionality and usability. Although websites offer much richer suites of account management tools, we found that in some cases, particularly person-to-person (P2P) payments, tablet apps’ relative simplicity led to strikingly higher success and efficiency. These and more findings available in our full whitepaper demonstrate the value that can be gained when you take the time to observe real users interacting with an interface.

For each of the five banks, we tested five participants, all of whom had experience with online banking but little or no familiarity with the interfaces they used in their session. A trained usability professional guided the participants through 13 tasks (eight on the bank’s website via a desktop computer, and five on the bank’s iPad application), while a second analyst monitored the session and took notes from an observation room separated by a one-way mirror.

We recorded audio and video from each session and encouraged users to “think aloud” in order to understand their state of mind and gain more insight into their decision-making process. At the end of both portions of the test, participants were asked to complete a Standard Usability Scale (SUS) survey, a ten-question Likert scale questionnaire that is used to determine individuals’ impressions of the overall usability of the interface.

Study Highlight: P2P Payments
For the most part, study participants were familiar with nearly all of the assigned tasks, having performed similar actions within their personal bank accounts in the past. The one exception was P2P payments. While several participants commented that they had experience sending money to friends via payment services such as PayPal, none had prior experience using a P2P feature on their bank’s website or apps. What’s more, most subjects were unaware that banks even offered this service.

There was a notably large disparity between task scores when looking at P2P payments on the desktop as compared to the iPad app. Less than half of all participants were able to complete a P2P payment on the desktop with relative ease, and nearly 25% failed completely.

One reason for such high failure rates on the website is that banking firms have yet to establish a consensus as to where the P2P option should be located within their website structure. Some firms offer P2P payments within the Transfers section of their site, while others locate the option alongside Bill Pay.

Two firms in particular illustrate these difficulties. On the Bank of America website, the P2P interface is located within the Transfers area of the website. Although this is the first place participants thought to look, nearly every participant failed to recognize that the bank categorizes these transactions as a transfer “Inside the bank.” Users immediately navigated to the “Outside the bank” tab and, when they couldn’t find the interface, began browsing the Bill Pay section of the site. Similar to the users’ expectations we observed on B of A’s site, U.S. Bank participants’ first instinct was to look in the Funds Transfer tab. Unfortunately, the firm places the P2P feature within Bill Pay. Although most of the test subjects eventually discovered the interface, they commented that they thought the term “bill pay” suggested paying a company, not a person.

Compared to the website, participants found it much easier to send money to a friend via the iPad apps. Participants were able to complete the task on the iPad almost 60% faster than they could on the corresponding bank’s website, although at the time most of the firms did not support adding new payees through the app interface (which we instructed them to do on the desktop). Users found it much less difficult to locate the P2P interface on the app, a result of the more streamlined and condensed tablet experience. Firms like Chase and Wells Fargo group transfers and payments together on their tablet apps, limiting any possible confusion between the two categories.

System Usability Scale (SUS) Survey Results
After completing the usability tests on both the website and the tablet application, participants were asked to complete a standard, ten-question System Usability Scale survey. This common questionnaire is used to determine an individual’s perception an interface’s ease-of-use and learnability. An average SUS score is 68. Based on our findings, participants expressed a strong overall preference for tablet apps – 86.5 average SUS score – over their traditional website counterparts – 61.1 average SUS score.

The data reinforces the idea that users prefer interfaces that are simple, straightforward and predictable. With most financial firms offering content-rich websites, it becomes even more important that the website is designed in a straightforward and user-friendly way. While complex tools and offerings can be a great resource for clients, they become useless if they are not simple enough for every client to find and utilize.


P2P Payment Process Proves Problematic
Should you feel that user testing is right for your company, Corporate Insight is here to help. We can:

  • Collaborate with you to design a study that tests the key features of your website or mobile app, and/or the platforms offered by your key competitors
  • Recruit participants that represent your target audience
  • Provide you with a detailed analysis of test sessions
  • Assist you with your existing usability efforts by moderating tests you have designed or hosting tests at our facility, conveniently located above Grand Central Station in Midtown Manhattan

For more information, please contact Adam Hulnick, VP of UX Research, at 929-575-4657 or