What Are The Best UX Testing Methods?

We spoke with Andru Dunn, Senior UI Developer at John Lewis


Andru Dunn has worked on the design team at John Lewis, one of the UK's premier retailers, for over two years. The company's mobile app gives users access to the store's 300,000 products, make price matches, scan products in to access further details, as well as building wish lists and checking product availability. Being a company as large and established as John Lewis, any digital product they roll out must cater to a wide audience, and so testing is key to determine the likelihood of a new product performing well.

Following his presentation at the UX & UI Innovation Summit last June, we sat down with Andru to discuss how he and his team perform testing, and how it influences the final product.

Hi Andru. So, how much does data influence UX and UI design at John Lewis?

Quite a lot. We have data backing up everything that we do, and we commonly test and multi-variant test things that we do, to validate anything that we've just released or things that are candidates that we want to change. It’s a big input in what we do.

There are a few different schools of thought on how to best test UX design. What's your view on the most effective?

I'd say the probably the most rewarding way to test designs is with user research. So, fortunately, we have an in-house lab where we get our customers in to test our prototypes on. And it's really rewarding because you get to see different ways of thinking, particularly about certain tasks. Being someone working in tech, you might think you know that everyone's going to use it this way, and you're often surprised about how customers are actually going to use your product.

Is that more, sort of, conversational, where you interact with them rather than the data?

Yes, it is. So we have UX researchers within our team that are shared resource and they will manage the interviews. So, we'll set out scenarios and prototypes that match those and then see how the customers react.

Does that massively change the final product?

It tends to, yeah. It tends to shape how we get to the actual end result. So we’ll tend to normally, if possible, just have two on maybe a phone (or whatever is easiest to get to prototype), but ideally based on their normal journey – we’ll test them on an iPad if they’re used to browsing on there, or such like that.

So what are the elements of modern UX design that you find the most exciting?

I think the most exciting thing coming up would be designing an offline experience, particularly for customers who are heading into li-fi on their phone, while they’re on the train for example. It's how we serve those customers where they are particularly in a patchy, or bad place with their network.

So it’s a big deal for developing countries, in particular?

Yeah, definitely. I know that’s not particularly a concern for at us John Lewis at the moment, but I think it’s probably the most exciting thing going.

Do you use personas in your user testing? And how effective do you find them?

Yes, we do. And very, I think they’re very good at the initial phase – when we’re designing. It just reminds you of, again, putting you in another person's shoes of how your core customers are going to be using a product. And it helps shape how we go forward and try and tackle and solve a problem.

And what do you think will be one of these challenges you can see coming in UX moving forward?

I think probably the biggest challenge is scaling effectively. I know it's a highly sought-after skill and role at the moment, and it's something that I think, as teams scale, to try to embed that into their culture and get everyone on board at the same time is tough. That's probably the most difficult I'd say, but I think it's also difficult to try and integrate into a big company who's not used to it. I know we're fortunate to be in a position where that's not the case, but from here on some of the people today in particularly enterprise size businesses that they struggle with that.

Find out more about managing digital products at the Chief Digital Officer Forum in London this April 25-26. Check out the full schedule here.


Read next:

Why Blockchain Hype Must End