3 Things I Learned About Usability Testing By Looking at Customers in Their Underwear

How resistance can get in the way of qualitative insights

“Please strip down to your underwear now,” I heard myself say. I tried to act casual about the situation I was in but the fact is, I felt pretty uncomfortable. When the middle-aged man walked back out of the dressing room, I was going to wrap a tape measure around his naked mid-section.

As a product manager for a technology company, you wouldn’t expect this task to be in my job description.

I’d been measuring people of all shapes and sizes for several hours. They were all there because they had answered an ad I put on Craigslist:

Wanted: Men and women over the age of 18 for a research study in the apparel industry. Please wear loose fitting clothing. Must be willing to strip down to underwear. Pizza will be served. Compensation: $25.

Being a naturally shy person, this was definitely outside of my comfort zone.

Resistance is a frenemy

Now, let me pause for a moment and introduce my buddy Resistance to the party. Oh, have you already met? I’ll bet you have.

Resistance often keeps us from talking to customers. Now, keep in mind that he has good reasons for what he does. He wants to protect us. He wants to keep us out of uncomfortable situations. Or he reminds us that we have so many other things to do … certainly we can find out everything we need to know from a survey. Turns out, Resistance is often more frenemy than friend. (And in the credit-where-credit-is-due department, let me say that author Stephen Pressfield is responsible for the concept of personifying Resistance in this way.)

Back in my days with the apparel company, the inescapable fact was that I needed try our fit-testing body scanner on real humans. I was forced to tell Resistance to beat it and push through my uneasiness. The result was that I ended up with loads of information we never would’ve had if we hadn’t brought in outside people.

What product managers should remember

Today, working for an email marketing company, I rarely have to ask people to strip down to their underwear to test out a product. That certainly makes talking to customers a whole lot easier.

Still, that awkward experience from many years ago taught me several important things about usability testing. Three of the most important are:

1. For a nominal fee and a slice of pizza, people will get nearly naked for you. Who knew? I certainly didn’t when I placed that Craigslist ad.

But the larger point is this: It’s not as hard as you might think to get customer feedback.

People generally like to be asked for their opinions. Don’t be afraid to reach out to customers. Not everyone will have time for you, but plenty of them will.

Today, instead of Craigslist and pizza, we use a screener in our application and offer participants a $25 Amazon gift card.

2. People are squishy. Data is not. Quantitative data is a great starting point, but it’s no substitute qualitative data.

During our fit testing experiment, we had one guy come in who swore he had a 32” waist. Our measurements said otherwise – he was closer to a 36. We soon figured out that he’d been stuffing himself into 32s even though they were much too small. Had we simply surveyed this man, we would’ve put him down as a 32. Because the human body is squishy, people could report all kinds of misinformation.

Human behavior can be squishy, too. People may report that they use your software a certain way, but you won’t know for sure until you find out for yourself.

Now, we start with quantitative data from analytics platforms like KISSmetrics and then we dig deeper with interviews.

3. If you’re making products for humans, you need to make human connections. With so many things competing for your time, talking to customers may feel like it’s not time well-spent. However, with just a conversation or two, you may discover an unexpected insight. You don’t know what you don’t know.

Just last week I was speaking to a long-time customer about a new product feature we’re developing. In the course of the conversation I had an epiphany: He’s not going to use this great new feature unless it works with his entire external ecosystem that he’s spent years developing. That put our plans into a whole different perspective for me.

It’s these types of insights that can only come from making a personal connection with someone.

Listen, I know I’m probably not telling you anything you don’t already know. But knowing it and doing it are two different things. We all know that we should eat right and exercise, too, but a booming diet industry is proof that not everyone follows that advice.

I promise you, blocking out even two hours to talk to customers can yield insights that you’ll incorporate for the next year. So if you’re a product manager, or if you’re involved in developing new products in any way, give Resistance his walking papers – or at least give him an afternoon off.

Lazy UX Design – Phone Number Fields

Grrr, I hate poorly designed forms. I’m especially annoyed by forms that want humans to think like computers. Case in point, take a look at this phone number field from Bed Bath and Beyond.

2016-05-15 Bad UX Patterns - Phone number field

Phone numbers in the US can take many forms – parens around the area and dash separated are the most common. What isn’t common is a form typed out without any separators. However, lazy design forces that on people. One side effect of this design is that the auto form completion built into browsers and password managers break when used here.

When you are thinking about your input fields, think about how your customers think about the information they are providing and provide support for the most common patterns.


When to Use Which User-Experience Research Methods

There are many different ways to research your users. So many in fact that it’s hard to know when to use a particular method. In this post, the author lays out framework you can follow to help you determine which UX research method to follow. In When to Use Which User-Experience Research Methods the author describes a framework for choosing which research method to use based on where you are in the lifecycle of a project.

While it’s not realistic to use the full set of methods on a given project, nearly all projects would benefit from multiple research methods and from combining insights. Unfortunately many design teams only use one or two methods that they are familiar with.  The key question is what to do when. To better understand when to use which method, it is helpful to view them along a 3-dimensional framework with the following axes:


Lazy Registration Examples

lazy registration example

Lazy Registration is a great design pattern if you want to allow your customers to kick the tires on your site or application without having to create an account.

The website ZenoBase has an example of this pattern in use.

UX Patterns for Expression Builders

I’m working on requirements that require an expression builder. I’m collecting some examples here.

In this example there are a number of expressions that are draggable so you can change the order of the operations.