The value of telephone user surveys

When you are building a web app you need to collect feedback. As someone who’s conducted both telephone and email surveys over a large user base for years, it’s my opinion that telephone interviews are vastly more valuable than email surveys.

From the earliest days we did telephone interviews with our customers. We prepared surveys and then we reached out to customers by email and by calling direct and scheduling times to talk about our web app. Making a call like that is scary, but time and time again our early adopters did nothing but give us encouragement and show us the way.

So what do you need to be successful with user interviews? You need to prepare a survey – or rather a call script – before you pick up the phone. You need it so you can make the best use of your precious time. You also need it for consistency so you can start to see trends in the questions you are most interested in.

For example, we always ask questions like:

- what were you using to invoice before?
- how long did you spend invoicing before?
- how long after?

Now those look like questions you could put in an email survey right? Well, let me tell you what I learned from the calls that I’ve never seen on an email survey.

On many calls people would tell me, “your software changed my behaviour”. At that point in time my heart would stop…but inevitably, after what felt like an eternity, they would tell me, “now I invoice as I do my work instead of putting it off until the end of the month!” and man, were they excited! I always thought it was bad if software changed your behaviour – turns out I was wrong, and I maintain that I would never have learned this from an email survey.

Other reasons the telephone is so much better? Intangibles – the important bits come out in the slight inflection of the voice, a quick gasp at a question or a thoughtful reply. When you hear a cue like this and you are on the phone, you can ask people what they are thinking and explore things. When you do that, you’ll start a conversation that will take you places you never imagined. I’d take a sample of 10 telephone interviews over 1000 email replies. Also, you’ll be amazed how energizing it is to talk with people who use your product. When you are sitting in your basement putting your life and soul into building your business, energy is priceless. In person interviews would be even better I’m sure, but it’s hard to make those happen when you are just getting started.

There are downsides to telephone interviews: they take longer to prepare, and they are more time consuming to carry out and you have to talk with people – which developers often don’t like to do. But I think those obstacles are well worth overcoming.