For as long as commerce has existed, sellers have been trying to better understand what exactly their buyers want and how they behave. In that sense, techniques that might be considered old school data collection--focus groups, customer interviews, surveys, etc.--are actually relatively recent, just a few decades old.
However, the digital revolution has led to many marketers abandoning these techniques in favor of digital solutions that let you automatically track and analyze customer behavior online. The big question for marketers now becomes: Which of these two schools is better suited to evaluating your customers? Should you stick with the tried and true methods of the past, or embrace digital data collection? Let's take a look at the pros and cons.
Where Digital Beats Traditional
All of the classic data gathering techniques that originated in the mid-20th century share a few big drawbacks. These are:
- They have high cost in both money and time.
- They can be hard to do at scale.
- They leave room for significant human error.
- They are potentially biased or leading.
Digital Is Cheaper And Easier
Collecting paper survey data can be costly. You have to pay for all the paper and ink, the postage to mail out all those surveys and the wages of whoever has to go through the surveys and enter the data. Focus groups often require paying an expensive expert to conduct them. Digital surveys and customer data collection techniques run automatically, and there are many solutions on the market can distill that data into actionable information.
Digital data collection techniques also allow for much larger sample sizes, improving the reliability of data. Focus groups and individual interviews are the ultimate example of small sample sizes, but even paper surveys can be difficult to conduct at any kind of large scale unless you have massive resources.
Companies often use special discounts to encourage people to give them information, but that's harder to do with traditional methods, as someone usually has to mail in a form and then wait to get a coupon in return. With modern tools, customers can just give you their contact information and then see the deal automatically show up in their inbox.
Traditional Techniques Are Unreliable
Paper surveys are notoriously unreliable. One study of paper versus digital surveys found that 35 percent of paper surveys had errors of omission against just 3 percent for digital. Gaps in the data can significantly diminish its effectiveness.
Finally, focus groups and interviews are too easily biased by leading questions or subtle body language signals from an interviewer. On the other hand, multivariate tests or A/B tests get run on users that don't even realize they're being tested, so they'll behave in a more natural manner to changes in your website design, products or customer experience.
Plus, customers don't always know what they want. Minor changes in wording on a landing page can increase signups up to 38 percent, even though customers probably wouldn't consciously register a big difference between the two. You have to observe them in action.
Where Traditional Data Collection Makes Sense
The evidence for digital data collection is pretty significant. It's cheaper, easier, can reach more people and tends to be more reliable. However, there may still be some places where traditional data collection techniques make sense.
Focus groups, for instance, allow you to go beyond quantitative data and into more qualitative discussions of why consumers react the way they do to your marketing and products. Steve Blank and other proponents of the "lean startup" philosophy advocate the usage of similar groups in helping develop minimum viable products for budding businesses. In-person interviews can shed insight on customer behavior, but generally only if you narrow the focus to understanding people's emotional responses and motivations behind their actions. For the collection of hard data, digital methods usually win on every count.