Studies can be useful. You might learn the type of decision making you use or even come to the conclusion that we're all dumber than we think.

But studies are also potentially confusing. Market research is often no better than flipping a coin. There are many potential problems, like not testing enough potential answers to a question. There might be bias because of someone's own interests or beliefs. Or you might have made unavoidable choices where any direction muddies understanding

A current example is a study from Upwork (a platform for freelance work) and Freelancers Union (a business organization for freelancers) touting that freelancers make up 35 percent of the US workforce, or 55 million. It's a remarkable number, and could be an important data point in creating a career or business strategy. But trying to reconcile that data with those from other reputable sources shows how little anyone grasps the reality of the situation.

Double check the data

The number sounded incredibly high, so I decided to do some checking. Freelancers should be a subset of self-employed people, which would include people who own businesses that employ other workers.

According to a year-old analysis from the Pew Research Center, 30 percent of US jobs are held by the self-employed and the workers they hire.

Their data source was the U.S. Census Bureau, which had made the specific information publicly available for the first time. Of the 146 million workers in the country, about 14.6 million, or 10 percent, were self-employed. They, in turn, employed another 29.4 million workers.

According to a March 2016 publication of the Bureau of Labor Statistics, the number of self-employed people last year was 15 million, or 10.1 percent of all US workers. Of those, 60 percent (9.5 million) were unincorporated. The remaining 40 percent (5.5 million) were incorporated.

When it comes to employees, 42.1 percent of the incorporated self-employed had them, compared to 14.3 percent of the unincorporated. Do the math and 3.7 million self-employed have employees. Take them from the overall 15 million and the remainder, 11.3 million, represent the self-employed without employees. That seemed like it should be an upper bound on freelancers, a number a fraction of the size that the Upwork/Freelancer Union study claims.

Why data gets tricky.

I contacted the PR contact from Upwork for more information about the study and my question about the validity of the numbers. They pointed out that the two organizations used the following definition of a freelancer: "Individuals who have engaged in supplemental, temporary, project-based or contract-based work, within the past 12 months."

As I've written about elsewhere, many "gig" workers, who would fit under this definition, don't realize they have to file taxes. As they don't think of themselves as running a business, they might be missed by government data and surveys.

Furthermore, while the Upwork/Freelancer Union asked about any freelance work in the last year, the Bureau of Labor Statistics, which conducts interviews every month, only asked people about their work situation in the previous week. And they don't ask whether people hold jobs and do contract or temporary work on the side.

The BLS is likely missing many who have done freelance or contract work, particularly part time. And yet, Upwork and Freelancers Union extend the net too wide by including anyone who had done anything in 12 months, if you're interested in people who freelance on a regular basis.

Another potential reason for a discrepancy is how the 6,000 respondents in the new study were recruited. The participants came from a commercial online panel of people generally willing to participate in studies, versus a more random sampling of the population that the government uses.

What seems fair to say is that neither this new study, nor the ones conducted by the government, actually get an accurate reading on how many active regular freelancers there are in the country. That means the further findings of either could be questionable.

Published on: Oct 12, 2016
The opinions expressed here by Inc.com columnists are their own, not those of Inc.com.