The struggle is real.

Dozens of small business owners came to me in 2011 and 2012, asking me for help with social media. And I turned them away. At the time, I was running Likeable Media, a social media and content marketing agency for big brands, and I simply couldn’t afford to work with small businesses.

But I remembered their struggles. And eventually, when the dozens of small business owners sharing their struggle with me became hundreds, I did something about it. I built software to automate social media for small businesses, and then in 2013 launched Likeable Local, a new company, to sell that software to thousands of SMB’s.

The rest is history.

In the "struggling moments" are the seeds of innovation. The trick for entrepreneurs, especially founders of small but agile startups, is to harvest meaningful and causal insights from those struggles.

I asked David Schonthal and Bob Moesta, who have long careers in entrepreneurship and startups, to give their thoughts on innovation. Schonthal is a professor of innovation and entrepreneurship at Northwestern University's Kellogg School of Management; a leader in the Business Design practice at IDEO, an award-winning innovational consultancy; and co-founder of MATTER, a healthcare startup co-working space. Moesta is a serial entrepreneur, engineer, and product developer who has helped more than 3,200 new product launches, and built and then sold five startups. He is currently president and CEO of the Re-Wired Group, helping companies from startups to Fortune 100 firms innovate on new products and services around the world. Here are their insights:

Successful innovative products and technologies start by focusing on the consumers' problems to be solved. What do consumers really want to do? What are they trying to do but cannot? What are the tradeoffs they are willing to make to achieve better outcomes?

The solutions target consumers' functional, social, and emotional needs and, even more important, their desired outcomes. Think Maslow's Hierarchy, with its tiers of psychological, safety, love, esteem, and self-actualization needs. The further a product's benefits extend up that pyramid the better chance the company will tap into massive pent-up consumer demand for innovation. One example might be Fitbit and similar products that perform functional exercise- and diet-related tasks, but on an emotional level are really about delivering personal accountability and motivation.

Identifying struggles also unlocks insights about consumers' decisions to "fire" the products and services they're currently using in order to "hire" new ones that promise to do the job better, faster, more efficiently, more cost effectively, etc. Yet such actions aren't as obvious as swapping out one household cleaner for another. For example, Facebook appeared to create a new category in social media. But when examined through the lens of consumer struggle, it becomes clear that Facebook was hired by consumers to update friends and family on what's going on--a job that used to be occupied by routines such as the Sunday night family phone call. In terms of industry categories, Facebook doesn't compete with AT&T, but in the consumers' minds, even unconsciously, they do.

Consider BucketFeet with its trendy and colorful line of casual footwear featuring designs by independent street artists. As a shoe company, BucketFeet would appear to go toe-to-toe with megabrands such as Nike and Converse. But in consumers' minds, BucketFeet is really wearable art--designs on a different kind of canvas.

Consumers' "hiring" and "firing" decisions are significant sources of insight because there really aren't any new jobs for the latest products to assume. Consumers' overcrowded lives can't handle one more thing. For entrepreneurs, that reality is a compelling case to stop focusing on the features and benefits of what they've invented and developed, and instead put their emphasis on moments of struggle and consumers' hire/fire decisions.

Successful innovation starts with finding the right struggle to solve. Many techniques such as the Jobs-To-Be-Done framework for consumer research and Design Research have long been applied to corporate innovation. The same approach can be utilized by startups.

Identifying moments of struggle can be accomplished by conducting research with as few as ten people if the questions and observations are of sufficient depth about how and why they buy and use products and technologies. Go into people's homes and witness how and what they are using. Study how they adapt products and technologies with their own workarounds and hacks, and improve or even push the limits of performance. Such small-scale but intensive ethnographic studies level the playing field for small startups and entrepreneurs who don't have huge research departments and massive consumer databases at their disposal.

The key is to frame the forces of progress (function, social, and emotional) that cause people to make progress in their lives. As the "Forces Diagram" (below) illustrates, the push of the situation and the pull of a new idea are weighed against the habit of current practice and the anxiety of adopting a new solution. Progress only happened when the push and pull are greater than the anxiety and habit.

(Sources: The Re-Wired Group)

Most consumer struggles from the functional to the emotional can't be hypothesized in a research lab or conference room or sketched at a white board. Doing that runs the risk of targeting the wrong market or a need that really doesn't exist. Consider the Segway, an intriguing personal mobility device that looked like it addressed a real functional need, but doesn't target a consumer struggle with a solution that allows them to make progress.

Just because something is possible, doesn't mean that consumers will clamor for it en masse. 3-D printing is a great technology, but still largely in search of a mainstream job. There are interesting applications, such as in making custom hearing aids, but no one has yet found a widespread use for this influential technology. Feasibility and desirability are not the same and need to be thought about differently.

Consumers are the source of most successful innovations; their desires and struggles are the sparks of genius that lead to disruptive opportunities. Technologies are the enablers to make it easier, faster, lower cost, or just better. Henry Ford understood this, noting that people didn't know they wanted an automobile. What they wanted was a faster horse. What followed was the firing of horses to transport the masses and the hiring of automobiles that could do the job better.

In the same way, entrepreneurs today need a keen understanding of human behavior and consumer decision-making. By focusing on moments of struggle and real problems to be solved, even small startups can compete head-to-head with big companies--and win.

Now it’s your turn. How has your company turned past struggles into triumphs? Tell me in the comments section below.