It’s a familiar story: You come up with a new idea for a product. You bring your product to market, and it turns out what you envisioned is not precisely what your customers want. Instead, they seize on a particular feature of your product and begin using it—a lot—in a way you had never planned. You’re wise enough to follow the market and so you adjust your plans to suit customer preferences. 

Sometimes that’s the right approach, and sometimes not. Knowing when to adjust your product to the market and when to stick stubbornly to your original vision may be the hardest question most entrepreneurs have to answer. 

Consider Smarterer, launched last spring. The concept is brilliant in its simplicity: A sort of Wikipedia of crowdsourced, self-administered tests. Any user can take a test in a wide range of skills and share his or her score with the world. Users can also comment on the tests, add questions to improve them, or create new tests of their own. The idea has a lot of appeal: From its launch in October through mid-January, users created more than 400 tests with 35,000 questions, according to CEO Jennifer Fremont-Smith. “In the nine months leading up to January 16th, we had 800,000 questions answered,” she adds. “That’s the metric we look at most of all.” 

Users had a different idea 

Many of those user-created tests, however, focused not on work-related skills, but on interests from Harry Potter to Gilmore Girls to the streets of Boston. The site’s users seemed to want to turn it into a giant, crowdsourced version of Trivial Pursuit. But that wasn’t the idea Smarterer’s founders had when they started the company: Their vision was to create “the gold standard in skills measurement,” Fremont-Smith says. Job-seekers would use Smarterer tests to demonstrate their technical, business, and social media skills, and employers would review test scores when evaluating job candidates. 

The move toward light-hearted quizzes came with the appealing prospect of taking Smarterer into the lucrative realm of social gaming. (Zynga, anyone?) “There was a sense that going in that direction would make the user base grow really fast because it would have been more light-hearted and engaging,” Fremont-Smith says. “There's always that temptation to go after whatever will push the growth the fastest.” 

To pivot or not to pivot 

In the end, the co-founders resisted the temptation. “We thought it would make it harder to be accepted as a credible source for professional skills validation and we thought there was a huge market for that,” Fremont-Smith says. “So we managed to stick to our guns and build the site we thought had the most long-term value.”  

So far, it looks like they made the right choice. In mid-January the company began a very limited beta of its B2B product, which allows employers to send job applicants directly to the site to take specific Smarterer tests, then returns applicants sorted in order of their test scores. Although only a few employers are using the tool it’s clearly caught the attention of job-seekers who now recognize Smarterer as a legitimate way to validate their skills. In the two months since the B2B beta launch, there have been 4.2 million questions answered, more than four times as many as in the previous four months combined. 

The business model can wait 

The site is “pre-revenue” as Fremont-Smith puts it and, she says, Smarterer is in no hurry to begin monetizing its site or its tests. “We're funded for growth; we'll raise more capital as needed and we're entirely focused on pushing forth this vision.” 

If the company gets there, she adds, building revenue will be no problem. One possibility would be charging employers for the use of its B2B testing tool, but another might be the Smarterer members themselves. “A lot of recruiters have already come calling to ask about using our platform as a candidate sourcing tool,” Fremont-Smith says. “The holy grail for them is to find ‘passive’ candidates (those not seeking jobs) who have hard-to-find skills and would not mind being contacted by a recruiter. There’s a lot of value possibility there.” 

The biggest lesson learned from the experience? “Stay true to your vision,” she says. “Too many entrepreneurs pivot too soon.”