Jessica Alba has at least one thing in common with less-famous entrepreneurs--at the beginning, no one thought her idea for a company made any sense.

Nor did they think that she, personally, could pull it off. Alba and one of her co-founders, CEO Brian Lee, spoke about these and other challenges at Inc.'s GrowCo conference in Nashville last week.

Alba knew it wouldn't be easy to create a company that, as she said, would "help people live an honest life and live in a nontoxic environment" by creating a variety of eco-friendly, price-conscious products. She also knew that starting the Honest Company would mean going up against huge consumer-products companies, in sectors as diverse as cleaning products, personal care, rugs, and furniture.

"People were like, 'It's way too big of an idea and I don't know how you're going to tackle it,'" said Alba. "Not one person said, sure, that's possible."

Alba admitted that her social circle is probably less familiar with high-growth entrepreneurship than some others. While celebrities are constantly shaping, leveraging, and selling their brands, most don't explore other types of entrepreneurship, Alba says. "It was so outside the scope of what anyone in my position had done in entertainment at the time."

Forming the Honest Company also meant fighting against the popular perception of Alba as a sex symbol. Alba was well aware that at least some of her naysayers were making lots of money off that version of her success.

"I wanted to make laundry detergent and baby diapers," she says. "They were like, why are you doing that? It's not sexy. It was against the way I had been marketed in entertainment, and how [people] were making money off of me."

Even Lee said no at first, recalled Alba. Lee responded that he was busy helping to run ShoeDazzle and LegalZoom at the time that Alba first approached him, and that he wasn't prepared to take on another project. But he also admitted that the birth of his first child, and his wife's resulting interest in organic products, made a difference.

"I saw the way she changed her life and how she started shopping at Whole Foods," Lee said. Then, tongue firmly in cheek: "Jessica's idea stayed in my head and I thought, maybe there's something to it. Maybe mothers do care."

Alba even had to overcome some concerns from her husband, movie producer Cash Warren, who was worried that she was spending too much on her idea. In response to an audience question, Alba said she invested in the Honest Company for three years. "I believed [the company] needed to happen," she said. "I needed to invest enough in the idea so people could see it was real. I invested in product development and quite a bit of some of the design."

That prompted a call from her husband to Lee. Recalled Lee: "I was talking to her husband and he was saying, 'She is spending all of her money. Like all of her money.'" Said Alba: "Cash knew I was passionate, but he wanted it to stop."

But it was Alba's comments on celebrity, not on entrepreneurship, that drew the biggest applause. She said investors were not terribly excited about being affiliated with her, but they had a great deal of respect for Lee. "He legitimized it," she said. "A lot of people can be a celebrity. You don't really have to have any talent to do that. You can, I don't know, let cameras follow your life."