For many entrepreneurs, growing a business from scratch is akin to raising a child -- which is perhaps why so many are reluctant to let go, according to a new survey.

Of 201 business owners whose companies have at least $10 million in annual revenue, 32 percent said they have no plans for choosing a successor, according to the survey conducted by GFK Roper Public Affairs for SunTrust Bank Private Wealth Management.

While 44 percent plan to pass their business on to a family member or promote someone within the company, only 14 percent plan to sell their business to a third party.

In addition, about 70 percent of respondents expect they will remain tied to their business in some capacity -- mostly as consultants or board members -- even after handing over the reigns to new leadership, the survey found.

Five percent said they do not plan on leaving their business at all.

The nature of some businesses may affect the owner's willingness to relinquish control, according to Mindy Harvey, co-founder of Ultimate Ears, which makes custom-fitted earplugs and headphones.

"In our situation, because of the custom nature of what we do and the personal relationships with our clients, I couldn't walk away," Harvey said. "It would be heartbreaking."

And Greg Wittstock, the founder and CEO of Aquascape known as "The Pond Guy," considers himself among the 5 percent of business owners who never want to walk away. "I'm 37 years old, and I plan on them dragging me out of here when I'm 97," he said. "I've been offered a lot of money to consider selling, but I look at this as my legacy."