Failure is hard. But you know what? There are real payoffs if you can figure out how to get over the fear of it, how to live with it, and ultimately, how to use it to push ahead

In my experience, one of the secrets to success is to embrace failure.

Now, I'm not talking about the kind of failure that's psychological, a state of mind. Nor am I talking about setting low expectations. I'm talking about real failure. Failure so close you can taste it, so strong that your clothes reek of it -- the kind of failure that makes people pretend they don't know you. That sort of failure is the best thing that can happen to you if you want to run your company creatively.

I learned that lesson when I was 23. I had a great job: I was assistant to the head of programming at ABC, and I ran the whole department while my boss was out creating new shows like The Flying Nun. I was sure my career would always be in administration. I would work for this talented guy, and as he moved onward and upward, I would follow him, happily making everything neat and clean so that the creative types could thrive. Never did I imagine that I could become one of them. Even though the programming department was the exciting place to be in television, it was much too scary on that side of the hall.

It seemed so much better where I was: I felt so comfortable, with so much power already at such a young age and with nothing ever to risk. My fear of putting myself on the line -- of risking the inevitable ridicule when you put forth new ideas -- prevented me from even thinking I might be qualified for creative work. The loftiest goal I would permit myself, my top fantasy, was that I'd be the best damn clerk in the world.

One fateful day, though, my boss threw me a script and said, "Read this and tell the producer what you think." I was terrified. That producer was the emperor of television -- he had something like 11 shows on the air.

I studied the script. I hated it. When the producer cornered me, I croaked out my opinion -- an inarticulate, incompetent response. The guy let me have it, up one side and down the other. But thank God, I'd gotten my feet wet. I was in the water -- hardly courageously, but in there nonetheless, barely able to breathe and trying to defend my opinion with whatever ideas I could come up with. And I didn't drown. That's when I discovered the secret to success: Plunge into the uncomfortable; push, or be lucky enough to have someone push you, beyond your fears and your sense of limitations. And that's what I've been doing ever since, overcoming my discomfort as I go along.

From ABC I moved to Paramount. This time I started at the top, as chairman of the board. But I was still the underdog, the first person at the studio to have come from the perceived ghetto of television. The man who ran Paramount's parent company, who had taken such a risk in hiring me, was being pressured daily to get rid of me. Outsiders said that Paramount was the last stop for scripts and new material, seventh on a list of seven companies -- until we did Saturday Night Fever, which propelled Paramount from seventh to first place.

I learned many things at Paramount, but one lesson stands out: Maybe it's a good thing not to be the first stop, getting what everyone else thinks is the best material. Maybe nobody really knows what's best. Maybe the best comes from making your own choice, on its merits, not its bloodlines. Maybe it's better to be uncomfortable, and to be left alone to believe in what you can put together based on your own judgment.

I took that lesson with me to Fox. This time when outsiders predicted failure, I welcomed it. All they knew was that fourth networks had always failed before and that even the Big Three were suffering a depressed ad market at the time. What they didn't know, what they couldn't know, was that we weren't interested in creating a fourth network. We were inventing an alternative.

They based their conclusion on one set of expectations. Meanwhile we were free to operate on another. But we didn't find our voice immediately. The truth is, our first shows were horrible. We were losing, just as everyone had expected. But we were also working it out, figuring out a process as we went along.

What all my experiences have had in common is a battle, a holy war if you will, between process and expertise. Expertise is a pack mentality that concludes something can't be done, or that it must be done this way. It's a mentality that relies too heavily on conventional wisdom. It has to. Because the awkward alternative would be to accept that a new thing can't be fully known or comfortably understood. Conventional wisdom, by definition, favors that which has come before, that which is known. That's great if you're building a house or flying a plane. But it's useless, and much worse, dangerously misleading, in creative positions.

Process, on the other hand, is ignoring the doomsayers and optimists alike. None of them matters. Process is fundamentally a human function. It can't be duplicated or automated. It's about finding a grain of an idea and following that through to its conclusion. And process can't be forced or rushed. It works for everyone, not just the four or five real geniuses out there. For them, God bless them, instinct is enough. For the rest of us, there's process.

What is this process? If I could put it clearly, I would. But I can only point to it. I know it when it happens, and you do too. It's a kind of tough yet tender anarchy. Creativity always starts off with new, half-baked ideas, with half-coherent musings.

And for me, at least, the fun is in the process. I never find great joy once the work is finished and everyone else starts cheering. Sure, victory is sweet. But it's just a moment. The absolute best time for me is always that period just before the victory, when you know the work is good and it's your secret. Even that is just a moment. When you're in the process, in the groove of the work itself in all its twists and turns, its mess, its mistakes, the nightmare days and anguished nights when you're working it through -- that's sustainable excitement. It's the journey. No one remembers myths for their endings -- we remember the epic adventures along the way. When you're propelled by curiosity and the claims of a self that dares to fail, that's the process.

Barry Diller, the former chairman of Paramount Pictures, Fox Inc., and QVC Inc., is chairman and CEO of Silver King Communications, the nation's sixth-largest broadcast group. This column was adapted from a commencement address he gave at Northwestern University's Kellogg Graduate School of Management.