I was serving on an expert panel at a recent innovation conference and an attendee asked about the Semmelweis effect, the tendency for people to reject new evidence that contradicts established beliefs. He wanted to know how aspiring innovators can overcome inherent bias against new ideas. 

The effect gets its name from the story of Ignaz Semmelweis, the Hungarian doctor who pioneered hand washing to prevent infections in hospitals during the 1840s. However, he was unable to get the medical establishment to accept his idea and thousands, if not millions, died unnecessarily because of it.

The Semmelweis effect is very real. We do get trapped in existing paradigms and that often blinds us to important new information. The Semmelweis story, however, is considerably more nuanced than most people give it credit for. The truth is that much of the blame falls on Semmelweis himself. The real story shows how we can overcome resistance to new ideas.

The Tragic Tale Of Ignaz Semmelweis

In the 1840's, Semmelweis was a doctor at the obstetric ward of Vienna General Hospital. Appalled by the number of deaths from childbed fever, he dedicated himself to finding a solution and quickly noticed an interesting fact. The death rates in the hospital's second clinic, staffed with midwives, was significantly lower than in the first, staffed by medical students.

His key insight came when his friend, Jakob Kolletschka, was pricked with a scalpel during an autopsy and soon came down with symptoms much like the women in the maternity ward. Semmelweis inferred that the medical students must be transferring "cadaverous particles" to the maternity patients and instituted a strict regime of hand washing. Mortality rates fell dramatically.

Yet instead of being lauded for his accomplishment, Semmelweis soon found himself castigated and considered a quack. Part of the problem was that Semmelweis's ideas about hand washing conflicted with the prevailing miasma theory of the day. It was widely thought at the time that "bad airs", not bacteria, caused disease. So hand washing simply didn't make any sense to the medical profession at the time.

Frustrated, Semmelweis wrote angry letters to prominent physicians and that, as well as political events at the time, led to his dismissal from the hospital. His mental state steadily declined and he was eventually confined to a medical institution, where he died, in morbid irony, from an infection he contracted under care.

What Really Happened

The basic facts of the Semmelweis story are true. He did, in fact, discover that hand washing can prevent infections in hospitals. The medical establishment, for its part, was not as receptive to his findings as it should have been, largely because of its commitment to the miasma theory. Old paradigms can be stubborn.

However, as Sherwin Nuland explains in The Doctor's Plague, there's more to the story than first meets the eye. Semmelweis, thinking his results were clear enough, didn't see the value in communicating his work effectively, formatting his publications clearly or even collecting data in a manner that would gain his ideas greater acceptance.

Luckily, those that came later, like Louis Pasteur, Joseph Lister and Robert Koch were more attentive and helped establish the germ theory of disease. The truth is that ideas alone, even breakthrough ideas, rarely amount to much. Innovations need to be communicated effectively if they are to spread and make an impact on society.

I've studied hundreds of innovators. Each and every one encountered resistance. The ones that succeeded were not necessarily smarter or more talented than the others, but learned to overcome obstacles they found in their path. In the end, that's what really makes the difference.

A Modern Day Semmelweis

Jim Allison spent most of his life as a fairly ordinary bench scientist and that's all he really wanted to be. He told me once that he "just liked figuring things out" and by doing so, he gained some level of prominence in the field of immunology, making discoveries that were primarily of interest to other immunologists.

His path diverged when he began to research the ability of our immune system to fight cancer. Using a novel approach, he was able to show amazing results in mice. "The tumors just melted away," he told me. Excited, he practical ran to tell pharmaceutical companies about his idea and get them to invest in his research.

Unfortunately, much like in the case of Semmelweis, they were not impressed. The problem wasn't that they didn't understand Jim's idea, but that they had already invested -- and lost -- billions of dollars on similar ideas. Hundreds of trials had been undertaken on immunological approaches to cancer and there hadn't been one real success.

Nonetheless, Jim persevered. He collected more data, pounded the pavement and made his case. It took three years, but he eventually got a small biotech company to invest in his idea and cancer immunotherapy is now considered to be a miracle cure. Tens of thousands of people are alive today because Jim had the courage and perseverance to stick it out.

Innovators Must Hold Themselves Accountable

By far the most common thing I hear from executives around the world is that they feel that no one is willing to listen to their ideas. Yes, that's frustrating, but it's also part of life. People don't give as much attention to our ideas because they are busy doing other things, like working their own ideas.

Even after Jim's idea was accepted, successfully navigated clinical trials and was approved by the FDA, he found that there were still significant obstacles in his path. As it turned out, the treatment was effective in less than 30% of patients. However, others had ideas that Jim could use to improve results. He readily adopted them and today, at the age of 68, he still goes to his lab every day, collaborating with scientists around the world to save lives.

To create a real impact on the world is no simple thing. Innovation is never a single event, but a process of discovery, engineering and transformation and those things rarely happen in the same place. That's why effective innovators are great collaborators, they get their ideas accepted by adopting the ideas of others, combining them with their own and showing that they can make an impact on the real world.

Things that change the world always arrive out of context, for the simple reason that the world hasn't changed yet. That's why innovation needs communication, because the world is a busy place, with lots of things demanding attention. We can't expect it to stop simply because we feel we have something to say.

The truth is that we need more Jim Allisons and fewer Ignaz Semmelweises. Innovation takes more than having ideas and expecting others to immediately accept them. If your idea is important enough, then it is your job to take responsibility for it and see it through.