(Coauthored with Ken Gray, Director of Innovation, Caterpillar, Inc.)

Many managers believe they have to lead by making decisions. Innovators lead by framing experiments and then getting out of the way. Managing innovation can be one of the greatest challenges any leader faces. On one hand, you are expected to make decisions about where to allocate resources and lead teams to greener pastures. On the other hand, by definition you are operating under uncertainty, and so, at best, can only guess where to lead your teams. How can you effectively lead innovation under such trying circumstances?

In our recent book, The Innovator's Method, Jeff Dyer and I argue that to manage the uncertainty of innovation, you need to shift from being the chief decision maker to becoming the chief experimenter. But what does that mean? Ken Gray, Director of Innovation at Caterpillar Inc., and I discussed these ideas and he shared his experience leading--by getting out of the way--the development of their successful 336E H hydraulic hybrid excavator. His team's experience provides profound lessons on how to become the chief experimenter and successfully lead innovation.

Let's take a trip back to January 2011, when Ken and his team had been under immense pressure for some time to develop a sustainable, diesel-electric hybrid excavator. The team was doing just that... until one meeting changed EVERYTHING, including Ken's definition of an innovative leader.

By this time, Caterpillar's design team had succeeded in demonstrating a best-in-class diesel-electric hybrid excavator that outperformed competitive hybrids from both a fuel and performance (tons/hour) standpoint. But then, in the final months before making the decision to scale up to production, Ken and his colleagues held what turned out to be a pivotal meeting with a group of competitive hybrid excavator users in Japan. The Japanese contractors sat on one side of a u-shaped room at the Hydraulic Excavator Development Center in Akashi, Japan, and the Caterpillar engineers on the other side. The Caterpillar team emphasized their commitment to sustainability and Ken asked the first contractor how he felt about the diesel-electric hybrid excavator's fuel savings and positive environmental impact.

After a moment's pause, the first customer said, "Ken, you are the product manager of the large excavator group. I have a large excavator on my job site because it has a big job to do. If I didn't need the performance of a large machine, I would buy a small one. In other words, if your machine comes to my site, it has to perform. If it can't do a big job, I'll throw it off the site." Another customer spoke up just as forcefully to emphasize the importance of reliability. Reading between the lines, Ken realized the customer was essentially saying the existing diesel-electric hybrid excavators--much like the one the Caterpillar team was developing--weren't reliable. Ken reassured the customer that the Caterpillar excavator would be incredibly reliable, and the team then asked several questions to assure themselves they understood the customer's definition of reliability. Then they asked about sustainability again.

A third contractor spoke up and said, "I want you to remember what my colleagues have said about performance and reliability. And I want you to take a step back and think about why Caterpillar buys the equipment it buys--the machine tools--to make these machines. When deciding to buy these tools you know that you have to make money within a certain payback period. Right now, my hands-on experience tells me the diesel-electric hybrid excavator payback period is about seven years, but I only have work scheduled for maybe two years. I want to be sustainable, of course, but seven years is just too long. I have to make money, too."

Ken and his team nodded in acknowledgement. They were worried. They had spent millions developing a diesel-electric hybrid excavator. Caterpillar is a results-oriented company. Although they had talked with Caterpillar customers during the entire development process, the message had never quite come across this way... this strongly. Still grasping the mandate they had received to develop a sustainable product, Ken asked one more time about sustainability.

After a pause, still another contractor spoke up: "We don't want to talk about sustainability. I want you to remember what my friends have said about performance, reliability, and profitability. If you are developing a diesel-electric hybrid excavator, don't waste my time or yours." Silence hovered in the room. Ken felt sick... like he had been kicked in the stomach. These customers had said in no uncertain terms that the choice Caterpillar had made to develop a diesel-electric hybrid excavator was wrong.

For a moment, Ken considered that his career at the company might be over, or at least severely limited. His team had made a significant investment, but the right thing might be to set their work aside. Soon after the meeting, a trusted engineer reminded Ken, "I know it sounds bad, but you know there is another way to do this." As Ken considered the personal and financial losses of stopping the project and the risk to his own career and others', he wondered what to do next. Many in leadership above Ken, who had not attended the meeting, still believed that Caterpillar needed to launch the diesel-electric hybrid excavator and were pushing hard. To follow the herd would have been the safest thing to do, particularly from a career point of view. Should he follow their lead and risk more losses, or abandon a project with significant investments already made?

The chief decision maker would attempt to guess the right course and then execute. Instead, Ken realized the low cost experiment a team of passionate engineers had been running might be the answer. In a sequestered location 'on the other side of' the Illinois River, he and a colleague had given a passionate team an old prototype machine and spare parts, but virtually no budget. The team often worked nights and weekends. They worked fast and dirty, attaching parts to the exterior of the machine rather than integrating them. The operating prototype--technically a proof of concept machine--had hoses and components attached in plain site and was so ugly the team called it "Medusa." But in a few short months, this team proved that, by leveraging technology that had been under development and shelved for years, they could create a hydraulic hybrid excavator that met all their customer requirements for performance and reliability, while lowering owning and operating costs, and improving sustainability.

So, that ugly prototype Medusa... it ended up becoming the answer to both those customers' requests and Ken and his team's prayers. It was a cheap experiment that led to the final hydraulic hybrid excavator. And that hydraulic hybrid excavator was a show stopper. It consumed 25% less fuel, was 50%-75% more efficient than traditional diesel excavators, performed just as well, and has become a major new product for Caterpillar.

Excavators represent roughly 40% of the construction equipment sold on a unit basis globally. It is a challenge to produce something novel for such a ubiquitous machine, but since its April 2013 launch, the 336E H hydraulic hybrid excavator has grown to nearly 1/3 of the machines Caterpillar sells in this size class. In stark contrast to its competitors, Caterpillar's new, industry-leading hydraulic hybrids save energy using hydraulic accumulators, improved engine efficiency, and a novel hydraulic system. In a future post, Ken and I will share more about how his team created this new innovation. In the meantime, two important lessons stand out:

Lessons Learned:

  • Because leaders are operating under uncertainty and come from a strong operating perspective, often they cannot accurately decide which way to go. If you try to play the role of decision maker, you will stumble. Instead, frame up a low cost experiment that tests the most critical assumption. Successful innovators play the role of chief experimenter by helping their teams frame the right assumption to test and then pushing the experiment. They provide just enough resources and pressure to move quickly and offer a haven for passionate inventors to complete their experiments.
  • Every corporation I know talks to customers to some degree. Ken had been in constant dialogue with customers. Why then did the team discover the bad news so late? Because they had only talked to Caterpillar customers to validate the views they already held. Instead they needed to talk to non-customers, those who saw the world completely differently. The lesson: when you validate demand, talk to non-users and non-customers as well as customers.