The Mystery-Shopper Questionnaire
How Au Bon Pain boosts customer service by rewarding employees who keep on their toes
The phrase mystery shopper may call to mind those television commercials in which persnickety houseguests surreptitiously examine their friends' homes for dusty tabletops or water-spotted dishes. But Au Bon Pain Co.'s mystery shoppers are nothing like those caricatures -- they are part of an ongoing, extensive program to measure whether the company's quality and service standards are being met.
The program's emphasis is on improving performance by rewarding employees for excellence. With 1,600 people on its payroll (another 400 are employed by franchisees) and systemwide sales of $70 million in 1990, Au Bon Pain uses its mystery-shopper program -- in which anonymous non-company-affiliated customers buy a meal and fill out a questionnaire about the restaurant, the food, and the service -- as a key element in its compensation system. Bonuses for everyone from line employees to vice-presidents are based in large part on how well they treat the average customer, i.e., the mystery shopper.
Here's how the program works: The corporate office compiles the questionnaires so managers can track trends and the long-term performance of the units. Individual servers and shift managers who have scored high are rewarded with certificates that can be traded in for gifts. Managers at all levels also receive cash profit-sharing bonuses based in part on their mystery-shopper evaluations.
In the 3 years that the 10-year-old company has systematically been mystery shopping, the number of questions has been whittled down from a high of 200 to the current 60, which span a four-page questionnaire. "We're still doing some rewording," says Joy Pomeroy, the company's first store manager and current head of Retail Quality Control, "to get shoppers to pay attention to the things we want."
The program appears to be improving customer service. An average of 20 people from the 69 company-owned restaurants and express units receive rave reviews each week. That "win rate" of 25% to 30% is about double what it was when the program was launched. Overall site performance scores also have risen, from an average of 72%, to 80%.
Here, Ron Shaich, Au Bon Pain's co-chief executive, and Pomeroy explain the questionnaire and the program, and what the company gets out of them.* * *
Where the Shoppers Come From
For the 40 Boston-area locations, Pomeroy recruits shoppers through newspaper advertisements and word of mouth, employing about 30 people at a time on the team. "I look for constant turnover," she says, to help guarantee the "mystery" part of the program. A national in-store research firm, Acker, Capozzi & Petersen Inc., based in Lyndhurst, N.J., monitors the out-of-state sites.
The Boston shoppers are paid $10 a visit; Acker gets $30 for each visit. All shoppers receive a two-to-three-hour training session on the company's customer service philosophy and are given samples of the food as it should be prepared. Shoppers are then expected to spend at least 20 minutes (30 is average) during their visits.
Overall, counting internal administration, the program costs more than $150,000. Shaich says it's worth it: "The results are as important as your profit-and-loss statement" for tracking the health of the company, he maintains.* * *
Frequency of the Visits
Each company-owned store is visited three times over four weeks, during breakfast, lunch, and dinner shifts. There are more visits if a district manager is concerned about the service or cleanliness of a particular unit. "It's a battle, figuring out how much is enough," says Shaich. "Many other food companies seem to shop once a month, although some do it 30 times a week. The more information the better. The important part is that it be systematic and statistically significant."
Reading the Overall Rating
PEGS -- Product, Environment, and Great Service -- encompasses 12 specific criteria Au Bon Pain has chosen to focus on, from food availability to length of waiting time to restating the order when it's handed to the customer. They are scattered throughout the form and added up for an overall score.
Managers whose shifts score 100% receive on-the-spot bonuses of 20 "Club Excellence Dollars" -- dubbed CDs -- which can be traded like green stamps for items in a company catalog. Half of all items are in the $10-to-$70 range, from Au Bon Pain sunglasses to portable cassette players.
In addition, the PEGS scores received by shift and district managers help determine their eligibility for monthly profit-sharing cash bonuses. A score of less than 78% cancels out any bonus, while higher numbers mean more money. Shaich also uses PEGS to track performance of both individual stores and larger regions. "I'm not worried about whether one shift was good or bad," he says. "That happens to everybody. But I'm certainly interested in whether we have a systematic problem across a specific district or a store, whether it's service speed or product quality."* * *
Choosing the Right Questions
Figuring out what to target in the areas of food, environment, and service wasn't easy, says Shaich. Some questions -- such as, Does the rail in front of the counter have smudge marks on it? -- were pulled in favor of those more critical to measuring quality and service.
Also, there are limits to how much mystery shoppers can observe. They can't evaluate "house standards" -- for example, whether the chicken is being kept at 140 degrees -- which district managers check up on. And they don't interact with bakers or prep people or clean-up people, who are monitored by in-house managers.* * *
Measuring Service Quality
Individual workers -- the servers -- get 10 CD points if they score 100% in six areas of customer interaction, from an overall willingness to serve, to greeting the shopper within three seconds.
Indeed, recognition for treating a mystery shopper well is a culmination of the constant reinforcement employees receive for good customer service. Winners' names are posted on each store's bulletin board next to a list of the judging criteria, and all winners get letters of congratulation from Pomeroy and the vice-president of operations. Notices reminding employees about the service priorities also are periodically included in paychecks.
Shoppers are expected to fill out the forms surreptitiously, so the crew members won't give them special treatment; things like how long it takes to be served are supposed to be noted subtly and jotted down later. But employees look out for them. "We encourage people to wonder," says Pomeroy. "It's a good motivator."* * *
Applicability to Other Businesses
"If you've got customers, you ought to be looking at what you do through a customer's eyes," argues Shaich. "We have a catering business, run out of Boston with an 800 number, and we started using the mystery-shop approach with it. Different form, but the concept's the same: Once a week we'll have a new customer call, then we ask questions like, How many rings did it take before someone answered the phone? How many times were you put on hold? Was the customer service rep friendly? These are questions any service company should ask."
Reading the General Comments
"I don't read every evaluation individually," says Shaich. "I'm mainly looking at the trend line on the scores by district, by the company overall, and in some cases, by the store. But I'll often pick up a bunch of them and read the food reviews and general comments. I'm looking for patterns.
"The biggest decision we've made from these? We fire people. Absolutely. The purpose of the program is still to reward people and find systematic trends, but if there are real problems, we'll go in and talk to that manager. I don't think we'd ever fire anybody over one bad review, but clearly if someone's consistently performing below par, they should be fired, shouldn't they?"