Browser Beware

Tweet

BizRate.com's ratings may seem free of commercial taint, since companies can't pay to influence the surveys, but there is still room for murkiness. For instance, a merchant's rating is based half on its front-end store and half on its back-end fulfillment, so stores with questionable practices can still receive a rating of 5.0 and above if they merely have a nice storefront.

"It can be difficult to tell exactly what the numbers mean," says Preston Gralla, author of The Complete Idiot's Guide to Online Shopping. "What's an 8.4 versus a 9.5? Those numbers might be based on criteria that aren't important to you." Gralla advises that shoppers take advantage of a BizRate.com feature that allows consumers to rank merchants according to specific criteria, such as on-time delivery. "If a merchant has 91% on-time delivery, that means something," says Gralla.

On Gomez.com, another online ratings site, it's especially difficult for consumers to assess whether a merchant's score reflects variables that are important to them, and for a very good reason: Gomez.com doesn't disclose all of the details about the criteria it uses to judge sites. "That would be like Coke giving away its special formula," says Julio Gomez, chairman and CEO of Gomez Inc., in Waltham, Mass. "For each of the 75 industries that we cover, we have about a dozen criteria that we use to judge each site." Gomez engages 30 industry analysts to generate the ratings rather than relying on consumers. (Gomez.com does publish basic information about its overall methodology, however.) In addition to running its own ratings site, Gomez.com provides merchant-ratings information to online comparison-shopping and consumer-information sites like MySimon.com, DealTime.com, and CNET.com.

What online shoppers who come across a Gomez.com rating may not realize is that the Gomez score doesn't take into account any aspect of the fulfillment experience. According to Julio Gomez, sites are scored only on the quality of the online experience they present to shoppers and on whether they have certain policies, such as accepting returned merchandise. So under Gomez.com's system, it's possible for a merchant with a slick online presence but questionable or even shabby business practices to receive a rating of two or even three stars out of three possible stars, as Jehremy Foster discovered when he encountered Family Photo and Video's two-star Gomez rating.

CEO Gomez answers such criticism by saying that his company concentrates on rating the consumer's online experience, period. "We're very focused on what we do. We are just about the customer experience on the Internet," he says. His company doesn't rate a merchant's fulfillment performance, he says, because it wouldn't be possible for his analysts to order products from all the 6,500 online merchants that are Gomez.com certified. "If someone out there would like to do that, they're welcome to it," he says.

As it happens, Julio Gomez's former employer, Forrester Research Inc., a Cambridge, Mass., market-research firm, appears to be doing just that. Forrester offers a free online consumer service called PowerRankings, which combines extensive consumer-survey data with expert analysis to rank the top E-commerce sites on the Web. According to Tom Rhinelander, research director at Forrester, no company can pay to get on (or off) the PowerRankings list.

Here's how the ranking system works: Forrester commissions a survey provider and research house (currently the NPD Group) to poll 20,000 customers of online E-tail and travel sites. From that data, Forrester analysts identify E-commerce sites that contain enough consumer data to justify inclusion in the PowerRankings. Since its resources are limited, Forrester ranks no more than 10 companies in each of the 13 industries that its rankings cover, which means that shoppers are unlikely to find unfamiliar merchants on the list. Forrester's analysts open accounts with, and shop at, the sites in question, placing real orders for products; they then return all the merchandise.

Based on those responses, Forrester prepares the rankings, giving the consumer-opinion data two-thirds weight and the in-house expert opinion one-third. "We believe the consumer data should carry more weight than expert opinion. That's part of our methodology," Rhinelander says.

The PowerRankings list isn't wholly free of the whiff of commercialism, since some of the companies on the list are Forrester's clients. But Rhinelander says the company's clients aren't able to influence the ratings, even if they want to. Clients sometimes "call up and moan about the rankings," he says. "The only way that will get any traction is if we get something wrong, and that can happen."

For consumers who are looking for a beacon of brand-name credibility on the Internet, there's always the Reliability Seal of the Better Business Bureau OnLine (BBBOnLine). BBBOnLine is drawing on the bureau's nearly 90 years of off-line credibility, making it a name to believe in, if not blindly trust. Consumers can search BBBOnLine to see if a particular merchant has received the organization's certification. In order to achieve the Reliability Seal, online businesses must have been in business for at least a year and be members of their local bureau.

The difference between BBBOnLine and other ratings services is that the companies that get the Reliability Seal must have already met certain qualifications. With straight sponsorship sites, virtually any company can pay to be listed. BBBOnLine has credibility because local bureau chapters are well situated to vouch for the legitimacy of businesses in their areas, according to Steve Salter, director of operations and administration for BBBOnLine, in Arlington, Va. Usually, local bureaus make an in-person visit to the companies that apply for membership, he says, which weeds out many of the worst types of operations.

 PREV  1 | 2 | 3 | 4  NEXT 

Read more:

Sign-up for our Technology Newsletter

Email Address: