Reference calls. We all have to make them. Whether you're considering hiring a new employee. Or if you're an investor, and you're looking to do a background check on the founders of a company.
My friend Jason Hirschhorn Tweeted about this today
References on hires are often useless.-; Jason Hirschhorn (@JasonHirschhorn) April 6, 2014
Here are some pointers that I've learned over the years:
1. Ask for at least 5 references
As your candidate for at least 5 references. You specifically want to ask for people who have directly worked with the person before. I like to get a mix of people who have reported to the candidate, whom the candidate has reported to (2x) and peers.
Don't worry about the fact that these are the references that the candidate has hand-picked. That's part of the process. I would start by asking the candidate, "How did you decide on these five people?" as part of your review process. You also should have mapped out their key employers. If they chose not to list their boss at one of them, that obviously forms a source of discussion for you (and possibly a person you might be more interested in calling).
2. Call each of the 5 references looking for "glowing" feedback
You should expect to get glowing references unless the candidate has told you in advance something like, "Look, I'm listing Susan as a reference. We had some great moments working together and some tough ones. I think you'll get a balanced perspective from her but I wanted you to get that anyways. Know that we had a conflict over "x" and that may come out."
- But that kind of reference being listed are 0.05 percent of the cases. Usually they are hand-selected to say the best possible things. If they don't here's what it tells you:
- The candidate isn't self aware of how others perceive him or her.
- The candidate isn't thorough enough to have pre-briefed his candidates to know the areas he would like covered. While on one hand you almost wouldn't want your reference calls to be "briefed," wouldn't you want to hire somebody professional enough to know the importance of doing so?
- The person lacks some judgment in whom they listed.
By the way, I also strongly advise that entrepreneurs reference VCs. In this instance my personal style is to have entrepreneurs whom I'm considering investing in to call my good references and those where we have gone through some tough times so in that sense I end up in the 0.05 percent, but I tell entrepreneurs I'm doing this and why. It's important to me that I have a really open relationship with entrepreneurs with whom I work, so I prefer that they know the good and the bad.
3. Know that all of your questions will go back to the source
One thing that isn't obvious to somebody who hasn't done a lot of referencing is that everything you discuss will go straight back to the candidate -- I promise. If you relay a story that during interviewing some of your colleagues found the candidate to be arrogant and you're "wondering whether that was a problem when he worked with you" that exact conversation we be relayed to the candidate.
You might not always care but it is important that you frame questions in a way that you feel won't make the candidate feel bad about ultimately joining your organization. Every statement of your "on list" reference calls should be made with the "everything I say will go back to the candidate" mentality. Once you're comfortable with that, ask away!
4. Seek "disconfirming evidence"
The strange thing about most reference calls is that by the time the hiring manager is calling she is often already pre-disposed to hiring the candidate. Most people delay reference calls until that point both due to expediency of time (why make phone calls unless you think you might hire the person?) and in fairness to the employee (why call a bunch of people and then not hire him -- leading others to wonder why he didn't get the job?).
I agree with the goal of waiting until late in the process. But the problem is that by the time you actually call people you really WANT to hire the candidate. So often people who do reference ask softball questions. That's not your job. Your job is to seek "disconfirming evidence" meaning you go in with the assumption that Stacy is great, but you want to be sure there isn't something you totally missed.
I usually try to disarm the person I'm calling this way. I usually start with, "Obviously Stacy has made it far into the process or I wouldn't be calling you. Many of the people at our company who interviewed her really like her. I just want to be sure that I get the full picture from her previous employers and make sure we didn't miss anything. Plus, if she does have areas where we could coach her on improvement, it would help to know that coming in."
5. "Everybody has negatives"
So you've started that glowing interview and learned that Stacy was well-liked, independent, achievement oriented, blah blah blah.
You expected no less. My next pivot after writing down all the good stuff (which of course is actually usual to know) is to say, "Those are some great strengths and some of the reasons we liked Stacy so much, which is great to have confirmed by you. Obviously everybody has some areas for improvement. It would be helpful to know these coming in so that if we do hire her we understand where to focus before she joins. What are Stacy's areas we should focus on?"
6. "Here is what some people have said"
After two or three reference calls you should have a short list of some things that Stacy needs to improve on. In more than 70 percent cases where you have done a good job in your interviews, these will sound minor and in fact probably will be. But your job is to avoid having to correct PURE mistakes (previously undetected recruiting errors), and it's significantly less costly if you catch in in the recruiting process rather than afterwards.
My strategy is to deflect the hard questions from me and project them on to an anonymous person asking the question, as in, "What some people have said about Stacy's performance is that while she is very achievement-oriented she doesn't do that well in team environments. Some people said she's a great individual contributor, but if we hire her we should be careful not to have her lead too many people on her team. Have you found this to be the case?"
It's easier for them to respond to an allegation that came from somebody else who worked with that person. Also when that reference calls, your candidate and talks about the "teamwork problem" it doesn't sound like a concern you have but one that was surfaces in a previous employer. You don't want that employee to join you already feeling judged.
7. Getting off-list references
The most obvious thing you need to do in order to better reference check people is to get "off list" references. For each person that the candidate gave you that was "on list" ask that person when you speak to them for two other people they think you should speak with in order to get a fuller picture of the candidate. You should be able to spider out pretty quickly to figure out who else can provide a reference.
Stating the obvious: Off-list references are less likely to be "friends" with the candidate, less likely to be scripted and less likely to feel accountable since it isn't clear they're on the short list of people the candidate has given.
Another strategy I have used is to go to LinkedIn and search through people who used to work for the company as a way of surfacing up old names of people either to be references or to suggest references.
Finally, if the person has some degree of seniority you should definitely make off-list references from general people in the community. You can ask VCs, customers, competitors or somebody who had a similar job function in the same general community. CFOs often know the other CFOs and same for heads of sales, marketing, tech, etc. And they have a general sense of reputation.
8. Having an accomplice
Another great technique is to have an accomplice make some of the tough calls. One obvious example is that HR can make a few calls that the hiring manager doesn't make. One reason I've found this to be effective is that it is far easier to maintain a good working relationship with the candidate when somebody else who made the calls asks the tough questions (because as per point three all information travels back to the candidate).
I know that in later-stage growth equity deals some firms even hire third-parties who will do the reference calls. I have been part of one deal where this happened, and the level of detail from a professional reference checker was unbelievable. This lady who made the calls was very sophisticated and very thorough in information gathering. This is when I realized just how helpful it can be to have somebody else making some of the calls.
9. A word on recruiter references
In point four under "seek disconfirming evidence" I talked about the problem of making the calls at the end of your search when emotionally you WANT to hear good news because you're positively biased or you wouldn't be making the calls. I have found this to be especially true when working with recruiting firms. I'm not questioning the integrity of recruiters. I have many great relationships with recruiting firms. I'm simply pointing out that sometimes when a recruiter is at the end of his or her project, one might not ask quite as tough of questions as might be asked at the start of a process.
I talked about that a little bit on this post about being careful about middle men in negotiations. In that post I wrote:
If you've never read Freakonomics you need to. One thing I learned from the book is that real estate agents always sell their personal property at higher relative prices than their clients' properties. They hold out. They wait for a better offer. They're not in a rush. When they rep you, the marginal cost of them trying to get you a slightly better deal is high for them relative to settling and moving on.
I think for most of us this is intuitive.
The same is true about references. When it's going to be somebody you need to work with daily going forward you're always going to be that extra bit more discerning than a third-party reference. And frankly, why not both do references?
10. How to interpret references
Finally, I'd like to point out that you need to be careful about how you actually interpret references. If you get somebody who doesn't say totally glowing things about your candidate make sure:
- You think critically about whether that person may have biases that led him to the conclusions he has about your candidates
- Make sure to ask other people about those specific qualities that the reference said weren't good -- even if you have to call back people with whom you're already spoken
- Understand whether any negative information is something that would stop you from wanting to hire the person. Everybody has weaknesses.
The things that are always non-starters for me? People with attitude problems. Negative people. People who cause conflicts unnecessarily. People I perceive as not having good moral compasses.
This article was originally published on Mark Suster's blog, Both Sides of the Table.