One of the most satisfying parts of being an columnist is feedback from readers. And the column “How Entrepreneurs Hire” drew an especially strong reaction. The photo paired with my column showed a woman in a suit shaking hands with a man, who presumably is the interviewer. “Wearing a monkey suit is a big mistake!” wrote one reader. That made me stop and think. This is a complex issue – more complex than it might first appear.

In some ways, it is discouraging that attire still matters so much, rather than our skills and experience. I’m sure part of it is just human nature: We make judgments based on the cues available, sometimes in a harsh way.

But I have a friend who bombed her interview at one well-known Silicon Valley tech company because she wore a suit. She had just returned from working in the more formal business cultures of Europe and India for a number of years, so to her, it made sense to wear a suit to a job interview. That darn suit was a gigantic distraction during the whole interview process. The folks on the other side of the table decided she was “too uptight” just based on the way she was dressed. Ouch!

Many small companies and start-ups have a very casual atmosphere, especially in California. However, jeans, flip-flops, and a t-shirt are not the answer to all situations. The truth is, there is no single right answer, but there are definitely times when you need to step it up—maybe for an interview. You also need to do some homework on company culture and think carefully about the role you would play. This goes not just for job seekers, but also for consultants and suppliers looking to do business with other companies.

A monkey suit is often over the top.  But there are at least three situations where you need to face the “suit or not to suit” decision head-on. And in these situations, if you decide against a suit, you still need to dress business casual. Leave the jeans and hoodie at home.

1. Customers. If you are visiting customers for sales or business development, think about the customer’s perception of your company when they see you. Do you inspire confidence? Professionalism? Discipline and integrity? You are the face of your company. You need to look like you can deliver. You don’t want the customer to feel like they’re taking a risk on you.

2. Fundraising. Not only do you need to inspire confidence when talking to investors, but you also need to communicate fiscal responsibility, steadiness, and dependability. Again, it depends on exactly who you’re meeting. While I’ve met young angel investors (often former Googlers) who wear jeans, most investors are from a generation that expects something more formal, especially if they are on the East Coast. Investors can make harsh judgments very quickly. I’ve seen investors write entrepreneurs off as “emotionally unstable” for sporting an unnatural hair color.

3. Meetings with overseas clients or investors. In my experience, business is generally more formal in Europe and Asia. So I definitely pack the monkey suit for business trips overseas. I generally dress more formally when I have overseas visitors come to see me in the U.S., but sometimes the visitors themselves are looking forward to a more relaxed atmosphere when they come to California. I’m always ready to ditch –or add—the jacket as needed. The goal is for visitors to be comfortable.

And yes, there is a double standard. Jeans, flip-flops, and a T-shirt might work for Mark Zuckerberg’s public image, but I’ve never seen Sheryl Sandberg photographed that way. Just accept it—there is no gender equality on this issue. Women need to look smart, professional, and on top of their game. That doesn’t always mean a suit, but the bar is higher.

It can also be more complicated. I no longer wear skirts on customer visits, in case the customer wants to show me something in a cleanroom. Skirts just don’t fit well under bunny suits.

The fact is that while we claim to celebrate those who ‘Think Different,” we’re still judged by superficial standards the moment we walk into a room. The key is to pay attention and walk the fine line so you don’t have to worry about sabotaging yourself.