He'd pushed some code on Friday afternoon. This move on its own wasn't the blunder. An unseen bug brought down our entire platform. It could have been catastrophic.
As a startup with lots of building going on, our development team pushes code sometimes twice a day. Most of these aren't major features. Usually just a pesky bug fix or two.
We tend to follow the "get features out in the wild and let real customers play with them" release plan. Which simply means we push a lot of imperfection. Intentionally. As customers kick the tires on any new idea we have to improve our video platform, feedback appears in real-time.
"We love this new field on the analytics page."
"This link on the new theme doesn't work."
"The change I made in the video lineup doesn't sync properly with the app."
Sometimes they just say nothing. Which could be good or bad depending on what new thing we just pushed.
In essence, we've hired our customers to play the role of a QA team. As a startup, we simply don't have the budget for the fully staffed unit. You don't go to battle with the army you wish you had, you go with who you've got.
Releasing broken or half baked code is commonplace.
Monday mornings are recognizable for us because of this all-hands meeting. We gather the whole company, all 18 of us in a rough circle near the back of our space. We each take a minute or two to talk about what success looks like for the coming week. The sales guys rarely need prompting to speak up while the tech people sometimes require a little nudging.
The session leans towards casual. Someone always shows up with the PG version of their weekend adventures. Along with our one room office, this gathering has created a community feel to our group. We like each other and pull for each other to win.
Chris sits mid-circle. He's a front end developer so success for him each week tends to take on the same face.
Release some new bells and whistles. Clean up a few nagging items. Don't break stuff.
He's understated, never taking full credit for victories. And while some sales guys tend to use this time to fluff their feathers, Chris seems to use these Monday sessions as a gateway to ease into the week.
So when Chris came out with this impromptu mea culpa, he immediately had our attention.
Chris went on, "I pushed some bug fixes on Friday afternoon without doing the right tests. I was sure I'd covered everything. But obviously not. It took down the whole of the platform for a bit. I had to bring in Al to fix it (our CTO). I couldn't sleep all weekend. And, I'm really sorry to you all."
There was a silent pause which I did not rush in to fill. How would the group handle this?
Before the silence became it's own character, a sales guy piped in. "You know, I had a real idiot move last week too. I got distracted by a couple of big accounts and didn't put in the time to build my pipeline for this week. I spent last night stressed about it. This week is going to be rough for me."
And then a New Account Specialist, "You know Chris, there were a few wobbles on Friday afternoon, but we were able to manage it. If that's the biggest screw up of your career, I think you're in good shape."
And from another developer, "Yeah -- no biggie Chris. You should have seen the hot mess I pushed last Tuesday. I'm surprised Al didn't fire me..."
As I sat back and witnessed this exchange I was filled from top to bottom. Pride. Respect. Reverence.
Chris had opened a box that leaders sometimes struggle to crack. His humble act made it OK to fail. His vulnerability endeared him to his colleagues. His courage to apologize pegged his respect meter off the charts.
As the circle made it's way to me -- I always go last -- I struggled with a punctuation for this unscripted session. I settled on this.
"Chris, thanks for that. Thanks to you all. Keep pushing hard enough to make big mistakes. Keep having each other's back when those mistakes inevitably show up. No matter what happens, this is going to be a great week."
It was a Monday to remember.