In an earlier column, I laid out some ways even technically-challenged CEOs can stay informed about the tech underlying their companies, including how not to fall behind Apple's constantly evolving software. But there's more to it than that. Non-technical leaders also need to evaluate, and keep up with, the engineering team, which to an outsider can feel like trying to impersonate Jordan Spieth on the golf course.

The short version is: Speak up and apply common sense. Engineers and other technical people such as User Interface and User Experience designers can seem remote and hard to understand. But remember: They are building something for regular old people and if they can't make their work intelligible to you, chances are it won't survive in the wild.

Let's look at a few more specific ways to keep a close watch on what's going on under the hood:

Hire Humans

Obviously a potential technical hire needs to be put through a serious interview by the relevant staffers at your company. But if engineers or UX designers can't communicate, especially with a non-technical leader, you will be driven insane. Engineers love metaphors (see rabbit hole here)--a car, an assembly line, the human body, whatever -- so use one as a tool for cracking open the interview: Get your candidate to describe a recent project, or the one they are auditioning to build for you. Sometimes I will ask prospective engineers to describe a technical issue or process to me as though I am the CTO; then I will ask them to describe the same thing as if I am a five year old. If they can't do both, be careful.

Don't be shy

Use your non-technical intuition in technical settings. If your engineers are good at hitting dates, that may tell you they can estimate tasks and get them done, but says nothing about the quality of what they're building. Examine the process as a whole. Sit in on some meetings, see how they're run. One of the telltale signs for me is whether they are talking about technology or process in those meetings. If all the talk is about the deep, gnarly security issues or debating whether to build using Swift or Objective C, that's a good sign. But if it's all about process and project management--how the build gets done rather than what is getting built--that's a yellow flag.

Challenging your tech team

Preparation and rigor are great for engineers. Schedule weekly demo calls to force them show you what they are working on. That gives you visibility into their progress and puts pressure on them to deliver on a rapid cycle. The best engineers will be able to consistently put together an intelligible demo. Ask a lot of questions. Challenge their thought process.

Sync early and often

If your engineering team is way ahead of your design team, or vice versa, that can also be a telltale sign that something isn't right. Those teams need to be in lockstep. Your Project Manager is key here.

Spend time with your customers

Obviously your PM should be dealing with customers and their needs more or less constantly. But it is critical for engineers to do so as well. They have to understand what's going on in the real world. If you just keep them in a closet, coding, don't expect big innovation. Give them the mandate--and the time--to get out in the wild.

Find a technical proxy

One of the problems at Scrollmotion when I got here was that we had silos--a front end team, a back end team, an iOS team--but no one knew the whole product. Especially with a non-technical leader, it is essential to have one person at the company who knows the product technically -- line by line, spec by spec, front to back, top to bottom. Someone who can see the complete horizon, and who can be relied on to give you the bad news as well as the good.