I had a reflex reaction to  Google Duplex.

It was not to give anyone a high-five or lean back in my chair and think about wonderful technological advances. It was more like that little ping on the back of your neck when you see a number calling on your phone that you don't recognize.

This is not good.

At its annual tech confab called  Google I/O, the search giant introduced new voices for the Google Assistant, including a few that sound like a co-worker and exactly like John Legend (because it is, in fact, John Legend). I liked the idea so far. I listened as Sundar Pichai, the Google CEO, explained how the new voices for the Google Assistant will replace the voice we all know from our Android phones and Google speakers.

And then things got really weird.

Pichai showed a demo of the Assistant bot calling a hair salon and impersonating a human, then repeated the same trick by having the bot call a restaurant (it was all pre-recorded). One robot calling one human, booking an appointment, but the human was blissfully unaware of the artificial intelligence at work. Nice. Google samples human voices and then extrapolates what to say during everyday conversations.

The robot even says "um." We've come a long way as a society, right?

But you can imagine where this will lead.

According to a recent report, robocalls are skyrocketing, mostly thanks to foreign entities that don't mind cold-calling you even if you are on a Do Not Call list.

I've noticed the uptick, and it's incredibly annoying--to the point where I don't bother looking at my phone that often and assume if someone wants to contact me he or she will send a text or wait for me to call back.

With a little ingenuity, the Assistant could be commandeered into making hundreds or thousands of calls, impersonating humans and conversing in a realistic way. We won't know the difference. This could cause real problems for legitimate businesses that have real customers they need to contact, either because the customer made an initial contact or needs a follow-up service call. And it could cause real problems for people who need to call colleagues or talk about an upcoming project with clients.

It gets a lot worse. Impersonating humans might save a few minutes if you need to book a hair appointment, but if bots can now pretend to be humans and people actually believe it, what's stopping them from tricking us in countless ways? Your child is sick at school, says the convincing female voice that seems to be the superintendent's at your local school. How about bomb scares, accident reports, divorce filings--it's all entirely possible.

You might even say this A.I. tech, which seems harmless and helpful at first, is the first step to ruining all phone calls. How would you ever trust anyone on the phone? I'm serious. The voice on the other end sounds perfectly human. We already know it's possible to make a video that impersonates President Obama (and President Trump). What's stopping this tech from falling into the wrong hands and impersonating someone you know?

So far, there are few answers. Google says the tech is in development. I'm waiting for the first robocall that tries to convince me I've won the lottery. And I don't even buy tickets.