Have you gotten an ominous phone call in the last few weeks? Is the voice on the other end of the line insistent that you owe the IRS money? Maybe an email is in your inbox right now instructing you to click a link to settle your IRS debt.

If these scenarios sound familiar, you're likely the target of a tax scam -- and you're not the only one.

Come tax season, scammers take to their phones, computers, and any other source they can get their hands on to try and turn American citizens' fear of an audit into some quick cash. Unfortunately, it works. Since October 2013, the U.S. Treasury Inspector General for Tax Administration projects that victims have paid more than $50 million to scammers posing as IRS officials, leading to an average loss of $5,200.

I recently had a chance to speak with Bill Smith, Managing Director for CBIZ MHM's National Tax Office. He has more than 30 years of experience in both the public and private sectors, including five years in the office of General Counsel at Deloitte & Touche LLP. He told me just how widespread these tactics are, and how easy it is to fall victim to them.

"Simply put, tax scams are a very lucrative business," Smith said. "Often times, criminals try to file fraudulent returns before the matching information returns go out, whether it's W2s or 1099s. So while you're waiting to collect all your information, they're filing false claims and making up the numbers, and the IRS has nothing to match it with."

Smith says that some of these scams include requesting fake tax payments via automated messages threatening legal action. Fake notices about the Affordable Care Act (ACA), "verifying" tax return information over the phone, scammers pretending to be taxpayers, phishing, and fake charities are other common tax scams.

"The phone scams are growing bigger and bigger all the time, because it's still proving to be successful," he said. "In my experience, older people are much more susceptible, because in general they're more likely to be rule followers, don't want to be in trouble with authority, and are less tech savvy, so if they get a phone call or an email that says they owe the IRS money, they're much more likely to believe it."

New and/or complicated legislation, like the ACA, can also be a scammer's best friend.

"Nobody understands the ACA, so it makes it easier to manipulate people," Smith said. "When something is new, scammers are going to get a higher hit ratio."

Smith reiterated a fact that many Americans should be mindful of: the IRS will never call to demand immediate payment, so don't be afraid to call a potential scammer's bluff. Furthermore, electronic filings can be a deterrent, thanks to the safeguards built into the process.

"By and large, we're electronically filing now for most of our clients, and when you file the return electronically, you'll immediately get a response saying that a return has been filed in connection with a social security number," he said. "So then you have to go through the process of filing the affidavit form verifying that it wasn't you who filed it. It's a very time-consuming process, and you're not going to get your refund very quickly, but at least you'll know right away if there is a problem, and you can work towards correcting it."

So as you're prepping your tax documents for the upcoming April 15 deadline, be skeptical of anyone claiming to be the IRS, because once a scammer sinks their claws in, it's hard to wriggle out of harm's way.