Feel like you're paying too much in taxes? You're not alone. Tax season can have that effect on everyone, no matter where they reside. But residents of some states have more of a gripe than others, as brought to light in the annual WalletHub tax rate study.

WalletHub's analysts searched for answers by comparing state and local rates for all types of taxes in the 50 states and the District of Columbia against national medians, and some of the largest bills coming in the Northeastern part of the country. The study found the annual state and local taxes on households were highest in New Jersey ($10,969), Connecticut ($10,155), and New York ($9,495). In contrast, residents of Tennessee ($3,566), followed by Montana ($3,596), and Delaware ($3,852), paid the lowest amount.

In effective total state and local tax rates on U.S. households, which is the average rate at which individuals' earned income is taxed, Illinois was the highest (14.76%), which is primarily attributable to property taxes, while Alaska was the lowest (5.64%).

"For the most part, a lot of these trends have remained the same year-over-year, and that shouldn't come as a surprise for residents of these areas. For example, if you went up to someone in Connecticut, New Jersey, or New York and told them they pay high taxes, they'd say they know," said Jill Gonzalez, a spokeswoman for Evolution Finance, the parent company of WalletHub, instead pointing to the effective tax rates as more of a surprising finding. "But what is interesting is the effective rates have been creeping up over the last three-to-five years in states like Wisconsin, Nebraska, and even in Ohio, which is something that's a newer trend."

The study also ranked the 50 states rates across different demographics including political leanings. A comparison of states that usually go Democratic in Presidential elections, or "blue states" (27.48 average rank) versus those that usually break Republican, or "red states" (24.97 average rank). Unsurprisingly, states that are more prone to go blue pay more in taxes, though the difference is smaller than you might expect. When I asked Gonzalez if those paying more are happier with the public services, she said it's something that's a factor residents should consider before simply voting for the lower tax burden, because it has an impact on their day-to-day lives.

"In some sense, you get what you pay for, such as in New Jersey and Connecticut, with top tier healthcare and public school systems," Gonzalez said, acknowledging that the issue would require more study for her opinion to be comprehensive. "You could say the same thing with some Southern states that have high dropout rates and higher crime rates, so that's why there's really a reason for residents to question their level of investment and the return on those investments."

The study also looked at who pays the lowest and highest gas tax (Alaska and Pennsylvania, respectively), and the lowest and highest rate for cigarette taxes (Missouri and New York, respectively). I asked Gonzalez if she thinks residents are really aware of these types of taxes, and she was skeptical.

"I think the only time residents really become aware of those issues is when they're put on a ballot," she said. "I don't think people even notice week-to-week when there are gas price increases, so most of these issues are really brought to full awareness when people are forced to have an opinion on them, and that's usually when they're up to vote."

Still, despite all the consternation around tax season and the polarization around the tax code, Americans are surprisingly optimistic on the current structure.

"I think the most surprising thing is that, at the end of the day, people think their tax rates are, for the most part, fair, and that the IRS is something we need, even though they think it needs some work," Gonzalez said. "People are also much more in favor of a progressive tax rate - you make more, you pay more - when it's put to them in a neutral setting, but when you assign a party to it, that's when it gets the axe."

So while tax reform works its way onto Congress' docket, and residents wait to see just how their burden may be different in 2018, it seems Americans are far more toward the center of the road on this issue than they get credit for.