This February, the US government appointed a former startup guru, Dhanurjay "DJ" Patil, to become its first Chief Data Scientist. The former Relate-IQ and LinkedIn employee said to Fast Company that the government was "more data-driven than most companies." The result is that there's a surge in startups helping both the government and its citizens deal with the amount of data out there.

That's the Ticket

The companies are dealing with all sorts of data-related issues, from parking ticket rules (and the government's occasional failure to deal with them properly) to information requests to the government. The Feds have thrown down a gauntlet and an admission at the same time; they've got so much data, your company might be the one to fix the problem.

Though it's hard to feel sorry for your city for giving you a parking ticket, it's almost impossible for every ticket to be fairly applied. This includes various laws behind whether a street sign is correctly shown (and a ticket fairly given), and thus Fixed was born. This is one of those companies you might've heard about that tells you if your ticket was unfairly put on your car.

For example, let's say you get a ticket because your car was parked in the wrong spot for street cleaning. Let's also say that there was inadequate signage. The app tells you a percentage chance of winning should you fight the ticket. Then, it will fight the ticket on your behalf, should you want that. It may be able to get the ticket thrown out on something as simple as the meter maid not jotting down your car's color correctly. It basically lets you keep a parking ticket lawyer in your pocket.

Managing Massive Numbers of Documents

Freedom of Information Act requests can be made by any citizen to get full or partial access to information that the US government has on file. You might've heard about this law on Discovery Channel documentaries where tireless conspiracy nuts search through endless documents on the JFK assassination. This nearly fifty-year-old law has been heralded by many because it shows that on some level the U.S. government can be slightly transparent. It all becomes difficult for the researcher however when the documents arrive. They find themselves sifting through thousands of docs in an effort to get at the ones with the right information.

Software startup Logikcull makes that easier by letting you search for and categorize such documents once you get them. The company, which raised $4 million in seed funding led by Storm Ventures, is also used by law firms to drag and drop thousands of files into the system, creating "cases" that are categorized, given OCR (optical character recognition, so you can search documents), and organized. While law firms use it to quickly speed through the analysis of entire corporations' email accounts, the cities of Chicago, Baltimore, D.C., and others use it to quickly respond to citizens' requests.

The Underbelly of the Internet

Big data has also become a problem outside of government hands. The so-called "dark web" has created potential risks for companies and their customers (banks in particular). Forums exist, and have for decades, that sell people's information, fake police badges and passwords. That's why London company Digital Shadows created a form of "search-engine for everything," both on the web we use every day and the dark web (hidden under the anonymous browser TOR), as well as search engines and other parts of the web.

The Dark Shadows interface can see illicit forums where drug sales (using anonymous cryptocurrency) might be occurring or where customer banking information is sold. It also monitors the web for scams designed to get people to give away login information. It's not a new phenomenon that criminals have used hidden parts of the web to do bad things, but Digital Shadows' program takes the formerly tedious and potentially dangerous search for criminal activity and manages it like an IT firm would. They also keep tabs on the world's hacker groups, mapping their attacks for their clients globally.

Our government's big data problem has only come into more sharp, unsettling focus in recent years, but it has created a perfect venue for companies trying to make money off of helping us manage the overload. In the same way that Marc Benioff is creating products around managing huge amounts of sales data, there are clear opportunities for hundreds of startups looking to profit from other big data problems. The most valuable companies in the world (like Palantir, now worth over $15 billion) mine data that most people may not even think about. With the right target, your next big data project could be huge.

Published on: Mar 24, 2015