Is the End Game of Big Data Actually … Little Data?

on November 30th, 2016
Big data

Big data is helping to cure some of the world’s most deadly diseases. It’s helping fight poverty in some of the poorest, most underprivileged areas of the globe. It’s revolutionizing how cities are run, assisting with every aspect of municipal management from alleviating traffic congestion to planning for future infrastructure.

Big data sells our wares, matches us to the perfect vacation spot, and even identifies the content we’d be most likely to read and sends it scrolling across our Facebook feeds. From the most monumental facets of our lives to the most inanely trivial, big data is making tremendous changes.

How We Get Little Data

But behind the big — how to feed the populations of an entire country devastated by earthquake or how to manage the relocation of hundreds of thousands of war refugees — is the little data. Little data comes from things like our Fitbits that help us meet and maintain our fitness goals. Apps that help us find a restaurant that’s open when we’ve worked until unholy hours of the night, and browsers that help us figure out the rest of the name of the book we want by filling in the search bar with the information we couldn’t remember. All that is little data; data on each of us, personally, that together comprises what we call big data.

Big data builds companies and cities and nations. Little data helps us manage our daily lives, though with a bit of intrusion. Google and Apple and Netflix know where we sleep, what we eat for breakfast, what route we take to work, and what movies we watch when our kids aren’t around.

How We Use Little Data

Little data, however, and the inferences we draw from it can be deceptive. Often, that’s because we’re trying to make a lot of conclusions based on metrics that don’t necessarily indicate cause and effect. For example, say your Fitbit indicates you’ve only burned 2,600 calories today and your goal is 3,000. You might hop off the subway a few blocks early or maybe take an extra long walk with the dog after dinner. But does that really mean that you’re a healthier person? Maybe something else — like your weekly indulgence in a cheeseburger or that occasional after-dinner cigar is more harmful to your health than burning a few hundred extra calories per day.

Little data usually involves tracking one such metric and making pretty far-reaching conclusions out of it. Often, little data and big data are right on the money. Sometimes, though, a single metric isn’t enough to make decisions with far-reaching consequences.

Why We Like Little Data

People are still rather uncomfortable with the little data derived from big data. They don’t like the fact that Netflix presumes to know that you’ll probably like the Divergent movies because you enjoyed Hunger Games or that Amazon recommends you order the House of Cards DVDs because you gave Game of Thrones a five-star review. But even when it gives us the creeps, we still keep clicking for more.

Once we’ve gotten over our aversion to big data knowing more about us than our parents and spouses and best friends ever could, the ease and convenience of scheduling apps, streaming music, and product recommendations easily wins us over. Hello, Little Data, here we are!

To learn even more than you ever wanted to know about big data, follow us on Twitter.