It’s been an interesting year. The Cubs won the World Series, America elected its 45th president amid one of the most hotly-contested election cycles in history, and the world lost a long list of ultra-talented folks like David Bowie, Arnold Palmer, and Gene Wilder.
Amid the mêlée, Strata + Hadoop 2016 celebrated Hadoop’s 10th birthday, as well as the overall success of the Hadoop ecosystem. Here are a few of the most notable happenings of Strata + Hadoop 2016.
Real-Time Analytics Goes Mainstream
The end goal of big data processing has always been real-time analytics, but until recently there weren’t platforms actually capable of reliable real-time processing. At Strata + Hadoop, it became apparent that Kafka is delivering everything necessary for real-time analytics to be mainstream. Expect most, if not all, big data projects to involve real-time analytics, and for Kafka to become an increasingly potent force in the realm of big data.
‘Spark’-ing Interest in Artificial Intelligence
Not to be outdone, Spark made waves of its own at Strata + Hadoop 2016, showcasing the potential for predictive analytics via Marvin the robot. Not to be confused with Marvin the Paranoid Android from A Hitchhiker’s Guide to the Galaxy, this Marvin plays the game “Rock, Paper, Scissors,” or Roshambo, using Spark and predictive analytics learning algorithms. Since Marvin is capable of learning, you may beat him once, but won’t likely do it a second time. Marvin generated lots of buzz at Strata + Hadoop for the capabilities of Spark and machine learning.
The future of big data is firmly in the cloud. All of the murmuring about data ownership and security have culminated in a single fact: cloud providers tend to do a better job with data management, governance, and ownership than the companies that own the data. This has driven cloud adoption to new heights in 2016, powered largely by the strong growth of all the major cloud for big data players: AWS, Azure, and Google Compute Engine. The cloud makes it far cheaper, easier, faster, and even perhaps more secure than on-prem data centers are capable of, and that was highlighted prominently at Strata + Hadoop 2016. Expect that next year will be more about which cloud providers offer what features and less about whether or not businesses are ‘going cloud’ because they are.
Mainframes Aren’t Going Anywhere
Mainframes were processing big data before big data was big data. While much of the data collected and stored by mainframes is now being offloaded to Hadoop and other big data tools and engines, the mainframe continues to be an MVP in the world of data. Mainframes still process about 70 to 80 percent of the world’s business transactions, and remain among the most secure means available for collecting and storing data. Hence, ETL tools and products for offloading data from the mainframe into Hadoop and big data engines, are critical to the field.
Getting excited about Strata + Hadoop World 2017? You can sign up for the March session in San Jose, CA now. Additional sessions are scheduled in London (in May), New York (in September), and Singapore (in December). To keep up with all the news on Strata + Hadoop and the world of big data, follow us on Twitter.