For the last five years, the term "big data" has evoked heady dreams of transforming business as we know it. The use cases were as broad and brash as as the imaginations of senior executives. Data scientists became rockstars and could name their own salaries. New approaches to storage and analytics, including distributed workloads on commodity hardware with open-source software, meant the performance and economics of extracting insights from data was radically shifted. The future's so bright, we've got to wear shades.
So now what? If ESG research is correct, and it always is, then the market now shifts from dreaming to working. Oh, wait, you thought these conditions lead to instant results? Sorry, but no, we have to the do the heavy lifting of implementation, administration, and support of the enterprise appetite for data and analytics. Businesses will evaluate technology on characteristics like security, reliability, availability, ease of administration, and TCO — same as they always have for mission critical enterprise systems. Shh, IT gets real now. This is going to spur a change in the way the vendors interact with prospective buyers.
Big Data Technology Companies
Upstart big data companies will emphasize their domain specific expertise and differentiation. "Good enough" isn't. Not only were we first on the bandwagon, we invented the wagon, so we know best. This cohort includes Cloudera, MapR, Hortonworks (all three for Hadoop), Databricks (Spark), Confluent (Kafka), Data Artisans (Flink), Continuum (Python), DataStax (Cassandra), etc. These folks will have to push ever harder to stay ahead of the curve. Time is the great equalizer, though, and it becomes harder to look different. How many companies have had their second product line be the same kind of breakout success as their first? No, they just have to keep proving themselves as the best and brightest, and worth the perceived risk. Some will succeed, some will run out of money, some will drift out of the spotlight.
Traditional database and data warehouse titans will emphasize their "big biz" strengths with their extensive install bases. Think Oracle, IBM, Microsoft, HPE, Teradata, and Dell here. Don't change horses mid-streaming analytics, they will say. Trust us, you already buy from us, we've supported you well to date, and we can deliver robustness, maturity, and services to work with what you are already do. This argument has a lot of merit. If it ain't broke, just augment it. Not always exciting, but certainly pragmatic.
Big Data Channel Partners
Then there are the service providers, systems integrators, and traditional resellers. They sell the pickaxes, whiskey, and boarding house beds to the '49er gold miners. Unfortunately, this group is usually market following, not market building. They respond to demand, standardize, drive out costs, and focus on efficient delivery. Not innovation. Innovation is costly, risky, time-intensive, and that's not how this crew looks for profits. Big data market maturity will be when this ecosystem comes fully online. We're still limited mostly to smaller, specialized consultants now, but they'll like get acquired by the usual suspects in time.
So, in all, same as it ever was in IT. I wonder how many more breathless articles the business press can publish about the miracles of big data before it just seems like boring old IT again. The lifecycle of a new industry looks a lot like the stages of the past. Still plenty of fun and excitement out there, and many, many implementations will indeed have a profound impact on each specific business, but we're beginning to see life settle back into the same patterns from an industry landscape point of view. The real wildcards and trick plays will be where big data intersects other complementary mega-trends like cloud and IoT. That'll be where the action goes next, with Amazon Web Services, Google Cloud Platform, Microsoft Azure, and then dozens of operational tech companies. Stay tuned.
Nik Rouda is a senior analyst at ESG, covering big data, business intelligence, analytics, visualization, and other technologies that help people understand what is happening in their business. Read more ESG blogs here.
