“What we really want to do is transform Oracle’s market,” Cockroach Labs co-founder and CEO Spencer Kimball told me in a recent interview. It was part of his explanation for why Cockroach Labs chose to build a relational database system in the middle of the cloud computing era.
When the company launched in 2015, NoSQL systems like Redis and MongoDB were all the rage. It was assumed at the time that Oracle, Microsoft and other SQL heavy hitters had the relational database market sewn up — so why compete directly with them?
Cockroach Labs had other ideas. Its main product, CockroachDB, is a so-called “distributed SQL” system. Kimball and his co-founders, Peter Mattis and Ben Darnell, got the idea from a 2012 paper describing Google’s “globally-distributed database” called Spanner. All three founders had worked at Google, so it was a short leap to launch a startup in 2015 to take the distributed SQL concept to market.
Five years on and CockroachDB is now one of the leading distributed SQL solutions, alongside Google’s Cloud Spanner (its current name), Amazon Aurora and relative newcomer Yugabyte. With that said, traditional relational database vendors like Oracle still hold a massive lead in the SQL database market. The graph below from DB-Engines shows that the top three relational DBMS products — Oracle is number one — are vastly more popular than CockroachDB (number 37 on the list, although its trend line is strong).
Source: DB-Engines
Given Oracle’s market advantage, I asked Kimball why an enterprise CIO today would choose a distributed SQL system? Especially since many corporate IT departments don’t necessarily need or want global distribution of data.
“So distributed SQL comes with, I think, three key benefits,” he replied. “One is scalability. In particular, the kind of scalability that you expect when you can scale horizontally.”
Kimball admitted that while “you can scale Oracle systems” — for example by using Oracle Exadata, either on-premises or in the cloud – ultimately this only achieves what he calls “enterprise scale.”
According to Kimball, enterprise-scale won’t cut it for “most of the world’s next generation of use cases, especially high growth tech companies.” What those companies need, he said, is the “Google-level scale” that a distributed SQL system gives.
“Nobody wants to buy extremely expensive hardware [and] extremely expensive software in the old model,” he continued. “They want to use public cloud infrastructure, they want commodity servers.”
The second key benefit to distributed SQL, says Kimball, is resilience. Again, he admits that “you can obviously have resilience in a traditional monolithic database” — for Oracle he cited GoldenGate, which uses asynchronous data replication. But the problem, he says, is that you could potentially lose data if you have to do a failover recovery (due to a software or hardware failure).
“Cockroach, inspired by Spanner, is using consensus-based replication,” Kimball explained, which means that “you don’t lose data even if you lose an entire replication facility, like a data center.”
“So those two [scalability and resilience] are very big departures from Oracle — and in addition, of course, the global data architecture side of things.”
To be fair, Oracle might counter that its product is one of the most highly rated by Gartner, which praised Oracle’s innovation, functionality and performance, and customer satisfaction. It’s number one in the relational DBMS market for a reason.
Even in cloud, despite being a late mover, Oracle has made significant investments to adapt.
So while it’s possible Cockroach Labs will over time make inroads into Oracle’s market share, as of now it’s equivalent to a gnat (or a cockroach) trying to take a bite out of an elephant.
Where I think Cockroach Labs is best positioned to make an impact is in the coming generation of AI applications. Scale and resilience will really matter when you’re dealing with the masses of data required to run those applications. Although it’s not that CockroachDB will be used to train the AI — for that you’ll need a platform like TensorFlow or PyTorch — but it can store and manage the data. This is similar to how Redis Labs has positioned itself for AI.
“Cockroach is really built to be the ultimate system of record,” is how Kimball put it, adding that “you don’t use a data lake, for example, as your system of record.”
Intelligent Agents
So what types of applications are we talking about? One of the areas Kimball thinks will benefit greatly from AI and machine learning is intelligent agents.
“So non-human agents that are taking actions, often with APIs serving as the front end to the wealth of services that are being created.”
Kimball says this will result in a shift from “human speed interaction to machine speed interaction,” which will lead to “transactional big data” becoming the standard. He thinks intelligent agents combined with the emerging Internet of Things (IoT) will bring about “potentially hundreds of billions [and] maybe even trillions of interactions.”
That is what Cockroach Labs ultimately aims to be: the database of record for AI interactions.
“Whether it’s IoT, whether it’s virtual agents that are watching stock tickers and doing robot autonomous training, all these things interact with APIs. And APIs are backed by applications and services, that have databases behind them. That’s where Cockroach fits in.”
This type of use case is also being explored by DataStax, a NoSQL database company built on the open source Apache Cassandra. Indeed, when I asked Kimball whether Cockroach Labs competes directly with any NoSQL systems, he told me that Cassandra is the only one.
“Cassandra has been the undisputed leader in terms of big scale — like, webscale — type applications. I mean, Apple’s got more than a hundred thousand instances of it.” [Author’s note: Apple doesn’t officially disclose its Cassandra usage numbers, but 150,000 instances was cited in an Apple presentation last November.]
However, Kimball argues that “although Cassandra handles scale very well, it doesn’t handle [transactional] consistency very well.” He says that while an application might start out simple, it can get more complex over time.
Which brings us back to the core reason SQL, as a programming language for databases, has survived the test of time (an IBM paper that led to the creation of SQL was published 50 years ago last month). Even with the innovations made over the past decade or so by the likes of Redis and Neo4j, relational databases still hold tremendous appeal in the cloud native world for their structure, powerful querying and ability to handle complexity.
Which is why CockroachDB has a SQL engine. Although right now, it’s like the little red caboose compared to Oracle’s steam engine at the front of the train.
Redis, MongoDB and DataStax are sponsors of InApps Technology.
Feature image via Pixabay.
At this time, InApps Technology does not allow comments directly on this website. We invite all readers who wish to discuss a story to visit us on Twitter or Facebook. We also welcome your news tips and feedback via email: [email protected].
