A number of new databases have taken on the challenges of distributed SQL, including Google Cloud Spanner, CockroachDB, TiDB, CrateDB and FaunaDB. Others, such as Splice Machine and Yugabyte have taken a hybrid approach to the whole SQL-vs.-NoSQL debate.
Rather than starting from scratch, as others have, San Francisco-based Citus Data is focused on making reliable, 20-year-old Postgres scale.
“We want to take care of the hard part of scaling the database for customers,” said Craig Kerstiens, head of cloud at Citus Data.
“The old way of scaling databases is you scale up and up until you can’t scale up anymore. There’s only so far you can go with that,” said Kerstiens. “We just let you keep scaling that database.”
The company wants customers to be able to focus on building features rather than mitigating their database pains.
Citus Data provides an extension relying on APIs to Postgres rather than a fork, which requires reworking each time a new version of Postgres comes out. That means it can stay current faster.
It provides auto-sharding and replication across a cluster of commodity nodes. If any node in the cluster becomes unavailable, Citus simply redirects any writes or queries to one of the other nodes, which maintains a copy of the affected shard.
Its largest customer has around 500 terabytes of data running on a 40-node cluster, Kerstiens said.
He cited two primary use cases for Citus Data:
- Fast-growing B-to-B apps for customers like ProsperWorks, Google’s recommended CRM. It was growing so fast the company thought it might have to stop taking new users. It found the answer was migrating beyond a single node.
- Customer-facing analytical dashboards for customers such as Cloudflare, one of the world’s top content delivery networks (CDNs). These customers have hundreds of thousands of users hitting these dashboards in real time and need subsecond response times across hundreds of terabytes of data.
Horizontal scalability
Stanford alums Ozgun Erdogan, Sumedh Pathak and Umur Cubukcu founded the company in 2010 and went through Y Combinator in summer 2011. The company open sourced its technology in March 2016. It has raised more than $13 million.
Citus Data provides the horizontal scalability of a NoSQL database with the transactional consistency and functionality of a relational database.
At a high level, Citus distributes the data across a cluster of commodity servers. Incoming SQL queries are then parallel processed across these servers.
It uses an architecture similar to Hadoop: one master node uses metadata about the shards and parcels out fragments of incoming queries to worker nodes that actually run the query pieces in in parallel.
Distributing the query across the cluster harnesses the processing power of all of the involved nodes and all the individual cores on each node.
The coordinator node looks like a single-node Postgres to the application. It aggregates results from the worker nodes and delivers a final result. It also looks to minimize the amount of data transferred across the network.
It provides:
- Massively parallel processing for SQL analytics.
- Real-time inserts/updates on distributed database tables.
- Dynamic scalability on commodity hardware with ability to easily add or remove nodes.
- JSON and structured data in one database.
- The expressiveness and familiarity of Postgres.
- Multi-tenancy to more than 100,000 tenants. It supports tenant isolation to provide performance guarantees for large tenants, and has the concept of reference tables to reduce data duplication across tenants.
“Citus makes things possible for us that would have been nightmarish in a NoSQL world,” said Jarred Nicholls, chief software architect at IronNet Cybersecurity. “Moving to Citus has improved the reliability and performance of our database, and the fact that it’s not a black box and all the Citus metadata is open and hackable gives us a lot of flexibility.”
The company primarily competes against enterprises doing manual sharding, which can take six months to two years of engineers’ time, Kerstiens said. It’s a big investment and quite time-consuming.
“If you care about performance and don’t want to have to stop feature development for six months while you work out performance issues, that’s where we’re seeing most of our adoption from customers,” Kerstiens said.
Focus on Cloud Service
Citus Data offers open source and enterprise versions, but more recently has been focused on its managed service offering on AWS.
It recently added new features to its Citus Cloud product including:
- Warp, which allows you to continue writing to your existing single-node Postgres database by streaming all of your updates into your Citus Cloud cluster as you prepare to migrate.
- Zero-downtime shard rebalancer, which enables customers to elastically scale out memory, compute, and storage as they add nodes without downtime.
- Point-in-time recovery, allows users to roll back the database to any point in
- the past 10 days.
- Followers, an up-to-date read-only copy of the database that enables developers and analysts to run real-time, complex queries without putting any query load on the production database.
- Support for distributed transactions.
Going forward, the company will continue to be focused on making the database something developers don’t have to worry about, and making adoption and migration easier, Kerstiens said.
Google Cloud is a sponsor of InApps.
InApps is a wholly owned subsidiary of Insight Partners, an investor in the following companies mentioned in this article: Real.
