Discussing Microservices, DevOps and Data with Kasten’s Georgi Matev – InApps 2022

Managing Data At Scale with Kasten’s Georgi Matev

Also available on Apple Podcasts, Google Podcasts, Overcast, PlayerFM, Pocket Casts, Spotify, Stitcher, TuneIn

For microservices environments, data can be tricky. In a system where everything is designed to be highly ephemeral, scalable and temporary, data storage doesn’t fit into any of those categories. Stateful applications require stable data platforms that don’t vanish when traffic goes down, or get overwhelmed when a site is overloaded.

Because of this square-peg-round-hole problem, a lot of DevOps practitioners can find themselves caught behind the weight of the data. It can drag down an agile process, creating large gaps in release cycles as teams wait for DBAs to adjust the store by hand, or to update the schema to accommodate an application change.

Georgi Matev, head of product at Kasten, spends a lot of time specifically thinking about how this issue plays out under Kubernetes. Kasten offers an open source data management platform, Kanister, that runs on Kubernetes. He said one of the problems facing data management, and DevOps in general, in container-based environments is the question of responsibility inside the teams.

“It’s really critical that when organizations are planning on a transition to DevOps, or they’re already doing that, really to understand how the relationships between the different teams need to change. What I mean by that is, what do the people who are responsible for the infrastructure side of things need to do differently, and what do people who are responsible for the development side need to do differently. There’s been quite a bit of convergence between those two, but there’s certainly some activities we see being somewhat specialized. This is particularly true when it comes to dealing with data that microservices are using,” said Matev.

This division of duties can become difficult to define when it comes to databases, he added. “Traditionally, you have a lot of the operators focused on some of the data management tasks, but in a DevOps world, a lot of those functions are transitioning to developers, and it’s really important that the two work together to provide applications still meet the requirements around business continuity, and making sure that data is stored safely.”

Adding Kubernetes to the mix can make things even more complex, but Matev is still convinced it’s the way databases will be managed in the future. “Kubernetes is a great platform to start building database-as-a-service type offerings for your organization. We often times work with centralized IT groups that are starting to think about this, and they find this to be a great platform that takes care of a lot of the challenges around auto-scaling, and handling additional loads. But as they are planning some of these transitions, they are thinking about, ‘How am I going to perform some of the traditional business requirements around business continuity, data protections, and disaster recovery in environments like that.’ The tools to do some of this are not quite there yet.”

In This Edition:

0:45: Holistic advice for people charged with automating some of the operating tasks around microservices

5:00: Have you noticed data agility has become something companies are specifically looking for?

6:45: Can you tell me about the work that you’re doing with Kubernetes?

8:03: What does hosting a database on Kubernetes look like?

12:02: Handling the controlling of new data points

14:05: Should people be running databases in containers?