Discussing DevOps, Data and Microservices with Vexata CTO Surya Varanasi
Also available on Apple Podcasts, Google Podcasts, Overcast, PlayerFM, Pocket Casts, Spotify, Stitcher, TuneIn
Data in microservice-based environments can be difficult to manage at scale. When application servers scale to near infinity, the datastores can’t necessarily expand to meet that demand; they can only be optimized to keep up, and perhaps sharded. Considering just how much enterprise information is stored in some of those large systems, it’s a worrying proposition to be asked by management to increase application performance when much of it is tied to an Oracle or Microsoft database.
Surya Varanasi, Chief Technology Officer of storage provider Vexata, has been dealing with large amounts of data for over a decade, now. While he once worked around the hardware layer at Brocade, today he focuses very heavily on the enterprise databases that power businesses around the world. From Oracle, to SAP, to SAS, and Microsoft, Vexata swims in an enterprise pool of customers.
The problem, said Varanasi, has always come down to application performance. Enterprises often have complex SLAs and agreements that can specify access times for crucial customer data, and be able to ensure that data gets to them unharmed and on time is a top priority.
“The amount of data you need to access really has grown a lot. There’s this notion of data gravity, which really refers to the difficulty of managing and moving data around, and how do you access it and keep the applications busy and get the results you want? That’s the number one issue we end up seeing in the market today,” said Varanasi.
Enterprises are experiencing a bloom of internal applications with the move to microservices, and yet, all these different services typically have one thing in common, said Varanasi. “All these applications have a lot of data. That’s the one constant they all have. And they are trying to be extremely agile, with application updates every week, or dealing with the number of clicks they get and trying to figure out how to scale with that… A lot of what you want to deliver is results based on your data, and how do you get to your data?”
Despite years of work from the open source community to replace these large, traditional, relational data stores, it is the weight of that data inside that holds agility back, not the nature of the store itself, said Varanasi. “The data gravity is driven by the data. It’s fundamentally the data. Microservices, and in general, compute and microservices, have to revolve around the data. It’s really very difficult to move large amounts of data impossibly fast. If you have a 100 TB database and it’s going through a 10 GB pipe, it’ll take forever. It’d just be a nightmare to move. It’s how you get your compute and microservices to hover around your data.”
In this Edition:
0:16: What is the problem with data in container-based or microservice-based environments?
1:01: How do you reconcile a large enterprise database with an agile process?
3:58: Are these data architectures becoming more complex because of the demands?
7:57: Does data gravity come from size or platform?
9:28: Where does database automation responsibility fall: Development or operations?
14:34: Do you have any advice for folks trying to sell software into enterprises?
Microsoft Azure is a sponsor of InApps.
Feature image via Pixabay.
