KubeCon + CloudNativeCon sponsored this post.
How PingCAP’s TikV Is Set to Process Transactions from China’s 800 Million Internet Shoppers
Also available on Apple Podcasts, Google Podcasts, Overcast, PlayerFM, Pocket Casts, Spotify, Stitcher, TuneIn
It wasn’t that long ago when Kevin Xu, general manager of global strategy and operations at PingCAP, remembers walking into what he described as a “dark and dingy” basement at an internet cafe to play World of Warcraft and eat noodles in his native China.
“Those days did exist, but not so much anymore — there are many mobile phone vendors now and [almost] everything is being done on your smartphone,” Xu said.
But after attending Stanford University Law School from 2014 until 2017 — when he took advanced computer science classes at the campus because he thought law school was “boring” — the Chinese Internet economy has become a much different place.
“The interesting thing about the Chinese internet economy is that, number one, it has the largest number of Internet users in the world of about 800 million people since my last count, which is like two and a half times the entire population of the United States and is probably more now,” Xu said. “And not only are they internet users, but they are also mostly mobile users.”
KubeCon + CloudNativeCon Open Source Summit China 2019 in Shanghai served as the perfect backdrop to record this latest edition of InApps Makers podcast with Xu. Among other things, Xu put into perspective how PingCap’s TikV open-source distributed transactional key-value database, which has over 5,500 GitHub stars and became an incubation-level hosted project by the Cloud Native Computing Foundation — are helping to meet the demand of the 800 million users that make up China’s Internet economy. (Zhihu, a China-based Q&A website, is also a TikV project maintainer.)
Processing such immense amounts of transactional data on the Internet in China requires a single view of the backend system, including, of course, all backed-up data. This is where the underlying programming language Rust and the Raft protocol for TikV play a role. “As you add more nodes to your cluster as your datasets grow, you don’t want to compromise on that and that’s where Raft comes in,” Xu said. “For us, that’s where all these other decision points come in for us when it comes to the design of TikV because…you don’t want your bank account number to be wrong. Even as a less consequential but still important example, like Amazon orders, you don’t want that to be wrong.”
For Xu, the requirements the Chinese Internet shoppers impose on engineers is what is interesting — and challenging. “A lot of people talk about how big the Chinese economy is and its valuation and the stock market,” Xu said. “What I find fascinating from a developer’s perspective is what kind of technical solutions can you come up with to absorb all of this demand.”
Xu described how TikV, and the open source TiDB (which provides the SQL processing layer), is able to scale to the needs of what he said can be “billions of dollars in transaction value on that one single day.” He said TiDB and TikV are separate, “so they’re two separate code bases.” This helps to allow for throughput to scale much easier by scaling the TiDB server as stateless nodes, so “you can expand and contract, depending on the kind of, raffic patterns or promotions or whatever other kind of things that you as a company might be running,” Xu said
“And that’s hugely flexible and helpful because they do have these crazy, you know,online shopping days, which is kind of ridiculous if you think about it from a money perspective, but even more ridiculous, if you’re like the engineer, who’s working on the backend trying of make sure your service doesn’t go down. So, in my perspective, all that demand is pulled out of the demand for better innovative technologies pulled out from the characteristic of this economy.”
In this Edition:
1:13: Tell us about TikV and PingCAP
6:17: So what are the ways that PingCAP has optimised TikV?
10:26: Why was it built in Rust?
15:56: How you see TIDB ad TikV working together among enterprises, in the context of the China market itself?
19:55: What are the challenges that are facing providers and your customers in China as the scale-out continues?
24:05: What does CNCF incubation status mean for TikV? What should we be expecting over the next several cycles?
Alex Williams, founder and editor in chief of InApps, hosted this podcast.
The Cloud Native Computing Foundation is a sponsor of InApps.
