The open source Apache Flink stream processing framework has been gaining interest and, naturally, as an enterprise-focused software project, Flink is starting to gain commercial support and products. The company that has picked up the Flink banner and run the farthest with it thus far is Data Artisans.
The company’s dA Platform 2 platform arrived this week as an early private beta, bringing with it new tools for managing and monitoring Flink applications in production. This was formerly a difficult task for operations teams to tackle. With the release of the new dA Platform 2, the package now includes the Kubernetes container orchestration tool and an application manager.
“It’s easy to build a Flink application and use the APIs, but once you want to have it running in production 24/7, then the challenges start. These are stateful applications, and it’s hard to maintain stateful applications in production,” said Kostas Tzoumas, CEO and co-founder of Data Artisans.
dA Platform 2 is built from three core components:
1) Flink.
2) an application manager.
3) A reference architecture and optional components for logging, and metrics.
This is also the first version of the dA platform that includes Kubernetes. The platform now utilizes the popular container orchestration platform to manage instances of Flink and to control the tool chain as a whole.
For the future, Tzoumas said that he sees a lot of opportunities to help enterprises operationalize and utilize Apache Flink for their stream processing needs. “What we are really focusing on now is bringing this technology to market and making it accessible to the enterprise as a whole. We really see streaming technologies being used for a much wider use case. We see applications in the apps domain, microservices, and batch processing. We’ve worked with users hardening and stress testing in large environments. Next, for us, is making operations much smoother for normal enterprise users.”
The company plans to fully release dA Platform 2 for production use in early 2018.
