Baidu Joins the Crowded Race of Open Source Machine Learning Frameworks – InApps 2022

Chinese Web services giant Baidu has joined the other big names in search and social media by releasing its machine learning toolkit as an open source project on GitHub.

PaddlePaddle — which stands for PArallel Distributed Deep Learning — lets you write pseudocode in Python, which the head of the project, Baidu Distinguished Scientist Wei Xu, said allows developers to “focus on the high-level structure of their model without worrying about the low-level details.”

Working at that high a level may mean less flexibility; Baidu Chief Scientist Andrew Ng noted at the announcement that “other deep learning platforms have been a great boon to researchers wanting to invent new deep learning algorithms. But their high degree of flexibility limits their ease of use.”

In contrast, PaddlePaddle is aimed at enthusiasts and mainstream developers rather than machine learning researchers developing their models and algorithms, and Xu told us that makes it more accessible than some current frameworks.

“For many tasks, PaddlePaddle requires significantly less code to be developed. For example, a machine translation model built on the PaddlePaddle engine would need about a quarter the amount of specially written code than the same task built on other AI platforms.”

PaddlePaddle lets you use some common deep network architectures; convolutional and recurrent neural networks (CNNs and RNNs), as well as attention networks and neural Turing machines. “It’s hard to enumerate,” Xu told us, “because it’s like playing with blocks; you can construct many different things with it.”

Some of the most interesting work with deep networks involves extremely deep networks with many more layers; PaddlePaddle doesn’t have specific tools for that, but there’s no fixed limit in the framework on the depth of networks; “it depends on the size of the model and the memory limit,” said Xu, pointing out that the framework scales to many GPUs and CPUs on multiple machines, and uses common Basic Linear Algebra Subprograms (BLAS) libraries to speed up its mathematical operations.

Baidu has been using PaddlePaddle to develop advertising, search ranking, large-scale image classification, optical character recognition and machine translation. According to Charles King, principal analyst at Pund-IT, that includes ad click-through rate (CTR) prediction, virus detection, and user recommendations; all standard scenarios for deep learning that show the framework is worth considering.

Xu claimed that it handles sequence models very well (like tagging parts of speech for recognition and translation), as well as “problems with high-dimensional sparse data, which is very common in industrial applications but lacks support in most other deep learning frameworks.”

Frameworks, Platforms and APIs

PaddlePaddle is competing with Google’s TensorFlow, which quickly became popular — especially after Google allowed the open source version to run on more than one machine, quickly overtaking the previous favorite, the Caffe image recognition framework from Berkeley Vision and Learning Center. Microsoft’s Computational Network Toolkit (CNTK) — originally developed for speech recognition but now applicable for many other models and available on GitHub for commercial use instead of its original academic limitation — is currently the third most popular machine learning framework on GitHub (by number of stars), followed by veteran open source tools Torch and Theano, and then Amazon’s Deep Scalable Sparse Tensor Network Engine (DSSTINE) framework, released in May.

All these projects want the developer community to use — and contribute to — their machine learning framework, to help improve it. “As vendors work to push new technologies and processes into broad adoption, it’s critical for them to engage as many interested participants as possible; that’s certainly the case in deep learning, and machine learning whose past technical requirements have discouraged non-specialists,” King noted.

But he added that “the news at this year’s fall technology conferences has been filled with vendors and their partners attempting to lower or dismantle those barriers to encourage AI development.” It’s a crowded market.

If PaddlePaddle really is faster and easier to work with, that would give non-expert developers a way to see if machine learning is useful for a project quickly. But for that audience, PaddlePaddle is also competing with services like AzureML, Microsoft’s drag and drop visual programming cloud service which offers multiple machine learning models including deep learning networks

It is also competing with APIs like those offered by IBM Watson and Microsoft Cognitive Services for image recognition and captioning, emotion and sentiment detection, voice matching and speech recognition, which mean developers don’t have to build a machine learning model themselves at all — just send their input and get the results back.

“For some standard problems, such as speech recognition, it is better to use some other existing services,” Xu admitted. However, he believes, “in many cases developers will encounter situations which do not have prebuilt services because of customized data or problems.”

One advantage of PaddlePaddle may have little to do with the technology, King points out. “Baidu’s framework should catch the attention of any company or developer hoping to do business in China,” he told us. “The sheer size of that market makes it hugely attractive, but history shows that collaborating with a recognized, respected Chinese organization is the only realistic way to proceed.”

He noted that Baidu has recently partnered with both Intel and NVidia, making sure that PaddlePaddle can take advantage of both CPU and GPU improvements; many existing frameworks are tuned to rely on GPU power. And despite the sudden dominance of TensorFlow, it is still early days.

“AI and deep networks and machine learning are really still in their infancies, with no single platform having a solid leadership position. That said, things are moving quickly, o it’s good for Baidu to make its move now. Next year might be too late.”

IBM and Intel are sponsors of InApps.

Feature image: Lido de Jesolo beachfront by Max Boettinger, via Unsplash.