Integration is Key – InApps 2022

“Connection is easy, it’s integration that’s hard,” said Ross Garrett, head of product marketing at Cloud Elements, an API integration platform.

Ross Garrett, head of Product Marketing at Cloud Elements, showing how connecting an API is just the tip of the iceberg.

The growth in the world of APIs has been exponential, Garrett said at the recent APIDays held in San Francisco. Between 2010 and 2016, API adoption grew 757 percent, and that’s just public APIs. With progress that fast come challenges and he laid out lessons learned and provided suggestions for how to move forward.

Now that there are many, many thousands of APIs, Garrett wanted to look at how well “those APIs are optimized for integration,” he said. So he looked at Cloud Foundry’s best practices using data collected from September 2016 through March 2017 from the Cloud Elements platform including 165 public apps and services, 28,000 integration instances and 1.6 billion API calls. His findings also used data from API data hubs such as ProgrammableWeb, SmartBear, and Datanyze.

All this research identified the top issues regarding installing APIs, said Garrett. “So you’re a CTO and your developers will tell you connecting an API is a one-week task. And it is. But that just the connection.” Just getting the system to talk end-to-end is one small step, not the end point, he said.

Also necessary is figuring out frameworks for error handling and eventing and how to manage custom data. “You have to ask questions like ‘Is there a potential for custom data in that service or how does that API deal with customer data? How do I deal with advancing within that service?’”

And suddenly, a one-week project turns into six months. To add insult to injury, he said, once you get all that in place, the API has changed so it’s all broken again.

So connecting is just the tip of the iceberg. And he brought seven tips for the challenges and opportunities for managing your APIs:

Authentication: Standardization across the board will bring familiarity to this new world, he said. For authentication, OAUTH2 is the preferred mechanism for token-based access, but 59 percent of respondents are not using it. Start using it, he suggested.

Documentation: Documentation is the first thing people see and the first experience that users have with your product. If you get it wrong, then the users may not ever see the other stuff you might have worked really hard on, he said. “Conversely, if you get this bit right and then encounter some issue, some functional issue down the road, then they’d be more willing to forgive you for that because at least the documentation was great to help them get started in the first place.”

There’s lots of room for improvement in API documentation, said Garrett, noting that Cloud Elements has identified this as an area in which they can do better.

Custom Data Discovery: Although fifty-eight percent of endpoints support some metadata discovery, the research found less than 15 percent of endpoints actually support true data discovery. This is important because a lot of companies capture custom data to manage custom reporting requirements, he said. “So this is like the meta data API, the Salesforce software, so I can effectively ask Salesforce and ask my specific incidents via API what can you do, and what fields can I talk to,” he explained. “That’s helpful, right?” But not every API handles this.

Suggested Status Code Rang

Status and Error codes: Garrett said he could do a whole presentation just on status codes and error handling. We can do so much better than “internal stack trace error”, he said. He suggested some standardization with HTTP status ranges and using some more natural language. Using the right code and providing some human contact for your clients will make them handle errors much more gracefully, he said.

Bulk Data Support: Enterprise users are often dealing with vast data lakes that need to be synchronized, analyzed, migrated, or even shared, he said, so bulk application support is critical. Application providers must think carefully about the functionality they offer to manage this scale of data.

Only forty-two percent of APIs support bulk data, noted Garrett, pointing out that it’s kind of shocking how many service providers don’t offer an easy way of doing that.

“They almost make it impossible, it’s like they don’t want you to go away,” he said. After realizing their customers appreciate bulk support built into interface, Cloud Elements provides this functionality wherever a service provider doesn’t. “It’s because product users are dealing with massive amounts of data that they need to do something with,” he said. “Synchronize it, share it, migrate it, whatever.”

Eventing Support: Polling is not a solution for event driven apps, he said. Too many applications contentiously poll with no useful data coming back. One stat says 98.5 percent of all polling results provide no usable data. “So we’ve got so much bandwidth and computational power being wasted just asking a system is it done yet? Is it done yet? Is it done yet?” he said.

Eighty percent of developers prefer to use webhooks to solve the event driven aspect of app integration, said Garrett. But only twenty-nine percent of applications offer webhooks today representing a huge missed opportunity.

Versioning: Another topic Garrett could spend all day on is the best strategies for versioning. “If something changes and you haven’t provided an easy way or any information to your API consumers, then the integration is broken,” he said. “It’s really that simple.” It’s often one of the things that developers don’t think of until it’s too late.

Know Before You Connect: So, what to do with all this information? Garrett created a scorecard to evaluate how good an API is from the perspective of integration.

The State of API Integration Scorecard, developed by Ross Garrett