Continuous Documentation in a CI/CD World – InApps 2022

We continuously integrate. We continuously deploy. We continuously test and monitor. And then we write documentation about it after the fact. Why do we suspend the continuity when it comes to documentation — the thing that helps users actually use our product?

Because no one wants to do it, as it seems like an ancillary chore to our normal workflow.

As DigitalOcean’s Developer Advocate Mason Egger puts it: “This century’s ‘Go get the coffee’ is now ‘Go write the docs’.”

This procrastination only puts farther distance between the code implementor and the documentation, meaning, while docs often exist, they tend to be inaccurate or outdated.

But docs are literally the make-or-break if developers are going to use your software — because without useful documentation, they won’t know how to use it.

“We live in a world that we do a lot of ReadMe engineering — if I can’t get your product up and running in the time it takes me to read your ReadMe documentation, I’ll go to another product which has better documentation.” — Mason Egger, DigitalOcean

The industry is thus demanding continuous documentation or, as Egger calls it, continuous integration for documentation or simply building docs like code.

This piece aims to understand what is continuous documentation and how to make it a part of not only your CI/CD pipeline, but also a part of your company culture.

What Is Continuous Documentation?

For Egger, it’s not just new features and code that need documenting, it’s anything really useful like bug fixes, how you use the code, and how you get started. He says as a first-class citizen, docs empower devs to use your tool in the way you want it to be used. After all, Egger says, people can use products in creative ways.

“If you didn’t explicitly say ‘This is what my tool does,’ then they may use it in ways you never imagined and you may get stuck supporting that for a very long time,” he explained.

Documenting continuously is seen as a core agile practice — albeit one that’s still only practiced by some. This is logical since agility is about quickly delivering value to your stakeholders and documentation is consistently demanded by software users.

Some even call it “deliverable documentation,” as in you can’t deliver code until it’s documented. This means that if you are continuously delivering, you need to be continuously documenting.

Of course, there are others like Bob Marshall who think any need for testing and documentation after the fact means you’ve failed to build quality, simplistic code.

“With agile your documentation is out of date as soon as it’s published — your docs need to be updated when your product is updated.” — Margaret Fero, Technical Writer

In the world of CI/CD and DevOps, in the same way we’ve automated our deployments and builds, we need to automate our documentation, so it stays relevant.

For Amalgam Insights Research Fellow Tom Petrocelli, continuous documentation is where a lot of companies start with CI/CD. If they have docs already as part of their release, when they start testing out there pipelines, they then go to continuation and then to code, working backwards to create a continuous delivery pipeline.

He says continuous documentation is logically automated with CI/CD.

“When you get to deterministic design, it becomes self-documenting. Before I had to write a manual that told me what my system did. Now I write this as code.” — Tom Petrocelli, Amalgam Insights

This could be in plain text with YAML or JSON, sitting alongside the code.

For Petrocelli, continuous documentation is hard to divorce from CI/CD pipelines in general. And since continuous docs is often documentation as code, it is viewed more as self-documenting. This process also appeals to industries that have a lot of regulations and compliance to work with. He says you simply teach the compliance folks to read code, instead of the business analyst translating it all into plain English before their review.

For the sake of this article, let’s define continuous documentation as, not shockingly, continuous and at least somewhat automated. Just like translation, we believe tech writers will be in demand for the foreseeable future to add the human touch to this human tool.

Continuous Docs Is a Culture Change, too

Continuous documentation isn’t just another piece of code, it needs a cultural and process change. There’s a pass-the-hot-potato culture with docs where you don’t want to do it so maybe an intern would? Doc writers are finding they need to sell the value of continuous documentation to the rest of the tech team.

“I have found a piece of the problem is that many developers/product owners don’t know what a docs team does or can do — they have a fuzzy idea it has to do with massaging words or writing a book no one really reads.” — Marya DeVoto, AppNexus

Marya DeVoto, senior technical writer at AppNexus, commented over Slack: “I found giving lunchtime talks that help developers understand that docs work much like code — planning, testing interactions, maintenance, old crufty stuff gets in the way of new content over time — and how we can actually make the product experience better has been productive —and also fun.”

After 15 years as a technical writer, Margaret Fero doesn’t think continuous documentation can be switched on overnight. If you haven’t been prioritizing docs, you won’t have the team already in place to support it. First you need to make a habit of contributing to docs more.

Once you have a functioning docs system in place, you move toward a shorter delivery window.

“If you already have a working docs process and a CI pipeline, it’s a lot easier to bring those two together and merge them,” Fero said.

Most application teams will have to build up to continuous documentation gradually.

Fero says to look for which teams the tech docs are closest to. If the docs are closer to the product team, then they say getting docs done is easier. If the docs are closer to the dev team, then continuous docs work better when integrated with issue notification.

This way Fero says you can work up to documented-equaling-done.

Egger agrees: “If you build documentation as a story into your sprint, it’s great and you get time to write good docs. When you create a story in Jira, the code is a task — you can’t close a story without unit tests and, if necessary, a documentation update.”

He continued that even internal documentation should be a consideration into “Done,” at least when addressing underlying infrastructure changes.

It’s not surprising that he also advocates that your docs should be a part of your CI/CD pipeline, making docs a part of your code deliverables, released at the same time.

Getting Started with Continuous Documentation

Fero offers the simple and widespread solution of a lint or linter as an easy sell because teams are probably using linters already. A linter is a tool that automatically analyzes source code to flag and fix programming errors, bugs, and anything suspicious, but can be used like a spell-checker and to flag stylistic errors. Linters introduce not only continuous documentation but house style and consistent formatting into your automated continuous integration pipeline. They say a linter checks for anything objectively wrong that can sneak through before the docs are committed alongside the regular code.

“Every time I fix a recurring or major issue, I implement” a linter action, they said.

One way Egger says you can achieve this is by keeping docs close to your users, using something simple like a WYSIWYG editor or markdown. He offered MkDocs which is a simple Python-based tool for static site generation for markdown. He says the MkDocs plug-ins allow you to do a lot including putting code snippets up against your library to see if the code is correct, sometimes picking up things the unit tests don’t.

You also need to make sure you have a tool that allows your devs to compose docs as code. Egger offers up DocSphinx for Python, Javadoc to generate HTML docs from Java source code, and RubyDoc for Ruby documentation as easy-to-use.

Linters help prevent repeated errors, but developers also have to be a part of any continuous documentation process. Fero says, especially when a developer is also the feature owner, that dev has the responsibility to also follow up on the documentation before being able to hit Done.

“Devs should be in charge of making sure that any words that come up in content in the commit be flagged up for style and correctness — making sure that the functionality of the code should be documented in some form and whatever shape your lInter has requested it to be in,” they explained.

Fero continued that this should then trigger a notification to a technical writer or the team releasing the code that the change has been implemented. Then they ask: Does there now need to be a doc change?

They explained, “The value of having especially a technical writer is making sure that the software and the function of the software is going to be communicated.”

Fero said that it’s not just about proper English but lingua franca — that it works as a point of communication between people who are non-native English speakers too. English is a continual barrier to entry for open source communities especially. Egger offered Django as being well-equipped for language localization.

Tech writers are also responsible for making sure docs remain accessible. Fero said a linter can be used here too, like making sure every image has alt text or flagging any buttons without labels. As a linter is best for avoiding repeated mistakes, they always flag for links with generic labels like “here” which don’t add value to reader or screen reader.

Where Does Continuous Documentation Live?

If the most essential part of continuous documentation is that the code functionality is documented in some form, then the docs need to be as close to the code as possible.

For Fero, if you’re using a Github flow, docs should be in your commit messages. Your pull request overview comments should include what the general functionality is.

They say if you aren’t using that flow or tooling, then you might want to do it in your bug tracker or wiki page. They said this decision all depends on how good your org is at itemizing and breaking down functionality.

“You don’t want to clutter the comments too much with things that should be evident in the code.

If there’s a complex change of parameters or precise tweaks, you won’t always be able to see at the glance what this interacts with,” Fero explained.

Particularly in CI/CD, you need to constantly make sure you aren’t holding up the release process. Fero suggested teams focus on issue automation on demand, like planning a 20-minute docs triage period at the end of each day.

There are a lot of tools that are trying to assist with continuous integration for documentation, which include OpenAPI (formerly Swagger) and Javadoc comments, but Fero warns these tools usually automate reference documentation — which helps but isn’t sufficient.

Richer documentation needs to also be task-focused and concept-focused.

Fero explained that “You’d be able to get away with no conceptual documentation for a word processor, but, for Docker, you need material on containerization, contextualizing what it’s for.”

“And you need task-based documentation so people can figure out the general flows that they want to work in.”

Additional Resources on Continuous Documentation:

InApps is a wholly owned subsidiary of Insight Partners, an investor in the following companies mentioned in this article: Docker, Real.

Feature image by Gerd Altmann from Pixabay.