All you need to know about DevOps

DevOps is a term used frequently by developers and other teams in an organization. Exactly what is it and why do we use it?

In this post, we’ll explore a definition of DevOps, the responsibilities of the DevOps engineer, DevOps’s relation to agile development, and the DevOps life cycle.

We’ll wrap up with some real-world stories from leading organizations that use DevOps to improve the speed and quality of software development.

What is DevOps?

DevOps brings together a combination of software development and operations. Development happens when software developers program an application and test it, and operations begins once the application is deployed and maintained on a server.

DevOps encompasses a set of practices that help organizations reduce the time between product ideation and its release in production to the end user. Done correctly, DevOps helps to eliminate silos and ultimately shorten long, drawn-out software release cycles by introducing smaller, more frequent releases that keep pace with customer needs.

DevOps engineers have to possess a broad range of skills, including:

• Developing a solid understanding of development concepts and the systems, tools and processes the developers use.
• Understanding how an application is configured to interact with services and data, and how it should be tested.
• Creating and configuring servers to run the application, either on-premises or in the cloud.
• Knowledge of server operating systems, such as Linux.
• Knowledge about networking, including DNS, DHCP, load balancing, etc.
• Understanding of virtualization, cloud Infrastructure as a Service (IaaS), and containerization.

While DevOps engineers don’t need to be experts in these areas, some may be specialists in one or more, which can help them succeed in the DevOps role.

One of the main tasks of a DevOps engineer is releasing a new application version into production in an efficient and automated way. Let’s take a look at the DevOps application-focused life cycle and how DevOps engineers can use this model for a successful release.

The Application Life Cycle

For DevOps, the application life cycle consists of five main phases: development, testing, integration, deployment and monitoring. These repeat in an infinite loop once an application is released into production. Here’s a summary of each stage:

Development

During this stage, a developer will collaborate with team members and potentially customers to define requirements for an application. They will determine an IDE (integrated development environment) and the programming language to use. DevOps engineers should familiarize themselves with the IDE and language so they can make the best infrastructure decisions for the application. Version control is critical, as well as a code repository for storing and collaborating on code. The repository can be public or private, and created on GitHub or GitLab, for example.

Development consists of three main steps:

• Plan: Planning is the process during which teams collaborate on features and bug fixes to roll out in the next sprint. The DevOps engineer can get involved to learn about the process and influence developers’ decisions based on the infrastructure requirements.
• Code: After planning, developers write the code. Then the code is merged into the repository.
• Build: This step involves building the code in the chosen language. It could involve compiling it or creating a Docker image, either of which requires the continuous integration/continuous delivery (CI/CD) pipeline.

Testing

Once the requirements are set and the application is being developed, testing code in various environments is the next step. During this phase, QA tests for bugs. Containers will be used to simulate the test environment, eliminating the need for physical or cloud infrastructure. Automation in this phase will contribute to continuous integration and help DevOps avoid having tens or even thousands of engineers do the work manually.

Integration and Release

Smack in the middle of the DevOps life cycle, integration involves committing changes to source code, usually on a daily or weekly basis. With every commit, automated testing can be employed to help detect issues or bugs prior to the next phase. Releasing the new code may be as simple as putting it into a registry or code repository where it’s accessible to your production servers.

Deployment

Now it’s time to deploy the application into production for users to consume. During this stage, the code is deployed to production servers. Different applications will require different hardware and configurations, which is why application configuration management and Infrastructure as Code come into play. If your application is containerized but also available on a virtual machine, platforms like Kubernetes will be needed to orchestrate those containers and ensure you have the desired state available to end users.

Monitoring

Once the application is in production, continuous application performance monitoring is essential to ensure end users are getting the experience they require. Part of this process is capturing user feedback to factor into the product roadmap. Reliability is also key — the application needs to be available to users when they need it. To that end, security and data management should also be continuously monitored. Some of the key parameters to monitor include CPU utilization, disk space and response time. Keeping logs enables developers to troubleshoot any issues.

The figure below illustrates the stages and shows how they operate in a continuous and infinite loop:

DevOps and Agile Development

Do you know the difference between DevOps and agile? In short, agile development is an approach that focuses on delivering small frequent changes, rather than releasing a major update, with the goal of providing an optimal end-user experience. DevOps describes software development and delivery best practices based on cooperation between software developers and operations specialists with the goal of simplifying development and minimizing miscommunications.

The two concepts are similar, but there are important differences:

• While agile aims to shorten iterations, that’s only possible with the automation DevOps provides. Whereas agile focuses on optimizing communication between end users and developers, DevOps targets developers and operations teams. The competencies of DevOps engineers lie in the intersection of development, QA and operations. They are involved in all stages of the product cycle.
• While agile offers various management frameworks to achieve flexibility and transparency — for example, Scrum, Kanban, Lean, Extreme, Crystal, Dynamic, and feature-driven — DevOps focuses on best practices such as Infrastructure as Code, architecture as code, monitoring, self-healing, end-to-end testing and automation, but no specific frameworks or methodologies.
• In agile, the main source of feedback is the end user, while in DevOps, feedback from stakeholders and the team itself has a higher priority.
• Agile focuses on software development more than on deployment and maintenance. DevOps focuses on software development as well, but its values and tools also cover deployment and post-release stages such as monitoring.
• Agile prioritizes flexibility and tasks over documentation, while DevOps regards project documentation as a top priority.
• In agile, projects are difficult to predict or evaluate as priorities and requirements are continually changing. In DevOps, risk comes from misunderstanding and lack of suitable tools.
• Agile tools are focused on management, communication, collaboration, metrics and feedback — think JIRA, Trello, Slack, Zoom, SurveyMonkey. DevOps uses tools for team communication, and software development and deployment — think Jenkins, GitHub Actions, BitBucket.

Combining agile with DevOps yields substantial benefits, including flexible management and powerful technologies, good communications, automation and better overall product quality. Best practices for merging agile and DevOps include:

• Unite the development and operation teams.
• Create build and run teams.
• Change your approach to sprints.
• Include QA in all development stages.
• Choose the right tools.
• Automate everything you can.
• Measure and control by using tangible numeric deliverables.

DevOps in Action: Amazon, Netflix and Etsy

DevOps is catching on among leading organizations in every industry that are realizing its many benefits. Here are three real-world examples of how a DevOps culture can be adopted successfully:

• Amazon: By adopting a continuous deployment process where developers can deploy code whenever and wherever they want to the servers of their choosing, Amazon is able to deploy new software to production servers in 11.6 seconds, on average.
• Netflix: Netflix developers can automatically build pieces of code into deployable web images without relying on IT operations. As the images are updated, they are integrated into Netflix’s infrastructure using a custom-built, web-based platform. Continuous monitoring is in place so that if deployment of the images fails, the new images are rolled back and traffic rerouted to the previous version. This enables Netflix to deliver services with speed, flexibility and attention to quality.
• Etsy: This company also lets developers deploy their own code — they’ve been doing it since 2009. Etsy realized that when developers are responsible for deployment, they also take responsibility for the application’s performance and uptime.

A Recap of This DevOps Overview

To summarize, DevOps is a combo of development and operations that allows a single team to manage the whole application development life cycle: development, testing, integration, deployment and monitoring.

Its goal is to accelerate the development life cycle while delivering innovative features and fixes that delight customers and align with business objectives.