Update AIOps: Greater Data Context Can Yield Richer Results

In 2017 Amazon Web Services famously rolled out a Snowmobile: a semitrailer to help customers move up to 100PB of data from an existing data center to the AWS cloud. Amazon decided that moving data by the truckload was the most efficient and cost-effective method of transferring large amounts of date from one place to another.

This is the inflection point in which we live today: the flywheel of data size and complexity is spinning at an unprecedented rate, and we now literally need to manage it by the truckload.

Data Is Unwieldy, but Expectations Are the Same

If it’s now difficult to simply transport data from one place to the next, it’s humanly impossible to monitor and manage the data produced from distributed, hybrid, multicloud applications and environments. Yet, there’s more pressure on IT operations teams than ever to embrace an agile framework encompassing multicloud platforms, containers and distributed application architectures, all while supporting DevOps and continuous delivery practices. This has created a huge challenge for traditional monitoring, management, and infrastructure tools.

IT practitioners often struggle with context. They jump to the data manipulation or aggregation, setting up models of data ingestion without thinking about the business first.

How do you move from static systems like data centers where you’d load balance and tune a system once, to the cloud, where everything can be optimized ad infinitum? How do you involve more stakeholders from development, managed service providers (MSPs), cloud providers and IT ops teams? And most importantly, how do you turn all of the data generated by your dynamic system into action?

Enter AIOps and the Rise of Data Context

This is the story behind the rise of automation and AIOps in digital operations. AIOps is the method of applying algorithmic intelligence to eliminate mundane tasks and drive machines to actually think. But in my experience, many practitioners forget that the success of AIOps and automation hinges on the quality of the data. Good, representative data is what any machine needs to operate accurately and perform the task that meets the needs of the business. We call this data contextual: that is to say, when taken together from a variety of sources, it provides an accurate, business-aware view of what’s really going on in the business.

IT practitioners often struggle with this context. They jump to the data manipulation or aggregation, setting up models of data ingestion without thinking about the business first. Instead, I advocate for a systemic approach to preparing data for a future of artificial intelligence that’s context-aware.

So what are the ingredients of contextual data? What really matters in the machine?

Building Futureproof Data Context

There are six steps to creating a rich, accurate data context for effective AIOps.

1. Focus on the business objectives and outcomes: Start with building the right data-driven culture, and the right data will follow. Contextual data is manufactured by teams with the right skills and vision for the task. Identify the opportunity first, and then put a plan in place.
2. Shift from process-based to data-based milestones: This requires a departure from establishing process first and hiring for that process. Instead, hire data experts who can help build the roadmap and set up appropriate data manipulation and governance. This last part is critical, since governance creates consistent context, and is easiest to establish at the beginning.
3. Get the data right! Once you have data governance established, it’s critical to discover, rationalized and understand all data sources. Create a data model that drives the relationship and context between technical and business services.
4. Bring quantitative and qualitative data together: Selectively apply data-driven business process, technology innovation, and corporate governance. What qualitative metrics matter to business goals? All types of information can inform data context depending on these criteria.
5. Make it consistent: Inconsistent data modeling applied inconsistently to existing data can result in a number of problems including blind spots, inaccurate diagnoses or nonsensical interpretations.
6. Start small with a targeted use case: Big things have small beginnings with plenty of validation. We’ve found most success by building a small automation framework to eliminate tasks that are repetitive, high-value, and have limited human interaction.

It may be easy to move data in a truck, but considerably harder to use it as an asset in an efficient framework of IT operations management. As data increases in volume, velocity and variety, it’s going to become increasingly important to start considering it in context for accuracy and usefulness. Because as competitive advantages become tougher to find, and digital operations teams are asked to do more with less, tools like artificial intelligence and machine learning will matter more than ever. And when it comes to effective AIOps, the winning strategies start with clean, contextual and useful data.

Feature image via Pixabay.