Infrastructure drift: A multidimensional problem with the need for new DevSecOps tools

As modern infrastructures get more complex everyday, DevOps teams have a hard time tracking infrastructure drift. The multiplicity of factors involved when running sophisticated infrastructures turns this situation into a multidimensional headache with consequences at stake on both production and security.

Modern infrastructures plagued with complex change tracking challenges

As modern infrastructures evolve towards complex, ever-moving “living-like” entities, keeping track of all changes is hardly feasible. The recent proliferation of managed services requiring additional tooling and IAM roles does nothing to temper this situation.

Beyond inevitable manual changes and despite the best GitOps process, some actions from authenticated apps and services will trigger unexpected changes to infrastructures.

Infrastructure drift: A multidimensional problem

In real-world Op’s life, DevOps teams usually manage multiple projects with several environments and various setups, sometimes over two or three clouds. That’s where things get worse. Indeed, the multiplicity of parameters turns infrastructure drift into a multidimensional issue as this situation implies tracking changes across a combination of setups over time. Among those factors count:

• IaaS accounts multiplicity: running one or several projects on different clouds.
• Providers heterogeneity: using different cloud providers versions depending on the projects and the environments.
• IaC languages multiplication: teams using different infrastructure automation tools (Terraform, CloudFormation, Pulumi…) within the same organization, sometimes on the same project.
• Unexpected changes over time of some basic default settings, based on unilateral decisions on the cloud providers side.

The need for generic tools to keep code and infrastructures in sync arises

One of the consequences of this complex multidimensional problem is a costly toil with a productivity impact for DevOps teams that need to fix issues on a regular basis. Another one, more DevSecOps related, is the fact that those changes open blind spots and are a source of potential security issues.

In the wade of this evolution, rises the need for generic tools, across clouds and automation languages to act as GitOps reconcilers and ensure that code and infrastructure stay in sync.

Multiple experiences on infrastructures of various sizes with similar issues made the team behind the driftctl project aware of the problem to solve. Before the initial release, they spent time asking hundreds of infrastructure teams, SREs, etc. around the world where they were standing in their Infrastructure as Code journey and describe their challenges.

The fact that changes were still happening outside of their infrastructure code was clearly one of the most pregnant issues they were facing with no obvious improvement of the situation in the near future. Some of them went as far as cobbling up some internal tool, but were clearly expecting a more complete off-the-shelf solution.

driftctl a free and open source CLI that catches drift outside of Terraform

driftctl is a free and open source CLI that warns of infrastructure drift and fills in the missing piece between static code analysis and runtime scanning in your DevSecOps toolbox.

Initially released mid December 2020, the tool presently compares the AWS API against Terraform state files to catch unexpected modifications and all manual changes (on the console or through the API) outside of the infrastructure code. More cloud providers and automation languages will come as the project moves forward.

A growing community emerges around this fully free and open source project (Apache 2.0 licence), with active contributions from various parts of the world, such as the USA, Japan, Europe… and GitHub discussions originating from many more places.

Towards an automation monitoring stack

Eric Mahé, CEO at CloudSkiff declares: “Infrastructure automation is a fantastic technical leap with lots of promises. But experience clearly shows us that automation should be monitored to ensure that code and platforms always stay in sync. driftctl is the first step of a journey that will lead us to ensure that automation provides all its benefits without triggering additional issues”.

“The mere notion of drift is wide and gets even wider the more you dig into it. So does the list of issues related to it for DevOps and DevSecOps teams. There are still a lot of aspects to address which is why we have additional tools coming up to ensure a full sync between code and infrastructures”, said Stephane Jourdan, CTO.