µs
Infrastructure as Code for DevOps Organizations

Connected dots symbolizing decentralization
Decentralizes Coordination

µs safely coordinates the order of resource creations, updates, and deletions across deployments of independent DevOps teams.

Circle of arrows symbolizing continuous, reactive execution
Continuous & Reactive

µs deployments are continuously running and automatically adjust the infrastructure to its environment whenever it changes.

Puzzle piece symbolizing compatibility
Broadly Compatible

µs builds on top of Pulumi and supports its resource providers and all existing Pulumi TypeScript programs out-of-the-box.

µs is open source: Find out more below, read the publications and try it out.

Talk to us if you are interested or have ideas!

Example

Webpage deployment illustration
Webpage.

Lets consider a simple static website with a single index.html page hosted in an AWS S3 bucket. While this is unrealistically simple, it suffices to showcase our approach. You can find the full source code and instructions to execute it yourself in our repository.

In descriptive infrastructure as code (IaC) systems, users define a directed acyclic graph (DAG) where each node is a resource, e.g., a db, container, load balancer, or network ACL entry, and arcs are dependencies between them. These dependencies are transitive and order the deployment, i.e., if resource R depends on S, S must be deployed before R and R must not be deployed when S is not deployed. For the webpage, the index must be deployed after and undeployed before the bucket. Have a look at the deployment graph and code describing it for µs using regular TypeScript.

Webpage deployment graph 
const bucket = new Bucket('bucket', {
    website: { indexDocument: 'index.html' }
}); // Creates the S3 bucket of the website
const index = new BucketObject('index', {
    bucket: bucket,
    content: content,
    key: 'index.html',
    contentType: 'text/html; charset=utf-8'
}); // Saves the index.html in the bucket
Webpage deployment graph and the µs deployment definition.

In DevOps organizations, ideally, each team operates its own resources, including deployment. To ensure correct deployment and undeployment order, teams need to manually coordinate whenever resources are deployed, updated or undeployed. This decreases the flexibility of the teams and wastes time due to synchronization. To decouple the deployment times, µs treats deployments not as one-off tasks – as it is the case with all common IaC systems – but as continuously running processes, updating the deployed resources reactively based on the deployment definition and external signals, e.g., changes in another deployment.

In the website example, the provider could be responsible for the bucket and the editor for the index page in it. To ensure that the page is only and correctly deployed when the bucket is, both teams need to manually coordinate whenever the bucket is deployed, updated or undeployed. This means that the editor starts their deployment independently and runs it continuously. Whenever the provider starts or updates their deployment, the editor’s deployment automatically deploys, updates and undeploys the index page, without manual intervention or synchronization.

To enable such behavior, the connections between deployments and inter-deployment dependencies of their resources must be explicit in the deployment definitions. µs introduces three new resource types: RemoteConnection for connections to other deployments, Offer to provide resources or information to a connected deployment, and Wish to access the offer of a connected deployment. Using them, the provider and editor specify their connection to the other deployment. The provider offers its bucket to the editor deployment. The editor specifies its expectation of the offer by defining a wish, allowing to use the offered bucket via wish.offer.

Decentralized webpage deployment graph using µs
Decentralized webpage deployment graph using µs.
const editor = new RemoteConnection('editor');
const bucket = new Bucket('bucket', {
    website: { indexDocument: 'index.html' }
});
new Offer(editor, 'bucket', bucket);
µs deployment definition of the provider.
const provider = new RemoteConnection('provider');
const wish = new Wish<Bucket>(provider, 'bucket');
const index = new BucketObject('index', {
    bucket: wish.offer,
    content: content,
    key: 'index.html',
    contentType: 'text/html; charset=utf-8'
});
µs deployment definition of the editor.

Use Cases

In addition to IaC use cases covered by solutions like Pulumi or AWS CDK, µs enables the following three use cases, which we illustrate with the webpage example.

Asynchronous Deployment

µs enables teams to start their deployments independently without synchronization. Resources are asynchronously deployed once their dependencies are satisfied. E.g., the editor starts its deployment before the provider. The index first remains not deployed as its dependency to the bucket is unsatisfied. Once the provider deploys the bucket, µs deploys the editor’s index page.

Safe Undeployment

µs ensures that all resources depending on a resource R are not deployed anymore, when R is undeployed. If R shall be undeployed, the undeployment of all resources depending on it is triggered and R is only undeployed after completion. E.g., when the provider undeploys the bucket, µs automatically undeploys the editor’s index page before.

Reactive Updates

µs automatically transports configuration changes across deployments, triggering reactive updates. E.g., the editor might show the bucket’s name (wish.offer.name) in the content of the page. If the provider updates the name of the bucket, this change is transported to the editor’s deployment, which subsequently automatically updates the page’s content to the new name.

Animated deployment use cases animation.

Publications

  1. Reliable infrastructure as Code for Decentralized Organizations

    Daniel Sokolowski
    Doctoral dissertation, Universität St. Gallen, 2024

    PDF

    IT must be reliable for organizations to thrive and quickly adaptable for their swift reaction to their environment. Agility is vital, and DevOps achieves these goals by empowering independent cross-functional teams in decentralized organizations and automating the entire software pipeline. Infrastructure as Code (IaC) is the critical tool to automate software operations, including infrastructure provisioning, application deployment, and configuration. Beyond simple IaC scripts, developers implement IaC programs in programming languages like TypeScript and Python. Such IaC programs are software, and their reliability is crucial to the functionality and security of the deployed systems. Still, techniques for the rapid development of reliable IaC programs are missing, limiting organizations’ agility. Specifically, developers lack automation for deployment coordination and updating and quality assurance tools for, e.g., testing and verification.

    We surveyed 134 IT professionals, finding that coordination across deployments is commonly needed and often requires manual coordination, even though IT professionals believe automated coordination yields better agility. However, automated approaches are centralized, limiting team independence and agility in decentralized organizations. To solve this issue, we propose automating coordination across deployments in a decentralized fashion through μs ([mju:z] “muse”), a novel IaC solution. With μs, teams have separate IaC programs, which express and jointly automate the correct order of operations across deployments. We further show how μs enables safe updating through IaC programs, preventing updates from breaking distributed transactions or workflows.

    Beyond automating the coordination of IaC programs, we address the reliability of IaC program code. To unblock studies, we built a dataset of 37 712 public IaC programs. In initial analyses, only a vanishing fraction implements tests. We identified that available testing techniques are either slow and resource-intensive or require excessive development effort. To solve this dilemma, we propose ACT, an extensible automated unit testing approach that enables testing IaC programs quickly in hundreds of configurations, often without writing additional testing code.

    This dissertation studies the coordination and testing of IaC programs. Empirically motivated, we present μs for safe deployment coordination and updating in decentralized setups and ACT for efficient testing of IaC programs. Our contributions nurture future research and reliable deployments in decentralized organizations, ensuring agility.

  2. IEEE Software
    Decentralizing Infrastructure as Code

    Daniel Sokolowski, Pascal Weisenburger, Guido Salvaneschi
    IEEE Software 40 (1), 2023

    PDF

    Infrastructure as Code (IaC) automates deployments for single teams, falling short of decentralized deployments across groups. We need mature IaC solutions that embrace and consolidate software engineering principles to enable testing and automation advances for decentralized organizations.

  3. ESEC/FSE DS
    Infrastructure as Code for Dynamic Deployments

    Daniel Sokolowski
    In Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/FSE, 2022

    PDF

    Modern DevOps organizations require a high degree of automation to achieve software stability at frequent changes. Further, there is a need for flexible, timely reconfiguration of the infrastructure, e.g., to use pay-per-use infrastructure efficiently based on application load. Infrastructure as Code (IaC) is the DevOps tool to automate infrastructure. However, modern static IaC solutions only support infrastructures that are deployed and do not change afterward. To implement infrastructures that change dynamically over time, static IaC programs have to be (updated and) re-run, e.g., in a CI/CD pipeline, or configure an external orchestrator that implements the dynamic behavior, e.g., an autoscaler or Kubernetes operator. Both do not capture the dynamic behavior in the IaC program and prevent analyzing and testing the infrastructure configuration jointly with its dynamic behavior.

    To fill this gap, we envision dynamic IaC, which augments static IaC with the ability to define dynamic behavior within the IaC program. In contrast to static IaC programs, dynamic IaC programs run continuously. They re-evaluate program parts that depend on external signals when these change and automatically adjust the infrastructure accordingly. We implement DIaC as the first dynamic IaC solution and demonstrate it in two realistic use cases of broader relevance. With dynamic IaC, ensuring the program’s correctness is even harder than for static IaC because programs may define many target configurations in contrast to only a few. However, for this reason, it is also more critical. To solve this issue, we propose automated, specialized property-based testing for IaC programs and implement it in ProTI.

  4. ESEC/FSE
    Automating Serverless Deployments for DevOps Organizations

    Daniel Sokolowski, Pascal Weisenburger, Guido Salvaneschi
    In Proceedings of the 29th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/FSE, 2021

    PDF Supp Code

    DevOps unifies software development and operations in cross-functional teams to improve software delivery and operations (SDO) performance. Ideally, cross-functional DevOps teams independently deploy their services, but the correct operation of a service often demands other services, requiring coordination to ensure the correct deployment order. This issue is currently solved either with a central deployment or manual out-of-band communication across teams, e.g., via phone, chat, or email. Unfortunately, both contradict the independence of teams, hindering SDO performance – the reason why DevOps is adopted in the first place.

    In this work, we conduct a study on 73 IT professionals, showing that, in practice, they resort to manual coordination for correct deployments even if they expect better SDO performance with fully automated approaches. To address this issue, we propose μs ([mju:z] “muse” ), a novel IaC system automating deployment coordination in a fully decentralized fashion, still retaining compatibility with DevOps practice – in contrast to today’s solutions. We implement µs, demonstrate that it effectively enables automated coordination, introduces negligible definition overhead, has no performance overhead, and is broadly applicable, as shown by the migration of 64 third-party IaC projects.

  5. ESEC/FSE
    Deployment Coordination for Cross-Functional DevOps Teams

    Daniel Sokolowski
    In Proceedings of the 29th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/FSE, 2021

    PDF

    Software stability and reliability are the core concerns of DevOps. They are improved by tightening the collaboration between developers and operators in cross-functional teams on the one hand and by automating operations through continuous integration (CI) and infrastructure as code (IaC) on the other hand. Ideally, teams in DevOps are fully independent. Still, their applications often depend on each other in practice, requiring them to coordinate their deployment through centralization or manual coordination.

    With this work, we propose and implement the novel IaC solution µs ([mju:z] ”muse”), which automates deployment coordination in a decentralized fashion. µs is the first approach that is compatible with the DevOps goals as it enables truly independent operations of the DevOps teams. We define our research problem through a questionnaire survey with IT professionals and evaluate the solution by comparing it to other modern IaC approaches, assessing its performance, and applying it to existing IaC programs.