µs
Infrastructure as Code for DevOps Organizations

Connected dots symbolizing decentralization
Decentralizes Coordination

µs safely coordinates the order of resource creations, updates, and deletions across deployments of independent DevOps teams.

Circle of arrows symbolizing continuous, reactive execution
Continuous & Reactive

µs deployments are continuously running and automatically adjust the infrastructure to its environment whenever it changes.

Puzzle piece symbolizing compatibility
Broadly Compatible

µs builds on top of Pulumi and supports its resource providers and all existing Pulumi TypeScript programs out-of-the-box.

µs is open source: Find out more below, read the publications and try it out.

Talk to us if you are interested or have ideas!

Example

Webpage deployment illustration

Lets consider a simple static website with a single index.html page hosted in an AWS S3 bucket. While this is unrealistically simple, it suffices to showcase our approach. You can find the full source code and instructions to execute it yourself in our repository.

In descriptive infrastructure as code (IaC) systems, users define a directed acyclic graph (DAG) where each node is a resource, e.g., a db, container, load balancer, or network ACL entry, and arcs are dependencies between them. These dependencies are transitive and order the deployment, i.e., if resource R depends on S, S must be deployed before R and R must not be deployed when S is not deployed. For the webpage, the index must be deployed after and undeployed before the bucket. Have a look at the deployment graph and code describing it for µs using regular TypeScript.

Webpage deployment graph
const bucket = new Bucket('bucket', {
    website: { indexDocument: 'index.html' }
}); // Creates the S3 bucket of the website
const index = new BucketObject('index', {
    bucket: bucket,
    content: content,
    key: 'index.html',
    contentType: 'text/html; charset=utf-8'
}); // Saves the index.html in the bucket

In DevOps organizations, ideally, each team operates its own resources, including deployment. To ensure correct deployment and undeployment order, teams need to manually coordinate whenever resources are deployed, updated or undeployed. This decreases the flexibility of the teams and wastes time due to synchronization. To decouple the deployment times, µs treats deployments not as one-off tasks – as it is the case with all common IaC systems – but as continuously running processes, updating the deployed resources reactively based on the deployment definition and external signals, e.g., changes in another deployment.

In the website example, the provider could be responsible for the bucket and the editor for the index page in it. To ensure that the page is only and correctly deployed when the bucket is, both teams need to manually coordinate whenever the bucket is deployed, updated or undeployed. This means that the editor starts their deployment independently and runs it continuously. Whenever the provider starts or updates their deployment, the editor’s deployment automatically deploys, updates and undeploys the index page, without manual intervention or synchronization.

To enable such behavior, the connections between deployments and inter-deployment dependencies of their resources must be explicit in the deployment definitions. µs introduces three new resource types: RemoteConnection for connections to other deployments, Offer to provide resources or information to a connected deployment, and Wish to access the offer of a connected deployment. Using them, the provider and editor specify their connection to the other deployment. The provider offers its bucket to the editor deployment. The editor specifies its expectation of the offer by defining a wish, allowing to use the offered bucket via wish.offer.

Decentralized webpage deployment graph using µs
const editor = new RemoteConnection('editor');
const bucket = new Bucket('bucket', {
    website: { indexDocument: 'index.html' }
});
new Offer(editor, 'bucket', bucket);
const provider = new RemoteConnection('provider');
const wish = new Wish<Bucket>(provider, 'bucket');
const index = new BucketObject('index', {
    bucket: wish.offer,
    content: content,
    key: 'index.html',
    contentType: 'text/html; charset=utf-8'
});

Use Cases

In addition to IaC use cases covered by solutions like Pulumi or AWS CDK, µs enables the following three use cases, which we illustrate with the webpage example.

Asynchronous Deployment

µs enables teams to start their deployments independently without synchronization. Resources are asynchronously deployed once their dependencies are satisfied. E.g., the editor starts its deployment before the provider. The index first remains not deployed as its dependency to the bucket is unsatisfied. Once the provider deploys the bucket, µs deploys the editor’s index page.

Safe Undeployment

µs ensures that all resources depending on a resource R are not deployed anymore, when R is undeployed. If R shall be undeployed, the undeployment of all resources depending on it is triggered and R is only undeployed after completion. E.g., when the provider undeploys the bucket, µs automatically undeploys the editor’s index page before.

Reactive Updates

µs automatically transports configuration changes across deployments, triggering reactive updates. E.g., the editor might show the bucket’s name (wish.offer.name) in the content of the page. If the provider updates the name of the bucket, this change is transported to the editor’s deployment, which subsequently automatically updates the page’s content to the new name.

Animated deployment use cases animation.

Publications

  1. ESEC/FSE
    Automating Serverless Deployments for DevOps Organizations

    In Proceedings of the 29th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/FSE, 2021

    PDF Supp Code

    DevOps unifies software development and operations in cross-functional teams to improve software delivery and operations (SDO) performance. Ideally, cross-functional DevOps teams independently deploy their services, but the correct operation of a service often demands other services, requiring coordination to ensure the correct deployment order. This issue is currently solved either with a central deployment or manual out-of-band communication across teams, e.g., via phone, chat, or email. Unfortunately, both contradict the independence of teams, hindering SDO performance – the reason why DevOps is adopted in the first place.

    In this work, we conduct a study on 73 IT professionals, showing that, in practice, they resort to manual coordination for correct deployments even if they expect better SDO performance with fully automated approaches. To address this issue, we propose μs ([mju:z] “muse” ), a novel IaC system automating deployment coordination in a fully decentralized fashion, still retaining compatibility with DevOps practice – in contrast to today’s solutions. We implement µs, demonstrate that it effectively enables automated coordination, introduces negligible definition overhead, has no performance overhead, and is broadly applicable, as shown by the migration of 64 third-party IaC projects.

  2. ESEC/FSE
    Deployment Coordination for Cross-Functional DevOps Teams

    In Proceedings of the 29th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/FSE, 2021

    PDF

    Software stability and reliability are the core concerns of DevOps. They are improved by tightening the collaboration between developers and operators in cross-functional teams on the one hand and by automating operations through continuous integration (CI) and infrastructure as code (IaC) on the other hand. Ideally, teams in DevOps are fully independent. Still, their applications often depend on each other in practice, requiring them to coordinate their deployment through centralization or manual coordination.

    With this work, we propose and implement the novel IaC solution µs ([mju:z] ”muse”), which automates deployment coordination in a decentralized fashion. µs is the first approach that is compatible with the DevOps goals as it enables truly independent operations of the DevOps teams. We define our research problem through a questionnaire survey with IT professionals and evaluate the solution by comparing it to other modern IaC approaches, assessing its performance, and applying it to existing IaC programs.