Introduction

As a member of an operations team you want things to be easy, you want things to “just work” the same way in a testing environment as they do in a production one. In this post we’ll cover applications that need to be configured, and how to remove those needed configurations in favor of an application that “just works”.

Environment configurable

Here is our example environment configurable application:

package main

import (
	"fmt"
	"os"

	"github.com/aws/aws-sdk-go/aws"
	"github.com/aws/aws-sdk-go/aws/session"
	"github.com/aws/aws-sdk-go/service/sqs"
)

func main() {
	// Setup aws session
	sess, err := session.NewSession(&aws.Config{
		Region: aws.String("us-west-2"),
	})
	if err != nil {
		panic(err)
	}

	// Setup sqs service
	sqsSvc := sqs.New(sess)

	// Get the queue url from environment variable
	queueUrl := os.Getenv("QUEUE_URL")
	fmt.Println("Queue:", queueUrl)

	// Get queue results
	queueResult, err := sqsSvc.ReceiveMessage(&sqs.ReceiveMessageInput{
		QueueUrl: aws.String(queueUrl),
	})
	if err != nil {
		panic(err)
	}

	// Print any messages we received from the queue
	for _, y := range queueResult.Messages {
		fmt.Println("Message:", y)
	}
}

This can be built with go build -o main .

When talking about environment configurable applications we typically see a lot of environment variables used, which is what we used, but this could also be config files or other methods as well.

Our demo application takes an environment variable in the form of an SQS Queue URL. This is considered to be an environment configurable application because it can be deployed to any environment, and as long as you give it the proper environment variable it will work as expected.

An example of how to run this locally: QUEUE_URL=https://test.queue/url ./application

Why this is good and what can be improved?

From an operations perspective this is great, we can deploy the same application to multiple environments without any code changes, this makes it easy to setup a CI/CD workflow for automation.

The downside? It requires specific configuration per environment and while this isn’t necessarily bad because we only have one environment variable to worry about, larger applications require much more configuration and this can take a toll on your operations staff trying to remember and manage.

So how do we fix it? Every application is going to have some level of configuration needed but we can cut down on the amount needed by getting information that would have previously been configured from the environment we’re deployed in.

For example, in the above configurable example we need an SQS Queue URL to run and we pass that in as an environment variable. If we were to name that needed SQS Queue the same in every environment, for example my-new-sqs-queue, we could then have the application use that to get the Queue URL with no configuration needed.

Environment agnostic

Updated application that is environment agnostic:

package main

import (
	"fmt"

	"github.com/aws/aws-sdk-go/aws"
	"github.com/aws/aws-sdk-go/aws/session"
	"github.com/aws/aws-sdk-go/service/sqs"
)

func main() {
	// Setup the aws session
	sess, err := session.NewSession(&aws.Config{
		Region: aws.String("us-west-2"),
	})
	if err != nil {
		panic(err)
	}

	// Setup the sqs service
	sqsSvc := sqs.New(sess)

	// Get the queue url from the environment we're deployed in
	queueUrl, err := sqsSvc.GetQueueUrl(&sqs.GetQueueUrlInput{
		QueueName: aws.String("my-new-sqs-queue"),
	})
	if err != nil {
		panic(err)
	}
	fmt.Println("Queue:", *queueUrl.QueueUrl)

	// Get qeuue results
	queueResult, err := sqsSvc.ReceiveMessage(&sqs.ReceiveMessageInput{
		QueueUrl: queueUrl.QueueUrl,
	})
	if err != nil {
		panic(err)
	}

	// Print any messages we received from the queue
	for _, y := range queueResult.Messages {
		fmt.Println("Message:", y)
	}
}

This can be built with go build -o main .

We’ve updated our application to now look for a SQS Queue named my-new-sqs-queue and get it’s own Queue URL from that based on the environment it’s deployed in.

An example command to run this is simply ./application, no further configuration is necessary.

This application will run in any AWS environment it’s deployed to, provided it has the proper access to read the SQS queue it needs.

Environment consistency

My goal, and the goal of almost every person working in a DevOps/Platform Engineering/SRE role is to reach a point where development, quality, and production environments are nearly identical. We want easily repeatable steps so it can be automated, and to reduce surprises when code is promoted from one environment to another.

This is doable with applications written in a configurable manner but it leads to more work, every application is likely going to need some level of configuration to run and wont be truly agnostic, but where and when possible things should be written to “just work”, your operations team will thank you.

End note

When deploying applications don’t add your environment name to services unless you have a good reason, for example in the case of S3 buckets you need a globally unique name and have to do this.

It may be tempting to do things like servicename-development or servicename-production, but please don’t do if there isn’t a good reason for doing it. As long as the environments are separated you don’t need name differences like this.

For example: my-new-sqs-queue instead of my-new-sqs-queue-[environment]. This allows us to reference the queue by my-new-sqs-queue without needing to know anything else, it “just works”.