Resumé-Driven Development Part 1: Setting up a Golang development environment with Kubernetes and Tilt

In which we set up our glorious dev environment.

Kris Kelly
Kris Kelly

So I've never done a blog tutorial series before... or a blog... or written anything longer than a social media post in longer than I care to admit. Naturally, I wanted to start out with something simple... Just kidding! This is going to be complicated.

In this series, we're going to be practicing something called resumé-driven development (RDD), starting with setting up our development environment and continuing through the basics of incorporating several unnecessary but buzzword-y technologies into our app.

In the process, hopefully you'll learn something without nearly as much frantic googling as I had to do. No familiarity with any of the particular technologies is required. In fact, it's much easier for me to pretend like you haven't heard about any of it. As long as you have a basic understanding of object-oriented programming and backend web development, you should be desperately confused good to go!

DISCLAIMER: While seeming to be satire, this tutorial is actually meant to be useful. If you find it to be neither amusing nor useful, then please film a video of yourself reacting disapprovingly, post it on youtube, send me a link, and I will watch that video as I cry myself to sleep late into the night.

Most of these tutorials involve building a todo app. Todo apps are boring. I'm a firm believer that the projects that motivate us best are the ones in which we "scratch our own itch", so I've decided to focus on a topic that is an endless source of frustration fascination for me: dating. I think many of us who are unfortunate enough to be single during this time of plague can agree that it's not fun. Anyway instead of learning to be alone or unpacking emotional baggage, let's build a dating app! Or at least, let's take a... swipe at it (I'm so sorry).

When it comes to apps, and particularly dating apps, user experience is everything. Therefore for this tutorial series, we'll be building the API portion that no one will ever see. Because this could otherwise potentially be very boring, we're going to overcomplicate this thing considerably. Thus I've decided to build it as a GraphQL API written in Go, running on Kubernetes, using both a traditional RDBMS (Postgres) and a NoSQL graph database (Dgraph), with a cloud service (AWS S3) thrown in there for good measure. I might do a series in the future about building the frontend app (most likely Next.js + Apollo), but then again, probably not.

Part 1 of this tutorial will involve setting up our project using Go, Kubernetes, and Tilt. In this case, we'll be setting up a local development Kubernetes cluster with a "Hello World" Go microservice. The first step is to use our big brain skills to give it a unique and exciting name:

$ mkdir dating-app-example
$ cd dating-app-example

The next step is to initialize our Go service. First, let's make sure Go is installed. You can install Go by following instructions here, or install it via homebrew. Now let's initialize the project:

$ go mod init github.com/[username]/dating-app-example

To structure the project, I took a few cues from golang-standards/project-layout, with a very simplified version consisting of the following layout:

$ tree
.
├── Dockerfile
├── Tiltfile
├── cmd
│   └── api.go
├── deployments
│   └── api.yaml
├── go.mod
└── internal

Don't worry about most of these files yet. In terms of the code we are going to write, the file cmd/api.go will be our executable, and all application code will eventually be kept in internal/. Many projects use pkg/ for application code, but since we aren't exposing any of this code as an API to be shared with other projects, we can leave it in internal/.

In the interests of getting our dev environment set up as quickly as possible, we'll stand up a very basic "Hello World" server.

Let's put the following into api.go:

package main

import (
	"log"
	"net/http"
)

func main() {
	http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
		w.WriteHeader(http.StatusOK)
		w.Write([]byte("ok"))
	})
	log.Fatal(http.ListenAndServe(":3000", nil))
}

You can now run this like so:

$ go run cmd/api.go

Now open up localhost:3000 in your browser. Congratulations, a running program! Even if you're not familiar with Go, this code should be relatively self-explanatory. It specifies a function that runs when you access the server at the root path "/", and it sends the string "ok" in the response. For a better introduction to the net/http library, Go has excellent documentation.

Now that that's done, feel free to take a water break. We are going to use a number of brain cells for this tutorial, so it's important to stay hydrated. When you're back, we're going to set up the Kubernetes cluster...

To set up our Kubernetes cluster development environment, we are going to install a tool called Tilt. Tilt's main feature as far as I can tell is that it enables a productive dev environment by managing your local Kubernetes cluster and rebuilding containers automatically as you go. It's best suited to multi-service architectures, so it's probably overkill for what we are trying to do right now. Perfect! Go ahead and follow the instructions to install it.

In order to use Tilt, we need to have a Kubernetes cluster running on our local machine. If you're on Mac, the fastest way to get that running is to install Docker for Mac. Once you've got that installed, Docker has a setting to enable Kubernetes. Enable that setting, and eventually you should see a "Kubernetes is running" message when you click on the Docker icon at the top of the screen.

"Why are we using Kubernetes for this dead-simple app?" you may ask. The quick answer is that Kubernetes is a required component when practicing resumé-driven development. That said, one of the nice things about Go is that you can build binaries that run without any dependencies, so if you wanted to skip this tutorial, you could just continue to compile and run the app manually in the terminal as we did before. Given Docker's abysmal performance on mac, that would probably even save us a few gray hairs.

But that's probably not why you're reading this. The fun starts when we run this app as a Docker container on a local k8s cluster, leveraging Tilt for live updating and k8s for managing our database(s), thus getting us as close as we can to what will eventually be our production environment.

Now that we've got Kubernetes running, we need to set up our Tilt dev environment. For our single Go service, that consists of three things:

  • Dockerfile for the service
  • Kubernetes deployment yaml file
  • Tiltfile

Let's start with the Dockerfile:

FROM golang:1.14-alpine

RUN apk add --no-cache git

WORKDIR /app

COPY go.mod .
# COPY go.sum .

RUN go mod download

COPY . .

RUN go build -o /app/build/api /app/cmd

CMD ["/app/build/api"]

I won't go over the basics of how Docker works, but the gist is that this copies the code over to the Docker image, downloads any dependencies listed in go.mod, builds the executable, and runs it. Since we'll be building this Docker image frequently during development, there's a "trick" for improving the build performance via the COPY go.mod . command that runs before the main COPY . . command. That allows Docker to cache the list of Go modules so that Docker will only re-download those dependencies if that list changes. NOTE: The go.sum file does not exist because there are no dependencies yet, so you'll want to uncomment that line of the Dockerfile once you've added some dependencies.

Next up, we want to add a basic deployment file for our k8s cluster. We'll call it deployments/api.yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: api
  labels:
    app: api
spec:
  selector:
    matchLabels:
      app: api
  template:
    metadata:
      labels:
        app: api
    spec:
      containers:
        - name: api
          image: dating-app/api
          command: ["/app/build/api"]
          ports:
            - containerPort: 3000

This defines a Kubernetes Deployment, the concept of which the Kubernetes docs will do a much better job of explaining than I can. In a nutshell, we define a container named api that is built using an image we've defined in our Tiltfile called dating-app/api.

The Tiltfile is a script that orchestrates our dev environment within Kubernetes. We are going to add the following to our Tiltfile in the root directory of the app:

load('ext://restart_process', 'docker_build_with_restart')

k8s_yaml('deployments/api.yaml')

docker_build_with_restart('dating-app/api', '.',
    entrypoint='/app/build/api',
    ignore=['./Dockerfile', '.git'],
    live_update=[
        sync('.', '/app'),
        run('go build -o /app/build/api /app/cmd'),
    ]
)

k8s_resource('api', port_forwards=[3000])

Right now this file does 3 things:

  • specifies the locations of the Kubernetes deployment yaml via k8s_yaml()
  • calls docker_build_with_restart() to specify that we want to build a Docker image called dating-app/api (the name we use in the deployment yaml)
  • sets up port-forwarding to make the app accessible via localhost:3000

The load() call at the top enables the docker_build_with_restart() feature, which is an important performance optimization because rebuilding the Docker image on every file change can be extremely slow. Instead, using the live_update argument, we sync file changes with the existing container and rebuild the app within the running container. This makes the development cycle snappy, and in my humble opinion, this should be a default rather than an "extension", because iterative development in Tilt is pretty slow without it.

So without further ado, let's go ahead and fire up Tilt. Open a new terminal tab for your project and run:

$ tilt up

In the browser, you should see your api container building properly and, once it finishes, the app should be accessible at localhost:3000.

Now we have a working dev environment! Stay tuned for Part 2 of the tutorial, setting up a rudimentary GraphQL API.