Overview

In this blog we’ll cover how the Astronomy Picture of the Day microservices are built and deployed with Docker and Kubernetes, and how to deploy the lightweight MQTT broker Mosquitto.

This is a follow up to my previous blog post System Scalability and Microservices with some additional information.

Requirements

• Docker installed locally
• Working Kubernetes cluster
• Helm installed

GitHub repositories

All of the code relating to this blog post can be found at the following links and is released under the GPL-3.0 license, with the exception of the official NASA api which is released under the Apache-2.0 license.

• Official NASA APOD API - The official NASA API for getting the Astronomy Picture of the Day.
• APOD - The API wrapper we use to query the NASA api.
• Apod-to-MQTT - This service uses the api wrapper above to actually do the query.
• Apod-MQTT-to-Discord - This service reads off an MQTT subscription and publishes the messages to Discord.

Kubernetes Setup

First we’ll setup two namespaces to run our services in. We’re putting Mosquitto in it’s own namespace so we can reuse it for other projects in the future.

kubectl create ns mqtt
kubectl create ns apod

Next we’ll need to create two Kubernetes secrets to hold the Docker credentials of our repository. This allows Kubernetes to connect to and pull images from a private repo, in this blog we’re using the GitHub Container Repository.

kubectl create secret docker-registry ghcr-credentials --docker-server=ghcr.io --docker-username=<myuser> --docker-email=<myemail> --docker-password=<mypersonaltoken> -n mqtt
kubectl create secret docker-registry ghcr-credentials --docker-server=ghcr.io --docker-username=<myuser> --docker-email=<myemail> --docker-password=<mypersonaltoken> -n apod

We’ll then need to create another Kubernetes secret to hold our APOD_API_KEY and DISCORD_BOT_TOKEN.

 kubectl create secret generic --from-literal=APOD_API_KEY=<apod-key> --from-literal=APOD_DISCORD_BOT_TOKEN=<discord-bot-token> -n apod

Lastly we’ll apply a configMap with some general configurations for our microservices. In the Apod-to-MQTT and Apod-MQTT-to-Discord repos there are example configMaps. Combine those two into a single file that looks like the one below with your configuration and apply it with kubectl apply -f <configMapfile.yaml> -n apod

apiVersion: v1
kind: ConfigMap
metadata:
  name: apod
data:
  MQTT_BROKER: <tcp://MQTT Broker>
  MQTT_TOPIC: <MQTT Topic>
  DISCORD_CHANNEL_ID: "<Discord channel ID>"

With all of that done our cluster is ready to host our services

MQTT - Mosquitto¹

Since our services use MQTT to communicate we’ll need to deploy that first.

Mosquitto is an incredibly lightweight MQTT broker, and the entire install is 3 commands.

helm repo add k8s-at-home https://k8s-at-home.com/charts/ # Add the helm repo
helm repo update # Update helm repos
helm install mosquitto k8s-at-home/mosquitto -n mqtt # Install mosquitto to the mqtt namespace

You can edit the values.yaml file to provide custom deployment options, but it’s not necessary. A default install will work for our purposes.

Discord Bot and Apod Publisher

Rename the apoddiscordbot.yaml.example file in the Apod-MQTT-to-Discord repo and edit it with any values you want to substitute (like the image location), and then deploy with kubectl apply -f <file.yaml> -n apod

Rename the apodtomqtt-cronjob.yaml.example file in the Apod-to-MQTT repo and edit it with any values to want to substitute (like the image location), and then deploy with kubectl apply -f <file.yaml> -n apod

Verification

You should be able to run kubectl get all -n apod and view the new services.

kubectl get all -n apod
NAME                   READY   STATUS    RESTARTS   AGE
pod/apod-discord-bot   1/1     Running   0          63m

NAME                                 SCHEDULE     SUSPEND   ACTIVE   LAST SCHEDULE   AGE
cronjob.batch/apod-to-mqtt-cronjob   0 16 * * *   False     0        7h6m            30d

Conclusion

This blog post provided more input on how to build and deploy the services we talked about previously. At some point in the future I may do a final post to wrap up the loose ends with debugging and an in-depth explaniation on what the Kubernetes manifest files are doing, but for now this post should answer the majority of questions I’ve received.