Using a Devpod

A devpod is a Pod running in your Yaook kubernetes cluster. You can use it to run Yaook operators when running them on your local machine is not sufficient for development purposes. The prime example of this is the infra operator, which requires full network connectivity into the cluster (to provision database/amqp users): Typically, your development machine does not have full network connectivity into the cluster and can thus not be used to run the infra operator.

In order to still be able to quickly iterate on the infra operator, you can run it inside a Pod in the Kubernetes cluster. By installing SSH, rsync and Python inside that Pod and using a ServiceAccount with the necessary privileges, you can run operators inside the Pod quickly.

Prerequisites

  • Working Yaook-compatible kubernetes cluster

  • The CRDs, roles and cluster roles of the operators must exist in the cluster

  • SSH keypair

Creating the Devpod

The repository comes with an example devpod which is geared toward development of the infra-operator. It assumes that your YAOOK_NAMESPACE is set to default. If you use a namespace other than ``default``, edit `tools/devpod.yaml` and replace the namespace accordingly.

$ kubectl apply -n $YAOOK_NAMESPACE tools/devpod.yaml

The pod will not start right away. You need to supply it with your SSH public key by creating a configmap:

---
apiVersion: v1
kind: ConfigMap
metadata:
name: devpubkey
data:
    pubkey: <insert your SSH public key here>

Apply the config map using kubectl.

The pod should start now.

First start of the Devpod

After the devpod has finished starting (it takes a minute or two to install and configure the SSH server as well as some Yaook dependencies), you can connect to it and prepare it for first use:

1. Start and let kubectl port-forward svc/devpod 2222:22 run in the background. This will make the SSH service of the devpod available at localhost:2222.

Note

kubectl port-forward (like other long-running kubectl commands such as kubectl logs) will exit after a specific timeout if no traffic is sent through the forwarding.

As long as you have an SSH session open, that will not be a problem because SSH is configured to send pings. However, if no SSH session is active, the forwarding may break after a few minutes.

  1. Upload your code directory to the devpod:

    $ rsync -e 'ssh -p 2222' -zraHEAXSP --exclude .git --exclude .direnv --exclude .mypy_cache --delete --exclude _build ./ localhost:/data/src/
    

    Note

    To make this command easier to use, it may be sensible to add an alias to your ssh config for devpod.

  2. Install Yaook operator. SSH into the devpod, enter /data/src and install the Yaook operator into a virtual environment. The easiest way is to use a .envrc file (direnv is installed in the Pod).

  3. Start an operator, for instance the infra operator, using:

    $ python3 -m yaook.op infra run
    

Continuous Development

Manually calling rsync is annoying, so here’s a nice litte inotifywatch based thing which will re-upload the code directory on any change to a .py file. Save the following script as watching:

#!/bin/bash
dirs=()
while [ "$#" -gt 0 ]; do
arg="$1"
shift
if [ "$arg" = ";" ]; then
    break
fi
dirs+=("$arg")
done
(echo; sleep 0.1; echo 'watching:' "${dirs[@]}" >&2; inotifywait -r -e CLOSE_WRITE -m "${dirs[@]}" | grep -P "\.(py|rst)$" --line-buffered) | while read _; do clear && echo -ne "\e[3J" && "$@"; done

Then invoke it as from your Yaook operator repository:

$ /path/to/watching yaook \; rsync -zraHEAXSP [...]

(The rsync command can be the same as above.)