Using a Devpod

A devpod is a Pod running in your Yaook kubernetes cluster. You can use it to run Yaook operators when running them on your local machine is not sufficient for development purposes. The prime example of this is the infra operator, which requires full network connectivity into the cluster (to provision database/amqp users): Typically, your development machine does not have full network connectivity into the cluster and can thus not be used to run the infra operator.

In order to still be able to quickly iterate on the infra operator, you can run it inside a Pod in the Kubernetes cluster. By installing SSH, rsync and Python inside that Pod and using a ServiceAccount with the necessary privileges, you can run operators inside the Pod quickly.

Prerequisites

  • Working Yaook-compatible kubernetes cluster

  • The CRDs, roles and cluster roles of the operators must exist in the cluster

  • SSH keypair

Creating the Devpod

The repository comes with an example devpod which is geared toward development of the infra-operator. It assumes that your YAOOK_OP_NAMESPACE is set to yaook. If you use a namespace other than ``yaook``, edit `tools/devpod.yaml` and replace the namespace accordingly.

Furthermore, if you need to give the devpod other permissions than of the ones of the infra-operator, replace the name of the cluster role and role that are referenced in its ClusterRoleBinding and RoleBinding, respectively.

$ kubectl apply -n $YAOOK_OP_NAMESPACE -f tools/devpod.yaml

The pod will not start right away. You need to supply it with your SSH public key by creating a file tools/devpod-pubkey.yaml with the configmap:

---
apiVersion: v1
kind: ConfigMap
metadata:
name: devpubkey
data:
    pubkey: <insert your SSH public key here>

Apply the config map using kubectl apply -n $YAOOK_OP_NAMESPACE  -f tools/devpod-pubkey.yaml and wait for the container to be up with kubectl -n $YAOOK_OP_NAMESPACE  get pods | grep dev

Configuring the Devpod for a specific service

Specify the service, which you want to replace by the devpod with export COMPONENT="...". To replace keystone for example use export COMPONENT="keystone"

To avoid conflicts with the original service the old service-pod has to be reduced to zero replicas to effectively disable the pod.

kubectl -n $YAOOK_OP_NAMESPACE  scale --replicas=0 deployment/$COMPONENT-operator

First start of the Devpod

After the devpod has finished starting (it takes a minute or two to install and configure the SSH server as well as some Yaook dependencies), you can connect to it and prepare it for first use:

1. Start and let kubectl port-forward svc/devpod -n $YAOOK_OP_NAMESPACE 2222:22 run in the background. This will make the SSH service of the devpod available at localhost:2222.

Note

kubectl port-forward (like other long-running kubectl commands such as kubectl logs) will exit after a specific timeout if no traffic is sent through the forwarding.

As long as you have an SSH session open, that will not be a problem because SSH is configured to send pings. However, if no SSH session is active, the forwarding may break after a few minutes.

  1. Upload your code directory to the devpod:

    $ rsync -e 'ssh -p 2222' -zraHEAXSP --exclude .git --exclude .direnv --exclude .mypy_cache --delete --exclude _build ./ localhost:/data/src/

    Note

    To make this command easier to use, it may be sensible to add an alias to your ssh config for devpod.

  2. Install Yaook operator. SSH into the devpod, enter /data/src and install the Yaook operator using pip install -e .. The easiest way is to do this into a virtual environment using a .envrc file (direnv is installed in the Pod).

  3. Start an operator, for instance the infra operator, using:

    $ python3 -m yaook.op infra run

Continuous Development

Manually calling rsync is annoying, so here’s a nice litte inotifywatch based thing which will re-upload the code directory on any change to a .py file. Save the following script as watching:

#!/bin/bash
dirs=()
while [ "$#" -gt 0 ]; do
arg="$1"
shift
if [ "$arg" = ";" ]; then
    break
fi
dirs+=("$arg")
done
(echo; sleep 0.1; echo 'watching:' "${dirs[@]}" >&2; inotifywait -r -e CLOSE_WRITE -m "${dirs[@]}" | grep -P "\.(py|rst)$" --line-buffered) | while read _; do clear && echo -ne "\e[3J" && "$@"; done

Then invoke it as from your Yaook operator repository:

$ /path/to/watching yaook \; rsync -zraHEAXSP [...]

(The rsync command can be the same as above.)