Working with CUE¶

For an introduction into how cue is used by Yaook, see the Developer Reference.

Create cue templates¶

To create another configuration template, create a new directory inside ./yaook/op/cue/pkg/yaook.cloud/<new_component>_config_by_yaook and add the file defaults.cue. Each dictionary inside <newcomponent_conf_spec> will later be mapped to a section inside the final configuration.

For example, this part of ./yaook/op/cue/pkg/yaook.cloud/rabbitmq_config_by_yaook/defaults.cue:

"net_ticktime":               *10 | uint64
"quorum_queue": {
    "continuous_membership_reconciliation": {
        "enabled":            *true | bool
        "target_group_size"?: int
    }
}

will be mapped to the following configuration under the usage of the sm.ConfigFlavor CUTTLEFISH and with a RabbitMQ cluster size of 3 replicas:

net_ticktime=10
quorum_queue.continuous_membership_reconciliation.enabled=true
quorum_queue.continuous_membership_reconciliation.target_group_size=3

Here is how the value of target_group_size ends up in the final configuration: The cue rendering process is initiated by the method yaook/statemachine/cue.py:_render_cue_configs, which passes available cue inputs generated by the method get_layer of the sm.CueLayer implementations to ./yaook/common/config.py:build_config. This will generate a file input.cue containing the values we need to inject. This file is then used by the cue export input.cue command which creates the json output of the final configuration content, which is then serialized to the correct configuration format.

input.cue¶

 import (
     "yaook.cloud/rabbitmq_config_by_yaook"
 )

 conf: {
     rabbitmq_config_by_yaook.rabbitmq_conf_spec
     ...
     quorum_queue: {
         "continuous_membership_reconciliation.target_group_size": 3
     }
 }

When you add configuration options to cue templates, the most common use cases will be the following:

Declare a non-overwritable value
```
ssl_ca_file: "/etc/pki/tls/certs/ca-bundle.crt"
```
Use these if it would make no sense to overwrite certain values inside a Yaook manifest.
Declare an overwritable (default) value

In the following example, ["admin"] serves as the default value and [string, ...] declares the type:
```
service_token_roles:  *["admin"] | [string, ...]
```
If in doubt, make the configuration variable overwritable. This allows easier configuration changes inside clusters if problems with the default configuration are detected.

If we do not want to provide a default value, we can do so by only providing type information:
```
service_token_roles:  [string, ...]
```
Note that injecting a value with a sm.CueLayer is mandatory in this case. To make the value optional, which simply omits the setting if not injected, add a question mark after the key:
```
service_token_roles?:  [string, ...]
```
Create and interpolate a variable

Here, the variable subnet_cidr is used to build the webserver-allow-from option. Variables prefixed with # like in this case are not included inside the the final output. Variables without the # prefix can also be refenced inside the interpolation.
```
#subnet_cidr: string
"webserver-allow-from": *"127.0.0.1,::1,\( #subnet_cidr )" | string
```

Create a cue layer¶

Sometimes, you need to create an additional class inheriting sm.CueLayer to inject needed information into the cue template. This could for example be case if you need to provide hostname and credentials for authentication at a new component. You then would pass the K8s service reference, a username and the K8s secret to the CueLayer during initialization. Before you do this, ensure that no class inheriting sm.CueLayer inside ./yaook/statemachine/cue.py already serves your purpose.

To build the configuration, you first need to assign dependencies such as resources.KubernetesReference to class members inside the constructor and pass KubernetesReferences to the _declare_dependencies call inside the constructor so the operator does not attempt to build the configuration part before the dependencies are ready.

The part of the configuration is then assembled inside the method get_layer with help of the method declare of the ConfigFlavor to generate the ConfigDeclaration. The get_layer method then needs to return a dictionary that maps targets to a ConfigDeclaration, where each target represents a section of the configuration.

Here is an example return value of get_layer taken from CephConfigLayer, which is injected into the glance and ceph section inside yaook/op/cue/pkg/yaook.cloud/glance_config_by_yaook/defaults.cue to create the glance.conf file:

{
    "glance": <ConfigDeclaration using OSLO_CONFIG>,
    "ceph": <ConfigDeclaration using CEPH_CONFIG>
}

If none of the existing ConfigFlavor matches your purpose, you can create a new ConfigFlavor inside yaook/common/config.py and write a serialization method.

The AMQPTransportLayer serves as a good example for how a CueLayer is set up:

./yaook/statemachine/cue.py¶

 class AMQPTransportLayer(CueLayer):
     def __init__(
             self,
             *,
             target: str,
             service: resources.KubernetesReference[kclient.V1Service],
             username: DynamicString,
             password_secret: resources.KubernetesReference[kclient.V1Secret],
             **kwargs: typing.Any):
         super().__init__(**kwargs)
         self._declare_dependencies(service, password_secret)
         self._target = target
         self._service = service
         self._username = _compile_string(username)
         self._password_secret = password_secret

     async def get_layer(self, ctx: context.Context) -> CueInput:
         return {
             self._target: yaook.common.config.OSLO_CONFIG.declare([
                 await resources.make_mq_config_overlay(
                     ctx,
                     mq_service=self._service,
                     mq_username=self._username(ctx),
                     mq_user_password_secret=self._password_secret,
                 ),
             ])
         }

./yaook/statemachine/resources/base.py¶

 async def make_mq_config_overlay(
     ctx: context.Context,
     mq_service: KubernetesReference[kclient.V1Service],
     mq_username: str,
     mq_user_password_secret: KubernetesReference[kclient.V1Secret],
     ) -> typing.MutableMapping[str, typing.Any]:
 service_ref = await mq_service.get(ctx)
 host = f"{service_ref.name}.{service_ref.namespace}"
 password = await extract_password(
     ctx,
     mq_user_password_secret,
 )
 return {
     "DEFAULT": {
         "#transport_url_hosts": [host],
         "#transport_url_username": mq_username,
         "#transport_url_password": password,
         "#transport_url_vhost": "",
     }
 }

Debug cue template rendering¶

For faster debugging of the rendering, you can set a breakpoint inside ./yaook/common/config.py:render_cue_template, write down the scratch space path and stop the operator. Then use this path to find out how the correct input.cue should look like by modifying it and running cue export input.cue inside that directory until the template is successfully rendered. Afterwards adjust the cue template or the operator code accordingly.