Working with CUE ================ For an introduction into how cue is used by Yaook, see the `Developer Reference `__. Create cue templates -------------------- To create another configuration template, create a new directory inside ``./yaook/op/cue/pkg/yaook.cloud/_config_by_yaook`` and add the file ``defaults.cue``. Each dictionary inside ```` will later be mapped to a section inside the final configuration. For example, this part of ``./yaook/op/cue/pkg/yaook.cloud/rabbitmq_config_by_yaook/defaults.cue``: .. code-block:: text "net_ticktime": *10 | uint64 "quorum_queue": { "continuous_membership_reconciliation": { "enabled": *true | bool "target_group_size"?: int } } will be mapped to the following configuration under the usage of the ``sm.ConfigFlavor`` ``CUTTLEFISH`` and with a RabbitMQ cluster size of 3 replicas: .. code-block:: text net_ticktime=10 quorum_queue.continuous_membership_reconciliation.enabled=true quorum_queue.continuous_membership_reconciliation.target_group_size=3 Here is how the value of ``target_group_size`` ends up in the final configuration: The cue rendering process is initiated by the method ``yaook/statemachine/cue.py:_render_cue_configs``, which passes available cue inputs generated by the method ``get_layer`` of the ``sm.CueLayer`` implementations to ``./yaook/common/config.py:build_config``. This will generate a file ``input.cue`` containing the values we need to inject. This file is then used by the ``cue export input.cue`` command which creates the json output of the final configuration content, which is then serialized to the correct configuration format. .. code-block:: text :caption: input.cue import ( "yaook.cloud/rabbitmq_config_by_yaook" ) conf: { rabbitmq_config_by_yaook.rabbitmq_conf_spec ... quorum_queue: { "continuous_membership_reconciliation.target_group_size": 3 } } When you add configuration options to cue templates, the most common use cases will be the following: #. Declare a non-overwritable value .. code-block:: text ssl_ca_file: "/etc/pki/tls/certs/ca-bundle.crt" Use these if it would make no sense to overwrite certain values inside a Yaook manifest. #. Declare an overwritable (default) value In the following example, ``["admin"]`` serves as the default value and ``[string, ...]`` declares the type: .. code-block:: text service_token_roles: *["admin"] | [string, ...] If in doubt, make the configuration variable overwritable. This allows easier configuration changes inside clusters if problems with the default configuration are detected. If we do not want to provide a default value, we can do so by only providing type information: .. code-block:: text service_token_roles: [string, ...] Note that injecting a value with a ``sm.CueLayer`` is mandatory in this case. To make the value optional, which simply omits the setting if not injected, add a question mark after the key: .. code-block:: text service_token_roles?: [string, ...] #. Create and interpolate a variable Here, the variable ``subnet_cidr`` is used to build the ``webserver-allow-from`` option. Variables prefixed with ``#`` like in this case are not included inside the the final output. Variables without the ``#`` prefix can also be refenced inside the interpolation. .. code-block:: text #subnet_cidr: string "webserver-allow-from": *"127.0.0.1,::1,\( #subnet_cidr )" | string Create a cue layer ------------------ Sometimes, you need to create an additional class inheriting ``sm.CueLayer`` to inject needed information into the cue template. This could for example be case if you need to provide hostname and credentials for authentication at a new component. You then would pass the K8s service reference, a username and the K8s secret to the CueLayer during initialization. Before you do this, ensure that no class inheriting ``sm.CueLayer`` inside ``./yaook/statemachine/cue.py`` already serves your purpose. To build the configuration, you first need to assign dependencies such as ``resources.KubernetesReference`` to class members inside the constructor and pass KubernetesReferences to the ``_declare_dependencies`` call inside the constructor so the operator does not attempt to build the configuration part before the dependencies are ready. The part of the configuration is then assembled inside the method ``get_layer`` with help of the method ``declare`` of the ConfigFlavor to generate the ``ConfigDeclaration``. The ``get_layer`` method then needs to return a dictionary that maps targets to a ``ConfigDeclaration``, where each target represents a section of the configuration. Here is an example return value of ``get_layer`` taken from ``CephConfigLayer``, which is injected into the ``glance`` and ``ceph`` section inside ``yaook/op/cue/pkg/yaook.cloud/glance_config_by_yaook/defaults.cue`` to create the ``glance.conf`` file: .. code-block:: python { "glance": , "ceph": } If none of the existing ConfigFlavor matches your purpose, you can create a new ConfigFlavor inside ``yaook/common/config.py`` and write a serialization method. The AMQPTransportLayer serves as a good example for how a CueLayer is set up: .. code-block:: python :caption: ``./yaook/statemachine/cue.py`` class AMQPTransportLayer(CueLayer): def __init__( self, *, target: str, service: resources.KubernetesReference[kclient.V1Service], username: DynamicString, password_secret: resources.KubernetesReference[kclient.V1Secret], **kwargs: typing.Any): super().__init__(**kwargs) self._declare_dependencies(service, password_secret) self._target = target self._service = service self._username = _compile_string(username) self._password_secret = password_secret async def get_layer(self, ctx: context.Context) -> CueInput: return { self._target: yaook.common.config.OSLO_CONFIG.declare([ await resources.make_mq_config_overlay( ctx, mq_service=self._service, mq_username=self._username(ctx), mq_user_password_secret=self._password_secret, ), ]) } .. code-block:: python :caption: ``./yaook/statemachine/resources/base.py`` async def make_mq_config_overlay( ctx: context.Context, mq_service: KubernetesReference[kclient.V1Service], mq_username: str, mq_user_password_secret: KubernetesReference[kclient.V1Secret], ) -> typing.MutableMapping[str, typing.Any]: service_ref = await mq_service.get(ctx) host = f"{service_ref.name}.{service_ref.namespace}" password = await extract_password( ctx, mq_user_password_secret, ) return { "DEFAULT": { "#transport_url_hosts": [host], "#transport_url_username": mq_username, "#transport_url_password": password, "#transport_url_vhost": "", } } Debug cue template rendering ---------------------------- For faster debugging of the rendering, you can set a breakpoint inside ``./yaook/common/config.py:render_cue_template``, write down the scratch space path and stop the operator. Then use this path to find out how the correct ``input.cue`` should look like by modifying it and running ``cue export input.cue`` inside that directory until the template is successfully rendered. Afterwards adjust the cue template or the operator code accordingly.