Evaluate requirements

Before building the operator, you first want to read documentation about the software architecture you want to deploy to evaluate which services and components need to be created with Kubernetes in which order and how they need to be configured. For example, with an OpenStack component, this usually includes the following as a bare minimum:

• Stateful components like database and messaging broker

• Stateless services (K8s deployments)

• Execution of commands for the initial setup (K8s jobs or preStart hooks)

• Configuration files (K8s secrets or configMaps)

• K8s service for the API service

When you add a subresource to the custom resource class, you will need to know which statemachine class is used by Yaook to manage it. The following table lists examples for frequently used mappings between K8s resources (including Yaook custom resources) and classes from the statemachine module.

Component	Templated class
Database	sm.TemplatedMySQLService
Database user	sm.SimpleMySQLUser
Messaging broker	sm.TemplatedAMQPServer
Messaging broker user	sm.SimpleAMQPUser
Keystone user	sm.StaticKeystoneUser or sm.StaticKeystoneUserWithParent
K8s Deployment	sm.TemplatedDeployment
K8s Statefulset	sm.TemplatedStatefulSet
K8s Job	sm.TemplatedJob
K8s Certificate	sm.TemplatedCertificate
K8s Secret	sm.CueSecret

You can find other availabe classes inside ./yaook/statemachine/resources/k8s_workload.py and ./yaook/statemachine/resources/yaook_infra.py, or you can inspect other operators to see how these handle certain scenarios.

Note that most custom resources managed by different operators like the sm.MySQLService are not shared by multiple OpenStack components and a new instance should be created for each OpenStack CR. There are some exception to this rule, with the central KeystoneDeployment being one example.

If you need another external component that can be referenced by other Yaook operators, add it to the infra operator under ./yaook/op/infra instead of creating a new operator.

Example evaluation

We will now use the Barbican installation guide as an example to find out which Yaook classes we can use to automate these steps. Note that this guide only serves as demonstration and is not intended for copy-pasting as parameters are subject to change and many are left out to focus on main concepts. The instructions were originally taken from here. For other OpenStack components a similiar guide should be part of the OpenStack documentation.

The prerequisites section first instructs us to setup a database with required priviledges for the database user. To achieve this, we add the following to the class of our CR:

class Barbican(sm.ReleaseAwareCustomResource):
    ...
    db = sm.TemplatedMySQLService(...)
    db_service = sm.ForeignResourceDependency(...)
    db_api_user_password = sm.AutoGeneratedPassword(...)
    db_api_user = sm.SimpleMySQLUser(
        password_secret=db_api_user_password,
    )

Likewise, we also need to create an AMQP server and user so services can communicate with each other:

class Barbican(sm.ReleaseAwareCustomResource):
    ...
    mq = sm.TemplatedAMQPServer(...)
    mq_service = sm.ForeignResourceDependency(...)
    mq_api_user_password = sm.AutoGeneratedPassword(...)
    mq_api_user = sm.SimpleAMQPUser(
        password_secret=mq_api_user_password,
    )

Next, the OpenStack CLI is used to create the OpenStack user barbican along with its role mappings. In Yaook, we will add the following:

class Barbican(sm.ReleaseAwareCustomResource):
    ...
    keystone = sm.KeystoneReference()
    keystone_user = sm.StaticKeystoneUser(
        keystone=keystone,
        username="barbican"
    )

And instead of running

openstack service create --name barbican --description "Key Manager" key-manager

we create the Keystone endpoint using the sm.TemplatedKeystoneEndpoint class:

class Barbican(sm.ReleaseAwareCustomResource):
    ...
    keystone_user_credentials = \
    yaook.op.common.keystone_user_credentials_reference(
        keystone_user
    )
    keystone_endpoint = sm.Optional(
        condition=yaook.op.common.publish_endpoint,
        wrapped_state=sm.TemplatedKeystoneEndpoint(
            template="barbican-keystone-endpoint.yaml",
            add_dependencies=[keystone],
        )
    )

Now, we need to create a configuration file to prepare connectivity to the MySQLService, the AMQPServer and Keystone. The configuration is placed inside a K8s secret which we can then mount to the deployments, statefulsets and jobs. If a configuration does not contain sensitive credentials, the class sm.CueConfigMap is more appropriate.

We first need to create a cue template, whose directory needs to be referenced with the sm.CueSecret, and which will be placed inside ./yaook/op/cue/pkg/yaook.cloud/barbican_config_by_yaook/defaults.cue, in this particular case. For each connected component, we also need to pass a separate CueLayer as parameters, where each of these layer requires customized parameters like credentials or service references that are used to assemble the configuration. Depending on your component and use case, you might have to add additional layers (if you look at the final Barbican class, there are actually more than three layers).

class Barbican(sm.ReleaseAwareCustomResource):
    ...
    config = sm.CueSecret(
        # "barbican-config-" references the directory
        metadata=("barbican-config-", True),
        add_cue_layers=[
            sm.KeystoneAuthLayer(
                target="barbican",
                credentials_secret=keystone_user_credentials,
                endpoint_config=keystone_internal_api,
            ),
            sm.DatabaseConnectionLayer(
                target="barbican",
                service=db_service,
                database_name=DATABASE_NAME,
                username=API_SVC_USERNAME,
                password_secret=db_api_user_password,
                config_section="database",
            ),
            sm.AMQPTransportLayer(
                target="barbican",
                service=mq_service,
                username=API_SVC_USERNAME,
                password_secret=mq_api_user_password,
            ),
            ...
        ]

Before the Barbican services can be started, the database needs to be populated via barbican-manage db upgrade which can be executed inside a Kubernetes job. Here, we also need to add our Docker image as dependency to access the command line utility, and we need to add the configuration we just created to the dependencies in order to gain database access. The latter will also ensure that the configuration secret is created before this job.

class Barbican(sm.ReleaseAwareCustomResource):
    barbican_docker_image = sm.ReleaseAwareVersionedDependency({
        ...
        "2025.1": sm.ConfigurableVersionedDockerImage(
                'barbican-2025.1',
                sm.YaookSemVerSelector(),
            ),
        },
        targetfn=lambda ctx: sm.version_utils.get_target_release(ctx)
    )
    ...
    db_sync = sm.Optional(
        condition=sm.optional_non_upgrade(),
        wrapped_state=sm.TemplatedJob(
            template="barbican-job-db-sync.yaml",
            add_dependencies=[config, ...],
            versioned_dependencies=[barbican_docker_image]
        )
    )

In the end the guide instructs us to start the Barbican services:

service barbican-keystone-listener restart
service barbican-worker restart
service apache2 restart

Instead of using Apache, we will run these services inside separate Kubernetes deployments using the images we created in the beginning and overwrite the spec.command with the corresponding command inside the jinja templates. Again, to ensure that the database sync job finished before the deployment was created, db_sync is added to the add_dependencies property of each sm.TemplatedDeployment, and again we need the config to create a volume mount inside the jinja templates.

class Barbican(sm.ReleaseAwareCustomResource):
    ...
    keystone_listener = sm.TemplatedDeployment(
        template="barbican-deployment-keystone-listener.yaml",
        add_dependencies=[
            config, db_sync, ...],
        ...
    )
    api_deployment = sm.TemplatedDeployment(
        template="barbican-deployment-api.yaml",
        add_dependencies=[
            config, db_sync, ...],
        ...
    )