yaook.statemachine

Each Resource class is responsible for one specific way to manage one type of Kubernetes resource (though it may manage multiple instances) per instance of a Custom Resource which uses that State.

Each CustomResource instantiates one or more instances of the specific State subclasses it needs. During instantiation, final parametrisation (such as paths to templates and the COMPONENT key) are configured.

resources – Objects representing resources

Permissions

The Service Account under which an operator is running needs the following permissions for any resource it manages:

  • create

  • delete

  • list

  • watch

  • patch

  • get

In addition, if any resource is modified with PerNode, the operator will also need the following resources on v1 Node resources:

  • get

  • list

  • watch

For the custom resources implemented by the operator, the following permissions are required:

  • get

  • list

  • watch

  • patch

To report concerning conditions, the operators need the following permissions on event objects:

  • create

Concrete resource classes

Templated Kubernetes resources

TemplatedConfigMapState(*, template[, ...])

Manage a jinja2-templated ConfigMap.

TemplatedSecretState(*, template[, ...])

Manage a jinja2-templated Secret.

TemplatedRoleState(*, template[, ...])

Manage a jinja2-templated Role.

TemplatedServiceAccountState(*, template[, ...])

Manage a jinja2-templated ServiceAccount.

TemplatedRoleBindingState(*, template[, ...])

Manage a jinja2-templated RoleBinding.

TemplatedServiceState(*, template[, ...])

Manage a jinja2-templated Service.

TemplatedDeploymentState(*, template[, ...])

Manage a jinja2-templated Deployment.

TemplatedJobState(*, template[, ...])

Manage a jinja2-templated Job.

TemplatedStatefulSetState(*, template[, ...])

Manage a jinja2-templated StatefulSet.

TemplatedKeystoneEndpointState(*, template)

Manage a jinja2-templated KeystoneEndpoint.

TemplatedConfiguredDaemonSetState(*, template)

Manage a jinja2-templated ConfiguredDaemonSet.

TemplatedPersistentVolumeClaimState(*, template)

Manage a jinja2-templated Deployment.

TemplatedCertificateState(*, template[, ...])

Manage a jinja2-templated CertificateState.

class yaook.statemachine.TemplatedConfigMapState(*, template: str, versioned_dependencies: List[yaook.statemachine.versioneddependencies.VersionedDependency] = [], **kwargs: Any)

Manage a jinja2-templated ConfigMap.

See also

BodyTemplateMixin:

for arguments related to templating.

class yaook.statemachine.TemplatedSecretState(*, template: str, versioned_dependencies: List[yaook.statemachine.versioneddependencies.VersionedDependency] = [], **kwargs: Any)

Manage a jinja2-templated Secret.

See also

BodyTemplateMixin:

for arguments related to templating.

class yaook.statemachine.TemplatedRoleState(*, template: str, versioned_dependencies: List[yaook.statemachine.versioneddependencies.VersionedDependency] = [], **kwargs: Any)

Manage a jinja2-templated Role.

See also

BodyTemplateMixin:

for arguments related to templating.

class yaook.statemachine.TemplatedServiceAccountState(*, template: str, versioned_dependencies: List[yaook.statemachine.versioneddependencies.VersionedDependency] = [], **kwargs: Any)

Manage a jinja2-templated ServiceAccount.

See also

BodyTemplateMixin:

for arguments related to templating.

class yaook.statemachine.TemplatedRoleBindingState(*, template: str, versioned_dependencies: List[yaook.statemachine.versioneddependencies.VersionedDependency] = [], **kwargs: Any)

Manage a jinja2-templated RoleBinding.

See also

BodyTemplateMixin:

for arguments related to templating.

class yaook.statemachine.TemplatedServiceState(*, template: str, versioned_dependencies: List[yaook.statemachine.versioneddependencies.VersionedDependency] = [], **kwargs: Any)

Manage a jinja2-templated Service.

See also

BodyTemplateMixin:

for arguments related to templating.

class yaook.statemachine.TemplatedDeploymentState(*, template: str, versioned_dependencies: List[yaook.statemachine.versioneddependencies.VersionedDependency] = [], **kwargs: Any)

Manage a jinja2-templated Deployment.

See also

BodyTemplateMixin:

for arguments related to templating.

DeploymentState:

for arguments specific to Deployments, such as the scheduling_keys.

class yaook.statemachine.TemplatedJobState(*, template: str, versioned_dependencies: List[yaook.statemachine.versioneddependencies.VersionedDependency] = [], **kwargs: Any)

Manage a jinja2-templated Job.

See also

BodyTemplateMixin:

for arguments related to templating.

JobState:

for specific information about managing Jobs and specific arguments, such as the scheduling_keys.

class yaook.statemachine.TemplatedStatefulSetState(*, template: str, versioned_dependencies: List[yaook.statemachine.versioneddependencies.VersionedDependency] = [], **kwargs: Any)

Manage a jinja2-templated StatefulSet.

See also

BodyTemplateMixin:

for arguments related to templating.

StatefulSetState:

for specific information about managing StatefulSets.

class yaook.statemachine.TemplatedKeystoneEndpointState(*, template: str, versioned_dependencies: List[yaook.statemachine.versioneddependencies.VersionedDependency] = [], **kwargs: Any)

Manage a jinja2-templated KeystoneEndpoint.

See also

BodyTemplateMixin:

for arguments related to templating.

class yaook.statemachine.TemplatedConfiguredDaemonSetState(*, template: str, versioned_dependencies: List[yaook.statemachine.versioneddependencies.VersionedDependency] = [], **kwargs: Any)

Manage a jinja2-templated ConfiguredDaemonSet.

This class adds some filters to the Jinja2 environment for easier generation of CDS specs:

See also

BodyTemplateMixin:

for arguments related to templating.

ConfiguredDaemonSetState:

for arguments specific to ConfiguredDaemonSets, such as the scheduling_keys.

class yaook.statemachine.TemplatedPersistentVolumeClaimState(*, template: str, versioned_dependencies: List[yaook.statemachine.versioneddependencies.VersionedDependency] = [], **kwargs: Any)

Manage a jinja2-templated Deployment.

See also

BodyTemplateMixin:

for arguments related to templating.

PersistentVolumeClaimState:

for arguments specific to PVCs.

class yaook.statemachine.TemplatedCertificateState(*, template: str, versioned_dependencies: List[yaook.statemachine.versioneddependencies.VersionedDependency] = [], **kwargs: Any)

Manage a jinja2-templated CertificateState.

See also

BodyTemplateMixin:

for arguments related to templating.

CertificateState:

for the original state.

Other resources

class yaook.statemachine.InstancedKeystoneUserState(*, prefix: str, keystone: yaook.statemachine.resources.KubernetesReference[Mapping], **kwargs: Any)
class yaook.statemachine.CAConfigMapState(metadata: Union[str, Tuple[str, bool], Callable[[yaook.statemachine.context.Context], str], Callable[[yaook.statemachine.context.Context], Tuple[str, bool]], Callable[[yaook.statemachine.context.Context], MutableMapping[str, Any]]], usercerts_spec_key: Optional[str] = None, certificate_secrets_states: List[yaook.statemachine.resources.SecretState] = [], copy_on_write: bool = True, **kwargs: Any)

ConfigMap generated with CA Certificate for use within directories and as CA-bundles. Since applications normally need to restart when their list of CA Certificates change we default to enable copy_on_write here.

Parameters
  • resource_name_base – The resource name or the prefix for dynamic generation.

  • generate_name – If the resource name should be generated dynamically.

  • usercerts_spec_key – If specified tries to extract a list of x509 certificates in pem format from the CustomResource.

  • certificate_secrets_states – If specified extracts the CAs used to sign the mentioned secrets (by taking the ca.crt from the secret) and adds them to the list of trusted CAs.

async _get_user_certs(ctx: yaook.statemachine.context.Context, dependencies: Mapping[str, yaook.statemachine.resources.Resource]) Set[str]

Generate a list of PEM Certificate Strings for the CA bundle. If usercerts_spec_key is set it gets the list from the specified key in the parent spec.

Parameters
  • ctx (Context) – The context of the current operation.

  • dependencies – The dependency instances of this resource.

class yaook.statemachine.EmptyTlsSecretState(metadata: Union[str, Tuple[str, bool], Callable[[yaook.statemachine.context.Context], str], Callable[[yaook.statemachine.context.Context], Tuple[str, bool]], Callable[[yaook.statemachine.context.Context], MutableMapping[str, Any]]], **kwargs: Any)

Generates an empty tls secret which is never updated. This can be used as a backing secret for CertificateState

Resource base classes

Basic kubernetes resources

ConfigMapState(*[, copy_on_write])

SecretState(*[, copy_on_write])

RoleState(*[, copy_on_write])

ServiceAccountState(*[, copy_on_write])

RoleBindingState(*[, copy_on_write])

ServiceState(*[, copy_on_write])

DeploymentState(*, scheduling_keys, **kwargs)

param scheduling_keys

Scheduling keys to use.

JobState(*, scheduling_keys, **kwargs)

Manage a Job resource.

StatefulSetState(*[, scheduling_keys])

KeystoneEndpointState(*[, copy_on_write])

ConfiguredDaemonSetState(*, scheduling_keys, ...)

param scheduling_keys

Scheduling keys to use.

class yaook.statemachine.ConfigMapState(*, copy_on_write: bool = False, **kwargs: Any)
class yaook.statemachine.SecretState(*, copy_on_write: bool = False, **kwargs: Any)
class yaook.statemachine.RoleState(*, copy_on_write: bool = False, **kwargs: Any)
class yaook.statemachine.ServiceAccountState(*, copy_on_write: bool = False, **kwargs: Any)
class yaook.statemachine.RoleBindingState(*, copy_on_write: bool = False, **kwargs: Any)
class yaook.statemachine.ServiceState(*, copy_on_write: bool = False, **kwargs: Any)
class yaook.statemachine.DeploymentState(*, scheduling_keys: Collection[str], **kwargs: Any)
Parameters

scheduling_keys (collection of str) – Scheduling keys to use.

When configuring the Deployment resource, the Pods will gain a nodeAffinity for only those nodes matching any of the scheduling keys.

A node matches a scheduling key if it has a label with that key; the value of that label is ignored.

In addition, tolerations for all the scheduling keys (for any taint effect) are added to the pods.

class yaook.statemachine.JobState(*, scheduling_keys: Collection[str], **kwargs: Any)

Manage a Job resource.

Parameters

scheduling_keys (collection of str) – Scheduling keys to use.

In contrast to other resources, the Job resource will only report as ready (via is_ready()) after the Job has had at least one successful run.

When configuring the Job resource, the Pods will gain a nodeAffinity for only those nodes matching any of the scheduling keys.

A node matches a scheduling key if it has a label with that key; the value of that label is ignored.

In addition, tolerations for all the scheduling keys (for any taint effect) are added to the pods.

class yaook.statemachine.StatefulSetState(*, scheduling_keys: Optional[Collection[str]] = None, **kwargs: Any)
class yaook.statemachine.KeystoneEndpointState(*, copy_on_write: bool = False, **kwargs: Any)
class yaook.statemachine.ConfiguredDaemonSetState(*, scheduling_keys: Collection[str], **kwargs: Any)
Parameters

scheduling_keys (collection of str) – Scheduling keys to use.

When configuring the ConfiguredDaemonSet resource, the targetNodes will be fored to match the nodes matching any of the scheduling_keys given.

A node matches a scheduling key if it has a label with that key; the value of that label is ignored.

In addition, tolerations for all the scheduling keys (for any taint effect) are added to the pods.

class yaook.statemachine.PersistentVolumeClaimState(*, copy_on_write: bool = False, **kwargs: Any)

Mixins

class yaook.statemachine.resources.TemplateMixin(*, templatedir: Optional[str] = None, params: Optional[Mapping] = None, **kwargs: Any)

A mixin to provide Jinja2 templating facilities in a resource.

Parameters
  • basedir – The base directory where the jinja2 instance will look for templates.

  • params – A dictionary with additional parameters which may be passed to templates.

If basedir is not set, it can be implicitly set by setting the TEMPLATE_DIR attribute on the CustomResource subclass on which the resource is used, provided that the resource is used as a descriptor.

The following functions are made available in the templating context:

class yaook.statemachine.resources.BodyTemplateMixin(*, template: str, versioned_dependencies: List[yaook.statemachine.versioneddependencies.VersionedDependency] = [], **kwargs: Any)

Mixin to create the body of a resource from a Jinja2 template.

Parameters

template (str) – Path to the template to render for the body.

See also

TemplateMixin

for more arguments of the constructor and which extra filters are available in templates.

async _get_template_parameters(ctx: yaook.statemachine.context.Context, dependencies: Mapping[str, yaook.statemachine.resources.Resource]) MutableMapping[str, Any]

In addition to the parameters exposed by _get_template_parameters(), this method offers the following:

versioned_dependencies

A mapping which maps the dependency name to the version string provided by the VersionedDependency.

Fundamental base classes

class yaook.statemachine.Resource(component: Optional[str] = None, *, add_dependencies: Sequence[yaook.statemachine.resources.Resource] = [])

Abstract base class to represent a resource.

Parameters
  • component – Unique (within a custom resource) identifier to identify this resource.

  • add_dependencies – Additional resources this state depends on.

This class defines the common interface of all resource managers which can be used to implement a CustomResource.

User documentation

If component is not given, the Resource must be assigned as a class attribute to the CustomResource, in which case the attribute name will be used as component.

Warning

The component of a logical resource MUST NOT change. If the component is changed (e.g. because the attribute is renamed in the code) and the operator is restarted, data which belongs to the old name will not be found by the instance with the new name.

Thus, if you have to rename the attribute, you are strongly advised to make use of the component argument (which takes precedence over the attribute assignment).

See also

Instantiating a Resource directly is not possible. You have to use on of the subclasses. See Concrete resource classes for an overview.

Public interface

component

The unique identifier of the logical component implemented by this resource.

See also

Resource for a discussion of the significance of the component.

get_listeners() List[yaook.statemachine.context.Listener]

Return a list of listener callbacks for this state.

The machinery around the CustomResource calls this method in order to set up watches for Kubernetes objects. This allows the resources to trigger additional reconcile runs based on changes to resources which are different than the CustomResource to which they belong (changes to the non-status of the custom resource always trigger a reconcile).

async get_used_resources(ctx: yaook.statemachine.context.Context) Iterable[yaook.statemachine.api_utils.ResourceReference]

Obtain an iterable of resources used by this resource.

The resulting set may include resources not managed by Yaook or by this operator. It also reflects the current state of the cluster and may be volatile.

async is_ready(ctx: yaook.statemachine.context.Context) bool

Test whether the managed resource is ready to be used by its dependents.

Parameters

ctx – The context of the operation.

Returns

True if the resource is ready to be used, false otherwise.

Note that returning false here does not necessarily imply that a reconcile run is needed right away. It is also possible that a long-running external process needs to finish first.

In that case, the Resource is responsible for registering a listener (see get_listeners()) which is able to trigger a reconcile once the resource is ready to be used in order to bring the dependents in shape.

The default implementation returns True.

abstract async reconcile(ctx: yaook.statemachine.context.Context, *, dependencies: Mapping[str, yaook.statemachine.resources.Resource]) bool

Reconcile the state.

Parameters
  • ctx (Context) – The context of the current operation.

  • dependencies – The dependencies which are injected into this State.

dependencies must be a dictionary mapping the state class to an instance of that class. It can be used to access information about dependent resources.

Return whether the resource was changed by the reconcile.

Documentation for subclasses

The following methods and attributes are meant for implementing subclasses only.

_set_component(component: str) None

Initialise the component.

This is called exactly once by the machinery in Resource to allow subclasses to initialise attributes based on the component value.

_set_owner(owner: Type) None

Set the owner of the Resource.

This is called up to once by the machinery in Resource when the resource instance is assigned to a CustomResource class attribute.

(An attempt to assign the same instance to different classes will be greeted with ValueError.)

_declare_dependencies(*dependencies: yaook.statemachine.resources.Resource) None

Declare other resources as dependencies of this resource.

Parameters

dependencies – The resources we dependend on.

This must only be called from constructors of subclasses, to register some of their arguments as resources they depend on.

class yaook.statemachine.KubernetesReference(component: Optional[str] = None, *, add_dependencies: Sequence[yaook.statemachine.resources.Resource] = [])

Reference to a kubernetes resource.

This type can be used in interfaces to require a Resource subclass which exposes access to a kubernetes resource of a specific type.

Public interface:

abstract async get(ctx: yaook.statemachine.context.Context) kubernetes_asyncio.client.models.v1_object_reference.V1ObjectReference

Retrieve the reference to the object of which the State is managed.

Parameters

ctx (Context) – The context of the current operation.

Raises
Return type

kubernetes_asyncio.client.V1ObjectReference

Returns

A reference containing the namespace and name of the object.

Both error conditions inherit from KeyError. In general, both error conditions should eventually be rectified by repeated calls to reconcile().

That means it should be safe to propagate them to the outer reconcile loop.

Note

This function may be expensive, as it may require round-trips to the kubernetes API.

abstract async get_all(ctx: yaook.statemachine.context.Context) Mapping[Optional[str], kubernetes_asyncio.client.models.v1_object_reference.V1ObjectReference]

Retrieve a mapping containing all instances of which the state is managed relating to the current context.

Parameters

ctx (Context) – The context of the current operation.

The mapping is keyed based on the instance string of the individual objects. If no objects exist, an empty mapping is returned. If this state only manages a single resource, the key is None instead of a string.

get_resource_interface(ctx: yaook.statemachine.context.Context) yaook.statemachine.interfaces.ResourceInterface[yaook.statemachine.resources.T]

Return a resource interface to access resources of the type managed by this object.

Parameters

ctx – The context to use to access the resources.

Returns

A resource interface to access the resources.

Subclass interface:

abstract _create_resource_interface(api_client: kubernetes_asyncio.client.api_client.ApiClient) yaook.statemachine.interfaces.ResourceInterface[yaook.statemachine.resources.T]

Construct a fresh resource interface to access the resources.

Parameters

api_client – The Kubernetes API client to use to access the resources.

Returns

A fresh resource interface to access the resources.`

The result of this method may be cached, keyed on api_client.

class yaook.statemachine.KubernetesResource(component: Optional[str] = None, *, add_dependencies: Sequence[yaook.statemachine.resources.Resource] = [])

Manage the state of a Kubernetes resource.

This is a specialisation of Resource, but still abstract. It manages one or more Kubernetes resources of the same type.

This is also a KubernetesReference and can thus be passed to functions which expect a reference to the specific resource.

See also

For specific implementations, see Concrete resource classes.

The Kubernetes resources associated with state managers have specifc labels in order to make them easily selectable through the Kubernetes API. The following labels are defined. Please see Labels and Annotations.

Public interface:

component

The unique identifier of the logical component implemented by this resource.

See also

Resource for a discussion of the significance of the component.

async cleanup_orphans(ctx: yaook.statemachine.context.Context, protect: Mapping[Tuple[str, str], Collection[Tuple[str, str]]]) None

Delete all unprotected resource orphans from the Kubernetes API server.

Parameters
  • ctx – The context to use.

  • protect – A collection of resources to keep.

protect must be a mapping which maps (apiVersion, plural) to a collection (preferably a set) of namespace-name-pairs of resources to keep. No resources from those sets will be deleted.

abstract async get(ctx: yaook.statemachine.context.Context) kubernetes_asyncio.client.models.v1_object_reference.V1ObjectReference

Retrieve the reference to the object of which the State is managed.

Parameters

ctx (Context) – The context of the current operation.

Raises
Return type

kubernetes_asyncio.client.V1ObjectReference

Returns

A reference containing the namespace and name of the object.

Both error conditions inherit from KeyError. In general, both error conditions should eventually be rectified by repeated calls to reconcile().

That means it should be safe to propagate them to the outer reconcile loop.

Note

This function may be expensive, as it may require round-trips to the kubernetes API.

abstract async get_all(ctx: yaook.statemachine.context.Context) Mapping[Optional[str], kubernetes_asyncio.client.models.v1_object_reference.V1ObjectReference]

Retrieve a mapping containing all instances of which the state is managed relating to the current context.

Parameters

ctx (Context) – The context of the current operation.

The mapping is keyed based on the instance string of the individual objects. If no objects exist, an empty mapping is returned. If this state only manages a single resource, the key is None instead of a string.

get_listeners() List[yaook.statemachine.context.Listener]

Return a list of listener callbacks for this state.

For the state to work correctly, the listeners must be invoked when the respective API objects change.

get_resource_interface(ctx: yaook.statemachine.context.Context) yaook.statemachine.interfaces.ResourceInterface[yaook.statemachine.resources.T]

Return a resource interface to access resources of the type managed by this object.

Parameters

ctx – The context to use to access the resources.

Returns

A resource interface to access the resources.

async is_ready(ctx: yaook.statemachine.context.Context) bool

Whether the managed object is ready to use for dependents.

abstract async reconcile(ctx: yaook.statemachine.context.Context, *, dependencies: Mapping[str, yaook.statemachine.resources.Resource]) bool

Reconcile the state.

Parameters
  • ctx (Context) – The context of the current operation.

  • dependencies – The dependencies which are injected into this State.

dependencies must be a dictionary mapping the state class to an instance of that class. It can be used to access information about dependent resources.

Return the completion state and the set of k8s resources belonging to the state.

Subclass interface:

abstract _create_resource_interface(api_client: kubernetes_asyncio.client.api_client.ApiClient) yaook.statemachine.interfaces.ResourceInterface[yaook.statemachine.resources.T]

Construct a fresh resource interface to access the resources.

Parameters

api_client – The Kubernetes API client to use to access the resources.

Returns

A fresh resource interface to access the resources.`

The result of this method may be cached, keyed on api_client.

Utilities for subclasses:

labels(ctx: yaook.statemachine.context.Context) Mapping[str, str]

Obtain the label match to retrieve all managed objects.

Parameters

ctx (Context) – The context of the operation.

Return type

dict

Returns

A (fresh) dictionary mapping the label keys to values.

async adopt_object(ctx: yaook.statemachine.context.Context, obj: MutableMapping[str, Any]) None

Update the obj in-place to adopt it into the given ctx.

Parameters
  • ctx (Context) – The context of the operation.

  • obj – The body of the resource to adopt.

This auto-creates an owner reference for the parent custom resource referenced by the ctx and it applies all labels from labels().

class yaook.statemachine.SingleObjectState(*, copy_on_write: bool = False, **kwargs: Any)

Manage the state of a single Kubernetes resource.

Parameters

copy_on_write – Initialises copy_on_write

See also

KubernetesResource, Resource

for more supported arguments.

Public interface:

copy_on_write

Enable the use of a copy-on-write strategy when updating the resource.

If copy_on_write is True, the resource will be orphaned (and later deleted by cleanup_orphans()) and re-created instead of updated in-place. It is effectively treated as completely immutable.

This strategy has advantages for situations where the old resource may still be in use and switching over to the new resource is not an atomic process. One prominent example of this is service configuration stored in Secrets.

component

The unique identifier of the logical component implemented by this resource.

See also

Resource for a discussion of the significance of the component.

async cleanup_orphans(ctx: yaook.statemachine.context.Context, protect: Mapping[Tuple[str, str], Collection[Tuple[str, str]]]) None

Delete all unprotected resource orphans from the Kubernetes API server.

Parameters
  • ctx – The context to use.

  • protect – A collection of resources to keep.

protect must be a mapping which maps (apiVersion, plural) to a collection (preferably a set) of namespace-name-pairs of resources to keep. No resources from those sets will be deleted.

async get(ctx: yaook.statemachine.context.Context) kubernetes_asyncio.client.models.v1_object_reference.V1ObjectReference

Retrieve the reference to the object of which the State is managed.

Parameters

ctx (Context) – The context of the current operation.

Raises
Return type

kubernetes_asyncio.client.V1ObjectReference

Returns

A reference containing the namespace and name of the object.

Both error conditions inherit from KeyError. In general, both error conditions should eventually be rectified by repeated calls to reconcile().

That means it should be safe to propagate them to the outer reconcile loop.

Note

This function may be expensive, as it may require round-trips to the kubernetes API.

async get_all(ctx: yaook.statemachine.context.Context) Mapping[Optional[str], kubernetes_asyncio.client.models.v1_object_reference.V1ObjectReference]

Retrieve a mapping containing all instances of which the state is managed relating to the current context.

Parameters

ctx (Context) – The context of the current operation.

The mapping is keyed based on the instance string of the individual objects. If no objects exist, an empty mapping is returned. If this state only manages a single resource, the key is None instead of a string.

get_listeners() List[yaook.statemachine.context.Listener]

Return a list of listener callbacks for this state.

For the state to work correctly, the listeners must be invoked when the respective API objects change.

get_resource_interface(ctx: yaook.statemachine.context.Context) yaook.statemachine.interfaces.ResourceInterface[yaook.statemachine.resources.T]

Return a resource interface to access resources of the type managed by this object.

Parameters

ctx – The context to use to access the resources.

Returns

A resource interface to access the resources.

async is_ready(ctx: yaook.statemachine.context.Context) bool

Whether the managed object is ready to use for dependents.

async reconcile(ctx: yaook.statemachine.context.Context, *, dependencies: Mapping[str, yaook.statemachine.resources.Resource]) bool

Reconcile the state of the resource.

Parameters
  • ctx – The context of the operation.

  • dependencies – The dependencies for the operation.

Returns

True if the resource was updated or (re-)created, false otherwise.

See also

reconcile()

for details on dependencies.

This compares a freshly generated intended body with the resource as is currently present in the Kubernetes API.

If the body differs (_needs_update()), the resource is updated according to the update strategy (see copy_on_write). If the resource does not exist at all, it will be created.

If a change was made to the resource (update or (re-)creation), this method returns true, false otherwise.

Methods to be implemented by subclasses:

abstract async _make_body(ctx: yaook.statemachine.context.Context, dependencies: Mapping[str, yaook.statemachine.resources.Resource]) MutableMapping[str, Any]

Construct the complete intended body of the resource.

Parameters
  • ctx – The context of the operation.

  • dependencies – The dependencies supplied to this resource.

See also

reconcile()

for details on dependencies.

Return the complete body as nested dictionaries, including the apiVersion, kind and metadata fields.

abstract _needs_update(old: yaook.statemachine.resources.T, new: Mapping) bool

Test whether the resource must be updated.

Parameters
  • old – The old resource as returned by the Kubernetes API.

  • new – The new resource as generated by _make_body().

Returns

True if the resource needs to be updated in Kubernetes, false otherwise.

Interface for subclasses:

async _get_current(ctx: yaook.statemachine.context.Context) yaook.statemachine.resources.T

Return the current instance of the resource, if it exists.

Parameters

ctx – The context of the operation.

Raises
Returns

The resource as returned by the Kubernetes API.

This lists all matching (see KubernetesResource.labels()) resources and filters out any orphaned or deleted resources.

If exactly one resource remains, it is returned. Otherwise, ResourceNotPresent or AmbiguousRequest is raised as is appropriate./

Internal API:

async _orphan(ctx: yaook.statemachine.context.Context, namespace: str, name: str) None

Orphan the current resource.

Parameters
  • ctx – The context of the operation.

  • namespace – The namespace of the resource to orphan.

  • name – The name of the resource to orphan.

This will add the LABEL_ORPHANED to the resource, if it is not present yet, to inform listers about the resource being deprecated.

Miscellaneous

class yaook.statemachine.resources.DependencyTemplateWrapper(ctx: yaook.statemachine.context.Context, obj: yaook.statemachine.resources.KubernetesReference)

A wrapper object used for templating.

resource_name() str

Return the Kubernetes name of the resource.

instances() Mapping[Optional[str], kubernetes_asyncio.client.models.v1_object_reference.V1ObjectReference]

Return a dictionary mapping all instance names to their Kubernetes object references (i.e. to objects with namespace and name attributes).

instance_name_map() Mapping[Optional[str], str]

Return a dictionary mapping all instance names to their Kubernetes object names.

last_update_timestamp() str

Return the last update timestamp as contained in the object’s annotations.

async yaook.statemachine.resources.write_ca_certificates(ca_config: kubernetes_asyncio.client.models.v1_object_reference.V1ObjectReference, ctx: yaook.statemachine.context.Context) str

Write the CA Certificates from the passed ca_config to a local file and return the filename.

This method is using the Context to choose an appropriate file name. Calls to this method with the same Context might overwrite the old file.

Parameters
  • ca_config – A reference to the Configmap holding the ca certificates.

  • ctx – the Context.

Returns

The path to the ca-bundle file.

Exception types

class yaook.statemachine.ResourceNotPresent(component: str, context: yaook.statemachine.context.Context, *, resource_version: Optional[str] = None)
class yaook.statemachine.AmbiguousRequest(component: str, context: yaook.statemachine.context.Context, *, resource_version: Optional[str] = None)
class yaook.statemachine.ContinueableError(msg: str, ready: bool = False)

An exception that indicates that an individual operation failed, but the process should nevertheless continue.

Parameters
  • msg – The message of the error.

  • ready – If the operator should regard the throwing state as ready for evaluating its dependencies (default is False)

This error should be raised by a State if the processing of this state failed, but the operator should not evaluate this as a failed reconcileing task. This Error is therefor used to differenciate between a bug in the operator itself (e.g. a KeyError) and an expected error state (e.g. failing to generate configuration, because the input is invalid). In the former case we can not do anything but abort the processing (as it is done with normal errors). In the latter case we can continue with all other states of the CR as this issue should not affect them.

Note that the operator might evaluate dependencies of this state after this exception has been raised if ready is True.

Helpers for Handling Secrets

async yaook.statemachine.extract_password(ctx: yaook.statemachine.context.Context, state: Union[yaook.statemachine.resources.KubernetesReference[kubernetes_asyncio.client.models.v1_secret.V1Secret], str], *, key: str = 'password') str
async yaook.statemachine.get_injected_secrets(ctx: yaook.statemachine.context.Context, injection_spec: Iterable[Mapping[str, Any]]) List[Any]

Construct a configuration parts from k8s secrets according to the specification given in the attribute injection_spec.

Parameters
  • ctx (Context) – The context of the current operation.

  • injection_spec – The injection specification.

Returns

The constructed configuration parts

This is intended to be called from implementations of CueSecretState._get_config_inputs() and the return value should be concatenated with the other configuration inputs.

An example injection specification (serialized as YAML):

- secretName: example-secret
  items:
  - key: foo
    path: /DEFAULT/foo
  - key: bar
    path: /DEFAULT/bar

With this spec, this method will take the contents of the keys foo and bar in the secret example-secret, say $foo and $bar, and return:

[{"DEFAULT": {"foo": "$foo"}}. {"DEFAULT": {"bar": "$bar"}}]

The path is a JSON Pointer following RFC 6901. All referenced non-terminal keys are assumed to be objects and created on demand.

Merging the configuration parts is the job of the caller, for cuelang you can just serialize the parts independently and let cuelang do the merging and verfication of consistency.

cue – Objects containing config generated with cuelang

Concept

For general information on cuelang, please visit the cuelang website.

In the context of Yaook, cuelang is used to generate and prevalidate configuration for OpenStack and other services. The key advantage with using cuelang is (simplified) that it enforces that if you see an input value for an option somewhere, it will also be the output value; if conflicting values for the same option come from different sources, cuelang rejects the input.

This means that even though Yaook supports multiple inputs for most configuration files (for example, snippets which are injected based on the labels of the target nodes), there will never be any surprises, because those inputs cannot override each other.

The classes in this document support working with cuelang by allowing the user to make most of those cuelang properties.

Layers

The cuelang generation uses the concept of layers or overlays. A single cuelang generated configuration is composed from multiple layers. Each layer is represented by a CueLayer object, while the composed configuration is managed by a class using the LayeredCueMixin.

Generally, each layer contains a single aspect of the configuration, such as a single database or rabbitmq connection. As described, each layer needs to merge with the other layers without errors for rendering with cuelang to succeed.

Quick reference

Configuration classes

To provide configuration to services in Kubernetes, secrets or config maps can be generated using cuelang:

CueConfigMapState

Manage a ConfigMap whose contents are generated using cuelang.

CueSecretState

Manage a Secret whose contents are generated using cuelang.

Provided layers

AMQPTransportLayer

Cue layer for a OpenStack AMQP transport URI.

DatabaseConnectionLayer

Cue layer for a OpenStack database connection URI.

KeystoneAuthLayer

Cue layer for OpenStack keystone credentials.

NodeLabelSelectedLayer

Cue layer to select configuration snippets based on the labels of the context’s node.

SecretInjectionLayer

Cue layer to inject Kubernetes Secrets referenced from the parent resource spec into the configuration.

SpecLayer

Cue layer to copy parts of the parent resource spec into the configuration.

ConfigSecretLayer

Cue layer to set a configuration option to a string.

Custom layers

To implement a layer, derive from the following class and implement the methods described in the documentation:

CueLayer

Base class to implement custom configuration overlays.

Full reference

Configuration class details

class yaook.statemachine.CueConfigMapState(*[, metadata][, add_cue_layers=[]][, copy_on_write=False])

Manage a ConfigMap whose contents are generated using cuelang.

Parameters
  • metadata – Description of the metadata to generate for the resource.

  • add_cue_layers – Collection of layers to add during initialisation.

  • copy_on_write – If true, create a new resource on changes instead of updating in-place.

All layers which are passed via add_cue_layers will be added to the configuration as if by calling add_layer() for each item.

See also

evaluate_metadata()

for details on the accepted values for metadata.

SingleObjectState

for details on copy_on_write

CueSecretState

for a usage example

add_layer(layer: yaook.statemachine.cue.CueLayer) None

Add an overlay to the configuration.

Parameters

layer – The layer to add.

The dependencies of the layer will be reflected in this resource’s dependencies.

class yaook.statemachine.CueSecretState(*[, metadata][, add_cue_layers=[]][, copy_on_write=False])

Manage a Secret whose contents are generated using cuelang.

Parameters
  • metadata – Description of the metadata to generate for the resource.

  • add_cue_layers – Collection of layers to add during initialisation.

  • copy_on_write – If true, create a new resource on changes instead of updating in-place.

All layers which are passed via add_cue_layers will be added to the configuration as if by calling add_layer() for each item.

The configuration generated by cuelang is encoded as UTF-8 and wrapped in base64 in order to transport it into a Kubernetes Secret.

Example use:

class FancyResource(sm.CustomResource):
    config = sm.CueSecretState(
        metadata=("my-config-", True),
        copy_on_write=True,
        add_cue_layers=[
            sm.DatabaseConnectionLayer(...),
        ]
    )

See also

evaluate_metadata()

for details on the accepted values for metadata.

SingleObjectState

for details on copy_on_write

add_layer(layer: yaook.statemachine.cue.CueLayer) None

Add an overlay to the configuration.

Parameters

layer – The layer to add.

The dependencies of the layer will be reflected in this resource’s dependencies.

Layer details

class yaook.statemachine.AMQPTransportLayer(*, target: str, service: yaook.statemachine.resources.KubernetesReference[kubernetes_asyncio.client.models.v1_service.V1Service], username: Union[str, Callable[[yaook.statemachine.context.Context], str]], password_secret: yaook.statemachine.resources.KubernetesReference[kubernetes_asyncio.client.models.v1_secret.V1Secret], **kwargs: Any)

Cue layer for a OpenStack AMQP transport URI.

Parameters
  • target – The cuelang package to target.

  • service – Reference to the Kubernetes Service under which the AMQP server is reachable.

  • username – Username to use for authenticating with the AMQP service.

  • password_secret – Reference to the Kubernetes Secret which holds the password to use for authenticating with the AMQP service.

This is a CueLayer for use with Configuration classes.

class yaook.statemachine.DatabaseConnectionLayer(*, target: str, service: yaook.statemachine.resources.KubernetesReference[kubernetes_asyncio.client.models.v1_service.V1Service], database_name: str, username: Union[str, Callable[[yaook.statemachine.context.Context], str]], password_secret: yaook.statemachine.resources.KubernetesReference[kubernetes_asyncio.client.models.v1_secret.V1Secret], config_section: str, **kwargs: Any)

Cue layer for a OpenStack database connection URI.

Parameters
  • target – The cuelang package to target.

  • service – Reference to the Kubernetes Service under which the database is reachable.

  • database_name – Name of the database.

  • username – Username to use for authenticating with the database service.

  • password_secret – Reference to the Kubernetes Secret which holds the password to use for authenticating with the database service.

  • config_section – The configuration section to inject the database credentials into.

This is a CueLayer for use with Configuration classes.

class yaook.statemachine.KeystoneAuthLayer(*, target: str, credentials_secret: yaook.statemachine.resources.KubernetesReference[kubernetes_asyncio.client.models.v1_secret.V1Secret], endpoint_config: yaook.statemachine.resources.KubernetesReference[kubernetes_asyncio.client.models.v1_config_map.V1ConfigMap], config_section: str = 'keystone_authtoken', **kwargs: Any)

Cue layer for OpenStack keystone credentials.

Parameters
  • target – The cuelang package to target.

  • credentials – Reference to the Kubernetes Secret which holds the OpenStack environment variables to access keystone.

  • config_section – The configuration section to inject the database credentials into.

This is a CueLayer for use with Configuration classes.

class yaook.statemachine.NodeLabelSelectedLayer(*, target_map: Mapping[str, str], accessor: Callable[[yaook.statemachine.context.Context], Collection[MutableMapping[str, Any]]], **kwargs: Any)

Cue layer to select configuration snippets based on the labels of the context’s node.

Parameters
  • target_map – A mapping which maps configuration keys from the templates to cuelang packages.

  • accessor – A function which extracts the configuration template collection from the parent resource’ spec.

When generating the layer, the labels of the node whose name matches the instance of the execution context are used to select the configuration snippets.

The snippets returned by accessor are expected to be dictionaries of the following format (in YAML):

nodeSelectors:
- matchLabels:
    ...
- ...
key1: ...
key2: ...

where nodeSelector is an array of label match definitions similar to those typically found in the k8s API. See from_dict() for the supported format. The remaining keys of the dictionary are taken as configuration snippets. The accessor receives the complete Context of the operation as only argument.

The configuration snippets will be associated to a cuelang package via the target_map. The keys of the target_map correspond to the keys from the above dictionary while the values are the corresponding cuelang packages.

This is a CueLayer for use with Configuration classes.

class yaook.statemachine.SecretInjectionLayer(*, target: str, accessor: Union[str, Callable[[Mapping[str, Any]], Any]], **kwargs: Any)

Cue layer to inject Kubernetes Secrets referenced from the parent resource spec into the configuration.

Parameters
  • target – The cuelang package to target.

  • accessor – A string key or a function to extract the secret injection spec from the parent resource.

This creates a configuration layer out secrets extracted from Kubernetes according to the secret injection specification referenced in the parent resource spec by accessor.

If accessor is a string, the injection spec is taken from parent["spec"][accessor]. Otherwise, the it is taken from accessor(parent["spec"]).

This is a CueLayer for use with Configuration classes.

class yaook.statemachine.SpecLayer(*, target: str, accessor: Union[str, Callable[[Mapping[str, Any]], Any]], flavor: yaook.common.config.ConfigFlavor = ConfigFlavor(name='oslo_config', from_json=<function <lambda>>, serialize=<function to_openstack_conf>, filename_format='{modulename}.conf'), **kwargs: Any)

Cue layer to copy parts of the parent resource spec into the configuration.

Parameters
  • target – The cuelang package to target.

  • accessor – A string key or a function to access the configuration to inject.

This creates a configuration layer out of an attribute of the resource spec of the parent resource. If accessor is a string, the configuration input is taken from parent["spec"][accessor]. Otherwise, the input is taken from accessor(parent["spec"]).

This is a CueLayer for use with Configuration classes.

class yaook.statemachine.ConfigSecretLayer(*, target: str, config_section: str, config_option_name: str, secret: yaook.statemachine.resources.SecretState, secret_key: str = 'password', flavor: yaook.common.config.ConfigFlavor = ConfigFlavor(name='oslo_config', from_json=<function <lambda>>, serialize=<function to_openstack_conf>, filename_format='{modulename}.conf'), **kwargs: Any)

Cue layer to set a configuration option to a string.

The value of the configuration option config_option_name in the configuration section config_group will set to config_option_value.

Parameters
  • target – The cuelang package to target.

  • config_section – The configuration section into which the secret should be injected

  • config_option_name – The name of the configuration option under which the secret should be injected

  • secret – The secret to inject

  • secret_key – the key in the secret’s data, whose value should be injected

  • config_flavor – configuration flavor. The default value is yaook.common.config.OSLO_CONFIG

This is a CueLayer for use with Configuration classes.

class yaook.statemachine.CueLayer(*, add_dependencies: Collection[yaook.statemachine.resources.Resource] = [])

Base class to implement custom configuration overlays.

The following interface is public:

get_dependencies() Collection[yaook.statemachine.resources.Resource]

Return the dependencies of this layer.

The following method must be implemented by subclasses and is also public:

abstract async get_layer(ctx: yaook.statemachine.context.Context) MutableMapping[str, yaook.common.config.ConfigDeclaration]

Compose the configuration overlay.

Parameters

ctx – The context of the operation.

Returns

The configuration overlay.

The format of the return value is a dictionary. The dictionary maps “targets” (cuelang package names) to lists of configuration inputs.

Each configuration input is typically a dictionary which maps string keys or configuration sections to configuration data.

The following interface is offered only to subclasses:

_declare_dependencies(*deps: yaook.statemachine.resources.Resource) None

Declare resources as dependencies of this layer.

Parameters

deps – Resources to declare as dependency.

The declared resources will be available through get_dependencies() to users of this class. Specifically, the default Configuration classes export the dependency information to the state machine for dependency resolution.

Instances of (subclasses of) this class can be used with Configuration classes to extend the generated output safely.

Implementation details

class yaook.statemachine.cue.CueMixin(**kwargs)

Mixin to generate cuelang-based configuration.

Subclasses must implement the following method:

abstract async _get_cue_inputs(ctx: yaook.statemachine.context.Context) MutableMapping[str, yaook.common.config.ConfigDeclaration]

Generate the cuelang input for the cuelang config generator.

The following method is provided to subclasess for consumption and is not part of the public interface:

async _render_cue_config(ctx: yaook.statemachine.context.Context) MutableMapping[str, Any]

Return the rendered cue configuration.

Raises

ContinueableError – If the configuration is invalid.

Internally, this uses _get_cue_inputs() to obtain the inputs to cuelang.

class yaook.statemachine.cue.LayeredCueMixin(*, add_cue_layers: Collection[yaook.statemachine.cue.CueLayer] = [], **kwargs: Any)

Mixin providing the layering functionality on top of CueMixin.

Parameters

add_cue_layers – Collection of layers to add during initialisation.

All layers which are passed via add_cue_layers will be added to the configuration as if by calling add_layer() for each item.

This mixin is used to form the Configuration classes.

add_layer(layer: yaook.statemachine.cue.CueLayer) None

Add an overlay to the configuration.

Parameters

layer – The layer to add.

The dependencies of the layer will be reflected in this resource’s dependencies.

extresources – Objects representing non-Kubernetes resources

ExternalResource(*, finalizer, **kwargs)

Represent a non-Kubernetes object’s state.

class yaook.statemachine.extresources.ExternalResource(*, finalizer: str, **kwargs: Any)

Represent a non-Kubernetes object’s state.

Parameters

finalizer – A Kubernetes finalizer string which is used to allow coordinated deletion of the external resource.

This state makes use of Kubernetes’ finalizer feature in order to prevent resource deletion without the cooperation of this state. This allows to require that the external resource has been cleaned up properly before the Kubernetes resource to which this state belongs gets deleted.

The finalizer is thus attached to the Kubernetes resource to which this state belongs. Based on the lifecycle events of that resource, the methods of this state are called.

Public interface:

component

The unique identifier of the logical component implemented by this resource.

See also

Resource for a discussion of the significance of the component.

get_listeners() List[yaook.statemachine.context.Listener]

Return a list of listener callbacks for this state.

The machinery around the CustomResource calls this method in order to set up watches for Kubernetes objects. This allows the resources to trigger additional reconcile runs based on changes to resources which are different than the CustomResource to which they belong (changes to the non-status of the custom resource always trigger a reconcile).

async is_ready(ctx: yaook.statemachine.context.Context) bool

Test whether the managed resource is ready to be used by its dependents.

Parameters

ctx – The context of the operation.

Returns

True if the resource is ready to be used, false otherwise.

Note that returning false here does not necessarily imply that a reconcile run is needed right away. It is also possible that a long-running external process needs to finish first.

In that case, the Resource is responsible for registering a listener (see get_listeners()) which is able to trigger a reconcile once the resource is ready to be used in order to bring the dependents in shape.

The default implementation returns True.

async reconcile(ctx: yaook.statemachine.context.Context, dependencies: Mapping[str, Resource]) bool

Reconcile the external resource state.

Parameters
  • ctx – The context of the operation.

  • dependencies – The dependencies of the operation.

See also

reconcile()

for details on dependencies.

This method uses finalizers to implement correct sequencing of calls to update() and delete() to maintain the external object instance.

Documentation for subclasses:

In order to implement a ExternalResource, the following methods must be implemented:

abstract async update(ctx: yaook.statemachine.context.Context, dependencies: Mapping[str, Resource]) bool

Update the object to conform to the declared state.

See also

reconcile()

for details on dependencies.

update() must be idempotent and it must be able to cope with having been interrupted at any point, including before its execution. That means that it has to be able to deal with all kinds of partially or not at all created states.

abstract async delete(ctx: yaook.statemachine.context.Context, dependencies: Mapping[str, Resource]) None

Delete the object to which the declared state refers.

See also

reconcile()

for details on dependencies.

delete() must be idempotent. It cannot assume that update() has completed successfully before it is called on an object. It must be able to deal with partially created instances, even if update() provides strong exception safety.

helm – Objects helm releases

HelmRelease(*, finalizer, **kwargs)

Manage a helm release.

class yaook.statemachine.helm.HelmRelease(*, finalizer: str, **kwargs: Any)

Manage a helm release.

This is an abstract external resource base class to manage a helm release inside the kubernetes cluster. For methods which need to be implemented by subclasses, see below.

While Helm also manages Kubernetes resources, we do not have direct access to those. Hence, this is managed as an external resource with a finalizer.

Warning

Use of helm releases in an operator is discouraged. It is preferable to port helm charts to operators.

Note

On every iteration of the operator, helm upgrade will be called. This may or may not be what you want, and it may even be dangerous depending on the helm chart used.

Note

This depends on having the helm binary in the search path.

Interface for subclasses:

abstract async _get_helm_release(ctx: yaook.statemachine.context.Context, dependencies: Mapping[str, Resource]) yaook.statemachine.helm.HelmReleaseInfo

Create the necessary information to manage the helm release.

Return type

HelmReleaseInfo

Returns

The information fully describing the desired helm release.

Warning

The name and namespace of the returned object must always be the same for an instance of a helm release. Otherwise, the previous release will be orphaned.`

infra – Objects representing infra.yaook.cloud resources

MySQLServiceState(*[, copy_on_write])

MySQLUserState(*[, copy_on_write])

AMQPServerState(*[, copy_on_write])

AMQPUserState(*[, copy_on_write])

TemplatedMySQLServiceState(*, template[, ...])

Manage a jinja2-templated MySQLService.

TemplatedMySQLUserState(*, template[, ...])

Manage a jinja2-templated MySQLUser.

TemplatedAMQPServerState(*, template[, ...])

Manage a jinja2-templated AMQPServer.

TemplatedAMQPUserState(*, template[, ...])

Manage a jinja2-templated AMQPUser.

class yaook.statemachine.infra.MySQLServiceState(*, copy_on_write: bool = False, **kwargs: Any)
class yaook.statemachine.infra.MySQLUserState(*, copy_on_write: bool = False, **kwargs: Any)
class yaook.statemachine.infra.AMQPServerState(*, copy_on_write: bool = False, **kwargs: Any)
class yaook.statemachine.infra.AMQPUserState(*, copy_on_write: bool = False, **kwargs: Any)
class yaook.statemachine.infra.TemplatedMySQLServiceState(*, template: str, versioned_dependencies: List[yaook.statemachine.versioneddependencies.VersionedDependency] = [], **kwargs: Any)

Manage a jinja2-templated MySQLService.

See also

BodyTemplateMixin:

for arguments related to templating.

MySQLServiceState:

for arguments specific to MySQLService.

class yaook.statemachine.infra.TemplatedMySQLUserState(*, template: str, versioned_dependencies: List[yaook.statemachine.versioneddependencies.VersionedDependency] = [], **kwargs: Any)

Manage a jinja2-templated MySQLUser.

See also

BodyTemplateMixin:

for arguments related to templating.

MySQLUserState:

for arguments specific to MySQLUser.

class yaook.statemachine.infra.TemplatedAMQPServerState(*, template: str, versioned_dependencies: List[yaook.statemachine.versioneddependencies.VersionedDependency] = [], **kwargs: Any)

Manage a jinja2-templated AMQPServer.

See also

BodyTemplateMixin:

for arguments related to templating.

AMQPServerState:

for arguments specific to AMQPServers.

class yaook.statemachine.infra.TemplatedAMQPUserState(*, template: str, versioned_dependencies: List[yaook.statemachine.versioneddependencies.VersionedDependency] = [], **kwargs: Any)

Manage a jinja2-templated AMQPUser.

See also

BodyTemplateMixin:

for arguments related to templating.

AMQPUser:

for arguments specific to AMQPUsers.

instancing – Manage groups of resources

This module contains a bunch of classes to manage sets of resources in different ways. The key difference is whether the resources are considered stateful (like as in StatefulSet: resources need active lifecycle management to not break) or stateless (think ConfigMaps, Secrets, …).

The following clasess implement node selector based instancing of resources using those two paradigms:

PerNode(*, scheduling_keys, wrapped_state, ...)

StatefulPerNode(*, scheduling_keys, **kwargs)

The main logic is implemented in:

StatelessInstancedResource(*, wrapped_state, ...)

Manage multiple instances of a resource which does not require state management.

StatefulInstancedResource(*, wrapped_state, ...)

Carefully Manage multiple instances of a stateful resource.

To achieve per-node semantics, the following mixin is used:

PerNodeMixin(*, scheduling_keys, **kwargs)

The lowest-level building block is:

InstancedResource(*, wrapped_state, **kwargs)

High level classes

class yaook.statemachine.PerNode(*, scheduling_keys: Collection[str], wrapped_state: yaook.statemachine.resources.SingleObjectState[yaook.statemachine.instancing.T], **kwargs: Any)
class yaook.statemachine.StatefulPerNode(*, scheduling_keys: Collection[str], **kwargs: Any)

Building blocks

Business logic base classes

class yaook.statemachine.instancing.StatelessInstancedResource(*, wrapped_state: yaook.statemachine.resources.SingleObjectState[yaook.statemachine.instancing.T], **kwargs: Any)

Manage multiple instances of a resource which does not require state management.

Parameters

wrapped_state – The resource to manage

class yaook.statemachine.instancing.StatefulInstancedResource(*, wrapped_state: yaook.statemachine.resources.SingleObjectState[yaook.statemachine.instancing.T], **kwargs: Any)

Carefully Manage multiple instances of a stateful resource.

Parameters

wrapped_state – The resource to manage

See reconcile() for the logic applied for the resource management.

Interface for subclasses:

_get_max_unavailable(ctx: yaook.statemachine.context.Context) int

Return the maximum number of resources the operator may willlingly bring into a non-Ready state.

abstract _get_run_state(ctx: yaook.statemachine.context.Context, instance: yaook.statemachine.instancing.T) yaook.statemachine.instancing.ResourceRunState

Return:

  • STARTING: If the resource is currently in the process of becoming ready. May also be used if the resource is currently updating.

  • READY: If the resource is ready to be used.

  • BROKEN: If the resource has crashed in a way which may not recover.

  • SHUTTING_DOWN: If the resource is getting cleaned up and will eventually disappear.

  • IMMEDIATE_DELETION_REQUIRED: If the resource is in a dangerous state and needs to be deleted immediately, even if it may violate the max_unavailable guarantees.

A resource which is broken or shutting down will be preferred during updating/deletion, followed by resources which are currently starting. Resources which are ready will be updated/deleted last.

abstract _get_spec_state(ctx: yaook.statemachine.context.Context, intent: Mapping, instance: yaook.statemachine.instancing.T) yaook.statemachine.instancing.ResourceSpecState

Get the state of the instance’s spec.

Parameters
  • ctx – The context of the operation.

  • intent – The intended state of the resource as returned by the wrapped_state.

  • instance – The current state of the resource as returned by the Kubernetes API.

Return:

  • UP_TO_DATE if the spec of the resource does not need updating.

  • STALE if the spec of the resource needs updating.

Mixins

class yaook.statemachine.instancing.PerNodeMixin(*, scheduling_keys: Collection[str], **kwargs: Any)

Base class

class yaook.statemachine.instancing.InstancedResource(*, wrapped_state: yaook.statemachine.resources.SingleObjectState[yaook.statemachine.instancing.T], **kwargs: Any)

context – Operative Context

class yaook.statemachine.Context(namespace: str, parent: Mapping[str, Any], parent_intf: yaook.statemachine.interfaces.ResourceInterfaceWithStatus[Mapping], instance: Optional[str], api_client: kubernetes_asyncio.client.api_client.ApiClient, logger: logging.Logger, field_manager: str)

Represent the context in which a state machine operation takes place.

It is comprised of:

  • Type and instance information about the parent custom resource

  • The namespace in which the operation takes place

  • An API client to use

  • A logger to use

  • And, if present, the instance of the component which is being worked on

namespace

The namespace in which all affected objects reside.

Note

This should only be used for listings. When a specific object is being looked for, you’ll usually have both its namespace and its name at your disposal, and then you should use the more specific source of the namespace value.

parent

The full body of the parent object.

parent_intf

The resource interface to access the parent resource.

parent_kind

The kind of the parent custom resource.

parent_plural

The plural of the parent custom resource.

parent_api_version

The apiVersion of the parent custom resource.

parent_name

The metadata.name of the parent custom resoucre.

parent_uid

The metadata.uid of the parent custom resource.

parent_spec

The JSON data representing the complete parent spec.

instance

If the current operation happens in the context of a instanced resource, for example due to PerNode, this field contains the string identifier of the instance.

field_manager

The fieldManager value to use when creating and updating Kubernetes API objects.

See also

Server-Side Apply in the Kubernetes documentation.

base_label_match() MutableMapping[str, str]
with_instance(instance: str) yaook.statemachine.context.Context

Create a child context for the given instance.

Parameters

instance (str) – The instance value to use.

Raises

ValueError – if the context already has an instance set.

Returns

A copy of the context with the instance field set to instance.

When attempting to call this method on a context which already has an instance set, ValueError is raised. This is because that usage indicates nested usage of state wrappers such as PerNode which is at this time not supported.

Labels and Annotations

yaook.statemachine.context.LABEL_PARENT_GROUP = 'state.yaook.cloud/parent-group'

The Kubernetes API group of the Custom Resource to which a managed object belongs. This is generally the string before the slash in the apiVersion value of the object.

yaook.statemachine.context.LABEL_PARENT_VERSION = 'state.yaook.cloud/parent-version'

The Kubernetes API version of the Custom Resource to which a managed object belongs. This is generally the string behind the slash in the apiVersion value of the object.

yaook.statemachine.context.LABEL_PARENT_PLURAL = 'state.yaook.cloud/parent-plural'

The Kubernetes API plural of the Custom Resource to which a managed object belongs.

yaook.statemachine.context.LABEL_PARENT_NAME = 'state.yaook.cloud/parent-name'

The name of the Custom Resource object to which a managed object belongs.

yaook.statemachine.context.LABEL_COMPONENT = 'state.yaook.cloud/component'

The name of the component represented by the object in the context of the custom resource. This is a freeform string assigned by the CustomResource instance.

yaook.statemachine.context.LABEL_INSTANCE = 'state.yaook.cloud/instance'

If multiple instances of a component exist, for example because of the use of PerNode, this label is used to distinguish the instances.

yaook.statemachine.context.LABEL_ORPHANED = 'state.yaook.cloud/orphaned'

If an object has been orhpaned by the state machine, it will be given this label.

yaook.statemachine.context.ANNOTATION_LAST_UPDATE = 'state.yaook.cloud/last-update-timestamp'

Mark the extended timestamp of the last update to the resource by the state machine.

templating – Template utilities

yaook.statemachine.templating.to_config_map_volumes(ref: kubernetes_asyncio.client.models.v1_object_reference.V1ObjectReference, extra_spec: Mapping) Mapping

Convert an instance mapping to a nodeMap for a configMap volume template for use with a ConfiguredDaemonSet.

yaook.statemachine.templating.to_secret_volumes(ref: kubernetes_asyncio.client.models.v1_object_reference.V1ObjectReference, extra_spec: Mapping) Mapping

Convert an instance mapping to a nodeMap for a secret volume template for use with a ConfiguredDaemonSet.

yaook.statemachine.templating.flat_merge_dict(d1: Mapping, d2: Mapping) MutableMapping

Merge d1 and d2 together and return the result.

This always returns a (flat) copy of the dictionaries and does not mutate any of the input dictionaries.

If the keys of d1 and d2 conflict, d2 wins.

yaook.statemachine.templating.parse_chmod(s: str) int

Convert the string representation of mode/permission bits into an integer.

Parameters

s – String representation of the permission bits.

Returns

The integer value represented.

This function supports two input formats:

  • Octal number with leading 0o (e.g. 0o755)

  • A subset of the syntax accepted by chmod. The following restrictions apply:

    • No support for sticky/setgid/setuid bits

    • No support for +/-

    • Setting the bits of the same subject multiple times in the same string is prohibited (i.e. u=r,u=w is illegal)

    • Leaving the subject blank (chmod does fun things based on umask in that case)

    Valid examples:

    • a=r (equal to 0o444)

    • u=rw,go=r (equal to 0o644)

    Invalid examples:

    • a=r,u=rw (duplicate assignment for u)

    • a=r,u+w (use of +)

    • =r (blank subject)

    • u=rws (setuid bit)

api_utils: Utilities for the k8s api

yaook.statemachine.label_match_to_selector(match: Mapping[str, str]) str

Convert a label match to a selector string.

Parameters

match – Dictionary mapping label keys to label values.

Returns

Label selector string which filters for the given labels and values.

yaook.statemachine.build_label_selector(selector: Union[Mapping[str, str], str]) str

Convert a selector into a proper string for the API.

Parameters

selector – Either a label match dictionary or a string which already is a proper selector.

Returns

The resulting label selector.

yaook.statemachine.extract_metadata(obj: Any) Dict[str, Any]

Extract the metadata dictionary from any kubernetes API object.

Parameters

obj – An object returned by the k8s API.

Returns

The metadata dictionary of that object.

This extract the following metadata:

  • annotations

  • creationTimestamp

  • deletionTimestamp

  • labels

  • name

  • namespace

  • resourceVersion

  • uid

If obj is a dictionary, this function returns the “metadata” key from that dictionary.

Otherwise, the metadata is extracted from the metadata attribute of obj and converted into a dictionary.

In addition, the following guarantees are made:

  • The labels key is never absent or None. Unless the input data contains a different type, it will always be a mapping.

yaook.statemachine.generate_update_timestamp() str

Generate a unique timestamp.

Returns

A string containing the current time as well as a random value to ensure uniqueness even in a distributed system.

The resulting value should be treated as opaque. Its internal structure may change at any time, however, it is supposed to be useful for human observers to reason about causality.

yaook.statemachine.deep_has_changes(old: Any, new: Any) bool

Check if new would change something in old when used in a patch.

Parameters
  • old – The current state of an object.

  • new – Input to a kubernetes PATCH API call.

This recursively compares old and new, returning True if and only if new changes or overwrites any keys in old.

yaook.statemachine.matches_labels(object_labels: Mapping[str, str], match_labels: Mapping[str, str]) bool

Check if the labels of an object match the given labels.

Parameters
  • object_labels – The label dictionary of the object to check.

  • match_labels – The label dictionary representing the labels to match.

Returns

True if the object is matched by the labels, false otherwise.

Returns true if and only if the object labels are a superset of the match labels and if the values for the labels are equal.

By extension, this means that:

  • An empty match_labels dict matches all objects.

  • If an object gains additional labels, it will still match an unchanged match_labels dict.

That means that this function can be used like label matchers in the Kubernetes API; they restrict the set of matched objects. The empty matcher matches all objects.

class yaook.statemachine.LabelSelector(*, match_labels: Mapping[str, str] = {}, match_expressions: Collection[yaook.statemachine.api_utils.LabelExpression] = ())

Match a set of object labels against the label selector.

Parameters
  • match_labels – A dictionary of labels to match on objects.

  • match_expressions – A collection of expressions.

When evaluating the selector, the expressions generated from match_labels and the expressions given via match_expressions are ANDed.

Note

The semantics of the LabelSelector are very similar to the selectors found in Kubernetes and which are represented by V1LabelSelector.

This class adds helper methods to work with such selectors.

classmethod from_dict(label_selector: Mapping) yaook.statemachine.api_utils.LabelSelector

Construct a LabelSelector from a dictionary.

Parameters

label_selector – The label selector to parse.

Raises

ValueError – if the label_selector has an unsupported format

The label_selector must be a dictionary with the following key:

  • matchLabels: a dictionary of label pairs to match

  • matchExpressions: a list of V1LabelSelectorRequirement JSON objects, i.e. dictionaries with key, operator, and values keys.

    See also

    LabelExpression

Note

The format of label_selector is the same as the V1LabelSelector of the Kubernetes API.

classmethod from_api_object(label_selector: kubernetes_asyncio.client.models.v1_label_selector.V1LabelSelector) yaook.statemachine.api_utils.LabelSelector
object_matches(object_labels: Mapping[str, str]) bool

Test if the labels of an object satisfy the selector.

Parameters

object_labels – The labels of an object as key-value pair mapping.

Returns

True if the labels satisfy the selector, false otherwise.

as_api_selector() str

Return a Kubernetes API compatible string representation of this selector.

Returns

A string which can be passed to the API as labelSelector argument.

class yaook.statemachine.LabelExpression(*, key: str, operator: yaook.statemachine.api_utils.SelectorOperator, values: Optional[Collection[str]])

Represent a single label selector expression for the Kubernetes API.

Parameters
  • key (str) – The label key to match

  • operator (SelectorOperator) – The operator to apply

  • values (collection of str or None (for In and NotIn operators)) – The right hand side of the operator.

classmethod from_dict(d: Mapping[str, Any]) yaook.statemachine.api_utils.LabelExpression

Parse a dict representing a V1LabelSelectorRequirement.

Parameters

d – The dictionary representing the JSON object representing the V1LabelSelectorRequirement.

Raises
  • KeyError – If a required key is missing

  • ValueError – If any value is invalid

  • ValueError – If values is missing or set to None but the operator requires a (possibly empty) set of values.

Returns

The resulting expression

Return type

LabelExpression

Parse the JSON object dictionary and return the corresponding label expression.

classmethod from_api_object(req: kubernetes_asyncio.client.models.v1_label_selector_requirement.V1LabelSelectorRequirement) yaook.statemachine.api_utils.LabelExpression

Convert a V1LabelSelectorRequirement into a LabelSelector.

Parameters

req – The requirement.

Raises
  • KeyError – If a required key is missing

  • ValueError – If any value is invalid

  • ValueError – If values is missing or set to None but the operator requires a (possibly empty) set of values.

Returns

The resulting expression

Return type

LabelExpression

object_matches(labels: Mapping[str, str]) bool

Test whether the labels of an object satisfy the expression.

Parameters

object_labels – The labels of an object as key-value pair mapping.

Returns

True if the labels satisfy the expression`, false otherwise.

as_api_selector() str

Return a string fragment representing this expression for use with the Kubernetes API.

If the operator is IN and only a single value is given, the shorthand version (key=value) is used instead of a set expression.

class yaook.statemachine.SelectorOperator(value)

See also

The operator key in the Kubernetes V1LabelSelectorRequirement object.

EXISTS

The Exists operator

IN

The In operator

NOT_IN

The NotIn operator

NOT_EXISTS

The DoesNotExist operator

class yaook.statemachine.ResourceReference(api_version: str, plural: str, namespace: typing.Optional[str], name: str)

Reference a resource by its API endpoint.

In contrast to the kubernetes_asyncio.client.V1ObjectReference, this reference directly refers to an API endpoint; the Kubernetes object reference uses kind instead of plural, which can not be used interchangably.

yaook.statemachine.split_api_version(s: str) Tuple[str, str]

Split a Kubernetes API version string into its API group and version parts.

Parameters

s – An API version string, such as v1 or apps/v1`.

Returns

The separate string as tuple of API group and version.

If the API version string refers to the core API group (i.e. only consits of a version number), the API group is returned as empty string.

yaook.statemachine.join_api_version(api_group: str, version: str) str

Compose API group and version into a single Kubernetes API version string.

Parameters
  • api_group – The API group. To refer to the core API group, use the empty string.

  • version – The version string.

Returns

The correctly composed API version string.

async yaook.statemachine.api_utils.multi_selector_list(listfn: Callable[[...], Coroutine[Any, Any, Iterable[yaook.statemachine.api_utils.T]]], selectors: Iterable[yaook.statemachine.api_utils.LabelSelector]) AsyncGenerator[yaook.statemachine.api_utils.T, None]

Request and return all listings for the given selectors.

Parameters
  • listfn – Listing function (see below)

  • selectors – The selectors to pass to the listing function

For each selector in selectors, this calls listfn with the label_selector keyword argument set to the API string of the selector.

The items from the function calls are yielded in an arbitrary order.

yaook.statemachine.api_utils.inject_scheduling_keys(pod_spec: MutableMapping[str, Any], scheduling_keys: Collection[str]) None

Add tolerations and a node affinity for the given set of scheduling keys.

Parameters
  • pod_spec – The specification of the Pod. Modified in-place.

  • scheduling_keys – Collection of scheduling keys to apply to the Pod.

Raises

ValueError – if the pod_spec already contains scheduling requirements, either in nodeAffinity or nodeSelector.

The Pod spec is modified in-place.

yaook.statemachine.api_utils.extract_pod_references(pod_spec: Mapping, namespace: str) Collection[yaook.statemachine.api_utils.ResourceReference]

Extract all referenced resources from a Pod specification.

Parameters
  • pod_spec (dict or kubernetes_asyncio.client.V1PodSpec) – The pod specification.

  • namespace (str) – The namespace in which the Pod exists.

Returns

A collection of references to the resources used by the Pod.

This function supports extracting references to config maps, secrets and persistent volume claims in environment variables and volumes of the Pod. Other resource references are currently not supported.

Note

This function expects a Pod spec, not a full Pod body.

The namespace is required to fill in the ResourceReference.namespace for those references where no explicit namespace is given in the Pod spec.

yaook.statemachine.api_utils.extract_cds_references(cds_spec: Mapping, namespace: str) Collection[yaook.statemachine.api_utils.ResourceReference]

Extract all referenced resources from a ConfiguredDaemonSet (CDS) specification.

Parameters
  • cds_spec (dict) – The cds specification.

  • namespace (str) – The namespace in which the CDS exists.

Returns

A collection of references to the resources used by the CDS.

This function supports extracting references to config maps, secrets and persistent volume claims in environment variables and volumes of the Pod, and the volume templates in the CDS itself. Other resource references are currently not supported.

Note

This function expects a CDS spec, not a full CDS body.

The namespace is required to fill in the ResourceReference.namespace for those references where no explicit namespace is given in the spec.

yaook.statemachine.api_utils.get_cluster_domain() str