yaook.op – Operator framework

common – Common utilities shared among operators

yaook.op.common.extract_labeled_configs(configset: List[Mapping], object_labels: Mapping[str, str]) List[Mapping]
yaook.op.common.transpose_config(configsets: Sequence[Mapping], keys: Sequence[str]) MutableMapping[str, List[Any]]
async yaook.op.common.get_node_labels_from_instance(ctx: yaook.statemachine.context.Context) Mapping[str, str]

Obtain the labels of the node referenced by the instance of the context.

Parameters

ctx (statemachine.Context) – The context to use.

Returns

The label dictionary of the node.

This looks up the node with the name matching the instance attribute of the context, using the API client of the context.

class yaook.op.common.SchedulingKey

This enumeration holds all scheduling keys used by the Yaook operators.

See also

Scheduling

for a general discussion of how Yaook handles scheduling.

Operators

OPERATOR_ANY = 'operator.yaook.cloud/any'

Used to schedule any of the Yaook operator Jobs.

OPERATOR_BARBICAN = 'operator.yaook.cloud/barbican'

Used to schedule yaook.op.barbican Jobs.

OPERATOR_CINDER = 'operator.yaook.cloud/cinder'

Used to schedule yaook.op.cinder Jobs.

OPERATOR_CONFIGURED_DAEMON_SET = 'operator.yaook.cloud/configured-daemon-set'
OPERATOR_GLANCE = 'operator.yaook.cloud/glance'

Used to schedule yaook.op.glance Jobs.

OPERATOR_HEAT = 'operator.yaook.cloud/heat'

Used to schedule yaook.op.heat Jobs.

OPERATOR_KEYSTONE = 'operator.yaook.cloud/keystone'

Used to schedule yaook.op.keystone Jobs.

OPERATOR_NEUTRON = 'operator.yaook.cloud/neutron'

Used to schedule yaook.op.neutron Jobs.

OPERATOR_NOVA = 'operator.yaook.cloud/nova'

Used to schedule yaook.op.nova Jobs.

OPERATOR_GNOCCHI = 'operator.yaook.cloud/gnocchi'

Used to schedule yaook.op.gnocchi Jobs.

OPERATOR_CEILOMETER = 'operator.yaook.cloud/ceilometer'

Used to schedule yaook.op.ceilometer Jobs.

Infrastructure services

ANY_INFRA = 'infra.yaook.cloud/any'

Used to schedule any infrastructure component.

INFRA_DATABASE = 'infra.yaook.cloud/db'

Used to schedule database-related services.

INFRA_MESSAGE_QUEUE = 'infra.yaook.cloud/mq'

Used to schedule services related to or implementing message queues.

API Services

ANY_API = 'any.yaook.cloud/api'

Used to schedule any of the OpenStack API service deployments.

BLOCK_STORAGE_API = 'block-storage.yaook.cloud/api'

Used to schedule the cinder-api Deployment.

IMAGE_API = 'image.yaook.cloud/api'

Used to schedule the glance-api Deployment.

IDENTITY_API = 'identity.yaook.cloud/api'

Used to schedule the keystone-api Deployment.

NETWORK_API = 'network.yaook.cloud/api'

Used to schedule the neutron-api Deployment.

COMPUTE_API = 'compute.yaook.cloud/api'

Used to schedule the nova-api Deployment.

COMPUTE_METADATA_API = 'compute.yaook.cloud/metadata-api'

Used to schedule the nova-metadata-api Deployment.

KEY_MANAGER_API = 'key-manager.yaook.cloud/api'

Used to schedule the barbican-api deployment.

PLACEMENT_API = 'placement.yaook.cloud/api'

Used to schedule the placement-api Deployment.

GNOCCHI_API = 'gnocchi.yaook.cloud/api'

Used to schedule gnocchi-api Deployment.

Barbican supporting services

KEY_MANAGER_BARBICAN_ANY_SERVICE = 'key-manager.yaook.cloud/barbican-any-service'

Used to schedule any of the barbican supporting services (barbican-keystone-listener).

KEY_MANAGER_BARBICAN_KEYSTONE_LISTENER = 'key-manager.yaook.cloud/barbican-keystone-listener'

Used to schedule the barbican-keystone-listener service.

Cinder supporting services

BLOCK_STORAGE_CINDER_ANY_SERVICE = 'block-storage.yaook.cloud/cinder-any-service'

Used to schedule any of the Cinder supporting services (backup, volume, scheduler).

BLOCK_STORAGE_CINDER_BACKUP = 'block-storage.yaook.cloud/cinder-backup'

Used to schedule the cinder-backup service.

BLOCK_STORAGE_CINDER_SCHEDULER = 'block-storage.yaook.cloud/cinder-scheduler'

Used to schedule the cinder-scheduler service.

BLOCK_STORAGE_CINDER_VOLUME = 'block-storage.yaook.cloud/cinder-volume'

Used to schedule the cinder-volume service.

Nova supporting services

COMPUTE_NOVA_ANY_SERVICE = 'compute.yaook.cloud/nova-any-service'

Used to schedule any of the Nova supporting services (conductor, scheduler).

COMPUTE_NOVA_CONDUCTOR = 'compute.yaook.cloud/nova-conductor'

Used to schedule the nova-conductor Deployment.

COMPUTE_NOVA_SCHEDULER = 'compute.yaook.cloud/nova-scheduler'

Used to schedule the nova-scheduler Deployment.

Network services

NETWORK_NEUTRON_L2_AGENT = 'network.yaook.cloud/neutron-l2-agent'

Used to schedule the neutron-l2-agent ConfiguredDaemonSet, among other scheduling keys.

See also

Neutron L2 Agent

for more details on L2 agent scheduling

NETWORK_NEUTRON_L3_AGENT = 'network.yaook.cloud/neutron-l3-agent'

Used to schedule the neutron-l3-agent ConfiguredDaemonSet.

NETWORK_NEUTRON_DHCP_AGENT = 'network.yaook.cloud/neutron-dhcp-agent'

Used to schedule the neutron-dhcp-agent ConfiguredDaemonSet.

Compute services

COMPUTE_HYPERVISOR = 'compute.yaook.cloud/hypervisor'

Used to schedule the nova-compute and libvirt ConfiguredDaemonSets.

Gnocchi supporting services

GNOCCHI_METRICD = 'gnocchi.yaook.cloud/metricd'

Used to schedule the gnocchi-metricd Deployment.

Ceilometer services

CEILOMETER_ANY_SERVICE = 'ceilometer.yaook.cloud/ceilometer-any-service'

Used to schedule any of the Ceilometer services (central, notification).

CEILOMETER_NOTIFICATION = 'ceilometer.yaook.cloud/notification'

Used to schedule the ceilometer-notification Deployment.

CEILOMETER_CENTRAL = 'ceilometer.yaook.cloud/central'

Used to schedule the ceilometer-central Deployment.

Note

If we ever support other hypervisor technologies than Libvirt, the Libvirt service will get its own label and Nova Compute may use a scheduling similar to the Neutron L2 Agent to appear on all relevant nodes.

tasks – Management of recurring and resilient tasks

class yaook.op.tasks.ExponentialBackOff(base: int, start: float, max_: int)

Iterable to represent the sleep intervals used for an exponential back-off.

Parameters
  • base – The factor by which the back-off value is multiplied on each iteration.

  • start – The initial value.

  • max – The maximum value for the back-off.

This object is iterable. Iterating it will yield values starting at start, increasing by a factor of base until they reach max_. At that point, only max_ is returned indefinitely.

The iterator can be reset to its original value by calling reset().

reset() None

Reset the back-off to its initial state and clear failing.

failing

Returns true if the iterator has been iterated since the last call to reset() or if the iterator was just created.

class yaook.op.tasks.TaskQueue(*, backoff_base: int = 2, backoff_start: float = 0.5, backoff_max: int = 120, jitter: float = 0.1, logger: Optional[logging.Logger] = None)

De-duplicating and re-trying queue of tasks.

Parameters
  • backoff_base – The base of the exponential back-off for failed tasks.

  • backoff_start – The start value of the exponential back-off for failed tasks.

  • backoff_max – The cutoff value for the exponential back-off for failed tasks.

  • jitter – The jitter ratio to use for rescheduling failed tasks.

  • logger – The logger to use for logging information about the queue itself.

Note

The queue does not do anything unless run() is running.

Task are initially run in the order they are enqueued.

If a task is run and fails, it will be rescheduled for a later attempt, without blocking other enqueued tasks. Hence, the order of the tasks can not be used to enforce dependencies between tasks. Repeatedly failing tasks will be rescheduled with exponential back-off, but never removed from the queue completely.

Tasks are rescheduled not in a strict exponential back-off, but are slightly jittered based on the jitter argument. This is to spread out the load on consumed resources even more if several tasks fail at the same time.

async run() None

Run the queue, forever.

class yaook.op.tasks.RestartingTask(coroutine_function: Callable, logger: Optional[logging.Logger] = None, loop: Optional[asyncio.events.AbstractEventLoop] = None)

Background task which gets restarted with exponential back-off if it crashes.

Parameters
  • coroutine_function – The coroutine function implementing the task.

  • logger – A logger used to log internals of the object.

  • loop – An optional asyncio event loop to use.

The initial state of the RestartingTask is stopped. To start it, call the start() method. At that point, the coroutine_function is called without arguments and scheduled to be run by the event loop.

If the coroutine raises an exception, it will be restarted with an exponential back-off (up to 120 seconds). If the coroutine exits without error or with a cancellation, it will not be restarted automatically.

When stop() is called and the task is currently running, it will be cancelled (and not automatically restarted).

start() None

Start the task to run it in the background.

This resets the internal exponential back-off for crashes and starts the task immediately, if it is not runnnig. If the task is already running, this method only resets the back-off.

restart() None

Stop and start the task.

If the task has been stopped before, this is equivalent to calling start().

stop() None

Stop the task.

If the task is not running, this does nothing. Otherwise, the task is cancelled.

async wait_for_termination() None

Stop the task and wait for it to terminate.

Exceptions from the task are not propagated but are logged as usual.