Nova

# NOTE: This file serves as an example only!
# It demonstrates a subset of the API features which are available and
# is NOT FIT FOR PRODUCTIVE USE. You need to adapt them to your
# specific environment, needs and scale.
# DO NOT DEPLOY THIS FILE WITH THE EXPECTATION THAT YOU WILL GET A
# YAOOK DEPLOYMENT FIT FOR ANY USE EXCEPT LOOKING AT IT.
apiVersion: yaook.cloud/v1
kind: NovaDeployment
metadata:
  name: nova
spec:
  keystoneRef:
    name: keystone
  database:
    api:
      replicas: 1
      # have to be the same value as cell0 and cell1 timeoutClient
      timeoutClient: 300
      proxy: {}
      mysqlConfig:
        mysqld:
          max_connections: 1337
          max_heap_table_size: 64M
      backup:
        schedule: "0 * * * *"
    placement:
      replicas: 1
      timeoutClient: 300
      proxy: {}
      mysqlConfig:
        mysqld:
          max_connections: 1337
          max_heap_table_size: 64M
      backup:
        schedule: "0 * * * *"
    cell0:
      replicas: 1
      # have to be the same value as cell1 and api timeoutClient
      timeoutClient: 300
      proxy: {}
      mysqlConfig:
        galera:
          wsrep_slave_threads: 3
      backup:
        schedule: "0 * * * *"
    cell1:
      replicas: 1
      resources:
        rabbitmq:
          limits:
            memory: 1G
            cpu: 1000m
      # have to be the same value as cell0 and api timeoutClient
      timeoutClient: 300
      proxy: {}
      backup:
        schedule: "0 * * * *"
  eviction:
    volumeLockDurationSeconds: 30
    netboxSetFailed: True
    netboxCreds:
      credentialsSecretRef:
        # Example netbox-token secret:
        # URL: "https://netbox.example.com"
        # TOKEN: "abcdef1234567890abcdef1234567890abcdef12"
        name: "<netbox-token>"
    ironicNodeShutdown:
      credentialsSecretRef:
        name: "<ironicNodeShutdownSecret>"
  messageQueue:
    cell1:
      replicas: 1
  memcached: {}
  api:
    replicas: 1
    ingress:
      fqdn: "nova.yaook.cloud"
      port: 32443
  conductor:
    replicas: 1
  placement:
    replicas: 1
    ingress:
      fqdn: "placement.yaook.cloud"
      port: 32443
  scheduler:
    replicas: 1
  vnc:
    ingress:
      fqdn: vnc.yaook.cloud
      port: 32443
    replicas: 1
  metadata:
    replicas: 1
  targetRelease: 2023.1
  novaConfig:
    DEFAULT:
      debug: True
    database:
      # (optional) connection_recycle_time hould always be ~10%
      # smaller than the haproxy timeoutClient
      connection_recycle_time: 280
  policy:
    "context_is_admin": "role:admin"
  compute:
    configTemplates:
      - nodeSelectors:
          - matchLabels: {}
        novaComputeConfig:
          DEFAULT:
            debug: True
          keystone_authtoken:
            os_region_name: MyRegion
        volumeBackends:
          ceph:
            enabled: True
            keyringSecretName: rook-ceph-client-cinder
            user: cinder
            uuid: "b3ab713d-912b-49ed-adaf-bd74368e567a"
            cephConfig: {}
      - nodeSelectors:
          - matchLabels:
              "compute.yaook.cloud/hypervisor-type": "qemu"
        novaComputeConfig:
          libvirt:
            virt_type: qemu
      - nodeSelectors:
          - matchLabels:
              "aggregate": "azandsomeproperties"
        hostAggregates:
          - aggregate-az-and-property
          - aggregate-dual-property
  issuerRef:
    name: ca-issuer
  region:
    name: MyRegion
  databaseCleanup:
    schedule: "0 0 * * *"
    clearAllEvents: False
    clearOnlyMigrationEvents: False
    deletionTimeRange: 60
  placementCleanup:
    schedule: "0 0 1 * *"

Ceph Config

Note

To use ceph you need to specify the same uuid and the same secret name as it is in use for cinder. The usage of different keys/users is not supported by openstack.

Add vTPM to instance

To add a vTPM, the following prequisites are to be fullfilled.

  1. Enable the swtpm settings in the nova-compute config for the nodes which are to provide the vTPM to the instance

# NOTE: This file serves as an example only!
# It demonstrates a subset of the API features which are available and
# is NOT FIT FOR PRODUCTIVE USE. You need to adapt them to your
# specific environment, needs and scale.
# DO NOT DEPLOY THIS FILE WITH THE EXPECTATION THAT YOU WILL GET A
# YAOOK DEPLOYMENT FIT FOR ANY USE EXCEPT LOOKING AT IT.
apiVersion: yaook.cloud/v1
kind: NovaDeployment
metadata:
  name: nova
spec:
  compute:
    configTemplates:
    - nodeSelectors:
      - matchLabels: {}
      novaComputeConfig:
        libvirt:
          swtpm_enabled: True
          swtpm_user: nova
          swtpm_group: libvirt

As we specify the same user/group within the qemu config for swtpm we highly recommend to use nova:libvirt within the novacompute config.

2. Set the image or flavor specs to provide a vTPM if an instance is created with either of those. openstack flavor set $FLAVOR -property hw:tpm_version=2.0 --property hw:tpm_model=tpm-crb or openstack image set $IMAGE -property hw_tpm_version=2.0 --property hw_tpm_model=tpm-crb (reference https://docs.openstack.org/nova/latest/admin/emulated-tpm.html)

  1. Have barbican as secret manager present

  2. Nova (and compute) of atleast Victoria Release

  3. Make sure the novacompute keystone user has permissions to create(store) barbican secrets (and read them).

Note

Yaook allows shelve and evacuate actions for vTPM enabled servers. To archive this the vTPM becomes ephemeral. This is comparable with the baremetal behavior where the TPM content is gone after moving a disk to a new hardware.

Add provider config to nova compute

Nova allows you to manage resource providers using config files for each compute node. (https://docs.openstack.org/nova/latest/admin/managing-resource-providers.html) This can be used for example to configure traits for isolated host aggregate filtering. (https://docs.openstack.org/nova/latest/reference/isolate-aggregates.html)

Yaook supports to set this config, like shown in this example:

# NOTE: This file serves as an example only!
# It demonstrates a subset of the API features which are available and
# is NOT FIT FOR PRODUCTIVE USE. You need to adapt them to your
# specific environment, needs and scale.
# DO NOT DEPLOY THIS FILE WITH THE EXPECTATION THAT YOU WILL GET A
# YAOOK DEPLOYMENT FIT FOR ANY USE EXCEPT LOOKING AT IT.
apiVersion: yaook.cloud/v1
kind: NovaDeployment
metadata:
  name: nova
spec:
  compute:
    configTemplates:
    - computeProviderConfig:
        providers:
        - traits:
            additional:
            - CUSTOM_LICENSED_WINDOWS
      nodeSelectors:
      - matchLabels:
          aggregate/imgos: windows

Eviction

the eviction part is mandatory. it contains two parameter.

  • ironicNodeShutdown (optional): It is used to power off the node via ironic when it is not available. It contains a secret with ironic openrc credentials.

  • volumeLockDurationSeconds (optional): wait for releasing the volume lock during the eviction. If NodeShutdown is not set, it is not needed to wait for the volume release. For this reason you can set it to 0. If ironicNodeShutdown is enabled, please set this parameter to the volume release time of your storage backend. This is especially needed for NetApp. If Ceph is used, the exclusive locks feature will handle the volume release, so please leave this setting at the default value.

apiVersion: v1
kind: Secret
metadata:
  name: ironic_creds
  namespace: yaook
type: Opaque
data:
  OS_AUTH_URL: <os_auth_url>
  OS_USERNAME: <os_username>
  OS_PASSWORD: <os_password>
  OS_PROJECT_NAME: <os_project_name>
  OS_REGION_NAME: <os_region_name>
  OS_INTERFACE: <os_infterface>
  OS_USER_DOMAIN_NAME: <os_user_domain_name>
  OS_PROJECT_DOMAIN_NAME: <os_project_domain_name>

NovaHostAggregate

Availability-Zone and Properties are both optional. A computenode can only reside in a single Availability-Zone. If there is a mapping for aggregates with conflicting Availability-Zone values, no mapping will take place. (there will be an Exception for that)

Example of a NovaHostAggregate

# NOTE: This file serves as an example only!
# It demonstrates a subset of the API features which are available and
# is NOT FIT FOR PRODUCTIVE USE. You need to adapt them to your
# specific environment, needs and scale.
# DO NOT DEPLOY THIS FILE WITH THE EXPECTATION THAT YOU WILL GET A
# YAOOK DEPLOYMENT FIT FOR ANY USE EXCEPT LOOKING AT IT.
apiVersion: compute.yaook.cloud/v1
kind: NovaHostAggregate
metadata:
  name: aggregate-az-and-property
  namespace: yaook
spec:
  novaRef:
    name: nova
  keystoneRef:
    name: keystone
  zone: southeast-asia-42
  properties:
    cake: lie
    foo: bar