Audit logging for OpenStack APIs
================================

OpenStack provides an `audit middleware <https://docs.openstack.org/keystonemiddleware/latest/audit.html>`_ that is able to record and emit auditing events in the `Cloud Auditing Data Federation (CADF) <https://www.dmtf.org/standards/cadf>`_ format based on API actions.

The `openstack-audit-middleware <https://github.com/sapcc/openstack-audit-middleware>`_ library is a continued and extended fork of the original keystonemiddleware implementation.

For a subset of OpenStack services supported by YAOOK, audit logging capability is provided by the YAOOK images.
This usually entails:

- **openstack-audit-middleware** installed in the service image (except for Keystone and Octavia)
- a WSGI Paste pipeline configuration ``api-paste-audit.ini`` file embedded in the service image
- an audit mapping YAML file for the audit middleware embedded in the service image

.. note::

    Keystone is an exception to this rule as it natively supports CADF logging through oslo.messaging without the use of the middleware and provides a dedicated configuration option for this.

Activating audit logging
------------------------

CADF audit logging using **openstack-audit-middleware** in OpenStack API services (except Keystone) can be configured to happen in one of two ways:

1. Native logging without the use of the oslo.messaging library.
2. Emitting event notifications using oslo.messaging via various drivers, including targets like log and message queue.

If oslo.messaging is not available or not enabled, the **openstack-audit-middleware** will fall back to native logging.

.. caution::

    Some guides will mention options like ``driver`` and ``use_oslo_messaging`` in the context of the audit logging middlewares.
    Note that the ``audit_middleware_notifications.driver`` setting is only effective if oslo.messaging is installed in the service image *and* ``audit_middleware_notifications.use_oslo_messaging`` is ``True``.
    Setting the driver to ``noop`` *does not* generally disable audit logging if oslo.messaging is not used via these options.
    The audit middleware will always fall back to native logging in such case!

The subsequent sections will describe how to enable audit logging in YAOOK depending on the service.

Keystone
^^^^^^^^

Keystone is different from the other services as it does not use a middleware and is able to emit CADF logging events on its own through the oslo.messaging library.

.. note::

    In YAOOK, each applicable OpenStack service has a dedicated message queue cluster (RabbitMQ).
    Some services, including Keystone, do not have any message queue at all because it is not required for normal operation.
    Setting ``spec.keystoneConfig.oslo_messaging_notifications.driver`` to any of the message queue targets (e.g. ``messagingv2``) will not work because Keystone lacks the necessary message queue connection.
    Use the ``log`` driver instead.

For audit logging in Keystone, set ``spec.keystoneConfig.DEFAULT.notification_format`` to ``cadf`` and add ``log`` to ``spec.keystoneConfig.oslo_messaging_notifications.driver`` in your KeystoneDeployment manifest, like so:

.. code-block:: yaml

    apiVersion: yaook.cloud/v1
    kind: KeystoneDeployment
    ...
    spec:
      keystoneConfig:
        DEFAULT:
          notification_format: "cadf"
        oslo_messaging_notifications:
          driver:
            - log

Keystone does not require a Paste configuration file or audit map.

Octavia
^^^^^^^

Octavia is similar to Keystone in a way that it has a strict dependency on oslo.messaging.
In contrast to other non-Keystone services, it does not use a WSGI Paste pipeline adjustment but instead incorporates the middleware directly through a custom ``audit`` config section:

.. code-block:: yaml

    apiVersion: yaook.cloud/v1
    kind: OctaviaDeployment
    ...
    spec:
      octaviaConfig:
        audit:
          enabled: True
          audit_map_file: "/usr/local/etc/octavia/octavia_api_audit_map.conf"
        audit_middleware_notifications:
          use_oslo_messaging: True
          driver: "log"

The ``octavia_api_audit_map.conf`` file is provided by the YAOOK image for Octavia by default.
It is based on the `template provided by Octavia itself <https://github.com/openstack/octavia/tree/master/etc/audit>`_.

.. note::

    Octavia is the only service aside from Keystone that is incompatible with the **openstack-audit-middleware** fork as it is hardwired to the audit module of keystonemiddleware, which uses an ini-based `.conf` audit map file instead of the YAML used by **openstack-audit-middleware**.

Other OpenStack APIs
^^^^^^^^^^^^^^^^^^^^

For OpenStack API services other than Keystone and Octavia adhere to the subsequent sections in order to enable direct audit logging through the middleware without the need for oslo.messaging.

.. note::

    Similar to Keystone, some OpenStack services lack a message queue in YAOOK (e.g. Glance).
    Enabling ``audit_middleware_notifications.use_oslo_messaging`` and setting ``audit_middleware_notifications.driver`` to any of the message queue targets (e.g. ``messagingv2``) will not work in these cases.
    It is recommended to use direct logging for all services instead.

To enable the middleware, an ``api-paste-audit.ini`` file is used that includes WSGI Paste pipeline definitions which inject the audit middleware into the API routes for consuming requests.
The individual file is included by the `yaook/images/ <https://gitlab.com/yaook/images/>`_ builds and the path is individual to each service image.
It usually follows a simple pattern: it is located within a subfolder of ``/usr/local/etc/`` named after the service.
If in doubt, consult the respective image build files.
The Glance variant has a ``glance-`` prefix, just as the default file shipped with Glance has.

An audit mapping YAML file is also built into the image and is referenced within the ini file itself.
It is usually taken from the `templates of openstack-audit-middleware <https://github.com/sapcc/openstack-audit-middleware/tree/master/etc>`_.

As ``audit_middleware_notifications.use_oslo_messaging`` is disabled by default in YAOOK, injecting the middleware via the ini file will automatically lead to direct logging of audit events in the affected service's regular log.
There is no toggle for this behavior other than removing the middleware from the WSGI Paste pipeline, i.e., by removing the ini file path from the config section.

Nova, Cinder, Neutron, Barbican
"""""""""""""""""""""""""""""""

In the main manifest of your respective service (e.g. CinderDeployment for Cinder), set ``DEFAULT.api_paste_config`` in the config section.
For Cinder for example, it would look like this:

.. code-block:: yaml

    apiVersion: yaook.cloud/v1
    kind: CinderDeployment
    ...
    spec:
      cinderConfig:
        DEFAULT:
          api_paste_config: "/usr/local/etc/cinder/api-paste-audit.ini"

As the spec section as well as the ini path contain the service's name, make sure to adjust both according to the respective service!

Designate
"""""""""

Designate uses different section:

.. code-block:: yaml

    apiVersion: yaook.cloud/v1
    kind: DesignateDeployment
    ...
    spec:
      designateConfig:
        service:api:
          api_paste_config: "/usr/local/etc/designate/api-paste-audit.ini"

Glance
""""""

Glance uses a different section, different config key and ini filename:

.. code-block:: yaml

    apiVersion: yaook.cloud/v1
    kind: GlanceDeployment
    ...
    spec:
      glanceConfig:
        paste_deploy:
          config_file: "/usr/local/etc/glance/glance-api-paste-audit.ini"

Verifying audit logging
-----------------------

To verify that audit logging is working correctly, issue an API command (e.g. creating a basic OpenStack resource) in each of the affected services and observe their log output.
The audit log messages will be in JSON format and include CADF ``typeURI`` references like ``schemas.dmtf.org/cloud/audit/1.0/event``.

For Keystone for example, creating a project would eventually emit a special audit log entry like the following:

.. code-block:: text

    {"message_id": "ee0e13a7-3cb6-4737-b283-58d90982562e"
    "publisher_id": "identity.keystone-api-797494746b-hvqsh",
    "event_type": "identity.project.created", "priority": "INFO",
    "payload": {"typeURI": "http://schemas.dmtf.org/cloud/audit/1.0/event", ...

Due to the difference in implementation, audit logs of services aside from Keystone and Octavia might look slightly different.
For example, creating an image in Glance would create audit log messages similar to the following:

.. code-block:: text

    {"message": "Event type: audit.cadf, Context: {},
    Payload: {'typeURI': 'http://schemas.dmtf.org/cloud/audit/1.0/event',
    'eventType': 'activity', 'id': '1015b9d8-1e25-529e-824f-5c8aac8fa5e3',
    'eventTime': '2025-06-30T13:47:49.067924+00:00', 'action': 'create', ...