Building Image Hotfix for CVE ============================= As OpenStack downstream project, we may get notified ahead of time about critical security vulnerabilities in OpenStack. This means, some people get patches before the public get's announced about a security advisory. .. important:: All information received via this must be treated as confidential, only shared with the people working on the issue and NEVER released to the public before the upstream advisory gets published!!! In the following a process how to handle the patches, so we can provide fixed images as soon the embargo falls. Yaook-Security Repository ------------------------- We use a separate repository to build the patched images. Main reason is to not loose opensource state at gitlab if we have to even one private repositories. The steps need to be done by a member of the yaook-security group. 1. Fork the repo to ``yaook-security`` group (may to subgroup ``images``) if not exist yet. **Create the repo as private!!** If the repo already exist, make sure it is set to private. 2. Create a branch, push the provided patches and backported patches there. Add the patches to the image build script. 3. Check that the pipeline runs and includes your patches. The images will get pushed to the (private) gitlab registry. We use the images of the feature branch and don't merge them at the fork. 4. At the day the CVE will get published, download the patched images from gitlab registry. You need to download the `-amd64` and `-arm64` version, as we want to provide both. Download for each supported OpenStack release. 5. Make sure the original image Repo can't get any changes on the main/devel branch. (e.g. by disabling push and merge for everyone.) 6. Check at the upstream yaook image what the next image version is (for the most images, this is in ``.next-version`` file.) 7. Tag the downloaded images with the original image name + tag of the next version. 8. Wait for the CVE to get published. 9. As soon the CVE got published (e.g. by making the bug public or message to openstack-announce mailinglist): 1. Upload both image versions for all OpenStack releases. 2. Create a manifest and push it, e.g.: ``` docker manifest create "registry.yaook.cloud/yaook/keystone-${OPENSTACK_VERSION}:${image_tag}" "registry.yaook.cloud/yaook/keystone-${OPENSTACK_VERSION}:${image_tag}-arm64" "registry.yaook.cloud/yaook/keystone-${OPENSTACK_VERSION}:${image_tag}-amd64" docker manifest push "registry.yaook.cloud/yaook/keystone-${OPENSTACK_VERSION}:${image_tag}" ``` 10. Set the repository to public to show how the images got build. 11. Create a MR from the fork and branch with patches to the original repository. 12. Delete or depublish the repository after at most one week. So no one can later see by checking the repos if there are new CVEs (based on that we took the repo offline again). CI Variable Settings ******************** The group and subgroups needs some settings, so pipelines works without changes to the repos. This is configured one time for the whole group/subgroup and don't need adjustment afterwards. At group level: ``REGISTRY_PASS`` Set to `$CI_REGISTRY_PASSWORD` with "Expand variable reference" enabled, Visibility: Visible ``REGISTRY_PROJECT`` Set to `yaook-security` Visibility: Visible ``REGISTRY_URL`` Set to `$CI_REGISTRY` with "Expand variable reference" enabled, Visibility: Visible ``REGISTRY_USER`` Set to `$CI_REGISTRY_USER` with "Expand variable reference" enabled, Visibility: Visible For each subgroup the ``REGISTRY_PROJECT`` needs to get overridden: ``REGISTRY_PROJECT`` Set to `yaook-security/images/$CI_PROJECT_NAME` with "Expand variable reference" enabled, Visibility: Visible Change the subgroup with the name your group has. Pipeline Runner *************** We have separate pipeline runners hosted by `ALASCA `__ for us. An ARM-Runner is currently hosted at STACKIT. Members of Yaook-Security Group ------------------------------- The list of the members is keept as small as possible. Only the two project maintainers and people known to be on the embargo list are part of the group. Based on each issue additional people can be add to only the affected project, if they work creating/backporting the patches.