2023.1 Series Release Notes

14.7.0-6

New Features

  • Adds the internal VIP to the NOPROXY/noproxy environment variables.

  • Adds support for using Cumulus switches (NCLU) with Networking Generic Switch.

Security Issues

  • When running API requests from a host configured with kayobe, traffic destined for the internal VIP is sent via the default proxy. This can be a security issue if not using TLS as the proxy will be able to intercept the traffic. If using an untrusted proxy, with TLS disabled on the internal VIP, it is recommended that you run kayobe overcloud host configure -t proxy, kayobe seed hypervisor host configure -t proxy, kayobe seed host configure -t proxy, and kayobe infra vm host configure -t proxy, to add the internal VIP to the no proxy configuration. This is considered a minor issue as traffic between containers will not use the proxy by default. LP#2087556

Bug Fixes

  • The proxysql image is now built when kolla_enable_proxysql is set to true.

  • Changes the default cloud image for seed and infra VMs to use Rocky Linux 9.3 when using bios boot mode, to fix boot failures with newer cloud images. When deploying new VMs, it is recommended to set infra_vm_boot_firmware and seed_vm_boot_firmware to efi.

14.7.0

New Features

  • Adds support for specifying boot_firmware and machine variables to seed and infra VMs. This can be used to launch VMs in UEFI boot mode with Q35 machine type.

  • Bumps stackhpc.libvirt-vm Ansible role to v1.16.1.

Bug Fixes

  • eos_config does not support the provider parameter since Ansible 7. Users are required to update their configuration according to Arista EOS documentation.

  • Fixes a bug where systemd-networkd was not permanently enabled when the unit was already in state runtime-enabled. LP#2073100

14.6.0

New Features

  • Adds support for custom RabbitMQ configuration.

  • Adds a new dev script dev/rabbitmq-migrate-queues.sh that will enable quorum queues and migrate RabbitMQ to use these.

Bug Fixes

  • Fixes an issue where Dell OS6 and Dell OS9 switch configuration was not applied correctly. LP#2061102.

  • Fixes issue of ironic files being left behind after node deprovision which prevents it from being enrolled and provisioned again.

  • Fixes default Ubuntu Apt keyrings location to the recommended /etc/apt/keyrings.

  • Fixes the bug where /etc/hosts was not populated correctly when running Kayobe using a host limit. LP#2051714

  • Fixes an issue with overcloud service destroy where it failed to remove the inspection store docker volume. See LP#2050092.

14.5.0

Bug Fixes

  • Fixes the wipe-disks role which was failing on supported host operating systems due to a change in the output format of lsblk -J in util-linux version 2.37. LP#2051859

14.4.0

Deprecation Notes

  • Support for the devicemapper Docker storage driver is deprecated following its removal from Docker Engine 25.0. Support will be fully removed in the Caracal 16.0.0 release. Operators using devicemapper should ensure that a compatible version of Docker Engine is installed (i.e. release 24.x or below).

14.3.0

New Features

  • Adds a new kolla_bifrost_deploy_image_user_data_content variable used to define custom user_data content used by cloud-init for overcloud provision.

Upgrade Notes

  • If the admin network does not have a gateway defined and seed_enable_snat is false, which is the default, overcloud hosts will not have a default gateway immediately after provisioning anymore. A default gateway on another network can still be applied during the host configuration step.

  • Introduces a new variable kolla_ansible_extra_custom_passwords to avoid the need to combine kolla_ansible_default_custom_passwords and kolla_ansible_custom_passwords when adding or overriding passwords.

  • Removes the kolla_docker_registry_insecure variable from etc/kayobe/kolla.yml as it is not used since the 2023.1 (Antelope) release. The replacement docker_registry_insecure variable has been added to etc/kayobe/docker.yml.

Bug Fixes

  • Fixes an issue where local configuration generation would be skipped when running in check mode. This would lead to Kolla Ansible checking with stale configuration. See story 2010526 for details.

  • Fixes an issue where kayobe configuration dump would fail when variables are encrypted using Ansible Vault. Encrypted variables are now sanitised in the dump output. LP#2031390

  • Fixes slow fact gathering in some environments by not configuring the seed host as the initial default gateway for overcloud hosts when seed_enable_snat is false, which is the default. LP#2039461

  • Fixes an issue where the Kolla Ansible variable kolla_admin_openrc_cacert was not set to the value of kolla_internal_fqdn_cacert.

  • Fixes gateway assignment when seed SNAT is disabled. In this circumstance Bifrost was generating ConfigDrive data with the default gateway unset even when one is available on the admin network.

  • Fixes a bug where NetworkManager would overwrite resolv.conf when resolv_is_managed is set to True. LP#2044537

  • When determining whether or not a host needs bootstrapping, we attempt to connect to the host using ansible_user, if the login fails, we then assume that the host needs bootstrapping. In previous releases we used a manually crafted ssh command. This did not respect any customisations to the SSH arguments made through Ansible configuration. We now use the raw module so that these customisations are used when connecting to the host. One possible use case is to configure a jump host between the control host and the target hosts. If bootstrapping was needed, hosts will now show as unreachable in the summary stats at the end of the run. This can safely be ignored.

  • Fixes an issue when user forgot to combine kolla_ansible_custom_passwords, kolla_ansible_default_custom_passwords and own dictionary with custom passwords in configuration files. Now kolla_ansible_extra_custom_passwords should provide only user custom passwords to add or override in kolla/passwords.yml.

  • Removes the kolla_docker_registry_insecure variable from etc/kayobe/kolla.yml as it is not used since the 2023.1 (Antelope) release. The replacement docker_registry_insecure variable has been added to etc/kayobe/docker.yml.

14.2.0

New Features

  • The Spanning Tree Protocol (STP) can now be configured on bridge interfaces. Enable or disable STP by setting the bridge_stp attribute for a network. Note that STP is not set by default on Ubuntu, but it is disabled on Rocky Linux 9 for compatibility with network scripts, as NetworkManager enables STP on all bridges by default.

  • Kolla Ansible inventories in the Kayobe configuration are now passed through without modification. Previously, only group_vars were passed through. When using multiple environments, the Kolla inventory from the base configuration layer and the Kolla inventory from the Kayobe environment layer will be passed through. The inventory from the environment takes precedence over the inventory from the base layer. This allows you to put any shared configuration in the base layer.

  • Attempts to log in to the kolla docker registry can be skipped by setting deploy_containers_registry_attempt_login to false.

    This is required for deployments using a non-standard registry deployed on the seed during the deploy-container step, since it takes place after the registry login attempt.

Upgrade Notes

  • For Rocky Linux 9, Kayobe now disables STP on a bridge by default. This action will cause the bridge interface to restart during the host configuration process.

  • As Kolla Ansible inventories are now passed through without modification, the inventory directory in Kayobe configuration (etc/kayobe/kolla/inventory/) must be a valid Ansible inventory, although *.j2 files used as Kolla Ansible inventory templates are ignored. For cases where only group_vars or hosts_vars are required, a blank inventory file in the same directory may be used.

  • It is no longer possible to create an environment named kayobe. This is reserved for internal use.

  • Adds an introspection rule to update the location of the deployment kernel registered in existing Ironic nodes. Nodes discovered on a deployment running the Train release or earlier may still be using the ipa.vmlinuz kernel, which stays unchanged when deployment images get updated. If only default introspection rules are in use, existing nodes may be updated from the Bifrost container with the following command:

    OS_CLOUD=bifrost baremetal introspection reprocess $NODE_UUID_OR_NAME

    If non-default rules are used, reprocessing may revert any customisation done by the operator. In this case, a more cautious approach is to update the deployment kernel location manually:

    OS_CLOUD=bifrost baremetal node set --driver-info deploy_kernel=<http://url/to/ipa.kernel> $NODE_UUID_OR_NAME

    If the kolla_bifrost_inspector_rules list is customised, the rule inspector_rule_legacy_deploy_kernel should be added to it.

Bug Fixes

  • Fixes failure to run kayobe overcloud deprovision after Bifrost is redeployed. LP#2038889

  • Improves performance of Bifrost operations by preventing unnecessary requests to the Ironic API.

  • Fixes detection of data file path when using editable installations with a recent pip.

  • Fixes the regression in configuring additional route options on CentOS / Rocky.

  • Fixed issue of seed containers being unable to use password protected registry by adding docker login function to kayobe deploy-containers role.

  • Adds a workaround to avoid NetworkManager setting the MTU of bridge VLAN interfaces to an incorrect value. LP#2039947

  • Fixes conflicts between NetworkManager nmconnection files generated by cloud-init and those generated by Kayobe by upgrading the MichaelRigart.interfaces role to version 1.14.4. LP#2039975

14.1.0

Bug Fixes

  • Fixes download of roles from Ansible Galaxy following the renaming of the mrlesmithjr.manage_lvm role. LP#2023502

  • Installs ncclient dependency for Juniper switch configuration when using Ansible check mode.

14.0.0

New Features

  • Adds the command kayobe overcloud service configuration validate to run the oslo-config-validator on all hosts via Kolla-Ansible.

  • Adds support for custom Multipathd configuration.

  • Improves failure handling in the kayobe * host configure commands by avoiding use of the kolla-ansible bootstrap-servers command, and moving all relevant functionality to Kayobe playbooks. This ensures that if a host fails during a host configuration command, other hosts are able to continue to completion. This is useful at scale, where host failures occur more frequently. See story 2009854 for details. Refer to the upgrade notes for information about the implications of this change.

  • Adds functionality into the kolla_passwords module to allow passwords that are generated for Kolla Ansible to be stored in Hashicorp Vault.

  • Adds support for configuring arbitrarily named VLAN interfaces using systemd-networkd. See story 2010266 for details.

  • Since Kolla containers can built with user provided repos.yaml Kayobe can override the file with their own content. The override files can be ${KAYOBE_CONFIG_PATH}/kolla/repos.yaml (default Kolla filename) or ${KAYOBE_CONFIG_PATH}/kolla/repos.yml. Multiple Environments supported.

Upgrade Notes

  • Updates the maximum supported version of Ansible from 6.x (ansible-core 2.13) to 7.x (ansible-core 2.14). The minimum supported version is updated from 5.x to 6.x. This is true for both Kayobe and Kolla Ansible.

  • ipa_build_dib_elements_default and ipa_build_dib_env_default now use os_distribution and os_release by default. This means that Ubuntu images will now be built with the Ironic Python Agent when running on Ubuntu. Rocky will still build CentOS images, as Rocky IPA images have not been tested yet.

  • The kayobe * host configure commands no longer use the kolla-ansible bootstrap-servers command, and associated baremetal role in Kolla Ansible. The functionality provided by the baremetal role has been extracted into a new openstack.kolla Ansible collection, and split into separate roles. This allows Kayobe to use it directly, and only the necessary parts.

    This change improves failure handling in these Kayobe commands, and aims to reduce confusion over which --limit and --tags arguments to provide.

    This change has implications for configuration of Kayobe, since some variables that were previously in Kolla Ansible are now in Kayobe. The following is an incomplete list of variables that have changed scoped from Kolla Ansible to Kayobe:

    * ``enable_docker_repo``
* ``docker_apt_url``
* ``docker_apt_repo``
* ``docker_apt_key_file``
* ``docker_apt_key_id``
* ``docker_apt_package``
* ``docker_yum_url``
* ``docker_yum_baseurl``
* ``docker_yum_gpgkey``
* ``docker_yum_gpgcheck``
* ``docker_yum_package``
* ``customize_etc_hosts``
* ``docker_storage_driver``
* ``docker_custom_option``
* ``docker_custom_config``
* ``docker_http_proxy``
* ``docker_https_proxy``
* ``docker_no_proxy``
* ``debian_pkg_install``
* ``redhat_pkg_install``
* ``ubuntu_pkg_removals``
* ``redhat_pkg_removals``

    The following Kolla Ansible variables are no longer relevant:

    * ``create_kolla_user``
* ``create_kolla_user_sudoers``
* ``kolla_user``
* ``kolla_group``
* ``change_selinux``
* ``selinux_state``
* ``host_python_version``
* ``virtualenv``
* ``virtualenv_site_packages``

  • Modifies the default value of kolla_ansible_venv_python to /usr/bin/python3. Using operating system python to create kolla-ansible venv fixes corner cases when using older venvs created with virtualenv command.

Bug Fixes

  • Fixes an issue where generation of passwords.yml for Kolla Ansible could fail if the directory containing the file does not exist. This is typical in a multiple environment setup, when creating a new environment. See story 2010293 for details.

  • Fixes an issue with systemd-networkd configuration on Ubuntu with multiple VLAN interfaces. See story 2009013 for details.

  • Fixes repositories files names in Rocky Linux 9. Distributions moved to lowercase names with RHEL 9 release.

  • Fixes various issues when applying network configuration on Rocky 9 hosts. See bugs: 2016970 and 2016971.

  • Synchronises the default value kolla_tag with the container image tagging scheme expected by Kolla Ansible. This ensures images are built with tags such as zed-ubuntu-jammy instead of zed.