oslo_config_validator¶

About the role¶

An Ansible role that will loop through all the containers on selected host, find the Openstack service configuration file and leverage the [oslo-config-validator](https://docs.openstack.org/oslo.config/latest/cli/validator.html) utility to validate the current running configuration.

It’s also possible to generate a report that contains all differences between the sample or default values with current running configuration.

Finally, it will also verify that the current running configuration doesn’t contain any known invalid settings that might have been deprecated and removed in previous versions.

Exceptions¶

Some services like cinder can have dynamic configuration sections. In cinder’s case, this is for the storage backends. To perform validation on these dynamic sections, we need to generate a yaml formatted config sample with oslo-config-generator beforehand, append a new sample configuration for each storage backends, and validate against that newly generated configuration file by passing --opt-data to the oslo-config-validator command instead of using --namespaces. Since generating a sample config adds some delay to the validation, this is not the default way of validating, we prefer to validate directly using --namespaces.

NOTE: At the time of writing this role, oslo-config-generator has a bug [1] when generating yaml config files, most notably with cinder. Since the inclusion of oslo.config patch can’t be garanteed, the role will inject this patch [2] to the oslo.config code, inside the validation container. This code change is ephemeral for the time of the configuration file generation. The reason why we want to inject this patch is because it’s possible that we run the validation on containers that were created before it was merged. This ensures a smooth validation across the board.

[1] https://bugs.launchpad.net/oslo.config/+bug/1928582 [2] https://review.opendev.org/c/openstack/oslo.config/+/790883

Requirements¶

This role needs to be run on an Undercloud with a deployed Overcloud.

Role Variables¶

oslo_config_validator_validation: Wether or not to run assertions on produced outputs. That also means that the role will fail if anything is output post-filtering. If this is enabled with the reporting, this will most likely trigger a failure unless executed against default configuration
oslo_config_validator_report: Wether or not we compare the configuration files found with the default config
oslo_config_validator_invalid_settings: When running validation, wether or not we should check for invalid settings. This adds to the time it takes to complete validation because of the way the validations_read_ini module works. This won’t work without oslo_config_validator_validation enabled.
oslo_config_validator_report_path: The folder used when generating the reports.
oslo_config_validator_global_ignored_messages: List of regular expressions that will filter out messages globally, across all namespaces
oslo_config_validator_namespaces_config: Specific namespace configurations. It contains namespace-specific ignored patterns as well as invalid settings configuration.
oslo_config_validator_service_configs: Mapping of known Openstack services with their namespace configuration.
oslo_config_validator_checked_services: List of services being validated.

Dependencies¶

podman_container
podman_container_info
validations_read_ini
https://review.opendev.org/c/openstack/oslo.config/+/790883

Example Reporting Playbook¶

- hosts: all
  vars:
  - oslo_config_validator_report: true
  - oslo_config_validator_validation: false
  roles:
  - { role: oslo_config_validator}

Example playbook to validate only one service¶

- hosts: all
  vars:
  - oslo_config_validator_checked_services:
  - nova
  roles:
  - { role: oslo_config_validator}

License¶

Apache

Author Information¶

Red Hat TripleO DFG:Compute Deployment Squad

Full Description¶

Role Documentation¶

Welcome to the “oslo_config_validator” role documentation.

Role Defaults¶

This section highlights all of the defaults and variables set within the “oslo_config_validator” role.

oslo_config_validator_checked_services: '{{ oslo_config_validator_service_configs.keys()
  | list }}'
oslo_config_validator_debug: false
oslo_config_validator_global_ignored_messages:
- INFO:keyring.backend:Loading.*
- WARNING:oslo_config.generator:normalizing group name.*
- WARNING:stevedore.named:Could not load .*
- WARNING:root:Deprecated opt .*
- INFO:oslo_utils.netutils:Could not determine default network interface, using .*
- ERROR:root:quotas/quota_[^\s]+ {{ invalid_setting_regex }}
- 'Error: error configuring network namespace for container'
- .*Ignoring option because it is part of the excluded patterns. This can be changed
  with the.*
oslo_config_validator_invalid_settings: true
oslo_config_validator_namespaces_config:
- ignored_messages:
  - 'ERROR:stevedore.extension:Could not load ''(glance.store.)*s3(.Store)*'': No
    module named ''boto3'''
  - ERROR:root:DEFAULT/enable_v1_api {{ invalid_setting_regex }}
  namespace: glance.store
- ignored_messages:
  - ERROR:root:(yaql|(resource_finder_)*cache)/[^\s]+ {{ invalid_setting_regex }}
  namespace: heat.common.config
- ignored_messages:
  - ERROR:root:pxe/ipxe_enabled {{ invalid_setting_regex }}
  - ERROR:root:conductor/max_time_interval {{ invalid_setting_regex }}
  namespace: ironic
- ignored_messages:
  - .*Ignoring missing option "(auth_url|username|password|(user|project)_(domain_)*name)"
    from group "keystone_authtoken" because the group is known to have incomplete
    sample config data and thus cannot be validated properly.*
  namespace: keystonemiddleware.auth_token
- ignored_messages:
  - ERROR:root:placement/[^\s]+ {{ invalid_setting_regex }}
  invalid_settings:
  - operator: eq
    option: tenant_name
    section: nova
    value_list:
    - service
  - operator: eq
    option: project_name
    section: nova
    value_list:
    - service
  - operator: not
    option: notify_nova_on_port_data_changes
    section: DEFAULT
    value_list:
    - 'True'
  - operator: not
    option: notify_nova_on_port_status_changes
    section: DEFAULT
    value_list:
    - 'True'
  namespace: neutron
- ignored_messages:
  - ERROR:root:DEFAULT/api_paste_config {{ invalid_setting_regex }}
  namespace: cinder
- ignored_messages:
  - ERROR:root:DEFAULT/ovsdb_connection {{ invalid_setting_regex }}
  - ERROR:root:vif_plug_ovs/ovsdb_connection {{ invalid_setting_regex }}
  - ERROR:root:(cinder|service_user)/region_name {{ invalid_setting_regex }}
  - WARNING:root:neutron/metadata_proxy_shared_secret sample value is empty but input-file
    has
  - ERROR:root:cache/tls_enabled not found
  invalid_settings:
  - option: enabled_filters
    section: filter_scheduler
    separator: ','
    value_list:
    - ExactCoreFilter
    - ExactRamFilter
    - ExactDiskFilter
    - CoreFilter
    - RamFilter
    - DiskFilter
  - operator: lt
    option: vif_plugging_timeout
    section: DEFAULT
    value_list:
    - 300
  - option: vif_plugging_timeout
    section: DEFAULT
    value_list:
    - 'True'
  namespace: nova.conf
oslo_config_validator_report: false
oslo_config_validator_report_archive: true
oslo_config_validator_report_path: /var/tmp/config_validator_report
oslo_config_validator_service_configs:
  cinder:
    config_files:
    - namespaces: []
      path: /etc/cinder/cinder.conf
    default_namespaces:
    - castellan.config
    - cinder
    - keystonemiddleware.auth_token
    - oslo.log
    - oslo.messaging
    - oslo.policy
    - oslo.privsep
    - oslo.service.periodic_task
    - oslo.service.service
    - oslo.db
    - oslo.db.concurrency
    - oslo.middleware
    - oslo.concurrency
    - osprofiler
    ignored_groups:
    - nova
    - service_user
    opt_data:
    - index_key:
        option: enabled_backends
        section: DEFAULT
        separator: ','
      template_section:
      - backend_defaults
      - backend
  glance:
    config_files:
    - namespaces: []
      path: /etc/glance/glance-api.conf
    - namespaces: []
      path: /etc/glance/glance-cache.conf
    - namespaces: []
      path: /etc/glance/glance-image-import.conf
    - namespaces: []
      path: /etc/glance/glance-registry.conf
    - namespaces: []
      path: /etc/glance/glance-scrubber.conf
    - namespaces: []
      path: /etc/glance/glance-swift.conf
    default_namespaces:
    - glance
    - glance.api
    - glance.store
    - glance.multi_store
    - keystonemiddleware.auth_token
    - oslo.log
    - oslo.messaging
    - oslo.policy
    - oslo.privsep
    - oslo.service.periodic_task
    - oslo.service.service
    - oslo.db
    - oslo.db.concurrency
    - oslo.middleware.cors
    - oslo.middleware.http_proxy_to_wsgi
    ignored_groups:
    - ref1
    - default_backend
  heat:
    config_files:
    - namespaces: []
      path: /etc/heat/heat.conf
    default_namespaces:
    - heat.common.config
    - heat.common.context
    - heat.common.crypt
    - heat.engine.clients.os.keystone.heat_keystoneclient
    - heat.common.wsgi
    - heat.engine.clients
    - heat.engine.notification
    - heat.engine.resources
    - heat.api.aws.ec2token
    - keystonemiddleware.auth_token
    - oslo.messaging
    - oslo.middleware
    - oslo.db
    - oslo.log
    - oslo.policy
    - oslo.service.service
    - oslo.service.periodic_task
    - oslo.service.sslutils
  ironic:
    config_files:
    - namespaces: []
      path: /etc/ironic/ironic.conf
    - namespaces: []
      path: /etc/ironic-inspector/inspector.conf
    default_namespaces:
    - ironic
    - ironic_lib.disk_utils
    - ironic_lib.disk_partitioner
    - ironic_lib.exception
    - ironic_lib.mdns
    - ironic_lib.metrics
    - ironic_lib.metrics_statsd
    - ironic_lib.utils
    - oslo.db
    - oslo.messaging
    - oslo.middleware.cors
    - oslo.middleware.healthcheck
    - oslo.middleware.http_proxy_to_wsgi
    - oslo.concurrency
    - oslo.policy
    - oslo.log
    - oslo.reports
    - oslo.service.service
    - oslo.service.periodic_task
    - oslo.service.sslutils
    - osprofiler
    - keystonemiddleware.auth_token
  keystone:
    config_files:
    - namespaces: []
      path: /etc/keystone/keystone.conf
    default_namespaces:
    - keystone
    - oslo.cache
    - oslo.log
    - oslo.messaging
    - oslo.policy
    - oslo.db
    - oslo.middleware
    - oslo.service.sslutils
    - osprofiler
  neutron:
    config_files:
    - namespaces: []
      path: /etc/neutron/neutron.conf
    - namespaces: []
      path: /etc/neutron/plugins/ml2/ml2_conf.ini
    default_namespaces:
    - neutron
    - neutron.agent
    - neutron.base.agent
    - neutron.db
    - neutron.extensions
    - nova.auth
    - ironic.auth
    - placement.auth
    - oslo.log
    - oslo.db
    - oslo.policy
    - oslo.privsep
    - oslo.concurrency
    - oslo.messaging
    - oslo.middleware.cors
    - oslo.middleware.http_proxy_to_wsgi
    - oslo.service.sslutils
    - oslo.service.wsgi
    - keystonemiddleware.auth_token
  nova:
    config_files:
    - namespaces: []
      path: /etc/nova/nova.conf
    default_namespaces:
    - nova.conf
    - keystonemiddleware.auth_token
    - oslo.log
    - oslo.messaging
    - oslo.policy
    - oslo.privsep
    - oslo.service.periodic_task
    - oslo.service.service
    - oslo.db
    - oslo.db.concurrency
    - oslo.cache
    - oslo.middleware
    - oslo.concurrency
    - osprofiler
  placement:
    config_files:
    - namespaces: []
      path: /etc/placement/placement.conf
    default_namespaces:
    - placement.conf
    - keystonemiddleware.auth_token
    - oslo.log
    - oslo.middleware.cors
    - oslo.policy
oslo_config_validator_validation: true
oslo_config_validator_work_path: /var/lib/tripleo-config/oslo_config_validator

Role Variables: main.yml¶

config_locations: {}
config_validations: []
invalid_setting_regex: (is not part of the sample config|not found)
metadata:
  description: 'This role is intended to leverage the `oslo-config-validator` on each
    one of the configuration files found on a deployment. The goal is to quickly catch
    erroneous configurations.

    When called manually, it will also be possible to generate a report returning
    all the differences between the current configuration and the default configuration

    '
  groups:
  - backup-and-restore
  - pre-upgrade
  - post-deployment
  - post-system-upgrade
  - post-update
  name: Openstack services configuration validation
validation_output: []

Molecule Scenarios¶

Molecule is being used to test the “oslo_config_validator” role. The following section highlights the drivers in service and provides an example playbook showing how the role is leveraged.

Scenario: default¶

Example default configuration¶

provisioner:
  env:
    ANSIBLE_LIBRARY: ${ANSIBLE_LIBRARY:-/usr/share/ansible/plugins/modules}:../../resources/library
    ANSIBLE_STDOUT_CALLBACK: yaml
    MOLECULE_OCV:
      config_file: etc/nova/nova.conf
      config_folder: /var/lib/config-data/puppet-generated/nova_libvirt
      service_name: nova_compute
      validator_out: 'ERROR:root:DEFAULT/ovsdb_connection not found

        ERROR:root:cinder/region_name not found

        INFO:root:Ignoring missing option "auth_url" from group "keystone_authtoken"
        because the group is known to have incomplete sample config data and thus
        cannot be validated properly.

        INFO:root:Ignoring missing option "username" from group "keystone_authtoken"
        because the group is known to have incomplete sample config data and thus
        cannot be validated properly.

        INFO:root:Ignoring missing option "password" from group "keystone_authtoken"
        because the group is known to have incomplete sample config data and thus
        cannot be validated properly.

        INFO:root:Ignoring missing option "user_domain_name" from group "keystone_authtoken"
        because the group is known to have incomplete sample config data and thus
        cannot be validated properly.

        INFO:root:Ignoring missing option "project_name" from group "keystone_authtoken"
        because the group is known to have incomplete sample config data and thus
        cannot be validated properly.

        INFO:root:Ignoring missing option "project_domain_name" from group "keystone_authtoken"
        because the group is known to have incomplete sample config data and thus
        cannot be validated properly.

        ERROR:root:service_user/region_name not found

        '
  inventory:
    hosts:
      all:
        hosts:
          centos:
            ansible_python_interpreter: /usr/bin/python3
  log: true
  name: ansible
  playbooks:
    converge: ../../resources/playbooks/converge.yml
    prepare: ../../resources/playbooks/prepare.yml

Molecule Inventory¶

hosts:
  all:
    hosts:
      centos:
        ansible_python_interpreter: /usr/bin/python3

Example default playbook¶

- environment:
    ANSIBLE_LIBRARY: ../../resources/library
    ANSIBLE_STDOUT_CALLBACK: yaml
  hosts: all
  name: Converge
  tasks:
  - block:
    - include_role:
        name: oslo_config_validator
      name: Include the oslo_config_validator role
    rescue:
    - fail:
        msg: Default test failed
      when: molecule_yml.scenario.name == "default"
    - fail:
        msg: Scenario {{ molecule_yml.scenario.name }} was suppsoed to fail
      when:
      - not validation_errors | count

Scenario: mocked_failure¶

Example mocked_failure configuration¶

provisioner:
  env:
    ANSIBLE_LIBRARY: ${ANSIBLE_LIBRARY:-/usr/share/ansible/plugins/modules}:../../resources/library
    ANSIBLE_STDOUT_CALLBACK: yaml
    MOLECULE_OCV:
      config_file: etc/nova/nova.conf
      config_folder: /var/lib/config-data/puppet-generated/nova_libvirt
      service_name: nova_compute
      validator_out: 'ERROR:root:Houston we''ve got a problem

        '
  inventory:
    hosts:
      all:
        hosts:
          centos:
            ansible_python_interpreter: /usr/bin/python3
  log: true
  name: ansible
  playbooks:
    converge: ../../resources/playbooks/converge.yml
    prepare: ../../resources/playbooks/prepare.yml

Molecule Inventory¶

hosts:
  all:
    hosts:
      centos:
        ansible_python_interpreter: /usr/bin/python3

Example mocked_failure playbook¶

- environment:
    ANSIBLE_LIBRARY: ../../resources/library
    ANSIBLE_STDOUT_CALLBACK: yaml
  hosts: all
  name: Converge
  tasks:
  - block:
    - include_role:
        name: oslo_config_validator
      name: Include the oslo_config_validator role
    rescue:
    - fail:
        msg: Default test failed
      when: molecule_yml.scenario.name == "default"
    - fail:
        msg: Scenario {{ molecule_yml.scenario.name }} was suppsoed to fail
      when:
      - not validation_errors | count