Compute schedulers¶

AggregateCoreFilter¶

Filters host by CPU core numbers with a per-aggregate cpu_allocation_ratio value. If the per-aggregate value is not found, the value falls back to the global setting. If the host is in more than one aggregate and more than one value is found, the minimum value will be used. For information about how to use this filter, see Host aggregates and availability zones. See also CoreFilter.

Note the cpu_allocation_ratio bug 1804125 restriction.

AggregateDiskFilter¶

Filters host by disk allocation with a per-aggregate disk_allocation_ratio value. If the per-aggregate value is not found, the value falls back to the global setting. If the host is in more than one aggregate and more than one value is found, the minimum value will be used. For information about how to use this filter, see Host aggregates and availability zones. See also DiskFilter.

Note the disk_allocation_ratio bug 1804125 restriction.

AggregateImagePropertiesIsolation¶

Matches properties defined in an image’s metadata against those of aggregates to determine host matches:

• If a host belongs to an aggregate and the aggregate defines one or more metadata that matches an image’s properties, that host is a candidate to boot the image’s instance.
• If a host does not belong to any aggregate, it can boot instances from all images.

For example, the following aggregate myWinAgg has the Windows operating system as metadata (named ‘windows’):

$ openstack aggregate show myWinAgg
+-------------------+----------------------------+
| Field             | Value                      |
+-------------------+----------------------------+
| availability_zone | zone1                      |
| created_at        | 2017-01-01T15:36:44.000000 |
| deleted           | False                      |
| deleted_at        | None                       |
| hosts             | [u'sf-devel']              |
| id                | 1                          |
| name              | myWinAgg                   |
| properties        | os_distro='windows'        |
| updated_at        | None                       |
+-------------------+----------------------------+

In this example, because the following Win-2012 image has the windows property, it boots on the sf-devel host (all other filters being equal):

$ openstack image show Win-2012
+------------------+------------------------------------------------------+
| Field            | Value                                                |
+------------------+------------------------------------------------------+
| checksum         | ee1eca47dc88f4879d8a229cc70a07c6                     |
| container_format | bare                                                 |
| created_at       | 2016-12-13T09:30:30Z                                 |
| disk_format      | qcow2                                                |
| ...                                                                     |
| name             | Win-2012                                             |
| ...                                                                     |
| properties       | os_distro='windows'                                  |
| ...                                                                     |

You can configure the AggregateImagePropertiesIsolation filter by using the following options in the nova.conf file:

# Considers only keys matching the given namespace (string).
# Multiple values can be given, as a comma-separated list.
aggregate_image_properties_isolation_namespace = <None>

# Separator used between the namespace and keys (string).
aggregate_image_properties_isolation_separator = .

AggregateInstanceExtraSpecsFilter¶

Matches properties defined in extra specs for an instance type against admin-defined properties on a host aggregate. Works with specifications that are scoped with aggregate_instance_extra_specs. Multiple values can be given, as a comma-separated list. For backward compatibility, also works with non-scoped specifications; this action is highly discouraged because it conflicts with ComputeCapabilitiesFilter filter when you enable both filters. For information about how to use this filter, see the Host aggregates and availability zones section.

AggregateIoOpsFilter¶

Filters host by disk allocation with a per-aggregate max_io_ops_per_host value. If the per-aggregate value is not found, the value falls back to the global setting. If the host is in more than one aggregate and more than one value is found, the minimum value will be used. For information about how to use this filter, see Host aggregates and availability zones. See also IoOpsFilter.

AggregateMultiTenancyIsolation¶

Ensures hosts in tenant-isolated Host aggregates and availability zones will only be available to a specified set of tenants. If a host is in an aggregate that has the filter_tenant_id metadata key, the host can build instances from only that tenant or comma-separated list of tenants. A host can be in different aggregates. If a host does not belong to an aggregate with the metadata key, the host can build instances from all tenants. This does not restrict the tenant from creating servers on hosts outside the tenant-isolated aggregate.

For example, consider there are two available hosts for scheduling, HostA and HostB. HostB is in an aggregate isolated to tenant X. A server create request from tenant X will result in either HostA or HostB as candidates during scheduling. A server create request from another tenant Y will result in only HostA being a scheduling candidate since HostA is not part of the tenant-isolated aggregate.

AggregateNumInstancesFilter¶

Filters host by number of instances with a per-aggregate max_instances_per_host value. If the per-aggregate value is not found, the value falls back to the global setting. If the host is in more than one aggregate and thus more than one value is found, the minimum value will be used. For information about how to use this filter, see Host aggregates and availability zones. See also NumInstancesFilter.

AggregateRamFilter¶

Filters host by RAM allocation of instances with a per-aggregate ram_allocation_ratio value. If the per-aggregate value is not found, the value falls back to the global setting. If the host is in more than one aggregate and thus more than one value is found, the minimum value will be used. For information about how to use this filter, see Host aggregates and availability zones. See also RamFilter.

Note the ram_allocation_ratio bug 1804125 restriction.

AggregateTypeAffinityFilter¶

This filter passes hosts if no instance_type key is set or the instance_type aggregate metadata value contains the name of the instance_type requested. The value of the instance_type metadata entry is a string that may contain either a single instance_type name or a comma-separated list of instance_type names, such as m1.nano or m1.nano,m1.small. For information about how to use this filter, see Host aggregates and availability zones.

AllHostsFilter¶

This is a no-op filter. It does not eliminate any of the available hosts.

AvailabilityZoneFilter¶

Filters hosts by availability zone. You must enable this filter for the scheduler to respect availability zones in requests.

ComputeCapabilitiesFilter¶

Matches properties defined in extra specs for an instance type against compute capabilities. If an extra specs key contains a colon (:), anything before the colon is treated as a namespace and anything after the colon is treated as the key to be matched. If a namespace is present and is not capabilities, the filter ignores the namespace. For backward compatibility, also treats the extra specs key as the key to be matched if no namespace is present; this action is highly discouraged because it conflicts with AggregateInstanceExtraSpecsFilter filter when you enable both filters.

Some virt drivers support reporting CPU traits to the Placement service. With that feature available, you should consider using traits in flavors instead of ComputeCapabilitiesFilter, because traits provide consistent naming for CPU features in some virt drivers and querying traits is efficient. For more detail, please see Support Matrix, Required traits, Forbidden traits and Report CPU features to the Placement service.

Also refer to Compute capabilities as traits.

ComputeFilter¶

Passes all hosts that are operational and enabled.

In general, you should always enable this filter.

CoreFilter¶

Only schedules instances on hosts if sufficient CPU cores are available. If this filter is not set, the scheduler might over-provision a host based on cores. For example, the virtual cores running on an instance may exceed the physical cores.

You can configure this filter to enable a fixed amount of vCPU overcommitment by using the cpu_allocation_ratio configuration option in nova.conf. The default setting is:

cpu_allocation_ratio = 16.0

With this setting, if 8 vCPUs are on a node, the scheduler allows instances up to 128 vCPU to be run on that node.

To disallow vCPU overcommitment set:

cpu_allocation_ratio = 1.0

DifferentHostFilter¶

Schedules the instance on a different host from a set of instances. To take advantage of this filter, the requester must pass a scheduler hint, using different_host as the key and a list of instance UUIDs as the value. This filter is the opposite of the SameHostFilter. Using the openstack server create command, use the --hint flag. For example:

$ openstack server create --image cedef40a-ed67-4d10-800e-17455edce175 \
  --flavor 1 --hint different_host=a0cf03a5-d921-4877-bb5c-86d26cf818e1 \
  --hint different_host=8c19174f-4220-44f0-824a-cd1eeef10287 server-1

With the API, use the os:scheduler_hints key. For example:

{
    "server": {
        "name": "server-1",
        "imageRef": "cedef40a-ed67-4d10-800e-17455edce175",
        "flavorRef": "1"
    },
    "os:scheduler_hints": {
        "different_host": [
            "a0cf03a5-d921-4877-bb5c-86d26cf818e1",
            "8c19174f-4220-44f0-824a-cd1eeef10287"
        ]
    }
}

DiskFilter¶

Only schedules instances on hosts if there is sufficient disk space available for root and ephemeral storage.

You can configure this filter to enable a fixed amount of disk overcommitment by using the disk_allocation_ratio configuration option in the nova.conf configuration file. The default setting disables the possibility of the overcommitment and allows launching a VM only if there is a sufficient amount of disk space available on a host:

disk_allocation_ratio = 1.0

DiskFilter always considers the value of the disk_available_least property and not the one of the free_disk_gb property of a hypervisor’s statistics:

$ openstack hypervisor stats show
+----------------------+-------+
| Field                | Value |
+----------------------+-------+
| count                | 1     |
| current_workload     | 0     |
| disk_available_least | 14    |
| free_disk_gb         | 27    |
| free_ram_mb          | 15374 |
| local_gb             | 27    |
| local_gb_used        | 0     |
| memory_mb            | 15886 |
| memory_mb_used       | 512   |
| running_vms          | 0     |
| vcpus                | 8     |
| vcpus_used           | 0     |
+----------------------+-------+

As it can be viewed from the command output above, the amount of the available disk space can be less than the amount of the free disk space. It happens because the disk_available_least property accounts for the virtual size rather than the actual size of images. If you use an image format that is sparse or copy on write so that each virtual instance does not require a 1:1 allocation of a virtual disk to a physical storage, it may be useful to allow the overcommitment of disk space.

To enable scheduling instances while overcommitting disk resources on the node, adjust the value of the disk_allocation_ratio configuration option to greater than 1.0:

disk_allocation_ratio > 1.0

ImagePropertiesFilter¶

Filters hosts based on properties defined on the instance’s image. It passes hosts that can support the specified image properties contained in the instance. Properties include the architecture, hypervisor type, hypervisor version (for Xen hypervisor type only), and virtual machine mode.

For example, an instance might require a host that runs an ARM-based processor, and QEMU as the hypervisor. You can decorate an image with these properties by using:

$ openstack image set --architecture arm --property hypervisor_type=qemu \
  img-uuid

The image properties that the filter checks for are:

architecture

describes the machine architecture required by the image. Examples are i686, x86_64, arm, and ppc64.

hypervisor_type

describes the hypervisor required by the image. Examples are xen, qemu, and xenapi.

Note

qemu is used for both QEMU and KVM hypervisor types.

hypervisor_version_requires

describes the hypervisor version required by the image. The property is supported for Xen hypervisor type only. It can be used to enable support for multiple hypervisor versions, and to prevent instances with newer Xen tools from being provisioned on an older version of a hypervisor. If available, the property value is compared to the hypervisor version of the compute host.

To filter the hosts by the hypervisor version, add the hypervisor_version_requires property on the image as metadata and pass an operator and a required hypervisor version as its value:

$ openstack image set --property hypervisor_type=xen --property \
  hypervisor_version_requires=">=4.3" img-uuid

vm_mode

describes the hypervisor application binary interface (ABI) required by the image. Examples are xen for Xen 3.0 paravirtual ABI, hvm for native ABI, uml for User Mode Linux paravirtual ABI, exe for container virt executable ABI.

IsolatedHostsFilter¶

Allows the admin to define a special (isolated) set of images and a special (isolated) set of hosts, such that the isolated images can only run on the isolated hosts, and the isolated hosts can only run isolated images. The flag restrict_isolated_hosts_to_isolated_images can be used to force isolated hosts to only run isolated images.

The logic within the filter depends on the restrict_isolated_hosts_to_isolated_images config option, which defaults to True. When True, a volume-backed instance will not be put on an isolated host. When False, a volume-backed instance can go on any host, isolated or not.

The admin must specify the isolated set of images and hosts in the nova.conf file using the isolated_hosts and isolated_images configuration options. For example:

[filter_scheduler]
isolated_hosts = server1, server2
isolated_images = 342b492c-128f-4a42-8d3a-c5088cf27d13, ebd267a6-ca86-4d6c-9a0e-bd132d6b7d09

IoOpsFilter¶

The IoOpsFilter filters hosts by concurrent I/O operations on it. Hosts with too many concurrent I/O operations will be filtered out. The max_io_ops_per_host option specifies the maximum number of I/O intensive instances allowed to run on a host. A host will be ignored by the scheduler if more than max_io_ops_per_host instances in build, resize, snapshot, migrate, rescue or unshelve task states are running on it.

JsonFilter¶

The JsonFilter allows a user to construct a custom filter by passing a scheduler hint in JSON format. The following operators are supported:

• =
• <
• >
• in
• <=
• >=
• not
• or
• and

The filter supports the following variables:

• $free_ram_mb
• $free_disk_mb
• $total_usable_ram_mb
• $vcpus_total
• $vcpus_used

Using the openstack server create command, use the --hint flag:

$ openstack server create --image 827d564a-e636-4fc4-a376-d36f7ebe1747 \
  --flavor 1 --hint query='[">=","$free_ram_mb",1024]' server1

With the API, use the os:scheduler_hints key:

{
    "server": {
        "name": "server-1",
        "imageRef": "cedef40a-ed67-4d10-800e-17455edce175",
        "flavorRef": "1"
    },
    "os:scheduler_hints": {
        "query": "[>=,$free_ram_mb,1024]"
    }
}

MetricsFilter¶

Filters hosts based on meters weight_setting. Only hosts with the available meters are passed so that the metrics weigher will not fail due to these hosts.

NUMATopologyFilter¶

Filters hosts based on the NUMA topology that was specified for the instance through the use of flavor extra_specs in combination with the image properties, as described in detail in the related nova-spec document. Filter will try to match the exact NUMA cells of the instance to those of the host. It will consider the standard over-subscription limits for each host NUMA cell, and provide limits to the compute host accordingly.

NumInstancesFilter¶

Hosts that have more instances running than specified by the max_instances_per_host option are filtered out when this filter is in place.

PciPassthroughFilter¶

The filter schedules instances on a host if the host has devices that meet the device requests in the extra_specs attribute for the flavor.

RamFilter¶

Only schedules instances on hosts that have sufficient RAM available. If this filter is not set, the scheduler may over provision a host based on RAM (for example, the RAM allocated by virtual machine instances may exceed the physical RAM).

You can configure this filter to enable a fixed amount of RAM overcommitment by using the ram_allocation_ratio configuration option in nova.conf. The default setting is:

ram_allocation_ratio = 1.5

This setting enables 1.5 GB instances to run on any compute node with 1 GB of free RAM.

RetryFilter¶

Filters out hosts that have already been attempted for scheduling purposes. If the scheduler selects a host to respond to a service request, and the host fails to respond to the request, this filter prevents the scheduler from retrying that host for the service request.

This filter is only useful if the scheduler_max_attempts configuration option is set to a value greater than zero.

SameHostFilter¶

Schedules the instance on the same host as another instance in a set of instances. To take advantage of this filter, the requester must pass a scheduler hint, using same_host as the key and a list of instance UUIDs as the value. This filter is the opposite of the DifferentHostFilter. Using the openstack server create command, use the --hint flag:

$ openstack server create --image cedef40a-ed67-4d10-800e-17455edce175 \
  --flavor 1 --hint same_host=a0cf03a5-d921-4877-bb5c-86d26cf818e1 \
  --hint same_host=8c19174f-4220-44f0-824a-cd1eeef10287 server-1

With the API, use the os:scheduler_hints key:

{
    "server": {
        "name": "server-1",
        "imageRef": "cedef40a-ed67-4d10-800e-17455edce175",
        "flavorRef": "1"
    },
    "os:scheduler_hints": {
        "same_host": [
            "a0cf03a5-d921-4877-bb5c-86d26cf818e1",
            "8c19174f-4220-44f0-824a-cd1eeef10287"
        ]
    }
}

ServerGroupAffinityFilter¶

The ServerGroupAffinityFilter ensures that an instance is scheduled on to a host from a set of group hosts. To take advantage of this filter, the requester must create a server group with an affinity policy, and pass a scheduler hint, using group as the key and the server group UUID as the value. Using the openstack server create command, use the --hint flag. For example:

$ openstack server group create --policy affinity group-1
$ openstack server create --image IMAGE_ID --flavor 1 \
  --hint group=SERVER_GROUP_UUID server-1

ServerGroupAntiAffinityFilter¶

The ServerGroupAntiAffinityFilter ensures that each instance in a group is on a different host. To take advantage of this filter, the requester must create a server group with an anti-affinity policy, and pass a scheduler hint, using group as the key and the server group UUID as the value. Using the openstack server create command, use the --hint flag. For example:

$ openstack server group create --policy anti-affinity group-1
$ openstack server create --image IMAGE_ID --flavor 1 \
  --hint group=SERVER_GROUP_UUID server-1

SimpleCIDRAffinityFilter¶

Schedules the instance based on host IP subnet range. To take advantage of this filter, the requester must specify a range of valid IP address in CIDR format, by passing two scheduler hints:

build_near_host_ip

The first IP address in the subnet (for example, 192.168.1.1)

cidr

The CIDR that corresponds to the subnet (for example, /24)

Using the openstack server create command, use the --hint flag. For example, to specify the IP subnet 192.168.1.1/24:

$ openstack server create --image cedef40a-ed67-4d10-800e-17455edce175 \
  --flavor 1 --hint build_near_host_ip=192.168.1.1 --hint cidr=/24 server-1

With the API, use the os:scheduler_hints key:

{
    "server": {
        "name": "server-1",
        "imageRef": "cedef40a-ed67-4d10-800e-17455edce175",
        "flavorRef": "1"
    },
    "os:scheduler_hints": {
        "build_near_host_ip": "192.168.1.1",
        "cidr": "24"
    }
}

Command-line interface¶

The nova command-line client supports the following aggregate-related commands.

nova aggregate-list

Print a list of all aggregates.

nova aggregate-create <name> [<availability-zone>]

Create a new aggregate named <name>, and optionally in availability zone [<availability-zone>] if specified. The command returns the ID of the newly created aggregate. Hosts can be made available to multiple host aggregates. Be careful when adding a host to an additional host aggregate when the host is also in an availability zone. Pay attention when using the nova aggregate-set-metadata and nova aggregate-update commands to avoid user confusion when they boot instances in different availability zones. An error occurs if you cannot add a particular host to an aggregate zone for which it is not intended.

nova aggregate-delete <aggregate>

Delete an aggregate with its <id> or <name>.

nova aggregate-show <aggregate>

Show details of the aggregate with its <id> or <name>.

nova aggregate-add-host <aggregate> <host>

Add host with name <host> to aggregate with its <id> or <name>.

nova aggregate-remove-host <aggregate> <host>

Remove the host with name <host> from the aggregate with its <id> or <name>.

nova aggregate-set-metadata <aggregate> <key=value> [<key=value> …]

Add or update metadata (key-value pairs) associated with the aggregate with its <id> or <name>.

nova aggregate-update [–name <name>] [–availability-zone <availability-zone>] <aggregate>

Update the name and/or availability zone for the aggregate.

nova host-list

List all hosts by service. It has been deprecated since microversion 2.43. Use nova hypervisor-list instead.

nova hypervisor-list [–matching <hostname>] [–marker <marker>] [–limit <limit>]

List hypervisors.

nova host-update [–status <enable|disable>] [–maintenance <enable|disable>] <hostname>

Put/resume host into/from maintenance. It has been deprecated since microversion 2.43. To enable or disable a service, use nova service-enable or nova service-disable instead.

nova service-enable <id>

Enable the service.

nova service-disable [–reason <reason>] <id>

Disable the service.

ERROR: Policy doesn't allow compute_extension:aggregates to be performed. (HTTP 403) (Request-ID: req-299fbff6-6729-4cef-93b2-e7e1f96b4864)

ERROR: Policy doesn't allow compute_extension:hosts to be performed. (HTTP 403) (Request-ID: req-ef2400f6-6776-4ea3-b6f1-7704085c27d1)

Configure scheduler to support host aggregates¶

One common use case for host aggregates is when you want to support scheduling instances to a subset of compute hosts because they have a specific capability. For example, you may want to allow users to request compute hosts that have SSD drives if they need access to faster disk I/O, or access to compute hosts that have GPU cards to take advantage of GPU-accelerated code.

To configure the scheduler to support host aggregates, the filter_scheduler.enabled_filters configuration option must contain the AggregateInstanceExtraSpecsFilter in addition to the other filters used by the scheduler. Add the following line to /etc/nova/nova.conf on the host that runs the nova-scheduler service to enable host aggregates filtering, as well as the other filters that are typically enabled:

[filter_scheduler]
enabled_filters=AggregateInstanceExtraSpecsFilter,RetryFilter,AvailabilityZoneFilter,ComputeCapabilitiesFilter,ImagePropertiesFilter,ServerGroupAntiAffinityFilter,ServerGroupAffinityFilter

Example: Specify compute hosts with SSDs¶

This example configures the Compute service to enable users to request nodes that have solid-state drives (SSDs). You create a fast-io host aggregate in the nova availability zone and you add the ssd=true key-value pair to the aggregate. Then, you add the node1, and node2 compute nodes to it.

$ openstack aggregate create --zone nova fast-io
+-------------------+----------------------------+
| Field             | Value                      |
+-------------------+----------------------------+
| availability_zone | nova                       |
| created_at        | 2016-12-22T07:31:13.013466 |
| deleted           | False                      |
| deleted_at        | None                       |
| id                | 1                          |
| name              | fast-io                    |
| updated_at        | None                       |
+-------------------+----------------------------+

$ openstack aggregate set --property ssd=true 1
+-------------------+----------------------------+
| Field             | Value                      |
+-------------------+----------------------------+
| availability_zone | nova                       |
| created_at        | 2016-12-22T07:31:13.000000 |
| deleted           | False                      |
| deleted_at        | None                       |
| hosts             | []                         |
| id                | 1                          |
| name              | fast-io                    |
| properties        | ssd='true'                 |
| updated_at        | None                       |
+-------------------+----------------------------+

$ openstack aggregate add host 1 node1
+-------------------+--------------------------------------------------+
| Field             | Value                                            |
+-------------------+--------------------------------------------------+
| availability_zone | nova                                             |
| created_at        | 2016-12-22T07:31:13.000000                       |
| deleted           | False                                            |
| deleted_at        | None                                             |
| hosts             | [u'node1']                                       |
| id                | 1                                                |
| metadata          | {u'ssd': u'true', u'availability_zone': u'nova'} |
| name              | fast-io                                          |
| updated_at        | None                                             |
+-------------------+--------------------------------------------------+

$ openstack aggregate add host 1 node2
+-------------------+--------------------------------------------------+
| Field             | Value                                            |
+-------------------+--------------------------------------------------+
| availability_zone | nova                                             |
| created_at        | 2016-12-22T07:31:13.000000                       |
| deleted           | False                                            |
| deleted_at        | None                                             |
| hosts             | [u'node1', u'node2']                             |
| id                | 1                                                |
| metadata          | {u'ssd': u'true', u'availability_zone': u'nova'} |
| name              | fast-io                                          |
| updated_at        | None                                             |
+-------------------+--------------------------------------------------+

Use the openstack flavor create command to create the ssd.large flavor called with an ID of 6, 8 GB of RAM, 80 GB root disk, and 4 vCPUs.

$ openstack flavor create --id 6 --ram 8192 --disk 80 --vcpus 4 ssd.large
+----------------------------+-----------+
| Field                      | Value     |
+----------------------------+-----------+
| OS-FLV-DISABLED:disabled   | False     |
| OS-FLV-EXT-DATA:ephemeral  | 0         |
| disk                       | 80        |
| id                         | 6         |
| name                       | ssd.large |
| os-flavor-access:is_public | True      |
| ram                        | 8192      |
| rxtx_factor                | 1.0       |
| swap                       |           |
| vcpus                      | 4         |
+----------------------------+-----------+

Once the flavor is created, specify one or more key-value pairs that match the key-value pairs on the host aggregates with scope aggregate_instance_extra_specs. In this case, that is the aggregate_instance_extra_specs:ssd=true key-value pair. Setting a key-value pair on a flavor is done using the openstack flavor set command.

$ openstack flavor set --property aggregate_instance_extra_specs:ssd=true ssd.large

Once it is set, you should see the extra_specs property of the ssd.large flavor populated with a key of ssd and a corresponding value of true.

$ openstack flavor show ssd.large
+----------------------------+-------------------------------------------+
| Field                      | Value                                     |
+----------------------------+-------------------------------------------+
| OS-FLV-DISABLED:disabled   | False                                     |
| OS-FLV-EXT-DATA:ephemeral  | 0                                         |
| disk                       | 80                                        |
| id                         | 6                                         |
| name                       | ssd.large                                 |
| os-flavor-access:is_public | True                                      |
| properties                 | aggregate_instance_extra_specs:ssd='true' |
| ram                        | 8192                                      |
| rxtx_factor                | 1.0                                       |
| swap                       |                                           |
| vcpus                      | 4                                         |
+----------------------------+-------------------------------------------+

Now, when a user requests an instance with the ssd.large flavor, the scheduler only considers hosts with the ssd=true key-value pair. In this example, these are node1 and node2.

Aggregates in Placement¶

Aggregates also exist in placement and are not the same thing as host aggregates in nova. These aggregates are defined (purely) as groupings of related resource providers. Since compute nodes in nova are represented in placement as resource providers, they can be added to a placement aggregate as well. For example, get the uuid of the compute node using openstack hypervisor list and add it to an aggregate in placement using openstack resource provider aggregate set.

$ openstack --os-compute-api-version=2.53 hypervisor list
+--------------------------------------+---------------------+-----------------+-----------------+-------+
| ID                                   | Hypervisor Hostname | Hypervisor Type | Host IP         | State |
+--------------------------------------+---------------------+-----------------+-----------------+-------+
| 815a5634-86fb-4e1e-8824-8a631fee3e06 | node1               | QEMU            | 192.168.1.123   | up    |
+--------------------------------------+---------------------+-----------------+-----------------+-------+

$ openstack --os-placement-api-version=1.2 resource provider aggregate set --aggregate df4c74f3-d2c4-4991-b461-f1a678e1d161 815a5634-86fb-4e1e-8824-8a631fee3e06

Some scheduling filter operations can be performed by placement for increased speed and efficiency.

Tenant Isolation with Placement¶

In order to use placement to isolate tenants, there must be placement aggregates that match the membership and UUID of nova host aggregates that you want to use for isolation. The same key pattern in aggregate metadata used by the AggregateMultiTenancyIsolation filter controls this function, and is enabled by setting [scheduler]/limit_tenants_to_placement_aggregate=True.

$ openstack --os-compute-api-version=2.53 aggregate create myagg
+-------------------+--------------------------------------+
| Field             | Value                                |
+-------------------+--------------------------------------+
| availability_zone | None                                 |
| created_at        | 2018-03-29T16:22:23.175884           |
| deleted           | False                                |
| deleted_at        | None                                 |
| id                | 4                                    |
| name              | myagg                                |
| updated_at        | None                                 |
| uuid              | 019e2189-31b3-49e1-aff2-b220ebd91c24 |
+-------------------+--------------------------------------+

$ openstack --os-compute-api-version=2.53 aggregate add host myagg node1
+-------------------+--------------------------------------+
| Field             | Value                                |
+-------------------+--------------------------------------+
| availability_zone | None                                 |
| created_at        | 2018-03-29T16:22:23.175884           |
| deleted           | False                                |
| deleted_at        | None                                 |
| hosts             | [u'node1']                           |
| id                | 4                                    |
| name              | myagg                                |
| updated_at        | None                                 |
| uuid              | 019e2189-31b3-49e1-aff2-b220ebd91c24 |
+-------------------+--------------------------------------+

$ openstack project list -f value | grep 'demo'
9691591f913949818a514f95286a6b90 demo

$ openstack aggregate set --property filter_tenant_id=9691591f913949818a514f95286a6b90 myagg

$ openstack --os-placement-api-version=1.2 resource provider aggregate set --aggregate 019e2189-31b3-49e1-aff2-b220ebd91c24 815a5634-86fb-4e1e-8824-8a631fee3e06

Note that the filter_tenant_id metadata key can be optionally suffixed with any string for multiple tenants, such as filter_tenant_id3=$tenantid.

Availability Zones with Placement¶

In order to use placement to honor availability zone requests, there must be placement aggregates that match the membership and UUID of nova host aggregates that you assign as availability zones. The same key in aggregate metadata used by the AvailabilityZoneFilter filter controls this function, and is enabled by setting [scheduler]/query_placement_for_availability_zone=True.

$ openstack --os-compute-api-version=2.53 aggregate create myaz
+-------------------+--------------------------------------+
| Field             | Value                                |
+-------------------+--------------------------------------+
| availability_zone | None                                 |
| created_at        | 2018-03-29T16:22:23.175884           |
| deleted           | False                                |
| deleted_at        | None                                 |
| id                | 4                                    |
| name              | myaz                                 |
| updated_at        | None                                 |
| uuid              | 019e2189-31b3-49e1-aff2-b220ebd91c24 |
+-------------------+--------------------------------------+

$ openstack --os-compute-api-version=2.53 aggregate add host myaz node1
+-------------------+--------------------------------------+
| Field             | Value                                |
+-------------------+--------------------------------------+
| availability_zone | None                                 |
| created_at        | 2018-03-29T16:22:23.175884           |
| deleted           | False                                |
| deleted_at        | None                                 |
| hosts             | [u'node1']                           |
| id                | 4                                    |
| name              | myagg                                |
| updated_at        | None                                 |
| uuid              | 019e2189-31b3-49e1-aff2-b220ebd91c24 |
+-------------------+--------------------------------------+

$ openstack aggregate set --property availability_zone=az002 myaz

$ openstack --os-placement-api-version=1.2 resource provider aggregate set --aggregate 019e2189-31b3-49e1-aff2-b220ebd91c24 815a5634-86fb-4e1e-8824-8a631fee3e06

With the above configuration, the AvailabilityZoneFilter filter can be disabled in [filter_scheduler]/enabled_filters while retaining proper behavior (and doing so with the higher performance of placement’s implementation).

XenServer hypervisor pools to support live migration¶

When using the XenAPI-based hypervisor, the Compute service uses host aggregates to manage XenServer Resource pools, which are used in supporting live migration.

Scheduling considerations¶

The allocation ratio configuration is used both during reporting of compute node resource provider inventory to the placement service and during scheduling.

The (deprecated) CoreFilter, DiskFilter and RamFilter filters will use the allocation ratio from the compute node directly when calculating available capacity on a given node during scheduling.

Usage scenarios¶

Since allocation ratios can be set via nova configuration, host aggregate metadata and the placement API, it can be confusing to know which should be used. This really depends on your scenario. A few common scenarios are detailed here.

1. When the deployer wants to always set an override value for a resource on a compute node, the deployer would ensure that the [DEFAULT]/cpu_allocation_ratio, [DEFAULT]/ram_allocation_ratio and [DEFAULT]/disk_allocation_ratio configuration options are set to a non-None value (or greater than 0.0 before the 19.0.0 Stein release). This will make the nova-compute service overwrite any externally-set allocation ratio values set via the placement REST API.

2. When the deployer wants to set an initial value for a compute node allocation ratio but wants to allow an admin to adjust this afterwards without making any configuration file changes, the deployer would set the [DEFAULT]/initial_cpu_allocation_ratio, [DEFAULT]/initial_ram_allocation_ratio and [DEFAULT]/initial_disk_allocation_ratio configuration options and then manage the allocation ratios using the placement REST API (or osc-placement command line interface). For example:

   $ openstack resource provider inventory set --resource VCPU:allocation_ratio=1.0 815a5634-86fb-4e1e-8824-8a631fee3e06
   

   Note the bug 1804125 restriction.

3. When the deployer wants to always use the placement API to set allocation ratios, then the deployer should ensure that [DEFAULT]/xxx_allocation_ratio options are all set to None (the default since 19.0.0 Stein, 0.0 before Stein) and then manage the allocation ratios using the placement REST API (or osc-placement command line interface).

   This scenario is the workaround for bug 1804125.

Command-line interface¶

The nova-manage command-line client supports the cell-disable related commands. To enable or disable a cell, use nova-manage cell_v2 update_cell and to create pre-disabled cells, use nova-manage cell_v2 create_cell. See the Nova Cells v2 man page for details on command usage.