6.4.3. OpenStack control plane performance report for 400 nodes

Abstract

This document includes OpenStack control plane performance test results for 379 nodes in fact (several more nodes were not able to be added to the cluster due to deployment issues). All tests have been performed regarding OpenStack control plane performance test plan

6.4.3.1. Environment description

Environment contains 4 types of servers:

  • rally node

  • controller node

  • compute-osd node

  • compute node

Amount of servers each role

Role

Servers count

Type

rally

1

1 or 2

controller

3

1 or 2

compute

291

1 or 2

compute-osd

34

3

compute-osd

49

1

6.4.3.1.1. Hardware configuration of each server

All servers have 3 types of configuration describing in table below

Description of servers hardware type 1

server

vendor,model

Dell PowerEdge R630

CPU

vendor,model

Intel,E5-2680 v3

processor_count

2

core_count

12

frequency_MHz

2500

RAM

vendor,model

Samsung, M393A2G40DB0-CPB

amount_MB

262144

NETWORK

interface_name s

eno1, eno2

vendor,model

Intel,X710 Dual Port

bandwidth

10G

interface_names

enp3s0f0, enp3s0f1

vendor,model

Intel,X710 Dual Port

bandwidth

10G

STORAGE

dev_name

/dev/sda

vendor,model
raid1 - Dell, PERC H730P Mini
2 disks Intel S3610

SSD/HDD

SSD

size

3,6TB

6.4.3.1.2. Network configuration of each server

All servers have same network configuration:

Network Scheme of the environment

6.4.3.1.3. Software configuration on servers with controller, compute and compute-osd roles

Services on servers by role

Role

Service name

controller
horizon
keystone
nova-api
nava-scheduler
nova-cert
nova-conductor
nova-consoleauth
nova-consoleproxy
cinder-api
cinder-backup
cinder-scheduler
cinder-volume
glance-api
glance-glare
glance-registry
neutron-dhcp-agent
neutron-l3-agent
neutron-metadata-agent
neutron-openvswitch-agent
neutron-server
heat-api
heat-api-cfn
heat-api-cloudwatch
ceph-mon
rados-gw
memcached
rabbitmq_server
mysqld
galera
corosync
pacemaker
haproxy

compute
nova-compute
neutron-l3-agent
neutron-metadata-agent
neutron-openvswitch-agent

compute-osd
nova-compute
neutron-l3-agent
neutron-metadata-agent
neutron-openvswitch-agent
ceph-osd
Software version on servers with controller, compute and compute-osd roles

Software

Version

OpenStack

Mitaka

Ceph

Hammer

Ubuntu

Ubuntu 14.04.3 LTS

You can find outputs of some commands and /etc folder in the following archives:

controller-1.tar.gz
controller-2.tar.gz
controller-3.tar.gz
compute-1.tar.gz
compute-osd-1.tar.gz

6.4.3.1.4. Software configuration on servers with rally role

On this server should be installed Rally. How to do it you can find in Rally installation documentation

Software version on server with rally role

Software

Version

Rally

0.5.0

Ubuntu

Ubuntu 14.04.3 LTS

6.4.3.2. Testing process

Some test parameters

Name

Value

Volume size to create in Cinder

1GB

Flavor to create VM from

m1.tiny

Image name to create VM from

cirros

  1. Create work directory on server with Rally role. In future we will call that directory as WORK_DIR

  2. Create directory “plugins” in WORK_DIR and copy to that directory nova_performance.py plugin.

  3. Create directory “scenarios” in WORK_DIR and copy to that directory boot_attach_live_migrate_and_delete_server_with_secgroups.json, create-and-delete-image.json and keystone.json scenarios.

  4. Perform tests:

    rally  --plugin-paths /home/rally/plugins task start \
keystone.json \
--task-args '{"gre_enabled": true, "compute": 375, "concurrency": 5}' \
--out=keystone.html

rally  --plugin-paths /home/rally/plugins task start \
boot_attach_live_migrate_and_delete_server_with_secgroups.json \
--task-args '{"gre_enabled": true, "compute": 375, "concurrency": 5}' \
--out=boot_attach_live_migrate_and_delete_server_with_secgroups.html

rally  --plugin-paths /home/rally/plugins task start \
create-and-delete-image.json \
--task-args '{"gre_enabled": true, "compute": 375, "concurrency": 5, "http_server_with_glance_images": "1.2.3.4"}' \
--out=create-and-delete-image.html

As a result of this part we got the following HTML file:

keystone.html
boot_attach_live_migrate_and_delete_server_with_secgroups.html
create-and-delete-image.html

6.4.3.3. Test results

All values in tables below are in seconds.

6.4.3.3.1. Cinder

Operation

Mean (sec)

90%ile (sec)

50%ile (sec)

Max (sec)

Min (sec)

create_volume

2.619

2.754

2.593

2.988

2.453

delete_volume

2.339

2.449

2.323

2.666

2.200

6.4.3.3.2. Glance

Iterations: 400 Failures: 17

Operation

Median (sec)

90%ile (sec)

95%ile (sec)

Max (sec)

Min (sec)

create_image

60.573

88.944

103.642

125.492

40.852

delete_image

6.428

9.804

12.221

28.287

1.072

6.4.3.3.3. Keystone

Operation

Median (sec)

90%ile (sec)

95%ile (sec)

Max (sec)

Min (sec)

authenticate

0.195

0.22

0.231

0.988

0.086

6.4.3.3.4. Neutron

Operation

Median (sec)

90%ile (sec)

95%ile (sec)

Max (sec)

Min (sec)

create_20_rules

5.957

6.347

6.467

8.227

5.202

create_2_security_groups

0.546

0.639

0.664

0.887

0.41

delete_2_security_groups

0.472

0.59

0.624

0.887

0.297

6.4.3.3.5. Nova

Operation

Median (sec)

90%ile (sec)

95%ile (sec)

Max (sec)

Min (sec)

attach_volume

3.063

3.256

3.304

5.349

2.726

boot_server

12.648

13.132

13.328

15.054

11.071

delete_server

2.622

2.862

3

5.078

2.446

detach_volume

2.901

3.067

3.13

3.319

2.629

find_host_to_migrate

0.807

1.021

1.069

1.385

0.655

live_migrate

16.317

18.477

18.689

31.738

14.371