5. Test Plans

• 5.1. 1000 Compute nodes resource consumption/scalability testing
  • 5.1.1. Test Plan
    • 5.1.1.1. Test Environment
      • 5.1.1.1.1. Preparation
      • 5.1.1.1.2. Environment description
        • 5.1.1.1.2.1. Hardware
        • 5.1.1.1.2.2. Network
        • 5.1.1.1.2.3. Software
        • 5.1.1.1.2.4. Configuration
    • 5.1.1.2. Test Case 1: Resources consumption under severe load
      • 5.1.1.2.1. Description
      • 5.1.1.2.2. Parameters
      • 5.1.1.2.3. List of performance metrics
  • 5.1.2. Reports
• 5.2. Measuring performance of container repositories
  • 5.2.1. List of performance metrics
  • 5.2.2. Test Plan
    • 5.2.2.1. Test Environment
      • 5.2.2.1.1. Preparation
      • 5.2.2.1.2. Environment description
    • 5.2.2.2. Test Case #1: Uploading to a docker repository.
      • 5.2.2.2.1. Description
      • 5.2.2.2.2. List of performance metrics
      • 5.2.2.2.3. Measuring PUSH_TIME(CONCURRENCY) values
    • 5.2.2.3. Test Case #2: Downloading from a docker repository.
      • 5.2.2.3.1. Description
      • 5.2.2.3.2. List of performance metrics
      • 5.2.2.3.3. Measuring PULL_TIME(CONCURRENCY) values
  • 5.2.3. Applications
    • 5.2.3.1. list of container repositories
    • 5.2.3.2. Script for collecting performance metrics
    • 5.2.3.3. Proposed docker file
  • 5.2.4. Reports
• 5.3. OpenStack control plane performance test plan
  • 5.3.1. Test Plan
    • 5.3.1.1. Test Environment
      • 5.3.1.1.1. Preparation
      • 5.3.1.1.2. Environment description
        • 5.3.1.1.2.1. Hardware
        • 5.3.1.1.2.2. Network
        • 5.3.1.1.2.3. Software
    • 5.3.1.2. Test tool
    • 5.3.1.3. Test Case 1: Boot, attach, migrate and delete server with security groups
      • 5.3.1.3.1. Description
      • 5.3.1.3.2. Parameters
      • 5.3.1.3.3. List of performance metrics
      • 5.3.1.3.4. Measuring performance values
    • 5.3.1.4. Cinder
    • 5.3.1.5. Neutron
    • 5.3.1.6. Nova
      • 5.3.1.6.1. Example of Rally scenario configuration
    • 5.3.1.7. Test Case 2: Create and delete image
      • 5.3.1.7.1. Description
      • 5.3.1.7.2. Parameters
      • 5.3.1.7.3. List of performance metrics
      • 5.3.1.7.4. Measuring performance values
    • 5.3.1.8. Glance
      • 5.3.1.8.1. Example of Rally scenario configuration
    • 5.3.1.9. Test case 3: Keystone authentication
      • 5.3.1.9.1. Description
      • 5.3.1.9.2. Parameters
      • 5.3.1.9.3. Measuring performance values
    • 5.3.1.10. Keystone
      • 5.3.1.10.1. List of performance metrics
      • 5.3.1.10.2. Example of Rally scenario configuration
  • 5.3.2. Reports
• 5.4. OpenStack control plane density testing
  • 5.4.1. Test Plan
    • 5.4.1.1. Test Environment
      • 5.4.1.1.1. Preparation
      • 5.4.1.1.2. Environment description
    • 5.4.1.2. Hardware configuration of each server
    • 5.4.1.3. Test Case 1: perform baseline Rally scenarios
      • 5.4.1.3.1. Description
      • 5.4.1.3.2. List of performance metrics
  • 5.4.2. Reports
• 5.5. SQL Database Test Plan
  • 5.5.1. Test Plan
    • 5.5.1.1. Test Environment
      • 5.5.1.1.1. Preparation
      • 5.5.1.1.2. Environment description
    • 5.5.1.2. Test Case 1: sysbench
      • 5.5.1.2.1. Description
      • 5.5.1.2.2. Parameters
      • 5.5.1.2.3. List of performance metrics
    • 5.5.1.3. Test Case 2: Database Testing Tool
      • 5.5.1.3.1. Description
      • 5.5.1.3.2. Parameters
      • 5.5.1.3.3. List of performance metrics
  • 5.5.2. Reports
• 5.6. Fuel Containerized Control Plane performance testing
  • 5.6.1. Test Plan
    • 5.6.1.1. Test Environment
      • 5.6.1.1.1. Preparation
      • 5.6.1.1.2. Environment description
        • 5.6.1.1.2.1. Hardware
        • 5.6.1.1.2.2. Network
        • 5.6.1.1.2.3. Software
    • 5.6.1.2. Test tool
    • 5.6.1.3. Test Case 1: Boot and delete server
      • 5.6.1.3.1. Description
      • 5.6.1.3.2. Parameters
      • 5.6.1.3.3. List of performance metrics
      • 5.6.1.3.4. Measuring performance values
    • 5.6.1.4. Nova
      • 5.6.1.4.1. Example of Rally scenario configuration
    • 5.6.1.5. Test Case 2: Boot and delete server with security groups
      • 5.6.1.5.1. Description
      • 5.6.1.5.2. Parameters
      • 5.6.1.5.3. List of performance metrics
      • 5.6.1.5.4. Measuring performance values
    • 5.6.1.6. Neutron
    • 5.6.1.7. Nova
      • 5.6.1.7.1. Example of Rally scenario configuration
    • 5.6.1.8. Test Case 3: Boot and list servers
      • 5.6.1.8.1. Description
      • 5.6.1.8.2. Parameters
      • 5.6.1.8.3. List of performance metrics
      • 5.6.1.8.4. Measuring performance values
    • 5.6.1.9. Nova
      • 5.6.1.9.1. Example of Rally scenario configuration
    • 5.6.1.10. Test Case 4: Create and delete image
      • 5.6.1.10.1. Description
      • 5.6.1.10.2. Parameters
      • 5.6.1.10.3. List of performance metrics
      • 5.6.1.10.4. Measuring performance values
    • 5.6.1.11. Glance
      • 5.6.1.11.1. Example of Rally scenario configuration
    • 5.6.1.12. Test Case 5: Create and list images
      • 5.6.1.12.1. Description
      • 5.6.1.12.2. Parameters
      • 5.6.1.12.3. List of performance metrics
      • 5.6.1.12.4. Measuring performance values
    • 5.6.1.13. Glance
      • 5.6.1.13.1. Example of Rally scenario configuration
    • 5.6.1.14. Test case 6: Keystone authentication
      • 5.6.1.14.1. Description
      • 5.6.1.14.2. Parameters
      • 5.6.1.14.3. Measuring performance values
    • 5.6.1.15. Keystone
      • 5.6.1.15.1. List of performance metrics
      • 5.6.1.15.2. Example of Rally scenario configuration
  • 5.6.2. Reports
• 5.7. Fuel Containerized Control Plane upgradability performance testing
  • 5.7.1. Test Plan
    • 5.7.1.1. Test Environment
      • 5.7.1.1.1. Preparation
      • 5.7.1.1.2. Environment description
        • 5.7.1.1.2.1. Hardware
        • 5.7.1.1.2.2. Network
        • 5.7.1.1.2.3. Software
    • 5.7.1.2. Test tool
    • 5.7.1.3. Test Case 1: Boot and delete server during Open Stack update
      • 5.7.1.3.1. Description
      • 5.7.1.3.2. Parameters
      • 5.7.1.3.3. List of performance metrics
      • 5.7.1.3.4. Measuring performance values
    • 5.7.1.4. Nova
      • 5.7.1.4.1. Example of Rally scenario configuration
• 5.8. Measuring performance of Cinder with Ceph backend
  • 5.8.1. Test Plan
    • 5.8.1.1. Test Environment
      • 5.8.1.1.1. Preparation
      • 5.8.1.1.2. Environment description
    • 5.8.1.2. Test Cases
      • 5.8.1.2.1. Description
      • 5.8.1.2.2. Parameters
      • 5.8.1.2.3. List of performance metrics
      • 5.8.1.2.4. Measuring performance values
  • 5.8.2. Applications
    • 5.8.2.1. Rally jobs templates:
    • 5.8.2.2. Heat templates
    • 5.8.2.3. Test script for heat
  • 5.8.3. Reports
• 5.9. Keystone Performance testing
  • 5.9.1. Test Plan
    • 5.9.1.1. Test Environment
      • 5.9.1.1.1. Preparation
      • 5.9.1.1.2. Environment description
        • 5.9.1.1.2.1. Hardware
        • 5.9.1.1.2.2. Network
        • 5.9.1.1.2.3. Software
    • 5.9.1.2. Test Case 1: Keystone DB / cache operations analysis
      • 5.9.1.2.1. Description
      • 5.9.1.2.2. Parameters
      • 5.9.1.2.3. List of performance metrics
    • 5.9.1.3. Test Case 2: Keystone DB / cache operations analysis (HA version)
      • 5.9.1.3.1. Description
      • 5.9.1.3.2. Parameters
      • 5.9.1.3.3. List of performance metrics
  • 5.9.2. Reports
• 5.10. Kubernetes density testing
  • 5.10.1. Test Plan
    • 5.10.1.1. Test Environment
      • 5.10.1.1.1. Preparation
      • 5.10.1.1.2. Environment description
    • 5.10.1.2. Test Case #1: Maximum pods per node
      • 5.10.1.2.1. Description
      • 5.10.1.2.2. List of performance metrics
    • 5.10.1.3. Test Case #2: Measure Kubelet capacity
      • 5.10.1.3.1. Description
      • 5.10.1.3.2. List of performance metrics
  • 5.10.2. Reports
• 5.11. Kubernetes DNS performance test plan
  • 5.11.1. Test Plan
    • 5.11.1.1. Test Environment
      • 5.11.1.1.1. Preparation
      • 5.11.1.1.2. Environment description
    • 5.11.1.2. Test Case #1: Performing DNS queries
      • 5.11.1.2.1. Description
      • 5.11.1.2.2. Parameters
      • 5.11.1.2.3. List of performance metrics
  • 5.11.2. Reports
• 5.12. Kubernetes POD recovery test plan
  • 5.12.1. Test Plan
    • 5.12.1.1. Test Environment
      • 5.12.1.1.1. Preparation
      • 5.12.1.1.2. Environment description
    • 5.12.1.2. Test Case #1: POD restart
      • 5.12.1.2.1. Description
      • 5.12.1.2.2. List of performance metrics
  • 5.12.2. Reports
• 5.13. Kubernetes proxy performance test plan
  • 5.13.1. Test Plan
    • 5.13.1.1. Test Environment
      • 5.13.1.1.1. Preparation
      • 5.13.1.1.2. Environment description
    • 5.13.1.2. Test Case #1: Performing kube-proxy
      • 5.13.1.2.1. Description
      • 5.13.1.2.2. Parameters
      • 5.13.1.2.3. List of performance metrics
  • 5.13.2. Reports
• 5.14. Massively Distributed RPCs
  • 5.14.1. Test Plan
    • 5.14.1.1. Test Environment
      • 5.14.1.1.1. Preparation
      • 5.14.1.1.2. Environment description
        • 5.14.1.1.2.1. Hardware
        • 5.14.1.1.2.2. Network
        • 5.14.1.1.2.3. Software
        • 5.14.1.1.2.4. Messaging middleware topology
        • 5.14.1.1.2.5. Openstack version
    • 5.14.1.2. Test Case 1 : One single large distributed target
      • 5.14.1.2.1. Description
      • 5.14.1.2.2. Methodology
      • 5.14.1.2.3. List of performance metrics
    • 5.14.1.3. Test Case 2: Multiple distributed targets
      • 5.14.1.3.1. Description
      • 5.14.1.3.2. Methodology
      • 5.14.1.3.3. List of performance metrics
    • 5.14.1.4. Test Case 3 : one single large distributed fanout
      • 5.14.1.4.1. Description
      • 5.14.1.4.2. Methodology
      • 5.14.1.4.3. List of performance metrics
    • 5.14.1.5. Test Case 4 : multiple distributed fanouts
      • 5.14.1.5.1. Description
      • 5.14.1.5.2. Methodology
      • 5.14.1.5.3. List of performance metrics
    • 5.14.1.6. Test Case 5 : Resilience
      • 5.14.1.6.1. Description
      • 5.14.1.6.2. Methodology
      • 5.14.1.6.3. List of performance metrics
    • 5.14.1.7. Common metrics to all test cases
    • 5.14.1.8. Test Case 6 : Operational testing
      • 5.14.1.8.1. Description
      • 5.14.1.8.2. List of performance metrics
  • 5.14.2. Reports
• 5.15. Message Queue Performance
  • 5.15.1. Test Plan
    • 5.15.1.1. Test Environment
      • 5.15.1.1.1. Preparation
      • 5.15.1.1.2. Environment description
        • 5.15.1.1.2.1. Hardware
        • 5.15.1.1.2.2. Network
        • 5.15.1.1.2.3. Software
    • 5.15.1.2. Test Case 1: RPC Call Throughput Test
      • 5.15.1.2.1. Description
      • 5.15.1.2.2. List of performance metrics
    • 5.15.1.3. Test Case 2: RPC Cast Throughput Test
      • 5.15.1.3.1. Description
      • 5.15.1.3.2. List of performance metrics
    • 5.15.1.4. Test Case 3: Notification Throughput Test
      • 5.15.1.4.1. Description
      • 5.15.1.4.2. List of performance metrics
  • 5.15.2. Tools
    • 5.15.2.1. Oslo.messaging Simulator
      • 5.15.2.1.1. Test environment preparation
      • 5.15.2.1.2. Test Case 1: RPC Call Throughput Test
      • 5.15.2.1.3. Test Case 2: RPC Cast Throughput Test
      • 5.15.2.1.4. Test Case 3: Notification Throughput Test
  • 5.15.3. Reports
• 5.16. Message Queue High Availability
  • 5.16.1. Test Plan
    • 5.16.1.1. Test Environment
      • 5.16.1.1.1. Preparation
      • 5.16.1.1.2. Environment description
        • 5.16.1.1.2.1. Hardware
        • 5.16.1.1.2.2. Network
        • 5.16.1.1.2.3. Software
    • 5.16.1.2. Test Case 1: Client and Server connected to Master, Master fails
      • 5.16.1.2.1. Description
      • 5.16.1.2.2. List of performance metrics
      • 5.16.1.2.3. Options
    • 5.16.1.3. Test Case 2: Client and Server connected to Slave 1, Slave 1 fails
      • 5.16.1.3.1. Description
      • 5.16.1.3.2. List of performance metrics
      • 5.16.1.3.3. Options
    • 5.16.1.4. Test Case 3: Client and Server on different slaves. Client Slave fails
      • 5.16.1.4.1. Description
      • 5.16.1.4.2. List of performance metrics
      • 5.16.1.4.3. Options
    • 5.16.1.5. Test Case 4: Client on Master and Server on Slave, Master fails
      • 5.16.1.5.1. Description
      • 5.16.1.5.2. List of performance metrics
      • 5.16.1.5.3. Options
  • 5.16.2. Tools
    • 5.16.2.1. Performa omsimulator
      • 5.16.2.1.1. Test environment preparation
      • 5.16.2.1.2. Execution
  • 5.16.3. Reports
• 5.17. OpenStack Networking (Neutron) control plane performance test plan
  • 5.17.1. Test Plan
    • 5.17.1.1. Test Environment
      • 5.17.1.1.1. Preparation
      • 5.17.1.1.2. Environment description
        • 5.17.1.1.2.1. Hardware
        • 5.17.1.1.2.2. Network
        • 5.17.1.1.2.3. Software
    • 5.17.1.2. Test tool
    • 5.17.1.3. Test Case 1: Basic Neutron test suite
      • 5.17.1.3.1. Description
      • 5.17.1.3.2. List of performance metrics
    • 5.17.1.4. Test Case 2: Stressful Neutron test suite
      • 5.17.1.4.1. Description
      • 5.17.1.4.2. List of performance metrics
    • 5.17.1.5. Test case 3: Neutron scalability test with many networks
      • 5.17.1.5.1. Description
      • 5.17.1.5.2. List of performance metrics
    • 5.17.1.6. Test case 4: Neutron scalability test with many servers
      • 5.17.1.6.1. List of performance metrics
  • 5.17.2. Reports
• 5.18. OpenStack Neutron Testing During Open Stack Upgrade Test Plan
  • 5.18.1. Test Plan
    • 5.18.1.1. Test Environment
      • 5.18.1.1.1. Preparation
      • 5.18.1.1.2. Environment description
        • 5.18.1.1.2.1. Hardware
        • 5.18.1.1.2.2. Network
        • 5.18.1.1.2.3. Software
    • 5.18.1.2. Test tool
    • 5.18.1.3. Test Case 1: Analysis of L2 metrics during Open Stack upgrade
      • 5.18.1.3.1. Description
      • 5.18.1.3.2. List of performance metrics
    • 5.18.1.4. Test Case 2: Analysis of L3 metrics during Open Stack upgrade
      • 5.18.1.4.1. Description
      • 5.18.1.4.2. List of performance metrics
  • 5.18.2. Reports
• 5.19. Neutron ML2/ODL Performance testing
  • 5.19.1. Test Plan
    • 5.19.1.1. Test Environment
      • 5.19.1.1.1. Preparation
      • 5.19.1.1.2. Environment description
        • 5.19.1.1.2.1. Hardware
        • 5.19.1.1.2.2. Software
        • 5.19.1.1.2.3. System Performance Monitoring
    • 5.19.1.2. Test Case 1
      • 5.19.1.2.1. Description
      • 5.19.1.2.2. List of performance metrics
    • 5.19.1.3. Test Case 2
      • 5.19.1.3.1. Description
      • 5.19.1.3.2. List of performance metrics
    • 5.19.1.4. Test Case 3
      • 5.19.1.4.1. Description
      • 5.19.1.4.2. List of performance metrics
    • 5.19.1.5. Tools
      • 5.19.1.5.1. Setup
      • 5.19.1.5.2. Analysis
• 5.20. OpenStack API Performance Metrics
  • 5.20.1. Test Plan
    • 5.20.1.1. Test Environment
      • 5.20.1.1.1. Preparation
      • 5.20.1.1.2. Environment description
        • 5.20.1.1.2.1. Hardware
        • 5.20.1.1.2.2. Network
        • 5.20.1.1.2.3. Software
    • 5.20.1.2. Test Case: Operation Performance Measurements
      • 5.20.1.2.1. Description
      • 5.20.1.2.2. Parameters
      • 5.20.1.2.3. List of performance metrics
  • 5.20.2. Tools
    • 5.20.2.1. Rally + Metrics2
      • 5.20.2.1.1. Rally atomic action measurement augmentation
      • 5.20.2.1.2. Scenarios
      • 5.20.2.1.3. Duration metrics
      • 5.20.2.1.4. Concurrency metrics
      • 5.20.2.1.5. Throughput metrics
      • 5.20.2.1.6. Scale impact metrics
• 5.21. OpenStack API worker performance and scalability testing with Browbeat
  • 5.21.1. Test Plan
    • 5.21.1.1. Test Environment
      • 5.21.1.1.1. Preparation
      • 5.21.1.1.2. Environment description
        • 5.21.1.1.2.1. Hardware
        • 5.21.1.1.2.2. Software
        • 5.21.1.1.2.3. Tuning/Configuration
        • 5.21.1.1.2.4. Test Diagram
    • 5.21.1.2. Test Case 1: Keystone
      • 5.21.1.2.1. Description
      • 5.21.1.2.2. Setup
      • 5.21.1.2.3. Analysis
      • 5.21.1.2.4. List of performance metrics
    • 5.21.1.3. Test Case 2: Neutron
      • 5.21.1.3.1. Description
      • 5.21.1.3.2. Setup
      • 5.21.1.3.3. Analysis
      • 5.21.1.3.4. List of performance metrics
• 5.22. OpenStack load testing
  • 5.22.1. Test Plan
    • 5.22.1.1. Test Environment
      • 5.22.1.1.1. Preparation
      • 5.22.1.1.2. Environment description
        • 5.22.1.1.2.1. Hardware
        • 5.22.1.1.2.2. Network
        • 5.22.1.1.2.3. Software
    • 5.22.1.2. Test Case 1: Neutron Load Testing
      • 5.22.1.2.1. Description
      • 5.22.1.2.2. List of performance metrics
  • 5.22.2. Tools
    • 5.22.2.1. Create Neutron networks
    • 5.22.2.2. Create Neutron network with 1 port
    • 5.22.2.3. Create Neutron ports
    • 5.22.2.4. Create Cinder volumes
    • 5.22.2.5. Boot Nova servers
  • 5.22.3. Reports
• 5.23. OpenStack in a WAN Context Testing
  • 5.23.1. Test Plan
    • 5.23.1.1. Test Environment
      • 5.23.1.1.1. Preparation
      • 5.23.1.1.2. Environment description
        • 5.23.1.1.2.1. Hardware
        • 5.23.1.1.2.2. Network
        • 5.23.1.1.2.3. Software
        • 5.23.1.1.2.4. Configuration
    • 5.23.1.2. Test Case 1: OpenStack control services behaviour under WAN
      • 5.23.1.2.1. Description
      • 5.23.1.2.2. Parameters
      • 5.23.1.2.3. Further parameters
      • 5.23.1.2.4. List of performance metrics
    • 5.23.1.3. Test Case 2: OpenStack data plane behaviour under WAN
      • 5.23.1.3.1. Description
      • 5.23.1.3.2. Parameters
      • 5.23.1.3.3. List of performance metrics
    • 5.23.1.4. Test tool
• 5.24. Measuring performance of provisioning systems
  • 5.24.1. Test Plan
    • 5.24.1.1. Test Environment
      • 5.24.1.1.1. Preparation
      • 5.24.1.1.2. Environment description
    • 5.24.1.2. Test Case
      • 5.24.1.2.1. Description
      • 5.24.1.2.2. Parameters
      • 5.24.1.2.3. List of performance metrics
      • 5.24.1.2.4. Measuring performance values
  • 5.24.2. Applications
    • 5.24.2.1. List of provisioning systems
  • 5.24.3. Full script for collecting performance metrics
  • 5.24.4. Reports
• 5.25. Telemetry Services resource consumption/scalability testing
  • 5.25.1. Test Plan
    • 5.25.1.1. Test Environment
      • 5.25.1.1.1. Preparation
      • 5.25.1.1.2. Environment description
        • 5.25.1.1.2.1. Hardware
        • 5.25.1.1.2.2. Software
        • 5.25.1.1.2.3. Tuning/Configuration
        • 5.25.1.1.2.4. System Performance Monitoring
        • 5.25.1.1.2.5. Test Diagram
    • 5.25.1.2. Test Case 1
      • 5.25.1.2.1. Description
      • 5.25.1.2.2. Setup
      • 5.25.1.2.3. Analysis
      • 5.25.1.2.4. List of performance metrics
  • 5.25.2. Reports
• 5.26. OpenStack tenant networking performance
  • 5.26.1. Test Plan
    • 5.26.1.1. Test Environment
      • 5.26.1.1.1. Preparation
      • 5.26.1.1.2. Environment description
        • 5.26.1.1.2.1. Hardware
        • 5.26.1.1.2.2. Network
        • 5.26.1.1.2.3. Software
    • 5.26.1.2. Test Case 1: single node L2 instance-to-instance performance
      • 5.26.1.2.1. Description
      • 5.26.1.2.2. List of performance metrics
    • 5.26.1.3. Test Case 2: L2 instance-to-instance performance
      • 5.26.1.3.1. Description
      • 5.26.1.3.2. List of performance metrics
    • 5.26.1.4. Test Case 3: L2 concurrent performance
      • 5.26.1.4.1. Description
      • 5.26.1.4.2. List of performance metrics
    • 5.26.1.5. Test Case 4: single node L3 east-west instance-to-instance performance
      • 5.26.1.5.1. Description
      • 5.26.1.5.2. List of performance metrics
    • 5.26.1.6. Test Case 5: L3 east-west instance-to-instance performance
      • 5.26.1.6.1. Description
      • 5.26.1.6.2. List of performance metrics
    • 5.26.1.7. Test Case 6: L3 east-west concurrent performance
      • 5.26.1.7.1. Description
      • 5.26.1.7.2. List of performance metrics
    • 5.26.1.8. Test Case 7: single node L3 north-south instance-to-instance performance
      • 5.26.1.8.1. Description
      • 5.26.1.8.2. List of performance metrics
    • 5.26.1.9. Test Case 8: L3 north-south instance-to-instance performance
      • 5.26.1.9.1. Description
      • 5.26.1.9.2. List of performance metrics
    • 5.26.1.10. Test Case 9: L3 north-south concurrent performance
      • 5.26.1.10.1. Description
      • 5.26.1.10.2. List of performance metrics
    • 5.26.1.11. Test Case 10: Neutron QoS testing
      • 5.26.1.11.1. Description
      • 5.26.1.11.2. List of performance metrics
  • 5.26.2. Tools
    • 5.26.2.1. Shaker
      • 5.26.2.1.1. Test environment preparation
      • 5.26.2.1.2. Test Case 1: L2 instance-to-instance performance
      • 5.26.2.1.3. Test Case 2: L2 concurrent performance
      • 5.26.2.1.4. Test Case 3: L3 east-west instance-to-instance performance
      • 5.26.2.1.5. Test Case 4: L3 east-west concurrent performance
      • 5.26.2.1.6. Test Case 5: L3 north-south instance-to-instance performance
      • 5.26.2.1.7. Test Case 6: L3 north_south concurrent performance
      • 5.26.2.1.8. Test Case 7: Neutron QoS testing
  • 5.26.3. Reports
• 5.27. Ceph RBD performance testing
  • 5.27.1. Test Plan
    • 5.27.1.1. Preparation
    • 5.27.1.2. Execution Strategy
    • 5.27.1.3. Test tool
    • 5.27.1.4. Test types
    • 5.27.1.5. Test Measurements and Metrics
    • 5.27.1.6. Expected Results and Pass/Fail Criteria
      • 5.27.1.6.1. Pass/Fail Criteria
      • 5.27.1.6.2. Expected results
    • 5.27.1.7. Load Description
    • 5.27.1.8. Test Environment
      • 5.27.1.8.1. Hardware
      • 5.27.1.8.2. Network
      • 5.27.1.8.3. Software
  • 5.27.2. Reports
• Container cluster systems
  • Abstract
  • Contents
    • 1. Measuring performance of Kargo
      • 1.1. Test Plan
        • 1.1.1. Test Environment
          • 1.1.1.1. Preparation
          • 1.1.1.2. Environment description
        • 1.1.2. Test Case
          • 1.1.2.1. Description
          • 1.1.2.2. Parameters
          • 1.1.2.3. List of performance metrics
          • 1.1.2.4. Measuring performance values
      • 1.2. Reports
    • 2. Measuring performance of container cluster systems
      • 2.1. Test Plan
        • 2.1.1. Test Environment
          • 2.1.1.1. Preparation
          • 2.1.1.2. Environment description
        • 2.1.2. Test Case #1: A new application creation time
          • 2.1.2.1. Description
          • 2.1.2.2. List of performance metrics
          • 2.1.2.3. Measuring values
        • 2.1.3. Test Case #2: A application CPU update time
          • 2.1.3.1. Description
          • 2.1.3.2. List of performance metrics
          • 2.1.3.3. Measuring values
        • 2.1.4. Test Case #3: A application memory update time
          • 2.1.4.1. Description
          • 2.1.4.2. List of performance metrics
          • 2.1.4.3. Measuring values
        • 2.1.5. Test Case #4: A application disk update time
          • 2.1.5.1. Description
          • 2.1.5.2. List of performance metrics
          • 2.1.5.3. Measuring values
        • 2.1.6. Test Case #5: A application instances update time
          • 2.1.6.1. Description
          • 2.1.6.2. List of performance metrics
          • 2.1.6.3. Measuring values
        • 2.1.7. Test Case #6: A application restart time
          • 2.1.7.1. Description
          • 2.1.7.2. List of performance metrics
          • 2.1.7.3. Measuring values
        • 2.1.8. Test Case #7: A application delete time
          • 2.1.8.1. Description
          • 2.1.8.2. List of performance metrics
          • 2.1.8.3. Measuring values
        • 2.1.9. Test Case #8: Health check Performance
          • 2.1.9.1. Description
          • 2.1.9.2. List of performance metrics
          • 2.1.9.3. Measuring REAL_INTERVAL(TASK_COUNT) values
      • 2.2. Applications
        • 2.2.1. list of container platforms
      • 2.3. Reports
    • 3. Measuring of API performance of container cluster systems
      • 3.1. Test Plan
        • 3.1.1. Test Environment
          • 3.1.1.1. Preparation
          • 3.1.1.2. Environment description
        • 3.1.2. Test Case #1: API latencies of requests which make actions on containers
          • 3.1.2.1. Description
          • 3.1.2.2. List of performance metrics
          • 3.1.2.3. Measuring values of API latencies of requests which make actions on containers
        • 3.1.3. Test Case #1.5: Container creation startup latency extended measurement
          • 3.1.3.1. Description
          • 3.1.3.2. List of performance metrics
        • 3.1.4. Test Case #2: API latencies of any other supported requests
          • 3.1.4.1. Description
          • 3.1.4.2. List of performance metrics
          • 3.1.4.3. Measuring values of API latencies of any other supported requests
      • 3.2. Reports
    • 4. ETCD health research test plan
      • 4.1. Test Plan
        • 4.1.1. Test Environment
          • 4.1.1.1. Preparation
          • 4.1.1.2. Environment description
        • 4.1.2. Test Cases
          • 4.1.2.1. Description
          • 4.1.2.2. List of performance metrics
          • 4.1.2.3. Collecting metrics
          • 4.1.2.4. Plotting ‘K8S vs ETCD dependency’
      • 4.2. Reports
• 5.28. Hardware features test plans
  • 5.28.1. Hardware Offloads - Comparative performance analysis
    • 5.28.1.1. Test Plan
      • 5.28.1.1.1. Test Environment
        • 5.28.1.1.1.1. Preparation
          • 5.28.1.1.1.1.1. Baseline physical host setup
          • 5.28.1.1.1.1.2. VM-to-VM on different nodes setup
        • 5.28.1.1.1.2. Environment description
          • 5.28.1.1.1.2.1. Hardware
          • 5.28.1.1.1.2.2. Software
      • 5.28.1.1.2. Test Case 1: Baseline physical scenario
        • 5.28.1.1.2.1. Description
        • 5.28.1.1.2.2. List of performance metrics
      • 5.28.1.1.3. Test Case 2: Baseline physical over VxLAN scenario
        • 5.28.1.1.3.1. Description
        • 5.28.1.1.3.2. List of performance metrics
      • 5.28.1.1.4. Test Case 3: VM-to-VM on different nodes scenario
        • 5.28.1.1.4.1. Description
        • 5.28.1.1.4.2. List of performance metrics
      • 5.28.1.1.5. Test Case 4: VM-to-VM on different nodes over VxLAN scenario
        • 5.28.1.1.5.1. Description
        • 5.28.1.1.5.2. List of performance metrics
    • 5.28.1.2. Reports
• 5.29. Neutron features test plans
  • 5.29.1. OpenStack Neutron Control Plane Performance and Agent Restart
    • 5.29.1.1. Test Plan
      • 5.29.1.1.1. Test Environment
        • 5.29.1.1.1.1. Preparation
        • 5.29.1.1.1.2. Environment description
          • 5.29.1.1.1.2.1. Hardware
          • 5.29.1.1.1.2.2. Network
          • 5.29.1.1.1.2.3. Software
      • 5.29.1.1.2. Test Case: mass restart of agents
        • 5.29.1.1.2.1. Description
    • 5.29.1.2. Reports
  • 5.29.2. OpenStack Neutron L3 HA Test Plan
    • 5.29.2.1. Test Plan
      • 5.29.2.1.1. Test Environment
        • 5.29.2.1.1.1. Preparation
        • 5.29.2.1.1.2. Environment description
          • 5.29.2.1.1.2.1. Hardware
          • 5.29.2.1.1.2.2. Network
          • 5.29.2.1.1.2.3. Software
      • 5.29.2.1.2. Test Case 1: Comparative analysis of metrics with and without L3 agents restart
        • 5.29.2.1.2.1. Description
        • 5.29.2.1.2.2. List of performance metrics
      • 5.29.2.1.3. Test Case 2: Rally tests execution
        • 5.29.2.1.3.1. Description
        • 5.29.2.1.3.2. List of performance metrics
      • 5.29.2.1.4. Test Case 3: Manual destruction test: Ping to external network from VM during reset of primary(non-primary) controller
        • 5.29.2.1.4.1. Description
        • 5.29.2.1.4.2. List of performance metrics
      • 5.29.2.1.5. Test Case 4: Manual destruction test: Ping from one VM to another VM in different network during ban L3 agent
        • 5.29.2.1.5.1. Description
        • 5.29.2.1.5.2. List of performance metrics
      • 5.29.2.1.6. Test Case 5: Manual destruction test: Iperf UPD testing between VMs in different networks ban L3 agent
        • 5.29.2.1.6.1. Description
        • 5.29.2.1.6.2. List of performance metrics
    • 5.29.2.2. Reports
  • 5.29.3. OpenStack Neutron Resource Density Test Plan
    • 5.29.3.1. Test Plan
      • 5.29.3.1.1. Test Environment
        • 5.29.3.1.1.1. Preparation
        • 5.29.3.1.1.2. Environment description
          • 5.29.3.1.1.2.1. Hardware
          • 5.29.3.1.1.2.2. Network
          • 5.29.3.1.1.2.3. Software
      • 5.29.3.1.2. Test Case: Data-plane performance measurements
        • 5.29.3.1.2.1. Description
        • 5.29.3.1.2.2. List of performance metrics
    • 5.29.3.2. Reports
  • 5.29.4. OpenStack Neutron Density Testing
    • 5.29.4.1. Test Plan
      • 5.29.4.1.1. Test Environment
        • 5.29.4.1.1.1. Preparation
        • 5.29.4.1.1.2. Environment description
          • 5.29.4.1.1.2.1. Hardware
          • 5.29.4.1.1.2.2. Network
          • 5.29.4.1.1.2.3. Software
      • 5.29.4.1.2. Test Case 1: VM density check
        • 5.29.4.1.2.1. Description
        • 5.29.4.1.2.2. List of performance metrics
      • 5.29.4.1.3. Tools
      • 5.29.4.1.4. Test Case 2: Additional integrity check
        • 5.29.4.1.4.1. Testing workflow
        • 5.29.4.1.4.2. Test steps
    • 5.29.4.2. Reports
• 5.30. OpenStack reliability testing
  • 5.30.1. OpenStack reliability testing
    • 5.30.1.1. Test Plan
      • 5.30.1.1.1. Test Environment
        • 5.30.1.1.1.1. Preparation
        • 5.30.1.1.1.2. Environment description
          • 5.30.1.1.1.2.1. Hardware
          • 5.30.1.1.1.2.2. Networking
      • 5.30.1.1.2. Factors description
      • 5.30.1.1.3. Test Case 1: NovaServers.boot_and_delete_server
        • 5.30.1.1.3.1. Description
        • 5.30.1.1.3.2. Service-level agreement
        • 5.30.1.1.3.3. Parameters
        • 5.30.1.1.3.4. List of reliability metrics
        • 5.30.1.1.3.5. Results
          • 5.30.1.1.3.5.1. reboot-random-controller
          • 5.30.1.1.3.5.2. Detailed results description
          • 5.30.1.1.3.5.3. reboot-random-rabbitmq
          • 5.30.1.1.3.5.4. Detailed results description
      • 5.30.1.1.4. Test Case 2: GlanceImages.create_and_delete_image
        • 5.30.1.1.4.1. Description
        • 5.30.1.1.4.2. Service-level agreement
        • 5.30.1.1.4.3. Parameters
        • 5.30.1.1.4.4. List of reliability metrics
        • 5.30.1.1.4.5. Results
          • 5.30.1.1.4.5.1. reboot-random-controller
          • 5.30.1.1.4.5.2. Detailed results description
          • 5.30.1.1.4.5.3. reboot-random-rabbitmq
          • 5.30.1.1.4.5.4. Detailed results description
    • 5.30.1.2. Reports
  • 5.30.2. OpenStack reliability testing. Version 2.0
    • 5.30.2.1. Test Plan
      • 5.30.2.1.1. Test Environment
        • 5.30.2.1.1.1. Preparation
        • 5.30.2.1.1.2. Environment description
          • 5.30.2.1.1.2.1. Hardware
          • 5.30.2.1.1.2.2. Network
          • 5.30.2.1.1.2.3. Software
      • 5.30.2.1.2. Test Case: Reliability Metrics Calculation
        • 5.30.2.1.2.1. Description
        • 5.30.2.1.2.2. Parameters
        • 5.30.2.1.2.3. List of performance metrics
    • 5.30.2.2. Tools
      • 5.30.2.2.1. Rally + os-faults
        • 5.30.2.2.1.1. Calculations
        • 5.30.2.2.1.2. How to run
    • 5.30.2.3. Reports