HowTo add new reporting mechanism

Reporting mechanism for verifications is pluggable. Custom plugins can be used for custom output formats or for exporting results to external systems.

We hardly recommend to read Rally Plugins page to understand how do Rally Plugins work.

Spec

All reporters should inherit rally.verification.reporter.VerificationReporter and implement all abstract methods. Here you can find its interface:

class rally.verification.reporter.VerificationReporter(verifications, output_destination)

Base class for all reporters for verifications.

base_ref

alias of VerificationReporter

generate()

Generate report

Returns:a dict with 3 optional elements:

• key “files” with a dictionary of files to save on disk. keys are paths, values are contents;
• key “print” - data to print at CLI level
• key “open” - path to file which should be open in case of –open flag

static make(reporter_cls, verifications, output_destination)

Initialize reporter, generate and validate report.

It is a base method which is called from API layer. It cannot be overridden. Do not even try! :)

Parameters:

• reporter_cls – class of VerificationReporter to be used
• verifications – list of results to generate report for
• output_destination – destination of report

classmethod validate(output_destination)

Validate destination of report.

Parameters:output_destination – Destination of report

Example of custom JSON Reporter

Basically, you need to implement only two methods “validate” and “generate”.

Method “validate” should check that destination of the report is right. Method “generate” should build a report or export results somewhere; actually, it is up to you what it should do but return format is strict, see Spec section for what it can return.

import json

from rally.verification import reporter


@reporter.configure("summary-in-json")
class SummaryInJsonReporter(reporter.VerificationReporter):
    """Store summary of verification(s) in JSON format"""

    # ISO 8601
    TIME_FORMAT = "%Y-%m-%dT%H:%M:%S%z"

    @classmethod
    def validate(cls, output_destination):
        # we do not have any restrictions for destination, so nothing to
        # check
        pass

    def generate(self):
        report = {}

        for v in self.verifications:
            report[v.uuid] = {
                "started_at": v.created_at.strftime(self.TIME_FORMAT),
                "finished_at": v.updated_at.strftime(self.TIME_FORMAT),
                "status": v.status,
                "run_args": v.run_args,
                "tests_count": v.tests_count,
                "tests_duration": v.tests_duration,
                "skipped": v.skipped,
                "success": v.success,
                "expected_failures": v.expected_failures,
                "unexpected_success": v.unexpected_success,
                "failures": v.failures,
                # v.tests includes all information about launched tests,
                # but for simplification of this fake reporters, let's
                # save just names
                "launched_tests": [test["name"]
                                   for test in v.tests.values()]
            }

        raw_report = json.dumps(report, indent=4)

        if self.output_destination:
            # In case of output_destination existence report will be saved
            # to hard drive and there is nothing to print to stdout, so
            # "print" key is not used
            return {"files": {self.output_destination: raw_report},
                    "open": self.output_destination}
        else:
            # it is something that will be print at CLI layer.
            return {"print": raw_report}