This 4This page describes a validation plan for OpenHPC deployment on AArch64 systems. It is not restrict to hardware, cloud or emulation systems and should be reasonably independent of the media.

...

• is accepted upstream (OpenHPC), so that it can be repeated by other members of the community, not just our members.
• is accepted by the infrastructure teams at Linaro, and that have had their review and input.
• is fully automated, with a few choices (compiler, MPI, etc.) and can be deployed on top of a vanilla image produced on the step #1.
• At least one OpenMP and one MPI test is performed successfully on each variation.

Step #3: Workload Testing

Once OpenHPC is being automatically deployed and tested for simple programs, we need to start looking at the packages and tests that the SIG members want us to continuously test.

Engineering Specification

For a first step, we want the workloads / libraries to be available through OpenHPC, and we're already working on getting some in there, but ultimately, we may be able to provide a git repository and a recipe (as a script or playbooks), and run that as additional testing.

We'd need to understand how to encapsulate each test, hopefully in similar steps: download, install, build, test. For example, installing an OpenHPC package means "download" and "install" are on the same process, so we'll need to allow those steps to be empty.

The testing phase can also be a benchmark, so not only we'll have outputs that infer the quality (proximity to certain floating point values or exact results) but also measure run time, or "reported time" and return that as integers, so we can track them in a separate database.

This last step is not necessary on the first run of step #3, but we ultimately want to track benchmark numbers and be able to identify regressions, spikes and trends.

Acceptance Criteria

The first iteration of this step needs to produce:

• A framework for how to create tests, separated in steps (download, install, build, test), and applicable to OpenHPC packages.
• A way to analyse the results based on the nature of the workload (statistical, exact, approximate) or to allow external scripts to validate output.
• A way to communicate pass/fail back to the process that initiated the deployment in the first place, so that we can have a nice green/red status.
• An example with at least one OpenHPC package being installed, compiled, tested and the results showing green/red on real errors.