TMP Spack

Owned by Mikael Rolfhamre
Mar 06, 2023
3 min read

Spack is a package manager for supercomputers, Linux, and macOS. It makes installing scientific software easy. Spack isn’t tied to a particular language; you can build a software stack in Python or R, link to libraries written in C, C++, or Fortran, and easily swap compilers or target specific microarchitectures. Learn more here.

Spack was developed in the Lawrence Livermore National Laboratory in the USA.

Spack is licensed under the Apache License, Version 2.0 (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0) or the MIT license, (LICENSE-MIT or http://opensource.org/licenses/MIT), at your option.

Technical background

  • SPACK is written in Python so is not itself affected by the CPU architecture it runs on.

  • In order to install a package SPACK makes use of spec files that describe the package and its dependencies. These spec files are written in Python.

  • Installs of packages by SPACK are parameterised so that the user can specify aspects of the installation such as which version is installed or the build flags used.

  • SPACK updates RPATHs in packages to ensure that the correct dependency is used at runtime.

  • SPACK builds on the target system so that the package can be built with the best optimisations for that system and so provide the best performance.

  • Multiple versions of the same package can exist on a machine at the same time without interfering with each other. Only the one that is active at the moment will be used.

  • SPACK uses hashes of configurations to distinguish between differing builds of packages.

  • SPACK currently lists 5064 packages with built in spec files.

  • SPACK can also handle independent groups of packages similarly in concept to Python virtual environments, which can be easily shared. Loading/unloading a group saves having to perform that operation on all the packages in the group individually.

Vision

SPACK exists to manage installations of multiple versions of the same package and give the user the ability to switch between them.

SPACK aims to handle all the complexities of differing version dependencies due to which version of a package you wish to install.

Community Collaboration

Spack is an open source project. Questions, discussion, and contributions are welcome. Contributions can be anything from new packages to bugfixes, documentation, or even new core features.

Contributing to Spack is relatively easy. Just send us a pull request. When you send your request, make develop the destination branch on the Spack repository.

Resources:

Challenges

The big challenge seems to be in writing and maintaining the spec files.

ARM considerations

The spec files need to be written in such a way as to cope with any ARM specific build requirements for each package. This will require in depth knowledge of each package.

Opportunities for Linaro

  1. Ensure that spec files exist and work for ARM builds of any packages that Linaro believes to be worth supporting.

  2. Have SPACK installed by default on developer hack boxes in the colo and use it to install packages that are missing the required version from the distro in question. This should save time and prevent frustration. At the same time it will allow SPACK to be exercised on ARM hardware potentially showing up any packages where ARM support is lacking.

Recommendations

Opportunity 1. above seems like a large excess of work over any potential benefit - not recommended.

Opportunity 2. above is worth giving a try as it may benefit Linaro developers but is not likely to result in huge improvements anywhere. Suggested as a low priority.

Â