The HPC Lab needs to use machines that are in pre-production and development stage, so a few quirks may be necessary to get them behaving like standard data centre machines.

Firmware / BIOS

D03/D05

BMC mode with HPM files (last resort)

Vendor firmware is signed and can be updated directly from the BMC web UI.

...

biosupdate 10.40.0.11 -u anonymous -p xxx -f D05.fd master

You need to download it and put into the local FTP server in Mr-Provisioner.

OS mode with Capsule files (preferred)

...

debian>root> apt install fwupdate
centos> |yum install -y fwupdate

root> any> fwupdate --info
# this will show an ID like {7978365d-7978-45fd-ad77-b27693cfe85b} and the current version


any>root> fwupdate --apply {...ID...} D0xFIRMWAREUPDATECAPSULE....Cap
# this will update the firmware


# reboot and try again
any>root> fwupdate --info
# this will show an ID and the current version, should be newer

...

Starting from 18.08, ERP firmware will allow capsule updates straight from the BIOS screenUEFI shell.

Use the latest firmware from releases.linaro.org.

...

Upgrade the BIOS, as described in their internal documents (not public), using IPMI and the new hpm images.

A few comments on TX2 BIOS:

• To be able to choose between booting on the internal PXE interface or the LAN one (e.g. Warewulf on the former, MrP on the latter), one needs to set one to be the 'default' boot order (which entails that it will PXE after a power cut off)
• Flashing the BIOS with ipmitool sometimes does NOT seem to work (despite ipmitool saying its fine). In those cases you'll end up with boot entries missing, network interfaces not showing up, and other errors of the sort. All you need in those cases is to just reflash the BIOS, and it should do the trick. NOTE: During the reflash, you'll notice a mismatch between the (re)uploaded version and the one on the machine (whereas if the first flash had gone according to plan they'd be the same).
• If it so happens that the ipmitool is spitting out errors of "version mismatch" and asking if you want to proceed, you need to update the BMC FW via the webinterface and using the .ima file (web paged is called "dual_firmware_upgrade"). Package 7.4 contains an .ima file that will reenable you to use the standard procedure via ipmitool.

Operating System

CentOS

AArch64 support in CentOS is done via AltArch. The repository and kernel images are available from their mirror, but that's always the latest release only.

NOTE: Make sure not to use Linaro's overlays with upstream CentOS, or the kernel will be overwritten and broken.

To install CentOS via PXE on Mr-Provisioner, use the following parameters:

...