/
Build Packages from Source

Build Packages from Source

Owned by Niyas Sait (Deactivated)
Last updated: Jun 23, 2022
2 min read

Overview

The following tutorials are targeting the currently known most simple and clean way of building any Python packages, even if those include platform specific extension (mostly C/C++) code.

These steps assume clean (Win10) system.

Third party dependencies should be solved case by case and as soon as those are solved, these can be the final steps to create release packages.

1. Setting the evironment

Common requirement

  • Install Visual Studio 2019

  • Install ARM64 build tools

1.2. Build machine: x64

Background: setuptools-distutils | 4. WoA (win arm64) support

1. Install latest (matching with the arm64win Python version in next step) win-x64 Python release to x64 machine from

2. Get latest (same version as x64 above) arm64win Python from http://nuget.org

 

3. Create and activate virtual environment by the x64 Python

C:\Users\<user>\AppData\Local\Programs\Python\Python310-32\python.exe --version Python 3.10.0rc1 C:\Users\<user>\AppData\Local\Programs\Python\Python310-32\python.exe -m venv <venv dir> <venv dir>\Scripts\activate.bat

4. Copy arm64win Python libs to x64 venv

  • copy

    • from

      <pythonarm64 unzipped dir>\libs

    • to

      <venv dir>\libs

5.Make sure latest distutils is used from setuptools (not older than setuptools-57.4.0)

  • Update setuptools within venv

    (venv_3.10_rc1) python -m pip install -U setuptools

  • set environment variableto use distutils from this module and not the global Python’s internal one

    set SETUPTOOLS_USE_DISTUTILS=local

6. Set environment variables to mimic cross-compilation

  • set VSCMD_ARG_TGT_ARCH=arm64 set SETUPTOOLS_EXT_SUFFIX=.cp310-win_arm64.pyd

7. Goto Package building tutorial | 2. Build the package

1.2. Build machine: WoA = win-arm64

Background: setuptools-distutils | 4. WoA (win arm64) support

  1. Get release version of arm64python

    • Current latest prerelease build: pythonarm64 3.10.0-rc2

    • Rename it to zip

    • Unzip and use python.exe from that path

  2. Create and activate virtual environment

    <arm64python>\python.exe -m venv <venv dir> <venv dir>\Scripts\activate.bat

  3. Install pre-release arm64win packages
    Python | Custom pre release package builds (wheel files)

    • pip

      <venv> python -m pip install <path to downloaded Pip-21.3.dev0-py3-none-any.whl> <venv> python -m pip install <path to downloaded Pip-21.3.dev0-py3-none-any.whl> --force-reinstall

      • note: the 2nd force-reinstall is required to generate the arm64win pip executables with the updated distlib of the 1st installation

    • setuptools

      <venv> pip install <path to downloaded Setuptools-57.4.0.post20210908-py3-none-any.whl>

  4. Goto Package building tutorial | 2. Build the package

2. Build the package

2.1. Test the system

The final goal here is to have a portable wheel file, but until we’re not sure the extension (mostly C/C++) build process (build_ext) is working, for try and error it doesn’t make sense to invoke the full process of wheel generation.

  • If the current project has any specific build steps, then use those.

  • Otherwise

    <venv> python setup.py build_ext

2.2. Create portable wheel file

  • note: this step requires to install wheel by pip

    <venv> pip install wheel

  • If project specific build process or generic build_ext is working, we can try wheel generation

    <venv> pip wheel --no-cache-dir --no-build-isolation -w <output directory> <path of the package source>

Related content