...

4. WoA (win-arm64) support

...

Limitation of the cross-compilation

• From Python point of view, we can only invoke native compilation on x64 and win-arm64 as well.

  • https://github.com/pypa/setuptools/blob/main/setuptools/_distutils/command/build_ext.py#L312

  • (self.plat_name == get_platform()) is set to true by our usecases

• But it's cross-compilation by the C compiler (MSVC), so I tend to call it fake native it’s not a perfectly clean cross-compilation as we convince Python to invoke the native build flow by a cross-compiler.

  • By VSCMD_ARG_TGT_ARCH env var, we're telling to Python it runs on the given platform

  • As there is no native compiler for win-arm64 this works (for now) - as seen in sources below - win-arm64 platform always results x86/arm64 compiler.

    • VSCMD_ARG_TGT_ARCH=arm64 -> x86_arm64  (PLAT_TO_VCVARS) -> VC\Tools\MSVC\<version>\bin\HostX86\ARM64\cl.exe

• build_ext has plat_name as cross-compiling target platform parameter, which looks hacky at first sight and I suppose that's why it’s not preferred to use

• I see one long-term problem here:

  • If there will be ever a native win-arm64/win-arm64 MSVC compiler, this method should be updated heavily as it can't fallback to win-arm64 = x86/win-arm64 anymore.

    • The current fake cross-compilation would fail as it would try to use the real native one.

  • It’s not a blocker, but it should be known.

...

• Described https://linaro.atlassian.net/wiki/spaces/WOAWOAR/pages/28593914166/setuptools-distutils#3.1.-Built-in-lib-in-Python-interpreter

• As this is going to be deprecated, I won’t expect any further support.

• Short summary

  • On WoA machine, the cross-compile MSVC is called by default, but on x64 it can’t be enforced.

• win-arm64 has been added as a host platform (in 2019)

  • This means on win-arm64 platform without any env var setting, it will invoke x86/arm64 MSVC compiler by default.

    • Which is good on WoA

  • https://github.com/python/cpython/blob/v3.10.0rc1/Lib/distutils/util.py#L39

    Code Block
    def get_host_platform(): if os.name == 'nt': if 'amd64' in sys.version.lower(): return 'win-amd64' if '(arm)' in sys.version.lower(): return 'win-arm32' if '(arm64)' in sys.version.lower(): return 'win-arm64' return sys.platform

  • https://github.com/python/cpython/blob/v3.10.0rc1/Lib/distutils/_msvccompiler.py#L163

    Code Block
    PLAT_TO_VCVARS = { 'win32' : 'x86', 'win-amd64' : 'x86_amd64', 'win-arm32' : 'x86_arm', 'win-arm64' : 'x86_arm64' }

• win-arm64 is not added as a target platform

  • it means win-arm64 as build target can’t be enforced by VSCMD_ARG_TGT_ARCH=arm64 environment variable on x64

  • as this is going to be deprecated and not recommended to use for any projects it’s not a blocker, but any project that is tied to old distutils, can’t be cross-compiled from x64 to win-arm64, for example numpy!

    • see https://linaro.atlassian.net/wiki/spaces/WOAWOAR/pages/28593914166/setuptools-distutils#2.-Important-issues

  • https://github.com/python/cpython/blob/v3.10.0rc1/Lib/distutils/util.py#L99

    Code Block
    def get_platform(): if os.name == 'nt': TARGET_TO_PLAT = { 'x86' : 'win32', 'x64' : 'win-amd64', 'arm' : 'win-arm32', } return TARGET_TO_PLAT.get(os.environ.get('VSCMD_ARG_TGT_ARCH')) or get_host_platform() else: return get_host_platform()

...

• Described https://linaro.atlassian.net/wiki/spaces/WOAWOAR/pages/28593914166/setuptools-distutils#3.2.-Within-stand-alone-setuptools-package

• As this deprecates the internal lib of CPython, any further support should be added and expected here.

• Short summary

  • On WoA machine, the cross-compile MSVC is called by default, while on x64 it can be enforced by VSCMD_ARG_TGT_ARCH=arm64 env var.

• win-arm64 is supported as host platform

  • https://github.com/pypa/setuptools/blob/main/setuptools/_distutils/util.py#L40

    Code Block
    def get_host_platform(): if os.name == 'nt': if 'amd64' in sys.version.lower(): return 'win-amd64' if '(arm)' in sys.version.lower(): return 'win-arm32' if '(arm64)' in sys.version.lower(): return 'win-arm64' return sys.platform

  • https://github.com/pypa/setuptools/blob/main/setuptools/_distutils/_msvccompiler.py#L92

    Code Block
    PLAT_TO_VCVARS = { 'win32' : 'x86', 'win-amd64' : 'x86_amd64', 'win-arm32' : 'x86_arm', 'win-arm64' : 'x86_arm64' }

• win-arm64 is supported as a target platform as well

  • https://github.com/pypa/setuptools/blob/0c302d232dd1a1f4ec531050532a84274edac2f4/setuptools/_distutils/util.py#L100

    Code Block
    def get_platform(): if os.name == 'nt': TARGET_TO_PLAT = { 'x86' : 'win32', 'x64' : 'win-amd64', 'arm' : 'win-arm32', 'arm64': 'win-arm64', } return TARGET_TO_PLAT.get(os.environ.get('VSCMD_ARG_TGT_ARCH')) or get_host_platform() else: return get_host_platform()

...