Issue found

Debug run-time dlls for arm64 are not installed properly by Windows 10 SDK.

C:\Windows\System32 path can’t be used for arm64 debug dll loading, even if dlls manual copied from Windows 10 SDK package directories.

Steps to reproduce

My system

Edition	Windows 10 Enterprise
Experience	Windows Feature Experience Pack 120.2212.3920.0

Install

Issue found

Visual Studio Community 2019 (Version 16.10.4)
Visual Studio Installer → Modify → Individual Components
- C++ Universal Windows Platform support for v142 build tools (ARM64)
- Windows 10 SDK (10.0.19041.0)

Re-tested and still reproduced with (at the time latest)

Visual Studio Community 2019 (Version 16.11.9)
Visual Studio Installer → Modify → Individual Components
- C++ Universal Windows Platform support for v142 build tools (ARM64)
- Windows 10 SDK (10.0.20348.0)

Sample project

hello_world.c

#include <stdio.h>

int main (void)
{
    printf("Hello World");
}

Alternatively a new Visual Studio C/C++ project can be created.

Steps

Invoke vcvars for arm64
Build sample project with debug dll run-time libraries: “/MDd”
https://docs.microsoft.com/en-us/cpp/build/reference/md-mt-ld-use-run-time-library?view=msvc-170
Try to run it, check it again in GUI to see error window.

/MDd Defines _DEBUG, _MT, and _DLL and causes the application to use the debug multithread-specific and DLL-specific version of the run-time library. It also causes the compiler to place the library name MSVCRTD.lib into the .obj file.

Commands in powershell

cmd /k '"C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Auxiliary\Build\vcvarsx86_arm64.bat" & powershell'
cd <project_dir>
cl /MDd .\hello_world.c
.\hello_world.exe

In Visual Studio the same flag can be set by (https://docs.microsoft.com/en-us/cpp/build/reference/md-mt-ld-use-run-time-library?view=msvc-170#to-set-this-compiler-option-in-the-visual-studio-development-environment)

Open the project's Property Pages dialog box. For details, see Set C++ compiler and build properties in Visual Studio.
Select the Configuration Properties > C/C++ > Code Generation property page.
Modify the Runtime Library property.

Expected behavior

Windows 10 SDK installs debug dlls into C:\Windows\System32 directory.
This directory is included in the PATH by default.
The hello_world sample application should be able to load the required and installed debug dlls, so it should print the hello world message, then exit successfully.
Just like as if it was built with non-debug dll loading requirements:

Actual behavior

The hello_world sample application fails to start as it can’t find the required debug dlls.

Sanity check

Are the debug dlls installed into System32? Yes
Is this directory included on the PATH? Yes
$env:PATH ... C:\Windows\system32 ...

Investigation

Check dll header with dumpbin: ucrtbase*.dll

Release

 dumpbin /HEADERS c:\Windows\system32\ucrtbase.dll

Debug

 dumpbin /HEADERS c:\Windows\system32\ucrtbased.dll

Copy the debug dll into another directory:

c:\Windows\system32\ucrtbased.dll -> c:\kg\dll_tmp\system32\ucrtbased.dll

Dumpbin the copy of the debug dll

Windows On Arm (WOA) > Debug run-time DLL issue > image-20220119-101332.png

Issues found here

The debug dll installed into System32 directory is for x86, while the release version is for ARM64.
In System32 directory the debug dll can’t be opened, but the same file is available on a different path!

Check debug dlls on other installed paths

ucrtbased.dll is packed by Windows 10 SDK to

c:\Program Files (x86)\Microsoft SDKs\Windows Kits\10\ExtensionSDKs\Microsoft.UniversalCRT.Debug\10.0.19041.0\Redist\Debug\arm64\ucrtbased.dll
c:\Program Files (x86)\Microsoft SDKs\Windows Kits\10\ExtensionSDKs\Microsoft.UniversalCRT.Debug\10.0.19041.0\Redist\Debug\arm\ucrtbased.dll
c:\Program Files (x86)\Microsoft SDKs\Windows Kits\10\ExtensionSDKs\Microsoft.UniversalCRT.Debug\10.0.19041.0\Redist\Debug\x86\ucrtbased.dll
c:\Program Files (x86)\Microsoft SDKs\Windows Kits\10\ExtensionSDKs\Microsoft.UniversalCRT.Debug\10.0.19041.0\Redist\Debug\x64\ucrtbased.dll

Issues verified here

x86/ucrtbased.dll is the same file (based on file sizes) as the installed at C:\Windows\System32.
Copying the arm64/ucrtbased.dll to C:\Windows\System32 doesn’t fix the original problem as even dumpbin still can’t open that file on that path.
1. Again, on another path dumbpin can reach it and the header is for ARM64 as expected.

Windows On Arm (WOA) > Debug run-time DLL issue > image-20220119-102721.png

Workaround

Add any missing debug dll’s Windows SDK package directory to the path.

Example:

1. ucrtbased.dll

Path:

"c:\Program Files (x86)\Microsoft SDKs\Windows Kits\10\ExtensionSDKs\Microsoft.UniversalCRT.Debug\10.0.19041.0\Redist\Debug\arm64\ucrtbased.dll"

Command in cmd:

PATH=c:\Program Files (x86)\Windows Kits\10\bin\10.0.19041.0\arm64\ucrt\;%PATH%

Command in PS:

$env:PATH="c:\Program Files (x86)\Windows Kits\10\bin\10.0.19041.0\arm64\ucrt\;$env:PATH"

2. vcruntime140.dll

Path:

c:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Redist\MSVC\14.29.30036\debug_nonredist\arm64\Microsoft.VC142.DebugCRT\MSVCP140D.dll

Command in cmd:

PATH=c:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Redist\MSVC\14.29.30036\debug_nonredist\arm64\Microsoft.VC142.DebugCRT\%PATH%

Command in PS:

$env:PATH="c:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Redist\MSVC\14.29.30036\debug_nonredist\arm64\Microsoft.VC142.DebugCRT;$env:PATH"

Tools

You can find missing dll using an msys prompt:

$ find /c/Program\ Files\ (x86)/ | grep -i ucrtbased.dll | grep arm64

To see dependencies of a given executable, you can use:

LDD

ldd (from cygwin/msys). It does not take into account arm64 arch, so it will only report first dll on your path. By using file, you can verify every dependency.

$ ldd python_d.exe
        ntdll.dll => /c/Windows/SYSTEM32/ntdll.dll (0x7ffc97690000)
        KERNEL32.DLL => /c/Windows/System32/KERNEL32.DLL (0x7ffc96fd0000)
        KERNELBASE.dll => /c/Windows/System32/KERNELBASE.dll (0x7ffc92fa0000)
        vcruntime140d.dll => /c/Windows/System32/vcruntime140d.dll (0x7ffc8c480000)
        python311_d.dll => /c/Users/tcwg/try/PCbuild/arm64/python311_d.dll (0x7ffc408b0000)
        WS2_32.dll => /c/Windows/System32/WS2_32.dll (0x7ffc955e0000)
        RPCRT4.dll => /c/Windows/System32/RPCRT4.dll (0x7ffc967a0000)
        ADVAPI32.dll => /c/Windows/System32/ADVAPI32.dll (0x7ffc94070000)
        msvcrt.dll => /c/Windows/System32/msvcrt.dll (0x7ffc95370000)
        sechost.dll => /c/Windows/System32/sechost.dll (0x7ffc97570000)
        ucrtbased.dll => /c/Windows/System32/ucrtbased.dll (0x7ffc744d0000)
        VCRUNTIME140D.dll => /c/Program Files (x86)/Microsoft Visual Studio/2019/BuildTools/VC/Redist/MSVC/14.29.30036/debug_nonredist/arm64/Microsoft.VC142.DebugCRT/VCRUNTIME140D.dll (0x7ffc8c450000)
        VERSION.dll => /c/Windows/SYSTEM32/VERSION.dll (0x7ffc829c0000)
        ucrtbased.dll => /c/Program Files (x86)/Windows Kits/10/bin/10.0.19041.0/arm64/ucrt/ucrtbased.dll (0x7ffc744e0000)
        bcrypt.dll => /c/Windows/SYSTEM32/bcrypt.dll (0x7ffc92360000)

$ file /c/Windows/SYSTEM32/bcrypt.dll
/c/Windows/SYSTEM32/bcrypt.dll: PE32+ executable (DLL) (console) Aarch64, for MS Windows

$ file /c/Windows/System32/vcruntime140d.dll
/c/Windows/System32/vcruntime140d.dll: PE32+ executable (DLL) (console) x86-64, for MS Windows

Dependencies

https://www.dependencywalker.com/ used to be a good solution to see DLL deps using a GUI. It is now outdated, and does not handle ARM64 arch.

A replacement, Dependencies, is now written to modernize it. It works with ARM64 arch: GitHub - lucasg/Dependencies

$ DependenciesGui.exe python_d.exe

Windows On Arm (WOA) > Debug run-time DLL issue > Screenshot_2022-04-07_09-46-29.png