Make PageAllocator use next_mmap_addr_hint on Windows

[squeek502]

This is a TODO in the code currently:

zig/lib/std/heap.zig

Lines 31 to 32 in 412d863

	/// TODO Utilize this on Windows.
	pub var next_mmap_addr_hint: ?[*]align(mem.page_size) u8 = null;

On non-Windows systems, PageAllocator uses this to make the addresses returned always increase to avoid memory addresses being re-used. This is useful/necessary for the GeneralPurposeAllocator to behave as documented:

zig/lib/std/heap/general_purpose_allocator.zig

Lines 18 to 20 in 412d863

	//! * Do not re-use memory slots, so that memory safety is upheld. For small
	//! allocations, this is handled here; for larger ones it is handled in the
	//! backing allocator (by default `std.heap.page_allocator`).

I'm converting this into an issue to better track progress towards addressing this on Windows.

Right now, VirtualAlloc is used in PageAllocator, but if the function VirtualAlloc2 was used instead (on >= Windows 10), it would allow the use of next_mmap_addr_hint via LowestStartingAddress

@dweiller has tried to make a binding for VirtualAlloc2 for unrelated reasons but ran into some problems. Relevant Discord links:

• https://discord.com/channels/605571803288698900/1155997268105756773/1155997268105756773
• https://discord.com/channels/605571803288698900/906306963942543370/1157910808710348840

I'm having problems trying to cross compile (from linux) a dll using VirtualAlloc2(). My investigations online have pointed to a few issues people have had with the microsoft docs (at least at one point) indicating incorrect dependencies and mingw not exporting it (mingw-w64/mingw-w64#27). I don't really understand what the files in lib/libc/mingw/lib{32,-common} are used for though I gather they define the symbols exported by some libraries and I see that zig only includes VirtualAlloc2FromApp() in them, with VirtualAlloc2() only present in libs/libc/include/any-windows-any/memoryapi.h, which may be one issue (the github link above links to a change to mingw on Jul 28 2023 that adds VirtualAlloc2 to api-ms-win-core-memory-l1-1-6.def). However, attempting to switch to using VirtualAlloc2FromApp results in the same error reported by Zig which is:

error: lld-link: undefined symbol: VirtualAlloc2

This is with -target x86_64-windows-gnu -lkernel32 -ladvapi32 - I'm not 100% confident these linker flags are correct.

My response:

mostly guessing:

• -target x86_64-windows-gnu will use the bundled-with-Zig MinGW which is not fully up-to-date, so the fix for VirtualAlloc2 and MapViewOfFile3 missing from mingw libs mingw-w64/mingw-w64#27 is not included
• mingw: build all CRT libraries #16273 is probably relevant for the fix on Zig's side

looks like the fix for mingw-w64/mingw-w64#27 isn't in the latest tagged mingw release either, so it might take even longer for the fix to make its way into zig

[andrewrk]

VirtualAlloc2 is from kernel32. I think you are looking for NtAllocateVirtualMemory.

Related:

• std.heap.PageAllocator: improve implementation on Windows using NtAllocateVirtualMemory #22846

[Trevor-Strong]

NtAllocateVirtualMemory is just the NT version of VirtualAlloc/VirtualAllocEx. VitualAlloc2 is a wrapper around a different system call, which seems to be NtAllocateVirtualMemoryEx

[alichraghi]

for anyone having trouble finding NtAllocateVirtualMemoryEx definition in win32 api docs:

pub const MEM_EXTENDED_PARAMETER = extern struct {
    BitFields: ULONG64,
    Value: extern union {
        ULong64: DWORD64,
        Pointer: PVOID,
        Size: SIZE_T,
        Handle: HANDLE,
        ULong: DWORD,
    },
};

pub extern "ntdll" fn NtAllocateVirtualMemoryEx(
    ProcessHandle: HANDLE,
    BaseAddress: *?PVOID,
    RegionSize: *SIZE_T,
    AllocationType: ULONG,
    PageProtection: ULONG,
    ExtendedParameters: ?*MEM_EXTENDED_PARAMETER,
    ExtendedParameterCount: ULONG,
) callconv(.winapi) NTSTATUS;