Skip to content

Misc. bug: Potential memory leak in backend registry #12986

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
dorpxam opened this issue Apr 16, 2025 · 0 comments
Open

Misc. bug: Potential memory leak in backend registry #12986

dorpxam opened this issue Apr 16, 2025 · 0 comments
Labels
bug-unconfirmed

Comments

@dorpxam
Copy link

dorpxam commented Apr 16, 2025

Name and Version

build: 5124 (bc091a4) with MSVC 19.43.34810.0 for x64 (debug)
static build (MT/MTd) with VS2022 / LLAMA & GGML
GGML_STATIC / GGML_USE_CPU / GGML_USE_BLAS / GGML_USE_CUDA

Operating systems

Windows

Which llama.cpp modules do you know to be affected?

Other (Please specify in the next section)

Command line

No command line. Using personal C++ (preliminary) implementation that follow the steps of the llama-cli.

Problem description & steps to reproduce

I'm currently only do :

llama_backend_init();
llama_model_load_from_file();
llama_model_free();
llama_backend_free();

There's no problem of memory leak until I offload the model on my GPU with n_gpu_layers > 0

The problem of leak comes with ggml_backend_cuda_reg() that act as a singleton-like fashion and never release the static context allocated as new. I don't know if the problem is specific from my build (GGML_STATIC) but I think that is potentially related to the comment:

ggml-backend-reg.cpp (line 196)

~ggml_backend_registry() {
// FIXME: backends cannot be safely unloaded without a function to destroy all the backend resources,
// since backend threads may still be running and accessing resources from the dynamic library

This is not a huge leak but here's the details of the leak detector:

---------- Block 11520 at 0x000000009F233040: 16 bytes ----------
  Leak Hash: 0xD208674E, Count: 1, Total 16 bytes
  Call Stack (TID 22032):
    ntdll.dll!RtlAllocateHeap()
    D:\a\_work\1\s\src\vctools\crt\vcstartup\src\heap\new_scalar.cpp (36): AIRoleplay.exe!operator new() + 0xA bytes
    C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.43.34808\include\xmemory (137): AIRoleplay.exe!std::_Default_allocate_traits::_Allocate()
    C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.43.34808\include\xmemory (258): AIRoleplay.exe!std::_Allocate<16,std::_Default_allocate_traits>()
    C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.43.34808\include\xmemory (987): AIRoleplay.exe!std::allocator<std::_Container_proxy>::allocate()
    C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.43.34808\include\xmemory (1216): AIRoleplay.exe!std::_Container_base12::_Alloc_proxy<std::allocator<std::_Container_proxy> >() + 0xF bytes
    C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.43.34808\include\vector (656): AIRoleplay.exe!std::vector<ggml_backend_device *,std::allocator<ggml_backend_device *> >::vector<ggml_backend_device *,std::allocator<ggml_backend_device *> >() + 0x2E bytes
    AIRoleplay.exe!ggml_backend_cuda_reg_context::ggml_backend_cuda_reg_context() + 0x17 bytes
    E:\Dependencies\ai.cpp\ggml\src\ggml-cuda\ggml-cuda.cu (3451): AIRoleplay.exe!ggml_backend_cuda_reg() + 0x2A bytes
    AIRoleplay.exe!ggml_backend_registry::ggml_backend_registry() + 0x5F bytes
    AIRoleplay.exe!ggml_backend_register() + 0x25A bytes
    AIRoleplay.exe!ggml_backend_dev_count() + 0x9 bytes
    AIRoleplay.exe!llama_print_system_info() + 0x583 bytes
    AIRoleplay.exe!llama_model_load_from_file() + 0x90 bytes
  Data:
    60 9A C5 A4    80 01 00 00    00 00 00 00    00 00 00 00     `....... ........

---------- Block 11519 at 0x00000000A4C59A60: 32 bytes ----------
  Leak Hash: 0x55976925, Count: 1, Total 32 bytes
  Call Stack (TID 22032):
    ntdll.dll!RtlAllocateHeap()
    D:\a\_work\1\s\src\vctools\crt\vcstartup\src\heap\new_scalar.cpp (36): AIRoleplay.exe!operator new() + 0xA bytes
    E:\Dependencies\ai.cpp\ggml\src\ggml-cuda\ggml-cuda.cu (3451): AIRoleplay.exe!ggml_backend_cuda_reg() + 0xA bytes
    AIRoleplay.exe!ggml_backend_registry::ggml_backend_registry() + 0x5F bytes
    AIRoleplay.exe!ggml_backend_register() + 0x25A bytes
    AIRoleplay.exe!ggml_backend_dev_count() + 0x9 bytes
    AIRoleplay.exe!llama_print_system_info() + 0x583 bytes
    AIRoleplay.exe!llama_model_load_from_file() + 0x90 bytes
Data:
  40 30 23 9F    80 01 00 00    F0 28 F4 A4    80 01 00 00     @0#..... .(......
  F8 28 F4 A4    80 01 00 00    F8 28 F4 A4    80 01 00 00     .(...... .(......

---------- Block 11588 at 0x00000000A4F3CE90: 88 bytes ----------
  Leak Hash: 0xDC22AD9D, Count: 1, Total 88 bytes
  Call Stack (TID 22032):
    ntdll.dll!RtlAllocateHeap()
    D:\a\_work\1\s\src\vctools\crt\vcstartup\src\heap\new_scalar.cpp (36): AIRoleplay.exe!operator new() + 0xA bytes
    E:\Dependencies\ai.cpp\ggml\src\ggml-cuda\ggml-cuda.cu (3454): AIRoleplay.exe!ggml_backend_cuda_reg() + 0xA bytes
    AIRoleplay.exe!ggml_backend_registry::ggml_backend_registry() + 0x5F bytes
    AIRoleplay.exe!ggml_backend_register() + 0x25A bytes
    AIRoleplay.exe!ggml_backend_dev_count() + 0x9 bytes
    AIRoleplay.exe!llama_print_system_info() + 0x583 bytes
    AIRoleplay.exe!llama_model_load_from_file() + 0x90 bytes
 Data:
   00 00 00 00    CD CD CD CD    60 1D F4 A4    80 01 00 00     ........ `.......
   43 55 44 41    30 00 00 00    00 00 00 00    00 00 00 00     CUDA0... ........
   05 00 00 00    00 00 00 00    0F 00 00 00    00 00 00 00     ........ ........
   A0 19 F4 A4    80 01 00 00    00 41 0D A5    80 01 00 00     ........ .A......
   00 00 00 00    00 00 00 00    17 00 00 00    00 00 00 00     ........ ........
   1F 00 00 00    00 00 00 00                                   ........ ........

---------- Block 11590 at 0x00000000A4F419A0: 16 bytes ----------
  Leak Hash: 0x31E4675E, Count: 1, Total 16 bytes
  Call Stack (TID 22032):
    ntdll.dll!RtlAllocateHeap()
    D:\a\_work\1\s\src\vctools\crt\vcstartup\src\heap\new_scalar.cpp (36): AIRoleplay.exe!operator new() + 0xA bytes
    C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.43.34808\include\xmemory (137): AIRoleplay.exe!std::_Default_allocate_traits::_Allocate()
    C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.43.34808\include\xmemory (258): AIRoleplay.exe!std::_Allocate<16,std::_Default_allocate_traits>()
    C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.43.34808\include\xmemory (987): AIRoleplay.exe!std::allocator<std::_Container_proxy>::allocate()
    C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.43.34808\include\xmemory (1216): AIRoleplay.exe!std::_Container_base12::_Alloc_proxy<std::allocator<std::_Container_proxy> >() + 0xF bytes
    C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.43.34808\include\xstring (833): AIRoleplay.exe!std::basic_string<char,std::char_traits<char>,std::allocator<char> >::_Construct_empty() + 0x24 bytes
    C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.43.34808\include\xstring (685): AIRoleplay.exe!std::basic_string<char,std::char_traits<char>,std::allocator<char> >::basic_string<char,std::char_traits<char>,std::allocator<char> >() + 0xA bytes
    AIRoleplay.exe!ggml_backend_cuda_device_context::ggml_backend_cuda_device_context() + 0x2C bytes
    E:\Dependencies\ai.cpp\ggml\src\ggml-cuda\ggml-cuda.cu (3454): AIRoleplay.exe!ggml_backend_cuda_reg() + 0x2A bytes
    AIRoleplay.exe!ggml_backend_registry::ggml_backend_registry() + 0x5F bytes
    AIRoleplay.exe!ggml_backend_register() + 0x25A bytes
    AIRoleplay.exe!ggml_backend_dev_count() + 0x9 bytes
    AIRoleplay.exe!llama_print_system_info() + 0x583 bytes
    AIRoleplay.exe!llama_model_load_from_file() + 0x90 bytes
Data:
  C0 CE F3 A4    80 01 00 00    00 00 00 00    00 00 00 00     ........ ........

---------- Block 11589 at 0x00000000A4F41D60: 16 bytes ----------
  Leak Hash: 0x81F556EB, Count: 1, Total 16 bytes
  Call Stack (TID 22032):
    ntdll.dll!RtlAllocateHeap()
    D:\a\_work\1\s\src\vctools\crt\vcstartup\src\heap\new_scalar.cpp (36): AIRoleplay.exe!operator new() + 0xA bytes
    C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.43.34808\include\xmemory (137): AIRoleplay.exe!std::_Default_allocate_traits::_Allocate()
    C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.43.34808\include\xmemory (258): AIRoleplay.exe!std::_Allocate<16,std::_Default_allocate_traits>()
    C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.43.34808\include\xmemory (987): AIRoleplay.exe!std::allocator<std::_Container_proxy>::allocate()
    C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.43.34808\include\xmemory (1216): AIRoleplay.exe!std::_Container_base12::_Alloc_proxy<std::allocator<std::_Container_proxy> >() + 0xF bytes
    C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.43.34808\include\xstring (833): AIRoleplay.exe!std::basic_string<char,std::char_traits<char>,std::allocator<char> >::_Construct_empty() + 0x24 bytes
    C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.43.34808\include\xstring (685): AIRoleplay.exe!std::basic_string<char,std::char_traits<char>,std::allocator<char> >::basic_string<char,std::char_traits<char>,std::allocator<char> >() + 0xA bytes
    AIRoleplay.exe!ggml_backend_cuda_device_context::ggml_backend_cuda_device_context() + 0x1B bytes
    E:\Dependencies\ai.cpp\ggml\src\ggml-cuda\ggml-cuda.cu (3454): AIRoleplay.exe!ggml_backend_cuda_reg() + 0x2A bytes
    AIRoleplay.exe!ggml_backend_registry::ggml_backend_registry() + 0x5F bytes
    AIRoleplay.exe!ggml_backend_register() + 0x25A bytes
    AIRoleplay.exe!ggml_backend_dev_count() + 0x9 bytes
    AIRoleplay.exe!llama_print_system_info() + 0x583 bytes
    AIRoleplay.exe!llama_model_load_from_file() + 0x90 bytes
 Data:
   98 CE F3 A4    80 01 00 00    00 00 00 00    00 00 00 00     ........ ........

---------- Block 11595 at 0x00000000A4F428F0: 8 bytes ----------
  Leak Hash: 0x051EE01B, Count: 1, Total 8 bytes
  Call Stack (TID 22032):
    ntdll.dll!RtlAllocateHeap()
    D:\a\_work\1\s\src\vctools\crt\vcstartup\src\heap\new_scalar.cpp (36): AIRoleplay.exe!operator new() + 0xA bytes
    C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.43.34808\include\xmemory (137): AIRoleplay.exe!std::_Default_allocate_traits::_Allocate()
    C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.43.34808\include\xmemory (258): AIRoleplay.exe!std::_Allocate<16,std::_Default_allocate_traits>()
    C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.43.34808\include\xmemory (987): AIRoleplay.exe!std::allocator<ggml_backend_device *>::allocate()
    C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.43.34808\include\xmemory (2268): AIRoleplay.exe!std::_Allocate_at_least_helper<std::allocator<ggml_backend_device *> >()
    AIRoleplay.exe!std::vector<ggml_backend_device * __ptr64,std::allocator<ggml_backend_device * __ptr64> >::_Emplace_reallocate<ggml_backend_device * __ptr64 const & __ptr64>() + 0xD9 bytes
    AIRoleplay.exe!std::vector<ggml_backend_device * __ptr64,std::allocator<ggml_backend_device * __ptr64> >::_Emplace_one_at_back<ggml_backend_device * __ptr64 const & __ptr64>() + 0x82 bytes
    AIRoleplay.exe!std::vector<ggml_backend_device * __ptr64,std::allocator<ggml_backend_device * __ptr64> >::push_back() + 0x1D bytes
    E:\Dependencies\ai.cpp\ggml\src\ggml-cuda\ggml-cuda.cu (3468): AIRoleplay.exe!ggml_backend_cuda_reg() + 0x22 bytes
    AIRoleplay.exe!ggml_backend_registry::ggml_backend_registry() + 0x5F bytes
    AIRoleplay.exe!ggml_backend_register() + 0x25A bytes
    AIRoleplay.exe!ggml_backend_dev_count() + 0x9 bytes
    AIRoleplay.exe!llama_print_system_info() + 0x583 bytes
    AIRoleplay.exe!llama_model_load_from_file() + 0x90 bytes
Data:
  90 7D D3 9E    80 01 00 00                                   .}...... ........

---------- Block 11593 at 0x00000000A50D4100: 32 bytes ----------
  Leak Hash: 0xBC55F72B, Count: 1, Total 32 bytes
  Call Stack (TID 22032):
    ntdll.dll!RtlAllocateHeap()
    D:\a\_work\1\s\src\vctools\crt\vcstartup\src\heap\new_scalar.cpp (36): AIRoleplay.exe!operator new() + 0xA bytes
    C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.43.34808\include\xmemory (137): AIRoleplay.exe!std::_Default_allocate_traits::_Allocate()
    C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.43.34808\include\xmemory (258): AIRoleplay.exe!std::_Allocate<16,std::_Default_allocate_traits>()
    C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.43.34808\include\xmemory (987): AIRoleplay.exe!std::allocator<char>::allocate()
    C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.43.34808\include\xmemory (2268): AIRoleplay.exe!std::_Allocate_at_least_helper<std::allocator<char> >()
    C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.43.34808\include\xstring (805): AIRoleplay.exe!std::basic_string<char,std::char_traits<char>,std::allocator<char> >::_Allocate_for_capacity<0>() + 0xF bytes
    C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.43.34808\include\xstring (2969): AIRoleplay.exe!std::basic_string<char,std::char_traits<char>,std::allocator<char> >::_Reallocate_for<`std::basic_string<char,std::char_traits<char>,std::allocator<char> >::assign'::`2'::<lambda_1>,char const *>() + 0xF bytes
    C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.43.34808\include\xstring (1611): AIRoleplay.exe!std::basic_string<char,std::char_traits<char>,std::allocator<char> >::assign()
    C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.43.34808\include\xstring (1615): AIRoleplay.exe!std::basic_string<char,std::char_traits<char>,std::allocator<char> >::assign()
    C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.43.34808\include\xstring (1423): AIRoleplay.exe!std::basic_string<char,std::char_traits<char>,std::allocator<char> >::operator=()
    E:\Dependencies\ai.cpp\ggml\src\ggml-cuda\ggml-cuda.cu (3461): AIRoleplay.exe!ggml_backend_cuda_reg() + 0x19 bytes
    AIRoleplay.exe!ggml_backend_registry::ggml_backend_registry() + 0x5F bytes
    AIRoleplay.exe!ggml_backend_register() + 0x25A bytes
    AIRoleplay.exe!ggml_backend_dev_count() + 0x9 bytes
    AIRoleplay.exe!llama_print_system_info() + 0x583 bytes
    AIRoleplay.exe!llama_model_load_from_file() + 0x90 bytes
 Data:
   4E 56 49 44    49 41 20 47    65 46 6F 72    63 65 20 52     NVIDIA.G eForce.R
   54 58 20 34    30 38 30 00    CD CD CD CD    CD CD CD CD     TX.4080. ........

---------- Block 10200651 at 0x000000000B2A9EF0: 48 bytes ----------
  Leak Hash: 0x6889E147, Count: 1, Total 48 bytes
  Call Stack (TID 22032):
    ntdll.dll!RtlAllocateHeap()
    D:\a\_work\1\s\src\vctools\crt\vcstartup\src\heap\new_scalar.cpp (36): AIRoleplay.exe!operator new() + 0xA bytes
    E:\Dependencies\ai.cpp\ggml\src\ggml-cuda\ggml-cuda.cu (713): AIRoleplay.exe!ggml_backend_cuda_buffer_type() + 0x50 bytes
    E:\Dependencies\ai.cpp\ggml\src\ggml-cuda\ggml-cuda.cu (2944): AIRoleplay.exe!ggml_backend_cuda_device_get_buffer_type()
    AIRoleplay.exe!ggml_backend_dev_buffer_type() + 0x20 bytes
    AIRoleplay.exe!llama_internal_get_tensor_map() + 0x10A7 bytes
    AIRoleplay.exe!llama_model::load_tensors() + 0x205 bytes
    AIRoleplay.exe!llama_print_system_info() + 0x36C bytes
    AIRoleplay.exe!llama_print_system_info() + 0xA3C bytes
    AIRoleplay.exe!llama_model_load_from_file() + 0x90 bytes
  Data:
    00 00 00 00    CD CD CD CD    C0 59 E5 14    81 01 00 00     ........ .Y......
    43 55 44 41    30 00 00 00    00 00 00 00    00 00 00 00     CUDA0... ........
    05 00 00 00    00 00 00 00    0F 00 00 00    00 00 00 00     ........ ........

---------- Block 10200653 at 0x0000000014E559C0: 16 bytes ----------
  Leak Hash: 0xBA21D9C7, Count: 1, Total 16 bytes
  Call Stack (TID 22032):
    ntdll.dll!RtlAllocateHeap()
    D:\a\_work\1\s\src\vctools\crt\vcstartup\src\heap\new_scalar.cpp (36): AIRoleplay.exe!operator new() + 0xA bytes
    C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.43.34808\include\xmemory (137): AIRoleplay.exe!std::_Default_allocate_traits::_Allocate()
    C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.43.34808\include\xmemory (258): AIRoleplay.exe!std::_Allocate<16,std::_Default_allocate_traits>()
    C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.43.34808\include\xmemory (987): AIRoleplay.exe!std::allocator<std::_Container_proxy>::allocate()
    C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.43.34808\include\xmemory (1216): AIRoleplay.exe!std::_Container_base12::_Alloc_proxy<std::allocator<std::_Container_proxy> >() + 0xF bytes
    C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.43.34808\include\xstring (1005): AIRoleplay.exe!std::basic_string<char,std::char_traits<char>,std::allocator<char> >::basic_string<char,std::char_traits<char>,std::allocator<char> >() + 0x2E bytes
    C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.43.34808\include\xstring (3198): AIRoleplay.exe!std::operator+<char,std::char_traits<char>,std::allocator<char> >() + 0x26 bytes
    E:\Dependencies\ai.cpp\ggml\src\ggml-cuda\ggml-cuda.cu (713): AIRoleplay.exe!ggml_backend_cuda_buffer_type() + 0xBF bytes
    E:\Dependencies\ai.cpp\ggml\src\ggml-cuda\ggml-cuda.cu (2944): AIRoleplay.exe!ggml_backend_cuda_device_get_buffer_type()
    AIRoleplay.exe!ggml_backend_dev_buffer_type() + 0x20 bytes
    AIRoleplay.exe!llama_internal_get_tensor_map() + 0x10A7 bytes
    AIRoleplay.exe!llama_model::load_tensors() + 0x205 bytes
    AIRoleplay.exe!llama_print_system_info() + 0x36C bytes
    AIRoleplay.exe!llama_print_system_info() + 0xA3C bytes
    AIRoleplay.exe!llama_model_load_from_file() + 0x90 bytes
Data:
  F8 9E 2A 0B    81 01 00 00    00 00 00 00    00 00 00 00     ..*..... ........

---------- Block 10200643 at 0x0000000014E56640: 8 bytes ----------
  Leak Hash: 0x93E3F1D6, Count: 1, Total 8 bytes
  Call Stack (TID 22032):
    ntdll.dll!RtlAllocateHeap()
    D:\a\_work\1\s\src\vctools\crt\vcstartup\src\heap\new_scalar.cpp (36): AIRoleplay.exe!operator new() + 0xA bytes
    AIRoleplay.exe!ggml_backend_cpu_aarch64_buffer_type() + 0x64 bytes
    AIRoleplay.exe!ggml_backend_cpu_get_extra_buffers_type() + 0xB6 bytes
    AIRoleplay.exe!ggml_backend_cpu_get_extra_buffers_type() + 0x62 bytes
    AIRoleplay.exe!ggml_backend_cpu_get_extra_buffers_type() + 0x13E bytes
    AIRoleplay.exe!llama_internal_get_tensor_map() + 0xEDC bytes
    AIRoleplay.exe!llama_model::load_tensors() + 0x109 bytes
    AIRoleplay.exe!llama_print_system_info() + 0x36C bytes
    AIRoleplay.exe!llama_print_system_info() + 0xA3C bytes
    AIRoleplay.exe!llama_model_load_from_file() + 0x90 bytes
Data:
  90 8D 07 92    F7 7F 00 00                                   ........ ........

The CRT library classic output (another execution):

Detected memory leaks!
Dumping objects ->
{10179386} normal block at 0x0000018EBE7F3950, 16 bytes long.
 Data: < r              > 98 72 A9 BD 8E 01 00 00 00 00 00 00 00 00 00 00 
{10179384} normal block at 0x0000018EBDA97290, 48 bytes long.
 Data: <        P9      > 00 00 00 00 CD CD CD CD 50 39 7F BE 8E 01 00 00 
{10179376} normal block at 0x0000018EBE7F3CC0, 8 bytes long.
 Data: <  )     > 90 8D 29 C7 F6 7F 00 00 
{7792} normal block at 0x0000018E9132B3F0, 8 bytes long.
 Data: <        > F0 07 F0 90 8E 01 00 00 
{7791} normal block at 0x0000018E90F007F0, 136 bytes long.
 Data: <@               > 40 14 E8 C5 F6 7F 00 00 80 14 E8 C5 F6 7F 00 00 
{7790} normal block at 0x0000018E912A1FF0, 32 bytes long.
 Data: <NVIDIA GeForce R> 4E 56 49 44 49 41 20 47 65 46 6F 72 63 65 20 52 
{7787} normal block at 0x0000018E9132B490, 16 bytes long.
 Data: <                > D0 AD FC 90 8E 01 00 00 00 00 00 00 00 00 00 00 
{7786} normal block at 0x0000018E9132B3A0, 16 bytes long.
 Data: <                > A8 AD FC 90 8E 01 00 00 00 00 00 00 00 00 00 00 
{7785} normal block at 0x0000018E90FCADA0, 88 bytes long.
 Data: <          2     > 00 00 00 00 CD CD CD CD A0 B3 32 91 8E 01 00 00 
{7783} normal block at 0x0000018E90E56420, 16 bytes long.
 Data: <P *             > 50 1D 2A 91 8E 01 00 00 00 00 00 00 00 00 00 00 
{7782} normal block at 0x0000018E912A1D50, 32 bytes long.
 Data: < d        2     > 20 64 E5 90 8E 01 00 00 F0 B3 32 91 8E 01 00 00 
Object dump complete.

I do not have found a way to release myself the memory because I cannot access to the underlaying ggml_backend_reg * of ggml_backend_reg_t that is hide in the ggml-backend-impl.h and not more accessible directly after building the static libraries.

I don't know if it's enough, but a code like that can eventually do the job:

ggml_backend_reg_t reg = ggml_backend_reg_by_name("CUDA");
free(reg->context);

Unfortunatly:
E0833 pointer or reference to incomplete type not allowed !

I hope that help.

Thank's a lot for ggml/llama.cpp ! You rock !

First Bad Commit

No response

Relevant log output


      		
    

      
  





        



            
            

              
            

        

      


      
    








      


  




  
  

              

  
    Sign up for free
    to join this conversation on GitHub.
    Already have an account?
    Sign in to comment


  



  






  
                  




    

  


      
  

    Assignees
  



        

    No one assigned







      


  


  

    Labels
  



    

      

    bug-unconfirmed









      

  

    

        

    Projects
  


        




    None yet




  



      


  

    
  

    Milestone
  


      No milestone





    
      
          


  

    

      
        

          
    

    Development
  




            
    
No branches or pull requests







      
    

  




      

    

  
      

  

    

      1 participant
    

    

        
          @dorpxam