Memory leak with Arc / Orc for allocation inside threads

[eagledot]

Description

Following code leads to a very high memory usage for memory allocated with either alloc or allocShared inside threads when arc / orc is used as mm option.

Using -d:useMalloc leads to expected behaviour !

when defined(windows):
    import std/winlean

var
   thr: array[2, Thread[tuple[a,b,: int]]]

proc threadFunc(interval: tuple[a,b: int]) {.thread.} =
    for i in interval.a..interval.b:
        let memory = allocShared(2048*4)
        
        # some work simulation.
        var sum = 0
        for i in 0..<2000:
            cast[ptr UncheckedArray[float32]](memory)[i] = float32(i)
        
        deallocShared(memory) # deallocate here.

proc some_function()=
    for i in 0..high(thr):
        createThread(thr[i], threadFunc, (i*10, i*10+5))
    joinThreads(thr)

    when defined(windows):
        for i in 0..high(thr):
            discard closeHandle(cast[Handle](thr[i].sys))

for i in 0..<1000000:
    some_function()

Nim Version

Nim Compiler Version 2.0.2 [Windows: amd64]
Compiled at 2023-12-15
Copyright (c) 2006-2023 by Andreas Rumpf

active boot switches: -d:release

(Same behaviour with nim 1.6.8)

Current Output

No response

Expected Output

Memory should be freed as expected when using `dealloc` or `deallocShared`.

Possible Solution

No response

Additional Information

No response

[simonkrauter]

I found the same or a similar memory leak.
My program uses threads too, but does not call allocShared explicitly.
I tested with Nim 2.0.2 on Linux amd64.

My minimal example:

import std / json

proc createJsonData(): string =
  for i in 1..50_000:
    if i == 1:
      result.add("{")
    else:
      result.add(",")
    result.add("\"foo")
    result.add($i)
    result.add("\":0")
  result.add("}")

type
  Request = ref object
    headers: seq[string]

var headers: seq[string]
headers.add("entry1")

proc threadFunc(req: Request) {.thread.} =
  req.headers.add("entry2")

proc callThread() =
  var req = new Request
  req.headers = headers

  var thread: Thread[Request]
  createThread(thread, threadFunc, req)
  joinThreads(thread)

for i in 1..10:
  callThread()

  discard createJsonData().parseJson()

  echo getOccupiedMem()

Output with arc/orc:

10739056
10739088
10739120
10739152
10739184
10739216
10739248
10739280
10739312
10739344

Output with refc:

13408304
15581280
14993520
14994768
14996688
14997072
14998896
14999472
14978928
14979888

Output with arc/orc and useMalloc:

0
0
0
0
0
0
0
0
0
0

[simonkrauter]

The example program above still fails with Nim 2.2.4.

[Araq]

The program is invalid though, when you do req.headers.add("entry2") in a thread, you should have used a lock around it afaict.