030 DEVICE_HEAP_ERROR codal error on long-running code using neopixel and display

[kevinjwalters]

neopixel-thirty-bug-repro-7.py is likely to eventually produce a 030 (DEVICE_HEAP_ERROR) on a micro:bit v2 perhap after 4-50 hours of running. It will do this without any WS2812 RGB LEDs attached, as expected.

This is on MicroPython v1.18 on 2023-10-30; micro:bit v2.1.2 with nRF52833.

Worth a second check but I don't think it has any memory leaks in the python code.

Some discussion about this in #227

I see one old issue which had unexplained 030 errors: #191

[microbit-carlos]

It's not very likely, but I wonder if the pattern list being constantly popped and appended into could be creating some memory fragmentation. If possible, would you mind converting that into a list of constant length and maybe using an index variable instead?

Could you also try commenting out the Neopixel code and see if that still triggers it? If it still happens there it points at maybe the leak being in the neopixel CODAL layer.

[kevinjwalters]

If there was a failure to allocate memory wouldn't that create a MemoryError exception? What expected cases are there for an 030 to be produced?

I may be out of touch for how GC works in MicroPython/CircuitPython. I thought it didn't have compaction but I'm wondering if it has segregated areas for small and large objects or heuristics/hints for short and long lived objects?

[microbit-carlos]

I was able to replicate this with the original code and the current MicroPython in the micro:bit Python Editor.
Creating a Neopixel stub that doesn't send anything to the PWM pin (and removing the built-in MicroPython import), and I can confirm the program doesn't crash after running all night.

This does point at the issue being in the CODAL neopixel implementation.

If there was a failure to allocate memory wouldn't that create a MemoryError exception? What expected cases are there for an 030 to be produced?

Yeah, I think you are right, that should have raised a MicroPython memory error instead.

I may be out of touch for how GC works in MicroPython/CircuitPython. I thought it didn't have compaction but I'm wondering if it has segregated areas for small and large objects or heuristics/hints for short and long lived objects?

I know CircuitPython did this a few years ago, I'm not sure if MicroPython has done the same or something similar.

[kevinjwalters]

It's not very likely, but I wonder if the pattern list being constantly popped and appended into could be creating some memory fragmentation. If possible, would you mind converting that into a list of constant length and maybe using an index variable instead?

@microbit-carlos That's actually how my real application code evolved. When I was doing a lot of list growing and shrinking I got MemoryErrors like this so I switched to a flattened array.array. I didn't think much about it at the time but the %u is weird.

Traceback (most recent call last):
  File "main.py", line 52, in <module>
  File "zc_bg_digitalrain.py", line 19, in render
MemoryError: memory allocation failed, allocating %u bytes