cuda error

[neverl805]

\llama.cpp\ggml\src\ggml-cuda\ggml-cuda.cu:75: CUDA error

[neverl805]

Sometimes it runs fine, but sometimes there will be an error message like \llama.cpp\ggml\src\ggml-cuda\ggml-cuda.cu:75: CUDA error. How to solve this? I'm using a 4070ti super GPU. I checked the VRAM and it seems sufficient. Could it be that the CUDA version is too high? It's 12.8.

[giladgd]

Are there more details when this error occurs?
Try using getLlama({debug: true}) to see extremely verbose logs which might provide more context when this error happens.

My quick guess is that the GPU might not be connected properly (physically), so a slight movement might disconnect something momentarily, which would be enough to trigger this issue occasionally.

If you build from source using this command, do you still get this issue occasionally?

npx --no node-llama-cpp source download