Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Crash previewing SVG with large dimensions #2562

Closed
3 tasks done
krir17 opened this issue Apr 2, 2025 · 21 comments · Fixed by #2564
Closed
3 tasks done

Crash previewing SVG with large dimensions #2562

krir17 opened this issue Apr 2, 2025 · 21 comments · Fixed by #2564
Labels
bug Something isn't working

Comments

@krir17
Copy link

krir17 commented Apr 2, 2025 •
edited
Loading

What system are you running Yazi on?

Linux Wayland

What terminal are you running Yazi in?

kitty 0.36.4

yazi --debug output

Yazi
    Version: 25.3.7 (1765aba 2025-03-30)
    Debug  : false
    Triple : x86_64-unknown-linux-gnu (linux-x86_64)
    Rustc  : 1.85.1 (4eb16125 2025-03-15)

Ya
    Version: 25.3.7 (1765aba 2025-03-30)

Emulator
    TERM                : Some("xterm-256color")
    TERM_PROGRAM        : None
    TERM_PROGRAM_VERSION: None
    Brand.from_env      : None
    Emulator.detect     : Emulator { kind: Right(Unknown { kgp: false, sixel: false }), light: false, cell_size: None }

Adapter
    Adapter.matches    : Wayland
    Dimension.available: WindowSize { rows: 51, columns: 237, width: 0, height: 0 }

Desktop
    XDG_SESSION_TYPE           : Some("wayland")
    WAYLAND_DISPLAY            : Some("wayland-1")
    DISPLAY                    : Some(":0")
    SWAYSOCK                   : Some("/run/user/1000/sway-ipc.1000.458567.sock")
    HYPRLAND_INSTANCE_SIGNATURE: None
    WAYFIRE_SOCKET             : None

SSH
    shared.in_ssh_connection: false

WSL
    WSL: false

Variables
    SHELL           : Some("/bin/bash")
    EDITOR          : Some("nvim")
    VISUAL          : Some("nvim")
    YAZI_FILE_ONE   : None
    YAZI_CONFIG_HOME: None
    YAZI_ZOXIDE_OPTS: None
    FZF_DEFAULT_OPTS: None

Text Opener
    default     : Some(OpenerRule { run: "${EDITOR:-vi} \"$@\"", block: true, orphan: false, desc: "$EDITOR", for_: None, spread: true })
    block-create: Some(OpenerRule { run: "${EDITOR:-vi} \"$@\"", block: true, orphan: false, desc: "$EDITOR", for_: None, spread: true })
    block-rename: Some(OpenerRule { run: "${EDITOR:-vi} \"$@\"", block: true, orphan: false, desc: "$EDITOR", for_: None, spread: true })

Multiplexers
    TMUX               : false
    tmux version       : tmux 3.5a
    tmux build flags   : enable-sixel=Unknown
    ZELLIJ_SESSION_NAME: None
    Zellij version     : No such file or directory (os error 2)

Dependencies
    file          : 5.45
    ueberzugpp    : No such file or directory (os error 2)
    ffmpeg/ffprobe: 7.0.2 / 7.0.2
    pdftoppm      : 24.08.0
    magick        : 7.1.1-41
    fzf           : 0.54.3
    fd/fdfind     : 10.1.0 / No such file or directory (os error 2)
    rg            : 14.1.1
    chafa         : 1.14.2
    zoxide        : No such file or directory (os error 2)
    7zz/7z        : No such file or directory (os error 2) / 16.02
    jq            : 1.7.1

Clipboard
    wl-copy/paste: 2.2.1 / 2.2.1
    xclip        : 0.13
    xsel         : 1.2.1

Routine
    `file -bL --mime-type`: text/plain


See https://yazi-rs.github.io/docs/plugins/overview#debugging on how to enable logging or debug runtime errors.

Describe the bug

While previewing a large (in dimensions, the file size can be tiny) SVG, (1) memory usage grows rapidly and (2) the / directory starts becoming filled, causing the system to crash. With all preloaders and previewers disabled, no crash occurs.

Minimal reproducer

(Note that this might crash the entire and system and potentially cause data loss)

  1. Save brick_system.svg to a local folder
  2. Monitor memory/disk usage (eg. using btop)
  3. Open yazi and try to preview the attached file

Anything else?

This is a partial duplicate of #2345. The memory issue is obviously on IM's side:

magick -density 200 brick_system.svg -flatten -resize 200x200 -auto-orient out.jpg takes ~13 GB of RAM on my system. Interestingly, setting MAGICK_MEMORY_LIMIT=10000 has no effect on this - and as a result, setting image_alloc doesn't help either. In both cases, magick produces a broken JPEG file but 'works'.

When yazi previews the file, something causes systemd-coredump to write >0.5 GBs (!) of core dumps to / every second (see attached video), crashing the entire session in ~10-20 seconds. This might be due to a crash in magick which doesn't happen when run directly.

Screencast of disk usage
yaziClip.mp4

TBH I expect none of this to be yazi's fault - and the attached SVG file is a bit extreme (although it is a info-stripped version of a real SVG file) - but it would be nice to prevent users from inadvertently running into this. I think using rsvg-convert over magick might be better (also for other reasons)... but that's probably a feature request.

Checklist

  • I tried the latest nightly build, and the issue is still reproducible
  • I updated the debug information (yazi --debug) input box to the nightly that I tried
  • I can reproduce it after disabling all custom configs/plugins (mv ~/.config/yazi ~/.config/yazi-backup)
@krir17 krir17 added the bug Something isn't working label Apr 2, 2025
@github-actions GitHub Actions
Copy link

github-actions bot commented Apr 2, 2025

Hey @krir17, I noticed that you did not correctly follow the issue template. Please ensure that:

  • The bug can still be reproduced on the newest nightly build.
  • The debug information (yazi --debug) is updated for the newest nightly.
  • The non-optional items in the checklist are checked.

Issues with needs info will be marked ready once edited with the proper content, or closed after 2 days of inactivity.

@ze0987
Copy link
Contributor

ze0987 commented Apr 2, 2025

I expect none of this to be yazi's fault

Actually, it is, because it tells magick to render document size, at 200 PPI, in this case it's 700 x 100 cm! That's about 6.4 GiB of RSS and a memory allocation error here when running magick -density 200 input.svg output.jpg (without transformations!). Another matter is the decoding delegate. It looks like magick will choose inkscape, if available.

Could you please test: magick -size 200x200 rsvg:brick_system.svg out.jpg if you have magick build with librsvg (magick --version | grep rsvg)? This should require about 25 MiB of RAM and a fraction of a second.

@krir17
Copy link
Author

krir17 commented Apr 3, 2025

Could you please test: magick -size 200x200 rsvg:brick_system.svg out.jpg if you have magick build with librsvg (magick --version | grep rsvg)? This should require about 25 MiB of RAM and a fraction of a second.

That takes about 26 ms and 20 MB of RAM; with inkscape it's a little over 6 seconds and >2 GBs.

@sxyazi
Copy link
Owner

sxyazi commented Apr 3, 2025

It looks like for some reason, ImageMagick is not complying MAGICK_MEMORY_LIMIT - I expected after setting it, in the worst case, even if rendering fails, it should not exceed the memory limit causing an OOM - this is also the strategy of the builtin image decoder.

At the moment, it is unclear how to properly limit the memory usage of ImageMagick during SVG rendering, I think a temporary fix would be to remove -density, which should help somewhat in mitigating the issue, although it might result in a loss of quality.

I plan to explore alternative options to ImageMagick in the future, and currently, resvg looks like a promising choice since it is cross-platform, and its SVG support seems good.

@sxyazi sxyazi mentioned this issue Apr 3, 2025
Merged
@sxyazi
Copy link
Owner

sxyazi commented Apr 3, 2025

Please try #2564

@krir17
Copy link
Author

krir17 commented Apr 3, 2025 •
edited
Loading

Please try #2564

Thanks - that does fix it for this file, but still fails when it encounters this inane file: image_10k.svg --- rsvg in this case bails out before doing anything while Inkscape churns out a huge SVG file anyway. I think a better short term fix might be to get magick to use rsvg even if Inkscape is present? Although if the intention is to move to a different renderer, this should be fine for now.

(Edit: Actually, it's probably not Inkscape's fault either: inkscape image_10k.svg --export-height=200 --export-filename=out.png works just fine. It's just that magick tries to make Inkscape/rsvg render a file that's >30k pixels wide and then shrink it.)

@sxyazi
Copy link
Owner

sxyazi commented Apr 3, 2025 via email

@krir17
Copy link
Author

krir17 commented Apr 3, 2025

ImageMagick to use rsvg — I haven't found such a parameter

Copying from @ze0987's comment:

magick rsvg:brick_sys.svg -flatten -resize 200x200 -auto-orient out.jpg doesn't seem to use Inkscape (the filename is prepended with rsvg:).

@sxyazi
Copy link
Owner

sxyazi commented Apr 3, 2025 via email

@krir17
Copy link
Author

krir17 commented Apr 3, 2025 •
edited
Loading

Unfortunately I couldn't find a definitive soruce, except for: https://stackoverflow.com/a/46586730 and maybe https://usage.imagemagick.org/files/

For a quick test, creating an inkscape executable in my local directory, with just echo "Inkscape is Running" and running the following commands

[krir17@fedora test_yazi]$ PATH=$(pwd):$PATH magick rsvg:new.svg out.jpg
[krir17@fedora test_yazi]$ PATH=$(pwd):$PATH magick new.svg out.jpg
Inkscape is running
[krir17@fedora test_yazi]$ PATH=$(pwd):$PATH magick wtvr:new.svg out.jpg
Inkscape is running

does suggest that it uses inkscape unless the rsvg delegate is specified

@sxyazi
Copy link
Owner

sxyazi commented Apr 3, 2025 via email

@krir17
Copy link
Author

krir17 commented Apr 3, 2025 •
edited
Loading

After forcing ImageMagick to use rsvg via rsvg:, does the -density 200 option still lead to an OOM? How do different -density parameters
perform with the rsvg backend?

Oh I just realized that the performance improvement observed in #2562 (comment) comes not from removing the density parameter but from the use of -size 200x200 instead of -resize 200x200. With resize I still end up with the OOM issue (and both Inkscape and rsvg perform similarly w.r.t memory). However, it looks like -size works only with the rsvg backend, so the inkscape backend still leads to an OOM.

Some performance tests: (mmem is in KiB)

[krir17@fedora test_yazi]$ /usr/bin/time magick -size 200x200 rsvg:brick_sys.svg -density 200 -flatten out.jpg
	0:00.02 real,	0.01 user,	0.01 sys,	0 amem,	19052 mmem
[krir17@fedora test_yazi]$ /usr/bin/time magick -size 200x200 rsvg:image_10k.svg -density 200 -flatten out.jpg
	0:00.03 real,	0.01 user,	0.01 sys,	0 amem,	19684 mmem
[krir17@fedora test_yazi]$ /usr/bin/time magick -size 200x200 brick_sys.svg -density 200 -flatten out.jpg
	0:10.29 real,	15.91 user,	8.43 sys,	0 amem,	3139204 mmem
[krir17@fedora test_yazi]$ /usr/bin/time magick -size 200x200 image_10k.svg -density 200 -flatten out.jpg
Command terminated by signal 9 (killed before OOM)
	1:19.26 real,	41.30 user,	36.32 sys,	0 amem,	1178300 mmem

But with the size specified beforehand, a higher density isn't necessary since rsvg takes care of that.

Density Tests

SVG file: Image

magick -size 200x200 rsvg:circleBug.svg -flatten out.jpg

Image

magick rsvg:circleBug.svg -flatten -resize 200x200 out_resized.jpg

Image

If you specify rsvg: but haven't installed rsvg and only have inkscape
installed, what will its behavior be? Will it fall back to ImageMagick's
default SVG backend or to inkscape?

Actually, not sure about this. I would expect it 'falls back' to inksape, but I can't test it quickly (without recompiling magick and uninstalling rsvg)

@sxyazi
Copy link
Owner

sxyazi commented Apr 3, 2025 via email

@krir17
Copy link
Author

krir17 commented Apr 3, 2025

Please try
b725b86
…

That works perfectly - thank you!

@ze0987
Copy link
Contributor

ze0987 commented Apr 3, 2025 •
edited
Loading

Oh I just realized that the performance improvement observed in #2562 (comment) comes not from removing the density parameter but from the use of -size 200x200 instead of -resize 200x200

Exactly. With -size magick w/ rsvg decoder always renders at the specified thumbnail size and not at the document size as before. In addition, we skip the expensive scaling (-resize), and the quality of the output is always perfect, even when the source is much smaller than the thumbnail size. Here is another comparison: #2533 (comment)

@ze0987
Copy link
Contributor

ze0987 commented Apr 6, 2025

I would expect it 'falls back' to inksape, but I can't test it quickly (without recompiling magick and uninstalling rsvg)

I checked it out and it doesn't look good.

Source file:

poster.svg
File Size: 475 kB
Image Size: 10000x10000

imagemagick w/o rsvg w/ inkscape installed:

magick -size 600x900 rsvg:poster.svg -strip -quality 75 thumbnail.jpg

Time: 1m10,122s
RAM: 2.755 GB
Thumbnail File Size: 1309 kB
Thumbnail Image Size: 10000x10000

imagemagick w/ rsvg w/ inkscape installed:

magick -size 600x900 rsvg:poster.svg -strip -quality 75 thumbnail.jpg

Time: 0m0,157s
RAM: 43.76 MB
Thumbnail File Size: 20 kB
Tumbnail Image Size: 600x600

@sxyazi
Copy link
Owner

sxyazi commented Apr 6, 2025 via email

@ze0987
Copy link
Contributor

ze0987 commented Apr 6, 2025

Yes, of course.

@sxyazi
Copy link
Owner

sxyazi commented Apr 6, 2025 via email

@ze0987
Copy link
Contributor

ze0987 commented Apr 6, 2025 •
edited
Loading

user doesn't have rsvg installed

To clarify. “imagemagick w/ rsvg” meant imagemagick built with --with-rsvg, with librsvg as a build dependency. librsvg comes with rsvg-convert, a tool that can be used independently of magick (does not require imagemagick). For example: rsvg-convert -w 600 -h 900 -a -o output input

@sxyazi sxyazi mentioned this issue Apr 7, 2025
Merged
@sxyazi
Copy link
Owner

sxyazi commented Apr 7, 2025

Please try #2581

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix: remove -density option from magick command in SVG previewer sxyazi/yazi
3 participants
@sxyazi @ze0987 @krir17