Skip to content

video: merge the tiles of grid heif and avif images#17738

Open
guidocella wants to merge 1 commit intompv-player:masterfrom
guidocella:tile
Open

video: merge the tiles of grid heif and avif images#17738
guidocella wants to merge 1 commit intompv-player:masterfrom
guidocella:tile

Conversation

@guidocella
Copy link
Copy Markdown
Contributor

@guidocella guidocella commented Apr 11, 2026

Fixes #13585.
Fixes #16486.

Switching between different grid images added with --external-files is also supported.

The prority of independent tracks is reverted for images, because we now want to select a dependent track to trigger the merging, rather than small preview tracks.

Mostly generated by Claude.

--hwdec=vaapi works for regular heif images, but breaks grid ones. nvdec works. I get CUDA_ERROR_OUT_OF_MEMORY but at least it fallsback to software decoding.
I can't test av1, but hardware decoding already doesn't work at least for HDR avif according to #14062.

Comment thread player/video.c
return 0;
}

static char *tile_grid_graph(void *ctx, const struct mp_tile_grid *grid)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All of these things belong to f_auto_filters.c, not here.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added an auto filter at first but it doesn't seem to be able to handle different streams.

@Andarwinux
Copy link
Copy Markdown
Contributor

I tested this PR. If the preferred hwdec is not a copy variant (--hwdec=d3d11va,d3d11va-copy), grid images fail to load entirely, and it is unable to failback to a copy hwdec and reload.

I can't test av1, but hardware decoding already doesn't work at least for HDR avif according to #14062.

MPV-00_00_00 000-N0001

I suspect the problem in #14062 is that the pixel format isn't yuv420, and currently there are no GPUs capable of yuv422/444 av1.

@guidocella
Copy link
Copy Markdown
Contributor Author

vaapi-copy also works.

@kasper93
Copy link
Copy Markdown
Member

kasper93 commented Apr 12, 2026

This should be merged after scaling individual tiles, preferably in libplacebo. Otherwise big images won't work.

EDIT: But after discussion, we don't care about big images > 16k, so it's not needed.

@guidocella
Copy link
Copy Markdown
Contributor Author

guidocella commented Apr 12, 2026

Is that related to tile grids? Non-grid large images don't work all the same. Also changing libplacebo won't work for dmabuf-wayland.

@llyyr
Copy link
Copy Markdown
Contributor

llyyr commented Apr 12, 2026

Also changing libplacebo won't work for dmabuf-wayland.

Nothing will, you need to make compositor support rendering larger textures or split textures in dmabuf-wayland and make compositor stitch it

Comment thread player/video.c
Comment thread demux/demux_lavf.c Outdated

// For a single-tile image the assembled canvas may be larger than
// the display area due to codec alignment padding. Encode that as a
// codec-level crop so mpv's normal pipeline trims it.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why this is needed for single tile? And not for multi-tile, when the tiles could overshoot the image display area in similar way?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tile_grid_graph sets the crop filter with multiple tiles.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess we can just always set a filter even with 1 tile? Not like images ever have a single tile in practice.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed the special casing of nb_tiles == 1.

Comment thread demux/stheader.h Outdated
Comment thread player/video.c Outdated

struct mp_tile_grid *grid = track->stream->tile_grid;
if (grid && grid->nb_tiles > 1) {
// Prevent crashing with heif-conformance/C021.heic in FATE.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is FATE? Needs to be more specific.

Also this looks like another workaround to something that should be handled better, you handle similar case for single-tile images above.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you prefer to not set mp_tile_grid beforehand in demux_lavf.c if invalid?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure, but this code belong in lavf generally. There is no reason to handle it here.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved to demux_lavf.c.

Fixes mpv-player#13585.
Fixes mpv-player#16486.

Switching between different grid images added with --external-files is
also supported.

The prority of independent tracks is reverted for images, because we now
want to select a dependent track to trigger the merging, rather than
small preview tracks.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

AVIF Grid images are not decoded correctly mpv can't properly parse tile HEIF still images

5 participants