-
Notifications
You must be signed in to change notification settings - Fork 2.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Responsive images causes performance issues on high-resolution devices #4421
Comments
^ This. IMHO, the spec is about ar art directionish as it should be and all this should be left to the browsers. But I wholeheartedly agree that this needs to be adressed. |
I've discussed this with @mor10 last week. It seems like the ubiquity of high resolution screens has made it so that there's a need for a cap on the DPR levels browsers take into account, either implicitly or explicitly. From my perspective, the main hurdle towards an on-by-default cap, automatically enforced by the browser, is the lack of data on the cut-off ratio, where higher-resolution doesn't necessarily mean better user-experience. If such data could be provided, that'd be helpful on that front. Barring that, the fastest route towards a solution here would be an opt-in cap (e.g. a Regarding bandwidth based restrictions, it's possible that @tarunban has looked into that. /cc @tabatkins @eeeps |
If memory serves, most browsers used to cap the selection process at 2x. If browsers are no longer specifically doing this, then I would agree that a markup solution would be nice so that developers could control the resolution caps themselves. I would suggest doing this via a meta value that applied to the whole page, rather than on an individual image basis, but I could see a use case for both options. |
The entire point of this feature is that browsers should be smart enough to be able to intelligently choose the "best" resolution to download; they have access to more information than the page author does, and whatever heuristics they come up with to intelligently pick between options get automatically applied to every page using this feature, rather than only being for whatever tiny % of people use a particularly good image-choosing JS library. If browsers aren't making good decisions, that's a bug on browsers. There shouldn't be any need for the page to further intervene here. It's definitely plausible that browsers are being naive here, so please file bugs showing bad results. ^_^ |
First of all: I agree with @tabatkins , the spec gives browsers great power here, and with that power comes all of the responsibility. The bugs belong with them. While you're all here though... I barfed out some initial research and thoughts re: Hi-DPR’s diminishing quality returns. https://observablehq.com/@eeeps/visual-acuity-and-device-pixel-ratio TL;DR @joemcgill I don't remember any browser limiting DPR to 2, but by the looks of it, maybe that's not such a terrible rule-of-thumb to start with. |
I too agree browsers should be doing this, but from what I see in the real world, they are not. Which gives us an impossible dilemma of either serving gigantic images for fancy 5x screens and making people on expensive data plans pay for image data they neither want nor need, or serving too-small images to all devices causing people on larger screens to question their eyesight. How do we move forward in a constructive way here? Is the next step reaching out to browser manufacturers and asking them where bandwidth / dataplan throttling is in the pipeline? Just to reiterate, this issue is currently causing 33% of the web to receive incorrectly sized images via WordPress (through no fault of WordPress). Finding a path toward a solution is non-trivial to the overall UX of the web. |
@eeeps - That's amazing work. Thank you for that!! @tabatkins - thoughts on the above and the conclusion? Should browsers use that to cut-off DPR calculations at ~2x? Higher? |
@mor10 - you already have 😺 Once Chrome ships this and proves that it's useful behavior, I suspect it won't take too long for other browser vendors to follow. |
As an aside, I do think that while a default should be spec'ed, that this should be overridden by the UA and/or markup. I can imagine scenarios where the UA might desire to have a max-DPR that is non-standard and >2x. I think of TVs as an example. While mobile devices (phones, laptops, watches) have a generally accepted interaction distance, TVs have a less standard use. Home viewing is one thing, but dashboards in the office or campus, or even signage in a store or trade-hall have very different viewing and interaction distances. For this reason I could reasonably see a TV manufacture wanting a 5x DPR to compensate for the viewing and interaction distances. |
I'm not sure we even want to specify a default, although we may add some data-backed recommendation of a good cut-off value, which browsers can then adopt as they see fit. |
@mor10 do you have any more detailed data on the “costs” side of things? I ran your test page through Chrome dev tools on a 1024x768 1x: 839K ...which, 1.4MB for a page with three images on it is not great, but I feel like there's a better story here about the sort of widespread performance impact that WordPress is trying to avoid by limiting images to |
I'll update the example with larger image sizes. The original was put together to demonstrate the negative effects of small images on wide screens. |
@eeeps Here's a new post with an extended range of For reference, my Pixel 3 in horizontal mode now pulls down the https://gutenberg.mor10.com/responsive-images-demo-extended-srcset-range/ |
I asked for this or a similar feature already in 2014 (I think As it was shut down I created a JS plugin for lazySizes. @eeeps @yoavweiss |
For reference, the WordPress core ticket to find a solution has been punted to a future release: https://core.trac.wordpress.org/ticket/45407#comment:33 |
Hi, has something changed in this topic? |
PSA: https://groups.google.com/a/chromium.org/d/msg/blink-dev/sIGNgOAC0oc/TFST-dJxAwAJ When srcset selects images, it takes the screen density into account, but beyond a certain density, that can result in excessive downloaded images. Currently, when adding `w` descriptors to srcset, developers run a risk that large `w` descriptor images will be used on small screens with very-high density values. For example, when defining `<img srcset="300.jpg 300w, 600.jpg 600w, 1200.jpg 1200w, 2400.jpg 2400w">`, developers that define the 2400px wide image typically intend for it to be used for very large screens (e.g. a 32" 2x screen), but it often ends up also being used on small screens (e.g. a phone with 4x DPR and viewport width of 600+px) . Research[1] shows that density above 2-2.2x is not visible by most people, and therefore we could avoid wasting those bytes and avoid the related slowdown. See whatwg/html#4421 for full discussion Change-Id: I52afd505b023767883088a939083f9f835e7fa14 BUG: 1125973 [1] https://observablehq.com/@eeeps/visual-acuity-and-device-pixel-ratio Change-Id: I52afd505b023767883088a939083f9f835e7fa14 Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2395619 Reviewed-by: Christian Biesinger <[email protected]> Reviewed-by: Mason Freed <[email protected]> Commit-Queue: Yoav Weiss <[email protected]> Cr-Commit-Position: refs/heads/master@{#812698}
I can see "SrcsetMaxDensity" was committed here, but I can't understand whether it shipped or not. @yoavweiss would you be able to send an update? Thanks PS: of course using |
I've never seen a good solution with You could use What I usually do is reduce the quality of the image (increasing the compression) on high density screens, as with Compressive Images, to limit network data. But it's really really bad for memory usage on the device. Here is some simplified pseudo-code for a full viewport width image: <picture>
<source
media="(min-resolution: 3dppx)"
sizes="100vw"
srcset="
img.jpg?w=300&q=15 300w,
img.jpg?w=600&q=15 600w, …">
<img
sizes="100vw"
srcset="
img.jpg?w=300&q=auto 300w,
img.jpg?w=600&q=auto 600w, …"
alt="…">
</picture> I use
If you want to limit quality above So yes, a simple way to limit the maximum resolution the browser uses to compute the size of the required image would be awesome. |
I wrote an article about it, How to cap image fidelity to 2x and save 45% image weight on high-end mobile phones, which uses some That solution works well to cap image fidelity on smaller viewports (smartphones), and let the The thing is that solution is That's why I think that a |
That's amazing data, thanks! |
@verlok I never thought about using both Depending on the target of the site, I would still consider feature phones with a As you say, it's quite difficult to generate such code, even more if the layout changes a lot on multiple viewport breakpoints. We need |
|
Initial conversations around max-useful-resolutions were predicated on
I strongly think there should be some kind of user preference here, too. Maybe that's just some spec language mandating that browsers can always override Long term, I want all of the above, structured according to the priority of constituencies:
|
@eeeps |
More generically, I think browsers should do more in automatically selecting smaller images from a
What do you think? |
I found someone else's post on StackOverflow that explores a strategy for controlling image pixel density for mobile phones: https://stackoverflow.com/questions/69586765/responsive-images-with-srcset-sizes-media-queries-prevent-loading-huge-image The basic idea is to associate media queries of the targeted pixel densities with an equivalent size. The browser uses the "sizes" element as a guide to select the appropriate image defined by the (w)idths in the srcset element. In this example, devices with 2x (as default), 3x and 4x pixel ratios will at most display 2x the image. To serve only 1x images to 4x, 3x and 2x devices, specify "24vw", "33vw" and "50vw" in the sizes element. The advantage of this approach is that only one srcset is required.
The disadvantage is that Safari doesn't support min-resolution. Using "-webkit-min-device-pixel-ratio" as well might give broader browser support. For example, What do ya'll think of this approach? |
That could work, @SamuelMiller, but the And things would get even more complicate when having to manage different image sizes at different media queries, for real. We already have a way to cap image pixel density on high dpi devices, and it’s to use the I really long for a Even if now it’s less complicated, thanks to my responsive images automator. :) |
@yoavweiss Do you think a web standard discussion may be possible at this point? I came to the conclusion that it's hard for developers to not only cap image qualities to the right viewport size, but it's also not easy to use other signals (like Data Saver, Memory, ETC) to customize the IMG size being requested without using JS... |
I'm highly supportive of API improvements that would help developers have more control here. Unfortunately, I won't have time to work on those myself in the near future, but more than happy to guide folks through this if anyone here is interested in contributing :) |
PSA: https://groups.google.com/a/chromium.org/d/msg/blink-dev/sIGNgOAC0oc/TFST-dJxAwAJ When srcset selects images, it takes the screen density into account, but beyond a certain density, that can result in excessive downloaded images. Currently, when adding `w` descriptors to srcset, developers run a risk that large `w` descriptor images will be used on small screens with very-high density values. For example, when defining `<img srcset="300.jpg 300w, 600.jpg 600w, 1200.jpg 1200w, 2400.jpg 2400w">`, developers that define the 2400px wide image typically intend for it to be used for very large screens (e.g. a 32" 2x screen), but it often ends up also being used on small screens (e.g. a phone with 4x DPR and viewport width of 600+px) . Research[1] shows that density above 2-2.2x is not visible by most people, and therefore we could avoid wasting those bytes and avoid the related slowdown. See whatwg/html#4421 for full discussion Change-Id: I52afd505b023767883088a939083f9f835e7fa14 BUG: 1125973 [1] https://observablehq.com/@eeeps/visual-acuity-and-device-pixel-ratio Change-Id: I52afd505b023767883088a939083f9f835e7fa14 Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2395619 Reviewed-by: Christian Biesinger <[email protected]> Reviewed-by: Mason Freed <[email protected]> Commit-Queue: Yoav Weiss <[email protected]> Cr-Commit-Position: refs/heads/master@{#812698} GitOrigin-RevId: 06f97a0fcf40024ca83284dff7b6bace790aaa35
I still think a A difficulty @yoavweiss expressed when we met last week at We Love Speed conference in Paris would be to define which default value it should have. I agree having a default value (not too agressive, maybe So I think it should have no default value:
I wish Yoav's code (or adapted for other browsers of course) was available behind a flag and/or origin trial. It still say "not supported", as reported by @verlok. |
@nhoizey - FWIW, you can test that experimental attribute in Chrome with the |
@yoavweiss as @verlok said, Chrome says it is not supported (here in French 😅): EDIT: But I did check, and it works! 😲 |
It doesn't change much if you just want to test the feature. The warning is there so that real users won't use this not-necessarily-tested feature. |
Ok, now I understand "unsupported" doesn't mean the "browser" doesn't support this feature, but "Google" doesn't support you if you use it. 🤦♂️ Sorry. |
^^ @progers - FYI |
Responsive images, eg. srcset/sizes, are not meant to reduce your image weight. What has changed? There is documentation saying the browsers do take network conditions into account when deciding what picture source or srcset size to use, but I am not sure on how this is actually calculated and how it effects what size/source the browser picks. I believe this is entirely an implementation issue with how browsers are currently selecting the srcset size or picture source to be used and the browsers are not properly taking network condition/speed into account during this selection process. The hard part is thresholds and knowing 'what is slow' relative to 'what source is going to be too large?' when the only known information to pick a proper size/source is the aspect ratio, device pixel density, and network condition. |
Being able to assign a So, on those devices, the multiplication browsers would have to do to find the ideal intrinsic width of the image to download would be:
|
I mean I wouldn’t point fingers to browser vendors, as the quality of the image might depend on the website we are building and on the page type. I usually advise to cap DPR to 2 on product listing pages, whilst I leave it to 3 on product detail pages, and it should be uncapped size on PDP Zoom images. |
What do we need to do to push for this attribute to become a standard? |
I understand. I just don't believe that this effectively resolves the underlying issue - it is a bandaid and I find it redundant. Why code in a 3x source when you are gonna add this maxdpr flag anyways? Just remove the source reference. Makes no sense to me. CDN scripts/app components creating srcset/sizes automatically is not an excuse. A proper resolution would not remove or hinder functionality from devices that can handle 3x 4x in order to 'fix this' for devices that are technically able, but in practically sense, incapable. The underlying variable is connection speed, not device density. If your connection is fiber optics, then maxdpr wouldn't even be discussed, and no one would bat an eye. There are good points made here regarding if we actually are able to distinguish 2x+ quality increases, etc, but I believe those conversations are off topic in regards to contributing towards a resolution for this specific issue. Just look at video. Best quality eg 4k, fullhd, or 720p, is used based on connection speed, unless the user specified a preference. Same deal, really. I see one path forward here, and it's connection speed assisting determination of device density browser uses, or something of similar effect to ensure assets load in a timely manner relative to connection. Bad connection? Consider your device now a 1x display. Edit: consider this Real life Scenario: Possible Browser Flow: Result: |
When you declare the srcset with the w descriptor, you might need a wider image to serve the correct size to a wide, desktop, hidpi display. |
I disagree. Users of a luxury e-commerce website still want to see the full quality image, especially on the product detail page, no matter what the connection speed. They are willing to wait. I have proven this with data. So it depends on the use case and this choice should be left to developers. |
A flag to enforce device density and ignore connection optimization makes more sense to me for what you describe. Why do you want to kneecap devices with maxdpr in first place? Cause it's slow if they have a 4x used for a thumbnail?.. Isn't that the point of srcset and whoever defined the 4x source - it gets used. Browsers: Satisfies maximum devices capability, satisfies ux improvement, satisfies user control / use cases I believe |
I read back and looked at this more. From my understanding:
I guess the viewpoints I am looking at this from is:
In conclusion: I mean even something like Additional Info: Why can't browsers implement this natively to address the underlying issue, ie. connection? |
@AndresInSpace I think you’re off track with your rational. You’re following the hack path and not what the |
My point of view on the hack is still this. And as @gilbertococchi was suggesting, I think this is time to try and make it a standard. I read again the comments from months ago and the difficulties at choosing a default value for this attribute. I agree the default shkukd be @yoavweiss can you point me to the right direction on how to propose it as a standard? |
@verlok Yes, I read that before posting actually! Made sure to refresh my memory lol! We need both, working together
Facts: The browsers currently do not optimize this process by down-scaling, ie changing dpr/dppx applied in srcset match calculation, to account for the underlying issues of 1 and this whole thread.
The Distinction: The proposed implementation of maxdpr (2), in my eyes, seems to rest all of this on the web developer and should instead be in tandem with browsers optimization to scale selection process (1). Edit - taking picture into account Use case: This was why I provided the size example using a 'x' unit size as it would address the maxdpr for img and work with picture, and still effectively is in-line with your statement that size should tell the size - we are - just the scaled size. (I mean, it's an interesting discussion in itself, an 'x' css unit size, but diverges from the topic at hand so I digress). Ensuring that the maxdpr is applied to |
I would define that |
tl;dr: Correctly marked up responsive images (
srcset
+sizes
) result in high-resolution devices (2x, 3x, 4x, 5x, 6x dpi) downloading and using very large files causing significant performance hits, especially on mobile devices on mobile networks.Scenario
srcset
list includes images of varying src options, from small (~400px wide) to very large (~3000px wide) to account for mobile, tablet, desktop, and large monitors, high-resolution laptops etc.sizes
attribute marked up to spec with natural breakpoints.Under these circumstances, a high-resolution smartphone will download larger image files to meet screen resolution demands, thus negating the original intent (as I understand it) of the RICG responsive images work which was to lower the bandwidth hit on mobile devices when viewing images.
This scenario is further complicated by modern design trends including full-bleed images as described above. Because the
srcset
list must include very large image sources to account for large high-resolution screens, small devices with high-resolution screens have access to and will use these same sources resulting in wasted bandwidth and significantly degraded performance.Practical use case
This issue came to my attention while working on the new content editor for WordPress ("Gutenberg"). The content editor allows for images to be aligned
wide
andfull
, the latter meaning the image takes up the full available width which often means the width of the viewport.When updating WordPress to generate appropriate
srcset
andsizes
attributes to account for these layouts, I found significant performance reductions on mobile devices caused by the download of larger-than-required images as per thesizes
attribute.A functional example, with detailed explanation, is available here:
https://gutenberg.mor10.com/responsive-images-demo-corrected-code/
The result of this discovery was a sub-optimal workaround for WordPress core which is live on every WordPress site running version 5.0 or above: To avoid the performance hit on mobile devices, the largest image source in the
srcset
list is 1024px unless the theme explicitly states otherwise. In other words, browsers are served with smaller-than-necessary images, causing a poor user experience.A functional example of the current auto-generated output as of WordPress 5.1 is available here:
https://gutenberg.mor10.com/image-test-current/
Summation: Due to the performance issues introduced by the
srcset
+sizes
combination, 33.3% of the web is currently shipping the wrong image sources on purpose as a workaround. That said, this same issue will be experienced by anyone setting up a site with full-bleed images as described above, WordPress or not.Possible solutions
From my perspective as a front-end developer, the ideal solution would be to amend the spec to allow developers to declare, through attributes or similar on each individual image, the pixel density the
sizes
attribute should be measured against. Something like this:Alternatively, the browser could detect available bandwidth and other factors and actively throttle the
srcset
list accordingly so high-resolution mobile devices on mobile networks would receive the appropriate resolution interpretation based on available data and performance. This of course brings into question how to measure bandwidth and download limits, especially for users on max-megabytes-per-month plans.I know this type of throttling is technically possible using client hints, but configuring client hints and server side solutions is beyond the capacity of most CMSes and site owners, and puts the onus of having the web work as expected on the individual user.
In lieu of the suggestions above type of bandwidth throttling, a third option could be to put browser limits on how high the resolution can be for images (2x limit on a 4x screen as an example).
cc @yoavweiss @joemcgill @getsource
The text was updated successfully, but these errors were encountered: