Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use img tag instead of picture #2067

Merged
merged 12 commits into from
Dec 4, 2020
Merged

Use img tag instead of picture #2067

merged 12 commits into from
Dec 4, 2020

Conversation

oliverlloyd
Copy link
Contributor

@oliverlloyd oliverlloyd commented Nov 16, 2020

What?

This PR proposes replacing the picture tag with an img tag as a stopgap ahead of using image-rendering

TLDR

Surprisingly, tests for this change showed a lot of deviation between Frontend vs. DCR, both DCR with a picture tag and DCR with an img tab. More surprisingly, DCR with an img tag often outperformed Frontend. These variations could be for a lot of reasons though, not purely because an img tag was used.

How?

There are two main code changes here:

  1. Moving all image manipulation related logic into the Picture (now called Img) component
  2. Removing the hdipi related logic and adding the ability to create sizes.

Why?

DCR has known issues where it can sometimes render substantially lower quality images than appropriate in the context of position, screen width and resolution. In the short term, this change brings us into line with Frontend in most cases, makes us actually do better than frontend in some cases, and leaves us at the same or an acceptable level below frontend in the rest.

Why is this a stopgap?

See this ongoing discussion for more detail on our medium to longer term goals. But to summarise we want to:

  1. Solve the Wealthy Guy problem. Mobile devices with small screens but high dpr values can end up being served excessively large files, wasting bandwidth.

  2. Finesse the quality. We want to be able to specify a different set of images to use above a certain resolution. This is because we've found that for high resolution screens, by using dpr=2&quality=45 together we can reduce the file size without noticeably impacting quality.

Both of these need a picture tag based solution.

Test methodology

I rendered different images using a variety of simulated devices (via Firefox and Chrome dev tools) and then checked the resulting src chosen by the browser. Test were done using code from Frontend, new DCR and current DCR.

Example urls:

https://www.theguardian.com/artanddesign/2020/may/11/cross-border-love-a-photo-essay?dcr=false
http://localhost:3030/Article?url=https://www.theguardian.com/artanddesign/2020/may/11/cross-border-love-a-photo-essay
https://www.theguardian.com/artanddesign/2020/may/11/cross-border-love-a-photo-essay?dcr=true

Results

Desktop DPR 2

immersive
Frontend: width=1300 | quality=45 | dpr=2
New DCR: width=1300 | quality=45 | dpr=2
Current DCR: width=1300 | quality=85

showcase (main media 1020 pixels)
Frontend: width=1300 | quality=45 | dpr=2
New DCR: width=1020 | quality=45 | dpr=2
Current DCR: width=1020 | quality=85

inline
Frontend: width=620 | quality=45 | dpr=2
New DCR: width=620 | quality=45 | dpr=2
Current DCR: width=620 | quality=85

showcase (860 pixels wide)
Frontend: width=880 | quality=45 | dpr=2
New DCR: width=940 | quality=45 | dpr=2
Current DCR: width=1020 | quality=85

thumbnail
Frontend: width=140 | quality=45 | dpr=2
New DCR: width=140 | quality=45 | dpr=2
Current DCR: width=140 | quality=85

Desktop DPR 1

inline
Frontend: width=620 | quality=85
New DCR: width=620 | quality=85
Current DCR: width=620 | quality=85

iphone (375 pixels wide)

inline
Frontend: width=300 | quality=85
New DCR: width=445 | quality=85
Current DCR: width=445 | quality=85

immersive
Frontend: width=300 | quality=85
New DCR: width=465 | quality=85
Current DCR: width=445 | quality=85

thumbnail (140 pixels)
Frontend: width=300 | quality=85
New DCR: width=140 | quality=45 | dpr=2
Current DCR: width=445 | quality=85

iphonex dpr3

main media
Frontend: width=2000 | quality=85
New DCR: width=1125 | quality=85
Current DCR: width=445 | quality=85

inline
Frontend: width=445 | quality=45 | dpr=2
New DCR: width=605 | quality=45 | dpr=2
Current DCR: width=445 | quality=85

galaxy s9 dpr4

main media
Frontend: width=2250 | quality=85
New DCR: width=725 | quality=45 | dpr=2
Current DCR: width=445 | quality=85

inline
Frontend: width=445 | quality=45 | dpr=2
New DCR: width=620 | quality=45 | dpr=2
Current DCR: width=445 | quality=85

immersive
Frontend: width=465 | quality=45 | dpr=2
New DCR: width=725 | quality=45 | dpr=2
Current DCR: width=445 | quality=85

@paperboyo
Copy link
Contributor

I welcome a closer look at (all) imagery! 👏
I cannot comment on code changes and from cursory read, img indeed provides support for everything we were using picture for.

Specifically,

  1. Removing the hdipi related logic and adding the ability to create sizes.

as long as this replacement provides parity support for higher resolution when one uses a device with higher density screen (or zooms in low-density desktop over 133%) and we can still control distinctive “steps” of HiDPI support generically by manipulating fastly’s dpr attribute (instead of worrying about specific widths), ’ts cool.

My (poor!) understanding regarding

Picture tags are useful for solving the art direction problem but are less useful where the focus is on saving bandwidth.

is that our use of picture provided everything to the browser to save the bandwidth, so this code change should be a no-op as far as readers are concerned (?).

@paperboyo
Copy link
Contributor

cc @sndrs coz he was the author of initial implementation and ads are boring…

@paperboyo
Copy link
Contributor

Related (beyond parity): guardian/frontend#12216 (comment).

@github-actions
Copy link

github-actions bot commented Nov 16, 2020

Size Change: -37 B (0%)

Total Size: 716 kB

Filename Size Change
dist/frontend.server.js 222 kB -37 B (0%)
ℹ️ View Unchanged
Filename Size Change
dist/atomIframe.js 1.21 kB 0 B
dist/atomIframe.legacy.js 1.21 kB 0 B
dist/dynamicImport.js 1.96 kB 0 B
dist/dynamicImport.legacy.js 2.01 kB 0 B
dist/embedIframe.js 1.21 kB 0 B
dist/embedIframe.legacy.js 1.22 kB 0 B
dist/ga.js 3.01 kB 0 B
dist/ga.legacy.js 3.47 kB 0 B
dist/GetMatchStats.js 3.23 kB 0 B
dist/GetMatchStats.legacy.js 3.32 kB 0 B
dist/lotame.js 1.14 kB 0 B
dist/lotame.legacy.js 1.15 kB 0 B
dist/MostViewedFooterData.js 5.63 kB 0 B
dist/MostViewedFooterData.legacy.js 5.84 kB 0 B
dist/MostViewedRightWrapper.js 4.08 kB 0 B
dist/MostViewedRightWrapper.legacy.js 4.3 kB 0 B
dist/newsletterEmbedIframe.js 1.14 kB 0 B
dist/newsletterEmbedIframe.legacy.js 1.15 kB 0 B
dist/OnwardsLower.js 8.8 kB 0 B
dist/OnwardsLower.legacy.js 9.05 kB 0 B
dist/OnwardsUpper.js 12.8 kB 0 B
dist/OnwardsUpper.legacy.js 13.2 kB 0 B
dist/ophan.js 6.94 kB 0 B
dist/ophan.legacy.js 6.93 kB 0 B
dist/react.js 113 kB 0 B
dist/react.legacy.js 116 kB 0 B
dist/sentry.js 22.8 kB 0 B
dist/sentry.legacy.js 22.7 kB 0 B
dist/shimport.js 3.21 kB 0 B
dist/shimport.legacy.js 3.21 kB 0 B
dist/SignInGateMain.js 1.87 kB 0 B
dist/SignInGateMain.legacy.js 1.89 kB 0 B
dist/vendors~braze-web-sdk-core.js 34.9 kB 0 B
dist/vendors~braze-web-sdk-core.legacy.js 34.9 kB 0 B
dist/vendors~guardian-braze-components.js 17.3 kB 0 B
dist/vendors~guardian-braze-components.legacy.js 17.4 kB 0 B

compressed-size-action

@gtrufitt
Copy link
Contributor

For reference, here is why we moved from img -> picture in Frontend: guardian/frontend#11400

@paperboyo
Copy link
Contributor

here is why we moved from img

Oh! Good find, I forgot completely. Makes sense. We don’t want a super-duper-highres phone to willy nilly pick up highest res source just because we provide one for half-super-duper desktops.

@JamieB-gu
Copy link
Contributor

For reference, here is why we moved from img -> picture in Frontend: guardian/frontend#11400

And here's why we moved from img -> picture in Apps-Rendering: guardian/apps-rendering#327

@JamieB-gu
Copy link
Contributor

An img tag is less explicit about which source in the srcset to use, letting the browser to make the choice with the guide of sizes but also, potentially, using other information such as the reader's network conditions and what they might have in their cache.

This is interesting. The original reason we migrated from img to picture in Apps-Rendering was because we had no way to capture screen density using srcset, which meant we had no way to specify the quality parameter. We use the quality parameter because we have determined that we want:

  • DPR 1 displays to use lower-res images, at higher quality (85%)
  • DPR 2 (and higher) displays to use higher-res images, at lower quality (45%)

It was found that this provided the best balance of performance/bandwidth/noticeable image quality.

How does this change manage flipping between the two quality parameters?

@oliverlloyd
Copy link
Contributor Author

Closing this in favour of image-rendering and based on discussions. See: https://docs.google.com/document/d/1LfSCS0e9KJFxQuehguEdYdbns_afDSrtxx8MI9zgiMg/edit

@oliverlloyd
Copy link
Contributor Author

Ressurecting as a potential stopgap ahead of image-rendering

@oliverlloyd oliverlloyd reopened this Dec 2, 2020
@oliverlloyd oliverlloyd marked this pull request as ready for review December 2, 2020 22:29
@oliverlloyd
Copy link
Contributor Author

@paperboyo @sndrs @JamieB-gu We're thinking of moving ahead with this change as a stopgap ahead of migrating to image-rendering. There are some painfully poor quality images on DCR right now and this step improves our baseline.

@paperboyo
Copy link
Contributor

paperboyo commented Dec 3, 2020

This is great! It will be such a pleasure to finally see good quality images on DCR! Do you think it may make sense to spell out in the PR description the two reasons we still think it makes sense to move to picture one day (for the future archaeologists)? IIUC, one Jamie gave here and the other Gareth – here.

Thank you for the examples, this is very useful, one could almost imagine automated tests that would compare CSS widths (and one day heights too) of images at different breakpoints with actual requests made on certain devices. They should be able to help find inconsistencies and mistakes, like eg iphonex dpr3: Main media which looks mad and wrong, but then you notice it’s a Photo essay – and everything suddenly makes sense, haha.

@oliverlloyd
Copy link
Contributor Author

Do you think it may make sense to spell out in the PR description the two reasons we still think it makes sense to move to picture

I've added to the description to make it clear what the future goals are

@oliverlloyd
Copy link
Contributor Author

one could almost imagine automated tests that would compare CSS widths of images at different breakpoints with actual requests made on certain devices

This would be a great contract to define. Eg.: If I have this device, at this screen width, and pass these srcsets, with this role, then I expect src to equal x. Superb idea.

@oliverlloyd
Copy link
Contributor Author

one could almost imagine automated tests that would compare CSS widths of images at different breakpoints with actual requests made on certain devices

This would be a great contract to define. Eg.: If I have this device, at this screen width, and pass these srcsets, with this role, then I expect src to equal x. Superb idea.

Okay, so I don't think this is realistically possible. We want to test that various browsers, on different devices, at different breakpoints, do what we expect. But test automation in this field is typically done headless, where the presentation part of the browser is removed and the focus is on the html. We could (and do) use Storybook to render images at various breakpoints but we still can't assert these to check that the right image was picked.

Sadly, unless someone else has a better idea or wants to write some nifty pupetter script or something, I think this process will continue to be a manual. This PR lists a good baseline for this.

@paperboyo
Copy link
Contributor

paperboyo commented Dec 3, 2020

Okay, so I don't think this is realistically possible.

Absolutely fine and kudos for having a look! In my naive mind I imagined two robots: first robot that builds srcsets based off designs at specific breakpoints in CSS pixels automagically (so that a rigid and arbitrary ImageProfile.scala goes away) and a second robot that runs through pages rendered on different devices and lists what millions of browsers have chosen from the menu supplied by the first robot. And then us, humans, looking over the popularity histograms of the output of the second robot over months to make informed changes to what range and steps we want the first robot to provide.
But, you know, errare robotum est 😜.

Copy link
Contributor

@jfsoul jfsoul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is a big improvement on the current situation, even if not the final solution. Great testing too, might even be worth pulling that procedure into a testing image changes advice doc.

@oliverlloyd oliverlloyd merged commit f3e76d2 into main Dec 4, 2020
@oliverlloyd oliverlloyd deleted the oliver/use-img-not-picture branch December 4, 2020 10:42
@paperboyo
Copy link
Contributor

paperboyo commented Dec 4, 2020

👏

Most popular inline images of 620px seem to be requested wrongly at 445px. But DPR 2 :-).
EDIT: Example URL, desktop widest breakpoint, zoomed in over 130% at dpr=2. Frontend serves 620px at dpr2.

@oliverlloyd
Copy link
Contributor Author

👏

Most popular inline images of 620px seem to be requested wrongly at 445px. But DPR 2 :-).
EDIT: Example URL, desktop widest breakpoint, zoomed in over 130% at dpr=2. Frontend serves 620px at dpr2.

I'm not able to replicate this; I'm getting the 620px, dp2 image everytime, on DCR, Frontend, same same. Is it still happening?

@paperboyo
Copy link
Contributor

Is it still happening?

Very, very weird. Chrome OK. Firefox (Mac and Win, different versions) OK in Private Browsing, but not in my main session (cleared site data, no extensions that aren’t also in Private, same zoom):

image

Probably something on my system, apologies, will report back if I find out what or repro somewhere else…

@oliverlloyd
Copy link
Contributor Author

Is this because the browser is favouring a cached img that it has locally over making a network request? I think the spec talks about this functionality.

@paperboyo
Copy link
Contributor

paperboyo commented Dec 4, 2020

Pls disregard. On a third machine, it works correctly. Gremlins in the machine.

EDIT: disregard the disregarding: this third machine was a genuine retina Mac (dpr 2)

@JamieB-gu
Copy link
Contributor

JamieB-gu commented Dec 4, 2020

Is this because the browser is favouring a cached img that it has locally over making a network request?

It's nice to see this happening in the wild. @paperboyo you can test this theory using the "Disable cache" checkbox in the Network tab of the Firefox devtools (it will only disable it while the tools are open AFAIK).

@paperboyo
Copy link
Contributor

paperboyo commented Dec 5, 2020

Yeah, I disabled the cache. Done some more tests. My current operating hypothesis is that Firefox seems to be choosing sources differently from this (img) than from frontend (picture) where zoomed-in on a dpr 1 device. Chrome behaves as it did, as is Firefox on a genuine dpr 2 device.

As I’m permanently zoomed-in to 133% (dpr 1 device) on the Guardian, unfortunately I’m getting smaller images than on frontend. But:

  • it’s much better than it used to be
  • I hope that as soon as we move to picture again, I will be getting correct* sources

*correct means: when zoomed-in over 130% (or whatever media query specifies), on a dpr 1 device, request an image of CSS pixels width (strangely, this is the broken part, I’m getting 445px requests for inline article images of 620 CSS pixels!) at dpr 2.

@JamieB-gu
Copy link
Contributor

Firefox on a genuine dpr 2 device

If you drop into responsive design mode on Firefox (Cmd-Opt-M) there's a dropdown to simulate different DPRs, don't know if that's helpful?

@paperboyo
Copy link
Contributor

Yeah, yeah, I know. But it doesn’t accept 1.3 which is what I would need to drop it down to on this machine I’m currently sitting on (drp 2) to simulate my home machines (both PC and Mac of dpr 1) being zoomed in over 130%.

Anyway :-). Can anyone with a Firefox on dpr 1 desktop can confirm 620px images are requested at 445px? (yes, I know, dpr 2, but still: too small dpr 2)

@oliverlloyd
Copy link
Contributor Author

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants