diff --git a/dotcom-rendering/docs/architecture/027-pictures.md b/dotcom-rendering/docs/architecture/027-pictures.md new file mode 100644 index 00000000000..5117a9ccbbc --- /dev/null +++ b/dotcom-rendering/docs/architecture/027-pictures.md @@ -0,0 +1,128 @@ +# Pictures in DCR + +This document outlines how (and why) pictures in DCR work the way they do. A lot of methodology comes from Frontend, however DCR didn't immediately have parity with Frontend for image/picture rendering, and it has been built on over time to improve it. + +The overall goal of the picture rendering to ensure that we serve the correct resolution image to the user in all cases. This can involve looking at things like the page & image layout, as well as client properties like DPR. + +## Background Info + +This background info section aims to give enough information for anyone who hasn't worked a lot with picture or source elements, or source sets. Feel free to skip it if you like! + +DCR uses the html `` tag to render images. This offers us a advantage over regular `` tags, and that is the use of `` elements to further help the browser understand what size & quality image to download, saving our users bandwidth & us money. + +### Fastly Image Optimiser + +They key enabler in this is the [fastly image optimiser](https://developer.fastly.com/reference/io/). This allows us to specify image widths when requesting an image URL. for example for a given image, say `https://i.guim.co.uk/img/media/xxxxx/yyy.jpg`, we are able to specify some important image transformation properties: `width`, `quality` & `dpr` (& others). + +- `width` allows us to specify the width of the image in pixels. +- `quality` allows us to specify how much to compress the image, 0 being very compressed, and 100 preserving the best quality possible. +- `dpr` or Device Pixel Ratio allows us to scale the size of the image by this number, for example using `?width=300&dpr=2` would return a 600px wide image. + +### Media queries (`media="query"`) + +Within any given `` tag, you can have multiple `` child elements. Using the `media` html attribute, which uses the same syntax as CSS media queries, you can tell the browser which `` element to look for an image source in. + +For example `` would tell the browser to pick this source if the viewport is 600px or wider. + +The browser will choose and stick with the first matching source element it finds, so it's important to ensure they're in the DOM in the right order. + +Another important media queries we use is (`(orientation: portrait)`) to check if it's a portrait device (e.g a smartphone). + +### Sizes (`sizes=`) + +The sizes html attribute acts as the translation layer between the size of the viewport and the size of the image source you'd like to pick. A good way to think about this is that, beyond a certain width, main media (inline) images never go beyond `620px` wide, and this html attribute gives us a way to communicate that to the browser. + +For example `` tells the browser, "Hey, if your viewport is 660px or wider, always look for an image source which is 620px wide. If not, default to an image which is the same width as 100% of the viewport width". We can provide as many sizes & queries as we want, with the `(query) px, (query) px, ... , px)` syntax, where the last argument without a query is the default if none others match. + +### Source Set (`srcset=`) + +Source sets work as our final piece of the puzzle. Our browser has already picked a `source` element, and has used `sizes` to figure out what size (in width only) image it is looking for. Our source set allows us to provide a list of URLs & the width of the image for each one, which the browser will then use to look for the best fitting image from. + +Source sets are formatted like: `` Our comma separated list of sources specifies first the URL to a given image source, then the real pixel width for that image. The unit `w` is used, to distinguish pixels inside the image from CCS `px` on the screen–there are many pixels per CSS `px` on high DPI screens. In our case where we use Fastly image optimiser, the only thing changing between these image urls is in the query parameters, e.g `?dpr=2` or `?width=300` and `?width=600`. + +### DPR?! + +DPR originates from the concept that the pixel widths which we use for CSS media queries is often different from the actual resolution of a devices display. +For example imagine a phone with a super high resolution 1200px wide (2400px high) screen. Following our breakpoint sizes, we'd try and render a desktop type experience for this user. However, the reality is this screen is only maybe 5 inches across, so the site would be totally unusable. + +CSS Pixels & DPR to the rescue! Our browser can use a different width for calculating breakpoints, media queries, etc than the real resolution of the screen. This is CSS pixels. So let's say for the sake of argument the browser uses 300px for our CSS pixel width - Wohoo, we're displaying a mobile experience, all is well. The DPR is the ratio between CSS pixels and actual resolution, so 1200 / 300 gives us a DPR of 4. Why this is important will be discussed later. + +#### The DPR Problem + +This problem comes from how the browser tries to compensate for high DPR displays when choosing an image source. In previous iterations of Frontend & DCR, we provided only 2 sources - a regular set, and a set of sources for high DPR displays; targeted with a media query to ensure it's picked. + +Unfortunately the browser itself would try to compensate for high DPR as well, but in a less efficient way. Once the browser had figured out what size source it wants from the `sizes` attribute on our source element, it's then multiplied by the DPR of the display to get the desired width it will look for in our `srcset`: + +``` +# desiredWidth is the width of an image that the browser will look for in `srcset` +# size is the chosen size based off our queries in the `sizes` attribute +# DPR is the ration between CSS Pixels & device resolution (e.g 2, 3, 4) +desiredWidth = size * DPR +``` + +This posed a problem for us, because if we tell the browser "Hey choose a 620px image", and the browser has a DPR of say 3, it will actually look for an image source for 1860px, far higher resolution than is needed for the user to have a good experience. + +#### The DPR Solution + +This problem was first solved in Frontend, then replicated in DCR + +Rather than having just 2 source elements, we instead have 2 source elements per breakpoint (one for high DPR, one for regular displays), and provide just one size & source set for that source element. + +Lets look at a simplified example (with only 1 source per breakpoint): + +```html + + + + + + +``` + +In this example, we have the logic that usually would have been in our sizes attribute (`sizes="(min-width: 660px) 620px, 100vw"`) extrapolated into individual source elements. All our breakpoint which are 660px or larger offer only 1 choice, the 620px source. The lower breakpoints have looked for sources which are closest to their own size, as a replacement for `100vw`. + +This solves our DPR problem because, the `media` attribute uses CSS pixels, and because we only offer 1 image source for each of these elements, once the browser has picked its source element, we basically strong arm it into which source to use. + +## What does DCR do? + +DCR Maintains some parity with Frontend's implementation of images, the key difference's being: + +1. DCR Relies on Frontend to generate it's image sources +2. Frontend offers higher resolution source for portait immersives +3. DCR Removes redundant sources to make the DOM more effecient. + +The implementation DCR picked involves creating 2 elements for each breakpoint, one regular, and one for high DPR displays. + +For Example: + +```html + + + + + + + + + + + + + + + + The Palace Theatre, London, showing Harry Potter and the Cursed Child + +``` diff --git a/dotcom-rendering/src/web/components/Picture.test.tsx b/dotcom-rendering/src/web/components/Picture.test.tsx new file mode 100644 index 00000000000..524378a81e7 --- /dev/null +++ b/dotcom-rendering/src/web/components/Picture.test.tsx @@ -0,0 +1,223 @@ +import { breakpoints } from '@guardian/src-foundations/mq'; +import type { DesiredWidth } from './Picture'; +import { getBestSourceForDesiredWidth, removeRedundantWidths } from './Picture'; + +const hdpiSources: SrcSetItem[] = [ + { + src: '1', + width: 1400, + }, + { + src: '2', + width: 1240, + }, + { + src: '3', + width: 930, + }, + { + src: '4', + width: 1290, + }, +]; + +const mdpiSources: SrcSetItem[] = [ + { + src: '1', + width: 620, + }, + { + src: '2', + width: 700, + }, + { + src: '3', + width: 465, + }, + { + src: '4', + width: 645, + }, +]; + +/** + * mobile: 320 + * mobileMedium: 375 + * mobileLandscape: 480 + * phablet: 660 + * tablet: 740 + * desktop: 980 + * leftCol: 1140 + * wide: 1300 + */ + +describe(`Picture`, () => { + describe('getClosestSetForWidth', () => { + it('Gets the closest source for a given width (hdpi)', () => { + // Breakpoints + expect( + getBestSourceForDesiredWidth( + breakpoints.mobile * 2, + hdpiSources, + ).width, + ).toBe(930); + expect( + getBestSourceForDesiredWidth( + breakpoints.mobileMedium * 2, + hdpiSources, + ).width, + ).toBe(930); + expect( + getBestSourceForDesiredWidth( + breakpoints.mobileLandscape * 2, + hdpiSources, + ).width, + ).toBe(1240); + expect( + getBestSourceForDesiredWidth( + breakpoints.phablet * 2, + hdpiSources, + ).width, + ).toBe(1400); + expect( + getBestSourceForDesiredWidth( + breakpoints.tablet * 2, + hdpiSources, + ).width, + ).toBe(1400); + expect( + getBestSourceForDesiredWidth( + breakpoints.desktop * 2, + hdpiSources, + ).width, + ).toBe(1400); + expect( + getBestSourceForDesiredWidth( + breakpoints.leftCol * 2, + hdpiSources, + ).width, + ).toBe(1400); + expect( + getBestSourceForDesiredWidth(breakpoints.wide * 2, hdpiSources) + .width, + ).toBe(1400); + + // Example widths + expect( + getBestSourceForDesiredWidth(620 * 2, hdpiSources).width, + ).toBe(1240); + }); + + it('Gets the closest source for a given width (mdpi)', () => { + // Breakpoints + expect( + getBestSourceForDesiredWidth(breakpoints.mobile, mdpiSources) + .width, + ).toBe(465); + expect( + getBestSourceForDesiredWidth( + breakpoints.mobileMedium, + mdpiSources, + ).width, + ).toBe(465); + expect( + getBestSourceForDesiredWidth( + breakpoints.mobileLandscape, + mdpiSources, + ).width, + ).toBe(620); + expect( + getBestSourceForDesiredWidth(breakpoints.phablet, mdpiSources) + .width, + ).toBe(700); + expect( + getBestSourceForDesiredWidth(breakpoints.tablet, mdpiSources) + .width, + ).toBe(700); + expect( + getBestSourceForDesiredWidth(breakpoints.desktop, mdpiSources) + .width, + ).toBe(700); + expect( + getBestSourceForDesiredWidth(breakpoints.leftCol, mdpiSources) + .width, + ).toBe(700); + expect( + getBestSourceForDesiredWidth(breakpoints.wide, mdpiSources) + .width, + ).toBe(700); + + // Example widths + expect(getBestSourceForDesiredWidth(620, mdpiSources).width).toBe( + 620, + ); + }); + }); + + describe('optimiseBreakpointSizes', () => { + it('Leaves un-optimisable breakpointSizes as-is', () => { + const breakPointSizes: DesiredWidth[] = [ + { breakpoint: 1000, width: 500 }, + { breakpoint: 800, width: 400 }, + { breakpoint: 600, width: 300 }, + { breakpoint: 400, width: 200 }, + ]; + expect(removeRedundantWidths(breakPointSizes)).toEqual( + breakPointSizes, + ); + }); + + it('Correctly removes optimisable breakpointSizes', () => { + expect( + removeRedundantWidths([ + { breakpoint: 1000, width: 500 }, + { breakpoint: 800, width: 400 }, + { breakpoint: 600, width: 400 }, + { breakpoint: 400, width: 200 }, + ]), + ).toEqual([ + { breakpoint: 1000, width: 500 }, + { breakpoint: 600, width: 400 }, + { breakpoint: 400, width: 200 }, + ]); + + expect( + removeRedundantWidths([ + { breakpoint: 1000, width: 500 }, + { breakpoint: 800, width: 400 }, + { breakpoint: 600, width: 200 }, + { breakpoint: 400, width: 200 }, + ]), + ).toEqual([ + { breakpoint: 1000, width: 500 }, + { breakpoint: 800, width: 400 }, + { breakpoint: 400, width: 200 }, + ]); + + expect( + removeRedundantWidths([ + { breakpoint: 1000, width: 500 }, + { breakpoint: 800, width: 200 }, + { breakpoint: 600, width: 200 }, + { breakpoint: 400, width: 200 }, + ]), + ).toEqual([ + { breakpoint: 1000, width: 500 }, + { breakpoint: 400, width: 200 }, + ]); + + expect( + removeRedundantWidths([ + { breakpoint: 1000, width: 500 }, + { breakpoint: 800, width: 500 }, + { breakpoint: 600, width: 300 }, + { breakpoint: 400, width: 200 }, + ]), + ).toEqual([ + { breakpoint: 800, width: 500 }, + { breakpoint: 600, width: 300 }, + { breakpoint: 400, width: 200 }, + ]); + }); + }); +}); diff --git a/dotcom-rendering/src/web/components/Picture.tsx b/dotcom-rendering/src/web/components/Picture.tsx index 18aeec43a8a..ef953e12b25 100644 --- a/dotcom-rendering/src/web/components/Picture.tsx +++ b/dotcom-rendering/src/web/components/Picture.tsx @@ -3,6 +3,10 @@ import { ArticleDisplay } from '@guardian/libs'; import { breakpoints } from '@guardian/src-foundations/mq'; +/* + * Working on this file? Checkout out 027-pictures.md for background information & context! + */ + type Props = { imageSources: ImageSource[]; role: RoleType; @@ -15,13 +19,27 @@ type Props = { }; type ResolutionType = 'hdpi' | 'mdpi'; +type Pixel = number; +type Breakpoint = number; + +export type DesiredWidth = { + breakpoint: Breakpoint; + width: Pixel; +}; -const getClosestSetForWidth = ( - desiredWidth: number, +/** + * Given a desired width & array of SrcSetItems, return the closest match. + * The match will always be greater than the desired width when available + * + * @param desiredWidth + * @param inlineSrcSets + * @returns {SrcSetItem} + */ +export const getBestSourceForDesiredWidth = ( + desiredWidth: Pixel, inlineSrcSets: SrcSetItem[], ): SrcSetItem => { - // For a desired width, find the SrcSetItem which is the closest match - const sorted = inlineSrcSets.sort((a, b) => b.width - a.width); + const sorted = inlineSrcSets.slice().sort((a, b) => b.width - a.width); return sorted.reduce((best, current) => { if (current.width < best.width && current.width >= desiredWidth) { return current; @@ -46,63 +64,61 @@ const getSourcesForRoleAndResolution = ( : setsForRole.filter((set) => !set.src.includes('dpr=2')); }; -const getFallback = ( - role: RoleType, - resolution: ResolutionType, - imageSources: ImageSource[], -): string | undefined => { - // Get the sources for this role and resolution - const sources: SrcSetItem[] = getSourcesForRoleAndResolution( - imageSources, - role, - resolution, - ); +const getFallback = (sources: SrcSetItem[]): string | undefined => { if (sources.length === 0) return undefined; // The assumption here is readers on devices that do not support srcset are likely to be on poor // network connections so we're going to fallback to a small image - return getClosestSetForWidth(300, sources).src; + return getBestSourceForDesiredWidth(300, sources).src; }; -const getSources = ( - role: RoleType, - resolution: ResolutionType, - imageSources: ImageSource[], -): string => { - // Get the sources for this role and resolution - const sources: SrcSetItem[] = getSourcesForRoleAndResolution( - imageSources, - role, - resolution, - ); +/** + * Removes redundant widths from an array of DesiredWidths + * + * This function specifically looks for *consecutive duplicates*, + * in which case we should always keep the last consecutive element. + * + * @param {DesiredWidth[]} desiredWidths + * @returns {DesiredWidth} + */ +export const removeRedundantWidths = ( + allDesiredWidths: DesiredWidth[], +): DesiredWidth[] => { + const desiredWidths: DesiredWidth[] = []; + + for (const desiredWidth of allDesiredWidths) { + if ( + desiredWidths[desiredWidths.length - 1]?.width === + desiredWidth.width + ) { + // We overwrite the end element as we want to keep the *last* consecutive duplicate + desiredWidths[desiredWidths.length - 1] = desiredWidth; + } else desiredWidths.push(desiredWidth); + } - return sources.map((srcSet) => `${srcSet.src} ${srcSet.width}w`).join(','); + return desiredWidths; }; /** - * mobile: 320 - * mobileMedium: 375 - * mobileLandscape: 480 - * phablet: 660 - * tablet: 740 - * desktop: 980 - * leftCol: 1140 - * wide: 1300 + * Returns the desired width for an image at the specified breakpoint, based on the image role, format, and if it's main media. + * This function acts as a source of truth for widths of images based on the provided parameters + * + * Returns 'breakpoint', or 'breakpoint / x' when the image is expected to fill a % of the viewport, + * instead of a fixed width. + * + * @param breakpoint + * @param role + * @param isMainMedia + * @param format + * @returns {Pixel} */ - -const getSizes = ( +const getDesiredWidthForBreakpoint = ( + breakpoint: number, role: RoleType, isMainMedia: boolean, format: ArticleFormat, -): string => { - if (format.display === ArticleDisplay.Immersive && isMainMedia) { - // Immersive MainMedia elements fill the height of the viewport, meaning - // on mobile devices even though the viewport width is small, we'll need - // a larger image to maintain quality. To solve this problem we're using - // the viewport height (vh) to calculate width. The value of 167vh - // relates to an assumed image ratio of 5:3 which is equal to - // 167 (viewport height) : 100 (viewport width). - return `(orientation: portrait) 167vh, 100vw`; - } +): Pixel => { + if (format.display === ArticleDisplay.Immersive && isMainMedia) + return breakpoint; if ( (format.display === ArticleDisplay.Showcase || @@ -111,25 +127,83 @@ const getSizes = ( ) { // Showcase main media images (which includes numbered list articles) appear // larger than in body showcase images so we use a different set of image sizes - return `(min-width: ${breakpoints.wide}px) 1020px, (min-width: ${breakpoints.leftCol}px) 940px, (min-width: ${breakpoints.tablet}px) 700px, (min-width: ${breakpoints.phablet}px) 660px, 100vw`; + if (breakpoint >= breakpoints.wide) return 1020; + if (breakpoint >= breakpoints.leftCol) return 940; + if (breakpoint >= breakpoints.tablet) return 700; + if (breakpoint >= breakpoints.phablet) return 700; + return breakpoint; } switch (role) { case 'inline': - return `(min-width: ${breakpoints.phablet}px) 620px, 100vw`; + if ( + breakpoint >= breakpoints.tablet && + breakpoint < breakpoints.desktop + ) + return 680; + if (breakpoint >= breakpoints.phablet) return 620; + return breakpoint; case 'halfWidth': - return `(min-width: ${breakpoints.phablet}px) 300px, 50vw`; + if (breakpoint >= breakpoints.phablet) return 300; + return Math.round(breakpoint / 2); case 'thumbnail': - return '140px'; + return 140; case 'immersive': - return `(min-width: ${breakpoints.wide}px) 1300px, 100vw`; + if (breakpoint >= breakpoints.wide) return 1300; + return breakpoint; case 'supporting': - return `(min-width: ${breakpoints.wide}px) 380px, 300px`; + if (breakpoint >= breakpoints.wide) return 380; + if (breakpoint >= breakpoints.tablet) return 300; + return breakpoint; case 'showcase': - return `(min-width: ${breakpoints.wide}px) 860px, (min-width: ${breakpoints.leftCol}px) 780px, (min-width: ${breakpoints.phablet}px) 620px, 100vw`; + if (breakpoint >= breakpoints.wide) return 860; + if (breakpoint >= breakpoints.leftCol) return 780; + if (breakpoint >= breakpoints.phablet) return 620; + return breakpoint; } }; +// Takes a size & sources, and returns a source set with 1 matching image with the desired size +const getSourceForDesiredWidth = ( + desiredWidth: Pixel, + sources: SrcSetItem[], + resolution: ResolutionType, +) => { + // The image sources we're provided use double-widths for HDPI images + // We should therefor multiply the width based on if it's HDPI or not + // e.g { url: '... ?width=500&dpr=2 ...', width: '1000' } + const source = getBestSourceForDesiredWidth( + resolution === 'hdpi' ? desiredWidth * 2 : desiredWidth, + sources, + ); + return `${source.src} ${source.width}w`; +}; + +// Create sourcesets for portrait immersive +// TODO: In a future PR this system will be updated to solve scaling issues with DPR +const portraitImmersiveSource = ( + sources: SrcSetItem[], + resolution: ResolutionType, +) => ( + `${srcSet.src} ${srcSet.width}w`) + .join(',')} + /> +); + export const Picture = ({ imageSources, role, @@ -140,21 +214,78 @@ export const Picture = ({ isMainMedia = false, isLazy = true, }: Props) => { - const hdpiSources = getSources(role, 'hdpi', imageSources); - const mdpiSources = getSources(role, 'mdpi', imageSources); - const fallbackSrc = getFallback(role, 'hdpi', imageSources); - const sizes = getSizes(role, isMainMedia, format); + const hdpiSourceSets = getSourcesForRoleAndResolution( + imageSources, + role, + 'hdpi', + ); + const mdpiSourceSets = getSourcesForRoleAndResolution( + imageSources, + role, + 'mdpi', + ); + const fallbackSrc = getFallback( + hdpiSourceSets.length ? hdpiSourceSets : mdpiSourceSets, + ); + + const allDesiredWidths: DesiredWidth[] = [ + // Create an array of breakpoints going from highest to lowest, with 0 as the final option + breakpoints.wide, + breakpoints.leftCol, + breakpoints.desktop, + breakpoints.tablet, + breakpoints.phablet, + breakpoints.mobileLandscape, + breakpoints.mobileMedium, + breakpoints.mobile, + 0, + ].map((breakpoint) => ({ + breakpoint, + width: getDesiredWidthForBreakpoint( + breakpoint, + role, + isMainMedia, + format, + ), + })); + + const desiredWidths: DesiredWidth[] = + removeRedundantWidths(allDesiredWidths); return ( - {/* HDPI Source (DPR2) - images in this srcset have `dpr=2&quality=45` in the url */} - - {/* MDPI Source (DPR1) - images in this srcset have `quality=85` in the url */} - + {format.display === ArticleDisplay.Immersive && isMainMedia && ( + <> + {portraitImmersiveSource(hdpiSourceSets, 'hdpi')} + {portraitImmersiveSource(mdpiSourceSets, 'mdpi')} + + )} + + {desiredWidths.map(({ breakpoint, width: desiredWidth }) => ( + <> + {/* HDPI Source (DPR2) - images in this srcset have `dpr=2&quality=45` in the url */} + {hdpiSourceSets.length > 0 && ( + + )} + {/* MDPI Source - images in this srcset have `quality=85` in the url */} + + + ))} +