Media library needs rendered thumbnails

[t1merickson]

- Do you want to request a feature or report a bug?

Feature/bug

- What is the current behavior?

Image library loads each image in at 100% original file size. If even one file is, say 14MB, the interface can slow to crawl in some browsers on some computers (heavy performance issues).

- If the current behavior is a bug, please provide the steps to reproduce.

Upload an image of a large size, say >10MB

- What is the expected behavior?

Media library will need to somehow generate thumbnails for images (eg. 276x160 @1x, 552x320 @2x) such that the media library interface is performant.

[t1merickson]

Ideally however: I think we're hoping to switch people off of using Git for media assets in the first place, so an integration with a service would allow for this resizing/thumbnail generation

[andreasremdt]

Confirmed on a fresh install of Netlify CMS on Windows 7 64-Bit with Google Chrome 63.

[tech4him1]

Agreed, I've seen performance issues as well.

[erquhart]

Would love to see this picked up by someone. We'll need to create thumbnail with canvas, and I don't believe there's a maintained library that handles that well, except maybe Origami.js. Should be simple enough to just use Canvas directly and avoid the dependency.

[tech4him1]

I wonder about using a generated SVG placeholder, I'm not sure what the performance difference is: https://jmperezperez.com/svg-placeholders/.

I guess the choice is whether we want a scaled-down version of the actual image, or something that looks similar to it (a placeholder). Probably a scaled-down version would be what people expect here?

[erquhart]

Yeah, that sort of placeholder is for loading full images in a production site. We'll need actual thumbnails for this.

[tech4him1]

Alternative libraries:

• https://github.com/raphamorim/origami.js (@erquhart mentioned)
• https://github.com/nodeca/pica May have better performance than plain canvas.
• https://github.com/kangax/fabric.js requires native library (Cairo)

[erquhart]

Hmm I don't think I saw Pica before, good find. I think Origami is probably the least ideal compared to these other two.

Pica seems the most straightforward, small footprint, well maintained. Fabric is the biggest, but may be more than we have any use for since it's so SVG focused.

[tech4him1]

Also, Pica uses WebAssembly and/or WebWorkers if they are available, so it should have better performance.

[owenhoskins]

@tech4him1 I've tested out Pica and it seems to perform better then canvas, great suggestion!

However, when I include it in my Gatsby project the build process throws an error.

12:09:23 PM:   Error: cms-919e2a556de9dd936850.js from UglifyJs
12:09:23 PM:   SyntaxError: Unexpected token: name (n) [./~/netlify-cms/dist/cms.js:81,14181]

The error only appears with pica imported. Here's my branch https://github.com/owenhoskins/netlify-cms/tree/image-preview-pica

Included via "netlify-cms": "owenhoskins/netlify-cms#image-preview-pica"

Here is the file where I import pica. I tried both require and import.

https://github.com/owenhoskins/netlify-cms/blob/image-preview-pica/src/components/EditorWidgets/Gallery/GalleryPreview.js

Any insights you might have would be appreciated!

[tech4him1]

@owenhoskins @erquhart pointed out that to get the compiled version of Pica, you have to import from pica/dist/pica. https://github.com/nodeca/pica#install

[erquhart]

@owenhoskins just checking in - is this still on your radar? Anything we can do to help?

[owenhoskins]

@erquhart: Yes! But I have been swamped by another project. I will be looking for a window of time to continue on this between now and mid March.

I've got a test-case going with a bit more then 1000 images at around 500kb each, totaling about 500MB. This made it clear that the Media Library also needs a lazy-loader to function at scale. I am thinking of approaching that with a window.IntersectionObserver implementation.

Also, I haven't worked out how to load / persist a set of images in netlify-cms when developing locally. I can upload via the library but each time the app hot reloads they clear, this works fine for simple tests but now that I am getting into large quantities I've got to find a better way!

Finally, where would you suggest a component like this should live within the cms directory structure?

import React, { Component } from 'react';
import picaImport from 'pica/dist/pica'
const pica = picaImport()

export class ImageCanvas extends Component {

  constructor(props) {
    super(props);
  }

  componentDidMount() {
    this.prepareCanvas(this.props.src)
  }

  componentWillReceiveProps(nextProps) {
    if (nextProps.src !== this.props.src) {
      this.prepareCanvas(nextProps.src)
    }
  }

  prepareCanvas = (src) => {
    const self = this
    const canvas = this.canvas
    const ctx = canvas.getContext('2d')
    const image = new Image()
    image.src = src
    image.onload = function(event){

      const aspectRatio = image.width / image.height
      canvas.width = aspectRatio * 210
      canvas.height = 210

      pica.resize(image, canvas, {
        unsharpAmount: 80,
        unsharpRadius: 0.6,
        unsharpThreshold: 2
      })
      .then(result => console.log('resize done!', result));
    }

  }

  render() {
    return (
      <canvas
        ref={ (ref) => this.canvas = ref }
        style={{
          ...this.props.style
        }}
        className={ this.props.className }
      />
    )
  }
}

[erquhart]

@owenhoskins sounds great!

So part of the goal is to create thumbnails that are very low weight so that a media library with 500MB of images wouldn't even be possible. The created previews would be persisted to the user's repo in the metadata.

As far as the location of the component you're building, that can go under src/components/MediaLibrary/ for now.

You can use a "real" backend during local development by editing the config in the example project in example/config.yml, and adding information for a live github repo. Any repo with at least one commit can store markdown files and any directories will be created automatically.

You can also always reach out on Gitter with any questions about things like local development.

[owenhoskins]

Thank you @erquhart, a few pointers if you would...

In #787 you comment:

that will introduce thumbnail creation on image upload, and the thumbnail will be committed as metadata, at which point we can begin pulling thumbnails through the API instead of by URL.

How do we commit that metadata?

My best guess is that we would generate a blob with pica when the image has been uploaded within something like the mediaLibraries' handlePresist method. https://github.com/netlify/netlify-cms/blob/master/src/actions/mediaLibrary.js#L75

Beyond that I would need some guidance on the API and how to access that metadata once committed!

Thanks again!

[erquhart]

Metadata handling is kind of buried in the GitHub API, it's only in use for tracking details about the editorial workflow. We really need metadata to take on a life of it's own and have centralized, abstracted methods for storage and retrieval. That said, you can just do something similar to what we're doing already, and keep it in the backend as we currently are.

Here are examples of retrieving and storing metadata.

Follow that linked code a bit to see how we're doing it, but it boils down to storing a tree in an orphan ref, meaning it's not actually on any branch. We do it this way to keep metadata out of the forefront, and to avoid things like accidental deletion, but we're very likely to start handling all metadata on a visible branch soon because GitLab and Bitbucket don't support orphan ref creation through their API's.

[huntercaron]

Any updates on this? Just launched the CMS on our site for 80ish users creating a bunch of image heavy design case studies and this is becoming a problem faaasssttt. Anything I can do to help?

[owenhoskins]

@huntercaron I've made a bit of progress on this but so far only to mitigate the issue until I have more time to get into saving thumbnails as metadata in git.

On this branch (https://github.com/owenhoskins/netlify-cms/tree/image-preview-pica) I've implemented pica.toCanvas which generates thumbnails on the fly along with a quick implementation of react-intersection-observer to load them lazily.

I am including this in my site's package.json via "netlify-cms": "owenhoskins/netlify-cms. And then running rm -rf ./node_modules/netlify-cms && npm install to get my branch.

This is allowing me to browse a media library with 5000 images equalling about 1.2gb. My next step is to test react-virtualized for infinite scrolling in the media library.

I am definitely at the upper limit of how many images one should manage via git and will be looking to offload these asap, but that's another issue!

[tech4him1]

@owenhoskins If you want to make a PR from that branch and just put WIP in the title so that it doesn't get merged yet, that would be great! That way we can get some more input on it as you progress.

[hanchennz]

Hi friends, I was wondering if this ended up being merged and released in 2.0.

Running into a problem where the images are disappearing after a short time, and I'm getting we'll run into performance issues once we go live.

[lukeburns]

Also experiencing disappearing media images. Status? Where is help needed?

[erquhart]

Guessing you both have private repos? The tokenized url expires in that case, we don't have a way around it yet. Covered in #787.

[Jinksi]

Is this issue fixed by utilising Netlify Large-Media?

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.