Memory leaks in webp render process

[Stafox]

Just move to new issue from this issue
Looks like the issue still exists for ios < 14.0
Can reproduce on devices with ios 12 and ios 13 while trying to render static WebP images (I see huge memory usage which cause app crash due to memory).
On ios14 works fine because SDWebImage v5.9.0 started to use native coder implementation.

In my pod manifest SDWebImageWebPCoder = 0.6.1.

• SDWebImage (5.9.4):
  • SDWebImage/Core (= 5.9.4)
• SDWebImage/Core (5.9.4)
• SDWebImageWebPCoder (0.6.1):
  • libwebp (~> 1.0)
  • SDWebImage/Core (~> 5.7)

On that image shown memory usage when 28 images were rendered. Size of each image about 40-80 kb.

About code usage.

I render images using ImageBackground (which uses Image component. Native code is here) component.

Also to add support of webp I've used react-native-webp-format (native code is here).

- (RCTImageLoaderCancellationBlock)decodeImageData:(NSData *)imageData
                                              size:(CGSize)size
                                             scale:(CGFloat)scale
                                        resizeMode:(RCTResizeMode)resizeMode
                                 completionHandler:(RCTImageLoaderCompletionBlock)completionHandler
{
    UIImage *image = [[SDImageWebPCoder sharedCoder] decodedImageWithData:imageData options:nil];
    if (!image) {
      completionHandler(nil, nil);
      return ^{};
    }
    completionHandler(nil, image);
    return ^{};
}

Also a little bit more info about images:
I've generated them using imagick with the following options.

            $image->getImagick()->setOption('webp:low-memory', 'true');
            $image->getImagick()->setOption('webp:method', '6');
            $image->getImagick()->setImageFormat('WEBP');
            $image->getImagick()->unsharpMaskImage(0.25, 0.25, 8, 0.065);
            $image->getImagick()->setInterlaceScheme(\Imagick::INTERLACE_NO);
            $image->getImagick()->setColorspace(\Imagick::COLORSPACE_SRGB);
            $image->getImagick()->setCompression(\Imagick::COMPRESSION_LZMA);
            $image->getImagick()->setImageCompressionQuality(80);

Also image sample attached (i had put it into zip because github does not allow to attach webp).
thumb_dcee16ff-ac19-4452-8784-605ff9e39848_photos_medium.webp.zip

[Stafox]

@dreampiggy hi. Have you found a time to take a look on that?

[dreampiggy]

Does this issue still exist ?

I doubt Memory Leak you described here. The image RAM usage follows the fomula:

width * height * bytesPerPixel (4 for normal alpha RGBA8888 image)

So, if there is one image with 2000x2000 pixels, it must allocate 15.2587890625 MB.
For 8000x8000 pixels, need 240MB.

[Stafox]

yes for iOS 12 & 13. in my case all images have a size 1080x835. So the expected memory usage should be ((10808354)/1024)/1024=3.44009399414 per image. In my mind the image should not be rendered in case when it is not visible.
So, I can't imagine how it possible to see memory usage over 500mb (because about 145 images should be rendered).

for now I can't use webp for ios because of lelaks for old OS versions. So, I use the same set of images but jpeg, and app memory usage about 70-80mb.

[DanielZanchi]

it can decode even animated images (webp).
Consider 80 frames for that amount and it will have a memory leak.

[dreampiggy]

yes for iOS 12 & 13. in my case all images have a size 1080x835. So the expected memory usage should be ((1080_835_4)/1024)/1024=3.44009399414 per image. In my mind the image should not be rendered in case when it is not visible.
So, I can't imagine how it possible to see memory usage over 500mb (because about 145 images should be rendered).

for now I can't use webp for ios because of lelaks for old OS versions. So, I use the same set of images but jpeg, and app memory usage about 70-80mb.

Apple's JPEG/GIF codec use some trick to avoid peak memory usage. It's tied Image/IO codec and UIKit render engine.

However, libwebp is outside of ImageIO, so once the image is not visible, UIImageView (iOS's rendering component) can not get the context to release the frame buffer.

If you use Static WebP mostlly. Using iOS 14's built-in support for WebP is better.

This WebPCoder is mostlly used for Animated WebP. Which is still not supported by Apple.

[dreampiggy]

Some trick for optimization about rendering RAM usage:
https://github.com/SDWebImage/SDWebImage/wiki/Common-Problems#configuration-to-reduce-memory-pressure-aggressive

The most important one is that SDWebImageAvoidDecodeImage, which will not force to re-draw the bitmap image decoded by libwebp. However, the bad effect that this may slove down some FPS (the decoding is happended during rendering on screen, not in advance during loading)

---

Actually, for JPEG/PNG, the hardware-accelerated decoding speed is fast enough so Apple decide to decoding when rendering on screen. It's a lazy algorithm, and they can also release the frame buffer when disappear (next time need re-decoding), works great for most small images.

However, for WebP, the decoding speed is slow, so if we decoding when rendering on screen, you will see a visible lag because the decoding blocking the rendering on main queue.

[Stafox]

Some trick for optimization about rendering RAM usage:
https://github.com/SDWebImage/SDWebImage/wiki/Common-Problems#configuration-to-reduce-memory-pressure-aggressive

Thanks. Will try this solution.

[DanielZanchi]

doesn't seem a possible solution if I need to convert an animated UIimage to webp Data.
At the moment I am using:
let webPData = SDImageCodersManager.shared.encodedData(with: newAnimatedimage, format: .webP, options: [SDImageCoderOption.encodeMaxFileSize: 1024 * 45])

But the conversion is very slow and takes up a lot of RAM, many times it fails for memory leak.

[dreampiggy]

If you can not get the suitable desired file size for WebP encoding, you'd better use compressionQuality, not that encodeMaxFileSize. For example, if one image which contains 4000x4000 pixels and it's a complicated graphics, product 45KB webp file size may cause libwebp to iterate more than 1000 times to reduce the accurate, which may cause RAM peak. (Not memory leak, just because it need that RAM for VP8 codec). You can just do the same thing using libwebp command line on the Mac to see about how slow libwebp encoding with best size is.

And, this issue and topic is talked about WebP Decoding, not Encoding...If there are something issue in encoding, we'd better open a new issue instead.

[dreampiggy]

And, for WebP Encoding, libwebp provide one config:

WebPConfig.low_memory = 1;

What about use this to have a try ? SDWebImageWebPCoder can map the options from Objective-C code to C-level API, see:
https://github.com/SDWebImage/SDWebImageWebPCoder/blob/master/SDWebImageWebPCoder/Classes/SDWebImageWebPCoderDefine.h#L51

[DanielZanchi]

Hello, thank you for the reply. I did try that option enabling low memory usage but it's very slow.
However i managed to do this even using more RAM but reducing the single frame size.

And yes, the encodeMaxFileSize was an issue. I am using compressionQuality for the moment.

I would like to do a faster encoding of an animated [SDImageFrame]. It takes about 10 seconds for a 60 frames animated image. (500x500px)

This is what I am doing at the moment:

let resizedData = SDImageCodersManager.shared.encodedData(with: newAnimatedimage, format: .webP, options: [SDImageCoderOption.encodeCompressionQuality: compression])

Where newAnimatedImage is [SDImageFrame] of about 20-100 frames.

[dreampiggy]

For faster encoding (don't care about the output size, it's a trade-off), there are more options from libwebp's manual:

1. WebPConfig.method = 6 (cwebp's -m6)
2. WebPConfig.quality = 0 (cwebp's -q)

More: https://developers.google.com/speed/webp/docs/cwebp, many lossy options may effect the encoding time and quality.

Actually I'm not expect at webp encoding. I focus on integrate the webp to iOS platform with easy-to-use interface...

[dreampiggy]

You can also use SDImageCoderEncodeMaxPixelSize to limit the pixel size for encoding. Don't need to scale down by yourself before encoding. We provide the common support for thumbnail encoding, actually.

https://github.com/SDWebImage/SDWebImageWebPCoder/blob/0.8.2/SDWebImageWebPCoder/Classes/SDImageWebPCoder.m#L870

[DanielZanchi]

I understand that there is not good solution for keeping an animated WebP under 500KB with a 500x500px with a fast encoding.
I am resizing the frames before because the resizing method I am using for the frames is very fast. I loose a lot of time during the encoding.