ScanTranscode - Convert New/Uncommon Image Formats #324
Conversation
| def scan(self, data, file, options, expire_at): | ||
| output_format = options.get("output_format", "jpeg") | ||
|
|
||
| def convert(im): | ||
| with io.BytesIO() as f: | ||
| im.save(f, format=f"{output_format}", quality=90) | ||
| return f.getvalue() | ||
|
|
||
| # Send extracted file back to Strelka | ||
| self.emit_file(convert(Image.open(io.BytesIO(data))), name=file.name) | ||
|
|
||
| self.flags.append("transcoded") |
There was a problem hiding this comment.
I know we discussed global exception handling - curious how you want to approach adding exceptions here. My first thought would be something like...
def scan(self, data, file, options, expire_at):
output_format = options.get("output_format", "jpeg")
def convert(im):
with io.BytesIO() as f:
try:
im.save(f, format=f"{output_format}", quality=90)
return f.getvalue()
except ValueError:
self.flags.append(f"{self.__class__.__name__} Exception: Invalid format or quality.")
except OSError:
self.flags.append(f"{self.__class__.__name__} Exception: Unsupported format or invalid image file.")
except AttributeError:
self.flags.append(f"{self.__class__.__name__} Exception: Data is not a bytes-like object.")
except Exception as e:
self.flags.append(f"{self.__class__.__name__} Exception: {str(e)[:50]}")
# Send extracted file back to Strelka
try:
self.emit_file(convert(Image.open(io.BytesIO(data))), name=file.name)
except Exception as e:
self.flags.append(f"{self.__class__.__name__} Exception: Failed to emit file")
return
self.flags.append("transcoded")
Too much? Too specific to scanner?
There was a problem hiding this comment.
I catch UnidentifiedImageError now, which is what's thrown when a broken image is loaded in to Image.open. Fair to do if we expect a lot of exceptions from badly formatted or truncated image files. I'd like to add in other specific exceptions as they come up while running Strelka.
Added a test for a broken image.
I'm not keen on a broader catch. If emit_file() itself fails, I think that should be handled inside of emit_file(). I added code to add a flag when emit_file fails, and log the exception. If emit_file fails, it's likely due to a coordinator connectivity problem, and we shouldn't suppress those exceptions.
Describe the change
HEIC/AVIF/HEIF images are appearing more frequently as platforms like mobile devices use these new encoding schemes to increase compression performance. Support for these codecs is missing in common tools and modules, like the
tesseractdependency in Strelka for OCR.The transcode process is relatively fast, e.g. 0.023 sec for a ~100KB image.
Adds a scanner that can convert e.g. HEIC/AVIF/HEIF files into other formats. Default conversion is to jpeg with quality 90 (preserves OCR quality without inflating file sizes).
Adds tests for three new fixtures transcoded into each of the six available formats, one test for broken images.
Adds negative matches in ScanJpeg, ScanEofPng, ScanEofBmp, ScanNf, ScanLsb for ScanTranscode.
Improves exception handling in emit_file().
Describe testing procedures
============================= test session starts ==============================
platform linux -- Python 3.10.6, pytest-7.2.0, pluggy-1.0.0
rootdir: /strelka
plugins: mock-3.10.0, unordered-0.5.2
collected 121 items
tests/test_scan_tar.py .
tests/test_scan_tlsh.py .
tests/test_scan_transcode.py ...................
tests/test_scan_upx.py .
tests/test_scan_url.py ..
====================== 122 passed, 28 warnings in 45.33s =======================
Sample output
Sample transcode:
{"file":{"depth":0,"flavors":{"mime":["image/avif"]},"name":"src/python/strelka/tests/fixtures/test_qr.avif","scanners":["ScanEntropy","ScanExiftool","ScanFooter","ScanHash","ScanHeader","ScanTlsh","ScanTranscode","ScanYara"],"size":49499,"tree":{"node":"d03763ca-52d3-4375-b1df-efeb62bfa1a1","root":"d03763ca-52d3-4375-b1df-efeb62bfa1a1"}},"request":{"attributes":{"filename":"src/python/strelka/tests/fixtures/test_qr.avif"},"client":"go-oneshot","id":"d03763ca-52d3-4375-b1df-efeb62bfa1a1","source":"ubuntu","time":1676416991},"scan":{"entropy":{"elapsed":0.000057,"entropy":7.995014927024274},"exiftool":{"elapsed":0.149846,"keys":[{"key":"ImageWidth","value":999},{"key":"ImageHeight","value":609}]},"footer":{"backslash":"\\x8c\\x98\\xd3\\xf0\\x0f\\x96\\r\\xce\\x14\\xcf\\xa0\\xda~P\\xdaOx\\\\A\\xa7br\\xd0A\\x04;S\\x87\\xe3 I\\xcdI\\x8c[b\\x80\\x85\\xcc\\x1f\\xdf]k\\xae\\x86\\xc5s\\x8dm\\xc0","elapsed":0.000036,"footer":"����\u000f�\r�\u0014Ϡ�~P�Ox\\A�br�A\u0004;S�� I�I�[b���\u001f�]k���s�m�"},"hash":{"elapsed":0.003897,"md5":"2609004bb9c371a869fa324b0fabdf5c","sha1":"9271dffbecf652db7803d0a930091c462428b18a","sha256":"f188590f54b3cbf60af59af4a797af5b6441a6de6b4bacb191417c2c8c71b1e9","ssdeep":"1536:DzNgGMG0z6S54BaP3phIkuboHbtwOeSwY:SB6aP5h6boHbtzr","tlsh":"T11D23027E3252DC22D96B857E5BD31316451210D901AE43AF3A7CF792432E227F9C793A"},"header":{"backslash":"\\x00\\x00\\x00\\x1cftypavif\\x00\\x00\\x00\\x00avifmif1miaf\\x00\\x00\\x00\\xeameta\\x00\\x00\\x00\\x00\\x00\\x00\\x00!hdlr\\x00\\x00","elapsed":0.000037,"header":"\u0000\u0000\u0000\u001cftypavif\u0000\u0000\u0000\u0000avifmif1miaf\u0000\u0000\u0000�meta\u0000\u0000\u0000\u0000\u0000\u0000\u0000!hdlr\u0000\u0000"},"tlsh":{"elapsed":0.001291},"transcode":{"elapsed":0.02379,"flags":["transcoded"]},"yara":{"elapsed":0.001789,"matches":["test"]}}} {"file":{"depth":1,"flavors":{"mime":["image/jpeg"],"yara":["jpeg_file"]},"name":"src/python/strelka/tests/fixtures/test_qr.avif","scanners":["ScanEntropy","ScanExiftool","ScanFooter","ScanHash","ScanHeader","ScanJpeg","ScanOcr","ScanQr","ScanTlsh","ScanYara"],"size":152620,"source":"ScanTranscode","tree":{"node":"8320996b-8446-4a10-b4c2-4e6b5317f58d","parent":"d03763ca-52d3-4375-b1df-efeb62bfa1a1","root":"d03763ca-52d3-4375-b1df-efeb62bfa1a1"}},"request":{"attributes":{"filename":"src/python/strelka/tests/fixtures/test_qr.avif"},"client":"go-oneshot","id":"d03763ca-52d3-4375-b1df-efeb62bfa1a1","source":"ubuntu","time":1676416991},"scan":{"entropy":{"elapsed":0.000138,"entropy":7.904183260122295},"exiftool":{"elapsed":0.134133,"keys":[{"key":"ImageWidth","value":999},{"key":"ImageHeight","value":609}]},"footer":{"backslash":"(\\xa2\\x80\\n(\\xa2\\x80\\n(\\xa2\\x80\\n(\\xa2\\x80\\n(\\xa2\\x80\\n(\\xa2\\x80\\n(\\xa2\\x80\\n(\\xa2\\x80\\n(\\xa2\\x80\\n(\\xa2\\x80\\n(\\xa2\\x80\\n(\\xa2\\x80?\\xff\\xd9","elapsed":0.000037,"footer":"(��\n(��\n(��\n(��\n(��\n(��\n(��\n(��\n(��\n(��\n(��\n(��?��"},"hash":{"elapsed":0.002865,"md5":"e22c8e84ca56433a9daca303d67d26f8","sha1":"93f040ddcf843092fd1583ff30c95045df3bd939","sha256":"4873a11562246dd8627e7688980934a76f0f4dd8223a58eee60dfab37ee926ce","ssdeep":"3072:2vqrjJqvKOpCGdZfF48LgIqo0VLEwMXs0OCPlj9AVBz1vv:2viOphFdLghosLFCPlSBxX","tlsh":"T19FE3E0138E658F93A5ADD3BCBE934E319F8C161CF5A232EA40660D8637A52264C4F51E"},"header":{"backslash":"\\xff\\xd8\\xff\\xe0\\x00\\x10JFIF\\x00\\x01\\x01\\x00\\x00\\x01\\x00\\x01\\x00\\x00\\xff\\xdb\\x00C\\x00\\x03\\x02\\x02\\x03\\x02\\x02\\x03\\x03\\x03\\x03\\x04\\x03\\x03\\x04\\x05\\x08\\x05\\x05\\x04\\x04\\x05\\n\\x07\\x07\\x06","elapsed":0.00002,"header":"����\u0000\u0010JFIF\u0000\u0001\u0001\u0000\u0000\u0001\u0000\u0001\u0000\u0000��\u0000C\u0000\u0003\u0002\u0002\u0003\u0002\u0002\u0003\u0003\u0003\u0003\u0004\u0003\u0003\u0004\u0005\u0008\u0005\u0005\u0004\u0004\u0005\n\u0007\u0007\u0006"},"jpeg":{"elapsed":0.007548,"flags":["no_trailer"]},"ocr":{"elapsed":0.900662,"text":["Lorem","Ipsum","Lorem","ipsum","dolor","sit","amet,","consectetur","adipiscing","elit.","Cras","lobortis","sem","dui.","Morbi","at","magna","quis","ligula","faucibus","consectetur","feugiat","at","purus.","Sed","nec","lorem","nibh.","Nam","vel","libero","odio.","Vivamus","tempus","non","enim","egestas","pretium.","Vestibulum","turpis","arcu,","maximus","nec","libero","quis,","imperdiet","suscipit","purus.","Vestibulum","blandit","quis","lacus","non","sollicitudin.","Nullam","non","convallis","dui,","et","aliquet","risus.","Sed","accumsan","ullamcorper","vehicula.","Proin","non","urna","facilisis,","condimentum","eros","quis,","suscipit","purus.","Morbi","euismod","imperdiet","neque","fermentum","dictum.","Integer","aliquam,","erat","sit","amet","fringilla","tempus,","mauris","ligula","blandit","sapien,","et","varius","sem","mauris","eu","diam.","Sed","fringilla","neque","est,","in","laoreet","felis","tristique","in.","Donec","luctus","velit","a","posuere","posuere.","Suspendisse","sodales","pellentesque","quam."]},"qr":{"data":"https://www.example.com/","elapsed":0.039702,"type":"url"},"tlsh":{"elapsed":0.001294},"yara":{"elapsed":0.000269,"matches":["test"]}}}Same image without transcoding:
Checklist