Inserted images don't show up in Adobe Illustrator

[vs-777]

I've written a script that generates qr codes and inserts them onto a pdf template using the insertImage module. While testing, I noticed that the inserted images do not show up in Illustrator, even though the pdf looks fine in Acrobat, Chrome, Firefox, etc. Although, I've noticed that the images sometimes only load a few seconds after the empty template. Any ideas what is causing this? I've attached the pdf.
page0.pdf

[JorjMcKie]

I am not using illustrator, so cannot really reproduce the problem.
But I saw a somewhat weird situation in the Optional Contents configuration:

>>> doc.pdf_catalog()
1
>>> print(doc.xref_object(1))
<<
  /Metadata 2 0 R
  /OCProperties <<
    /D <<
      /ON [ 5 0 R 244 0 R ]
      /Order 245 0 R
      /RBGroups [ ]
    >>
    /OCGs [ 5 0 R 244 0 R ]
  >>
  /Pages 3 0 R
  /Type /Catalog
>>
>>> print(doc.xref_object(245))
[ 244 0 R ]
>>> 

The /Order entry (indirectly) says that OCG 244 is offered as a switchable layer. If switched OFF, the text entries disappear and only the QR images remain displayed.
Optional Content Group 5 0 R seems never to be used. Its definition also exactly equals that of 244 0 R. So maybe Illustrator cannot handle optional content at all or cannot cope with this unusual configuration.
After all, the page definition consists of 88,618 command lines (and has a size of almost 2 MB), of which the first 88,199 are dealing with all those text rectangles (and are under control of the optional content group 244 0 R), and only the remaining 418 lines display all the QR images.
The size of the page definition also explains, why there is a noticeable response time displaying it.
So, what can you do?

• You could insert your QR with overlay=False, this will cause the QR display commands come first in the page definition. Maybe makes it easier for Illustrator to understand what to do.
• Depending on whether those text boxes are needed at all, you could delete them permanently
• Or you could remove their conditional display and always show them, effectively ignoring the optional content mechanism
• Or you could try to cleanup the optional content configuration a little: doc.set_layer(-1, on=[244]), so that there is no more mentioning of the unused OCG 5 0 R.

[vs-777]

Hi, thank you very much for the quick reply!

Still stuck on the following:

• When I set overlay=False all the qr codes get put below the template.
• How can I remove the conditional display and always show them?
• set_layer does not work, I've looked in the documentation and I presume you meant set_layer_config? I tried the latter, but get the following error: self.template_pdf.set_layer_config(-1, on=[244]) TypeError: setLayerConfig() got an unexpected keyword argument 'on'

[JorjMcKie]

When I set overlay=False all the qr codes get put below the template.

Of course - just an idea. Thought the template might be transparent. Bad luck then.

>>> import fitz
>>> help(fitz.Document.set_layer)
Help on function set_layer in module fitz.fitz:

set_layer(self, config, basestate=None, on=None, off=None, rbgroups=None)
    Set the ON, OFF, RBGroups PDF keys of an OC layer.

>>> 

What is your version?

Anyway, to remove the optional content instructions, use the following snippet:

# after finishing your image inserts
page.cleanContents()  # clean the syntax, join multiple contetns objects
xref = page.getContents()[0]  # get xref of resulting single object
cont_lines = doc.xrefStream(xref).splitlines()  # read it, break up into lines
for i in range(len(cont_lines)):
    if cont_lines[i].startswith((b"/OC", b"EMC")):
        cont_lines[i] = b""
cont = b"\n".join(cont_lines)
doc.updateStream(xref, cont)
# now save the document
# using "garbage=4, deflate=True" should result in a file of about 60% of original size

This makes everything unconditionally visible. Hopefully Illustrator is at least smart enough to digest the result ...

[vs-777]

Hi, unfortunately none of the fixes worked. Still getting the same problems!

Would it be possible to merge the layers of the qr codes and the empty template, i.e. flattening the pdf? Is this what the unconditional visibility snippet you provided does?

Looking through the documentation, the set_layer_config seemed to be relevant Activates the ON / OFF states of OCGs as defined in this layer. If as_default=True, then additionally all layers, including the standard one, are merged and the result is written back to the standard layer, and all optional layers are deleted. I played around with it, but didn't get it work. Also, it seems that the function is gone in 1.18.5.

[JorjMcKie]

No that function is still there, but has changed name.
Anyway it won't solve your problem, if the one I mentioned did not do that.

i.e. flattening the pdf? Is this what the unconditional visibility snippet you provided does?

No, my snippet let the template content completely ignore the optional content instructions.
This was based on the hope that Illustrator could now it.

But flattening may be an excellent idea, like this:

• Before inserting the QR images, flatten the page by replacing it with its image - still showing the template content of course.
• Onto that image insert the QR images

The steps would be:

1. make a page pixmap
2. delete the page
3. make a new page of same size
4. insert pixmap into new page covering it completely
5. insert QR images into new page

pix = page.getPixmap()
rect = page.rect
pno = page.number
doc.deletePage(pno)  # delete the page, create equal sized new one
page=doc.newPage(pno, width=rect.width, height=rect.height)
page.insertImage(page.rect, pixmap=pix)  # insert image of prev page
# insert QR images here
# ...
doc.save(..., garbage=4, deflate=True)

Depending on the pixmap precision, the result will look very much like before.
If the template content seems too blurred, increase the pixmap resolution via pix = page.getPixmap(matrix=fitz.Matrix(zoom, zoom)), where zoom > 1.

[JorjMcKie]

Just did a few experiments: you should probably use at least fitz.Matrix(2, 2) to see a satisfactory quality of the template content.
By the way: if it is important to keep the template content switchable between on and off, that can also be done with this solution ... let me know.

[vs-777]

Hi! Unfortunately I require the output to be in pdf/vector format as the templates will require high quality printing. I've decided to leave it as is, as long as printing goes directly through a software that isn't Illustrator, it should be fine.

[vs-777]

I appreciate all the help. Thanks a lot for making this library, it is a life saver.