Page Labels -- IMPLEMENTED

[chapmanwilliam]

Amazing library. But it is missing something fundamental: support for page labels.

Mr Bloomfield claims he has provided such a feature here in Sep 2015, but I don’t think it has been incorporated.

https://bugs.ghostscript.com/show_bug.cgi?id=695351

Any chance that could be done?

With many thanks in advance.

[JorjMcKie]

It has not been incorporated in MuPDF.
It certainly could be done in PyMuPDF - although I am reluctant in those cases. But I also have to admit, that I have done this sort of thing several times, for example PDF embedded files and advanced appearance features for annotations.

But anyway: could you please elaborate a bit on what you would like to see?

• Just detecting the label of a page, e.g. and provide it as an attribute page.label, whichg could be None or a string?
• Or also make it usable via doc.loadPage(label="string")?
• What else? Assigning labels to pages?

[chapmanwilliam]

Can you send me your script that returns the label for a given page? That would give me a start.

I'll have a crack at the rest in python.

What's PR?

[JorjMcKie]

PR = Pull Request
You make changes to your PyMuPDF fork as you feel appropriate. Once finished testing, tell GitHub to create a PR. This automatically creates source code deltas relative to the main repo.
I will be notified, look into it and make it part of the next version.

page-labels.zip

The script creates a Python dict from the array stored as /PageLabels<</Nums[...]>> in the catalog. As I wrote, this is now simply a string extracted from the catalog object - to be later replaced by a C function.
It supports numeric labels and Roman numbers currently. Other formats (prefixes, etc.) would have to be added, but shouldn't be a major challenge.
A decision must be made, when this dict is created: at the time (and every time), the label is asked for, or rather during PDF open ...
I imagine to create a page method def label(self): ... and make it a @property.

The reverse thing (label to page number list) should be a Document method, but otherwise use that same dictionary and walk through its values.

The harder part is building such a Python dictionary and then convert and store it as a PDF array.

Technically, it is probably a lot of work to make up your own environment to build PyMuPDF from sources using your fork as source code.
The better idea may therefore be adding these methods as ordinary functions to utils.py. Also modify __init__.py to assign your functions to the Page and Document classes. There are already several examples like this in the init file.
This will allow testing without building PyMuPDF from sources.

[chapmanwilliam]

Many thanks. Let you know how I get on. William

[chapmanwilliam]

Here is my attempt: here

Note that pdf storage of page labels is more complicated than your code assumed. Where there are many different pagelabels, adobe stores these by reference. So the references have to be expanded first.

There are two basic functions: reading (getpgLabelMapping(doc)) and writing (update_catalog_nums(doc, labels)). The comments at the top should be sufficient.

William

[JorjMcKie]

ok, thanks.
I will have a look into this asap.

[JorjMcKie]

@chapmanwilliam
Looks promising, thank you!
As I mentioned before, there will be C functions that directly read and write the stuff between the square brackets /PageLables<</Nums[stuff]>>. These functions also automatically take care of any indirect references (nnn 0 R type of things). So there is no need to resolve those yourself. Same thing with parsing your way through the zillions of other PDF catalog entries.
This is taken care of.

As per writing the labels entry, I think your contribution is more or less final enough to become a Document method.
I see have performance / memory concerns however with getting a page's label, or finding a label's page(s):

• Finding the page label should ideally become a Page method (or property). This should not lead to creating two potentially very large Python objects every single time.
• Not making a page label property, but just creating those two objects "on request" as the result of a Document method is of course possible too, but seems less user-friendly. But even in this case, label computation is possible "on the fly" with minimal memory usage.
• For computing the list of pages, that match a given label, must of course be a Document method. The question here again is whether having two large objects sitting around and doing nothing most of the time is really the best thing to have ...

What I will do next is to make two (private) Document methods:

1. Document._get_page_labels() directly returns "stuff" as indicated above, as a string. If no labels exists: "". No need to bother with indirect references or other PDF catalog entries.
2. Document._set_page_labels(stuff) directly writes the result of your function create_nums() - without the wrappers Nums[ and ] - to the catalog. Automatically either overwrites what is there or creates that entry.

Looking forward to your reaction.

[chapmanwilliam]

Dear Mr McKie,

Delighted at the prospect.

Is it possible to access the C functions that write entries to and from the catalog directly from PyMuPDF?

Page label - I agree this should be a property, or, probably more likely a method of page. The method could simply process your document._get_page_labels() method.

Happy with the proposed document methods ._get_page_labels() and .set_page_labels().

._get_page_labels(). I think you are proposing to return a string (or a list of strings) where each entry looks like 0<</S/D>>2<</S/r>>8<</S/D>>. That would still require some parsing to make it usable but that's not a biggie. Would you consider parsing the string as part of the functions?

Ditto for ._set_page_labels() but in reverse.

Thank you for looking at this.

Best wishes,

William

[JorjMcKie]

Hi William,

just call me Jorj 😎.
I already made the methods. they work like described:

>>> doc=fitz.open("pillow.pdf")
>>> doc._get_page_labels()
'0<</S/D>>2<</S/r>>6<</S/D>>'
>>> # if without labels, "" is returned

So your code parsing that string is fully applicable. The function is extremely fast - as is typical for MuPDF. So computing the label for any page number simply needs to check in which of the intervals (0, 1), (2, 5), (6, <end>) the page number falls and apply the corresponding rule.

For a computation "label => pno-list" one could use a Python list comprehension:

pno_list = [page.number for page in doc if page.label() == label]

Which is not the most performant way, but extremely simple.
I suspect, that a list of all applicable page numbers is not usually required. So one could accept the first page.number hit:

for page in doc:
    if page.label() == label:
        break

This is obviously faster, because not all pages are loaded.

Even better would be a function that returns a label for a given integer ... without needing a page object for this. This function can then be called for computing page.label(), but also in the list comprhension: instead of using the pages, we would use the documents page number range:

pno_list = [pno for pno in range(doc.pageCount) if label_function(pno) == label]
# etc.

For Creating / updating the page label info, the code would be like this:

doc._set_page_labels(create_nums(labels))
# with **your** function 'create_nums' - that's it.
# only omit prefix "Nums[" and suffix "]"

I believe you use Max OSX? Let me generate a pre-release wheel. When done I will let you from where to download.

[JorjMcKie]

but page.label() did not work. I got an error that Page object has no attribute 'label'.

Of course ... that is something I thought you would implement 😆 ... based on what you already coded.

the _get_page_labels() works (albeit it returns indirect references for me)

Oha! Do you have an example PDF? I know what needs to be done, but need a reproducer

[chapmanwilliam]

Yes, of course, I can do a page.label()

Here is an example. The first dozen pages are roman numerals. The _get_label_pages() function returns: 0 5512 0 R 12 5513 0 R

[chapmanwilliam]

PS - on a large file (4,000 pages) with complicated page labelling, the _get_label_pages() returned "" even though there is a clear /PageLabels with an indirect reference.

The catalog looks like this here. Unfortunately, I can't send you the pdf file because it is confidential.

[JorjMcKie]

The first dozen pages are roman numerals. The _get_label_pages() function returns: 0 5512 0 R 12 5513 0 R

Incredible - but a valuable example, thank you.
This requires to modify the _get_page_labels() method significantly.
I will let you know when I have a new version generated.

As per the page.label()`` method: Try to code a function in utils.py, e.g. def page_label(page): ...`. Then add the following statement in __init__.py:

fitz.Page.label = fitz.utils.page_label

This approach will allow to test your function without having to recompile PyMuPDF.
Similar for the document methods ...

[chapmanwilliam]

When I run my expand_indirect_references() function on this reference I get:

/PageLabels <</Kids[<</Limits[0 640]/Nums[0 <</S/r>> 29 <</P(A)/S/D>> 35 <</P(A6.)/S/D>> 36 <</P(A)/S/D/St 9>> 38 <</P(A8.)/S/D>> 39 <</P(A)/S/D/St 9>> 164 <</P(B)/S/D>> 257 <</S/D>> 272 <</P(15)/S/a>> 273 <</S/D/St 16>> 276 <</P(18)/S/a>> 278 <</S/D/St 19>> 285 <</P(25)/S/a>> 286 <</S/D/St 26>> 292 <</P(31)/S/a>> 293 <</S/D/St 32>> 389 <</P(127)/S/a>> 390 <</S/D/St 128>> 394 <</P(131)/S/a>> 397 <</S/D/St 132>> 398 <</P(132)/S/a>> 399 <</S/D/St 133>> 401 <</P(134)/S/a>> 402 <</S/D/St 135>> 403 <</P(135)/S/a>> 404 <</S/D/St 136>> 406 <</P(137)/S/a>> 407 <</S/D/St 138>> 411 <</P(141)/S/a>> 415 <</S/D/St 142>> 417 <</P(143)/S/a>> 418 <</S/D/St 144>> 584 <</P(309)/S/a>> 586 <</S/D/St 310>> 640 <</P(363)/S/a>>]>> <</Limits[643 2768]/Nums[643 <</S/D/St 364>> 647 <</P(367)/S/a>> 651 <</S/D/St 368>> 677 <</P(393)/S/a>> 679 <</S/D/St 394>> 707 <</P(421)/S/a>> 708 <</S/D/St 422>> 710 <</P(423)/S/a>> 711 <</S/D/St 424>> 741 <</P(453)/S/a>> 742 <</S/D/St 454>> 762 <</P(473)/S/a>> 763 <</S/D/St 474>> 765 <</P(475)/S/a>> 767 <</S/D/St 476>> 773 <</P(481)/S/a>> 774 <</S/D/St 482>> 1149 <</P(856)/S/a>> 1150 <</S/D/St 857>> 1610 <</P(1316)/S/a>> 1611 <</S/D/St 1317>> 1923 <</P(1628)/S/a>> 1924 <</S/D/St 1629>> 2160 <</P(1864)/S/a>> 2161 <</S/D/St 1865>> 2165 <</P(1868)/S/a>> 2169 <</S/D/St 1869>> 2257 <</P(1956)/S/a>> 2259 <</S/D/St 1957>> 2469 <</P(2166)/S/a>> 2470 <</S/D/St 2167>> 2474 <</P(2170)/S/a>> 2477 <</S/D/St 2171>> 2481 <</P(2174)/S/a>> 2484 <</S/D/St 2175>> 2747 <</P(2437)/S/a>> 2750 <</S/D/St 2438>> 2766 <</P(2453)/S/a>> 2768 <</S/D/St 2454>>]>> <</Limits[2800 4113]/Nums[2800 <</P(2485)/S/a>> 2804 <</S/D/St 2486>> 2853 <</P(2534)/S/a>> 2857 <</S/D/St 2535>> 2905 <</P(2582)/S/a>> 2907 <</S/D/St 2583>> 2910 <</P(2585)/S/a>> 2912 <</S/D/St 2586>> 2937 <</P(2610)/S/a>> 2940 <</S/D/St 2611>> 3001 <</P(2671)/S/a>> 3004 <</S/D/St 2672>> 3027 <</P(2694)/S/a>> 3029 <</S/D/St 2695>> 3069 <</P(2734)/S/a>> 3072 <</S/D/St 2735>> 3228 <</P(2890)/S/a>> 3230 <</S/D/St 2891>> 3278 <</P(2938)/S/a>> 3279 <</S/D/St 2939>> 3356 <</P(3015)/S/a>> 3358 <</S/D/St 3016>> 3417 <</P(3074)/S/a>> 3418 <</S/D/St 3075>> 3455 <</P(3111)/S/a>> 3456 <</S/D/St 3112>> 3483 <</P(3138)/S/a>> 3486 <</S/D/St 3139>> 3487 <</P(3139)/S/a>> 3488 <</S/D/St 3140>> 3501 <</P(3152)/S/a>> 3502 <</S/D/St 3153>> 3516 <</P(3166)/S/a>> 3520 <</S/D/St 3167>> 3521 <</P(3167)/S/a>> 3522 <</S/D/St 3168>> 3524 <</P(3169)/S/a>> 3526 <</S/D/St 3170>> 3631 <</P(3274.)/S/D>> 3639 <</S/D/St 3275>> 3878 <</P(3513.)/S/D>> 3883 <</S/D/St 3514>> 3914 <</P(D)/S/D>> 4109 <</P(D195.)/S/D>> 4113 <</P(D)/S/D/St 196>>]>>]/Nums[]>>

[JorjMcKie]

on a large file (4,000 pages) ...

The catalog just contains /PageLabels 49945 0 R, so could you please send me the print of xref 49945 too please?

[chapmanwilliam]

When I run my expand_indirect_references() function on this reference I get:

/PageLabels <</Kids[<</Limits[0 640]/Nums[0 <</S/r>> 29 <</P(A)/S/D>> 35 <</P(A6.)/S/D>> 36 <</P(A)/S/D/St 9>> 38 <</P(A8.)/S/D>> 39 <</P(A)/S/D/St 9>> 164 <</P(B)/S/D>> 257 <</S/D>> 272 <</P(15)/S/a>> 273 <</S/D/St 16>> 276 <</P(18)/S/a>> 278 <</S/D/St 19>> 285 <</P(25)/S/a>> 286 <</S/D/St 26>> 292 <</P(31)/S/a>> 293 <</S/D/St 32>> 389 <</P(127)/S/a>> 390 <</S/D/St 128>> 394 <</P(131)/S/a>> 397 <</S/D/St 132>> 398 <</P(132)/S/a>> 399 <</S/D/St 133>> 401 <</P(134)/S/a>> 402 <</S/D/St 135>> 403 <</P(135)/S/a>> 404 <</S/D/St 136>> 406 <</P(137)/S/a>> 407 <</S/D/St 138>> 411 <</P(141)/S/a>> 415 <</S/D/St 142>> 417 <</P(143)/S/a>> 418 <</S/D/St 144>> 584 <</P(309)/S/a>> 586 <</S/D/St 310>> 640 <</P(363)/S/a>>]>> <</Limits[643 2768]/Nums[643 <</S/D/St 364>> 647 <</P(367)/S/a>> 651 <</S/D/St 368>> 677 <</P(393)/S/a>> 679 <</S/D/St 394>> 707 <</P(421)/S/a>> 708 <</S/D/St 422>> 710 <</P(423)/S/a>> 711 <</S/D/St 424>> 741 <</P(453)/S/a>> 742 <</S/D/St 454>> 762 <</P(473)/S/a>> 763 <</S/D/St 474>> 765 <</P(475)/S/a>> 767 <</S/D/St 476>> 773 <</P(481)/S/a>> 774 <</S/D/St 482>> 1149 <</P(856)/S/a>> 1150 <</S/D/St 857>> 1610 <</P(1316)/S/a>> 1611 <</S/D/St 1317>> 1923 <</P(1628)/S/a>> 1924 <</S/D/St 1629>> 2160 <</P(1864)/S/a>> 2161 <</S/D/St 1865>> 2165 <</P(1868)/S/a>> 2169 <</S/D/St 1869>> 2257 <</P(1956)/S/a>> 2259 <</S/D/St 1957>> 2469 <</P(2166)/S/a>> 2470 <</S/D/St 2167>> 2474 <</P(2170)/S/a>> 2477 <</S/D/St 2171>> 2481 <</P(2174)/S/a>> 2484 <</S/D/St 2175>> 2747 <</P(2437)/S/a>> 2750 <</S/D/St 2438>> 2766 <</P(2453)/S/a>> 2768 <</S/D/St 2454>>]>> <</Limits[2800 4113]/Nums[2800 <</P(2485)/S/a>> 2804 <</S/D/St 2486>> 2853 <</P(2534)/S/a>> 2857 <</S/D/St 2535>> 2905 <</P(2582)/S/a>> 2907 <</S/D/St 2583>> 2910 <</P(2585)/S/a>> 2912 <</S/D/St 2586>> 2937 <</P(2610)/S/a>> 2940 <</S/D/St 2611>> 3001 <</P(2671)/S/a>> 3004 <</S/D/St 2672>> 3027 <</P(2694)/S/a>> 3029 <</S/D/St 2695>> 3069 <</P(2734)/S/a>> 3072 <</S/D/St 2735>> 3228 <</P(2890)/S/a>> 3230 <</S/D/St 2891>> 3278 <</P(2938)/S/a>> 3279 <</S/D/St 2939>> 3356 <</P(3015)/S/a>> 3358 <</S/D/St 3016>> 3417 <</P(3074)/S/a>> 3418 <</S/D/St 3075>> 3455 <</P(3111)/S/a>> 3456 <</S/D/St 3112>> 3483 <</P(3138)/S/a>> 3486 <</S/D/St 3139>> 3487 <</P(3139)/S/a>> 3488 <</S/D/St 3140>> 3501 <</P(3152)/S/a>> 3502 <</S/D/St 3153>> 3516 <</P(3166)/S/a>> 3520 <</S/D/St 3167>> 3521 <</P(3167)/S/a>> 3522 <</S/D/St 3168>> 3524 <</P(3169)/S/a>> 3526 <</S/D/St 3170>> 3631 <</P(3274.)/S/D>> 3639 <</S/D/St 3275>> 3878 <</P(3513.)/S/D>> 3883 <</S/D/St 3514>> 3914 <</P(D)/S/D>> 4109 <</P(D195.)/S/D>> 4113 <</P(D)/S/D/St 196>>]>>]/Nums[]>>

[JorjMcKie]

Okay, modified the Document._get_page_labels(). It will now no longer return a string, but a Python list. Each list item consists of a tuple (pno, rule).
I had to do this to let MuPDF resolve any indirect references for the rulesspecification. I hope we never stumble across an idiot who also provides the page number as an indirect reference ... well I actually could as well right now take of this too.

Anyway this is the output for your example PDF:

>>> import fitz
>>> doc=fitz.open("1 Court Guides.pdf")
>>> from pprint import pprint
>>> pprint(doc._get_page_labels())
[(0, '<</S/r>>'), (12, '<</S/D>>')]  # this is what you now have to parse
>>> doc.pdf_catalog()
4166
>>> print(doc.xref_object(4166))
<<
  /AcroForm 4176 0 R
  /Metadata 3415 0 R
  /PageLabels 4082 0 R  # indirect ref successfully resolved
  /Pages 4085 0 R
  /Type /Catalog
>>
>>> print(doc.xref_object(4082))
<<
  /Nums [ 0 4083 0 R 12 4084 0 R ]  # indirect refs successfully resolved
>>
>>> print(doc.xref_object(4083))
<<
  /S /r
>>
>>> print(doc.xref_object(4084))
<<
  /S /D
>>
>>> 

Old example also still works:

>>> doc=fitz.open("pillow.pdf")
>>> pprint(doc._get_page_labels())
[(0, '<</S/D>>'), (2, '<</S/r>>'), (6, '<</S/D>>')]
>>> 

I will let you know a new wheel generation.

[JorjMcKie]

New wheels are ready btw.

It amazes me how convoluted Adobe made pdf.

PDF spec is decades old. So lots of amendments, bells and whistles were added over time. Without being ever able to retire old stuff because of backward compatibility issues.
So you often have many alternatives to arrive at the same result. In addition, the PDF viewers also each have their own lifecycles and may or may not support the latest, greatest specification ...

[chapmanwilliam]

I tried the new wheel.

It works for the file I sent you.

Unfortunately, it still returns an empty string for the more complicated file I have here with (when expanded):

/PageLabels <</Kids[<</Limits[0 640]/Nums[0 <</S/r>> 29 <</P(A)/S/D>> 35 <</P(A6.)/S/D>> 36 <</P(A)/S/D/St 9>> 38 <</P(A8.)/S/D>> 39 <</P(A)/S/D/St 9>> 164 <</P(B)/S/D>> 257 <</S/D>> 272 <</P(15)/S/a>> 273 <</S/D/St 16>> 276 <</P(18)/S/a>> 278 <</S/D/St 19>> 285 <</P(25)/S/a>> 286 <</S/D/St 26>> 292 <</P(31)/S/a>> 293 <</S/D/St 32>> 389 <</P(127)/S/a>> 390 <</S/D/St 128>> 394 <</P(131)/S/a>> 397 <</S/D/St 132>> 398 <</P(132)/S/a>> 399 <</S/D/St 133>> 401 <</P(134)/S/a>> 402 <</S/D/St 135>> 403 <</P(135)/S/a>> 404 <</S/D/St 136>> 406 <</P(137)/S/a>> 407 <</S/D/St 138>> 411 <</P(141)/S/a>> 415 <</S/D/St 142>> 417 <</P(143)/S/a>> 418 <</S/D/St 144>> 584 <</P(309)/S/a>> 586 <</S/D/St 310>> 640 <</P(363)/S/a>>]>> <</Limits[643 2768]/Nums[643 <</S/D/St 364>> 647 <</P(367)/S/a>> 651 <</S/D/St 368>> 677 <</P(393)/S/a>> 679 <</S/D/St 394>> 707 <</P(421)/S/a>> 708 <</S/D/St 422>> 710 <</P(423)/S/a>> 711 <</S/D/St 424>> 741 <</P(453)/S/a>> 742 <</S/D/St 454>> 762 <</P(473)/S/a>> 763 <</S/D/St 474>> 765 <</P(475)/S/a>> 767 <</S/D/St 476>> 773 <</P(481)/S/a>> 774 <</S/D/St 482>> 1149 <</P(856)/S/a>> 1150 <</S/D/St 857>> 1610 <</P(1316)/S/a>> 1611 <</S/D/St 1317>> 1923 <</P(1628)/S/a>> 1924 <</S/D/St 1629>> 2160 <</P(1864)/S/a>> 2161 <</S/D/St 1865>> 2165 <</P(1868)/S/a>> 2169 <</S/D/St 1869>> 2257 <</P(1956)/S/a>> 2259 <</S/D/St 1957>> 2469 <</P(2166)/S/a>> 2470 <</S/D/St 2167>> 2474 <</P(2170)/S/a>> 2477 <</S/D/St 2171>> 2481 <</P(2174)/S/a>> 2484 <</S/D/St 2175>> 2747 <</P(2437)/S/a>> 2750 <</S/D/St 2438>> 2766 <</P(2453)/S/a>> 2768 <</S/D/St 2454>>]>> <</Limits[2800 4113]/Nums[2800 <</P(2485)/S/a>> 2804 <</S/D/St 2486>> 2853 <</P(2534)/S/a>> 2857 <</S/D/St 2535>> 2905 <</P(2582)/S/a>> 2907 <</S/D/St 2583>> 2910 <</P(2585)/S/a>> 2912 <</S/D/St 2586>> 2937 <</P(2610)/S/a>> 2940 <</S/D/St 2611>> 3001 <</P(2671)/S/a>> 3004 <</S/D/St 2672>> 3027 <</P(2694)/S/a>> 3029 <</S/D/St 2695>> 3069 <</P(2734)/S/a>> 3072 <</S/D/St 2735>> 3228 <</P(2890)/S/a>> 3230 <</S/D/St 2891>> 3278 <</P(2938)/S/a>> 3279 <</S/D/St 2939>> 3356 <</P(3015)/S/a>> 3358 <</S/D/St 3016>> 3417 <</P(3074)/S/a>> 3418 <</S/D/St 3075>> 3455 <</P(3111)/S/a>> 3456 <</S/D/St 3112>> 3483 <</P(3138)/S/a>> 3486 <</S/D/St 3139>> 3487 <</P(3139)/S/a>> 3488 <</S/D/St 3140>> 3501 <</P(3152)/S/a>> 3502 <</S/D/St 3153>> 3516 <</P(3166)/S/a>> 3520 <</S/D/St 3167>> 3521 <</P(3167)/S/a>> 3522 <</S/D/St 3168>> 3524 <</P(3169)/S/a>> 3526 <</S/D/St 3170>> 3631 <</P(3274.)/S/D>> 3639 <</S/D/St 3275>> 3878 <</P(3513.)/S/D>> 3883 <</S/D/St 3514>> 3914 <</P(D)/S/D>> 4109 <</P(D195.)/S/D>> 4113 <</P(D)/S/D/St 196>>]>>]>>

I wonder if the problem is the unusual keywords: /Kids and /Limits.

[JorjMcKie]

Right - I am not sure this is actually a legal specification. Need to look that up.
The keyword hierarchy supported so far is /PageLabels/Nums, hre we have an extra level /PageLabels/Kids/Nums.

[JorjMcKie]

As suspected: this is not a legal specification. Quote from the spec (p.166):

What I can do is checking for the second hierarchy if the first check delivered nothing.
If have no such illegal file, so I am dependent on you to test the new version coming ...

[chapmanwilliam]

I’ll check for you.

[JorjMcKie]

New wheel is generated. You can check now if your large example works.

[JorjMcKie]

In that new version, I have also integrated a first version of color support for table of contents as follows:

• reporting back color and text properties (bold, italic) is still missing
• you can set color properties of single arbitrary toc items by modifying the destination dictionary:

toc = doc.get_toc(simple=False)
item = toc[n]  # we want to modify item 'n'
d = item[3]  # the destination details dictionary of that item
d["color"] = (1,0,0)  # we want it red
d["italic"] = True  # and italic
d["bold"] = True  # and bold
doc.set_toc_item(n, dest_dict=d)  # update this item
doc.save(...)

Doing the above for item n=0 of the PyMuPDF docu yields this in some supporting viewer:

[JorjMcKie]

wheel generation finished.

[chapmanwilliam]

OK Jorj, I think it is reading the page labels fine now.

I am having difficulty saving the them using _set_page_labels(). What is the correct format of the parameter for this function.

I tried passing this string:

0<</S/r/St 1>>29<</P(A)/S/D/St 1>>35<</P(A6.)/S/D/St 1>>36<</P(A)/S/D/St 9>>38<</P(A8.)/S/D/St 1>>39<</P(A)/S/D/St 9>>164<</P(B)/S/D/St 1>>257<</S/D/St 1>>272<</P(15)/S/a/St 1>>273<</S/D/St 16>>276<</P(18)/S/a/St 1>>278<</S/D/St 19>>285<</P(25)/S/a/St 1>>286<</S/D/St 26>>292<</P(31)/S/a/St 1>>293<</S/D/St 32>>389<</P(127)/S/a/St 1>>390<</S/D/St 128>>394<</P(131)/S/a/St 1>>397<</S/D/St 132>>398<</P(132)/S/a/St 1>>399<</S/D/St 133>>401<</P(134)/S/a/St 1>>402<</S/D/St 135>>403<</P(135)/S/a/St 1>>404<</S/D/St 136>>406<</P(137)/S/a/St 1>>407<</S/D/St 138>>411<</P(141)/S/a/St 1>>415<</S/D/St 142>>417<</P(143)/S/a/St 1>>418<</S/D/St 144>>584<</P(309)/S/a/St 1>>586<</S/D/St 310>>640<</P(363)/S/a/St 1>>643<</S/D/St 364>>647<</P(367)/S/a/St 1>>651<</S/D/St 368>>677<</P(393)/S/a/St 1>>679<</S/D/St 394>>707<</P(421)/S/a/St 1>>708<</S/D/St 422>>710<</P(423)/S/a/St 1>>711<</S/D/St 424>>741<</P(453)/S/a/St 1>>742<</S/D/St 454>>762<</P(473)/S/a/St 1>>763<</S/D/St 474>>765<</P(475)/S/a/St 1>>767<</S/D/St 476>>773<</P(481)/S/a/St 1>>774<</S/D/St 482>>1149<</P(856)/S/a/St 1>>1150<</S/D/St 857>>1610<</P(1316)/S/a/St 1>>1611<</S/D/St 1317>>1923<</P(1628)/S/a/St 1>>1924<</S/D/St 1629>>2160<</P(1864)/S/a/St 1>>2161<</S/D/St 1865>>2165<</P(1868)/S/a/St 1>>2169<</S/D/St 1869>>2257<</P(1956)/S/a/St 1>>2259<</S/D/St 1957>>2469<</P(2166)/S/a/St 1>>2470<</S/D/St 2167>>2474<</P(2170)/S/a/St 1>>2477<</S/D/St 2171>>2481<</P(2174)/S/a/St 1>>2484<</S/D/St 2175>>2747<</P(2437)/S/a/St 1>>2750<</S/D/St 2438>>2766<</P(2453)/S/a/St 1>>2768<</S/D/St 2454>>2800<</P(2485)/S/a/St 1>>2804<</S/D/St 2486>>2853<</P(2534)/S/a/St 1>>2857<</S/D/St 2535>>2905<</P(2582)/S/a/St 1>>2907<</S/D/St 2583>>2910<</P(2585)/S/a/St 1>>2912<</S/D/St 2586>>2937<</P(2610)/S/a/St 1>>2940<</S/D/St 2611>>3001<</P(2671)/S/a/St 1>>3004<</S/D/St 2672>>3027<</P(2694)/S/a/St 1>>3029<</S/D/St 2695>>3069<</P(2734)/S/a/St 1>>3072<</S/D/St 2735>>3228<</P(2890)/S/a/St 1>>3230<</S/D/St 2891>>3278<</P(2938)/S/a/St 1>>3279<</S/D/St 2939>>3356<</P(3015)/S/a/St 1>>3358<</S/D/St 3016>>3417<</P(3074)/S/a/St 1>>3418<</S/D/St 3075>>3455<</P(3111)/S/a/St 1>>3456<</S/D/St 3112>>3483<</P(3138)/S/a/St 1>>3486<</S/D/St 3139>>3487<</P(3139)/S/a/St 1>>3488<</S/D/St 3140>>3501<</P(3152)/S/a/St 1>>3502<</S/D/St 3153>>3516<</P(3166)/S/a/St 1>>3520<</S/D/St 3167>>3521<</P(3167)/S/a/St 1>>3522<</S/D/St 3168>>3524<</P(3169)/S/a/St 1>>3526<</S/D/St 3170>>3631<</P(3274.)/S/D/St 1>>3639<</S/D/St 3275>>3878<</P(3513.)/S/D/St 1>>3883<</S/D/St 3514>>3914<</P(D)/S/D/St 1>>4109<</P(D195.)/S/D/St 1>>4113<</P(D)/S/D/St 196>>

but it failed to create any pagelabels.

[chapmanwilliam]

Strange - the _get_page_labels() function has now failed again. It returns an empty list.

[chapmanwilliam]

I installed wheel 1.18.6.

[JorjMcKie]

we are approaching the goal millimeter-wise:
When writing back the labels, any previously existing entry must be removed ... completely!
This does not happen now: I just insert a new /Nums entry, but the old /Kids entry continues to exist.
So ... another version is required 🙄

[chapmanwilliam]

Dear Jorj,

I have written a getLabel(page) function along with some other minor modifications to the page_labels file here

I have also put in some dummy functions awaiting your perfected _get_page_labels() and _set_page_labels....

Best wishes and Happy New Year.

William

[JorjMcKie]

Thanks William,
I will look into it. Some general comment: please do never redefine a builtin function or type, even if the effect may be local - I saw str = ....

I have submitted wheel generation based on what we discussed / found. Should be available in half an hour or so.

Note: This version includes your other request for identifyable link insertions and the ability to modify toc item colors.
Returning toc item color properties will have to wait a bit. The main reason for this is that I am still looking for an acceptable programming API ... and this is not trivial, because TOC genearation works for alldocument types, not just PDFs. This very feature however will only work for PDF ...

[JorjMcKie]

wheels are ready to go!

[chapmanwilliam]

Thanks Jorj.

Yes, about 'str'. My mistake.

Tried the new wheel.

Unfortunately, I get an error when setting the table of contents using doc.setToC():

File "/Users/William/Dropbox (Personal)/Python/Code/pyCharm/pythonProject2/Display.py", line 973, in readTree
self.doc.setToC(nTOC) # replace table of contents
File "/Users/William/Dropbox (Personal)/Python/Code/pyCharm/pythonProject2/venv/lib/python3.9/site-packages/fitz/utils.py", line 1187, in setToC
if ol["color"] and len(ol["color"]) == 3:
KeyError: 'color'

I tried this on a couple of different files with the same result.

Best wishes,

William

[chapmanwilliam]

Similarly, there is an error raised when using doc.set_toc_item:

File "/Users/William/Dropbox (Personal)/Python/Code/pyCharm/pythonProject2/venv/lib/python3.9/site-packages/fitz/utils.py", line 857, in set_toc_item
raise ValueError("bad bookmark dest")

Not sure why this wheel is failing. Your previous wheel dealt with setting the colours fine.

[JorjMcKie]

just built another set of pre-release wheels here.
They contain some corrections, such that doc.set_toc(...) works again, plus the other discussed changes.
So I am only missing your methods around page labels for an official publication ... 😎:

• page.label() to return the label
• doc.page_numbers(label) to return page numbers (list) for a given label
• doc.set_page_labels(labels_string) to modify a PDF's page label definitons.

As I said, you can define them as normal Python functions that can be included in utils.py and assigned to the Page and Document classes via appropriate instructions in the __init__.py file.

[chapmanwilliam]

I've added those functions in two amended files (utils.py and init.py) here

I've tried the latest wheel. Only problem i've noticed is that the colour of the bookmarks is not being saved.

Best wishes,
William

[chapmanwilliam]

Forgive me: my error. There is no problem saving the colours. It's my problem when reading the colours and translating them for Treeview.

[JorjMcKie]

... the colour of the bookmarks is not being saved

Huh?
Please show me the complete code you used: it works for me.

[chapmanwilliam]

Jorj, you are quite right. My mistake, apologies. My problem was reading the colours and putting them into Treeview. William

…

On 6 Jan 2021, at 17:23, Jorj X. McKie ***@***.***> wrote: ... the colour of the bookmarks is not being saved Huh? Please show me the complete code you used: it works for me. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#782 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AI5FHCG23UQEMF4M3CZZD73SYSMCJANCNFSM4VEIFDNA>.

[JorjMcKie]

Hi William,
here is utils.py based on your contributions, and with some changes on my behalf. Page label code is now at the end of that file.
utils.zip

Main changes I made:

• more prominently acknowledge your contributions,
• removed OrderedDict requirement - a tuple of tuples does the same job in this case, is faster and requires less storage
• removed regular expression (re) requirement, basic string manipulation is sufficient here and (probably) much faster, too.
• replaced the list of letters of the ASCII alphabet with string.ascii_uppercase, which is much faster, requires no additional storage and - adhering to the Python sequence protocol - also supports indexing.
• some minor streamlining, especially around getting the page numbers for a given label.

With this code, the following new methods become available:

• page.get_label() - returns the label (string) or ""
• doc.get_page_numbers(label) - returns a list of page numbers having page.get_label() == label. Added an option to stop searching after the first hit. This offers the chance to prevent scanning through all page numbers: presumably many / most PDFs have no label duplicates. It would be a shame checking all 3071 page numbers in the "Pandas" manual if I want to know which is the one for label "vii".
• doc.set_page_labels(labels) - updates the PDF catalog by adding or replacing the /PageLabels entry. Argument is the list of dictionaries as you defined it, per label rule.

[chapmanwilliam]

Dear Jorj,

I think that is it. Delighted. Well done and thanks for the credit; my contribution was small.

Best wishes,

William

[chapmanwilliam]

Jorj,

I've added this function, below, to utils. It returns a list of labels in the rule_dict format i.e [{'startpage': int, 'prefix': str, 'style': str, 'firstpagenum': int},....] to enable easier handling of [(0, '<</P(TOC_)/S/r/St 1>>'), (112, '<</S/D/St 3>>').

William

def get_labels_rule_dict(doc):
ls=doc.get_page_labels()
#ls is is a list like
#[(0, '<</P(TOC)/S/r/St 1>>'), (112, '<</S/D/St 3>>')
#returns a list like that in rule_dict
def parse_flags(flags):
# flags looks like this /P(A)/S/r/St3
prefix = ''
style = ''
firstpagenum = ''
# look for prefix
x=flags.find('/P')
if x>-1:
st=flags.find('(',x+1)
en=flags.find(')',st+1)
prefix=flags[st+1:en]
# look for style
x=flags.find('/S')
if x>-1:
st=flags.find('/',x+1)
en=flags.find('/',st+1)
style=flags[st+1:en]
# look for startnumber
if x>-1:
st=flags.find(' ',x)
en=len(flags)
firstpagenum=int(flags[st+1:en])

    return prefix, style, firstpagenum

labels=[]
if len(ls)>0:
    for l in ls:
        label = {}
        label['startpage']=l[0]
        x1=l[1].find('<<')
        x2=l[1].find('>>')
        flechedStr=l[1][x1+2:x2] #strip << and >>
        label['prefix'], label['style'], label['firstpagenum'] = parse_flags(flechedStr)
        labels.append(label)
    # check if labels start at page 0
    if len(labels) > 0:
        s = int(labels[0]['startpage'])
        if s > 0:
            # create an initial label
            l = {'startpage': 0, 'prefix': '', 'style': 'D', 'firstpagenum': 1}
            labels.insert(0, l)
else:
    # create an initial label
    l = {'startpage': 0, 'prefix': '', 'style': 'D', 'firstpagenum': 1}
    labels.append(l)
return labels

[chapmanwilliam]

Sorry, that's not very helpful. Here in the file.

[JorjMcKie]

Ah ok, thanks!
Will integrate it in the next version!

[chapmanwilliam]

Sorry, use this here

[JorjMcKie]

The new version 1.18.6 has been uploaded.

[chapmanwilliam]

Jorj,

Two things:

1. page label functions were generating errors when there were no page labels at all in the pdf. I've altered the code to return (pgNo+1) as the label by default.
2. I added function doc.parse_page_string(txt) function. This takes a string of page label references (e.g. (1-3, iii-iv) and returns a list of page numbers.

See here

[JorjMcKie]

page label functions were generating errors when there were no page labels at all in the pdf.

Which ones? I am not getting errors if no labels are defined: page.get_label() = "" in that case.

get_labels_rule_dict

I already have a method doc.get_page_labels() which returns a list of these label dictionaries. It consists of one statement only:
return [rule_dict(item) for item in doc._get_page_labels()]

[chapmanwilliam]

Ah, I get:

'Document' object has no attribute 'get_page_labels'

Perhaps this is the latest wheel?

[JorjMcKie]

yes it is not yet productive
just wanted to mention how I implemented it

[JorjMcKie]

BTW returning a non-empty label for a page, for which there is no rule in the file, contradicts the PDF specification: the response in those cases must be "".

[JorjMcKie]

The generation of new wheels reflecting the discussed (and more) is underway and should be available in half an hour or so.
It also contains a few fixes for corner cases of rules, e.g. without a numbering parameter.

[JorjMcKie]

As per a method like parse_page_string: I do not think it makes much sense.

• it seems a rather mundane requirement - which realistic situation would require an answer like this? And then it would probably be better to document how the respective functions in utils.py which are not "published" as Document methods, can be used to build a specific own solution. One could also explain how to use the single label rule dictionaries can be used to build whatever special solution: those dictionaries are well designed and easy to access and able to answer any inquiry - just by using Python capabilities.
• the parameter specification as a string does not make much sense either: after all how can we know that there never will be rule prefixes containing commas? or "-"? It practically impossible to disambiguate a string parameter in that respect.

As a general comment:
I have put our names - when mentioned as authors - outside the """ ... """ comment blocks for a reason: they would be displayed if someone does a help(fitz.Document.function) inquiry. Embarrassing and immodest.

[chapmanwilliam]

Ok - I’ll handle it my side

…

Sent from my iPhone

On 14 Jan 2021, at 13:19, Jorj X. McKie ***@***.***> wrote:  As per a method like parse_page_string: I do not think it makes much sense. it seems a rather mundane requirement - which realistic situation would require an answer like this? And then it would probably be better to document how the respective functions in utils.py which are not "published" as Document methods, can be used to build a specific own solution. One could also explain how to use the single label rule dictionaries can be used to build whatever special solution: those dictionaries are well designed and easy to access and able to answer any inquiry - just by using Python capabilities. the parameter specification as a string does not make much sense either: after all how can we know that there never will be rule prefixes containing commas? or "-"? It practically impossible to disambiguate a string parameter in that respect. As a general comment: I have put our names - when mentioned as authors - outside the """ ... """ comment blocks for a reason: they would be displayed if someone does a help(fitz.Document.function) inquiry. Embarrassing and immodest. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[chapmanwilliam]

Yes, fair point, although Adobe acrobat seems to accept page labels as user input for page ranges.

…

Sent from my iPhone

On 14 Jan 2021, at 15:49, William Chapman ***@***.***> wrote: Ok - I’ll handle it my side Sent from my iPhone >> On 14 Jan 2021, at 13:19, Jorj X. McKie ***@***.***> wrote: >> >  > As per a method like parse_page_string: I do not think it makes much sense. > > it seems a rather mundane requirement - which realistic situation would require an answer like this? And then it would probably be better to document how the respective functions in utils.py which are not "published" as Document methods, can be used to build a specific own solution. One could also explain how to use the single label rule dictionaries can be used to build whatever special solution: those dictionaries are well designed and easy to access and able to answer any inquiry - just by using Python capabilities. > the parameter specification as a string does not make much sense either: after all how can we know that there never will be rule prefixes containing commas? or "-"? It practically impossible to disambiguate a string parameter in that respect. > As a general comment: > I have put our names - when mentioned as authors - outside the """ ... """ comment blocks for a reason: they would be displayed if someone does a help(fitz.Document.function) inquiry. Embarrassing and immodest. > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub, or unsubscribe.

[JorjMcKie]

Yes, fair point, although Adobe acrobat seems to accept page labels as user input for page ranges.

Adobe Acrobat is a viewer not a programming library. Also I doubt that this logic is fool-proof: how does it disguindish between a page range "5-7" and a prefix looking exactly like this? or like this: "1,4,3-II"?
At best with an error message requesting disambiguation, worst with a crap answer.
I just don't want to be bothered with the load of issues generated by all sorts of corner cases breaking that code.

[chapmanwilliam]

I hear you Jorj!