Add option to export_to_markdown to mark page breaks

[cau-git]

As suggested in this discussion, we should add a placeholder feature for page breaks, the same way we support placeholders for pictures.

• The placeholder text for page breaks should by default contain the page number.
• It should be disabled as a default.

[chakravarthik27]

Hi @cau-git

I am interested in working on this issue.

[cau-git]

@chakravarthik27 Thanks, we would welcome that you make a contribution for this issue!

As a starting point, it would require extension of this method.

[sunwoongc]

@chakravarthik27
Hi, I'm also very interested in resolving this issue. I've found a solution that works in my case, though I'm not sure if it’s a general fix. Would you mind if I also worked on this?

[PeterStaar-IBM]

@chakravarthik27 Absolutely not, go ahead! Just make a PR in https://github.com/DS4SD/docling-core!

[chakravarthik27]

I'm getting confusion, Still I didn't started, so please continue @sunwoongc

[PeterStaar-IBM]

@sunwoongc @chakravarthik27 Please coordinate with each other for the markdown pagebreaks and let us know when you expect it to be done.

[sunwoongc]

@chakravarthik27 @PeterStaar-IBM

Thank you for your input!

I noticed a simple but important detail: most items inheriting from DocItem have an attribute called prov, which includes a page_no field for tracking provenance. For reference, here's the ProvenanceItem.

However, the GroupItem class lacks this attribute, as it's designated as a container type. See GroupItem.

To handle this in the export_to_markdown function, I've added the following code:

prev_page_no = -1
page_change_flag = False
for ix, (item, level) in enumerate(doc.iterate_items(doc.body, with_groups=True)):  
    if not isinstance(item, GroupItem):
        cur_page_no = item.prov[0].page_no
        if prev_page_no != cur_page_no:
            page_change_flag = True
        else:
            page_change_flag = False

        # Append text if page has changed
        if page_change_flag:
            mdtexts.append(f"Page {cur_page_no}")

        # Update previous page number after handling change
        prev_page_no = cur_page_no

I'm concerned this solution may encounter edge cases. If you have any suggestions or foresee potential issues, I'd appreciate your feedback!