How to extract alll paragraphs instead of lines?

[nissansz]

How to extract alll paragraphs instead of lines?

[JorjMcKie]

The only way without a lot of your own code:
Accept the blocks of the TextPage i.e. the text contained in each block as delivered by page.get_text("blocks").

[nissansz]

If it is a table, each cell will be a separate block?

[JorjMcKie]

Sometimes yes, sometimes no. Neither PDF nor MuPDF know what a "table" is.

Depends on how messy the PDF was made. MuPDF's algorithm uses a set of heuristics to determine what to regard as a block or a line. You can look up details in their C code - I don't want to do it here.

As I wrote: if you don't like the result, you will have to write your code and see if that fits your situation better.

[nissansz]

page.get_text("blocks")

it seems, most blocks are lines, others are images etc.

For writing text to pdf,
how to write text above image, or text under image?

There are page.insert_text() and page.insert_textbox(). Anything wrong with them?
Or use TextWriter which makes it easier to support different fonts.

[JorjMcKie]

how to write text above image, or text under image?

Use the overlay parameter. This is present in all relevant functions, including draw functions.

it seems, most blocks are lines, others are images etc.

If you do not care about the flagsparameter, you will be serviced with the default per get_text variant. You can force to omit images by using flags=fitz.TEXTFLAGS_TEXT.

[nissansz]

There is error, whey I tried to use textwriter.

RuntimeError: cannot create font

    text = "PyMuPDF: the Python bindings for MuPDF我们"
    font_path = r'L:\paddle\字体制作\大合集\GTAN00B.TTF'
    # font = fitz.Font("figo")  # choose a font with small caps

    font = fitz.Font(font_path)  # choose a font with small caps
    tw = fitz.TextWriter(page.rect)
    tw.append((50, 100), text, font=font, small_caps=True)
    tw.write_text(page)
doc.ez_save(pdf_file+'writetest.pdf')
doc.close()

[JorjMcKie]

font = fitz.Font(fontfile=font_path) # choose a font with small caps

[nissansz]

How to add overlay parameters? I tried to write, and the result is overlay.

def append(self, pos, text, font=None, fontsize=11, language=None, right_to_left=0, small_caps=0):

[JorjMcKie]

write_text(page, opacity=None, color=None, morph=None, overlay=True, oc=0, render_mode=0)

[nissansz]

It works to be below image when overlay = False.

How to get image for each block including image?
for block in page.get_text("dict", flags=fitz.TEXTFLAGS_TEXT)["blocks"]:
for line in block["lines"]:
j=j+1
bbox = line["bbox"] # the line bbox
text = ''.join([span["text"] for span in line["spans"]]) # text in line
# print(bbox)
# print(text)
if len(text.strip())>0:

                pix = page.get_pixmap(clip=bbox, dpi=300)  # pixmap of line bbox

[JorjMcKie]

It works to be below image when overlay = False.

No: it will always be below everything else - not just images! If subsequent stuff however is not covering that area completely (because it is transparent, or other text) then ugly results will occur.

I don't understand the rest of your post.

[nissansz]

How to get background color of the original line background, and then use the background to fill the text box background?

[JorjMcKie]

You can make a pixmap of the bbox and compute the predominent color used in that rectangle: pix.color_topusage().

[nissansz]

And how to set the angle of the write_text textbox?

[JorjMcKie]

Use the morph parameter. Needs a fixpoint and a matrix. The fixpoint probably is the bbox bottom-left and the matrix could be fitz.Matrix(angle), angle in degrees.

[nissansz]

For example, at the original text line position,
I want to first insert a small image without text to cover the original position, then write_text on the new small image,
how to write the code?

[JorjMcKie]

First determine the background color of the bbox, then page.draw_rect(bbox, color=None, fill=background_color), then write new text in the bbox.
Be aware though that the original text is still there and would appear in a text extraction.

[nissansz]

page.insert_image(
rect, # where to place the image (rect-like)
filename=None, # image in a file
stream=None, # image in memory (bytes)
pixmap=None, # image from pixmap
mask=None, # specify alpha channel separately
rotate=0, # rotate (int, multiple of 90)
xref=0, # re-use existing image
oc=0, # control visibility via OCG / OCMD
keep_proportion=True, # keep aspect ratio
overlay=True, # put in foreground
)

[nissansz]

Overlay position seems not maching perfect, any better matching method? By character instead of line?

[JorjMcKie]

No, you are causing this by not providing the correct insertion point. Text extractions with the "dict" option return "origin" as part of the "span" dictionary. The y-component of that "origin" should be used.

[nissansz]

Don't know how to do it.

[JorjMcKie]

"blocks" and "words" extractions only return the text and their surrounding bbox. Almost never the original insertion point (which you use yourself when doing e.g. insert_text) is equal to the bboxes bottom-left. The originally used font size can neither be derived from the bbox (directly). Depending on the font used, this y is a bit higher up than bbox.bl.
If you need to be so precise, you have to go the longer way and extract with the "dict" option. The first span of each line could be taken to determine that y (from span["origin"]) and the used font size (from span["size"].

for block in get_text("dict",...)["blocks"]:
    for line in b["lines"]:
        bbox = fitz.Rect(line["bbox"])
        span0 = line["spans"][0]
        origin_y = span["origin"][1]
        fontsize = span["size]
        # insert text using (bbox.x0, y) as insertion point and the font size

[nissansz]

Thank you. Where to see all properties for "spans"?

[JorjMcKie]

https://pymupdf.readthedocs.io/en/latest/textpage.html#structure-of-dictionary-outputs

[nissansz]

Information seems complete.

for block in page.get_text("dict")["blocks"]:
for line in block["lines"]:
bbox = fitz.Rect(line["bbox"])
span0 = line["spans"][0]
origin_y = span0["origin"][1]
fontsize = span0["size"]
print('-------------------')

[nissansz]

I cannot see angle information. Any place to get angle from dict?

[JorjMcKie]

That's part of the line dictionary, because this is what has an angle with the x-axis. Look at the line's "dir" key ... just a few sections upwards in the documentation.

[nissansz]

for block in page.get_text("dict")["words"]:

KeyError: 'words'

When try to extract words, there is above error.

[nissansz]

for block in page.get_text("dict")["words"]:

KeyError: 'words'

When try to extract words, there is above error.

[JorjMcKie]

"words" is alternative of "blocks", "dict" etc., not part of any key with the "dict" output.

[nissansz]

    for word in page.get_text("words"):
        print(word)

What is the value for [5] to [7]?