Fix/pdf miner source property

[benjats07]

This PR adds three possible values for source field:

• pdfminer as source for elements directly obtained from PDFs.
• OCR-tesseract and OCR-paddle for elements obtained with the respective OCR engines.

All those new values are stored in a new class Source in unstructured_inference>constants.py

This would help users to filter certain elements depending on how were obtained.

For testing purposes, you can execute this script:

from unstructured_inference.constants import OCRMode
from unstructured_inference.inference import layout
from unstructured_inference.models.base import get_model

file = "sample-docs/loremipsum-flat.pdf"
model = get_model("yolox_tiny")
doc = layout.DocumentLayout.from_file(
    file,
    model,
    ocr_mode=OCRMode.FULL_PAGE.value,
    supplement_with_ocr_elements=True,
    ocr_strategy="force",
)
assert "OCR-tesseract" in {e.source for e in doc.pages[0].elements}
print("OCR-tesseract in sources")

elements = layout.load_pdf(file)
assert elements[0][0][0].source == 'pdfminer'
print("pdfminer in sources")

The output should be:

OCR-tesseract in sources
pdfminer in sources

[qued]

LGTM. I wonder if as a follow on we could make the source property mandatory... I don't know if we ever want to allow an element without a source?

[badGarnet]

can we have those string constants defined as constants? this is so that user can import the names instead of need to read the code to find out how they are spelled and what options are there.

[benjats07]

Sure, but...will be a special case: the merged type in

https://github.com/Unstructured-IO/unstructured-inference/blob/2de61a4d5c60dfda4d789e7d7fdcdf49f1f04960/unstructured_inference/inference/layoutelement.py#L314C62-L314C62

If we want to transform this into constants, then the source for those elements needs to be just "merged" 🤔 (losing the source of origin elements), any ideas about how to solve this? (apart from enumerating all possible combinations)

Edit: I added other attribute to merged elements, but not sure if is the best approach (see

unstructured-inference/unstructured_inference/inference/layoutelement.py

Line 320 in 46f2c8d

setattr(element, "merged_sources", sources)

)

[benjats07]

LGTM. I wonder if as a follow on we could make the source property mandatory... I don't know if we ever want to allow an element without a source?

🤔 Yeah, this must be mandatory, no makes sense elements coming from an unknown source.