Release 1.8.8

What's Changed

• Updates to document parsing.
• Automated metadata attribute generation
• Image summary bugfixes
• Upload with APIv2 to enable tabular data understanding

Full Changelog: 1.8.7...1.8.8

Release 1.8.7

What's Changed

• Add docling support
• Dockerfile based on Python 3.11 and smaller image

Full Changelog: 1.8.6...1.8.7

Release 1.8.6

What's Changed

• Updates for folder and FMP/Edgar crawlers
• Added processing of images, and improved table processing with unstructured. Also supports Unstructured chunking
• Bug fixes and updates to README

Release 1.8.5

What's Changed

• Fixes to CSV crawler

Release 1.8.4

What's Changed

• Add twitter crawler
• Fix issues with CSV crawler and other bug fixes

Full Changelog: 1.8.3...1.8.4

Release 1.8.3

What's Changed

• Update website_crawler.py to keep clean urls
• Updates to PMC crawler, CSV crawler and hfdataset crawler
• Store docs locally
• some bug fixes

New Contributors

• @nespera made their first contribution in #114

Full Changelog: 1.8.2...1.8.3

Release 1.8.2

What's Changed

• updated notion crawler
• fixed bug in csv/DB crawlers
• fixed OOM issue with large datasets, and removed local files to avoid out of disk space
• bug fixes to GDrive crawler

Full Changelog: 1.8.1...1.8.2

Release 1.8.1

Hotfix

Full Changelog: 1.8.0...1.8.1

Release 1.8.0

What's Changed

• Update to FMP crawler
• Update SECURITY.md
• Huggingface crawler
• Update to how website content is extracted from HTML
• google drive crawler

New Contributors

• @eskibars made their first contribution in #98
• @AbhilashaLodha made their first contribution in #101

Full Changelog: 1.7.10...1.8.0

Release 1.7.10

What's Changed

• Improve edgar crawler
• Bug fix for html parsing

Full Changelog: 1.7.9...1.7.10