Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add aside Tag to Pruned Tags List in HTML Crawler Configuration #2803

Closed
marevol opened this issue Feb 5, 2024 · 0 comments
Closed

Add aside Tag to Pruned Tags List in HTML Crawler Configuration #2803

marevol opened this issue Feb 5, 2024 · 0 comments
Assignees
Milestone

Comments

@marevol
Copy link
Contributor

marevol commented Feb 5, 2024

We have updated the HTML crawler configuration to include the aside tag in the list of pruned tags. This change aims to improve the relevance of crawled content by excluding sections typically not central to the main content, such as sidebars and supplementary information.

This modification ensures that the HTML crawler will skip content within aside tags during the crawling process, focusing more on the primary content of the pages.

@marevol marevol added this to the 14.12.0 milestone Feb 5, 2024
@marevol marevol self-assigned this Feb 5, 2024
@marevol marevol closed this as completed in f6b2ef3 Feb 5, 2024
marevol added a commit that referenced this issue Feb 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant