Support non-breaking content in Python #251
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This patch supports non-breaking content in Python. In Java and Python implementations, the "Skip" operation includes the skipped content to the BudouX parser, so no changes to the text for the parser is needed. This patch changes following items: 1. Changed `to_skip` to a stack of elements, rather than always reset to `False` at the end of an element. 2. When there's a phrase boundary right before the "skip" element, insert a break before the "skip" element. Note `<NOBR>` is added to `skip_nodes.json` at: google#248.
tushuhei
approved these changes
Aug 17, 2023
kojiishi
added a commit
to kojiishi/budoux
that referenced
this pull request
Nov 10, 2023
google#251 assumed that all tags are closed properly. This assumption doesn't stand for cases like: 1. Self-closing tags such as `<img>` don't have corresponding close tags. 2. Unpaired close tags are still valid HTML. This patch supports these cases by assuming all open tags that doesn't nest correctly or that doesn't close are automatically closed. This isn't the full HTML "adoption agency algorithm", but it should be good enough for the needs of BudouX. Fixes google#355
kojiishi
added a commit
that referenced
this pull request
Nov 11, 2023
* Fix unpaired close tags and self-closing tags #251 assumed that all tags are closed properly. This assumption doesn't stand for cases like: 1. Self-closing tags such as `<img>` don't have corresponding close tags. 2. Unpaired close tags are still valid HTML. This patch supports these cases by assuming all open tags that doesn't nest correctly or that doesn't close are automatically closed. This isn't the full HTML "adoption agency algorithm", but it should be good enough for the needs of BudouX. Fixes #355
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This patch supports non-breaking content in Python.
In Java and Python implementations, the "Skip" operation includes the skipped content to the BudouX parser, so no changes to the text for the parser is needed.
This patch changes following items:
to_skipto a stack of elements, rather than always reset toFalseat the end of an element.Note
<NOBR>is added toskip_nodes.jsonat: #248.