Fix issue with markdown translation where code blocks were split across chunks #21
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Purpose
Problem
During markdown translation, code blocks and inline code were sometimes split across multiple chunks, causing errors or incomplete translations. This often led to broken translations, especially for longer documents with numerous code snippets. Additionally, in some cases, inline code and code blocks contained variables or syntax that could cause errors in the Semantic Kernel translation process.
Solution
```
) and inline code (``
) with placeholders like@@CODE_BLOCK_x@@
and@@INLINE_CODE_x@@
before sending the document for translation.Testing
The solution was tested on documents with extensive code snippets to ensure code blocks and inline code remain intact during translation, fixing the previous issues where chunks were improperly split. It was also confirmed that this method prevents Semantic Kernel errors caused by specific variables or syntax within the code blocks.
Additionally, in the Phi-3 CookBook, the previously problematic translation of the 06.E2ESamples/E2E_Phi-3-FineTuning_PromptFlow_Integration.md file, which is around 1200 lines long, was successfully completed without any chunk loss or formatting errors. (Solved #18)
Does this introduce a breaking change?
When developers merge from main and run the server, azd up, or azd deploy, will this produce an error?
If you're not sure, try it out on an old environment.
Type of change