Reference markdown splitter in features table

benbrandt · Mar 25, 2024 · 9ba3da5 · 9ba3da5
1 parent abf7777
commit 9ba3da5
Showing 1 changed file with 4 additions and 4 deletions.
diff --git a/README.md b/README.md
@@ -188,10 +188,10 @@ There are lots of methods of determining sentence breaks, all to varying degrees
 
 ### Tokenizer Support
 
-| Dependency Feature | Version Supported | Description                                                                                                                                                               |
-| ------------------ | ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| `tiktoken-rs`      | `0.5.8`           | Enables the `TextSplitter::new` to take a `tiktoken_rs::CoreBPE` as an argument. This is useful for splitting text for OpenAI models.                                     |
-| `tokenizers`       | `0.15.2`          | Enables the `TextSplitter::new` to take a `tokenizers::Tokenizer` as an argument. This is useful for splitting text models that have a Hugging Face-compatible tokenizer. |
+| Dependency Feature | Version Supported | Description    |
+| ------------------ | ----------------- | -------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `tiktoken-rs`      | `0.5.8`           | Enables `[Text | Markdown]Splitter::new`to take`tiktoken_rs::CoreBPE` as an argument. This is useful for splitting text for OpenAI models.                                     |
+| `tokenizers`       | `0.15.2`          | Enables `[Text | Markdown]Splitter::new`to take`tokenizers::Tokenizer` as an argument. This is useful for splitting text models that have a Hugging Face-compatible tokenizer. |
 
 ## Inspiration