Skip to content

Commit

Permalink
chore(release): v0.6.0 [skip ci]
Browse files Browse the repository at this point in the history
  • Loading branch information
actions-user committed Dec 28, 2023
1 parent a5a7bb4 commit 7b8348d
Show file tree
Hide file tree
Showing 3 changed files with 38 additions and 2 deletions.
35 changes: 35 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,43 @@



## v0.6.0 (2023-12-28)

### Chore

* chore: add test case for encode with is_start=False (#74)

* chore: add test case for encode with is_start=False

* fix: split is_start=False to a different testcase ([`77c0a39`](https://github.com/AI21Labs/ai21-tokenizer/commit/77c0a39d1bcde81cc0166a512eb454dad6d3c569))

### Feature

* feat: Add decode with offsets (#73)

* feat: Add decode_with_offsets() to JurassicTokenizer

* refactor: remove kwargs from decode_with_offsets since it's not used

* chore: Add unittest for decode and for offsets

* fix: test only decode_with_offsets

* fix: dummy for returned offsets in decode_with_offsets ([`a5a7bb4`](https://github.com/AI21Labs/ai21-tokenizer/commit/a5a7bb4b27fa4f74a0b1a1d6874599556a35c1c5))

* feat: Add the is_start parameter to JurassicTokenizer.encode() (#72)

* feat: Add the is_start parameter to JurassicTokenizer.encode()

* refactor: take 'is_start' from kwargs ([`296bda5`](https://github.com/AI21Labs/ai21-tokenizer/commit/296bda5578edd57ff58d6763b3ccd5b9ba709795))


## v0.5.0 (2023-12-28)

### Chore

* chore(release): v0.5.0 [skip ci] ([`96f384f`](https://github.com/AI21Labs/ai21-tokenizer/commit/96f384f2873ec363016c8def483f59ac535acba9))

### Feature

* feat: Add more special tokens (#71)
Expand Down
2 changes: 1 addition & 1 deletion ai21_tokenizer/version.py
Original file line number Diff line number Diff line change
@@ -1 +1 @@
VERSION = "0.5.0"
VERSION = "0.6.0"
3 changes: 2 additions & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ schema_pattern = "(build|ci|docs|feat|fix|perf|refactor|style|test|chore|revert|

[tool.poetry]
name = "ai21-tokenizer"
version = "0.5.0"
version = "0.6.0"
description = ""
authors = ["AI21 Labs"]
readme = "README.md"
Expand Down Expand Up @@ -110,6 +110,7 @@ newline_sequence = "\n"




[tool.poetry.group.test.dependencies]
coverage = "^7.1.0"
pytest = "7.2.1"
Expand Down

0 comments on commit 7b8348d

Please sign in to comment.