v0.12.0

v0.12.0 (2024-08-21)

Feature

• feat: add jamba 1.5 tokenizers (mini and large) (#96)

• feat: add jamba 1.5 tokenizers (mini and large)

• test: fix value error on conftest.py

• refactor: rename file name

• chore: fix path

• fix: enter (e052b22)

v0.11.3

v0.11.3 (2024-08-05)

Chore

• chore(deps-dev): bump authlib from 1.2.1 to 1.3.1 (#91)

Bumps authlib from 1.2.1 to 1.3.1.

• Release notes
• Changelog
• Commits

---

updated-dependencies:

• dependency-name: authlib
  dependency-type: indirect
  ...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> (fe24518)

• chore(deps-dev): bump black from 23.3.0 to 24.3.0 (#90)

Bumps black from 23.3.0 to 24.3.0.

• Release notes
• Changelog
• Commits

---

updated-dependencies:

• dependency-name: black
  dependency-type: direct:development
  ...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> (5257f3d)

• chore(deps-dev): bump python-semantic-release from 8.3.0 to 8.7.0 (#80)

Bumps python-semantic-release from 8.3.0 to 8.7.0.

• Release notes
• Changelog
• Commits

---

updated-dependencies:

• dependency-name: python-semantic-release
  dependency-type: direct:development
  update-type: version-update:semver-minor
  ...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Asaf Gardin <[email protected]>
Co-authored-by: Asaf Joseph Gardin <[email protected]> (376894f)

Ci

• ci: allow running tests and semantic checks on external prs (#93) (c08604d)

Fix

• fix(deps): bump requests from 2.31.0 to 2.32.2 (#92)

Bumps requests from 2.31.0 to 2.32.2.

• Release notes
• Changelog
• Commits

---

updated-dependencies:

• dependency-name: requests
  dependency-type: indirect
  ...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Asaf Joseph Gardin <[email protected]> (3216898)

v0.11.2

v0.11.2 (2024-06-20)

Fix

• fix: add missing space tokens init on jursssic (#89)

• fix: missing space tokens init on jursssic, fix import in example

• test: add test for space_tokens init (4ec43a0)

v0.11.1

v0.11.1 (2024-06-20)

Fix

• fix: Added backwards compatibility for jamba tokenizer (#88)

• fix: Added backwards compatibility for jamba tokenizer

• fix: Added to async

---

Co-authored-by: asafg <[email protected]> (9ad3475)

v0.11.0

v0.11.0 (2024-06-18)

Feature

• feat: Async tokenizer (#86)

• feat: support async, wip

• feat: fix and add tests, examples, update readme

• fix: poetry lock

• fix: anyio -> aiofiles

• fix: try 3.8

• fix: remove 3.7 from tests

• fix: poetry lock

• fix: add 3.7 back

• fix: poetry lock

• fix: poetry.lock

• ci: pipenv

• fix: pipenv

• fix: pipenv

• fix: pyproject

• fix: lock

• fix: version

• fix: Removed aiofiles

• ci: update python version,

• fix: switch from aiofiles to anyio, remove redundant comments

• chore: poetry lock

• fix: disable initializing async classes directly, cr comments

• test: fix import

• ci: add asyncio-mode to test workflow

• fix: to_thread -> run_in_executor

• ci: add asyncio

• fix: cr comments

• fix: cr comments

---

Co-authored-by: asafg <[email protected]> (3006cda)

v0.10.0

v0.10.0 (2024-06-16)

Feature

• feat: remove python 3.7 support (#87) (58482ab)

v0.9.1

v0.9.1 (2024-05-14)

Fix

• fix: depend on less restrictive version of tokenizers (#85) (eab6a14)

v0.9.0

v0.9.0 (2024-03-28)

Feature

• feat: Jamba instruct tokenizer (#84) (88ff9af)

v0.8.2

v0.8.2 (2024-03-11)

Ci

• ci: add Python 3.12 to test matrix (#82)

• ci: add Python 3.12 to test matrix

• chore: use sentencepiece 0.2.0 or higher

• fix: update poetry.lock (8084117)

Fix

• fix: docs (#83) (c26949a)

Unknown

• Update issue templates (86ea6e7)

v0.8.1

v0.8.1 (2024-01-07)

Fix

• fix: re-ordered parameters in ctor to avoid a breaking change (#79) (6c1b608)