Skip to content

0.6.2

Latest
Compare
Choose a tag to compare
@mhconradt mhconradt released this 14 Sep 20:49

This release updates all tree-sitter grammars to 0.23, their most recent version at time of release.
This allows getting rid of some code that did pointer arithmetic in Python using the ctypes library, which threatened various memory safety issues.
Not quite included in the code, but it also adds tests for all supported languages.
These are fairly basic and will be expanded upon in the future, but should prevent anything super silly from happening.

Also, this release reduced the concurrency of making requests to the OpenAI API to avoid exceeding rate limits.

Essentially, making requests from multiple threads is great for performance on small projects, because you can make requests at higher than the sustainable throughput (1M, 5M, 10M tokens per minute) for under 1 minute.
i.e. if your rate limit is 1M tokens per minute, you could potentially make 500k tokens of requests in 5 seconds, which is 6x your sustained rate limit.
This was bad for larger projects, so I'm temporarily reducing request concurrency to 1, but the most effective strategy (already in place) for boosting performance is to batch requests, so we're able to sustain approximately 100% of the allowed throughput on the OpenAI API (5M tokens, at least for tiers 3 and 4).