Skip to content

Optimize byte pair merge for really big tokens (40x faster for a 2500 token word) #219

Optimize byte pair merge for really big tokens (40x faster for a 2500 token word)

Optimize byte pair merge for really big tokens (40x faster for a 2500 token word) #219