Preserve spaces when replacing token

[culturedniichan]

When choosing another token from the probabilities, the token's text (content) that may have had a space before it to indicate a new word was not preserved. So for example, if we had

The maid cafe was located in Tokyo

And I wanted to replace "located" (assuming it was its own token) with "situated", what happens is as follows:

• The text for located is actually " located"
• However, when replaced, we put "situated", no starting space. This means it becomes

The maid cafe wassituated in Tokyo

What the updated code does is:

1. Check if the token that will be replaced at position i starts with a space, If not, we do nothing, as we're replacing a token that was part of a word (e.g. joi-ned if we replace the token 'ned')
2. Adds to the new token a starting space, unless the new token is an English punctuation mark. For example, if we replace 'went' in 'the man went', if the token is the apostrophe ' (e.g. the man's luggage) then there will be no space prepended.

[lmg-anon]

The token's text should have that space, if it doesn't have then it's a bug in the backend. I assume you are using oobabooga?

[culturedniichan]

The token's text should have that space, if it doesn't have then it's a bug in the backend. I assume you are using oobabooga?

Yes, I'm using oobabooga

[lmg-anon]

I made a PR in oobabooga's repository to fix this issue there, hopefully it will get merged.
oobabooga/text-generation-webui#5339

[culturedniichan]

Great. I didn't know it was an ooba issue. I don't have an OpenAI key, and I don't usually use Kobold, so all my tests and use are based on oobabooga. I'll just keep this branch on my computer until (hopefully) it gets merged into ooba

[culturedniichan]

Latest Ooba commit seems to have solved the issue. Thanks