[Feature] Support tokenizer wrapper and support EmbeddingLayerWithFixes

[LeoXing1996]

Thanks for your contribution and we appreciate it a lot. The following instructions would make your pull request more healthy and more easily get feedback. If you do not understand some items, don't worry, just make the pull request and seek help from maintainers.

Motivation

Textual inversion need add external embeddings to the original embedding layer. Therefore we support a tokenizer wrapper and a module named EmbeddingLayerWithFixes to support this feature.

Modification

As title.

BC-breaking (Optional)

Does the modification introduce changes that break the backward-compatibility of the downstream repositories?
If so, please describe how it breaks the compatibility and how the downstream projects should modify their code to keep compatibility with this PR.

Use cases (Optional)

>>> 1. Add token to tokenizer and get the token id.
>>> tokenizer = TokenizerWrapper('openai/clip-vit-base-patch32')
>>> # 'how much' in kiswahili
>>> tokenizer.add_placeholder_tokens('ngapi', num_vec_per_token=4)
>>>
>>> 2. Add external embeddings to the model.
>>> new_embedding = {
>>>     'name': 'ngapi',  # 'how much' in kiswahili
>>>     'embedding': torch.ones(1, 15) * 4,
>>>     'start': tokenizer.get_token_info('kwaheri')['start'],
>>>     'end': tokenizer.get_token_info('kwaheri')['end']
>>> }
>>> embedding_layer = nn.Embedding(10, 15)
>>> embedding_layer_wrapper = EmbeddingLayerWithFixes(embedding_layer)
>>> embedding_layer_wrapper.add_embeddings(new_embedding)
>>>
>>> 3. Forward tokenizer and embedding layer!
>>> input_text = ['hello, ngapi!', 'hello my friend, ngapi?']
>>> input_ids = tokenizer(
>>>     input_text, padding='max_length', truncation=True,
>>>     return_tensors='pt')['input_ids']
>>> out_feat = embedding_layer_wrapper(input_ids)
>>>
>>> 4. Let's validate the result!
>>> assert (out_feat[0, 3: 7] == 2.3).all()
>>> assert (out_feat[2, 5: 9] == 2.3).all()

Checklist

Before PR:

• I have read and followed the workflow indicated in the CONTRIBUTING.md to create this PR.
• Pre-commit or linting tools indicated in CONTRIBUTING.md are used to fix the potential lint issues.
• Bug fixes are covered by unit tests, the case that causes the bug should be added in the unit tests.
• New functionalities are covered by complete unit tests. If not, please add more unit test to ensure the correctness.
• The documentation has been modified accordingly, including docstring or example tutorials.

After PR:

• If the modification has potential influence on downstream or other related projects, this PR should be tested with some of those projects.
• CLA has been signed and all committers have signed the CLA in this PR.

[codecov]

Codecov Report

Patch coverage: 73.91% and project coverage change: -0.07 ⚠️

Comparison is base (e248c7c) 83.13% compared to head (78ca163) 83.07%.

❗ Current head 78ca163 differs from pull request most recent head 1191927. Consider uploading reports for the commit 1191927 to get more accurate results

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1846      +/-   ##
==========================================
- Coverage   83.13%   83.07%   -0.07%     
==========================================
  Files         410      411       +1     
  Lines       27339    27523     +184     
  Branches     4257     4296      +39     
==========================================
+ Hits        22728    22864     +136     
- Misses       3700     3737      +37     
- Partials      911      922      +11     

Flag	Coverage Δ
unittests	83.07% <73.91%> (-0.07%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
mmagic/models/archs/tokenizer.py	68.36% <68.36%> (ø)
...gic/models/editors/disco_diffusion/clip_wrapper.py	84.29% <80.00%> (-10.15%)	⬇️
mmagic/models/archs/__init__.py	92.85% <100.00%> (+0.17%)	⬆️

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.