how to convert huggingface model to megatron-deepspeed?

[yayaQAQ]

as title said.

[mayank31398]

This is not possible.
To download DS checkpoints refer to this issue:
#319

[yayaQAQ]

This is not possible. To download DS checkpoints refer to this issue: #319

why? So I have to training from the ground up?
It's hard.

[mayank31398]

I don't understand the issue. Do you just need to run inference?
If that is the case, DS-inference is compatible with all Huggingface models.

[AnShengqiang]

I don't understand the issue. Do you just need to run inference? If that is the case, DS-inference is compatible with all Huggingface models.

Hello, I have the same problem.

I want to load the model on Huggingface as a pre-training model weight and continue the training using the Megatron Deepspeed framework.

But I found that I didn't know how to convert the weight of Huggingface into the weight of Megatron Deepspeed.

I look forward to your help. Thank you.

[AnShengqiang]

By the way:

model structure: gpt
model link: https://huggingface.co/TsinghuaAI/CPM-Generate

I want to train the model with 4 pipeline parallel and deepspeed.

[mayank31398]

@AnShengqiang Its non-trivial to convert models for training.
People are actively exploring this as far as I know.
This repository saves something called a universal checkpoint which can be converted to other checkpoints.
However, I am quite new here so, I don't really know how that works.

[AnShengqiang]

@AnShengqiang Its non-trivial to convert models for training. People are actively exploring this as far as I know. This repository saves something called a universal checkpoint which can be converted to other checkpoints. However, I am quite new here so, I don't really know how that works.

Thank you for your reply, I will go to find the answer, if there is good news, I will put it here.

[stgzr]

I don't understand the issue. Do you just need to run inference? If that is the case, DS-inference is compatible with all Huggingface models.

Hello, I have the same problem.

I want to load the model on Huggingface as a pre-training model weight and continue the training using the Megatron Deepspeed framework.

But I found that I didn't know how to convert the weight of Huggingface into the weight of Megatron Deepspeed.

I look forward to your help. Thank you.

Same problem. Any tools can do this?