Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

A question about Wizardcoder paper #225

Open
Smith-xuan opened this issue Dec 15, 2023 · 0 comments
Open

A question about Wizardcoder paper #225

Smith-xuan opened this issue Dec 15, 2023 · 0 comments

Comments

@Smith-xuan
Copy link

Dear author, hello! I have a question about the WizardCoder paper.
According to my understanding from the paper, after each round of evolution, you merge the data from that round with the data from previous rounds for fine-tuning, resulting in a performance iteration growth effect similar to Figure 3 in the paper.
{D81DA96B-46C9-4ea2-AC5B-D70D30D87850}

However, since the evolution of the dataset is independent of the fine-tuning process, why not merge the data from each round together and perform fine-tuning only once? For example, if the original data used for fine-tuning is (0), (0,1), (0,1,2), (0,1,2,3), why not directly use only the data from (0,1,2,3) for fine-tuning? Does this iterative fine-tuning approach lead to performance improvements in the model? I would like to ask if your team has conducted experiments in this regard.
If you are willing to provide an answer, I would be extremely grateful!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant