require -l in run.cpp

[metascroy]

We pass the llama version in run.cpp with -l arg. Today it has a default value of 2 (for llama2), but I don't think this should have a default value.

[mikekgfb]

Does it actually matter what it is? Or is it just the tokenizer anyway? Maybe don't call it model if it's tokenizer?

[metascroy]

@mikekgfb it's the model in the sense that it's used to select the tokenizer class, prompt structure, and default vocab size, all of which are different for llama2 vs. llama3.

[metascroy]

@larryliu0820 and I were talking that maybe the tokenizer should get the vocab size from the tokenizer.model file, but that's not how they work today.

[metascroy]

After #626 lands, the tokenizer will get the vocab size from the *.model file.

So we should be able to automatically deduce 2 or 3 by a tokenizer load failure, and clean up some of the edge cases in the runner.

[mikekgfb]

Agreed. Let's make sure changing this to remove the default doesn't break documented workflows.
(And in long term, will autodetect).

[mikekgfb]

Thank you!

[mikekgfb]

@metascroy do we still need to land this, or did you already land a version of this?

[mikekgfb]

Thank you!

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/torchchat/594

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 5 New Failures

As of commit 84d8e5c with merge base a80ae4d ():

NEW FAILURES - The following jobs have failed:

• pull / runner-aoti (16-core-ubuntu) (gh)
• pull / runner-aoti (macos-14-xlarge) (gh)
  Process completed with exit code 1.
• pull / runner-et (16-core-ubuntu) (gh)
• pull / runner-et (macos-14-xlarge) (gh)
  Process completed with exit code 1.
• Run the aoti runner with CUDA using stories / test-runner-aot-cuda / linux-job (gh)
  RuntimeError: Command docker exec -t 38bbe48e00a868adf8c190da0b91e18eb38357346f377529567f15c4dc7d6b15 /exec failed with exit code 1

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[Jack-Khuu]

Closing Stale PR