Modify the args_default usage

[yuanwu2017]

While args_defaults is compatible with the original usage, the value of argments can also be set through function call usage.

[conglongli]

@yuanwu2017 your description is unclear. If your PR fixes an existing issue, please describe what issue does it fix with an example. If your PR is a new feature, please give an example of when it is helpful.

[yuanwu2017]

We are integrating the Megatron-deepspeed into Ray. We need initialize_megatron to initialize megatron by passing in function arguments instead of arguments from the startup command.
For example:

args = {
  "num_layers": 32,
            "hidden_size": 4096,
            "ffn_hidden_size": 11008,
            "num_attention_heads": 32,
            "max_position_embeddings":4096,
            "num_key_value_heads": 32
              .......
}
initialize_megatron(ignore_unknown_args=True, args_defaults=args)

[JiafeiSun]

with the patch, I run pretrain_llama2_distributed.sh which set --tokenizer-type GPTSentencePieceTokenizer, but in pretrain_gpt.py args_defaults={'tokenizer_type': 'GPT2BPETokenizer'}, as result, tokenizer-type=GPT2BPETokenizer, but for llama2, tokenizer-type should be GPTSentencePieceTokenizer, how to solve it? ths

[yuanwu2017]

Sorry for the introduction of bug and your inconvenience. I reverted the patch and made a new one.

[JiafeiSun]

Sorry for the introduction of bug and your inconvenience. I reverted the patch and made a new one.

thx