Enhancing get_all_bundles_list under monai.bundle to support model zoo NGC hosting

[KumoLiu]

Follow-up issue #6761.

In this PR, I have updated the download API under monai.bundle to support model zoo NGC hosting.

It appears that there are still several other methods that need to be updated, such as get_all_bundles_list, get_bundle_versions, and get_bundle_info which may be used in MONAILabel.
https://github.com/Project-MONAI/MONAILabel/blob/1d034726c655102cf612053842b1c45727966612/monailabel/utils/others/generic.py#L261

We can't use the search API in NGC API doc to find all models. May need the NGC team to add support for this one.

cc @wyli @ericspod @Nic-Ma @yiheng-wang-nv

[wyli]

(This is useful for here as well: https://github.com/Project-MONAI/SlicerMONAIViz/blob/22cc28103b0b2167d9f5e973c54f2a376703cedf/MONAIViz/MONAIVizLib/utils.py#L106)

[KumoLiu]

Looks like we can use the raw file link of "model_info.json" as a workaround. But seems not a recommend way.

request_url = f"https://raw.githubusercontent.com/Project-MONAI/model-zoo/dev/models/model_info.json"
resp = get(request_url)

[yiheng-wang-nv]

Thanks @KumoLiu . Did you confirm that request this file have no rate limit restriction?

[KumoLiu]

GitHub's rate limiting typically applies to API requests, I didn't meet rate limit restriction in a 6000loop try with this raw file link but it's not the recommended way to interact with GitHub programmatically.

[yiheng-wang-nv]

GitHub's rate limiting typically applies to API requests, I didn't meet rate limit restriction in a 6000loop try with this raw file link but it's not the recommended way to interact with GitHub programmatically.

I see. If not have other methods so far, I think we can add error handling to process exceptions such as if not get expected content, and then this way is fine. I'm not sure if there are any other drawbacks, and we may need @wyli @ericspod 's suggestions here.

[ericspod]

GitHub's rate limiting typically applies to API requests, I didn't meet rate limit restriction in a 6000loop try with this raw file link but it's not the recommended way to interact with GitHub programmatically.

I see. If not have other methods so far, I think we can add error handling to process exceptions such as if not get expected content, and then this way is fine. I'm not sure if there are any other drawbacks, and we may need @wyli @ericspod 's suggestions here.

Let's see if this continues to work first because it is an easy solution, if not we just have to be more careful with the API calls. The rate limits are to stop people from hammering the servers intentionally or otherwise, specifically to avoid using it as some sort of real time system. Requested information shouldn't change often so caching would be expected on the client side, eg. using something like https://pypi.org/project/requests-cache/ for our case. MONAI Label could use this to avoid checking the model zoo constantly.