Is there a way to retrieve all dataset refs in a project?

[akiyamasho]

In the Datasets documentation, there is a way to publish and get by ref, but there seems to be no way to get a list of all existing dataset refs in a project.

(NOTE: I also checked the codebase and it doesn't seem to include any way to retrieve a list of dataset refs from the project
https://github.com/wandb/weave/blob/master/weave/flow/dataset.py ・ https://github.com/wandb/weave/blob/master/weave/trace/api.py)

[TeoZosa]

For context, @akiyamasho and I are deploying Weave over multiple projects. Grabbing datasets programmatically would make that easier vs. now having to hard-code dataset refs as they're published for each project. 🙇

[m-rgba]

I don't believe this is currently in the SDK - you can query the API though to retrieve the datasets like the following:

import requests

### Your Info:
team_id=""
project_id=""
wandb_token=""
###

url = "https://trace.wandb.ai/objs/query"
headers = {
  'Content-Type': 'application/json',
  'Accept': 'application/json'
}
payload ={
  "project_id": f"{team_id}/{project_id}",
  "filter": {
    "base_object_classes": [
      "Dataset"
    ],
  },
  "metadata_only": True
}
response = requests.post(
  url, headers=headers, json=payload, auth=("api", wandb_token)
)
data = response.json()
print(data)

Depending on your use case you may want to return the actual datasets by setting metadata_only to False.

You can find more information on the request object here:
https://weave-docs.wandb.ai/reference/service-api/objs-query-objs-query-post

[akiyamasho]

Thank you @gtarpenning!