Improved Filter Syntax

[blowmage]

It would be super cool to have a better way to express the filter criteria rather than the filter syntax supported by the API. We don't know what that syntax looks like, but it would be super cool.

See #342 (comment)

[blowmage]

in #342 (comment) @jgeewax said:

What's unclear to me in that syntax is how I filter with multiple labels... and I'm no Ruby expert, but I'd expect the map to be an actual map, not a string looking like a map, so ideally...

filters = { :labels => { :env => "production" } }
# or if we can't do nesting
filters = { "labels.env" => "production" }
# or (if we want multiple options)
filters = { "labels.env" => ["production", "prod"], "name" => "name" } # (production *or* prod) *and* name
projects = resource_manager.projects filter: filters

[blowmage]

According to the documentation, multiple expressions are AND. There doesn't seem to be any support for OR.

"labels.color:red labels.size:big" The project's label color has the value red and its label size has the value big.

[blowmage]

The above example:

filters = { "labels.env" => ["production", "prod"], "name" => "name" } # (production *or* prod) *and* name
projects = resource_manager.projects filter: filters

would have to become:

filters = { labels: { env: :production, name: :name } } # production *and* name
projects = resource_manager.projects filter: filters

would be equivalent to:

filters = "labels.env:production labels.name:name"
projects = resource_manager.projects filter: filters

[blowmage]

One thing I noticed is you can't do a partial match. I kinda hoped that you could find all projects with "gcloud" in the name with the following:

projects = resource_manager.projects filter: "name:*gcloud*"

But that sadly does not work. The filter needs to be an exact match, albeit case-insensitive. Because of this, you wont filter by labels, and name or id. In fact, of the 3, only labels would have multiple criteria being other labels. So, nesting multiple hashes under filter to me seems to be deep.

What if we left filter alone, so you can provide the string as it works today, and we introduced new options that create (or append to) the filter criteria? Something like this:

projects = resource_manager.projects id: "gcloud-example-789" # "id.=:gcloud-example-789"
projects = resource_manager.projects name: "Gcloud" # "name:Gcloud"
projects = resource_manager.projects labels: { env: :production, name: "*" } # "labels.env:production labels.name:*"

[blowmage]

BTW, I have yet to figure out how to filter on a name with spaces in it.

projects = resource_manager.projects filter: "name:Gcloud Example" # raises "Request contains an invalid argument."
projects = resource_manager.projects filter: "name:\"Gcloud Example\"" # no results
projects = resource_manager.projects filter: "name:GcloudNoSpacesInName" # works, finds project
projects = resource_manager.projects filter: "name:\"GcloudNoSpacesInName\"" # works even while quoted!

[jackfirth]

@blowmage We're looking into this internally now, we'll get back to you

[blowmage]

Thanks @jackfirth!

[quartzmo]

I'm going to remove this issue from the 0.5.0 Milestone, but keep it open.

[jackfirth]

@blowmage We've found a bug that may explain parts of the behavior you're seeing. If you have a project named "Foo Bar" and try to filter your projects by name through the api with this shell script:

export OAUTH_TOKEN=<your token without angle brackets>
curl \
    --header "Authorization: Bearer ${OAUTH_TOKEN}" \
    --header "Content-Type: application/json" \
    --data-urlencode "filter=name:\"Foo Bar\"" \
    -G https://cloudresourcemanager.googleapis.com/v1beta1/projects

Then the api fails to find the project, even though the name has been properly escaped and quoted. However, if your request looks like this:

export OAUTH_TOKEN=<your token without angle brackets>
curl \
    --header "Authorization: Bearer ${OAUTH_TOKEN}" \
    --header "Content-Type: application/json" \
    --data-urlencode "filter=name:\"foo bar\"" \
    -G https://cloudresourcemanager.googleapis.com/v1beta1/projects

Then the api finds the project. This is a case sensitivity bug that we're working on resolving. In the meantime, see if you have issues escaping spaces in project names that are all lowercase, as that would mean a second bug exists.

[jackfirth]

Relevant documentation for this request is here: https://cloud.google.com/resource-manager/reference/rest/v1beta1/projects/list

[blowmage]

Fantastic! Sure enough my filter value was mixed case. Very glad to know this. Should we down case the string in gcloud to work around this? Or will the service be updated soon?

Also, on the URL you shared there is a typo. There is a filter value example that is "labels.color:red label.size:big", but best we can tell it should be "labels.color:red labels.size:big". I'm not sure if a bug was ever filed for that, and I have no way of checking.

Thank you very much for your help!

[jackfirth]

@blowmage Update: We've fixed the issue and will push the fix to prod soon.

[blowmage]

Thanks! Looking forward to it. :)

[blowmage]

Moved to feature backlog.