Fix libraries in filter.go to use the correct operators supported by GCE

[prameshj]

Reverts #6

Most GCE APIs don't support regexp filter, hence reverting this.

Picked this up in k/k and Routes Controller started failing when listing routes with:

GCERoutes.List(context.Background.WithDeadline(2019-08-19 23:56:59.880823189 +0000 UTC m=+5923.032348891 [59m59.8650081s]), ..., (name ~ e2e-test-.) (network ~ https://www.googleapis.com/compute/v1/projects/pavithrar-k8s-dev/global/networks/e2e-test) (description ~ k8s-node-route)) = , googleapi: Error 400: Invalid value for field 'filter': '(name ~ e2e-test-pavithrar-.) (network ~ https://www.googleapis.com/compute/v1/projects/pavithrar-k8s-dev/global/networks/e2e-test-pavithrar) (description ~ k8s-node-route)'. Invalid list filter expression., invalid

The GCE APIs only support filter of the form "name = value" and not regexp. gcloud seems to be implementing regex filter and invoking API to get the full list.
ingress-gce uses composite types for most resources and no filter so far, which is probably why it has not broken any ingress tests.

[prameshj]

/assign @bowei @rramkumar1
@spencerhance

[prameshj]

/hold

Actually we need a part of the changes in this PR. Equals needs to translate to "=" and NotEquals needs to translate to " != ".
From the GCE API docs:

"A filter expression that filters resources listed in the response. The expression must specify the field name, a comparison operator, and the value that you want to use for filtering. The value must be a string, a number, or a boolean. The comparison operator must be either =, !=, >, or <."

This is the same for Instances, Routes, Addresses, ForwardingRules..
We can keep RegexpEquals as an option still, but not the default. I will work on the changes and submit for review.

[prameshj]

/hold

Actually we need a part of the changes in this PR. Equals needs to translate to "=" and NotEquals needs to translate to " != ".
From the GCE API docs:

"A filter expression that filters resources listed in the response. The expression must specify the field name, a comparison operator, and the value that you want to use for filtering. The value must be a string, a number, or a boolean. The comparison operator must be either =, !=, >, or <."

This is the same for Instances, Routes, Addresses, ForwardingRules..
We can keep RegexpEquals as an option still, but not the default. I will work on the changes and submit for review.

UPDATE:

Turns out the GCE documentation is a bit confusing here, I tried this out using the "Try this API" section here: https://cloud.google.com/compute/docs/reference/rest/v1/routes/list

1. Using "eq" for string equality/regex works, The following filter returned the correct matches:
   '(name eq e2e-test-pavithrar-.*) (network eq https://www.googleapis.com/compute/v1/projects/pavithrar-gke-dev/global/networks/e2e-test-pavithrar) (description eq k8s-node-route)'

2. Using "=" for works only for exact match. The following filter returned a 400 "Invalid value for filter ..."
   '(name = e2e-test-pavithrar-.*) (network = https://www.googleapis.com/compute/v1/projects/pavithrar-gke-dev/global/networks/e2e-test-pavithrar) (description = k8s-node-route)'

3. Using "=" for exact string match works, but not for regex. The following filter returns the correct matches:
   '(network = "https://www.googleapis.com/compute/v1/projects/pavithrar-gke-dev/global/networks/e2e-test-pavithrar") (description = "k8s-node-route")'

4. Using "eq" works for int and bool matches.
   a) '(network eq https://www.googleapis.com/compute/v1/projects/pavithrar-gke-dev/global/networks/e2e-test-pavithrar) (description eq k8s-node-route) (priority eq 1000)'
   b) '(autoCreateSubnetworks eq false)' for networks api works.

So, for exact string match, the docs are correct. We need to use "=". For regex match, the operator to use is "eq". "eq" also works for int and bool equality. Finally, "~" is not supported for regex yet.

Reverting this PR will replace all equality filters with "eq", which (based on 1 and 4 above) should work.

[prameshj]

Added a new test and verified that the filters work after the revert(and did not work before)

/assign @bowei

[rramkumar1]

LGTM

[bowei]

It would be better if we are keeping the upper layer Regexp() functions to keep the semantic meaning until the translation into GCE API.

[prameshj]

It would be better if we are keeping the upper layer Regexp() functions to keep the semantic meaning until the translation into GCE API.

I just tried making the change so we keep Regexp functions, but use 'eq' operator and use '=' for exact matches. Turns out we cannot do a mix of 2operators in the API query:
'(name eq e2e-test-.) (description eq k8s-node-route) (priority = 1000)' fails, but '(name eq e2e-test-.) (description eq k8s-node-route) (priority eq 1000)' works.

We could keep Regexp and equals but fill in the same operator 'eq' always.

[bowei]

add comment "GCE API maps regexp comparison to eq"