Project from client (bill-stackdriver-experiment) should agree with project from resource(1054618175867).

[waprin]

I'm not sure if this is a gcloud-python issue or a bug on the server-side.

Doing list_log.entries() gives the error:

  File "/Users/waprin/.virtualenvs/docsamples10/lib/python2.7/site-packages/gcloud/_helpers.py", line 464, in _name_from_project_path
    'project from resource(%s).' % (project, found_project))
ValueError: Project from client (bill-stackdriver-experiment) should agree with project from resource(1054618175867).

Adding some print statements reveals the reason:

LOGGER NAME IS projects/1054618175867/logs/appengine.googleapis.com%2Frequest_log

It seems some of the app engine logs return the project as the project number rather then the project ID. Note that 1054618175867 is the correct project number for the project. I do feel like there should be consistency and only the project id should be used so maybe this is a server side bug, but I'm not sure if there's any official external docs to reference. Getting the project number from a project id also might mean an extra API call is required just to do a validation.

My vote would be to temporarily disable the validation for now and follow up internally about the format, thoughts?

[waprin]

Ok, I missed the docs earlier:

https://cloud.google.com/logging/docs/api/ref_v2beta1/rest/v2beta1/LogEntry

Required. The resource name of the log to which this log entry belongs. The format of the name is "projects/<project-id>/logs/<log-id>". Examples: "projects/my-projectid/logs/syslog", "projects/1234567890/logs/library.googleapis.com%2Fbook_log".
So the docs do suggest that project number should be supported. I am not a fan though.

[dhermes]

@waprin Can you get a deeper stacktrace? The offending line isn't very helpful without the RegEx passed in.

[dhermes]

Update: This seems to be the correct RegEx, and it is used in only one place

[waprin]

Full stacktrace at the bottom, but the issue is not with the regex ,the issue AFAICT is that the validation checks that the client project ID equals the project ID returned by the resource, which is not going to be true when the resource returns the project number instead. I have filed an internal bug about the fact that using project number interchangably with project id is bad but doubt it will get resolved soon.

Traceback (most recent call last):
  File "list_logs.py", line 74, in <module>
    main(args.project_id)
  File "list_logs.py", line 62, in main
    list_logs(client)
  File "list_logs.py", line 39, in list_logs
    entries, token = client.list_entries()  # API call
  File "/Users/waprin/.virtualenvs/docsamples10/lib/python2.7/site-packages/gcloud/logging/client.py", line 141, in list_entries
    for resource in resp.get('entries', ())]
  File "/Users/waprin/.virtualenvs/docsamples10/lib/python2.7/site-packages/gcloud/logging/client.py", line 83, in _entry_from_resource
    return ProtobufEntry.from_api_repr(resource, self, loggers)
  File "/Users/waprin/.virtualenvs/docsamples10/lib/python2.7/site-packages/gcloud/logging/entries.py", line 86, in from_api_repr
    logger_fullname, client.project)
  File "/Users/waprin/.virtualenvs/docsamples10/lib/python2.7/site-packages/gcloud/logging/_helpers.py", line 46, in logger_name_from_path
    return _name_from_project_path(path, project, _LOGGER_TEMPLATE)
  File "/Users/waprin/.virtualenvs/docsamples10/lib/python2.7/site-packages/gcloud/_helpers.py", line 464, in _name_from_project_path
    'project from resource(%s).' % (project, found_project))
ValueError: Project from client (bill-stackdriver-experiment) should agree with project from resource(1054618175867).

[dhermes]

Yeah, as you mentioned in your original post, the root cause is validating the project, I just wanted to see where the code traveled to make it there.

[dhermes]

@tseaver WDYT about a match_project=True (or validate_project or validate) flag somewhere in the pipeline?

[tseaver]

For other types which share the use of gcloud._helpers._name_from_project_path, the validation is important: instances make API calls through their "host" client, and need to be "within" its project. Log entry instances don't have any methods which wrap API calls, which means we could probably get away with dropping the validation (for them).