SignedJwtAssertionCredentials has been removed: Why?

[jefftune]

To: @dhermes

I am currently using class SignedJwtAssertionCredentials, and as of 2 days ago, it was removed.

oauth2client/client.py

dcd20c9#diff-d8b1525b7f161cbce2f6438636be9c38

What am I to use instead now?

Signed JWT is expected for Google Cloud API Authentication.

See SignedJwtAssertionCredentials._generate_assertion()

[dhermes]

Thanks for noticing! The behavior has been moved onto oauth2client.service_account.ServiceAccountCredentials. We will be sure to provide details in the release notes on how to migrate.

There is a factory constructor:

creds = ServiceAccountCredentials.from_p12_keyfile('[email protected]', '/path/to/keyfile.p12')

[nathanielmanistaatgoogle]

@jefftune: are you able to point us to documentation that needs to be updated as a result of the change that we made on Friday?

[voscausa]

Example code using ServiceAccountCredentials:

import webapp2
from oauth2client.service_account import ServiceAccountCredentials
import gspread

SCOPES = ['https://spreadsheets.google.com/feeds']

class GssHandler(webapp2.RequestHandler):
    """ replaces the SignedJwtAssertionCredentials """

    def get(self):

        credentials = ServiceAccountCredentials.from_json_keyfile_name('voscausa-sa-test.json', SCOPES)
        gss_client = gspread.authorize(credentials)
        gss = gss_client.open('test-gspread')
        worksheet = gss.sheet1
        self.response.write(worksheet.acell('A1').value)

To make in work in the appengine dev_server I installed:

• httplib2 0.9.2
• gspread 0.3.0
• oauth2client 1.5.2 from github. Now 2.0.0
• requests 2.3.0
• pyasn1 0.1.9
• payasn1-modules 0.0.8
• rsa 3.3
• six.py 1.10.0

[allenpc]

@dhermes one problem I'm running into when I tried to migrate to .from_p12_keyfile is that SignedJwtAssertionCredentials allowed you to pass in a File buffer yourself. Now, .from_p12_keyfile requires a local file path.

We pull in our p12 keys dynamically, and the old behavior allowed us to not have to write the contents to a temp file locally.

Any chance we could get a version of .from_p12_keyfile that takes in a raw File buffer?

[dhermes]

Ahhh @allenpc thanks for the heads up! We had discussed adding that as an intermediate constructor but decided "no one wants it". Proved wrong in less than 1 week!

[dhermes]

@nathanielmanistaatgoogle Shouldn't this issue be closed? Should a new issue be filed for the legit feature request of @allenpc?

[bood]

@dhermes @nathanielmanistaatgoogle
This page definitely needs an update:
https://developers.google.com/api-client-library/python/auth/service-accounts

I suggest do not close this issue until all documents are updated

[dhermes]

👍 good call

[allenpc]

@dhermes, moved my request out as a separate feature: #412

Agreed on keeping this open until all documentation is updated though. Here is another page that needs to get updated: https://developers.google.com/api-client-library/python/guide/aaa_oauth

[dhermes]

Thanks.

[alexbenfica]

Not that I like it... but keep the old version will be the solution for now... oauth2client==1.5.2

[dukedougal]

Authentication - one of the hardest things to get working in modern development and I've worked with Google authentication over the years and it's always been an absolute disaster, mainly because developers change things and leave the documentation out of date. I was hoping things might be different this time around but here it is again. The Python documentation is just plain wrong. Sorry so grumble to hard but man Google really puts in the effort to make things hard for developers trying to use its authentication. Again Google has just burned hours of my time - thanks.

[dhermes]

Thanks for taking the time to share @dukedougal. I can take the brunt of the blame for this change. We had 2 competing implementations for service accounts and as a programmer I just had to fix it. It didn't even occur to me to fix the various sets of docs on the Google developers site. I suppose there should be more great docstrings like in flask_util (w00t @jonparrott and @waprin) and the out-of-project docs should probably just point to oauth2client.

As for making a breaking change catching people off guard, I'm a bit unclear on what could have been done better. The major version was upgraded and the release notes have migration instructions.

[dhermes]

from oauth2client.service_account import ServiceAccountCredentials
credentials = ServiceAccountCredentials.from_json_keyfile_name(
    key_file_name, scopes=[...])

[patroqueeet]

Dear,

I can confirm @dhermes code snippet and add my two cents for the p12 (confirmed to be working today).

        # Create an httplib2.Http object to handle our HTTP requests and to
        # authorize them correctly.
        credentials = ServiceAccountCredentials.from_p12_keyfile(
            self.SERVICE_ACCOUNT_EMAIL,
            self.KEY_FILE,
            scopes=['https://www.googleapis.com/auth/fusiontables']
        )

        http = httplib2.Http()
        self.http = credentials.authorize(http)

        # Read the first page of features in a Table.
        self.service = discovery.build('fusiontables', 'v1', http=self.http)

but I had to switch this to master due to a bugfix just done today:

requirements.txt:

# https://github.com/google/google-api-python-client/issues/179 breaks
# google-api-python-client
git+https://github.com/google/google-api-python-client.git@master#egg=google-api-python-client

[dhermes]

Thanks @patroqueeet

[dukedougal]

I got it to work, thanks for your quick response.

But PLEASE update both the version on pip and your documentation. I'll probably come back here some day soon and think damn! I can't remember what I did to fix it, I hope the pip version is current and the documentation up to date. Don't do "the Google thing" and leave it for some other time.

i.e. this is wrong and out of date:

https://developers.google.com/api-client-library/python/auth/service-accounts

If you were super awesome then you would also search Google for SignedJwtAssertionCredentials+python and look for where else this is referred to on the Google documentation and elsewhere on the net and fix the Google documentation, email the bloggers who have written posts about it and put comments on StackOverflow and other support forums where people might be going to try to work out why SignedJwtAssertionCredentials doesn't seem to work anymore.

And finally, don't just chop the method out of the code. Leave it there with a nice error message for developers saying "this is now replaced with blah and you can find more information here".

One final thing, don't just delete the old stuff from the docs, explain that they have been moved and replaced. That way developers won't remain in the dark as to why the function they expected does not seem to exist at all.

[samdeepk]

I am not sure how many times referenced this import I wrote across last 5 years in various projects in different companies. I see i need to update 5 current projects of mine and ask other teams working with projects to upgrade their libraries.

I wonder what would be the recommendation, I should fix a version of google-api-python-client/oauth2client in our code repos or ask all developers working to upgrade to this change and fix all projects to avoids reference issues in future !?

I just realized this not affecting one branch we need to fix this on dev branches and manage merge issues later!

[theacodes]

I wonder what would be the recommendation, I should fix a version of google-api-python-client/oauth2client in our code repos or ask all developers working to upgrade to this change and fix all projects to avoids reference issues in future !?

In general the recommendation is to pin your packages. Major version changes will almost always introduce breaking changes, and even minor version release should be vetted before upgrading.

We can't advance the state of this project without breaking changes. We've bumped the major version number for these changes as expected.

@nathanielmanistaatgoogle @dhermes I can work on getting those docs updated today.

[dhermes]

Thanks a lot @jonparrott!

[theacodes]

Docs updated. Will need to be bumped again once #418 is resolved.

[dhermes]

Wow that was fast! Thanks

[bood]

@dhermes @jonparrott I noticed that the documents are updated, however, if I'm still using an old version, where could I find the document for the old version?

[theacodes]

Earth engine is likely using an older version of this library. You should file an issue on their repository asking them to upgrade. In the meantime, you can install oauth2client<2.0.0 and it should work.

[cgmorton]

Usually, rolling back to oauth2client 1.5.2 after updating the Earth Engine API (pip install earthengine-api==1.5.2) fixes the problem. I started an issue google/earthengine-api/issues/5, but feel free to add any additional information.

[yogesh4mantlelabs]

Hi, @jonparrott & @cgmorton Thanks for the reply. I will check what is earthengine-api=1.5.2 equivalent in windows as in windows versions range from 0.0.3 to 0.1.88 .

Initially i was using earthengine-api-0.1.87 (note that I am using windows) and oauth2client<2.
I got following error:
Traceback (most recent call last):
File "C:\Program Files (x86)\Google\Cloud SDK\google-cloud-sdk\platform\google
_appengine\google\appengine\runtime\wsgi.py", line 240, in Handle
handler = _config_handle.add_wsgi_middleware(self._LoadHandler())
File "C:\Program Files (x86)\Google\Cloud SDK\google-cloud-sdk\platform\google
_appengine\google\appengine\runtime\wsgi.py", line 299, in _LoadHandler
handler, path, err = LoadObject(self._handler)
File "C:\Program Files (x86)\Google\Cloud SDK\google-cloud-sdk\platform\google
_appengine\google\appengine\runtime\wsgi.py", line 85, in LoadObject
obj = import(path[0])
File "E:\googleearth\sample\crop_earthengine\demos\trendy-lights\server.py", l
ine 226, in
config.EE_ACCOUNT, config.EE_PRIVATE_KEY_FILE)
INFO 2016-07-16 11:16:36,957 module.py:788] default: "GET / HTTP/1.1" 500 -
INFO 2016-07-16 11:16:36,957 module.py:788] default: "GET / HTTP/1.1" 500 -
File "E:\googleearth\sample\crop_earthengine\demos\trendy-lights\lib\ee__init
__.py", line 182, in ServiceAccountCredentials
email, key_data, oauth.SCOPE)
File "E:\googleearth\sample\crop_earthengine\demos\trendy-lights\lib\oauth2cli
ent\util.py", line 140, in positional_wrapper
return wrapped(_args, *_kwargs)
File "E:\googleearth\sample\crop_earthengine\demos\trendy-lights\lib\oauth2cli
ent\client.py", line 1630, in init
_RequireCryptoOrDie()
File "E:\googleearth\sample\crop_earthengine\demos\trendy-lights\lib\oauth2cli
ent\client.py", line 1581, in _RequireCryptoOrDie
raise CryptoUnavailableError('No crypto library available')
CryptoUnavailableError: No crypto library available

---

To remove this error, i had tried installing pycrypto again, also i have cross checked with app.yaml for libraries but error remains as it is.

[theacodes]

Can you get earth engine to work outside of Google App Engine? Or better question, if you do:

C:\> python
>>> import Crypto

Do you get an error?

[yogesh4mantlelabs]

Actually after my last post, i had updated gcloud which had also updated Crypto libraries to latest version and then I got SignedJwtAssertionCredentials error posted earlier.

Meaning, when now when I run simple python with import Crypto, it doesnot give error.

For trial purpose, from ' %ANACONDA_PATH%\lib\site-packages ' i shifted following folders to a backup folder:
Crypto
cryptography
cryptography-1.4.dist-info
pycrypto-2.6.1.dist-info

So, i did the same for my lib folder in ' trendy-lights\lib '. Shifted those same packages to other backup folder.
Now when I run import Crypto, it gives error.

Additional

Without installing Crypto again, i tried to start trendy-lights app using dev_appserver.py ./ which is as below:

E:\googleearth\sample\crop_earthengine\demos\trendy-lights>dev_appserver.py ./

Updates are available for some Cloud SDK components. To install them,
please run:
$ gcloud components update

INFO 2016-07-19 11:09:40,792 devappserver2.py:769] Skipping SDK update check
.
INFO 2016-07-19 11:09:40,836 api_server.py:205] Starting API server at: http
://localhost:50165
INFO 2016-07-19 11:09:40,841 dispatcher.py:197] Starting module "default" ru
nning at: http://localhost:8080
INFO 2016-07-19 11:09:40,842 admin_server.py:116] Starting admin server at:
http://localhost:8000
ERROR 2016-07-19 05:39:56,780 wsgi.py:263]
Traceback (most recent call last):
File "C:\Program Files (x86)\Google\Cloud SDK\google-cloud-sdk\platform\google
_appengine\google\appengine\runtime\wsgi.py", line 240, in Handle
handler = _config_handle.add_wsgi_middleware(self._LoadHandler())
File "C:\Program Files (x86)\Google\Cloud SDK\google-cloud-sdk\platform\google
_appengine\google\appengine\runtime\wsgi.py", line 299, in _LoadHandler
handler, path, err = LoadObject(self._handler)
File "C:\Program Files (x86)\Google\Cloud SDK\google-cloud-sdk\platform\google
_appengine\google\appengine\runtime\wsgi.py", line 85, in LoadObject
obj = import(path[0])
File "E:\googleearth\sample\crop_earthengine\demos\trendy-lights\server.py", l
ine 226, in
config.EE_ACCOUNT, config.EE_PRIVATE_KEY_FILE)
File "E:\googleearth\sample\crop_earthengine\demos\trendy-lights\lib\ee__init
__.py", line 173, in ServiceAccountCredentials
return oauth2client.client.SignedJwtAssertionCredentials(
AttributeError: 'module' object has no attribute 'SignedJwtAssertionCredentials'

INFO 2016-07-19 11:09:56,788 module.py:788] default: "GET / HTTP/1.1" 500 -

[theacodes]

Can you do this in your code while running under dev-appserver?

import oauth2client
print(oauth2client.__version__)
print(oauth2client.__file__)

[dongseok0]

This page has incorrect info about p12 based setup - https://developers.google.com/api-client-library/python/auth/service-accounts

[theacodes]

@dongseok0 those seem correct, which part in particular is incorrect?

[dongseok0]

@jonparrott

private_key_password need to be added for from_p12_keyfile method argument.

credentials = ServiceAccountCredentials.from_p12_keyfile(
    client_email, '/path/to/keyfile.p12', scopes)

Need to be

credentials = ServiceAccountCredentials.from_p12_keyfile(
    client_email, '/path/to/keyfile.p12', private_key_password, scopes)

[theacodes]

The docs have been updated, but may not be published until later today.

[daguy666]

This docs still references from oauth2client.client import SignedJwtAssertionCredentials https://developers.google.com/admin-sdk/directory/v1/guides/delegation

[MeLight]

I second @daguy666 We're struggling to migrate from earlier versions of oauth2client and the SignedJwtAssertionCredentials class for 3 days now. I'm yet to find one working example. Please update delegations docs (https://developers.google.com/admin-sdk/directory/v1/guides/delegation) for Python.

[theacodes]

@MeLight delegated credentials can be created with ServiceAccountCredentials.create_delegated:

delegated_credentials = credentials.create_delegated('[email protected]')

Please let me know if this doesn't work for you.

[theacodes]

And FYI, I'm in the process of updating the docs. We don't directly own those docs so it's difficult to coordinate across multiple teams.

[MeLight]

@jonparrott thanks for the reply. This is my code:

from oauth2client.service_account import ServiceAccountCredentials
import httplib2
from googleapiclient import discovery    
import oauth2client
def main():
    """Shows basic usage of the Google Admin SDK Directory API.

    Creates a Google Admin SDK API service object and outputs a list of first
    10 users in the domain.
    """
    credentials = ServiceAccountCredentials.from_json_keyfile_name('qa-app-key.json', scopes=[
        'https://apps-apis.google.com/a/feeds/emailsettings/2.0/',
        'https://www.googleapis.com/auth/admin.directory.user',
        ])

    credentials = credentials.create_delegated('[email protected]')
    http = credentials.authorize(httplib2.Http())
    service = discovery.build('admin', 'directory_v1', http=http)

    print('Getting the first 10 users in the domain')
    results = service.users().list(customer='my_customer', orderBy='email').execute()

if __name__ == '__main__':
    main()

And this is the error I'm getting:

Traceback (most recent call last):
  File "auth.py", line 39, in <module>
    main()
  File "auth.py", line 23, in main
    results = service.users().list(domain='my_customer', orderBy='email').execute()
  File "C:\Python27\lib\site-packages\oauth2client\util.py", line 137, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "C:\Python27\lib\site-packages\googleapiclient\http.py", line 827, in execute
    method=str(self.method), body=self.body, headers=self.headers)
  File "C:\Python27\lib\site-packages\googleapiclient\http.py", line 154, in _retry_request
    resp, content = http.request(uri, method, *args, **kwargs)
  File "C:\Python27\lib\site-packages\oauth2client\transport.py", line 153, in new_request
    credentials._refresh(orig_request_method)
  File "C:\Python27\lib\site-packages\oauth2client\client.py", line 765, in _refresh
    self._do_refresh_request(http_request)
  File "C:\Python27\lib\site-packages\oauth2client\client.py", line 834, in _do_refresh_request
    raise HttpAccessTokenRefreshError(error_msg, status=resp.status)
oauth2client.client.HttpAccessTokenRefreshError: unauthorized_client: Unauthorized client or scope in request.

Please advise. Thanks

[theacodes]

@MeLight I don't think 'https://apps-apis.google.com/a/feeds/emailsettings/2.0/' is a valid scope. Does this work with oauth2client < 2.0.0?

[MeLight]

@jonparrott yes it works with previous versions, also you can find this scope documented here: https://developers.google.com/admin-sdk/email-settings/auth

[theacodes]

@MeLight can you share the working previous version code? Are you using the same service account for both?

[MeLight]

@jonparrott sure:

    @staticmethod
    def check_access(admin_email):
        http = httplib2.Http()
        credentials = SignedJwtAssertionCredentials(
            SERVICE_EMAIL, 
            PRIVATE_KEY, 
            scope='https://apps-apis.google.com/a/feeds/emailsettings/2.0/ https://www.googleapis.com/auth/admin.directory.user.readonly', 
            sub=str(admin_email),
        )
        http = credentials.authorize(http)
        try:
            service = build(serviceName='admin', version='directory_v1', http=http)
            logging.info("App has access to admin's %s domain" % (admin_email))
            return True
        except Exception as e:
            logging.info("App does not have access to admin's %s domain (exception: %s)" % (admin_email, e.message))
            return False

[MeLight]

@jonparrott very embrassing - I think I've found the error.

[theacodes]

@MeLight I'm glad you figured it out. Do you mind sharing what the issue was?

[MeLight]

After permutating many combinations of keys/ids/permissions/scopes I forgot to change back the scope of admin sdk to admin.directory.user.readonly. Our app does not have admin.directory.user scope permission, rather only the readonly version. Deepest apologies for the confusion.

Note for future generations: the above posted code can be used for domain delegation, and for your convenience I've pasted a gist with a working example:
https://gist.github.com/MeLight/1f4517560a9761317d13ebb2cdc670d3

[theacodes]

Thanks! Glad you got it figured out.

On Wed, Aug 10, 2016, 4:14 PM Yuri Sh [email protected] wrote:

After permutating many combinations of keys/ids/permissions/scopes I
forgot to change back the scope of admin sdk to
admin.directory.user.readonly. Our app does not have admin.directory.user
scope permission, rather only the readonly version. Sorry for the
confusion.

Note for future generations: the above posted code can be used for domain
delegation, and for your convenience I've pasted a gist with a working
example::
https://gist.github.com/MeLight/1f4517560a9761317d13ebb2cdc670d3

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#401 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAPUc6L0_ZRQRo0B57XS8beNw-dekXd1ks5qeltegaJpZM4HVDZY
.

[aeijdenberg]

Please consider the following PR to re-add the legacy method signature:
#687