Authenticating Service Accounts

[Joehutz]

Hi,

I'm trying to build an application and hope to use service account credentials for uploading and downloading. I'm having some issues authenticating the service account by following PyDrive's documentation.

After creating my service account, I renamed the "secret" to 'client_secrets.json' and I encounter the following error message when I authenticate:

Traceback (most recent call last):
  File "downloader.py", line 24, in <module>
    file_list = drive.ListFile().GetList()
  File "C:\Python36-32\lib\site-packages\pydrive-1.3.1-py3.6.egg\pydrive\apiattr.py", line 162, in GetList
  File "C:\Python36-32\lib\site-packages\pydrive-1.3.1-py3.6.egg\pydrive\apiattr.py", line 146, in __next__
  File "C:\Python36-32\lib\site-packages\pydrive-1.3.1-py3.6.egg\pydrive\auth.py", line 57, in _decorated
  File "C:\Python36-32\lib\site-packages\pydrive-1.3.1-py3.6.egg\pydrive\auth.py", line 113, in _decorated
  File "C:\Python36-32\lib\site-packages\pydrive-1.3.1-py3.6.egg\pydrive\auth.py", line 443, in GetFlow
  File "C:\Python36-32\lib\site-packages\pydrive-1.3.1-py3.6.egg\pydrive\auth.py", line 366, in LoadClientConfig
  File "C:\Python36-32\lib\site-packages\pydrive-1.3.1-py3.6.egg\pydrive\auth.py", line 388, in LoadClientConfigFile
pydrive.settings.InvalidConfigError: Invalid client secrets file Invalid file format. See https://developers.google.com/api-client-library/python/guide/aaa_client_secrets Expected a JSON object with a single property for a "web" or "installed" application

When I look at auth.py, I see the function defined as ServiceAuth().
I'm able to use google's stock api to achieve listfile:

scopes = ['https://www.googleapis.com/auth/drive']

credentials = ServiceAccountCredentials.from_json_keyfile_name('client_secrets.json', scopes)
http_auth = credentials.authorize(Http())
drive = build('drive','v3',http=http_auth)

response = drive.files().list().execute()

How do I go about authenticating a service account via PyDrive?

[rodrigorodriguescosta]

Same error here, I need to use service account because my server will upload files direct to my account, any news about this issue?

[georgep7n]

i'm getting the same error too for a service account.

[RNabel]

Hi all, could you post the version of oath2client you have currently installed (using pip freeze)? Have you updated it recently?

[georgep7n]

I've got 4.1.2

[wtluke]

Has anyone had any success with this yet?

[paullombard]

Any more on this?

[ghostku]

It works for me as:

gauth = GoogleAuth()
scope = ['https://www.googleapis.com/auth/drive']
gauth.credentials = ServiceAccountCredentials.from_json_keyfile_name(JSON_FILE, scope)
drive = GoogleDrive(gauth)

my_file = drive.CreateFile({'id': 'FILE_ID'}) 

[Californian]

The code that @ghostku posted worked perfectly for me (fyi, GoogleAuth and ServiceAccountCredentials are imported from pydrive.auth); thanks so much for that.

I don't understand why, but gauth.ServiceAuth() did not work, and there is no documentation for it either. That should probably be the way service authentication is done, since the code exists and that's how the rest of the authentication methods are executed in the docs; if one of the project maintainers has time, it would probably help a lot of people going forward to do a short writeup of that (or describe how in comments here, and if I get it working I'll gladly submit a PR to add it to the docs myself).

[clytemnestra]

@ghostku 's method works perfect, would be great to have it documented somewhere. Or to be able to have the json file be taken from the default or yaml config and not hardcoded.

[jrake-revelant]

Using @ghostku 's method and the following example code from the projects documentation

# Paginate file lists by specifying number of max results
for file_list in drive.ListFile({'q': 'trashed=true', 'maxResults': 10}):
  print('Received %s files from Files.list()' % len(file_list)) # <= 10
  for file1 in file_list:
      print('title: %s, id: %s' % (file1['title'], file1['id']))

results in the follow error for me

AttributeError: 'Credentials' object has no attribute 'access_token_expired'

[jrake-revelant]

Never mind, I was using the wrong credentials method.

[mithunmanohar]

#107 (comment)
Where to find the file that is created by this ?

[shlomiLan]

Tried to use @ghostku code, but I'm getting no files (empty list) when running:

file_list = drive.ListFile({'q': "'root' in parents and trashed=false"}).GetList()

When using googleapiclient I do see files, what I'm missing?

    from googleapiclient.discovery import build
    drive_service = build('drive', 'v3', credentials=creds)
    # Call the Drive v3 API
    results = drive_service.files().list(
        pageSize=10, fields="nextPageToken, files(id, name)").execute()
    print(results)

[tamilarasumk731]

@ghostku Same issue as @shlomiLan

[shcheklein]

@tamilarasumk731 could you try the PyDrive2 fork? We've fixed a bunch of errors related to the services accounts and missing options to different API calls to work properly with shared drives/shared folders (usual reason for list to be empty, but not sure if that's the case in your case)

[shlomiLan]

I want to be able to read the credentials from an environment variable, what will be the best way to do that?
This is my code:

def init_connection():
    credentials_data = os.environ.get('GOOGLE_SERVICE_ACCOUNT_CREDENTIALS')
    if not credentials_data:
        raise KeyError('No credentials data, missing environment variable')

    credentials_data = json.loads(credentials_data)
    # Fix the 'private_key' escaping
    credentials_data['private_key'] = credentials_data.get('private_key').encode().decode('unicode-escape')

    from oauth2client.service_account import ServiceAccountCredentials
    scope = ['https://www.googleapis.com/auth/drive']

    credentials = ServiceAccountCredentials.from_json_keyfile_dict(credentials_data, scope)

    return credentials

def list_files():
    creds = init_connection()
    drive_service = build('drive', 'v3', credentials=creds)
    # Call the Drive v3 API
    results1 = drive_service.files().list(
        pageSize=10, fields="nextPageToken, files(id, name)").execute()

    print(results1)

    from pydrive2.auth import GoogleAuth
    from pydrive2.drive import GoogleDrive

    gauth = GoogleAuth()
    gauth.credentials = creds

    drive = GoogleDrive(gauth)
    result2 = drive.ListFile({'q': "'root' in parents"}).GetList()
    print(result2)

In result1 I get results that are in the bucket and in result2 I got an empty list. What I'm going wrong?

[fish-face]

When using @ghostku's workaround, you should also set the auth method:

gauth.auth_method = 'service'

Otherwise if you do many operations over a long time your access token will expire, and PyDrive(2) will incorrectly try to refresh the access token via the LocalServerAuth() method, failing with oauth2client.clientsecrets.InvalidClientSecretsError: ('Error opening file', 'client_secrets.json', 'No such file or directory', 2) (because you didn't specify client_secrets.json since it's not necessary if you do things this way.)

[im8080]

When using @ghostku's workaround, you should also set the auth method:

gauth.auth_method = 'service'

Otherwise if you do many operations over a long time your access token will expire, and PyDrive(2) will incorrectly try to refresh the access token via the LocalServerAuth() method, failing with oauth2client.clientsecrets.InvalidClientSecretsError: ('Error opening file', 'client_secrets.json', 'No such file or directory', 2) (because you didn't specify client_secrets.json since it's not necessary if you do things this way.)

@fish-face @ghostku

I set gauth.auth_method = 'service', but after 1 hour of execution, the same exception still occurs.
failing with oauth2client.clientsecrets.InvalidClientSecretsError: ('Error opening file', 'client_secrets.json', 'No such file or directory'.

Do you know the reason? Or, is there any solution? I use service account's delegated credentials. Thank you very much.

code show as below:

scope = ['https://www.googleapis.com/auth/drive']
credentials = ServiceAccountCredentials.from_json_keyfile_name(SERVICE_ACCOUNT_FILE, scope)
delegated_credentials = credentials.create_delegated(delegated_email)

gauth = GoogleAuth()
gauth.credentials = delegated_credentials
gauth.auth_method = 'service'