`airbyte/source-google-ads` connector's SPEC command does not succeed #13580

evantahler · 2022-06-07T22:11:18Z

$ docker run airbyte/source-google-ads spec

{"type": "LOG", "log": {"level": "FATAL", "message": "'NoneType' object has no attribute 'message_types_by_name'\nTraceback (most recent call last):\n  File \"/airbyte/integration_code/main.py\", line 9, in <module>\n    from source_google_ads import SourceGoogleAds\n  File \"/airbyte/integration_code/source_google_ads/__init__.py\", line 25, in <module>\n    from .source import SourceGoogleAds\n  File \"/airbyte/integration_code/source_google_ads/source.py\", line 13, in <module>\n    from google.ads.googleads.errors import GoogleAdsException\n  File \"/usr/local/lib/python3.9/site-packages/google/ads/googleads/__init__.py\", line 16, in <module>\n    import google.ads.googleads.client\n  File \"/usr/local/lib/python3.9/site-packages/google/ads/googleads/client.py\", line 20, in <module>\n    from google.api_core.gapic_v1.client_info import ClientInfo\n  File \"/usr/local/lib/python3.9/site-packages/google/api_core/gapic_v1/__init__.py\", line 16, in <module>\n    from google.api_core.gapic_v1 import config\n  File \"/usr/local/lib/python3.9/site-packages/google/api_core/gapic_v1/config.py\", line 25, in <module>\n    from google.api_core import exceptions\n  File \"/usr/local/lib/python3.9/site-packages/google/api_core/exceptions.py\", line 29, in <module>\n    from google.rpc import error_details_pb2\n  File \"/usr/local/lib/python3.9/site-packages/google/rpc/error_details_pb2.py\", line 39, in <module>\n    _RETRYINFO = DESCRIPTOR.message_types_by_name[\"RetryInfo\"]\nAttributeError: 'NoneType' object has no attribute 'message_types_by_name'"}}
{"type": "TRACE", "trace": {"type": "ERROR", "emitted_at": 1654639830233.4658, "error": {"message": "Something went wrong in the connector. See the logs for more details.", "internal_message": "'NoneType' object has no attribute 'message_types_by_name'", "stack_trace": "Traceback (most recent call last):\n  File \"/airbyte/integration_code/main.py\", line 9, in <module>\n    from source_google_ads import SourceGoogleAds\n  File \"/airbyte/integration_code/source_google_ads/__init__.py\", line 25, in <module>\n    from .source import SourceGoogleAds\n  File \"/airbyte/integration_code/source_google_ads/source.py\", line 13, in <module>\n    from google.ads.googleads.errors import GoogleAdsException\n  File \"/usr/local/lib/python3.9/site-packages/google/ads/googleads/__init__.py\", line 16, in <module>\n    import google.ads.googleads.client\n  File \"/usr/local/lib/python3.9/site-packages/google/ads/googleads/client.py\", line 20, in <module>\n    from google.api_core.gapic_v1.client_info import ClientInfo\n  File \"/usr/local/lib/python3.9/site-packages/google/api_core/gapic_v1/__init__.py\", line 16, in <module>\n    from google.api_core.gapic_v1 import config\n  File \"/usr/local/lib/python3.9/site-packages/google/api_core/gapic_v1/config.py\", line 25, in <module>\n    from google.api_core import exceptions\n  File \"/usr/local/lib/python3.9/site-packages/google/api_core/exceptions.py\", line 29, in <module>\n    from google.rpc import error_details_pb2\n  File \"/usr/local/lib/python3.9/site-packages/google/rpc/error_details_pb2.py\", line 39, in <module>\n    _RETRYINFO = DESCRIPTOR.message_types_by_name[\"RetryInfo\"]\nAttributeError: 'NoneType' object has no attribute 'message_types_by_name'\n", "failure_type": "system_error"}}}

The text was updated successfully, but these errors were encountered:

davydov-d · 2022-06-08T19:13:37Z

@evantahler are you running it against the latest image? although I've figured out the possible cause, I can not reproduce it on the just pulled image:

(.venv) ddavydov@LWO1-LHP-A10584:~/airbyte/airbyte-integrations/connectors/source-google-ads$ docker pull airbyte/source-google-ads
Using default tag: latest
latest: Pulling from airbyte/source-google-ads
42c077c10790: Pull complete 
f63e77b7563a: Pull complete 
bd5fdadae976: Pull complete 
205491f53016: Pull complete 
2030d9f0f012: Pull complete 
f050921cccf1: Pull complete 
307b6cbb80d2: Pull complete 
bc89ee555c9c: Pull complete 
a25e412ae77a: Pull complete 
aa7c31888cd5: Pull complete 
ba0404f4e68b: Pull complete 
Digest: sha256:1a55590f2ca6d9e0543a6bf3725378b994d5be6249d6e1a23fe158264f6733c5
Status: Downloaded newer image for airbyte/source-google-ads:latest
docker.io/airbyte/source-google-ads:latest
(.venv) ddavydov@LWO1-LHP-A10584:~/airbyte/airbyte-integrations/connectors/source-google-ads$ docker run airbyte/source-google-ads spec
{"type": "SPEC", "spec": {"documentationUrl": "https://docs.airbyte.com/integrations/sources/google-ads", "connectionSpecification": {"$schema": "http://json-schema.org/draft-07/schema#", "title": "Google Ads Spec", "type": "object", "required": ["credentials", "start_date", "customer_id"], "additionalProperties": true, "properties": {"credentials": {"type": "object", "description": "", "title": "Google Credentials", "order": 0, "required": ["developer_token", "client_id", "client_secret", "refresh_token"], "properties": {"developer_token": {"type": "string", "title": "Developer Token", "order": 0, "description": "Developer token granted by Google to use their APIs. More instruction on how to find this value in our <a href=\"https://docs.airbyte.com/integrations/sources/google-ads#setup-guide\">docs</a>", "airbyte_secret": true}, "client_id": {"type": "string", "title": "Client ID", "order": 1, "description": "The Client ID of your Google Ads developer application. More instruction on how to find this value in our <a href=\"https://docs.airbyte.com/integrations/sources/google-ads#setup-guide\">docs</a>"}, "client_secret": {"type": "string", "title": "Client Secret", "order": 2, "description": "The Client Secret of your Google Ads developer application. More instruction on how to find this value in our <a href=\"https://docs.airbyte.com/integrations/sources/google-ads#setup-guide\">docs</a>", "airbyte_secret": true}, "refresh_token": {"type": "string", "title": "Refresh Token", "order": 3, "description": "The token for obtaining a new access token. More instruction on how to find this value in our <a href=\"https://docs.airbyte.com/integrations/sources/google-ads#setup-guide\">docs</a>", "airbyte_secret": true}, "access_token": {"type": "string", "title": "Access Token (Optional)", "order": 4, "description": "Access Token for making authenticated requests. More instruction on how to find this value in our <a href=\"https://docs.airbyte.com/integrations/sources/google-ads#setup-guide\">docs</a>", "airbyte_secret": true}}}, "customer_id": {"title": "Customer ID(s)", "type": "string", "description": "Comma separated list of (client) customer IDs. Each customer ID must be specified as a 10-digit number without dashes. More instruction on how to find this value in our <a href=\"https://docs.airbyte.com/integrations/sources/google-ads#setup-guide\">docs</a>. Metrics streams like AdGroupAdReport cannot be requested for a manager account.", "pattern": "^[0-9]{10}(,[0-9]{10})*$", "examples": ["6783948572,5839201945"], "order": 1}, "start_date": {"type": "string", "title": "Start Date", "description": "UTC date and time in the format 2017-01-25. Any data before this date will not be replicated.", "pattern": "^[0-9]{4}-[0-9]{2}-[0-9]{2}$", "examples": ["2017-01-25"], "order": 2}, "end_date": {"type": "string", "title": "End Date (Optional)", "description": "UTC date and time in the format 2017-01-25. Any data after this date will not be replicated.", "pattern": "^[0-9]{4}-[0-9]{2}-[0-9]{2}$", "examples": ["2017-01-30"], "order": 6}, "custom_queries": {"type": "array", "title": "Custom GAQL Queries (Optional)", "description": "", "order": 3, "items": {"type": "object", "properties": {"query": {"type": "string", "title": "Custom Query", "description": "A custom defined GAQL query for building the report. Should not contain segments.date expression because it is used by incremental streams. See Google's <a href=\"https://developers.google.com/google-ads/api/fields/v9/overview_query_builder\">query builder</a> for more information.", "examples": ["SELECT segments.ad_destination_type, campaign.advertising_channel_sub_type FROM campaign WHERE campaign.status = 'PAUSED'"]}, "table_name": {"type": "string", "title": "Destination Table Name", "description": "The table name in your destination database for choosen query."}}}}, "login_customer_id": {"type": "string", "title": "Login Customer ID for Managed Accounts (Optional)", "description": "If your access to the customer account is through a manager account, this field is required and must be set to the customer ID of the manager account (10-digit number without dashes). More information about this field you can see <a href=\"https://developers.google.com/google-ads/api/docs/concepts/call-structure#cid\">here</a>", "pattern": "^([0-9]{10})?$", "examples": ["7349206847"], "order": 4}, "conversion_window_days": {"title": "Conversion Window (Optional)", "type": "integer", "description": "A conversion window is the period of time after an ad interaction (such as an ad click or video view) during which a conversion, such as a purchase, is recorded in Google Ads. For more information, see Google's <a href=\"https://support.google.com/google-ads/answer/3123169?hl=en\">documentation</a>.", "minimum": 0, "maximum": 1095, "default": 14, "examples": [14], "order": 5}}}, "authSpecification": {"auth_type": "oauth2.0", "oauth2Specification": {"rootObject": ["credentials"], "oauthFlowInitParameters": [["client_id"], ["client_secret"], ["developer_token"]], "oauthFlowOutputParameters": [["access_token"], ["refresh_token"]]}}}}
(.venv) ddavydov@LWO1-LHP-A10584:~/airbyte/airbyte-integrations/connectors/source-google-ads$

evantahler · 2022-06-08T20:37:50Z

Weird! I just confirmed again and see the same error. Our Docker image SHAs match

➜  ~ docker pull airbyte/source-google-ads
Using default tag: latest
latest: Pulling from airbyte/source-google-ads
Digest: sha256:1a55590f2ca6d9e0543a6bf3725378b994d5be6249d6e1a23fe158264f6733c5
Status: Image is up to date for airbyte/source-google-ads:latest
docker.io/airbyte/source-google-ads:latest
➜  ~ docker run airbyte/source-google-ads spec
{"type": "LOG", "log": {"level": "FATAL", "message": "'NoneType' object has no attribute 'message_types_by_name'\nTraceback (most recent call last):\n  File \"/airbyte/integration_code/main.py\", line 9, in <module>\n    from source_google_ads import SourceGoogleAds\n  File \"/airbyte/integration_code/source_google_ads/__init__.py\", line 25, in <module>\n    from .source import SourceGoogleAds\n  File \"/airbyte/integration_code/source_google_ads/source.py\", line 13, in <module>\n    from google.ads.googleads.errors import GoogleAdsException\n  File \"/usr/local/lib/python3.9/site-packages/google/ads/googleads/__init__.py\", line 16, in <module>\n    import google.ads.googleads.client\n  File \"/usr/local/lib/python3.9/site-packages/google/ads/googleads/client.py\", line 20, in <module>\n    from google.api_core.gapic_v1.client_info import ClientInfo\n  File \"/usr/local/lib/python3.9/site-packages/google/api_core/gapic_v1/__init__.py\", line 16, in <module>\n    from google.api_core.gapic_v1 import config\n  File \"/usr/local/lib/python3.9/site-packages/google/api_core/gapic_v1/config.py\", line 25, in <module>\n    from google.api_core import exceptions\n  File \"/usr/local/lib/python3.9/site-packages/google/api_core/exceptions.py\", line 29, in <module>\n    from google.rpc import error_details_pb2\n  File \"/usr/local/lib/python3.9/site-packages/google/rpc/error_details_pb2.py\", line 39, in <module>\n    _RETRYINFO = DESCRIPTOR.message_types_by_name[\"RetryInfo\"]\nAttributeError: 'NoneType' object has no attribute 'message_types_by_name'"}}
{"type": "TRACE", "trace": {"type": "ERROR", "emitted_at": 1654720608597.7239, "error": {"message": "Something went wrong in the connector. See the logs for more details.", "internal_message": "'NoneType' object has no attribute 'message_types_by_name'", "stack_trace": "Traceback (most recent call last):\n  File \"/airbyte/integration_code/main.py\", line 9, in <module>\n    from source_google_ads import SourceGoogleAds\n  File \"/airbyte/integration_code/source_google_ads/__init__.py\", line 25, in <module>\n    from .source import SourceGoogleAds\n  File \"/airbyte/integration_code/source_google_ads/source.py\", line 13, in <module>\n    from google.ads.googleads.errors import GoogleAdsException\n  File \"/usr/local/lib/python3.9/site-packages/google/ads/googleads/__init__.py\", line 16, in <module>\n    import google.ads.googleads.client\n  File \"/usr/local/lib/python3.9/site-packages/google/ads/googleads/client.py\", line 20, in <module>\n    from google.api_core.gapic_v1.client_info import ClientInfo\n  File \"/usr/local/lib/python3.9/site-packages/google/api_core/gapic_v1/__init__.py\", line 16, in <module>\n    from google.api_core.gapic_v1 import config\n  File \"/usr/local/lib/python3.9/site-packages/google/api_core/gapic_v1/config.py\", line 25, in <module>\n    from google.api_core import exceptions\n  File \"/usr/local/lib/python3.9/site-packages/google/api_core/exceptions.py\", line 29, in <module>\n    from google.rpc import error_details_pb2\n  File \"/usr/local/lib/python3.9/site-packages/google/rpc/error_details_pb2.py\", line 39, in <module>\n    _RETRYINFO = DESCRIPTOR.message_types_by_name[\"RetryInfo\"]\nAttributeError: 'NoneType' object has no attribute 'message_types_by_name'\n", "failure_type": "system_error"}}}

I'll ask others to see if this is perhaps os-dependent. I'm on an M1 Mac...

pedroslopez · 2022-06-08T20:41:16Z

I can confirm, I am also getting the same error as @evantahler:

docker pull airbyte/source-google-ads && docker run airbyte/source-google-ads spec
Using default tag: latest
latest: Pulling from airbyte/source-google-ads
dc1f00a5d701: Already exists 
a2bfcfb5c8a7: Already exists 
c7690aa477f8: Already exists 
4f8939f4afff: Already exists 
96ee4e8d6b7f: Pull complete 
8a092cce2ab8: Pull complete 
e94a7c67d26e: Pull complete 
d7a29d8d1c06: Pull complete 
bc8c0e24df6c: Pull complete 
ee10dd37ea7a: Pull complete 
41f23b67842a: Pull complete 
Digest: sha256:1a55590f2ca6d9e0543a6bf3725378b994d5be6249d6e1a23fe158264f6733c5
Status: Downloaded newer image for airbyte/source-google-ads:latest
docker.io/airbyte/source-google-ads:latest
{"type": "LOG", "log": {"level": "FATAL", "message": "'NoneType' object has no attribute 'message_types_by_name'\nTraceback (most recent call last):\n  File \"/airbyte/integration_code/main.py\", line 9, in <module>\n    from source_google_ads import SourceGoogleAds\n  File \"/airbyte/integration_code/source_google_ads/__init__.py\", line 25, in <module>\n    from .source import SourceGoogleAds\n  File \"/airbyte/integration_code/source_google_ads/source.py\", line 13, in <module>\n    from google.ads.googleads.errors import GoogleAdsException\n  File \"/usr/local/lib/python3.9/site-packages/google/ads/googleads/__init__.py\", line 16, in <module>\n    import google.ads.googleads.client\n  File \"/usr/local/lib/python3.9/site-packages/google/ads/googleads/client.py\", line 20, in <module>\n    from google.api_core.gapic_v1.client_info import ClientInfo\n  File \"/usr/local/lib/python3.9/site-packages/google/api_core/gapic_v1/__init__.py\", line 16, in <module>\n    from google.api_core.gapic_v1 import config\n  File \"/usr/local/lib/python3.9/site-packages/google/api_core/gapic_v1/config.py\", line 25, in <module>\n    from google.api_core import exceptions\n  File \"/usr/local/lib/python3.9/site-packages/google/api_core/exceptions.py\", line 29, in <module>\n    from google.rpc import error_details_pb2\n  File \"/usr/local/lib/python3.9/site-packages/google/rpc/error_details_pb2.py\", line 39, in <module>\n    _RETRYINFO = DESCRIPTOR.message_types_by_name[\"RetryInfo\"]\nAttributeError: 'NoneType' object has no attribute 'message_types_by_name'"}}
{"type": "TRACE", "trace": {"type": "ERROR", "emitted_at": 1654720814469.962, "error": {"message": "Something went wrong in the connector. See the logs for more details.", "internal_message": "'NoneType' object has no attribute 'message_types_by_name'", "stack_trace": "Traceback (most recent call last):\n  File \"/airbyte/integration_code/main.py\", line 9, in <module>\n    from source_google_ads import SourceGoogleAds\n  File \"/airbyte/integration_code/source_google_ads/__init__.py\", line 25, in <module>\n    from .source import SourceGoogleAds\n  File \"/airbyte/integration_code/source_google_ads/source.py\", line 13, in <module>\n    from google.ads.googleads.errors import GoogleAdsException\n  File \"/usr/local/lib/python3.9/site-packages/google/ads/googleads/__init__.py\", line 16, in <module>\n    import google.ads.googleads.client\n  File \"/usr/local/lib/python3.9/site-packages/google/ads/googleads/client.py\", line 20, in <module>\n    from google.api_core.gapic_v1.client_info import ClientInfo\n  File \"/usr/local/lib/python3.9/site-packages/google/api_core/gapic_v1/__init__.py\", line 16, in <module>\n    from google.api_core.gapic_v1 import config\n  File \"/usr/local/lib/python3.9/site-packages/google/api_core/gapic_v1/config.py\", line 25, in <module>\n    from google.api_core import exceptions\n  File \"/usr/local/lib/python3.9/site-packages/google/api_core/exceptions.py\", line 29, in <module>\n    from google.rpc import error_details_pb2\n  File \"/usr/local/lib/python3.9/site-packages/google/rpc/error_details_pb2.py\", line 39, in <module>\n    _RETRYINFO = DESCRIPTOR.message_types_by_name[\"RetryInfo\"]\nAttributeError: 'NoneType' object has no attribute 'message_types_by_name'\n", "failure_type": "system_error"}}}

(also on an M1 Mac)

evantahler · 2022-06-08T20:45:46Z

And on an Ubuntu box, things work OK!

root@ubuntu-s-1vcpu-2gb-amd-sfo3-01:~# uname -a
Linux ubuntu-s-1vcpu-2gb-amd-sfo3-01 5.4.0-107-generic #121-Ubuntu SMP Thu Mar 24 16:04:27 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
root@ubuntu-s-1vcpu-2gb-amd-sfo3-01:~# docker --version
Docker version 20.10.12, build 20.10.12-0ubuntu2~20.04.1
root@ubuntu-s-1vcpu-2gb-amd-sfo3-01:~# docker pull airbyte/source-google-ads && docker run airbyte/source-google-ads spec
Using default tag: latest
latest: Pulling from airbyte/source-google-ads
42c077c10790: Pull complete
f63e77b7563a: Pull complete
bd5fdadae976: Pull complete
205491f53016: Pull complete
2030d9f0f012: Pull complete
f050921cccf1: Pull complete
307b6cbb80d2: Pull complete
bc89ee555c9c: Pull complete
a25e412ae77a: Pull complete
aa7c31888cd5: Pull complete
ba0404f4e68b: Pull complete
Digest: sha256:1a55590f2ca6d9e0543a6bf3725378b994d5be6249d6e1a23fe158264f6733c5
Status: Downloaded newer image for airbyte/source-google-ads:latest
docker.io/airbyte/source-google-ads:latest
{"type": "SPEC", "spec": {"documentationUrl": "https://docs.airbyte.com/integrations/sources/google-ads", "connectionSpecification": {"$schema": "http://json-schema.org/draft-07/schema#", "title": "Google Ads Spec", "type": "object", "required": ["credentials", "start_date", "customer_id"], "additionalProperties": true, "properties": {"credentials": {"type": "object", "description": "", "title": "Google Credentials", "order": 0, "required": ["developer_token", "client_id", "client_secret", "refresh_token"], "properties": {"developer_token": {"type": "string", "title": "Developer Token", "order": 0, "description": "Developer token granted by Google to use their APIs. More instruction on how to find this value in our <a href=\"https://docs.airbyte.com/integrations/sources/google-ads#setup-guide\">docs</a>", "airbyte_secret": true}, "client_id": {"type": "string", "title": "Client ID", "order": 1, "description": "The Client ID of your Google Ads developer application. More instruction on how to find this value in our <a href=\"https://docs.airbyte.com/integrations/sources/google-ads#setup-guide\">docs</a>"}, "client_secret": {"type": "string", "title": "Client Secret", "order": 2, "description": "The Client Secret of your Google Ads developer application. More instruction on how to find this value in our <a href=\"https://docs.airbyte.com/integrations/sources/google-ads#setup-guide\">docs</a>", "airbyte_secret": true}, "refresh_token": {"type": "string", "title": "Refresh Token", "order": 3, "description": "The token for obtaining a new access token. More instruction on how to find this value in our <a href=\"https://docs.airbyte.com/integrations/sources/google-ads#setup-guide\">docs</a>", "airbyte_secret": true}, "access_token": {"type": "string", "title": "Access Token (Optional)", "order": 4, "description": "Access Token for making authenticated requests. More instruction on how to find this value in our <a href=\"https://docs.airbyte.com/integrations/sources/google-ads#setup-guide\">docs</a>", "airbyte_secret": true}}}, "customer_id": {"title": "Customer ID(s)", "type": "string", "description": "Comma separated list of (client) customer IDs. Each customer ID must be specified as a 10-digit number without dashes. More instruction on how to find this value in our <a href=\"https://docs.airbyte.com/integrations/sources/google-ads#setup-guide\">docs</a>. Metrics streams like AdGroupAdReport cannot be requested for a manager account.", "pattern": "^[0-9]{10}(,[0-9]{10})*$", "examples": ["6783948572,5839201945"], "order": 1}, "start_date": {"type": "string", "title": "Start Date", "description": "UTC date and time in the format 2017-01-25. Any data before this date will not be replicated.", "pattern": "^[0-9]{4}-[0-9]{2}-[0-9]{2}$", "examples": ["2017-01-25"], "order": 2}, "end_date": {"type": "string", "title": "End Date (Optional)", "description": "UTC date and time in the format 2017-01-25. Any data after this date will not be replicated.", "pattern": "^[0-9]{4}-[0-9]{2}-[0-9]{2}$", "examples": ["2017-01-30"], "order": 6}, "custom_queries": {"type": "array", "title": "Custom GAQL Queries (Optional)", "description": "", "order": 3, "items": {"type": "object", "properties": {"query": {"type": "string", "title": "Custom Query", "description": "A custom defined GAQL query for building the report. Should not contain segments.date expression because it is used by incremental streams. See Google's <a href=\"https://developers.google.com/google-ads/api/fields/v9/overview_query_builder\">query builder</a> for more information.", "examples": ["SELECT segments.ad_destination_type, campaign.advertising_channel_sub_type FROM campaign WHERE campaign.status = 'PAUSED'"]}, "table_name": {"type": "string", "title": "Destination Table Name", "description": "The table name in your destination database for choosen query."}}}}, "login_customer_id": {"type": "string", "title": "Login Customer ID for Managed Accounts (Optional)", "description": "If your access to the customer account is through a manager account, this field is required and must be set to the customer ID of the manager account (10-digit number without dashes). More information about this field you can see <a href=\"https://developers.google.com/google-ads/api/docs/concepts/call-structure#cid\">here</a>", "pattern": "^([0-9]{10})?$", "examples": ["7349206847"], "order": 4}, "conversion_window_days": {"title": "Conversion Window (Optional)", "type": "integer", "description": "A conversion window is the period of time after an ad interaction (such as an ad click or video view) during which a conversion, such as a purchase, is recorded in Google Ads. For more information, see Google's <a href=\"https://support.google.com/google-ads/answer/3123169?hl=en\">documentation</a>.", "minimum": 0, "maximum": 1095, "default": 14, "examples": [14], "order": 5}}}, "authSpecification": {"auth_type": "oauth2.0", "oauth2Specification": {"rootObject": ["credentials"], "oauthFlowInitParameters": [["client_id"], ["client_secret"], ["developer_token"]], "oauthFlowOutputParameters": [["access_token"], ["refresh_token"]]}}}}

I guess the thing to do is bump the version and publish the connector again with both ARM and AMD builds. I'll do that via #13624

evantahler · 2022-06-08T20:58:28Z

Oops! The image is already published multi-arch (thanks @pedroslopez) - https://hub.docker.com/r/airbyte/source-google-ads/tags

However, ./gradlew :airbyte-integrations:connectors:source-google-ads:build fails with the following error, so I do think that something is wrong in our code.

> Task :airbyte-integrations:connectors:source-google-ads:_unitTestCoverage
[python] .venv/bin/python -m coverage run --data-file=unit_tests/.coverage.unitTest --rcfile=/Users/evan/workspace/airbyte/airbyte/pyproject.toml -m pytest -s unit_tests -c /Users/evan/workspace/airbyte/airbyte/pyproject.toml
         ImportError while loading conftest '/Users/evan/workspace/airbyte/airbyte/airbyte-integrations/connectors/source-google-ads/unit_tests/conftest.py'.
         unit_tests/conftest.py:6: in <module>
             from source_google_ads.models import Customer
         source_google_ads/__init__.py:25: in <module>
             from .source import SourceGoogleAds
         source_google_ads/source.py:13: in <module>
             from google.ads.googleads.errors import GoogleAdsException
         .venv/lib/python3.9/site-packages/google/ads/googleads/__init__.py:16: in <module>
             import google.ads.googleads.client
         .venv/lib/python3.9/site-packages/google/ads/googleads/client.py:20: in <module>
             from google.api_core.gapic_v1.client_info import ClientInfo
         .venv/lib/python3.9/site-packages/google/api_core/gapic_v1/__init__.py:16: in <module>
             from google.api_core.gapic_v1 import config
         .venv/lib/python3.9/site-packages/google/api_core/gapic_v1/config.py:25: in <module>
             from google.api_core import exceptions
         .venv/lib/python3.9/site-packages/google/api_core/exceptions.py:29: in <module>
             from google.rpc import error_details_pb2
         .venv/lib/python3.9/site-packages/google/rpc/error_details_pb2.py:39: in <module>
             _RETRYINFO = DESCRIPTOR.message_types_by_name["RetryInfo"]
         E   AttributeError: 'NoneType' object has no attribute 'message_types_by_name'

> Task :airbyte-integrations:connectors:source-google-ads:_unitTestCoverage FAILED

pedroslopez · 2022-06-08T21:02:25Z

Interestingly enough, running the image for amd64 does work fine (airbyte/source-google-ads is published for multiple architectures):

$ docker pull airbyte/source-google-ads --platform linux/amd64               
Using default tag: latest
latest: Pulling from airbyte/source-google-ads
42c077c10790: Pull complete 
f63e77b7563a: Pull complete 
bd5fdadae976: Pull complete 
205491f53016: Pull complete 
2030d9f0f012: Pull complete 
f050921cccf1: Pull complete 
307b6cbb80d2: Pull complete 
bc89ee555c9c: Pull complete 
a25e412ae77a: Pull complete 
aa7c31888cd5: Pull complete 
ba0404f4e68b: Pull complete 
Digest: sha256:1a55590f2ca6d9e0543a6bf3725378b994d5be6249d6e1a23fe158264f6733c5
Status: Downloaded newer image for airbyte/source-google-ads:latest
docker.io/airbyte/source-google-ads:latest

$ docker run airbyte/source-google-ads spec                   

WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
{"type": "SPEC", "spec": {"documentationUrl": "https://docs.airbyte.com/integrations/sources/google-ads", "connectionSpecification": {"$schema": "http://json-schema.org/draft-07/schema#", "title": "Google Ads Spec", "type": "object", "required": ["credentials", "start_date", "customer_id"], "additionalProperties": true, "properties": {"credentials": {"type": "object", "description": "", "title": "Google Credentials", "order": 0, "required": ["developer_token", "client_id", "client_secret", "refresh_token"], "properties": {"developer_token": {"type": "string", "title": "Developer Token", "order": 0, "description": "Developer token granted by Google to use their APIs. More instruction on how to find this value in our <a href=\"https://docs.airbyte.com/integrations/sources/google-ads#setup-guide\">docs</a>", "airbyte_secret": true}, "client_id": {"type": "string", "title": "Client ID", "order": 1, "description": "The Client ID of your Google Ads developer application. More instruction on how to find this value in our <a href=\"https://docs.airbyte.com/integrations/sources/google-ads#setup-guide\">docs</a>"}, "client_secret": {"type": "string", "title": "Client Secret", "order": 2, "description": "The Client Secret of your Google Ads developer application. More instruction on how to find this value in our <a href=\"https://docs.airbyte.com/integrations/sources/google-ads#setup-guide\">docs</a>", "airbyte_secret": true}, "refresh_token": {"type": "string", "title": "Refresh Token", "order": 3, "description": "The token for obtaining a new access token. More instruction on how to find this value in our <a href=\"https://docs.airbyte.com/integrations/sources/google-ads#setup-guide\">docs</a>", "airbyte_secret": true}, "access_token": {"type": "string", "title": "Access Token (Optional)", "order": 4, "description": "Access Token for making authenticated requests. More instruction on how to find this value in our <a href=\"https://docs.airbyte.com/integrations/sources/google-ads#setup-guide\">docs</a>", "airbyte_secret": true}}}, "customer_id": {"title": "Customer ID(s)", "type": "string", "description": "Comma separated list of (client) customer IDs. Each customer ID must be specified as a 10-digit number without dashes. More instruction on how to find this value in our <a href=\"https://docs.airbyte.com/integrations/sources/google-ads#setup-guide\">docs</a>. Metrics streams like AdGroupAdReport cannot be requested for a manager account.", "pattern": "^[0-9]{10}(,[0-9]{10})*$", "examples": ["6783948572,5839201945"], "order": 1}, "start_date": {"type": "string", "title": "Start Date", "description": "UTC date and time in the format 2017-01-25. Any data before this date will not be replicated.", "pattern": "^[0-9]{4}-[0-9]{2}-[0-9]{2}$", "examples": ["2017-01-25"], "order": 2}, "end_date": {"type": "string", "title": "End Date (Optional)", "description": "UTC date and time in the format 2017-01-25. Any data after this date will not be replicated.", "pattern": "^[0-9]{4}-[0-9]{2}-[0-9]{2}$", "examples": ["2017-01-30"], "order": 6}, "custom_queries": {"type": "array", "title": "Custom GAQL Queries (Optional)", "description": "", "order": 3, "items": {"type": "object", "properties": {"query": {"type": "string", "title": "Custom Query", "description": "A custom defined GAQL query for building the report. Should not contain segments.date expression because it is used by incremental streams. See Google's <a href=\"https://developers.google.com/google-ads/api/fields/v9/overview_query_builder\">query builder</a> for more information.", "examples": ["SELECT segments.ad_destination_type, campaign.advertising_channel_sub_type FROM campaign WHERE campaign.status = 'PAUSED'"]}, "table_name": {"type": "string", "title": "Destination Table Name", "description": "The table name in your destination database for choosen query."}}}}, "login_customer_id": {"type": "string", "title": "Login Customer ID for Managed Accounts (Optional)", "description": "If your access to the customer account is through a manager account, this field is required and must be set to the customer ID of the manager account (10-digit number without dashes). More information about this field you can see <a href=\"https://developers.google.com/google-ads/api/docs/concepts/call-structure#cid\">here</a>", "pattern": "^([0-9]{10})?$", "examples": ["7349206847"], "order": 4}, "conversion_window_days": {"title": "Conversion Window (Optional)", "type": "integer", "description": "A conversion window is the period of time after an ad interaction (such as an ad click or video view) during which a conversion, such as a purchase, is recorded in Google Ads. For more information, see Google's <a href=\"https://support.google.com/google-ads/answer/3123169?hl=en\">documentation</a>.", "minimum": 0, "maximum": 1095, "default": 14, "examples": [14], "order": 5}}}, "authSpecification": {"auth_type": "oauth2.0", "oauth2Specification": {"rootObject": ["credentials"], "oauthFlowInitParameters": [["client_id"], ["client_secret"], ["developer_token"]], "oauthFlowOutputParameters": [["access_token"], ["refresh_token"]]}}}}

So there seems to be something different between the builds

evantahler · 2022-06-08T21:16:21Z

Ok! #13624 builds and works on M1 now - thanks for the protobuf hint @davydov-d!

davydov-d · 2022-06-09T09:01:18Z

it seems like pinning that package degraded performance..I'm now trying to figure out how big is the time gap. if it's not critical - I will increase the timeout

davydov-d · 2022-06-09T09:39:51Z

Unbelievable, reading time increased from 150-200 sec on my local machine to more than 600 sec with the same config. I did not even wait after 600 seconds for the connector to finish reading. And one more thing:

I guess I'll continue the investigation

davydov-d · 2022-06-09T10:10:54Z

@evantahler could you please try version 3.20.0 or 3.20.1? both were released after the google-ads 14.1.0 we use, and it seems they are quite compatible.

upd: 3.20.0 works fine on both Mac M1 and Ubuntu, but conflicts with google-ads dependency. I'll try to bump google-ads to 15.1.1, that should do the trick

davydov-d · 2022-06-09T16:56:00Z

Done! Validated both for Ubuntu and Mac M1 - works. @evantahler pls have a look at the PR, I will publish new version as soon as you approve it

evantahler added type/bug Something isn't working needs-triage labels Jun 7, 2022

octavia-squidington-iii added the team/triage label Jun 7, 2022

evantahler changed the title ~~airbyte/source-google-ads connector's CHECK command does not succeed~~ airbyte/source-google-ads connector's SPEC command does not succeed Jun 7, 2022

evantahler added the team/extensibility label Jun 8, 2022

octavia-squidington-iii removed the team/triage label Jun 8, 2022

evantahler mentioned this issue Jun 8, 2022

Fix source-google-ads on M1 Macs by pinning protobuf==3.14 #13624

Merged

davydov-d added the team/connectors-python label Jun 9, 2022

davydov-d self-assigned this Jun 9, 2022

evantahler closed this as completed in #13624 Jun 9, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`airbyte/source-google-ads` connector's SPEC command does not succeed #13580

`airbyte/source-google-ads` connector's SPEC command does not succeed #13580

evantahler commented Jun 7, 2022 •

edited

Loading

davydov-d commented Jun 8, 2022

evantahler commented Jun 8, 2022 •

edited

Loading

pedroslopez commented Jun 8, 2022 •

edited

Loading

evantahler commented Jun 8, 2022 •

edited

Loading

evantahler commented Jun 8, 2022 •

edited

Loading

pedroslopez commented Jun 8, 2022 •

edited

Loading

evantahler commented Jun 8, 2022

davydov-d commented Jun 9, 2022

davydov-d commented Jun 9, 2022

davydov-d commented Jun 9, 2022 •

edited

Loading

davydov-d commented Jun 9, 2022

airbyte/source-google-ads connector's SPEC command does not succeed #13580

airbyte/source-google-ads connector's SPEC command does not succeed #13580

Comments

evantahler commented Jun 7, 2022 • edited Loading

davydov-d commented Jun 8, 2022

evantahler commented Jun 8, 2022 • edited Loading

pedroslopez commented Jun 8, 2022 • edited Loading

evantahler commented Jun 8, 2022 • edited Loading

evantahler commented Jun 8, 2022 • edited Loading

pedroslopez commented Jun 8, 2022 • edited Loading

evantahler commented Jun 8, 2022

davydov-d commented Jun 9, 2022

davydov-d commented Jun 9, 2022

davydov-d commented Jun 9, 2022 • edited Loading

davydov-d commented Jun 9, 2022

`airbyte/source-google-ads` connector's SPEC command does not succeed #13580

`airbyte/source-google-ads` connector's SPEC command does not succeed #13580

evantahler commented Jun 7, 2022 •

edited

Loading

evantahler commented Jun 8, 2022 •

edited

Loading

pedroslopez commented Jun 8, 2022 •

edited

Loading

evantahler commented Jun 8, 2022 •

edited

Loading

evantahler commented Jun 8, 2022 •

edited

Loading

pedroslopez commented Jun 8, 2022 •

edited

Loading

davydov-d commented Jun 9, 2022 •

edited

Loading