-
Notifications
You must be signed in to change notification settings - Fork 4.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[source-sentry] [source-sendgrid] [source-greenhouse] add .yaml to list of file imports in setup.py #15800
Conversation
@brianjlai how does this pass SAT but then fail on cloud? 🤔 |
That I'm not sure about. I haven't been able to replicate this locally when I package up a connecter on my own machine either. But it is also the only thing I can think of that could cause the file to not be read, since we copy the whole directory over. I wouldn't expect this to work in any capacity. Figuring out why can be part of a post-fix retro given that this affects all 3 connectors |
I think this is a pretty safe change to publish and test, given that there are no code changes, and a minimal config adjustment that we have configured for other connectors in prod |
/test connector=connectors/source-sentry
Build PassedTest summary info:
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
publishing this is fine w me but feels like a total shot in the dark. If the connector works locally then that means the yaml was packaged, no? is the problem an issue of relative paths? Does this connector work on a local instance of airbyte for you?
/test connector=connectors/source-sentry
Build PassedTest summary info:
|
/test connector=connectors/source-sendgrid
Build PassedTest summary info:
|
/test connector=connectors/source-greenhouse
Build FailedTest summary info:
|
/test connector=connectors/source-greenhouse
Build FailedTest summary info:
|
/publish connector=connectors/source-sentry
if you have connectors that successfully published but failed definition generation, follow step 4 here |
/test connector=connectors/source-greenhouse
Build FailedTest summary info:
|
…bytehq/airbyte into brian/fix_declarative_sentry
/test connector=connectors/source-greenhouse
Build FailedTest summary info:
|
/publish connector=connectors/source-sendgrid
if you have connectors that successfully published but failed definition generation, follow step 4 here |
/publish connector=connectors/source-greenhouse
if you have connectors that successfully published but failed definition generation, follow step 4 here |
…irbyte into brian/fix_declarative_sentry
{"stream":"demographics_questions","data":{"translations":[{"name":"Are you a veteran or active member of the United States Armed Forces? (select one)","language":"en"}],"required":false,"name":"Are you a veteran or active member of the United States Armed Forces? (select one)","id":4015594003,"demographic_question_set_id":4002702003,"answer_type":"multi_value_single_select","active":true},"emitted_at":1660156527389} | ||
{"stream":"demographics_questions","data":{"translations":[{"name":"Do you have a disability or chronic condition (physical, visual, auditory, cognitive, mental, emotional, or other) that substantially limits one or more of your major life activities, including mobility, communication (seeing, hearing, speaking), and learning? (select one)","language":"en"}],"required":false,"name":"Do you have a disability or chronic condition (physical, visual, auditory, cognitive, mental, emotional, or other) that substantially limits one or more of your major life activities, including mobility, communication (seeing, hearing, speaking), and learning? (select one)","id":4015596003,"demographic_question_set_id":4002702003,"answer_type":"multi_value_single_select","active":true},"emitted_at":1660156527389} | ||
{"stream":"demographics_questions","data":{"translations":[{"name":"Do you identify as transgender? (select one)","language":"en"}],"required":false,"name":"Do you identify as transgender? (select one)","id":4015598003,"demographic_question_set_id":4002702003,"answer_type":"multi_value_single_select","active":true},"emitted_at":1660156527389} | ||
{"stream":"demographics_questions","data":{"translations":[{"name":"Are you a veteran or active member of the United States Armed Forces?","language":"en"}],"required":false,"name":"Are you a veteran or active member of the United States Armed Forces?","id":4015594003,"demographic_question_set_id":4002702003,"answer_type":"multi_value_single_select","active":true},"emitted_at":1660156527389} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not sure what exactly happened here, but in the last week greenhouse might've changed their default question vernacular, but it no longer lines up with the expected records from last week. I doubled checked this against making the direct query via Postman and it does not include (select one)
in the response so I'm confident this was a change on their end or in our test data
@@ -6,7 +6,7 @@ | |||
from setuptools import find_packages, setup | |||
|
|||
MAIN_REQUIREMENTS = [ | |||
"airbyte-cdk~=0.1.76", | |||
"airbyte-cdk~=0.1.79", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
technically this doesn't exist yet, but I will merge #15814 and publish 0.1.79
first which contains the fix for retrieving schemas while running from a job
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it now exists as of 8/22/22
/test connector=connectors/source-greenhouse
Build FailedTest summary info:
|
/test connector=connectors/source-greenhouse
Build PassedTest summary info:
|
/publish connector=connectors/source-greenhouse
if you have connectors that successfully published but failed definition generation, follow step 4 here |
This latest release of greenhouse includes the fix for both the config and schemas read issues. Version And rather than publish two new images under this same PR that already published, I'm gonna open a new PR to fix the remaining schema read issues by bumping the latest CDK. I validated those changes on a local version of Airbyte. This PR should be all set to merge in. |
…st of file imports in setup.py (#15800) * add .yaml to list of file imports in setup.py * bump dockerfile version for sentry * update sentry changelog * correct sentry, greenhouse, and sendgrid sources to correctly read configs * add changelog * update Dockerfile versions * auto-bump connector version [ci skip] * auto-bump connector version [ci skip] * fix greenhouse SAT tests and update to next version of cdk w/ schema read fix * auto-bump connector version [ci skip] Co-authored-by: Octavia Squidington III <[email protected]>
What
Addresses https://github.com/airbytehq/oncall/issues/462
We're getting file not found errors on the container on Cloud when we try to run syncs with declarative connectors. This is likely because we don't specify that .yaml files in the working directory are usable by the program. I'm going to test this with one source first, and if it works, then I'll roll it out for the other two.
How
Adds
*.yaml
topackage_data
so that it can be used at runtime by the declarative source being executed