🐛 Fix hubspot datetime empty string #5798

avida · 2021-09-02T08:00:10Z

What

Fix for #5714

How

Describe the solution

Pre-merge Checklist

Expand the relevant checklist and delete the others.

New Connector

Community member or Airbyter

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

Create a non-forked branch based on this PR and test the below items on it
Build is successful
Credentials added to Github CI. Instructions.
/test connector=connectors/<name> command is passing.
New Connector version released on Dockerhub by running the /publish command described here

Updating a connector

Community member or Airbyter

Grant edit access to maintainers (instructions)
Secrets in the connector's spec are annotated with airbyte_secret
Unit & integration tests added and passing. Community members, please provide proof of success locally e.g: screenshot or copy-paste unit, integration, and acceptance test output. To run acceptance tests for a Python connector, follow instructions in the README. For java connectors run ./gradlew :airbyte-integrations:connectors:<name>:integrationTest.
Code reviews completed
Documentation updated
- Connector's README.md
- Changelog updated in docs/integrations/<source or destination>/<name>.md including changelog. See changelog example
PR name follows PR naming conventions
Connector version bumped like described here

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

Create a non-forked branch based on this PR and test the below items on it
Build is successful
Credentials added to Github CI. Instructions.
/test connector=connectors/<name> command is passing.
New Connector version released on Dockerhub by running the /publish command described here

Connector Generator

Issue acceptance criteria met
PR name follows PR naming conventions
If adding a new generator, add it to the list of scaffold modules being tested
The generator test modules (all connectors with -scaffold in their name) have been updated with the latest scaffold by running ./gradlew :airbyte-integrations:connector-templates:generator:testScaffoldTemplates then checking in your changes
Documentation which references the generator is updated as needed.

avida · 2021-09-02T08:07:56Z

/test connector=connectors/source-hubspot

🕑 connectors/source-hubspot https://github.com/airbytehq/airbyte/actions/runs/1193363183
❌ connectors/source-hubspot https://github.com/airbytehq/airbyte/actions/runs/1193363183

avida · 2021-09-02T08:41:46Z

/test connector=connectors/source-hubspot

🕑 connectors/source-hubspot https://github.com/airbytehq/airbyte/actions/runs/1193468820
✅ connectors/source-hubspot https://github.com/airbytehq/airbyte/actions/runs/1193468820

yevhenii-ldv · 2021-09-02T14:21:51Z

buildSrc/src/main/groovy/airbyte-source-acceptance-test.gradle

@@ -19,7 +19,7 @@ class AirbyteSourceAcceptanceTestPlugin implements Plugin<Project> {
                        '-w', "$targetMountDirectory",
                        '-e', "AIRBYTE_SAT_CONNECTOR_DIR=${project.projectDir.absolutePath}",
                        'airbyte/source-acceptance-test',
-                        '-p', 'integration_tests.acceptance',
+                        '-p', 'source_acceptance_test.plugin',


I don’t understand a little, a change in this line leads to the fact that we can delete the acceptance.py and integration_test.py files, but still use the SAT tests?

yes, you don't need acceptance.py, all test cases will be picked up by plugin according to acceptance test config

yevhenii-ldv · 2021-09-02T14:24:00Z

airbyte-integrations/bases/base-python/setup.py

@@ -37,7 +37,7 @@
        "PyYAML==5.4",
        "pydantic==1.6.*",
        "airbyte-protocol",
-        "jsonschema==2.6.0",
+        "jsonschema==3.2.0",


Are you sure that jsonschema version 3 is backward compatible with version 2 and this change will not generate new bugs?

not 100% sure but acceptance test passed fine. There were dependencies conflicts with SAT that failed a build so Ive made this change. AFAIK base-python is outdated and we should move away from it.

yevhenii-ldv

LGTM, please, resolve conflicts

sherifnada · 2021-09-03T15:15:56Z

airbyte-integrations/bases/source-acceptance-test/source_acceptance_test/utils/asserts.py

@@ -52,7 +52,7 @@ def check_datetime(value: str) -> bool:
        return valid_format and valid_time

    def check(self, instance, format):
-        if format == "date-time":
+        if instance is not None and format == "date-time":


make sure to publish SAT

avida · 2021-09-03T15:19:43Z

/publish connector=connectors/source-hubspot

🕑 connectors/source-hubspot https://github.com/airbytehq/airbyte/actions/runs/1198514756
❌ connectors/source-hubspot https://github.com/airbytehq/airbyte/actions/runs/1198514756

…spot-fix

avida · 2021-09-03T15:23:21Z

/publish connector=bases/source-acceptance-test

🕑 bases/source-acceptance-test https://github.com/airbytehq/airbyte/actions/runs/1198526770
❌ bases/source-acceptance-test https://github.com/airbytehq/airbyte/actions/runs/1198526770

avida · 2021-09-03T15:47:40Z

Cannot publish cause github runner' ubuntu-latest image have python with setuptools defect: pypa/setuptools-scm#608 Will try later when image will be updated (hopefully)

avida · 2021-09-06T06:45:11Z

/publish connector=connectors/source-hubspot

🕑 connectors/source-hubspot https://github.com/airbytehq/airbyte/actions/runs/1204970168
❌ connectors/source-hubspot https://github.com/airbytehq/airbyte/actions/runs/1204970168

avida · 2021-09-06T06:56:39Z

/publish connector=bases/source-acceptance-test

🕑 bases/source-acceptance-test https://github.com/airbytehq/airbyte/actions/runs/1204996942
✅ bases/source-acceptance-test https://github.com/airbytehq/airbyte/actions/runs/1204996942

avida · 2021-09-06T07:02:39Z

/publish connector=connectors/source-hubspot

🕑 connectors/source-hubspot https://github.com/airbytehq/airbyte/actions/runs/1205013037
✅ connectors/source-hubspot https://github.com/airbytehq/airbyte/actions/runs/1205013037

github-actions bot added the area/connectors Connector related issues label Sep 2, 2021

avida force-pushed the drezchykov/5714-hubspot-fix branch from a6fc442 to f31d001 Compare September 2, 2021 08:07

github-actions bot added the area/documentation Improvements or additions to documentation label Sep 2, 2021

jrhizor temporarily deployed to more-secrets September 2, 2021 08:10 Inactive

github-actions bot added the area/protocol label Sep 2, 2021

jrhizor temporarily deployed to more-secrets September 2, 2021 08:44 Inactive

github-actions bot added the CDK Connector Development Kit label Sep 2, 2021

avida force-pushed the drezchykov/5714-hubspot-fix branch from 256aa64 to 3175184 Compare September 2, 2021 11:05

avida requested review from htrueman, yevhenii-ldv and Zirochkaa September 2, 2021 11:05

avida force-pushed the drezchykov/5714-hubspot-fix branch from 3175184 to 22dffb2 Compare September 2, 2021 11:09

github-actions bot removed the CDK Connector Development Kit label Sep 2, 2021

yevhenii-ldv reviewed Sep 2, 2021

View reviewed changes

avida changed the title ~~Fix hubspot datetime empty string~~ 🐛 Fix hubspot datetime empty string Sep 2, 2021

yevhenii-ldv approved these changes Sep 3, 2021

View reviewed changes

avida force-pushed the drezchykov/5714-hubspot-fix branch from 22dffb2 to f8893ac Compare September 3, 2021 08:25

github-actions bot removed the area/protocol label Sep 3, 2021

avida requested a review from sherifnada September 3, 2021 08:26

avida linked an issue Sep 3, 2021 that may be closed by this pull request

Hubspot Connector 0.1.11 Fails on Some Tables due to outputting empty strings as dates #5714

Closed

Fix hubspot datetime empty string

973cf6d

sherifnada approved these changes Sep 3, 2021

View reviewed changes

avida force-pushed the drezchykov/5714-hubspot-fix branch from f8893ac to 973cf6d Compare September 3, 2021 15:16

Bump SAT version

e46cf33

Merge remote-tracking branch 'origin/master' into drezchykov/5714-hub…

4c3b768

…spot-fix

jrhizor temporarily deployed to more-secrets September 3, 2021 15:25 Inactive

jrhizor temporarily deployed to more-secrets September 6, 2021 06:47 Inactive

jrhizor temporarily deployed to more-secrets September 6, 2021 06:59 Inactive

jrhizor temporarily deployed to more-secrets September 6, 2021 07:05 Inactive

avida merged commit 60fd9d5 into master Sep 6, 2021

avida deleted the drezchykov/5714-hubspot-fix branch September 6, 2021 07:11

bazarnov mentioned this pull request Sep 6, 2021

🐛 SAT: fix for handling None/null values of format date-time #5855

Closed

10 tasks

jrhizor mentioned this pull request Sep 8, 2021

Bump Airbyte version from 0.29.15-alpha to 0.29.16-alpha #5925

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

🐛 Fix hubspot datetime empty string #5798

🐛 Fix hubspot datetime empty string #5798

avida commented Sep 2, 2021 •

edited

Loading

avida commented Sep 2, 2021 •

edited by github-actions bot

Loading

avida commented Sep 2, 2021 •

edited by github-actions bot

Loading

yevhenii-ldv Sep 2, 2021 •

edited

Loading

avida Sep 2, 2021

yevhenii-ldv Sep 2, 2021

avida Sep 2, 2021

yevhenii-ldv left a comment

sherifnada Sep 3, 2021

avida commented Sep 3, 2021 •

edited by github-actions bot

Loading

avida commented Sep 3, 2021 •

edited by github-actions bot

Loading

avida commented Sep 3, 2021

avida commented Sep 6, 2021 •

edited by github-actions bot

Loading

avida commented Sep 6, 2021 •

edited by github-actions bot

Loading

avida commented Sep 6, 2021 •

edited by github-actions bot

Loading

🐛 Fix hubspot datetime empty string #5798

🐛 Fix hubspot datetime empty string #5798

Conversation

avida commented Sep 2, 2021 • edited Loading

What

How

Recommended reading order

Pre-merge Checklist

Community member or Airbyter

Airbyter

Community member or Airbyter

Airbyter

avida commented Sep 2, 2021 • edited by github-actions bot Loading

avida commented Sep 2, 2021 • edited by github-actions bot Loading

yevhenii-ldv Sep 2, 2021 • edited Loading

Choose a reason for hiding this comment

avida Sep 2, 2021

Choose a reason for hiding this comment

yevhenii-ldv Sep 2, 2021

Choose a reason for hiding this comment

avida Sep 2, 2021

Choose a reason for hiding this comment

yevhenii-ldv left a comment

Choose a reason for hiding this comment

sherifnada Sep 3, 2021

Choose a reason for hiding this comment

avida commented Sep 3, 2021 • edited by github-actions bot Loading

avida commented Sep 3, 2021 • edited by github-actions bot Loading

avida commented Sep 3, 2021

avida commented Sep 6, 2021 • edited by github-actions bot Loading

avida commented Sep 6, 2021 • edited by github-actions bot Loading

avida commented Sep 6, 2021 • edited by github-actions bot Loading

avida commented Sep 2, 2021 •

edited

Loading

avida commented Sep 2, 2021 •

edited by github-actions bot

Loading

avida commented Sep 2, 2021 •

edited by github-actions bot

Loading

yevhenii-ldv Sep 2, 2021 •

edited

Loading

avida commented Sep 3, 2021 •

edited by github-actions bot

Loading

avida commented Sep 3, 2021 •

edited by github-actions bot

Loading

avida commented Sep 6, 2021 •

edited by github-actions bot

Loading

avida commented Sep 6, 2021 •

edited by github-actions bot

Loading

avida commented Sep 6, 2021 •

edited by github-actions bot

Loading