Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TER-94 export from TD TPT to ABS #165

Merged
merged 26 commits into from
Jan 27, 2023
Merged

TER-94 export from TD TPT to ABS #165

merged 26 commits into from
Jan 27, 2023

Conversation

martinjunger
Copy link
Contributor

@martinjunger martinjunger commented Jan 12, 2023

Jira: TER-94 (export), TER-98 (import)

Before asking for review make sure that:


  • load test files to ABS: composer loadABS
  • update .env:
    • fill ABS_* variables
    • change STORAGE_TYPE variable to ABS
    • ABS import a export:
      • run test \Tests\Keboola\Db\ImportExportFunctional\Teradata\ExportTest::testExportSimple
      • run test \Tests\Keboola\Db\ImportExportFunctional\Teradata\ToFinal\FullImportTest::testFullImportWithDataSet

@martinjunger martinjunger changed the title TER-94 abs export TER-94 export from TD TPT to ABS Jan 12, 2023
@jirkasemmler
Copy link
Contributor

na prvni pohled to vypada dobre, nezkoumal jsem to detailne, chtelo by to prvne videt zelene testy

@martinjunger martinjunger marked this pull request as ready for review January 16, 2023 16:59
@martinjunger martinjunger marked this pull request as draft January 16, 2023 16:59
Copy link
Contributor

@jirkasemmler jirkasemmler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cool, potrebujeme poresit ten manifest. jen se mi nezda ten helper, nebo jsem ho aspon nepochopil

src/Backend/Teradata/Helper/QuoteHelper.php Outdated Show resolved Hide resolved
src/Storage/ABS/SourceFile.php Show resolved Hide resolved
tests/functional/Teradata/ExportTest.php Outdated Show resolved Hide resolved
@martinjunger
Copy link
Contributor Author

martinjunger commented Jan 19, 2023

Tenhle test je zatim jako incomplete, protoze na ABS trva prilis dlouho a test timeoutuje po 2 hodinach.

...\ToFinal\FullImportTest::testFullImportWithDataSet with data set "large manifest"
resp.
...\Snowflake\FullImportTest::testFullImport with data set #0

Casy behu pro vybrane test suity:

suite time link
snowflake-s3 1m 00s / 1m 06s link
snoflake-abs 17m 33s / 18m 00s link
teradata-tpt-s3 1m 24s link
teradata-tpt-abs timeout link

@jirkasemmler jirkasemmler marked this pull request as ready for review January 25, 2023 08:43
@jirkasemmler
Copy link
Contributor

@martinjunger za me je to takto ok. snad ty testy tedka projdou. Prosim jen to nejak po-fixupuj, at pak neni v masteru takovej mrdnik commitu :)

@martinjunger
Copy link
Contributor Author

@martinjunger za me je to takto ok. snad ty testy tedka projdou. Prosim jen to nejak po-fixupuj, at pak neni v masteru takovej mrdnik commitu :)

Trochu jsem to squasnul.

@jirkasemmler jirkasemmler self-requested a review January 26, 2023 10:50
@martinjunger martinjunger merged commit 95fabbc into master Jan 27, 2023
@martinjunger martinjunger deleted the TER-94-abs-export branch January 27, 2023 15:10
romanbracinik pushed a commit that referenced this pull request Jul 23, 2024
CT-1642 Fix support for GEOMETRY and GEOGRAPHY for incremental load
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants