Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix python model test #406

Merged
merged 36 commits into from
Jul 26, 2022
Merged

Conversation

ChenyuLInx
Copy link
Contributor

No description provided.

McKnight-42 and others added 22 commits May 4, 2022 15:31
…#344)

* rename dev_requirements.txt -> dev-requirements.txt to match dbt-core

* add changelog entry
* Add commit hook tooling. (#346)
* Make pre commit code changes (#345)
* Refactor exception function using accurate type specs.
* Upgrade the mypy version in order to use TypeAlias-es.
* Upgrade the mypy version in the worfklow. Fix bug introduced by using str's instead of Optional[str]'s
* Address code review comments: Remove integration test command since there are multiple ways we can run tests in spark.
* Add changelog entry
* Altering names of dev_requirements references.
* More consistent results from get_columns_in_relation

* Not dispatched, full name

* Add changelog entry
* First draft of adding contributing.md to each adapter repo

* updates after kyle review, and minor changes regarding review process and CI as spark still uses CircleCI and not GHA

* minor addition

* add test.env.example

* fix eof black errors

* added example for functional tests
* Add invocation env to user agent string

* Consistency + fixups

* Changelog entry

* Try diff pattern
* Initialize lift + shift, dateadd + datediff

* Fixups

* More fixups

* Next round of utilities

* Reorgnanize, skip, max for bool_or

* fail -> skip_profile

* Rm branch names
* Bumping version to 1.2.0b1

* Remove whitespace

* Update CHANGELOG.md

Co-authored-by: Github Build Bot <[email protected]>
Co-authored-by: leahwicz <[email protected]>
* Testing Spark3 upgrade.WIP

* Skip tests.WIP

* update readme and setup for pyspark.WIP

* Fix circle ci version and bump mem value

* Bump memory, fix nit, bump pyhive version

* Pyhive version change

* enabled sasl for metastore

* Explicit server2 host port

* Try showing debug-level logs

* Rm -n4

* move to godatadriven lates spark image

* restore to 2 to check output

* Restore debug and parallelized to check output

* Revert to 3.0

* Revert to normal state

* open source spark image

* Change to pyspark image

* Testing with gdd spark 3.0 for thrift

* Switch back to dbt user pass

* Spark 3.1.1 gdd image without configs

* Clean up

* Skip session test

* Clean up for review

* Update to CHANGELOG

Co-authored-by: Jeremy Cohen <[email protected]>
* Disable TestDateDiff on apache_spark

* i dont really know what im doing
* Not dropping table for incremental full refresh with delta

* Updated changelog

* Simplified conditional logic according to suggestion

* Updated changelog

* Only drop table if not delta table

Co-authored-by: Jeremy Cohen <[email protected]>

* Update changelog, trigger CircleCI tests

Co-authored-by: Jeremy Cohen <[email protected]>
Co-authored-by: Jeremy Cohen <[email protected]>
* Run tests for data type macros. Fine tune numeric_type

* Hard code seed loading types for float + int

* Repoint, fixup, changelog entry
* Add apply_grants call to materialization macros

* add standardize_grants_dict

* Working grant macros

* Initialize tests in CI

* Refactor to account for core macro changes. Passing tests

* Fix code checks

* Try default__reset_csv_table

* Code checks

* Revert "Try default__reset_csv_table"

This reverts commit 8bd4145.

* Account for refactor in dbt-labs/dbt-core@c763601

* Account for test changes in dbt-labs/dbt-core@debc867

* add changelog

* Empty-Commit

* rerun ci

* rerun ci

* readd a persist_docs call to snapshot.sql

* fix whitespace

Co-authored-by: Jeremy Cohen <[email protected]>
Co-authored-by: Matthew McKnight <[email protected]>
* Bumping version to 1.3.0a1

* Update CHANGELOG.md

* Fix whitespace

Co-authored-by: Github Build Bot <[email protected]>
Co-authored-by: leahwicz <[email protected]>
* [CT-868] Pin pyodbc in dbt-spark 

Pin pyodbc to a known-working version until pyodbc>=4.0.35 releases.

* Update CHANGELOG.md

* Update CHANGELOG.md

* Update CHANGELOG.md
@cla-bot cla-bot bot added the cla:yes label Jul 22, 2022
@ChenyuLInx
Copy link
Contributor Author

All test passed, merging to feature branch for final review

@ChenyuLInx ChenyuLInx merged commit 9eea396 into feature/python-model-v1 Jul 26, 2022
@ChenyuLInx ChenyuLInx deleted the chenyu/fix_python_model_test branch July 26, 2022 04:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.