Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open HashMethod to all types #1171

Merged
merged 2 commits into from
Sep 23, 2022
Merged

Conversation

eapolinario
Copy link
Collaborator

@eapolinario eapolinario commented Sep 22, 2022

TL;DR

Ensure all types can have their hash values overriden

Type

  • Bug Fix
  • Feature
  • Plugin

Are all requirements met?

  • Code completed
  • Smoke tested
  • Unit tests added
  • Code documentation added
  • Any pending items have an associated Issue

Complete description

Initially we made it so that only pandas dataframes and StructuredDatasets could leverage the ability to override their hash values for the purpose of producing cache keys. This PR lifts that restriction and opens up that feature to all types.

Tracking Issue

flyteorg/flyte#2912

Follow-up issue

NA
OR
https://github.com/flyteorg/flyte/issues/

Signed-off-by: Eduardo Apolinario <[email protected]>
@codecov
Copy link

codecov bot commented Sep 22, 2022

Codecov Report

Merging #1171 (afc4eed) into master (2ccaed7) will increase coverage by 0.02%.
The diff coverage is 90.90%.

@@            Coverage Diff             @@
##           master    #1171      +/-   ##
==========================================
+ Coverage   68.51%   68.53%   +0.02%     
==========================================
  Files         288      288              
  Lines       26085    26191     +106     
  Branches     2918     2925       +7     
==========================================
+ Hits        17871    17951      +80     
- Misses       7735     7762      +27     
+ Partials      479      478       -1     
Impacted Files Coverage Δ
flytekit/types/schema/types_pandas.py 24.59% <ø> (-1.22%) ⬇️
flytekit/types/structured/structured_dataset.py 63.32% <ø> (-0.11%) ⬇️
flytekit/core/type_engine.py 59.38% <50.00%> (+0.04%) ⬆️
tests/flytekit/unit/core/test_local_cache.py 100.00% <100.00%> (+0.37%) ⬆️
tests/flytekit/unit/core/test_type_engine.py 98.39% <100.00%> (ø)
tests/flytekit/unit/core/test_type_hints.py 96.23% <100.00%> (ø)
plugins/setup.py 0.00% <0.00%> (ø)
flytekit/extras/sqlite3/task.py 90.74% <0.00%> (ø)
tests/flytekit/unit/models/test_execution.py 100.00% <0.00%> (ø)
tests/flytekit/unit/extras/sqlite3/test_task.py 100.00% <0.00%> (ø)
... and 5 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@eapolinario eapolinario merged commit 245c414 into master Sep 23, 2022
Ln11211 pushed a commit to Ln11211/flytekit that referenced this pull request Oct 2, 2022
* Open HashMethod to all types

* Fix test in test_type_hints.py

Signed-off-by: Eduardo Apolinario <[email protected]>

Signed-off-by: Eduardo Apolinario <[email protected]>
Co-authored-by: Eduardo Apolinario <[email protected]>
Signed-off-by: LN <[email protected]>
eapolinario added a commit that referenced this pull request Oct 7, 2022
…mote.sync (#1212)

* Open HashMethod to all types (#1171)

* Open HashMethod to all types

* Fix test in test_type_hints.py

Signed-off-by: Eduardo Apolinario <[email protected]>

Signed-off-by: Eduardo Apolinario <[email protected]>
Co-authored-by: Eduardo Apolinario <[email protected]>
Signed-off-by: LN <[email protected]>

* Return None for SyncCheckpoint.read() when src is empty (#1189)

Signed-off-by: Andrew Dye <[email protected]>

Signed-off-by: Andrew Dye <[email protected]>
Signed-off-by: LN <[email protected]>

* fix: plugins/flytekit-k8s-pod/requirements.txt to reduce vulnerabilities (#1192)

The following vulnerabilities are fixed by pinning transitive dependencies:
- https://snyk.io/vuln/SNYK-PYTHON-PROTOBUF-3031740

Signed-off-by: LN <[email protected]>

* pyflyte non-fast register (#1205)

* pyflyte run non-fast register

Signed-off-by: Kevin Su <[email protected]>

* lint

Signed-off-by: Kevin Su <[email protected]>

Signed-off-by: Kevin Su <[email protected]>
Signed-off-by: LN <[email protected]>

* Update control_plane.rst

Added about large response  while using Remote.Sync() and how to handle it in the doc

Signed-off-by: LN <[email protected]>

* Update docs/source/design/control_plane.rst

updating FlyteRemote sync message

Co-authored-by: Samhita Alla <[email protected]>

* Update control_plane.rst

Updated the fix for the error and also removed the repeated error message line.

Signed-off-by: LN <[email protected]>

Signed-off-by: Eduardo Apolinario <[email protected]>
Signed-off-by: LN <[email protected]>
Signed-off-by: Andrew Dye <[email protected]>
Signed-off-by: Kevin Su <[email protected]>
Signed-off-by: LN <[email protected]>
Co-authored-by: Eduardo Apolinario <[email protected]>
Co-authored-by: Eduardo Apolinario <[email protected]>
Co-authored-by: Andrew Dye <[email protected]>
Co-authored-by: Snyk bot <[email protected]>
Co-authored-by: Kevin Su <[email protected]>
Co-authored-by: Samhita Alla <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants