Code

Caching of offloaded objects#762

Code

Merged

eapolinario merged 41 commits intomasterflyteorg/flytekit:masterfrom offloaded-objects-cachingflyteorg/flytekit:offloaded-objects-cachingCopy head branch name to clipboardMar 2, 2022

+1,020-336

Commits

Commits on Dec 2, 2021

• Remove flyteidl from install_requires

  Show description for 06e960f
  eapolinario

  committed
  06e960f

• Expose hash in Literal

  Show description for c9047a8
  eapolinario

  committed
  c9047a8

• Set hash in TypeEngine

  Show description for 4d21957
  eapolinario

  committed
  4d21957

• Modify cache key calculation to take hash into account

  Show description for f9273db
  eapolinario

  committed
  f9273db

• Opt-in PandasDataFrameTransformer

  Show description for 53dff4d
  eapolinario

  committed
  53dff4d

• Add unit tests

  Show description for 5ae247c
  eapolinario

  committed
  5ae247c

Commits on Dec 3, 2021

• Iterate using a flyteidl branch

  Show description for 9305746
  eapolinario

  committed
  9305746

• Merge remote-tracking branch 'origin' into offloaded-objects-caching

  Show description for faff038
  eapolinario

  committed
  faff038

• Regenerate requirements files

  Show description for ecd8b93
  eapolinario

  committed
  ecd8b93

• Regenerate requirements files

  Show description for 44b72f6
  eapolinario

  committed
  44b72f6

Commits on Jan 19, 2022

• Merge remote-tracking branch 'origin' into offloaded-objects-caching

  Show description for f46dc74
  eapolinario

  committed
  f46dc74

Commits on Jan 21, 2022

• Move _hash_overridable to StructureDatasetTransformerEngine

  Show description for 22c90d3
  eapolinario

  committed
  22c90d3

Commits on Jan 25, 2022

• Merge remote-tracking branch 'origin' into offloaded-objects-caching

  Show description for fe7e8f7
  eapolinario

  committed
  fe7e8f7

Commits on Jan 26, 2022

• Move HashMethod to flytekit.core.hash

  Show description for 6ac4b44
  eapolinario

  committed
  6ac4b44

Commits on Feb 17, 2022

• Merge remote-tracking branch 'origin' into offloaded-objects-caching

  Show description for 1702961
  eapolinario

  committed
  1702961

• Fix unit_test make target

  Show description for 4552e78
  eapolinario

  committed
  4552e78

• Merge remote-tracking branch 'origin' into offloaded-objects-caching

  Show description for c9fc044
  eapolinario

  committed
  c9fc044

• Split unit_test make target in two lines

  Show description for 24f67a0
  eapolinario

  committed
  24f67a0

• Add assert to structured dataset compatibility test

  Show description for 60ecf3a
  eapolinario

  committed
  60ecf3a

• Remove TODO

  Show description for 3aeedeb
  eapolinario

  committed
  3aeedeb

• Regenerate plugins requirements files pointing to the right version of flyteidl.

  Show description for c2dbb54
  eapolinario

  committed
  c2dbb54

• Set hash as a property of the literal

  Show description for 0e58199
  eapolinario

  committed
  0e58199

• Install plugins requirements in CI.

  Show description for 5054861
  eapolinario

  committed
  5054861

Commits on Feb 18, 2022

• Add hash.setter

  Show description for 9f2d06f
  eapolinario

  committed
  9f2d06f

• Install flyteidl directly

  Show description for e039836
  eapolinario

  committed
  e039836

• Revert "Regenerate plugins requirements files pointing to the right version of flyteidl."

  Show description for 2da76f3
  eapolinario

  committed
  2da76f3

• wip - Add support for univariate lists

  Show description for adaa448
  eapolinario

  committed
  adaa448

• Add support for lists of annotated objects

  Show description for 4b5f608
  eapolinario

  committed
  4b5f608

Commits on Feb 23, 2022

• Revamp generation of cache key (to cover case of literals collections and maps)

  Show description for ecffa04
  eapolinario

  committed
  ecffa04

• Leave TODO for warning

  Show description for d4b0b49
  eapolinario

  committed
  d4b0b49

• Revert "Add support for lists of annotated objects"

  Show description for 4d54c59
  eapolinario

  committed
  4d54c59

• Revert "wip - Add support for univariate lists"

  Show description for 82bbc1f
  eapolinario

  committed
  82bbc1f

Commits on Mar 1, 2022

• Remove docstring

  Show description for a21631b
  eapolinario

  committed
  a21631b

• Merge remote-tracking branch 'origin' into offloaded-objects-caching

  Show description for a432b1e
  eapolinario

  committed
  a432b1e

Commits on Mar 2, 2022

• Add flyteidl>=0.23.0

  Show description for 3519149
  eapolinario

  committed
  3519149

• Remove mentions to branch flyteidl@add-hash-to-literal

  Show description for 09e736d
  eapolinario

  committed
  09e736d

• Bump flyteidl in plugins requirements

  Show description for 9ec103c
  eapolinario

  committed
  9ec103c

• Regenerate plugins requirements again

  Show description for fa4e3b2
  eapolinario

  committed
  fa4e3b2

• Restore papermill/requirements.txt

  Show description for 373b3bb
  eapolinario

  committed
  373b3bb

• Point flytekitplugins-spark to the offloaded-objects-caching branch in papermill tests

  Show description for 403830d
  eapolinario

  committed
  403830d

• Set flyteidl>=0.23.0 in papermill dev-requirements

  eapolinario

  committed
  9754a0f