Releases: neptune-ai/kedro-neptune
0.6.0
0.5.0
Features
Better support for distributed, async workflows
The kedro-neptune
plugin now respects the NEPTUNE_CUSTOM_RUN_ID
environmental variable.
Previously, the plugin could not be used in a distributed asynchronous workflow (such as Kubeflow, AWS Batch, etc.), where multiple containers run independently as each container would create a new run in Neptune to log into during the after_catalog_created()
execution in NeptuneHooks
.
With this change, you can set NEPTUNE_CUSTOM_RUN_ID
to the same value for each container before the kedro run execution and Neptune will only create one run for each Custom Run ID, letting multiple nodes executed asynchronously to log into the same Neptune run.
More about the Custom Run ID in the docs.
Enhanced pipeline execution status tracking
By @SiddhantSadangi in #82
We have added new fields to better track pipeline execution status in realtime:
-
The
log
field shows which nodes have run and are currently running
-
The
execution_order
field exports a description of the pipeline (pipeline.describe()
), showing the order of execution of the nodes, and expected free input and output variables
-
Finally, each node now has a
status
field that shows if the node is "running" or "done".
Fixes
- Fixed the case when node input parameters were not logged to Neptune if passed as a list (by @SiddhantSadangi in #82)
New Contributors
Full Changelog: 0.4.0...0.5.0
0.4.0
What's Changed
Features
- Added support for
dependencies
parameter by @SiddhantSadangi in #80
Fixes
- Replaced
*DataSet
anddata_set
with*Dataset
anddataset
respectively by @SiddhantSadangi in #79
Full Changelog: 0.3.0...0.4.0
0.3.0
What's Changed
Features
- Added
OmegaConfig
support by @MarkusSagen and @SiddhantSadangi (#73)
Fixes
- Replaced
AbstractDataSet
,TextDataSet
andCSVDataSet
withAbstractDataset
,TextDataset
andCSVDataSet
respectively by @SiddhantSadangi (#73) - Replaced
kedro.extras.datasets
withkedro_datasets
by @SiddhantSadangi (#73)
Changes
- Added
kedro_datasets
to requirements, and bumpedkedro
to>=0.18.5
by @SiddhantSadangi (#73)
Full Changelog: 0.2.0...0.3.0
0.2.0
0.1.6
0.1.5
Fixes
- Fixed some of latest
neptune-client
warning messages (#58) - Fixed failing run on latest MacOS 12 (#58)
Changes
- Removed
neptune
andneptune-client
from base requirements for backward compatibility (#62) - Simplified example project and removed unrelated files (#58)
- Better life-time of the internally created Run in
NeptuneRunDataSet
(#58) - Updated the integration for compatibility with
neptune-client
1.0.0
. (#59)