Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add migration steps for ConfigLoader to OmegaConfigLoader #2887

Merged
merged 14 commits into from
Aug 8, 2023
Merged
65 changes: 65 additions & 0 deletions docs/source/configuration/config_loader_migration.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
# Migration guide for config loaders
The `ConfigLoader` and `TemplatedConfigLoader` classes have been deprecated since Kedro `0.18.12` and will be removed in Kedro `0.19.0`. To ensure a smooth transition, we strongly recommend you adopt the [`OmegaConfigLoader`](/kedro.config.OmegaConfigLoader) as soon as possible.
This migration guide outlines the primary distinctions between the old loaders and the `OmegaConfigLoader`, providing step-by-step instructions on updating your code base to utilise the new class effectively.

merelcht marked this conversation as resolved.
Show resolved Hide resolved
## [`ConfigLoader`](/kedro.config.ConfigLoader) to [`OmegaConfigLoader`](/kedro.config.OmegaConfigLoader)

### 1. Install the Required Library
The [`OmegaConfigLoader`](advanced_configuration.md#omegaconfigloader) was introduced in Kedro `0.18.5` and is based on [OmegaConf](https://omegaconf.readthedocs.io/). In order to use it you need to ensure you have both a version of Kedro of `0.18.5` or above and `omegaconf` installed.
You can install both using `pip`:

```bash
pip install kedro==0.18.5
```
This would be the minimum required Kedro version which includes `omegaconf` as dependency.
merelcht marked this conversation as resolved.
Show resolved Hide resolved
Or you can run:
```bash
pip install -U kedro
```

This command installs the most recent version of Kedro which also includes `omegaconf` as dependency.
merelcht marked this conversation as resolved.
Show resolved Hide resolved

### 2. Use the `OmegaConfigLoader`
To use `OmegaConfigLoader` in your project, set the `CONFIG_LOADER_CLASS` constant in your [`src/<package_name>/settings.py`](../kedro_project_setup/settings.md):

```python
from kedro.config import OmegaConfigLoader # new import

CONFIG_LOADER_CLASS = OmegaConfigLoader
merelcht marked this conversation as resolved.
Show resolved Hide resolved
```

### 3. Import Statements
Replace the import statement for `ConfigLoader` with the one for `OmegaConfigLoader`:

```python
# Before:
from kedro.config import ConfigLoader

# After:
from kedro.config import OmegaConfigLoader
merelcht marked this conversation as resolved.
Show resolved Hide resolved
```

### 4. File Format Support
`OmegaConfigLoader` supports only `yaml` and `json` file formats. Make sure that all your configuration files are in one of these formats. If you were using other formats with `ConfigLoader`, convert them to `yaml` or `json`.
merelcht marked this conversation as resolved.
Show resolved Hide resolved

### 5. Load Configuration
The method to load the configuration using `OmegaConfigLoader` differs slightly from that used by `ConfigLoader`, which allowed users to access configuration through the `.get()` method and required patterns as argument.
When you migrate to use `OmegaConfigLoader`it requires you to fetch configuration through a configuration key that points to [configuration patterns specified in the loader class](configuration_basics.md#configuration-patterns) or [provided in the `CONFIG_LOADER_ARGS`](advanced_configuration.md#how-to-change-which-configuration-files-are-loaded) in `settings.py`.
merelcht marked this conversation as resolved.
Show resolved Hide resolved

```python
# Before:
conf_path = str(project_path / settings.CONF_SOURCE)
merelcht marked this conversation as resolved.
Show resolved Hide resolved
conf_loader = ConfigLoader(conf_source=conf_path, env="local")
catalog = conf_loader.get("catalog*")

# After:
conf_path = str(project_path / settings.CONF_SOURCE)
config_loader = OmegaConfigLoader(conf_source=conf_path, env="local")
catalog = config_loader["catalog"]
merelcht marked this conversation as resolved.
Show resolved Hide resolved
```

In this example, `"catalog"` is the key to the default catalog patterns specified in the `OmegaConfigLoader` class.

### 6. Exception Handling
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Am I correct in thinking this guide is mostly flagging differences rather than a full set of migration steps. So, in this section, you're showing where errors could be different. Are there things that are the same that you need to call out, as in "For error and exception handling, most errors are the same. Those you need to be aware of that are different between the original ConfigLoader and OmegaConfigLoader are as follows:"

Or is everything different, and set out below ? I'm just thinking about setting user expectations I guess.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good point. I have indeed mostly focussed on differences, because all the other behaviour is still the same. This section about errors is quite specific and most users probably won't care, but I added it in case users are catching a specific type of error and need to now change that. I'll add the sentence in mentioning most errors are still the same.

* `OmegaConfigLoader` throws a `MissingConfigException` when configuration paths don't exist, rather than the `ValueError` used in `ConfigLoader`.
* In `OmegaConfigLoader`, if there is bad syntax in your configuration files, it will trigger a `ParserError` instead of a `BadConfigException` used in `ConfigLoader`.
1 change: 1 addition & 0 deletions docs/source/configuration/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,5 +6,6 @@
configuration_basics
credentials
parameters
config_loader_migration
advanced_configuration
```