Skip to content

Commit

Permalink
Add en tutorials
Browse files Browse the repository at this point in the history
  • Loading branch information
jbwang1997 committed Dec 18, 2022
1 parent 0e0b2d5 commit bb7769d
Show file tree
Hide file tree
Showing 2 changed files with 81 additions and 9 deletions.
72 changes: 72 additions & 0 deletions docs/en/advanced_tutorials/config.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,8 @@ wget https://raw.githubusercontent.com/open-mmlab/mmengine/main/docs/resources/c
wget https://raw.githubusercontent.com/open-mmlab/mmengine/main/docs/resources/config/my_module.py
wget https://raw.githubusercontent.com/open-mmlab/mmengine/main/docs/resources/config/optimizer_cfg.py
wget https://raw.githubusercontent.com/open-mmlab/mmengine/main/docs/resources/config/predefined_var.py
wget https://raw.githubusercontent.com/open-mmlab/mmengine/main/docs/resources/config/replace_data_root.py
wget https://raw.githubusercontent.com/open-mmlab/mmengine/main/docs/resources/config/replace_num_classes.py
wget https://raw.githubusercontent.com/open-mmlab/mmengine/main/docs/resources/config/refer_base_var.py
wget https://raw.githubusercontent.com/open-mmlab/mmengine/main/docs/resources/config/resnet50_delete_key.py
wget https://raw.githubusercontent.com/open-mmlab/mmengine/main/docs/resources/config/resnet50_lr0.01.py
Expand Down Expand Up @@ -490,6 +492,76 @@ Config (path: ./example.py): {'model': {'type': 'CustomModel', 'in_channels': [1

:::

### Replace fields with environment variables

When a field is deeply nested, we need to add a long prefix at the command line to locate it. To alleviate this problem, MMEngine allows users to substitute fields in configuration with environment variables.

Before parsing the configuration file, the program will search all `{{$ENV_VAR:DEF_VAL}}` fields and substitute those sections with environment variables. Here, `ENV_VAR` is the name of the environment variable used to replace this section, `DEF_VAL` is the default value if `ENV_VAR` is not set.

When we want to modify the dataset path at the command line, we can take `replace_data_root.py` as an example:

```python
dataset_type = 'CocoDataset'
data_root = '{{$DATASET:/data/coco/}}'
dataset=dict(ann_file= data_root + 'train.json')
```

If we run `demo_train.py` to parse this configuration file.

```bash
python demo_train.py replace_data_root.py
```

```
Config (path: replace_data_root.py): {'dataset_type': 'CocoDataset', 'data_root': '/data/coco/', 'dataset': {'ann_file': '/data/coco/train.json'}}
```

Here, we don't set the environment variable `DATASET`. Thus, the program directly replaces `{{$DATASET:/data/coco/}}` with the default value `/data/coco/`. If we set `DATASET` at the command line:

```bash
DATASET=/new/dataset/path/ python demo_train.py replace_data_root.py
```

```
Config (path: replace_data_root.py): {'dataset_type': 'CocoDataset', 'data_root': '/new/dataset/path/', 'dataset': {'ann_file': '/new/dataset/path/train.json'}}
```

The value of `data_root` has been substituted with the value of `DATASET` as `/new/dataset/path`.

```note
Environment variable substitution occurs before the configuration parsing. If the replaced field is also involved in other fields assignment, the environment variable substitution will also affect the other fields. In the above example, dataset.ann_file = data_root + 'train.json', so when data_root is replaced, dataset.ann_file will also be changed. Meanwhile, --cfg-options replace a field after the configuration file is parsed. --cfg-options data_root='/new/dataset/path/' does not affect dataset.ann_file
```

Environment variables can also be used to replace other types of fields. We can use `{{'$ENV_VAR:DEF_VAL'}}` or `{{"$ENV_VAR:DEF_VAL"}}` format to ensure the configuration file conforms to python syntax.

We can task `replace_num_classes.py` as an example:

```
model=dict(
bbox_head=dict(
num_classes={{'$NUM_CLASSES:80'}}))
```

If we run `demo_train.py` to parse this configuration file.

```bash
python demo_train.py replace_num_classes.py
```

```
Config (path: replace_num_classes.py): {'model': {'bbox_head': {'num_classes': 80}}}
```

Let us set the environment variable `NUM_CLASSES`

```bash
NUM_CLASSES=20 python demo_train.py replace_num_classes.py
```

```
Config (path: replace_num_classes.py): {'model': {'bbox_head': {'num_classes': 20}}}
```

### import the custom module

If we customize a module and register it into the corresponding registry, could we directly build it from the configuration file as the previous [section](#how-to-use-config) does? The answer is "I don't know" since I'm not sure the registration process has been triggered. To solve this "unknown" case, `Config` provides the `custom_imports` function, to make sure your module could be registered as expected.
Expand Down
18 changes: 9 additions & 9 deletions docs/zh_cn/advanced_tutorials/config.md
Original file line number Diff line number Diff line change
Expand Up @@ -492,13 +492,13 @@ Config (path: ./example.py): {'model': {'type': 'CustomModel', 'in_channels': [1

:::

### 替换环境变量
### 使用环境变量替换配置

当要修改的变量嵌套很深时,我们在命令行中也需要加上很长的前缀来表示这个变量的位置。为了更方便地在命令行中修改配置文件,MMEngine 提供了一套通过环境变量来替换固定片段的方法
当要修改的配置嵌套很深时,我们在命令行中需要加上很长的前缀来进行定位。为了更方便地在命令行中修改配置,MMEngine 提供了一套通过环境变量来替换配置的方法

在解析配置文件之前,MMEngine 会搜索所有的 `{{$ENV_VAR:DEF_VAL}}` 字段,并使用字段指定的环境变量来替换这一部分。这里 `ENV_VAR` 为这一字段指定的环境变量`DEF_VAL` 表示没有设置环境变量时的默认值
在解析配置文件之前,MMEngine 会搜索所有的 `{{$ENV_VAR:DEF_VAL}}` 字段,并使用特定的环境变量来替换这一部分。这里 `ENV_VAR` 为替换这一部分所用的环境变量`DEF_VAL` 为没有设置环境变量时的默认值

例如,当我们想在命令行中修改数据集的路径时,我们可以在配置文件 `replace_data_root.py` 中这样写:
例如,当我们想在命令行中修改数据集路径时,我们可以在配置文件 `replace_data_root.py` 中这样写:

```python
dataset_type = 'CocoDataset'
Expand All @@ -516,7 +516,7 @@ python demo_train.py replace_data_root.py
Config (path: replace_data_root.py): {'dataset_type': 'CocoDataset', 'data_root': '/data/coco/', 'dataset': {'ann_file': '/data/coco/train.json'}}
```

这里因为没有设置环境变量 `DATASET`, 程序直接使用默认值 `/data/coco/` 作为这一部分的值。如果在命令行前设置设置环境变量则会有如下结果:
这里没有设置环境变量 `DATASET`, 程序直接使用默认值 `/data/coco/` 来替换 `{{$DATASET:/data/coco/}}`。如果在命令行前设置设置环境变量则会有如下结果:

```bash
DATASET=/new/dataset/path/ python demo_train.py replace_data_root.py
Expand All @@ -526,15 +526,15 @@ DATASET=/new/dataset/path/ python demo_train.py replace_data_root.py
Config (path: replace_data_root.py): {'dataset_type': 'CocoDataset', 'data_root': '/new/dataset/path/', 'dataset': {'ann_file': '/new/dataset/path/train.json'}}
```

`data_root` 被替换成了新的路径 `/new/dataset/path/`
`data_root` 被替换成了环境变量 `DATASET` 的值 `/new/dataset/path/`

```note
环境变量的替换发生在配置文件解析之前。如果该变量还参与到其他变量的定义时,环境变量替换也会影响到其他变量。在上例中 dataset.ann_file 等于 data_root + 'train.json', 因此当 data_root 被替换时,dataset.ann_file 也会发生变化。而 --cfg-options 是在配置文件解析后去替换特定变量。--cfg-options data_root='/new/dataset/path/' 不会影响到 dataset.ann_file
环境变量的替换发生在配置文件解析之前。如果该配置还参与到其他配置的定义时,环境变量替换也会影响到其他配置。在上例中 dataset.ann_file 等于 data_root + 'train.json', 因此当 data_root 被替换时,dataset.ann_file 也会发生变化。而 --cfg-options 是在配置文件解析后去替换特定配置。--cfg-options data_root='/new/dataset/path/' 不会影响到 dataset.ann_file
```

环境变量也可以用来替换字符串以外的字段,这时可以使用 `{{'$ENV_VAR:DEF_VAL'}}` 或者 `{{"$ENV_VAR:DEF_VAL"}}``''``""` 保证配置文件合乎 python 语法。
环境变量也可以用来替换字符串以外的配置,这时可以使用 `{{'$ENV_VAR:DEF_VAL'}}` 或者 `{{"$ENV_VAR:DEF_VAL"}}` 格式`''``""` 用来保证配置文件合乎 python 语法。

例如当我们想替换模型预测的类别数时,可以在配置文件 `replace_num_classes.py` 中这样写:
例如,当我们想替换模型预测的类别数时,可以在配置文件 `replace_num_classes.py` 中这样写:

```
model=dict(
Expand Down

0 comments on commit bb7769d

Please sign in to comment.