Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Minimal major configuration overhaul #4161

Closed
0x009922 opened this issue Dec 21, 2023 · 3 comments
Closed

Minimal major configuration overhaul #4161

0x009922 opened this issue Dec 21, 2023 · 3 comments
Assignees
Labels
config-changes Changes in configuration and start up of the Iroha Enhancement New feature or request iroha2-dev The re-implementation of a BFT hyperledger in RUST Refactor Improvement to overall code quality UI Something about the interface

Comments

@0x009922
Copy link
Contributor

0x009922 commented Dec 21, 2023

Description

Part of the RFC: #2585

After making semi-minor changes to the config and CLI, I think it is possible to make a major step in Configuration Overhaul. In this issue I describe how I see this next major step.

At the very least it should match with the configuration reference (hyperledger/iroha-2-docs#397), which I will enhance at the same time.

Planned changes

  • Drop JSON/JSON5 support for the configuration. Use TOML exclusively.
  • Rename and restructure parameters, re-think ENV variables
  • Support extends mechanism for easy configuration composition
  • Move all parsing and validation logic into iroha_config, clearly distinguishing user-provided config from the parsed one
  • Implement configuration tracing mechanism
  • Move chain-wide parameters into a separate section until it is moved from the config entirely: [discussion] Align configuration-related instructions with the RFC #4028
  • Perhaps, remove complex JSON values from ENV entirely: leave this space for primitives only
  • Perhaps, use miette for comprehensive error reports

Goals in mind

Since there will be a lot of boilerplate, it makes sense to continue development of project soukousei which will provide a generic way to work with configuration. However, I think it would be too costly to bother about it right now. Instead I want to re-write iroha_config fully manually. Although there will be a lot of boilerplate, the process will be streamlined and focused on the final result from the start. Moreover, after manual implementation it will be clearer how to design soukousei.

Therefore:

  • Re-write iroha_config fully manually, despite having a lot of boilerplate
  • Write code while keeping in mind that boilerplate will be moved to the library
  • Prioritise errors handling quality: collect as much info as possible, build comprehensive reports
@0x009922 0x009922 added Enhancement New feature or request iroha2-dev The re-implementation of a BFT hyperledger in RUST config-changes Changes in configuration and start up of the Iroha Refactor Improvement to overall code quality UI Something about the interface labels Dec 21, 2023
@0x009922 0x009922 self-assigned this Dec 21, 2023
0x009922 added a commit to 0x009922/iroha that referenced this issue Dec 21, 2023
Signed-off-by: Dmitry Balashov <[email protected]>
@DCNick3
Copy link
Contributor

DCNick3 commented Dec 21, 2023

re: miette: might also want to consider ariadne, which is focused solely on formatting the output nicely without trying to cover the use-cases of thiserror which might or might not be useful in case of config.

What would also be nice to see diagnostics-wise is emission of all validation errors in one go, so that the user would not have to re-try launching iroha after fixing errors one-by-one and could instead see all (or, at least, most) of the validation problems right away. This can be achieved by using a error accumulator pattern, like was done in iroha macros (the emit! macro doesn't fail the validation right away, allowing the execution to proceed and collect some more possible errors)

0x009922 added a commit to 0x009922/iroha that referenced this issue Dec 22, 2023
Signed-off-by: Dmitry Balashov <[email protected]>
@0x009922
Copy link
Contributor Author

might also want to consider ariadne

Thanks, will consider

What would also be nice to see diagnostics-wise is emission of all validation errors in one go

For sure! It is one of the main points of the RFC.

0x009922 added a commit that referenced this issue Dec 22, 2023
* [refactor]: refactor cli(s)

- `iroha`: use `clap` `v4`
- `iroha`: update CLI (`--config`, `--terminal-colors`)
- `iroha`: handle user-provided config file strictly
- `iroha`: parse ENV config after file config
- `iroha_config`: add `genesis.file`, remove `account_*` prefix
- `iroha_config`: move some genesis config validation logic
  from `iroha`
- `iroha_client_cli`: upgrade `clap` to `v4`
- `iroha_genesis`: remove side effects (bail, logs)
- `iroha_genesis`: clarify validation logic (#3075)
- `iroha_genesis`: remove `pub` to guard invariant of `GenesisNetwork`

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: chore

Signed-off-by: Dmitry Balashov <[email protected]>

* [fix]: fix & test config

- `iroha_config`: use `PathBuf` when appropriate
- `iroha_config`: fix `Path` behaviour
- `iroha_config`: finally, make `disable_panic_terminal_colors`
  truly optional (#3506)
- `iroha`: resolve all relative paths in config file
- `iroha`: improve config error messages
- `iroha`: add config integration tests

Signed-off-by: Dmitry Balashov <[email protected]>

* [test]: test & refactor

- `iroha_client_cli`: adopt `--config` from `iroha`
- `iroha_client_cli`: refactor `--metadata` arg
- `iroha`: extend tests
- update test code overall

Signed-off-by: Dmitry Balashov <[email protected]>

* [fix]: clippy, tests

Signed-off-by: Dmitry Balashov <[email protected]>

* [fix]: use `serde_as_str` for paths

Signed-off-by: Dmitry Balashov <[email protected]>

* [chore]: re-generate peer config

Signed-off-by: Dmitry Balashov <[email protected]>

* [test]: update `test_env.py`

Signed-off-by: Dmitry Balashov <[email protected]>

* [test]: remove deprecated option from test config

Signed-off-by: Dmitry Balashov <[email protected]>

* [docs]: update README

Signed-off-by: Dmitry Balashov <[email protected]>

* [chore]: remove obsolete comment

Signed-off-by: Dmitry Balashov <[email protected]>

* [fix]: update `iroha_swarm`

Now it relies on that `genesis.file` is set in the config file

Signed-off-by: Dmitry Balashov <[email protected]>

* [feat]: `kagami config peer` + `--genesis-file-in-config`

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: chores

Signed-off-by: Dmitry Balashov <[email protected]>

* Apply suggestions from code review

Co-authored-by: Ekaterina Mekhnetsova <[email protected]>
Signed-off-by: 0x009922 <[email protected]>

* [feat]: use `PATH` as value name

Signed-off-by: Dmitry Balashov <[email protected]>

* [docs]: align `StatusFailure` message

Signed-off-by: Dmitry Balashov <[email protected]>

* [docs]: adopt suggestion in `iroha_client_cli`

#4153 (comment)
Signed-off-by: Dmitry Balashov <[email protected]>

* [build]: exclude `genesis.file` from default config

Signed-off-by: Dmitry Balashov <[email protected]>

* [feat]: refine genesis config parse error

Signed-off-by: Dmitry Balashov <[email protected]>

* [test]: fix & refactor `test_env.py`

Signed-off-by: Dmitry Balashov <[email protected]>

* [test]: fix CANNOT_BE_EMPTY message

It is updated with a new version of clap

Signed-off-by: Dmitry Balashov <[email protected]>

* [test]: return `disable_panic_terminal_colors: null` to test config

It is needed so that only `iroha_test_config.json` without
defaults layer is valid (will fix on config refactoring)

Signed-off-by: Dmitry Balashov <[email protected]>

* [fix]: handle configuration properly in `iroha_swarm`

- set `GENESIS_FILE` in env, since `config.json` doesn't have it
- strictly specify `config.json` and `genesis.json` files

Signed-off-by: Dmitry Balashov <[email protected]>

* Apply suggestions from code review

Co-authored-by: ⭐️NINIKA⭐️ <[email protected]>
Signed-off-by: 0x009922 <[email protected]>

* Update client_cli/src/main.rs

Co-authored-by: ⭐️NINIKA⭐️ <[email protected]>
Signed-off-by: 0x009922 <[email protected]>

* [refactor]: eliminate extra `iroha_config` dependency

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: remove dead code

Signed-off-by: Dmitry Balashov <[email protected]>

* [test]: use `serde_json::json!`

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: remove `disable_panic_terminal_colors` parameter

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: update genesis & its testing

- remove `cfg(test)` behaviour from `GenesisNetwork::new()`
- refactor `impl TestGenesis for Genesis`
  - remove extra `bool` + `Option` invariant
  - allow creating genesis with additional ISIs
- remove `test-utils` feature & `GenesisNetwork::transactions_mut()`
- update related tests
- update error message on signing failure

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: remove extension check from `Path::default()`

Signed-off-by: Dmitry Balashov <[email protected]>

* [docs]: update genesis topology messages

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: use `Popen`'s `env` instead of global `os.environ`

Signed-off-by: Dmitry Balashov <[email protected]>

* [fix]: typo

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: use `Partial`/`Full` terminology

Signed-off-by: Dmitry Balashov <[email protected]>

* [test]: inherit env from OS

Signed-off-by: Dmitry Balashov <[email protected]>

* [docs]: link issue #4161

Signed-off-by: Dmitry Balashov <[email protected]>

* [fix]: format

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: apply lints

Signed-off-by: Dmitry Balashov <[email protected]>

* [feat]: panic on default path extension

Signed-off-by: Dmitry Balashov <[email protected]>

* [test]: update err message

Signed-off-by: Dmitry Balashov <[email protected]>

* [docs]: update docs of `read_config`

Signed-off-by: Dmitry Balashov <[email protected]>

---------

Signed-off-by: Dmitry Balashov <[email protected]>
Signed-off-by: 0x009922 <[email protected]>
Co-authored-by: Ekaterina Mekhnetsova <[email protected]>
Co-authored-by: ⭐️NINIKA⭐️ <[email protected]>
Asem-Abdelhady pushed a commit to Asem-Abdelhady/iroha that referenced this issue Jan 9, 2024
…ctor CLIs (hyperledger#4153)

* [refactor]: refactor cli(s)

- `iroha`: use `clap` `v4`
- `iroha`: update CLI (`--config`, `--terminal-colors`)
- `iroha`: handle user-provided config file strictly
- `iroha`: parse ENV config after file config
- `iroha_config`: add `genesis.file`, remove `account_*` prefix
- `iroha_config`: move some genesis config validation logic
  from `iroha`
- `iroha_client_cli`: upgrade `clap` to `v4`
- `iroha_genesis`: remove side effects (bail, logs)
- `iroha_genesis`: clarify validation logic (hyperledger#3075)
- `iroha_genesis`: remove `pub` to guard invariant of `GenesisNetwork`

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: chore

Signed-off-by: Dmitry Balashov <[email protected]>

* [fix]: fix & test config

- `iroha_config`: use `PathBuf` when appropriate
- `iroha_config`: fix `Path` behaviour
- `iroha_config`: finally, make `disable_panic_terminal_colors`
  truly optional (hyperledger#3506)
- `iroha`: resolve all relative paths in config file
- `iroha`: improve config error messages
- `iroha`: add config integration tests

Signed-off-by: Dmitry Balashov <[email protected]>

* [test]: test & refactor

- `iroha_client_cli`: adopt `--config` from `iroha`
- `iroha_client_cli`: refactor `--metadata` arg
- `iroha`: extend tests
- update test code overall

Signed-off-by: Dmitry Balashov <[email protected]>

* [fix]: clippy, tests

Signed-off-by: Dmitry Balashov <[email protected]>

* [fix]: use `serde_as_str` for paths

Signed-off-by: Dmitry Balashov <[email protected]>

* [chore]: re-generate peer config

Signed-off-by: Dmitry Balashov <[email protected]>

* [test]: update `test_env.py`

Signed-off-by: Dmitry Balashov <[email protected]>

* [test]: remove deprecated option from test config

Signed-off-by: Dmitry Balashov <[email protected]>

* [docs]: update README

Signed-off-by: Dmitry Balashov <[email protected]>

* [chore]: remove obsolete comment

Signed-off-by: Dmitry Balashov <[email protected]>

* [fix]: update `iroha_swarm`

Now it relies on that `genesis.file` is set in the config file

Signed-off-by: Dmitry Balashov <[email protected]>

* [feat]: `kagami config peer` + `--genesis-file-in-config`

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: chores

Signed-off-by: Dmitry Balashov <[email protected]>

* Apply suggestions from code review

Co-authored-by: Ekaterina Mekhnetsova <[email protected]>
Signed-off-by: 0x009922 <[email protected]>

* [feat]: use `PATH` as value name

Signed-off-by: Dmitry Balashov <[email protected]>

* [docs]: align `StatusFailure` message

Signed-off-by: Dmitry Balashov <[email protected]>

* [docs]: adopt suggestion in `iroha_client_cli`

hyperledger#4153 (comment)
Signed-off-by: Dmitry Balashov <[email protected]>

* [build]: exclude `genesis.file` from default config

Signed-off-by: Dmitry Balashov <[email protected]>

* [feat]: refine genesis config parse error

Signed-off-by: Dmitry Balashov <[email protected]>

* [test]: fix & refactor `test_env.py`

Signed-off-by: Dmitry Balashov <[email protected]>

* [test]: fix CANNOT_BE_EMPTY message

It is updated with a new version of clap

Signed-off-by: Dmitry Balashov <[email protected]>

* [test]: return `disable_panic_terminal_colors: null` to test config

It is needed so that only `iroha_test_config.json` without
defaults layer is valid (will fix on config refactoring)

Signed-off-by: Dmitry Balashov <[email protected]>

* [fix]: handle configuration properly in `iroha_swarm`

- set `GENESIS_FILE` in env, since `config.json` doesn't have it
- strictly specify `config.json` and `genesis.json` files

Signed-off-by: Dmitry Balashov <[email protected]>

* Apply suggestions from code review

Co-authored-by: ⭐️NINIKA⭐️ <[email protected]>
Signed-off-by: 0x009922 <[email protected]>

* Update client_cli/src/main.rs

Co-authored-by: ⭐️NINIKA⭐️ <[email protected]>
Signed-off-by: 0x009922 <[email protected]>

* [refactor]: eliminate extra `iroha_config` dependency

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: remove dead code

Signed-off-by: Dmitry Balashov <[email protected]>

* [test]: use `serde_json::json!`

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: remove `disable_panic_terminal_colors` parameter

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: update genesis & its testing

- remove `cfg(test)` behaviour from `GenesisNetwork::new()`
- refactor `impl TestGenesis for Genesis`
  - remove extra `bool` + `Option` invariant
  - allow creating genesis with additional ISIs
- remove `test-utils` feature & `GenesisNetwork::transactions_mut()`
- update related tests
- update error message on signing failure

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: remove extension check from `Path::default()`

Signed-off-by: Dmitry Balashov <[email protected]>

* [docs]: update genesis topology messages

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: use `Popen`'s `env` instead of global `os.environ`

Signed-off-by: Dmitry Balashov <[email protected]>

* [fix]: typo

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: use `Partial`/`Full` terminology

Signed-off-by: Dmitry Balashov <[email protected]>

* [test]: inherit env from OS

Signed-off-by: Dmitry Balashov <[email protected]>

* [docs]: link issue hyperledger#4161

Signed-off-by: Dmitry Balashov <[email protected]>

* [fix]: format

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: apply lints

Signed-off-by: Dmitry Balashov <[email protected]>

* [feat]: panic on default path extension

Signed-off-by: Dmitry Balashov <[email protected]>

* [test]: update err message

Signed-off-by: Dmitry Balashov <[email protected]>

* [docs]: update docs of `read_config`

Signed-off-by: Dmitry Balashov <[email protected]>

---------

Signed-off-by: Dmitry Balashov <[email protected]>
Signed-off-by: 0x009922 <[email protected]>
Co-authored-by: Ekaterina Mekhnetsova <[email protected]>
Co-authored-by: ⭐️NINIKA⭐️ <[email protected]>
@0x009922 0x009922 mentioned this issue Jan 17, 2024
12 tasks
Asem-Abdelhady pushed a commit to Asem-Abdelhady/iroha that referenced this issue Jan 22, 2024
…ctor CLIs (hyperledger#4153)

* [refactor]: refactor cli(s)

- `iroha`: use `clap` `v4`
- `iroha`: update CLI (`--config`, `--terminal-colors`)
- `iroha`: handle user-provided config file strictly
- `iroha`: parse ENV config after file config
- `iroha_config`: add `genesis.file`, remove `account_*` prefix
- `iroha_config`: move some genesis config validation logic
  from `iroha`
- `iroha_client_cli`: upgrade `clap` to `v4`
- `iroha_genesis`: remove side effects (bail, logs)
- `iroha_genesis`: clarify validation logic (hyperledger#3075)
- `iroha_genesis`: remove `pub` to guard invariant of `GenesisNetwork`

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: chore

Signed-off-by: Dmitry Balashov <[email protected]>

* [fix]: fix & test config

- `iroha_config`: use `PathBuf` when appropriate
- `iroha_config`: fix `Path` behaviour
- `iroha_config`: finally, make `disable_panic_terminal_colors`
  truly optional (hyperledger#3506)
- `iroha`: resolve all relative paths in config file
- `iroha`: improve config error messages
- `iroha`: add config integration tests

Signed-off-by: Dmitry Balashov <[email protected]>

* [test]: test & refactor

- `iroha_client_cli`: adopt `--config` from `iroha`
- `iroha_client_cli`: refactor `--metadata` arg
- `iroha`: extend tests
- update test code overall

Signed-off-by: Dmitry Balashov <[email protected]>

* [fix]: clippy, tests

Signed-off-by: Dmitry Balashov <[email protected]>

* [fix]: use `serde_as_str` for paths

Signed-off-by: Dmitry Balashov <[email protected]>

* [chore]: re-generate peer config

Signed-off-by: Dmitry Balashov <[email protected]>

* [test]: update `test_env.py`

Signed-off-by: Dmitry Balashov <[email protected]>

* [test]: remove deprecated option from test config

Signed-off-by: Dmitry Balashov <[email protected]>

* [docs]: update README

Signed-off-by: Dmitry Balashov <[email protected]>

* [chore]: remove obsolete comment

Signed-off-by: Dmitry Balashov <[email protected]>

* [fix]: update `iroha_swarm`

Now it relies on that `genesis.file` is set in the config file

Signed-off-by: Dmitry Balashov <[email protected]>

* [feat]: `kagami config peer` + `--genesis-file-in-config`

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: chores

Signed-off-by: Dmitry Balashov <[email protected]>

* Apply suggestions from code review

Co-authored-by: Ekaterina Mekhnetsova <[email protected]>
Signed-off-by: 0x009922 <[email protected]>

* [feat]: use `PATH` as value name

Signed-off-by: Dmitry Balashov <[email protected]>

* [docs]: align `StatusFailure` message

Signed-off-by: Dmitry Balashov <[email protected]>

* [docs]: adopt suggestion in `iroha_client_cli`

hyperledger#4153 (comment)
Signed-off-by: Dmitry Balashov <[email protected]>

* [build]: exclude `genesis.file` from default config

Signed-off-by: Dmitry Balashov <[email protected]>

* [feat]: refine genesis config parse error

Signed-off-by: Dmitry Balashov <[email protected]>

* [test]: fix & refactor `test_env.py`

Signed-off-by: Dmitry Balashov <[email protected]>

* [test]: fix CANNOT_BE_EMPTY message

It is updated with a new version of clap

Signed-off-by: Dmitry Balashov <[email protected]>

* [test]: return `disable_panic_terminal_colors: null` to test config

It is needed so that only `iroha_test_config.json` without
defaults layer is valid (will fix on config refactoring)

Signed-off-by: Dmitry Balashov <[email protected]>

* [fix]: handle configuration properly in `iroha_swarm`

- set `GENESIS_FILE` in env, since `config.json` doesn't have it
- strictly specify `config.json` and `genesis.json` files

Signed-off-by: Dmitry Balashov <[email protected]>

* Apply suggestions from code review

Co-authored-by: ⭐️NINIKA⭐️ <[email protected]>
Signed-off-by: 0x009922 <[email protected]>

* Update client_cli/src/main.rs

Co-authored-by: ⭐️NINIKA⭐️ <[email protected]>
Signed-off-by: 0x009922 <[email protected]>

* [refactor]: eliminate extra `iroha_config` dependency

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: remove dead code

Signed-off-by: Dmitry Balashov <[email protected]>

* [test]: use `serde_json::json!`

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: remove `disable_panic_terminal_colors` parameter

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: update genesis & its testing

- remove `cfg(test)` behaviour from `GenesisNetwork::new()`
- refactor `impl TestGenesis for Genesis`
  - remove extra `bool` + `Option` invariant
  - allow creating genesis with additional ISIs
- remove `test-utils` feature & `GenesisNetwork::transactions_mut()`
- update related tests
- update error message on signing failure

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: remove extension check from `Path::default()`

Signed-off-by: Dmitry Balashov <[email protected]>

* [docs]: update genesis topology messages

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: use `Popen`'s `env` instead of global `os.environ`

Signed-off-by: Dmitry Balashov <[email protected]>

* [fix]: typo

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: use `Partial`/`Full` terminology

Signed-off-by: Dmitry Balashov <[email protected]>

* [test]: inherit env from OS

Signed-off-by: Dmitry Balashov <[email protected]>

* [docs]: link issue hyperledger#4161

Signed-off-by: Dmitry Balashov <[email protected]>

* [fix]: format

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: apply lints

Signed-off-by: Dmitry Balashov <[email protected]>

* [feat]: panic on default path extension

Signed-off-by: Dmitry Balashov <[email protected]>

* [test]: update err message

Signed-off-by: Dmitry Balashov <[email protected]>

* [docs]: update docs of `read_config`

Signed-off-by: Dmitry Balashov <[email protected]>

---------

Signed-off-by: Dmitry Balashov <[email protected]>
Signed-off-by: 0x009922 <[email protected]>
Co-authored-by: Ekaterina Mekhnetsova <[email protected]>
Co-authored-by: ⭐️NINIKA⭐️ <[email protected]>
Signed-off-by: Asem-Abdelhady <[email protected]>
Asem-Abdelhady pushed a commit to Asem-Abdelhady/iroha that referenced this issue Feb 9, 2024
…ctor CLIs (hyperledger#4153)

* [refactor]: refactor cli(s)

- `iroha`: use `clap` `v4`
- `iroha`: update CLI (`--config`, `--terminal-colors`)
- `iroha`: handle user-provided config file strictly
- `iroha`: parse ENV config after file config
- `iroha_config`: add `genesis.file`, remove `account_*` prefix
- `iroha_config`: move some genesis config validation logic
  from `iroha`
- `iroha_client_cli`: upgrade `clap` to `v4`
- `iroha_genesis`: remove side effects (bail, logs)
- `iroha_genesis`: clarify validation logic (hyperledger#3075)
- `iroha_genesis`: remove `pub` to guard invariant of `GenesisNetwork`

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: chore

Signed-off-by: Dmitry Balashov <[email protected]>

* [fix]: fix & test config

- `iroha_config`: use `PathBuf` when appropriate
- `iroha_config`: fix `Path` behaviour
- `iroha_config`: finally, make `disable_panic_terminal_colors`
  truly optional (hyperledger#3506)
- `iroha`: resolve all relative paths in config file
- `iroha`: improve config error messages
- `iroha`: add config integration tests

Signed-off-by: Dmitry Balashov <[email protected]>

* [test]: test & refactor

- `iroha_client_cli`: adopt `--config` from `iroha`
- `iroha_client_cli`: refactor `--metadata` arg
- `iroha`: extend tests
- update test code overall

Signed-off-by: Dmitry Balashov <[email protected]>

* [fix]: clippy, tests

Signed-off-by: Dmitry Balashov <[email protected]>

* [fix]: use `serde_as_str` for paths

Signed-off-by: Dmitry Balashov <[email protected]>

* [chore]: re-generate peer config

Signed-off-by: Dmitry Balashov <[email protected]>

* [test]: update `test_env.py`

Signed-off-by: Dmitry Balashov <[email protected]>

* [test]: remove deprecated option from test config

Signed-off-by: Dmitry Balashov <[email protected]>

* [docs]: update README

Signed-off-by: Dmitry Balashov <[email protected]>

* [chore]: remove obsolete comment

Signed-off-by: Dmitry Balashov <[email protected]>

* [fix]: update `iroha_swarm`

Now it relies on that `genesis.file` is set in the config file

Signed-off-by: Dmitry Balashov <[email protected]>

* [feat]: `kagami config peer` + `--genesis-file-in-config`

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: chores

Signed-off-by: Dmitry Balashov <[email protected]>

* Apply suggestions from code review

Co-authored-by: Ekaterina Mekhnetsova <[email protected]>
Signed-off-by: 0x009922 <[email protected]>

* [feat]: use `PATH` as value name

Signed-off-by: Dmitry Balashov <[email protected]>

* [docs]: align `StatusFailure` message

Signed-off-by: Dmitry Balashov <[email protected]>

* [docs]: adopt suggestion in `iroha_client_cli`

hyperledger#4153 (comment)
Signed-off-by: Dmitry Balashov <[email protected]>

* [build]: exclude `genesis.file` from default config

Signed-off-by: Dmitry Balashov <[email protected]>

* [feat]: refine genesis config parse error

Signed-off-by: Dmitry Balashov <[email protected]>

* [test]: fix & refactor `test_env.py`

Signed-off-by: Dmitry Balashov <[email protected]>

* [test]: fix CANNOT_BE_EMPTY message

It is updated with a new version of clap

Signed-off-by: Dmitry Balashov <[email protected]>

* [test]: return `disable_panic_terminal_colors: null` to test config

It is needed so that only `iroha_test_config.json` without
defaults layer is valid (will fix on config refactoring)

Signed-off-by: Dmitry Balashov <[email protected]>

* [fix]: handle configuration properly in `iroha_swarm`

- set `GENESIS_FILE` in env, since `config.json` doesn't have it
- strictly specify `config.json` and `genesis.json` files

Signed-off-by: Dmitry Balashov <[email protected]>

* Apply suggestions from code review

Co-authored-by: ⭐️NINIKA⭐️ <[email protected]>
Signed-off-by: 0x009922 <[email protected]>

* Update client_cli/src/main.rs

Co-authored-by: ⭐️NINIKA⭐️ <[email protected]>
Signed-off-by: 0x009922 <[email protected]>

* [refactor]: eliminate extra `iroha_config` dependency

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: remove dead code

Signed-off-by: Dmitry Balashov <[email protected]>

* [test]: use `serde_json::json!`

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: remove `disable_panic_terminal_colors` parameter

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: update genesis & its testing

- remove `cfg(test)` behaviour from `GenesisNetwork::new()`
- refactor `impl TestGenesis for Genesis`
  - remove extra `bool` + `Option` invariant
  - allow creating genesis with additional ISIs
- remove `test-utils` feature & `GenesisNetwork::transactions_mut()`
- update related tests
- update error message on signing failure

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: remove extension check from `Path::default()`

Signed-off-by: Dmitry Balashov <[email protected]>

* [docs]: update genesis topology messages

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: use `Popen`'s `env` instead of global `os.environ`

Signed-off-by: Dmitry Balashov <[email protected]>

* [fix]: typo

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: use `Partial`/`Full` terminology

Signed-off-by: Dmitry Balashov <[email protected]>

* [test]: inherit env from OS

Signed-off-by: Dmitry Balashov <[email protected]>

* [docs]: link issue hyperledger#4161

Signed-off-by: Dmitry Balashov <[email protected]>

* [fix]: format

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: apply lints

Signed-off-by: Dmitry Balashov <[email protected]>

* [feat]: panic on default path extension

Signed-off-by: Dmitry Balashov <[email protected]>

* [test]: update err message

Signed-off-by: Dmitry Balashov <[email protected]>

* [docs]: update docs of `read_config`

Signed-off-by: Dmitry Balashov <[email protected]>

---------

Signed-off-by: Dmitry Balashov <[email protected]>
Signed-off-by: 0x009922 <[email protected]>
Co-authored-by: Ekaterina Mekhnetsova <[email protected]>
Co-authored-by: ⭐️NINIKA⭐️ <[email protected]>
0x009922 added a commit that referenced this issue Feb 16, 2024
* [refactor]: wip

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: update structure

- exclude genesis block loading from config
- construct `KeyPair` on `iroha` completion
- use full field names in `complete()`s

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: apply lints

Signed-off-by: Dmitry Balashov <[email protected]>

* [feat]: include more ENV vars, refactor

Signed-off-by: Dmitry Balashov <[email protected]>

* [feat]: impl merging

- `UserField` wrap instead of `Option`
- move trusted peers uniqueness check

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: update Kagami

- Remove `config` subcommand
- Update default `genesis` building

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: restructure code

- move generic tools to `iroha_config_base`
- remove `iroha_client_config` crate
- define client config in `iroha_client::config`
- add client config sample TOML

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: update `iroha_logger`

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: update `iroha_telemetry`

- update usage of `iroha_config`
- use `Duration` in `RetryPeriod`

Signed-off-by: Dmitry Balashov <[email protected]>

* [fix]: fix util macro

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: re-struct config, update `iroha_core`

- split "user-layer" and "actual" config modules
- update logger, telemetry, and kagami (genesis)

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: update more crates

- `iroha_client`
- `iroha_torii`
- move Torii `uri` to `iroha_torii_const`

Signed-off-by: Dmitry Balashov <[email protected]>

* [feat]: compile `iroha`!

Signed-off-by: Dmitry Balashov <[email protected]>

* [feat]: compile `iroha_client`!

Signed-off-by: Dmitry Balashov <[email protected]>

* [feat]: compile `iroha_client_cli`!

Signed-off-by: Dmitry Balashov <[email protected]>

* [feat]: compile everything *_*

Signed-off-by: Dmitry Balashov <[email protected]>

* [fix]: chores

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: pass tests

- update signature of `PeerId::new` (avoid extra clone)
- fix `UserField::set`
- fix deps of `iroha_config_base`
- resolve runtime todos in `iroha_config`
- update `iroha_swarm` generation output
- some other refactoring

Signed-off-by: Dmitry Balashov <[email protected]>

* [feat]: implement `extends`

Signed-off-by: Dmitry Balashov <[email protected]>

* [chore]: update default snapshot storage

Signed-off-by: Dmitry Balashov <[email protected]>

* [fix]: update after rebase

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: update just everything

- `test_env.py`: use toml configs
- remove other channel configs
- put example configs into `config_samples` dir
- put docker setup into `config_samples/swarm` dir
- pytests: split settings for CLI and config paths
- make test env runnable

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: refactored config boilerplate into dedicated modules

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: apply some lints

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: update config naming in `[queue]`

also run tests with all features enabled

Signed-off-by: Dmitry Balashov <[email protected]>

* [feat]: implement `Config::load` shorthand

Signed-off-by: Dmitry Balashov <[email protected]>

* [docs]: add comment

Signed-off-by: Dmitry Balashov <[email protected]>

* [ci]: fix typo

Signed-off-by: Dmitry Balashov <[email protected]>

* [ci]: install `tomli_w` via pacman

Signed-off-by: Dmitry Balashov <[email protected]>

* [chore]: fix whitespace

Signed-off-by: Dmitry Balashov <[email protected]>

* [revert]: update the snapshot store path

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: fix pytests

- do not mutate client config from tests,
  override via env instead
- add `TORII_URL` env var to client config

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: apply suggestions from code review

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: curl up `SumeragiStartArgs`

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: refine telemetry

- simplify `regular_telemetry` to just `telemetry`
- clearer config passing

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: use `Infallible`

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: simplify client config

- `nonce` instead of `add_nonce`
- move `[api]` to root

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: update `--config` arg

- remove default value
- remove `IROHA_CONFIG` env

Signed-off-by: Dmitry Balashov <[email protected]>

* [docs]: update `read_config_and_genesis` docs

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: chore

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: just `idle_time`, without `query_`

Signed-off-by: Dmitry Balashov <[email protected]>

* [revert]: use `ident_length_limits` in _actual_ config

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: use `Config`, not vague `Root`

Signed-off-by: Dmitry Balashov <[email protected]>

* [chore]: remove comment

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: use `capacity` term in Queue

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: `set_creation_time_ms`, use `*` instead of `mul`

Signed-off-by: Dmitry Balashov <[email protected]>

* [fix]: remove dead code

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: update `--config` args

- use `PathBuf` again
- remove default value at Client CLI

Signed-off-by: Dmitry Balashov <[email protected]>

* [chore]: cleaning

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: use `strum` in place of `parse_display`

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: no unsafe code any more

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: docs & refinements

- document code in:
  - `iroha_config_base`
  - `iroha_config`
  - `iroha_client::config`
- refactor `iroha_config_base` APIs
- move `ExtendsPaths` into `iroha_config_base`
- remove `[iroha]` user config section
  - move `chain_id` and key pair to the root
  - move `p2p_address` to `network.address`
- rename `user_layer` modules to `user`
- add `_bytes` suffix for relevant fields,
  put a TODO to add a newtype for it

Signed-off-by: Dmitry Balashov <[email protected]>

* [docs]: fill `peer.example.toml`

Signed-off-by: Dmitry Balashov <[email protected]>

* [chore]: re-export `ConfigurationDTO` from client

Signed-off-by: Dmitry Balashov <[email protected]>

* [test]: fix them

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: apply lints

Signed-off-by: Dmitry Balashov <[email protected]>

* [test]: fix pytests

Signed-off-by: Dmitry Balashov <[email protected]>

* [misc]: re-arrange sample configurations

- use `configs` dir again
- name example configs as _templates_

Signed-off-by: Dmitry Balashov <[email protected]>

* [docs]: update README

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: remove `parse-display` from deps

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: remove extra `iroha_config_base` exposure from the client

Signed-off-by: Dmitry Balashov <[email protected]>

* [fix]: import `Deserialize` in the macro

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: use `PrivateKey::into_raw` instead

Signed-off-by: Dmitry Balashov <[email protected]>

* [ci]: use `--break-system-packages` `pip` flag

Signed-off-by: Dmitry Balashov <[email protected]>

* [fix]: regenerate swarms

Signed-off-by: Dmitry Balashov <[email protected]>

* [ci]: fix pytests workflow

Signed-off-by: Dmitry Balashov <[email protected]>

* [fix]: remove `PrivateKey::payload()` access

Signed-off-by: Dmitry Balashov <[email protected]>

* [docs]: fix "unable to validate" doc

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: accept `Duration` for `set_creation_time`

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: `PrivateKey::to_raw`

Signed-off-by: Dmitry Balashov <[email protected]>

* [fix]: do not extend trusted peers in config

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: hide user view from `iroha_client::config`

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: rename `*configuration` to `config` everywhere﹡

﹡except crypto

Signed-off-by: Dmitry Balashov <[email protected]>

* [fix]: update default wasm fuel limit

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: change docs and methods of `WebLogin`

Signed-off-by: Dmitry Balashov <[email protected]>

* [fix]: trusted peers and config tests

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: rename `telemetry.dev.out_file`

Signed-off-by: Dmitry Balashov <[email protected]>

* [docs]: update `GenesisNetwork::new` errors

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: use `serde_with`

Signed-off-by: Dmitry Balashov <[email protected]>

* [misc]: move `nonzero_ext` to workspace level

Signed-off-by: Dmitry Balashov <[email protected]>

* [fix]: make `--config` optional

It is still possible to set full config via env
(e.g. `iroha_swarm` case)

Signed-off-by: Dmitry Balashov <[email protected]>

* [chore]: remove "regular" from telemetry re-export

Signed-off-by: Dmitry Balashov <[email protected]>

* [chore]: rename imports in `wasm.rs`

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: lints

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: use `serde_with` in swarm

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: rename parameters, cover full config in tests, fix bugs

Signed-off-by: Dmitry Balashov <[email protected]>

* [test]: cover absolute paths

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: update client configs, cover full in tests

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: lints

Signed-off-by: Dmitry Balashov <[email protected]>

* [fix]: also rename ENVs

Signed-off-by: Dmitry Balashov <[email protected]>

* [build]: add notes to Dockerfile

Signed-off-by: Dmitry Balashov <[email protected]>

* [chore]: update style in `iroha_test_config.toml`

Signed-off-by: Dmitry Balashov <[email protected]>

* [test]: fix old params in `test_env.py`

Signed-off-by: Dmitry Balashov <[email protected]>

* [test]: add a note to `panic_on_invalid_genesis.sh`

Signed-off-by: Dmitry Balashov <[email protected]>

---------

Signed-off-by: Dmitry Balashov <[email protected]>
@0x009922
Copy link
Contributor Author

Closed by #4239

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
config-changes Changes in configuration and start up of the Iroha Enhancement New feature or request iroha2-dev The re-implementation of a BFT hyperledger in RUST Refactor Improvement to overall code quality UI Something about the interface
Projects
None yet
Development

No branches or pull requests

2 participants