[CHIA-299] DL separate DAT files in folders by store id

[fchirica]

Adds new config setting group_files_by_store which then organizes all the DAT files into subfolders where the directory name is the store id. The default setting is false, which means there is no change to the current behaviour on upgrade.

Currently, all the DAT files are stored in one single directory.

This PR allows the user to organize their DAT files using a sub-directory for each store. The directories will be automatically created if the setting is true.

Note that a new JSON key has been added to the plugin API (also called group_files_by_store) which instructs the plugin on whether to use sub-directories per store.

Note that the setting will only affect new files created or downloaded. Existing files will not be automatically moved when the setting is changed to true. Users may manually move files.

[github-actions]

This pull request has conflicts, please resolve those before we can evaluate the pull request.

[github-actions]

Conflicts have been resolved. A maintainer will review the pull request shortly.

[github-actions]

This pull request has conflicts, please resolve those before we can evaluate the pull request.

[github-actions]

Conflicts have been resolved. A maintainer will review the pull request shortly.

[github-actions]

This pull request has conflicts, please resolve those before we can evaluate the pull request.

[coveralls-official]

Pull Request Test Coverage Report for Build 10001146000

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

• For more information on this, see Tracking coverage changes with pull request builds.
• To avoid this issue with future PRs, see these Recommended CI Configurations.
• For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

• 167 of 252 (66.27%) changed or added relevant lines in 6 files are covered.
• 1948 unchanged lines in 78 files lost coverage.
• Overall coverage decreased (-0.1%) to 90.869%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
chia/data_layer/data_layer.py	14	19	73.68%
chia/data_layer/data_layer_server.py	0	10	0.0%
chia/data_layer/download_data.py	52	67	77.61%
chia/data_layer/s3_plugin_service.py	0	55	0.0%

Files with Coverage Reduction	New Missed Lines	%
chia/_tests/wallet/test_wallet_retry.py	1	98.55%
chia/types/eligible_coin_spends.py	1	98.06%
chia/_tests/wallet/db_wallet/test_dl_wallet.py	1	99.42%
chia/_tests/wallet/db_wallet/test_dl_offers.py	1	99.24%
chia/util/keyring_wrapper.py	1	73.83%
chia/daemon/keychain_proxy.py	1	72.57%
chia/full_node/mempool.py	1	99.71%
chia/_tests/wallet/dao_wallet/test_dao_wallets.py	1	99.95%
chia/data_layer/data_layer_server.py	1	0.0%
chia/farmer/farmer.py	1	72.23%

Totals
Change from base Build 9719381064:	-0.1%
Covered Lines:	72256
Relevant Lines:	79499

---

💛 - Coveralls

[github-actions]

This pull request has conflicts, please resolve those before we can evaluate the pull request.

[github-actions]

Conflicts have been resolved. A maintainer will review the pull request shortly.

[Starttoaster]

This PR is awaiting a coverage-diff override (or coverage of the diff) prior to merge

[github-actions]

File	Coverage	Missing Lines
chia/data_layer/data_layer.py	85.7%	lines 751
chia/data_layer/data_layer_server.py	0.0%	lines 105-113, 118
chia/data_layer/download_data.py	77.6%	lines 19-20, 57-60, 83, 214-215, 221-222, 227-229, 373
chia/data_layer/s3_plugin_service.py	0.0%	lines 14, 139-140, 142-143, 145, 148-149, 151-153, 155-156, 158-159, 161, 164-165, 167-169, 179, 182-183, 185-187, 189, 192, 194-195, 197-200, 202-205, 262, 265, 282-283, 302, 310-313, 315-317, 319-320, 322-323

Total	Missing	Coverage
240 lines	81 lines	66%

[emlowe]

Coverage exemption