Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add -checkpoints-file flag to defid #1347

Merged
merged 7 commits into from
Jun 21, 2022
Merged

Conversation

Jouzo
Copy link
Collaborator

@Jouzo Jouzo commented Jun 14, 2022

What kind of PR is this?:

/kind feature

What this PR does / why we need it:

Add -checkpoint-file=<file_path> flag to overwrite chainparams checkpoints.

To reduce defid datadir memory usage, the prune option is limited as it only prunes bitcoin related data.
Checkpoints are hardcoded in chainparams but some blocks segment are still reaching high memory usage. (50GiB around 800k blocks mark).
This PR let's a user add additional checkpoint to reduce resources usage.

The checkpoints file should be formatted as height blockhash
checkpoints_file.txt example :

50000 a45e6bf6ae858a287eb39021ea23880b4115c94e882e2b7c0fcfc98c317922cd
100000 3acd556dbd5e6e75bf463a15eeeeb54b6eab4a1f28039bdc343cc8c851cce45c
150000 46b231d42e5b002852708d48dec119bbc2d550fb67908f1e9f35102c1b45b94d
200000 414076e74894aaed3e1b52d64937f23289d59fe80e287c328a1281398bf9cb31
250000 d50a44503fa55cd01a78b98dea125e63b65aac720c96cca696857722e8149d77
300000 351c82cb8f77fba73e24223a9dd50954630560602c3a38f4d1c03dfa5cf1fd10
350000 ebc8737cb2caa77397f446e9a5aff72a2ca9e8305a6a6f8eb4b6c22f389bef08
400000 97c1014a66c9f327e04a59b3e1b4f551122d0698b6b1a98ec99555fffb474e9d
450000 03701a440b02d61b875ba2503bb53f1f1360cf66b4f0cf472e660a6809534379
500000 6a5b285bc68362deb66148069f55f82c02974056e73f5cc96971f7661ecd5880
550000 3f9aab70727d3cc76a3d406f520a71ccc6095aeea2d185e489f563320d429d5b
600000 79ddf4537e40cb59335a0551e5edc7bd396e6949aa2864c3200ca66f9c455405
650000 f18d64dd75c53590e833d3068132a65644963d5c5aebb4c73d42cbde8dc28d68
700000 0d0f779d7f43cc3fd9f01e71e136f7e9319d73791f8839720ab495327075e272
750000 d65c1f983b015ad665b2db9a206e2a7a0e8c51789918ea5bfda2f8121adc6f0f
800000 8d9ade23a54f16a0a5a6ce5d14788f93bfd7e703be8a3a07a85c935b47511f95
850000 2d7d58ae18a74f73b9836a8fffd3f65ce409536e654a6c644ce735215238a004
900000 02818c71fe7d8bcba7c778a73832a7e7b6603a8537bf61915ace497d2b0dec40
950000 51f243157205fb88f7cf5fe31cca7754f1b701c4482e9d43231cddcf931dc617
1000000 253d7137cfcda8d63e5f352ce1162c51a263dba6f9cad7a52f93308c91f04301
1050000 59df07db6d89dd1ea567dcbef1a605e434bdfb4008309c05843d482e7c04cc36
1100000 96d30c51e1617a12454a9ffa68a0e5746377063bdf09d1bf80bc5053ef8a392b
1150000 eeccab1fe80fe61498f6b264b56e6dc656e3d92bf1a1f6f544ff559df09306cd
1200000 0859876a88c08242c77a1deea7d01757e3c6c1b3155e73bff60e28c8f6e65911
1250000 cdbe56d09f344821b356f3d73862e8a8a6a939042e32d3e9b384da8228b9b5c4
1300000 be6dc303c212243899478b8908c1f7c54217bb735655cdef8d85ae2c3c548229
1350000 f8a18112409e689a55eeccb56ad0c7b18927ca8eda4b9f289dbaccc0685dcd2f
1400000 a222eb0930af41d1d5a6f367a9532ee319630fd72d72055fde5da6b57feabbc1
1450000 3a7e289cb3b0a3a460fb726f36e2bee4217492e8f8d996b8e342ab5082dc0462
1500000 c35a8a2fa6d076208dbb47e37b46b9da2da778bc06e2f224d5d5f370b8d71485
1550000 685a961abceadd38dbafafe0bdb528f6fb53cc9d3b4cbe2443446260f37e888c
1600000 213c49a84a8c01b412e13169ac0c1e070b828a73fa50b02e7ac18520155cb98a
1650000 7be36820df33d3a307d9b0c6c03f5e043fac3cf65d5fb19d111d52e31eda06ea
1700000 724af81603f524a0b54dea8348d8275e329b80e028982a706608099d68d01c9d
1750000 a787060561a8b2fced6aae34de52300b5a8721ce1a4e6ac6c37f995185441894
1800000 324cafa596501d4dda404727fc466eb493e072b669e00b007508fc1f69443991
1850000 1c9f6e6b7122f1d2e6ca0c6751453647e905e068ba8791e13c3722bbe5ff5987
1900000 65843174fc86bf26d65d3e10290c3e5f64fc946617ed0c9a391ecb17f55ecad5
1950000 70155c7089bffdf1f14745a4e5f819a40f590dc613bfda6d11502801ca9d657e

@Jouzo Jouzo marked this pull request as ready for review June 16, 2022 09:45
Res UpdateCheckpointsFromFile(CChainParams &params, const std::string &fileName) {
std::ifstream file(fileName);
if (!file.good()) {
return Res::Err("File \"%s\" does not exist.", fileName);
Copy link
Member

@prasannavl prasannavl Jun 20, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's make the error message is adaptable to all errors:
Could not read %s. Ensure it exists and has read permissions

@prasannavl prasannavl merged commit 59cd82f into master Jun 21, 2022
@prasannavl prasannavl deleted the feature/checkpoints_file branch June 21, 2022 22:01
Bushstar pushed a commit that referenced this pull request Jul 5, 2022
* Add -checkpoints-file flag to defid

* Fix lint: use ParseInt32 and add -checkpoints-file to SET_DOC_OPTIONAL

* Accepts comments and empty line in checkpoints file

* Generic file error message

* Trim lines

* Move trim_ws from masternodes/token to strencodings
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants