Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Remove 0.2% restore bench to prevent nightly timeouts #2477

Merged
merged 1 commit into from
Jan 29, 2021
Merged

Remove 0.2% restore bench to prevent nightly timeouts #2477

merged 1 commit into from
Jan 29, 2021

Conversation

Anviking
Copy link
Member

@Anviking Anviking commented Jan 29, 2021 •
edited
Loading

Context

The original problem in ADP-639 just disappeared... But now, a few recent nightlies are timing out:

  • Build #833 -> about to timeout
    • Node sync took 2.3h
      • [2021-01-29 00:32:52.77 UTC]
      • [2021-01-29 02:52:00.75 UTC]
  • Build #832 timed out
    • Node sync took 2.4h
      • [2021-01-28 00:31:59.32 UTC]
      • [2021-01-28 02:56:13.76 UTC]
  • Build #830 succeeded
    • Node sync took 2.3h
      • [2021-01-27 01:08:39.55 UTC]
      • [2021-01-27 03:28:23.57 UTC]

Overview

  • Removed 0.2% seq and 0.2% rnd benchmarks to gain about an hour

Comments

  • Now we have 0%, 0.1% and 0.4%. Don't think 0.2% provides anything that 0.4% doesn't.
  • Haven't looked close into what causes the variability, but I think simply removing 0.2% is a fine thing still?

@Anviking
Remove 0.2% restore bench to prevent nightly timeouts
7cabfad 
A few recent nightlies are timing out:

- [Build #833](https://buildkite.com/input-output-hk/cardano-wallet-nightly/builds/833#f00662b3-a0d0-4e59-bdaf-d097ef06f86c) -> about to timeout
    - Node sync took 2.3h
        - [2021-01-29 00:32:52.77 UTC]
        - [2021-01-29 02:52:00.75 UTC]
- [Build #832](https://buildkite.com/input-output-hk/cardano-wallet-nightly/builds/832#55f4c9bb-40fa-41ee-a6a8-c028b6bdcc4b) timed out
    - Node sync took 2.4h
        - [2021-01-28 00:31:59.32 UTC]
        - [2021-01-28 02:56:13.76 UTC]
- [Build #830](https://buildkite.com/input-output-hk/cardano-wallet-nightly/builds/830#31d4a3e2-eec2-4344-bad5-e8bbddffbe85) succeeded
    - Node sync took 2.3h
        - [2021-01-27 01:08:39.55 UTC]
        - [2021-01-27 03:28:23.57 UTC]

To be safe, we could make the tests shorter.

We should gain about an hour from removing 0.2% seq and rnd.

Now we have 0%, 0.1% and 0.4%.

I don't think there's a better candidate for removing:

All results:
BenchSeqResults:
  benchName: 1-wallets-to-0.1
  restoreTime: 41.27 s
  readWalletTime: 7.077 ms
  listAddressesTime: 84.37 Î?s
  listTransactionsTime: 401.7 Î?s
  estimateFeesTime: 95.49 ms
  walletOverview:
     number of addresses: 20
     number of transactions: -0
     = Total value of -0 lovelace across 0 UTxOs

BenchSeqResults:
  benchName: 10-wallets-to-0.01
  restoreTime: 38.70 s
  readWalletTime: 203.1 ms
  listAddressesTime: 219.6 ms
  listTransactionsTime: 133.9 ms
  estimateFeesTime: 452.7 ms
  walletOverview:
     number of addresses: 20
     number of transactions: -0
     = Total value of -0 lovelace across 0 UTxOs

BenchSeqResults:
  benchName: 20-wallets-to-0.01
  restoreTime: 74.61 s
  readWalletTime: 797.4 ms
  listAddressesTime: 132.9 ms
  listTransactionsTime: 151.0 ms
  estimateFeesTime: 106.3 ms
  walletOverview:
     number of addresses: 20
     number of transactions: -0
     = Total value of -0 lovelace across 0 UTxOs

BenchSeqResults:
  benchName: 40-wallets-to-0.01
  restoreTime: 165.5 s
  readWalletTime: 1.407 ms
  listAddressesTime: 519.9 Î?s
  listTransactionsTime: 580.7 Î?s
  estimateFeesTime: 94.64 ms
  walletOverview:
     number of addresses: 20
     number of transactions: -0
     = Total value of -0 lovelace across 0 UTxOs

BenchSeqResults:
  benchName: 80-wallets-to-0.01
  restoreTime: 409.6 s
  readWalletTime: 11.70 ms
  listAddressesTime: 2.015 ms
  listTransactionsTime: 2.020 ms
  estimateFeesTime: 103.4 ms
  walletOverview:
     number of addresses: 20
     number of transactions: -0
     = Total value of -0 lovelace across 0 UTxOs

BenchSeqResults:
  benchName: 100-wallets-to-0.01
  restoreTime: 563.5 s
  readWalletTime: 8.196 ms
  listAddressesTime: 2.235 ms
  listTransactionsTime: 380.9 Î?s
  estimateFeesTime: 238.3 ms
  walletOverview:
     number of addresses: 20
     number of transactions: -0
     = Total value of -0 lovelace across 0 UTxOs

BenchSeqResults:
  benchName: 0-percent-seq
  restoreTime: 490.2 s
  readWalletTime: 414.0 Î?s
  listAddressesTime: 72.47 Î?s
  listTransactionsTime: 270.5 Î?s
  estimateFeesTime: 67.40 ms
  walletOverview:
     number of addresses: 20
     number of transactions: -0
     = Total value of -0 lovelace across 0 UTxOs

BenchSeqResults:
  benchName: 0.1-percent-seq
  restoreTime: 1053 s
  readWalletTime: 23.79 ms
  listAddressesTime: 29.23 ms
  listTransactionsTime: 13.13 s
  estimateFeesTime: 291.8 ms
  walletOverview:
     number of addresses: 7744
     number of transactions: 13243
     = Total value of 28889686415572 lovelace across 916 UTxOs

BenchSeqResults:
  benchName: 0.2-percent-seq
  restoreTime: 1982 s
  readWalletTime: 88.99 ms
  listAddressesTime: 134.3 ms
  listTransactionsTime: 49.45 s
  estimateFeesTime: 941.2 ms
  walletOverview:
     number of addresses: 21521
     number of transactions: 35377
     = Total value of 44102340460783 lovelace across 1766 UTxOs

BenchSeqResults:
  benchName: 0.4-percent-seq
  restoreTime: 3701 s
  readWalletTime: 773.8 ms
  listAddressesTime: 396.3 ms
  listTransactionsTime: 163.5 s
  estimateFeesTime: 1.612 s
  walletOverview:
     number of addresses: 38287
     number of transactions: 63070
     = Total value of 84999797989729 lovelace across 3642 UTxOs

BenchRndResults:
  benchName: 0-percent-rnd
  restoreTime: 607.6 s
  readWalletTime: 477.1 Î?s
  listAddressesTime: 11.23 Î?s
  listTransactionsTime: 163.7 Î?s
  importOneAddressTime: 2.802 ms
  importManyAddressesTime: 105.1 ms
  estimateFeesTime: 30.74 ms
  walletOverview:
     number of addresses: -0
     number of transactions: -0
     = Total value of -0 lovelace across 0 UTxOs

BenchRndResults:
  benchName: 0.1-percent-rnd
  restoreTime: 1168 s
  readWalletTime: 12.19 ms
  listAddressesTime: 826.5 Î?s
  listTransactionsTime: 14.27 s
  importOneAddressTime: 163.6 ms
  importManyAddressesTime: 826.7 ms
  estimateFeesTime: 13.49 ms
  walletOverview:
     number of addresses: 7729
     number of transactions: 13247
     = Total value of 25389808016281 lovelace across 917 UTxOs

BenchRndResults:
  benchName: 0.2-percent-rnd
  restoreTime: 2135 s
  readWalletTime: 141.7 ms
  listAddressesTime: 2.489 ms
  listTransactionsTime: 55.26 s
  importOneAddressTime: 542.2 ms
  importManyAddressesTime: 2.966 s
  estimateFeesTime: 889.1 ms
  walletOverview:
     number of addresses: 21509
     number of transactions: 35384
     = Total value of 40607592679608 lovelace across 1770 UTxOs

BenchRndResults:
  benchName: 0.4-percent-rnd
  restoreTime: 3973 s
  readWalletTime: 187.2 ms
  listAddressesTime: 4.467 ms
  listTransactionsTime: 185.3 s
  importOneAddressTime: 1.116 s
  importManyAddressesTime: 6.352 s
  estimateFeesTime: 1.787 s
  walletOverview:
     number of addresses: 38275
     number of transactions: 63080
     = Total value of 84998315991650 lovelace across 3645 UTxOs
@Anviking Anviking self-assigned this Jan 29, 2021
@Anviking Anviking mentioned this pull request Jan 29, 2021
Closed
1 task
@Anviking Anviking requested a review from piotr-iohk January 29, 2021 11:05
piotr-iohk
piotr-iohk approved these changes Jan 29, 2021
View reviewed changes
Copy link
Contributor

@piotr-iohk piotr-iohk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sounds good.

@Anviking
Copy link
Member Author

bors r+

@iohk-bors
Copy link
Contributor

iohk-bors bot commented Jan 29, 2021

Build succeeded:

@iohk-bors iohk-bors bot merged commit d96b11d into master Jan 29, 2021
@iohk-bors iohk-bors bot deleted the anviking/ADP-639/slim-down-bench branch January 29, 2021 12:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@piotr-iohk piotr-iohk piotr-iohk approved these changes

Assignees

@Anviking Anviking

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@Anviking @piotr-iohk