Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cost model 1.7 #20188

Merged
merged 31 commits into from
Oct 6, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
d74a587
Cost Model to limit transactions which are not parallelizeable (#16694)
tao-stones Jun 1, 2021
5c81d02
replay stage feed back program cost (#17731)
tao-stones Jun 9, 2021
31e5e55
investigate system performance test degradation (#17919)
tao-stones Jun 29, 2021
484526f
Persist cost table to blockstore (#18123)
tao-stones Jul 1, 2021
ee885dd
log warning when channel send fails (#18391)
tao-stones Jul 2, 2021
1887884
Aggregate cost_model into cost_tracker (#18374)
tao-stones Jul 6, 2021
317b196
update ledger tool to restore cost table from blockstore (#18489)
tao-stones Jul 8, 2021
6ff0104
manually fix merge conflicts
tao-stones Jul 16, 2021
731d1bc
Per-program id timings (#17554)
sakridge Jun 4, 2021
fcabd5b
more manual fixing
tao-stones Jul 16, 2021
20c6ed4
solve a merge conflict
tao-stones Jul 16, 2021
ddfe8ae
featurize cost model
tao-stones Jul 15, 2021
2b06c03
more merge fix
tao-stones Jul 16, 2021
a01dfb1
cost model uses compute_unit to replace microsecond as cost unit
tao-stones Jul 27, 2021
5e54e89
Reject blocks for costs above the max block cost (#18994)
tao-stones Jul 30, 2021
8accd75
Update block max cost limit to fix performance regession (#19276)
tao-stones Aug 17, 2021
0e90064
replace function with const var for better readability (#19285)
tao-stones Aug 18, 2021
1c0c473
Add few more metrics data points (#19624)
tao-stones Sep 2, 2021
d9db3b9
periodically report sigverify_stage stats (#19674)
jbiseda Sep 21, 2021
e9f38f5
manual merge
tao-stones Sep 25, 2021
4741aea
cost model nits (#18528)
jackcmay Jul 9, 2021
263d1aa
Accumulate consumed units (#18714)
jackcmay Jul 16, 2021
8c14e49
tx wide compute budget (#18631)
jackcmay Jul 16, 2021
f943c09
more manual merge
tao-stones Sep 25, 2021
cc1442b
ignore zerorize drop security
tao-stones Sep 25, 2021
1a0e3c9
- update const cost values with data collected by #19627
tao-stones Sep 28, 2021
673046d
add transaction cost histogram metrics (#20350)
tao-stones Sep 30, 2021
cee202c
rebase to 1.7.15
tao-stones Sep 30, 2021
d4162c0
add tx count and thread id to stats (#20451)
tao-stones Oct 5, 2021
d0a5161
remove cost_model feature_set
tao-stones Oct 6, 2021
a71e098
ignore vote transactions from cost model
tao-stones Oct 6, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
246 changes: 139 additions & 107 deletions Cargo.lock

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ members = [
"poh-bench",
"program-test",
"programs/bpf_loader",
"programs/compute-budget",
"programs/config",
"programs/exchange",
"programs/failure",
Expand Down
7 changes: 5 additions & 2 deletions banking-bench/src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ use crossbeam_channel::unbounded;
use log::*;
use rand::{thread_rng, Rng};
use rayon::prelude::*;
use solana_core::banking_stage::BankingStage;
use solana_core::{banking_stage::BankingStage, cost_model::CostModel, cost_tracker::CostTracker};
use solana_gossip::{cluster_info::ClusterInfo, cluster_info::Node};
use solana_ledger::{
blockstore::Blockstore,
Expand All @@ -27,7 +27,7 @@ use solana_sdk::{
};
use solana_streamer::socket::SocketAddrSpace;
use std::{
sync::{atomic::Ordering, mpsc::Receiver, Arc, Mutex},
sync::{atomic::Ordering, mpsc::Receiver, Arc, Mutex, RwLock},
thread::sleep,
time::{Duration, Instant},
};
Expand Down Expand Up @@ -231,6 +231,9 @@ fn main() {
vote_receiver,
None,
replay_vote_sender,
Arc::new(RwLock::new(CostTracker::new(Arc::new(RwLock::new(
CostModel::default(),
))))),
);
poh_recorder.lock().unwrap().set_bank(&bank);

Expand Down
1 change: 1 addition & 0 deletions ci/do-audit.sh
Original file line number Diff line number Diff line change
Expand Up @@ -45,5 +45,6 @@ cargo_audit_ignores=(
# Blocked on jsonrpc removing dependency on unmaintained `websocket`
# https://github.com/paritytech/jsonrpc/issues/605
--ignore RUSTSEC-2021-0079

)
scripts/cargo-for-all-lock-files.sh stable audit "${cargo_audit_ignores[@]}"
7 changes: 4 additions & 3 deletions core/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -27,13 +27,14 @@ ed25519-dalek = "=1.0.1"
fs_extra = "1.2.0"
flate2 = "1.0"
indexmap = { version = "1.5", features = ["rayon"] }
itertools = "0.9.0"
libc = "0.2.81"
log = "0.4.11"
lru = "0.6.1"
miow = "0.2.2"
net2 = "0.2.37"
num-traits = "0.2"
histogram = "0.6.9"
itertools = "0.10.1"
log = "0.4.14"
lru = "0.6.6"
rand = "0.7.0"
rand_chacha = "0.2.2"
rand_core = "0.6.2"
Expand Down
12 changes: 11 additions & 1 deletion core/benches/banking_stage.rs
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,9 @@ use log::*;
use rand::{thread_rng, Rng};
use rayon::prelude::*;
use solana_core::banking_stage::{BankingStage, BankingStageStats};
use solana_core::cost_model::CostModel;
use solana_core::cost_tracker::CostTracker;
use solana_core::cost_tracker_stats::CostTrackerStats;
use solana_gossip::cluster_info::ClusterInfo;
use solana_gossip::cluster_info::Node;
use solana_ledger::blockstore_processor::process_entries;
Expand All @@ -33,7 +36,7 @@ use solana_streamer::socket::SocketAddrSpace;
use std::collections::VecDeque;
use std::sync::atomic::Ordering;
use std::sync::mpsc::Receiver;
use std::sync::Arc;
use std::sync::{Arc, RwLock};
use std::time::{Duration, Instant};
use test::Bencher;

Expand Down Expand Up @@ -92,6 +95,10 @@ fn bench_consume_buffered(bencher: &mut Bencher) {
None::<Box<dyn Fn()>>,
&BankingStageStats::default(),
&recorder,
&Arc::new(RwLock::new(CostTracker::new(Arc::new(RwLock::new(
CostModel::new(std::u64::MAX, std::u64::MAX),
))))),
&mut CostTrackerStats::default(),
);
});

Expand Down Expand Up @@ -218,6 +225,9 @@ fn bench_banking(bencher: &mut Bencher, tx_type: TransactionType) {
vote_receiver,
None,
s,
Arc::new(RwLock::new(CostTracker::new(Arc::new(RwLock::new(
CostModel::new(std::u64::MAX, std::u64::MAX),
))))),
);
poh_recorder.lock().unwrap().set_bank(&bank);

Expand Down
Loading