Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: move busread and lookup block construction at the top of the trace #10707

Merged
merged 26 commits into from
Dec 18, 2024

Conversation

maramihali
Copy link
Contributor

@maramihali maramihali commented Dec 13, 2024

Constructing the lookup block and lookup table data at the top of the trace removes the dependence of active ranges on the dyadic circuit size which was causing problems for the overflow scenario and also reduces the number of active rows to be close to the real size (modulo AztecProtocol/barretenberg#1152 which still needs investigation).

@maramihali maramihali force-pushed the mm/refine-active-ranges branch from 0e9e0df to 5656dac Compare December 16, 2024 12:11
@maramihali maramihali changed the title Mm/refine active ranges chore: move the busread and lookup block at the top of the trace Dec 16, 2024
@maramihali maramihali marked this pull request as ready for review December 16, 2024 18:20
Copy link
Contributor

@ledwards2225 ledwards2225 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - couple of minor comments/suggestions

@@ -25,7 +25,10 @@ void construct_lookup_table_polynomials(const RefArray<typename Flavor::Polynomi
const size_t tables_size = circuit.get_tables_size();
ASSERT(tables_size <= MAX_LOOKUP_TABLES_SIZE); // if false, may need to increase constant
ASSERT(dyadic_circuit_size > tables_size + additional_offset);
size_t offset = dyadic_circuit_size - tables_size - additional_offset;
size_t offset = circuit.blocks.lookup.trace_offset;
if constexpr (IsPlonkFlavor<Flavor>) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is the right thing to do but ugh. Plonk.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

getting closer and closer to removing it is what one would hope

@@ -55,7 +55,7 @@ TEST_F(ComposerLibTests, LookupReadCounts)

// The table polys are constructed at the bottom of the trace, thus so to are the counts/tags
// TODO(https://github.com/AztecProtocol/barretenberg/issues/1033): construct tables and counts at top of trace
size_t offset = circuit_size - builder.get_tables_size();
size_t offset = builder.blocks.lookup.trace_offset;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if this offset should be codified into the flavor or something..

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's a bit risky it will cause someone some annoying bugs :/ the composer lib stuff can be integrated in the decider proving key once we get rid of plonk so it would not look as odd to access trace_offsets there

@@ -84,25 +84,23 @@ struct ExecutionTraceUsageTracker {
for (auto [max_size, fixed_block] : zip_view(max_sizes.get(), fixed_sizes.get())) {
size_t start_idx = fixed_block.trace_offset;
size_t end_idx = start_idx + max_size;
active_ranges.push_back(Range{ start_idx, end_idx });
active_ranges.emplace_back(start_idx, end_idx);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this will not break the mac build since Range is just a std::pair which probably has an explicit constructor but you might want to make sure that's right. What you have is obviously better form but I'm suspicious as to why it was using the push_back syntax previously.. may have just been because the Range struct used to be something custom.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be clear the mac build will break if you try to use emplace_back(<constructor_inputs>) for a struct which may be trivially constructible but does not have an explicitly defined constructor. For example it would not work if you had

struct Range {
    size_t start;
    size_t end;
};

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ohh interesting, i will double check this is ok

@maramihali maramihali changed the title chore: move the busread and lookup block at the top of the trace feat move busread and lookup block construction at the top of the trace Dec 17, 2024
@maramihali maramihali changed the title feat move busread and lookup block construction at the top of the trace feat: move busread and lookup block construction at the top of the trace Dec 17, 2024
@@ -11,7 +11,7 @@ namespace bb {
template <typename Flavor>
void construct_lookup_table_polynomials(const RefArray<typename Flavor::Polynomial, 4>& table_polynomials,
const typename Flavor::CircuitBuilder& circuit,
const size_t dyadic_circuit_size,
[[maybe_unused]] const size_t dyadic_circuit_size,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

needed?

@@ -33,16 +33,16 @@ struct ExecutionTraceUsageTracker {
size_t max_tables_size = 0;

// For printing only. Must match the order of the members in the arithmetization
static constexpr std::array<std::string_view, 13> block_labels{ "ecc_op",
static constexpr std::array<std::string_view, 13> block_labels{ "busread",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these are wrong no? ecc_op should still be first

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, thanks

@maramihali maramihali enabled auto-merge (squash) December 18, 2024 10:18
Copy link
Contributor

Changes to circuit sizes

Generated at commit: f81e8814e83ebfe88d4a39cf5f7114dcb3bdeeff, compared to commit: c90bb16a5880c42752809f383f517181e6f8a53a

🧾 Summary (100% most significant diffs)

Program ACIR opcodes (+/-) % Circuit size (+/-) %
rollup_base_public 0 ➖ 0.00% -37,029 ✅ -0.29%
rollup_base_private 0 ➖ 0.00% -37,030 ✅ -0.33%
rollup_block_root 0 ➖ 0.00% -111,087 ✅ -2.52%
rollup_block_merge 0 ➖ 0.00% -74,057 ✅ -3.75%
rollup_root 0 ➖ 0.00% -74,057 ✅ -3.75%
rollup_merge 0 ➖ 0.00% -74,025 ✅ -4.07%
parity_root 0 ➖ 0.00% -148,083 ✅ -4.07%
private_kernel_empty 0 ➖ 0.00% -36,997 ✅ -4.10%

Full diff report 👇
Program ACIR opcodes (+/-) % Circuit size (+/-) %
rollup_base_public 3,851,252 (0) 0.00% 12,617,705 (-37,029) -0.29%
rollup_base_private 3,619,601 (0) 0.00% 11,224,473 (-37,030) -0.33%
rollup_block_root 682,326 (0) 0.00% 4,293,399 (-111,087) -2.52%
rollup_block_merge 32,406 (0) 0.00% 1,900,801 (-74,057) -3.75%
rollup_root 32,390 (0) 0.00% 1,900,787 (-74,057) -3.75%
rollup_merge 1,791 (0) 0.00% 1,744,952 (-74,025) -4.07%
parity_root 4,290 (0) 0.00% 3,487,933 (-148,083) -4.07%
private_kernel_empty 965 (0) 0.00% 865,003 (-36,997) -4.10%

@maramihali maramihali merged commit e8480cb into master Dec 18, 2024
77 checks passed
@maramihali maramihali deleted the mm/refine-active-ranges branch December 18, 2024 11:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants