Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixing source map locations when peephole optimizations are applied. #15384

Merged
merged 2 commits into from
Nov 26, 2024

Conversation

vineethk
Copy link
Contributor

@vineethk vineethk commented Nov 24, 2024

Description

Fixes #14167.

Peephole optimization changes the original file format bytecode after it is generated from stackless bytecode. Source mapping is computed when the original file format bytecode is created, so the source mapping is incorrect after peephole optimization changes the file format bytecode.

This resulted in several downstream issues, including move test, prover, coverage, etc.

This PR contains the fix by having the peephole optimization maintain a mapping to the original offsets and propagating them, and updating the source map accordingly after the optimization.

How Has This Been Tested?

  • Existing tests
  • Added another test reported by a user
  • Manually tested the user's report before and after the fix (details below).

When we ran aptos move test --move-2 for the following program:

module anto::Example {
    use std::option::{Option};

    const E_CONDITION_A: u64 = 1;
    const E_CONDITION_B: u64 = 2;
    const E_CONDITION_C: u64 = 3;
    const E_CONDITION_D: u64 = 4;

    fun validate_parameters(
        param_a: Option<u64>, // Optional first parameter to validate
        param_b: Option<u64>, // Optional second parameter to validate
        reference_value: u64, // Reference value for comparisons
        threshold_value: u64, // Threshold for additional validation
        flag: bool // Indicates a positive or negative condition
    ) {
        // Determine the context based on the flag
        let is_positive = flag;

        // Check and validate param_a if it exists
        if (param_a.is_some()) {
            // Extract the value from the Option
            let value_a = param_a.extract();

            // Ensure the value is non-zero
            assert!(value_a > 0, E_CONDITION_A);

            // Validate based on the condition (is_positive)
            let is_valid_a =
                if (is_positive) {
                    value_a > reference_value // For positive condition, value_a must be greater than reference_value
                } else {
                    value_a < reference_value // For negative condition, value_a must be less than reference_value
                };

            // Assert that the validation passed
            assert!(
                is_valid_a,
                if (is_positive) E_CONDITION_B else E_CONDITION_C
            );
        };

        // Check and validate param_b if it exists
        if (param_b.is_some()) {
            // Extract the value from the Option
            let value_b = param_b.extract();

            // Ensure the value is non-zero
            assert!(value_b > 0, E_CONDITION_A);

            // Validate based on the condition (is_positive)
            let is_valid_b =
                if (is_positive) {
                    value_b < reference_value // For positive condition, value_b must be less than reference_value
                } else {
                    value_b > reference_value // For negative condition, value_b must be greater than reference_value
                };

            // Assert that the validation passed
            assert!(
                is_valid_b,
                if (is_positive) E_CONDITION_C else E_CONDITION_D
            );

            // Additional validation against the threshold value if it exists
            if (threshold_value > 0) {
                let is_valid_threshold =
                    if (is_positive) {
                        value_b > threshold_value // For positive condition, value_b must be greater than threshold_value
                    } else {
                        value_b < threshold_value // For negative condition, value_b must be less than threshold_value
                    };

                // Assert that the threshold validation passed
                assert!(is_valid_threshold, E_CONDITION_A);
            }
        };
    }

    #[test_only]
    use std::option;

    #[test]
    fun test_validate_parameters() {
        // Passing Invalid param_a for positive condition
        // This should throw E_CONDITION_B error
        validate_parameters(
            option::some(40), // Less than reference_value (60)
            option::some(50),
            60,
            30,
            true
        );
    }
}

on the main branch, we get:

│  9 │     fun validate_parameters(
│    │         ------------------- In this function in 0xffeeede::Example
│    ·
│ 48 │             assert!(value_b > 0, E_CONDITION_A);
│    │             ^^^^^^ Test was not expected to error, but it aborted with code 2 originating in the module 000000000000000000000000000000000000000000000000000000000ffeeede::Example rooted here

with this PR, we get (correctly):

│  9 │     fun validate_parameters(
│    │         ------------------- In this function in 0xffeeede::Example
│    ·
│ 36 │             assert!(
│    │             ^^^^^^ Test was not expected to error, but it aborted with code 2 originating in the module 000000000000000000000000000000000000000000000000000000000ffeeede::Example rooted here

Key Areas to Review

  • Correctness of the offset propagation
  • It is expected that there are no changes to the generated file format bytecode itself, if you find anything that suggests otherwise, please comment.

Type of Change

  • Bug fix

Which Components or Systems Does This Change Impact?

  • Move Compiler

Copy link

trunk-io bot commented Nov 24, 2024

⏱️ 2h 55m total CI duration on this PR
Slowest 15 Jobs Cumulative Duration Recent Runs
execution-performance / single-node-performance 46m 🟩🟩
rust-move-tests 13m 🟩
rust-move-tests 13m 🟩
rust-move-tests 12m
rust-move-tests 12m 🟩
rust-move-tests 12m 🟩
check-dynamic-deps 11m 🟩🟩🟩🟩🟩
test-target-determinator 10m 🟩🟩
rust-cargo-deny 9m 🟩🟩🟩🟩🟩
execution-performance / test-target-determinator 9m 🟩🟩
check 7m 🟩🟩
rust-doc-tests 6m 🟩
rust-doc-tests 5m 🟩
fetch-last-released-docker-image-tag 3m 🟩🟩
semgrep/ci 2m 🟩🟩🟩🟩🟩

🚨 1 job on the last run was significantly faster/slower than expected

Job Duration vs 7d avg Delta
check-dynamic-deps 3m 1m +136%

settingsfeedbackdocs ⋅ learn more about trunk.io

Copy link
Contributor Author

This stack of pull requests is managed by Graphite. Learn more about stacking.

@vineethk vineethk force-pushed the vk/peephole-location-fix branch from a21b337 to e4f7584 Compare November 24, 2024 18:02
@vineethk vineethk marked this pull request as ready for review November 24, 2024 18:04
Copy link
Contributor

@rahxephon89 rahxephon89 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks

Copy link
Contributor

@brmataptos brmataptos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks ok, though I'd at least add a comment for code clarity.

@vineethk vineethk force-pushed the vk/peephole-location-fix branch from e4f7584 to 33c6f61 Compare November 26, 2024 00:53
@vineethk vineethk enabled auto-merge (squash) November 26, 2024 01:06

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Copy link
Contributor

✅ Forge suite realistic_env_max_load success on 2e87c37dbc23907eac35c6cdcba79f2b4652e4c7

two traffics test: inner traffic : committed: 14204.33 txn/s, latency: 2799.72 ms, (p50: 2700 ms, p70: 2700, p90: 3000 ms, p99: 6600 ms), latency samples: 5400820
two traffics test : committed: 99.89 txn/s, latency: 1815.46 ms, (p50: 1400 ms, p70: 1900, p90: 2200 ms, p99: 12400 ms), latency samples: 1660
Latency breakdown for phase 0: ["MempoolToBlockCreation: max: 2.061, avg: 1.365", "ConsensusProposalToOrdered: max: 0.326, avg: 0.293", "ConsensusOrderedToCommit: max: 0.379, avg: 0.367", "ConsensusProposalToCommit: max: 0.669, avg: 0.660"]
Max non-epoch-change gap was: 0 rounds at version 0 (avg 0.00) [limit 4], 0.81s no progress at version 2374168 (avg 0.20s) [limit 15].
Max epoch-change gap was: 0 rounds at version 0 (avg 0.00) [limit 4], 15.66s no progress at version 2374166 (avg 15.66s) [limit 16].
Test Ok

Copy link
Contributor

✅ Forge suite framework_upgrade success on f436adbe4384d6c5fd296addbb7f52d4be77231b ==> 2e87c37dbc23907eac35c6cdcba79f2b4652e4c7

Compatibility test results for f436adbe4384d6c5fd296addbb7f52d4be77231b ==> 2e87c37dbc23907eac35c6cdcba79f2b4652e4c7 (PR)
Upgrade the nodes to version: 2e87c37dbc23907eac35c6cdcba79f2b4652e4c7
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1383.85 txn/s, submitted: 1386.39 txn/s, failed submission: 2.54 txn/s, expired: 2.54 txn/s, latency: 2204.49 ms, (p50: 1800 ms, p70: 2300, p90: 3300 ms, p99: 5100 ms), latency samples: 119960
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1434.38 txn/s, submitted: 1437.54 txn/s, failed submission: 3.16 txn/s, expired: 3.16 txn/s, latency: 2196.27 ms, (p50: 2100 ms, p70: 2100, p90: 3600 ms, p99: 4500 ms), latency samples: 127060
5. check swarm health
Compatibility test for f436adbe4384d6c5fd296addbb7f52d4be77231b ==> 2e87c37dbc23907eac35c6cdcba79f2b4652e4c7 passed
Upgrade the remaining nodes to version: 2e87c37dbc23907eac35c6cdcba79f2b4652e4c7
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1514.82 txn/s, submitted: 1517.94 txn/s, failed submission: 3.12 txn/s, expired: 3.12 txn/s, latency: 2190.45 ms, (p50: 2100 ms, p70: 2300, p90: 3400 ms, p99: 4200 ms), latency samples: 126140
Test Ok

Copy link
Contributor

✅ Forge suite compat success on f436adbe4384d6c5fd296addbb7f52d4be77231b ==> 2e87c37dbc23907eac35c6cdcba79f2b4652e4c7

Compatibility test results for f436adbe4384d6c5fd296addbb7f52d4be77231b ==> 2e87c37dbc23907eac35c6cdcba79f2b4652e4c7 (PR)
1. Check liveness of validators at old version: f436adbe4384d6c5fd296addbb7f52d4be77231b
compatibility::simple-validator-upgrade::liveness-check : committed: 17228.60 txn/s, latency: 1970.77 ms, (p50: 2100 ms, p70: 2100, p90: 2200 ms, p99: 2500 ms), latency samples: 554520
2. Upgrading first Validator to new version: 2e87c37dbc23907eac35c6cdcba79f2b4652e4c7
compatibility::simple-validator-upgrade::single-validator-upgrading : committed: 7665.05 txn/s, latency: 3759.01 ms, (p50: 4200 ms, p70: 4400, p90: 4500 ms, p99: 4600 ms), latency samples: 144100
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 7034.80 txn/s, latency: 4534.80 ms, (p50: 4500 ms, p70: 4600, p90: 6800 ms, p99: 7000 ms), latency samples: 233640
3. Upgrading rest of first batch to new version: 2e87c37dbc23907eac35c6cdcba79f2b4652e4c7
compatibility::simple-validator-upgrade::half-validator-upgrading : committed: 7717.34 txn/s, latency: 3771.19 ms, (p50: 4200 ms, p70: 4400, p90: 4400 ms, p99: 4700 ms), latency samples: 145260
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 7074.72 txn/s, latency: 4525.80 ms, (p50: 4400 ms, p70: 4600, p90: 6800 ms, p99: 7100 ms), latency samples: 268180
4. upgrading second batch to new version: 2e87c37dbc23907eac35c6cdcba79f2b4652e4c7
compatibility::simple-validator-upgrade::rest-validator-upgrading : committed: 11031.78 txn/s, latency: 2528.09 ms, (p50: 2400 ms, p70: 2800, p90: 4100 ms, p99: 4500 ms), latency samples: 201400
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 10933.26 txn/s, latency: 2830.92 ms, (p50: 2500 ms, p70: 2900, p90: 5100 ms, p99: 6900 ms), latency samples: 357600
5. check swarm health
Compatibility test for f436adbe4384d6c5fd296addbb7f52d4be77231b ==> 2e87c37dbc23907eac35c6cdcba79f2b4652e4c7 passed
Test Ok

@vineethk vineethk merged commit ac0447c into main Nov 26, 2024
48 checks passed
@vineethk vineethk deleted the vk/peephole-location-fix branch November 26, 2024 02:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Bug][compiler-v2] Fix source location mapping for peephole optimized code
3 participants