Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

G_BUILD_VECTOR & G_UNMERGE_VALUES legalization for 32-bit vectors #38

Merged
merged 2 commits into from
Jun 7, 2024

Conversation

ValentijnvdBeek
Copy link
Collaborator

In this merge request, 32-bit vectors are iterated over and added iteratively to a 32-bit number. It does not support 16-bit build vectors since this type cannot occur within a properly legalized program. The intuition for this is as follows:

  1. The legalizer removes code that it considers dead
  2. The legalizer considers code dead when it does a "not safe to move" operation (e.x. move, call or sequential load)
  3. Build Vector does fall under those operations, therefore its result must be used somewhere
  4. Global ISel uses Single Static Assignment, so variable types cannot change and must be defined exactly once
  5. The legalizer works backward from the last instruction to the first instruction
  6. Types must be the same when they're defined and used
  7. If a 16-bit vector reaches the BV, there is another operation uses it and didn't set its type correctly. If that happens, we cannot fix it in the Build Vector since that would cause a conflict between the two types
  8. Ergo such conflicts need to be resolved by legalizing them with the "not safe to move" operations

The opposite logic is done for G_UNMERGE_VALUES that iterates over a 32-bit number, shifts truncates them into separate scalar numbers and finishes once it has extracted all values.

@ValentijnvdBeek ValentijnvdBeek added llvm:globalisel Code that modifies the Global Intruction Selection backend:aie Code that modifies AIE code vectorization Support for vector instructions labels May 17, 2024
@ValentijnvdBeek ValentijnvdBeek self-assigned this May 17, 2024
@ValentijnvdBeek
Copy link
Collaborator Author

To ease the reviewing process and to help prove correctness of the transformation, I have written property tests for the outputted code. Those are tests where you give an invariant which must hold, the test framework then uses a random/all options to see if the invariant holds. If not, it finds the minimal case where the test fails. The main invariant here is that unpack(pack(x)) = x.

Implementation is done using the rapidcheck testing framework, you can run it on a local machine if you'd like.

#include <cstdint>
#include <rapidcheck.h>
#include <vector>
#include <cstdio>

#define G_TRUNC(val) ((val) & 0xFF)
#define G_ANYEXT(val) (static_cast<uint32_t>(val))

std::vector<uint8_t> unpack_vector(uint32_t source) {
    uint32_t offset = 0;
    uint32_t dstcast = source;
    std::vector<uint8_t> abc;

    for (int i = 0; i < 4; i++) {
        if (offset != 0) {
            uint32_t mask = dstcast >> offset;
            uint32_t cut = G_TRUNC(mask);
            abc.push_back(cut);
        } else {
            uint32_t cut = G_TRUNC(dstcast);
            abc.push_back(cut);
        }
        offset += 8;
    }
    return abc;
}

uint32_t pack_vector(const std::vector<uint8_t> abc) {
    uint32_t offset = 0;
    uint32_t dstcast = 0;
    uint32_t acc = 0;

    for (int i = 0; i < 4; i++) {
        uint8_t op = abc.at(i);
        uint32_t destination_operand = G_ANYEXT(op);

        if (offset != 0) {
            uint32_t mask = destination_operand << offset;
            dstcast |= mask;
            acc = dstcast | mask;
        } else {
            acc = dstcast | destination_operand;
        }

        dstcast = acc;
        offset += 8;
    }
    return dstcast;
}

int main() {
    rc::check("pack unsigned numbers", []() {
            const uint32_t value = 0xDEADBEEF;
            const std::vector<uint8_t> split {0xEF, 0xBE, 0xAD, 0xDE};
            const uint32_t packed = pack_vector(split);
            RC_ASSERT(packed == value);
        });

    rc::check("pack unsigned numbers random",
        [](const uint8_t a, const uint8_t b, const uint8_t c, const uint8_t d) {
           std::vector<uint8_t> vec {a, b, c, d};
           const uint32_t packed = pack_vector(vec);
           const uint32_t custom = a | b << 8 | c << 16 | d << 24;
           RC_ASSERT(packed == custom);
        });

    rc::check("unpack unsigned numbers", []() {
            const uint32_t values = 0xDEADBEEF;
            const std::vector<uint8_t> unpacked = unpack_vector(values);
            const std::vector<uint8_t> custom { 0xEF, 0xBE, 0xAD, 0xDE };
            RC_ASSERT(unpacked == custom);
            for (size_t i = 0; i < unpacked.size(); ++i)
                RC_ASSERT(unpacked.at(i) == custom.at(i));
        });

    rc::check("unpack unsigned numbers random",
        [](const uint8_t a, const uint8_t b, const uint8_t c, const uint8_t d) {
            const uint32_t value = a | b << 8 | c << 16 | d << 24;
            const std::vector<uint8_t> packed = unpack_vector(value);
            const std::vector<uint8_t> custom {a, b, c, d};
            RC_ASSERT(packed == custom);
        });

  rc::check("pack and unpack cancel each other out",
        [](const uint8_t a, const uint8_t b, const uint8_t c, const uint8_t d) {
            const std::vector<uint8_t> vec {a, b, c, d};
            std::vector newvec = unpack_vector(pack_vector(vec));
            RC_ASSERT(vec == newvec);
        });

  rc::check("Signedness is preserved whenever values are cast",
        [](const int8_t a, const int8_t b, const int8_t c, const int8_t d) {
            const std::vector<uint8_t> vec { static_cast<uint8_t>(a),
                static_cast<uint8_t>(b), static_cast<uint8_t>(c),
                static_cast<uint8_t>(d) };
           std::vector newvec = unpack_vector(pack_vector(vec));
           RC_ASSERT(vec == newvec);
           RC_ASSERT(static_cast<int8_t>(newvec.at(0)) == a);
           RC_ASSERT(static_cast<int8_t>(newvec.at(1)) == b);
           RC_ASSERT(static_cast<int8_t>(newvec.at(2)) == c);
           RC_ASSERT(static_cast<int8_t>(newvec.at(3)) == d);
        });
}

@ValentijnvdBeek
Copy link
Collaborator Author

This code has already been reviewed off-repo before the move to Github. Some of the issues there an incorrect opcode was used (ANYEXT instead of ZEXT), lack of tests for negative numbers and the lack of tests where operands are from registers.

@ValentijnvdBeek ValentijnvdBeek force-pushed the vvandebe.32bit.merge.unmerge branch 2 times, most recently from daf156f to a6556e8 Compare May 31, 2024 08:25
@@ -630,6 +671,9 @@ bool AIELegalizerInfo::legalizeG_UNMERGE_VALUES(LegalizerHelper &Helper,
assert(CurrentTy.isScalar() &&
"this operation is only supported for scalar types");

if (LastTy.getSizeInBits() == 32)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Move outside of the for loop, below the assert

Copy link
Collaborator Author

@ValentijnvdBeek ValentijnvdBeek Jun 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the review, that makes sense. I will update it, thank you Konstantin.

@ValentijnvdBeek ValentijnvdBeek merged commit 07926c9 into aie-public Jun 7, 2024
8 checks passed
@konstantinschwarz konstantinschwarz deleted the vvandebe.32bit.merge.unmerge branch June 7, 2024 08:15
stephenneuendorffer pushed a commit that referenced this pull request Jul 19, 2024
…(#80904)"

This reverts commit b1ac052.

This commit breaks coroutine splitting for non-swift calling convention
functions. In this example:

```ll
; ModuleID = 'repro.ll'
source_filename = "stdlib/test/runtime/test_llcl.mojo"
target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-i128:128-f80:128-n8:16:32:64-S128"
target triple = "x86_64-unknown-linux-gnu"

@0 = internal constant { i32, i32 } { i32 trunc (i64 sub (i64 ptrtoint (ptr @craSH to i64), i64 ptrtoint (ptr getelementptr inbounds ({ i32, i32 }, ptr @0, i32 0, i32 1) to i64)) to i32), i32 64 }

define dso_local void @af_suspend_fn(ptr %0, i64 %1, ptr %2) #0 {
  ret void
}

define dso_local void @craSH(ptr %0) #0 {
  %2 = call token @llvm.coro.id.async(i32 64, i32 8, i32 0, ptr @0)
  %3 = call ptr @llvm.coro.begin(token %2, ptr null)
  %4 = getelementptr inbounds { ptr, { ptr, ptr }, i64, { ptr, i1 }, i64, i64 }, ptr poison, i32 0, i32 0
  %5 = call ptr @llvm.coro.async.resume()
  store ptr %5, ptr %4, align 8
  %6 = call { ptr, ptr, ptr } (i32, ptr, ptr, ...) @llvm.coro.suspend.async.sl_p0p0p0s(i32 0, ptr %5, ptr @ctxt_proj_fn, ptr @af_suspend_fn, ptr poison, i64 -1, ptr poison)
  ret void
}

define dso_local ptr @ctxt_proj_fn(ptr %0) #0 {
  ret ptr %0
}

; Function Attrs: nomerge nounwind
declare { ptr, ptr, ptr } @llvm.coro.suspend.async.sl_p0p0p0s(i32, ptr, ptr, ...) #1

; Function Attrs: nounwind
declare token @llvm.coro.id.async(i32, i32, i32, ptr) #2

; Function Attrs: nounwind
declare ptr @llvm.coro.begin(token, ptr writeonly) #2

; Function Attrs: nomerge nounwind
declare ptr @llvm.coro.async.resume() #1

attributes #0 = { "target-features"="+adx,+aes,+avx,+avx2,+bmi,+bmi2,+clflushopt,+clwb,+clzero,+crc32,+cx16,+cx8,+f16c,+fma,+fsgsbase,+fxsr,+invpcid,+lzcnt,+mmx,+movbe,+mwaitx,+pclmul,+pku,+popcnt,+prfchw,+rdpid,+rdpru,+rdrnd,+rdseed,+sahf,+sha,+sse,+sse2,+sse3,+sse4.1,+sse4.2,+sse4a,+ssse3,+vaes,+vpclmulqdq,+wbnoinvd,+x87,+xsave,+xsavec,+xsaveopt,+xsaves" }
attributes #1 = { nomerge nounwind }
attributes #2 = { nounwind }
```

This verifier crashes after the `coro-split` pass with

```
cannot guarantee tail call due to mismatched parameter counts
  musttail call void @af_suspend_fn(ptr poison, i64 -1, ptr poison)
LLVM ERROR: Broken function
PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace.
Stack dump:
0.      Program arguments: opt ../../../reduced.ll -O0
 #0 0x00007f1d89645c0e __interceptor_backtrace.part.0 /build/gcc-11-XeT9lY/gcc-11-11.4.0/build/x86_64-linux-gnu/libsanitizer/asan/../../../../src/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:4193:28
 #1 0x0000556d94d254f7 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) /home/ubuntu/modular/third-party/llvm-project/llvm/lib/Support/Unix/Signals.inc:723:22
 #2 0x0000556d94d19a2f llvm::sys::RunSignalHandlers() /home/ubuntu/modular/third-party/llvm-project/llvm/lib/Support/Signals.cpp:105:20
 #3 0x0000556d94d1aa42 SignalHandler(int) /home/ubuntu/modular/third-party/llvm-project/llvm/lib/Support/Unix/Signals.inc:371:36
 #4 0x00007f1d88e42520 (/lib/x86_64-linux-gnu/libc.so.6+0x42520)
 #5 0x00007f1d88e969fc __pthread_kill_implementation ./nptl/pthread_kill.c:44:76
 #6 0x00007f1d88e969fc __pthread_kill_internal ./nptl/pthread_kill.c:78:10
 #7 0x00007f1d88e969fc pthread_kill ./nptl/pthread_kill.c:89:10
 #8 0x00007f1d88e42476 gsignal ./signal/../sysdeps/posix/raise.c:27:6
 #9 0x00007f1d88e287f3 abort ./stdlib/abort.c:81:7
 #10 0x0000556d8944be01 std::vector<llvm::json::Value, std::allocator<llvm::json::Value>>::size() const /usr/include/c++/11/bits/stl_vector.h:919:40
 #11 0x0000556d8944be01 bool std::operator==<llvm::json::Value, std::allocator<llvm::json::Value>>(std::vector<llvm::json::Value, std::allocator<llvm::json::Value>> const&, std::vector<llvm::json::Value, std::allocator<llvm::json::Value>> const&) /usr/include/c++/11/bits/stl_vector.h:1893:23
 #12 0x0000556d8944be01 llvm::json::operator==(llvm::json::Array const&, llvm::json::Array const&) /home/ubuntu/modular/third-party/llvm-project/llvm/include/llvm/Support/JSON.h:572:69
 #13 0x0000556d8944be01 llvm::json::operator==(llvm::json::Value const&, llvm::json::Value const&) (.cold) /home/ubuntu/modular/third-party/llvm-project/llvm/lib/Support/JSON.cpp:204:28
 #14 0x0000556d949ed2bd llvm::report_fatal_error(char const*, bool) /home/ubuntu/modular/third-party/llvm-project/llvm/lib/Support/ErrorHandling.cpp:82:70
 #15 0x0000556d8e37e876 llvm::SmallVectorBase<unsigned int>::size() const /home/ubuntu/modular/third-party/llvm-project/llvm/include/llvm/ADT/SmallVector.h:91:32
 #16 0x0000556d8e37e876 llvm::SmallVectorTemplateCommon<llvm::DiagnosticInfoOptimizationBase::Argument, void>::end() /home/ubuntu/modular/third-party/llvm-project/llvm/include/llvm/ADT/SmallVector.h:282:41
 #17 0x0000556d8e37e876 llvm::SmallVector<llvm::DiagnosticInfoOptimizationBase::Argument, 4u>::~SmallVector() /home/ubuntu/modular/third-party/llvm-project/llvm/include/llvm/ADT/SmallVector.h:1215:24
 #18 0x0000556d8e37e876 llvm::DiagnosticInfoOptimizationBase::~DiagnosticInfoOptimizationBase() /home/ubuntu/modular/third-party/llvm-project/llvm/include/llvm/IR/DiagnosticInfo.h:413:7
 #19 0x0000556d8e37e876 llvm::DiagnosticInfoIROptimization::~DiagnosticInfoIROptimization() /home/ubuntu/modular/third-party/llvm-project/llvm/include/llvm/IR/DiagnosticInfo.h:622:7
 #20 0x0000556d8e37e876 llvm::OptimizationRemark::~OptimizationRemark() /home/ubuntu/modular/third-party/llvm-project/llvm/include/llvm/IR/DiagnosticInfo.h:689:7
 #21 0x0000556d8e37e876 operator() /home/ubuntu/modular/third-party/llvm-project/llvm/lib/Transforms/Coroutines/CoroSplit.cpp:2213:14
 #22 0x0000556d8e37e876 emit<llvm::CoroSplitPass::run(llvm::LazyCallGraph::SCC&, llvm::CGSCCAnalysisManager&, llvm::LazyCallGraph&, llvm::CGSCCUpdateResult&)::<lambda()> > /home/ubuntu/modular/third-party/llvm-project/llvm/include/llvm/Analysis/OptimizationRemarkEmitter.h:83:12
 #23 0x0000556d8e37e876 llvm::CoroSplitPass::run(llvm::LazyCallGraph::SCC&, llvm::AnalysisManager<llvm::LazyCallGraph::SCC, llvm::LazyCallGraph&>&, llvm::LazyCallGraph&, llvm::CGSCCUpdateResult&) /home/ubuntu/modular/third-party/llvm-project/llvm/lib/Transforms/Coroutines/CoroSplit.cpp:2212:13
 #24 0x0000556d8c36ecb1 llvm::detail::PassModel<llvm::LazyCallGraph::SCC, llvm::CoroSplitPass, llvm::AnalysisManager<llvm::LazyCallGraph::SCC, llvm::LazyCallGraph&>, llvm::LazyCallGraph&, llvm::CGSCCUpdateResult&>::run(llvm::LazyCallGraph::SCC&, llvm::AnalysisManager<llvm::LazyCallGraph::SCC, llvm::LazyCallGraph&>&, llvm::LazyCallGraph&, llvm::CGSCCUpdateResult&) /home/ubuntu/modular/third-party/llvm-project/llvm/include/llvm/IR/PassManagerInternal.h:91:3
 #25 0x0000556d91c1a84f llvm::PassManager<llvm::LazyCallGraph::SCC, llvm::AnalysisManager<llvm::LazyCallGraph::SCC, llvm::LazyCallGraph&>, llvm::LazyCallGraph&, llvm::CGSCCUpdateResult&>::run(llvm::LazyCallGraph::SCC&, llvm::AnalysisManager<llvm::LazyCallGraph::SCC, llvm::LazyCallGraph&>&, llvm::LazyCallGraph&, llvm::CGSCCUpdateResult&) /home/ubuntu/modular/third-party/llvm-project/llvm/lib/Analysis/CGSCCPassManager.cpp:90:12
 #26 0x0000556d8c3690d1 llvm::detail::PassModel<llvm::LazyCallGraph::SCC, llvm::PassManager<llvm::LazyCallGraph::SCC, llvm::AnalysisManager<llvm::LazyCallGraph::SCC, llvm::LazyCallGraph&>, llvm::LazyCallGraph&, llvm::CGSCCUpdateResult&>, llvm::AnalysisManager<llvm::LazyCallGraph::SCC, llvm::LazyCallGraph&>, llvm::LazyCallGraph&, llvm::CGSCCUpdateResult&>::run(llvm::LazyCallGraph::SCC&, llvm::AnalysisManager<llvm::LazyCallGraph::SCC, llvm::LazyCallGraph&>&, llvm::LazyCallGraph&, llvm::CGSCCUpdateResult&) /home/ubuntu/modular/third-party/llvm-project/llvm/include/llvm/IR/PassManagerInternal.h:91:3
 #27 0x0000556d91c2162d llvm::ModuleToPostOrderCGSCCPassAdaptor::run(llvm::Module&, llvm::AnalysisManager<llvm::Module>&) /home/ubuntu/modular/third-party/llvm-project/llvm/lib/Analysis/CGSCCPassManager.cpp:278:18
 #28 0x0000556d8c369035 llvm::detail::PassModel<llvm::Module, llvm::ModuleToPostOrderCGSCCPassAdaptor, llvm::AnalysisManager<llvm::Module>>::run(llvm::Module&, llvm::AnalysisManager<llvm::Module>&) /home/ubuntu/modular/third-party/llvm-project/llvm/include/llvm/IR/PassManagerInternal.h:91:3
 #29 0x0000556d9457abc5 llvm::PassManager<llvm::Module, llvm::AnalysisManager<llvm::Module>>::run(llvm::Module&, llvm::AnalysisManager<llvm::Module>&) /home/ubuntu/modular/third-party/llvm-project/llvm/include/llvm/IR/PassManager.h:247:20
 #30 0x0000556d8e30979e llvm::CoroConditionalWrapper::run(llvm::Module&, llvm::AnalysisManager<llvm::Module>&) /home/ubuntu/modular/third-party/llvm-project/llvm/lib/Transforms/Coroutines/CoroConditionalWrapper.cpp:19:74
 #31 0x0000556d8c365755 llvm::detail::PassModel<llvm::Module, llvm::CoroConditionalWrapper, llvm::AnalysisManager<llvm::Module>>::run(llvm::Module&, llvm::AnalysisManager<llvm::Module>&) /home/ubuntu/modular/third-party/llvm-project/llvm/include/llvm/IR/PassManagerInternal.h:91:3
 #32 0x0000556d9457abc5 llvm::PassManager<llvm::Module, llvm::AnalysisManager<llvm::Module>>::run(llvm::Module&, llvm::AnalysisManager<llvm::Module>&) /home/ubuntu/modular/third-party/llvm-project/llvm/include/llvm/IR/PassManager.h:247:20
 #33 0x0000556d89818556 llvm::SmallPtrSetImplBase::isSmall() const /home/ubuntu/modular/third-party/llvm-project/llvm/include/llvm/ADT/SmallPtrSet.h:196:33
 #34 0x0000556d89818556 llvm::SmallPtrSetImplBase::~SmallPtrSetImplBase() /home/ubuntu/modular/third-party/llvm-project/llvm/include/llvm/ADT/SmallPtrSet.h:84:17
 #35 0x0000556d89818556 llvm::SmallPtrSetImpl<llvm::AnalysisKey*>::~SmallPtrSetImpl() /home/ubuntu/modular/third-party/llvm-project/llvm/include/llvm/ADT/SmallPtrSet.h:321:7
 #36 0x0000556d89818556 llvm::SmallPtrSet<llvm::AnalysisKey*, 2u>::~SmallPtrSet() /home/ubuntu/modular/third-party/llvm-project/llvm/include/llvm/ADT/SmallPtrSet.h:427:7
 #37 0x0000556d89818556 llvm::PreservedAnalyses::~PreservedAnalyses() /home/ubuntu/modular/third-party/llvm-project/llvm/include/llvm/IR/Analysis.h:109:7
 #38 0x0000556d89818556 llvm::runPassPipeline(llvm::StringRef, llvm::Module&, llvm::TargetMachine*, llvm::TargetLibraryInfoImpl*, llvm::ToolOutputFile*, llvm::ToolOutputFile*, llvm::ToolOutputFile*, llvm::StringRef, llvm::ArrayRef<llvm::PassPlugin>, llvm::ArrayRef<std::function<void (llvm::PassBuilder&)>>, llvm::opt_tool::OutputKind, llvm::opt_tool::VerifierKind, bool, bool, bool, bool, bool, bool, bool) /home/ubuntu/modular/third-party/llvm-project/llvm/tools/opt/NewPMDriver.cpp:532:10
 #39 0x0000556d897e3939 optMain /home/ubuntu/modular/third-party/llvm-project/llvm/tools/opt/optdriver.cpp:737:27
 #40 0x0000556d89455461 main /home/ubuntu/modular/third-party/llvm-project/llvm/tools/opt/opt.cpp:25:33
 #41 0x00007f1d88e29d90 __libc_start_call_main ./csu/../sysdeps/nptl/libc_start_call_main.h:58:16
 #42 0x00007f1d88e29e40 call_init ./csu/../csu/libc-start.c:128:20
 #43 0x00007f1d88e29e40 __libc_start_main ./csu/../csu/libc-start.c:379:5
 #44 0x0000556d897b6335 _start (/home/ubuntu/modular/.derived/third-party/llvm-project/build-relwithdebinfo-asan/bin/opt+0x150c335)
Aborted (core dumped)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backend:aie Code that modifies AIE code llvm:globalisel Code that modifies the Global Intruction Selection vectorization Support for vector instructions
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants