Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

aot/jit native stack bound check improvement #2244

Merged
merged 107 commits into from
Jun 21, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
107 commits
Select commit Hold shift + click to select a range
991834a
aot_llvm.c: create a global to store stack sizes
yamt May 23, 2023
11910cf
aot: update stack_sizes (with a dummy value for now)
yamt May 23, 2023
32d2213
aot_add_llvm_func: extract llvm function creation logic
yamt May 23, 2023
62ec1b5
aot/llvm-jit: generate "precheck" function wrappers
yamt May 23, 2023
0cbd140
aot: debug
yamt May 23, 2023
22b5d14
mark the function body noinline
yamt May 24, 2023
699913d
aot/jit: separate aot_add_precheck_function
yamt May 25, 2023
f08429e
record stack_sizes global in in comp ctx
yamt May 25, 2023
3a5ab29
aot_add_precheck_function: add some IRs (WIP)
yamt May 25, 2023
f206aea
wip
yamt May 26, 2023
07cbe04
wip
yamt May 26, 2023
f8a75f6
revert a few (wrong) constifications
yamt May 29, 2023
f8b6be9
aot: read stack usage file
yamt May 29, 2023
d373c01
fix a few inverted conditions
yamt May 29, 2023
c5d26e4
clang-format
yamt May 30, 2023
4e0641b
call correct func
yamt May 30, 2023
4a7ed3e
make the precheck func noinline as well
yamt May 30, 2023
07114a8
debug code
yamt May 30, 2023
52dea3d
retire create_native_stack_bound_from_exec_env
yamt May 30, 2023
0a56499
start making the new thing conditional on enable_stack_bound_check
yamt May 30, 2023
7458c8e
retire old native stack check logic
yamt May 30, 2023
a22a086
create a temporary stack usage file automatically
yamt May 30, 2023
4ffa025
enable module verification for jit as well for now
yamt May 31, 2023
0d56e76
precheck function numbering
yamt May 31, 2023
78982bc
debug log
yamt May 31, 2023
4f33a13
g/c last_alloca
yamt May 31, 2023
aee2c96
debug log
yamt May 31, 2023
b531c85
aot_add_precheck_function: fix a type mismatch
yamt May 31, 2023
01099c1
experiment to query stack sizes for jit
yamt May 15, 2023
078a302
clang-format
yamt May 31, 2023
8e193df
aot_add_llvm_func: don't bother to use internal linkage for jit
yamt May 31, 2023
9f92bbd
use another prefix for wrapped functions
yamt Jun 1, 2023
0671205
jit: ensure precheck wrapper and wrapped func are compiled together
yamt Jun 1, 2023
d959f35
disable jit compilation threads for now
yamt Jun 1, 2023
2b817f2
restructure a bit
yamt Jun 1, 2023
2732e67
jit: fill stack_sizes
yamt Jun 1, 2023
87fdb6c
aot_add_precheck_function: implement 32-bit case
yamt Jun 1, 2023
9f40049
read_stack_usage_file: add a few sanity checks
yamt Jun 1, 2023
73e1d17
comment
yamt Jun 1, 2023
d27710d
aot_resolve_stack_sizes: relax checks a bit
yamt Jun 1, 2023
a745de0
Revert "debug code"
yamt Jun 1, 2023
8592e3d
Revert "disable jit compilation threads for now"
yamt Jun 1, 2023
b4f55da
clang-format
yamt Jun 5, 2023
a91419d
aot: fix enable_stack_bound_check with external llc
yamt Jun 6, 2023
49962ac
re-implement native stack estimation
yamt Jun 6, 2023
d4d43e7
calculate new sp first
yamt Jun 6, 2023
c053c33
Revert "Revert "debug code""
yamt Jun 6, 2023
d0e11fa
underflow check
yamt Jun 6, 2023
78a51c2
clang-format
yamt Jun 6, 2023
91f0e7b
fix a type mismatch
yamt Jun 6, 2023
bd5a26b
aot_runtime: debug log to print stack size
yamt Jun 6, 2023
9a23ca5
Revert "aot_runtime: debug log to print stack size"
yamt Jun 6, 2023
35061ec
Revert "Revert "Revert "debug code"""
yamt Jun 6, 2023
2caa19e
revert a debug log
yamt Jun 6, 2023
11735f0
aot: enable_stack_estimation needs stack_sizes as well
yamt Jun 6, 2023
81fb815
jit: enable_stack_estimation needs stack_sizes as well
yamt Jun 6, 2023
ff5ff8a
implement stack estimation for function calls
yamt Jun 7, 2023
90df3ca
add some error checks
yamt Jun 7, 2023
92d9b1b
fix an assertion
yamt Jun 8, 2023
855ba8c
add a comment
yamt Jun 8, 2023
bdcb96c
interpreter: fix app heap __heap_base adjustment
yamt Jun 8, 2023
e616c6a
Revert "interpreter: fix app heap __heap_base adjustment"
yamt Jun 8, 2023
5038166
aot: fix WASM_ENABLE_DEBUG_AOT
yamt Jun 8, 2023
3080061
Revert "debug log"
yamt Jun 8, 2023
2d3a0d7
Revert "enable module verification for jit as well for now"
yamt Jun 8, 2023
ff0621a
remove stale XXX comments
yamt Jun 8, 2023
9420e85
comment
yamt Jun 9, 2023
66f9815
comment
yamt Jun 9, 2023
a113803
remove a stale comment
yamt Jun 9, 2023
6259e2e
comment
yamt Jun 9, 2023
c8d05f8
copyright comment
yamt Jun 9, 2023
be166e8
reduce code dup
yamt Jun 9, 2023
c892b8f
disable underflow check for comparison
yamt Jun 13, 2023
c605057
Revert "disable underflow check for comparison"
yamt Jun 13, 2023
cd5880e
use a constant threshold to see load-from-table overhead
yamt Jun 13, 2023
7eb3246
Revert "use a constant threshold to see load-from-table overhead"
yamt Jun 13, 2023
57e128b
aot_move_file: bump the buffer size
yamt Jun 15, 2023
3894278
aot_move_file: remove the incomplete file on error
yamt Jun 15, 2023
24f90e4
remove redundant memcpy
yamt Jun 15, 2023
8000285
comment fixes
yamt Jun 15, 2023
1664ae8
use align_uint
yamt Jun 15, 2023
a5250d4
style
yamt Jun 15, 2023
d45df4c
read_stack_usage_file: error on unexpected number of precheck functions
yamt Jun 15, 2023
7ace9bd
remove extra LLVMDisposeSectionIterator call
yamt Jun 15, 2023
cf7a705
aot_move_file: fix swapped arguments
yamt Jun 15, 2023
c863450
add a few error log
yamt Jun 15, 2023
7362a1d
read_stack_usage_file: add more error check
yamt Jun 15, 2023
0317e24
aot_resolve_stack_sizes: don't forget to add the offset
yamt Jun 15, 2023
e41ed73
aot_emit_aot_file.c: fix segv in some cases
yamt Jun 15, 2023
3fe3a32
use uint32 instead of uint32_t in a few places
yamt Jun 15, 2023
3f87e8f
Add LLVMSetTailCallKind
yamt Jun 16, 2023
b7babbd
separate aot_estimate_stack_usage_for_function_call into two functions
yamt Jun 19, 2023
7487102
aot_estimate_stack_usage_for_function_call: tweak for xtensa
yamt Jun 19, 2023
34fd5b1
aot_add_precheck_function: use musttail where it's safe
yamt Jun 19, 2023
6475975
export aot_estimate_stack_usage_for_function_call
yamt Jun 19, 2023
d082d69
move aot_estimate_stack_usage_for_function_call to aot_llvm.c
yamt Jun 19, 2023
2575b0c
aot_resolve_stack_sizes: fix the estimation w/o tail call optimization
yamt Jun 19, 2023
b9da93f
add a warning
yamt Jun 19, 2023
f9c32bb
jit_stack_size_callback: sync with the aot version
yamt Jun 19, 2023
9976da8
aot_emit_aot_file.c: downgrade a few scary warnings
yamt Jun 19, 2023
04ffbc4
a comment
yamt Jun 19, 2023
36bd181
whitespace
yamt Jun 19, 2023
18ef14e
rename aot_func2# -> aot_func_internal#
yamt Jun 20, 2023
699ebaf
use bh_memcpy_s
yamt Jun 20, 2023
090552c
remove an extra LLVMOrcDisposeThreadSafeModule call
yamt Jun 20, 2023
3c0d82d
use uint64 instead of uint64_t
yamt Jun 20, 2023
a338714
add an assertion and comments
yamt Jun 20, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions core/iwasm/compilation/aot.h
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,15 @@ extern "C" {
#define AOT_FUNC_PREFIX "aot_func#"
#endif

#ifndef AOT_FUNC_INTERNAL_PREFIX
#define AOT_FUNC_INTERNAL_PREFIX "aot_func_internal#"
#endif

#ifndef AOT_STACK_SIZES_NAME
#define AOT_STACK_SIZES_NAME "aot_stack_sizes"
#endif
extern const char *aot_stack_sizes_name;

typedef InitializerExpression AOTInitExpr;
typedef WASMType AOTFuncType;
typedef WASMExport AOTExport;
Expand Down
97 changes: 95 additions & 2 deletions core/iwasm/compilation/aot_compiler.c
Original file line number Diff line number Diff line change
Expand Up @@ -2761,6 +2761,16 @@ aot_compile_wasm(AOTCompContext *comp_ctx)
aot_handle_llvm_errmsg("failed to addIRModule", err);
return false;
}

if (comp_ctx->stack_sizes != NULL) {
LLVMOrcJITTargetAddress addr;
if ((err = LLVMOrcLLLazyJITLookup(comp_ctx->orc_jit, &addr,
aot_stack_sizes_name))) {
aot_handle_llvm_errmsg("failed to look up stack_sizes", err);
return false;
}
comp_ctx->jit_stack_sizes = (uint32 *)addr;
}
}

return true;
Expand Down Expand Up @@ -2815,6 +2825,55 @@ aot_emit_llvm_file(AOTCompContext *comp_ctx, const char *file_name)
return true;
}

static bool
aot_move_file(const char *dest, const char *src)
{
FILE *dfp = fopen(dest, "w");
FILE *sfp = fopen(src, "r");
size_t rsz;
char buf[128];
bool success = false;
wenyongh marked this conversation as resolved.
Show resolved Hide resolved

if (dfp == NULL || sfp == NULL) {
LOG_DEBUG("open error %s %s", dest, src);
goto fail;
}
do {
rsz = fread(buf, 1, sizeof(buf), sfp);
if (rsz > 0) {
size_t wsz = fwrite(buf, 1, rsz, dfp);
if (wsz < rsz) {
LOG_DEBUG("write error");
goto fail;
}
}
if (rsz < sizeof(buf)) {
wenyongh marked this conversation as resolved.
Show resolved Hide resolved
if (ferror(sfp)) {
LOG_DEBUG("read error");
goto fail;
}
}
} while (rsz > 0);
success = true;
fail:
if (dfp != NULL) {
if (fclose(dfp)) {
LOG_DEBUG("close error");
success = false;
}
wenyongh marked this conversation as resolved.
Show resolved Hide resolved
if (!success) {
(void)unlink(dest);
}
}
if (sfp != NULL) {
(void)fclose(sfp);
}
if (success) {
(void)unlink(src);
}
return success;
}

bool
aot_emit_object_file(AOTCompContext *comp_ctx, char *file_name)
{
Expand All @@ -2830,7 +2889,25 @@ aot_emit_object_file(AOTCompContext *comp_ctx, char *file_name)
int ret;

if (comp_ctx->external_llc_compiler) {
const char *stack_usage_flag = "";
char bc_file_name[64];
char su_file_name[65]; /* See the comment below */

if (comp_ctx->stack_usage_file != NULL) {
/*
* Note: we know the caller uses 64 byte buffer for
* file_name. It will get 1 byte longer because we
* replace ".o" with ".su".
*/
size_t len = strlen(file_name);
bh_assert(len + 1 <= sizeof(su_file_name));
bh_assert(len > 3);
bh_assert(file_name[len - 2] == '.');
bh_assert(file_name[len - 1] == 'o');
snprintf(su_file_name, sizeof(su_file_name), "%.*s.su",
(int)(len - 2), file_name);
wenyongh marked this conversation as resolved.
Show resolved Hide resolved
stack_usage_flag = " -fstack-usage";
}

if (!aot_generate_tempfile_name("wamrc-bc", "bc", bc_file_name,
sizeof(bc_file_name))) {
Expand All @@ -2842,8 +2919,8 @@ aot_emit_object_file(AOTCompContext *comp_ctx, char *file_name)
return false;
}

snprintf(cmd, sizeof(cmd), "%s %s -o %s %s",
comp_ctx->external_llc_compiler,
snprintf(cmd, sizeof(cmd), "%s%s %s -o %s %s",
comp_ctx->external_llc_compiler, stack_usage_flag,
comp_ctx->llc_compiler_flags ? comp_ctx->llc_compiler_flags
: "-O3 -c",
file_name, bc_file_name);
Expand All @@ -2858,6 +2935,22 @@ aot_emit_object_file(AOTCompContext *comp_ctx, char *file_name)
"with external LLC compiler.");
return false;
}
if (comp_ctx->stack_usage_file != NULL) {
/*
* move the temporary .su file to the specified location.
*
* Note: the former is automatimally inferred from the output
* filename (file_name here) by clang.
*
* Note: the latter might be user-specified.
* (wamrc --stack-usage=<file>)
*/
if (!aot_move_file(comp_ctx->stack_usage_file, su_file_name)) {
aot_set_last_error("failed to move su file.");
(void)unlink(su_file_name);
return false;
}
}
}
else if (comp_ctx->external_asm_compiler) {
char asm_file_name[64];
Expand Down
Loading