From 8457f7f3c10151b433b4101133f24bca11729c4d Mon Sep 17 00:00:00 2001 From: Morgan Date: Mon, 18 Dec 2023 23:34:16 +0100 Subject: [PATCH] feat(gnovm): improved native bindings (#859) This PR provides an initial transition and groundwork to move from native bindings as "package injections", whereby an injector function is called after package parsing and loading to provide built-in natives, to the new design described in #814. ## Status The PR is currently ready for review. - [x] Fix all tests and ensure precompiling has no issues - [x] Find out if the errors that arise when running the tests with `DEBUG=1` are bugs introduced by this PR or not - [x] Create temporary exceptions for linked types (like `std.{Coins,Address,Realm}`), so that all normal native bindings run exclusively on the new system, awaiting `//gno:link` (see #814). - [x] Finding a permanent place for ExecContext (see comment on `gnovm/stdlibs/context.go`); suggestions welcome (tm2/vm?) - [x] Include code generation in Makefiles and add a CI check to make sure PRs do generate code when necessary - [x] And some unit tests for the new functionality / potential edge cases. - [x] make sure all injected functions have `// injected` as a comment Next steps, after merging this PR, include: - Supporting precompilation of native functions by linking directly to the Go source in stdlibs/, removing the necessity of `stdshim`. - Supporting the test-only tests/stdlibs/ directory in `gno doc`; as well as documenting native functions explicitly by adding the `// injected` comment after their declaration. - Making all of the native-only packages defined in tests/imports.go native bindings; so that the special packages and native values are restricted only to ImportModeNativePreferred, while everything else uses documentable native bindings also for tests. ## Implementation Summary A new API has been added to `gnolang.Store`: `NativeStore`. This is a simple function which resolves a package and a name of a function to a native function.[^1] This is used to resolve native bindings first and foremost in `preprocess.go`, which also adds information about the native binding in `NativePkg` and `NativeName`, two new fields in `*gnolang.FuncValue`. The purpose of `NativePkg` and `NativeName` is to 1. enable assigning native functions to other function types, including function parameters or func type variables and 2. being able to easily and effectively persist the FuncValue during realm de/serialization. This way, `op_call.go` can recover the value of `nativeBody` if it is nil. On the other side, examples of the new stdlibs can be seen in the `gnovm/stdlibs` directory, where everything except a few functions in package `std` have been ported to the new system. The code generator in `misc/genstd` parses the AST of both the Go and Gno files, and ultimately generates `native.go`, which contains the porting code from the Gno function call to Go and back. To support the test contexts, which needs to have different function implementation or entirely new functions, an additional stdlibs directory was added to the `tests` directory. This is still code generated with the same generator and follows the same structure. When stdlibs are loaded for tests, the code in tests/stdlibs/, if it exists, is loaded with "higher priority" compared to normal `stdlibs` code.[^2] ### Suggested order for reviewing (and some notes) 1. `pkg/gnolang` for the core of the changes, starting from `values.go`, `preprocess.go`, `op_call.go`, `store.go`, `realm.go` which show how NativePkg, NativeName and nativeBody interact * `gnovm/pkg/gnolang/op_eval.go` adds the type of the evaluated expression, as I think it's useful to understand in what terms something is being evaluated; specifically I added it to better distinguish between `DEBUG: EVAL: (*gnolang.CallExpr) std.TestCurrentRealm()` and `DEBUG: EVAL: (*gnolang.SelectorExpr) std.TestCurrentRealm`. I think it's useful, but we can remove it if we prefer to keep it simpler. * `gnovm/pkg/gnolang/op_expressions.go` adds a new debug print pretending we're popping and pushing the value stack. This is to make clear what's happening in the debug logs (we're really swapping the value at the pointer), and I've made it `v[S]` so it's clear that the log is not coming from `PushValue/PopValue`. * I removed a goto-loop in `gnovm/pkg/gnolang/values.go` because I don't see it as beneficial when it's essentially just doing a for loop; if the concern was inlining, even with goto the function is too "costly" for the go compiler to inline. 2. `misc/genstd` contains the code generator and the template it uses to generate the files, ie. `gnovm/stdlibs/native.go` and `gnovm/tests/stdlibs/native.go`. 3. `stdlibs` and `tests/stdlibs` changes show the new stdlib directories, which genstd uses to generate code. The `tests/stdlibs` is available only in test environments, and there are also some overriding definitions to those in normal stdlibs. * `gnovm/stdlibs/encoding/base64/base64.gno` removes a check on `strconv.IntSize`. This is because `strconv.IntSize` is dependent on the system and thus is non-deterministic behaviour on the Gnovm. I don't know if `int` in Gno is == Go's `int` or it is precisely defined to be 64-bits; in any case I don't think it should be dependent on the system as this wouldn't work in a blockchain scenario. 4. Changes to `tm2` and `gnovm/tests/imports.go` mostly relate to wiring up the new NativeStore to the gnovm where it's used. 5. Changes to `examples` and `tests/files` (except for float5) are all updates to golden tests.
Checklists...

## Contributors Checklist - [x] Added new tests, or not needed, or not feasible - [x] Provided an example (e.g. screenshot) to aid review or the PR is self-explanatory - [x] Updated the official documentation or not needed - [x] No breaking changes were made, or a `BREAKING CHANGE: xxx` message was included in the description - [x] Added references to related issues and PRs - [x] Provided any useful hints for running manual tests ## Maintainers Checklist - [ ] Checked that the author followed the guidelines in `CONTRIBUTING.md` - [ ] Checked the conventional-commit (especially PR title and verb, presence of `BREAKING CHANGE:` in the body) - [ ] Ensured that this PR is not a significant change or confirmed that the review/consideration process was appropriate for the change

[^1]: Currently, the only native bindings that are supported with the new system are top-level functions. I think we can keep it this way in the spirit of [KISS](https://en.wikipedia.org/wiki/KISS_principle); maybe think of extending it if we find appropriate usage. [^2]: See gnolang.RunMemPackageWithOverrides --------- Co-authored-by: Thomas Bruyelle Co-authored-by: jaekwon --- .gitattributes | 2 + .github/codecov.yml | 3 + .github/workflows/codegen.yml | 36 + .github/workflows/lint.yml | 90 +++ .github/workflows/misc.yml | 110 ++- Makefile | 1 + docs/reference/standard-library.md | 226 ++++-- examples/gno.land/p/demo/avl/z_0_filetest.gno | 4 + examples/gno.land/p/demo/avl/z_1_filetest.gno | 4 + .../gno.land/r/demo/boards/z_4_filetest.gno | 8 +- gno.land/pkg/gnoland/app.go | 2 +- gno.land/pkg/sdk/vm/builtins.go | 39 +- gnovm/Makefile | 9 + gnovm/cmd/gno/test.go | 36 +- .../gno/testdata/gno_test/realm_correct.txtar | 2 + .../testdata/gno_test/realm_incorrect.txtar | 2 +- .../gno/testdata/gno_test/realm_sync.txtar | 2 + gnovm/pkg/gnolang/gno_test.go | 4 +- gnovm/pkg/gnolang/go2gno.go | 15 +- gnovm/pkg/gnolang/gonative.go | 12 + gnovm/pkg/gnolang/helpers.go | 6 + gnovm/pkg/gnolang/machine.go | 105 ++- gnovm/pkg/gnolang/machine_test.go | 50 +- gnovm/pkg/gnolang/nodes.go | 134 ++- gnovm/pkg/gnolang/op_call.go | 7 + gnovm/pkg/gnolang/op_eval.go | 2 +- gnovm/pkg/gnolang/op_expressions.go | 8 +- gnovm/pkg/gnolang/preprocess.go | 68 +- gnovm/pkg/gnolang/realm.go | 21 +- gnovm/pkg/gnolang/store.go | 26 +- gnovm/pkg/gnolang/types.go | 46 ++ gnovm/pkg/gnolang/values.go | 36 +- gnovm/stdlibs/crypto/sha256/sha256.gno | 10 +- gnovm/stdlibs/crypto/sha256/sha256.go | 7 + gnovm/stdlibs/encoding/base64/base64.gno | 4 +- .../stdlibs/internal/crypto/sha256/sha256.gno | 3 - gnovm/stdlibs/internal/math/math.gno | 3 - gnovm/stdlibs/math/native.gno | 21 + gnovm/stdlibs/math/native.go | 8 + gnovm/stdlibs/math/unsafe.gno | 29 - gnovm/stdlibs/native.go | 764 ++++++++++++++++++ gnovm/stdlibs/{ => std}/banker.go | 6 +- gnovm/stdlibs/{ => std}/context.go | 2 +- gnovm/stdlibs/{ => std}/frame.go | 2 +- gnovm/stdlibs/std/native.gno | 18 + gnovm/stdlibs/std/native.go | 187 +++++ gnovm/stdlibs/stdlibs.go | 577 +------------ gnovm/stdlibs/strconv/strconv.gno | 10 +- gnovm/stdlibs/strconv/strconv.go | 12 + gnovm/stdlibs/time/time.gno | 12 +- gnovm/stdlibs/time/time.go | 17 + gnovm/tests/file.go | 1 + .../files/extern/redeclaration1/README.md | 4 + .../extern/redeclaration1/redeclaration.gno | 3 + .../extern/redeclaration1/redeclaration2.gno | 4 + .../files/extern/redeclaration2/README.md | 4 + .../extern/redeclaration2/redeclaration.gno | 3 + .../extern/redeclaration2/redeclaration2.gno | 4 + .../files/extern/redeclaration3/README.md | 4 + .../extern/redeclaration3/redeclaration.gno | 5 + .../extern/redeclaration3/redeclaration2.gno | 4 + gnovm/tests/files/float5_native.gno | 4 +- gnovm/tests/files/float5_stdlibs.gno | 6 +- gnovm/tests/files/redeclaration10.gno | 12 + gnovm/tests/files/redeclaration6.gno | 12 + gnovm/tests/files/redeclaration7.gno | 13 + gnovm/tests/files/redeclaration8.gno | 12 + gnovm/tests/files/redeclaration9.gno | 14 + gnovm/tests/files/zrealm0.gno | 2 + gnovm/tests/files/zrealm1.gno | 2 + gnovm/tests/files/zrealm2.gno | 4 + gnovm/tests/files/zrealm3.gno | 4 + gnovm/tests/files/zrealm4.gno | 4 + gnovm/tests/files/zrealm5.gno | 4 + gnovm/tests/files/zrealm6.gno | 4 + gnovm/tests/files/zrealm7.gno | 4 + gnovm/tests/files/zrealm_avl0.gno | 4 + gnovm/tests/files/zrealm_avl1.gno | 4 + gnovm/tests/files/zrealm_natbind0.gno | 197 +++++ gnovm/tests/files/zrealm_tests0.gno | 86 +- gnovm/tests/imports.go | 347 ++------ gnovm/tests/stdlibs/README.md | 6 + gnovm/tests/stdlibs/native.go | 226 ++++++ gnovm/tests/stdlibs/std/std.gno | 12 + gnovm/tests/stdlibs/std/std.go | 116 +++ gnovm/tests/stdlibs/stdlibs.go | 18 + misc/Makefile | 28 + misc/genstd/exprstring.go | 290 +++++++ misc/genstd/genstd.go | 210 +++++ misc/genstd/mapping.go | 471 +++++++++++ misc/genstd/mapping_test.go | 294 +++++++ misc/genstd/template.tmpl | 88 ++ .../genstd/testdata/linkFunctions/std/std.gno | 23 + misc/genstd/testdata/linkFunctions/std/std.go | 47 ++ .../linkFunctions_TypedValue/std/std.gno | 11 + .../linkFunctions_TypedValue/std/std.go | 16 + .../linkFunctions_noMatch/std/std.gno | 3 + .../testdata/linkFunctions_noMatch/std/std.go | 3 + .../linkFunctions_noMatchSig/std/std.gno | 3 + .../linkFunctions_noMatchSig/std/std.go | 8 + .../testdata/linkFunctions_unexp/std/std.gno | 4 + .../testdata/linkFunctions_unexp/std/std.go | 13 + misc/genstd/util.go | 119 +++ misc/genstd/util_test.go | 31 + tm2/pkg/std/memfile.go | 8 +- 105 files changed, 4392 insertions(+), 1239 deletions(-) create mode 100644 .github/workflows/codegen.yml create mode 100644 .github/workflows/lint.yml create mode 100644 gnovm/stdlibs/crypto/sha256/sha256.go delete mode 100644 gnovm/stdlibs/internal/crypto/sha256/sha256.gno delete mode 100644 gnovm/stdlibs/internal/math/math.gno create mode 100644 gnovm/stdlibs/math/native.gno create mode 100644 gnovm/stdlibs/math/native.go delete mode 100644 gnovm/stdlibs/math/unsafe.gno create mode 100644 gnovm/stdlibs/native.go rename gnovm/stdlibs/{ => std}/banker.go (97%) rename gnovm/stdlibs/{ => std}/context.go (96%) rename gnovm/stdlibs/{ => std}/frame.go (94%) create mode 100644 gnovm/stdlibs/std/native.gno create mode 100644 gnovm/stdlibs/std/native.go create mode 100644 gnovm/stdlibs/strconv/strconv.go create mode 100644 gnovm/stdlibs/time/time.go create mode 100644 gnovm/tests/files/extern/redeclaration1/README.md create mode 100644 gnovm/tests/files/extern/redeclaration1/redeclaration.gno create mode 100644 gnovm/tests/files/extern/redeclaration1/redeclaration2.gno create mode 100644 gnovm/tests/files/extern/redeclaration2/README.md create mode 100644 gnovm/tests/files/extern/redeclaration2/redeclaration.gno create mode 100644 gnovm/tests/files/extern/redeclaration2/redeclaration2.gno create mode 100644 gnovm/tests/files/extern/redeclaration3/README.md create mode 100644 gnovm/tests/files/extern/redeclaration3/redeclaration.gno create mode 100644 gnovm/tests/files/extern/redeclaration3/redeclaration2.gno create mode 100644 gnovm/tests/files/redeclaration10.gno create mode 100644 gnovm/tests/files/redeclaration6.gno create mode 100644 gnovm/tests/files/redeclaration7.gno create mode 100644 gnovm/tests/files/redeclaration8.gno create mode 100644 gnovm/tests/files/redeclaration9.gno create mode 100644 gnovm/tests/files/zrealm_natbind0.gno create mode 100644 gnovm/tests/stdlibs/README.md create mode 100644 gnovm/tests/stdlibs/native.go create mode 100644 gnovm/tests/stdlibs/std/std.gno create mode 100644 gnovm/tests/stdlibs/std/std.go create mode 100644 gnovm/tests/stdlibs/stdlibs.go create mode 100644 misc/Makefile create mode 100644 misc/genstd/exprstring.go create mode 100644 misc/genstd/genstd.go create mode 100644 misc/genstd/mapping.go create mode 100644 misc/genstd/mapping_test.go create mode 100644 misc/genstd/template.tmpl create mode 100644 misc/genstd/testdata/linkFunctions/std/std.gno create mode 100644 misc/genstd/testdata/linkFunctions/std/std.go create mode 100644 misc/genstd/testdata/linkFunctions_TypedValue/std/std.gno create mode 100644 misc/genstd/testdata/linkFunctions_TypedValue/std/std.go create mode 100644 misc/genstd/testdata/linkFunctions_noMatch/std/std.gno create mode 100644 misc/genstd/testdata/linkFunctions_noMatch/std/std.go create mode 100644 misc/genstd/testdata/linkFunctions_noMatchSig/std/std.gno create mode 100644 misc/genstd/testdata/linkFunctions_noMatchSig/std/std.go create mode 100644 misc/genstd/testdata/linkFunctions_unexp/std/std.gno create mode 100644 misc/genstd/testdata/linkFunctions_unexp/std/std.go create mode 100644 misc/genstd/util.go create mode 100644 misc/genstd/util_test.go diff --git a/.gitattributes b/.gitattributes index a0c478da1fc..c22d136ec50 100644 --- a/.gitattributes +++ b/.gitattributes @@ -1,3 +1,5 @@ *.gno linguist-language=Go *.pb.go linguist-generated merge=ours -diff go.sum linguist-generated text +gnovm/stdlibs/native.go linguist-generated +gnovm/tests/stdlibs/native.go linguist-generated diff --git a/.github/codecov.yml b/.github/codecov.yml index 65609743a74..ecd223f0e84 100644 --- a/.github/codecov.yml +++ b/.github/codecov.yml @@ -52,3 +52,6 @@ flag_management: - name: gno.land paths: - gno.land + - name: misc + paths: + - misc diff --git a/.github/workflows/codegen.yml b/.github/workflows/codegen.yml new file mode 100644 index 00000000000..f941dd69855 --- /dev/null +++ b/.github/workflows/codegen.yml @@ -0,0 +1,36 @@ +name: code generation + +on: + push: + branches: [ "master" ] + pull_request: + paths: + - 'gnovm/stdlibs/**' + - 'gnovm/tests/stdlibs/**' + - 'misc/genstd' + +concurrency: + group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }} + cancel-in-progress: true + +jobs: + generated: + runs-on: ubuntu-latest + steps: + - name: Install Go + uses: actions/setup-go@v4 + with: + go-version: 1.21.x + + - name: Checkout code + uses: actions/checkout@v4 + + - name: Check generated files are up to date + run: | + go generate -x ./... + if [ "$(git status -s)" != "" ]; then + echo "command 'go generate' creates file that differ from git tree, please run 'go generate' and commit:" + git status -s + exit 1 + fi + diff --git a/.github/workflows/lint.yml b/.github/workflows/lint.yml new file mode 100644 index 00000000000..a4c5b6dc85e --- /dev/null +++ b/.github/workflows/lint.yml @@ -0,0 +1,90 @@ +name: lint + +on: + push: + branches: [ "master" ] + pull_request: + +concurrency: + group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }} + cancel-in-progress: true + +jobs: + lint: + runs-on: ubuntu-latest + steps: + - name: Checkout code + uses: actions/checkout@v4 + + - name: Install Go + uses: actions/setup-go@v4 + with: + go-version: 1.21.x + + - name: Lint + uses: golangci/golangci-lint-action@v3 + with: + # sync with misc/devdeps/go.mod + version: v1.54 + args: + --config=./.github/golangci.yml + fmt: + runs-on: ubuntu-latest + steps: + - name: Checkout code + uses: actions/checkout@v4 + + - name: Install Go + uses: actions/setup-go@v4 + with: + go-version: 1.21.x + + - name: Install make + run: sudo apt-get install -y make + + # prefill dependencies so that mod messages don't show up in make output + - name: Fetch dependencies + run: go mod download -modfile ./misc/devdeps/go.mod -x + + # inspired by: + # https://github.com/Jerome1337/gofmt-action/blob/d5eabd189843f1d568286a54578159978b7c0fb1/entrypoint.sh + - name: Check gofumpt + run: | + output="$(GOFMT_FLAGS=-l make -s fmt)" + if [ ! -z "$output" ]; then + echo "The following files are not properly formatted; run 'make fmt' to format them." + echo "$output" + exit 1 + else + echo 'Succeeded.' + fi + modtidy: + runs-on: ubuntu-latest + steps: + - name: Checkout code + uses: actions/checkout@v4 + + - name: Install Go + uses: actions/setup-go@v4 + with: + go-version: 1.21.x + + - name: Check go.mods + run: | + set -xe + # Find all go.mod files + gomods=$(find . -type f -name go.mod) + + # Calculate sums for all go.mod files + sums=$(sha256sum $gomods) + + # Iterate over each go.mod file + for modfile in $gomods; do + dir=$(dirname "$modfile") + + # Run go mod tidy in the directory + (cd "$dir" && go mod tidy -v) || exit 1 + done + + # Verify the sums + echo "$sums" | sha256sum -c diff --git a/.github/workflows/misc.yml b/.github/workflows/misc.yml index bd8b53e7a25..9c666675fa4 100644 --- a/.github/workflows/misc.yml +++ b/.github/workflows/misc.yml @@ -1,65 +1,59 @@ +# tests the "misc" directory & tools +# (not meant for miscellaneous workflows) name: misc on: + pull_request: + paths: + - "misc/genstd/**.go" + - "misc/Makefile" + - ".github/workflows/misc.yml" + # Until the codecov issue is resolved, it's essential to run the tests for gnovm, tm2, misc, and gno.land concurrently. + - "gnovm/**" + - "tm2/**" + - "gno.land/**" + - "examples/**" + - ".github/workflows/**" push: branches: [ "master" ] - pull_request: concurrency: group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }} cancel-in-progress: true jobs: - lint: + build: + strategy: + fail-fast: false + matrix: + goversion: + - "1.21.x" + program: + - "genstd" runs-on: ubuntu-latest + timeout-minutes: 5 steps: - - name: Checkout code - uses: actions/checkout@v4 - - name: Install Go uses: actions/setup-go@v4 with: - go-version: 1.21.x - - - name: Lint - uses: golangci/golangci-lint-action@v3 - with: - # sync with misc/devdeps/go.mod - version: v1.54 - args: - --config=./.github/golangci.yml - fmt: - runs-on: ubuntu-latest - steps: + go-version: ${{ matrix.goversion }} - name: Checkout code uses: actions/checkout@v4 - - name: Install Go - uses: actions/setup-go@v4 - with: - go-version: 1.21.x - - - name: Install make - run: sudo apt-get install -y make + - name: go install + working-directory: misc + run: go install ./${{ matrix.program }} - # prefill dependencies so that mod messages don't show up in make output - - name: Fetch dependencies - run: go mod download -modfile ./misc/devdeps/go.mod -x - - # inspired by: - # https://github.com/Jerome1337/gofmt-action/blob/d5eabd189843f1d568286a54578159978b7c0fb1/entrypoint.sh - - name: Check gofumpt - run: | - output="$(GOFMT_FLAGS=-l make -s fmt)" - if [ ! -z "$output" ]; then - echo "The following files are not properly formatted; run 'make fmt' to format them." - echo "$output" - exit 1 - else - echo 'Succeeded.' - fi - modtidy: + test: + strategy: + fail-fast: false + matrix: + goversion: + - "1.21.x" + args: + - _test.genstd runs-on: ubuntu-latest + timeout-minutes: 15 steps: - name: Checkout code uses: actions/checkout@v4 @@ -67,24 +61,20 @@ jobs: - name: Install Go uses: actions/setup-go@v4 with: - go-version: 1.21.x + go-version: ${{ matrix.goversion }} - - name: Check go.mods + - name: Test + working-directory: misc run: | - set -xe - # Find all go.mod files - gomods=$(find . -type f -name go.mod) - - # Calculate sums for all go.mod files - sums=$(sha256sum $gomods) - - # Iterate over each go.mod file - for modfile in $gomods; do - dir=$(dirname "$modfile") - - # Run go mod tidy in the directory - (cd "$dir" && go mod tidy -v) || exit 1 - done - - # Verify the sums - echo "$sums" | sha256sum -c + export GOPATH=$HOME/go + export GOTEST_FLAGS="-v -p 1 -timeout=30m -coverprofile=coverage.out -covermode=atomic" + make ${{ matrix.args }} + + - if: runner.os == 'Linux' + uses: codecov/codecov-action@v3 + with: + token: ${{ secrets.CODECOV_TOKEN }} + name: misc + flags: misc,misc-${{matrix.args}},go-${{ matrix.goversion }} + files: ./misc/coverage.out + fail_ci_if_error: ${{ github.repository == 'gnolang/gno' }} diff --git a/Makefile b/Makefile index 358094e5c4a..80c9bef2ed7 100644 --- a/Makefile +++ b/Makefile @@ -25,6 +25,7 @@ test.components: $(MAKE) --no-print-directory -C gnovm test $(MAKE) --no-print-directory -C gno.land test $(MAKE) --no-print-directory -C examples test + $(MAKE) --no-print-directory -C misc test .PHONY: test.docker test.docker: diff --git a/docs/reference/standard-library.md b/docs/reference/standard-library.md index f6ab5b307ce..8776c62faf8 100644 --- a/docs/reference/standard-library.md +++ b/docs/reference/standard-library.md @@ -2,67 +2,193 @@ id: standard-library --- -# Gno Standard Library - -When developing a realm in Gnolang, developers may utilize libraries in [stdlibs](https://github.com/gnolang/gno/tree/master/gnovm/stdlibs). These are the core standard packages provided for Gnolang [Realms ](../explanation/realms.md)& [Packages](../explanation/packages.md). - -Libraries can be imported in a manner similar to how libraries are imported in Golang. - -An example of importing a `std` library in Gnolang is demonstrated in the following command: - -```go -import "std" +# Standard Libraries + +Gno comes with a set of standard libraries which are included whenever you +execute Gno code. These are distinguishable from imports of packages from the +chain by not referring to a "domain" as the first part of their import path. For +instance, `import "encoding/binary"` refers to a standard library, while +`import "gno.land/p/demo/avl"` refers to an on-chain package. + +Standard libraries packages follow the same semantics as on-chain packages (ie. +they don't persist state like realms do) and come as a part of the Gno +programming language rather than with the Gno.land chain. + +Many standard libaries are near-identical copies of the equivalent Go standard +libraries; in fact, you can check the current status of implementation of each +Go standard libarary on [Go\<\>Gno compatibility](go-gno-compatibility.md). + +## Gathering documentation + +At the time being, there is no "list" of the available standard libraries +available from Gno tooling or documentation, but you can obtain a list of all +the available packages with the following commands: + +```console +$ cd gnovm/stdlibs # go to correct directory +$ find -type d +./testing +./math +./crypto +./crypto/chacha20 +./crypto/chacha20/chacha +./crypto/chacha20/rand +./crypto/sha256 +./crypto/cipher +... ``` -Let's explore some of the most commonly used modules in the library. - -## `stdshim` - -### `banker.gno` +All of the packages have automatic, generated documentation through the use of +`gno doc`, which has similar functionality and features to `go doc`: -A library for manipulating `Coins`. Interfaces that must be implemented when using this library are as follows: +```console +$ gno doc encoding/binary +package binary // import "encoding/binary" -[embedmd]:# (../assets/reference/standard-library/std-1.gno go) -```go -// returns the list of coins owned by the address -GetCoins(addr Address) (dst Coins) +Package binary implements simple translation between numbers and byte sequences +and encoding and decoding of varints. -// sends coins from one address to another -SendCoins(from, to Address, amt Coins) +[...] -// returns the total supply of the coin -TotalCoin(denom string) int64 +var BigEndian bigEndian +var LittleEndian littleEndian +type AppendByteOrder interface{ ... } +type ByteOrder interface{ ... } +$ gno doc -u -src encoding/binary littleEndian.AppendUint16 +package binary // import "encoding/binary" -// issues coins to the address -IssueCoin(addr Address, denom string, amount int64) - -// burns coins from the address -RemoveCoin(addr Address, denom string, amount int64) +func (littleEndian) AppendUint16(b []byte, v uint16) []byte { + return append(b, + byte(v), + byte(v>>8), + ) +} ``` -### `coins.gno` +`gno doc` will work automatically when used within the Gno repository or any +repository which has a `go.mod` dependency on `github.com/gnolang/gno`, which +can be a simple way to set up your Gno repositories to automatically support +`gno` commands (aside from `doc`, also `test`, `run`, etc.). -A library that declares structs for expressing `Coins`. The struct looks like the following: +Another alternative is setting your enviornment variable `GNOROOT` to point to +where you cloned the Gno repository. You can set this in your `~/.profile` file +to be automatically set up in your console: -[embedmd]:# (../assets/reference/standard-library/std-2.gno go) -```go -type Coin struct { - Denom string `json:"denom"` // the symbol of the coin - Amount int64 `json:"amount"` // the quantity of the coin -} +```sh +export GNOROOT=$HOME/gno ``` -### `testing` - -A library that declares `*testing`, which is a tool used for the creation and execution of test cases during the development and testing phase of realms utilizing the `gno` CLI tool with the `test` option. - -There are 3 types of testing in `gno`. +## Test standard libraries + +There are some additional standard library functions and packages which are +currently available only in `_test.gno` and `_filetest.gno` files. At the time +of writing, these are only some additions in the `std` package to support +changing some values in test functions. + +`gno doc` currently doesn't support reading from the test standard libraries, +though support is planned to be added. For now, you can inspect the directory +`gnovm/tests/stdlibs`. + +## Adding new standard libraries + +New standard libraries may be added by simply creating a new directory (whose +path relative to the `stdlibs` directory will be the import path used in Gno +programs). Following that, the suggested approach for adding a Go standard +libary is to copy the original files from the Go source tree, and renaming their +extensions from `.go` to `.gno`. + +> As a small aid, this bash one-liner can be useful to convert all the file +> extensions: +> +> ```sh +> for i in *.go; do mv $i "$(echo $i | sed 's/\.go$/.gno/')"; done +> ``` + +Following that, the suggested approach is to iteratively try running `gno test .`, +while fixing any errors that may come out of trying to test the package. + +Some things to keep in mind: + +- Gno doesn't support assembly functions and build tags. Some Go packages may + contain assembly versions for different architecture and a `generic.go` file + containing the architecture-independent version. The general approach is that + of removing everything architecture/os-specific except for the `generic.go` file. +- Gno doesn't support reflection at the time of writing, which means that for + now many packages which rely heavily on reflection have to be delayed or + reduced while we figure out the details on how to implement reflection. + Aside from the `reflect` package itself, this also translates to very common + packages still not available in Gno, such as `fmt` or `encoding/json`. +- In the package documentation, specify the Go version from which the library + was taken. +- All changes from the Go standard libaries must be explicitly marked, possibly + with `// XXX` comments as needed. + +If you intend to create a PR to add a new standard library, remember to update +[Go\<\>Gno compatibility](go-gno-compatibility.md) accordingly. + +## Native bindings + +Gno has support for "natively-defined functions" exclusively within the standard +libaries. These are functions which are _declared_ in Gno code, but only _defined_ +in Go. There are generally three reasons why a function should be natively +defined: + +1. It relies on inspecting the Gno Virtual Machine itself.\ + For example: `std.AssertOriginCall`, `std.CurrentRealmPath`. +2. It relies on `unsafe`, or other features which are not planned to be + available in the GnoVM.\ + For example: `math.Float64frombits`. +3. Its native Go performance significantly outperforms the Gno counterpart by + several orders of magnitude, and it is used in crucial code or hot paths in + many programs.\ + For example: `sha256.Sum256`. + +The takeaway here is that native bindings are a special feature which can be +useful to overcome pure Gno limitations, but it is not a substitute for writing +standard libaries in Gno. + +There are three components to a natively bound function in Gno: + +1. The Gno function declaration, which must be a top-level function with no body + (and no brackets).\ + For example: `crypto/sha256/sha256.gno`. +2. The Go function definition, which must be a top-level function with the same + name and signature.\ + For example: `crypto/sha256/sha256.go`. +3. When the two above are present and valid, the native binding can be created + by executing the code generator: either execute `go generate` from the + `stdlibs` directory, or run `make generate` from the `gnovm` directory.\ + This generates the `native.go` file available in the `stdlibs` directory, + which provides the binding itself to then be used by the GnoVM. + +The code generator in question is available in the `misc/genstd` directory. +There are some quirks and features that must be kept in mind when writing native +bindings, which are the following: + +- Unexported functions (for instance, `func sum256(b []byte)`) must have their + Go counterpart prefixed with `X_` in order to make the functions exported (ie. + `func X_sum256(b []byte)`). +- The Go function declaration may specify as the first argument + `m *gno.Machine`, where `gno` is an import for + `github.com/gnolang/gno/gnovm/pkg/gnolang`. This gives the function access to + the Virtual Machine state, and is used by functions like `std.AssertOriginCall()`. +- The Go function may change the type of any parameter or result to + `gno.TypedValue` (where `gno` is an import for the above import path). This + means that the `native.go` generated code will not attempt to automatically + convert the Gno value into the Go value, and can be useful for unsupported + conversions like interface values. +- A small set of named types are "linked" between their Gno version and Go + counterpart. For instance, `std.Address` in Gno is + `(".../tm2/pkg/crypto").Bech32Address` in Go. A list of these can be found in + `misc/genstd/mapping.go`. +- Not all type literals are currently supported when converting from their Gno + version to their Go counterpart. Notable omissions at the time of writing + include struct and map literals. If you intend to use these, modify the code + generator to support them. +- The code generator does not inspect any imported packages from the Go native code + to determine the default package identifier (ie. the `package` clause). + Ie. if a package is in `foo/bar`, but declares `package xyz`, when importing + foo/bar the generator will assume the name to be `bar` instead of `xyz`. + You can add an identifier to the import to fix this and use the identifier + you want/need, ie.: `import gno "github.com/gnolang/gno/gnovm/pkg/gnolang"`. -* Type `T` - * Type passed to Test functions to manage test state and support formatted test logs. -* Type `B` - * Type passed to Benchmark functions. - * Manage benchmark timing. - * Specify the number of iterations to run. -* Type `PB` - * Used by `RunParallel` for running parallel benchmarks. diff --git a/examples/gno.land/p/demo/avl/z_0_filetest.gno b/examples/gno.land/p/demo/avl/z_0_filetest.gno index 814e19d6d49..e91788ac8eb 100644 --- a/examples/gno.land/p/demo/avl/z_0_filetest.gno +++ b/examples/gno.land/p/demo/avl/z_0_filetest.gno @@ -267,6 +267,8 @@ func main() { // "FileName": "main.gno", // "IsMethod": false, // "Name": "init.0", +// "NativeName": "", +// "NativePkg": "", // "PkgPath": "gno.land/r/test", // "Source": { // "@type": "/gno.RefNode", @@ -301,6 +303,8 @@ func main() { // "FileName": "main.gno", // "IsMethod": false, // "Name": "main", +// "NativeName": "", +// "NativePkg": "", // "PkgPath": "gno.land/r/test", // "Source": { // "@type": "/gno.RefNode", diff --git a/examples/gno.land/p/demo/avl/z_1_filetest.gno b/examples/gno.land/p/demo/avl/z_1_filetest.gno index 410e9e93601..cdd56a5ad89 100644 --- a/examples/gno.land/p/demo/avl/z_1_filetest.gno +++ b/examples/gno.land/p/demo/avl/z_1_filetest.gno @@ -291,6 +291,8 @@ func main() { // "FileName": "main.gno", // "IsMethod": false, // "Name": "init.0", +// "NativeName": "", +// "NativePkg": "", // "PkgPath": "gno.land/r/test", // "Source": { // "@type": "/gno.RefNode", @@ -325,6 +327,8 @@ func main() { // "FileName": "main.gno", // "IsMethod": false, // "Name": "main", +// "NativeName": "", +// "NativePkg": "", // "PkgPath": "gno.land/r/test", // "Source": { // "@type": "/gno.RefNode", diff --git a/examples/gno.land/r/demo/boards/z_4_filetest.gno b/examples/gno.land/r/demo/boards/z_4_filetest.gno index 749566ea5bc..c891a352d8c 100644 --- a/examples/gno.land/r/demo/boards/z_4_filetest.gno +++ b/examples/gno.land/r/demo/boards/z_4_filetest.gno @@ -374,7 +374,7 @@ func main() { // "Escaped": true, // "ObjectID": "336074805fc853987abe6f7fe3ad97a6a6f3077a:2" // }, -// "Index": "188", +// "Index": "189", // "TV": null // } // } @@ -541,7 +541,7 @@ func main() { // }, // "V": { // "@type": "/gno.RefValue", -// "Hash": "8164abed5231309c88497013f7da72a1b5d427b0", +// "Hash": "25ffc45509708ca0ae17271cb4c3a1dfb367b965", // "ObjectID": "f6dbf411da22e67d74cd7ddba6a76cd7e14a4822:115" // } // }, @@ -847,7 +847,7 @@ func main() { // }, // "V": { // "@type": "/gno.RefValue", -// "Hash": "5b4b593f1d4b37cb99166247ea28174f91087fdd", +// "Hash": "a8e67b9881af89ca2ec2f05778bf7528a54a5833", // "ObjectID": "f6dbf411da22e67d74cd7ddba6a76cd7e14a4822:82" // } // }, @@ -865,7 +865,7 @@ func main() { // }, // "V": { // "@type": "/gno.RefValue", -// "Hash": "7e9fd9bb5e90a06c7751585cd80f23aedddde25b", +// "Hash": "d8ae14a4620e3c6dedabd76cd0c5d7e3c205d647", // "ObjectID": "f6dbf411da22e67d74cd7ddba6a76cd7e14a4822:83" // } // }, diff --git a/gno.land/pkg/gnoland/app.go b/gno.land/pkg/gnoland/app.go index 412930e51dd..689ae1ae106 100644 --- a/gno.land/pkg/gnoland/app.go +++ b/gno.land/pkg/gnoland/app.go @@ -158,7 +158,7 @@ func InitChainer(baseApp *sdk.BaseApp, acctKpr auth.AccountKeeperI, bankKpr bank // NOTE: comment out to ignore. if !skipFailingGenesisTxs { - panic(res.Error) + panic(res.Log) } } else { ctx.Logger().Info("SUCCESS:", string(amino.MustMarshalJSON(tx))) diff --git a/gno.land/pkg/sdk/vm/builtins.go b/gno.land/pkg/sdk/vm/builtins.go index 270a4f7c37e..ef2ac93f617 100644 --- a/gno.land/pkg/sdk/vm/builtins.go +++ b/gno.land/pkg/sdk/vm/builtins.go @@ -40,47 +40,10 @@ func (vm *VMKeeper) initBuiltinPackagesAndTypes(store gno.Store) { return m2.RunMemPackage(memPkg, true) } store.SetPackageGetter(getPackage) - store.SetPackageInjector(vm.packageInjector) + store.SetNativeStore(stdlibs.NativeStore) stdlibs.InjectNativeMappings(store) } -func (vm *VMKeeper) packageInjector(store gno.Store, pn *gno.PackageNode) { - // Also inject stdlibs native functions. - stdlibs.InjectPackage(store, pn) - // vm (this package) specific injections: - switch pn.PkgPath { - case "std": - /* XXX deleteme - // Also see stdlibs/InjectPackage. - pn.DefineNative("AssertOriginCall", - gno.Flds( // params - ), - gno.Flds( // results - ), - func(m *gno.Machine) { - isOrigin := len(m.Frames) == 2 - if !isOrigin { - panic("invalid non-origin call") - } - }, - ) - pn.DefineNative("IsOriginCall", - gno.Flds( // params - ), - gno.Flds( // results - "isOrigin", "bool", - ), - func(m *gno.Machine) { - isOrigin := len(m.Frames) == 2 - res0 := gno.TypedValue{T: gno.BoolType} - res0.SetBool(isOrigin) - m.PushValue(res0) - }, - ) - */ - } -} - // ---------------------------------------- // SDKBanker diff --git a/gnovm/Makefile b/gnovm/Makefile index 29510e9a1da..cc7154492d8 100644 --- a/gnovm/Makefile +++ b/gnovm/Makefile @@ -63,6 +63,11 @@ _test.gnolang.stdlibs.sync:; go test tests/*.go -test.short -run 'TestFiles$$/' ######################################## # Code gen +# TODO: move _dev.stringer to go:generate instructions, simplify generate +# to just go generate. +.PHONY: generate +generate: _dev.stringer _dev.generate + stringer_cmd=$(rundep) golang.org/x/tools/cmd/stringer .PHONY: _dev.stringer _dev.stringer: @@ -73,5 +78,9 @@ _dev.stringer: $(stringer_cmd) -type=VPType ./pkg/gnolang $(stringer_cmd) -type=Word ./pkg/gnolang +.PHONY: _dev.generate +_dev.generate: + go generate -x ./... + # genproto: # see top-level Makefile. diff --git a/gnovm/cmd/gno/test.go b/gnovm/cmd/gno/test.go index 38615c88c4e..86ca33f3dc8 100644 --- a/gnovm/cmd/gno/test.go +++ b/gnovm/cmd/gno/test.go @@ -313,8 +313,11 @@ func gnoTestPkg( if err == nil { gnoPkgPath = modfile.Module.Mod.Path } else { - // unable to read pkgPath from gno.mod, generate a random realm path - gnoPkgPath = gno.GnoRealmPkgsPrefixBefore + random.RandStr(8) + gnoPkgPath = pkgPathFromRootDir(pkgPath, rootDir) + if gnoPkgPath == "" { + // unable to read pkgPath from gno.mod, generate a random realm path + gnoPkgPath = gno.GnoRealmPkgsPrefixBefore + random.RandStr(8) + } } memPkg := gno.ReadMemPackage(pkgPath, gnoPkgPath) @@ -431,6 +434,35 @@ func gnoTestPkg( return errs } +// attempts to determine the full gno pkg path by analyzing the directory. +func pkgPathFromRootDir(pkgPath, rootDir string) string { + abPkgPath, err := filepath.Abs(pkgPath) + if err != nil { + log.Printf("could not determine abs path: %v", err) + return "" + } + abRootDir, err := filepath.Abs(rootDir) + if err != nil { + log.Printf("could not determine abs path: %v", err) + return "" + } + abRootDir += string(filepath.Separator) + if !strings.HasPrefix(abPkgPath, abRootDir) { + return "" + } + impPath := strings.ReplaceAll(abPkgPath[len(abRootDir):], string(filepath.Separator), "/") + for _, prefix := range [...]string{ + "examples/", + "gnovm/stdlibs/", + "gnovm/tests/stdlibs/", + } { + if strings.HasPrefix(impPath, prefix) { + return impPath[len(prefix):] + } + } + return "" +} + func runTestFiles( m *gno.Machine, files *gno.FileSet, diff --git a/gnovm/cmd/gno/testdata/gno_test/realm_correct.txtar b/gnovm/cmd/gno/testdata/gno_test/realm_correct.txtar index c3d3b983e34..f17f28055f2 100644 --- a/gnovm/cmd/gno/testdata/gno_test/realm_correct.txtar +++ b/gnovm/cmd/gno/testdata/gno_test/realm_correct.txtar @@ -55,6 +55,8 @@ func main() { // "FileName": "main.gno", // "IsMethod": false, // "Name": "main", +// "NativeName": "", +// "NativePkg": "", // "PkgPath": "gno.land/r/x", // "Source": { // "@type": "/gno.RefNode", diff --git a/gnovm/cmd/gno/testdata/gno_test/realm_incorrect.txtar b/gnovm/cmd/gno/testdata/gno_test/realm_incorrect.txtar index 3922c34fec8..38794a3e645 100644 --- a/gnovm/cmd/gno/testdata/gno_test/realm_incorrect.txtar +++ b/gnovm/cmd/gno/testdata/gno_test/realm_incorrect.txtar @@ -7,7 +7,7 @@ stderr '=== RUN file/x_filetest.gno' stderr 'panic: fail on x_filetest.gno: diff:' stderr '--- Expected' stderr '\+\+\+ Actual' -stderr '@@ -1 \+1,64 @@' +stderr '@@ -1 \+1,66 @@' stderr '-xxx' stderr '\+switchrealm\["gno.land/r/x"\]' diff --git a/gnovm/cmd/gno/testdata/gno_test/realm_sync.txtar b/gnovm/cmd/gno/testdata/gno_test/realm_sync.txtar index 236e69f8641..e8c643af0ba 100644 --- a/gnovm/cmd/gno/testdata/gno_test/realm_sync.txtar +++ b/gnovm/cmd/gno/testdata/gno_test/realm_sync.txtar @@ -70,6 +70,8 @@ func main() { // "FileName": "main.gno", // "IsMethod": false, // "Name": "main", +// "NativeName": "", +// "NativePkg": "", // "PkgPath": "gno.land/r/x", // "Source": { // "@type": "/gno.RefNode", diff --git a/gnovm/pkg/gnolang/gno_test.go b/gnovm/pkg/gnolang/gno_test.go index 751534e1e16..5c50b0830a6 100644 --- a/gnovm/pkg/gnolang/gno_test.go +++ b/gnovm/pkg/gnolang/gno_test.go @@ -16,7 +16,9 @@ func TestRunEmptyMain(t *testing.T) { t.Parallel() m := NewMachine("test", nil) - main := FuncD("main", nil, nil, nil) + // []Stmt{} != nil, as nil means that in the source code not even the + // brackets are present and is reserved for external (ie. native) functions. + main := FuncD("main", nil, nil, []Stmt{}) m.RunDeclaration(main) m.RunMain() } diff --git a/gnovm/pkg/gnolang/go2gno.go b/gnovm/pkg/gnolang/go2gno.go index c3ab4e95b73..ee82cb39555 100644 --- a/gnovm/pkg/gnolang/go2gno.go +++ b/gnovm/pkg/gnolang/go2gno.go @@ -96,9 +96,11 @@ func MustParseExpr(expr string) Expr { return x } -// filename must not include the path. +// ParseFile uses the Go parser to parse body. It then runs [Go2Gno] on the +// resulting AST -- the resulting FileNode is returned, together with any other +// error (including panics, which are recovered) from [Go2Gno]. func ParseFile(filename string, body string) (fn *FileNode, err error) { - // Parse src but stop after processing the imports. + // Use go parser to parse the body. fs := token.NewFileSet() f, err := parser.ParseFile(fs, filename, body, parser.ParseComments|parser.DeclarationErrors) if err != nil { @@ -112,9 +114,9 @@ func ParseFile(filename string, body string) (fn *FileNode, err error) { defer func() { if r := recover(); r != nil { if rerr, ok := r.(error); ok { - err = rerr + err = errors.Wrap(rerr, "parsing file") } else { - err = errors.New(fmt.Sprintf("%v", r)) + err = errors.New(fmt.Sprintf("%v", r)).Stacktrace() } return } @@ -431,7 +433,10 @@ func Go2Gno(fs *token.FileSet, gon ast.Node) (n Node) { } name := toName(gon.Name) type_ := Go2Gno(fs, gon.Type).(*FuncTypeExpr) - body := Go2Gno(fs, gon.Body).(*BlockStmt).Body + var body []Stmt + if gon.Body != nil { + body = Go2Gno(fs, gon.Body).(*BlockStmt).Body + } return &FuncDecl{ IsMethod: isMethod, Recv: recv, diff --git a/gnovm/pkg/gnolang/gonative.go b/gnovm/pkg/gnolang/gonative.go index 69e468f755a..f3739dad0f3 100644 --- a/gnovm/pkg/gnolang/gonative.go +++ b/gnovm/pkg/gnolang/gonative.go @@ -676,6 +676,9 @@ func go2GnoValueUpdate(alloc *Allocator, rlm *Realm, lvl int, tv *TypedValue, rv return } +// used for direct comparison to error types +var tError = reflect.TypeOf(new(error)).Elem() + // If recursive is false, this function is like go2GnoValue() but less lazy // (but still not recursive/eager). When recursive is false, it is for // converting Go types to Gno types upon an explicit conversion (via @@ -757,6 +760,15 @@ func go2GnoValue2(alloc *Allocator, store Store, rv reflect.Value, recursive boo // regardless. tv.V = alloc.NewNative(rv) case reflect.Interface: + // special case for errors, which are very often used especially in + // native bindings + if rv.Type() == tError { + tv.T = gErrorType + if !rv.IsNil() { + tv.V = alloc.NewNative(rv.Elem()) + } + return + } panic("not yet implemented") case reflect.Map: panic("not yet implemented") diff --git a/gnovm/pkg/gnolang/helpers.go b/gnovm/pkg/gnolang/helpers.go index 4810a67304a..b163b6a52a7 100644 --- a/gnovm/pkg/gnolang/helpers.go +++ b/gnovm/pkg/gnolang/helpers.go @@ -105,6 +105,12 @@ func MaybeNativeT(tx interface{}) *MaybeNativeTypeExpr { } } +// FuncD creates a new function declaration. +// +// There is a difference between passing nil to body or passing []Stmt{}: +// nil means that the curly brackets are missing in the source code, indicating +// a declaration for an externally-defined function, while []Stmt{} is simply a +// functions with no statements (func() {}). func FuncD(name interface{}, params, results FieldTypeExprs, body []Stmt) *FuncDecl { return &FuncDecl{ NameExpr: *Nx(name), diff --git a/gnovm/pkg/gnolang/machine.go b/gnovm/pkg/gnolang/machine.go index 94232e014d2..a22d335ad3b 100644 --- a/gnovm/pkg/gnolang/machine.go +++ b/gnovm/pkg/gnolang/machine.go @@ -46,14 +46,15 @@ type Machine struct { Context interface{} } -// machine.Release() must be called on objects -// created via this constructor -// Machine with new package of given path. -// Creates a new MemRealmer for any new realms. -// Looks in store for package of pkgPath; if not found, -// creates new instances as necessary. -// If pkgPath is zero, the machine has no active package -// and one must be set prior to usage. +// NewMachine initializes a new gno virtual machine, acting as a shorthand +// for [NewMachineWithOptions], setting the given options PkgPath and Store. +// +// The machine will run on the package at the given path, which will be +// retrieved through the given store. If it is not set, the machine has no +// active package, and one must be set prior to usage. +// +// Like for [NewMachineWithOptions], Machines initialized through this +// constructor must be finalized with [Machine.Release]. func NewMachine(pkgPath string, store Store) *Machine { return NewMachineWithOptions( MachineOptions{ @@ -62,12 +63,14 @@ func NewMachine(pkgPath string, store Store) *Machine { }) } +// MachineOptions is used to pass options to [NewMachineWithOptions]. type MachineOptions struct { + // Active package of the given machine; must be set before execution. PkgPath string CheckTypes bool // not yet used ReadOnly bool - Output io.Writer - Store Store + Output io.Writer // default os.Stdout + Store Store // default NewStore(Alloc, nil, nil) Context interface{} Alloc *Allocator // or see MaxAllocBytes. MaxAllocBytes int64 // or 0 for no limit. @@ -87,6 +90,11 @@ var machinePool = sync.Pool{ }, } +// NewMachineWithOptions initializes a new gno virtual machine with the given +// options. +// +// Machines initialized through this constructor must be finalized with +// [Machine.Release]. func NewMachineWithOptions(opts MachineOptions) *Machine { checkTypes := opts.CheckTypes readOnly := opts.ReadOnly @@ -141,11 +149,10 @@ var ( valueZeroed [VMSliceSize]TypedValue ) -// m should not be used after this call -// if m is nil, this will panic -// this is on purpose, to discourage misuse -// and prevent objects that were not taken from -// the pool, to call Release +// Release resets some of the values of *Machine and puts back m into the +// machine pool; for this reason, Release() should be called as a finalizer, +// and m should not be used after this call. Only Machines initialized with this +// package's constructors should be released. func (m *Machine) Release() { // here we zero in the values for the next user m.NumOps = 0 @@ -175,6 +182,9 @@ func (m *Machine) SetActivePackage(pv *PackageValue) { // Upon restart, preprocess all MemPackage and save blocknodes. // This is a temporary measure until we optimize/make-lazy. +// +// NOTE: package paths not beginning with gno.land will be allowed to override, +// to support cases of stdlibs processed through [RunMemPackagesWithOverrides]. func (m *Machine) PreprocessAllFilesAndSaveBlockNodes() { ch := m.Store.IterMemPackage() for memPkg := range ch { @@ -209,8 +219,23 @@ func (m *Machine) PreprocessAllFilesAndSaveBlockNodes() { // and corresponding package node, package value, and types to store. Save // is set to false for tests where package values may be native. func (m *Machine) RunMemPackage(memPkg *std.MemPackage, save bool) (*PackageNode, *PackageValue) { + return m.runMemPackage(memPkg, save, false) +} + +// RunMemPackageWithOverrides works as [RunMemPackage], however after parsing, +// declarations are filtered removing duplicate declarations. +// To control which declaration overrides which, use [ReadMemPackageFromList], +// putting the overrides at the top of the list. +func (m *Machine) RunMemPackageWithOverrides(memPkg *std.MemPackage, save bool) (*PackageNode, *PackageValue) { + return m.runMemPackage(memPkg, save, true) +} + +func (m *Machine) runMemPackage(memPkg *std.MemPackage, save, overrides bool) (*PackageNode, *PackageValue) { // parse files. files := ParseMemPackage(memPkg) + if !overrides && checkDuplicates(files) { + panic(fmt.Errorf("running package %q: duplicate declarations not allowed", memPkg.Path)) + } // make and set package if doesn't exist. pn := (*PackageNode)(nil) pv := (*PackageValue)(nil) @@ -237,6 +262,56 @@ func (m *Machine) RunMemPackage(memPkg *std.MemPackage, save bool) (*PackageNode return pn, pv } +// checkDuplicates returns true if there duplicate declarations in the fset. +func checkDuplicates(fset *FileSet) bool { + defined := make(map[Name]struct{}, 128) + for _, f := range fset.Files { + for _, d := range f.Decls { + var name Name + switch d := d.(type) { + case *FuncDecl: + if d.Name == "init" { //nolint:goconst + continue + } + name = d.Name + if d.IsMethod { + name = Name(destar(d.Recv.Type).String()) + "." + name + } + case *TypeDecl: + name = d.Name + case *ValueDecl: + for _, nx := range d.NameExprs { + if nx.Name == "_" { + continue + } + if _, ok := defined[nx.Name]; ok { + return true + } + defined[nx.Name] = struct{}{} + } + continue + default: + continue + } + if name == "_" { + continue + } + if _, ok := defined[name]; ok { + return true + } + defined[name] = struct{}{} + } + } + return false +} + +func destar(x Expr) Expr { + if x, ok := x.(*StarExpr); ok { + return x.X + } + return x +} + // Tests all test files in a mempackage. // Assumes that the importing of packages is handled elsewhere. // The resulting package value and node become injected with TestMethods and diff --git a/gnovm/pkg/gnolang/machine_test.go b/gnovm/pkg/gnolang/machine_test.go index e37fc508cb6..4268e3f3332 100644 --- a/gnovm/pkg/gnolang/machine_test.go +++ b/gnovm/pkg/gnolang/machine_test.go @@ -1,6 +1,16 @@ package gnolang -import "testing" +import ( + "fmt" + "testing" + + dbm "github.com/gnolang/gno/tm2/pkg/db" + "github.com/gnolang/gno/tm2/pkg/std" + "github.com/gnolang/gno/tm2/pkg/store/dbadapter" + "github.com/gnolang/gno/tm2/pkg/store/iavl" + stypes "github.com/gnolang/gno/tm2/pkg/store/types" + "github.com/jaekwon/testify/assert" +) func BenchmarkCreateNewMachine(b *testing.B) { for i := 0; i < b.N; i++ { @@ -8,3 +18,41 @@ func BenchmarkCreateNewMachine(b *testing.B) { m.Release() } } + +func TestRunMemPackageWithOverrides_revertToOld(t *testing.T) { + // A test to check revertToOld is correctly putting back an old value, + // after preprocessing fails. + db := dbm.NewMemDB() + baseStore := dbadapter.StoreConstructor(db, stypes.StoreOptions{}) + iavlStore := iavl.StoreConstructor(db, stypes.StoreOptions{}) + store := NewStore(nil, baseStore, iavlStore) + m := NewMachine("std", store) + m.RunMemPackageWithOverrides(&std.MemPackage{ + Name: "std", + Path: "std", + Files: []*std.MemFile{ + {Name: "a.gno", Body: `package std; func Redecl(x int) string { return "1" }`}, + }, + }, true) + result := func() (p string) { + defer func() { + p = fmt.Sprint(recover()) + }() + m.RunMemPackageWithOverrides(&std.MemPackage{ + Name: "std", + Path: "std", + Files: []*std.MemFile{ + {Name: "b.gno", Body: `package std; func Redecl(x int) string { var y string; _, _ = y; return "2" }`}, + }, + }, true) + return + }() + t.Log("panic trying to redeclare invalid func", result) + m.RunStatement(S(Call(X("Redecl"), 11))) + + // Check last value, assuming it is the result of Redecl. + v := m.Values[0] + assert.NotNil(t, v) + assert.Equal(t, v.T.Kind(), StringKind) + assert.Equal(t, v.V, StringValue("1")) +} diff --git a/gnovm/pkg/gnolang/nodes.go b/gnovm/pkg/gnolang/nodes.go index 2a9e0b51a97..b127cd32421 100644 --- a/gnovm/pkg/gnolang/nodes.go +++ b/gnovm/pkg/gnolang/nodes.go @@ -1092,13 +1092,21 @@ func PackageNameFromFileBody(name, body string) Name { return Name(astFile.Name.Name) } -// NOTE: panics if package name is invalid. +// ReadMemPackage initializes a new MemPackage by reading the OS directory +// at dir, and saving it with the given pkgPath (import path). +// The resulting MemPackage will contain the names and content of all *.gno files, +// and additionally README.md, LICENSE, and gno.mod. +// +// ReadMemPackage does not perform validation aside from the package's name; +// the files are not parsed but their contents are merely stored inside a MemFile. +// +// NOTE: panics if package name is invalid (characters must be alphanumeric or _, +// lowercase, and must start with a letter). func ReadMemPackage(dir string, pkgPath string) *std.MemPackage { files, err := os.ReadDir(dir) if err != nil { panic(err) } - memPkg := &std.MemPackage{Path: pkgPath} allowedFiles := []string{ // make case insensitive? "gno.mod", "LICENSE", @@ -1107,27 +1115,43 @@ func ReadMemPackage(dir string, pkgPath string) *std.MemPackage { allowedFileExtensions := []string{ ".gno", } - var pkgName Name + list := make([]string, 0, len(files)) for _, file := range files { if file.IsDir() || strings.HasPrefix(file.Name(), ".") || (!endsWith(file.Name(), allowedFileExtensions) && !contains(allowedFiles, file.Name())) { continue } - fpath := filepath.Join(dir, file.Name()) + list = append(list, filepath.Join(dir, file.Name())) + } + return ReadMemPackageFromList(list, pkgPath) +} + +// ReadMemPackageFromList creates a new [std.MemPackage] with the specified pkgPath, +// containing the contents of all the files provided in the list slice. +// No parsing or validation is done on the filenames. +// +// NOTE: panics if package name is invalid (characters must be alphanumeric or _, +// lowercase, and must start with a letter). +func ReadMemPackageFromList(list []string, pkgPath string) *std.MemPackage { + memPkg := &std.MemPackage{Path: pkgPath} + var pkgName Name + for _, fpath := range list { + fname := filepath.Base(fpath) bz, err := os.ReadFile(fpath) if err != nil { panic(err) } - if pkgName == "" && strings.HasSuffix(file.Name(), ".gno") { - pkgName = PackageNameFromFileBody(file.Name(), string(bz)) + // XXX: should check that all pkg names are the same (else package is invalid) + if pkgName == "" && strings.HasSuffix(fname, ".gno") { + pkgName = PackageNameFromFileBody(fname, string(bz)) if strings.HasSuffix(string(pkgName), "_test") { pkgName = pkgName[:len(pkgName)-len("_test")] } } memPkg.Files = append(memPkg.Files, &std.MemFile{ - Name: file.Name(), + Name: fname, Body: string(bz), }) } @@ -1141,33 +1165,29 @@ func ReadMemPackage(dir string, pkgPath string) *std.MemPackage { return memPkg } -func PrecompileMemPackage(memPkg *std.MemPackage) error { - return nil -} - -// Returns the code fileset minus any spurious or test files. +// ParseMemPackage executes [ParseFile] on each file of the memPkg, excluding +// test and spurious (non-gno) files. The resulting *FileSet is returned. +// +// If one of the files has a different package name than memPkg.Name, +// or [ParseFile] returns an error, ParseMemPackage panics. func ParseMemPackage(memPkg *std.MemPackage) (fset *FileSet) { fset = &FileSet{} for _, mfile := range memPkg.Files { - if !strings.HasSuffix(mfile.Name, ".gno") { - continue // skip spurious file. + if !strings.HasSuffix(mfile.Name, ".gno") || + endsWith(mfile.Name, []string{"_test.gno", "_filetest.gno"}) { + continue // skip spurious or test file. } n, err := ParseFile(mfile.Name, mfile.Body) if err != nil { panic(errors.Wrap(err, "parsing file "+mfile.Name)) } - if strings.HasSuffix(mfile.Name, "_test.gno") { - // skip test file. - } else if strings.HasSuffix(mfile.Name, "_filetest.gno") { - // skip test file. - } else if memPkg.Name == string(n.PkgName) { - // add package file. - fset.AddFiles(n) - } else { + if memPkg.Name != string(n.PkgName) { panic(fmt.Sprintf( "expected package name [%s] but got [%s]", memPkg.Name, n.PkgName)) } + // add package file. + fset.AddFiles(n) } return fset } @@ -1236,7 +1256,11 @@ func (fs *FileSet) GetDeclFor(n Name) (*FileNode, *Decl) { func (fs *FileSet) GetDeclForSafe(n Name) (*FileNode, *Decl, bool) { // XXX index to bound to linear time. - for _, fn := range fs.Files { + + // Iteration happens reversing fs.Files; this is because the LAST declaration + // of n is what we are looking for. + for i := len(fs.Files) - 1; i >= 0; i-- { + fn := fs.Files[i] for i, dn := range fn.Decls { if _, isImport := dn.(*ImportDecl); isImport { // imports in other files don't count. @@ -1377,6 +1401,7 @@ func (x *PackageNode) DefineNative(n Name, ps, rs FieldTypeExprs, native func(*M if native == nil { panic("DefineNative expects a function, but got nil") } + fd := FuncD(n, ps, rs, nil) fd = Preprocess(nil, x, fd).(*FuncDecl) ft := evalStaticType(nil, x, &fd.Type).(*FuncType) @@ -1457,6 +1482,22 @@ type StaticBlock struct { Consts []Name // TODO consider merging with Names. Externs []Name Loc Location + + // temporary storage for rolling back redefinitions. + oldValues []oldValue +} + +type oldValue struct { + idx uint16 + value Value +} + +// revert values upon failure of redefinitions. +func (sb *StaticBlock) revertToOld() { + for _, ov := range sb.oldValues { + sb.Block.Values[ov.idx].V = ov.value + } + sb.oldValues = nil } // Implements BlockNode @@ -1656,7 +1697,6 @@ func (sb *StaticBlock) GetStaticTypeOfAt(store Store, path ValuePath) Type { path.Depth -= 1 } } - panic("should not happen") } // Implements BlockNode. @@ -1706,12 +1746,11 @@ func (sb *StaticBlock) GetValueRef(store Store, n Name) *TypedValue { // values, which are pre-computeed in the preprocessor. // Once a typed value is defined, it cannot be changed. // -// NOTE: Currently tv.V is only set when the value -// represents a Type(Value). The purpose of tv is to describe -// the invariant of a named value, at the minimum its type, -// but also sometimes the typeval value; but we could go -// further and store preprocessed constant results here -// too. See "anyValue()" and "asValue()" for usage. +// NOTE: Currently tv.V is only set when the value represents a Type(Value) or +// a FuncValue. The purpose of tv is to describe the invariant of a named +// value, at the minimum its type, but also sometimes the typeval value; but we +// could go further and store preprocessed constant results here too. See +// "anyValue()" and "asValue()" for usage. func (sb *StaticBlock) Define(n Name, tv TypedValue) { sb.Define2(false, n, tv.T, tv) } @@ -1754,16 +1793,32 @@ func (sb *StaticBlock) Define2(isConst bool, n Name, st Type, tv TypedValue) { } old := sb.Block.Values[idx] if !old.IsUndefined() { - if tv.T.TypeID() != old.T.TypeID() { - panic(fmt.Sprintf( - "StaticBlock.Define2(%s) cannot change .T; was %v, new %v", - n, old.T, tv.T)) - } - if tv.V != old.V { - panic(fmt.Sprintf( - "StaticBlock.Define2(%s) cannot change .V", - n)) + if tv.T.Kind() == FuncKind && tv.T.(*FuncType).IsZero() { + // special case, + // allow re-predefining for func upgrades. + // keep the old type so we can check it at preprocessor. + // fmt.Println("QWEQWEQWE>>>", old.String()) + // fmt.Println("QWEQWEQWE>>>", tv.String()) + tv.T = old.T + fv := tv.V.(*FuncValue) + fv.Type = old.T + st = old.T + sb.oldValues = append(sb.oldValues, + oldValue{idx, old.V}) + } else { + if tv.T.TypeID() != old.T.TypeID() { + panic(fmt.Sprintf( + "StaticBlock.Define2(%s) cannot change .T; was %v, new %v", + n, old.T, tv.T)) + } + if tv.V != old.V { + panic(fmt.Sprintf( + "StaticBlock.Define2(%s) cannot change .V", + n)) + } } + // Allow re-definitions if they have the same type. + // (In normal scenarios, duplicate declarations are "caught" by RunMemPackage.) } sb.Block.Values[idx] = tv sb.Types[idx] = st @@ -2024,6 +2079,7 @@ const ( ) // TODO: consider length restrictions. +// If this function is changed, ReadMemPackage's documentation should be updated accordingly. func validatePkgName(name string) { if nameOK, _ := regexp.MatchString( `^[a-z][a-z0-9_]+$`, name); !nameOK { diff --git a/gnovm/pkg/gnolang/op_call.go b/gnovm/pkg/gnolang/op_call.go index 8d652667111..eebee978919 100644 --- a/gnovm/pkg/gnolang/op_call.go +++ b/gnovm/pkg/gnolang/op_call.go @@ -58,6 +58,13 @@ func (m *Machine) doOpCall() { clo := fr.Func.GetClosure(m.Store) b := m.Alloc.NewBlock(fr.Func.GetSource(m.Store), clo) m.PushBlock(b) + if fv.nativeBody == nil && fv.NativePkg != "" { + // native function, unmarshaled so doesn't have nativeBody yet + fv.nativeBody = m.Store.GetNative(fv.NativePkg, fv.NativeName) + if fv.nativeBody == nil { + panic(fmt.Sprintf("natively defined function (%q).%s could not be resolved", fv.NativePkg, fv.NativeName)) + } + } if fv.nativeBody == nil { fbody := fv.GetBodyFromSource(m.Store) if len(ft.Results) == 0 { diff --git a/gnovm/pkg/gnolang/op_eval.go b/gnovm/pkg/gnolang/op_eval.go index a4c05382c18..283d035dca2 100644 --- a/gnovm/pkg/gnolang/op_eval.go +++ b/gnovm/pkg/gnolang/op_eval.go @@ -15,7 +15,7 @@ import ( func (m *Machine) doOpEval() { x := m.PeekExpr(1) if debug { - debug.Printf("EVAL: %v\n", x) + debug.Printf("EVAL: (%T) %v\n", x, x) // fmt.Println(m.String()) } // This case moved out of switch for performance. diff --git a/gnovm/pkg/gnolang/op_expressions.go b/gnovm/pkg/gnolang/op_expressions.go index b46e343b00e..b3bf240aea1 100644 --- a/gnovm/pkg/gnolang/op_expressions.go +++ b/gnovm/pkg/gnolang/op_expressions.go @@ -78,8 +78,12 @@ func (m *Machine) doOpIndex2() { func (m *Machine) doOpSelector() { sx := m.PopExpr().(*SelectorExpr) xv := m.PeekValue(1) - res := xv.GetPointerTo(m.Alloc, m.Store, sx.Path) - *xv = res.Deref() // reuse as result + res := xv.GetPointerTo(m.Alloc, m.Store, sx.Path).Deref() + if debug { + m.Printf("-v[S] %v\n", xv) + m.Printf("+v[S] %v\n", res) + } + *xv = res // reuse as result } func (m *Machine) doOpSlice() { diff --git a/gnovm/pkg/gnolang/preprocess.go b/gnovm/pkg/gnolang/preprocess.go index 2e428cfd344..ee943226c85 100644 --- a/gnovm/pkg/gnolang/preprocess.go +++ b/gnovm/pkg/gnolang/preprocess.go @@ -118,6 +118,18 @@ var preprocessing int // - Assigns BlockValuePath to NameExprs. // - TODO document what it does. func Preprocess(store Store, ctx BlockNode, n Node) Node { + // When panic, revert any package updates. + defer func() { + // Revert all new values. + // this is needed to revert top level + // function redeclarations. + if r := recover(); r != nil { + pkg := packageOf(ctx) + pkg.StaticBlock.revertToOld() + panic(r) + } + }() + // Increment preprocessing counter while preprocessing. { preprocessing += 1 @@ -1824,8 +1836,13 @@ func Preprocess(store Store, ctx BlockNode, n Node) Node { for i, vx := range n.Values { if cx, ok := vx.(*ConstExpr); ok && !cx.TypedValue.IsUndefined() { - // if value is non-nil const expr: - tvs[i] = cx.TypedValue + if n.Const { + // const _ = : static block should contain value + tvs[i] = cx.TypedValue + } else { + // var _ = : static block should NOT contain value + tvs[i] = anyValue(cx.TypedValue.T) + } } else { // for var decls of non-const expr. st := sts[i] @@ -2932,10 +2949,22 @@ func predefineNow2(store Store, last BlockNode, d Decl, m map[Name]struct{}) (De ft := ftv.T.(*FuncType) cd.Type = *Preprocess(store, last, &cd.Type).(*FuncTypeExpr) ft2 := evalStaticType(store, last, &cd.Type).(*FuncType) - *ft = *ft2 + if !ft.IsZero() { + // redefining function. + // make sure the type is the same. + if ft.TypeID() != ft2.TypeID() { + panic(fmt.Sprintf( + "Redefinition (%s) cannot change .T; was %v, new %v", + cd, ft, ft2)) + } + // keep the orig type. + } else { + *ft = *ft2 + } // XXX replace attr w/ ft? // return Preprocess(store, last, cd).(Decl), true } + // Full type declaration/preprocessing already done in tryPredefine return d, false case *ValueDecl: return Preprocess(store, last, cd).(Decl), true @@ -3129,19 +3158,30 @@ func tryPredefine(store Store, last BlockNode, d Decl) (un Name) { } // define a FuncValue w/ above type as d.Name. // fill in later during *FuncDecl:BLOCK. + fv := &FuncValue{ + Type: ft, + IsMethod: false, + Source: d, + Name: d.Name, + Closure: nil, // set lazily. + FileName: fileNameOf(last), + PkgPath: pkg.PkgPath, + body: d.Body, + nativeBody: nil, + } + // NOTE: fv.body == nil means no body (ie. not even curly braces) + // len(fv.body) == 0 could mean also {} (ie. no statements inside) + if fv.body == nil && store != nil { + fv.nativeBody = store.GetNative(pkg.PkgPath, d.Name) + if fv.nativeBody == nil { + panic(fmt.Sprintf("function %s does not have a body but is not natively defined", d.Name)) + } + fv.NativePkg = pkg.PkgPath + fv.NativeName = d.Name + } pkg.Define(d.Name, TypedValue{ T: ft, - V: &FuncValue{ - Type: ft, - IsMethod: false, - Source: d, - Name: d.Name, - Closure: nil, // set lazily. - FileName: fileNameOf(last), - PkgPath: pkg.PkgPath, - body: d.Body, - nativeBody: nil, - }, + V: fv, }) if d.Name == "init" { // init functions can't be referenced. diff --git a/gnovm/pkg/gnolang/realm.go b/gnovm/pkg/gnolang/realm.go index 567aea58284..d10a9280301 100644 --- a/gnovm/pkg/gnolang/realm.go +++ b/gnovm/pkg/gnolang/realm.go @@ -1129,18 +1129,23 @@ func copyValueWithRefs(parent Object, val Value) Value { if cv.Closure != nil { closure = toRefValue(parent, cv.Closure) } - if cv.nativeBody != nil { + // nativeBody funcs which don't come from NativeStore (and thus don't + // have NativePkg/Name) can't be persisted, and should not be able + // to get here anyway. + if cv.nativeBody != nil && cv.NativePkg == "" { panic("should not happen") } ft := copyTypeWithRefs(cv.Type) return &FuncValue{ - Type: ft, - IsMethod: cv.IsMethod, - Source: source, - Name: cv.Name, - Closure: closure, - FileName: cv.FileName, - PkgPath: cv.PkgPath, + Type: ft, + IsMethod: cv.IsMethod, + Source: source, + Name: cv.Name, + Closure: closure, + FileName: cv.FileName, + PkgPath: cv.PkgPath, + NativePkg: cv.NativePkg, + NativeName: cv.NativeName, } case *BoundMethodValue: fnc := copyValueWithRefs(cv, cv.Func).(*FuncValue) diff --git a/gnovm/pkg/gnolang/store.go b/gnovm/pkg/gnolang/store.go index 24aff4936f3..be4a854e7ce 100644 --- a/gnovm/pkg/gnolang/store.go +++ b/gnovm/pkg/gnolang/store.go @@ -17,6 +17,9 @@ type PackageGetter func(pkgPath string) (*PackageNode, *PackageValue) // inject natives into a new or loaded package (value and node) type PackageInjector func(store Store, pn *PackageNode) +// NativeStore is a function which can retrieve native bodies of native functions. +type NativeStore func(pkgName string, name Name) func(m *Machine) + type Store interface { // STABLE SetPackageGetter(PackageGetter) @@ -48,10 +51,12 @@ type Store interface { GetMemPackage(path string) *std.MemPackage GetMemFile(path string, name string) *std.MemFile IterMemPackage() <-chan *std.MemPackage - ClearObjectCache() // for each delivertx. - Fork() Store // for checktx, simulate, and queries. - SwapStores(baseStore, iavlStore store.Store) // for gas wrappers. - SetPackageInjector(PackageInjector) // for natives + ClearObjectCache() // for each delivertx. + Fork() Store // for checktx, simulate, and queries. + SwapStores(baseStore, iavlStore store.Store) // for gas wrappers. + SetPackageInjector(PackageInjector) // for natives + SetNativeStore(NativeStore) // for "new" natives XXX + GetNative(pkgPath string, name Name) func(m *Machine) // for "new" natives XXX SetLogStoreOps(enabled bool) SprintStoreOps() string LogSwitchRealm(rlmpath string) // to mark change of realm boundaries @@ -70,6 +75,7 @@ type defaultStore struct { baseStore store.Store // for objects, types, nodes iavlStore store.Store // for escaped object hashes pkgInjector PackageInjector // for injecting natives + nativeStore NativeStore // for injecting natives go2gnoMap map[string]string // go pkgpath.name -> gno pkgpath.name go2gnoStrict bool // if true, native->gno type conversion must be registered. @@ -601,6 +607,7 @@ func (ds *defaultStore) Fork() Store { baseStore: ds.baseStore, iavlStore: ds.iavlStore, pkgInjector: ds.pkgInjector, + nativeStore: ds.nativeStore, go2gnoMap: ds.go2gnoMap, go2gnoStrict: ds.go2gnoStrict, opslog: nil, // new ops log. @@ -620,6 +627,17 @@ func (ds *defaultStore) SetPackageInjector(inj PackageInjector) { ds.pkgInjector = inj } +func (ds *defaultStore) SetNativeStore(ns NativeStore) { + ds.nativeStore = ns +} + +func (ds *defaultStore) GetNative(pkgPath string, name Name) func(m *Machine) { + if ds.nativeStore != nil { + return ds.nativeStore(pkgPath, name) + } + return nil +} + func (ds *defaultStore) Flush() { // XXX } diff --git a/gnovm/pkg/gnolang/types.go b/gnovm/pkg/gnolang/types.go index 6aed71fcf9b..65bf7227458 100644 --- a/gnovm/pkg/gnolang/types.go +++ b/gnovm/pkg/gnolang/types.go @@ -1084,6 +1084,12 @@ type FuncType struct { bound *FuncType } +// true for predefined func types that are not filled in yet. +func (ft *FuncType) IsZero() bool { + // XXX be explicit. + return ft.Params == nil && ft.Results == nil && ft.typeid.IsZero() && ft.bound == nil +} + // if ft is a method, returns whether method takes a pointer receiver. func (ft *FuncType) HasPointerReceiver() bool { if debug { @@ -1445,6 +1451,46 @@ func (dt *DeclaredType) GetPkgPath() string { } func (dt *DeclaredType) DefineMethod(fv *FuncValue) { + name := fv.Name + + // Handle redeclarations. + for i, tv := range dt.Methods { + ofv := tv.V.(*FuncValue) + if ofv.Name == name { + // Do not allow redeclaring (override) a method. + // In the future we may allow this, just like we + // allow package-level function overrides. + + // Special case: if the type and location are the same, + // ignore and do not redefine. + // This is due to PreprocessAllFilesAndSaveBlocknodes, + // and because the preprocessor fills some of the + // method's FuncValue. Since the method was already + // filled in prior to PreprocessAllFilesAndSaveBlocks, + // there is no need to re-set it. + // Keep this or move this check outside. + if fv.Type.TypeID() == ofv.Type.TypeID() && + fv.Source.GetLocation() == ofv.Source.GetLocation() { + return + } + + // Special case: allow defining a native body. + if fv.Type.TypeID() == ofv.Type.TypeID() && + !ofv.IsNative() && fv.IsNative() { + dt.Methods[i] = TypedValue{ + T: fv.Type, // keep old type. + V: fv, + } + return + } + + // Otherwise panic. + panic(fmt.Sprintf("redeclaration of method %s.%s", + dt.Name, name)) + } + } + + // If not redeclaring, just append. dt.Methods = append(dt.Methods, TypedValue{ T: fv.Type, V: fv, diff --git a/gnovm/pkg/gnolang/values.go b/gnovm/pkg/gnolang/values.go index d8c656d2ca3..3607ccc4669 100644 --- a/gnovm/pkg/gnolang/values.go +++ b/gnovm/pkg/gnolang/values.go @@ -527,18 +527,32 @@ func (sv *StructValue) Copy(alloc *Allocator) *StructValue { // makes construction TypedValue{T:*FuncType{},V:*FuncValue{}} // faster. type FuncValue struct { - Type Type // includes unbound receiver(s) - IsMethod bool // is an (unbound) method - Source BlockNode // for block mem allocation - Name Name // name of function/method - Closure Value // *Block or RefValue to closure (may be nil for file blocks; lazy) - FileName Name // file name where declared - PkgPath string + Type Type // includes unbound receiver(s) + IsMethod bool // is an (unbound) method + Source BlockNode // for block mem allocation + Name Name // name of function/method + Closure Value // *Block or RefValue to closure (may be nil for file blocks; lazy) + FileName Name // file name where declared + PkgPath string + NativePkg string // for native bindings through NativeStore + NativeName Name // not redundant with Name; this cannot be changed in userspace body []Stmt // function body nativeBody func(*Machine) // alternative to Body } +func (fv *FuncValue) IsNative() bool { + if fv.NativePkg == "" && fv.NativeName == "" { + return false + } + if fv.NativePkg == "" || fv.NativeName == "" { + panic(fmt.Sprintf("function (%q).%s has invalid native pkg/name ((%q).%s)", + fv.Source.GetLocation().PkgPath, fv.Name, + fv.NativePkg, fv.NativeName)) + } + return true +} + func (fv *FuncValue) Copy(alloc *Allocator) *FuncValue { alloc.AllocateFunc() return &FuncValue{ @@ -549,6 +563,8 @@ func (fv *FuncValue) Copy(alloc *Allocator) *FuncValue { Closure: fv.Closure, FileName: fv.FileName, PkgPath: fv.PkgPath, + NativePkg: fv.NativePkg, + NativeName: fv.NativeName, body: fv.body, nativeBody: fv.nativeBody, } @@ -2321,12 +2337,8 @@ func (b *Block) GetPointerTo(store Store, path ValuePath) PointerValue { // the generation for uverse is 0. If path.Depth is // 0, it implies that b == uverse, and the condition // would fail as if it were 1. - i := uint8(1) -LOOP: - if i < path.Depth { + for i := uint8(1); i < path.Depth; i++ { b = b.GetParent(store) - i++ - goto LOOP } return b.GetPointerToInt(store, int(path.Index)) } diff --git a/gnovm/stdlibs/crypto/sha256/sha256.gno b/gnovm/stdlibs/crypto/sha256/sha256.gno index efb8ebb8e37..c36313f5482 100644 --- a/gnovm/stdlibs/crypto/sha256/sha256.gno +++ b/gnovm/stdlibs/crypto/sha256/sha256.gno @@ -1,12 +1,8 @@ package sha256 -import ( - isha256 "internal/crypto/sha256" -) - const Size = 32 // Sum returns the SHA-256 checksum of the data. -func Sum256(data []byte) [Size]byte { - return isha256.Sum256(data) -} +func Sum256(data []byte) [Size]byte { return sum256(data) } + +func sum256(data []byte) [32]byte // injected diff --git a/gnovm/stdlibs/crypto/sha256/sha256.go b/gnovm/stdlibs/crypto/sha256/sha256.go new file mode 100644 index 00000000000..c3aac1106e2 --- /dev/null +++ b/gnovm/stdlibs/crypto/sha256/sha256.go @@ -0,0 +1,7 @@ +package sha256 + +import "crypto/sha256" + +func X_sum256(data []byte) [32]byte { + return sha256.Sum256(data) +} diff --git a/gnovm/stdlibs/encoding/base64/base64.gno b/gnovm/stdlibs/encoding/base64/base64.gno index 7889a548832..ea3b0a55c2a 100644 --- a/gnovm/stdlibs/encoding/base64/base64.gno +++ b/gnovm/stdlibs/encoding/base64/base64.gno @@ -496,7 +496,9 @@ func (enc *Encoding) Decode(dst, src []byte) (n int, err error) { _ = enc.decodeMap si := 0 - for strconv.IntSize >= 64 && len(src)-si >= 8 && len(dst)-n >= 8 { + // XXX: Go source checks for strconv.IntSize >= 64 as well in this loop. + // In the gnovm, int size is always guaranteed to be 64 bits. + for len(src)-si >= 8 && len(dst)-n >= 8 { src2 := src[si : si+8] if dn, ok := assemble64( enc.decodeMap[src2[0]], diff --git a/gnovm/stdlibs/internal/crypto/sha256/sha256.gno b/gnovm/stdlibs/internal/crypto/sha256/sha256.gno deleted file mode 100644 index 458b2123f59..00000000000 --- a/gnovm/stdlibs/internal/crypto/sha256/sha256.gno +++ /dev/null @@ -1,3 +0,0 @@ -package sha256 - -// XXX injected via stdlibs/stdlibs.go diff --git a/gnovm/stdlibs/internal/math/math.gno b/gnovm/stdlibs/internal/math/math.gno deleted file mode 100644 index 42245392caf..00000000000 --- a/gnovm/stdlibs/internal/math/math.gno +++ /dev/null @@ -1,3 +0,0 @@ -package math - -// XXX injected via stdlibs/stdlibs.go diff --git a/gnovm/stdlibs/math/native.gno b/gnovm/stdlibs/math/native.gno new file mode 100644 index 00000000000..b1b5684f9af --- /dev/null +++ b/gnovm/stdlibs/math/native.gno @@ -0,0 +1,21 @@ +package math + +// Float32bits returns the IEEE 754 binary representation of f, with the sign +// bit of f and the result in the same bit position. +// Float32bits(Float32frombits(x)) == x. +func Float32bits(f float32) uint32 // injected + +// Float32frombits returns the floating-point number corresponding to the IEEE +// 754 binary representation b, with the sign bit of b and the result in the +// same bit position. Float32frombits(Float32bits(x)) == x. +func Float32frombits(b uint32) float32 // injected + +// Float64bits returns the IEEE 754 binary representation of f, with the sign +// bit of f and the result in the same bit position. +// Float64bits(Float64frombits(x)) == x. +func Float64bits(f float64) uint64 // injected + +// Float64frombits returns the floating-point number corresponding to the IEEE +// 754 binary representation b, with the sign bit of b and the result in the +// same bit position. Float64frombits(Float64bits(x)) == x. +func Float64frombits(b uint64) float64 // injected diff --git a/gnovm/stdlibs/math/native.go b/gnovm/stdlibs/math/native.go new file mode 100644 index 00000000000..21021085f6d --- /dev/null +++ b/gnovm/stdlibs/math/native.go @@ -0,0 +1,8 @@ +package math + +import "math" + +func Float32bits(f float32) uint32 { return math.Float32bits(f) } +func Float32frombits(b uint32) float32 { return math.Float32frombits(b) } +func Float64bits(f float64) uint64 { return math.Float64bits(f) } +func Float64frombits(b uint64) float64 { return math.Float64frombits(b) } diff --git a/gnovm/stdlibs/math/unsafe.gno b/gnovm/stdlibs/math/unsafe.gno deleted file mode 100644 index 92d316bab2c..00000000000 --- a/gnovm/stdlibs/math/unsafe.gno +++ /dev/null @@ -1,29 +0,0 @@ -// Copyright 2009 The Go Authors. All rights reserved. -// Use of this source code is governed by a BSD-style -// license that can be found in the LICENSE file. - -package math - -import imath "internal/math" - -// Float32bits returns the IEEE 754 binary representation of f, -// with the sign bit of f and the result in the same bit position. -// Float32bits(Float32frombits(x)) == x. -func Float32bits(f float32) uint32 { return imath.Float32bits(f) } - -// Float32frombits returns the floating-point number corresponding -// to the IEEE 754 binary representation b, with the sign bit of b -// and the result in the same bit position. -// Float32frombits(Float32bits(x)) == x. -func Float32frombits(b uint32) float32 { return imath.Float32frombits(b) } - -// Float64bits returns the IEEE 754 binary representation of f, -// with the sign bit of f and the result in the same bit position, -// and Float64bits(Float64frombits(x)) == x. -func Float64bits(f float64) uint64 { return imath.Float64bits(f) } - -// Float64frombits returns the floating-point number corresponding -// to the IEEE 754 binary representation b, with the sign bit of b -// and the result in the same bit position. -// Float64frombits(Float64bits(x)) == x. -func Float64frombits(b uint64) float64 { return imath.Float64frombits(b) } diff --git a/gnovm/stdlibs/native.go b/gnovm/stdlibs/native.go new file mode 100644 index 00000000000..f85780f2021 --- /dev/null +++ b/gnovm/stdlibs/native.go @@ -0,0 +1,764 @@ +// This file is autogenerated from the genstd tool (@/misc/genstd); do not edit. +// To regenerate it, run `go generate` from this directory. + +package stdlibs + +import ( + "reflect" + + gno "github.com/gnolang/gno/gnovm/pkg/gnolang" + libs_crypto_sha256 "github.com/gnolang/gno/gnovm/stdlibs/crypto/sha256" + libs_math "github.com/gnolang/gno/gnovm/stdlibs/math" + libs_std "github.com/gnolang/gno/gnovm/stdlibs/std" + libs_strconv "github.com/gnolang/gno/gnovm/stdlibs/strconv" + libs_time "github.com/gnolang/gno/gnovm/stdlibs/time" + tm2_crypto "github.com/gnolang/gno/tm2/pkg/crypto" +) + +type nativeFunc struct { + gnoPkg string + gnoFunc gno.Name + params []gno.FieldTypeExpr + results []gno.FieldTypeExpr + f func(m *gno.Machine) +} + +var nativeFuncs = [...]nativeFunc{ + { + "crypto/sha256", + "sum256", + []gno.FieldTypeExpr{ + {Name: gno.N("p0"), Type: gno.X("[]byte")}, + }, + []gno.FieldTypeExpr{ + {Name: gno.N("r0"), Type: gno.X("[32]byte")}, + }, + func(m *gno.Machine) { + b := m.LastBlock() + var ( + p0 []byte + rp0 = reflect.ValueOf(&p0).Elem() + ) + + gno.Gno2GoValue(b.GetPointerTo(nil, gno.NewValuePathBlock(1, 0, "")).TV, rp0) + + r0 := libs_crypto_sha256.X_sum256(p0) + + m.PushValue(gno.Go2GnoValue( + m.Alloc, + m.Store, + reflect.ValueOf(&r0).Elem(), + )) + }, + }, + { + "math", + "Float32bits", + []gno.FieldTypeExpr{ + {Name: gno.N("p0"), Type: gno.X("float32")}, + }, + []gno.FieldTypeExpr{ + {Name: gno.N("r0"), Type: gno.X("uint32")}, + }, + func(m *gno.Machine) { + b := m.LastBlock() + var ( + p0 float32 + rp0 = reflect.ValueOf(&p0).Elem() + ) + + gno.Gno2GoValue(b.GetPointerTo(nil, gno.NewValuePathBlock(1, 0, "")).TV, rp0) + + r0 := libs_math.Float32bits(p0) + + m.PushValue(gno.Go2GnoValue( + m.Alloc, + m.Store, + reflect.ValueOf(&r0).Elem(), + )) + }, + }, + { + "math", + "Float32frombits", + []gno.FieldTypeExpr{ + {Name: gno.N("p0"), Type: gno.X("uint32")}, + }, + []gno.FieldTypeExpr{ + {Name: gno.N("r0"), Type: gno.X("float32")}, + }, + func(m *gno.Machine) { + b := m.LastBlock() + var ( + p0 uint32 + rp0 = reflect.ValueOf(&p0).Elem() + ) + + gno.Gno2GoValue(b.GetPointerTo(nil, gno.NewValuePathBlock(1, 0, "")).TV, rp0) + + r0 := libs_math.Float32frombits(p0) + + m.PushValue(gno.Go2GnoValue( + m.Alloc, + m.Store, + reflect.ValueOf(&r0).Elem(), + )) + }, + }, + { + "math", + "Float64bits", + []gno.FieldTypeExpr{ + {Name: gno.N("p0"), Type: gno.X("float64")}, + }, + []gno.FieldTypeExpr{ + {Name: gno.N("r0"), Type: gno.X("uint64")}, + }, + func(m *gno.Machine) { + b := m.LastBlock() + var ( + p0 float64 + rp0 = reflect.ValueOf(&p0).Elem() + ) + + gno.Gno2GoValue(b.GetPointerTo(nil, gno.NewValuePathBlock(1, 0, "")).TV, rp0) + + r0 := libs_math.Float64bits(p0) + + m.PushValue(gno.Go2GnoValue( + m.Alloc, + m.Store, + reflect.ValueOf(&r0).Elem(), + )) + }, + }, + { + "math", + "Float64frombits", + []gno.FieldTypeExpr{ + {Name: gno.N("p0"), Type: gno.X("uint64")}, + }, + []gno.FieldTypeExpr{ + {Name: gno.N("r0"), Type: gno.X("float64")}, + }, + func(m *gno.Machine) { + b := m.LastBlock() + var ( + p0 uint64 + rp0 = reflect.ValueOf(&p0).Elem() + ) + + gno.Gno2GoValue(b.GetPointerTo(nil, gno.NewValuePathBlock(1, 0, "")).TV, rp0) + + r0 := libs_math.Float64frombits(p0) + + m.PushValue(gno.Go2GnoValue( + m.Alloc, + m.Store, + reflect.ValueOf(&r0).Elem(), + )) + }, + }, + { + "std", + "AssertOriginCall", + []gno.FieldTypeExpr{}, + []gno.FieldTypeExpr{}, + func(m *gno.Machine) { + libs_std.AssertOriginCall( + m, + ) + }, + }, + { + "std", + "IsOriginCall", + []gno.FieldTypeExpr{}, + []gno.FieldTypeExpr{ + {Name: gno.N("r0"), Type: gno.X("bool")}, + }, + func(m *gno.Machine) { + r0 := libs_std.IsOriginCall( + m, + ) + + m.PushValue(gno.Go2GnoValue( + m.Alloc, + m.Store, + reflect.ValueOf(&r0).Elem(), + )) + }, + }, + { + "std", + "CurrentRealmPath", + []gno.FieldTypeExpr{}, + []gno.FieldTypeExpr{ + {Name: gno.N("r0"), Type: gno.X("string")}, + }, + func(m *gno.Machine) { + r0 := libs_std.CurrentRealmPath( + m, + ) + + m.PushValue(gno.Go2GnoValue( + m.Alloc, + m.Store, + reflect.ValueOf(&r0).Elem(), + )) + }, + }, + { + "std", + "GetChainID", + []gno.FieldTypeExpr{}, + []gno.FieldTypeExpr{ + {Name: gno.N("r0"), Type: gno.X("string")}, + }, + func(m *gno.Machine) { + r0 := libs_std.GetChainID( + m, + ) + + m.PushValue(gno.Go2GnoValue( + m.Alloc, + m.Store, + reflect.ValueOf(&r0).Elem(), + )) + }, + }, + { + "std", + "GetHeight", + []gno.FieldTypeExpr{}, + []gno.FieldTypeExpr{ + {Name: gno.N("r0"), Type: gno.X("int64")}, + }, + func(m *gno.Machine) { + r0 := libs_std.GetHeight( + m, + ) + + m.PushValue(gno.Go2GnoValue( + m.Alloc, + m.Store, + reflect.ValueOf(&r0).Elem(), + )) + }, + }, + { + "std", + "GetOrigSend", + []gno.FieldTypeExpr{}, + []gno.FieldTypeExpr{ + {Name: gno.N("r0"), Type: gno.X("Coins")}, + }, + func(m *gno.Machine) { + r0 := libs_std.GetOrigSend( + m, + ) + + m.PushValue(gno.Go2GnoValue( + m.Alloc, + m.Store, + reflect.ValueOf(&r0).Elem(), + )) + }, + }, + { + "std", + "GetOrigCaller", + []gno.FieldTypeExpr{}, + []gno.FieldTypeExpr{ + {Name: gno.N("r0"), Type: gno.X("Address")}, + }, + func(m *gno.Machine) { + r0 := libs_std.GetOrigCaller( + m, + ) + + m.PushValue(gno.Go2GnoValue( + m.Alloc, + m.Store, + reflect.ValueOf(&r0).Elem(), + )) + }, + }, + { + "std", + "CurrentRealm", + []gno.FieldTypeExpr{}, + []gno.FieldTypeExpr{ + {Name: gno.N("r0"), Type: gno.X("Realm")}, + }, + func(m *gno.Machine) { + r0 := libs_std.CurrentRealm( + m, + ) + + m.PushValue(gno.Go2GnoValue( + m.Alloc, + m.Store, + reflect.ValueOf(&r0).Elem(), + )) + }, + }, + { + "std", + "PrevRealm", + []gno.FieldTypeExpr{}, + []gno.FieldTypeExpr{ + {Name: gno.N("r0"), Type: gno.X("Realm")}, + }, + func(m *gno.Machine) { + r0 := libs_std.PrevRealm( + m, + ) + + m.PushValue(gno.Go2GnoValue( + m.Alloc, + m.Store, + reflect.ValueOf(&r0).Elem(), + )) + }, + }, + { + "std", + "GetOrigPkgAddr", + []gno.FieldTypeExpr{}, + []gno.FieldTypeExpr{ + {Name: gno.N("r0"), Type: gno.X("Address")}, + }, + func(m *gno.Machine) { + r0 := libs_std.GetOrigPkgAddr( + m, + ) + + m.PushValue(gno.Go2GnoValue( + m.Alloc, + m.Store, + reflect.ValueOf(&r0).Elem(), + )) + }, + }, + { + "std", + "GetCallerAt", + []gno.FieldTypeExpr{ + {Name: gno.N("p0"), Type: gno.X("int")}, + }, + []gno.FieldTypeExpr{ + {Name: gno.N("r0"), Type: gno.X("Address")}, + }, + func(m *gno.Machine) { + b := m.LastBlock() + var ( + p0 int + rp0 = reflect.ValueOf(&p0).Elem() + ) + + gno.Gno2GoValue(b.GetPointerTo(nil, gno.NewValuePathBlock(1, 0, "")).TV, rp0) + + r0 := libs_std.GetCallerAt( + m, + p0) + + m.PushValue(gno.Go2GnoValue( + m.Alloc, + m.Store, + reflect.ValueOf(&r0).Elem(), + )) + }, + }, + { + "std", + "GetBanker", + []gno.FieldTypeExpr{ + {Name: gno.N("p0"), Type: gno.X("BankerType")}, + }, + []gno.FieldTypeExpr{ + {Name: gno.N("r0"), Type: gno.X("Banker")}, + }, + func(m *gno.Machine) { + b := m.LastBlock() + var ( + p0 libs_std.BankerType + rp0 = reflect.ValueOf(&p0).Elem() + ) + + gno.Gno2GoValue(b.GetPointerTo(nil, gno.NewValuePathBlock(1, 0, "")).TV, rp0) + + r0 := libs_std.GetBanker( + m, + p0) + + m.PushValue(r0) + }, + }, + { + "std", + "DerivePkgAddr", + []gno.FieldTypeExpr{ + {Name: gno.N("p0"), Type: gno.X("string")}, + }, + []gno.FieldTypeExpr{ + {Name: gno.N("r0"), Type: gno.X("Address")}, + }, + func(m *gno.Machine) { + b := m.LastBlock() + var ( + p0 string + rp0 = reflect.ValueOf(&p0).Elem() + ) + + gno.Gno2GoValue(b.GetPointerTo(nil, gno.NewValuePathBlock(1, 0, "")).TV, rp0) + + r0 := libs_std.DerivePkgAddr(p0) + + m.PushValue(gno.Go2GnoValue( + m.Alloc, + m.Store, + reflect.ValueOf(&r0).Elem(), + )) + }, + }, + { + "std", + "EncodeBech32", + []gno.FieldTypeExpr{ + {Name: gno.N("p0"), Type: gno.X("string")}, + {Name: gno.N("p1"), Type: gno.X("[20]byte")}, + }, + []gno.FieldTypeExpr{ + {Name: gno.N("r0"), Type: gno.X("Address")}, + }, + func(m *gno.Machine) { + b := m.LastBlock() + var ( + p0 string + rp0 = reflect.ValueOf(&p0).Elem() + p1 [20]byte + rp1 = reflect.ValueOf(&p1).Elem() + ) + + gno.Gno2GoValue(b.GetPointerTo(nil, gno.NewValuePathBlock(1, 0, "")).TV, rp0) + gno.Gno2GoValue(b.GetPointerTo(nil, gno.NewValuePathBlock(1, 1, "")).TV, rp1) + + r0 := libs_std.EncodeBech32(p0, p1) + + m.PushValue(gno.Go2GnoValue( + m.Alloc, + m.Store, + reflect.ValueOf(&r0).Elem(), + )) + }, + }, + { + "std", + "DecodeBech32", + []gno.FieldTypeExpr{ + {Name: gno.N("p0"), Type: gno.X("Address")}, + }, + []gno.FieldTypeExpr{ + {Name: gno.N("r0"), Type: gno.X("string")}, + {Name: gno.N("r1"), Type: gno.X("[20]byte")}, + {Name: gno.N("r2"), Type: gno.X("bool")}, + }, + func(m *gno.Machine) { + b := m.LastBlock() + var ( + p0 tm2_crypto.Bech32Address + rp0 = reflect.ValueOf(&p0).Elem() + ) + + gno.Gno2GoValue(b.GetPointerTo(nil, gno.NewValuePathBlock(1, 0, "")).TV, rp0) + + r0, r1, r2 := libs_std.DecodeBech32(p0) + + m.PushValue(gno.Go2GnoValue( + m.Alloc, + m.Store, + reflect.ValueOf(&r0).Elem(), + )) + m.PushValue(gno.Go2GnoValue( + m.Alloc, + m.Store, + reflect.ValueOf(&r1).Elem(), + )) + m.PushValue(gno.Go2GnoValue( + m.Alloc, + m.Store, + reflect.ValueOf(&r2).Elem(), + )) + }, + }, + { + "strconv", + "Itoa", + []gno.FieldTypeExpr{ + {Name: gno.N("p0"), Type: gno.X("int")}, + }, + []gno.FieldTypeExpr{ + {Name: gno.N("r0"), Type: gno.X("string")}, + }, + func(m *gno.Machine) { + b := m.LastBlock() + var ( + p0 int + rp0 = reflect.ValueOf(&p0).Elem() + ) + + gno.Gno2GoValue(b.GetPointerTo(nil, gno.NewValuePathBlock(1, 0, "")).TV, rp0) + + r0 := libs_strconv.Itoa(p0) + + m.PushValue(gno.Go2GnoValue( + m.Alloc, + m.Store, + reflect.ValueOf(&r0).Elem(), + )) + }, + }, + { + "strconv", + "AppendUint", + []gno.FieldTypeExpr{ + {Name: gno.N("p0"), Type: gno.X("[]byte")}, + {Name: gno.N("p1"), Type: gno.X("uint64")}, + {Name: gno.N("p2"), Type: gno.X("int")}, + }, + []gno.FieldTypeExpr{ + {Name: gno.N("r0"), Type: gno.X("[]byte")}, + }, + func(m *gno.Machine) { + b := m.LastBlock() + var ( + p0 []byte + rp0 = reflect.ValueOf(&p0).Elem() + p1 uint64 + rp1 = reflect.ValueOf(&p1).Elem() + p2 int + rp2 = reflect.ValueOf(&p2).Elem() + ) + + gno.Gno2GoValue(b.GetPointerTo(nil, gno.NewValuePathBlock(1, 0, "")).TV, rp0) + gno.Gno2GoValue(b.GetPointerTo(nil, gno.NewValuePathBlock(1, 1, "")).TV, rp1) + gno.Gno2GoValue(b.GetPointerTo(nil, gno.NewValuePathBlock(1, 2, "")).TV, rp2) + + r0 := libs_strconv.AppendUint(p0, p1, p2) + + m.PushValue(gno.Go2GnoValue( + m.Alloc, + m.Store, + reflect.ValueOf(&r0).Elem(), + )) + }, + }, + { + "strconv", + "Atoi", + []gno.FieldTypeExpr{ + {Name: gno.N("p0"), Type: gno.X("string")}, + }, + []gno.FieldTypeExpr{ + {Name: gno.N("r0"), Type: gno.X("int")}, + {Name: gno.N("r1"), Type: gno.X("error")}, + }, + func(m *gno.Machine) { + b := m.LastBlock() + var ( + p0 string + rp0 = reflect.ValueOf(&p0).Elem() + ) + + gno.Gno2GoValue(b.GetPointerTo(nil, gno.NewValuePathBlock(1, 0, "")).TV, rp0) + + r0, r1 := libs_strconv.Atoi(p0) + + m.PushValue(gno.Go2GnoValue( + m.Alloc, + m.Store, + reflect.ValueOf(&r0).Elem(), + )) + m.PushValue(gno.Go2GnoValue( + m.Alloc, + m.Store, + reflect.ValueOf(&r1).Elem(), + )) + }, + }, + { + "strconv", + "CanBackquote", + []gno.FieldTypeExpr{ + {Name: gno.N("p0"), Type: gno.X("string")}, + }, + []gno.FieldTypeExpr{ + {Name: gno.N("r0"), Type: gno.X("bool")}, + }, + func(m *gno.Machine) { + b := m.LastBlock() + var ( + p0 string + rp0 = reflect.ValueOf(&p0).Elem() + ) + + gno.Gno2GoValue(b.GetPointerTo(nil, gno.NewValuePathBlock(1, 0, "")).TV, rp0) + + r0 := libs_strconv.CanBackquote(p0) + + m.PushValue(gno.Go2GnoValue( + m.Alloc, + m.Store, + reflect.ValueOf(&r0).Elem(), + )) + }, + }, + { + "strconv", + "FormatInt", + []gno.FieldTypeExpr{ + {Name: gno.N("p0"), Type: gno.X("int64")}, + {Name: gno.N("p1"), Type: gno.X("int")}, + }, + []gno.FieldTypeExpr{ + {Name: gno.N("r0"), Type: gno.X("string")}, + }, + func(m *gno.Machine) { + b := m.LastBlock() + var ( + p0 int64 + rp0 = reflect.ValueOf(&p0).Elem() + p1 int + rp1 = reflect.ValueOf(&p1).Elem() + ) + + gno.Gno2GoValue(b.GetPointerTo(nil, gno.NewValuePathBlock(1, 0, "")).TV, rp0) + gno.Gno2GoValue(b.GetPointerTo(nil, gno.NewValuePathBlock(1, 1, "")).TV, rp1) + + r0 := libs_strconv.FormatInt(p0, p1) + + m.PushValue(gno.Go2GnoValue( + m.Alloc, + m.Store, + reflect.ValueOf(&r0).Elem(), + )) + }, + }, + { + "strconv", + "FormatUint", + []gno.FieldTypeExpr{ + {Name: gno.N("p0"), Type: gno.X("uint64")}, + {Name: gno.N("p1"), Type: gno.X("int")}, + }, + []gno.FieldTypeExpr{ + {Name: gno.N("r0"), Type: gno.X("string")}, + }, + func(m *gno.Machine) { + b := m.LastBlock() + var ( + p0 uint64 + rp0 = reflect.ValueOf(&p0).Elem() + p1 int + rp1 = reflect.ValueOf(&p1).Elem() + ) + + gno.Gno2GoValue(b.GetPointerTo(nil, gno.NewValuePathBlock(1, 0, "")).TV, rp0) + gno.Gno2GoValue(b.GetPointerTo(nil, gno.NewValuePathBlock(1, 1, "")).TV, rp1) + + r0 := libs_strconv.FormatUint(p0, p1) + + m.PushValue(gno.Go2GnoValue( + m.Alloc, + m.Store, + reflect.ValueOf(&r0).Elem(), + )) + }, + }, + { + "strconv", + "Quote", + []gno.FieldTypeExpr{ + {Name: gno.N("p0"), Type: gno.X("string")}, + }, + []gno.FieldTypeExpr{ + {Name: gno.N("r0"), Type: gno.X("string")}, + }, + func(m *gno.Machine) { + b := m.LastBlock() + var ( + p0 string + rp0 = reflect.ValueOf(&p0).Elem() + ) + + gno.Gno2GoValue(b.GetPointerTo(nil, gno.NewValuePathBlock(1, 0, "")).TV, rp0) + + r0 := libs_strconv.Quote(p0) + + m.PushValue(gno.Go2GnoValue( + m.Alloc, + m.Store, + reflect.ValueOf(&r0).Elem(), + )) + }, + }, + { + "strconv", + "QuoteToASCII", + []gno.FieldTypeExpr{ + {Name: gno.N("p0"), Type: gno.X("string")}, + }, + []gno.FieldTypeExpr{ + {Name: gno.N("r0"), Type: gno.X("string")}, + }, + func(m *gno.Machine) { + b := m.LastBlock() + var ( + p0 string + rp0 = reflect.ValueOf(&p0).Elem() + ) + + gno.Gno2GoValue(b.GetPointerTo(nil, gno.NewValuePathBlock(1, 0, "")).TV, rp0) + + r0 := libs_strconv.QuoteToASCII(p0) + + m.PushValue(gno.Go2GnoValue( + m.Alloc, + m.Store, + reflect.ValueOf(&r0).Elem(), + )) + }, + }, + { + "time", + "now", + []gno.FieldTypeExpr{}, + []gno.FieldTypeExpr{ + {Name: gno.N("r0"), Type: gno.X("int64")}, + {Name: gno.N("r1"), Type: gno.X("int32")}, + {Name: gno.N("r2"), Type: gno.X("int64")}, + }, + func(m *gno.Machine) { + r0, r1, r2 := libs_time.X_now( + m, + ) + + m.PushValue(gno.Go2GnoValue( + m.Alloc, + m.Store, + reflect.ValueOf(&r0).Elem(), + )) + m.PushValue(gno.Go2GnoValue( + m.Alloc, + m.Store, + reflect.ValueOf(&r1).Elem(), + )) + m.PushValue(gno.Go2GnoValue( + m.Alloc, + m.Store, + reflect.ValueOf(&r2).Elem(), + )) + }, + }, +} diff --git a/gnovm/stdlibs/banker.go b/gnovm/stdlibs/std/banker.go similarity index 97% rename from gnovm/stdlibs/banker.go rename to gnovm/stdlibs/std/banker.go index 82bf1bad42a..7653f2a519f 100644 --- a/gnovm/stdlibs/banker.go +++ b/gnovm/stdlibs/std/banker.go @@ -1,4 +1,4 @@ -package stdlibs +package std import ( "fmt" @@ -21,10 +21,10 @@ type Banker interface { } // Used in std.GetBanker(options). -// Also available as Gno in stdlibs/std/banker.go +// Also available as Gno in stdlibs/std/banker.gno type BankerType uint8 -// Also available as Gno in stdlibs/std/banker.go +// Also available as Gno in stdlibs/std/banker.gno const ( // Can only read state. BankerTypeReadonly BankerType = iota diff --git a/gnovm/stdlibs/context.go b/gnovm/stdlibs/std/context.go similarity index 96% rename from gnovm/stdlibs/context.go rename to gnovm/stdlibs/std/context.go index 5f140c344d4..c50e2e5e1b9 100644 --- a/gnovm/stdlibs/context.go +++ b/gnovm/stdlibs/std/context.go @@ -1,4 +1,4 @@ -package stdlibs +package std import ( "github.com/gnolang/gno/tm2/pkg/crypto" diff --git a/gnovm/stdlibs/frame.go b/gnovm/stdlibs/std/frame.go similarity index 94% rename from gnovm/stdlibs/frame.go rename to gnovm/stdlibs/std/frame.go index e428bb1776d..2948813ad0f 100644 --- a/gnovm/stdlibs/frame.go +++ b/gnovm/stdlibs/std/frame.go @@ -1,4 +1,4 @@ -package stdlibs +package std import "github.com/gnolang/gno/tm2/pkg/crypto" diff --git a/gnovm/stdlibs/std/native.gno b/gnovm/stdlibs/std/native.gno new file mode 100644 index 00000000000..2f7da810bcb --- /dev/null +++ b/gnovm/stdlibs/std/native.gno @@ -0,0 +1,18 @@ +package std + +func AssertOriginCall() // injected +func IsOriginCall() bool // injected +func CurrentRealmPath() string // injected +func GetChainID() string // injected +func GetHeight() int64 // injected +func GetOrigSend() Coins // injected +func GetOrigCaller() Address // injected +func CurrentRealm() Realm // injected +func PrevRealm() Realm // injected +func GetOrigPkgAddr() Address // injected +func GetCallerAt(n int) Address // injected +func GetBanker(bt BankerType) Banker // injected +func DerivePkgAddr(pkgPath string) Address // injected + +func EncodeBech32(prefix string, bz [20]byte) Address // injected +func DecodeBech32(addr Address) (prefix string, bz [20]byte, ok bool) // injected diff --git a/gnovm/stdlibs/std/native.go b/gnovm/stdlibs/std/native.go new file mode 100644 index 00000000000..5f6cc8a0061 --- /dev/null +++ b/gnovm/stdlibs/std/native.go @@ -0,0 +1,187 @@ +package std + +import ( + "fmt" + "reflect" + + gno "github.com/gnolang/gno/gnovm/pkg/gnolang" + "github.com/gnolang/gno/tm2/pkg/bech32" + "github.com/gnolang/gno/tm2/pkg/crypto" + "github.com/gnolang/gno/tm2/pkg/std" +) + +func AssertOriginCall(m *gno.Machine) { + if !IsOriginCall(m) { + m.Panic(typedString("invalid non-origin call")) + } +} + +func IsOriginCall(m *gno.Machine) bool { + return len(m.Frames) == 2 +} + +func CurrentRealmPath(m *gno.Machine) string { + if m.Realm != nil { + return m.Realm.Path + } + return "" +} + +func GetChainID(m *gno.Machine) string { + return m.Context.(ExecContext).ChainID +} + +func GetHeight(m *gno.Machine) int64 { + return m.Context.(ExecContext).Height +} + +func GetOrigSend(m *gno.Machine) std.Coins { + return m.Context.(ExecContext).OrigSend +} + +func GetOrigCaller(m *gno.Machine) crypto.Bech32Address { + return m.Context.(ExecContext).OrigCaller +} + +func CurrentRealm(m *gno.Machine) Realm { + var ( + ctx = m.Context.(ExecContext) + // Default lastCaller is OrigCaller, the signer of the tx + lastCaller = ctx.OrigCaller + lastPkgPath = "" + ) + + for i := m.NumFrames() - 1; i > 0; i-- { + fr := m.Frames[i] + if fr.LastPackage != nil && fr.LastPackage.IsRealm() { + lastCaller = fr.LastPackage.GetPkgAddr().Bech32() + lastPkgPath = fr.LastPackage.PkgPath + break + } + } + + return Realm{ + addr: lastCaller, + pkgPath: lastPkgPath, + } +} + +func PrevRealm(m *gno.Machine) Realm { + var ( + ctx = m.Context.(ExecContext) + // Default lastCaller is OrigCaller, the signer of the tx + lastCaller = ctx.OrigCaller + lastPkgPath = "" + ) + + for i := m.NumFrames() - 1; i > 0; i-- { + fr := m.Frames[i] + if fr.LastPackage == nil || !fr.LastPackage.IsRealm() { + // Ignore non-realm frame + continue + } + pkgPath := fr.LastPackage.PkgPath + // The first realm we encounter will be the one calling + // this function; to get the calling realm determine the first frame + // where fr.LastPackage changes. + if lastPkgPath == "" { + lastPkgPath = pkgPath + } else if lastPkgPath == pkgPath { + continue + } else { + lastCaller = fr.LastPackage.GetPkgAddr().Bech32() + lastPkgPath = pkgPath + break + } + } + + // Empty the pkgPath if we return a user + if ctx.OrigCaller == lastCaller { + lastPkgPath = "" + } + + return Realm{ + addr: lastCaller, + pkgPath: lastPkgPath, + } +} + +func GetOrigPkgAddr(m *gno.Machine) crypto.Bech32Address { + return m.Context.(ExecContext).OrigPkgAddr +} + +func GetCallerAt(m *gno.Machine, n int) crypto.Bech32Address { + if n <= 0 { + fmt.Println("QWEQWEQWEQWE", n) + m.Panic(typedString("GetCallerAt requires positive arg")) + return "" + } + if n > m.NumFrames() { + // NOTE: the last frame's LastPackage + // is set to the original non-frame + // package, so need this check. + m.Panic(typedString("frame not found")) + return "" + } + if n == m.NumFrames() { + // This makes it consistent with GetOrigCaller. + ctx := m.Context.(ExecContext) + return ctx.OrigCaller + } + return m.LastCallFrame(n).LastPackage.GetPkgAddr().Bech32() +} + +func GetBanker(m *gno.Machine, bankerType BankerType) gno.TypedValue { + ctx := m.Context.(ExecContext) + banker := ctx.Banker + switch bankerType { + case BankerTypeReadonly: + banker = NewReadonlyBanker(banker) + case BankerTypeOrigSend: + banker = NewOrigSendBanker(banker, ctx.OrigPkgAddr, ctx.OrigSend, ctx.OrigSendSpent) + case BankerTypeRealmSend: + banker = NewRealmSendBanker(banker, ctx.OrigPkgAddr) + case BankerTypeRealmIssue: + banker = banker + default: + panic("should not happen") // defensive + } + m.Alloc.AllocateStruct() // defensive; native space not allocated. + m.Alloc.AllocateStructFields(10) // defensive 10; native space not allocated. + + // make gno bankAdapter{rv} + btv := gno.Go2GnoNativeValue(m.Alloc, reflect.ValueOf(banker)) + bsv := m.Alloc.NewStructWithFields(btv) + bankAdapterType := m.Store.GetType(gno.DeclaredTypeID("std", "bankAdapter")) + res0 := gno.TypedValue{T: bankAdapterType, V: bsv} + + return res0 +} + +func EncodeBech32(prefix string, bytes [20]byte) crypto.Bech32Address { + b32, err := bech32.ConvertAndEncode(prefix, bytes[:]) + if err != nil { + panic(err) // should not happen + } + return crypto.Bech32Address(b32) +} + +func DerivePkgAddr(pkgPath string) crypto.Bech32Address { + return gno.DerivePkgAddr(pkgPath).Bech32() +} + +func DecodeBech32(addr crypto.Bech32Address) (prefix string, bytes [20]byte, ok bool) { + prefix, bz, err := bech32.Decode(string(addr)) + if err != nil || len(bz) != 20 { + return "", [20]byte{}, false + } + // TODO: can be simplified when we switch to go1.20 in go mod to be a simple [20]byte(bz) + copy(bytes[:], bz) + return prefix, bytes, true +} + +func typedString(s gno.StringValue) gno.TypedValue { + tv := gno.TypedValue{T: gno.StringType} + tv.SetString(s) + return tv +} diff --git a/gnovm/stdlibs/stdlibs.go b/gnovm/stdlibs/stdlibs.go index 8931266eb9a..22054613c03 100644 --- a/gnovm/stdlibs/stdlibs.go +++ b/gnovm/stdlibs/stdlibs.go @@ -1,583 +1,30 @@ package stdlibs +//go:generate go run github.com/gnolang/gno/misc/genstd + import ( - "crypto/sha256" - "math" "reflect" - "strconv" - "time" gno "github.com/gnolang/gno/gnovm/pkg/gnolang" - "github.com/gnolang/gno/tm2/pkg/bech32" + libsstd "github.com/gnolang/gno/gnovm/stdlibs/std" "github.com/gnolang/gno/tm2/pkg/crypto" "github.com/gnolang/gno/tm2/pkg/std" ) +type ExecContext = libsstd.ExecContext + func InjectNativeMappings(store gno.Store) { store.AddGo2GnoMapping(reflect.TypeOf(crypto.Bech32Address("")), "std", "Address") store.AddGo2GnoMapping(reflect.TypeOf(std.Coins{}), "std", "Coins") store.AddGo2GnoMapping(reflect.TypeOf(std.Coin{}), "std", "Coin") - store.AddGo2GnoMapping(reflect.TypeOf(Realm{}), "std", "Realm") -} - -func InjectPackage(store gno.Store, pn *gno.PackageNode) { - switch pn.PkgPath { - case "internal/crypto/sha256": - pn.DefineNative("Sum256", - gno.Flds( // params - "data", "[]byte", - ), - gno.Flds( // results - "bz", "[32]byte", - ), - func(m *gno.Machine) { - arg0 := m.LastBlock().GetParams1().TV - bz := []byte(nil) - - if arg0.V != nil { - slice := arg0.V.(*gno.SliceValue) - array := slice.GetBase(m.Store) - bz = array.GetReadonlyBytes()[:slice.Length] - } - - hash := sha256.Sum256(bz) - res0 := gno.Go2GnoValue( - m.Alloc, - m.Store, - reflect.ValueOf(hash), - ) - m.PushValue(res0) - }, - ) - case "internal/math": - pn.DefineNative("Float32bits", - gno.Flds( // params - "f", "float32", - ), - gno.Flds( // results - "b", "uint32", - ), - func(m *gno.Machine) { - arg0 := m.LastBlock().GetParams1().TV - res0 := typedUint32(math.Float32bits(arg0.GetFloat32())) - m.PushValue(res0) - }, - ) - pn.DefineNative("Float32frombits", - gno.Flds( // params - "b", "uint32", - ), - gno.Flds( // results - "f", "float32", - ), - func(m *gno.Machine) { - arg0 := m.LastBlock().GetParams1().TV - res0 := typedFloat32(math.Float32frombits(arg0.GetUint32())) - m.PushValue(res0) - }, - ) - pn.DefineNative("Float64bits", - gno.Flds( // params - "f", "float64", - ), - gno.Flds( // results - "b", "uint64", - ), - func(m *gno.Machine) { - arg0 := m.LastBlock().GetParams1().TV - res0 := typedUint64(math.Float64bits(arg0.GetFloat64())) - m.PushValue(res0) - }, - ) - pn.DefineNative("Float64frombits", - gno.Flds( // params - "b", "uint64", - ), - gno.Flds( // results - "f", "float64", - ), - func(m *gno.Machine) { - arg0 := m.LastBlock().GetParams1().TV - res0 := typedFloat64(math.Float64frombits(arg0.GetUint64())) - m.PushValue(res0) - }, - ) - case "internal/os": - pn.DefineNative("Now", - gno.Flds( // params - ), - gno.Flds( // results - "sec", "int64", - "nsec", "int32", - "mono", "int64", - ), - func(m *gno.Machine) { - if m.Context == nil { - res0 := typedInt64(0) - res1 := typedInt32(0) - res2 := typedInt64(0) - m.PushValue(res0) - m.PushValue(res1) - m.PushValue(res2) - } else { - ctx := m.Context.(ExecContext) - res0 := typedInt64(ctx.Timestamp) - res1 := typedInt32(int32(ctx.TimestampNano)) - res2 := typedInt64(ctx.Timestamp*int64(time.Second) + ctx.TimestampNano) - m.PushValue(res0) - m.PushValue(res1) - m.PushValue(res2) - } - }, - ) - // case "internal/os_test": - // XXX defined in tests/imports.go - case "strconv": - pn.DefineGoNativeValue("Itoa", strconv.Itoa) - pn.DefineGoNativeValue("Atoi", strconv.Atoi) - pn.DefineGoNativeValue("FormatInt", strconv.FormatInt) - pn.DefineGoNativeValue("FormatUint", strconv.FormatUint) - pn.DefineGoNativeValue("Quote", strconv.Quote) - pn.DefineGoNativeValue("QuoteToASCII", strconv.QuoteToASCII) - pn.DefineGoNativeValue("CanBackquote", strconv.CanBackquote) - pn.DefineGoNativeValue("IntSize", strconv.IntSize) - pn.DefineGoNativeValue("AppendUint", strconv.AppendUint) - case "std": - // NOTE: some of these are overridden in tests/imports.go - // Also see stdlibs/InjectPackage. - pn.DefineNative("AssertOriginCall", - gno.Flds( // params - ), - gno.Flds( // results - ), - func(m *gno.Machine) { - isOrigin := len(m.Frames) == 2 - if !isOrigin { - m.Panic(typedString("invalid non-origin call")) - return - } - }, - ) - pn.DefineNative("IsOriginCall", - gno.Flds( // params - ), - gno.Flds( // results - "isOrigin", "bool", - ), - func(m *gno.Machine) { - isOrigin := len(m.Frames) == 2 - res0 := gno.TypedValue{T: gno.BoolType} - res0.SetBool(isOrigin) - m.PushValue(res0) - }, - ) - pn.DefineNative("CurrentRealmPath", - gno.Flds( // params - ), - gno.Flds( // results - "", "string", - ), - func(m *gno.Machine) { - realmPath := "" - if m.Realm != nil { - realmPath = m.Realm.Path - } - res0 := gno.Go2GnoValue( - m.Alloc, - m.Store, - reflect.ValueOf(realmPath), - ) - m.PushValue(res0) - }, - ) - pn.DefineNative("GetChainID", - gno.Flds( // params - ), - gno.Flds( // results - "", "string", - ), - func(m *gno.Machine) { - ctx := m.Context.(ExecContext) - res0 := gno.Go2GnoValue( - m.Alloc, - m.Store, - reflect.ValueOf(ctx.ChainID), - ) - m.PushValue(res0) - }, - ) - pn.DefineNative("GetHeight", - gno.Flds( // params - ), - gno.Flds( // results - "", "int64", - ), - func(m *gno.Machine) { - ctx := m.Context.(ExecContext) - res0 := gno.Go2GnoValue( - m.Alloc, - m.Store, - reflect.ValueOf(ctx.Height), - ) - m.PushValue(res0) - }, - ) - pn.DefineNative("GetOrigSend", - gno.Flds( // params - ), - gno.Flds( // results - "", "Coins", - ), - func(m *gno.Machine) { - ctx := m.Context.(ExecContext) - res0 := gno.Go2GnoValue( - m.Alloc, - m.Store, - reflect.ValueOf(ctx.OrigSend), - ) - coinT := store.GetType(gno.DeclaredTypeID("std", "Coin")) - coinsT := store.GetType(gno.DeclaredTypeID("std", "Coins")) - res0.T = coinsT - av := res0.V.(*gno.SliceValue).Base.(*gno.ArrayValue) - for i := range av.List { - av.List[i].T = coinT - } - m.PushValue(res0) - }, - ) - pn.DefineNative("GetOrigCaller", - gno.Flds( // params - ), - gno.Flds( // results - "", "Address", - ), - func(m *gno.Machine) { - ctx := m.Context.(ExecContext) - res0 := gno.Go2GnoValue( - m.Alloc, - m.Store, - reflect.ValueOf(ctx.OrigCaller), - ) - addrT := store.GetType(gno.DeclaredTypeID("std", "Address")) - res0.T = addrT - m.PushValue(res0) - }, - ) - pn.DefineNative("CurrentRealm", - gno.Flds( // params - ), - gno.Flds( // results - "", "Realm", - ), - func(m *gno.Machine) { - var ( - ctx = m.Context.(ExecContext) - // Default lastCaller is OrigCaller, the signer of the tx - lastCaller = ctx.OrigCaller - lastPkgPath = "" - ) - - for i := m.NumFrames() - 1; i > 0; i-- { - fr := m.Frames[i] - if fr.LastPackage != nil && fr.LastPackage.IsRealm() { - lastCaller = fr.LastPackage.GetPkgAddr().Bech32() - lastPkgPath = fr.LastPackage.PkgPath - break - } - } - - // Return the result - res0 := gno.Go2GnoValue( - m.Alloc, - m.Store, - reflect.ValueOf(Realm{ - addr: lastCaller, - pkgPath: lastPkgPath, - }), - ) - - realmT := store.GetType(gno.DeclaredTypeID("std", "Realm")) - res0.T = realmT - m.PushValue(res0) - }, - ) - pn.DefineNative("PrevRealm", - gno.Flds( // params - ), - gno.Flds( // results - "", "Realm", - ), - func(m *gno.Machine) { - var ( - ctx = m.Context.(ExecContext) - // Default lastCaller is OrigCaller, the signer of the tx - lastCaller = ctx.OrigCaller - lastPkgPath = "" - ) - - for i := m.NumFrames() - 1; i > 0; i-- { - fr := m.Frames[i] - if fr.LastPackage == nil || !fr.LastPackage.IsRealm() { - // Ignore non-realm frame - continue - } - pkgPath := fr.LastPackage.PkgPath - // The first realm we encounter will be the one calling - // this function; to get the calling realm determine the first frame - // where fr.LastPackage changes. - if lastPkgPath == "" { - lastPkgPath = pkgPath - } else if lastPkgPath == pkgPath { - continue - } else { - lastCaller = fr.LastPackage.GetPkgAddr().Bech32() - lastPkgPath = pkgPath - break - } - } - - // Empty the pkgPath if we return a user - if ctx.OrigCaller == lastCaller { - lastPkgPath = "" - } - - // Return the result - res0 := gno.Go2GnoValue( - m.Alloc, - m.Store, - reflect.ValueOf(Realm{ - addr: lastCaller, - pkgPath: lastPkgPath, - }), - ) - - realmT := store.GetType(gno.DeclaredTypeID("std", "Realm")) - res0.T = realmT - m.PushValue(res0) - }, - ) - pn.DefineNative("GetOrigPkgAddr", - gno.Flds( // params - ), - gno.Flds( // results - "", "Address", - ), - func(m *gno.Machine) { - ctx := m.Context.(ExecContext) - res0 := gno.Go2GnoValue( - m.Alloc, - m.Store, - reflect.ValueOf(ctx.OrigPkgAddr), - ) - addrT := store.GetType(gno.DeclaredTypeID("std", "Address")) - res0.T = addrT - m.PushValue(res0) - }, - ) - pn.DefineNative("GetCallerAt", - gno.Flds( // params - "n", "int", - ), - gno.Flds( // results - "", "Address", - ), - func(m *gno.Machine) { - arg0 := m.LastBlock().GetParams1().TV - n := arg0.GetInt() - if n <= 0 { - m.Panic(typedString("GetCallerAt requires positive arg")) - return - } - if n > m.NumFrames() { - // NOTE: the last frame's LastPackage - // is set to the original non-frame - // package, so need this check. - m.Panic(typedString("frame not found")) - return - } - var pkgAddr string - if n == m.NumFrames() { - // This makes it consistent with GetOrigCaller. - ctx := m.Context.(ExecContext) - pkgAddr = string(ctx.OrigCaller) - } else { - pkgAddr = string(m.LastCallFrame(n).LastPackage.GetPkgAddr().Bech32()) - } - res0 := gno.Go2GnoValue( - m.Alloc, - m.Store, - reflect.ValueOf(pkgAddr), - ) - addrT := store.GetType(gno.DeclaredTypeID("std", "Address")) - res0.T = addrT - m.PushValue(res0) - }, - ) - pn.DefineNative("GetBanker", - gno.Flds( // params - "bankerType", "BankerType", - ), - gno.Flds( // results - "", "Banker", - ), - func(m *gno.Machine) { - ctx := m.Context.(ExecContext) - arg0 := m.LastBlock().GetParams1().TV - bankerType := BankerType(arg0.GetUint8()) - banker := ctx.Banker - switch bankerType { - case BankerTypeReadonly: - banker = NewReadonlyBanker(banker) - case BankerTypeOrigSend: - banker = NewOrigSendBanker(banker, ctx.OrigPkgAddr, ctx.OrigSend, ctx.OrigSendSpent) - case BankerTypeRealmSend: - banker = NewRealmSendBanker(banker, ctx.OrigPkgAddr) - case BankerTypeRealmIssue: - banker = banker - default: - panic("should not happen") // defensive - } - rv := reflect.ValueOf(banker) - m.Alloc.AllocateStruct() // defensive; native space not allocated. - m.Alloc.AllocateStructFields(10) // defensive 10; native space not allocated. - - // make gno bankAdapter{rv} - btv := gno.Go2GnoNativeValue(m.Alloc, rv) - bsv := m.Alloc.NewStructWithFields(btv) - bankAdapterType := store.GetType(gno.DeclaredTypeID("std", "bankAdapter")) - res0 := gno.TypedValue{T: bankAdapterType, V: bsv} - m.PushValue(res0) - }, - ) - pn.DefineNative("EncodeBech32", - gno.Flds( // params - "prefix", "string", - "bytes", "[20]byte", - ), - gno.Flds( // results - "addr", "Address", - ), - func(m *gno.Machine) { - arg0, arg1 := m.LastBlock().GetParams2() - prefix := arg0.TV.GetString() - bz := arg1.TV.V.(*gno.ArrayValue).GetReadonlyBytes() - if len(bz) != crypto.AddressSize { - panic("should not happen") - } - b32, err := bech32.ConvertAndEncode(prefix, bz) - if err != nil { - panic(err) // should not happen - } - res0 := gno.Go2GnoValue( - m.Alloc, - m.Store, - reflect.ValueOf(b32), - ) - addrT := store.GetType(gno.DeclaredTypeID("std", "Address")) - res0.T = addrT - m.PushValue(res0) - }, - ) - pn.DefineNative("DecodeBech32", - gno.Flds( // params - "addr", "Address", - ), - gno.Flds( // results - "prefix", "string", - "bytes", "[20]byte", - "ok", "bool", - ), - func(m *gno.Machine) { - arg0 := m.LastBlock().GetParams1() - addr := arg0.TV.GetString() - prefix, bz, err := bech32.Decode(addr) - if err != nil || len(bz) != 20 { - m.PushValue(typedString(m.Alloc.NewString(""))) - m.PushValue(typedByteArray(20, m.Alloc.NewDataArray(20))) - m.PushValue(typedBool(false)) - } else { - m.PushValue(typedString(m.Alloc.NewString(prefix))) - m.PushValue(typedByteArray(20, m.Alloc.NewArrayFromData(bz))) - m.PushValue(typedBool(true)) - } - }, - ) - pn.DefineNative("DerivePkgAddr", - gno.Flds( // params - "pkgPath", "string", - ), - gno.Flds( // results - "addr", "Address", - ), - func(m *gno.Machine) { - arg0 := m.LastBlock().GetParams1().TV - pkgPath := arg0.GetString() - pkgAddr := gno.DerivePkgAddr(pkgPath).Bech32() - res0 := gno.Go2GnoValue( - m.Alloc, - m.Store, - reflect.ValueOf(pkgAddr), - ) - addrT := store.GetType(gno.DeclaredTypeID("std", "Address")) - res0.T = addrT - m.PushValue(res0) - }, - ) - } -} - -func typedInt32(i32 int32) gno.TypedValue { - tv := gno.TypedValue{T: gno.Int32Type} - tv.SetInt32(i32) - return tv -} - -func typedInt64(i64 int64) gno.TypedValue { - tv := gno.TypedValue{T: gno.Int64Type} - tv.SetInt64(i64) - return tv -} - -func typedUint32(u32 uint32) gno.TypedValue { - tv := gno.TypedValue{T: gno.Uint32Type} - tv.SetUint32(u32) - return tv -} - -func typedUint64(u64 uint64) gno.TypedValue { - tv := gno.TypedValue{T: gno.Uint64Type} - tv.SetUint64(u64) - return tv -} - -func typedFloat32(f32 float32) gno.TypedValue { - tv := gno.TypedValue{T: gno.Float32Type} - tv.SetFloat32(f32) - return tv -} - -func typedFloat64(f64 float64) gno.TypedValue { - tv := gno.TypedValue{T: gno.Float64Type} - tv.SetFloat64(f64) - return tv -} - -func typedString(s gno.StringValue) gno.TypedValue { - tv := gno.TypedValue{T: gno.StringType} - tv.SetString(s) - return tv -} - -func typedBool(b bool) gno.TypedValue { - tv := gno.TypedValue{T: gno.BoolType} - tv.SetBool(b) - return tv + store.AddGo2GnoMapping(reflect.TypeOf(libsstd.Realm{}), "std", "Realm") } -func typedByteArray(ln int, bz *gno.ArrayValue) gno.TypedValue { - if bz != nil && bz.GetLength() != ln { - panic("array length mismatch") +func NativeStore(pkgPath string, name gno.Name) func(*gno.Machine) { + for _, nf := range nativeFuncs { + if nf.gnoPkg == pkgPath && name == nf.gnoFunc { + return nf.f + } } - tv := gno.TypedValue{T: &gno.ArrayType{Len: ln, Elt: gno.Uint8Type}, V: bz} - return tv + return nil } diff --git a/gnovm/stdlibs/strconv/strconv.gno b/gnovm/stdlibs/strconv/strconv.gno index dce62b890e3..bc7b5d8d1b6 100644 --- a/gnovm/stdlibs/strconv/strconv.gno +++ b/gnovm/stdlibs/strconv/strconv.gno @@ -1,4 +1,10 @@ package strconv -// NOTE: currently these are implemented as native functions. -// See InjectNatives(). +func Itoa(n int) string // injected +func AppendUint(dst []byte, i uint64, base int) []byte // injected +func Atoi(s string) (int, error) // injected +func CanBackquote(s string) bool // injected +func FormatInt(i int64, base int) string // injected +func FormatUint(i uint64, base int) string // injected +func Quote(s string) string // injected +func QuoteToASCII(s string) string // injected diff --git a/gnovm/stdlibs/strconv/strconv.go b/gnovm/stdlibs/strconv/strconv.go new file mode 100644 index 00000000000..782a63e84b6 --- /dev/null +++ b/gnovm/stdlibs/strconv/strconv.go @@ -0,0 +1,12 @@ +package strconv + +import "strconv" + +func Itoa(n int) string { return strconv.Itoa(n) } +func AppendUint(dst []byte, i uint64, base int) []byte { return strconv.AppendUint(dst, i, base) } +func Atoi(s string) (int, error) { return strconv.Atoi(s) } +func CanBackquote(s string) bool { return strconv.CanBackquote(s) } +func FormatInt(i int64, base int) string { return strconv.FormatInt(i, base) } +func FormatUint(i uint64, base int) string { return strconv.FormatUint(i, base) } +func Quote(s string) string { return strconv.Quote(s) } +func QuoteToASCII(r string) string { return strconv.QuoteToASCII(r) } diff --git a/gnovm/stdlibs/time/time.gno b/gnovm/stdlibs/time/time.gno index 7b7e45ba9b6..ceed70452f6 100644 --- a/gnovm/stdlibs/time/time.gno +++ b/gnovm/stdlibs/time/time.gno @@ -80,8 +80,6 @@ package time import ( "errors" - - ios "internal/os" // XXX to access time. // XXX _ "unsafe" // for go:linkname ) @@ -1071,17 +1069,13 @@ func daysSinceEpoch(year int) uint64 { return d } -/* XXX replaced with ios.Now() -// Provided by package runtime. -func now() (sec int64, nsec int32, mono int64) -*/ +func now() (sec int64, nsec int32, mono int64) // injected -// XXX SHIM // runtimeNano returns the current value of the runtime clock in nanoseconds. // //go:linkname runtimeNano runtime.nanotime func runtimeNano() int64 { - _, _, mono := ios.Now() // XXX now() + _, _, mono := now() return mono } @@ -1095,7 +1089,7 @@ var startNano int64 = runtimeNano() - 1 // Now returns the current local time. func Now() Time { - sec, nsec, mono := ios.Now() // XXX now() + sec, nsec, mono := now() mono -= startNano sec += unixToInternal - minWall if uint64(sec)>>33 != 0 { diff --git a/gnovm/stdlibs/time/time.go b/gnovm/stdlibs/time/time.go new file mode 100644 index 00000000000..8c1c768715e --- /dev/null +++ b/gnovm/stdlibs/time/time.go @@ -0,0 +1,17 @@ +package time + +import ( + "time" + + gno "github.com/gnolang/gno/gnovm/pkg/gnolang" + "github.com/gnolang/gno/gnovm/stdlibs/std" +) + +func X_now(m *gno.Machine) (sec int64, nsec int32, mono int64) { + if m == nil || m.Context == nil { + return 0, 0, 0 + } + + ctx := m.Context.(std.ExecContext) + return ctx.Timestamp, int32(ctx.TimestampNano), ctx.Timestamp*int64(time.Second) + ctx.TimestampNano +} diff --git a/gnovm/tests/file.go b/gnovm/tests/file.go index 451bf0677dc..a021628a385 100644 --- a/gnovm/tests/file.go +++ b/gnovm/tests/file.go @@ -183,6 +183,7 @@ func RunFileTest(rootDir string, path string, opts ...RunFileTestOption) error { }, }, } + // run decls and init functions. m.RunMemPackage(memPkg, true) // reconstruct machine and clear store cache. // whether package is realm or not, since non-realm diff --git a/gnovm/tests/files/extern/redeclaration1/README.md b/gnovm/tests/files/extern/redeclaration1/README.md new file mode 100644 index 00000000000..6c8514c3ca1 --- /dev/null +++ b/gnovm/tests/files/extern/redeclaration1/README.md @@ -0,0 +1,4 @@ +This package is invalid because 'a' is defined twice. +NOTE: the Go parser itself returns an error for redefinitions in the same file, +but testing for redeclarations across files requires our own custom logic. +(arguably we should check ourself either way). diff --git a/gnovm/tests/files/extern/redeclaration1/redeclaration.gno b/gnovm/tests/files/extern/redeclaration1/redeclaration.gno new file mode 100644 index 00000000000..b9b17a50e80 --- /dev/null +++ b/gnovm/tests/files/extern/redeclaration1/redeclaration.gno @@ -0,0 +1,3 @@ +package redeclaration + +var a = 1 diff --git a/gnovm/tests/files/extern/redeclaration1/redeclaration2.gno b/gnovm/tests/files/extern/redeclaration1/redeclaration2.gno new file mode 100644 index 00000000000..11f2f095cb4 --- /dev/null +++ b/gnovm/tests/files/extern/redeclaration1/redeclaration2.gno @@ -0,0 +1,4 @@ +package redeclaration + +// redeclared +var a = 2 diff --git a/gnovm/tests/files/extern/redeclaration2/README.md b/gnovm/tests/files/extern/redeclaration2/README.md new file mode 100644 index 00000000000..6c8514c3ca1 --- /dev/null +++ b/gnovm/tests/files/extern/redeclaration2/README.md @@ -0,0 +1,4 @@ +This package is invalid because 'a' is defined twice. +NOTE: the Go parser itself returns an error for redefinitions in the same file, +but testing for redeclarations across files requires our own custom logic. +(arguably we should check ourself either way). diff --git a/gnovm/tests/files/extern/redeclaration2/redeclaration.gno b/gnovm/tests/files/extern/redeclaration2/redeclaration.gno new file mode 100644 index 00000000000..ef6cac198db --- /dev/null +++ b/gnovm/tests/files/extern/redeclaration2/redeclaration.gno @@ -0,0 +1,3 @@ +package redeclaration + +func a() int { return 1 } diff --git a/gnovm/tests/files/extern/redeclaration2/redeclaration2.gno b/gnovm/tests/files/extern/redeclaration2/redeclaration2.gno new file mode 100644 index 00000000000..3f58a963502 --- /dev/null +++ b/gnovm/tests/files/extern/redeclaration2/redeclaration2.gno @@ -0,0 +1,4 @@ +package redeclaration + +// redeclared (with same signature) +func a() int { return 2 } diff --git a/gnovm/tests/files/extern/redeclaration3/README.md b/gnovm/tests/files/extern/redeclaration3/README.md new file mode 100644 index 00000000000..6c8514c3ca1 --- /dev/null +++ b/gnovm/tests/files/extern/redeclaration3/README.md @@ -0,0 +1,4 @@ +This package is invalid because 'a' is defined twice. +NOTE: the Go parser itself returns an error for redefinitions in the same file, +but testing for redeclarations across files requires our own custom logic. +(arguably we should check ourself either way). diff --git a/gnovm/tests/files/extern/redeclaration3/redeclaration.gno b/gnovm/tests/files/extern/redeclaration3/redeclaration.gno new file mode 100644 index 00000000000..380a1a4288f --- /dev/null +++ b/gnovm/tests/files/extern/redeclaration3/redeclaration.gno @@ -0,0 +1,5 @@ +package redeclaration + +type a struct{} + +func (_ a) method() int { return 1 } diff --git a/gnovm/tests/files/extern/redeclaration3/redeclaration2.gno b/gnovm/tests/files/extern/redeclaration3/redeclaration2.gno new file mode 100644 index 00000000000..c2256bdf7cb --- /dev/null +++ b/gnovm/tests/files/extern/redeclaration3/redeclaration2.gno @@ -0,0 +1,4 @@ +package redeclaration + +// redeclared (with same signature) +func (_ a) method() int { return 2 } diff --git a/gnovm/tests/files/float5_native.gno b/gnovm/tests/files/float5_native.gno index 1050d277605..b2d9228c0bc 100644 --- a/gnovm/tests/files/float5_native.gno +++ b/gnovm/tests/files/float5_native.gno @@ -1,13 +1,13 @@ package main import ( - imath "internal/math" + "math" ) func main() { // test float64 f := float64(0.3) - x := imath.Float64bits(f) + x := math.Float64bits(f) e := uint(40) println(f, x, e, (1 << (64 - e))) diff --git a/gnovm/tests/files/float5_stdlibs.gno b/gnovm/tests/files/float5_stdlibs.gno index 8449ddff820..b3d8cd84713 100644 --- a/gnovm/tests/files/float5_stdlibs.gno +++ b/gnovm/tests/files/float5_stdlibs.gno @@ -2,13 +2,11 @@ package main import ( "math" - - imath "internal/math" ) func main() { - println(math.MaxFloat32, imath.Float32bits(math.MaxFloat32)) - println(math.MaxFloat64, imath.Float64bits(math.MaxFloat64)) + println(math.MaxFloat32, math.Float32bits(math.MaxFloat32)) + println(math.MaxFloat64, math.Float64bits(math.MaxFloat64)) } // Output: diff --git a/gnovm/tests/files/redeclaration10.gno b/gnovm/tests/files/redeclaration10.gno new file mode 100644 index 00000000000..01584b1755c --- /dev/null +++ b/gnovm/tests/files/redeclaration10.gno @@ -0,0 +1,12 @@ +package main + +import _ "github.com/gnolang/gno/_test/redeclaration3" + +func main() { + println("should not happen") +} + +// XXX show what was redeclared. + +// Error: +// running package "github.com/gnolang/gno/_test/redeclaration3": duplicate declarations not allowed diff --git a/gnovm/tests/files/redeclaration6.gno b/gnovm/tests/files/redeclaration6.gno new file mode 100644 index 00000000000..25e36fa61aa --- /dev/null +++ b/gnovm/tests/files/redeclaration6.gno @@ -0,0 +1,12 @@ +package main + +import _ "github.com/gnolang/gno/_test/redeclaration1" + +func main() { + println("should not happen") +} + +// XXX show what was redeclared. + +// Error: +// running package "github.com/gnolang/gno/_test/redeclaration1": duplicate declarations not allowed diff --git a/gnovm/tests/files/redeclaration7.gno b/gnovm/tests/files/redeclaration7.gno new file mode 100644 index 00000000000..d987a58d7eb --- /dev/null +++ b/gnovm/tests/files/redeclaration7.gno @@ -0,0 +1,13 @@ +package main + +func f1() int { return 1 } + +func f1() int { return 2 } + +func main() { + println("hello", f1()) +} + +// Error: +// files/redeclaration7.gno:5:6: f1 redeclared in this block +// previous declaration at files/redeclaration7.gno:3:6 diff --git a/gnovm/tests/files/redeclaration8.gno b/gnovm/tests/files/redeclaration8.gno new file mode 100644 index 00000000000..d0e5b958030 --- /dev/null +++ b/gnovm/tests/files/redeclaration8.gno @@ -0,0 +1,12 @@ +package main + +import _ "github.com/gnolang/gno/_test/redeclaration2" + +func main() { + println("should not happen") +} + +// XXX show what was redeclared. + +// Error: +// running package "github.com/gnolang/gno/_test/redeclaration2": duplicate declarations not allowed diff --git a/gnovm/tests/files/redeclaration9.gno b/gnovm/tests/files/redeclaration9.gno new file mode 100644 index 00000000000..89a63683c13 --- /dev/null +++ b/gnovm/tests/files/redeclaration9.gno @@ -0,0 +1,14 @@ +package main + +type a struct{} + +func (_ a) method() int { return 1 } + +func (_ a) method() int { return 2 } + +func main() { + println("hello") +} + +// Error: +// main/files/redeclaration9.gno:7: redeclaration of method a.method diff --git a/gnovm/tests/files/zrealm0.gno b/gnovm/tests/files/zrealm0.gno index 1b8f37540b3..7578781e503 100644 --- a/gnovm/tests/files/zrealm0.gno +++ b/gnovm/tests/files/zrealm0.gno @@ -56,6 +56,8 @@ func main() { // "FileName": "main.gno", // "IsMethod": false, // "Name": "main", +// "NativeName": "", +// "NativePkg": "", // "PkgPath": "gno.land/r/test", // "Source": { // "@type": "/gno.RefNode", diff --git a/gnovm/tests/files/zrealm1.gno b/gnovm/tests/files/zrealm1.gno index 1dea983a49d..d90c5e8621a 100644 --- a/gnovm/tests/files/zrealm1.gno +++ b/gnovm/tests/files/zrealm1.gno @@ -170,6 +170,8 @@ func main() { // "FileName": "main.gno", // "IsMethod": false, // "Name": "main", +// "NativeName": "", +// "NativePkg": "", // "PkgPath": "gno.land/r/test", // "Source": { // "@type": "/gno.RefNode", diff --git a/gnovm/tests/files/zrealm2.gno b/gnovm/tests/files/zrealm2.gno index bf321c42d31..67ba2f5a768 100644 --- a/gnovm/tests/files/zrealm2.gno +++ b/gnovm/tests/files/zrealm2.gno @@ -173,6 +173,8 @@ func main() { // "FileName": "main.gno", // "IsMethod": false, // "Name": "init.3", +// "NativeName": "", +// "NativePkg": "", // "PkgPath": "gno.land/r/test", // "Source": { // "@type": "/gno.RefNode", @@ -207,6 +209,8 @@ func main() { // "FileName": "main.gno", // "IsMethod": false, // "Name": "main", +// "NativeName": "", +// "NativePkg": "", // "PkgPath": "gno.land/r/test", // "Source": { // "@type": "/gno.RefNode", diff --git a/gnovm/tests/files/zrealm3.gno b/gnovm/tests/files/zrealm3.gno index 4ff8dd1a531..da8a581375c 100644 --- a/gnovm/tests/files/zrealm3.gno +++ b/gnovm/tests/files/zrealm3.gno @@ -172,6 +172,8 @@ func main() { // "FileName": "main.gno", // "IsMethod": false, // "Name": "init.2", +// "NativeName": "", +// "NativePkg": "", // "PkgPath": "gno.land/r/test", // "Source": { // "@type": "/gno.RefNode", @@ -206,6 +208,8 @@ func main() { // "FileName": "main.gno", // "IsMethod": false, // "Name": "main", +// "NativeName": "", +// "NativePkg": "", // "PkgPath": "gno.land/r/test", // "Source": { // "@type": "/gno.RefNode", diff --git a/gnovm/tests/files/zrealm4.gno b/gnovm/tests/files/zrealm4.gno index 2e2fa4e8d09..dc3c48c774b 100644 --- a/gnovm/tests/files/zrealm4.gno +++ b/gnovm/tests/files/zrealm4.gno @@ -114,6 +114,8 @@ func main() { // "FileName": "main.gno", // "IsMethod": false, // "Name": "init.0", +// "NativeName": "", +// "NativePkg": "", // "PkgPath": "gno.land/r/test", // "Source": { // "@type": "/gno.RefNode", @@ -148,6 +150,8 @@ func main() { // "FileName": "main.gno", // "IsMethod": false, // "Name": "main", +// "NativeName": "", +// "NativePkg": "", // "PkgPath": "gno.land/r/test", // "Source": { // "@type": "/gno.RefNode", diff --git a/gnovm/tests/files/zrealm5.gno b/gnovm/tests/files/zrealm5.gno index 8ad3e7400b3..e65b089c18d 100644 --- a/gnovm/tests/files/zrealm5.gno +++ b/gnovm/tests/files/zrealm5.gno @@ -185,6 +185,8 @@ func main() { // "FileName": "main.gno", // "IsMethod": false, // "Name": "init.0", +// "NativeName": "", +// "NativePkg": "", // "PkgPath": "gno.land/r/test", // "Source": { // "@type": "/gno.RefNode", @@ -219,6 +221,8 @@ func main() { // "FileName": "main.gno", // "IsMethod": false, // "Name": "main", +// "NativeName": "", +// "NativePkg": "", // "PkgPath": "gno.land/r/test", // "Source": { // "@type": "/gno.RefNode", diff --git a/gnovm/tests/files/zrealm6.gno b/gnovm/tests/files/zrealm6.gno index fbe320ad962..20615fa7d39 100644 --- a/gnovm/tests/files/zrealm6.gno +++ b/gnovm/tests/files/zrealm6.gno @@ -257,6 +257,8 @@ func main() { // "FileName": "main.gno", // "IsMethod": false, // "Name": "init.0", +// "NativeName": "", +// "NativePkg": "", // "PkgPath": "gno.land/r/test", // "Source": { // "@type": "/gno.RefNode", @@ -291,6 +293,8 @@ func main() { // "FileName": "main.gno", // "IsMethod": false, // "Name": "main", +// "NativeName": "", +// "NativePkg": "", // "PkgPath": "gno.land/r/test", // "Source": { // "@type": "/gno.RefNode", diff --git a/gnovm/tests/files/zrealm7.gno b/gnovm/tests/files/zrealm7.gno index 689d55d3916..9decb0dae10 100644 --- a/gnovm/tests/files/zrealm7.gno +++ b/gnovm/tests/files/zrealm7.gno @@ -329,6 +329,8 @@ func main() { // "FileName": "main.gno", // "IsMethod": false, // "Name": "init.0", +// "NativeName": "", +// "NativePkg": "", // "PkgPath": "gno.land/r/test", // "Source": { // "@type": "/gno.RefNode", @@ -363,6 +365,8 @@ func main() { // "FileName": "main.gno", // "IsMethod": false, // "Name": "main", +// "NativeName": "", +// "NativePkg": "", // "PkgPath": "gno.land/r/test", // "Source": { // "@type": "/gno.RefNode", diff --git a/gnovm/tests/files/zrealm_avl0.gno b/gnovm/tests/files/zrealm_avl0.gno index 814e19d6d49..e91788ac8eb 100644 --- a/gnovm/tests/files/zrealm_avl0.gno +++ b/gnovm/tests/files/zrealm_avl0.gno @@ -267,6 +267,8 @@ func main() { // "FileName": "main.gno", // "IsMethod": false, // "Name": "init.0", +// "NativeName": "", +// "NativePkg": "", // "PkgPath": "gno.land/r/test", // "Source": { // "@type": "/gno.RefNode", @@ -301,6 +303,8 @@ func main() { // "FileName": "main.gno", // "IsMethod": false, // "Name": "main", +// "NativeName": "", +// "NativePkg": "", // "PkgPath": "gno.land/r/test", // "Source": { // "@type": "/gno.RefNode", diff --git a/gnovm/tests/files/zrealm_avl1.gno b/gnovm/tests/files/zrealm_avl1.gno index 410e9e93601..cdd56a5ad89 100644 --- a/gnovm/tests/files/zrealm_avl1.gno +++ b/gnovm/tests/files/zrealm_avl1.gno @@ -291,6 +291,8 @@ func main() { // "FileName": "main.gno", // "IsMethod": false, // "Name": "init.0", +// "NativeName": "", +// "NativePkg": "", // "PkgPath": "gno.land/r/test", // "Source": { // "@type": "/gno.RefNode", @@ -325,6 +327,8 @@ func main() { // "FileName": "main.gno", // "IsMethod": false, // "Name": "main", +// "NativeName": "", +// "NativePkg": "", // "PkgPath": "gno.land/r/test", // "Source": { // "@type": "/gno.RefNode", diff --git a/gnovm/tests/files/zrealm_natbind0.gno b/gnovm/tests/files/zrealm_natbind0.gno new file mode 100644 index 00000000000..60e0d448202 --- /dev/null +++ b/gnovm/tests/files/zrealm_natbind0.gno @@ -0,0 +1,197 @@ +// PKGPATH: gno.land/r/test +package test + +import ( + "std" +) + +var node interface{} + +func init() { + node = std.GetOrigCaller +} + +func main() { + f := node.(func() std.Address) + println(f()) + node = std.DerivePkgAddr + g := node.(func(path string) std.Address) + println(g("x")) +} + +// Output: +// g1wymu47drhr0kuq2098m792lytgtj2nyx77yrsm +// g19kt9e22k34ny5jf5plrjdltmws0jc0qqd2cwky + +// Realm: +// switchrealm["gno.land/r/test"] +// u[a8ada09dee16d791fd406d629fe29bb0ed084a30:2]={ +// "Blank": {}, +// "ObjectInfo": { +// "ID": "a8ada09dee16d791fd406d629fe29bb0ed084a30:2", +// "IsEscaped": true, +// "ModTime": "3", +// "RefCount": "2" +// }, +// "Parent": null, +// "Source": { +// "@type": "/gno.RefNode", +// "BlockNode": null, +// "Location": { +// "File": "", +// "Line": "0", +// "Nonce": "0", +// "PkgPath": "gno.land/r/test" +// } +// }, +// "Values": [ +// { +// "T": { +// "@type": "/gno.FuncType", +// "Params": [], +// "Results": [] +// }, +// "V": { +// "@type": "/gno.FuncValue", +// "Closure": { +// "@type": "/gno.RefValue", +// "Escaped": true, +// "ObjectID": "a8ada09dee16d791fd406d629fe29bb0ed084a30:3" +// }, +// "FileName": "main.gno", +// "IsMethod": false, +// "Name": "init.0", +// "NativeName": "", +// "NativePkg": "", +// "PkgPath": "gno.land/r/test", +// "Source": { +// "@type": "/gno.RefNode", +// "BlockNode": null, +// "Location": { +// "File": "main.gno", +// "Line": "10", +// "Nonce": "0", +// "PkgPath": "gno.land/r/test" +// } +// }, +// "Type": { +// "@type": "/gno.FuncType", +// "Params": [], +// "Results": [] +// } +// } +// }, +// { +// "T": { +// "@type": "/gno.FuncType", +// "Params": [], +// "Results": [] +// }, +// "V": { +// "@type": "/gno.FuncValue", +// "Closure": { +// "@type": "/gno.RefValue", +// "Escaped": true, +// "ObjectID": "a8ada09dee16d791fd406d629fe29bb0ed084a30:3" +// }, +// "FileName": "main.gno", +// "IsMethod": false, +// "Name": "main", +// "NativeName": "", +// "NativePkg": "", +// "PkgPath": "gno.land/r/test", +// "Source": { +// "@type": "/gno.RefNode", +// "BlockNode": null, +// "Location": { +// "File": "main.gno", +// "Line": "14", +// "Nonce": "0", +// "PkgPath": "gno.land/r/test" +// } +// }, +// "Type": { +// "@type": "/gno.FuncType", +// "Params": [], +// "Results": [] +// } +// } +// }, +// { +// "T": { +// "@type": "/gno.FuncType", +// "Params": [ +// { +// "Embedded": false, +// "Name": "pkgPath", +// "Tag": "", +// "Type": { +// "@type": "/gno.PrimitiveType", +// "value": "16" +// } +// } +// ], +// "Results": [ +// { +// "Embedded": false, +// "Name": "", +// "Tag": "", +// "Type": { +// "@type": "/gno.RefType", +// "ID": "std.Address" +// } +// } +// ] +// }, +// "V": { +// "@type": "/gno.FuncValue", +// "Closure": { +// "@type": "/gno.RefValue", +// "Escaped": true, +// "ObjectID": "a7f5397443359ea76c50be82c77f1f893a060925:5" +// }, +// "FileName": "native.gno", +// "IsMethod": false, +// "Name": "DerivePkgAddr", +// "NativeName": "DerivePkgAddr", +// "NativePkg": "std", +// "PkgPath": "std", +// "Source": { +// "@type": "/gno.RefNode", +// "BlockNode": null, +// "Location": { +// "File": "native.gno", +// "Line": "15", +// "Nonce": "0", +// "PkgPath": "std" +// } +// }, +// "Type": { +// "@type": "/gno.FuncType", +// "Params": [ +// { +// "Embedded": false, +// "Name": "pkgPath", +// "Tag": "", +// "Type": { +// "@type": "/gno.PrimitiveType", +// "value": "16" +// } +// } +// ], +// "Results": [ +// { +// "Embedded": false, +// "Name": "", +// "Tag": "", +// "Type": { +// "@type": "/gno.RefType", +// "ID": "std.Address" +// } +// } +// ] +// } +// } +// } +// ] +// } diff --git a/gnovm/tests/files/zrealm_tests0.gno b/gnovm/tests/files/zrealm_tests0.gno index cfb1f08c6f4..73d07f726eb 100644 --- a/gnovm/tests/files/zrealm_tests0.gno +++ b/gnovm/tests/files/zrealm_tests0.gno @@ -239,62 +239,8 @@ func main() { // "FileName": "tests.gno", // "IsMethod": true, // "Name": "Modify", -// "PkgPath": "gno.land/r/demo/tests", -// "Source": { -// "@type": "/gno.RefNode", -// "BlockNode": null, -// "Location": { -// "File": "tests.gno", -// "Line": "42", -// "Nonce": "0", -// "PkgPath": "gno.land/r/demo/tests" -// } -// }, -// "Type": { -// "@type": "/gno.FuncType", -// "Params": [ -// { -// "Embedded": false, -// "Name": "t", -// "Tag": "", -// "Type": { -// "@type": "/gno.PointerType", -// "Elt": { -// "@type": "/gno.RefType", -// "ID": "gno.land/r/demo/tests.TestRealmObject" -// } -// } -// } -// ], -// "Results": [] -// } -// } -// }, -// { -// "T": { -// "@type": "/gno.FuncType", -// "Params": [ -// { -// "Embedded": false, -// "Name": "t", -// "Tag": "", -// "Type": { -// "@type": "/gno.PointerType", -// "Elt": { -// "@type": "/gno.RefType", -// "ID": "gno.land/r/demo/tests.TestRealmObject" -// } -// } -// } -// ], -// "Results": [] -// }, -// "V": { -// "@type": "/gno.FuncValue", -// "Closure": null, -// "FileName": "tests.gno", -// "IsMethod": true, -// "Name": "Modify", +// "NativeName": "", +// "NativePkg": "", // "PkgPath": "gno.land/r/demo/tests", // "Source": { // "@type": "/gno.RefNode", @@ -399,6 +345,8 @@ func main() { // "FileName": "interfaces.gno", // "IsMethod": false, // "Name": "AddStringer", +// "NativeName": "", +// "NativePkg": "", // "PkgPath": "gno.land/r/demo/tests", // "Source": { // "@type": "/gno.RefNode", @@ -463,6 +411,8 @@ func main() { // "FileName": "interfaces.gno", // "IsMethod": false, // "Name": "Render", +// "NativeName": "", +// "NativePkg": "", // "PkgPath": "gno.land/r/demo/tests", // "Source": { // "@type": "/gno.RefNode", @@ -517,6 +467,8 @@ func main() { // "FileName": "tests.gno", // "IsMethod": false, // "Name": "IncCounter", +// "NativeName": "", +// "NativePkg": "", // "PkgPath": "gno.land/r/demo/tests", // "Source": { // "@type": "/gno.RefNode", @@ -561,6 +513,8 @@ func main() { // "FileName": "tests.gno", // "IsMethod": false, // "Name": "Counter", +// "NativeName": "", +// "NativePkg": "", // "PkgPath": "gno.land/r/demo/tests", // "Source": { // "@type": "/gno.RefNode", @@ -615,6 +569,8 @@ func main() { // "FileName": "tests.gno", // "IsMethod": false, // "Name": "CurrentRealmPath", +// "NativeName": "", +// "NativePkg": "", // "PkgPath": "gno.land/r/demo/tests", // "Source": { // "@type": "/gno.RefNode", @@ -659,6 +615,8 @@ func main() { // "FileName": "tests.gno", // "IsMethod": false, // "Name": "AssertOriginCall", +// "NativeName": "", +// "NativePkg": "", // "PkgPath": "gno.land/r/demo/tests", // "Source": { // "@type": "/gno.RefNode", @@ -703,6 +661,8 @@ func main() { // "FileName": "tests.gno", // "IsMethod": false, // "Name": "IsOriginCall", +// "NativeName": "", +// "NativePkg": "", // "PkgPath": "gno.land/r/demo/tests", // "Source": { // "@type": "/gno.RefNode", @@ -760,6 +720,8 @@ func main() { // "FileName": "tests.gno", // "IsMethod": false, // "Name": "ModifyTestRealmObject", +// "NativeName": "", +// "NativePkg": "", // "PkgPath": "gno.land/r/demo/tests", // "Source": { // "@type": "/gno.RefNode", @@ -807,6 +769,8 @@ func main() { // "FileName": "tests.gno", // "IsMethod": false, // "Name": "InitTestNodes", +// "NativeName": "", +// "NativePkg": "", // "PkgPath": "gno.land/r/demo/tests", // "Source": { // "@type": "/gno.RefNode", @@ -841,6 +805,8 @@ func main() { // "FileName": "tests.gno", // "IsMethod": false, // "Name": "ModTestNodes", +// "NativeName": "", +// "NativePkg": "", // "PkgPath": "gno.land/r/demo/tests", // "Source": { // "@type": "/gno.RefNode", @@ -875,6 +841,8 @@ func main() { // "FileName": "tests.gno", // "IsMethod": false, // "Name": "PrintTestNodes", +// "NativeName": "", +// "NativePkg": "", // "PkgPath": "gno.land/r/demo/tests", // "Source": { // "@type": "/gno.RefNode", @@ -919,6 +887,8 @@ func main() { // "FileName": "tests.gno", // "IsMethod": false, // "Name": "GetPrevRealm", +// "NativeName": "", +// "NativePkg": "", // "PkgPath": "gno.land/r/demo/tests", // "Source": { // "@type": "/gno.RefNode", @@ -973,6 +943,8 @@ func main() { // "FileName": "tests.gno", // "IsMethod": false, // "Name": "GetRSubtestsPrevRealm", +// "NativeName": "", +// "NativePkg": "", // "PkgPath": "gno.land/r/demo/tests", // "Source": { // "@type": "/gno.RefNode", @@ -1028,6 +1000,8 @@ func main() { // "FileName": "tests.gno", // "IsMethod": false, // "Name": "Exec", +// "NativeName": "", +// "NativePkg": "", // "PkgPath": "gno.land/r/demo/tests", // "Source": { // "@type": "/gno.RefNode", diff --git a/gnovm/tests/imports.go b/gnovm/tests/imports.go index 0741d0b466a..0db5651fbcc 100644 --- a/gnovm/tests/imports.go +++ b/gnovm/tests/imports.go @@ -26,6 +26,7 @@ import ( "math/rand" "net" "net/url" + "os" "path/filepath" "reflect" "sort" @@ -33,17 +34,15 @@ import ( "strings" "sync" "sync/atomic" - "testing" "text/template" "time" "unicode/utf8" gno "github.com/gnolang/gno/gnovm/pkg/gnolang" "github.com/gnolang/gno/gnovm/stdlibs" - "github.com/gnolang/gno/tm2/pkg/crypto" + teststdlibs "github.com/gnolang/gno/gnovm/tests/stdlibs" dbm "github.com/gnolang/gno/tm2/pkg/db" osm "github.com/gnolang/gno/tm2/pkg/os" - "github.com/gnolang/gno/tm2/pkg/std" "github.com/gnolang/gno/tm2/pkg/store/dbadapter" "github.com/gnolang/gno/tm2/pkg/store/iavl" stypes "github.com/gnolang/gno/tm2/pkg/store/types" @@ -51,15 +50,16 @@ import ( type importMode uint64 +// Import modes to control the import behaviour of TestStore. const ( + // use stdlibs/* only (except a few exceptions). for stdlibs/* and examples/* testing. ImportModeStdlibsOnly importMode = iota + // use stdlibs/* if present, otherwise use native. used in files/tests, excluded for *_native.go ImportModeStdlibsPreferred + // do not use stdlibs/* if native registered. used in files/tests, excluded for *_stdlibs.go ImportModeNativePreferred ) -// ImportModeStdlibsOnly: use stdlibs/* only (except a few exceptions). for stdlibs/* and examples/* testing. -// ImportModeStdlibsPreferred: use stdlibs/* if present, otherwise use native. for files/tests2/*. -// ImportModeNativePreferred: do not use stdlibs/* if native registered. for files/tests/*. // NOTE: this isn't safe, should only be used for testing. func TestStore(rootDir, filesPath string, stdin io.Reader, stdout, stderr io.Writer, mode importMode) (store gno.Store) { getPackage := func(pkgPath string) (pn *gno.PackageNode, pv *gno.PackageValue) { @@ -93,23 +93,9 @@ func TestStore(rootDir, filesPath string, stdin io.Reader, stdout, stderr io.Wri // if stdlibs package is preferred , try to load it first. if mode == ImportModeStdlibsOnly || mode == ImportModeStdlibsPreferred { - stdlibPath := filepath.Join(rootDir, "gnovm", "stdlibs", pkgPath) - if osm.DirExists(stdlibPath) { - memPkg := gno.ReadMemPackage(stdlibPath, pkgPath) - if !memPkg.IsEmpty() { - m2 := gno.NewMachineWithOptions(gno.MachineOptions{ - // NOTE: see also pkgs/sdk/vm/builtins.go - // XXX: why does this fail when just pkgPath? - PkgPath: "gno.land/r/stdlibs/" + pkgPath, - Output: stdout, - Store: store, - }) - save := pkgPath != "testing" // never save the "testing" package - return m2.RunMemPackage(memPkg, save) - } - - // There is no package there, but maybe we have a - // native counterpart below. + pn, pv = loadStdlib(rootDir, pkgPath, store, stdout) + if pn != nil { + return } } @@ -120,12 +106,10 @@ func TestStore(rootDir, filesPath string, stdin io.Reader, stdout, stderr io.Wri pkgPath == "crypto/rand" || pkgPath == "crypto/md5" || pkgPath == "crypto/sha1" || - pkgPath == "encoding/base64" || pkgPath == "encoding/binary" || pkgPath == "encoding/json" || pkgPath == "encoding/xml" || pkgPath == "internal/os_test" || - pkgPath == "math" || pkgPath == "math/big" || pkgPath == "math/rand" || mode == ImportModeStdlibsPreferred || @@ -215,17 +199,6 @@ func TestStore(rootDir, filesPath string, stdin io.Reader, stdout, stderr io.Wri pkg.DefineGoNativeType(reflect.TypeOf(net.TCPAddr{})) pkg.DefineGoNativeValue("IPv4", net.IPv4) return pkg, pkg.NewPackage() - case "net/http": - // XXX UNSAFE - // There's no reason why we can't replace these with safer alternatives. - panic("just say gno") - /* - pkg := gno.NewPackageNode("http", pkgPath, nil) - pkg.DefineGoNativeType(reflect.TypeOf(http.Request{})) - pkg.DefineGoNativeValue("DefaultClient", http.DefaultClient) - pkg.DefineGoNativeType(reflect.TypeOf(http.Client{})) - return pkg, pkg.NewPackage() - */ case "net/url": pkg := gno.NewPackageNode("url", pkgPath, nil) pkg.DefineGoNativeType(reflect.TypeOf(url.Values{})) @@ -277,6 +250,8 @@ func TestStore(rootDir, filesPath string, stdin io.Reader, stdout, stderr io.Wri pkg.DefineGoNativeValue("Abs", math.Abs) pkg.DefineGoNativeValue("Cos", math.Cos) pkg.DefineGoNativeValue("Pi", math.Pi) + pkg.DefineGoNativeValue("Float64bits", math.Float64bits) + pkg.DefineGoNativeValue("Pi", math.Pi) pkg.DefineGoNativeValue("MaxFloat32", math.MaxFloat32) pkg.DefineGoNativeValue("MaxFloat64", math.MaxFloat64) return pkg, pkg.NewPackage() @@ -385,25 +360,6 @@ func TestStore(rootDir, filesPath string, stdin io.Reader, stdout, stderr io.Wri pkg := gno.NewPackageNode("fnv", pkgPath, nil) pkg.DefineGoNativeValue("New32a", fnv.New32a) return pkg, pkg.NewPackage() - /* XXX support somehow for speed. for now, generic implemented in stdlibs. - case "internal/bytealg": - pkg := gno.NewPackageNode("bytealg", pkgPath, nil) - pkg.DefineGoNativeValue("Compare", bytealg.Compare) - pkg.DefineGoNativeValue("CountString", bytealg.CountString) - pkg.DefineGoNativeValue("Cutover", bytealg.Cutover) - pkg.DefineGoNativeValue("Equal", bytealg.Equal) - pkg.DefineGoNativeValue("HashStr", bytealg.HashStr) - pkg.DefineGoNativeValue("HashStrBytes", bytealg.HashStrBytes) - pkg.DefineGoNativeValue("HashStrRev", bytealg.HashStrRev) - pkg.DefineGoNativeValue("HashStrRevBytes", bytealg.HashStrRevBytes) - pkg.DefineGoNativeValue("Index", bytealg.Index) - pkg.DefineGoNativeValue("IndexByte", bytealg.IndexByte) - pkg.DefineGoNativeValue("IndexByteString", bytealg.IndexByteString) - pkg.DefineGoNativeValue("IndexRabinKarp", bytealg.IndexRabinKarp) - pkg.DefineGoNativeValue("IndexRabinKarpBytes", bytealg.IndexRabinKarpBytes) - pkg.DefineGoNativeValue("IndexString", bytealg.IndexString) - return pkg, pkg.NewPackage() - */ default: // continue on... } @@ -411,19 +367,8 @@ func TestStore(rootDir, filesPath string, stdin io.Reader, stdout, stderr io.Wri // if native package is preferred, try to load stdlibs/* as backup. if mode == ImportModeNativePreferred { - stdlibPath := filepath.Join(rootDir, "gnovm", "stdlibs", pkgPath) - if osm.DirExists(stdlibPath) { - memPkg := gno.ReadMemPackage(stdlibPath, pkgPath) - if memPkg.IsEmpty() { - panic(fmt.Sprintf("found an empty package %q", pkgPath)) - } - - m2 := gno.NewMachineWithOptions(gno.MachineOptions{ - PkgPath: "test", - Output: stdout, - Store: store, - }) - pn, pv = m2.RunMemPackage(memPkg, true) + pn, pv = loadStdlib(rootDir, pkgPath, store, stdout) + if pn != nil { return } } @@ -452,6 +397,7 @@ func TestStore(rootDir, filesPath string, stdin io.Reader, stdout, stderr io.Wri iavlStore := iavl.StoreConstructor(db, stypes.StoreOptions{}) store = gno.NewStore(nil, baseStore, iavlStore) store.SetPackageGetter(getPackage) + store.SetNativeStore(teststdlibs.NativeStore) store.SetPackageInjector(testPackageInjector) store.SetStrictGo2GnoMapping(false) // native mappings @@ -459,30 +405,49 @@ func TestStore(rootDir, filesPath string, stdin io.Reader, stdout, stderr io.Wri return } -//---------------------------------------- -// testInjectNatives -// analogous to stdlibs.InjectNatives, but with -// native methods suitable for the testing environment. - -func testPackageInjector(store gno.Store, pn *gno.PackageNode) { - // Also inject stdlibs native functions. - stdlibs.InjectPackage(store, pn) - isOriginCall := func(m *gno.Machine) bool { - tname := m.Frames[0].Func.Name - switch tname { - case "main": // test is a _filetest - return len(m.Frames) == 3 - case "runtest": // test is a _test - return len(m.Frames) == 7 +func loadStdlib(rootDir, pkgPath string, store gno.Store, stdout io.Writer) (*gno.PackageNode, *gno.PackageValue) { + dirs := [...]string{ + // normal stdlib path. + filepath.Join(rootDir, "gnovm", "stdlibs", pkgPath), + // override path. definitions here override the previous if duplicate. + filepath.Join(rootDir, "gnovm", "tests", "stdlibs", pkgPath), + } + files := make([]string, 0, 32) // pre-alloc 32 as a likely high number of files + for _, path := range dirs { + dl, err := os.ReadDir(path) + if err != nil { + if os.IsNotExist(err) { + continue + } + panic(fmt.Errorf("could not access dir %q: %w", path, err)) } - // support init() in _filetest - // XXX do we need to distinguish from 'runtest'/_test? - // XXX pretty hacky even if not. - if strings.HasPrefix(string(tname), "init.") { - return len(m.Frames) == 3 + + for _, f := range dl { + // NOTE: RunMemPackage has other rules; those should be mostly useful + // for on-chain packages (ie. include README and gno.mod). + if !f.IsDir() && strings.HasSuffix(f.Name(), ".gno") { + files = append(files, filepath.Join(path, f.Name())) + } } - panic("unable to determine if test is a _test or a _filetest") } + if len(files) == 0 { + return nil, nil + } + + memPkg := gno.ReadMemPackageFromList(files, pkgPath) + m2 := gno.NewMachineWithOptions(gno.MachineOptions{ + // NOTE: see also pkgs/sdk/vm/builtins.go + // Needs PkgPath != its name because TestStore.getPackage is the package + // getter for the store, which calls loadStdlib, so it would be recursively called. + PkgPath: "stdlibload", + Output: stdout, + Store: store, + }) + save := pkgPath != "testing" // never save the "testing" package + return m2.RunMemPackageWithOverrides(memPkg, save) +} + +func testPackageInjector(store gno.Store, pn *gno.PackageNode) { // Test specific injections: switch pn.PkgPath { case "strconv": @@ -490,203 +455,6 @@ func testPackageInjector(store gno.Store, pn *gno.PackageNode) { // from stdlibs.InjectNatives. pn.DefineGoNativeType(reflect.TypeOf(strconv.NumError{})) pn.DefineGoNativeValue("ParseInt", strconv.ParseInt) - case "std": - // NOTE: some of these are overrides. - // Also see stdlibs/InjectPackage. - pn.DefineNativeOverride("AssertOriginCall", - /* - gno.Flds( // params - ), - gno.Flds( // results - ), - */ - func(m *gno.Machine) { - if !isOriginCall(m) { - m.Panic(typedString("invalid non-origin call")) - return - } - }, - ) - pn.DefineNativeOverride("IsOriginCall", - /* - gno.Flds( // params - ), - gno.Flds( // results - "isOrigin", "bool", - ), - */ - func(m *gno.Machine) { - res0 := gno.TypedValue{T: gno.BoolType} - res0.SetBool(isOriginCall(m)) - m.PushValue(res0) - }, - ) - pn.DefineNativeOverride("GetCallerAt", - /* - gno.Flds( // params - "n", "int", - ), - gno.Flds( // results - "", "Address", - ), - */ - func(m *gno.Machine) { - arg0 := m.LastBlock().GetParams1().TV - n := arg0.GetInt() - if n <= 0 { - m.Panic(typedString("GetCallerAt requires positive arg")) - return - } - if n > m.NumFrames()-1 { - // NOTE: the last frame's LastPackage - // is set to the original non-frame - // package, so need this check. - m.Panic(typedString("frame not found")) - return - } - var pkgAddr string - if n == m.NumFrames()-1 { - // This makes it consistent with GetOrigCaller and TestSetOrigCaller. - ctx := m.Context.(stdlibs.ExecContext) - pkgAddr = string(ctx.OrigCaller) - } else { - pkgAddr = string(m.LastCallFrame(n).LastPackage.GetPkgAddr().Bech32()) - } - res0 := gno.Go2GnoValue( - m.Alloc, - m.Store, - reflect.ValueOf(pkgAddr), - ) - addrT := store.GetType(gno.DeclaredTypeID("std", "Address")) - res0.T = addrT - m.PushValue(res0) - }, - ) - pn.DefineNative("TestSetOrigCaller", - gno.Flds( // params - "", "Address", - ), - gno.Flds( // results - ), - func(m *gno.Machine) { - arg0 := m.LastBlock().GetParams1().TV - addr := arg0.GetString() - // overwrite context - ctx := m.Context.(stdlibs.ExecContext) - ctx.OrigCaller = crypto.Bech32Address(addr) - m.Context = ctx - }, - ) - pn.DefineNative("TestSetOrigPkgAddr", - gno.Flds( // params - "", "Address", - ), - gno.Flds( // results - ), - func(m *gno.Machine) { - arg0 := m.LastBlock().GetParams1().TV - addr := crypto.Bech32Address(arg0.GetString()) - // overwrite context - ctx := m.Context.(stdlibs.ExecContext) - ctx.OrigPkgAddr = addr - m.Context = ctx - }, - ) - pn.DefineNative("TestSetOrigSend", - gno.Flds( // params - "sent", "Coins", - "spent", "Coins", - ), - gno.Flds( // results - ), - func(m *gno.Machine) { - arg0, arg1 := m.LastBlock().GetParams2() - var sent std.Coins - rvSent := reflect.ValueOf(&sent).Elem() - gno.Gno2GoValue(arg0.TV, rvSent) - sent = rvSent.Interface().(std.Coins) // needed? - var spent std.Coins - rvSpent := reflect.ValueOf(&spent).Elem() - gno.Gno2GoValue(arg1.TV, rvSpent) - spent = rvSpent.Interface().(std.Coins) // needed? - // overwrite context. - ctx := m.Context.(stdlibs.ExecContext) - ctx.OrigSend = sent - ctx.OrigSendSpent = &spent - m.Context = ctx - }, - ) - pn.DefineNative("TestIssueCoins", - gno.Flds( // params - "addr", "Address", - "coins", "Coins", - ), - gno.Flds( // results - ), - func(m *gno.Machine) { - arg0, arg1 := m.LastBlock().GetParams2() - addr := crypto.Bech32Address(arg0.TV.GetString()) - var coins std.Coins - rvCoins := reflect.ValueOf(&coins).Elem() - gno.Gno2GoValue(arg1.TV, rvCoins) - coins = rvCoins.Interface().(std.Coins) // needed? - // overwrite context. - ctx := m.Context.(stdlibs.ExecContext) - banker := ctx.Banker - for _, coin := range coins { - banker.IssueCoin(addr, coin.Denom, coin.Amount) - } - }, - ) - pn.DefineNative("TestCurrentRealm", - gno.Flds( // params - ), - gno.Flds( // results - "realm", "string", - ), - func(m *gno.Machine) { - rlmpath := m.Realm.Path - m.PushValue(typedString(rlmpath)) - }, - ) - pn.DefineNative("TestSkipHeights", - gno.Flds( // params - "count", "int64", - ), - gno.Flds( // results - ), - func(m *gno.Machine) { - arg0 := m.LastBlock().GetParams1().TV - count := arg0.GetInt64() - - ctx := m.Context.(stdlibs.ExecContext) - ctx.Height += count - m.Context = ctx - }, - ) - // TODO: move elsewhere. - pn.DefineNative("ClearStoreCache", - gno.Flds( // params - ), - gno.Flds( // results - ), - func(m *gno.Machine) { - if gno.IsDebug() && testing.Verbose() { - store.Print() - fmt.Println("========================================") - fmt.Println("CLEAR CACHE (RUNTIME)") - fmt.Println("========================================") - } - m.Store.ClearCache() - m.PreprocessAllFilesAndSaveBlockNodes() - if gno.IsDebug() && testing.Verbose() { - store.Print() - fmt.Println("========================================") - fmt.Println("CLEAR CACHE DONE") - fmt.Println("========================================") - } - }, - ) } } @@ -703,13 +471,6 @@ func (*dummyReader) Read(b []byte) (n int, err error) { //---------------------------------------- -// NOTE: does not allocate; used for panics. -func typedString(s string) gno.TypedValue { - tv := gno.TypedValue{T: gno.StringType} - tv.V = gno.StringValue(s) - return tv -} - type TestReport struct { Name string Verbose bool diff --git a/gnovm/tests/stdlibs/README.md b/gnovm/tests/stdlibs/README.md new file mode 100644 index 00000000000..16d5d171342 --- /dev/null +++ b/gnovm/tests/stdlibs/README.md @@ -0,0 +1,6 @@ +# tests/stdlibs + +This directory contains test-specific standard libraries. These are only +available when testing gno code in `_test.gno` and `_filetest.gno` files. +Re-declarations of functions already existing override the definitions of the +normal stdlibs directory. diff --git a/gnovm/tests/stdlibs/native.go b/gnovm/tests/stdlibs/native.go new file mode 100644 index 00000000000..18d281bebe0 --- /dev/null +++ b/gnovm/tests/stdlibs/native.go @@ -0,0 +1,226 @@ +// This file is autogenerated from the genstd tool (@/misc/genstd); do not edit. +// To regenerate it, run `go generate` from this directory. + +package stdlibs + +import ( + "reflect" + + gno "github.com/gnolang/gno/gnovm/pkg/gnolang" + testlibs_std "github.com/gnolang/gno/gnovm/tests/stdlibs/std" + tm2_crypto "github.com/gnolang/gno/tm2/pkg/crypto" + tm2_std "github.com/gnolang/gno/tm2/pkg/std" +) + +type nativeFunc struct { + gnoPkg string + gnoFunc gno.Name + params []gno.FieldTypeExpr + results []gno.FieldTypeExpr + f func(m *gno.Machine) +} + +var nativeFuncs = [...]nativeFunc{ + { + "std", + "AssertOriginCall", + []gno.FieldTypeExpr{}, + []gno.FieldTypeExpr{}, + func(m *gno.Machine) { + testlibs_std.AssertOriginCall( + m, + ) + }, + }, + { + "std", + "IsOriginCall", + []gno.FieldTypeExpr{}, + []gno.FieldTypeExpr{ + {Name: gno.N("r0"), Type: gno.X("bool")}, + }, + func(m *gno.Machine) { + r0 := testlibs_std.IsOriginCall( + m, + ) + + m.PushValue(gno.Go2GnoValue( + m.Alloc, + m.Store, + reflect.ValueOf(&r0).Elem(), + )) + }, + }, + { + "std", + "TestCurrentRealm", + []gno.FieldTypeExpr{}, + []gno.FieldTypeExpr{ + {Name: gno.N("r0"), Type: gno.X("string")}, + }, + func(m *gno.Machine) { + r0 := testlibs_std.TestCurrentRealm( + m, + ) + + m.PushValue(gno.Go2GnoValue( + m.Alloc, + m.Store, + reflect.ValueOf(&r0).Elem(), + )) + }, + }, + { + "std", + "TestSkipHeights", + []gno.FieldTypeExpr{ + {Name: gno.N("p0"), Type: gno.X("int64")}, + }, + []gno.FieldTypeExpr{}, + func(m *gno.Machine) { + b := m.LastBlock() + var ( + p0 int64 + rp0 = reflect.ValueOf(&p0).Elem() + ) + + gno.Gno2GoValue(b.GetPointerTo(nil, gno.NewValuePathBlock(1, 0, "")).TV, rp0) + + testlibs_std.TestSkipHeights( + m, + p0) + }, + }, + { + "std", + "ClearStoreCache", + []gno.FieldTypeExpr{}, + []gno.FieldTypeExpr{}, + func(m *gno.Machine) { + testlibs_std.ClearStoreCache( + m, + ) + }, + }, + { + "std", + "GetCallerAt", + []gno.FieldTypeExpr{ + {Name: gno.N("p0"), Type: gno.X("int")}, + }, + []gno.FieldTypeExpr{ + {Name: gno.N("r0"), Type: gno.X("Address")}, + }, + func(m *gno.Machine) { + b := m.LastBlock() + var ( + p0 int + rp0 = reflect.ValueOf(&p0).Elem() + ) + + gno.Gno2GoValue(b.GetPointerTo(nil, gno.NewValuePathBlock(1, 0, "")).TV, rp0) + + r0 := testlibs_std.GetCallerAt( + m, + p0) + + m.PushValue(gno.Go2GnoValue( + m.Alloc, + m.Store, + reflect.ValueOf(&r0).Elem(), + )) + }, + }, + { + "std", + "TestSetOrigCaller", + []gno.FieldTypeExpr{ + {Name: gno.N("p0"), Type: gno.X("Address")}, + }, + []gno.FieldTypeExpr{}, + func(m *gno.Machine) { + b := m.LastBlock() + var ( + p0 tm2_crypto.Bech32Address + rp0 = reflect.ValueOf(&p0).Elem() + ) + + gno.Gno2GoValue(b.GetPointerTo(nil, gno.NewValuePathBlock(1, 0, "")).TV, rp0) + + testlibs_std.TestSetOrigCaller( + m, + p0) + }, + }, + { + "std", + "TestSetOrigPkgAddr", + []gno.FieldTypeExpr{ + {Name: gno.N("p0"), Type: gno.X("Address")}, + }, + []gno.FieldTypeExpr{}, + func(m *gno.Machine) { + b := m.LastBlock() + var ( + p0 tm2_crypto.Bech32Address + rp0 = reflect.ValueOf(&p0).Elem() + ) + + gno.Gno2GoValue(b.GetPointerTo(nil, gno.NewValuePathBlock(1, 0, "")).TV, rp0) + + testlibs_std.TestSetOrigPkgAddr( + m, + p0) + }, + }, + { + "std", + "TestSetOrigSend", + []gno.FieldTypeExpr{ + {Name: gno.N("p0"), Type: gno.X("Coins")}, + {Name: gno.N("p1"), Type: gno.X("Coins")}, + }, + []gno.FieldTypeExpr{}, + func(m *gno.Machine) { + b := m.LastBlock() + var ( + p0 tm2_std.Coins + rp0 = reflect.ValueOf(&p0).Elem() + p1 tm2_std.Coins + rp1 = reflect.ValueOf(&p1).Elem() + ) + + gno.Gno2GoValue(b.GetPointerTo(nil, gno.NewValuePathBlock(1, 0, "")).TV, rp0) + gno.Gno2GoValue(b.GetPointerTo(nil, gno.NewValuePathBlock(1, 1, "")).TV, rp1) + + testlibs_std.TestSetOrigSend( + m, + p0, p1) + }, + }, + { + "std", + "TestIssueCoins", + []gno.FieldTypeExpr{ + {Name: gno.N("p0"), Type: gno.X("Address")}, + {Name: gno.N("p1"), Type: gno.X("Coins")}, + }, + []gno.FieldTypeExpr{}, + func(m *gno.Machine) { + b := m.LastBlock() + var ( + p0 tm2_crypto.Bech32Address + rp0 = reflect.ValueOf(&p0).Elem() + p1 tm2_std.Coins + rp1 = reflect.ValueOf(&p1).Elem() + ) + + gno.Gno2GoValue(b.GetPointerTo(nil, gno.NewValuePathBlock(1, 0, "")).TV, rp0) + gno.Gno2GoValue(b.GetPointerTo(nil, gno.NewValuePathBlock(1, 1, "")).TV, rp1) + + testlibs_std.TestIssueCoins( + m, + p0, p1) + }, + }, +} diff --git a/gnovm/tests/stdlibs/std/std.gno b/gnovm/tests/stdlibs/std/std.gno new file mode 100644 index 00000000000..380549be694 --- /dev/null +++ b/gnovm/tests/stdlibs/std/std.gno @@ -0,0 +1,12 @@ +package std + +func AssertOriginCall() // injected +func IsOriginCall() bool // injected +func TestCurrentRealm() string // injected +func TestSkipHeights(count int64) // injected +func ClearStoreCache() // injected +func GetCallerAt(n int) Address // injected +func TestSetOrigCaller(addr Address) // injected +func TestSetOrigPkgAddr(addr Address) // injected +func TestSetOrigSend(sent, spent Coins) // injected +func TestIssueCoins(addr Address, coins Coins) // injected diff --git a/gnovm/tests/stdlibs/std/std.go b/gnovm/tests/stdlibs/std/std.go new file mode 100644 index 00000000000..27ad079b4a6 --- /dev/null +++ b/gnovm/tests/stdlibs/std/std.go @@ -0,0 +1,116 @@ +package std + +import ( + "fmt" + "strings" + "testing" + + gno "github.com/gnolang/gno/gnovm/pkg/gnolang" + "github.com/gnolang/gno/gnovm/stdlibs" + "github.com/gnolang/gno/gnovm/stdlibs/std" + "github.com/gnolang/gno/tm2/pkg/crypto" + tm2std "github.com/gnolang/gno/tm2/pkg/std" +) + +func AssertOriginCall(m *gno.Machine) { + if !IsOriginCall(m) { + m.Panic(typedString("invalid non-origin call")) + } +} + +func typedString(s gno.StringValue) gno.TypedValue { + tv := gno.TypedValue{T: gno.StringType} + tv.SetString(s) + return tv +} + +func IsOriginCall(m *gno.Machine) bool { + tname := m.Frames[0].Func.Name + switch tname { + case "main": // test is a _filetest + return len(m.Frames) == 3 + case "runtest": // test is a _test + return len(m.Frames) == 7 + } + // support init() in _filetest + // XXX do we need to distinguish from 'runtest'/_test? + // XXX pretty hacky even if not. + if strings.HasPrefix(string(tname), "init.") { + return len(m.Frames) == 3 + } + panic("unable to determine if test is a _test or a _filetest") +} + +func TestCurrentRealm(m *gno.Machine) string { + return m.Realm.Path +} + +func TestSkipHeights(m *gno.Machine, count int64) { + ctx := m.Context.(std.ExecContext) + ctx.Height += count + m.Context = ctx +} + +func ClearStoreCache(m *gno.Machine) { + if gno.IsDebug() && testing.Verbose() { + m.Store.Print() + fmt.Println("========================================") + fmt.Println("CLEAR CACHE (RUNTIME)") + fmt.Println("========================================") + } + m.Store.ClearCache() + m.PreprocessAllFilesAndSaveBlockNodes() + if gno.IsDebug() && testing.Verbose() { + m.Store.Print() + fmt.Println("========================================") + fmt.Println("CLEAR CACHE DONE") + fmt.Println("========================================") + } +} + +func GetCallerAt(m *gno.Machine, n int) crypto.Bech32Address { + if n <= 0 { + m.Panic(typedString("GetCallerAt requires positive arg")) + return "" + } + if n > m.NumFrames()-1 { + // NOTE: the last frame's LastPackage + // is set to the original non-frame + // package, so need this check. + m.Panic(typedString("frame not found")) + return "" + } + if n == m.NumFrames()-1 { + // This makes it consistent with GetOrigCaller and TestSetOrigCaller. + ctx := m.Context.(stdlibs.ExecContext) + return ctx.OrigCaller + } + return m.LastCallFrame(n).LastPackage.GetPkgAddr().Bech32() +} + +func TestSetOrigCaller(m *gno.Machine, addr crypto.Bech32Address) { + ctx := m.Context.(std.ExecContext) + ctx.OrigCaller = addr + m.Context = ctx +} + +func TestSetOrigPkgAddr(m *gno.Machine, addr crypto.Bech32Address) { + ctx := m.Context.(stdlibs.ExecContext) + ctx.OrigPkgAddr = addr + m.Context = ctx +} + +func TestSetOrigSend(m *gno.Machine, sent, spent tm2std.Coins) { + ctx := m.Context.(stdlibs.ExecContext) + ctx.OrigSend = sent + ctx.OrigSendSpent = &spent + m.Context = ctx +} + +func TestIssueCoins(m *gno.Machine, addr crypto.Bech32Address, coins tm2std.Coins) { + ctx := m.Context.(stdlibs.ExecContext) + banker := ctx.Banker + for _, coin := range coins { + banker.IssueCoin(addr, coin.Denom, coin.Amount) + } +} diff --git a/gnovm/tests/stdlibs/stdlibs.go b/gnovm/tests/stdlibs/stdlibs.go new file mode 100644 index 00000000000..b0a1050af41 --- /dev/null +++ b/gnovm/tests/stdlibs/stdlibs.go @@ -0,0 +1,18 @@ +// Package stdlibs provides supplemental stdlibs for the testing environment. +package stdlibs + +import ( + gno "github.com/gnolang/gno/gnovm/pkg/gnolang" + "github.com/gnolang/gno/gnovm/stdlibs" +) + +//go:generate go run github.com/gnolang/gno/misc/genstd + +func NativeStore(pkgPath string, name gno.Name) func(*gno.Machine) { + for _, nf := range nativeFuncs { + if nf.gnoPkg == pkgPath && name == nf.gnoFunc { + return nf.f + } + } + return stdlibs.NativeStore(pkgPath, name) +} diff --git a/misc/Makefile b/misc/Makefile new file mode 100644 index 00000000000..84acc40e387 --- /dev/null +++ b/misc/Makefile @@ -0,0 +1,28 @@ +.PHONY: help +help: + @echo "Available make commands:" + @cat Makefile | grep '^[a-z][^:]*:' | cut -d: -f1 | sort | sed 's/^/ /' + +rundep=go run -modfile ./devdeps/go.mod + +######################################## +# Dev tools +.PHONY: lint +lint: + $(rundep) github.com/golangci/golangci-lint/cmd/golangci-lint run --config ../.github/golangci.yml ./... + +.PHONY: fmt +GOFMT_FLAGS ?= -w +fmt: + $(rundep) mvdan.cc/gofumpt $(GOFMT_FLAGS) . + +######################################## +# Test suite +.PHONY: test +test: _test.genstd + +GOTEST_FLAGS ?= -v -p 1 -timeout=30m + +.PHONY: _test.genstd +_test.genstd: + go test ./genstd/... $(GOTEST_FLAGS) diff --git a/misc/genstd/exprstring.go b/misc/genstd/exprstring.go new file mode 100644 index 00000000000..c95c05c584e --- /dev/null +++ b/misc/genstd/exprstring.go @@ -0,0 +1,290 @@ +// Forked from go/types (go 1.20.3) to implement support for *linkedIdent. +// It cannot be easily split from the original as WriteExpr is highly recursive. + +// Copyright 2013 The Go Authors. All rights reserved. +// Use of this source code is governed by a BSD-style +// license that can be found in the LICENSE file. + +// This file implements printing of expressions. + +package main + +import ( + "bytes" + "fmt" + "go/ast" + "go/types" +) + +const ( + printerModeGoQualified = iota + printerModeGnoType +) + +type exprPrinter struct { + mode int +} + +// ExprString returns the (possibly shortened) string representation for x. +// Shortened representations are suitable for user interfaces but may not +// necessarily follow Go syntax. +// +// ExprString is identical to [types.ExprString] with the difference that it +// supports *linkedIdent. +func (ep *exprPrinter) ExprString(x ast.Expr) string { + var buf bytes.Buffer + ep.WriteExpr(&buf, x) + return buf.String() +} + +// WriteExpr writes the (possibly shortened) string representation for x to buf. +// Shortened representations are suitable for user interfaces but may not +// necessarily follow Go syntax. +// +// WriteExpr is identical to [types.WriteExpr] with the difference that it +// supports *linkedIdent. +func (ep *exprPrinter) WriteExpr(buf *bytes.Buffer, x ast.Expr) { + // The AST preserves source-level parentheses so there is + // no need to introduce them here to correct for different + // operator precedences. (This assumes that the AST was + // generated by a Go parser.) + + switch x := x.(type) { + default: + // fallback to go original -- for all non-recursive ast.Expr types + types.WriteExpr(buf, x) + + case *linkedIdent: + switch ep.mode { + case printerModeGoQualified: + n := pkgNameFromPath(x.lt.goPackage) + buf.WriteString(n) + buf.WriteByte('.') + buf.WriteString(x.lt.goName) + case printerModeGnoType: + buf.WriteString(x.lt.gnoName) + default: + panic(fmt.Errorf("invalid mode %d", ep.mode)) + } + + case *ast.Ellipsis: + buf.WriteString("...") + if x.Elt != nil { + ep.WriteExpr(buf, x.Elt) + } + + case *ast.FuncLit: + buf.WriteByte('(') + ep.WriteExpr(buf, x.Type) + buf.WriteString(" literal)") // shortened + + case *ast.CompositeLit: + ep.WriteExpr(buf, x.Type) + buf.WriteByte('{') + if len(x.Elts) > 0 { + buf.WriteString("…") + } + buf.WriteByte('}') + + case *ast.ParenExpr: + buf.WriteByte('(') + ep.WriteExpr(buf, x.X) + buf.WriteByte(')') + + case *ast.SelectorExpr: + ep.WriteExpr(buf, x.X) + buf.WriteByte('.') + buf.WriteString(x.Sel.Name) + + case *ast.IndexExpr, *ast.IndexListExpr: + ix := tpUnpackIndexExpr(x) + ep.WriteExpr(buf, ix.X) + buf.WriteByte('[') + ep.writeExprList(buf, ix.Indices) + buf.WriteByte(']') + + case *ast.SliceExpr: + ep.WriteExpr(buf, x.X) + buf.WriteByte('[') + if x.Low != nil { + ep.WriteExpr(buf, x.Low) + } + buf.WriteByte(':') + if x.High != nil { + ep.WriteExpr(buf, x.High) + } + if x.Slice3 { + buf.WriteByte(':') + if x.Max != nil { + ep.WriteExpr(buf, x.Max) + } + } + buf.WriteByte(']') + + case *ast.TypeAssertExpr: + ep.WriteExpr(buf, x.X) + buf.WriteString(".(") + ep.WriteExpr(buf, x.Type) + buf.WriteByte(')') + + case *ast.CallExpr: + ep.WriteExpr(buf, x.Fun) + buf.WriteByte('(') + ep.writeExprList(buf, x.Args) + if x.Ellipsis.IsValid() { + buf.WriteString("...") + } + buf.WriteByte(')') + + case *ast.StarExpr: + buf.WriteByte('*') + ep.WriteExpr(buf, x.X) + + case *ast.UnaryExpr: + buf.WriteString(x.Op.String()) + ep.WriteExpr(buf, x.X) + + case *ast.BinaryExpr: + ep.WriteExpr(buf, x.X) + buf.WriteByte(' ') + buf.WriteString(x.Op.String()) + buf.WriteByte(' ') + ep.WriteExpr(buf, x.Y) + + case *ast.ArrayType: + buf.WriteByte('[') + if x.Len != nil { + ep.WriteExpr(buf, x.Len) + } + buf.WriteByte(']') + ep.WriteExpr(buf, x.Elt) + + case *ast.StructType: + buf.WriteString("struct{") + ep.writeFieldList(buf, x.Fields.List, "; ", false) + buf.WriteByte('}') + + case *ast.FuncType: + buf.WriteString("func") + ep.writeSigExpr(buf, x) + + case *ast.InterfaceType: + buf.WriteString("interface{") + ep.writeFieldList(buf, x.Methods.List, "; ", true) + buf.WriteByte('}') + + case *ast.MapType: + buf.WriteString("map[") + ep.WriteExpr(buf, x.Key) + buf.WriteByte(']') + ep.WriteExpr(buf, x.Value) + + case *ast.ChanType: + var s string + switch x.Dir { + case ast.SEND: + s = "chan<- " + case ast.RECV: + s = "<-chan " + default: + s = "chan " + } + buf.WriteString(s) + ep.WriteExpr(buf, x.Value) + } +} + +func (ep *exprPrinter) writeSigExpr(buf *bytes.Buffer, sig *ast.FuncType) { + buf.WriteByte('(') + ep.writeFieldList(buf, sig.Params.List, ", ", false) + buf.WriteByte(')') + + res := sig.Results + n := res.NumFields() + if n == 0 { + // no result + return + } + + buf.WriteByte(' ') + if n == 1 && len(res.List[0].Names) == 0 { + // single unnamed result + ep.WriteExpr(buf, res.List[0].Type) + return + } + + // multiple or named result(s) + buf.WriteByte('(') + ep.writeFieldList(buf, res.List, ", ", false) + buf.WriteByte(')') +} + +func (ep *exprPrinter) writeFieldList(buf *bytes.Buffer, list []*ast.Field, sep string, iface bool) { + for i, f := range list { + if i > 0 { + buf.WriteString(sep) + } + + // field list names + ep.writeIdentList(buf, f.Names) + + // types of interface methods consist of signatures only + if sig, _ := f.Type.(*ast.FuncType); sig != nil && iface { + ep.writeSigExpr(buf, sig) + continue + } + + // named fields are separated with a blank from the field type + if len(f.Names) > 0 { + buf.WriteByte(' ') + } + + ep.WriteExpr(buf, f.Type) + + // ignore tag + } +} + +func (ep *exprPrinter) writeIdentList(buf *bytes.Buffer, list []*ast.Ident) { + for i, x := range list { + if i > 0 { + buf.WriteString(", ") + } + buf.WriteString(x.Name) + } +} + +func (ep *exprPrinter) writeExprList(buf *bytes.Buffer, list []ast.Expr) { + for i, x := range list { + if i > 0 { + buf.WriteString(", ") + } + ep.WriteExpr(buf, x) + } +} + +// The following are copied from go/internal/typeparams. +// We cannot use the original directly as it comes from an "internal" package. + +// tpIndexExpr wraps an ast.IndexExpr or ast.IndexListExpr. +// +// Orig holds the original ast.Expr from which this IndexExpr was derived. +type tpIndexExpr struct { + Orig ast.Expr // the wrapped expr, which may be distinct from the IndexListExpr below. + *ast.IndexListExpr +} + +func tpUnpackIndexExpr(n ast.Node) *tpIndexExpr { + switch e := n.(type) { + case *ast.IndexExpr: + return &tpIndexExpr{e, &ast.IndexListExpr{ + X: e.X, + Lbrack: e.Lbrack, + Indices: []ast.Expr{e.Index}, + Rbrack: e.Rbrack, + }} + case *ast.IndexListExpr: + return &tpIndexExpr{e, e} + } + return nil +} diff --git a/misc/genstd/genstd.go b/misc/genstd/genstd.go new file mode 100644 index 00000000000..318a63e5ee8 --- /dev/null +++ b/misc/genstd/genstd.go @@ -0,0 +1,210 @@ +// Command genstd provides static code generation for standard library native +// bindings. +package main + +import ( + "fmt" + "go/ast" + "go/parser" + "go/token" + "io/fs" + "os" + "path/filepath" + "strconv" + "strings" + "text/template" + + _ "embed" +) + +func main() { + path := "." + if len(os.Args) > 1 { + path = os.Args[1] + } + if err := _main(path); err != nil { + fmt.Fprintf(os.Stderr, "%+v\n", err) + os.Exit(1) + } +} + +func _main(stdlibsPath string) error { + stdlibsPath = filepath.Clean(stdlibsPath) + if s, err := os.Stat(stdlibsPath); err != nil { + return err + } else if !s.IsDir() { + return fmt.Errorf("not a directory: %q", stdlibsPath) + } + + // Gather data about each package, getting functions of interest + // (gno bodyless + go exported). + pkgs, err := walkStdlibs(stdlibsPath) + if err != nil { + return err + } + + // Link up each Gno function with its matching Go function. + mappings := linkFunctions(pkgs) + + // Create generated file. + f, err := os.Create("native.go") + if err != nil { + return fmt.Errorf("create native.go: %w", err) + } + defer f.Close() + + // Execute template. + td := &tplData{ + Mappings: mappings, + } + if err := tpl.Execute(f, td); err != nil { + return fmt.Errorf("execute template: %w", err) + } + if err := f.Close(); err != nil { + return err + } + + // gofumpt doesn't do "import fixing" like goimports: + // https://github.com/mvdan/gofumpt#frequently-asked-questions + if err := runTool("golang.org/x/tools/cmd/goimports"); err != nil { + return err + } + return runTool("mvdan.cc/gofumpt") +} + +type pkgData struct { + importPath string + fsDir string + gnoBodyless []funcDecl + goExported []funcDecl +} + +type funcDecl struct { + *ast.FuncDecl + imports []*ast.ImportSpec +} + +func addImports(fds []*ast.FuncDecl, imports []*ast.ImportSpec) []funcDecl { + r := make([]funcDecl, len(fds)) + for i, fd := range fds { + r[i] = funcDecl{fd, imports} + } + return r +} + +// walkStdlibs does a BFS walk through the given directory, expected to be a +// "stdlib" directory, parsing and keeping track of Go and Gno functions of +// interest. +func walkStdlibs(stdlibsPath string) ([]*pkgData, error) { + pkgs := make([]*pkgData, 0, 64) + err := filepath.WalkDir(stdlibsPath, func(fpath string, d fs.DirEntry, err error) error { + // skip dirs and top-level directory. + if d.IsDir() || filepath.Dir(fpath) == stdlibsPath { + return nil + } + + // skip non-source and test files. + ext := filepath.Ext(fpath) + noExt := fpath[:len(fpath)-len(ext)] + if (ext != ".go" && ext != ".gno") || strings.HasSuffix(noExt, "_test") { + return nil + } + + dir := filepath.Dir(fpath) + var pkg *pkgData + // because of bfs, we know that if we've already been in this directory + // in a previous file, it must be in the last entry of pkgs. + if len(pkgs) == 0 || pkgs[len(pkgs)-1].fsDir != dir { + pkg = &pkgData{ + importPath: strings.ReplaceAll(strings.TrimPrefix(dir, stdlibsPath+"/"), string(filepath.Separator), "/"), + fsDir: dir, + } + pkgs = append(pkgs, pkg) + } else { + pkg = pkgs[len(pkgs)-1] + } + fs := token.NewFileSet() + f, err := parser.ParseFile(fs, fpath, nil, parser.SkipObjectResolution) + if err != nil { + return err + } + if ext == ".go" { + // keep track of exported function declarations. + // warn about all exported type, const and var declarations. + if exp := filterExported(f); len(exp) > 0 { + pkg.goExported = append(pkg.goExported, addImports(exp, f.Imports)...) + } + } else if bd := filterBodylessFuncDecls(f); len(bd) > 0 { + // gno file -- keep track of function declarations without body. + pkg.gnoBodyless = append(pkg.gnoBodyless, addImports(bd, f.Imports)...) + } + return nil + }) + return pkgs, err +} + +// filterBodylessFuncDecls returns the function declarations in the given file +// which don't contain a body. +func filterBodylessFuncDecls(f *ast.File) (bodyless []*ast.FuncDecl) { + for _, decl := range f.Decls { + fd, ok := decl.(*ast.FuncDecl) + if !ok || fd.Body != nil { + continue + } + bodyless = append(bodyless, fd) + } + return +} + +// filterExported returns the exported function declarations of the given file. +func filterExported(f *ast.File) (exported []*ast.FuncDecl) { + for _, decl := range f.Decls { + switch d := decl.(type) { + case *ast.GenDecl: + // TODO: complain if there are exported types/vars/consts + continue + case *ast.FuncDecl: + if d.Name.IsExported() { + exported = append(exported, d) + } + } + } + return +} + +//go:embed template.tmpl +var templateText string + +var tpl = template.Must(template.New("").Parse(templateText)) + +// tplData is the data passed to the template. +type tplData struct { + Mappings []mapping +} + +type tplImport struct{ Name, Path string } + +func (t tplData) Imports() (res []tplImport) { + add := func(path string) { + for _, v := range res { + if v.Path == path { + return + } + } + res = append(res, tplImport{Name: pkgNameFromPath(path), Path: path}) + } + for _, m := range t.Mappings { + add(m.GoImportPath) + // There might be a bit more than we need - but we run goimports to fix that. + for _, v := range m.goImports { + s, err := strconv.Unquote(v.Path.Value) + if err != nil { + panic(fmt.Errorf("could not unquote go import string literal: %s", v.Path.Value)) + } + add(s) + } + } + return +} + +func (tplData) PkgName(path string) string { return pkgNameFromPath(path) } diff --git a/misc/genstd/mapping.go b/misc/genstd/mapping.go new file mode 100644 index 00000000000..0e4034a1ab9 --- /dev/null +++ b/misc/genstd/mapping.go @@ -0,0 +1,471 @@ +package main + +import ( + "errors" + "fmt" + "go/ast" + "path" + "strconv" +) + +const gnoPackagePath = "github.com/gnolang/gno/gnovm/pkg/gnolang" + +type mapping struct { + GnoImportPath string // time + GnoFunc string // now + GoImportPath string // github.com/gnolang/gno/gnovm/stdlibs/time + GoFunc string // X_now + Params []mappingType + Results []mappingType + MachineParam bool + + gnoImports []*ast.ImportSpec + goImports []*ast.ImportSpec +} + +type mappingType struct { + // type of ast.Expr is from the normal ast.Expr types + // + *linkedIdent. + Type ast.Expr + + // IsTypedValue is set to true if the parameter or result in go is of type + // gno.TypedValue. This prevents the generated code from performing + // Go2Gno/Gno2Go reflection-based conversion. + IsTypedValue bool +} + +func (mt mappingType) GoQualifiedName() string { + return (&exprPrinter{ + mode: printerModeGoQualified, + }).ExprString(mt.Type) +} + +func (mt mappingType) GnoType() string { + return (&exprPrinter{ + mode: printerModeGnoType, + }).ExprString(mt.Type) +} + +type linkedIdent struct { + ast.BadExpr // Unused, but it makes *linkedIdent implement ast.Expr + + lt linkedType +} + +func linkFunctions(pkgs []*pkgData) []mapping { + var mappings []mapping + for _, pkg := range pkgs { + for _, gb := range pkg.gnoBodyless { + nameWant := gb.Name.Name + if !gb.Name.IsExported() { + nameWant = "X_" + nameWant + } + fn := findFuncByName(pkg.goExported, nameWant) + if fn.FuncDecl == nil { + panic( + fmt.Errorf("package %q: no matching go function declaration (%q) exists for function %q", + pkg.importPath, nameWant, gb.Name.Name), + ) + } + mp := mapping{ + GnoImportPath: pkg.importPath, + GnoFunc: gb.Name.Name, + GoImportPath: "github.com/gnolang/gno/" + relPath() + "/" + pkg.importPath, + GoFunc: fn.Name.Name, + + gnoImports: gb.imports, + goImports: fn.imports, + } + if !mp.signaturesMatch(gb, fn) { + panic( + fmt.Errorf("package %q: signature of gno function %s doesn't match signature of go function %s", + pkg.importPath, gb.Name.Name, fn.Name.Name), + ) + } + mp.loadParamsResults(gb, fn) + mappings = append(mappings, mp) + } + } + return mappings +} + +func findFuncByName(fns []funcDecl, name string) funcDecl { + for _, fn := range fns { + if fn.Name.Name == name { + return fn + } + } + return funcDecl{} +} + +func (m *mapping) loadParamsResults(gnof, gof funcDecl) { + // initialise with lengths + m.Params = make([]mappingType, 0, gnof.Type.Params.NumFields()) + m.Results = make([]mappingType, 0, gnof.Type.Results.NumFields()) + + gofpl := gof.Type.Params.List + if m.MachineParam { + // skip machine parameter + gofpl = gofpl[1:] + } + if gnof.Type.Params != nil { + m._loadParamsResults(&m.Params, gnof.Type.Params.List, gofpl) + } + if gnof.Type.Results != nil { + m._loadParamsResults(&m.Results, gnof.Type.Results.List, gof.Type.Results.List) + } +} + +func (m *mapping) _loadParamsResults(dst *[]mappingType, gnol, gol []*ast.Field) { + iterFields(gnol, gol, func(gnoe, goe ast.Expr) error { + if m.isTypedValue(goe) { + *dst = append(*dst, mappingType{Type: gnoe, IsTypedValue: true}) + } else { + merged := m.mergeTypes(gnoe, goe) + *dst = append(*dst, mappingType{Type: merged}) + } + return nil + }) +} + +// isGnoMachine checks whether field is of type *gno.Machine, +// and it has at most 1 name. +func (m *mapping) isGnoMachine(field *ast.Field) bool { + if len(field.Names) > 1 { + return false + } + + return m.isGnoType(field.Type, true, "Machine") +} + +// isTypedValue checks whether e is type gno.TypedValue. +func (m *mapping) isTypedValue(e ast.Expr) bool { + return m.isGnoType(e, false, "TypedValue") +} + +func (m *mapping) isGnoType(e ast.Expr, star bool, typeName string) bool { + if star { + px, ok := e.(*ast.StarExpr) + if !ok { + return false + } + e = px.X + } + + sx, ok := e.(*ast.SelectorExpr) + if !ok { + return false + } + + imp := resolveSelectorImport(m.goImports, sx) + return imp == gnoPackagePath && sx.Sel.Name == typeName +} + +// iterFields iterates over gnol and gol, calling callback for each matching +// parameter. iterFields assumes the caller already checked for the "true" number +// of parameters in the two arrays to be equal (can be checked using +// (*ast.FieldList).NumFields()). +// +// If callback returns an error, iterFields returns that error immediately. +// No errors are otherwise generated. +func iterFields(gnol, gol []*ast.Field, callback func(gnoType, goType ast.Expr) error) error { + var goIdx, goNameIdx int + + for _, l := range gnol { + n := len(l.Names) + if n == 0 { + n = 1 + } + gnoe := l.Type + for i := 0; i < n; i++ { + goe := gol[goIdx].Type + + if err := callback(gnoe, goe); err != nil { + return err + } + + goNameIdx++ + if goNameIdx >= len(gol[goIdx].Names) { + goIdx++ + goNameIdx = 0 + } + } + } + return nil +} + +// mergeTypes merges gnoe and goe into a single ast.Expr. +// +// gnoe and goe are expected to have the same underlying structure, but they +// may differ in their type identifiers (possibly qualified, ie pkg.T). +// if they differ, mergeTypes returns nil. +// +// When two type identifiers are found, they are checked against the list of +// linkedTypes to determine if they refer to a linkedType. If they are not, +// mergeTypes returns nil. If they are, the *ast.Ident/*ast.SelectorExpr is +// replaced with a *linkedIdent. +// +// mergeTypes does not modify the given gnoe or goe; the returned ast.Expr is +// (recursively) newly allocated. +func (m *mapping) mergeTypes(gnoe, goe ast.Expr) ast.Expr { + resolveGoNamed := func(lt *linkedType) bool { + switch goe := goe.(type) { + case *ast.SelectorExpr: + // selector - resolve pkg ident to path + lt.goPackage = resolveSelectorImport(m.goImports, goe) + lt.goName = goe.Sel.Name + case *ast.Ident: + // local name -- use import path of go pkg + lt.goPackage = m.GoImportPath + lt.goName = goe.Name + default: + return false + } + return true + } + + switch gnoe := gnoe.(type) { + // We're working with a subset of all expressions: + // https://go.dev/ref/spec#Type + + case *ast.SelectorExpr: + lt := linkedType{ + gnoPackage: resolveSelectorImport(m.gnoImports, gnoe), + gnoName: gnoe.Sel.Name, + } + if !resolveGoNamed(<) || !linkedTypeExists(lt) { + return nil + } + return &linkedIdent{lt: lt} + case *ast.Ident: + // easy case - built-in identifiers + goi, ok := goe.(*ast.Ident) + if ok && isBuiltin(gnoe.Name) && gnoe.Name == goi.Name { + return &ast.Ident{Name: gnoe.Name} + } + + lt := linkedType{ + gnoPackage: m.GnoImportPath, + gnoName: gnoe.Name, + } + if !resolveGoNamed(<) || !linkedTypeExists(lt) { + return nil + } + return &linkedIdent{lt: lt} + + // easier cases -- check for equality of structure and underlying types + case *ast.StarExpr: + goe, ok := goe.(*ast.StarExpr) + if !ok { + return nil + } + x := m.mergeTypes(gnoe.X, goe.X) + if x == nil { + return nil + } + return &ast.StarExpr{X: x} + case *ast.ArrayType: + goe, ok := goe.(*ast.ArrayType) + if !ok || !basicLitsEqual(gnoe.Len, goe.Len) { + return nil + } + elt := m.mergeTypes(gnoe.Elt, goe.Elt) + if elt == nil { + return nil + } + var l ast.Expr + if gnoe.Len != nil { + l = &ast.BasicLit{Value: gnoe.Len.(*ast.BasicLit).Value} + } + return &ast.ArrayType{Len: l, Elt: elt} + + case *ast.StructType, + *ast.FuncType, + *ast.InterfaceType, + *ast.MapType, + *ast.Ellipsis: + // TODO + panic("not implemented") + default: + panic(fmt.Errorf("invalid expression as func param/return type: %T (%v)", gnoe, gnoe)) + } +} + +// returns full import path from package ident +func resolveImport(imports []*ast.ImportSpec, ident string) string { + for _, i := range imports { + s, err := strconv.Unquote(i.Path.Value) + if err != nil { + panic(fmt.Errorf("could not unquote import path literal: %s", i.Path.Value)) + } + + // TODO: for simplicity, if i.Name is nil we assume the name to be == + // to the last part of the import path. + // ideally, use importer to resolve package directory on user's FS and + // resolve by parsing and reading package clause + var name string + if i.Name != nil { + name = i.Name.Name + } else { + name = path.Base(s) + } + + if name == ident { + return s + } + } + return "" +} + +func resolveSelectorImport(imports []*ast.ImportSpec, sx *ast.SelectorExpr) string { + pkgIdent, ok := sx.X.(*ast.Ident) + if !ok { + panic(fmt.Errorf("encountered unhandled SelectorExpr.X type: %T (%v)", sx.X, sx)) + } + impPath := resolveImport(imports, pkgIdent.Name) + if impPath == "" { + panic(fmt.Errorf( + "unknown identifier %q (for resolving type %q)", + pkgIdent.Name, pkgIdent.Name+"."+sx.Sel.Name, + )) + } + return impPath +} + +// simple equivalence between two BasicLits. +// Note that this returns true only if the expressions are exactly the same; +// ie. 16 != 0x10, only 16 == 16. +func basicLitsEqual(x1, x2 ast.Expr) bool { + if x1 == nil || x2 == nil { + return x1 == nil && x2 == nil + } + l1, ok1 := x1.(*ast.BasicLit) + l2, ok2 := x2.(*ast.BasicLit) + if !ok1 || !ok2 { + return false + } + return l1.Value == l2.Value +} + +// Signatures match when they accept the same elementary types, or a linked +// type mapping (see [linkedTypes]). +// +// Additionally, if the first parameter to the Go function is +// *[gnolang.Machine], it is ignored when matching to the Gno function. +func (m *mapping) signaturesMatch(gnof, gof funcDecl) bool { + if gnof.Type.TypeParams != nil || gof.Type.TypeParams != nil { + panic("type parameters not supported") + } + + // if first param of go function is *gno.Machine, remove it + gofp := gof.Type.Params + if gofp != nil && len(gofp.List) > 0 && m.isGnoMachine(gofp.List[0]) { + // avoid touching original struct + n := *gofp + n.List = n.List[1:] + gofp = &n + + m.MachineParam = true + } + + return m.fieldListsMatch(gnof.Type.Params, gofp) && + m.fieldListsMatch(gnof.Type.Results, gof.Type.Results) +} + +var errNoMatch = errors.New("no match") + +func (m *mapping) fieldListsMatch(gnofl, gofl *ast.FieldList) bool { + if gnofl == nil || gofl == nil { + return gnofl == nil && gofl == nil + } + if gnofl.NumFields() != gofl.NumFields() { + return false + } + err := iterFields(gnofl.List, gofl.List, func(gnoe, goe ast.Expr) error { + // if the go type is gno.TypedValue, we just don't perform reflect-based conversion. + if m.isTypedValue(goe) { + return nil + } + if m.mergeTypes(gnoe, goe) == nil { + return errNoMatch + } + return nil + }) + return err == nil +} + +// TODO: this is created based on the uverse definitions. This should be +// centralized, or at least have a CI/make check to make sure this stays the +// same +var builtinTypes = [...]string{ + "bool", + "string", + "int", + "int8", + "int16", + "rune", + "int32", + "int64", + "uint", + "byte", + "uint8", + "uint16", + "uint32", + "uint64", + "bigint", + "float32", + "float64", + "error", +} + +func isBuiltin(name string) bool { + for _, x := range builtinTypes { + if x == name { + return true + } + } + return false +} + +type linkedType struct { + gnoPackage string + gnoName string + goPackage string + goName string +} + +var linkedTypes = [...]linkedType{ + { + "std", "Address", + "github.com/gnolang/gno/tm2/pkg/crypto", "Bech32Address", + }, + { + "std", "Coin", + "github.com/gnolang/gno/tm2/pkg/std", "Coin", + }, + { + "std", "Coins", + "github.com/gnolang/gno/tm2/pkg/std", "Coins", + }, + { + "std", "Realm", + "github.com/gnolang/gno/gnovm/stdlibs/std", "Realm", + }, + { + "std", "BankerType", + "github.com/gnolang/gno/gnovm/stdlibs/std", "BankerType", + }, + { + "std", "Banker", + "github.com/gnolang/gno/gnovm/stdlibs/std", "Banker", + }, +} + +func linkedTypeExists(lt linkedType) bool { + for _, ltx := range linkedTypes { + if lt == ltx { + return true + } + } + return false +} diff --git a/misc/genstd/mapping_test.go b/misc/genstd/mapping_test.go new file mode 100644 index 00000000000..b0cfa1bd4a7 --- /dev/null +++ b/misc/genstd/mapping_test.go @@ -0,0 +1,294 @@ +package main + +import ( + "fmt" + "go/ast" + "go/parser" + "os" + "path/filepath" + "sync" + "testing" + + "github.com/jaekwon/testify/assert" + "github.com/jaekwon/testify/require" +) + +const testdataDir = "github.com/gnolang/gno/misc/genstd/testdata/" + +var initWD = func() string { + d, err := os.Getwd() + if err != nil { + panic(err) + } + return d +}() + +func chdir(t *testing.T, s string) { + t.Helper() + + os.Chdir(filepath.Join(initWD, s)) + t.Cleanup(func() { + os.Chdir(initWD) + dirsOnce = sync.Once{} + memoGitRoot, memoRelPath = "", "" + }) +} + +func Test_linkFunctions(t *testing.T) { + chdir(t, "testdata/linkFunctions") + + pkgs, err := walkStdlibs(".") + require.NoError(t, err) + + mappings := linkFunctions(pkgs) + require.Len(t, mappings, 8) + + const ( + ret = 1 << iota + param + machine + ) + str := func(i int) string { + s := "Fn" + if i&machine != 0 { + s += "Machine" + } + if i¶m != 0 { + s += "Param" + } + if i&ret != 0 { + s += "Ret" + } + return s + } + + for i, v := range mappings { + exp := str(i) + assert.Equal(t, v.GnoFunc, exp) + assert.Equal(t, v.GoFunc, exp) + assert.Equal(t, v.GnoImportPath, "std") + assert.Equal(t, v.GoImportPath, testdataDir+"linkFunctions/std") + + assert.Equal(t, v.MachineParam, i&machine != 0, "MachineParam should match expected value") + if i¶m != 0 { + // require, otherwise the following would panic + require.Len(t, v.Params, 1) + p := v.Params[0] + assert.Equal(t, p.GnoType(), "int") + assert.Equal(t, p.GoQualifiedName(), "int") + assert.False(t, p.IsTypedValue) + } else { + assert.Len(t, v.Params, 0) + } + if i&ret != 0 { + // require, otherwise the following would panic + require.Len(t, v.Results, 1) + p := v.Results[0] + assert.Equal(t, p.GnoType(), "int") + assert.Equal(t, p.GoQualifiedName(), "int") + assert.False(t, p.IsTypedValue) + } else { + assert.Len(t, v.Results, 0) + } + } +} + +func Test_linkFunctions_unexp(t *testing.T) { + chdir(t, "testdata/linkFunctions_unexp") + + pkgs, err := walkStdlibs(".") + require.NoError(t, err) + + mappings := linkFunctions(pkgs) + require.Len(t, mappings, 2) + + assert.Equal(t, mappings[0].MachineParam, false) + assert.Equal(t, mappings[0].GnoFunc, "t1") + assert.Equal(t, mappings[0].GoFunc, "X_t1") + + assert.Equal(t, mappings[1].MachineParam, true) + assert.Equal(t, mappings[1].GnoFunc, "t2") + assert.Equal(t, mappings[1].GoFunc, "X_t2") +} + +func Test_linkFunctions_TypedValue(t *testing.T) { + chdir(t, "testdata/linkFunctions_TypedValue") + + pkgs, err := walkStdlibs(".") + require.NoError(t, err) + + mappings := linkFunctions(pkgs) + require.Len(t, mappings, 3) + + assert.Equal(t, mappings[0].MachineParam, false) + assert.Equal(t, mappings[0].GnoFunc, "TVParam") + assert.Equal(t, mappings[0].GoFunc, "TVParam") + assert.Len(t, mappings[0].Results, 0) + _ = assert.Len(t, mappings[0].Params, 1) && + assert.Equal(t, mappings[0].Params[0].IsTypedValue, true) && + assert.Equal(t, mappings[0].Params[0].GnoType(), "struct{m1 map[string]interface{}}") + + assert.Equal(t, mappings[1].MachineParam, false) + assert.Equal(t, mappings[1].GnoFunc, "TVResult") + assert.Equal(t, mappings[1].GoFunc, "TVResult") + assert.Len(t, mappings[1].Params, 0) + _ = assert.Len(t, mappings[1].Results, 1) && + assert.Equal(t, mappings[1].Results[0].IsTypedValue, true) && + assert.Equal(t, mappings[1].Results[0].GnoType(), "interface{S() map[int]Banker}") + + assert.Equal(t, mappings[2].MachineParam, true) + assert.Equal(t, mappings[2].GnoFunc, "TVFull") + assert.Equal(t, mappings[2].GoFunc, "TVFull") + assert.Len(t, mappings[2].Params, 1) + assert.Len(t, mappings[2].Results, 1) +} + +func Test_linkFunctions_noMatch(t *testing.T) { + chdir(t, "testdata/linkFunctions_noMatch") + + pkgs, err := walkStdlibs(".") + require.NoError(t, err) + + defer func() { + r := recover() + assert.NotNil(t, r) + assert.Contains(t, fmt.Sprint(r), "no matching go function declaration") + }() + + linkFunctions(pkgs) +} + +func Test_linkFunctions_noMatchSig(t *testing.T) { + chdir(t, "testdata/linkFunctions_noMatchSig") + + pkgs, err := walkStdlibs(".") + require.NoError(t, err) + + defer func() { + r := recover() + assert.NotNil(t, r) + assert.Contains(t, fmt.Sprint(r), "doesn't match signature of go function") + }() + + linkFunctions(pkgs) +} + +// mergeTypes - separate tests. + +var mergeTypesMapping = &mapping{ + GnoImportPath: "std", + GnoFunc: "Fn", + GoImportPath: "github.com/gnolang/gno/gnovm/stdlibs/std", + GoFunc: "Fn", + goImports: []*ast.ImportSpec{ + { + Name: &ast.Ident{Name: "gno"}, + Path: &ast.BasicLit{Value: `"github.com/gnolang/gno/gnovm/pkg/gnolang"`}, + }, + { + Path: &ast.BasicLit{Value: `"github.com/gnolang/gno/tm2/pkg/crypto"`}, + }, + }, + gnoImports: []*ast.ImportSpec{ + { + // cheating a bit -- but we currently only have linked types in `std`. + Path: &ast.BasicLit{Value: `"std"`}, + }, + { + Path: &ast.BasicLit{Value: `"math"`}, + }, + }, +} + +func Test_mergeTypes(t *testing.T) { + tt := []struct { + gnoe, goe string + result ast.Expr + }{ + {"int", "int", &ast.Ident{Name: "int"}}, + {"*[11][]rune", "*[11][]rune", &ast.StarExpr{ + X: &ast.ArrayType{Len: &ast.BasicLit{Value: "11"}, Elt: &ast.ArrayType{ + Elt: &ast.Ident{Name: "rune"}, + }}, + }}, + + {"Address", "crypto.Bech32Address", &linkedIdent{lt: linkedType{ + gnoPackage: "std", + gnoName: "Address", + goPackage: "github.com/gnolang/gno/tm2/pkg/crypto", + goName: "Bech32Address", + }}}, + {"std.Realm", "Realm", &linkedIdent{lt: linkedType{ + gnoPackage: "std", + gnoName: "Realm", + goPackage: "github.com/gnolang/gno/gnovm/stdlibs/std", + goName: "Realm", + }}}, + } + + for _, tv := range tt { + t.Run(tv.gnoe, func(t *testing.T) { + gnoe, err := parser.ParseExpr(tv.gnoe) + require.NoError(t, err) + goe, err := parser.ParseExpr(tv.goe) + require.NoError(t, err) + + result := mergeTypesMapping.mergeTypes(gnoe, goe) + assert.Equal(t, result, tv.result) + }) + } +} + +func Test_mergeTypes_invalid(t *testing.T) { + tt := []struct { + gnoe, goe string + panic string + }{ + {"int", "string", ""}, + + {"*int", "int", ""}, + {"string", "*string", ""}, + {"*string", "*int", ""}, + + {"[]int", "[1]int", ""}, + {"[1]int", "[]int", ""}, + {"[2]int", "[2]string", ""}, + // valid, but unsupported (only BasicLits) + {"[(11)]int", "[(11)]string", ""}, + + {"Address", "string", ""}, + {"math.X", "X", ""}, + + {"map[string]string", "map[string]string", "not implemented"}, + {"func(s string)", "func(s string)", "not implemented"}, + {"interface{}", "interface{}", "not implemented"}, + {"struct{}", "struct{}", "not implemented"}, + + {"1 + 2", "1 + 2", "invalid expression"}, + + // even though semantically equal, for simplicity we don't implement + // "true" basic lit equivalence + {"[8]int", "[0x8]int", ""}, + } + + for _, tv := range tt { + t.Run(tv.gnoe, func(t *testing.T) { + gnoe, err := parser.ParseExpr(tv.gnoe) + require.NoError(t, err) + goe, err := parser.ParseExpr(tv.goe) + require.NoError(t, err) + + defer func() { + r := recover() + if tv.panic == "" { + assert.Nil(t, r) + } else { + assert.Contains(t, fmt.Sprint(r), tv.panic) + } + }() + + result := mergeTypesMapping.mergeTypes(gnoe, goe) + assert.Nil(t, result) + }) + } +} diff --git a/misc/genstd/template.tmpl b/misc/genstd/template.tmpl new file mode 100644 index 00000000000..f2cad0a851b --- /dev/null +++ b/misc/genstd/template.tmpl @@ -0,0 +1,88 @@ +// This file is autogenerated from the genstd tool (@/misc/genstd); do not edit. +// To regenerate it, run `go generate` from this directory. + +package stdlibs + +import ( + "reflect" + + gno "github.com/gnolang/gno/gnovm/pkg/gnolang" +{{- range .Imports }} + {{ .Name }} {{ printf "%q" .Path }} +{{- end }} +) + +type nativeFunc struct { + gnoPkg string + gnoFunc gno.Name + params []gno.FieldTypeExpr + results []gno.FieldTypeExpr + f func(m *gno.Machine) +} + +var nativeFuncs = [...]nativeFunc{ +{{- range $i, $m := .Mappings }} + { + {{ printf "%q" $m.GnoImportPath }}, + {{ printf "%q" $m.GnoFunc }}, + {{- /* TODO: set nil if empty */}} + []gno.FieldTypeExpr{ + {{- range $i, $p := $m.Params }} + {Name: gno.N("p{{ $i }}"), Type: gno.X({{ printf "%q" $p.GnoType }})}, + {{- end }} + }, + []gno.FieldTypeExpr{ + {{- range $i, $r := $m.Results }} + {Name: gno.N("r{{ $i }}"), Type: gno.X({{ printf "%q" $r.GnoType }})}, + {{- end }} + }, + func(m *gno.Machine) { + {{ if $m.Params -}} + b := m.LastBlock() + var ( + {{- range $pn, $pv := $m.Params -}} + {{- if $pv.IsTypedValue }} + p{{ $pn }} = gno.NewValuePathBlock(1, {{ $pn }}, "")).TV + {{- else }} + p{{ $pn }} {{ $pv.GoQualifiedName }} + rp{{ $pn }} = reflect.ValueOf(&p{{ $pn }}).Elem() + {{- end }} + {{- end }} + ) + + {{ range $pn, $pv := $m.Params -}} + {{- if not $pv.IsTypedValue }} + gno.Gno2GoValue(b.GetPointerTo(nil, gno.NewValuePathBlock(1, {{ $pn }}, "")).TV, rp{{ $pn }}) + {{- end -}} + {{ end }} + {{- end }} + + {{ range $rn, $rv := $m.Results -}} + {{- if gt $rn 0 -}}, {{ end -}} + r{{ $rn }} + {{- end -}} + {{- if $m.Results }} := {{ end -}} + {{ $.PkgName $m.GoImportPath }}.{{ $m.GoFunc }}( + {{- if $m.MachineParam }} + m, + {{ end -}} + {{- range $pn, $pv := $m.Params -}} + p{{ $pn }}, + {{- end -}} + ) + + {{ range $rn, $rv := $m.Results -}} + {{- if $rv.IsTypedValue }} + m.PushValue(r{{ $rn }}) + {{- else }} + m.PushValue(gno.Go2GnoValue( + m.Alloc, + m.Store, + reflect.ValueOf(&r{{ $rn }}).Elem(), {{- /* necessary to support interfaces (ie. error) */}} + )) + {{- end }} + {{- end }} + }, + }, +{{- end }} +} diff --git a/misc/genstd/testdata/linkFunctions/std/std.gno b/misc/genstd/testdata/linkFunctions/std/std.gno new file mode 100644 index 00000000000..ab04b4084ba --- /dev/null +++ b/misc/genstd/testdata/linkFunctions/std/std.gno @@ -0,0 +1,23 @@ +package std + +func Fn() + +func FnRet() int + +func FnParam(n int) + +func FnParamRet(n int) int + +func FnMachine() + +func FnMachineRet() int + +func FnMachineParam(n int) + +func FnMachineParamRet(n int) int + +func Ignored() int { + // Ignored even if it has a matching go definition - + // as this one has a body. + return 1 +} diff --git a/misc/genstd/testdata/linkFunctions/std/std.go b/misc/genstd/testdata/linkFunctions/std/std.go new file mode 100644 index 00000000000..1b7a791c6cc --- /dev/null +++ b/misc/genstd/testdata/linkFunctions/std/std.go @@ -0,0 +1,47 @@ +package std + +import ( + gno "github.com/gnolang/gno/gnovm/pkg/gnolang" +) + +func Fn() { + println("call Fn") +} + +func FnRet() int { + println("call FnRet") + return 1 +} + +func FnParam(n int) { + println("call FnParam", n) +} + +func FnParamRet(n int) int { + println("call FnParamRet", n) + return 1 +} + +func FnMachine(m *gno.Machine) { + println("call FnMachine") +} + +func FnMachineRet(m *gno.Machine) int { + println("call FnMachineRet") + return 1 +} + +func FnMachineParam(m *gno.Machine, n int) { + println("call FnMachineParam", n) +} + +func FnMachineParamRet(m *gno.Machine, n int) int { + println("call FnMachineParamRet", n) + return 1 +} + +func Ignored() int { + // Ignored even if it has a matching go definition - + // as gno's has a body. + return 1 +} diff --git a/misc/genstd/testdata/linkFunctions_TypedValue/std/std.gno b/misc/genstd/testdata/linkFunctions_TypedValue/std/std.gno new file mode 100644 index 00000000000..3bba36774e3 --- /dev/null +++ b/misc/genstd/testdata/linkFunctions_TypedValue/std/std.gno @@ -0,0 +1,11 @@ +package std + +type Banker interface { + B() +} + +func TVParam(m struct{ m1 map[string]interface{} }) + +func TVResult() interface{ S() map[int]Banker } + +func TVFull(map[Banker]map[string]interface{}) (n [86]map[string]bool) diff --git a/misc/genstd/testdata/linkFunctions_TypedValue/std/std.go b/misc/genstd/testdata/linkFunctions_TypedValue/std/std.go new file mode 100644 index 00000000000..03d95721438 --- /dev/null +++ b/misc/genstd/testdata/linkFunctions_TypedValue/std/std.go @@ -0,0 +1,16 @@ +package std + +import ( + gno "github.com/gnolang/gno/gnovm/pkg/gnolang" +) + +func TVParam(p gno.TypedValue) { +} + +func TVResult() gno.TypedValue { + return gno.TypedValue{} +} + +func TVFull(m *gno.Machine, v gno.TypedValue) gno.TypedValue { + return gno.TypedValue{} +} diff --git a/misc/genstd/testdata/linkFunctions_noMatch/std/std.gno b/misc/genstd/testdata/linkFunctions_noMatch/std/std.gno new file mode 100644 index 00000000000..2ef4be8abc6 --- /dev/null +++ b/misc/genstd/testdata/linkFunctions_noMatch/std/std.gno @@ -0,0 +1,3 @@ +package std + +func X() int diff --git a/misc/genstd/testdata/linkFunctions_noMatch/std/std.go b/misc/genstd/testdata/linkFunctions_noMatch/std/std.go new file mode 100644 index 00000000000..97399743533 --- /dev/null +++ b/misc/genstd/testdata/linkFunctions_noMatch/std/std.go @@ -0,0 +1,3 @@ +package std + +func Y() {} diff --git a/misc/genstd/testdata/linkFunctions_noMatchSig/std/std.gno b/misc/genstd/testdata/linkFunctions_noMatchSig/std/std.gno new file mode 100644 index 00000000000..75e8e10e2e3 --- /dev/null +++ b/misc/genstd/testdata/linkFunctions_noMatchSig/std/std.gno @@ -0,0 +1,3 @@ +package std + +func X(n int) int diff --git a/misc/genstd/testdata/linkFunctions_noMatchSig/std/std.go b/misc/genstd/testdata/linkFunctions_noMatchSig/std/std.go new file mode 100644 index 00000000000..7a5a0e5893b --- /dev/null +++ b/misc/genstd/testdata/linkFunctions_noMatchSig/std/std.go @@ -0,0 +1,8 @@ +package std + +import ( + gno "github.com/gnolang/gno/gnovm/pkg/gnolang" +) + +func X(m *gno.Machine, n string) { +} diff --git a/misc/genstd/testdata/linkFunctions_unexp/std/std.gno b/misc/genstd/testdata/linkFunctions_unexp/std/std.gno new file mode 100644 index 00000000000..c4811e5e837 --- /dev/null +++ b/misc/genstd/testdata/linkFunctions_unexp/std/std.gno @@ -0,0 +1,4 @@ +package std + +func t1() int +func t2() int diff --git a/misc/genstd/testdata/linkFunctions_unexp/std/std.go b/misc/genstd/testdata/linkFunctions_unexp/std/std.go new file mode 100644 index 00000000000..023b424e87c --- /dev/null +++ b/misc/genstd/testdata/linkFunctions_unexp/std/std.go @@ -0,0 +1,13 @@ +package std + +import ( + gno "github.com/gnolang/gno/gnovm/pkg/gnolang" +) + +func X_t1() int { + return 1 +} + +func X_t2(m *gno.Machine) int { + return m.NumOps +} diff --git a/misc/genstd/util.go b/misc/genstd/util.go new file mode 100644 index 00000000000..061a9604c67 --- /dev/null +++ b/misc/genstd/util.go @@ -0,0 +1,119 @@ +package main + +import ( + "errors" + "fmt" + "os" + "os/exec" + "path" + "path/filepath" + "strings" + "sync" +) + +func runTool(importPath string) error { + shortName := path.Base(importPath) + gr := gitRoot() + + cmd := exec.Command( + "go", "run", "-modfile", filepath.Join(gr, "misc/devdeps/go.mod"), + importPath, "-w", "native.go", + ) + _, err := cmd.Output() + if err != nil { + if err, ok := err.(*exec.ExitError); ok { + return fmt.Errorf("error executing %s: %w; output: %v", shortName, err, string(err.Stderr)) + } + return fmt.Errorf("error executing %s: %w", shortName, err) + } + return nil +} + +var ( + memoGitRoot string + memoRelPath string + + dirsOnce sync.Once +) + +func gitRoot() string { + dirsOnceDo() + return memoGitRoot +} + +func relPath() string { + dirsOnceDo() + return memoRelPath +} + +func dirsOnceDo() { + dirsOnce.Do(func() { + var err error + memoGitRoot, memoRelPath, err = findDirs() + if err != nil { + panic(fmt.Errorf("could not determine git root: %w", err)) + } + }) +} + +func findDirs() (gitRoot string, relPath string, err error) { + wd, err := os.Getwd() + if err != nil { + return + } + p := wd + for { + if s, e := os.Stat(filepath.Join(p, ".git")); e == nil && s.IsDir() { + // make relPath relative to the git root + rp := strings.TrimPrefix(wd, p+string(filepath.Separator)) + // normalize separator to / + rp = strings.ReplaceAll(rp, string(filepath.Separator), "/") + return p, rp, nil + } + + if strings.HasSuffix(p, string(filepath.Separator)) { + return "", "", errors.New("root git not found") + } + + p = filepath.Dir(p) + } +} + +// pkgNameFromPath derives the package name from the given path, +// unambiguously for the most part (so safe for the code generation). +// +// The path is taken and possibly shortened if it starts with a known prefix. +// For instance, github.com/gnolang/gno/stdlibs/std simply becomes "libs_std". +// "Unsafe" characters are removed (ie. invalid for go identifiers). +func pkgNameFromPath(path string) string { + const ( + repoPrefix = "github.com/gnolang/gno/" + vmPrefix = repoPrefix + "gnovm/" + tm2Prefix = repoPrefix + "tm2/pkg/" + libsPrefix = vmPrefix + "stdlibs/" + testlibsPrefix = vmPrefix + "tests/stdlibs/" + ) + + ns := "ext" + switch { + case strings.HasPrefix(path, testlibsPrefix): + ns, path = "testlibs", path[len(testlibsPrefix):] + case strings.HasPrefix(path, libsPrefix): + ns, path = "libs", path[len(libsPrefix):] + case strings.HasPrefix(path, vmPrefix): + ns, path = "vm", path[len(vmPrefix):] + case strings.HasPrefix(path, tm2Prefix): + ns, path = "tm2", path[len(tm2Prefix):] + case strings.HasPrefix(path, repoPrefix): + ns, path = "repo", path[len(repoPrefix):] + case !strings.Contains(path, "."): + ns = "go" + } + + flds := strings.FieldsFunc(path, func(r rune) bool { + return (r < 'a' || r > 'z') && + (r < 'A' || r > 'Z') && + (r < '0' || r > '9') + }) + return ns + "_" + strings.Join(flds, "_") +} diff --git a/misc/genstd/util_test.go b/misc/genstd/util_test.go new file mode 100644 index 00000000000..f6e804d545f --- /dev/null +++ b/misc/genstd/util_test.go @@ -0,0 +1,31 @@ +package main + +import ( + "fmt" + "testing" + + "github.com/jaekwon/testify/assert" +) + +func Test_pkgNameFromPath(t *testing.T) { + tt := []struct { + input, result string + }{ + {"math", "go_math"}, + {"crypto/sha256", "go_crypto_sha256"}, + {"github.com/import/path", "ext_github_com_import_path"}, + // consecutive unsupported characters => _ + {"kebab----------case", "go_kebab_case"}, + + {"github.com/gnolang/gno/misc/test", "repo_misc_test"}, + {"github.com/gnolang/gno/tm2/pkg/crypto", "tm2_crypto"}, + {"github.com/gnolang/gno/gnovm/test", "vm_test"}, + {"github.com/gnolang/gno/gnovm/stdlibs/std", "libs_std"}, + {"github.com/gnolang/gno/gnovm/tests/stdlibs/std", "testlibs_std"}, + } + for i, tv := range tt { + t.Run(fmt.Sprintf("n%d", i+1), func(t *testing.T) { + assert.Equal(t, pkgNameFromPath(tv.input), tv.result) + }) + } +} diff --git a/tm2/pkg/std/memfile.go b/tm2/pkg/std/memfile.go index 99b8061ea3b..c632d3026d0 100644 --- a/tm2/pkg/std/memfile.go +++ b/tm2/pkg/std/memfile.go @@ -13,11 +13,15 @@ type MemFile struct { Body string } +// MemPackage represents the information and files of a package which will be +// stored in memory. It will generally be initialized by package gnolang's +// ReadMemPackage. +// // NOTE: in the future, a MemPackage may represent // updates/additional-files for an existing package. type MemPackage struct { - Name string - Path string + Name string // package name as declared by `package` + Path string // import path Files []*MemFile }