Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: new realm /r/demo/bytebeat #840

Open
wants to merge 30 commits into
base: master
Choose a base branch
from

Conversation

schollz
Copy link
Contributor

@schollz schollz commented May 22, 2023

Please note - this PR is dependent on merging of #965.

bytebeat realm

The intention behind this realm is to provide a proof-of-concept for generating audio through smart contracts (for possible future use with web3 audio nfts or similar).

There are several packages added to accomplish this:

  • examples/gno.land/p/demo/audio/riff: generates riff headers for .wav files
  • examples/gno.land/p/demo/audio/wav: provides Writer interface for writing .wav files
  • examples/gno.land/p/demo/audio/biquad: implements basic 2nd-order digital biquad filter for doing low-pass and high-pass filtering
  • examples/gno.land/p/demo/audio/bytebeat: core library for generating 16-bit byte-beat audio that is post-processed with a DC offset filter and a high pass filter to eliminate aliasing

Demo

(turn audio on)

bytebeat-realm.mp4

Caveats

Currently, there is a limitation to the number of CPU cycles available to a contract and the realm is currently limited to generating only 2 seconds of audio at 8 kHz.

Usage

To use, add the packages for the audio library and then add the realm:

$ gnokey maketx addpkg --pkgpath "gno.land/p/demo/audio/biquad" --pkgdir "examples/gno.land/p/demo/audio/biquad" --deposit 100000000ugnot --gas-fee 1000000ugnot --gas-wanted 2000000 --broadcast --chainid dev --remote localhost:26657 <yourkey>
$ gnokey maketx addpkg --pkgpath "gno.land/p/demo/audio/riff" --pkgdir "examples/gno.land/p/demo/audio/riff" --deposit 100000000ugnot --gas-fee 1000000ugnot --gas-wanted 2000000 --broadcast --chainid dev --remote localhost:26657 <yourkey>
$ gnokey maketx addpkg --pkgpath "gno.land/p/demo/audio/wav" --pkgdir "examples/gno.land/p/demo/audio/wav" --deposit 100000000ugnot --gas-fee 1000000ugnot --gas-wanted 2000000 --broadcast --chainid dev --remote localhost:26657 <yourkey>
$ gnokey maketx addpkg --pkgpath "gno.land/p/demo/audio/bytebeat" --pkgdir "examples/gno.land/p/demo/audio/bytebeat" --deposit 100000000ugnot --gas-fee 1000000ugnot --gas-wanted 8000000 --broadcast --chainid dev --remote localhost:26657  <yourkey>
$ gnokey maketx addpkg --pkgpath "gno.land/r/demo/bytebeat" --pkgdir "examples/gno.land/r/demo/bytebeat" --deposit 100000000ugnot --gas-fee 1000000ugnot --gas-wanted 3000000 --broadcast --chainid dev --remote localhost:26657  <yourkey>

Adding the gno.land/p/demo/audio/bytebeat requires adding more to --gas-wanted (at least 8000000).

@schollz schollz requested a review from a team as a code owner May 22, 2023 21:57
@github-actions github-actions bot added the 🧾 package/realm Tag used for new Realms or Packages. label May 22, 2023
@schollz
Copy link
Contributor Author

schollz commented May 22, 2023

(CI will fail until #805 and #816 are merged).

Copy link
Member

@thehowl thehowl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You'll probably need to add some corresponding gno.mod files, seeing as we restructured the examples directory to have them.

Lots of comments; mostly formatting and doc requests :) I know you added the README, which I really appreciate, but I think some code comments on usage and the function parameters can be really useful to other potential users.

Sorry it took long to review this, going through a long backlog. Hopefully I'll also get to the haiku package soon enough. Thank you for your work! 🎉

@@ -0,0 +1,57 @@
package biquad
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pkgdoc with some references to what biquad implements, for the ignorant reader (such as me?)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added in 3536794! Sorry I should've added more info, basically biquad is an implementation of basic digital signal processing tools (filters).

x1_f, x2_f, y1_f, y2_f float64
}

func New(fc float64, fs float64, q float64, db float64, filter_type string) (f *Filter) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
func New(fc float64, fs float64, q float64, db float64, filter_type string) (f *Filter) {
func New(fc float64, fs float64, q float64, db float64, filterType string) *Filter {

(we don't used the named return parameter)

also add some doc, especially on what each parameter is?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, didn't know about the return convention, will follow here on out. Update is here: 6939ecd

}

func New(fc float64, fs float64, q float64, db float64, filter_type string) (f *Filter) {
w0 := 2 * 3.1415926535897932384626 * (fc / fs)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
w0 := 2 * 3.1415926535897932384626 * (fc / fs)
w0 := 2 * math.Pi * (fc / fs)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Added in 1ce2f76

Comment on lines 14 to 16
A := 1.0 // db = 0
_ = A
// A := math.Pow(10, (db / 40))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

??

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Those coefficients are for other types of filters (notch, shelf) which are not implemented here. I kept them in for posterity, but I'll just remove them: ba2fd7d

Comment on lines 17 to 33
alpha := sinW / (2 * q)
beta := 1 // db = 0
// beta := A ^ 0.5/q
_ = beta
b0 := (1 - cosW) / 2
b1 := 1 - cosW
b2 := (1 - cosW) / 2
a0 := 1 + alpha
a1 := -2 * cosW
a2 := 1 - alpha
if filter_type == "highpass" {
b0 = (1 + cosW) / 2
b1 = -(1 + cosW)
b2 = (1 + cosW) / 2
} else if filter_type == "lowpass" {
// do nothing, default
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

black magic; can it be demystified? or at least have a reference to the implemented algorithm?

Copy link
Contributor Author

@schollz schollz Jul 31, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ha yeah! It does indeed look like black magic. Its basically ways to derive coefficients to the solution of a "biquadratic" equation, which allows you to extract multiple filter types from the same topology, just changing coefficients. I added a reference that explains it well. (c6d5413)

Comment on lines 43 to 48
// highpass filter
hpf := biquad.New(10, float64(sampleRate), 0.707, 0, "highpass")
for i := uint32(0); i < numSamples; i++ {
samples[i].Values[0] = int(math.Round(hpf.Update(float64(samples[i].Values[0]))))
}
// lowpass
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we have some references for these magic values? :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, added in ee06123

@@ -0,0 +1,70 @@
package riff
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pkgdoc and func docs? :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, added in 285626a

@@ -0,0 +1,100 @@
package wav
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know I'm more annoying than golint, but I would also appreciate the same here :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No worries! I sometimes work so fast I forget about these things, added in 97081f6

gnokey maketx addpkg --pkgpath "gno.land/r/demo/bytebeat" --pkgdir "examples/gno.land/r/demo/bytebeat" --deposit 100000000ugnot --gas-fee 1000000ugnot --gas-wanted 3000000 --broadcast --chainid dev --remote localhost:26657 <yourkey>
```

Note that adding the `gno.land/p/demo/audio/bytebeat` requires adding more to `--gas-wanted` (at least 8000000).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Note that adding the `gno.land/p/demo/audio/bytebeat` requires adding more to `--gas-wanted` (at least 8000000).
Note that adding the `gno.land/p/demo/audio/bytebeat` package requires adding more to `--gas-wanted` (at least 8000000).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 959d90b


func TestByteBeat(t *testing.T) {
s := Render("")
if len(s) != 128194 {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

very dependent on the format, don't you think?

what about strings.Contains(s, data)?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, this is fixed now (fixed a bit ago, not sure which commit).

@schollz
Copy link
Contributor Author

schollz commented Jul 6, 2023

@thehowl Thanks for the review. No problem about the time :), I am happy to get started with your feedback.

@schollz schollz mentioned this pull request Jul 10, 2023
10 tasks
@schollz
Copy link
Contributor Author

schollz commented Jul 31, 2023

This package has been updated to utilize the math_eval package so that any strings can be submitted directly from the API or Realm to generate a bytebeat audio. Merging is now blocked until #965 is merged.

@moul moul added this to the 🌟 main.gno.land (wanted) milestone Sep 8, 2023
moul added a commit that referenced this pull request Sep 16, 2023
This eval realm is capable of evaluating 32-bit integer expressions as
they would appear in Go. For example, you can use this package (and
realm) to evaluate an expression like `"(32*32+4)-1"` when presented as
a string.

This could have several applications. This current "eval" realm is
simply demonstrating it as a 32-bit integer calculator.

I would like to also add floats, but it seems `strconv.ParseFloat` is
not available yet.

In the future I would like to use the int32 version of this package in
the bytebeat realm (#840) which could
then utilize this library to generate audio directly from string
expressions, instead of programmed realms - which ultimately gets closer
to an audio NFT platform where expressions are uploaded if they are
parsed correctly.

To test, you can just add the `eval` package and realm:

```
> gnokey maketx addpkg --pkgpath "gno.land/p/demo/math_eval_int32" \
    --pkgdir "examples/gno.land/p/demo/math_eval/int32" \
    --deposit 100000000ugnot --gas-fee 1000000ugnot \
    --gas-wanted 2000000 --broadcast --chainid dev --remote localhost:26657 \
    YOURKEY
> gnokey maketx addpkg --pkgpath "gno.land/r/demo/math_eval" \
    --pkgdir "examples/gno.land/r/demo/math_eval" \
    --deposit 100000000ugnot --gas-fee 1000000ugnot \
    --gas-wanted 2000000 --broadcast --chainid dev --remote localhost:26657 \
    YOURKEY
```

Then open up to `localhost:8888/r/demo/math_eval` and try expressions in
the URL query, like `localhost:8888/r/demo/math_eval:1+1`.



![image](https://github.com/gnolang/gno/assets/6550035/5b227a22-2ce5-4217-9b9b-99c967c9af86)


<details><summary>Checklists...</summary>

## Contributors Checklist

- [x] Added new tests, or not needed, or not feasible
- [x] Provided an example (e.g. screenshot) to aid review or the PR is
self-explanatory
- [x] Updated the official documentation or not needed
- [x] No breaking changes were made, or a `BREAKING CHANGE: xxx` message
was included in the description
- [x] Added references to related issues and PRs
- [x] Provided any useful hints for running manual tests
- [ ] Added new benchmarks to [generated
graphs](https://gnoland.github.io/benchmarks), if any. More info
[here](../.benchmarks/README.md).

## Maintainers Checklist

- [ ] Checked that the author followed the guidelines in
`CONTRIBUTING.md`
- [ ] Checked the conventional-commit (especially PR title and verb,
presence of `BREAKING CHANGE:` in the body)
- [ ] Ensured that this PR is not a significant change or confirmed that
the review/consideration process was appropriate for the change
</details>

---------

Co-authored-by: Manfred Touron <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🧾 package/realm Tag used for new Realms or Packages.
Projects
Status: No status
Status: 🌟 Wanted for Launch
Status: No status
Status: In Review
Development

Successfully merging this pull request may close these issues.

4 participants