Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(performance,vrl): Add RFC for "LLVM Backend for VRL" #10518

Merged
merged 24 commits into from
Jan 31, 2022

Conversation

pablosichert
Copy link
Contributor

@pablosichert pablosichert commented Dec 20, 2021

Closes #10517.

Rendered version 👀.

Some preliminary results from my local machine (executing the script 1M times each):

.hostname = "vector"

if .status == "warning" {
    .thing = upcase(.hostname)
} else if .status == "notice" {
    .thung = downcase(.hostname)
} else {
    .nong = upcase(.hostname)
}

.matches = { "name": .message, "num": "2" }

Traverse: 1.653175s
VM: 715.790583ms
LLVM: 608.121458ms


.foo = string("hi!")

Traverse: 248.525166ms
VM: 102.147416ms
LLVM: 41.894708ms


.key = "foo"

Traverse: 275.768333ms
VM: 57.462375ms
LLVM: 33.156291ms


.num = 123

Traverse: 270.735291ms
VM: 43.291541ms
LLVM: 27.7585ms


if .status == 123 {
    .foo = "bar"
}

Traverse: 241.31875ms
VM: 50.283333ms
LLVM: 31.510625ms


Additional material: slides for the presentation on LLVM Backend for VRL.

@pablosichert pablosichert added domain: performance Anything related to Vector's performance domain: vrl Anything related to the Vector Remap Language domain: rfc labels Dec 20, 2021
@netlify
Copy link

netlify bot commented Dec 20, 2021

✔️ Deploy Preview for vector-project canceled.

🔨 Explore the source changes: 7e1b77b

🔍 Inspect the deploy log: https://app.netlify.com/sites/vector-project/deploys/61f7e2ee76c7a60007504113

@github-actions github-actions bot removed the domain: vrl Anything related to the Vector Remap Language label Dec 20, 2021
@pablosichert pablosichert added the domain: vrl Anything related to the Vector Remap Language label Dec 20, 2021
@pablosichert pablosichert force-pushed the pablosichert/llvm-backend-for-vrl-rfc branch from 8c47e60 to 7645459 Compare December 20, 2021 12:58
@github-actions github-actions bot removed the domain: vrl Anything related to the Vector Remap Language label Dec 20, 2021
@pablosichert pablosichert force-pushed the pablosichert/llvm-backend-for-vrl-rfc branch from 7645459 to 6ca41f0 Compare December 20, 2021 13:18
rfcs/2021-12-20-10517-llvm-backend-for-vrl.md Outdated Show resolved Hide resolved
rfcs/2021-12-20-10517-llvm-backend-for-vrl.md Outdated Show resolved Hide resolved
@StephenWakely
Copy link
Contributor

I think it could be worth adding a section to discuss calling functions in the VRL stdlib. We need to give some thought to how it will handle optional and named parameters. I'm not sure what the best solution would be here.

@jszwedko jszwedko requested review from fuchsnj, JeanMertz and tobz and removed request for tobz January 25, 2022 16:05
@pablosichert pablosichert changed the title chore(performance,vrl,rfc): Add RFC for "LLVM Backend for VRL" chore(performance,vrl): Add RFC for "LLVM Backend for VRL" Jan 26, 2022
@pablosichert
Copy link
Contributor Author

@StephenWakely I agree that we still need to figure out the specifics of handling function parameters. However, I don't think the implementation details matter too much to the general approach lined out here other than "we will find a solution". If there's demand to discuss the details of these, we can add another RFC.

I added it to the action items: 1e8b6ab.

Copy link
Contributor

@JeanMertz JeanMertz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a great RFC, nice work @pablosichert.

I did leave some remarks and questions inline. Additionally, I'd like there to be a section specifically around testing (how we will test, what does this require us to have in terms of testing infrastructure, what parts of this are difficult to test and require manual checks during PR reviews).

(I know there are two items on testing in the "Plan Of Attack" section, but I'd like that to be expanded into its own section in the actual RFC, with an explanation on why we need certain tests, etc)

rfcs/2021-12-20-10517-llvm-backend-for-vrl.md Outdated Show resolved Hide resolved
rfcs/2021-12-20-10517-llvm-backend-for-vrl.md Outdated Show resolved Hide resolved
rfcs/2021-12-20-10517-llvm-backend-for-vrl.md Show resolved Hide resolved
rfcs/2021-12-20-10517-llvm-backend-for-vrl.md Outdated Show resolved Hide resolved
rfcs/2021-12-20-10517-llvm-backend-for-vrl.md Show resolved Hide resolved
rfcs/2021-12-20-10517-llvm-backend-for-vrl.md Outdated Show resolved Hide resolved
rfcs/2021-12-20-10517-llvm-backend-for-vrl.md Show resolved Hide resolved
rfcs/2021-12-20-10517-llvm-backend-for-vrl.md Outdated Show resolved Hide resolved
rfcs/2021-12-20-10517-llvm-backend-for-vrl.md Show resolved Hide resolved
Copy link
Contributor

@StephenWakely StephenWakely left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a great start to a very promising project!

Signed-off-by: Pablo Sichert <[email protected]>
@pablosichert pablosichert force-pushed the pablosichert/llvm-backend-for-vrl-rfc branch from 2df80e5 to ea3c206 Compare January 27, 2022 17:31
Copy link
Contributor

@JeanMertz JeanMertz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎉

@pablosichert pablosichert merged commit ab2141c into master Jan 31, 2022
@pablosichert pablosichert deleted the pablosichert/llvm-backend-for-vrl-rfc branch January 31, 2022 19:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
domain: performance Anything related to Vector's performance domain: rfc
Projects
None yet
Development

Successfully merging this pull request may close these issues.

RFC for "LLVM Backend for VRL"
8 participants