docs(blog): add post on stream-batch unification #8293
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
chloeh13q
force-pushed
the
docs/flink-stream-batch
branch
3 times, most recently
from
3f2cc65 to
42a409c
Compare
chloeh13q
commented
Feb 9, 2024
| @@ -0,0 +1,414 @@ | |||
| --- | |||
| title: "Stream-batch unification through Ibis lens" | |||
There was a problem hiding this comment.
I considered titling this "ELI5: How Ibis unifies streaming and batch"... LOL Perhaps there is a better title
chloeh13q
force-pushed
the
docs/flink-stream-batch
branch
from
d752854 to
74c90cf
Compare
|
|
lostmygithubaccount
added
docs
ibis-docs-bot
bot
removed
the
docs-preview
lostmygithubaccount
requested changes
Feb 13, 2024
| --- | ||
| title: "Stream-batch unification through Ibis lens" | ||
| author: "Chloe" | ||
| date: 2024-02-08 |
There was a problem hiding this comment.
update -- let's either do this Thursday (this week) or Monday -- I want to get the roadmap out, so maybe roadmap Thursday and this Monday? then again maybe it's better to capitalize on the 8.0 launch? then again maybe it's better to wait and re-ignite the streaming fire next week? idk
let's plan on this going out Monday tentatively, roadmap this Thursday
Comment on lines
11
to
22
| One of my focuses in the past 10 months has been to implement the Flink backend | ||
| for Ibis. I was working with Apache Flink and building a feature engineering | ||
| tool, and we stumbled upon Ibis as we attempted to build our own translation | ||
| layer that could turn user declarations into relation trees, then optimize and | ||
| deploy the query plan, all while maintaining the underlying infrastructure for | ||
| the user. We considered and prototyped with a number of tools and eventually | ||
| chose Ibis. It had already established a position in the batch world and had | ||
| support for 10+ of the most popular batch engines (at the time). We loved the | ||
| idea of decoupling the user-facing interface from the execution engine, so that | ||
| users can swap out the execution engine depending on their needs, without | ||
| having to rewrite code. And, of course, it was open-source. It was everything | ||
| we dreamed of. |
There was a problem hiding this comment.
some form of this could be good for the testimonials page
also unnecessary spaces on lines 20-22
lostmygithubaccount
added
the
docs-preview
ibis-docs-bot
bot
removed
the
docs-preview
chloeh13q
force-pushed
the
docs/flink-stream-batch
branch
from
4d9ed20 to
8051858
Compare
ibis-docs-bot
bot
removed
the
docs-preview
ibis-docs-bot
bot
removed
the
docs-preview
lostmygithubaccount
requested changes
Feb 26, 2024
… unbound expressions instead of defining the transformation twice
chloeh13q
force-pushed
the
docs/flink-stream-batch
branch
from
33741fd to
28f039c
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description of changes
Add a new blog post on stream-batch unification.