-
Notifications
You must be signed in to change notification settings - Fork 374
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
RFC: Built in buffer type and library #739
Conversation
I want to add that this is a feature that Chickynoid, a server-authoritative character controller, would greatly benefit from. In our test bench for Chickynoid, we found that it could only reliably support up to ~60 players because the bottleneck was, somewhat surprisingly, packet serialization for the custom replication. Chickynoid created a unique replication packet for each player at 20hz. Beyond 60 players, the time spent in each frame serializing packets exceeded the frame budget and caused massive performance degradation on the server. Unfortunately, there was no way around this because we were limited to slower Lua-side byte buffer implementations. Our goal was to comfortably support 100 players in each match, but this was impossible without native byte buffers. EDIT: Just to note, the byte buffer is still serialized to a string so that it can be sent over remotes. I'm not asking for direct binary support over Roblox remotes because that's out of scope here. |
👍 Outside of integers and floats, bytes should be supported as well for the datatypes. They are very common as well, and this could make working with certain strings more performant as it wouldn't add new strings to the heap when changing them (and making changing them easier) (A byte is just a number with 8 bits) |
8 bit integers (signed and unsigned) are already included in the RFC. |
Existing solutions to working with binary data aren't unworkably slow, but they are incredibly inefficient in terms of memory. Well-made solutions are also prohibitively complex, so the number of people who can write and maintain them isn't very high. This has lead us to the point where as far as I know there's less than a dozen 'good' modules for working with binary data in the wild and they vary wildly in performance. This is obviously not ideal under any circumstance but it's most impactful when writing performance-sensitive code like you do in game development. Something as simple as saving game data might use a custom binary format, and it pays to be both fast and efficient when serializing to it. Right now, that's not really an option though, so users of Luau are either being thorough with their storage formats or being quick. Not both. While I'm hesitant to support an RFC that suggests adding a new 'primitive' type to Luau, I think it has to be done for binary manipulation to be done properly. Strings are far too slow to use for this, even though @Rerumu At the moment this feels like it's just a proposal to add native (resizable?) arrays and then a library to manipulate binary data on top of them. While obviously the implementation details would have to be bashed out, is there something I'm missing there? Because if not, this may benefit from being split into two RFCs: one for native arrays and one for binary functions. |
I would suggest proposing this as a library implemented with a userdata type. |
This could be the case, and it might be better to have a general solution, but I have not encountered a problem yet where Lua's built in arrays weren't appropriately fast or memory efficient except for the case of working directly on byte streams. I would be open to a general solution, I just feel as is, one wouldn't find much general use. |
Don't mean to be a pessimist here, but as Dekkonaut says Luau already has functionality like this: It uses an internal C API type called a "Buffer", which is a super fast append byte array which can be converted to a lua string. The format string's documentation can be found here. I naively rewrote @grilme99 's motivating example, a server-authoritative character & movement library, to serialize it's |
I wonder if mutable strings used internally for string buffer operations (like in |
I wasn't aware of the mutable string stuff, so that raises a good question on whether it's worth exposing. Also, if there is a better design proposal that works I'd like to know what changes (or rewrite) of the RFC should be made. As long as the use case is covered I think how the API looks won't matter too much. |
Luau needs to add better documentation for Many people don't know about thwm and adding documentation would fix a lot of the afformentioned issues as people dont knoe about their existence |
I think there has been a misunderstanding. I am aware of |
This looks exciting to see! My main reason for not encoding things as strings/chars in Roblox projects is that non-UTF8 strings can't be serialized. APIs such as DataStoreService will throw an error when an invalid UTF8 grapheme is ever encountered in a string, and I believe you can't actually represent certain character sequences in strings in general. This leads to very specific edge cases that you probably won't encounter while developing and playtesting, but exploiters and unlucky users will, and could cause undefined behavior and player data corruption potentially. Would be awesome if raw bytes could be sent through HTTP APIs on roblox in the future too. |
After looking into it, having a mutable string value will bring more issues if the LUA_TSTRING tag is reused for it. To conclude, I think the best way is to implement this as a library that creates a tagged userdata object. Tag can be reserved similar to |
I started an experimental implementation of this RFC as a library and there are some things to consider for this RFC. To aim for the best performance and especially to be able to map function to fast-calls and later lower them to single instruction in our native code generation, removing the endianness from the API might be a good decision. We also need to explicitly say that unaligned reads are allowed. Once again, JS engines go the other way of requiring a Typed view so that the implementation is reading 32 bit integers at offsets that are multiple of 4, as an example. I believe this was relaxed in WebAssembly, but I don't have a citation on that. For methods that are reading floating-point numbers I would like for us to have some flexibility around NaN values. Finally, something to note in the drawbacks is that resizing will prevent us from embedding data inside the userdata and will require a secondary allocation (small-buffer optimization might help, but that's optional). |
Good catch, this solution is probably much better and I agree endianness should probably be handled outside the buffer. My only concern might be endianness of 64 bit integers as those would be truncated in
Yeah the WebAssembly memory instructions are explicit that unaligned access must succeed, but it's okay for it to be slower. It makes sense to add something like this to the RFC too.
This I understand though I am a bit wary of; it is an admittedly niche use case but the WASM specification's reinterpret instructions require that bit patterns be preserved to and from float conversions. I have in fact faced issues around that in the past, as LuaJIT does NaN boxing for its value types, and it's resulted in memory corruption due to reading otherwise normal NaN values out of an FFI array. LLVM also somewhat commonly will generate code that relies on this property which eventually makes it down to the WASM output, and anecdotally I've found most of it to be in hashmap implementations.
This seems reasonable too if it allows for much better optimization potential. The original use case was that of keeping a single buffer for reuse and expanding similar to vectors in native languages but I see now that it might not be as applicable to this sort of data structure. |
Is the goal of the implementation to avoid atomization or is it to simply allow strings to be constructed and deconstructed from smaller components? As in, are we going to be making a brand new type that is completely separate from strings, or are we just making fancier APIs to encode data into a lua string type? |
The buffer type would be a separate userdata type. There were some discussions about exposing the mutable strings used internally in the compiler, but it was decided against. |
I've written an experimental implementation of this here master...vegorov-rbx:luau:bytebuf RFC is currently lacking names, so I named the library
Fastcalls and IR is supported for:
Including names in the RFC will help bikeshed for better names in the discussion. Some choices that are omitted from the RFC that were made:
|
I guess the most difficult problem now is the naming. I don't really know what to name it, As for the choices, For doubles converted to ints, are they truncated to 32 bits or 64 bits? On Roblox it might be that 4gb+ buffers are never reasonable but in standalone Luau I can see the use case for it, and using the ~52 bit integer limit of doubles might make sense here. Everything else looks good though. Should I update the RFC to include these specific changes? |
32 bits. Extending to 2GB-smallN userdata limit is possible, extending further will require a separate feature request (that will require a strong use case of an actual project). |
Yes. Let's also use 'buffer' as the name. |
I think we can avoid placing the specific memory limit in the specification and just have this 1GB as the current implementation limit. |
My initial concern with this is that constructing a buffer with a specific contents or dumping the contents is going to be boilerplate despite it being a core use of buffers. I'd like to propose something akin to It's not terribly cumbersome to do either of these things with the functions as presented above, but I'd rather the boilerplate be removed entirely rather than exist in basically every use case where you need to work with pre-existing data or export the data. |
I don't think there's a definite right answer here, but I weigh much more heavily on the VM type side:
All of this to say is that I think a new VM type is actually more cohesive with what we have right now, than a custom tag for a userdata that happens to correspond to a type/library that we support out of the box with tight runtime integration for performance. It's easier to explain - we're not just adding a buffer library that happens to be able to create buffer objects and by the way the runtime secretly knows about a userdata tag, we're just adding a buffer type which, just like string or table, has a first-class VM type, library, and runtime integration. I also don't think we are at the risk of an explosive growth of types - not just because we have limits, and our current pace of adding types is slow, but also because we are fundamentally trying to keep the language and implementation reasonably small and expect most of the "extras" to be userdata types outside of the confines of language specification and implementation - which is to say, I think we should by default resist the addition of new types anyway, and in some sense making it easier to add new types to Luau-the-language (as opposed to, for example, Roblox-the-platform-that-embeds-Luau) is an anti-goal ;-) (I do like this proposal and think we're getting close to being able to merge it.) |
Rewrite design/motivation sections.
I've also rewritten design/motivation sections (without changing the substance). |
Minor question about
I expect that at some point in the future we might want to add
Unsure if there are other considerations here (apart from |
Updated the RFC to switch from userdata type to a new tag value type. Update also includes extra motivation for offsets starting at 0 and describes the C API. |
Re:
Yeah, I agree this is not ideal. However, the luaL_Buffer API has been fairly idiosyncratic for a while. It's mostly an internal API for the implementation of various string libraries. It may be a little useful for embedders, but it's not ABI-stable, it's not API-stable (we've changed details compared to Lua 5.x when reworking these with no notice), and while you can still find a small subset between Lua 5.1, Lua 5.4 and Luau API that compiles everywhere (
I think we can probably improve this gradually. For example, we can call the type We could do an analysis of open source libraries that target Luau API to see if they use luaL_Buffer or not to understand the impact of these changes better. mlua and LuaBridge3 don't use this and neither does Roblox so this gives me hope that this is something we can gradually rework to be nicer without too many issues. |
Updated the RFC:
Added For a moment, I thought about
It could be seen that current copy would match |
Thanks. I agree with I propose to lock this RFC for new functionality and just focus on adjusting details or finding inconsistencies in the existing function set from now on. |
Should the format of signed integers be specified in the RFC? We presumably all assume it will be two's complement, but I would also generally assume that IEEE-754 floats would be used and that was specified. It may be worth including it just for clarity. |
Co-authored-by: Micah <[email protected]>
…ers (#1052) This function might be of particular use if #739 is accepted, but as it stands, it still brings a measurable reduction in complexity for swapping byte order and is measurably faster (around a 50% reduction in time per operation on my machine), albeit on the order of magnitude of nanoseconds.
Simplified Alternatives section
I've cleaned up and simplified alternatives section further. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks good to me but let's wait for Oct 19th so that there's two full weeks between this RFC going into "in review" status and us merging it.
### What's Changed - Improve readability of unions and intersections by limiting the number of elements of those types that can be presented on a single line (gated under `FFlag::LuauToStringSimpleCompositeTypesSingleLine`) - Adds a new option to the compiler `--record-stats` to record and output compilation statistics - `if...then...else` expressions are now optimized into `AND/OR` form when possible. ### VM - Add a new `buffer` type to Luau based on the [buffer RFC](#739) and additional C API functions to work with it; this release does not include the library. - Internal C API to work with string buffers has been updated to align with Lua version more closely ### Native Codegen - Added support for new X64 instruction (rev) and new A64 instruction (bswap) in the assembler - Simplified the way numerical loop condition is translated to IR ### New Type Solver - Operator inference now handled by type families - Created a new system called `Type Paths` to explain why subtyping tests fail in order to improve the quality of error messages. - Systematic changes to implement Data Flow analysis in the new solver (`Breadcrumb` removed and replaced with `RefinementKey`) --- Co-authored-by: Aaron Weiss <[email protected]> Co-authored-by: Alexander McCord <[email protected]> Co-authored-by: Andy Friesen <[email protected]> Co-authored-by: Aviral Goel <[email protected]> Co-authored-by: Lily Brown <[email protected]> Co-authored-by: Vighnesh Vijay <[email protected]> Co-authored-by: Vyacheslav Egorov <[email protected]> --------- Co-authored-by: Arseny Kapoulkine <[email protected]> Co-authored-by: Vyacheslav Egorov <[email protected]> Co-authored-by: Andy Friesen <[email protected]> Co-authored-by: Lily Brown <[email protected]> Co-authored-by: Aaron Weiss <[email protected]> Co-authored-by: Alexander McCord <[email protected]> Co-authored-by: Aviral Goel <[email protected]>
### What's Changed - Improve readability of unions and intersections by limiting the number of elements of those types that can be presented on a single line (gated under `FFlag::LuauToStringSimpleCompositeTypesSingleLine`) - Adds a new option to the compiler `--record-stats` to record and output compilation statistics - `if...then...else` expressions are now optimized into `AND/OR` form when possible. ### VM - Add a new `buffer` type to Luau based on the [buffer RFC](luau-lang/luau#739) and additional C API functions to work with it; this release does not include the library. - Internal C API to work with string buffers has been updated to align with Lua version more closely ### Native Codegen - Added support for new X64 instruction (rev) and new A64 instruction (bswap) in the assembler - Simplified the way numerical loop condition is translated to IR ### New Type Solver - Operator inference now handled by type families - Created a new system called `Type Paths` to explain why subtyping tests fail in order to improve the quality of error messages. - Systematic changes to implement Data Flow analysis in the new solver (`Breadcrumb` removed and replaced with `RefinementKey`) --- Co-authored-by: Aaron Weiss <[email protected]> Co-authored-by: Alexander McCord <[email protected]> Co-authored-by: Andy Friesen <[email protected]> Co-authored-by: Aviral Goel <[email protected]> Co-authored-by: Lily Brown <[email protected]> Co-authored-by: Vighnesh Vijay <[email protected]> Co-authored-by: Vyacheslav Egorov <[email protected]> --------- Co-authored-by: Arseny Kapoulkine <[email protected]> Co-authored-by: Vyacheslav Egorov <[email protected]> Co-authored-by: Andy Friesen <[email protected]> Co-authored-by: Lily Brown <[email protected]> Co-authored-by: Aaron Weiss <[email protected]> Co-authored-by: Alexander McCord <[email protected]> Co-authored-by: Aviral Goel <[email protected]>
Rendered document.