Benchmark Truffle "add" for arrow-language and Column.+

[JaroslavTulach]

There already is an implementation of arrow-language written by @hubertp. There is an interest in cleaning up the current Storage architecture particularly to clearly separate storage from operations. @JaroslavTulach claims that the most efficient (and at the end also simplest) implementation can be achieved by using Truffle. Let's run an experiment to verify such a claim!

The documentation of Column.+ mentions two examples:

             vector_plus = Examples.float_column + Examples.integer_column
             scalar_plus = Examples.integer_column + 10

let's reimplement them with the Arrow language. Then let's generate a million item column and benchmark them against each other.

API

The Arrow language already supports two operations: new[type] and cast[type]. E.g. one can write new[Int32] and gets a function that can be called to allocate array of provided (as function argument) size. Let's introduce new syntax:

+[type]

parsing such statement is going to produce a function (e.g. isExecutable object) that takes two arguments. When invoked, it'd check whether both arguments hasArrayElements. If so, they have to have the same getArraySize(). Then the function would create new[type] for appropriate size and filled the arrow column with added content of both arrays. If one (or both) arguments are scalars, then the same behavior as scalar_plus would "happen".

One of the important steps in the implementation are conversions. The values need of arguments need to be converted to the target type of the resulting array. Such conversions can benefit from Truffle @Specialization and effectively compute the optimal result.

Expected Result

After final benchmarking we shall see that the Truffle results are faster, yet the code remains simple.

[enso-bot]

Jaroslav Tulach reports a new STANDUP for yesterday (2024-06-03):

Progress: - arrow language +: #10150

• discovered interop error: Cannot execute foreign function value  #10151
• arrow + benchmark runs: 2f16308
• searching for "null bug": f9a802e
• fighting with IGV
• standup & meetings It should be finished by 2024-06-07.

[enso-bot]

Jaroslav Tulach reports a new STANDUP for yesterday (2024-06-04):

Progress: - RoundingUtil: #10150 (comment)

• removed overhead of null checks: Implement and benchmark ArrowOperationPlus node #10150 (comment)
• inline ByteBuffer calls: Implement and benchmark ArrowOperationPlus node #10150 (comment)
• QED: Implement and benchmark ArrowOperationPlus node #10150 (comment)
• What's the name of my method PR: What's the name of my method? #10164 (comment)
• review mutable builder: Extract mutable builder from ModuleScope #9914 (review)
• review Ydoc test: Implement the ydoc js bundle test #10160 (comment)
• planging, standup, reviews It should be finished by 2024-06-07.

[JaroslavTulach]

• What's the name of my method? #10164 just got merged into the main line

[enso-bot]

Jaroslav Tulach reports a new STANDUP for yesterday (2024-06-06):

Progress: - Hands on IGV lab! https://drive.google.com/file/d/11o2eHSy0Ptux0SWW9JpRiGQp5wtVZshh/view

• improving IGV documentation: Updating IGV documentation #10193
• discussion about import/export with Marcin: https://discord.com/channels/@me/944471092771295253/1248144379969933363
• PR merged: Execute foreign function and check autoscoped constructor result #10187
• fix for Partially applied autoscope constructors get thru ascribed type check #10180: ab0724c
• fix for Autoscoping can build ClosureRootNode with a null ModuleScope #10186: 55dce2f
• meeting about Remove expression UUIDs from metadata section of a source file #10182 It should be finished by 2024-06-07.

Discord

Discord - Group Chat That’s All Fun & Games

Discord is great for playing games and chilling with friends, or even building a worldwide community. Customize your own space to talk, play, and hang out.

[enso-bot]

Jaroslav Tulach reports a new STANDUP for yesterday (2024-06-07):

Progress: - Hands on IGV lab recording: https://drive.google.com/file/d/11o2eHSy0Ptux0SWW9JpRiGQp5wtVZshh/view?usp=sharing

• long with nulls benchmark
• Using builder.append again
• Speeding append up: 801184a
• PutNode: f10bd06
• proper (but slow) support for nulls: 72c89a1 It should be finished by 2024-06-07.

Google Docs

Hands on IGV Lab (2024-06-06 17:05 GMT+2)