Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement and benchmark ArrowOperationPlus node #10150

Merged
merged 43 commits into from
Jun 11, 2024
Merged
Changes from 1 commit
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
a56aa05
Test for +[Int8] behavior
JaroslavTulach May 31, 2024
a9e07f1
Generalizing parser to support +
JaroslavTulach May 31, 2024
f6d7e84
Trivial implementation of ArrowOperationPlus
JaroslavTulach Jun 3, 2024
2f78ea9
Merge remote-tracking branch 'origin/develop' into wip/jtulach/PocArr…
JaroslavTulach Jun 3, 2024
a044cb0
Benchmark Arrow + implementation against Table one
JaroslavTulach Jun 3, 2024
9c70d17
Invoke foreign function with arguments
JaroslavTulach Jun 3, 2024
a008b3d
Allow Int64 value to be null
JaroslavTulach Jun 3, 2024
a6248a1
Support isNull arguments
JaroslavTulach Jun 3, 2024
2f16308
Measure just arrow_plus performance
JaroslavTulach Jun 3, 2024
2bfba6e
Speeding up by using different interop library for different element
JaroslavTulach Jun 3, 2024
760a106
Tests for Int64 buffer
JaroslavTulach Jun 3, 2024
f9a802e
For some reasons there are nulls in the array
JaroslavTulach Jun 3, 2024
68d9306
Fix byte mask used for index calculation
hubertp Jun 3, 2024
1dbbae1
Tell the compiler static values are final constants
JaroslavTulach Jun 4, 2024
ea36864
Prefer final fields and hope they will be constant
JaroslavTulach Jun 4, 2024
82ca3e7
Use null bitmapBuffer when there are no null values
JaroslavTulach Jun 4, 2024
c36d22e
Make sure unit tests work
JaroslavTulach Jun 4, 2024
33f8694
Measure arrow x + u (and not y)
JaroslavTulach Jun 4, 2024
ac3eda6
Inline ByteBuffer.getLong and putLong methods
JaroslavTulach Jun 4, 2024
869fd76
Using Specialized Libraries
JaroslavTulach Jun 4, 2024
87ff552
Brute force speedup
JaroslavTulach Jun 4, 2024
d78e305
Eliminating write overhead
JaroslavTulach Jun 4, 2024
13e244b
Let's abstract direct buffer access with a Truffle Iterator
JaroslavTulach Jun 5, 2024
62cf4fb
Merge remote-tracking branch 'origin/develop' into wip/jtulach/PocArr…
JaroslavTulach Jun 5, 2024
1b126a8
Benchmarking overflowing plus
JaroslavTulach Jun 7, 2024
fa140c0
Introducing AppendNode
JaroslavTulach Jun 7, 2024
f8eefc3
Using (unoptimized) builder.append again
JaroslavTulach Jun 7, 2024
801184a
Cache logical layout in the plus operation
JaroslavTulach Jun 7, 2024
5dc5d77
Value to number conversion node
JaroslavTulach Jun 7, 2024
46f28d0
AppendNode via ValueToNumberNode
JaroslavTulach Jun 7, 2024
fc42f06
Removing unused crap
JaroslavTulach Jun 7, 2024
c993a91
Runtime.assert that the Column results are the same as Arrow results
JaroslavTulach Jun 7, 2024
f10bd06
PutNode to optimize AppendNode
JaroslavTulach Jun 7, 2024
a661aca
Merging with develop
JaroslavTulach Jun 7, 2024
72c89a1
Proper, but slow, support for nulls in the arrow vectors
JaroslavTulach Jun 7, 2024
5e54a10
Manipulate bitmapBuffer from nodes
JaroslavTulach Jun 8, 2024
5fcba2d
Introducing NullIterator that reads bitmapBuffer
JaroslavTulach Jun 8, 2024
d03e484
Introducing executeOp and eliminating overhead of ArithmeticException
JaroslavTulach Jun 8, 2024
3f49737
Making the graph smaller by eliminating handling of exceptional state
JaroslavTulach Jun 8, 2024
55ee7cc
Math.addExact is intrinsified. Use it! But only until first Arithmeti…
JaroslavTulach Jun 8, 2024
2bb6ebc
Benchmark work with an array with occational Nothing values
JaroslavTulach Jun 8, 2024
bce1892
Merge branch 'develop' into wip/jtulach/PocArrowPlus10056
mergify[bot] Jun 10, 2024
807fd6d
create_arrow_columns function
JaroslavTulach Jun 11, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -184,15 +184,15 @@ public int capacity() throws UnsupportedMessageException {
public boolean isNull(int index) {
var bufferIndex = index >> 3;
var slot = bitmapBuffer.get(bufferIndex);
var byteIndex = index & ~(1 << 3);
var byteIndex = index & byteMask;
var mask = 1 << byteIndex;
return (slot & mask) == 0;
}

public void setNull(int index) {
var bufferIndex = index >> 3;
var slot = bitmapBuffer.get(bufferIndex);
var byteIndex = index & ~(1 << 3);
var byteIndex = index & byteMask;
var mask = ~(1 << byteIndex);
bitmapBuffer.put(bufferIndex, (byte) (slot & mask));
}
Expand All @@ -201,12 +201,15 @@ private void setValidityBitmap(int index0, int unitSize) {
var index = index0 / unitSize;
var bufferIndex = index >> 3;
var slot = bitmapBuffer.get(bufferIndex);
var byteIndex = index & ~(1 << 3);
var byteIndex = index & byteMask;

var mask = 1 << byteIndex;
var updated = (slot | mask);
bitmapBuffer.put(bufferIndex, (byte) (updated));
}

private static final int byteMask = ~(~(1 << 3) + 1); // 7
JaroslavTulach marked this conversation as resolved.
Show resolved Hide resolved

@Override
public void close() throws Exception {
this.dataBuffer.clear();
Expand Down
Loading