-
Notifications
You must be signed in to change notification settings - Fork 323
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implement and benchmark ArrowOperationPlus
node
#10150
Merged
Merged
Changes from 3 commits
Commits
Show all changes
43 commits
Select commit
Hold shift + click to select a range
a56aa05
Test for +[Int8] behavior
JaroslavTulach a9e07f1
Generalizing parser to support +
JaroslavTulach f6d7e84
Trivial implementation of ArrowOperationPlus
JaroslavTulach 2f78ea9
Merge remote-tracking branch 'origin/develop' into wip/jtulach/PocArr…
JaroslavTulach a044cb0
Benchmark Arrow + implementation against Table one
JaroslavTulach 9c70d17
Invoke foreign function with arguments
JaroslavTulach a008b3d
Allow Int64 value to be null
JaroslavTulach a6248a1
Support isNull arguments
JaroslavTulach 2f16308
Measure just arrow_plus performance
JaroslavTulach 2bfba6e
Speeding up by using different interop library for different element
JaroslavTulach 760a106
Tests for Int64 buffer
JaroslavTulach f9a802e
For some reasons there are nulls in the array
JaroslavTulach 68d9306
Fix byte mask used for index calculation
hubertp 1dbbae1
Tell the compiler static values are final constants
JaroslavTulach ea36864
Prefer final fields and hope they will be constant
JaroslavTulach 82ca3e7
Use null bitmapBuffer when there are no null values
JaroslavTulach c36d22e
Make sure unit tests work
JaroslavTulach 33f8694
Measure arrow x + u (and not y)
JaroslavTulach ac3eda6
Inline ByteBuffer.getLong and putLong methods
JaroslavTulach 869fd76
Using Specialized Libraries
JaroslavTulach 87ff552
Brute force speedup
JaroslavTulach d78e305
Eliminating write overhead
JaroslavTulach 13e244b
Let's abstract direct buffer access with a Truffle Iterator
JaroslavTulach 62cf4fb
Merge remote-tracking branch 'origin/develop' into wip/jtulach/PocArr…
JaroslavTulach 1b126a8
Benchmarking overflowing plus
JaroslavTulach fa140c0
Introducing AppendNode
JaroslavTulach f8eefc3
Using (unoptimized) builder.append again
JaroslavTulach 801184a
Cache logical layout in the plus operation
JaroslavTulach 5dc5d77
Value to number conversion node
JaroslavTulach 46f28d0
AppendNode via ValueToNumberNode
JaroslavTulach fc42f06
Removing unused crap
JaroslavTulach c993a91
Runtime.assert that the Column results are the same as Arrow results
JaroslavTulach f10bd06
PutNode to optimize AppendNode
JaroslavTulach a661aca
Merging with develop
JaroslavTulach 72c89a1
Proper, but slow, support for nulls in the arrow vectors
JaroslavTulach 5e54a10
Manipulate bitmapBuffer from nodes
JaroslavTulach 5fcba2d
Introducing NullIterator that reads bitmapBuffer
JaroslavTulach d03e484
Introducing executeOp and eliminating overhead of ArithmeticException
JaroslavTulach 3f49737
Making the graph smaller by eliminating handling of exceptional state
JaroslavTulach 55ee7cc
Math.addExact is intrinsified. Use it! But only until first Arithmeti…
JaroslavTulach 2bb6ebc
Benchmark work with an array with occational Nothing values
JaroslavTulach bce1892
Merge branch 'develop' into wip/jtulach/PocArrowPlus10056
mergify[bot] 807fd6d
create_arrow_columns function
JaroslavTulach File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
16 changes: 0 additions & 16 deletions
16
...-language-arrow/src/main/java/org/enso/interpreter/arrow/node/ArrowCastFixedSizeNode.java
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
16 changes: 0 additions & 16 deletions
16
...time-language-arrow/src/main/java/org/enso/interpreter/arrow/node/ArrowFixedSizeNode.java
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
61 changes: 61 additions & 0 deletions
61
...e-language-arrow/src/main/java/org/enso/interpreter/arrow/runtime/ArrowOperationPlus.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,61 @@ | ||
package org.enso.interpreter.arrow.runtime; | ||
|
||
import com.oracle.truffle.api.interop.ArityException; | ||
import com.oracle.truffle.api.interop.InteropLibrary; | ||
import com.oracle.truffle.api.interop.InvalidArrayIndexException; | ||
import com.oracle.truffle.api.interop.TruffleObject; | ||
import com.oracle.truffle.api.interop.UnknownIdentifierException; | ||
import com.oracle.truffle.api.interop.UnsupportedMessageException; | ||
import com.oracle.truffle.api.interop.UnsupportedTypeException; | ||
import com.oracle.truffle.api.library.CachedLibrary; | ||
import com.oracle.truffle.api.library.ExportLibrary; | ||
import com.oracle.truffle.api.library.ExportMessage; | ||
|
||
@ExportLibrary(InteropLibrary.class) | ||
public final class ArrowOperationPlus implements TruffleObject { | ||
private final ArrowFixedSizeArrayFactory factory; | ||
|
||
public ArrowOperationPlus(ArrowFixedSizeArrayFactory factory) { | ||
this.factory = factory; | ||
} | ||
|
||
@ExportMessage | ||
boolean isExecutable() { | ||
return true; | ||
} | ||
|
||
@ExportMessage | ||
Object execute(Object[] args, @CachedLibrary(limit = "3") InteropLibrary iop) | ||
throws ArityException, UnsupportedTypeException, UnsupportedMessageException { | ||
if (args.length != 2) { | ||
throw ArityException.create(2, 2, args.length); | ||
} | ||
var arr0 = args[0]; | ||
var arr1 = args[1]; | ||
if (!iop.hasArrayElements(arr0) || !iop.hasArrayElements(arr1)) { | ||
throw UnsupportedTypeException.create(args); | ||
} | ||
var len = iop.getArraySize(arr0); | ||
if (len != iop.getArraySize(arr1)) { | ||
throw UnsupportedTypeException.create(args, "Arrays must have the same length"); | ||
} | ||
var builder = iop.instantiate(factory, len); | ||
try { | ||
for (long i = 0; i < len; i++) { | ||
var elem0 = iop.readArrayElement(arr0, i); | ||
var elem1 = iop.readArrayElement(arr1, i); | ||
var l0 = iop.asLong(elem0); | ||
var l1 = iop.asLong(elem1); | ||
iop.invokeMember(builder, "append", l0 + l1); | ||
} | ||
return iop.invokeMember(builder, "build"); | ||
} catch (InvalidArrayIndexException | UnknownIdentifierException ex) { | ||
throw raise(RuntimeException.class, ex); | ||
} | ||
} | ||
|
||
@SuppressWarnings("unchecked") | ||
private static <E extends Throwable> E raise(Class<E> type, Throwable t) throws E { | ||
throw (E) t; | ||
} | ||
} |
71 changes: 71 additions & 0 deletions
71
engine/runtime-language-arrow/src/test/java/org/enso/interpreter/arrow/AddArrowTest.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,71 @@ | ||
package org.enso.interpreter.arrow; | ||
|
||
import static org.junit.Assert.*; | ||
|
||
import org.graalvm.polyglot.Context; | ||
import org.graalvm.polyglot.io.IOAccess; | ||
import org.junit.AfterClass; | ||
import org.junit.BeforeClass; | ||
import org.junit.Test; | ||
|
||
public class AddArrowTest { | ||
private static Context ctx; | ||
|
||
@BeforeClass | ||
public static void initEnsoContext() { | ||
ctx = | ||
Context.newBuilder() | ||
.allowExperimentalOptions(true) | ||
.allowIO(IOAccess.ALL) | ||
.out(System.out) | ||
.err(System.err) | ||
.allowAllAccess(true) | ||
.build(); | ||
} | ||
|
||
@AfterClass | ||
public static void closeEnsoContext() throws Exception { | ||
if (ctx != null) { | ||
ctx.close(); | ||
} | ||
} | ||
|
||
@Test | ||
public void addTwoInt8ArrowArrays() { | ||
var arrow = ctx.getEngine().getLanguages().get("arrow"); | ||
assertNotNull("Arrow is available", arrow); | ||
var int8Constr = ctx.eval("arrow", "new[Int8]"); | ||
assertNotNull(int8Constr); | ||
|
||
var arrLength = 10; | ||
var builder1 = int8Constr.newInstance(arrLength); | ||
var builder2 = int8Constr.newInstance(arrLength); | ||
|
||
for (var i = 0; i < arrLength; i++) { | ||
var ni = arrLength - i - 1; | ||
var v = i * i; | ||
builder1.invokeMember("append", i, (byte) v); | ||
builder2.invokeMember("append", ni, (byte) v); | ||
} | ||
|
||
var arr1 = builder1.invokeMember("build"); | ||
assertEquals("Right size of arr1", arrLength, arr1.getArraySize()); | ||
var arr2 = builder2.invokeMember("build"); | ||
assertEquals("Right size of arr2", arrLength, arr2.getArraySize()); | ||
|
||
var int8Plus = ctx.eval("arrow", "+[Int8]"); | ||
var resultArr = int8Plus.execute(arr1, arr2); | ||
|
||
assertTrue("Result is an array", resultArr.hasArrayElements()); | ||
assertEquals("Right size", arrLength, resultArr.getArraySize()); | ||
|
||
for (var i = 0; i < arrLength; i++) { | ||
var ni = arrLength - i - 1; | ||
var v1 = resultArr.getArrayElement(i).asInt(); | ||
var v2 = resultArr.getArrayElement(ni).asInt(); | ||
|
||
assertEquals("Values at " + i + " and " + ni + " are the same", v1, v2); | ||
assertTrue("Values are always bigger than zero: " + v1, v1 > 0); | ||
} | ||
} | ||
} |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Use ContextUtils and declare dependency
runtime-language-arrow/Test
-->test-utils
. Being able to use context and project utils fromtest-utils
was the main motivation to movetest-utils
into separate project in #10112