Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run node in a different execution environment #11173

Merged
merged 27 commits into from
Oct 9, 2024
Merged
Show file tree
Hide file tree
Changes from 18 commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
108b230
DRAFT: execution config
4e6 Sep 19, 2024
d79a9fd
DRAFT: thread local
4e6 Sep 20, 2024
75a66c8
DRAFT: cache invalidation
4e6 Sep 24, 2024
3994fe3
refactor: RuntimeRecomputeTest
4e6 Sep 25, 2024
a55f906
update: language server api
4e6 Sep 25, 2024
169e386
doc: update language server api
4e6 Sep 26, 2024
95f2c13
update: truffle boundaries
4e6 Sep 26, 2024
a25b2e5
misc: javafmt
4e6 Sep 26, 2024
b5ed0c0
feat: ExpressionExecutionState
4e6 Oct 2, 2024
1b8b9f8
DRAFT: interrupt test
4e6 Oct 2, 2024
7df660f
Merge branch 'develop' into wip/db/10719-run-node-in-a-different-context
4e6 Oct 2, 2024
1be1988
test: interruption
4e6 Oct 2, 2024
cd365aa
fix: native image
4e6 Oct 2, 2024
ff97389
misc: javafmt
4e6 Oct 2, 2024
4662720
DRAFT: ContextThreadLocal
4e6 Oct 2, 2024
e77618c
Merge branch 'develop' into wip/db/10719-run-node-in-a-different-context
4e6 Oct 7, 2024
4250a3b
update: callbacks interface
4e6 Oct 7, 2024
0ccd2cd
misc: javafmt
4e6 Oct 7, 2024
c05283d
remove: getRootNode
4e6 Oct 8, 2024
51b3e6c
rename: execution environment getters
4e6 Oct 8, 2024
220605e
refactor: resetExecutionEnvironment
4e6 Oct 8, 2024
21f8675
misc: cleanup ExecutionConfig
4e6 Oct 8, 2024
4d29bfa
remove: ExecutionEnvironmentReference
4e6 Oct 8, 2024
c5b3c0c
refactor: ContextThreadLocal execution environment
4e6 Oct 8, 2024
6d64253
misc: fmt
4e6 Oct 8, 2024
9eeb917
fix: ExecutionConfig compilation
4e6 Oct 8, 2024
398827f
Merge branch 'develop' into wip/db/10719-run-node-in-a-different-context
4e6 Oct 9, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 29 additions & 1 deletion docs/language-server/protocol-language-server.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,8 @@ transport formats, please look [here](./protocol-architecture).
- [`MethodCall`](#methodcall)
- [`MethodPointer`](#methodpointer)
- [`ProfilingInfo`](#profilinginfo)
- [`ExecutionEnvironment`](#executionenvironment)
- [`ExpressionConfig`](#expressionConfig)
- [`ExpressionUpdate`](#expressionupdate)
- [`ExpressionUpdatePayload`](#expressionupdatepayload)
- [`VisualizationConfiguration`](#visualizationconfiguration)
Expand Down Expand Up @@ -343,6 +345,19 @@ The execution environment of Enso runtime.
type ExecutionEnvironment = Design | Live;
```

### `ExpressionConfig`

The expression configuration used in the recompute request.

```typescript
interface ExpressionConfig {
/** The expression identifier. */
expressionId: ExpressionId;
/** The execution environment that should be used to run this expression. */
executionEnvironment?: ExecutionEnvironment;
}
```

### `ExpressionUpdate`

An update about the computed expression.
Expand Down Expand Up @@ -3633,10 +3648,23 @@ May include a list of expressions for which caches should be invalidated.
interface ExecutionContextRecomputeParameters {
/** The execution context identifier. */
contextId: ContextId;
/** The expressions that will be invalidated before the execution. */

/** The expressions that will be invalidated before the execution.
*
* Only the provided expression ids are invalidated excluding the dependencies.
*/
invalidatedExpressions?: "all" | ExpressionId[];

/** The execution environment that will be used in the execution. */
executionEnvironment?: ExecutionEnvironment;

/** The execution configurations for particular expressions.
*
* The provided expressions will be invalidated from the cache with the
* dependencies. The result of the execution will stay in the cache until the
* cache is invalidated by editing the node or other means.
*/
expressionConfigs?: ExpressionConfig[];
}
```

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,8 @@ class RecomputeHandler(
session,
msg.params.contextId,
msg.params.invalidatedExpressions,
msg.params.executionEnvironment
msg.params.executionEnvironment,
msg.params.expressionConfigs.getOrElse(Seq())
)

override protected def positiveResponse(
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -193,7 +193,8 @@ final class ContextRegistry(
client,
contextId,
expressions,
environment
environment,
expressionConfigs
) =>
if (store.hasContext(client.clientId, contextId)) {
val handler =
Expand All @@ -210,7 +211,8 @@ final class ContextRegistry(
Api.RecomputeContextRequest(
contextId,
invalidatedExpressions,
environment.map(ExecutionEnvironments.toApi)
environment.map(ExecutionEnvironments.toApi),
expressionConfigs.map(_.toApi)
)
)
} else {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -98,12 +98,14 @@ object ContextRegistryProtocol {
* @param contextId execution context identifier
* @param invalidatedExpressions the expressions that should be invalidated
* @param executionEnvironment the environment that should be used for execution
* @param expressionConfigs the execution configurations for particular expressions
*/
case class RecomputeContextRequest(
rpcSession: JsonSession,
contextId: ContextId,
invalidatedExpressions: Option[InvalidatedExpressions],
executionEnvironment: Option[ExecutionEnvironments.ExecutionEnvironment]
executionEnvironment: Option[ExecutionEnvironments.ExecutionEnvironment],
expressionConfigs: Seq[ExpressionConfig]
)

/** A response about recomputing the context.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -108,7 +108,8 @@ object ExecutionApi {
case class Params(
contextId: ContextId,
invalidatedExpressions: Option[InvalidatedExpressions],
executionEnvironment: Option[ExecutionEnvironments.ExecutionEnvironment]
executionEnvironment: Option[ExecutionEnvironments.ExecutionEnvironment],
expressionConfigs: Option[Seq[ExpressionConfig]]
)

implicit
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
package org.enso.languageserver.runtime

import org.enso.polyglot.runtime.Runtime.Api

import java.util.UUID

/** The expression configuration used in the recompute request.
*
* @param expressionId the expression identifier
* @param executionEnvironment the execution environment used to run this expression
*/
case class ExpressionConfig(
expressionId: UUID,
executionEnvironment: Option[ExecutionEnvironments.ExecutionEnvironment]
) {

/** Convert this expression config to the runtime API. */
def toApi: Api.ExpressionConfig =
Api.ExpressionConfig(
expressionId,
executionEnvironment.map(ExecutionEnvironments.toApi)
)
}
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,7 @@ package org.enso.languageserver.runtime
import io.circe.syntax._
import io.circe.{Decoder, DecodingFailure, Encoder}

/** A request to invalidate expressions.
*/
/** A request to invalidate expressions. */
sealed trait InvalidatedExpressions

object InvalidatedExpressions {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -449,7 +449,8 @@ class ContextRegistryTest extends BaseServerTest with ReportLogsOnFailure {
Api.RecomputeContextRequest(
`contextId`,
Some(Api.InvalidatedExpressions.All()),
None
None,
Seq()
)
) =>
requestId
Expand Down Expand Up @@ -527,7 +528,92 @@ class ContextRegistryTest extends BaseServerTest with ReportLogsOnFailure {
Vector(`expressionId`)
)
),
None
None,
Seq()
)
) =>
requestId
case msg =>
fail(s"Unexpected message: $msg")
}
runtimeConnectorProbe.lastSender ! Api.Response(
requestId3,
Api.RecomputeContextResponse(contextId)
)
client.expectJson(json.ok(3))
}

"recompute with expression configs" in {
val client = getInitialisedWsClient()

// create context
client.send(json.executionContextCreateRequest(1))
val (requestId, contextId) =
runtimeConnectorProbe.receiveN(1).head match {
case Api.Request(requestId, Api.CreateContextRequest(contextId)) =>
(requestId, contextId)
case msg =>
fail(s"Unexpected message: $msg")
}
runtimeConnectorProbe.lastSender ! Api.Response(
requestId,
Api.CreateContextResponse(contextId)
)
client.expectJson(json.executionContextCreateResponse(1, contextId))

// push stack item
val expressionId = UUID.randomUUID()
client.send(json.executionContextPushRequest(2, contextId, expressionId))
val requestId2 =
runtimeConnectorProbe.receiveN(1).head match {
case Api.Request(
requestId,
Api.PushContextRequest(
`contextId`,
Api.StackItem.LocalCall(`expressionId`)
)
) =>
requestId
case msg =>
fail(s"Unexpected message: $msg")
}
runtimeConnectorProbe.lastSender ! Api.Response(
requestId2,
Api.PushContextResponse(contextId)
)
client.expectJson(json.ok(2))

// recompute
client.send(
json"""
{ "jsonrpc": "2.0",
"method": "executionContext/recompute",
"id": 3,
"params": {
"contextId": $contextId,
"expressionConfigs": [
{ "expressionId": $expressionId,
"executionEnvironment": "Live"
}
]
}
}
"""
)
val requestId3 =
runtimeConnectorProbe.receiveN(1).head match {
case Api.Request(
requestId,
Api.RecomputeContextRequest(
`contextId`,
None,
None,
Seq(
Api.ExpressionConfig(
`expressionId`,
Some(Api.ExecutionEnvironment.Live())
)
)
)
) =>
requestId
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
import com.oracle.truffle.api.instrumentation.EventBinding;
import com.oracle.truffle.api.instrumentation.ExecutionEventNodeFactory;
import com.oracle.truffle.api.interop.TruffleObject;
import com.oracle.truffle.api.nodes.RootNode;
import java.util.UUID;

public interface IdExecutionService {
Expand All @@ -21,6 +22,11 @@ public abstract class Info {
*/
public abstract Object getResult();

/**
* @return the root of this node.
*/
public abstract RootNode getRootNode();
Copy link
Member

@JaroslavTulach JaroslavTulach Oct 4, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, we do not want to expose getRootNode(). It is only used by:

EnsoContext context = EnsoContext.get(info.getRootNode());
originalExecutionEnvironment = context.getExecutionEnvironment();

and corresponding setter. Just call it getExecutionEnvironment() and setExecutionEnvironment.

Btw. There already is setExceptionEnvironment below why we have two ways of doing the same thing? Can you document the methods and describe different among them?


/**
* @return {@code true} when the result is panic, {@code false} otherwise.
*/
Expand Down Expand Up @@ -69,6 +75,8 @@ public interface Callbacks {
* the execution and return the value as a result.
*/
Object onFunctionReturn(Info info);

Object getExecutionEnvironment(Info info);
JaroslavTulach marked this conversation as resolved.
Show resolved Hide resolved
}

/**
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -691,6 +691,16 @@ object Runtime {
}
}

/** The configuration of how to execute the expression.
*
* @param expressionId the expression identifier
* @param executionEnvironment the execution environment for the expression
*/
sealed case class ExpressionConfig(
expressionId: ExpressionId,
executionEnvironment: Option[ExecutionEnvironment]
)

/** The notification about the execution status.
*
* @param contextId the context's id
Expand Down Expand Up @@ -914,12 +924,14 @@ object Runtime {
* @param expressions the selector specifying which expressions should be
* recomputed.
* @param executionEnvironment the environment used for execution
* @param expressionConfigs execution configurations for selected expressions
*/
@named("recomputeContextRequest")
final case class RecomputeContextRequest(
contextId: ContextId,
expressions: Option[InvalidatedExpressions],
executionEnvironment: Option[ExecutionEnvironment]
executionEnvironment: Option[ExecutionEnvironment],
expressionConfigs: Seq[ExpressionConfig]
) extends ApiRequest

/** A response sent from the server upon handling the
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
package org.enso.interpreter.instrument;

import java.util.Collections;
import java.util.HashMap;
import java.util.Map;
import java.util.UUID;
import org.enso.interpreter.runtime.state.ExecutionEnvironment;
import org.enso.polyglot.runtime.Runtime$Api$ExecutionEnvironment;
import org.enso.polyglot.runtime.Runtime$Api$ExpressionConfig;
import scala.Option;
import scala.collection.immutable.Seq;

/**
* The program execution config.
*
* @param executionEnvironment the global execution environment of the program
* @param expressionConfigs execution configs for each expression
*/
public record ExecutionConfig(
ExecutionEnvironment executionEnvironment, Map<UUID, ExecutionEnvironment> expressionConfigs) {

public static ExecutionConfig empty() {
return new ExecutionConfig(null, Collections.emptyMap());
}

@SuppressWarnings("unchecked")
public static ExecutionConfig create(
Object executionEnvironmentOption1, Object expressionConfigs1) {
Map<UUID, ExecutionEnvironment> expressionConfigsBuilder = new HashMap<>();
JaroslavTulach marked this conversation as resolved.
Show resolved Hide resolved
Option<Runtime$Api$ExecutionEnvironment> executionEnvironmentOption =
(Option<Runtime$Api$ExecutionEnvironment>) executionEnvironmentOption1;
Seq<Runtime$Api$ExpressionConfig> expressionConfigs =
(Seq<Runtime$Api$ExpressionConfig>) expressionConfigs1;

expressionConfigs.foreach(
expressionConfig -> {
expressionConfig
.executionEnvironment()
.foreach(
executionEnvironment -> {
expressionConfigsBuilder.put(
expressionConfig.expressionId(),
ExecutionEnvironment.forName(executionEnvironment.name()));
return null;
});
return null;
});

ExecutionEnvironment executionEnvironment =
executionEnvironmentOption
.map(env -> ExecutionEnvironment.forName(env.name()))
.getOrElse(() -> null);

return new ExecutionConfig(executionEnvironment, expressionConfigsBuilder);
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
package org.enso.interpreter.instrument;

import java.util.HashMap;
import java.util.Map;
import java.util.UUID;
import org.enso.interpreter.runtime.state.ExecutionEnvironment;

public final class ExpressionExecutionState {

private final Map<UUID, ExecutionEnvironment> expressionConfigs;

public ExpressionExecutionState() {
this.expressionConfigs = new HashMap<>();
}

public ExpressionExecutionState(Map<UUID, ExecutionEnvironment> expressionConfigs) {
this.expressionConfigs = expressionConfigs;
}

public void setExpressionConfigs(Map<UUID, ExecutionEnvironment> expressionConfigs) {
this.expressionConfigs.putAll(expressionConfigs);
}

public void setExpressionExecuted(UUID expressionId) {
expressionConfigs.remove(expressionId);
}

public ExecutionEnvironment getExpressionExecutionEnvironment(UUID expressionId) {
return expressionConfigs.get(expressionId);
}
}
Loading
Loading