Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

IDE uses new visualization API #3661

Merged
merged 28 commits into from
Sep 1, 2022
Merged
Show file tree
Hide file tree
Changes from 19 commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,8 +58,9 @@
- [Fixed error handling during startup.][3648] This prevents entering IDE into a
"zombie" state, where processes were started but not visible to user. They
could cause issues with starting further IDE instances.
- [New nodes are created in the project source when the searcher is opened
and a new node is created.][5250]
- [New nodes are created in the project source when the searcher is opened and a
new node is created.][3645]
- [IDE uses new visualization API.][3661]

#### EnsoGL (rendering engine)

Expand Down Expand Up @@ -286,7 +287,8 @@
[3643]: https://github.com/enso-org/enso/pull/3643
[3644]: https://github.com/enso-org/enso/pull/3644
[3648]: https://github.com/enso-org/enso/pull/3648
[5250]: https://github.com/enso-org/enso/pull/5250
[3645]: https://github.com/enso-org/enso/pull/3645
[3661]: https://github.com/enso-org/enso/pull/3661

#### Enso Compiler

Expand Down
24 changes: 18 additions & 6 deletions app/gui/controller/engine-protocol/src/language_server/tests.rs
Original file line number Diff line number Diff line change
Expand Up @@ -424,12 +424,18 @@ fn test_execution_context() {
);
let visualisation_id = uuid::Uuid::default();
let expression_id = uuid::Uuid::default();
let expression = "1 + 1".to_string();
let visualisation_module = "[Foo.Bar.Baz]".to_string();
let visualization_function = "foo";
let visualization_module = "[Foo.Bar.Baz]";
let expression = MethodPointer {
module: visualization_module.to_string(),
defined_on_type: visualization_module.to_string(),
name: visualization_function.to_string(),
};
let positional_arguments_expressions = vec![];
let visualisation_config = VisualisationConfiguration {
execution_context_id: context_id,
expression,
visualisation_module,
positional_arguments_expressions,
};
test_request(
|client| {
Expand Down Expand Up @@ -459,12 +465,18 @@ fn test_execution_context() {
unit_json.clone(),
(),
);
let expression = "1 + 1".to_string();
let visualisation_module = "[Foo.Bar.Baz]".to_string();
let visualization_function = "foo";
let visualization_module = "[Foo.Bar.Baz]";
let expression = MethodPointer {
module: visualization_module.to_string(),
defined_on_type: visualization_module.to_string(),
name: visualization_function.to_string(),
};
let positional_arguments_expressions = vec![];
let visualisation_config = VisualisationConfiguration {
execution_context_id: context_id,
expression,
visualisation_module,
positional_arguments_expressions,
};
test_request(
|client| client.modify_visualisation(&visualisation_id, &visualisation_config),
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -649,12 +649,12 @@ pub type ExpressionId = Uuid;
#[serde(rename_all = "camelCase")]
#[allow(missing_docs)]
pub struct VisualisationConfiguration {
#[allow(missing_docs)]
/// An execution context of the visualization.
pub execution_context_id: ContextId,
/// A qualified name of the module containing the expression which creates visualisation.
pub visualisation_module: String,
/// An enso lambda that will transform the data into expected format, i.e. `a -> a.json`.
pub expression: String,
/// An enso function that will transform the data into expected format.
pub expression: MethodPointer,
/// A list of arguments to pass to the visualization expression.
pub positional_arguments_expressions: Vec<String>,
}

/// Used to enter deeper in the execution context stack. In general, all consequent stack items
Expand Down
111 changes: 48 additions & 63 deletions app/gui/docs/product/visualizations.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,22 +11,22 @@ tags: [product]

Visualizations have two main purposes:

- **Display results of nodes**
Each node can be assigned with one or more visualization. After a node
computes its new value, the visualization shows it in an understandable way to
the user. Please note that a single node can be assigned with multiple
visualizations at the same time. For example, a node might want to display a
map of locations, and their list at the same time next to each other.

- **Provide interactive way to generate new data**
In a widget mode (described in detail later), visualizations provide users
with an interactive GUI to define data. For example, a map visualization can
both display locations, as well as allowing the user to pick locations by
clicking with a mouse. Similarly, the histogram can both display a list of
numbers, and can be manually draw with the mouse producing such a list.
Several numbers can be visualized as a table of sliders, which can also be
used to interactively generate a table of numbers. Image visualizations can
behave like an image editor, etc.
- **Display results of nodes** Each node can be assigned with one or more
visualization. After a node computes its new value, the visualization shows it
in an understandable way to the user. Please note that a single node can be
assigned with multiple visualizations at the same time. For example, a node
might want to display a map of locations, and their list at the same time next
to each other.

- **Provide interactive way to generate new data** In a widget mode (described
in detail later), visualizations provide users with an interactive GUI to
define data. For example, a map visualization can both display locations, as
well as allowing the user to pick locations by clicking with a mouse.
Similarly, the histogram can both display a list of numbers, and can be
manually draw with the mouse producing such a list. Several numbers can be
visualized as a table of sliders, which can also be used to interactively
generate a table of numbers. Image visualizations can behave like an image
editor, etc.

## Visualization Display Forms

Expand All @@ -37,34 +37,33 @@ Visualizations can be displayed in the following ways:
you move the node, the visualization moves as well. This mode can be toggled
by tapping the spacebar.

- **Fullscreen**
Visualization attached to node can grow (animate) to ocupy full IDE visual
space. This mode can be triggered on the recently selected node (in case many
nodes are selected, the last selected node will be used) by either pressing
keeping the spacebar pressed for longer than approx 0.5s, or by tapping it
twice. In the former case, the visualization shrinks to each original form
whenever we release space, in the later, whenever we press space again.

- **Detached**
Visualizations attached to nodes can be detached, scaled, and placed freely
across the visual canvas (we might introduce a special place where you can put
such visualizations). This is useful when defining dashboards or reports. We
also plan to provide a notebook-like experience where you can write text mixed
with visualizations (including widgets for an interactive experience).

- **Widgets**
In this mode visualizations behave like nodes but do not display expressions.
They have one input and one output port. If the input port is connected, the
visualization displays its value and passes its to the output port. In case it
is not connected, the visualization becomes an interactive widget allowing the
user to specify data. For example, a map visualization will allow the user to
manually pick locations. After each change, the new locations will be sent to
the output port. Under the hood, widgets are represented as nodes and their
code lines are assigned with a dedicated "visualization" metadata.
Visualizations generate expressions always in the form of `name = data`, where
data is a hardcoded data produced from the visualization. For example, when
user clicks the map to define locations, the data could be a string literal
containing locations encoded in JSON.
- **Fullscreen** Visualization attached to node can grow (animate) to occupy
full IDE visual space. This mode can be triggered on the recently selected
node (in case many nodes are selected, the last selected node will be used) by
either pressing keeping the spacebar pressed for longer than approx 0.5s, or
by tapping it twice. In the former case, the visualization shrinks to each
original form whenever we release space, in the later, whenever we press space
again.

- **Detached** Visualizations attached to nodes can be detached, scaled, and
placed freely across the visual canvas (we might introduce a special place
where you can put such visualizations). This is useful when defining
dashboards or reports. We also plan to provide a notebook-like experience
where you can write text mixed with visualizations (including widgets for an
interactive experience).

- **Widgets** In this mode visualizations behave like nodes but do not display
expressions. They have one input and one output port. If the input port is
connected, the visualization displays its value and passes its to the output
port. In case it is not connected, the visualization becomes an interactive
widget allowing the user to specify data. For example, a map visualization
will allow the user to manually pick locations. After each change, the new
locations will be sent to the output port. Under the hood, widgets are
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That almost sounds like "dynamic widgets" to me!

represented as nodes and their code lines are assigned with a dedicated
"visualization" metadata. Visualizations generate expressions always in the
form of `name = data`, where data is a hardcoded data produced from the
visualization. For example, when user clicks the map to define locations, the
data could be a string literal containing locations encoded in JSON.

### Choosing a Visualization Type.

Expand Down Expand Up @@ -201,24 +200,11 @@ In particular:
as visualizations. The superclass defines a default constructor and a set of
utilities:

- #### Method `setPreprocessorCode(code)`
Set an Enso code which will be evaluated on the server-side before sending
data to visualization. If not called, a default unspecified code is used
- #### Method `setPreprocessor(module,method)`
Set an Enso method which will be evaluated on the server-side before sending
data to visualization. If not called, a default unspecified method is used
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

default unspecified? That's a strange formulation in a specification of the behavior ;-)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why? Specification explicitly opts-out from defining this.
There'll be some code present, but custom visualization should not make any assumptions about it. Thanks to that, we can adjust it without breaking public API.

that will provide some JSON representation of the value. See
[Lazy visualizations](#lazy-visualizations) section for details.
- #### Method `setPreprocessorModule(module)`
Define in which module's context the preprocessor code should be evaluated.
If not called, the `Main` module of the project that defines visualization
will be used. See [Lazy visualizations](#lazy-visualizations) section for
details.
- #### Method `setPreprocessor(code,mode)`
Set both code and its module context at once. If both need to be updated,
using this method can save an update processing and needless evaluation.
Note that using both `setPreprocessorCode` and `setPreprocessorModule` from
the visualization's custom constructor will not cause any unnecessary
updates, as the preprocessor is applied only after visualization is fully
constructed. See [Lazy visualizations](#lazy-visualizations) section for
details.
- #### Field `dom`
It is initialized in the constructor to the DOM symbol used to host the
visualization content. Users are free to modify the DOM element, including
Expand Down Expand Up @@ -297,9 +283,8 @@ visualization you may use the `setPreprocessor` method). This code defines an
Enso function, which will be run by the compiler on data the visualization is
attached to. Only the results of this code will be sent to the GUI. In the case
of the JSON input format, the result of the call should be a valid JSON string.
The code will be evaluated in the context of the `Main` module in the project
where visualization is defined - you may use any symbol defined or imported in
that module.
The code will be evaluated in the context of the module where the preprocessor
method is defined - you may use any symbol defined or imported in that module.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How can one pass arguments to the method? #3655 needs to be able to invoke the Enso engine part of the visualization with different arguments.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd expect that arguments are part of the expression, as the arguments change we update the expression.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, it was removed. Then some mechanism for arguments is necessary,


For example, imagine you want to display a heatmap of 10 million points on a
map, and these points change rapidly. Sending such an amount of information via
Expand Down
11 changes: 5 additions & 6 deletions app/gui/src/controller/graph/executed.rs
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ use crate::prelude::*;
use crate::model::execution_context::ComponentGroup;
use crate::model::execution_context::ComputedValueInfoRegistry;
use crate::model::execution_context::LocalCall;
use crate::model::execution_context::QualifiedMethodPointer;
use crate::model::execution_context::Visualization;
use crate::model::execution_context::VisualizationId;
use crate::model::execution_context::VisualizationUpdateData;
Expand Down Expand Up @@ -142,10 +143,9 @@ impl Handle {
pub fn modify_visualization(
&self,
id: VisualizationId,
expression: Option<String>,
module: Option<model::module::QualifiedName>,
method_pointer: Option<QualifiedMethodPointer>,
) -> BoxFuture<FallibleResult> {
self.execution_ctx.modify_visualization(id, expression, module)
self.execution_ctx.modify_visualization(id, method_pointer)
}

/// See [`model::ExecutionContext::detach_visualization`].
Expand All @@ -171,10 +171,9 @@ impl Handle {
pub async fn set_visualization_preprocessor(
&self,
id: VisualizationId,
code: String,
module: model::module::QualifiedName,
method_pointer: QualifiedMethodPointer,
) -> FallibleResult {
self.execution_ctx.modify_visualization(id, Some(code), Some(module)).await
self.execution_ctx.modify_visualization(id, Some(method_pointer)).await
}

/// See [`model::ExecutionContext::component_groups`].
Expand Down
Loading