Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rename fixes, get data from xml docs and sample #3083 #3086 #3090

Merged
merged 3 commits into from
Jul 17, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,7 @@ All of this combined makes it hard to maintain and debug this type of loops.

== Pipeline and Workflow executor

The xref:pipeline/transforms/workflowexecutor.adoc[Workflow executor^] and xref:pipeline/transforms/pipelineexecutor.adoc[Pipeline executor^] offer flexible and elegant ways to run workflows and pipelines from within an existing pipeline.
The xref:pipeline/transforms/workflow-executor.adoc[Workflow executor^] and xref:pipeline/transforms/pipeline-executor.adoc[Pipeline executor^] offer flexible and elegant ways to run workflows and pipelines from within an existing pipeline.

=== Pipeline Executor

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,14 @@ The Get Data From XML transform provides the ability to read data from any type

Get Data From XML can read data from 3 kind of sources (files, stream and url) in 2 modes (user can define files and urls at static mode or in a dynamic way).

See also:

* xref:pipeline/transforms/xmlinputstream.adoc[XML Input Stream (StAX)] transform.

Samples (Samples project):

* transforms/get-data-from-xml.hpl

|
== Supported Engines
[%noheader,cols="2,1a",frame=none, role="table-supported-engines"]
Expand All @@ -47,7 +55,7 @@ Get Data From XML can read data from 3 kind of sources (files, stream and url) i
The files tab is where you define the location of the XML files from which you want to read.
The table below contains options associated with the Files tab.

[options="header"]
[options="header", cols="1,3"]
|===
|Option|Description
|Transform name|Name of the transform.
Expand All @@ -58,8 +66,9 @@ These are read.
* Read source as URL : the previous transform is giving URLs in a certain field in the input stream.
These are read.
* Get XML source from a field : specify the field to read XML, filename or URL from.
|File or directory|Specifies the location and/or name of the input text file.
Note: Click Add to add the file/directory/wildcard combination to the list of selected files (grid) below.
|File or directory a|Specifies the location and/or name of the input text file.

TIP: Click Add to add the file/directory/wildcard combination to the list of selected files (grid) below.
|Regular expression|Specifies the regular expression you want to use to select the files in the directory specified in the previous option.
|Selected Files|Contains a list of selected files (or wildcard selections) and a property specifying if file is required or not.
If a file is required and it is not found, an error is generated;otherwise, the file name is skipped.
Expand All @@ -68,7 +77,7 @@ If a file is required and it is not found, an error is generated;otherwise, the

=== Content Tab

[options="header"]
[options="header", cols="1,3"]
|===
|Option|Description
|Settings a|
Expand All @@ -88,7 +97,7 @@ Please see the Example 1 to see how it works.
Check this if you want to ignore those altogether.
* Do not raise an error if no file: Don't raise a stink if no files are found.
* Limit : Limits the number of rows to this number (zero (0) means all rows).
* Prune path to handle large files: almost the same value as the "Loop XPath" property with some exceptions, see Get Data from XML - Handling Large Files for more details.
* Prune path to handle large files: almost the same value as the "Loop XPath" property with some exceptions. Use the prune path to speed up processing of large files, or consider using the xref:pipeline/transforms/xmlinputstream.adoc[XML Input Stream (StAX)] transform.
Note that you can use this parameter to avoid multiple HTTP URL requests.

|Additional fields a|
Expand All @@ -105,12 +114,17 @@ A unique list is being kept in memory that can be used in the next workflow acti

=== Fields Tab

[options="header"]
[options="header", cols="1,3"]
|===
|Option|Description
|Name|The name of the output field
|XPath|The path to the element node or attribute to read
|Element|The element type to read: Node or Attribute
|Result Type a|"Value of" or "Single node"

* Value of: retrieve the value of your XPath expression, e.g. the contents of an element or the value of an attribute.
* Single node: retrieve the XML contents returned by an XPath expression. Contrary to "Value of", the result of "Single node" is an XML snippet.

|Type|The data type to convert to
|Format|The format or conversion mask to use in the data type conversion
|Length|The length of the output data type
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -56,10 +56,11 @@ See also:

* The xref:workflow/actions/workflow.adoc[Workflow action] that executes a sub-workflow from a workflow.
* The xref:workflow/actions/pipeline.adoc[Pipeline action] that executes a pipeline from a workflow.
* The xref:pipeline/transforms/workflowexecutor.adoc[Workflow Executor transform] that executes a workflow from a pipeline.
* The xref:pipeline/transforms/workflow-executor.adoc[Workflow Executor transform] that executes a workflow from a pipeline.
* The xref:how-to-guides/loops-in-apache-hop.adoc[Loops in Apache Hop] how-to guide.

Samples (samples project):

* loops/pipeline-executor.hpl

== Options
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -52,10 +52,11 @@ See also:

* The xref:workflow/actions/workflow.adoc[Workflow action] that executes a sub-workflow from a workflow.
* The xref:workflow/actions/pipeline.adoc[Pipeline action] that executes a pipeline from a workflow.
* The xref:pipeline/transforms/pipelineexecutor.adoc[Pipeline Executor transform] that executes a sub-pipeline from a pipeline.
* The xref:pipeline/transforms/pipeline-executor.adoc[Pipeline Executor transform] that executes a sub-pipeline from a pipeline.
* The xref:how-to-guides/loops-in-apache-hop.adoc[Loops in Apache Hop] how-to guide.

Samples (samples project)

* loops/workflow-executor.hpl

== Options
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ under the License.
Avoid looping in workflows.
The easiest way to loop over a set of values, rows, files, ... is to use an Executor transform.

* xref:pipeline/transforms/pipelineexecutor.adoc[Pipeline Executor] : run a pipeline for each input row
* xref:pipeline/transforms/workflowexecutor.adoc[Workflow Executor] : run a workflow for each input row
* xref:pipeline/transforms/pipeline-executor.adoc[Pipeline Executor] : run a pipeline for each input row
* xref:pipeline/transforms/workflow-executor.adoc[Workflow Executor] : run a workflow for each input row

You can nicely map field values to parameters of the pipeline or workflow making loops a breeze.
Original file line number Diff line number Diff line change
Expand Up @@ -35,8 +35,8 @@ For this example, the Pipeline action defines which pipeline to run to populate
See also:

* The xref:workflow/actions/workflow.adoc[Workflow action] that executes a sub-workflow from a workflow.
* The xref:pipeline/transforms/workflowexecutor.adoc[Workflow Executor transform] that executes a workflow from a pipeline.
* The xref:pipeline/transforms/pipelineexecutor.adoc[Pipeline Executor transform] that executes a sub-pipeline from a pipeline.
* The xref:pipeline/transforms/workflow-executor.adoc[Workflow Executor transform] that executes a workflow from a pipeline.
* The xref:pipeline/transforms/pipeline-executor.adoc[Pipeline Executor transform] that executes a sub-pipeline from a pipeline.

== Options

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -40,8 +40,8 @@ It is better to create smaller workflows and compose them.
See also:

* The xref:workflow/actions/pipeline.adoc[Pipeline action] that executes a pipeline from a workflow.
* The xref:pipeline/transforms/workflowexecutor.adoc[Workflow Executor transform] that executes a workflow from a pipeline.
* The xref:pipeline/transforms/pipelineexecutor.adoc[Pipeline Executor transform] that executes a sub-pipeline from a pipeline.
* The xref:pipeline/transforms/workflow-executor.adoc[Workflow Executor transform] that executes a workflow from a pipeline.
* The xref:pipeline/transforms/pipeline-executor.adoc[Pipeline Executor transform] that executes a sub-pipeline from a pipeline.

== Options

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -175,7 +175,7 @@ public void setInputDataSet(HopGuiPipelineTransformContext context) {
Collections.sort(setNames);
EnterSelectionDialog esd =
new EnterSelectionDialog(
hopGui.getDisplay().getActiveShell(),
hopGui.getShell(),
setNames.toArray(new String[setNames.size()]),
BaseMessages.getString(PKG, "TestingGuiPlugin.ContextAction.SetInputDataset.Header"),
BaseMessages.getString(
Expand All @@ -192,7 +192,7 @@ public void setInputDataSet(HopGuiPipelineTransformContext context) {
}
} catch (Exception e) {
new ErrorDialog(
hopGui.getDisplay().getActiveShell(),
hopGui.getShell(),
BaseMessages.getString(
PKG, "TestingGuiPlugin.ContextAction.SetInputDataset.Error.Header"),
BaseMessages.getString(
Expand Down Expand Up @@ -230,7 +230,7 @@ private boolean setInputDataSetOnTransform(
String[] setFieldNames = setFields.getFieldNames();

EnterMappingDialog mappingDialog =
new EnterMappingDialog(hopGui.getDisplay().getActiveShell(), setFieldNames, transformFieldNames);
new EnterMappingDialog(hopGui.getShell(), setFieldNames, transformFieldNames);
List<SourceToTargetMapping> mappings = mappingDialog.open();
if (mappings == null) {
return false;
Expand All @@ -249,7 +249,7 @@ private boolean setInputDataSetOnTransform(
}
EditRowsDialog orderDialog =
new EditRowsDialog(
hopGui.getDisplay().getActiveShell(),
hopGui.getShell(),
SWT.NONE,
BaseMessages.getString(PKG, "TestingGuiPlugin.SortOrder.Title"),
BaseMessages.getString(PKG, "TestingGuiPlugin.SortOrder.Message"),
Expand Down Expand Up @@ -330,7 +330,7 @@ public void clearInputDataSet(HopGuiPipelineTransformContext context) {
context.getPipelineGraph().updateGui();
} catch (Exception e) {
new ErrorDialog(
hopGui.getDisplay().getActiveShell(),
hopGui.getShell(),
BaseMessages.getString(
PKG, "TestingGuiPlugin.ContextAction.ClearInputDataset.Error.Header"),
BaseMessages.getString(
Expand All @@ -349,7 +349,7 @@ private boolean checkTestPresent(HopGui hopGui, PipelineMeta pipelineMeta) {
// there is no test defined of selected in the pipeline.
// Show a warning
//
MessageBox box = new MessageBox(hopGui.getDisplay().getActiveShell(), SWT.OK | SWT.ICON_INFORMATION);
MessageBox box = new MessageBox(hopGui.getShell(), SWT.OK | SWT.ICON_INFORMATION);
box.setMessage(
BaseMessages.getString(PKG, "TestingGuiPlugin.ContextAction.CheckTestPresent.Message"));
box.setText(
Expand Down Expand Up @@ -395,7 +395,7 @@ public void setGoldenDataSet(HopGuiPipelineTransformContext context) {
Collections.sort(setNames);
EnterSelectionDialog esd =
new EnterSelectionDialog(
hopGui.getDisplay().getActiveShell(),
hopGui.getShell(),
setNames.toArray(new String[setNames.size()]),
BaseMessages.getString(PKG, "TestingGuiPlugin.ContextAction.SetGoldenDataset.Header"),
BaseMessages.getString(
Expand All @@ -412,7 +412,7 @@ public void setGoldenDataSet(HopGuiPipelineTransformContext context) {
}
} catch (Exception e) {
new ErrorDialog(
hopGui.getDisplay().getActiveShell(),
hopGui.getShell(),
BaseMessages.getString(
PKG, "TestingGuiPlugin.ContextAction.SetGoldenDataset.Error.Header"),
BaseMessages.getString(
Expand Down Expand Up @@ -545,7 +545,7 @@ public void clearGoldenDataSet(HopGuiPipelineTransformContext context) {
saveUnitTest(variables, hopGui.getMetadataProvider(), currentUnitTest, pipelineMeta);
} catch (Exception e) {
new ErrorDialog(
hopGui.getDisplay().getActiveShell(),
hopGui.getShell(),
BaseMessages.getString(
PKG, "TestingGuiPlugin.ContextAction.ClearGoldenDataset.Error.Header"),
BaseMessages.getString(
Expand Down Expand Up @@ -655,7 +655,7 @@ public void createDataSetFromTransform(HopGuiPipelineTransformContext context) {
// to use it on the transform?
//
MessageBox box =
new MessageBox(hopGui.getDisplay().getActiveShell(), SWT.YES | SWT.NO | SWT.CANCEL | SWT.ICON_QUESTION);
new MessageBox(hopGui.getShell(), SWT.YES | SWT.NO | SWT.CANCEL | SWT.ICON_QUESTION);
box.setText(
BaseMessages.getString(
PKG, "TestingGuiPlugin.ContextAction.CreateDataset.DatasetType.Header"));
Expand Down Expand Up @@ -689,7 +689,7 @@ public void createDataSetFromTransform(HopGuiPipelineTransformContext context) {
}
} catch (Exception e) {
new ErrorDialog(
hopGui.getDisplay().getActiveShell(),
hopGui.getShell(),
BaseMessages.getString(PKG, "TestingGuiPlugin.ContextAction.CreateDataset.Error.Header"),
BaseMessages.getString(PKG, "TestingGuiPlugin.ContextAction.CreateDataset.Error.Message"),
e);
Expand Down Expand Up @@ -719,7 +719,7 @@ public void writeTransformDataToDataSet(HopGuiPipelineTransformContext context)
PipelineMeta pipelineMeta = context.getPipelineMeta();

if (pipelineMeta.hasChanged()) {
MessageBox box = new MessageBox(hopGui.getDisplay().getActiveShell(), SWT.OK | SWT.ICON_INFORMATION);
MessageBox box = new MessageBox(hopGui.getShell(), SWT.OK | SWT.ICON_INFORMATION);
box.setText(
BaseMessages.getString(
PKG, "TestingGuiPlugin.ContextAction.Run.SavePipelineDialog.Header"));
Expand All @@ -739,7 +739,7 @@ public void writeTransformDataToDataSet(HopGuiPipelineTransformContext context)
Collections.sort(setNames);
EnterSelectionDialog esd =
new EnterSelectionDialog(
hopGui.getDisplay().getActiveShell(),
hopGui.getShell(),
setNames.toArray(new String[setNames.size()]),
BaseMessages.getString(PKG, "TestingGuiPlugin.ContextAction.Run.ActionList.Item1"),
BaseMessages.getString(PKG, "TestingGuiPlugin.ContextAction.Run.ActionList.Item2"));
Expand All @@ -765,7 +765,7 @@ public void writeTransformDataToDataSet(HopGuiPipelineTransformContext context)
// Ask for the mapping between the output row and the data set field
//
EnterMappingDialog mappingDialog =
new EnterMappingDialog(hopGui.getDisplay().getActiveShell(), transformFields, setFields);
new EnterMappingDialog(hopGui.getShell(), transformFields, setFields);
List<SourceToTargetMapping> mapping = mappingDialog.open();
if (mapping == null) {
return;
Expand All @@ -790,7 +790,7 @@ public void writeTransformDataToDataSet(HopGuiPipelineTransformContext context)

} catch (Exception e) {
new ErrorDialog(
hopGui.getDisplay().getActiveShell(),
hopGui.getShell(),
BaseMessages.getString(PKG, "TestingGuiPlugin.ContextAction.Run.Error.Header"),
BaseMessages.getString(PKG, "TestingGuiPlugin.ContextAction.Run.Error.Message"),
e);
Expand Down Expand Up @@ -850,7 +850,7 @@ public void detachUnitTest() {
pipelineGraph.updateGui();
} catch (Exception e) {
new ErrorDialog(
hopGui.getDisplay().getActiveShell(),
hopGui.getShell(),
BaseMessages.getString(PKG, "TestingGuiPlugin.ToolbarElement.Detach.Error.Header"),
BaseMessages.getString(PKG, "TestingGuiPlugin.ToolbarElement.Detach.Error.Message"),
e);
Expand Down Expand Up @@ -935,7 +935,7 @@ public void deleteUnitTest() {
return; // doesn't exist
}

MessageBox box = new MessageBox(hopGui.getDisplay().getActiveShell(), SWT.YES | SWT.NO | SWT.ICON_QUESTION);
MessageBox box = new MessageBox(hopGui.getShell(), SWT.YES | SWT.NO | SWT.ICON_QUESTION);
box.setMessage(
BaseMessages.getString(
PKG,
Expand All @@ -960,7 +960,7 @@ public void deleteUnitTest() {
refreshUnitTestsList();
} catch (Exception e) {
new ErrorDialog(
hopGui.getDisplay().getActiveShell(),
hopGui.getShell(),
BaseMessages.getString(PKG, "TestingGuiPlugin.ToolbarElement.Delete.Error.Header"),
BaseMessages.getString(PKG, "TestingGuiPlugin.ToolbarElement.Delete.Error.Message"),
e);
Expand Down Expand Up @@ -1021,7 +1021,7 @@ public void editPipelineUnitTest() {
}
} catch (Exception e) {
new ErrorDialog(
hopGui.getDisplay().getActiveShell(),
hopGui.getShell(),
BaseMessages.getString(PKG, "TestingGuiPlugin.ToolbarElement.UnitTest.Error.Header"),
BaseMessages.getString(
PKG, "TestingGuiPlugin.ToolbarElement.UnitTest.Error.Message", unitTestName),
Expand Down Expand Up @@ -1144,7 +1144,7 @@ public void selectUnitTest() {
}
} catch (Exception e) {
new ErrorDialog(
hopGui.getDisplay().getActiveShell(),
hopGui.getShell(),
BaseMessages.getString(
PKG, "TestingGuiPlugin.ToolbarElement.GetUnitTestList.Error.Header"),
BaseMessages.getString(
Expand Down Expand Up @@ -1304,7 +1304,7 @@ private void tweakUnitTestTransform(
hopGui.getActiveFileTypeHandler().updateGui();
} catch (Exception exception) {
new ErrorDialog(
hopGui.getDisplay().getActiveShell(),
hopGui.getShell(),
BaseMessages.getString(PKG, "TestingGuiPlugin.TweakUnitTestTransform.Error.Header"),
BaseMessages.getString(
PKG,
Expand Down Expand Up @@ -1347,15 +1347,15 @@ public RowMetaAndData selectUnitTestFromAllTests() {
//
SelectRowDialog dialog =
new SelectRowDialog(
hopGui.getDisplay().getActiveShell(), new Variables(), SWT.DIALOG_TRIM | SWT.MAX | SWT.RESIZE, rows);
hopGui.getShell(), new Variables(), SWT.DIALOG_TRIM | SWT.MAX | SWT.RESIZE, rows);
RowMetaAndData selection = dialog.open();
if (selection != null) {
return selection;
}
return null;
} catch (Exception e) {
new ErrorDialog(
hopGui.getDisplay().getActiveShell(),
hopGui.getShell(),
BaseMessages.getString(PKG, "TestingGuiPlugin.SelectUnitTestFromAllTests.Error.Header"),
BaseMessages.getString(PKG, "TestingGuiPlugin.SelectUnitTestFromAllTests.Error.Message"),
e);
Expand Down
Loading