Skip to content

Commit

Permalink
docs: Added examples for batching and multithreading
Browse files Browse the repository at this point in the history
  • Loading branch information
jruaux committed Jun 19, 2023
1 parent 32c74e3 commit d3b94aa
Showing 1 changed file with 18 additions and 3 deletions.
21 changes: 18 additions & 3 deletions docs/guide/src/docs/asciidoc/architecture.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,12 @@ If the target is Redis, writing is done in a single command {link_redis_pipeline
You can change the batch size (and hence pipeline size) using the `--batch` option.
The optimal batch size in terms of throughput depends on many factors like record size and command types (see {link_pipeline_tuning} for details).

.`--batch` example
[source]
----
include::{testdir}/faker-import-tsadd[]
----

[[_architecture_threads]]
== Multi-threading

Expand All @@ -31,14 +37,23 @@ Here, only one reader is being accessed from multiple threads.

To set the number of threads use the `--threads` option.

.`--threads` example
[source]
----
include::{testdir}/db-import-postgresql-multithreaded
----

[[_architecture_processing]]
== Processing

Processors are applied to records in the following order:

* Transforms
* Regular expressions
* Filters

[[_architecture_transforms]]
== Transforms
=== Transforms

Transforms allow you to create/update/delete fields using the {link_spel} (SpEL):

Expand All @@ -63,7 +78,7 @@ riot file-import --process epoch="#date.parse(mydate).getTime()" location="#geo(
----

[[_architecture_regex]]
== Regular Expressions
=== Regular Expressions

Extract patterns from source fields using regular expressions:
[source]
Expand All @@ -72,7 +87,7 @@ riot file-import --regex name="(?<first>\w+)\/(?<last>\w+)" ...
----

[[_architecture_filters]]
== Filters
=== Filters

Filters allow you to exclude records that don't match a SpEL boolean expression.

Expand Down

0 comments on commit d3b94aa

Please sign in to comment.