From 27a19f54ab38393bb776fe4fad0b4f70cea5a2c0 Mon Sep 17 00:00:00 2001 From: Laurence Morgan Date: Thu, 19 Sep 2024 22:33:40 +0100 Subject: [PATCH 01/28] website: user-guide --- app/app.go | 4 +- docs/changelog/v2.0.md | 2 +- docs/changelog/v2.10.md | 2 +- docs/changelog/v2.4.md | 2 +- docs/changelog/v5.0.md | 2 +- docs/commands/bexists.md | 2 +- docs/commands/murex-package.md | 2 +- docs/commands/openagent.md | 2 +- docs/user-guide/README.md | 4 +- docs/user-guide/fileref.md | 2 +- docs/user-guide/integrations.md | 2 +- docs/user-guide/modules.md | 2 +- docs/user-guide/murex-arrays.md | 35 ++ docs/user-guide/profile.md | 2 +- docs/user-guide/reserved-vars.md | 2 +- docs/variables/argv.md | 2 +- docs/variables/params.md | 2 +- docs/variables/pwdhist.md | 2 +- docs/variables/self.md | 2 +- gen/docgen.yaml | 23 +- ...e-nushell.md => json-merge-nushell.inc.md} | 2 +- ...shell.yaml => json-merge-nushell_doc.yaml} | 4 +- gen/user-guide/ansi_doc.yaml | 2 + gen/user-guide/arrays.inc.md | 315 ++++++++++++++++++ gen/user-guide/arrays_doc.yaml | 25 ++ gen/user-guide/bang-prefix_doc.yaml | 4 +- gen/user-guide/fileref_doc.yaml | 2 + gen/user-guide/hint-text_doc.yaml | 2 + gen/user-guide/integrations_doc.yaml | 2 + gen/user-guide/interactive-shell_doc.yaml | 2 + gen/user-guide/job-control_doc.yaml | 2 + gen/user-guide/modules_doc.yaml | 4 +- gen/user-guide/named-pipes_doc.yaml | 2 + gen/user-guide/operators-tokens_doc.yaml | 2 + gen/user-guide/profile_doc.yaml | 2 + gen/user-guide/reserved_vars_doc.yaml | 2 + gen/user-guide/rosetta-stone_doc.yaml | 2 + gen/user-guide/schedulers_doc.yaml | 2 + gen/user-guide/scoping_doc.yaml | 2 + gen/user-guide/strict-types_doc.yaml | 2 + gen/user-guide/terminal-keys_doc.yaml | 3 + gen/userguide-vue-cat.tmpl | 1 + gen/vuepress/sidebar.ts | 5 +- gen/vuepress/variables_generated.json | 2 +- version.svg | 2 +- 45 files changed, 463 insertions(+), 30 deletions(-) create mode 100644 docs/user-guide/murex-arrays.md rename gen/examples/{json-merge-nushell.md => json-merge-nushell.inc.md} (96%) rename gen/examples/{json-merge-nushell.yaml => json-merge-nushell_doc.yaml} (82%) create mode 100644 gen/user-guide/arrays.inc.md create mode 100644 gen/user-guide/arrays_doc.yaml create mode 100644 gen/userguide-vue-cat.tmpl diff --git a/app/app.go b/app/app.go index 48dc71d0d..56712dd92 100644 --- a/app/app.go +++ b/app/app.go @@ -16,9 +16,9 @@ const Name = "murex" const ( Major = 6 Minor = 3 - Revision = 4225 + Revision = 4231 Branch = "develop" - BuildDate = "2024-09-18 22:09:49" + BuildDate = "2024-09-19 22:33:50" ) // Copyright is the copyright owner string diff --git a/docs/changelog/v2.0.md b/docs/changelog/v2.0.md index 1490ccb1b..1aa0c8f4a 100644 --- a/docs/changelog/v2.0.md +++ b/docs/changelog/v2.0.md @@ -34,7 +34,7 @@ Published: 17.04.2021 at 12:49 * [Interactive Shell](../user-guide/interactive-shell.md): What's different about Murex's interactive shell? * [Modules And Packages](../user-guide/modules.md): - An introduction to Murex modules and packages + Modules and packages: An Introduction * [Murex Package Management (`murex-package`)](../commands/murex-package.md): Murex's package manager * [Open File (`open`)](../commands/open.md): diff --git a/docs/changelog/v2.10.md b/docs/changelog/v2.10.md index 0bc361e5a..47906ecbd 100644 --- a/docs/changelog/v2.10.md +++ b/docs/changelog/v2.10.md @@ -59,7 +59,7 @@ Published: 01.08.2022 at 20:10 * [List Filesystem Objects (`f`)](../commands/f.md): Lists or filters file system objects (eg files) * [Modules And Packages](../user-guide/modules.md): - An introduction to Murex modules and packages + Modules and packages: An Introduction * [Pipe Fail (`trypipe`)](../commands/trypipe.md): Checks for non-zero exits of each function in a pipeline * [Profile Files](../user-guide/profile.md): diff --git a/docs/changelog/v2.4.md b/docs/changelog/v2.4.md index 417294b55..24ae79b8d 100644 --- a/docs/changelog/v2.4.md +++ b/docs/changelog/v2.4.md @@ -111,7 +111,7 @@ Published: 09.12.2021 at 08:00 * [FileRef](../user-guide/fileref.md): How to track what code was loaded and from where * [Modules And Packages](../user-guide/modules.md): - An introduction to Murex modules and packages + Modules and packages: An Introduction * [Open File (`open`)](../commands/open.md): Open a file with a preferred handler * [Profile Files](../user-guide/profile.md): diff --git a/docs/changelog/v5.0.md b/docs/changelog/v5.0.md index 66174d51b..f8fef0473 100644 --- a/docs/changelog/v5.0.md +++ b/docs/changelog/v5.0.md @@ -92,7 +92,7 @@ Published: 07.09.2023 at 00:12 * [Location Of Command (`which`)](../commands/which.md): Locate command origin * [Modules And Packages](../user-guide/modules.md): - An introduction to Murex modules and packages + Modules and packages: An Introduction * [Murex Event Subsystem (`event`)](../commands/event.md): Event driven programming for shell scripts * [Murex Package Management (`murex-package`)](../commands/murex-package.md): diff --git a/docs/commands/bexists.md b/docs/commands/bexists.md index ed91c8e83..c5f3aeda0 100644 --- a/docs/commands/bexists.md +++ b/docs/commands/bexists.md @@ -48,7 +48,7 @@ that reason alone it has been spared from the axe. * [Execute Shell Function or Builtin (`fexec`)](../commands/fexec.md): Execute a command or function, bypassing the usual order of precedence. * [Modules And Packages](../user-guide/modules.md): - An introduction to Murex modules and packages + Modules and packages: An Introduction * [Shell Runtime (`runtime`)](../commands/runtime.md): Returns runtime information on the internal state of Murex diff --git a/docs/commands/murex-package.md b/docs/commands/murex-package.md index 0894f4797..f706310cd 100644 --- a/docs/commands/murex-package.md +++ b/docs/commands/murex-package.md @@ -121,7 +121,7 @@ If you wish to view what modules are loaded in a current session then use * [Alias Pointer (`alias`)](../commands/alias.md): Create an alias for a command * [Modules And Packages](../user-guide/modules.md): - An introduction to Murex modules and packages + Modules and packages: An Introduction * [Murex's Offline Documentation (`murex-docs`)](../commands/murex-docs.md): Displays the man pages for Murex builtins * [Private Function (`private`)](../commands/private.md): diff --git a/docs/commands/openagent.md b/docs/commands/openagent.md index 5383195ac..f085dcbbc 100644 --- a/docs/commands/openagent.md +++ b/docs/commands/openagent.md @@ -47,7 +47,7 @@ handler by checking `runtime --open-agents` and checking it's **FileRef**. * [FileRef](../user-guide/fileref.md): How to track what code was loaded and from where * [Modules And Packages](../user-guide/modules.md): - An introduction to Murex modules and packages + Modules and packages: An Introduction * [Open File (`open`)](../commands/open.md): Open a file with a preferred handler * [Shell Runtime (`runtime`)](../commands/runtime.md): diff --git a/docs/user-guide/README.md b/docs/user-guide/README.md index ffc02d2a4..9df400715 100644 --- a/docs/user-guide/README.md +++ b/docs/user-guide/README.md @@ -46,7 +46,7 @@ The [Language Tour](/tour.md) is a great introduction into the Murex language. * [Job Control](../user-guide/job-control.md): How to manage jobs with Murex * [Modules And Packages](../user-guide/modules.md): - An introduction to Murex modules and packages + Modules and packages: An Introduction * [Named Pipes](../user-guide/namedpipes.md): A detailed breakdown of named pipes in Murex * [Operators And Tokens](../user-guide/operators-and-tokens.md): @@ -67,6 +67,8 @@ The [Language Tour](/tour.md) is a great introduction into the Murex language. A list of all the terminal hotkeys and their uses * [Variable And Config Scoping](../user-guide/scoping.md): How scoping works within Murex +* [Working With Arrays](../user-guide/murex-arrays.md): + Examples using arrays within Murex ## Operators And Tokens diff --git a/docs/user-guide/fileref.md b/docs/user-guide/fileref.md index 57cfd9784..6c94d3f47 100644 --- a/docs/user-guide/fileref.md +++ b/docs/user-guide/fileref.md @@ -59,7 +59,7 @@ empty Filename string. * [Include / Evaluate Murex Code (`source`)](../commands/source.md): Import Murex code from another file or code block * [Modules And Packages](../user-guide/modules.md): - An introduction to Murex modules and packages + Modules and packages: An Introduction * [Murex Package Management (`murex-package`)](../commands/murex-package.md): Murex's package manager * [Shell Runtime (`runtime`)](../commands/runtime.md): diff --git a/docs/user-guide/integrations.md b/docs/user-guide/integrations.md index dd210d766..9204d58be 100644 --- a/docs/user-guide/integrations.md +++ b/docs/user-guide/integrations.md @@ -46,7 +46,7 @@ Murex's base install. * [FileRef](../user-guide/fileref.md): How to track what code was loaded and from where * [Modules And Packages](../user-guide/modules.md): - An introduction to Murex modules and packages + Modules and packages: An Introduction * [Profile Files](../user-guide/profile.md): A breakdown of the different files loaded on start up * [Spellcheck](../integrations/spellcheck.md): diff --git a/docs/user-guide/modules.md b/docs/user-guide/modules.md index 31e4b1c42..dcba3cc3f 100644 --- a/docs/user-guide/modules.md +++ b/docs/user-guide/modules.md @@ -1,6 +1,6 @@ # Modules And Packages -> An introduction to Murex modules and packages +> Modules and packages: An Introduction ## Description diff --git a/docs/user-guide/murex-arrays.md b/docs/user-guide/murex-arrays.md new file mode 100644 index 000000000..ec0906189 --- /dev/null +++ b/docs/user-guide/murex-arrays.md @@ -0,0 +1,35 @@ +# Working With Arrays + +> Examples using arrays within Murex + +In this example we'll merge two JSON files together, using example code from +Nushell to illustrate Murex's syntax. + +## See Also + +* [Define Type (`cast`)](../commands/cast.md): + Alters the data-type of the previous function without altering its output +* [Expressions (`expr`)](../commands/expr.md): + Expressions: mathematical, string comparisons, logical operators +* [Operators And Tokens](../user-guide/operators-and-tokens.md): + All supported operators and tokens +* [Shell Configuration And Settings (`config`)](../commands/config.md): + Query or define Murex runtime settings +* [`*=` Multiply By Operator](../parser/multiply-by.md): + Multiplies a variable by the right hand value (expression) +* [`+` Addition Operator](../parser/addition.md): + Adds two numeric values together (expression) +* [`-=` Subtract By Operator](../parser/subtract-by.md): + Subtracts a variable by the right hand value (expression) +* [`/=` Divide By Operator](../parser/divide-by.md): + Divides a variable by the right hand value (expression) +* [`float` (floating point number)](../types/float.md): + Floating point number (primitive) +* [`int`](../types/int.md): + Whole number (primitive) +* [`num` (number)](../types/num.md): + Floating point number (primitive) + +
+ +This document was generated from [gen/user-guide/arrays_doc.yaml](https://github.com/lmorg/murex/blob/master/gen/user-guide/arrays_doc.yaml). \ No newline at end of file diff --git a/docs/user-guide/profile.md b/docs/user-guide/profile.md index 0a02a6922..94eb75f4a 100644 --- a/docs/user-guide/profile.md +++ b/docs/user-guide/profile.md @@ -80,7 +80,7 @@ how you define that value. * [Define Environmental Variable (`export`)](../commands/export.md): Define an environmental variable and set it's value * [Modules And Packages](../user-guide/modules.md): - An introduction to Murex modules and packages + Modules and packages: An Introduction * [Murex Package Management (`murex-package`)](../commands/murex-package.md): Murex's package manager diff --git a/docs/user-guide/reserved-vars.md b/docs/user-guide/reserved-vars.md index 8392a4814..42ccdf991 100644 --- a/docs/user-guide/reserved-vars.md +++ b/docs/user-guide/reserved-vars.md @@ -38,7 +38,7 @@ Error in `set` (0,1): cannot set a reserved variable: SELF * [Define Variable (`set`)](../commands/set.md): Define a variable (typically local) and set it's value * [Modules And Packages](../user-guide/modules.md): - An introduction to Murex modules and packages + Modules and packages: An Introduction * [Pipeline](../user-guide/pipeline.md): Overview of what a "pipeline" is * [Variable And Config Scoping](../user-guide/scoping.md): diff --git a/docs/variables/argv.md b/docs/variables/argv.md index d4cc5980b..1e7ad24bc 100644 --- a/docs/variables/argv.md +++ b/docs/variables/argv.md @@ -41,7 +41,7 @@ deprecated and may be removed from future releases. * [Define Variable (`set`)](../commands/set.md): Define a variable (typically local) and set it's value * [Modules And Packages](../user-guide/modules.md): - An introduction to Murex modules and packages + Modules and packages: An Introduction * [Output String (`out`)](../commands/out.md): Print a string to the stdout with a trailing new line character * [Pipeline](../user-guide/pipeline.md): diff --git a/docs/variables/params.md b/docs/variables/params.md index 7a19062d7..301962d92 100644 --- a/docs/variables/params.md +++ b/docs/variables/params.md @@ -29,7 +29,7 @@ This is a [reserved variable](/docs/user-guide/reserved-vars.md) so it cannot be * [Define Variable (`set`)](../commands/set.md): Define a variable (typically local) and set it's value * [Modules And Packages](../user-guide/modules.md): - An introduction to Murex modules and packages + Modules and packages: An Introduction * [Output String (`out`)](../commands/out.md): Print a string to the stdout with a trailing new line character * [Pipeline](../user-guide/pipeline.md): diff --git a/docs/variables/pwdhist.md b/docs/variables/pwdhist.md index ef23b181b..0b0e82c87 100644 --- a/docs/variables/pwdhist.md +++ b/docs/variables/pwdhist.md @@ -28,7 +28,7 @@ It is updated via `cd` however you can overwrite its value manually via `set`. * [Define Variable (`set`)](../commands/set.md): Define a variable (typically local) and set it's value * [Modules And Packages](../user-guide/modules.md): - An introduction to Murex modules and packages + Modules and packages: An Introduction * [Pipeline](../user-guide/pipeline.md): Overview of what a "pipeline" is * [Reserved Variables](../user-guide/reserved-vars.md): diff --git a/docs/variables/self.md b/docs/variables/self.md index 19c6a6f9e..61700612e 100644 --- a/docs/variables/self.md +++ b/docs/variables/self.md @@ -85,7 +85,7 @@ terminal (TTY) or a pipe?) * [Interactive Shell](../user-guide/interactive-shell.md): What's different about Murex's interactive shell? * [Modules And Packages](../user-guide/modules.md): - An introduction to Murex modules and packages + Modules and packages: An Introduction * [Pipeline](../user-guide/pipeline.md): Overview of what a "pipeline" is * [Public Function (`function`)](../commands/function.md): diff --git a/gen/docgen.yaml b/gen/docgen.yaml index 78a44afbf..dee2af915 100644 --- a/gen/docgen.yaml +++ b/gen/docgen.yaml @@ -224,7 +224,6 @@ Categories: DocumentTemplate: gen/types-md-doc.tmpl # special variables: - - ID: variables Title: Special Variables Summary: >- @@ -249,7 +248,7 @@ Categories: Description: |- Environmental variables are variables held at an operating system level and thus will get passed between running processes. - ID: vars.scoped - Title: Scoped Variables + Title: Locally Scoped Variables VueIcon: arrows-turn-to-dots Description: |- Scoped variables are defined within a specific function's scope. These might be reserved variables but they might also be used to provide structured returns from a particular scope. @@ -296,6 +295,26 @@ Categories: CategoryTemplate: gen/userguide-md-cat.tmpl.md DocumentTemplate: gen/userguide-md-doc.tmpl + - OutputPath: gen/vuepress + CategoryFile: userguide_generated.json + CategoryTemplate: gen/userguide-vue-cat.tmpl + + SubCategories: + - ID: guide.beginners + Title: Beginners Guides + VueIcon: + Description: |- + Just getting started in Murex? Then have a read of these guides. + - ID: guide.reference + Title: Cheat Sheets + Description: |- + The guides are useful for beginners and experiences users alike. + - ID: guide.readmore + Title: Read More + Description: |- + Read more about specific features within Murex. This section contains more detailed contents rather than broader documents providing an overview. The contents is suitable for beginners and experienced users alike. + + # integrations - ID: integrations Title: Integrations diff --git a/gen/examples/json-merge-nushell.md b/gen/examples/json-merge-nushell.inc.md similarity index 96% rename from gen/examples/json-merge-nushell.md rename to gen/examples/json-merge-nushell.inc.md index 90af55b18..9c70f45c2 100644 --- a/gen/examples/json-merge-nushell.md +++ b/gen/examples/json-merge-nushell.inc.md @@ -4,7 +4,7 @@ First we need to create two JSON files. Here the syntax is pretty similar however with the Nushell code we are using the command `save` whereas the Murex equivalent uses the `|>` operator, which -is like the Bash `>` redirection pipe. +is equivalent to the Bash redirection pipe, `>`. Nushell: diff --git a/gen/examples/json-merge-nushell.yaml b/gen/examples/json-merge-nushell_doc.yaml similarity index 82% rename from gen/examples/json-merge-nushell.yaml rename to gen/examples/json-merge-nushell_doc.yaml index 026d3696a..ddfd8ffda 100644 --- a/gen/examples/json-merge-nushell.yaml +++ b/gen/examples/json-merge-nushell_doc.yaml @@ -2,13 +2,15 @@ Title: >- JSON Merge CategoryID: examples + SubCategoryIDs: + - eg.nushell Summary: >- Merging two JSON arrays (comparing Murex to Nushell) Description: |- In this example we'll merge two JSON files together, using example code from Nushell to illustrate Murex's syntax. Detail: |- - {{ include "gen/examples/json-merge-nushell.md" }} + {{ include "gen/examples/json-merge-nushell.inc.md" }} Related: - expr - addition diff --git a/gen/user-guide/ansi_doc.yaml b/gen/user-guide/ansi_doc.yaml index e12383006..a6aa20830 100644 --- a/gen/user-guide/ansi_doc.yaml +++ b/gen/user-guide/ansi_doc.yaml @@ -2,6 +2,8 @@ Title: >- ANSI Constants CategoryID: user-guide + SubCategoryIDs: + - guide.reference Summary: >- Infixed constants that return ANSI escape sequences Description: |- diff --git a/gen/user-guide/arrays.inc.md b/gen/user-guide/arrays.inc.md new file mode 100644 index 000000000..b5a90f887 --- /dev/null +++ b/gen/user-guide/arrays.inc.md @@ -0,0 +1,315 @@ +### Creating Arrays + +Arrays can be defined with `%[ ... ]`. + +The syntax is a superset of JSON. So that any JSON array can be a Murex array +when prefixed with `%`. For example + +``` +%["Monday", "Tuesday", "Wednesday", "Thursday", "Friday"] +``` + +...however while this is readable, this is not convenient when working in the +interactive command line. We want something that's a little more comfortable +for "write many, read once" type environments like a shell REPL. + +So Murex makes the parser-defined punctuation optional: + +``` +%[Monday Tuesday Wednesday Thursday Friday] +``` + +That's better, however surely the computer already knows what days there are in +a week? I think we can improve this syntax further... + +``` +%[Monday..Friday] +``` + +The `..` describes a range. So we are saying "return everyday from Monday to +Friday, inclusive". + +``` +» %[Monday..Friday] +[ + "Monday", + "Tuesday", + "Wednesday", + "Thursday", + "Friday" +] +``` + +It's not just days of the week that can be completed like this. Most forms of +dates, number bases and other familiar sequences can be. + +You can also have multiple parts of arrays expanded: + +``` +» %[[2024..2026]\\, [spring..winter]] +[ + "2024, spring", + "2024, summer", + "2024, autumn", + "2024, winter", + "2025, spring", + "2025, summer", + "2025, autumn", + "2025, winter", + "2026, spring", + "2026, summer", + "2026, autumn", + "2026, winter" +] +``` + +Did you notice how the seasons are lower case this time? Murex respects the +text case of the ranges. For example: + +``` +» %[MON..WED] +[ + "MON", + "TUE", + "WED", +] + +» %[mon..wed] +[ + "mon", + "tue", + "wed", +] +``` + +Returning to our multi-part array, perhaps you want **fall** instead of +**autumn**? (here you do need to include a comma) + +``` +» %[[2024..2026]\\, [spring,summer,fall,winter]] +[ + "2024, spring", + "2024, summer", + "2024, fall", + "2024, winter", + "2025, spring", + "2025, summer", + "2025, fall", + "2025, winter", + "2026, spring", + "2026, summer", + "2026, fall", + "2026, winter" +] +``` + +### Creating Multi-Dimensional Arrays + +But what if they were supposed to be nested rather than flattened arrays? Well +that's not a problem either. Just make sure your first nested array is also +prefixed with `%`: + +``` +» %[%[2024..2026] [Spring..Winter]] +[ + [ + 2024, + 2025, + 2026 + ], + [ + "Spring", + "Summer", + "Autumn", + "Winter" + ] +] +``` + +### Streaming Arrays + +Ok, that's great, but marshalling a data structure and passing it to functions +requires allocating that entire data structure to memory. Whereas shells best +excel when they're streaming data because it allows processes to perform +concurrent operations across massive data sets. + +If that's a problem for you too, then Murex has you sorted: `a`. + +The `a` builtin returns a line separated list. And thus can be operated on via +your traditional pipes and UNIX core utilities. + +Lets say, for some reason, you wanted to run a job against every IPv4 address +available. Marshalling that into a data structure just to run commands +sequentially would be rather silly. So lets stream that array using `a`: + +``` +a [0..255].[0..255].[0..255].[0..255] | foreach $ip { ping -c 1 -t 1 $ip } +``` + +> If you do happen to run this and wonder how to cancel the underlying `a` and +> `foreach` routines, you can press `ctrl`+`\` -- which is a Murex shortcut to +> kill everything in that shell session. + +This is obviously an absurd example because nobody in their right mind would +want to ping every valid IPv4 address. But it does demonstrate the advantages +of streaming lists rather than creating arrays. + +### Accessing Array Values + +There are two main ways to access values inside an array: + +* square brackets for immutable copies: `$my_array[index]` + +* dot notation, which allows being written to: `$my_array.index` + +Why two? Because they support different features. + +#### Square Brackets (immutable) + +With square brackets you can select more than just a single element. For +example, if you wanted the first element you can reference it the same way +you'd reference arrays in any other language: + +``` +$my_array[0] +``` + +> Murex arrays begin at `0`. + +If you wanted to count from the end of the array, you can use negative values: + +``` +$my_array[-1] +``` + +> Watch out here because negative indexes count from `1` because -0 isn't a +> valid number. + +#### Multiple Elements + +However what if you wanted multiple elements from the array, like 2nd and 4th? +Then just specify multiple elements inside the square brackets: + +``` +$my_array[1 4] +``` + +#### Ranges + +That's handy, but if I actually want a range of elements? Well then you can use +the range `..` operator like before: + +``` +$my_array[1..4] +``` + +And if you don't know the size of your array, you can ignore the index value +entirely. For example: + +**Everything from and including the 2nd element:** + +``` +$my_array[2..] +``` + +**Everything up to and including the 4th element:** + +``` +$my_array[..4] +``` + +> Ranges are indexed from 1. Yes, I know that's stupid and confusing. + +#### As A Function + +The square brackets can also be used as a function too. Which means any kind of +array or list can be queried from stdin, you don't have to first convert it to +a variable. + +``` +» %[mon..fri] | [1 4] +[ + "tue", + "fri" +] +``` + +This is especially helpful if say something writes JSON, YAML, or any other +structured document, and you only want specific values. For example you want +the last container in your cloud infrastructure, and you know your cloud CLI +tool (`cloud-api` for our made up purposes here) returns JSON: + +``` +cloud-api list-containers | :json: [-1] +``` + +#### Making Changes + +That's all great, but what if I want to make a change to the host array? + +Well this is where dot notation comes in... + +### Dot Notation (mutable) + +Dot notation is a lot more limited in what you can do because it's designed for +making careful edits of the underlying data structure. So it can only be used +with variables. + +#### Assignment + +You can edit an element, for example renaming **Wednesday** to **Humpday**: + +``` +» $days = %[Monday..Friday] + +» $days.2 = "Humpday" + +» $days +[ + "Monday", + "Tuesday", + "Humpday", + "Thursday", + "Friday" +] +``` + +> Remember: arrays are zero based + +#### Printing + +You can also use dot notation to return a value, just like you would with the +square braces solution above. But dot notation doesn't support any special +magic and still only works on variables: + +``` +$my_array.2 +``` + +### Appending Arrays + +There are several ways to append an array. You can create a copy of that array +and append via the pipeline: + +``` +$my_array | append foo bar +``` + +> You can also prepend using `prepend`) + +...or... + +``` +$my_array ~> %[foo bar] +``` + +However if you want to update a variable in place, you can use the merge +operator, `<~`: + +``` +$my_array <~ %[foo bar] +``` + +## Fin + +Now you're an expert in arrays. Hooray `\o/` \ No newline at end of file diff --git a/gen/user-guide/arrays_doc.yaml b/gen/user-guide/arrays_doc.yaml new file mode 100644 index 000000000..7cfaf16c1 --- /dev/null +++ b/gen/user-guide/arrays_doc.yaml @@ -0,0 +1,25 @@ +- DocumentID: murex-arrays + Title: >- + Working With Arrays + CategoryID: user-guide + SubCategoryIDs: + - guide.beginners + Summary: >- + Examples using arrays within Murex + Description: |- + In this example we'll merge two JSON files together, using example code from + Nushell to illustrate Murex's syntax. + Detail: |- + {{ include "gen/examples/arrays.inc.md" }} + Related: + - expr + - addition + - subtract-by + - multiply-by + - divide-by + - int + - float + - num + - config + - cast + - operators-and-tokens \ No newline at end of file diff --git a/gen/user-guide/bang-prefix_doc.yaml b/gen/user-guide/bang-prefix_doc.yaml index 8ba68afbb..6cc608b36 100644 --- a/gen/user-guide/bang-prefix_doc.yaml +++ b/gen/user-guide/bang-prefix_doc.yaml @@ -2,6 +2,8 @@ Title: >- Bang Prefix CategoryID: user-guide + SubCategoryIDs: + - guide.readmore Summary: >- Bang prefixing to reverse default actions Description: |- @@ -32,5 +34,3 @@ - and - or - config - - diff --git a/gen/user-guide/fileref_doc.yaml b/gen/user-guide/fileref_doc.yaml index c92741256..d14e2a9c1 100644 --- a/gen/user-guide/fileref_doc.yaml +++ b/gen/user-guide/fileref_doc.yaml @@ -2,6 +2,8 @@ Title: >- FileRef CategoryID: user-guide + SubCategoryIDs: + - guide.readmore Summary: >- How to track what code was loaded and from where Description: |- diff --git a/gen/user-guide/hint-text_doc.yaml b/gen/user-guide/hint-text_doc.yaml index 9ad6bb0ee..097745b5c 100644 --- a/gen/user-guide/hint-text_doc.yaml +++ b/gen/user-guide/hint-text_doc.yaml @@ -2,6 +2,8 @@ Title: >- Hint Text CategoryID: user-guide + SubCategoryIDs: + - guide.readmore Summary: >- A status bar for your shell Description: |- diff --git a/gen/user-guide/integrations_doc.yaml b/gen/user-guide/integrations_doc.yaml index d80f8d3ff..16d2d4014 100644 --- a/gen/user-guide/integrations_doc.yaml +++ b/gen/user-guide/integrations_doc.yaml @@ -2,6 +2,8 @@ Title: >- Integrations CategoryID: user-guide + SubCategoryIDs: + - guide.readmore Summary: >- Default integrations shipped with Murex Description: |- diff --git a/gen/user-guide/interactive-shell_doc.yaml b/gen/user-guide/interactive-shell_doc.yaml index 043241fbc..46ab05772 100644 --- a/gen/user-guide/interactive-shell_doc.yaml +++ b/gen/user-guide/interactive-shell_doc.yaml @@ -2,6 +2,8 @@ Title: >- Interactive Shell CategoryID: user-guide + SubCategoryIDs: + - guide.beginners Summary: >- What's different about Murex's interactive shell? Description: |- diff --git a/gen/user-guide/job-control_doc.yaml b/gen/user-guide/job-control_doc.yaml index 125ffbbc3..8d4b0eae0 100644 --- a/gen/user-guide/job-control_doc.yaml +++ b/gen/user-guide/job-control_doc.yaml @@ -2,6 +2,8 @@ Title: >- Job Control CategoryID: user-guide + SubCategoryIDs: + - guide.readmore Summary: >- How to manage jobs with Murex Description: |- diff --git a/gen/user-guide/modules_doc.yaml b/gen/user-guide/modules_doc.yaml index d9ad121d8..6608cfd3c 100644 --- a/gen/user-guide/modules_doc.yaml +++ b/gen/user-guide/modules_doc.yaml @@ -2,8 +2,10 @@ Title: >- Modules And Packages CategoryID: user-guide + SubCategoryIDs: + - guide.beginners Summary: >- - An introduction to Murex modules and packages + Modules and packages: An Introduction Description: |- ## Description diff --git a/gen/user-guide/named-pipes_doc.yaml b/gen/user-guide/named-pipes_doc.yaml index 8b7a71764..aec3eeb4c 100644 --- a/gen/user-guide/named-pipes_doc.yaml +++ b/gen/user-guide/named-pipes_doc.yaml @@ -2,6 +2,8 @@ Title: >- Named Pipes CategoryID: user-guide + SubCategoryIDs: + - guide.readmore Summary: >- A detailed breakdown of named pipes in Murex Description: |- diff --git a/gen/user-guide/operators-tokens_doc.yaml b/gen/user-guide/operators-tokens_doc.yaml index 9284b8787..0b1ebb398 100644 --- a/gen/user-guide/operators-tokens_doc.yaml +++ b/gen/user-guide/operators-tokens_doc.yaml @@ -2,6 +2,8 @@ Title: >- Operators And Tokens CategoryID: user-guide + SubCategoryIDs: + - guide.reference Summary: >- All supported operators and tokens Description: |- diff --git a/gen/user-guide/profile_doc.yaml b/gen/user-guide/profile_doc.yaml index 3440a96c5..bd26eea24 100644 --- a/gen/user-guide/profile_doc.yaml +++ b/gen/user-guide/profile_doc.yaml @@ -2,6 +2,8 @@ Title: >- Profile Files CategoryID: user-guide + SubCategoryIDs: + - guide.readmore Summary: >- A breakdown of the different files loaded on start up Description: |- diff --git a/gen/user-guide/reserved_vars_doc.yaml b/gen/user-guide/reserved_vars_doc.yaml index 7ceb0f8ec..09b664dbe 100644 --- a/gen/user-guide/reserved_vars_doc.yaml +++ b/gen/user-guide/reserved_vars_doc.yaml @@ -2,6 +2,8 @@ Title: >- Reserved Variables CategoryID: user-guide + SubCategoryIDs: + - guide.readmore Summary: >- Special variables reserved by Murex Description: |- diff --git a/gen/user-guide/rosetta-stone_doc.yaml b/gen/user-guide/rosetta-stone_doc.yaml index af3244f4f..aaa6fdd83 100644 --- a/gen/user-guide/rosetta-stone_doc.yaml +++ b/gen/user-guide/rosetta-stone_doc.yaml @@ -2,6 +2,8 @@ Title: >- Rosetta Stone CategoryID: user-guide + SubCategoryIDs: + - guide.reference Summary: >- A tabulated list of Bashism's and their equivalent Murex syntax Description: |- diff --git a/gen/user-guide/schedulers_doc.yaml b/gen/user-guide/schedulers_doc.yaml index 1cadd0a39..b3228790a 100644 --- a/gen/user-guide/schedulers_doc.yaml +++ b/gen/user-guide/schedulers_doc.yaml @@ -2,6 +2,8 @@ Title: >- Schedulers CategoryID: user-guide + SubCategoryIDs: + - guide.readmore Summary: >- Overview of the different schedulers (or 'run modes') in Murex Description: |- diff --git a/gen/user-guide/scoping_doc.yaml b/gen/user-guide/scoping_doc.yaml index c03dee9b5..09679d917 100644 --- a/gen/user-guide/scoping_doc.yaml +++ b/gen/user-guide/scoping_doc.yaml @@ -2,6 +2,8 @@ Title: >- Variable And Config Scoping CategoryID: user-guide + SubCategoryIDs: + - guide.readmore Summary: >- How scoping works within Murex Description: |- diff --git a/gen/user-guide/strict-types_doc.yaml b/gen/user-guide/strict-types_doc.yaml index 45a976316..f8dd0f1b3 100644 --- a/gen/user-guide/strict-types_doc.yaml +++ b/gen/user-guide/strict-types_doc.yaml @@ -2,6 +2,8 @@ Title: >- Strict Types In Expressions CategoryID: user-guide + SubCategoryIDs: + - guide.readmore Summary: >- Expressions can auto-convert types or strictly honour data types Description: |- diff --git a/gen/user-guide/terminal-keys_doc.yaml b/gen/user-guide/terminal-keys_doc.yaml index a9d472488..786c33879 100644 --- a/gen/user-guide/terminal-keys_doc.yaml +++ b/gen/user-guide/terminal-keys_doc.yaml @@ -2,6 +2,9 @@ Title: >- Terminal Hotkeys CategoryID: user-guide + SubCategoryIDs: + - guide.beginners + - guide.reference Summary: >- A list of all the terminal hotkeys and their uses Description: |- diff --git a/gen/userguide-vue-cat.tmpl b/gen/userguide-vue-cat.tmpl new file mode 100644 index 000000000..cbbe043f6 --- /dev/null +++ b/gen/userguide-vue-cat.tmpl @@ -0,0 +1 @@ +{{ vuepressmenu "user-guide" }} \ No newline at end of file diff --git a/gen/vuepress/sidebar.ts b/gen/vuepress/sidebar.ts index c4140b03d..f8a0c7a6d 100644 --- a/gen/vuepress/sidebar.ts +++ b/gen/vuepress/sidebar.ts @@ -2,6 +2,7 @@ import { sidebar } from "vuepress-theme-hope"; import commandsMenu from "./commands_generated.json" import parserMenu from "./parser_generated.json" import variablesMenu from "./variables_generated.json" +import userguideMenu from "./userguide_generated.json" export default sidebar({ "/": [ @@ -23,8 +24,8 @@ export default sidebar({ { text: "User Guide", icon: "book", - prefix: "user-guide/", - children: "structure", + prefix: "/", + children: userguideMenu, collapsible: true, }, { diff --git a/gen/vuepress/variables_generated.json b/gen/vuepress/variables_generated.json index 75646f8b2..01ea03bb0 100644 --- a/gen/vuepress/variables_generated.json +++ b/gen/vuepress/variables_generated.json @@ -62,7 +62,7 @@ ], "collapsible": true, "icon": "folder-closed", - "text": "Scoped Variables" + "text": "Locally Scoped Variables" }, { "children": [ diff --git a/version.svg b/version.svg index 6ece2a9fa..a473d8759 100644 --- a/version.svg +++ b/version.svg @@ -1 +1 @@ -Version: 6.3.4225Version6.3.4225 +Version: 6.3.4231Version6.3.4231 From 297098bb964cc78584e69ac3bf9ddf900a047904 Mon Sep 17 00:00:00 2001 From: Laurence Morgan Date: Thu, 19 Sep 2024 22:34:35 +0100 Subject: [PATCH 02/28] website: reformatted user-guide menu + arrays doc --- app/app.go | 4 +- builtins/docs/summaries.go | 2110 +++++++++++++------------ docs/user-guide/murex-arrays.md | 317 +++- gen/user-guide/arrays_doc.yaml | 6 +- gen/vuepress/userguide_generated.json | 134 ++ version.svg | 2 +- 6 files changed, 1515 insertions(+), 1058 deletions(-) create mode 100644 gen/vuepress/userguide_generated.json diff --git a/app/app.go b/app/app.go index 56712dd92..4e7a096f9 100644 --- a/app/app.go +++ b/app/app.go @@ -16,9 +16,9 @@ const Name = "murex" const ( Major = 6 Minor = 3 - Revision = 4231 + Revision = 4234 Branch = "develop" - BuildDate = "2024-09-19 22:33:50" + BuildDate = "2024-09-19 22:54:17" ) // Copyright is the copyright owner string diff --git a/builtins/docs/summaries.go b/builtins/docs/summaries.go index dce3019c3..ef73c936c 100644 --- a/builtins/docs/summaries.go +++ b/builtins/docs/summaries.go @@ -5,1072 +5,1084 @@ package docs func init() { Summary = map[string]string{ - - "key-code": "Returns character sequences for any key pressed (ie sent from the terminal)", - "addheading": "Adds headings to a table", - "prefix": "Prefix a string to every item in a list", - "suffix": "Prefix a string to every item in a list", - "alias": "Create an alias for a command", - "alter": "Change a value within a structured data-type and pass that change along the pipeline without altering the original source input", - "append": "Add data to the end of an array", - "bg": "Run processes in the background", - "cpuarch": "Output the hosts CPU architecture", - "cpucount": "Output the number of CPU cores available on your host", - "catch": "Handles the exception code raised by `try` or `trypipe`", - "cd": "Change (working) directory", - "list.case": "Changes the character case of a string or all elements in an array", - "bexists": "Check which builtins exist", - "history": "Outputs murex's command history", - "count": "Count items in a map, list or array", - "2darray": "Create a 2D JSON array from multiple input sources", - "ja": "A sophisticated yet simply way to build a JSON array", - "map": "Creates a map from two data sources", - "pipe": "Manage Murex named pipes", - "ta": "A sophisticated yet simple way to build an array of a user defined data-type", - "tmp": "Create a temporary file and write to it", - "datetime": "A date and/or time conversion tool (like `printf` but for date and time values)", - "debug": "Debugging information", - "export": "Define an environmental variable and set it's value", - "args": "Command line flag parser for Murex shell scripting", - "global": "Define a global variable and set it's value", - "openagent": "Creates a handler function for `open`", - "method": "Define a methods supported data-types", - "cast": "Alters the data-type of the previous function without altering its output", - "set": "Define a variable (typically local) and set it's value", - "unsafe": "Execute a block of code, always returning a zero exit number", - "type": "Command type (function, builtin, alias, etc)", - "fid-list": "Lists all running functions within the current Murex session", - "getfile": "Makes a standard HTTP request and return the contents as Murex-aware data type for passing along Murex pipelines.", - "err": "Print a line to the stderr", - "esccli": "Escapes an array so output is valid shell code", - "eschtml": "Encode or decodes text for HTML", - "escurl": "Encode or decodes text for the URL", - "exec": "Runs an executable", - "fexec": "Execute a command or function, bypassing the usual order of precedence.", - "break": "Terminate execution of a block within your processes scope", - "return": "Exits current function scope", - "exit": "Exit murex", - "expr": "Expressions: mathematical, string comparisons, logical operators", - "false": "Returns a `false` value", - "foreach": "Iterate through an array", - "formap": "Iterate through a map or other collection of data", - "for": "A more familiar iteration loop to existing developers", - "fg": "Sends a background process into the foreground", - "runmode": "Alter the scheduler's behaviour at higher scoping level", - "rand": "Random field generator", - "get-type": "Returns the data-type of a variable or pipe", - "exitnum": "Output the exit number of the previous process", - "pt": "Pipe telemetry. Writes data-types and bytes written", - "get": "Makes a standard HTTP request and returns the result as a JSON object", - "g": "Glob pattern matching for file system objects (eg `*.txt`)", - "if": "Conditional statement to execute different blocks of code depending on the result of the condition", - "source": "Import Murex code from another file or code block", - "is-null": "Checks if a variable is null or undefined", - "mjoin": "Joins a list or array into a single string", - "fid-killall": "Terminate all running Murex functions in current session", - "fid-kill": "Terminate a running Murex function", - "left": "Left substring every item in a list", - "f": "Lists or filters file system objects (eg files)", - "which": "Locate command origin", - "lockfile": "Create and manage lock files", - "and": "Returns `true` or `false` depending on whether multiple conditions are met", - "or": "Returns `true` or `false` depending on whether one code-block out of multiple ones supplied is successful or unsuccessful.", - "while": "Loop until condition false", - "man-summary": "Outputs a man page summary of a command", - "match": "Match an exact value in an array", - "event": "Event driven programming for shell scripts", - "murex-package": "Murex's package manager", - "version": "Get Murex version", - "murex-docs": "Displays the man pages for Murex builtins", - "continue": "Terminate process of a block within a caller function", - "not-func": "Reads the stdin and exit number from previous process and not's it's condition", - "devnull": "null function. Similar to /dev/null", - "open": "Open a file with a preferred handler", - "os": "Output the auto-detected OS name", - "out": "Print a string to the stdout with a trailing new line character", - "tout": "Print a string to the stdout and set it's data-type", - "man-get-flags": "Parses man page files for command line flags", - "trypipe": "Checks for non-zero exits of each function in a pipeline", - "post": "HTTP POST request with a JSON-parsable return", - "prepend": "Add data to the start of an array", - "pretty": "Prettifies JSON to make it human readable", - "struct-keys": "Outputs all the keys in a structure as a file path", - "private": "Define a private function block", - "time": "Returns the execution run time of a command or block", - "function": "Define a function block", - "escape": "Escape or unescape input", + + + "key-code": "Returns character sequences for any key pressed (ie sent from the terminal)", + "addheading": "Adds headings to a table", + "prefix": "Prefix a string to every item in a list", + "suffix": "Prefix a string to every item in a list", + "alias": "Create an alias for a command", + "alter": "Change a value within a structured data-type and pass that change along the pipeline without altering the original source input", + "append": "Add data to the end of an array", + "bg": "Run processes in the background", + "cpuarch": "Output the hosts CPU architecture", + "cpucount": "Output the number of CPU cores available on your host", + "catch": "Handles the exception code raised by `try` or `trypipe`", + "cd": "Change (working) directory", + "list.case": "Changes the character case of a string or all elements in an array", + "bexists": "Check which builtins exist", + "history": "Outputs murex's command history", + "count": "Count items in a map, list or array", + "2darray": "Create a 2D JSON array from multiple input sources", + "ja": "A sophisticated yet simply way to build a JSON array", + "map": "Creates a map from two data sources", + "pipe": "Manage Murex named pipes", + "ta": "A sophisticated yet simple way to build an array of a user defined data-type", + "tmp": "Create a temporary file and write to it", + "datetime": "A date and/or time conversion tool (like `printf` but for date and time values)", + "debug": "Debugging information", + "export": "Define an environmental variable and set it's value", + "args": "Command line flag parser for Murex shell scripting", + "global": "Define a global variable and set it's value", + "openagent": "Creates a handler function for `open`", + "method": "Define a methods supported data-types", + "cast": "Alters the data-type of the previous function without altering its output", + "set": "Define a variable (typically local) and set it's value", + "unsafe": "Execute a block of code, always returning a zero exit number", + "type": "Command type (function, builtin, alias, etc)", + "fid-list": "Lists all running functions within the current Murex session", + "getfile": "Makes a standard HTTP request and return the contents as Murex-aware data type for passing along Murex pipelines.", + "err": "Print a line to the stderr", + "esccli": "Escapes an array so output is valid shell code", + "eschtml": "Encode or decodes text for HTML", + "escurl": "Encode or decodes text for the URL", + "exec": "Runs an executable", + "fexec": "Execute a command or function, bypassing the usual order of precedence.", + "break": "Terminate execution of a block within your processes scope", + "return": "Exits current function scope", + "exit": "Exit murex", + "expr": "Expressions: mathematical, string comparisons, logical operators", + "false": "Returns a `false` value", + "foreach": "Iterate through an array", + "formap": "Iterate through a map or other collection of data", + "for": "A more familiar iteration loop to existing developers", + "fg": "Sends a background process into the foreground", + "runmode": "Alter the scheduler's behaviour at higher scoping level", + "rand": "Random field generator", + "get-type": "Returns the data-type of a variable or pipe", + "exitnum": "Output the exit number of the previous process", + "pt": "Pipe telemetry. Writes data-types and bytes written", + "get": "Makes a standard HTTP request and returns the result as a JSON object", + "g": "Glob pattern matching for file system objects (eg `*.txt`)", + "if": "Conditional statement to execute different blocks of code depending on the result of the condition", + "source": "Import Murex code from another file or code block", + "is-null": "Checks if a variable is null or undefined", + "mjoin": "Joins a list or array into a single string", + "fid-killall": "Terminate all running Murex functions in current session", + "fid-kill": "Terminate a running Murex function", + "left": "Left substring every item in a list", + "f": "Lists or filters file system objects (eg files)", + "which": "Locate command origin", + "lockfile": "Create and manage lock files", + "and": "Returns `true` or `false` depending on whether multiple conditions are met", + "or": "Returns `true` or `false` depending on whether one code-block out of multiple ones supplied is successful or unsuccessful.", + "while": "Loop until condition false", + "man-summary": "Outputs a man page summary of a command", + "match": "Match an exact value in an array", + "event": "Event driven programming for shell scripts", + "murex-package": "Murex's package manager", + "version": "Get Murex version", + "murex-docs": "Displays the man pages for Murex builtins", + "continue": "Terminate process of a block within a caller function", + "not-func": "Reads the stdin and exit number from previous process and not's it's condition", + "devnull": "null function. Similar to /dev/null", + "open": "Open a file with a preferred handler", + "os": "Output the auto-detected OS name", + "out": "Print a string to the stdout with a trailing new line character", + "tout": "Print a string to the stdout and set it's data-type", + "man-get-flags": "Parses man page files for command line flags", + "trypipe": "Checks for non-zero exits of each function in a pipeline", + "post": "HTTP POST request with a JSON-parsable return", + "prepend": "Add data to the start of an array", + "pretty": "Prettifies JSON to make it human readable", + "struct-keys": "Outputs all the keys in a structure as a file path", + "private": "Define a private function block", + "time": "Returns the execution run time of a command or block", + "function": "Define a function block", + "escape": "Escape or unescape input", "murex-update-exe-list": "Forces Murex to rescan $PATH looking for executables", - "read": "`read` a line of input from the user and store as a variable", - "tread": "`read` a line of input from the user and store as a user defined *typed* variable (deprecated)", - "format": "Reformat one data-type into another data-type", - "rx": "Regexp pattern matching for file system objects (eg `.*\\\\.txt`)", - "regexp": "Regexp tools for arrays / lists of strings", - "open-image": "Renders bitmap image data on your terminal", - "mtac": "Reverse the order of an array", - "right": "Right substring every item in a list", - "round": "Round a number by a user defined precision", - "signal": "Sends a signal RPC", - "summary": "Defines a summary help text for a command", - "config": "Query or define Murex runtime settings", - "runtime": "Returns runtime information on the internal state of Murex", - "test": "Murex's test framework - define tests, run tests and debug shell scripts", - "msort": "Sorts an array - data type agnostic", - "jsplit": "Splits stdin into a JSON array based on a regex parameter", - "trypipeerr": "Checks state of each function in a pipeline and exits block on error", - "tryerr": "Handles errors inside a block of code", - "a": "A sophisticated yet simple way to stream an array or list (mkarray)", - "switch": "Blocks of cascading conditionals", - "autocomplete": "Set definitions for tab-completion in the command line", - "tabulate": "Table transformation tools", - "true": "Returns a `true` value", - "try": "Handles non-zero exits inside a block of code", - "die": "Terminate murex with an exit number of 1 (deprecated)", - "let": "Evaluate a mathematical function and assign to variable (deprecated)", - "murex-parser": "Runs the Murex parser against a block of code", + "read": "`read` a line of input from the user and store as a variable", + "tread": "`read` a line of input from the user and store as a user defined *typed* variable (deprecated)", + "format": "Reformat one data-type into another data-type", + "rx": "Regexp pattern matching for file system objects (eg `.*\\\\.txt`)", + "regexp": "Regexp tools for arrays / lists of strings", + "open-image": "Renders bitmap image data on your terminal", + "mtac": "Reverse the order of an array", + "right": "Right substring every item in a list", + "round": "Round a number by a user defined precision", + "signal": "Sends a signal RPC", + "summary": "Defines a summary help text for a command", + "config": "Query or define Murex runtime settings", + "runtime": "Returns runtime information on the internal state of Murex", + "test": "Murex's test framework - define tests, run tests and debug shell scripts", + "msort": "Sorts an array - data type agnostic", + "jsplit": "Splits stdin into a JSON array based on a regex parameter", + "trypipeerr": "Checks state of each function in a pipeline and exits block on error", + "tryerr": "Handles errors inside a block of code", + "a": "A sophisticated yet simple way to stream an array or list (mkarray)", + "switch": "Blocks of cascading conditionals", + "autocomplete": "Set definitions for tab-completion in the command line", + "tabulate": "Table transformation tools", + "true": "Returns a `true` value", + "try": "Handles non-zero exits inside a block of code", + "die": "Terminate murex with an exit number of 1 (deprecated)", + "let": "Evaluate a mathematical function and assign to variable (deprecated)", + "murex-parser": "Runs the Murex parser against a block of code", + "select": "Inlining SQL into shell pipelines", - "bz2": "Decompress a bz2 file", + "bz2": "Decompress a bz2 file", "base64": "Encode or decode a base64 string", - "gz": "Compress or decompress a gzip file", - "qr": "Creates a QR code from stdin", - "sleep": "Suspends the shell for a number of seconds", + "gz": "Compress or decompress a gzip file", + "qr": "Creates a QR code from stdin", + "sleep": "Suspends the shell for a number of seconds", - "commands/key-code": "Returns character sequences for any key pressed (ie sent from the terminal)", - "commands/addheading": "Adds headings to a table", - "commands/prefix": "Prefix a string to every item in a list", - "commands/suffix": "Prefix a string to every item in a list", - "commands/alias": "Create an alias for a command", - "commands/alter": "Change a value within a structured data-type and pass that change along the pipeline without altering the original source input", - "commands/append": "Add data to the end of an array", - "commands/bg": "Run processes in the background", - "commands/cpuarch": "Output the hosts CPU architecture", - "commands/cpucount": "Output the number of CPU cores available on your host", - "commands/catch": "Handles the exception code raised by `try` or `trypipe`", - "commands/cd": "Change (working) directory", - "commands/list.case": "Changes the character case of a string or all elements in an array", - "commands/bexists": "Check which builtins exist", - "commands/history": "Outputs murex's command history", - "commands/count": "Count items in a map, list or array", - "commands/2darray": "Create a 2D JSON array from multiple input sources", - "commands/ja": "A sophisticated yet simply way to build a JSON array", - "commands/map": "Creates a map from two data sources", - "commands/pipe": "Manage Murex named pipes", - "commands/ta": "A sophisticated yet simple way to build an array of a user defined data-type", - "commands/tmp": "Create a temporary file and write to it", - "commands/datetime": "A date and/or time conversion tool (like `printf` but for date and time values)", - "commands/debug": "Debugging information", - "commands/export": "Define an environmental variable and set it's value", - "commands/args": "Command line flag parser for Murex shell scripting", - "commands/global": "Define a global variable and set it's value", - "commands/openagent": "Creates a handler function for `open`", - "commands/method": "Define a methods supported data-types", - "commands/cast": "Alters the data-type of the previous function without altering its output", - "commands/set": "Define a variable (typically local) and set it's value", - "commands/unsafe": "Execute a block of code, always returning a zero exit number", - "commands/type": "Command type (function, builtin, alias, etc)", - "commands/fid-list": "Lists all running functions within the current Murex session", - "commands/getfile": "Makes a standard HTTP request and return the contents as Murex-aware data type for passing along Murex pipelines.", - "commands/err": "Print a line to the stderr", - "commands/esccli": "Escapes an array so output is valid shell code", - "commands/eschtml": "Encode or decodes text for HTML", - "commands/escurl": "Encode or decodes text for the URL", - "commands/exec": "Runs an executable", - "commands/fexec": "Execute a command or function, bypassing the usual order of precedence.", - "commands/break": "Terminate execution of a block within your processes scope", - "commands/return": "Exits current function scope", - "commands/exit": "Exit murex", - "commands/expr": "Expressions: mathematical, string comparisons, logical operators", - "commands/false": "Returns a `false` value", - "commands/foreach": "Iterate through an array", - "commands/formap": "Iterate through a map or other collection of data", - "commands/for": "A more familiar iteration loop to existing developers", - "commands/fg": "Sends a background process into the foreground", - "commands/runmode": "Alter the scheduler's behaviour at higher scoping level", - "commands/rand": "Random field generator", - "commands/get-type": "Returns the data-type of a variable or pipe", - "commands/exitnum": "Output the exit number of the previous process", - "commands/pt": "Pipe telemetry. Writes data-types and bytes written", - "commands/get": "Makes a standard HTTP request and returns the result as a JSON object", - "commands/g": "Glob pattern matching for file system objects (eg `*.txt`)", - "commands/if": "Conditional statement to execute different blocks of code depending on the result of the condition", - "commands/source": "Import Murex code from another file or code block", - "commands/is-null": "Checks if a variable is null or undefined", - "commands/mjoin": "Joins a list or array into a single string", - "commands/fid-killall": "Terminate all running Murex functions in current session", - "commands/fid-kill": "Terminate a running Murex function", - "commands/left": "Left substring every item in a list", - "commands/f": "Lists or filters file system objects (eg files)", - "commands/which": "Locate command origin", - "commands/lockfile": "Create and manage lock files", - "commands/and": "Returns `true` or `false` depending on whether multiple conditions are met", - "commands/or": "Returns `true` or `false` depending on whether one code-block out of multiple ones supplied is successful or unsuccessful.", - "commands/while": "Loop until condition false", - "commands/man-summary": "Outputs a man page summary of a command", - "commands/match": "Match an exact value in an array", - "commands/event": "Event driven programming for shell scripts", - "commands/murex-package": "Murex's package manager", - "commands/version": "Get Murex version", - "commands/murex-docs": "Displays the man pages for Murex builtins", - "commands/continue": "Terminate process of a block within a caller function", - "commands/not-func": "Reads the stdin and exit number from previous process and not's it's condition", - "commands/devnull": "null function. Similar to /dev/null", - "commands/open": "Open a file with a preferred handler", - "commands/os": "Output the auto-detected OS name", - "commands/out": "Print a string to the stdout with a trailing new line character", - "commands/tout": "Print a string to the stdout and set it's data-type", - "commands/man-get-flags": "Parses man page files for command line flags", - "commands/trypipe": "Checks for non-zero exits of each function in a pipeline", - "commands/post": "HTTP POST request with a JSON-parsable return", - "commands/prepend": "Add data to the start of an array", - "commands/pretty": "Prettifies JSON to make it human readable", - "commands/struct-keys": "Outputs all the keys in a structure as a file path", - "commands/private": "Define a private function block", - "commands/time": "Returns the execution run time of a command or block", - "commands/function": "Define a function block", - "commands/escape": "Escape or unescape input", - "commands/murex-update-exe-list": "Forces Murex to rescan $PATH looking for executables", - "commands/read": "`read` a line of input from the user and store as a variable", - "commands/tread": "`read` a line of input from the user and store as a user defined *typed* variable (deprecated)", - "commands/format": "Reformat one data-type into another data-type", - "commands/rx": "Regexp pattern matching for file system objects (eg `.*\\\\.txt`)", - "commands/regexp": "Regexp tools for arrays / lists of strings", - "commands/open-image": "Renders bitmap image data on your terminal", - "commands/mtac": "Reverse the order of an array", - "commands/right": "Right substring every item in a list", - "commands/round": "Round a number by a user defined precision", - "commands/signal": "Sends a signal RPC", - "commands/summary": "Defines a summary help text for a command", - "commands/config": "Query or define Murex runtime settings", - "commands/runtime": "Returns runtime information on the internal state of Murex", - "commands/test": "Murex's test framework - define tests, run tests and debug shell scripts", - "commands/msort": "Sorts an array - data type agnostic", - "commands/jsplit": "Splits stdin into a JSON array based on a regex parameter", - "commands/trypipeerr": "Checks state of each function in a pipeline and exits block on error", - "commands/tryerr": "Handles errors inside a block of code", - "commands/a": "A sophisticated yet simple way to stream an array or list (mkarray)", - "commands/switch": "Blocks of cascading conditionals", - "commands/autocomplete": "Set definitions for tab-completion in the command line", - "commands/tabulate": "Table transformation tools", - "commands/true": "Returns a `true` value", - "commands/try": "Handles non-zero exits inside a block of code", - "commands/die": "Terminate murex with an exit number of 1 (deprecated)", - "commands/let": "Evaluate a mathematical function and assign to variable (deprecated)", - "commands/murex-parser": "Runs the Murex parser against a block of code", - "mkarray/date": "Create arrays of dates", - "mkarray/character": "Making character arrays (a to z)", - "mkarray/decimal": "Create arrays of decimal integers", - "mkarray/non-decimal": "Create arrays of integers from non-decimal number bases", - "mkarray/special": "Create arrays from ranges of dictionary terms (eg weekdays, months, seasons, etc)", - "optional/select": "Inlining SQL into shell pipelines", - "optional/bz2": "Decompress a bz2 file", - "optional/base64": "Encode or decode a base64 string", - "optional/gz": "Compress or decompress a gzip file", - "optional/qr": "Creates a QR code from stdin", - "optional/sleep": "Suspends the shell for a number of seconds", - "parser/expr-inlined": "Inline expressions", - "parser/c-style-fun": "Inlined commands for expressions and statements", - "parser/range": "Outputs a ranged subset of data from stdin", - "parser/item-index": "Outputs an element from an array, map or table", - "parser/element": "Outputs an element from a nested structure", - "parser/namedpipe": "Reads from a Murex named pipe", - "parser/stdin": "Read the stdin belonging to the parent code block", - "parser/file-truncate": "Writes stdin to disk - overwriting contents if file already exists", - "parser/double-quote": "Initiates or terminates a string (variables expanded)", - "parser/scalar": "Expand values as a scalar", - "parser/brace-quote": "Initiates or terminates a string (variables expanded)", - "parser/create-array": "Quickly generate arrays", - "parser/create-object": "Quickly generate objects (dictionaries / maps)", - "parser/logical-and": "Continues next operation if previous operation passes", - "parser/single-quote": "Initiates or terminates a string (variables not expanded)", - "parser/brace-quote-func": "Write a string to the stdout without new line (deprecated)", - "parser/multiply-by": "Multiplies a variable by the right hand value (expression)", - "parser/multiplication": "Multiplies one numeric value with another (expression)", - "parser/add-with": "Adds the right hand value to a variable (expression)", - "parser/addition": "Adds two numeric values together (expression)", - "parser/subtract-by": "Subtracts a variable by the right hand value (expression)", - "parser/pipe-arrow": "Pipes stdout from the left hand command to stdin of the right hand command", - "parser/subtraction": "Subtracts one numeric value from another (expression)", - "parser/divide-by": "Divides a variable by the right hand value (expression)", - "parser/division": "Divides one numeric value from another (expression)", - "parser/assign-or-merge": "Merges the right hand value to a variable on the left hand side (expression)", - "parser/pipe-generic": "Pipes a reformatted stdout stream from the left hand command to stdin of the right hand command", - "parser/equ": "Evaluate a mathematical function (deprecated)", - "parser/file-append": "Writes stdin to disk - appending contents if file already exists", - "parser/elvis": "Returns the right operand if the left operand is falsy (expression)", - "parser/null-coalescing": "Returns the right operand if the left operand is empty / undefined (expression)", - "parser/pipe-err": "Pipes stderr from the left hand command to stdin of the right hand command (DEPRECATED)", - "parser/array": "Expand values as an array", - "parser/lambda": "Iterate through structured data", - "parser/curly-brace": "Initiates or terminates a code block", - "parser/pipe-posix": "Pipes stdout from the left hand command to stdin of the right hand command", - "parser/logical-or": "Continues next operation only if previous operation fails", - "parser/tilde": "Home directory path variable", - "events/oncommandcompletion": "Trigger an event upon a command's completion", - "events/onfilesystemchange": "Add a filesystem watch", - "events/onkeypress": "Custom definable key bindings and macros", - "events/onpreview": "Full screen previews for files and command documentation", - "events/onprompt": "Events triggered by changes in state of the interactive shell", - "events/onsecondselapsed": "Events triggered by time intervals", - "events/onsignalreceived": "Trap OS signals", - "types/generic": "generic (primitive)", - "types/bool": "Boolean (primitive)", - "types/commonlog": "Apache httpd \"common\" log format", - "types/csv": "CSV files (and other character delimited tables)", - "types/float": "Floating point number (primitive)", - "types/hcl": "HashiCorp Configuration Language (HCL)", - "types/int": "Whole number (primitive)", - "types/json": "JavaScript Object Notation (JSON)", - "types/jsonc": "Concatenated JSON", - "types/jsonl": "JSON Lines", - "types/num": "Floating point number (primitive)", - "types/path": "Structured object for working with file and directory paths", - "types/paths": "Structured array for working with `$PATH` style data", - "types/str": "string (primitive)", - "types/toml": "Tom's Obvious, Minimal Language (TOML)", - "types/yaml": "YAML Ain't Markup Language (YAML)", - "types/mxjson": "Murex-flavoured JSON (deprecated)", - "variables/numeric": "Variables who's name is a positive integer, eg `0`, `1`, `2`, `3` and above", - "variables/meta-values": "State information for iteration blocks", - "variables/argv": "Array of the command name and parameters within a given scope", - "variables/columns": "Character width of terminal", - "variables/event_return": "Return values for events", - "variables/home": "Return the home directory for the current session user", - "variables/hostname": "Hostname of the current machine", - "variables/lines": "Character height of terminal", - "variables/logname": "Username for the current session (historic)", - "variables/murex_argv": "Array of the command name and parameters passed to the current shell", - "variables/murex_exe": "Absolute path to running shell", - "variables/oldpwd": "Return the home directory for the current session user", - "variables/params": "Array of the parameters within a given scope", - "variables/pwdhist": "History of each change to the sessions working directory", - "variables/pwd": "Current working directory", - "variables/random": "Return a random 32-bit integer (historical)", - "variables/self": "Meta information about the running scope.", - "variables/shell": "Path of current shell", - "variables/tmpdir": "Return the temporary directory", - "variables/user": "Username for the current session", - "apis/Marshal": "Converts structured memory into a structured file format (eg for stdio)", - "apis/ReadArray": "Read from a data type one array element at a time", - "apis/ReadArrayWithType": "Read from a data type one array element at a time and return the elements contents and data type", - "apis/ReadIndex": "Data type handler for the index, `[`, builtin", - "apis/ReadMap": "Treat data type as a key/value structure and read its contents", - "apis/ReadNotIndex": "Data type handler for the bang-prefixed index, `![`, builtin", - "apis/Unmarshal": "Converts a structured file format into structured memory", - "apis/WriteArray": "Write a data type, one array element at a time", - "apis/lang.ArrayTemplate": "Unmarshals a data type into a Go struct and returns the results as an array", + + "commands/key-code": "Returns character sequences for any key pressed (ie sent from the terminal)", + "commands/addheading": "Adds headings to a table", + "commands/prefix": "Prefix a string to every item in a list", + "commands/suffix": "Prefix a string to every item in a list", + "commands/alias": "Create an alias for a command", + "commands/alter": "Change a value within a structured data-type and pass that change along the pipeline without altering the original source input", + "commands/append": "Add data to the end of an array", + "commands/bg": "Run processes in the background", + "commands/cpuarch": "Output the hosts CPU architecture", + "commands/cpucount": "Output the number of CPU cores available on your host", + "commands/catch": "Handles the exception code raised by `try` or `trypipe`", + "commands/cd": "Change (working) directory", + "commands/list.case": "Changes the character case of a string or all elements in an array", + "commands/bexists": "Check which builtins exist", + "commands/history": "Outputs murex's command history", + "commands/count": "Count items in a map, list or array", + "commands/2darray": "Create a 2D JSON array from multiple input sources", + "commands/ja": "A sophisticated yet simply way to build a JSON array", + "commands/map": "Creates a map from two data sources", + "commands/pipe": "Manage Murex named pipes", + "commands/ta": "A sophisticated yet simple way to build an array of a user defined data-type", + "commands/tmp": "Create a temporary file and write to it", + "commands/datetime": "A date and/or time conversion tool (like `printf` but for date and time values)", + "commands/debug": "Debugging information", + "commands/export": "Define an environmental variable and set it's value", + "commands/args": "Command line flag parser for Murex shell scripting", + "commands/global": "Define a global variable and set it's value", + "commands/openagent": "Creates a handler function for `open`", + "commands/method": "Define a methods supported data-types", + "commands/cast": "Alters the data-type of the previous function without altering its output", + "commands/set": "Define a variable (typically local) and set it's value", + "commands/unsafe": "Execute a block of code, always returning a zero exit number", + "commands/type": "Command type (function, builtin, alias, etc)", + "commands/fid-list": "Lists all running functions within the current Murex session", + "commands/getfile": "Makes a standard HTTP request and return the contents as Murex-aware data type for passing along Murex pipelines.", + "commands/err": "Print a line to the stderr", + "commands/esccli": "Escapes an array so output is valid shell code", + "commands/eschtml": "Encode or decodes text for HTML", + "commands/escurl": "Encode or decodes text for the URL", + "commands/exec": "Runs an executable", + "commands/fexec": "Execute a command or function, bypassing the usual order of precedence.", + "commands/break": "Terminate execution of a block within your processes scope", + "commands/return": "Exits current function scope", + "commands/exit": "Exit murex", + "commands/expr": "Expressions: mathematical, string comparisons, logical operators", + "commands/false": "Returns a `false` value", + "commands/foreach": "Iterate through an array", + "commands/formap": "Iterate through a map or other collection of data", + "commands/for": "A more familiar iteration loop to existing developers", + "commands/fg": "Sends a background process into the foreground", + "commands/runmode": "Alter the scheduler's behaviour at higher scoping level", + "commands/rand": "Random field generator", + "commands/get-type": "Returns the data-type of a variable or pipe", + "commands/exitnum": "Output the exit number of the previous process", + "commands/pt": "Pipe telemetry. Writes data-types and bytes written", + "commands/get": "Makes a standard HTTP request and returns the result as a JSON object", + "commands/g": "Glob pattern matching for file system objects (eg `*.txt`)", + "commands/if": "Conditional statement to execute different blocks of code depending on the result of the condition", + "commands/source": "Import Murex code from another file or code block", + "commands/is-null": "Checks if a variable is null or undefined", + "commands/mjoin": "Joins a list or array into a single string", + "commands/fid-killall": "Terminate all running Murex functions in current session", + "commands/fid-kill": "Terminate a running Murex function", + "commands/left": "Left substring every item in a list", + "commands/f": "Lists or filters file system objects (eg files)", + "commands/which": "Locate command origin", + "commands/lockfile": "Create and manage lock files", + "commands/and": "Returns `true` or `false` depending on whether multiple conditions are met", + "commands/or": "Returns `true` or `false` depending on whether one code-block out of multiple ones supplied is successful or unsuccessful.", + "commands/while": "Loop until condition false", + "commands/man-summary": "Outputs a man page summary of a command", + "commands/match": "Match an exact value in an array", + "commands/event": "Event driven programming for shell scripts", + "commands/murex-package": "Murex's package manager", + "commands/version": "Get Murex version", + "commands/murex-docs": "Displays the man pages for Murex builtins", + "commands/continue": "Terminate process of a block within a caller function", + "commands/not-func": "Reads the stdin and exit number from previous process and not's it's condition", + "commands/devnull": "null function. Similar to /dev/null", + "commands/open": "Open a file with a preferred handler", + "commands/os": "Output the auto-detected OS name", + "commands/out": "Print a string to the stdout with a trailing new line character", + "commands/tout": "Print a string to the stdout and set it's data-type", + "commands/man-get-flags": "Parses man page files for command line flags", + "commands/trypipe": "Checks for non-zero exits of each function in a pipeline", + "commands/post": "HTTP POST request with a JSON-parsable return", + "commands/prepend": "Add data to the start of an array", + "commands/pretty": "Prettifies JSON to make it human readable", + "commands/struct-keys": "Outputs all the keys in a structure as a file path", + "commands/private": "Define a private function block", + "commands/time": "Returns the execution run time of a command or block", + "commands/function": "Define a function block", + "commands/escape": "Escape or unescape input", + "commands/murex-update-exe-list": "Forces Murex to rescan $PATH looking for executables", + "commands/read": "`read` a line of input from the user and store as a variable", + "commands/tread": "`read` a line of input from the user and store as a user defined *typed* variable (deprecated)", + "commands/format": "Reformat one data-type into another data-type", + "commands/rx": "Regexp pattern matching for file system objects (eg `.*\\\\.txt`)", + "commands/regexp": "Regexp tools for arrays / lists of strings", + "commands/open-image": "Renders bitmap image data on your terminal", + "commands/mtac": "Reverse the order of an array", + "commands/right": "Right substring every item in a list", + "commands/round": "Round a number by a user defined precision", + "commands/signal": "Sends a signal RPC", + "commands/summary": "Defines a summary help text for a command", + "commands/config": "Query or define Murex runtime settings", + "commands/runtime": "Returns runtime information on the internal state of Murex", + "commands/test": "Murex's test framework - define tests, run tests and debug shell scripts", + "commands/msort": "Sorts an array - data type agnostic", + "commands/jsplit": "Splits stdin into a JSON array based on a regex parameter", + "commands/trypipeerr": "Checks state of each function in a pipeline and exits block on error", + "commands/tryerr": "Handles errors inside a block of code", + "commands/a": "A sophisticated yet simple way to stream an array or list (mkarray)", + "commands/switch": "Blocks of cascading conditionals", + "commands/autocomplete": "Set definitions for tab-completion in the command line", + "commands/tabulate": "Table transformation tools", + "commands/true": "Returns a `true` value", + "commands/try": "Handles non-zero exits inside a block of code", + "commands/die": "Terminate murex with an exit number of 1 (deprecated)", + "commands/let": "Evaluate a mathematical function and assign to variable (deprecated)", + "commands/murex-parser": "Runs the Murex parser against a block of code", + "mkarray/date": "Create arrays of dates", + "mkarray/character": "Making character arrays (a to z)", + "mkarray/decimal": "Create arrays of decimal integers", + "mkarray/non-decimal": "Create arrays of integers from non-decimal number bases", + "mkarray/special": "Create arrays from ranges of dictionary terms (eg weekdays, months, seasons, etc)", + "optional/select": "Inlining SQL into shell pipelines", + "optional/bz2": "Decompress a bz2 file", + "optional/base64": "Encode or decode a base64 string", + "optional/gz": "Compress or decompress a gzip file", + "optional/qr": "Creates a QR code from stdin", + "optional/sleep": "Suspends the shell for a number of seconds", + "parser/expr-inlined": "Inline expressions", + "parser/c-style-fun": "Inlined commands for expressions and statements", + "parser/range": "Outputs a ranged subset of data from stdin", + "parser/item-index": "Outputs an element from an array, map or table", + "parser/element": "Outputs an element from a nested structure", + "parser/namedpipe": "Reads from a Murex named pipe", + "parser/stdin": "Read the stdin belonging to the parent code block", + "parser/file-truncate": "Writes stdin to disk - overwriting contents if file already exists", + "parser/double-quote": "Initiates or terminates a string (variables expanded)", + "parser/scalar": "Expand values as a scalar", + "parser/brace-quote": "Initiates or terminates a string (variables expanded)", + "parser/create-array": "Quickly generate arrays", + "parser/create-object": "Quickly generate objects (dictionaries / maps)", + "parser/logical-and": "Continues next operation if previous operation passes", + "parser/single-quote": "Initiates or terminates a string (variables not expanded)", + "parser/brace-quote-func": "Write a string to the stdout without new line (deprecated)", + "parser/multiply-by": "Multiplies a variable by the right hand value (expression)", + "parser/multiplication": "Multiplies one numeric value with another (expression)", + "parser/add-with": "Adds the right hand value to a variable (expression)", + "parser/addition": "Adds two numeric values together (expression)", + "parser/subtract-by": "Subtracts a variable by the right hand value (expression)", + "parser/pipe-arrow": "Pipes stdout from the left hand command to stdin of the right hand command", + "parser/subtraction": "Subtracts one numeric value from another (expression)", + "parser/divide-by": "Divides a variable by the right hand value (expression)", + "parser/division": "Divides one numeric value from another (expression)", + "parser/assign-or-merge": "Merges the right hand value to a variable on the left hand side (expression)", + "parser/pipe-generic": "Pipes a reformatted stdout stream from the left hand command to stdin of the right hand command", + "parser/equ": "Evaluate a mathematical function (deprecated)", + "parser/file-append": "Writes stdin to disk - appending contents if file already exists", + "parser/elvis": "Returns the right operand if the left operand is falsy (expression)", + "parser/null-coalescing": "Returns the right operand if the left operand is empty / undefined (expression)", + "parser/pipe-err": "Pipes stderr from the left hand command to stdin of the right hand command (DEPRECATED)", + "parser/array": "Expand values as an array", + "parser/lambda": "Iterate through structured data", + "parser/curly-brace": "Initiates or terminates a code block", + "parser/pipe-posix": "Pipes stdout from the left hand command to stdin of the right hand command", + "parser/logical-or": "Continues next operation only if previous operation fails", + "parser/tilde": "Home directory path variable", + "events/oncommandcompletion": "Trigger an event upon a command's completion", + "events/onfilesystemchange": "Add a filesystem watch", + "events/onkeypress": "Custom definable key bindings and macros", + "events/onpreview": "Full screen previews for files and command documentation", + "events/onprompt": "Events triggered by changes in state of the interactive shell", + "events/onsecondselapsed": "Events triggered by time intervals", + "events/onsignalreceived": "Trap OS signals", + "types/generic": "generic (primitive)", + "types/bool": "Boolean (primitive)", + "types/commonlog": "Apache httpd \"common\" log format", + "types/csv": "CSV files (and other character delimited tables)", + "types/float": "Floating point number (primitive)", + "types/hcl": "HashiCorp Configuration Language (HCL)", + "types/int": "Whole number (primitive)", + "types/json": "JavaScript Object Notation (JSON)", + "types/jsonc": "Concatenated JSON", + "types/jsonl": "JSON Lines", + "types/num": "Floating point number (primitive)", + "types/path": "Structured object for working with file and directory paths", + "types/paths": "Structured array for working with `$PATH` style data", + "types/str": "string (primitive)", + "types/toml": "Tom's Obvious, Minimal Language (TOML)", + "types/yaml": "YAML Ain't Markup Language (YAML)", + "types/mxjson": "Murex-flavoured JSON (deprecated)", + "variables/numeric": "Variables who's name is a positive integer, eg `0`, `1`, `2`, `3` and above", + "variables/meta-values": "State information for iteration blocks", + "variables/argv": "Array of the command name and parameters within a given scope", + "variables/columns": "Character width of terminal", + "variables/event_return": "Return values for events", + "variables/home": "Return the home directory for the current session user", + "variables/hostname": "Hostname of the current machine", + "variables/lines": "Character height of terminal", + "variables/logname": "Username for the current session (historic)", + "variables/murex_argv": "Array of the command name and parameters passed to the current shell", + "variables/murex_exe": "Absolute path to running shell", + "variables/oldpwd": "Return the home directory for the current session user", + "variables/params": "Array of the parameters within a given scope", + "variables/pwdhist": "History of each change to the sessions working directory", + "variables/pwd": "Current working directory", + "variables/random": "Return a random 32-bit integer (historical)", + "variables/self": "Meta information about the running scope.", + "variables/shell": "Path of current shell", + "variables/tmpdir": "Return the temporary directory", + "variables/user": "Username for the current session", + "apis/Marshal": "Converts structured memory into a structured file format (eg for stdio)", + "apis/ReadArray": "Read from a data type one array element at a time", + "apis/ReadArrayWithType": "Read from a data type one array element at a time and return the elements contents and data type", + "apis/ReadIndex": "Data type handler for the index, `[`, builtin", + "apis/ReadMap": "Treat data type as a key/value structure and read its contents", + "apis/ReadNotIndex": "Data type handler for the bang-prefixed index, `![`, builtin", + "apis/Unmarshal": "Converts a structured file format into structured memory", + "apis/WriteArray": "Write a data type, one array element at a time", + "apis/lang.ArrayTemplate": "Unmarshals a data type into a Go struct and returns the results as an array", "apis/lang.ArrayWithTypeTemplate": "Unmarshals a data type into a Go struct and returns the results as an array with data type included", - "apis/lang.IndexTemplateObject": "Returns element(s) from a data structure", - "apis/lang.IndexTemplateTable": "Returns element(s) from a table", - "apis/lang.MarshalData": "Converts structured memory into a Murex data-type (eg for stdio)", - "apis/lang.UnmarshalData": "Converts a Murex data-type into structured memory", - "user-guide/ansi": "Infixed constants that return ANSI escape sequences", - "user-guide/bang-prefix": "Bang prefixing to reverse default actions", - "user-guide/code-block": "Overview of how code blocks are parsed", - "user-guide/fileref": "How to track what code was loaded and from where", - "user-guide/hint-text": "A status bar for your shell", - "user-guide/integrations": "Default integrations shipped with Murex", - "user-guide/interactive-shell": "What's different about Murex's interactive shell?", - "user-guide/job-control": "How to manage jobs with Murex", - "user-guide/modules": "An introduction to Murex modules and packages", - "user-guide/namedpipes": "A detailed breakdown of named pipes in Murex", + "apis/lang.IndexTemplateObject": "Returns element(s) from a data structure", + "apis/lang.IndexTemplateTable": "Returns element(s) from a table", + "apis/lang.MarshalData": "Converts structured memory into a Murex data-type (eg for stdio)", + "apis/lang.UnmarshalData": "Converts a Murex data-type into structured memory", + "user-guide/ansi": "Infixed constants that return ANSI escape sequences", + "user-guide/bang-prefix": "Bang prefixing to reverse default actions", + "user-guide/code-block": "Overview of how code blocks are parsed", + "user-guide/fileref": "How to track what code was loaded and from where", + "user-guide/hint-text": "A status bar for your shell", + "user-guide/integrations": "Default integrations shipped with Murex", + "user-guide/interactive-shell": "What's different about Murex's interactive shell?", + "user-guide/job-control": "How to manage jobs with Murex", + "user-guide/modules": "Modules and packages: An Introduction", + "user-guide/namedpipes": "A detailed breakdown of named pipes in Murex", "user-guide/operators-and-tokens": "All supported operators and tokens", - "user-guide/pipeline": "Overview of what a \"pipeline\" is", - "user-guide/profile": "A breakdown of the different files loaded on start up", - "user-guide/reserved-vars": "Special variables reserved by Murex", - "user-guide/rosetta-stone": "A tabulated list of Bashism's and their equivalent Murex syntax", - "user-guide/schedulers": "Overview of the different schedulers (or 'run modes') in Murex", - "user-guide/strict-types": "Expressions can auto-convert types or strictly honour data types", - "user-guide/terminal-keys": "A list of all the terminal hotkeys and their uses", - "user-guide/scoping": "How scoping works within Murex", - "integrations/chatgpt": "How to enable ChatGPT hints", - "integrations/cheatsh": "Cheatsheets provided by cheat.sh", - "integrations/kitty": "Get more out of Kitty terminal emulator", - "integrations/make": "`make` integrations", - "integrations/man-pages": "Linux/UNIX `man` page integrations", - "integrations/spellcheck": "How to enable inline spellchecking", - "integrations/terminology": "Get more out of Terminology terminal emulator", - "integrations/direnv": "Directory specific environmental variables", - "integrations/yarn": "Working with `yarn` and `package.json`", - "integrations/iterm2": "Get more out of iTerm2 terminal emulator", - "changelog/v2.0": "This release comes with spellchecking, inlined images, smarter syntax completion and more", - "changelog/v2.1": "This release comes with support for inlining SQL and some major bug fixes plus a breaking change for `config`. Please read for details.", - "changelog/v2.10": "This release brings a few minor improvements and bug fixes rather than big new headline features.", - "changelog/v2.11": "This release mainly focuses on refinements in performance and usability, rather than introducing new features", - "changelog/v2.2": "This is mainly a bug fix release but it does include one breaking change for `config`. Please read for details.", - "changelog/v2.3": "This release includes significant changes to the interactive terminal", - "changelog/v2.4": "This release introduces a strict mode for variables, new builtin, performance improvements, and better error messages; plus a potential breaking change", - "changelog/v2.5": "This release introduces a number of new builtins, fixes some regression bugs and supercharges the `select` optional builtin (which I plan to include into the core builtins for non-Windows users in the next release).", - "changelog/v2.6": "This update has introduced a potential breaking change: variables now need to be defined before usage otherwise the commandline will fail. Read notes to learn how to disable this feature where needed. Also included in this release is the `select` command as part of the standard build.", - "changelog/v2.7": "This update has introduced another potential breaking change for your safety: zero length arrays now fail by default. Also errors inside subshells will cause the parent command to fail if ran inside a `try` or `trypipe` block.", - "changelog/v2.8": "This release comes with a number of experimental but stable features that might eventually become standard practice. The features are there to use if you with but adjacent from the older code so there is zero risk in updating to this version.", - "changelog/v2.9": "This release focuses on testing and REPL usability improvements but also includes updates several new run modes to make error handling easier in larger scripts.", - "changelog/v3.0": "This is a major release that brings a significant number of changes and improvements, including a complete overhaul of the parser. Backwards compatibility is a high priority however these new features bring greater readability and consistency to shell scripting. So while the older syntax remains for compatibility, it is worth migrating over to the newer syntax for all new code being written", - "changelog/v3.1": "This release includes mostly bug fixes and new experimental features which are opt into. To enable all experimental features, set the environmental variable `MUREX_EXPERIMENTAL` to any value. Or you can enable specific features individually via `config`", - "changelog/v4.0": "This release sees significant improvements for use with non-latin characters in both the interactive prompt and shell scripting. It introduces new syntax to make working with structured data even easier than before. As well as new data types and smoother user experience.", - "changelog/v4.1": "The previous releases have brought significant advancements to Murex's syntax but at the cost of longer gaps between releases. So the 4.1.x versions will be shorter releases but focusing on bug fixes. The 4.1.x release notes will be appended to [murex.rocks changelog](https://murex.rocks/changelog/v4.1.html) and available on [Github releases](https://github.com/lmorg/murex/releases) too", - "changelog/v4.2": "Murex usage has raised considerably in recent weeks. This release addresses a number of feature requests and bugs raised on Github.", - "changelog/v4.3": "This brings improved support on Windows plus one breaking change from the previous release (v4.2)", - "changelog/v4.4": "v4.4 features two new builtins, improvements in testing, and automatic generation of autocompletion suggestions backed by man page parsing. Plus there has been a lot of focus on improving _readline_ responsiveness", - "changelog/v5.0": "v5.0 is a massive release. It brings along changes to syntax, new operators as well as new builtins, reserved variables and a new event", - "changelog/v5.1": "This release brings new operators and a builtin, all for managing null types. There is also a substantial revamp to readline's responsiveness.", - "changelog/v5.2": "The v5.2 release introduces significant new features and improvements for those using Murex as their interactive shell. Many of these features are unique to Murex.", - "changelog/v5.3": "Caching has been vastly improved in this release due to a new sqlite3-backed persistent `cache.db`. There have also been some improvements to `[f1]` help pages", - "changelog/v6.0": "Despite this being a new major version release, it is a vary minor update. Aside from a handful of bugfixes, the most significant change is notice of deprecation for `=`, `let`, and `?`.", - "changelog/v6.1": "This release sees a massive jump in event-driven capabilities as well as several new features and bug fixes.", - "changelog/v6.2": "Bug fix release", - "changelog/v6.3": "This is a massive release ahead of the v7.0. This brings notifications of new deprecations, new builtins, new flags, improved CI/CD flow, and changes to the website. Unfortunately it also carries 3 breaking changes.", + "user-guide/pipeline": "Overview of what a \"pipeline\" is", + "user-guide/profile": "A breakdown of the different files loaded on start up", + "user-guide/reserved-vars": "Special variables reserved by Murex", + "user-guide/rosetta-stone": "A tabulated list of Bashism's and their equivalent Murex syntax", + "user-guide/schedulers": "Overview of the different schedulers (or 'run modes') in Murex", + "user-guide/strict-types": "Expressions can auto-convert types or strictly honour data types", + "user-guide/terminal-keys": "A list of all the terminal hotkeys and their uses", + "user-guide/scoping": "How scoping works within Murex", + "user-guide/murex-arrays": "Examples using arrays within Murex", + "integrations/chatgpt": "How to enable ChatGPT hints", + "integrations/cheatsh": "Cheatsheets provided by cheat.sh", + "integrations/kitty": "Get more out of Kitty terminal emulator", + "integrations/make": "`make` integrations", + "integrations/man-pages": "Linux/UNIX `man` page integrations", + "integrations/spellcheck": "How to enable inline spellchecking", + "integrations/terminology": "Get more out of Terminology terminal emulator", + "integrations/direnv": "Directory specific environmental variables", + "integrations/yarn": "Working with `yarn` and `package.json`", + "integrations/iterm2": "Get more out of iTerm2 terminal emulator", + "changelog/v2.0": "This release comes with spellchecking, inlined images, smarter syntax completion and more", + "changelog/v2.1": "This release comes with support for inlining SQL and some major bug fixes plus a breaking change for `config`. Please read for details.", + "changelog/v2.10": "This release brings a few minor improvements and bug fixes rather than big new headline features.", + "changelog/v2.11": "This release mainly focuses on refinements in performance and usability, rather than introducing new features", + "changelog/v2.2": "This is mainly a bug fix release but it does include one breaking change for `config`. Please read for details.", + "changelog/v2.3": "This release includes significant changes to the interactive terminal", + "changelog/v2.4": "This release introduces a strict mode for variables, new builtin, performance improvements, and better error messages; plus a potential breaking change", + "changelog/v2.5": "This release introduces a number of new builtins, fixes some regression bugs and supercharges the `select` optional builtin (which I plan to include into the core builtins for non-Windows users in the next release).", + "changelog/v2.6": "This update has introduced a potential breaking change: variables now need to be defined before usage otherwise the commandline will fail. Read notes to learn how to disable this feature where needed. Also included in this release is the `select` command as part of the standard build.", + "changelog/v2.7": "This update has introduced another potential breaking change for your safety: zero length arrays now fail by default. Also errors inside subshells will cause the parent command to fail if ran inside a `try` or `trypipe` block.", + "changelog/v2.8": "This release comes with a number of experimental but stable features that might eventually become standard practice. The features are there to use if you with but adjacent from the older code so there is zero risk in updating to this version.", + "changelog/v2.9": "This release focuses on testing and REPL usability improvements but also includes updates several new run modes to make error handling easier in larger scripts.", + "changelog/v3.0": "This is a major release that brings a significant number of changes and improvements, including a complete overhaul of the parser. Backwards compatibility is a high priority however these new features bring greater readability and consistency to shell scripting. So while the older syntax remains for compatibility, it is worth migrating over to the newer syntax for all new code being written", + "changelog/v3.1": "This release includes mostly bug fixes and new experimental features which are opt into. To enable all experimental features, set the environmental variable `MUREX_EXPERIMENTAL` to any value. Or you can enable specific features individually via `config`", + "changelog/v4.0": "This release sees significant improvements for use with non-latin characters in both the interactive prompt and shell scripting. It introduces new syntax to make working with structured data even easier than before. As well as new data types and smoother user experience.", + "changelog/v4.1": "The previous releases have brought significant advancements to Murex's syntax but at the cost of longer gaps between releases. So the 4.1.x versions will be shorter releases but focusing on bug fixes. The 4.1.x release notes will be appended to [murex.rocks changelog](https://murex.rocks/changelog/v4.1.html) and available on [Github releases](https://github.com/lmorg/murex/releases) too", + "changelog/v4.2": "Murex usage has raised considerably in recent weeks. This release addresses a number of feature requests and bugs raised on Github.", + "changelog/v4.3": "This brings improved support on Windows plus one breaking change from the previous release (v4.2)", + "changelog/v4.4": "v4.4 features two new builtins, improvements in testing, and automatic generation of autocompletion suggestions backed by man page parsing. Plus there has been a lot of focus on improving _readline_ responsiveness", + "changelog/v5.0": "v5.0 is a massive release. It brings along changes to syntax, new operators as well as new builtins, reserved variables and a new event", + "changelog/v5.1": "This release brings new operators and a builtin, all for managing null types. There is also a substantial revamp to readline's responsiveness.", + "changelog/v5.2": "The v5.2 release introduces significant new features and improvements for those using Murex as their interactive shell. Many of these features are unique to Murex.", + "changelog/v5.3": "Caching has been vastly improved in this release due to a new sqlite3-backed persistent `cache.db`. There have also been some improvements to `[f1]` help pages", + "changelog/v6.0": "Despite this being a new major version release, it is a vary minor update. Aside from a handful of bugfixes, the most significant change is notice of deprecation for `=`, `let`, and `?`.", + "changelog/v6.1": "This release sees a massive jump in event-driven capabilities as well as several new features and bug fixes.", + "changelog/v6.2": "Bug fix release", + "changelog/v6.3": "This is a massive release ahead of the v7.0. This brings notifications of new deprecations, new builtins, new flags, improved CI/CD flow, and changes to the website. Unfortunately it also carries 3 breaking changes.", + } + Synonym = map[string]string{ - "key-code": "key-code", - "addheading": "addheading", - "prefix": "prefix", - "list.prefix": "prefix", - "suffix": "suffix", - "list.suffix": "suffix", - "alias": "alias", - "!alias": "alias", - "alter": "alter", - "~>": "alter", - "append": "append", - "list.append": "append", - "bg": "bg", - "cpuarch": "cpuarch", - "sys.cpu.arch": "cpuarch", - "cpucount": "cpucount", - "sys.cpu.count": "cpucount", - "catch": "catch", - "!catch": "catch", - "cd": "cd", - "list.case": "list.case", - "bexists": "bexists", - "history": "history", - "count": "count", - "len": "count", - "2darray": "2darray", - "ja": "ja", - "map": "map", - "pipe": "pipe", - "!pipe": "pipe", - "ta": "ta", - "tmp": "tmp", - "datetime": "datetime", - "str.datetime": "datetime", - "debug": "debug", - "export": "export", - "!export": "export", - "unset": "export", - "var.env": "export", - "!var.env": "export", - "args": "args", - "global": "global", - "!global": "global", - "openagent": "openagent", - "!openagent": "openagent", - "method": "method", - "cast": "cast", - "set": "set", - "!set": "set", - "unsafe": "unsafe", - "type": "type", - "fid-list": "fid-list", - "jobs": "fid-list", - "getfile": "getfile", - "err": "err", - "esccli": "esccli", - "eschtml": "eschtml", - "!eschtml": "eschtml", - "escurl": "escurl", - "!escurl": "escurl", - "exec": "exec", - "command": "exec", - "exec.file": "exec", - "fexec": "fexec", - "builtin": "fexec", - "exec.builtin": "fexec", - "exec.function": "fexec", - "exec.private": "fexec", - "break": "break", - "return": "return", - "exit": "exit", - "expr": "expr", - "false": "false", - "foreach": "foreach", - "formap": "formap", - "for": "for", - "fg": "fg", - "runmode": "runmode", - "rand": "rand", - "get-type": "get-type", - "exitnum": "exitnum", - "pt": "pt", - "get": "get", - "g": "g", - "!g": "g", - "if": "if", - "!if": "if", - "source": "source", - ".": "source", - "is-null": "is-null", - "mjoin": "mjoin", - "list.join": "mjoin", - "fid-killall": "fid-killall", - "fid-kill": "fid-kill", - "left": "left", - "list.left": "left", - "f": "f", - "which": "which", - "lockfile": "lockfile", - "and": "and", - "!and": "and", - "or": "or", - "!or": "or", - "while": "while", - "!while": "while", - "man-summary": "man-summary", - "help.man.summary": "man-summary", - "match": "match", - "!match": "match", - "list.str": "match", - "!list.str": "match", - "event": "event", - "!event": "event", - "murex-package": "murex-package", - "version": "version", - "murex-docs": "murex-docs", - "help": "murex-docs", - "continue": "continue", - "!": "not-func", - "not": "not-func", - "null": "devnull", - "open": "open", - "os": "os", - "sys.os": "os", - "out": "out", - "echo": "out", - "tout": "tout", - "man-get-flags": "man-get-flags", - "trypipe": "trypipe", - "post": "post", - "prepend": "prepend", - "list.prepend": "prepend", - "pretty": "pretty", - "struct-keys": "struct-keys", - "private": "private", - "time": "time", - "function": "function", - "!function": "function", - "escape": "escape", - "!escape": "escape", + + "key-code": "key-code", + "addheading": "addheading", + "prefix": "prefix", + "list.prefix": "prefix", + "suffix": "suffix", + "list.suffix": "suffix", + "alias": "alias", + "!alias": "alias", + "alter": "alter", + "~>": "alter", + "append": "append", + "list.append": "append", + "bg": "bg", + "cpuarch": "cpuarch", + "sys.cpu.arch": "cpuarch", + "cpucount": "cpucount", + "sys.cpu.count": "cpucount", + "catch": "catch", + "!catch": "catch", + "cd": "cd", + "list.case": "list.case", + "bexists": "bexists", + "history": "history", + "count": "count", + "len": "count", + "2darray": "2darray", + "ja": "ja", + "map": "map", + "pipe": "pipe", + "!pipe": "pipe", + "ta": "ta", + "tmp": "tmp", + "datetime": "datetime", + "str.datetime": "datetime", + "debug": "debug", + "export": "export", + "!export": "export", + "unset": "export", + "var.env": "export", + "!var.env": "export", + "args": "args", + "global": "global", + "!global": "global", + "openagent": "openagent", + "!openagent": "openagent", + "method": "method", + "cast": "cast", + "set": "set", + "!set": "set", + "unsafe": "unsafe", + "type": "type", + "fid-list": "fid-list", + "jobs": "fid-list", + "getfile": "getfile", + "err": "err", + "esccli": "esccli", + "eschtml": "eschtml", + "!eschtml": "eschtml", + "escurl": "escurl", + "!escurl": "escurl", + "exec": "exec", + "command": "exec", + "exec.file": "exec", + "fexec": "fexec", + "builtin": "fexec", + "exec.builtin": "fexec", + "exec.function": "fexec", + "exec.private": "fexec", + "break": "break", + "return": "return", + "exit": "exit", + "expr": "expr", + "false": "false", + "foreach": "foreach", + "formap": "formap", + "for": "for", + "fg": "fg", + "runmode": "runmode", + "rand": "rand", + "get-type": "get-type", + "exitnum": "exitnum", + "pt": "pt", + "get": "get", + "g": "g", + "!g": "g", + "if": "if", + "!if": "if", + "source": "source", + ".": "source", + "is-null": "is-null", + "mjoin": "mjoin", + "list.join": "mjoin", + "fid-killall": "fid-killall", + "fid-kill": "fid-kill", + "left": "left", + "list.left": "left", + "f": "f", + "which": "which", + "lockfile": "lockfile", + "and": "and", + "!and": "and", + "or": "or", + "!or": "or", + "while": "while", + "!while": "while", + "man-summary": "man-summary", + "help.man.summary": "man-summary", + "match": "match", + "!match": "match", + "list.str": "match", + "!list.str": "match", + "event": "event", + "!event": "event", + "murex-package": "murex-package", + "version": "version", + "murex-docs": "murex-docs", + "help": "murex-docs", + "continue": "continue", + "!": "not-func", + "not": "not-func", + "null": "devnull", + "open": "open", + "os": "os", + "sys.os": "os", + "out": "out", + "echo": "out", + "tout": "tout", + "man-get-flags": "man-get-flags", + "trypipe": "trypipe", + "post": "post", + "prepend": "prepend", + "list.prepend": "prepend", + "pretty": "pretty", + "struct-keys": "struct-keys", + "private": "private", + "time": "time", + "function": "function", + "!function": "function", + "escape": "escape", + "!escape": "escape", "murex-update-exe-list": "murex-update-exe-list", - "read": "read", - "tread": "tread", - "format": "format", - "rx": "rx", - "!rx": "rx", - "regexp": "regexp", - "!regexp": "regexp", - "list.regex": "regexp", - "!list.regex": "regexp", - "open-image": "open-image", - "mtac": "mtac", - "list.reverse": "mtac", - "right": "right", - "list.right": "right", - "round": "round", - "num.round": "round", - "signal": "signal", - "summary": "summary", - "!summary": "summary", - "config": "config", - "!config": "config", - "runtime": "runtime", - "builtins": "runtime", - "shell.runtime": "runtime", - "test": "test", - "!test": "test", - "msort": "msort", - "list.sort": "msort", - "jsplit": "jsplit", - "str.split": "jsplit", - "trypipeerr": "trypipeerr", - "tryerr": "tryerr", - "a": "a", - "mkarray": "a", - "switch": "switch", - "autocomplete": "autocomplete", - "tabulate": "tabulate", - "true": "true", - "try": "try", - "die": "die", - "let": "let", - "murex-parser": "murex-parser", + "read": "read", + "tread": "tread", + "format": "format", + "rx": "rx", + "!rx": "rx", + "regexp": "regexp", + "!regexp": "regexp", + "list.regex": "regexp", + "!list.regex": "regexp", + "open-image": "open-image", + "mtac": "mtac", + "list.reverse": "mtac", + "right": "right", + "list.right": "right", + "round": "round", + "num.round": "round", + "signal": "signal", + "summary": "summary", + "!summary": "summary", + "config": "config", + "!config": "config", + "runtime": "runtime", + "builtins": "runtime", + "shell.runtime": "runtime", + "test": "test", + "!test": "test", + "msort": "msort", + "list.sort": "msort", + "jsplit": "jsplit", + "str.split": "jsplit", + "trypipeerr": "trypipeerr", + "tryerr": "tryerr", + "a": "a", + "mkarray": "a", + "switch": "switch", + "autocomplete": "autocomplete", + "tabulate": "tabulate", + "true": "true", + "try": "try", + "die": "die", + "let": "let", + "murex-parser": "murex-parser", - "select": "optional/select", + + "select": "optional/select", "table.select": "optional/select", - "!bz2": "optional/bz2", - "base64": "optional/base64", - "!base64": "optional/base64", - "gz": "optional/gz", - "!gz": "optional/gz", - "qr": "optional/qr", - "sleep": "optional/sleep", + "!bz2": "optional/bz2", + "base64": "optional/base64", + "!base64": "optional/base64", + "gz": "optional/gz", + "!gz": "optional/gz", + "qr": "optional/qr", + "sleep": "optional/sleep", - "expr-inlined": "parser/expr-inlined", - "c-style-fun": "parser/c-style-fun", - "@[": "parser/range", - "[": "parser/item-index", - "![": "parser/item-index", - "item-index": "parser/item-index", - "index": "parser/item-index", - "[[": "parser/element", - "element": "parser/element", + + "expr-inlined": "parser/expr-inlined", + "c-style-fun": "parser/c-style-fun", + "@[": "parser/range", + "[": "parser/item-index", + "![": "parser/item-index", + "item-index": "parser/item-index", + "index": "parser/item-index", + "[[": "parser/element", + "element": "parser/element", "(murex named pipe)": "parser/namedpipe", - "<>": "parser/namedpipe", - "read-named-pipe": "parser/namedpipe", - "": "parser/stdin", - ">": "parser/file-truncate", - "|>": "parser/file-truncate", - "fwrite": "parser/file-truncate", - "double-quote": "parser/double-quote", - "scalar": "parser/scalar", - "brace-quote": "parser/brace-quote", - "create-array": "parser/create-array", - "create-object": "parser/create-object", - "logical-and": "parser/logical-and", - "single-quote": "parser/single-quote", - "(": "parser/brace-quote-func", - "multiply-by": "parser/multiply-by", - "multiplication": "parser/multiplication", - "add-with": "parser/add-with", - "addition": "parser/addition", - "subtract-by": "parser/subtract-by", - "pipe-arrow": "parser/pipe-arrow", - "subtraction": "parser/subtraction", - "divide-by": "parser/divide-by", - "division": "parser/division", - "assign-or-merge": "parser/assign-or-merge", - "pipe-generic": "parser/pipe-generic", - "=": "parser/equ", - ">>": "parser/file-append", - "fappend": "parser/file-append", - "elvis": "parser/elvis", - "null-coalescing": "parser/null-coalescing", - "pipe-err": "parser/pipe-err", - "array": "parser/array", - "lambda": "parser/lambda", - "curly-brace": "parser/curly-brace", - "pipe-posix": "parser/pipe-posix", - "logical-or": "parser/logical-or", - "tilde": "parser/tilde", + "<>": "parser/namedpipe", + "read-named-pipe": "parser/namedpipe", + "": "parser/stdin", + ">": "parser/file-truncate", + "|>": "parser/file-truncate", + "fwrite": "parser/file-truncate", + "double-quote": "parser/double-quote", + "scalar": "parser/scalar", + "brace-quote": "parser/brace-quote", + "create-array": "parser/create-array", + "create-object": "parser/create-object", + "logical-and": "parser/logical-and", + "single-quote": "parser/single-quote", + "(": "parser/brace-quote-func", + "multiply-by": "parser/multiply-by", + "multiplication": "parser/multiplication", + "add-with": "parser/add-with", + "addition": "parser/addition", + "subtract-by": "parser/subtract-by", + "pipe-arrow": "parser/pipe-arrow", + "subtraction": "parser/subtraction", + "divide-by": "parser/divide-by", + "division": "parser/division", + "assign-or-merge": "parser/assign-or-merge", + "pipe-generic": "parser/pipe-generic", + "=": "parser/equ", + ">>": "parser/file-append", + "fappend": "parser/file-append", + "elvis": "parser/elvis", + "null-coalescing": "parser/null-coalescing", + "pipe-err": "parser/pipe-err", + "array": "parser/array", + "lambda": "parser/lambda", + "curly-brace": "parser/curly-brace", + "pipe-posix": "parser/pipe-posix", + "logical-or": "parser/logical-or", + "tilde": "parser/tilde", - "commands/key-code": "commands/key-code", - "commands/addheading": "commands/addheading", - "commands/prefix": "commands/prefix", - "commands/list.prefix": "commands/prefix", - "commands/suffix": "commands/suffix", - "commands/list.suffix": "commands/suffix", - "commands/alias": "commands/alias", - "commands/!alias": "commands/alias", - "commands/alter": "commands/alter", - "commands/~>": "commands/alter", - "commands/append": "commands/append", - "commands/list.append": "commands/append", - "commands/bg": "commands/bg", - "commands/cpuarch": "commands/cpuarch", - "commands/sys.cpu.arch": "commands/cpuarch", - "commands/cpucount": "commands/cpucount", - "commands/sys.cpu.count": "commands/cpucount", - "commands/catch": "commands/catch", - "commands/!catch": "commands/catch", - "commands/cd": "commands/cd", - "commands/list.case": "commands/list.case", - "commands/bexists": "commands/bexists", - "commands/history": "commands/history", - "commands/count": "commands/count", - "commands/len": "commands/count", - "commands/2darray": "commands/2darray", - "commands/ja": "commands/ja", - "commands/map": "commands/map", - "commands/pipe": "commands/pipe", - "commands/!pipe": "commands/pipe", - "commands/ta": "commands/ta", - "commands/tmp": "commands/tmp", - "commands/datetime": "commands/datetime", - "commands/str.datetime": "commands/datetime", - "commands/debug": "commands/debug", - "commands/export": "commands/export", - "commands/!export": "commands/export", - "commands/unset": "commands/export", - "commands/var.env": "commands/export", - "commands/!var.env": "commands/export", - "commands/args": "commands/args", - "commands/global": "commands/global", - "commands/!global": "commands/global", - "commands/openagent": "commands/openagent", - "commands/!openagent": "commands/openagent", - "commands/method": "commands/method", - "commands/cast": "commands/cast", - "commands/set": "commands/set", - "commands/!set": "commands/set", - "commands/unsafe": "commands/unsafe", - "commands/type": "commands/type", - "commands/fid-list": "commands/fid-list", - "commands/jobs": "commands/fid-list", - "commands/getfile": "commands/getfile", - "commands/err": "commands/err", - "commands/esccli": "commands/esccli", - "commands/eschtml": "commands/eschtml", - "commands/!eschtml": "commands/eschtml", - "commands/escurl": "commands/escurl", - "commands/!escurl": "commands/escurl", - "commands/exec": "commands/exec", - "commands/command": "commands/exec", - "commands/exec.file": "commands/exec", - "commands/fexec": "commands/fexec", - "commands/builtin": "commands/fexec", - "commands/exec.builtin": "commands/fexec", - "commands/exec.function": "commands/fexec", - "commands/exec.private": "commands/fexec", - "commands/break": "commands/break", - "commands/return": "commands/return", - "commands/exit": "commands/exit", - "commands/expr": "commands/expr", - "commands/false": "commands/false", - "commands/foreach": "commands/foreach", - "commands/formap": "commands/formap", - "commands/for": "commands/for", - "commands/fg": "commands/fg", - "commands/runmode": "commands/runmode", - "commands/rand": "commands/rand", - "commands/get-type": "commands/get-type", - "commands/exitnum": "commands/exitnum", - "commands/pt": "commands/pt", - "commands/get": "commands/get", - "commands/g": "commands/g", - "commands/!g": "commands/g", - "commands/if": "commands/if", - "commands/!if": "commands/if", - "commands/source": "commands/source", - "commands/.": "commands/source", - "commands/is-null": "commands/is-null", - "commands/mjoin": "commands/mjoin", - "commands/list.join": "commands/mjoin", - "commands/fid-killall": "commands/fid-killall", - "commands/fid-kill": "commands/fid-kill", - "commands/left": "commands/left", - "commands/list.left": "commands/left", - "commands/f": "commands/f", - "commands/which": "commands/which", - "commands/lockfile": "commands/lockfile", - "commands/and": "commands/and", - "commands/!and": "commands/and", - "commands/or": "commands/or", - "commands/!or": "commands/or", - "commands/while": "commands/while", - "commands/!while": "commands/while", - "commands/man-summary": "commands/man-summary", - "commands/help.man.summary": "commands/man-summary", - "commands/match": "commands/match", - "commands/!match": "commands/match", - "commands/list.str": "commands/match", - "commands/!list.str": "commands/match", - "commands/event": "commands/event", - "commands/!event": "commands/event", - "commands/murex-package": "commands/murex-package", - "commands/version": "commands/version", - "commands/murex-docs": "commands/murex-docs", - "commands/help": "commands/murex-docs", - "commands/continue": "commands/continue", - "commands/!": "commands/not-func", - "commands/not": "commands/not-func", - "commands/null": "commands/devnull", - "commands/open": "commands/open", - "commands/os": "commands/os", - "commands/sys.os": "commands/os", - "commands/out": "commands/out", - "commands/echo": "commands/out", - "commands/tout": "commands/tout", - "commands/man-get-flags": "commands/man-get-flags", - "commands/trypipe": "commands/trypipe", - "commands/post": "commands/post", - "commands/prepend": "commands/prepend", - "commands/list.prepend": "commands/prepend", - "commands/pretty": "commands/pretty", - "commands/struct-keys": "commands/struct-keys", - "commands/private": "commands/private", - "commands/time": "commands/time", - "commands/function": "commands/function", - "commands/!function": "commands/function", - "commands/escape": "commands/escape", - "commands/!escape": "commands/escape", - "commands/murex-update-exe-list": "commands/murex-update-exe-list", - "commands/read": "commands/read", - "commands/tread": "commands/tread", - "commands/format": "commands/format", - "commands/rx": "commands/rx", - "commands/!rx": "commands/rx", - "commands/regexp": "commands/regexp", - "commands/!regexp": "commands/regexp", - "commands/list.regex": "commands/regexp", - "commands/!list.regex": "commands/regexp", - "commands/open-image": "commands/open-image", - "commands/mtac": "commands/mtac", - "commands/list.reverse": "commands/mtac", - "commands/right": "commands/right", - "commands/list.right": "commands/right", - "commands/round": "commands/round", - "commands/num.round": "commands/round", - "commands/signal": "commands/signal", - "commands/summary": "commands/summary", - "commands/!summary": "commands/summary", - "commands/config": "commands/config", - "commands/!config": "commands/config", - "commands/runtime": "commands/runtime", - "commands/builtins": "commands/runtime", - "commands/shell.runtime": "commands/runtime", - "commands/test": "commands/test", - "commands/!test": "commands/test", - "commands/msort": "commands/msort", - "commands/list.sort": "commands/msort", - "commands/jsplit": "commands/jsplit", - "commands/str.split": "commands/jsplit", - "commands/trypipeerr": "commands/trypipeerr", - "commands/tryerr": "commands/tryerr", - "commands/a": "commands/a", - "commands/mkarray": "commands/a", - "commands/switch": "commands/switch", - "commands/autocomplete": "commands/autocomplete", - "commands/tabulate": "commands/tabulate", - "commands/true": "commands/true", - "commands/try": "commands/try", - "commands/die": "commands/die", - "commands/let": "commands/let", - "commands/murex-parser": "commands/murex-parser", - "mkarray/date": "mkarray/date", - "mkarray/character": "mkarray/character", - "mkarray/decimal": "mkarray/decimal", - "mkarray/non-decimal": "mkarray/non-decimal", - "mkarray/special": "mkarray/special", - "optional/select": "optional/select", - "optional/table.select": "optional/select", - "optional/!bz2": "optional/bz2", - "optional/base64": "optional/base64", - "optional/!base64": "optional/base64", - "optional/gz": "optional/gz", - "optional/!gz": "optional/gz", - "optional/qr": "optional/qr", - "optional/sleep": "optional/sleep", - "parser/expr-inlined": "parser/expr-inlined", - "parser/c-style-fun": "parser/c-style-fun", - "parser/@[": "parser/range", - "parser/[": "parser/item-index", - "parser/![": "parser/item-index", - "parser/item-index": "parser/item-index", - "parser/index": "parser/item-index", - "parser/[[": "parser/element", - "parser/element": "parser/element", - "parser/(murex named pipe)": "parser/namedpipe", - "parser/<>": "parser/namedpipe", - "parser/read-named-pipe": "parser/namedpipe", - "parser/": "parser/stdin", - "parser/>": "parser/file-truncate", - "parser/|>": "parser/file-truncate", - "parser/fwrite": "parser/file-truncate", - "parser/double-quote": "parser/double-quote", - "parser/scalar": "parser/scalar", - "parser/brace-quote": "parser/brace-quote", - "parser/create-array": "parser/create-array", - "parser/create-object": "parser/create-object", - "parser/logical-and": "parser/logical-and", - "parser/single-quote": "parser/single-quote", - "parser/(": "parser/brace-quote-func", - "parser/multiply-by": "parser/multiply-by", - "parser/multiplication": "parser/multiplication", - "parser/add-with": "parser/add-with", - "parser/addition": "parser/addition", - "parser/subtract-by": "parser/subtract-by", - "parser/pipe-arrow": "parser/pipe-arrow", - "parser/subtraction": "parser/subtraction", - "parser/divide-by": "parser/divide-by", - "parser/division": "parser/division", - "parser/assign-or-merge": "parser/assign-or-merge", - "parser/pipe-generic": "parser/pipe-generic", - "parser/=": "parser/equ", - "parser/>>": "parser/file-append", - "parser/fappend": "parser/file-append", - "parser/elvis": "parser/elvis", - "parser/null-coalescing": "parser/null-coalescing", - "parser/pipe-err": "parser/pipe-err", - "parser/array": "parser/array", - "parser/lambda": "parser/lambda", - "parser/curly-brace": "parser/curly-brace", - "parser/pipe-posix": "parser/pipe-posix", - "parser/logical-or": "parser/logical-or", - "parser/tilde": "parser/tilde", - "events/oncommandcompletion": "events/oncommandcompletion", - "events/onCommandCompletion": "events/oncommandcompletion", - "events/onfilesystemchange": "events/onfilesystemchange", - "events/onFileSystemChange": "events/onfilesystemchange", - "events/onkeypress": "events/onkeypress", - "events/onKeyPress": "events/onkeypress", - "events/onpreview": "events/onpreview", - "events/onPreview": "events/onpreview", - "events/onprompt": "events/onprompt", - "events/onPrompt": "events/onprompt", - "events/onsecondselapsed": "events/onsecondselapsed", - "events/onSecondsElapsed": "events/onsecondselapsed", - "events/onsignalreceived": "events/onsignalreceived", - "events/onSignalReceived": "events/onsignalreceived", - "types/generic": "types/generic", - "types/*": "types/generic", - "types/bool": "types/bool", - "types/commonlog": "types/commonlog", - "types/csv": "types/csv", - "types/float": "types/float", - "types/hcl": "types/hcl", - "types/int": "types/int", - "types/json": "types/json", - "types/jsonc": "types/jsonc", - "types/jsonl": "types/jsonl", - "types/num": "types/num", - "types/path": "types/path", - "types/paths": "types/paths", - "types/str": "types/str", - "types/string": "types/str", - "types/toml": "types/toml", - "types/yaml": "types/yaml", - "types/mxjson": "types/mxjson", - "variables/numeric": "variables/numeric", - "variables/meta-values": "variables/meta-values", - "variables/$.": "variables/meta-values", - "variables/argv": "variables/argv", - "variables/ARGV": "variables/argv", - "variables/ARGS": "variables/argv", - "variables/columns": "variables/columns", - "variables/COLUMNS": "variables/columns", - "variables/event_return": "variables/event_return", - "variables/EVENT_RETURN": "variables/event_return", - "variables/home": "variables/home", - "variables/HOME": "variables/home", - "variables/hostname": "variables/hostname", - "variables/HOSTNAME": "variables/hostname", - "variables/lines": "variables/lines", - "variables/LINES": "variables/lines", - "variables/logname": "variables/logname", - "variables/LOGNAME": "variables/logname", - "variables/murex_argv": "variables/murex_argv", - "variables/MUREX_ARGV": "variables/murex_argv", - "variables/MUREX_ARGS": "variables/murex_argv", - "variables/murex_exe": "variables/murex_exe", - "variables/MUREX_EXE": "variables/murex_exe", - "variables/oldpwd": "variables/oldpwd", - "variables/OLDPWD": "variables/oldpwd", - "variables/params": "variables/params", - "variables/PARAMS": "variables/params", - "variables/pwdhist": "variables/pwdhist", - "variables/PWDHIST": "variables/pwdhist", - "variables/pwd": "variables/pwd", - "variables/PWD": "variables/pwd", - "variables/random": "variables/random", - "variables/RANDOM": "variables/random", - "variables/self": "variables/self", - "variables/SELF": "variables/self", - "variables/shell": "variables/shell", - "variables/SHELL": "variables/shell", - "variables/tmpdir": "variables/tmpdir", - "variables/TMPDIR": "variables/tmpdir", - "variables/user": "variables/user", - "variables/USER": "variables/user", - "apis/Marshal": "apis/Marshal", - "apis/ReadArray": "apis/ReadArray", - "apis/ReadArrayWithType": "apis/ReadArrayWithType", - "apis/ReadIndex": "apis/ReadIndex", - "apis/ReadMap": "apis/ReadMap", - "apis/ReadNotIndex": "apis/ReadNotIndex", - "apis/Unmarshal": "apis/Unmarshal", - "apis/WriteArray": "apis/WriteArray", - "apis/lang.ArrayTemplate": "apis/lang.ArrayTemplate", + + "commands/key-code": "commands/key-code", + "commands/addheading": "commands/addheading", + "commands/prefix": "commands/prefix", + "commands/list.prefix": "commands/prefix", + "commands/suffix": "commands/suffix", + "commands/list.suffix": "commands/suffix", + "commands/alias": "commands/alias", + "commands/!alias": "commands/alias", + "commands/alter": "commands/alter", + "commands/~>": "commands/alter", + "commands/append": "commands/append", + "commands/list.append": "commands/append", + "commands/bg": "commands/bg", + "commands/cpuarch": "commands/cpuarch", + "commands/sys.cpu.arch": "commands/cpuarch", + "commands/cpucount": "commands/cpucount", + "commands/sys.cpu.count": "commands/cpucount", + "commands/catch": "commands/catch", + "commands/!catch": "commands/catch", + "commands/cd": "commands/cd", + "commands/list.case": "commands/list.case", + "commands/bexists": "commands/bexists", + "commands/history": "commands/history", + "commands/count": "commands/count", + "commands/len": "commands/count", + "commands/2darray": "commands/2darray", + "commands/ja": "commands/ja", + "commands/map": "commands/map", + "commands/pipe": "commands/pipe", + "commands/!pipe": "commands/pipe", + "commands/ta": "commands/ta", + "commands/tmp": "commands/tmp", + "commands/datetime": "commands/datetime", + "commands/str.datetime": "commands/datetime", + "commands/debug": "commands/debug", + "commands/export": "commands/export", + "commands/!export": "commands/export", + "commands/unset": "commands/export", + "commands/var.env": "commands/export", + "commands/!var.env": "commands/export", + "commands/args": "commands/args", + "commands/global": "commands/global", + "commands/!global": "commands/global", + "commands/openagent": "commands/openagent", + "commands/!openagent": "commands/openagent", + "commands/method": "commands/method", + "commands/cast": "commands/cast", + "commands/set": "commands/set", + "commands/!set": "commands/set", + "commands/unsafe": "commands/unsafe", + "commands/type": "commands/type", + "commands/fid-list": "commands/fid-list", + "commands/jobs": "commands/fid-list", + "commands/getfile": "commands/getfile", + "commands/err": "commands/err", + "commands/esccli": "commands/esccli", + "commands/eschtml": "commands/eschtml", + "commands/!eschtml": "commands/eschtml", + "commands/escurl": "commands/escurl", + "commands/!escurl": "commands/escurl", + "commands/exec": "commands/exec", + "commands/command": "commands/exec", + "commands/exec.file": "commands/exec", + "commands/fexec": "commands/fexec", + "commands/builtin": "commands/fexec", + "commands/exec.builtin": "commands/fexec", + "commands/exec.function": "commands/fexec", + "commands/exec.private": "commands/fexec", + "commands/break": "commands/break", + "commands/return": "commands/return", + "commands/exit": "commands/exit", + "commands/expr": "commands/expr", + "commands/false": "commands/false", + "commands/foreach": "commands/foreach", + "commands/formap": "commands/formap", + "commands/for": "commands/for", + "commands/fg": "commands/fg", + "commands/runmode": "commands/runmode", + "commands/rand": "commands/rand", + "commands/get-type": "commands/get-type", + "commands/exitnum": "commands/exitnum", + "commands/pt": "commands/pt", + "commands/get": "commands/get", + "commands/g": "commands/g", + "commands/!g": "commands/g", + "commands/if": "commands/if", + "commands/!if": "commands/if", + "commands/source": "commands/source", + "commands/.": "commands/source", + "commands/is-null": "commands/is-null", + "commands/mjoin": "commands/mjoin", + "commands/list.join": "commands/mjoin", + "commands/fid-killall": "commands/fid-killall", + "commands/fid-kill": "commands/fid-kill", + "commands/left": "commands/left", + "commands/list.left": "commands/left", + "commands/f": "commands/f", + "commands/which": "commands/which", + "commands/lockfile": "commands/lockfile", + "commands/and": "commands/and", + "commands/!and": "commands/and", + "commands/or": "commands/or", + "commands/!or": "commands/or", + "commands/while": "commands/while", + "commands/!while": "commands/while", + "commands/man-summary": "commands/man-summary", + "commands/help.man.summary": "commands/man-summary", + "commands/match": "commands/match", + "commands/!match": "commands/match", + "commands/list.str": "commands/match", + "commands/!list.str": "commands/match", + "commands/event": "commands/event", + "commands/!event": "commands/event", + "commands/murex-package": "commands/murex-package", + "commands/version": "commands/version", + "commands/murex-docs": "commands/murex-docs", + "commands/help": "commands/murex-docs", + "commands/continue": "commands/continue", + "commands/!": "commands/not-func", + "commands/not": "commands/not-func", + "commands/null": "commands/devnull", + "commands/open": "commands/open", + "commands/os": "commands/os", + "commands/sys.os": "commands/os", + "commands/out": "commands/out", + "commands/echo": "commands/out", + "commands/tout": "commands/tout", + "commands/man-get-flags": "commands/man-get-flags", + "commands/trypipe": "commands/trypipe", + "commands/post": "commands/post", + "commands/prepend": "commands/prepend", + "commands/list.prepend": "commands/prepend", + "commands/pretty": "commands/pretty", + "commands/struct-keys": "commands/struct-keys", + "commands/private": "commands/private", + "commands/time": "commands/time", + "commands/function": "commands/function", + "commands/!function": "commands/function", + "commands/escape": "commands/escape", + "commands/!escape": "commands/escape", + "commands/murex-update-exe-list": "commands/murex-update-exe-list", + "commands/read": "commands/read", + "commands/tread": "commands/tread", + "commands/format": "commands/format", + "commands/rx": "commands/rx", + "commands/!rx": "commands/rx", + "commands/regexp": "commands/regexp", + "commands/!regexp": "commands/regexp", + "commands/list.regex": "commands/regexp", + "commands/!list.regex": "commands/regexp", + "commands/open-image": "commands/open-image", + "commands/mtac": "commands/mtac", + "commands/list.reverse": "commands/mtac", + "commands/right": "commands/right", + "commands/list.right": "commands/right", + "commands/round": "commands/round", + "commands/num.round": "commands/round", + "commands/signal": "commands/signal", + "commands/summary": "commands/summary", + "commands/!summary": "commands/summary", + "commands/config": "commands/config", + "commands/!config": "commands/config", + "commands/runtime": "commands/runtime", + "commands/builtins": "commands/runtime", + "commands/shell.runtime": "commands/runtime", + "commands/test": "commands/test", + "commands/!test": "commands/test", + "commands/msort": "commands/msort", + "commands/list.sort": "commands/msort", + "commands/jsplit": "commands/jsplit", + "commands/str.split": "commands/jsplit", + "commands/trypipeerr": "commands/trypipeerr", + "commands/tryerr": "commands/tryerr", + "commands/a": "commands/a", + "commands/mkarray": "commands/a", + "commands/switch": "commands/switch", + "commands/autocomplete": "commands/autocomplete", + "commands/tabulate": "commands/tabulate", + "commands/true": "commands/true", + "commands/try": "commands/try", + "commands/die": "commands/die", + "commands/let": "commands/let", + "commands/murex-parser": "commands/murex-parser", + "mkarray/date": "mkarray/date", + "mkarray/character": "mkarray/character", + "mkarray/decimal": "mkarray/decimal", + "mkarray/non-decimal": "mkarray/non-decimal", + "mkarray/special": "mkarray/special", + "optional/select": "optional/select", + "optional/table.select": "optional/select", + "optional/!bz2": "optional/bz2", + "optional/base64": "optional/base64", + "optional/!base64": "optional/base64", + "optional/gz": "optional/gz", + "optional/!gz": "optional/gz", + "optional/qr": "optional/qr", + "optional/sleep": "optional/sleep", + "parser/expr-inlined": "parser/expr-inlined", + "parser/c-style-fun": "parser/c-style-fun", + "parser/@[": "parser/range", + "parser/[": "parser/item-index", + "parser/![": "parser/item-index", + "parser/item-index": "parser/item-index", + "parser/index": "parser/item-index", + "parser/[[": "parser/element", + "parser/element": "parser/element", + "parser/(murex named pipe)": "parser/namedpipe", + "parser/<>": "parser/namedpipe", + "parser/read-named-pipe": "parser/namedpipe", + "parser/": "parser/stdin", + "parser/>": "parser/file-truncate", + "parser/|>": "parser/file-truncate", + "parser/fwrite": "parser/file-truncate", + "parser/double-quote": "parser/double-quote", + "parser/scalar": "parser/scalar", + "parser/brace-quote": "parser/brace-quote", + "parser/create-array": "parser/create-array", + "parser/create-object": "parser/create-object", + "parser/logical-and": "parser/logical-and", + "parser/single-quote": "parser/single-quote", + "parser/(": "parser/brace-quote-func", + "parser/multiply-by": "parser/multiply-by", + "parser/multiplication": "parser/multiplication", + "parser/add-with": "parser/add-with", + "parser/addition": "parser/addition", + "parser/subtract-by": "parser/subtract-by", + "parser/pipe-arrow": "parser/pipe-arrow", + "parser/subtraction": "parser/subtraction", + "parser/divide-by": "parser/divide-by", + "parser/division": "parser/division", + "parser/assign-or-merge": "parser/assign-or-merge", + "parser/pipe-generic": "parser/pipe-generic", + "parser/=": "parser/equ", + "parser/>>": "parser/file-append", + "parser/fappend": "parser/file-append", + "parser/elvis": "parser/elvis", + "parser/null-coalescing": "parser/null-coalescing", + "parser/pipe-err": "parser/pipe-err", + "parser/array": "parser/array", + "parser/lambda": "parser/lambda", + "parser/curly-brace": "parser/curly-brace", + "parser/pipe-posix": "parser/pipe-posix", + "parser/logical-or": "parser/logical-or", + "parser/tilde": "parser/tilde", + "events/oncommandcompletion": "events/oncommandcompletion", + "events/onCommandCompletion": "events/oncommandcompletion", + "events/onfilesystemchange": "events/onfilesystemchange", + "events/onFileSystemChange": "events/onfilesystemchange", + "events/onkeypress": "events/onkeypress", + "events/onKeyPress": "events/onkeypress", + "events/onpreview": "events/onpreview", + "events/onPreview": "events/onpreview", + "events/onprompt": "events/onprompt", + "events/onPrompt": "events/onprompt", + "events/onsecondselapsed": "events/onsecondselapsed", + "events/onSecondsElapsed": "events/onsecondselapsed", + "events/onsignalreceived": "events/onsignalreceived", + "events/onSignalReceived": "events/onsignalreceived", + "types/generic": "types/generic", + "types/*": "types/generic", + "types/bool": "types/bool", + "types/commonlog": "types/commonlog", + "types/csv": "types/csv", + "types/float": "types/float", + "types/hcl": "types/hcl", + "types/int": "types/int", + "types/json": "types/json", + "types/jsonc": "types/jsonc", + "types/jsonl": "types/jsonl", + "types/num": "types/num", + "types/path": "types/path", + "types/paths": "types/paths", + "types/str": "types/str", + "types/string": "types/str", + "types/toml": "types/toml", + "types/yaml": "types/yaml", + "types/mxjson": "types/mxjson", + "variables/numeric": "variables/numeric", + "variables/meta-values": "variables/meta-values", + "variables/$.": "variables/meta-values", + "variables/argv": "variables/argv", + "variables/ARGV": "variables/argv", + "variables/ARGS": "variables/argv", + "variables/columns": "variables/columns", + "variables/COLUMNS": "variables/columns", + "variables/event_return": "variables/event_return", + "variables/EVENT_RETURN": "variables/event_return", + "variables/home": "variables/home", + "variables/HOME": "variables/home", + "variables/hostname": "variables/hostname", + "variables/HOSTNAME": "variables/hostname", + "variables/lines": "variables/lines", + "variables/LINES": "variables/lines", + "variables/logname": "variables/logname", + "variables/LOGNAME": "variables/logname", + "variables/murex_argv": "variables/murex_argv", + "variables/MUREX_ARGV": "variables/murex_argv", + "variables/MUREX_ARGS": "variables/murex_argv", + "variables/murex_exe": "variables/murex_exe", + "variables/MUREX_EXE": "variables/murex_exe", + "variables/oldpwd": "variables/oldpwd", + "variables/OLDPWD": "variables/oldpwd", + "variables/params": "variables/params", + "variables/PARAMS": "variables/params", + "variables/pwdhist": "variables/pwdhist", + "variables/PWDHIST": "variables/pwdhist", + "variables/pwd": "variables/pwd", + "variables/PWD": "variables/pwd", + "variables/random": "variables/random", + "variables/RANDOM": "variables/random", + "variables/self": "variables/self", + "variables/SELF": "variables/self", + "variables/shell": "variables/shell", + "variables/SHELL": "variables/shell", + "variables/tmpdir": "variables/tmpdir", + "variables/TMPDIR": "variables/tmpdir", + "variables/user": "variables/user", + "variables/USER": "variables/user", + "apis/Marshal": "apis/Marshal", + "apis/ReadArray": "apis/ReadArray", + "apis/ReadArrayWithType": "apis/ReadArrayWithType", + "apis/ReadIndex": "apis/ReadIndex", + "apis/ReadMap": "apis/ReadMap", + "apis/ReadNotIndex": "apis/ReadNotIndex", + "apis/Unmarshal": "apis/Unmarshal", + "apis/WriteArray": "apis/WriteArray", + "apis/lang.ArrayTemplate": "apis/lang.ArrayTemplate", "apis/lang.ArrayWithTypeTemplate": "apis/lang.ArrayWithTypeTemplate", - "apis/lang.IndexTemplateObject": "apis/lang.IndexTemplateObject", - "apis/lang.IndexTemplateTable": "apis/lang.IndexTemplateTable", - "apis/lang.MarshalData": "apis/lang.MarshalData", - "apis/lang.UnmarshalData": "apis/lang.UnmarshalData", - "user-guide/ansi": "user-guide/ansi", - "user-guide/bang-prefix": "user-guide/bang-prefix", - "user-guide/bang": "user-guide/bang-prefix", - "user-guide/code-block": "user-guide/code-block", - "user-guide/fileref": "user-guide/fileref", - "user-guide/FileRef": "user-guide/fileref", - "user-guide/hint-text": "user-guide/hint-text", - "user-guide/integrations": "user-guide/integrations", - "user-guide/interactive-shell": "user-guide/interactive-shell", - "user-guide/repl": "user-guide/interactive-shell", - "user-guide/readline": "user-guide/interactive-shell", - "user-guide/job-control": "user-guide/job-control", - "user-guide/module": "user-guide/modules", - "user-guide/modules": "user-guide/modules", - "user-guide/package": "user-guide/modules", - "user-guide/packages": "user-guide/modules", - "user-guide/namedpipes": "user-guide/namedpipes", + "apis/lang.IndexTemplateObject": "apis/lang.IndexTemplateObject", + "apis/lang.IndexTemplateTable": "apis/lang.IndexTemplateTable", + "apis/lang.MarshalData": "apis/lang.MarshalData", + "apis/lang.UnmarshalData": "apis/lang.UnmarshalData", + "user-guide/ansi": "user-guide/ansi", + "user-guide/bang-prefix": "user-guide/bang-prefix", + "user-guide/bang": "user-guide/bang-prefix", + "user-guide/code-block": "user-guide/code-block", + "user-guide/fileref": "user-guide/fileref", + "user-guide/FileRef": "user-guide/fileref", + "user-guide/hint-text": "user-guide/hint-text", + "user-guide/integrations": "user-guide/integrations", + "user-guide/interactive-shell": "user-guide/interactive-shell", + "user-guide/repl": "user-guide/interactive-shell", + "user-guide/readline": "user-guide/interactive-shell", + "user-guide/job-control": "user-guide/job-control", + "user-guide/module": "user-guide/modules", + "user-guide/modules": "user-guide/modules", + "user-guide/package": "user-guide/modules", + "user-guide/packages": "user-guide/modules", + "user-guide/namedpipes": "user-guide/namedpipes", "user-guide/operators-and-tokens": "user-guide/operators-and-tokens", - "user-guide/pipeline": "user-guide/pipeline", - "user-guide/profile": "user-guide/profile", - "user-guide/.murex_profile": "user-guide/profile", - "user-guide/murex_profile": "user-guide/profile", - "user-guide/.murex_preload": "user-guide/profile", - "user-guide/murex_preload": "user-guide/profile", - "user-guide/reserved-vars": "user-guide/reserved-vars", - "user-guide/reserved": "user-guide/reserved-vars", - "user-guide/reserved-variables": "user-guide/reserved-vars", - "user-guide/special-vars": "user-guide/reserved-vars", - "user-guide/special-variables": "user-guide/reserved-vars", - "user-guide/rosetta-stone": "user-guide/rosetta-stone", - "user-guide/schedulers": "user-guide/schedulers", - "user-guide/strict-types": "user-guide/strict-types", - "user-guide/hotkeys": "user-guide/terminal-keys", - "user-guide/scoping": "user-guide/scoping", - "integrations/chatgpt": "integrations/chatgpt", - "integrations/ChatGPT": "integrations/chatgpt", - "integrations/cheatsh": "integrations/cheatsh", - "integrations/cheat.sh": "integrations/cheatsh", - "integrations/kitty": "integrations/kitty", - "integrations/make": "integrations/make", - "integrations/Makefile": "integrations/make", - "integrations/man-pages": "integrations/man-pages", - "integrations/spellcheck": "integrations/spellcheck", - "integrations/terminology": "integrations/terminology", - "integrations/direnv": "integrations/direnv", - "integrations/yarn": "integrations/yarn", - "integrations/package.json": "integrations/yarn", - "integrations/iterm2": "integrations/iterm2", - "changelog/v2.0": "changelog/v2.0", - "changelog/v2.1": "changelog/v2.1", - "changelog/v2.10": "changelog/v2.10", - "changelog/v2.11": "changelog/v2.11", - "changelog/v2.2": "changelog/v2.2", - "changelog/v2.3": "changelog/v2.3", - "changelog/v2.4": "changelog/v2.4", - "changelog/v2.5": "changelog/v2.5", - "changelog/v2.6": "changelog/v2.6", - "changelog/v2.7": "changelog/v2.7", - "changelog/v2.8": "changelog/v2.8", - "changelog/v2.9": "changelog/v2.9", - "changelog/v3.0": "changelog/v3.0", - "changelog/v3.1": "changelog/v3.1", - "changelog/v4.0": "changelog/v4.0", - "changelog/v4.1": "changelog/v4.1", - "changelog/v4.2": "changelog/v4.2", - "changelog/v4.3": "changelog/v4.3", - "changelog/v4.4": "changelog/v4.4", - "changelog/v5.0": "changelog/v5.0", - "changelog/v5.1": "changelog/v5.1", - "changelog/v5.2": "changelog/v5.2", - "changelog/v5.3": "changelog/v5.3", - "changelog/v6.0": "changelog/v6.0", - "changelog/v6.1": "changelog/v6.1", - "changelog/v6.2": "changelog/v6.2", - "changelog/v6.3": "changelog/v6.3", + "user-guide/pipeline": "user-guide/pipeline", + "user-guide/profile": "user-guide/profile", + "user-guide/.murex_profile": "user-guide/profile", + "user-guide/murex_profile": "user-guide/profile", + "user-guide/.murex_preload": "user-guide/profile", + "user-guide/murex_preload": "user-guide/profile", + "user-guide/reserved-vars": "user-guide/reserved-vars", + "user-guide/reserved": "user-guide/reserved-vars", + "user-guide/reserved-variables": "user-guide/reserved-vars", + "user-guide/special-vars": "user-guide/reserved-vars", + "user-guide/special-variables": "user-guide/reserved-vars", + "user-guide/rosetta-stone": "user-guide/rosetta-stone", + "user-guide/schedulers": "user-guide/schedulers", + "user-guide/strict-types": "user-guide/strict-types", + "user-guide/hotkeys": "user-guide/terminal-keys", + "user-guide/scoping": "user-guide/scoping", + "user-guide/murex-arrays": "user-guide/murex-arrays", + "integrations/chatgpt": "integrations/chatgpt", + "integrations/ChatGPT": "integrations/chatgpt", + "integrations/cheatsh": "integrations/cheatsh", + "integrations/cheat.sh": "integrations/cheatsh", + "integrations/kitty": "integrations/kitty", + "integrations/make": "integrations/make", + "integrations/Makefile": "integrations/make", + "integrations/man-pages": "integrations/man-pages", + "integrations/spellcheck": "integrations/spellcheck", + "integrations/terminology": "integrations/terminology", + "integrations/direnv": "integrations/direnv", + "integrations/yarn": "integrations/yarn", + "integrations/package.json": "integrations/yarn", + "integrations/iterm2": "integrations/iterm2", + "changelog/v2.0": "changelog/v2.0", + "changelog/v2.1": "changelog/v2.1", + "changelog/v2.10": "changelog/v2.10", + "changelog/v2.11": "changelog/v2.11", + "changelog/v2.2": "changelog/v2.2", + "changelog/v2.3": "changelog/v2.3", + "changelog/v2.4": "changelog/v2.4", + "changelog/v2.5": "changelog/v2.5", + "changelog/v2.6": "changelog/v2.6", + "changelog/v2.7": "changelog/v2.7", + "changelog/v2.8": "changelog/v2.8", + "changelog/v2.9": "changelog/v2.9", + "changelog/v3.0": "changelog/v3.0", + "changelog/v3.1": "changelog/v3.1", + "changelog/v4.0": "changelog/v4.0", + "changelog/v4.1": "changelog/v4.1", + "changelog/v4.2": "changelog/v4.2", + "changelog/v4.3": "changelog/v4.3", + "changelog/v4.4": "changelog/v4.4", + "changelog/v5.0": "changelog/v5.0", + "changelog/v5.1": "changelog/v5.1", + "changelog/v5.2": "changelog/v5.2", + "changelog/v5.3": "changelog/v5.3", + "changelog/v6.0": "changelog/v6.0", + "changelog/v6.1": "changelog/v6.1", + "changelog/v6.2": "changelog/v6.2", + "changelog/v6.3": "changelog/v6.3", + } -} +} \ No newline at end of file diff --git a/docs/user-guide/murex-arrays.md b/docs/user-guide/murex-arrays.md index ec0906189..9222015f0 100644 --- a/docs/user-guide/murex-arrays.md +++ b/docs/user-guide/murex-arrays.md @@ -2,8 +2,321 @@ > Examples using arrays within Murex -In this example we'll merge two JSON files together, using example code from -Nushell to illustrate Murex's syntax. +### Creating Arrays + +Arrays can be defined with `%[ ... ]`. + +The syntax is a superset of JSON. So that any JSON array can be a Murex array +when prefixed with `%`. For example + +``` +%["Monday", "Tuesday", "Wednesday", "Thursday", "Friday"] +``` + +...however while this is readable, this is not convenient when working in the +interactive command line. We want something that's a little more comfortable +for "write many, read once" type environments like a shell REPL. + +So Murex makes the parser-defined punctuation optional: + +``` +%[Monday Tuesday Wednesday Thursday Friday] +``` + +That's better, however surely the computer already knows what days there are in +a week? I think we can improve this syntax further... + +``` +%[Monday..Friday] +``` + +The `..` describes a range. So we are saying "return everyday from Monday to +Friday, inclusive". + +``` +» %[Monday..Friday] +[ + "Monday", + "Tuesday", + "Wednesday", + "Thursday", + "Friday" +] +``` + +It's not just days of the week that can be completed like this. Most forms of +dates, number bases and other familiar sequences can be. + +You can also have multiple parts of arrays expanded: + +``` +» %[[2024..2026]\\, [spring..winter]] +[ + "2024, spring", + "2024, summer", + "2024, autumn", + "2024, winter", + "2025, spring", + "2025, summer", + "2025, autumn", + "2025, winter", + "2026, spring", + "2026, summer", + "2026, autumn", + "2026, winter" +] +``` + +Did you notice how the seasons are lower case this time? Murex respects the +text case of the ranges. For example: + +``` +» %[MON..WED] +[ + "MON", + "TUE", + "WED", +] + +» %[mon..wed] +[ + "mon", + "tue", + "wed", +] +``` + +Returning to our multi-part array, perhaps you want **fall** instead of +**autumn**? (here you do need to include a comma) + +``` +» %[[2024..2026]\\, [spring,summer,fall,winter]] +[ + "2024, spring", + "2024, summer", + "2024, fall", + "2024, winter", + "2025, spring", + "2025, summer", + "2025, fall", + "2025, winter", + "2026, spring", + "2026, summer", + "2026, fall", + "2026, winter" +] +``` + +### Creating Multi-Dimensional Arrays + +But what if they were supposed to be nested rather than flattened arrays? Well +that's not a problem either. Just make sure your first nested array is also +prefixed with `%`: + +``` +» %[%[2024..2026] [Spring..Winter]] +[ + [ + 2024, + 2025, + 2026 + ], + [ + "Spring", + "Summer", + "Autumn", + "Winter" + ] +] +``` + +### Streaming Arrays + +Ok, that's great, but marshalling a data structure and passing it to functions +requires allocating that entire data structure to memory. Whereas shells best +excel when they're streaming data because it allows processes to perform +concurrent operations across massive data sets. + +If that's a problem for you too, then Murex has you sorted: `a`. + +The `a` builtin returns a line separated list. And thus can be operated on via +your traditional pipes and UNIX core utilities. + +Lets say, for some reason, you wanted to run a job against every IPv4 address +available. Marshalling that into a data structure just to run commands +sequentially would be rather silly. So lets stream that array using `a`: + +``` +a [0..255].[0..255].[0..255].[0..255] | foreach $ip { ping -c 1 -t 1 $ip } +``` + +> If you do happen to run this and wonder how to cancel the underlying `a` and +> `foreach` routines, you can press `ctrl`+`\` -- which is a Murex shortcut to +> kill everything in that shell session. + +This is obviously an absurd example because nobody in their right mind would +want to ping every valid IPv4 address. But it does demonstrate the advantages +of streaming lists rather than creating arrays. + +### Accessing Array Values + +There are two main ways to access values inside an array: + +* square brackets for immutable copies: `$my_array[index]` + +* dot notation, which allows being written to: `$my_array.index` + +Why two? Because they support different features. + +#### Square Brackets (immutable) + +With square brackets you can select more than just a single element. For +example, if you wanted the first element you can reference it the same way +you'd reference arrays in any other language: + +``` +$my_array[0] +``` + +> Murex arrays begin at `0`. + +If you wanted to count from the end of the array, you can use negative values: + +``` +$my_array[-1] +``` + +> Watch out here because negative indexes count from `1` because -0 isn't a +> valid number. + +#### Multiple Elements + +However what if you wanted multiple elements from the array, like 2nd and 4th? +Then just specify multiple elements inside the square brackets: + +``` +$my_array[1 4] +``` + +#### Ranges + +That's handy, but if I actually want a range of elements? Well then you can use +the range `..` operator like before: + +``` +$my_array[1..4] +``` + +And if you don't know the size of your array, you can ignore the index value +entirely. For example: + +**Everything from and including the 2nd element:** + +``` +$my_array[2..] +``` + +**Everything up to and including the 4th element:** + +``` +$my_array[..4] +``` + +> Ranges are indexed from 1. Yes, I know that's stupid and confusing. + +#### As A Function + +The square brackets can also be used as a function too. Which means any kind of +array or list can be queried from stdin, you don't have to first convert it to +a variable. + +``` +» %[mon..fri] | [1 4] +[ + "tue", + "fri" +] +``` + +This is especially helpful if say something writes JSON, YAML, or any other +structured document, and you only want specific values. For example you want +the last container in your cloud infrastructure, and you know your cloud CLI +tool (`cloud-api` for our made up purposes here) returns JSON: + +``` +cloud-api list-containers | :json: [-1] +``` + +#### Making Changes + +That's all great, but what if I want to make a change to the host array? + +Well this is where dot notation comes in... + +### Dot Notation (mutable) + +Dot notation is a lot more limited in what you can do because it's designed for +making careful edits of the underlying data structure. So it can only be used +with variables. + +#### Assignment + +You can edit an element, for example renaming **Wednesday** to **Humpday**: + +``` +» $days = %[Monday..Friday] + +» $days.2 = "Humpday" + +» $days +[ + "Monday", + "Tuesday", + "Humpday", + "Thursday", + "Friday" +] +``` + +> Remember: arrays are zero based + +#### Printing + +You can also use dot notation to return a value, just like you would with the +square braces solution above. But dot notation doesn't support any special +magic and still only works on variables: + +``` +$my_array.2 +``` + +### Appending Arrays + +There are several ways to append an array. You can create a copy of that array +and append via the pipeline: + +``` +$my_array | append foo bar +``` + +> You can also prepend using `prepend`) + +...or... + +``` +$my_array ~> %[foo bar] +``` + +However if you want to update a variable in place, you can use the merge +operator, `<~`: + +``` +$my_array <~ %[foo bar] +``` + +## Fin + +Now you're an expert in arrays. Hooray `\o/` ## See Also diff --git a/gen/user-guide/arrays_doc.yaml b/gen/user-guide/arrays_doc.yaml index 7cfaf16c1..4ee8e5338 100644 --- a/gen/user-guide/arrays_doc.yaml +++ b/gen/user-guide/arrays_doc.yaml @@ -7,10 +7,8 @@ Summary: >- Examples using arrays within Murex Description: |- - In this example we'll merge two JSON files together, using example code from - Nushell to illustrate Murex's syntax. - Detail: |- - {{ include "gen/examples/arrays.inc.md" }} + {{ include "gen/user-guide/arrays.inc.md" }} + Detail: Related: - expr - addition diff --git a/gen/vuepress/userguide_generated.json b/gen/vuepress/userguide_generated.json new file mode 100644 index 000000000..3d7c605c1 --- /dev/null +++ b/gen/vuepress/userguide_generated.json @@ -0,0 +1,134 @@ +[ + { + "icon": "file-lines", + "index": true, + "link": "/user-guide/", + "text": "User Guide" + }, + { + "children": [ + { + "icon": "file-lines", + "link": "user-guide/interactive-shell.html", + "text": "Interactive Shell" + }, + { + "icon": "file-lines", + "link": "user-guide/modules.html", + "text": "Modules And Packages" + }, + { + "icon": "file-lines", + "link": "user-guide/terminal-keys.html", + "text": "Terminal Hotkeys" + }, + { + "icon": "file-lines", + "link": "user-guide/murex-arrays.html", + "text": "Working With Arrays" + } + ], + "collapsible": true, + "icon": "folder-closed", + "text": "Beginners Guides" + }, + { + "children": [ + { + "icon": "file-lines", + "link": "user-guide/ansi.html", + "text": "ANSI Constants" + }, + { + "icon": "file-lines", + "link": "user-guide/operators-and-tokens.html", + "text": "Operators And Tokens" + }, + { + "icon": "file-lines", + "link": "user-guide/rosetta-stone.html", + "text": "Rosetta Stone" + }, + { + "icon": "file-lines", + "link": "user-guide/terminal-keys.html", + "text": "Terminal Hotkeys" + } + ], + "collapsible": true, + "icon": "folder-closed", + "text": "Cheat Sheets" + }, + { + "children": [ + { + "icon": "file-lines", + "link": "user-guide/bang-prefix.html", + "text": "Bang Prefix" + }, + { + "icon": "file-lines", + "link": "user-guide/fileref.html", + "text": "FileRef" + }, + { + "icon": "file-lines", + "link": "user-guide/hint-text.html", + "text": "Hint Text" + }, + { + "icon": "file-lines", + "link": "user-guide/integrations.html", + "text": "Integrations" + }, + { + "icon": "file-lines", + "link": "user-guide/job-control.html", + "text": "Job Control" + }, + { + "icon": "file-lines", + "link": "user-guide/namedpipes.html", + "text": "Named Pipes" + }, + { + "icon": "file-lines", + "link": "user-guide/profile.html", + "text": "Profile Files" + }, + { + "icon": "file-lines", + "link": "user-guide/reserved-vars.html", + "text": "Reserved Variables" + }, + { + "icon": "file-lines", + "link": "user-guide/schedulers.html", + "text": "Schedulers" + }, + { + "icon": "file-lines", + "link": "user-guide/strict-types.html", + "text": "Strict Types In Expressions" + }, + { + "icon": "file-lines", + "link": "user-guide/scoping.html", + "text": "Variable And Config Scoping" + } + ], + "collapsible": true, + "icon": "folder-closed", + "text": "Read More" + }, + { + "icon": "file-lines", + "link": "user-guide/code-block.html", + "text": "Code Block Parsing" + }, + { + "icon": "file-lines", + "link": "user-guide/pipeline.html", + "text": "Pipeline" + } +] \ No newline at end of file diff --git a/version.svg b/version.svg index a473d8759..4e2fba4f5 100644 --- a/version.svg +++ b/version.svg @@ -1 +1 @@ -Version: 6.3.4231Version6.3.4231 +Version: 6.3.4234Version6.3.4234 From e6c0c084cd5465e87c1e2f3967d30cf677fbd7ee Mon Sep 17 00:00:00 2001 From: Laurence Morgan Date: Thu, 19 Sep 2024 23:09:26 +0100 Subject: [PATCH 03/28] website: arrays doc + sidebar --- app/app.go | 4 +- builtins/docs/summaries.go | 2112 +++++++++++++++---------------- docs/user-guide/murex-arrays.md | 20 +- gen/root/README.inc.md | 1 + gen/user-guide/arrays.inc.md | 20 +- version.svg | 2 +- 6 files changed, 1075 insertions(+), 1084 deletions(-) diff --git a/app/app.go b/app/app.go index 4e7a096f9..086fdb18c 100644 --- a/app/app.go +++ b/app/app.go @@ -16,9 +16,9 @@ const Name = "murex" const ( Major = 6 Minor = 3 - Revision = 4234 + Revision = 4235 Branch = "develop" - BuildDate = "2024-09-19 22:54:17" + BuildDate = "2024-09-19 23:09:30" ) // Copyright is the copyright owner string diff --git a/builtins/docs/summaries.go b/builtins/docs/summaries.go index ef73c936c..3e8f5e752 100644 --- a/builtins/docs/summaries.go +++ b/builtins/docs/summaries.go @@ -5,1084 +5,1074 @@ package docs func init() { Summary = map[string]string{ - - - "key-code": "Returns character sequences for any key pressed (ie sent from the terminal)", - "addheading": "Adds headings to a table", - "prefix": "Prefix a string to every item in a list", - "suffix": "Prefix a string to every item in a list", - "alias": "Create an alias for a command", - "alter": "Change a value within a structured data-type and pass that change along the pipeline without altering the original source input", - "append": "Add data to the end of an array", - "bg": "Run processes in the background", - "cpuarch": "Output the hosts CPU architecture", - "cpucount": "Output the number of CPU cores available on your host", - "catch": "Handles the exception code raised by `try` or `trypipe`", - "cd": "Change (working) directory", - "list.case": "Changes the character case of a string or all elements in an array", - "bexists": "Check which builtins exist", - "history": "Outputs murex's command history", - "count": "Count items in a map, list or array", - "2darray": "Create a 2D JSON array from multiple input sources", - "ja": "A sophisticated yet simply way to build a JSON array", - "map": "Creates a map from two data sources", - "pipe": "Manage Murex named pipes", - "ta": "A sophisticated yet simple way to build an array of a user defined data-type", - "tmp": "Create a temporary file and write to it", - "datetime": "A date and/or time conversion tool (like `printf` but for date and time values)", - "debug": "Debugging information", - "export": "Define an environmental variable and set it's value", - "args": "Command line flag parser for Murex shell scripting", - "global": "Define a global variable and set it's value", - "openagent": "Creates a handler function for `open`", - "method": "Define a methods supported data-types", - "cast": "Alters the data-type of the previous function without altering its output", - "set": "Define a variable (typically local) and set it's value", - "unsafe": "Execute a block of code, always returning a zero exit number", - "type": "Command type (function, builtin, alias, etc)", - "fid-list": "Lists all running functions within the current Murex session", - "getfile": "Makes a standard HTTP request and return the contents as Murex-aware data type for passing along Murex pipelines.", - "err": "Print a line to the stderr", - "esccli": "Escapes an array so output is valid shell code", - "eschtml": "Encode or decodes text for HTML", - "escurl": "Encode or decodes text for the URL", - "exec": "Runs an executable", - "fexec": "Execute a command or function, bypassing the usual order of precedence.", - "break": "Terminate execution of a block within your processes scope", - "return": "Exits current function scope", - "exit": "Exit murex", - "expr": "Expressions: mathematical, string comparisons, logical operators", - "false": "Returns a `false` value", - "foreach": "Iterate through an array", - "formap": "Iterate through a map or other collection of data", - "for": "A more familiar iteration loop to existing developers", - "fg": "Sends a background process into the foreground", - "runmode": "Alter the scheduler's behaviour at higher scoping level", - "rand": "Random field generator", - "get-type": "Returns the data-type of a variable or pipe", - "exitnum": "Output the exit number of the previous process", - "pt": "Pipe telemetry. Writes data-types and bytes written", - "get": "Makes a standard HTTP request and returns the result as a JSON object", - "g": "Glob pattern matching for file system objects (eg `*.txt`)", - "if": "Conditional statement to execute different blocks of code depending on the result of the condition", - "source": "Import Murex code from another file or code block", - "is-null": "Checks if a variable is null or undefined", - "mjoin": "Joins a list or array into a single string", - "fid-killall": "Terminate all running Murex functions in current session", - "fid-kill": "Terminate a running Murex function", - "left": "Left substring every item in a list", - "f": "Lists or filters file system objects (eg files)", - "which": "Locate command origin", - "lockfile": "Create and manage lock files", - "and": "Returns `true` or `false` depending on whether multiple conditions are met", - "or": "Returns `true` or `false` depending on whether one code-block out of multiple ones supplied is successful or unsuccessful.", - "while": "Loop until condition false", - "man-summary": "Outputs a man page summary of a command", - "match": "Match an exact value in an array", - "event": "Event driven programming for shell scripts", - "murex-package": "Murex's package manager", - "version": "Get Murex version", - "murex-docs": "Displays the man pages for Murex builtins", - "continue": "Terminate process of a block within a caller function", - "not-func": "Reads the stdin and exit number from previous process and not's it's condition", - "devnull": "null function. Similar to /dev/null", - "open": "Open a file with a preferred handler", - "os": "Output the auto-detected OS name", - "out": "Print a string to the stdout with a trailing new line character", - "tout": "Print a string to the stdout and set it's data-type", - "man-get-flags": "Parses man page files for command line flags", - "trypipe": "Checks for non-zero exits of each function in a pipeline", - "post": "HTTP POST request with a JSON-parsable return", - "prepend": "Add data to the start of an array", - "pretty": "Prettifies JSON to make it human readable", - "struct-keys": "Outputs all the keys in a structure as a file path", - "private": "Define a private function block", - "time": "Returns the execution run time of a command or block", - "function": "Define a function block", - "escape": "Escape or unescape input", + + "key-code": "Returns character sequences for any key pressed (ie sent from the terminal)", + "addheading": "Adds headings to a table", + "prefix": "Prefix a string to every item in a list", + "suffix": "Prefix a string to every item in a list", + "alias": "Create an alias for a command", + "alter": "Change a value within a structured data-type and pass that change along the pipeline without altering the original source input", + "append": "Add data to the end of an array", + "bg": "Run processes in the background", + "cpuarch": "Output the hosts CPU architecture", + "cpucount": "Output the number of CPU cores available on your host", + "catch": "Handles the exception code raised by `try` or `trypipe`", + "cd": "Change (working) directory", + "list.case": "Changes the character case of a string or all elements in an array", + "bexists": "Check which builtins exist", + "history": "Outputs murex's command history", + "count": "Count items in a map, list or array", + "2darray": "Create a 2D JSON array from multiple input sources", + "ja": "A sophisticated yet simply way to build a JSON array", + "map": "Creates a map from two data sources", + "pipe": "Manage Murex named pipes", + "ta": "A sophisticated yet simple way to build an array of a user defined data-type", + "tmp": "Create a temporary file and write to it", + "datetime": "A date and/or time conversion tool (like `printf` but for date and time values)", + "debug": "Debugging information", + "export": "Define an environmental variable and set it's value", + "args": "Command line flag parser for Murex shell scripting", + "global": "Define a global variable and set it's value", + "openagent": "Creates a handler function for `open`", + "method": "Define a methods supported data-types", + "cast": "Alters the data-type of the previous function without altering its output", + "set": "Define a variable (typically local) and set it's value", + "unsafe": "Execute a block of code, always returning a zero exit number", + "type": "Command type (function, builtin, alias, etc)", + "fid-list": "Lists all running functions within the current Murex session", + "getfile": "Makes a standard HTTP request and return the contents as Murex-aware data type for passing along Murex pipelines.", + "err": "Print a line to the stderr", + "esccli": "Escapes an array so output is valid shell code", + "eschtml": "Encode or decodes text for HTML", + "escurl": "Encode or decodes text for the URL", + "exec": "Runs an executable", + "fexec": "Execute a command or function, bypassing the usual order of precedence.", + "break": "Terminate execution of a block within your processes scope", + "return": "Exits current function scope", + "exit": "Exit murex", + "expr": "Expressions: mathematical, string comparisons, logical operators", + "false": "Returns a `false` value", + "foreach": "Iterate through an array", + "formap": "Iterate through a map or other collection of data", + "for": "A more familiar iteration loop to existing developers", + "fg": "Sends a background process into the foreground", + "runmode": "Alter the scheduler's behaviour at higher scoping level", + "rand": "Random field generator", + "get-type": "Returns the data-type of a variable or pipe", + "exitnum": "Output the exit number of the previous process", + "pt": "Pipe telemetry. Writes data-types and bytes written", + "get": "Makes a standard HTTP request and returns the result as a JSON object", + "g": "Glob pattern matching for file system objects (eg `*.txt`)", + "if": "Conditional statement to execute different blocks of code depending on the result of the condition", + "source": "Import Murex code from another file or code block", + "is-null": "Checks if a variable is null or undefined", + "mjoin": "Joins a list or array into a single string", + "fid-killall": "Terminate all running Murex functions in current session", + "fid-kill": "Terminate a running Murex function", + "left": "Left substring every item in a list", + "f": "Lists or filters file system objects (eg files)", + "which": "Locate command origin", + "lockfile": "Create and manage lock files", + "and": "Returns `true` or `false` depending on whether multiple conditions are met", + "or": "Returns `true` or `false` depending on whether one code-block out of multiple ones supplied is successful or unsuccessful.", + "while": "Loop until condition false", + "man-summary": "Outputs a man page summary of a command", + "match": "Match an exact value in an array", + "event": "Event driven programming for shell scripts", + "murex-package": "Murex's package manager", + "version": "Get Murex version", + "murex-docs": "Displays the man pages for Murex builtins", + "continue": "Terminate process of a block within a caller function", + "not-func": "Reads the stdin and exit number from previous process and not's it's condition", + "devnull": "null function. Similar to /dev/null", + "open": "Open a file with a preferred handler", + "os": "Output the auto-detected OS name", + "out": "Print a string to the stdout with a trailing new line character", + "tout": "Print a string to the stdout and set it's data-type", + "man-get-flags": "Parses man page files for command line flags", + "trypipe": "Checks for non-zero exits of each function in a pipeline", + "post": "HTTP POST request with a JSON-parsable return", + "prepend": "Add data to the start of an array", + "pretty": "Prettifies JSON to make it human readable", + "struct-keys": "Outputs all the keys in a structure as a file path", + "private": "Define a private function block", + "time": "Returns the execution run time of a command or block", + "function": "Define a function block", + "escape": "Escape or unescape input", "murex-update-exe-list": "Forces Murex to rescan $PATH looking for executables", - "read": "`read` a line of input from the user and store as a variable", - "tread": "`read` a line of input from the user and store as a user defined *typed* variable (deprecated)", - "format": "Reformat one data-type into another data-type", - "rx": "Regexp pattern matching for file system objects (eg `.*\\\\.txt`)", - "regexp": "Regexp tools for arrays / lists of strings", - "open-image": "Renders bitmap image data on your terminal", - "mtac": "Reverse the order of an array", - "right": "Right substring every item in a list", - "round": "Round a number by a user defined precision", - "signal": "Sends a signal RPC", - "summary": "Defines a summary help text for a command", - "config": "Query or define Murex runtime settings", - "runtime": "Returns runtime information on the internal state of Murex", - "test": "Murex's test framework - define tests, run tests and debug shell scripts", - "msort": "Sorts an array - data type agnostic", - "jsplit": "Splits stdin into a JSON array based on a regex parameter", - "trypipeerr": "Checks state of each function in a pipeline and exits block on error", - "tryerr": "Handles errors inside a block of code", - "a": "A sophisticated yet simple way to stream an array or list (mkarray)", - "switch": "Blocks of cascading conditionals", - "autocomplete": "Set definitions for tab-completion in the command line", - "tabulate": "Table transformation tools", - "true": "Returns a `true` value", - "try": "Handles non-zero exits inside a block of code", - "die": "Terminate murex with an exit number of 1 (deprecated)", - "let": "Evaluate a mathematical function and assign to variable (deprecated)", - "murex-parser": "Runs the Murex parser against a block of code", + "read": "`read` a line of input from the user and store as a variable", + "tread": "`read` a line of input from the user and store as a user defined *typed* variable (deprecated)", + "format": "Reformat one data-type into another data-type", + "rx": "Regexp pattern matching for file system objects (eg `.*\\\\.txt`)", + "regexp": "Regexp tools for arrays / lists of strings", + "open-image": "Renders bitmap image data on your terminal", + "mtac": "Reverse the order of an array", + "right": "Right substring every item in a list", + "round": "Round a number by a user defined precision", + "signal": "Sends a signal RPC", + "summary": "Defines a summary help text for a command", + "config": "Query or define Murex runtime settings", + "runtime": "Returns runtime information on the internal state of Murex", + "test": "Murex's test framework - define tests, run tests and debug shell scripts", + "msort": "Sorts an array - data type agnostic", + "jsplit": "Splits stdin into a JSON array based on a regex parameter", + "trypipeerr": "Checks state of each function in a pipeline and exits block on error", + "tryerr": "Handles errors inside a block of code", + "a": "A sophisticated yet simple way to stream an array or list (mkarray)", + "switch": "Blocks of cascading conditionals", + "autocomplete": "Set definitions for tab-completion in the command line", + "tabulate": "Table transformation tools", + "true": "Returns a `true` value", + "try": "Handles non-zero exits inside a block of code", + "die": "Terminate murex with an exit number of 1 (deprecated)", + "let": "Evaluate a mathematical function and assign to variable (deprecated)", + "murex-parser": "Runs the Murex parser against a block of code", - "select": "Inlining SQL into shell pipelines", - "bz2": "Decompress a bz2 file", + "bz2": "Decompress a bz2 file", "base64": "Encode or decode a base64 string", - "gz": "Compress or decompress a gzip file", - "qr": "Creates a QR code from stdin", - "sleep": "Suspends the shell for a number of seconds", + "gz": "Compress or decompress a gzip file", + "qr": "Creates a QR code from stdin", + "sleep": "Suspends the shell for a number of seconds", - - "commands/key-code": "Returns character sequences for any key pressed (ie sent from the terminal)", - "commands/addheading": "Adds headings to a table", - "commands/prefix": "Prefix a string to every item in a list", - "commands/suffix": "Prefix a string to every item in a list", - "commands/alias": "Create an alias for a command", - "commands/alter": "Change a value within a structured data-type and pass that change along the pipeline without altering the original source input", - "commands/append": "Add data to the end of an array", - "commands/bg": "Run processes in the background", - "commands/cpuarch": "Output the hosts CPU architecture", - "commands/cpucount": "Output the number of CPU cores available on your host", - "commands/catch": "Handles the exception code raised by `try` or `trypipe`", - "commands/cd": "Change (working) directory", - "commands/list.case": "Changes the character case of a string or all elements in an array", - "commands/bexists": "Check which builtins exist", - "commands/history": "Outputs murex's command history", - "commands/count": "Count items in a map, list or array", - "commands/2darray": "Create a 2D JSON array from multiple input sources", - "commands/ja": "A sophisticated yet simply way to build a JSON array", - "commands/map": "Creates a map from two data sources", - "commands/pipe": "Manage Murex named pipes", - "commands/ta": "A sophisticated yet simple way to build an array of a user defined data-type", - "commands/tmp": "Create a temporary file and write to it", - "commands/datetime": "A date and/or time conversion tool (like `printf` but for date and time values)", - "commands/debug": "Debugging information", - "commands/export": "Define an environmental variable and set it's value", - "commands/args": "Command line flag parser for Murex shell scripting", - "commands/global": "Define a global variable and set it's value", - "commands/openagent": "Creates a handler function for `open`", - "commands/method": "Define a methods supported data-types", - "commands/cast": "Alters the data-type of the previous function without altering its output", - "commands/set": "Define a variable (typically local) and set it's value", - "commands/unsafe": "Execute a block of code, always returning a zero exit number", - "commands/type": "Command type (function, builtin, alias, etc)", - "commands/fid-list": "Lists all running functions within the current Murex session", - "commands/getfile": "Makes a standard HTTP request and return the contents as Murex-aware data type for passing along Murex pipelines.", - "commands/err": "Print a line to the stderr", - "commands/esccli": "Escapes an array so output is valid shell code", - "commands/eschtml": "Encode or decodes text for HTML", - "commands/escurl": "Encode or decodes text for the URL", - "commands/exec": "Runs an executable", - "commands/fexec": "Execute a command or function, bypassing the usual order of precedence.", - "commands/break": "Terminate execution of a block within your processes scope", - "commands/return": "Exits current function scope", - "commands/exit": "Exit murex", - "commands/expr": "Expressions: mathematical, string comparisons, logical operators", - "commands/false": "Returns a `false` value", - "commands/foreach": "Iterate through an array", - "commands/formap": "Iterate through a map or other collection of data", - "commands/for": "A more familiar iteration loop to existing developers", - "commands/fg": "Sends a background process into the foreground", - "commands/runmode": "Alter the scheduler's behaviour at higher scoping level", - "commands/rand": "Random field generator", - "commands/get-type": "Returns the data-type of a variable or pipe", - "commands/exitnum": "Output the exit number of the previous process", - "commands/pt": "Pipe telemetry. Writes data-types and bytes written", - "commands/get": "Makes a standard HTTP request and returns the result as a JSON object", - "commands/g": "Glob pattern matching for file system objects (eg `*.txt`)", - "commands/if": "Conditional statement to execute different blocks of code depending on the result of the condition", - "commands/source": "Import Murex code from another file or code block", - "commands/is-null": "Checks if a variable is null or undefined", - "commands/mjoin": "Joins a list or array into a single string", - "commands/fid-killall": "Terminate all running Murex functions in current session", - "commands/fid-kill": "Terminate a running Murex function", - "commands/left": "Left substring every item in a list", - "commands/f": "Lists or filters file system objects (eg files)", - "commands/which": "Locate command origin", - "commands/lockfile": "Create and manage lock files", - "commands/and": "Returns `true` or `false` depending on whether multiple conditions are met", - "commands/or": "Returns `true` or `false` depending on whether one code-block out of multiple ones supplied is successful or unsuccessful.", - "commands/while": "Loop until condition false", - "commands/man-summary": "Outputs a man page summary of a command", - "commands/match": "Match an exact value in an array", - "commands/event": "Event driven programming for shell scripts", - "commands/murex-package": "Murex's package manager", - "commands/version": "Get Murex version", - "commands/murex-docs": "Displays the man pages for Murex builtins", - "commands/continue": "Terminate process of a block within a caller function", - "commands/not-func": "Reads the stdin and exit number from previous process and not's it's condition", - "commands/devnull": "null function. Similar to /dev/null", - "commands/open": "Open a file with a preferred handler", - "commands/os": "Output the auto-detected OS name", - "commands/out": "Print a string to the stdout with a trailing new line character", - "commands/tout": "Print a string to the stdout and set it's data-type", - "commands/man-get-flags": "Parses man page files for command line flags", - "commands/trypipe": "Checks for non-zero exits of each function in a pipeline", - "commands/post": "HTTP POST request with a JSON-parsable return", - "commands/prepend": "Add data to the start of an array", - "commands/pretty": "Prettifies JSON to make it human readable", - "commands/struct-keys": "Outputs all the keys in a structure as a file path", - "commands/private": "Define a private function block", - "commands/time": "Returns the execution run time of a command or block", - "commands/function": "Define a function block", - "commands/escape": "Escape or unescape input", - "commands/murex-update-exe-list": "Forces Murex to rescan $PATH looking for executables", - "commands/read": "`read` a line of input from the user and store as a variable", - "commands/tread": "`read` a line of input from the user and store as a user defined *typed* variable (deprecated)", - "commands/format": "Reformat one data-type into another data-type", - "commands/rx": "Regexp pattern matching for file system objects (eg `.*\\\\.txt`)", - "commands/regexp": "Regexp tools for arrays / lists of strings", - "commands/open-image": "Renders bitmap image data on your terminal", - "commands/mtac": "Reverse the order of an array", - "commands/right": "Right substring every item in a list", - "commands/round": "Round a number by a user defined precision", - "commands/signal": "Sends a signal RPC", - "commands/summary": "Defines a summary help text for a command", - "commands/config": "Query or define Murex runtime settings", - "commands/runtime": "Returns runtime information on the internal state of Murex", - "commands/test": "Murex's test framework - define tests, run tests and debug shell scripts", - "commands/msort": "Sorts an array - data type agnostic", - "commands/jsplit": "Splits stdin into a JSON array based on a regex parameter", - "commands/trypipeerr": "Checks state of each function in a pipeline and exits block on error", - "commands/tryerr": "Handles errors inside a block of code", - "commands/a": "A sophisticated yet simple way to stream an array or list (mkarray)", - "commands/switch": "Blocks of cascading conditionals", - "commands/autocomplete": "Set definitions for tab-completion in the command line", - "commands/tabulate": "Table transformation tools", - "commands/true": "Returns a `true` value", - "commands/try": "Handles non-zero exits inside a block of code", - "commands/die": "Terminate murex with an exit number of 1 (deprecated)", - "commands/let": "Evaluate a mathematical function and assign to variable (deprecated)", - "commands/murex-parser": "Runs the Murex parser against a block of code", - "mkarray/date": "Create arrays of dates", - "mkarray/character": "Making character arrays (a to z)", - "mkarray/decimal": "Create arrays of decimal integers", - "mkarray/non-decimal": "Create arrays of integers from non-decimal number bases", - "mkarray/special": "Create arrays from ranges of dictionary terms (eg weekdays, months, seasons, etc)", - "optional/select": "Inlining SQL into shell pipelines", - "optional/bz2": "Decompress a bz2 file", - "optional/base64": "Encode or decode a base64 string", - "optional/gz": "Compress or decompress a gzip file", - "optional/qr": "Creates a QR code from stdin", - "optional/sleep": "Suspends the shell for a number of seconds", - "parser/expr-inlined": "Inline expressions", - "parser/c-style-fun": "Inlined commands for expressions and statements", - "parser/range": "Outputs a ranged subset of data from stdin", - "parser/item-index": "Outputs an element from an array, map or table", - "parser/element": "Outputs an element from a nested structure", - "parser/namedpipe": "Reads from a Murex named pipe", - "parser/stdin": "Read the stdin belonging to the parent code block", - "parser/file-truncate": "Writes stdin to disk - overwriting contents if file already exists", - "parser/double-quote": "Initiates or terminates a string (variables expanded)", - "parser/scalar": "Expand values as a scalar", - "parser/brace-quote": "Initiates or terminates a string (variables expanded)", - "parser/create-array": "Quickly generate arrays", - "parser/create-object": "Quickly generate objects (dictionaries / maps)", - "parser/logical-and": "Continues next operation if previous operation passes", - "parser/single-quote": "Initiates or terminates a string (variables not expanded)", - "parser/brace-quote-func": "Write a string to the stdout without new line (deprecated)", - "parser/multiply-by": "Multiplies a variable by the right hand value (expression)", - "parser/multiplication": "Multiplies one numeric value with another (expression)", - "parser/add-with": "Adds the right hand value to a variable (expression)", - "parser/addition": "Adds two numeric values together (expression)", - "parser/subtract-by": "Subtracts a variable by the right hand value (expression)", - "parser/pipe-arrow": "Pipes stdout from the left hand command to stdin of the right hand command", - "parser/subtraction": "Subtracts one numeric value from another (expression)", - "parser/divide-by": "Divides a variable by the right hand value (expression)", - "parser/division": "Divides one numeric value from another (expression)", - "parser/assign-or-merge": "Merges the right hand value to a variable on the left hand side (expression)", - "parser/pipe-generic": "Pipes a reformatted stdout stream from the left hand command to stdin of the right hand command", - "parser/equ": "Evaluate a mathematical function (deprecated)", - "parser/file-append": "Writes stdin to disk - appending contents if file already exists", - "parser/elvis": "Returns the right operand if the left operand is falsy (expression)", - "parser/null-coalescing": "Returns the right operand if the left operand is empty / undefined (expression)", - "parser/pipe-err": "Pipes stderr from the left hand command to stdin of the right hand command (DEPRECATED)", - "parser/array": "Expand values as an array", - "parser/lambda": "Iterate through structured data", - "parser/curly-brace": "Initiates or terminates a code block", - "parser/pipe-posix": "Pipes stdout from the left hand command to stdin of the right hand command", - "parser/logical-or": "Continues next operation only if previous operation fails", - "parser/tilde": "Home directory path variable", - "events/oncommandcompletion": "Trigger an event upon a command's completion", - "events/onfilesystemchange": "Add a filesystem watch", - "events/onkeypress": "Custom definable key bindings and macros", - "events/onpreview": "Full screen previews for files and command documentation", - "events/onprompt": "Events triggered by changes in state of the interactive shell", - "events/onsecondselapsed": "Events triggered by time intervals", - "events/onsignalreceived": "Trap OS signals", - "types/generic": "generic (primitive)", - "types/bool": "Boolean (primitive)", - "types/commonlog": "Apache httpd \"common\" log format", - "types/csv": "CSV files (and other character delimited tables)", - "types/float": "Floating point number (primitive)", - "types/hcl": "HashiCorp Configuration Language (HCL)", - "types/int": "Whole number (primitive)", - "types/json": "JavaScript Object Notation (JSON)", - "types/jsonc": "Concatenated JSON", - "types/jsonl": "JSON Lines", - "types/num": "Floating point number (primitive)", - "types/path": "Structured object for working with file and directory paths", - "types/paths": "Structured array for working with `$PATH` style data", - "types/str": "string (primitive)", - "types/toml": "Tom's Obvious, Minimal Language (TOML)", - "types/yaml": "YAML Ain't Markup Language (YAML)", - "types/mxjson": "Murex-flavoured JSON (deprecated)", - "variables/numeric": "Variables who's name is a positive integer, eg `0`, `1`, `2`, `3` and above", - "variables/meta-values": "State information for iteration blocks", - "variables/argv": "Array of the command name and parameters within a given scope", - "variables/columns": "Character width of terminal", - "variables/event_return": "Return values for events", - "variables/home": "Return the home directory for the current session user", - "variables/hostname": "Hostname of the current machine", - "variables/lines": "Character height of terminal", - "variables/logname": "Username for the current session (historic)", - "variables/murex_argv": "Array of the command name and parameters passed to the current shell", - "variables/murex_exe": "Absolute path to running shell", - "variables/oldpwd": "Return the home directory for the current session user", - "variables/params": "Array of the parameters within a given scope", - "variables/pwdhist": "History of each change to the sessions working directory", - "variables/pwd": "Current working directory", - "variables/random": "Return a random 32-bit integer (historical)", - "variables/self": "Meta information about the running scope.", - "variables/shell": "Path of current shell", - "variables/tmpdir": "Return the temporary directory", - "variables/user": "Username for the current session", - "apis/Marshal": "Converts structured memory into a structured file format (eg for stdio)", - "apis/ReadArray": "Read from a data type one array element at a time", - "apis/ReadArrayWithType": "Read from a data type one array element at a time and return the elements contents and data type", - "apis/ReadIndex": "Data type handler for the index, `[`, builtin", - "apis/ReadMap": "Treat data type as a key/value structure and read its contents", - "apis/ReadNotIndex": "Data type handler for the bang-prefixed index, `![`, builtin", - "apis/Unmarshal": "Converts a structured file format into structured memory", - "apis/WriteArray": "Write a data type, one array element at a time", - "apis/lang.ArrayTemplate": "Unmarshals a data type into a Go struct and returns the results as an array", + "commands/key-code": "Returns character sequences for any key pressed (ie sent from the terminal)", + "commands/addheading": "Adds headings to a table", + "commands/prefix": "Prefix a string to every item in a list", + "commands/suffix": "Prefix a string to every item in a list", + "commands/alias": "Create an alias for a command", + "commands/alter": "Change a value within a structured data-type and pass that change along the pipeline without altering the original source input", + "commands/append": "Add data to the end of an array", + "commands/bg": "Run processes in the background", + "commands/cpuarch": "Output the hosts CPU architecture", + "commands/cpucount": "Output the number of CPU cores available on your host", + "commands/catch": "Handles the exception code raised by `try` or `trypipe`", + "commands/cd": "Change (working) directory", + "commands/list.case": "Changes the character case of a string or all elements in an array", + "commands/bexists": "Check which builtins exist", + "commands/history": "Outputs murex's command history", + "commands/count": "Count items in a map, list or array", + "commands/2darray": "Create a 2D JSON array from multiple input sources", + "commands/ja": "A sophisticated yet simply way to build a JSON array", + "commands/map": "Creates a map from two data sources", + "commands/pipe": "Manage Murex named pipes", + "commands/ta": "A sophisticated yet simple way to build an array of a user defined data-type", + "commands/tmp": "Create a temporary file and write to it", + "commands/datetime": "A date and/or time conversion tool (like `printf` but for date and time values)", + "commands/debug": "Debugging information", + "commands/export": "Define an environmental variable and set it's value", + "commands/args": "Command line flag parser for Murex shell scripting", + "commands/global": "Define a global variable and set it's value", + "commands/openagent": "Creates a handler function for `open`", + "commands/method": "Define a methods supported data-types", + "commands/cast": "Alters the data-type of the previous function without altering its output", + "commands/set": "Define a variable (typically local) and set it's value", + "commands/unsafe": "Execute a block of code, always returning a zero exit number", + "commands/type": "Command type (function, builtin, alias, etc)", + "commands/fid-list": "Lists all running functions within the current Murex session", + "commands/getfile": "Makes a standard HTTP request and return the contents as Murex-aware data type for passing along Murex pipelines.", + "commands/err": "Print a line to the stderr", + "commands/esccli": "Escapes an array so output is valid shell code", + "commands/eschtml": "Encode or decodes text for HTML", + "commands/escurl": "Encode or decodes text for the URL", + "commands/exec": "Runs an executable", + "commands/fexec": "Execute a command or function, bypassing the usual order of precedence.", + "commands/break": "Terminate execution of a block within your processes scope", + "commands/return": "Exits current function scope", + "commands/exit": "Exit murex", + "commands/expr": "Expressions: mathematical, string comparisons, logical operators", + "commands/false": "Returns a `false` value", + "commands/foreach": "Iterate through an array", + "commands/formap": "Iterate through a map or other collection of data", + "commands/for": "A more familiar iteration loop to existing developers", + "commands/fg": "Sends a background process into the foreground", + "commands/runmode": "Alter the scheduler's behaviour at higher scoping level", + "commands/rand": "Random field generator", + "commands/get-type": "Returns the data-type of a variable or pipe", + "commands/exitnum": "Output the exit number of the previous process", + "commands/pt": "Pipe telemetry. Writes data-types and bytes written", + "commands/get": "Makes a standard HTTP request and returns the result as a JSON object", + "commands/g": "Glob pattern matching for file system objects (eg `*.txt`)", + "commands/if": "Conditional statement to execute different blocks of code depending on the result of the condition", + "commands/source": "Import Murex code from another file or code block", + "commands/is-null": "Checks if a variable is null or undefined", + "commands/mjoin": "Joins a list or array into a single string", + "commands/fid-killall": "Terminate all running Murex functions in current session", + "commands/fid-kill": "Terminate a running Murex function", + "commands/left": "Left substring every item in a list", + "commands/f": "Lists or filters file system objects (eg files)", + "commands/which": "Locate command origin", + "commands/lockfile": "Create and manage lock files", + "commands/and": "Returns `true` or `false` depending on whether multiple conditions are met", + "commands/or": "Returns `true` or `false` depending on whether one code-block out of multiple ones supplied is successful or unsuccessful.", + "commands/while": "Loop until condition false", + "commands/man-summary": "Outputs a man page summary of a command", + "commands/match": "Match an exact value in an array", + "commands/event": "Event driven programming for shell scripts", + "commands/murex-package": "Murex's package manager", + "commands/version": "Get Murex version", + "commands/murex-docs": "Displays the man pages for Murex builtins", + "commands/continue": "Terminate process of a block within a caller function", + "commands/not-func": "Reads the stdin and exit number from previous process and not's it's condition", + "commands/devnull": "null function. Similar to /dev/null", + "commands/open": "Open a file with a preferred handler", + "commands/os": "Output the auto-detected OS name", + "commands/out": "Print a string to the stdout with a trailing new line character", + "commands/tout": "Print a string to the stdout and set it's data-type", + "commands/man-get-flags": "Parses man page files for command line flags", + "commands/trypipe": "Checks for non-zero exits of each function in a pipeline", + "commands/post": "HTTP POST request with a JSON-parsable return", + "commands/prepend": "Add data to the start of an array", + "commands/pretty": "Prettifies JSON to make it human readable", + "commands/struct-keys": "Outputs all the keys in a structure as a file path", + "commands/private": "Define a private function block", + "commands/time": "Returns the execution run time of a command or block", + "commands/function": "Define a function block", + "commands/escape": "Escape or unescape input", + "commands/murex-update-exe-list": "Forces Murex to rescan $PATH looking for executables", + "commands/read": "`read` a line of input from the user and store as a variable", + "commands/tread": "`read` a line of input from the user and store as a user defined *typed* variable (deprecated)", + "commands/format": "Reformat one data-type into another data-type", + "commands/rx": "Regexp pattern matching for file system objects (eg `.*\\\\.txt`)", + "commands/regexp": "Regexp tools for arrays / lists of strings", + "commands/open-image": "Renders bitmap image data on your terminal", + "commands/mtac": "Reverse the order of an array", + "commands/right": "Right substring every item in a list", + "commands/round": "Round a number by a user defined precision", + "commands/signal": "Sends a signal RPC", + "commands/summary": "Defines a summary help text for a command", + "commands/config": "Query or define Murex runtime settings", + "commands/runtime": "Returns runtime information on the internal state of Murex", + "commands/test": "Murex's test framework - define tests, run tests and debug shell scripts", + "commands/msort": "Sorts an array - data type agnostic", + "commands/jsplit": "Splits stdin into a JSON array based on a regex parameter", + "commands/trypipeerr": "Checks state of each function in a pipeline and exits block on error", + "commands/tryerr": "Handles errors inside a block of code", + "commands/a": "A sophisticated yet simple way to stream an array or list (mkarray)", + "commands/switch": "Blocks of cascading conditionals", + "commands/autocomplete": "Set definitions for tab-completion in the command line", + "commands/tabulate": "Table transformation tools", + "commands/true": "Returns a `true` value", + "commands/try": "Handles non-zero exits inside a block of code", + "commands/die": "Terminate murex with an exit number of 1 (deprecated)", + "commands/let": "Evaluate a mathematical function and assign to variable (deprecated)", + "commands/murex-parser": "Runs the Murex parser against a block of code", + "mkarray/date": "Create arrays of dates", + "mkarray/character": "Making character arrays (a to z)", + "mkarray/decimal": "Create arrays of decimal integers", + "mkarray/non-decimal": "Create arrays of integers from non-decimal number bases", + "mkarray/special": "Create arrays from ranges of dictionary terms (eg weekdays, months, seasons, etc)", + "optional/select": "Inlining SQL into shell pipelines", + "optional/bz2": "Decompress a bz2 file", + "optional/base64": "Encode or decode a base64 string", + "optional/gz": "Compress or decompress a gzip file", + "optional/qr": "Creates a QR code from stdin", + "optional/sleep": "Suspends the shell for a number of seconds", + "parser/expr-inlined": "Inline expressions", + "parser/c-style-fun": "Inlined commands for expressions and statements", + "parser/range": "Outputs a ranged subset of data from stdin", + "parser/item-index": "Outputs an element from an array, map or table", + "parser/element": "Outputs an element from a nested structure", + "parser/namedpipe": "Reads from a Murex named pipe", + "parser/stdin": "Read the stdin belonging to the parent code block", + "parser/file-truncate": "Writes stdin to disk - overwriting contents if file already exists", + "parser/double-quote": "Initiates or terminates a string (variables expanded)", + "parser/scalar": "Expand values as a scalar", + "parser/brace-quote": "Initiates or terminates a string (variables expanded)", + "parser/create-array": "Quickly generate arrays", + "parser/create-object": "Quickly generate objects (dictionaries / maps)", + "parser/logical-and": "Continues next operation if previous operation passes", + "parser/single-quote": "Initiates or terminates a string (variables not expanded)", + "parser/brace-quote-func": "Write a string to the stdout without new line (deprecated)", + "parser/multiply-by": "Multiplies a variable by the right hand value (expression)", + "parser/multiplication": "Multiplies one numeric value with another (expression)", + "parser/add-with": "Adds the right hand value to a variable (expression)", + "parser/addition": "Adds two numeric values together (expression)", + "parser/subtract-by": "Subtracts a variable by the right hand value (expression)", + "parser/pipe-arrow": "Pipes stdout from the left hand command to stdin of the right hand command", + "parser/subtraction": "Subtracts one numeric value from another (expression)", + "parser/divide-by": "Divides a variable by the right hand value (expression)", + "parser/division": "Divides one numeric value from another (expression)", + "parser/assign-or-merge": "Merges the right hand value to a variable on the left hand side (expression)", + "parser/pipe-generic": "Pipes a reformatted stdout stream from the left hand command to stdin of the right hand command", + "parser/equ": "Evaluate a mathematical function (deprecated)", + "parser/file-append": "Writes stdin to disk - appending contents if file already exists", + "parser/elvis": "Returns the right operand if the left operand is falsy (expression)", + "parser/null-coalescing": "Returns the right operand if the left operand is empty / undefined (expression)", + "parser/pipe-err": "Pipes stderr from the left hand command to stdin of the right hand command (DEPRECATED)", + "parser/array": "Expand values as an array", + "parser/lambda": "Iterate through structured data", + "parser/curly-brace": "Initiates or terminates a code block", + "parser/pipe-posix": "Pipes stdout from the left hand command to stdin of the right hand command", + "parser/logical-or": "Continues next operation only if previous operation fails", + "parser/tilde": "Home directory path variable", + "events/oncommandcompletion": "Trigger an event upon a command's completion", + "events/onfilesystemchange": "Add a filesystem watch", + "events/onkeypress": "Custom definable key bindings and macros", + "events/onpreview": "Full screen previews for files and command documentation", + "events/onprompt": "Events triggered by changes in state of the interactive shell", + "events/onsecondselapsed": "Events triggered by time intervals", + "events/onsignalreceived": "Trap OS signals", + "types/generic": "generic (primitive)", + "types/bool": "Boolean (primitive)", + "types/commonlog": "Apache httpd \"common\" log format", + "types/csv": "CSV files (and other character delimited tables)", + "types/float": "Floating point number (primitive)", + "types/hcl": "HashiCorp Configuration Language (HCL)", + "types/int": "Whole number (primitive)", + "types/json": "JavaScript Object Notation (JSON)", + "types/jsonc": "Concatenated JSON", + "types/jsonl": "JSON Lines", + "types/num": "Floating point number (primitive)", + "types/path": "Structured object for working with file and directory paths", + "types/paths": "Structured array for working with `$PATH` style data", + "types/str": "string (primitive)", + "types/toml": "Tom's Obvious, Minimal Language (TOML)", + "types/yaml": "YAML Ain't Markup Language (YAML)", + "types/mxjson": "Murex-flavoured JSON (deprecated)", + "variables/numeric": "Variables who's name is a positive integer, eg `0`, `1`, `2`, `3` and above", + "variables/meta-values": "State information for iteration blocks", + "variables/argv": "Array of the command name and parameters within a given scope", + "variables/columns": "Character width of terminal", + "variables/event_return": "Return values for events", + "variables/home": "Return the home directory for the current session user", + "variables/hostname": "Hostname of the current machine", + "variables/lines": "Character height of terminal", + "variables/logname": "Username for the current session (historic)", + "variables/murex_argv": "Array of the command name and parameters passed to the current shell", + "variables/murex_exe": "Absolute path to running shell", + "variables/oldpwd": "Return the home directory for the current session user", + "variables/params": "Array of the parameters within a given scope", + "variables/pwdhist": "History of each change to the sessions working directory", + "variables/pwd": "Current working directory", + "variables/random": "Return a random 32-bit integer (historical)", + "variables/self": "Meta information about the running scope.", + "variables/shell": "Path of current shell", + "variables/tmpdir": "Return the temporary directory", + "variables/user": "Username for the current session", + "apis/Marshal": "Converts structured memory into a structured file format (eg for stdio)", + "apis/ReadArray": "Read from a data type one array element at a time", + "apis/ReadArrayWithType": "Read from a data type one array element at a time and return the elements contents and data type", + "apis/ReadIndex": "Data type handler for the index, `[`, builtin", + "apis/ReadMap": "Treat data type as a key/value structure and read its contents", + "apis/ReadNotIndex": "Data type handler for the bang-prefixed index, `![`, builtin", + "apis/Unmarshal": "Converts a structured file format into structured memory", + "apis/WriteArray": "Write a data type, one array element at a time", + "apis/lang.ArrayTemplate": "Unmarshals a data type into a Go struct and returns the results as an array", "apis/lang.ArrayWithTypeTemplate": "Unmarshals a data type into a Go struct and returns the results as an array with data type included", - "apis/lang.IndexTemplateObject": "Returns element(s) from a data structure", - "apis/lang.IndexTemplateTable": "Returns element(s) from a table", - "apis/lang.MarshalData": "Converts structured memory into a Murex data-type (eg for stdio)", - "apis/lang.UnmarshalData": "Converts a Murex data-type into structured memory", - "user-guide/ansi": "Infixed constants that return ANSI escape sequences", - "user-guide/bang-prefix": "Bang prefixing to reverse default actions", - "user-guide/code-block": "Overview of how code blocks are parsed", - "user-guide/fileref": "How to track what code was loaded and from where", - "user-guide/hint-text": "A status bar for your shell", - "user-guide/integrations": "Default integrations shipped with Murex", - "user-guide/interactive-shell": "What's different about Murex's interactive shell?", - "user-guide/job-control": "How to manage jobs with Murex", - "user-guide/modules": "Modules and packages: An Introduction", - "user-guide/namedpipes": "A detailed breakdown of named pipes in Murex", + "apis/lang.IndexTemplateObject": "Returns element(s) from a data structure", + "apis/lang.IndexTemplateTable": "Returns element(s) from a table", + "apis/lang.MarshalData": "Converts structured memory into a Murex data-type (eg for stdio)", + "apis/lang.UnmarshalData": "Converts a Murex data-type into structured memory", + "user-guide/ansi": "Infixed constants that return ANSI escape sequences", + "user-guide/bang-prefix": "Bang prefixing to reverse default actions", + "user-guide/code-block": "Overview of how code blocks are parsed", + "user-guide/fileref": "How to track what code was loaded and from where", + "user-guide/hint-text": "A status bar for your shell", + "user-guide/integrations": "Default integrations shipped with Murex", + "user-guide/interactive-shell": "What's different about Murex's interactive shell?", + "user-guide/job-control": "How to manage jobs with Murex", + "user-guide/modules": "Modules and packages: An Introduction", + "user-guide/namedpipes": "A detailed breakdown of named pipes in Murex", "user-guide/operators-and-tokens": "All supported operators and tokens", - "user-guide/pipeline": "Overview of what a \"pipeline\" is", - "user-guide/profile": "A breakdown of the different files loaded on start up", - "user-guide/reserved-vars": "Special variables reserved by Murex", - "user-guide/rosetta-stone": "A tabulated list of Bashism's and their equivalent Murex syntax", - "user-guide/schedulers": "Overview of the different schedulers (or 'run modes') in Murex", - "user-guide/strict-types": "Expressions can auto-convert types or strictly honour data types", - "user-guide/terminal-keys": "A list of all the terminal hotkeys and their uses", - "user-guide/scoping": "How scoping works within Murex", - "user-guide/murex-arrays": "Examples using arrays within Murex", - "integrations/chatgpt": "How to enable ChatGPT hints", - "integrations/cheatsh": "Cheatsheets provided by cheat.sh", - "integrations/kitty": "Get more out of Kitty terminal emulator", - "integrations/make": "`make` integrations", - "integrations/man-pages": "Linux/UNIX `man` page integrations", - "integrations/spellcheck": "How to enable inline spellchecking", - "integrations/terminology": "Get more out of Terminology terminal emulator", - "integrations/direnv": "Directory specific environmental variables", - "integrations/yarn": "Working with `yarn` and `package.json`", - "integrations/iterm2": "Get more out of iTerm2 terminal emulator", - "changelog/v2.0": "This release comes with spellchecking, inlined images, smarter syntax completion and more", - "changelog/v2.1": "This release comes with support for inlining SQL and some major bug fixes plus a breaking change for `config`. Please read for details.", - "changelog/v2.10": "This release brings a few minor improvements and bug fixes rather than big new headline features.", - "changelog/v2.11": "This release mainly focuses on refinements in performance and usability, rather than introducing new features", - "changelog/v2.2": "This is mainly a bug fix release but it does include one breaking change for `config`. Please read for details.", - "changelog/v2.3": "This release includes significant changes to the interactive terminal", - "changelog/v2.4": "This release introduces a strict mode for variables, new builtin, performance improvements, and better error messages; plus a potential breaking change", - "changelog/v2.5": "This release introduces a number of new builtins, fixes some regression bugs and supercharges the `select` optional builtin (which I plan to include into the core builtins for non-Windows users in the next release).", - "changelog/v2.6": "This update has introduced a potential breaking change: variables now need to be defined before usage otherwise the commandline will fail. Read notes to learn how to disable this feature where needed. Also included in this release is the `select` command as part of the standard build.", - "changelog/v2.7": "This update has introduced another potential breaking change for your safety: zero length arrays now fail by default. Also errors inside subshells will cause the parent command to fail if ran inside a `try` or `trypipe` block.", - "changelog/v2.8": "This release comes with a number of experimental but stable features that might eventually become standard practice. The features are there to use if you with but adjacent from the older code so there is zero risk in updating to this version.", - "changelog/v2.9": "This release focuses on testing and REPL usability improvements but also includes updates several new run modes to make error handling easier in larger scripts.", - "changelog/v3.0": "This is a major release that brings a significant number of changes and improvements, including a complete overhaul of the parser. Backwards compatibility is a high priority however these new features bring greater readability and consistency to shell scripting. So while the older syntax remains for compatibility, it is worth migrating over to the newer syntax for all new code being written", - "changelog/v3.1": "This release includes mostly bug fixes and new experimental features which are opt into. To enable all experimental features, set the environmental variable `MUREX_EXPERIMENTAL` to any value. Or you can enable specific features individually via `config`", - "changelog/v4.0": "This release sees significant improvements for use with non-latin characters in both the interactive prompt and shell scripting. It introduces new syntax to make working with structured data even easier than before. As well as new data types and smoother user experience.", - "changelog/v4.1": "The previous releases have brought significant advancements to Murex's syntax but at the cost of longer gaps between releases. So the 4.1.x versions will be shorter releases but focusing on bug fixes. The 4.1.x release notes will be appended to [murex.rocks changelog](https://murex.rocks/changelog/v4.1.html) and available on [Github releases](https://github.com/lmorg/murex/releases) too", - "changelog/v4.2": "Murex usage has raised considerably in recent weeks. This release addresses a number of feature requests and bugs raised on Github.", - "changelog/v4.3": "This brings improved support on Windows plus one breaking change from the previous release (v4.2)", - "changelog/v4.4": "v4.4 features two new builtins, improvements in testing, and automatic generation of autocompletion suggestions backed by man page parsing. Plus there has been a lot of focus on improving _readline_ responsiveness", - "changelog/v5.0": "v5.0 is a massive release. It brings along changes to syntax, new operators as well as new builtins, reserved variables and a new event", - "changelog/v5.1": "This release brings new operators and a builtin, all for managing null types. There is also a substantial revamp to readline's responsiveness.", - "changelog/v5.2": "The v5.2 release introduces significant new features and improvements for those using Murex as their interactive shell. Many of these features are unique to Murex.", - "changelog/v5.3": "Caching has been vastly improved in this release due to a new sqlite3-backed persistent `cache.db`. There have also been some improvements to `[f1]` help pages", - "changelog/v6.0": "Despite this being a new major version release, it is a vary minor update. Aside from a handful of bugfixes, the most significant change is notice of deprecation for `=`, `let`, and `?`.", - "changelog/v6.1": "This release sees a massive jump in event-driven capabilities as well as several new features and bug fixes.", - "changelog/v6.2": "Bug fix release", - "changelog/v6.3": "This is a massive release ahead of the v7.0. This brings notifications of new deprecations, new builtins, new flags, improved CI/CD flow, and changes to the website. Unfortunately it also carries 3 breaking changes.", - + "user-guide/pipeline": "Overview of what a \"pipeline\" is", + "user-guide/profile": "A breakdown of the different files loaded on start up", + "user-guide/reserved-vars": "Special variables reserved by Murex", + "user-guide/rosetta-stone": "A tabulated list of Bashism's and their equivalent Murex syntax", + "user-guide/schedulers": "Overview of the different schedulers (or 'run modes') in Murex", + "user-guide/strict-types": "Expressions can auto-convert types or strictly honour data types", + "user-guide/terminal-keys": "A list of all the terminal hotkeys and their uses", + "user-guide/scoping": "How scoping works within Murex", + "user-guide/murex-arrays": "Examples using arrays within Murex", + "integrations/chatgpt": "How to enable ChatGPT hints", + "integrations/cheatsh": "Cheatsheets provided by cheat.sh", + "integrations/kitty": "Get more out of Kitty terminal emulator", + "integrations/make": "`make` integrations", + "integrations/man-pages": "Linux/UNIX `man` page integrations", + "integrations/spellcheck": "How to enable inline spellchecking", + "integrations/terminology": "Get more out of Terminology terminal emulator", + "integrations/direnv": "Directory specific environmental variables", + "integrations/yarn": "Working with `yarn` and `package.json`", + "integrations/iterm2": "Get more out of iTerm2 terminal emulator", + "changelog/v2.0": "This release comes with spellchecking, inlined images, smarter syntax completion and more", + "changelog/v2.1": "This release comes with support for inlining SQL and some major bug fixes plus a breaking change for `config`. Please read for details.", + "changelog/v2.10": "This release brings a few minor improvements and bug fixes rather than big new headline features.", + "changelog/v2.11": "This release mainly focuses on refinements in performance and usability, rather than introducing new features", + "changelog/v2.2": "This is mainly a bug fix release but it does include one breaking change for `config`. Please read for details.", + "changelog/v2.3": "This release includes significant changes to the interactive terminal", + "changelog/v2.4": "This release introduces a strict mode for variables, new builtin, performance improvements, and better error messages; plus a potential breaking change", + "changelog/v2.5": "This release introduces a number of new builtins, fixes some regression bugs and supercharges the `select` optional builtin (which I plan to include into the core builtins for non-Windows users in the next release).", + "changelog/v2.6": "This update has introduced a potential breaking change: variables now need to be defined before usage otherwise the commandline will fail. Read notes to learn how to disable this feature where needed. Also included in this release is the `select` command as part of the standard build.", + "changelog/v2.7": "This update has introduced another potential breaking change for your safety: zero length arrays now fail by default. Also errors inside subshells will cause the parent command to fail if ran inside a `try` or `trypipe` block.", + "changelog/v2.8": "This release comes with a number of experimental but stable features that might eventually become standard practice. The features are there to use if you with but adjacent from the older code so there is zero risk in updating to this version.", + "changelog/v2.9": "This release focuses on testing and REPL usability improvements but also includes updates several new run modes to make error handling easier in larger scripts.", + "changelog/v3.0": "This is a major release that brings a significant number of changes and improvements, including a complete overhaul of the parser. Backwards compatibility is a high priority however these new features bring greater readability and consistency to shell scripting. So while the older syntax remains for compatibility, it is worth migrating over to the newer syntax for all new code being written", + "changelog/v3.1": "This release includes mostly bug fixes and new experimental features which are opt into. To enable all experimental features, set the environmental variable `MUREX_EXPERIMENTAL` to any value. Or you can enable specific features individually via `config`", + "changelog/v4.0": "This release sees significant improvements for use with non-latin characters in both the interactive prompt and shell scripting. It introduces new syntax to make working with structured data even easier than before. As well as new data types and smoother user experience.", + "changelog/v4.1": "The previous releases have brought significant advancements to Murex's syntax but at the cost of longer gaps between releases. So the 4.1.x versions will be shorter releases but focusing on bug fixes. The 4.1.x release notes will be appended to [murex.rocks changelog](https://murex.rocks/changelog/v4.1.html) and available on [Github releases](https://github.com/lmorg/murex/releases) too", + "changelog/v4.2": "Murex usage has raised considerably in recent weeks. This release addresses a number of feature requests and bugs raised on Github.", + "changelog/v4.3": "This brings improved support on Windows plus one breaking change from the previous release (v4.2)", + "changelog/v4.4": "v4.4 features two new builtins, improvements in testing, and automatic generation of autocompletion suggestions backed by man page parsing. Plus there has been a lot of focus on improving _readline_ responsiveness", + "changelog/v5.0": "v5.0 is a massive release. It brings along changes to syntax, new operators as well as new builtins, reserved variables and a new event", + "changelog/v5.1": "This release brings new operators and a builtin, all for managing null types. There is also a substantial revamp to readline's responsiveness.", + "changelog/v5.2": "The v5.2 release introduces significant new features and improvements for those using Murex as their interactive shell. Many of these features are unique to Murex.", + "changelog/v5.3": "Caching has been vastly improved in this release due to a new sqlite3-backed persistent `cache.db`. There have also been some improvements to `[f1]` help pages", + "changelog/v6.0": "Despite this being a new major version release, it is a vary minor update. Aside from a handful of bugfixes, the most significant change is notice of deprecation for `=`, `let`, and `?`.", + "changelog/v6.1": "This release sees a massive jump in event-driven capabilities as well as several new features and bug fixes.", + "changelog/v6.2": "Bug fix release", + "changelog/v6.3": "This is a massive release ahead of the v7.0. This brings notifications of new deprecations, new builtins, new flags, improved CI/CD flow, and changes to the website. Unfortunately it also carries 3 breaking changes.", } - Synonym = map[string]string{ - - "key-code": "key-code", - "addheading": "addheading", - "prefix": "prefix", - "list.prefix": "prefix", - "suffix": "suffix", - "list.suffix": "suffix", - "alias": "alias", - "!alias": "alias", - "alter": "alter", - "~>": "alter", - "append": "append", - "list.append": "append", - "bg": "bg", - "cpuarch": "cpuarch", - "sys.cpu.arch": "cpuarch", - "cpucount": "cpucount", - "sys.cpu.count": "cpucount", - "catch": "catch", - "!catch": "catch", - "cd": "cd", - "list.case": "list.case", - "bexists": "bexists", - "history": "history", - "count": "count", - "len": "count", - "2darray": "2darray", - "ja": "ja", - "map": "map", - "pipe": "pipe", - "!pipe": "pipe", - "ta": "ta", - "tmp": "tmp", - "datetime": "datetime", - "str.datetime": "datetime", - "debug": "debug", - "export": "export", - "!export": "export", - "unset": "export", - "var.env": "export", - "!var.env": "export", - "args": "args", - "global": "global", - "!global": "global", - "openagent": "openagent", - "!openagent": "openagent", - "method": "method", - "cast": "cast", - "set": "set", - "!set": "set", - "unsafe": "unsafe", - "type": "type", - "fid-list": "fid-list", - "jobs": "fid-list", - "getfile": "getfile", - "err": "err", - "esccli": "esccli", - "eschtml": "eschtml", - "!eschtml": "eschtml", - "escurl": "escurl", - "!escurl": "escurl", - "exec": "exec", - "command": "exec", - "exec.file": "exec", - "fexec": "fexec", - "builtin": "fexec", - "exec.builtin": "fexec", - "exec.function": "fexec", - "exec.private": "fexec", - "break": "break", - "return": "return", - "exit": "exit", - "expr": "expr", - "false": "false", - "foreach": "foreach", - "formap": "formap", - "for": "for", - "fg": "fg", - "runmode": "runmode", - "rand": "rand", - "get-type": "get-type", - "exitnum": "exitnum", - "pt": "pt", - "get": "get", - "g": "g", - "!g": "g", - "if": "if", - "!if": "if", - "source": "source", - ".": "source", - "is-null": "is-null", - "mjoin": "mjoin", - "list.join": "mjoin", - "fid-killall": "fid-killall", - "fid-kill": "fid-kill", - "left": "left", - "list.left": "left", - "f": "f", - "which": "which", - "lockfile": "lockfile", - "and": "and", - "!and": "and", - "or": "or", - "!or": "or", - "while": "while", - "!while": "while", - "man-summary": "man-summary", - "help.man.summary": "man-summary", - "match": "match", - "!match": "match", - "list.str": "match", - "!list.str": "match", - "event": "event", - "!event": "event", - "murex-package": "murex-package", - "version": "version", - "murex-docs": "murex-docs", - "help": "murex-docs", - "continue": "continue", - "!": "not-func", - "not": "not-func", - "null": "devnull", - "open": "open", - "os": "os", - "sys.os": "os", - "out": "out", - "echo": "out", - "tout": "tout", - "man-get-flags": "man-get-flags", - "trypipe": "trypipe", - "post": "post", - "prepend": "prepend", - "list.prepend": "prepend", - "pretty": "pretty", - "struct-keys": "struct-keys", - "private": "private", - "time": "time", - "function": "function", - "!function": "function", - "escape": "escape", - "!escape": "escape", + "key-code": "key-code", + "addheading": "addheading", + "prefix": "prefix", + "list.prefix": "prefix", + "suffix": "suffix", + "list.suffix": "suffix", + "alias": "alias", + "!alias": "alias", + "alter": "alter", + "~>": "alter", + "append": "append", + "list.append": "append", + "bg": "bg", + "cpuarch": "cpuarch", + "sys.cpu.arch": "cpuarch", + "cpucount": "cpucount", + "sys.cpu.count": "cpucount", + "catch": "catch", + "!catch": "catch", + "cd": "cd", + "list.case": "list.case", + "bexists": "bexists", + "history": "history", + "count": "count", + "len": "count", + "2darray": "2darray", + "ja": "ja", + "map": "map", + "pipe": "pipe", + "!pipe": "pipe", + "ta": "ta", + "tmp": "tmp", + "datetime": "datetime", + "str.datetime": "datetime", + "debug": "debug", + "export": "export", + "!export": "export", + "unset": "export", + "var.env": "export", + "!var.env": "export", + "args": "args", + "global": "global", + "!global": "global", + "openagent": "openagent", + "!openagent": "openagent", + "method": "method", + "cast": "cast", + "set": "set", + "!set": "set", + "unsafe": "unsafe", + "type": "type", + "fid-list": "fid-list", + "jobs": "fid-list", + "getfile": "getfile", + "err": "err", + "esccli": "esccli", + "eschtml": "eschtml", + "!eschtml": "eschtml", + "escurl": "escurl", + "!escurl": "escurl", + "exec": "exec", + "command": "exec", + "exec.file": "exec", + "fexec": "fexec", + "builtin": "fexec", + "exec.builtin": "fexec", + "exec.function": "fexec", + "exec.private": "fexec", + "break": "break", + "return": "return", + "exit": "exit", + "expr": "expr", + "false": "false", + "foreach": "foreach", + "formap": "formap", + "for": "for", + "fg": "fg", + "runmode": "runmode", + "rand": "rand", + "get-type": "get-type", + "exitnum": "exitnum", + "pt": "pt", + "get": "get", + "g": "g", + "!g": "g", + "if": "if", + "!if": "if", + "source": "source", + ".": "source", + "is-null": "is-null", + "mjoin": "mjoin", + "list.join": "mjoin", + "fid-killall": "fid-killall", + "fid-kill": "fid-kill", + "left": "left", + "list.left": "left", + "f": "f", + "which": "which", + "lockfile": "lockfile", + "and": "and", + "!and": "and", + "or": "or", + "!or": "or", + "while": "while", + "!while": "while", + "man-summary": "man-summary", + "help.man.summary": "man-summary", + "match": "match", + "!match": "match", + "list.str": "match", + "!list.str": "match", + "event": "event", + "!event": "event", + "murex-package": "murex-package", + "version": "version", + "murex-docs": "murex-docs", + "help": "murex-docs", + "continue": "continue", + "!": "not-func", + "not": "not-func", + "null": "devnull", + "open": "open", + "os": "os", + "sys.os": "os", + "out": "out", + "echo": "out", + "tout": "tout", + "man-get-flags": "man-get-flags", + "trypipe": "trypipe", + "post": "post", + "prepend": "prepend", + "list.prepend": "prepend", + "pretty": "pretty", + "struct-keys": "struct-keys", + "private": "private", + "time": "time", + "function": "function", + "!function": "function", + "escape": "escape", + "!escape": "escape", "murex-update-exe-list": "murex-update-exe-list", - "read": "read", - "tread": "tread", - "format": "format", - "rx": "rx", - "!rx": "rx", - "regexp": "regexp", - "!regexp": "regexp", - "list.regex": "regexp", - "!list.regex": "regexp", - "open-image": "open-image", - "mtac": "mtac", - "list.reverse": "mtac", - "right": "right", - "list.right": "right", - "round": "round", - "num.round": "round", - "signal": "signal", - "summary": "summary", - "!summary": "summary", - "config": "config", - "!config": "config", - "runtime": "runtime", - "builtins": "runtime", - "shell.runtime": "runtime", - "test": "test", - "!test": "test", - "msort": "msort", - "list.sort": "msort", - "jsplit": "jsplit", - "str.split": "jsplit", - "trypipeerr": "trypipeerr", - "tryerr": "tryerr", - "a": "a", - "mkarray": "a", - "switch": "switch", - "autocomplete": "autocomplete", - "tabulate": "tabulate", - "true": "true", - "try": "try", - "die": "die", - "let": "let", - "murex-parser": "murex-parser", + "read": "read", + "tread": "tread", + "format": "format", + "rx": "rx", + "!rx": "rx", + "regexp": "regexp", + "!regexp": "regexp", + "list.regex": "regexp", + "!list.regex": "regexp", + "open-image": "open-image", + "mtac": "mtac", + "list.reverse": "mtac", + "right": "right", + "list.right": "right", + "round": "round", + "num.round": "round", + "signal": "signal", + "summary": "summary", + "!summary": "summary", + "config": "config", + "!config": "config", + "runtime": "runtime", + "builtins": "runtime", + "shell.runtime": "runtime", + "test": "test", + "!test": "test", + "msort": "msort", + "list.sort": "msort", + "jsplit": "jsplit", + "str.split": "jsplit", + "trypipeerr": "trypipeerr", + "tryerr": "tryerr", + "a": "a", + "mkarray": "a", + "switch": "switch", + "autocomplete": "autocomplete", + "tabulate": "tabulate", + "true": "true", + "try": "try", + "die": "die", + "let": "let", + "murex-parser": "murex-parser", - - "select": "optional/select", + "select": "optional/select", "table.select": "optional/select", - "!bz2": "optional/bz2", - "base64": "optional/base64", - "!base64": "optional/base64", - "gz": "optional/gz", - "!gz": "optional/gz", - "qr": "optional/qr", - "sleep": "optional/sleep", + "!bz2": "optional/bz2", + "base64": "optional/base64", + "!base64": "optional/base64", + "gz": "optional/gz", + "!gz": "optional/gz", + "qr": "optional/qr", + "sleep": "optional/sleep", - - "expr-inlined": "parser/expr-inlined", - "c-style-fun": "parser/c-style-fun", - "@[": "parser/range", - "[": "parser/item-index", - "![": "parser/item-index", - "item-index": "parser/item-index", - "index": "parser/item-index", - "[[": "parser/element", - "element": "parser/element", + "expr-inlined": "parser/expr-inlined", + "c-style-fun": "parser/c-style-fun", + "@[": "parser/range", + "[": "parser/item-index", + "![": "parser/item-index", + "item-index": "parser/item-index", + "index": "parser/item-index", + "[[": "parser/element", + "element": "parser/element", "(murex named pipe)": "parser/namedpipe", - "<>": "parser/namedpipe", - "read-named-pipe": "parser/namedpipe", - "": "parser/stdin", - ">": "parser/file-truncate", - "|>": "parser/file-truncate", - "fwrite": "parser/file-truncate", - "double-quote": "parser/double-quote", - "scalar": "parser/scalar", - "brace-quote": "parser/brace-quote", - "create-array": "parser/create-array", - "create-object": "parser/create-object", - "logical-and": "parser/logical-and", - "single-quote": "parser/single-quote", - "(": "parser/brace-quote-func", - "multiply-by": "parser/multiply-by", - "multiplication": "parser/multiplication", - "add-with": "parser/add-with", - "addition": "parser/addition", - "subtract-by": "parser/subtract-by", - "pipe-arrow": "parser/pipe-arrow", - "subtraction": "parser/subtraction", - "divide-by": "parser/divide-by", - "division": "parser/division", - "assign-or-merge": "parser/assign-or-merge", - "pipe-generic": "parser/pipe-generic", - "=": "parser/equ", - ">>": "parser/file-append", - "fappend": "parser/file-append", - "elvis": "parser/elvis", - "null-coalescing": "parser/null-coalescing", - "pipe-err": "parser/pipe-err", - "array": "parser/array", - "lambda": "parser/lambda", - "curly-brace": "parser/curly-brace", - "pipe-posix": "parser/pipe-posix", - "logical-or": "parser/logical-or", - "tilde": "parser/tilde", + "<>": "parser/namedpipe", + "read-named-pipe": "parser/namedpipe", + "": "parser/stdin", + ">": "parser/file-truncate", + "|>": "parser/file-truncate", + "fwrite": "parser/file-truncate", + "double-quote": "parser/double-quote", + "scalar": "parser/scalar", + "brace-quote": "parser/brace-quote", + "create-array": "parser/create-array", + "create-object": "parser/create-object", + "logical-and": "parser/logical-and", + "single-quote": "parser/single-quote", + "(": "parser/brace-quote-func", + "multiply-by": "parser/multiply-by", + "multiplication": "parser/multiplication", + "add-with": "parser/add-with", + "addition": "parser/addition", + "subtract-by": "parser/subtract-by", + "pipe-arrow": "parser/pipe-arrow", + "subtraction": "parser/subtraction", + "divide-by": "parser/divide-by", + "division": "parser/division", + "assign-or-merge": "parser/assign-or-merge", + "pipe-generic": "parser/pipe-generic", + "=": "parser/equ", + ">>": "parser/file-append", + "fappend": "parser/file-append", + "elvis": "parser/elvis", + "null-coalescing": "parser/null-coalescing", + "pipe-err": "parser/pipe-err", + "array": "parser/array", + "lambda": "parser/lambda", + "curly-brace": "parser/curly-brace", + "pipe-posix": "parser/pipe-posix", + "logical-or": "parser/logical-or", + "tilde": "parser/tilde", - - "commands/key-code": "commands/key-code", - "commands/addheading": "commands/addheading", - "commands/prefix": "commands/prefix", - "commands/list.prefix": "commands/prefix", - "commands/suffix": "commands/suffix", - "commands/list.suffix": "commands/suffix", - "commands/alias": "commands/alias", - "commands/!alias": "commands/alias", - "commands/alter": "commands/alter", - "commands/~>": "commands/alter", - "commands/append": "commands/append", - "commands/list.append": "commands/append", - "commands/bg": "commands/bg", - "commands/cpuarch": "commands/cpuarch", - "commands/sys.cpu.arch": "commands/cpuarch", - "commands/cpucount": "commands/cpucount", - "commands/sys.cpu.count": "commands/cpucount", - "commands/catch": "commands/catch", - "commands/!catch": "commands/catch", - "commands/cd": "commands/cd", - "commands/list.case": "commands/list.case", - "commands/bexists": "commands/bexists", - "commands/history": "commands/history", - "commands/count": "commands/count", - "commands/len": "commands/count", - "commands/2darray": "commands/2darray", - "commands/ja": "commands/ja", - "commands/map": "commands/map", - "commands/pipe": "commands/pipe", - "commands/!pipe": "commands/pipe", - "commands/ta": "commands/ta", - "commands/tmp": "commands/tmp", - "commands/datetime": "commands/datetime", - "commands/str.datetime": "commands/datetime", - "commands/debug": "commands/debug", - "commands/export": "commands/export", - "commands/!export": "commands/export", - "commands/unset": "commands/export", - "commands/var.env": "commands/export", - "commands/!var.env": "commands/export", - "commands/args": "commands/args", - "commands/global": "commands/global", - "commands/!global": "commands/global", - "commands/openagent": "commands/openagent", - "commands/!openagent": "commands/openagent", - "commands/method": "commands/method", - "commands/cast": "commands/cast", - "commands/set": "commands/set", - "commands/!set": "commands/set", - "commands/unsafe": "commands/unsafe", - "commands/type": "commands/type", - "commands/fid-list": "commands/fid-list", - "commands/jobs": "commands/fid-list", - "commands/getfile": "commands/getfile", - "commands/err": "commands/err", - "commands/esccli": "commands/esccli", - "commands/eschtml": "commands/eschtml", - "commands/!eschtml": "commands/eschtml", - "commands/escurl": "commands/escurl", - "commands/!escurl": "commands/escurl", - "commands/exec": "commands/exec", - "commands/command": "commands/exec", - "commands/exec.file": "commands/exec", - "commands/fexec": "commands/fexec", - "commands/builtin": "commands/fexec", - "commands/exec.builtin": "commands/fexec", - "commands/exec.function": "commands/fexec", - "commands/exec.private": "commands/fexec", - "commands/break": "commands/break", - "commands/return": "commands/return", - "commands/exit": "commands/exit", - "commands/expr": "commands/expr", - "commands/false": "commands/false", - "commands/foreach": "commands/foreach", - "commands/formap": "commands/formap", - "commands/for": "commands/for", - "commands/fg": "commands/fg", - "commands/runmode": "commands/runmode", - "commands/rand": "commands/rand", - "commands/get-type": "commands/get-type", - "commands/exitnum": "commands/exitnum", - "commands/pt": "commands/pt", - "commands/get": "commands/get", - "commands/g": "commands/g", - "commands/!g": "commands/g", - "commands/if": "commands/if", - "commands/!if": "commands/if", - "commands/source": "commands/source", - "commands/.": "commands/source", - "commands/is-null": "commands/is-null", - "commands/mjoin": "commands/mjoin", - "commands/list.join": "commands/mjoin", - "commands/fid-killall": "commands/fid-killall", - "commands/fid-kill": "commands/fid-kill", - "commands/left": "commands/left", - "commands/list.left": "commands/left", - "commands/f": "commands/f", - "commands/which": "commands/which", - "commands/lockfile": "commands/lockfile", - "commands/and": "commands/and", - "commands/!and": "commands/and", - "commands/or": "commands/or", - "commands/!or": "commands/or", - "commands/while": "commands/while", - "commands/!while": "commands/while", - "commands/man-summary": "commands/man-summary", - "commands/help.man.summary": "commands/man-summary", - "commands/match": "commands/match", - "commands/!match": "commands/match", - "commands/list.str": "commands/match", - "commands/!list.str": "commands/match", - "commands/event": "commands/event", - "commands/!event": "commands/event", - "commands/murex-package": "commands/murex-package", - "commands/version": "commands/version", - "commands/murex-docs": "commands/murex-docs", - "commands/help": "commands/murex-docs", - "commands/continue": "commands/continue", - "commands/!": "commands/not-func", - "commands/not": "commands/not-func", - "commands/null": "commands/devnull", - "commands/open": "commands/open", - "commands/os": "commands/os", - "commands/sys.os": "commands/os", - "commands/out": "commands/out", - "commands/echo": "commands/out", - "commands/tout": "commands/tout", - "commands/man-get-flags": "commands/man-get-flags", - "commands/trypipe": "commands/trypipe", - "commands/post": "commands/post", - "commands/prepend": "commands/prepend", - "commands/list.prepend": "commands/prepend", - "commands/pretty": "commands/pretty", - "commands/struct-keys": "commands/struct-keys", - "commands/private": "commands/private", - "commands/time": "commands/time", - "commands/function": "commands/function", - "commands/!function": "commands/function", - "commands/escape": "commands/escape", - "commands/!escape": "commands/escape", - "commands/murex-update-exe-list": "commands/murex-update-exe-list", - "commands/read": "commands/read", - "commands/tread": "commands/tread", - "commands/format": "commands/format", - "commands/rx": "commands/rx", - "commands/!rx": "commands/rx", - "commands/regexp": "commands/regexp", - "commands/!regexp": "commands/regexp", - "commands/list.regex": "commands/regexp", - "commands/!list.regex": "commands/regexp", - "commands/open-image": "commands/open-image", - "commands/mtac": "commands/mtac", - "commands/list.reverse": "commands/mtac", - "commands/right": "commands/right", - "commands/list.right": "commands/right", - "commands/round": "commands/round", - "commands/num.round": "commands/round", - "commands/signal": "commands/signal", - "commands/summary": "commands/summary", - "commands/!summary": "commands/summary", - "commands/config": "commands/config", - "commands/!config": "commands/config", - "commands/runtime": "commands/runtime", - "commands/builtins": "commands/runtime", - "commands/shell.runtime": "commands/runtime", - "commands/test": "commands/test", - "commands/!test": "commands/test", - "commands/msort": "commands/msort", - "commands/list.sort": "commands/msort", - "commands/jsplit": "commands/jsplit", - "commands/str.split": "commands/jsplit", - "commands/trypipeerr": "commands/trypipeerr", - "commands/tryerr": "commands/tryerr", - "commands/a": "commands/a", - "commands/mkarray": "commands/a", - "commands/switch": "commands/switch", - "commands/autocomplete": "commands/autocomplete", - "commands/tabulate": "commands/tabulate", - "commands/true": "commands/true", - "commands/try": "commands/try", - "commands/die": "commands/die", - "commands/let": "commands/let", - "commands/murex-parser": "commands/murex-parser", - "mkarray/date": "mkarray/date", - "mkarray/character": "mkarray/character", - "mkarray/decimal": "mkarray/decimal", - "mkarray/non-decimal": "mkarray/non-decimal", - "mkarray/special": "mkarray/special", - "optional/select": "optional/select", - "optional/table.select": "optional/select", - "optional/!bz2": "optional/bz2", - "optional/base64": "optional/base64", - "optional/!base64": "optional/base64", - "optional/gz": "optional/gz", - "optional/!gz": "optional/gz", - "optional/qr": "optional/qr", - "optional/sleep": "optional/sleep", - "parser/expr-inlined": "parser/expr-inlined", - "parser/c-style-fun": "parser/c-style-fun", - "parser/@[": "parser/range", - "parser/[": "parser/item-index", - "parser/![": "parser/item-index", - "parser/item-index": "parser/item-index", - "parser/index": "parser/item-index", - "parser/[[": "parser/element", - "parser/element": "parser/element", - "parser/(murex named pipe)": "parser/namedpipe", - "parser/<>": "parser/namedpipe", - "parser/read-named-pipe": "parser/namedpipe", - "parser/": "parser/stdin", - "parser/>": "parser/file-truncate", - "parser/|>": "parser/file-truncate", - "parser/fwrite": "parser/file-truncate", - "parser/double-quote": "parser/double-quote", - "parser/scalar": "parser/scalar", - "parser/brace-quote": "parser/brace-quote", - "parser/create-array": "parser/create-array", - "parser/create-object": "parser/create-object", - "parser/logical-and": "parser/logical-and", - "parser/single-quote": "parser/single-quote", - "parser/(": "parser/brace-quote-func", - "parser/multiply-by": "parser/multiply-by", - "parser/multiplication": "parser/multiplication", - "parser/add-with": "parser/add-with", - "parser/addition": "parser/addition", - "parser/subtract-by": "parser/subtract-by", - "parser/pipe-arrow": "parser/pipe-arrow", - "parser/subtraction": "parser/subtraction", - "parser/divide-by": "parser/divide-by", - "parser/division": "parser/division", - "parser/assign-or-merge": "parser/assign-or-merge", - "parser/pipe-generic": "parser/pipe-generic", - "parser/=": "parser/equ", - "parser/>>": "parser/file-append", - "parser/fappend": "parser/file-append", - "parser/elvis": "parser/elvis", - "parser/null-coalescing": "parser/null-coalescing", - "parser/pipe-err": "parser/pipe-err", - "parser/array": "parser/array", - "parser/lambda": "parser/lambda", - "parser/curly-brace": "parser/curly-brace", - "parser/pipe-posix": "parser/pipe-posix", - "parser/logical-or": "parser/logical-or", - "parser/tilde": "parser/tilde", - "events/oncommandcompletion": "events/oncommandcompletion", - "events/onCommandCompletion": "events/oncommandcompletion", - "events/onfilesystemchange": "events/onfilesystemchange", - "events/onFileSystemChange": "events/onfilesystemchange", - "events/onkeypress": "events/onkeypress", - "events/onKeyPress": "events/onkeypress", - "events/onpreview": "events/onpreview", - "events/onPreview": "events/onpreview", - "events/onprompt": "events/onprompt", - "events/onPrompt": "events/onprompt", - "events/onsecondselapsed": "events/onsecondselapsed", - "events/onSecondsElapsed": "events/onsecondselapsed", - "events/onsignalreceived": "events/onsignalreceived", - "events/onSignalReceived": "events/onsignalreceived", - "types/generic": "types/generic", - "types/*": "types/generic", - "types/bool": "types/bool", - "types/commonlog": "types/commonlog", - "types/csv": "types/csv", - "types/float": "types/float", - "types/hcl": "types/hcl", - "types/int": "types/int", - "types/json": "types/json", - "types/jsonc": "types/jsonc", - "types/jsonl": "types/jsonl", - "types/num": "types/num", - "types/path": "types/path", - "types/paths": "types/paths", - "types/str": "types/str", - "types/string": "types/str", - "types/toml": "types/toml", - "types/yaml": "types/yaml", - "types/mxjson": "types/mxjson", - "variables/numeric": "variables/numeric", - "variables/meta-values": "variables/meta-values", - "variables/$.": "variables/meta-values", - "variables/argv": "variables/argv", - "variables/ARGV": "variables/argv", - "variables/ARGS": "variables/argv", - "variables/columns": "variables/columns", - "variables/COLUMNS": "variables/columns", - "variables/event_return": "variables/event_return", - "variables/EVENT_RETURN": "variables/event_return", - "variables/home": "variables/home", - "variables/HOME": "variables/home", - "variables/hostname": "variables/hostname", - "variables/HOSTNAME": "variables/hostname", - "variables/lines": "variables/lines", - "variables/LINES": "variables/lines", - "variables/logname": "variables/logname", - "variables/LOGNAME": "variables/logname", - "variables/murex_argv": "variables/murex_argv", - "variables/MUREX_ARGV": "variables/murex_argv", - "variables/MUREX_ARGS": "variables/murex_argv", - "variables/murex_exe": "variables/murex_exe", - "variables/MUREX_EXE": "variables/murex_exe", - "variables/oldpwd": "variables/oldpwd", - "variables/OLDPWD": "variables/oldpwd", - "variables/params": "variables/params", - "variables/PARAMS": "variables/params", - "variables/pwdhist": "variables/pwdhist", - "variables/PWDHIST": "variables/pwdhist", - "variables/pwd": "variables/pwd", - "variables/PWD": "variables/pwd", - "variables/random": "variables/random", - "variables/RANDOM": "variables/random", - "variables/self": "variables/self", - "variables/SELF": "variables/self", - "variables/shell": "variables/shell", - "variables/SHELL": "variables/shell", - "variables/tmpdir": "variables/tmpdir", - "variables/TMPDIR": "variables/tmpdir", - "variables/user": "variables/user", - "variables/USER": "variables/user", - "apis/Marshal": "apis/Marshal", - "apis/ReadArray": "apis/ReadArray", - "apis/ReadArrayWithType": "apis/ReadArrayWithType", - "apis/ReadIndex": "apis/ReadIndex", - "apis/ReadMap": "apis/ReadMap", - "apis/ReadNotIndex": "apis/ReadNotIndex", - "apis/Unmarshal": "apis/Unmarshal", - "apis/WriteArray": "apis/WriteArray", - "apis/lang.ArrayTemplate": "apis/lang.ArrayTemplate", + "commands/key-code": "commands/key-code", + "commands/addheading": "commands/addheading", + "commands/prefix": "commands/prefix", + "commands/list.prefix": "commands/prefix", + "commands/suffix": "commands/suffix", + "commands/list.suffix": "commands/suffix", + "commands/alias": "commands/alias", + "commands/!alias": "commands/alias", + "commands/alter": "commands/alter", + "commands/~>": "commands/alter", + "commands/append": "commands/append", + "commands/list.append": "commands/append", + "commands/bg": "commands/bg", + "commands/cpuarch": "commands/cpuarch", + "commands/sys.cpu.arch": "commands/cpuarch", + "commands/cpucount": "commands/cpucount", + "commands/sys.cpu.count": "commands/cpucount", + "commands/catch": "commands/catch", + "commands/!catch": "commands/catch", + "commands/cd": "commands/cd", + "commands/list.case": "commands/list.case", + "commands/bexists": "commands/bexists", + "commands/history": "commands/history", + "commands/count": "commands/count", + "commands/len": "commands/count", + "commands/2darray": "commands/2darray", + "commands/ja": "commands/ja", + "commands/map": "commands/map", + "commands/pipe": "commands/pipe", + "commands/!pipe": "commands/pipe", + "commands/ta": "commands/ta", + "commands/tmp": "commands/tmp", + "commands/datetime": "commands/datetime", + "commands/str.datetime": "commands/datetime", + "commands/debug": "commands/debug", + "commands/export": "commands/export", + "commands/!export": "commands/export", + "commands/unset": "commands/export", + "commands/var.env": "commands/export", + "commands/!var.env": "commands/export", + "commands/args": "commands/args", + "commands/global": "commands/global", + "commands/!global": "commands/global", + "commands/openagent": "commands/openagent", + "commands/!openagent": "commands/openagent", + "commands/method": "commands/method", + "commands/cast": "commands/cast", + "commands/set": "commands/set", + "commands/!set": "commands/set", + "commands/unsafe": "commands/unsafe", + "commands/type": "commands/type", + "commands/fid-list": "commands/fid-list", + "commands/jobs": "commands/fid-list", + "commands/getfile": "commands/getfile", + "commands/err": "commands/err", + "commands/esccli": "commands/esccli", + "commands/eschtml": "commands/eschtml", + "commands/!eschtml": "commands/eschtml", + "commands/escurl": "commands/escurl", + "commands/!escurl": "commands/escurl", + "commands/exec": "commands/exec", + "commands/command": "commands/exec", + "commands/exec.file": "commands/exec", + "commands/fexec": "commands/fexec", + "commands/builtin": "commands/fexec", + "commands/exec.builtin": "commands/fexec", + "commands/exec.function": "commands/fexec", + "commands/exec.private": "commands/fexec", + "commands/break": "commands/break", + "commands/return": "commands/return", + "commands/exit": "commands/exit", + "commands/expr": "commands/expr", + "commands/false": "commands/false", + "commands/foreach": "commands/foreach", + "commands/formap": "commands/formap", + "commands/for": "commands/for", + "commands/fg": "commands/fg", + "commands/runmode": "commands/runmode", + "commands/rand": "commands/rand", + "commands/get-type": "commands/get-type", + "commands/exitnum": "commands/exitnum", + "commands/pt": "commands/pt", + "commands/get": "commands/get", + "commands/g": "commands/g", + "commands/!g": "commands/g", + "commands/if": "commands/if", + "commands/!if": "commands/if", + "commands/source": "commands/source", + "commands/.": "commands/source", + "commands/is-null": "commands/is-null", + "commands/mjoin": "commands/mjoin", + "commands/list.join": "commands/mjoin", + "commands/fid-killall": "commands/fid-killall", + "commands/fid-kill": "commands/fid-kill", + "commands/left": "commands/left", + "commands/list.left": "commands/left", + "commands/f": "commands/f", + "commands/which": "commands/which", + "commands/lockfile": "commands/lockfile", + "commands/and": "commands/and", + "commands/!and": "commands/and", + "commands/or": "commands/or", + "commands/!or": "commands/or", + "commands/while": "commands/while", + "commands/!while": "commands/while", + "commands/man-summary": "commands/man-summary", + "commands/help.man.summary": "commands/man-summary", + "commands/match": "commands/match", + "commands/!match": "commands/match", + "commands/list.str": "commands/match", + "commands/!list.str": "commands/match", + "commands/event": "commands/event", + "commands/!event": "commands/event", + "commands/murex-package": "commands/murex-package", + "commands/version": "commands/version", + "commands/murex-docs": "commands/murex-docs", + "commands/help": "commands/murex-docs", + "commands/continue": "commands/continue", + "commands/!": "commands/not-func", + "commands/not": "commands/not-func", + "commands/null": "commands/devnull", + "commands/open": "commands/open", + "commands/os": "commands/os", + "commands/sys.os": "commands/os", + "commands/out": "commands/out", + "commands/echo": "commands/out", + "commands/tout": "commands/tout", + "commands/man-get-flags": "commands/man-get-flags", + "commands/trypipe": "commands/trypipe", + "commands/post": "commands/post", + "commands/prepend": "commands/prepend", + "commands/list.prepend": "commands/prepend", + "commands/pretty": "commands/pretty", + "commands/struct-keys": "commands/struct-keys", + "commands/private": "commands/private", + "commands/time": "commands/time", + "commands/function": "commands/function", + "commands/!function": "commands/function", + "commands/escape": "commands/escape", + "commands/!escape": "commands/escape", + "commands/murex-update-exe-list": "commands/murex-update-exe-list", + "commands/read": "commands/read", + "commands/tread": "commands/tread", + "commands/format": "commands/format", + "commands/rx": "commands/rx", + "commands/!rx": "commands/rx", + "commands/regexp": "commands/regexp", + "commands/!regexp": "commands/regexp", + "commands/list.regex": "commands/regexp", + "commands/!list.regex": "commands/regexp", + "commands/open-image": "commands/open-image", + "commands/mtac": "commands/mtac", + "commands/list.reverse": "commands/mtac", + "commands/right": "commands/right", + "commands/list.right": "commands/right", + "commands/round": "commands/round", + "commands/num.round": "commands/round", + "commands/signal": "commands/signal", + "commands/summary": "commands/summary", + "commands/!summary": "commands/summary", + "commands/config": "commands/config", + "commands/!config": "commands/config", + "commands/runtime": "commands/runtime", + "commands/builtins": "commands/runtime", + "commands/shell.runtime": "commands/runtime", + "commands/test": "commands/test", + "commands/!test": "commands/test", + "commands/msort": "commands/msort", + "commands/list.sort": "commands/msort", + "commands/jsplit": "commands/jsplit", + "commands/str.split": "commands/jsplit", + "commands/trypipeerr": "commands/trypipeerr", + "commands/tryerr": "commands/tryerr", + "commands/a": "commands/a", + "commands/mkarray": "commands/a", + "commands/switch": "commands/switch", + "commands/autocomplete": "commands/autocomplete", + "commands/tabulate": "commands/tabulate", + "commands/true": "commands/true", + "commands/try": "commands/try", + "commands/die": "commands/die", + "commands/let": "commands/let", + "commands/murex-parser": "commands/murex-parser", + "mkarray/date": "mkarray/date", + "mkarray/character": "mkarray/character", + "mkarray/decimal": "mkarray/decimal", + "mkarray/non-decimal": "mkarray/non-decimal", + "mkarray/special": "mkarray/special", + "optional/select": "optional/select", + "optional/table.select": "optional/select", + "optional/!bz2": "optional/bz2", + "optional/base64": "optional/base64", + "optional/!base64": "optional/base64", + "optional/gz": "optional/gz", + "optional/!gz": "optional/gz", + "optional/qr": "optional/qr", + "optional/sleep": "optional/sleep", + "parser/expr-inlined": "parser/expr-inlined", + "parser/c-style-fun": "parser/c-style-fun", + "parser/@[": "parser/range", + "parser/[": "parser/item-index", + "parser/![": "parser/item-index", + "parser/item-index": "parser/item-index", + "parser/index": "parser/item-index", + "parser/[[": "parser/element", + "parser/element": "parser/element", + "parser/(murex named pipe)": "parser/namedpipe", + "parser/<>": "parser/namedpipe", + "parser/read-named-pipe": "parser/namedpipe", + "parser/": "parser/stdin", + "parser/>": "parser/file-truncate", + "parser/|>": "parser/file-truncate", + "parser/fwrite": "parser/file-truncate", + "parser/double-quote": "parser/double-quote", + "parser/scalar": "parser/scalar", + "parser/brace-quote": "parser/brace-quote", + "parser/create-array": "parser/create-array", + "parser/create-object": "parser/create-object", + "parser/logical-and": "parser/logical-and", + "parser/single-quote": "parser/single-quote", + "parser/(": "parser/brace-quote-func", + "parser/multiply-by": "parser/multiply-by", + "parser/multiplication": "parser/multiplication", + "parser/add-with": "parser/add-with", + "parser/addition": "parser/addition", + "parser/subtract-by": "parser/subtract-by", + "parser/pipe-arrow": "parser/pipe-arrow", + "parser/subtraction": "parser/subtraction", + "parser/divide-by": "parser/divide-by", + "parser/division": "parser/division", + "parser/assign-or-merge": "parser/assign-or-merge", + "parser/pipe-generic": "parser/pipe-generic", + "parser/=": "parser/equ", + "parser/>>": "parser/file-append", + "parser/fappend": "parser/file-append", + "parser/elvis": "parser/elvis", + "parser/null-coalescing": "parser/null-coalescing", + "parser/pipe-err": "parser/pipe-err", + "parser/array": "parser/array", + "parser/lambda": "parser/lambda", + "parser/curly-brace": "parser/curly-brace", + "parser/pipe-posix": "parser/pipe-posix", + "parser/logical-or": "parser/logical-or", + "parser/tilde": "parser/tilde", + "events/oncommandcompletion": "events/oncommandcompletion", + "events/onCommandCompletion": "events/oncommandcompletion", + "events/onfilesystemchange": "events/onfilesystemchange", + "events/onFileSystemChange": "events/onfilesystemchange", + "events/onkeypress": "events/onkeypress", + "events/onKeyPress": "events/onkeypress", + "events/onpreview": "events/onpreview", + "events/onPreview": "events/onpreview", + "events/onprompt": "events/onprompt", + "events/onPrompt": "events/onprompt", + "events/onsecondselapsed": "events/onsecondselapsed", + "events/onSecondsElapsed": "events/onsecondselapsed", + "events/onsignalreceived": "events/onsignalreceived", + "events/onSignalReceived": "events/onsignalreceived", + "types/generic": "types/generic", + "types/*": "types/generic", + "types/bool": "types/bool", + "types/commonlog": "types/commonlog", + "types/csv": "types/csv", + "types/float": "types/float", + "types/hcl": "types/hcl", + "types/int": "types/int", + "types/json": "types/json", + "types/jsonc": "types/jsonc", + "types/jsonl": "types/jsonl", + "types/num": "types/num", + "types/path": "types/path", + "types/paths": "types/paths", + "types/str": "types/str", + "types/string": "types/str", + "types/toml": "types/toml", + "types/yaml": "types/yaml", + "types/mxjson": "types/mxjson", + "variables/numeric": "variables/numeric", + "variables/meta-values": "variables/meta-values", + "variables/$.": "variables/meta-values", + "variables/argv": "variables/argv", + "variables/ARGV": "variables/argv", + "variables/ARGS": "variables/argv", + "variables/columns": "variables/columns", + "variables/COLUMNS": "variables/columns", + "variables/event_return": "variables/event_return", + "variables/EVENT_RETURN": "variables/event_return", + "variables/home": "variables/home", + "variables/HOME": "variables/home", + "variables/hostname": "variables/hostname", + "variables/HOSTNAME": "variables/hostname", + "variables/lines": "variables/lines", + "variables/LINES": "variables/lines", + "variables/logname": "variables/logname", + "variables/LOGNAME": "variables/logname", + "variables/murex_argv": "variables/murex_argv", + "variables/MUREX_ARGV": "variables/murex_argv", + "variables/MUREX_ARGS": "variables/murex_argv", + "variables/murex_exe": "variables/murex_exe", + "variables/MUREX_EXE": "variables/murex_exe", + "variables/oldpwd": "variables/oldpwd", + "variables/OLDPWD": "variables/oldpwd", + "variables/params": "variables/params", + "variables/PARAMS": "variables/params", + "variables/pwdhist": "variables/pwdhist", + "variables/PWDHIST": "variables/pwdhist", + "variables/pwd": "variables/pwd", + "variables/PWD": "variables/pwd", + "variables/random": "variables/random", + "variables/RANDOM": "variables/random", + "variables/self": "variables/self", + "variables/SELF": "variables/self", + "variables/shell": "variables/shell", + "variables/SHELL": "variables/shell", + "variables/tmpdir": "variables/tmpdir", + "variables/TMPDIR": "variables/tmpdir", + "variables/user": "variables/user", + "variables/USER": "variables/user", + "apis/Marshal": "apis/Marshal", + "apis/ReadArray": "apis/ReadArray", + "apis/ReadArrayWithType": "apis/ReadArrayWithType", + "apis/ReadIndex": "apis/ReadIndex", + "apis/ReadMap": "apis/ReadMap", + "apis/ReadNotIndex": "apis/ReadNotIndex", + "apis/Unmarshal": "apis/Unmarshal", + "apis/WriteArray": "apis/WriteArray", + "apis/lang.ArrayTemplate": "apis/lang.ArrayTemplate", "apis/lang.ArrayWithTypeTemplate": "apis/lang.ArrayWithTypeTemplate", - "apis/lang.IndexTemplateObject": "apis/lang.IndexTemplateObject", - "apis/lang.IndexTemplateTable": "apis/lang.IndexTemplateTable", - "apis/lang.MarshalData": "apis/lang.MarshalData", - "apis/lang.UnmarshalData": "apis/lang.UnmarshalData", - "user-guide/ansi": "user-guide/ansi", - "user-guide/bang-prefix": "user-guide/bang-prefix", - "user-guide/bang": "user-guide/bang-prefix", - "user-guide/code-block": "user-guide/code-block", - "user-guide/fileref": "user-guide/fileref", - "user-guide/FileRef": "user-guide/fileref", - "user-guide/hint-text": "user-guide/hint-text", - "user-guide/integrations": "user-guide/integrations", - "user-guide/interactive-shell": "user-guide/interactive-shell", - "user-guide/repl": "user-guide/interactive-shell", - "user-guide/readline": "user-guide/interactive-shell", - "user-guide/job-control": "user-guide/job-control", - "user-guide/module": "user-guide/modules", - "user-guide/modules": "user-guide/modules", - "user-guide/package": "user-guide/modules", - "user-guide/packages": "user-guide/modules", - "user-guide/namedpipes": "user-guide/namedpipes", + "apis/lang.IndexTemplateObject": "apis/lang.IndexTemplateObject", + "apis/lang.IndexTemplateTable": "apis/lang.IndexTemplateTable", + "apis/lang.MarshalData": "apis/lang.MarshalData", + "apis/lang.UnmarshalData": "apis/lang.UnmarshalData", + "user-guide/ansi": "user-guide/ansi", + "user-guide/bang-prefix": "user-guide/bang-prefix", + "user-guide/bang": "user-guide/bang-prefix", + "user-guide/code-block": "user-guide/code-block", + "user-guide/fileref": "user-guide/fileref", + "user-guide/FileRef": "user-guide/fileref", + "user-guide/hint-text": "user-guide/hint-text", + "user-guide/integrations": "user-guide/integrations", + "user-guide/interactive-shell": "user-guide/interactive-shell", + "user-guide/repl": "user-guide/interactive-shell", + "user-guide/readline": "user-guide/interactive-shell", + "user-guide/job-control": "user-guide/job-control", + "user-guide/module": "user-guide/modules", + "user-guide/modules": "user-guide/modules", + "user-guide/package": "user-guide/modules", + "user-guide/packages": "user-guide/modules", + "user-guide/namedpipes": "user-guide/namedpipes", "user-guide/operators-and-tokens": "user-guide/operators-and-tokens", - "user-guide/pipeline": "user-guide/pipeline", - "user-guide/profile": "user-guide/profile", - "user-guide/.murex_profile": "user-guide/profile", - "user-guide/murex_profile": "user-guide/profile", - "user-guide/.murex_preload": "user-guide/profile", - "user-guide/murex_preload": "user-guide/profile", - "user-guide/reserved-vars": "user-guide/reserved-vars", - "user-guide/reserved": "user-guide/reserved-vars", - "user-guide/reserved-variables": "user-guide/reserved-vars", - "user-guide/special-vars": "user-guide/reserved-vars", - "user-guide/special-variables": "user-guide/reserved-vars", - "user-guide/rosetta-stone": "user-guide/rosetta-stone", - "user-guide/schedulers": "user-guide/schedulers", - "user-guide/strict-types": "user-guide/strict-types", - "user-guide/hotkeys": "user-guide/terminal-keys", - "user-guide/scoping": "user-guide/scoping", - "user-guide/murex-arrays": "user-guide/murex-arrays", - "integrations/chatgpt": "integrations/chatgpt", - "integrations/ChatGPT": "integrations/chatgpt", - "integrations/cheatsh": "integrations/cheatsh", - "integrations/cheat.sh": "integrations/cheatsh", - "integrations/kitty": "integrations/kitty", - "integrations/make": "integrations/make", - "integrations/Makefile": "integrations/make", - "integrations/man-pages": "integrations/man-pages", - "integrations/spellcheck": "integrations/spellcheck", - "integrations/terminology": "integrations/terminology", - "integrations/direnv": "integrations/direnv", - "integrations/yarn": "integrations/yarn", - "integrations/package.json": "integrations/yarn", - "integrations/iterm2": "integrations/iterm2", - "changelog/v2.0": "changelog/v2.0", - "changelog/v2.1": "changelog/v2.1", - "changelog/v2.10": "changelog/v2.10", - "changelog/v2.11": "changelog/v2.11", - "changelog/v2.2": "changelog/v2.2", - "changelog/v2.3": "changelog/v2.3", - "changelog/v2.4": "changelog/v2.4", - "changelog/v2.5": "changelog/v2.5", - "changelog/v2.6": "changelog/v2.6", - "changelog/v2.7": "changelog/v2.7", - "changelog/v2.8": "changelog/v2.8", - "changelog/v2.9": "changelog/v2.9", - "changelog/v3.0": "changelog/v3.0", - "changelog/v3.1": "changelog/v3.1", - "changelog/v4.0": "changelog/v4.0", - "changelog/v4.1": "changelog/v4.1", - "changelog/v4.2": "changelog/v4.2", - "changelog/v4.3": "changelog/v4.3", - "changelog/v4.4": "changelog/v4.4", - "changelog/v5.0": "changelog/v5.0", - "changelog/v5.1": "changelog/v5.1", - "changelog/v5.2": "changelog/v5.2", - "changelog/v5.3": "changelog/v5.3", - "changelog/v6.0": "changelog/v6.0", - "changelog/v6.1": "changelog/v6.1", - "changelog/v6.2": "changelog/v6.2", - "changelog/v6.3": "changelog/v6.3", - + "user-guide/pipeline": "user-guide/pipeline", + "user-guide/profile": "user-guide/profile", + "user-guide/.murex_profile": "user-guide/profile", + "user-guide/murex_profile": "user-guide/profile", + "user-guide/.murex_preload": "user-guide/profile", + "user-guide/murex_preload": "user-guide/profile", + "user-guide/reserved-vars": "user-guide/reserved-vars", + "user-guide/reserved": "user-guide/reserved-vars", + "user-guide/reserved-variables": "user-guide/reserved-vars", + "user-guide/special-vars": "user-guide/reserved-vars", + "user-guide/special-variables": "user-guide/reserved-vars", + "user-guide/rosetta-stone": "user-guide/rosetta-stone", + "user-guide/schedulers": "user-guide/schedulers", + "user-guide/strict-types": "user-guide/strict-types", + "user-guide/hotkeys": "user-guide/terminal-keys", + "user-guide/scoping": "user-guide/scoping", + "user-guide/murex-arrays": "user-guide/murex-arrays", + "integrations/chatgpt": "integrations/chatgpt", + "integrations/ChatGPT": "integrations/chatgpt", + "integrations/cheatsh": "integrations/cheatsh", + "integrations/cheat.sh": "integrations/cheatsh", + "integrations/kitty": "integrations/kitty", + "integrations/make": "integrations/make", + "integrations/Makefile": "integrations/make", + "integrations/man-pages": "integrations/man-pages", + "integrations/spellcheck": "integrations/spellcheck", + "integrations/terminology": "integrations/terminology", + "integrations/direnv": "integrations/direnv", + "integrations/yarn": "integrations/yarn", + "integrations/package.json": "integrations/yarn", + "integrations/iterm2": "integrations/iterm2", + "changelog/v2.0": "changelog/v2.0", + "changelog/v2.1": "changelog/v2.1", + "changelog/v2.10": "changelog/v2.10", + "changelog/v2.11": "changelog/v2.11", + "changelog/v2.2": "changelog/v2.2", + "changelog/v2.3": "changelog/v2.3", + "changelog/v2.4": "changelog/v2.4", + "changelog/v2.5": "changelog/v2.5", + "changelog/v2.6": "changelog/v2.6", + "changelog/v2.7": "changelog/v2.7", + "changelog/v2.8": "changelog/v2.8", + "changelog/v2.9": "changelog/v2.9", + "changelog/v3.0": "changelog/v3.0", + "changelog/v3.1": "changelog/v3.1", + "changelog/v4.0": "changelog/v4.0", + "changelog/v4.1": "changelog/v4.1", + "changelog/v4.2": "changelog/v4.2", + "changelog/v4.3": "changelog/v4.3", + "changelog/v4.4": "changelog/v4.4", + "changelog/v5.0": "changelog/v5.0", + "changelog/v5.1": "changelog/v5.1", + "changelog/v5.2": "changelog/v5.2", + "changelog/v5.3": "changelog/v5.3", + "changelog/v6.0": "changelog/v6.0", + "changelog/v6.1": "changelog/v6.1", + "changelog/v6.2": "changelog/v6.2", + "changelog/v6.3": "changelog/v6.3", } -} \ No newline at end of file +} diff --git a/docs/user-guide/murex-arrays.md b/docs/user-guide/murex-arrays.md index 9222015f0..7141e7f5b 100644 --- a/docs/user-guide/murex-arrays.md +++ b/docs/user-guide/murex-arrays.md @@ -2,7 +2,7 @@ > Examples using arrays within Murex -### Creating Arrays +## Creating Arrays Arrays can be defined with `%[ ... ]`. @@ -107,7 +107,7 @@ Returning to our multi-part array, perhaps you want **fall** instead of ] ``` -### Creating Multi-Dimensional Arrays +### Multi-Dimensional Arrays But what if they were supposed to be nested rather than flattened arrays? Well that's not a problem either. Just make sure your first nested array is also @@ -158,7 +158,7 @@ This is obviously an absurd example because nobody in their right mind would want to ping every valid IPv4 address. But it does demonstrate the advantages of streaming lists rather than creating arrays. -### Accessing Array Values +## Accessing Array Values There are two main ways to access values inside an array: @@ -168,7 +168,7 @@ There are two main ways to access values inside an array: Why two? Because they support different features. -#### Square Brackets (immutable) +### Square Brackets (immutable) With square brackets you can select more than just a single element. For example, if you wanted the first element you can reference it the same way @@ -224,7 +224,7 @@ $my_array[..4] > Ranges are indexed from 1. Yes, I know that's stupid and confusing. -#### As A Function +#### Elements From A Pipeline The square brackets can also be used as a function too. Which means any kind of array or list can be queried from stdin, you don't have to first convert it to @@ -247,19 +247,19 @@ tool (`cloud-api` for our made up purposes here) returns JSON: cloud-api list-containers | :json: [-1] ``` -#### Making Changes +### Making Changes That's all great, but what if I want to make a change to the host array? Well this is where dot notation comes in... -### Dot Notation (mutable) +## Dot Notation (mutable) Dot notation is a lot more limited in what you can do because it's designed for making careful edits of the underlying data structure. So it can only be used with variables. -#### Assignment +### Assignment You can edit an element, for example renaming **Wednesday** to **Humpday**: @@ -280,7 +280,7 @@ You can edit an element, for example renaming **Wednesday** to **Humpday**: > Remember: arrays are zero based -#### Printing +### Printing You can also use dot notation to return a value, just like you would with the square braces solution above. But dot notation doesn't support any special @@ -290,7 +290,7 @@ magic and still only works on variables: $my_array.2 ``` -### Appending Arrays +## Appending Arrays There are several ways to append an array. You can create a copy of that array and append via the pipeline: diff --git a/gen/root/README.inc.md b/gen/root/README.inc.md index e9f79565c..9028d97cb 100644 --- a/gen/root/README.inc.md +++ b/gen/root/README.inc.md @@ -1,6 +1,7 @@ {{ if env "DOCGEN_TARGET=vuepress" }} home: true icon: home +sidebar: true heroImage: murex-logo-shell.svg?v={{ env "COMMITHASHSHORT" }} heroText: Murex.Rocks tagline: An intuitive and content aware shell for a modern command line diff --git a/gen/user-guide/arrays.inc.md b/gen/user-guide/arrays.inc.md index b5a90f887..5b371d8c7 100644 --- a/gen/user-guide/arrays.inc.md +++ b/gen/user-guide/arrays.inc.md @@ -1,4 +1,4 @@ -### Creating Arrays +## Creating Arrays Arrays can be defined with `%[ ... ]`. @@ -103,7 +103,7 @@ Returning to our multi-part array, perhaps you want **fall** instead of ] ``` -### Creating Multi-Dimensional Arrays +### Multi-Dimensional Arrays But what if they were supposed to be nested rather than flattened arrays? Well that's not a problem either. Just make sure your first nested array is also @@ -154,7 +154,7 @@ This is obviously an absurd example because nobody in their right mind would want to ping every valid IPv4 address. But it does demonstrate the advantages of streaming lists rather than creating arrays. -### Accessing Array Values +## Accessing Array Values There are two main ways to access values inside an array: @@ -164,7 +164,7 @@ There are two main ways to access values inside an array: Why two? Because they support different features. -#### Square Brackets (immutable) +### Square Brackets (immutable) With square brackets you can select more than just a single element. For example, if you wanted the first element you can reference it the same way @@ -220,7 +220,7 @@ $my_array[..4] > Ranges are indexed from 1. Yes, I know that's stupid and confusing. -#### As A Function +#### Elements From A Pipeline The square brackets can also be used as a function too. Which means any kind of array or list can be queried from stdin, you don't have to first convert it to @@ -243,19 +243,19 @@ tool (`cloud-api` for our made up purposes here) returns JSON: cloud-api list-containers | :json: [-1] ``` -#### Making Changes +### Making Changes That's all great, but what if I want to make a change to the host array? Well this is where dot notation comes in... -### Dot Notation (mutable) +## Dot Notation (mutable) Dot notation is a lot more limited in what you can do because it's designed for making careful edits of the underlying data structure. So it can only be used with variables. -#### Assignment +### Assignment You can edit an element, for example renaming **Wednesday** to **Humpday**: @@ -276,7 +276,7 @@ You can edit an element, for example renaming **Wednesday** to **Humpday**: > Remember: arrays are zero based -#### Printing +### Printing You can also use dot notation to return a value, just like you would with the square braces solution above. But dot notation doesn't support any special @@ -286,7 +286,7 @@ magic and still only works on variables: $my_array.2 ``` -### Appending Arrays +## Appending Arrays There are several ways to append an array. You can create a copy of that array and append via the pipeline: diff --git a/version.svg b/version.svg index 4e2fba4f5..efeb860b6 100644 --- a/version.svg +++ b/version.svg @@ -1 +1 @@ -Version: 6.3.4234Version6.3.4234 +Version: 6.3.4235Version6.3.4235 From 7946b34088183987a02512a40490466da131685e Mon Sep 17 00:00:00 2001 From: Laurence Morgan Date: Fri, 20 Sep 2024 23:46:34 +0100 Subject: [PATCH 04/28] #851 write file pipeline check --- app/app.go | 10 +++---- builtins/core/io/file.go | 65 ++++++++++++++++++++++++++++++++++++++-- version.svg | 2 +- 3 files changed, 68 insertions(+), 9 deletions(-) diff --git a/app/app.go b/app/app.go index 48dc71d0d..6625c2d9d 100644 --- a/app/app.go +++ b/app/app.go @@ -14,11 +14,11 @@ const Name = "murex" // Version number of $SHELL // Format of version string should be "$(Major).$(Minor).$(Revision) ($Branch)" const ( - Major = 6 - Minor = 3 - Revision = 4225 - Branch = "develop" - BuildDate = "2024-09-18 22:09:49" + Major = 7 + Minor = 0 + Revision = 101 + Branch = "851/file-redirect" + BuildDate = "2024-09-20 23:49:34" ) // Copyright is the copyright owner string diff --git a/builtins/core/io/file.go b/builtins/core/io/file.go index 357ea2195..cffd9c1cf 100644 --- a/builtins/core/io/file.go +++ b/builtins/core/io/file.go @@ -1,14 +1,17 @@ package io import ( + "bytes" "fmt" "io" "os" "time" "github.com/lmorg/murex/lang" + "github.com/lmorg/murex/lang/state" "github.com/lmorg/murex/lang/types" "github.com/lmorg/murex/utils/humannumbers" + "github.com/lmorg/murex/utils/lists" ) func init() { @@ -55,22 +58,78 @@ func cmdPipeTelemetry(p *lang.Process) error { return err } +const ( + _WAIT_EOF_LONG = "--wait-for-eof" + _WAIT_EOF_SHORT = "-w" + _DONT_CHECK_PIPELINE = "--ignore-pipeline-check" +) + func cmdWriteFile(p *lang.Process) error { p.Stdout.SetDataType(types.Null) - name, err := p.Parameters.String(0) + filename, err := p.Parameters.String(0) if err != nil { return err } - file, err := os.Create(name) + if filename == _DONT_CHECK_PIPELINE { + filename, err = p.Parameters.String(1) + if err != nil { + return err + } + return writeFile(p.Stdin, filename) + } + + wait := filename == _WAIT_EOF_SHORT || filename == _WAIT_EOF_LONG + + if wait { + filename, err = p.Parameters.String(1) + if err != nil { + return err + } + } else { + wait = isFileOpen(p, filename) + } + + if wait { + _, _ = p.Stderr.Writeln([]byte(fmt.Sprintf("warning: '%s' appears as a parameter elsewhere in the pipeline so I'm going to cache the file in RAM before writing to disk.\n : this message can be suppressed using `%s` or `%s`.", filename, _WAIT_EOF_LONG, _DONT_CHECK_PIPELINE))) + } else { + return writeFile(p.Stdin, filename) + } + + b, err := p.Stdin.ReadAll() + if err != nil { + return err + } + + return writeFile(bytes.NewReader(b), filename) +} + +func isFileOpen(p *lang.Process, filename string) bool { + p = p.Previous + for { + if p.State.Get() < state.Executing { + continue + } + if lists.Match(p.Parameters.StringArray(), filename) { + return true + } + if !p.IsMethod || p.Id == lang.ShellProcess.Id { + return false + } + p = p.Previous + } +} + +func writeFile(reader io.Reader, filename string) error { + file, err := os.Create(filename) if err != nil { return err } defer file.Close() - _, err = io.Copy(file, p.Stdin) + _, err = io.Copy(file, reader) return err } diff --git a/version.svg b/version.svg index 6ece2a9fa..c5033a256 100644 --- a/version.svg +++ b/version.svg @@ -1 +1 @@ -Version: 6.3.4225Version6.3.4225 +Version: 7.0.0101Version7.0.0101 From 266111e0b8b2f52a6bc7a301b12e1785e670e361 Mon Sep 17 00:00:00 2001 From: Laurence Morgan Date: Sat, 21 Sep 2024 00:06:50 +0100 Subject: [PATCH 05/28] #851 add -w support to append file --- app/app.go | 4 +- builtins/core/io/pt.go | 52 ++++++++++++++ .../core/io/{file_doc.yaml => pt_doc.yaml} | 37 ---------- builtins/core/io/pt_test.go | 18 +++++ builtins/core/io/{file.go => write.go} | 69 ++++--------------- builtins/core/io/write_doc.yaml | 34 +++++++++ .../core/io/{file_test.go => write_test.go} | 11 --- docs/commands/pt.md | 2 +- docs/parser/file-truncate.md | 2 +- version.svg | 2 +- 10 files changed, 122 insertions(+), 109 deletions(-) create mode 100644 builtins/core/io/pt.go rename builtins/core/io/{file_doc.yaml => pt_doc.yaml} (51%) create mode 100644 builtins/core/io/pt_test.go rename builtins/core/io/{file.go => write.go} (56%) create mode 100644 builtins/core/io/write_doc.yaml rename builtins/core/io/{file_test.go => write_test.go} (79%) diff --git a/app/app.go b/app/app.go index 6625c2d9d..4889a9741 100644 --- a/app/app.go +++ b/app/app.go @@ -16,9 +16,9 @@ const Name = "murex" const ( Major = 7 Minor = 0 - Revision = 101 + Revision = 102 Branch = "851/file-redirect" - BuildDate = "2024-09-20 23:49:34" + BuildDate = "2024-09-21 00:06:53" ) // Copyright is the copyright owner string diff --git a/builtins/core/io/pt.go b/builtins/core/io/pt.go new file mode 100644 index 000000000..ac35cc9b5 --- /dev/null +++ b/builtins/core/io/pt.go @@ -0,0 +1,52 @@ +package io + +import ( + "fmt" + "io" + "os" + "time" + + "github.com/lmorg/murex/lang" + "github.com/lmorg/murex/lang/types" + "github.com/lmorg/murex/utils/humannumbers" +) + +func init() { + lang.DefineMethod("pt", cmdPipeTelemetry, types.Any, types.Any) +} + +func cmdPipeTelemetry(p *lang.Process) error { + dt := p.Stdin.GetDataType() + p.Stdout.SetDataType(dt) + quit := make(chan bool) + stats := func() { + written, _ := p.Stdin.Stats() + _, read := p.Stdout.Stats() + os.Stderr.WriteString( + fmt.Sprintf("Pipe telemetry: `%s` written %s -> pt -> `%s` read %s (Data type: %s)\n", + p.Previous.Name.String(), + humannumbers.Bytes(written), + p.Next.Name.String(), + humannumbers.Bytes(read), + dt), + ) + } + + go func() { + for { + time.Sleep(1 * time.Second) + select { + case <-quit: + return + default: + stats() + } + + } + }() + + _, err := io.Copy(p.Stdout, p.Stdin) + quit <- true + stats() + return err +} diff --git a/builtins/core/io/file_doc.yaml b/builtins/core/io/pt_doc.yaml similarity index 51% rename from builtins/core/io/file_doc.yaml rename to builtins/core/io/pt_doc.yaml index d0276b91e..6a31508f5 100644 --- a/builtins/core/io/file_doc.yaml +++ b/builtins/core/io/pt_doc.yaml @@ -1,40 +1,3 @@ -- DocumentID: file-truncate - Title: >+ - Truncate File (`>`) - CategoryID: parser - SubCategoryIDs: [ commands.fs, parser.pipes ] - Summary: >- - Writes stdin to disk - overwriting contents if file already exists - Description: |- - Redirects output to file. - - If a file already exists, the contents will be truncated (overwritten). - Otherwise a new file is created. - Usage: |- - ``` - |> filename - ``` - Examples: |- - ``` - g * |> files.txt - ``` - Detail: - Synonyms: - - ">" - - "|>" - - "fwrite" - Related: - - file-append - - pipe - - g - - tmp - - pipe-arrow - - pipe-posix - - namedpipe - - pipe-err - - - - DocumentID: pt Title: >+ Get Pipe Status (`pt`) diff --git a/builtins/core/io/pt_test.go b/builtins/core/io/pt_test.go new file mode 100644 index 000000000..bc238b626 --- /dev/null +++ b/builtins/core/io/pt_test.go @@ -0,0 +1,18 @@ +package io_test + +import ( + "testing" + + "github.com/lmorg/murex/test" +) + +func TestPipeTelemetry(t *testing.T) { + tests := []test.MurexTest{ + { + Block: `tout * 12345 -> pt`, + Stdout: `12345`, + }, + } + + test.RunMurexTests(tests, t) +} diff --git a/builtins/core/io/file.go b/builtins/core/io/write.go similarity index 56% rename from builtins/core/io/file.go rename to builtins/core/io/write.go index cffd9c1cf..bb2c24432 100644 --- a/builtins/core/io/file.go +++ b/builtins/core/io/write.go @@ -5,66 +5,30 @@ import ( "fmt" "io" "os" - "time" "github.com/lmorg/murex/lang" "github.com/lmorg/murex/lang/state" "github.com/lmorg/murex/lang/types" - "github.com/lmorg/murex/utils/humannumbers" "github.com/lmorg/murex/utils/lists" ) func init() { - lang.DefineMethod("pt", cmdPipeTelemetry, types.Any, types.Any) - lang.DefineMethod(">", cmdWriteFile, types.Any, types.Null) - lang.DefineMethod("fwrite", cmdWriteFile, types.Any, types.Null) + lang.DefineMethod(">", cmdTruncateFile, types.Any, types.Null) + lang.DefineMethod("fwrite", cmdTruncateFile, types.Any, types.Null) lang.DefineMethod(">>", cmdAppendFile, types.Any, types.Null) lang.DefineMethod("fappend", cmdAppendFile, types.Any, types.Null) } -func cmdPipeTelemetry(p *lang.Process) error { - dt := p.Stdin.GetDataType() - p.Stdout.SetDataType(dt) - quit := make(chan bool) - stats := func() { - written, _ := p.Stdin.Stats() - _, read := p.Stdout.Stats() - os.Stderr.WriteString( - fmt.Sprintf("Pipe telemetry: `%s` written %s -> pt -> `%s` read %s (Data type: %s)\n", - p.Previous.Name.String(), - humannumbers.Bytes(written), - p.Next.Name.String(), - humannumbers.Bytes(read), - dt), - ) - } - - go func() { - for { - time.Sleep(1 * time.Second) - select { - case <-quit: - return - default: - stats() - } - - } - }() - - _, err := io.Copy(p.Stdout, p.Stdin) - quit <- true - stats() - return err -} - const ( _WAIT_EOF_LONG = "--wait-for-eof" _WAIT_EOF_SHORT = "-w" _DONT_CHECK_PIPELINE = "--ignore-pipeline-check" ) -func cmdWriteFile(p *lang.Process) error { +func cmdTruncateFile(p *lang.Process) error { return writeFile(p, truncateFile) } +func cmdAppendFile(p *lang.Process) error { return writeFile(p, appendFile) } + +func writeFile(p *lang.Process, fn func(io.Reader, string) error) error { p.Stdout.SetDataType(types.Null) filename, err := p.Parameters.String(0) @@ -77,7 +41,7 @@ func cmdWriteFile(p *lang.Process) error { if err != nil { return err } - return writeFile(p.Stdin, filename) + return fn(p.Stdin, filename) } wait := filename == _WAIT_EOF_SHORT || filename == _WAIT_EOF_LONG @@ -94,7 +58,7 @@ func cmdWriteFile(p *lang.Process) error { if wait { _, _ = p.Stderr.Writeln([]byte(fmt.Sprintf("warning: '%s' appears as a parameter elsewhere in the pipeline so I'm going to cache the file in RAM before writing to disk.\n : this message can be suppressed using `%s` or `%s`.", filename, _WAIT_EOF_LONG, _DONT_CHECK_PIPELINE))) } else { - return writeFile(p.Stdin, filename) + return fn(p.Stdin, filename) } b, err := p.Stdin.ReadAll() @@ -102,7 +66,7 @@ func cmdWriteFile(p *lang.Process) error { return err } - return writeFile(bytes.NewReader(b), filename) + return fn(bytes.NewReader(b), filename) } func isFileOpen(p *lang.Process, filename string) bool { @@ -121,7 +85,7 @@ func isFileOpen(p *lang.Process, filename string) bool { } } -func writeFile(reader io.Reader, filename string) error { +func truncateFile(reader io.Reader, filename string) error { file, err := os.Create(filename) if err != nil { return err @@ -133,21 +97,14 @@ func writeFile(reader io.Reader, filename string) error { return err } -func cmdAppendFile(p *lang.Process) error { - p.Stdout.SetDataType(types.Null) - - name, err := p.Parameters.String(0) - if err != nil { - return err - } - - file, err := os.OpenFile(name, os.O_APPEND|os.O_WRONLY|os.O_CREATE, 0664) +func appendFile(reader io.Reader, filename string) error { + file, err := os.OpenFile(filename, os.O_APPEND|os.O_WRONLY|os.O_CREATE, 0664) if err != nil { return err } defer file.Close() - _, err = io.Copy(file, p.Stdin) + _, err = io.Copy(file, reader) return err } diff --git a/builtins/core/io/write_doc.yaml b/builtins/core/io/write_doc.yaml new file mode 100644 index 000000000..15f757b90 --- /dev/null +++ b/builtins/core/io/write_doc.yaml @@ -0,0 +1,34 @@ +- DocumentID: file-truncate + Title: >+ + Truncate File (`>`) + CategoryID: parser + SubCategoryIDs: [ commands.fs, parser.pipes ] + Summary: >- + Writes stdin to disk - overwriting contents if file already exists + Description: |- + Redirects output to file. + + If a file already exists, the contents will be truncated (overwritten). + Otherwise a new file is created. + Usage: |- + ``` + |> filename + ``` + Examples: |- + ``` + g * |> files.txt + ``` + Detail: + Synonyms: + - ">" + - "|>" + - "fwrite" + Related: + - file-append + - pipe + - g + - tmp + - pipe-arrow + - pipe-posix + - namedpipe + - pipe-err diff --git a/builtins/core/io/file_test.go b/builtins/core/io/write_test.go similarity index 79% rename from builtins/core/io/file_test.go rename to builtins/core/io/write_test.go index 7dada897d..ddfb9f4e1 100644 --- a/builtins/core/io/file_test.go +++ b/builtins/core/io/write_test.go @@ -8,17 +8,6 @@ import ( "github.com/lmorg/murex/test" ) -func TestPipeTelemetry(t *testing.T) { - tests := []test.MurexTest{ - { - Block: `tout * 12345 -> pt`, - Stdout: `12345`, - }, - } - - test.RunMurexTests(tests, t) -} - func TestWriteFile(t *testing.T) { file := t.TempDir() file += "/TestWriteFile.txt" diff --git a/docs/commands/pt.md b/docs/commands/pt.md index 71d8eadcb..e308bfe5a 100644 --- a/docs/commands/pt.md +++ b/docs/commands/pt.md @@ -42,4 +42,4 @@ curl -s https://example.com/bigfile.bin -> pt |> bigfile.bin
-This document was generated from [builtins/core/io/file_doc.yaml](https://github.com/lmorg/murex/blob/master/builtins/core/io/file_doc.yaml). \ No newline at end of file +This document was generated from [builtins/core/io/pt_doc.yaml](https://github.com/lmorg/murex/blob/master/builtins/core/io/pt_doc.yaml). \ No newline at end of file diff --git a/docs/parser/file-truncate.md b/docs/parser/file-truncate.md index 276607216..dd35d7b42 100644 --- a/docs/parser/file-truncate.md +++ b/docs/parser/file-truncate.md @@ -49,4 +49,4 @@ g * |> files.txt
-This document was generated from [builtins/core/io/file_doc.yaml](https://github.com/lmorg/murex/blob/master/builtins/core/io/file_doc.yaml). \ No newline at end of file +This document was generated from [builtins/core/io/write_doc.yaml](https://github.com/lmorg/murex/blob/master/builtins/core/io/write_doc.yaml). \ No newline at end of file diff --git a/version.svg b/version.svg index c5033a256..2777e53bc 100644 --- a/version.svg +++ b/version.svg @@ -1 +1 @@ -Version: 7.0.0101Version7.0.0101 +Version: 7.0.0102Version7.0.0102 From a95095ca0b555dd262f51a04b6ff9ae4c6cbe867 Mon Sep 17 00:00:00 2001 From: Laurence Morgan Date: Sat, 21 Sep 2024 01:33:27 +0100 Subject: [PATCH 06/28] #851 send zero data to writer if not method --- app/app.go | 4 ++-- builtins/core/io/write.go | 5 +++++ lang/expressions/parse_block_test.go | 7 ++++--- test/murex.go | 11 ++++++++++- utils/readline/tabgrid_test.go | 6 +++--- utils/readline/{unicide_test.go => unicode_test.go} | 0 version.svg | 2 +- 7 files changed, 25 insertions(+), 10 deletions(-) rename utils/readline/{unicide_test.go => unicode_test.go} (100%) diff --git a/app/app.go b/app/app.go index 4889a9741..f140a57ef 100644 --- a/app/app.go +++ b/app/app.go @@ -16,9 +16,9 @@ const Name = "murex" const ( Major = 7 Minor = 0 - Revision = 102 + Revision = 103 Branch = "851/file-redirect" - BuildDate = "2024-09-21 00:06:53" + BuildDate = "2024-09-21 01:33:30" ) // Copyright is the copyright owner string diff --git a/builtins/core/io/write.go b/builtins/core/io/write.go index bb2c24432..4a9aa916a 100644 --- a/builtins/core/io/write.go +++ b/builtins/core/io/write.go @@ -36,6 +36,11 @@ func writeFile(p *lang.Process, fn func(io.Reader, string) error) error { return err } + if !p.IsMethod { + // nothing to write + return fn(bytes.NewBuffer([]byte{}), filename) + } + if filename == _DONT_CHECK_PIPELINE { filename, err = p.Parameters.String(1) if err != nil { diff --git a/lang/expressions/parse_block_test.go b/lang/expressions/parse_block_test.go index 5c8777f2e..bfe0f684a 100644 --- a/lang/expressions/parse_block_test.go +++ b/lang/expressions/parse_block_test.go @@ -196,7 +196,7 @@ func TestParseBlockPipeAppendFile1(t *testing.T) { tests := []test.MurexTest{ { Block: fmt.Sprintf(`%%(%s) >> %s; %%(%s) >> %s; open %s; rm %s`, filename, filename, filename, filename, filename, filename), - Stdout: filename + filename, + Stdout: fmt.Sprintf(`%s%s`, filename, filename), }, } @@ -208,11 +208,12 @@ func TestParseBlockPipeAppendFile2(t *testing.T) { tests := []test.MurexTest{ { Block: fmt.Sprintf(`echo %s >> %s; echo %s >> %s; open %s; rm %s`, filename, filename, filename, filename, filename, filename), - Stdout: fmt.Sprintf("%s\n%s\n", filename, filename), + Stdout: fmt.Sprintf("^%s\n%s\n$", filename, filename), + Stderr: `^warning:`, }, } - test.RunMurexTests(tests, t) + test.RunMurexTestsRx(tests, t) } func TestParseBlockLogicOperators(t *testing.T) { diff --git a/test/murex.go b/test/murex.go index f597307ee..58bb18461 100644 --- a/test/murex.go +++ b/test/murex.go @@ -14,6 +14,8 @@ import ( "github.com/lmorg/murex/test/count" ) +const _TIMEOUT_SECONDS = 30 + // MurexTest is a basic framework to test murex code. // Please note this shouldn't be confused with the murex scripting language's inbuilt testing framework! type MurexTest struct { @@ -36,7 +38,7 @@ func RunMurexTests(tests []MurexTest, t *testing.T) { fork := lang.ShellProcess.Fork(lang.F_FUNCTION | lang.F_NEW_MODULE | lang.F_NO_STDIN | lang.F_CREATE_STDOUT | lang.F_CREATE_STDERR) go func(testNum int) { - time.Sleep(30 * time.Second) + time.Sleep(_TIMEOUT_SECONDS * time.Second) if !fork.Process.HasCancelled() { panic(fmt.Sprintf("timeout in %s() test %d", t.Name(), testNum)) } @@ -109,6 +111,13 @@ func RunMurexTestsRx(tests []MurexTest, t *testing.T) { hasError := false fork := lang.ShellProcess.Fork(lang.F_FUNCTION | lang.F_NEW_MODULE | lang.F_NO_STDIN | lang.F_CREATE_STDOUT | lang.F_CREATE_STDERR) + go func(testNum int) { + time.Sleep(_TIMEOUT_SECONDS * time.Second) + if !fork.Process.HasCancelled() { + panic(fmt.Sprintf("timeout in %s() test %d", t.Name(), testNum)) + } + }(i) + fork.Name.Set("RunMurexTestsRx()") fork.FileRef = &ref.File{Source: &ref.Source{Module: fmt.Sprintf("murex/%s-%d", t.Name(), i)}} exitNum, err := fork.Execute([]rune(tests[i].Block)) diff --git a/utils/readline/tabgrid_test.go b/utils/readline/tabgrid_test.go index 4e54ddfd3..a24e035ab 100644 --- a/utils/readline/tabgrid_test.go +++ b/utils/readline/tabgrid_test.go @@ -21,9 +21,9 @@ func TestCropCaption(t *testing.T) { } }() - for caption = 0; caption < 101; caption++ { - for maxLen = 0; maxLen < 101; maxLen++ { - for cellWidth = 0; cellWidth < 101; cellWidth++ { + for caption = 0; caption < 10; caption++ { + for maxLen = 0; maxLen < 10; maxLen++ { + for cellWidth = 0; cellWidth < 10; cellWidth++ { _ = cropCaption(strings.Repeat("s", caption), maxLen, cellWidth) } } diff --git a/utils/readline/unicide_test.go b/utils/readline/unicode_test.go similarity index 100% rename from utils/readline/unicide_test.go rename to utils/readline/unicode_test.go diff --git a/version.svg b/version.svg index 2777e53bc..4927bd243 100644 --- a/version.svg +++ b/version.svg @@ -1 +1 @@ -Version: 7.0.0102Version7.0.0102 +Version: 7.0.0103Version7.0.0103 From e23799dd3d8efa5f96a42b3b0dacd42874bcdb9a Mon Sep 17 00:00:00 2001 From: Laurence Morgan Date: Sun, 22 Sep 2024 23:15:44 +0100 Subject: [PATCH 07/28] truncate: updated docs --- app/app.go | 4 +- builtins/core/io/write.go | 38 ++++--- builtins/core/io/write_doc.yaml | 153 +++++++++++++++++++++++++++- builtins/core/io/write_test.go | 1 - builtins/core/io/write_unix_test.go | 132 ++++++++++++++++++++++++ docs/parser/file-append.md | 6 +- docs/parser/file-truncate.md | 100 +++++++++++++++++- gen/parser/pipes_doc.yaml | 53 ---------- version.svg | 2 +- 9 files changed, 412 insertions(+), 77 deletions(-) create mode 100644 builtins/core/io/write_unix_test.go diff --git a/app/app.go b/app/app.go index f140a57ef..d53d7512d 100644 --- a/app/app.go +++ b/app/app.go @@ -16,9 +16,9 @@ const Name = "murex" const ( Major = 7 Minor = 0 - Revision = 103 + Revision = 107 Branch = "851/file-redirect" - BuildDate = "2024-09-21 01:33:30" + BuildDate = "2024-09-22 23:17:58" ) // Copyright is the copyright owner string diff --git a/builtins/core/io/write.go b/builtins/core/io/write.go index 4a9aa916a..9d1a024e4 100644 --- a/builtins/core/io/write.go +++ b/builtins/core/io/write.go @@ -20,9 +20,10 @@ func init() { } const ( - _WAIT_EOF_LONG = "--wait-for-eof" - _WAIT_EOF_SHORT = "-w" - _DONT_CHECK_PIPELINE = "--ignore-pipeline-check" + _WAIT_EOF_SHORT = "-w" + _WAIT_EOF_LONG = "--wait-for-eof" + _IGNORE_PIPELINE_SHORT = "-i" + _IGNORE_PIPELINE_LONG = "--ignore-pipeline-check" ) func cmdTruncateFile(p *lang.Process) error { return writeFile(p, truncateFile) } @@ -41,28 +42,35 @@ func writeFile(p *lang.Process, fn func(io.Reader, string) error) error { return fn(bytes.NewBuffer([]byte{}), filename) } - if filename == _DONT_CHECK_PIPELINE { - filename, err = p.Parameters.String(1) - if err != nil { - return err + if filename == _IGNORE_PIPELINE_SHORT || filename == _IGNORE_PIPELINE_LONG { + parameter2, err := p.Parameters.String(1) + if err == nil { + return fn(p.Stdin, parameter2) } - return fn(p.Stdin, filename) + // no second parameter so lets assume the flag was actually a file name } wait := filename == _WAIT_EOF_SHORT || filename == _WAIT_EOF_LONG if wait { - filename, err = p.Parameters.String(1) - if err != nil { - return err + parameter2, err := p.Parameters.String(1) + if err == nil { + filename = parameter2 + + } else { + // no second parameter so lets assume the flag was actually a file name + wait = false } - } else { + } + + if !wait { wait = isFileOpen(p, filename) + if wait { + _, _ = p.Stderr.Writeln([]byte(fmt.Sprintf("warning: '%s' appears as a parameter elsewhere in the pipeline so I'm going to cache the file in RAM before writing to disk.\n : This message can be suppressed using `%s` or `%s`.", filename, _WAIT_EOF_LONG, _IGNORE_PIPELINE_LONG))) + } } - if wait { - _, _ = p.Stderr.Writeln([]byte(fmt.Sprintf("warning: '%s' appears as a parameter elsewhere in the pipeline so I'm going to cache the file in RAM before writing to disk.\n : this message can be suppressed using `%s` or `%s`.", filename, _WAIT_EOF_LONG, _DONT_CHECK_PIPELINE))) - } else { + if !wait { return fn(p.Stdin, filename) } diff --git a/builtins/core/io/write_doc.yaml b/builtins/core/io/write_doc.yaml index 15f757b90..e553cc4e6 100644 --- a/builtins/core/io/write_doc.yaml +++ b/builtins/core/io/write_doc.yaml @@ -12,13 +12,105 @@ Otherwise a new file is created. Usage: |- ``` - |> filename + |> [ -i | --ignore-pipeline-check ] filename + + |> [ -w | --wait-for-eof ] filename ``` Examples: |- ``` g * |> files.txt ``` - Detail: + Flags: + "--ignore-pipeline-check": >- + Don't check if _filename_ is a parameter for an earlier command in the pipeline + "-i": >- + Alias for `--ignore-pipeline` + + "--wait-for-eof": >- + Wait for stdin to return an EOF before opening _filename_ + "-w": >- + Alias for `--wait-for-eof` + Detail: |- + ### Race Condition Detection + + If no flags are specified, then `|>` will check if the filename supplied is + used in any parameters for other commands in the pipeline. If it has been, then + `|>` will wait for an **EOF** (End Of File) from stdin before opening _filename_. + + This is to allow pipelines like the following: + + ``` + open example.log | regexp m/error/ |> example.log + ``` + + Under traditional shells and Murex's normal scheduler, all commands in a + pipeline would run concurrently. This leads to a race condition where `|>` + opens (and thus truncates) a file before other commands can read from it. + + However by default, `|>` will check the pipeline to look for any other + references of _filename_ and if it exists, it will wait for an EOF before + `|>` truncates _filename_. + + This wait for EOF behaviour can be forced with the `--wait-for-eof` / `-w` + flag. + + Alternatively, if you want to force `|>` to run concurrently then you can + disable the pipeline check with the `--ignore-pipeline-check` / `-i` flag. + + #### High Memory Usage + + > WARNING! Waiting for EOF will cause `|>` to cache the pipeline into RAM. + > If your pipeline is parsing multi-gigabyte or larger files then you may + > experience performance issues. + + For large datasets, it might be preferable to write to a temporary file first. + + ``` + open example.log | regexp m/error/ |> example.log.tmp + mv example.log.tmp example.log + ``` + + The move operation should be instantaneous on most filesystems because your + operating system will just alter filesystem metadata rather than move the file + contents. + + ### Flag Without A Filename + + If you specify a flag without a filename, eg `|> --wait-for-eof`, then it is + assumed that the flag _is_ the filename. + + ### Syntactic Sugar + + While `|>` is referred to as an operator, it's actually a pipe followed by a + builtin: + + ``` + out "foobar" | > example.txt + ``` + + Thus you can actually use `>` by itself. + + ### Creating An Empty File + + If `>` is at the start of a pipeline then it is treated as null input. This a + convenient shortcut to create an empty file or blank an existing file. + + **Create a new empty file:** + + ``` + > newfile + ``` + + **Clear a large log file without deleting the file itself:** + + ``` + > large.log + ``` + + ### Appending A File + + To append a file (ie write at the end of the file without overwriting its + contents) use `>>` instead. Synonyms: - ">" - "|>" @@ -32,3 +124,60 @@ - pipe-posix - namedpipe - pipe-err + - schedulers + + + +- DocumentID: file-append + Title: >- + `>>` Append File + CategoryID: parser + SubCategoryIDs: [ parser.pipes ] + Summary: >- + Writes stdin to disk - appending contents if file already exists + Description: |- + This is used to redirect the stdout of a command and append it to a file. If + that file does not exist, then the file is created. + + This behaves similarly to the [Bash (et al) token](https://www.gnu.org/software/bash/manual/bash.html#Appending-Redirected-Output) + except it doesn't support adding alternative file descriptor numbers. Instead + you will need to use named pipes to achieve the same effect in Murex. + Examples: |- + ``` + » out "Hello" >> example.txt + » out "World!" >> example.txt + » open example.txt + Hello + World! + ``` + Detail: |- + ### Syntactic Sugar + + This is just syntactic sugar for `-> >>`. Thus when the parser reads code like + the following: + + ``` + out "foobar" >> example.txt + ``` + + it will compile an abstract syntax tree which would reflect the following code + instead: + + ``` + out "foobar" | >> example.txt + ``` + + ### Truncating A File + + To truncate a file (ie overwrite its contents) use `|>` instead. + Synonyms: + - ">>" + - "fappend" + Related: + - pipe + - pipe-arrow + - pipe-posix + - namedpipe + - pipeline + - file-truncate + - out diff --git a/builtins/core/io/write_test.go b/builtins/core/io/write_test.go index ddfb9f4e1..3823eb574 100644 --- a/builtins/core/io/write_test.go +++ b/builtins/core/io/write_test.go @@ -4,7 +4,6 @@ import ( "fmt" "testing" - _ "github.com/lmorg/murex/builtins" "github.com/lmorg/murex/test" ) diff --git a/builtins/core/io/write_unix_test.go b/builtins/core/io/write_unix_test.go new file mode 100644 index 000000000..737b6b27f --- /dev/null +++ b/builtins/core/io/write_unix_test.go @@ -0,0 +1,132 @@ +//go:build !windows +// +build !windows + +package io + +import ( + "fmt" + "testing" + + "github.com/lmorg/murex/test" +) + +func TestWriteFilePipelineFlags(t *testing.T) { + file := t.TempDir() + file += "/TestWriteFilePipelineFlags" + + tests := []test.MurexTest{ + { + Block: fmt.Sprintf(`a [0..20] |> %[1]s.%[2]d; open %[1]s.%[2]d -> regexp m/0/ |> %[1]s.%[2]d; open %[1]s.%[2]d`, + file, 0), + Stdout: "^0\n10\n20\n$", + Stderr: "^warning", + }, + { + Block: fmt.Sprintf(`a [0..20] |> %[1]s.%[2]d; open %[1]s.%[2]d -> regexp m/0/ |> %[3]s %[1]s.%[2]d; open %[1]s.%[2]d`, + file, 1, _WAIT_EOF_SHORT), + Stdout: "^0\n10\n20\n$", + Stderr: "^$", + }, + { + Block: fmt.Sprintf(`a [0..20] |> %[1]s.%[2]d; open %[1]s.%[2]d -> regexp m/0/ |> %[3]s %[1]s.%[2]d; open %[1]s.%[2]d`, + file, 2, _WAIT_EOF_LONG), + Stdout: "^0\n10\n20\n$", + Stderr: "^$", + }, + { + Block: fmt.Sprintf(`a [0..20] |> %[1]s.%[2]d; open %[1]s.%[2]d -> regexp s/0// |> %[3]s %[1]s.%[2]d; g %[3]s`, + file, 3, _IGNORE_PIPELINE_SHORT), + Stdout: "^$", + Stderr: "^$", + ExitNum: 1, // just because of ending g + }, + { + Block: fmt.Sprintf(`a [0..20] |> %[1]s.%[2]d; open %[1]s.%[2]d -> regexp s/0// |> %[3]s %[1]s.%[2]d; g %[3]s`, + file, 4, _IGNORE_PIPELINE_LONG), Stdout: "^$", + Stderr: "^$", + ExitNum: 1, // just because of ending g + }, + + // two tests here because the regexp is being quirky and I want to check beginning and end of string + { + Block: fmt.Sprintf(`g %[3]s; a [0..20] |> %[1]s.%[2]d; open %[1]s.%[2]d -> regexp s/0// |> %[3]s; rm -- %[3]s`, + file, 5, _WAIT_EOF_SHORT), + Stdout: "^$", + Stderr: `^Error in .g.`, + }, + { + Block: fmt.Sprintf(`g %[3]s; a [0..20] |> %[1]s.%[2]d; open %[1]s.%[2]d -> regexp s/0// |> %[3]s; rm -- %[3]s`, + file, 6, _WAIT_EOF_SHORT), + Stdout: "^$", + Stderr: `Error: no data returned\n$`, + }, + // two tests here because the regexp is being quirky and I want to check beginning and end of string + { + Block: fmt.Sprintf(`g %[3]s; a [0..20] |> %[1]s.%[2]d; open %[1]s.%[2]d -> regexp s/0// |> %[3]s; rm -- %[3]s`, + file, 5, _WAIT_EOF_LONG), + Stdout: "^$", + Stderr: `^Error in .g.`, + }, + { + Block: fmt.Sprintf(`g %[3]s; a [0..20] |> %[1]s.%[2]d; open %[1]s.%[2]d -> regexp s/0// |> %[3]s; rm -- %[3]s`, + file, 6, _WAIT_EOF_LONG), + Stdout: "^$", + Stderr: `Error: no data returned\n$`, + }, + // two tests here because the regexp is being quirky and I want to check beginning and end of string + { + Block: fmt.Sprintf(`g %[3]s; a [0..20] |> %[1]s.%[2]d; open %[1]s.%[2]d -> regexp s/0// |> %[3]s; rm -- %[3]s`, + file, 7, _IGNORE_PIPELINE_SHORT), + Stdout: "^$", + Stderr: `^Error in .g.`, + }, + { + Block: fmt.Sprintf(`g %[3]s; a [0..20] |> %[1]s.%[2]d; open %[1]s.%[2]d -> regexp s/0// |> %[3]s; rm -- %[3]s`, + file, 8, _IGNORE_PIPELINE_SHORT), + Stdout: "^$", + Stderr: `Error: no data returned\n$`, + }, + // two tests here because the regexp is being quirky and I want to check beginning and end of string + { + Block: fmt.Sprintf(`g %[3]s; a [0..20] |> %[1]s.%[2]d; open %[1]s.%[2]d -> regexp s/0// |> %[3]s; rm -- %[3]s`, + file, 5, _IGNORE_PIPELINE_LONG), + Stdout: "^$", + Stderr: `^Error in .g.`, + }, + { + Block: fmt.Sprintf(`g %[3]s; a [0..20] |> %[1]s.%[2]d; open %[1]s.%[2]d -> regexp s/0// |> %[3]s; rm -- %[3]s`, + file, 6, _IGNORE_PIPELINE_LONG), + Stdout: "^$", + Stderr: `Error: no data returned\n$`, + }, + } + + test.RunMurexTestsRx(tests, t) +} + +func TestWriteEmptyFile(t *testing.T) { + file := t.TempDir() + file += "/TestWriteFilePipelineFlags" + + tests := []test.MurexTest{ + // two tests here because the regexp is being quirky and I want to check beginning and end of string + { + Block: fmt.Sprintf(`g %[1]s.%[2]d; > %[1]s.%[2]d; g %[1]s.%[2]d`, + file, 0), + Stdout: fmt.Sprintf(`^\[\"%[1]s.%[2]d\"\]\n$`, + file, 0), + Stderr: `^Error in .g.`, + ExitNum: 0, // no error because last command succeeded + }, + { + Block: fmt.Sprintf(`g %[1]s.%[2]d; > %[1]s.%[2]d; g %[1]s.%[2]d`, + file, 1), + Stdout: fmt.Sprintf(`^\[\"%[1]s.%[2]d\"\]\n$`, + file, 1), + Stderr: `Error: no data returned\n$`, + ExitNum: 0, // no error because last command succeeded + }, + } + + test.RunMurexTestsRx(tests, t) +} diff --git a/docs/parser/file-append.md b/docs/parser/file-append.md index 89648f318..ef3a39c91 100644 --- a/docs/parser/file-append.md +++ b/docs/parser/file-append.md @@ -25,6 +25,8 @@ World! ## Detail +### Syntactic Sugar + This is just syntactic sugar for `-> >>`. Thus when the parser reads code like the following: @@ -39,7 +41,7 @@ instead: out "foobar" | >> example.txt ``` -### Truncating a file +### Truncating A File To truncate a file (ie overwrite its contents) use `|>` instead. @@ -68,4 +70,4 @@ To truncate a file (ie overwrite its contents) use `|>` instead.
-This document was generated from [gen/parser/pipes_doc.yaml](https://github.com/lmorg/murex/blob/master/gen/parser/pipes_doc.yaml). \ No newline at end of file +This document was generated from [builtins/core/io/write_doc.yaml](https://github.com/lmorg/murex/blob/master/builtins/core/io/write_doc.yaml). \ No newline at end of file diff --git a/docs/parser/file-truncate.md b/docs/parser/file-truncate.md index dd35d7b42..66d1b5706 100644 --- a/docs/parser/file-truncate.md +++ b/docs/parser/file-truncate.md @@ -12,7 +12,9 @@ Otherwise a new file is created. ## Usage ``` - |> filename + |> [ -i | --ignore-pipeline-check ] filename + + |> [ -w | --wait-for-eof ] filename ``` ## Examples @@ -21,6 +23,100 @@ Otherwise a new file is created. g * |> files.txt ``` +## Flags + +* `--ignore-pipeline-check` + Don't check if _filename_ is a parameter for an earlier command in the pipeline +* `--wait-for-eof` + Wait for stdin to return an EOF before opening _filename_ +* `-i` + Alias for `--ignore-pipeline` +* `-w` + Alias for `--wait-for-eof` + +## Detail + +### Race Condition Detection + +If no flags are specified, then `|>` will check if the filename supplied is +used in any parameters for other commands in the pipeline. If it has been, then +`|>` will wait for an **EOF** (End Of File) from stdin before opening _filename_. + +This is to allow pipelines like the following: + +``` +open example.log | regexp m/error/ |> example.log +``` + +Under traditional shells and Murex's normal scheduler, all commands in a +pipeline would run concurrently. This leads to a race condition where `|>` +opens (and thus truncates) a file before other commands can read from it. + +However by default, `|>` will check the pipeline to look for any other +references of _filename_ and if it exists, it will wait for an EOF before +`|>` truncates _filename_. + +This wait for EOF behaviour can be forced with the `--wait-for-eof` / `-w` +flag. + +Alternatively, if you want to force `|>` to run concurrently then you can +disable the pipeline check with the `--ignore-pipeline-check` / `-i` flag. + +#### High Memory Usage + +> WARNING! Waiting for EOF will cause `|>` to cache the pipeline into RAM. +> If your pipeline is parsing multi-gigabyte or larger files then you may +> experience performance issues. + +For large datasets, it might be preferable to write to a temporary file first. + +``` +open example.log | regexp m/error/ |> example.log.tmp +mv example.log.tmp example.log +``` + +The move operation should be instantaneous on most filesystems because your +operating system will just alter filesystem metadata rather than move the file +contents. + +### Flag Without A Filename + +If you specify a flag without a filename, eg `|> --wait-for-eof`, then it is +assumed that the flag _is_ the filename. + +### Syntactic Sugar + +While `|>` is referred to as an operator, it's actually a pipe followed by a +builtin: + +``` +out "foobar" | > example.txt +``` + +Thus you can actually use `>` by itself. + +### Creating An Empty File + +If `>` is at the start of a pipeline then it is treated as null input. This a +convenient shortcut to create an empty file or blank an existing file. + +**Create a new empty file:** + +``` +> newfile +``` + +**Clear a large log file without deleting the file itself:** + +``` +> large.log +``` + +### Appending A File + +To append a file (ie write at the end of the file without overwriting its +contents) use `>>` instead. + ## Synonyms * `>` @@ -38,6 +134,8 @@ g * |> files.txt Glob pattern matching for file system objects (eg `*.txt`) * [Read / Write To A Named Pipe (``)](../parser/namedpipe.md): Reads from a Murex named pipe +* [Schedulers](../user-guide/schedulers.md): + Overview of the different schedulers (or 'run modes') in Murex * [`->` Arrow Pipe](../parser/pipe-arrow.md): Pipes stdout from the left hand command to stdin of the right hand command * [`>>` Append File](../parser/file-append.md): diff --git a/gen/parser/pipes_doc.yaml b/gen/parser/pipes_doc.yaml index 1c7d331c7..58ff298e4 100644 --- a/gen/parser/pipes_doc.yaml +++ b/gen/parser/pipes_doc.yaml @@ -221,56 +221,3 @@ - err - regexp - - -- DocumentID: file-append - Title: >- - `>>` Append File - CategoryID: parser - SubCategoryIDs: [ parser.pipes ] - Summary: >- - Writes stdin to disk - appending contents if file already exists - Description: |- - This is used to redirect the stdout of a command and append it to a file. If - that file does not exist, then the file is created. - - This behaves similarly to the [Bash (et al) token](https://www.gnu.org/software/bash/manual/bash.html#Appending-Redirected-Output) - except it doesn't support adding alternative file descriptor numbers. Instead - you will need to use named pipes to achieve the same effect in Murex. - Examples: |- - ``` - » out "Hello" >> example.txt - » out "World!" >> example.txt - » open example.txt - Hello - World! - ``` - Detail: |- - This is just syntactic sugar for `-> >>`. Thus when the parser reads code like - the following: - - ``` - out "foobar" >> example.txt - ``` - - it will compile an abstract syntax tree which would reflect the following code - instead: - - ``` - out "foobar" | >> example.txt - ``` - - ### Truncating a file - - To truncate a file (ie overwrite its contents) use `|>` instead. - Synonyms: - - ">>" - - "fappend" - Related: - - pipe - - pipe-arrow - - pipe-posix - - namedpipe - - pipeline - - file-truncate - - out diff --git a/version.svg b/version.svg index 4927bd243..a5a887ecb 100644 --- a/version.svg +++ b/version.svg @@ -1 +1 @@ -Version: 7.0.0103Version7.0.0103 +Version: 7.0.0107Version7.0.0107 From 58844ebcae26017507418c9811e03077cf91d8e5 Mon Sep 17 00:00:00 2001 From: Laurence Morgan Date: Wed, 25 Sep 2024 20:02:10 +0100 Subject: [PATCH 08/28] #883 mutex added to private struct --- app/app.go | 4 ++-- lang/privates.go | 36 +++++++++++++++++++++++++++++++++--- version.svg | 2 +- 3 files changed, 36 insertions(+), 6 deletions(-) diff --git a/app/app.go b/app/app.go index 086fdb18c..ef1f5cadf 100644 --- a/app/app.go +++ b/app/app.go @@ -16,9 +16,9 @@ const Name = "murex" const ( Major = 6 Minor = 3 - Revision = 4235 + Revision = 4237 Branch = "develop" - BuildDate = "2024-09-19 23:09:30" + BuildDate = "2024-09-25 20:02:12" ) // Copyright is the copyright owner string diff --git a/lang/privates.go b/lang/privates.go index 7380d8fab..3420918dd 100644 --- a/lang/privates.go +++ b/lang/privates.go @@ -3,6 +3,7 @@ package lang import ( "fmt" "strings" + "sync" "github.com/lmorg/murex/lang/ref" ) @@ -13,6 +14,7 @@ func ErrPrivateNotFound(module string) error { type privateFunctions struct { module map[string]*MurexFuncs + mutex sync.RWMutex } func NewMurexPrivs() *privateFunctions { @@ -22,14 +24,21 @@ func NewMurexPrivs() *privateFunctions { } func (pf *privateFunctions) Define(name string, parameters []MxFunctionParams, block []rune, fileRef *ref.File) { + pf.mutex.Lock() + if pf.module[fileRef.Source.Module] == nil { pf.module[fileRef.Source.Module] = NewMurexFuncs() } pf.module[fileRef.Source.Module].Define(name, parameters, block, fileRef) + + pf.mutex.Unlock() } func (pf *privateFunctions) get(name string, fileRef *ref.File) *murexFuncDetails { + pf.mutex.RLock() + defer pf.mutex.RUnlock() + if pf.module[fileRef.Source.Module] == nil { return nil } @@ -38,6 +47,9 @@ func (pf *privateFunctions) get(name string, fileRef *ref.File) *murexFuncDetail } func (pf *privateFunctions) GetString(name string, module string) *murexFuncDetails { + pf.mutex.RLock() + defer pf.mutex.RUnlock() + if pf.module[module] == nil { return nil } @@ -46,6 +58,9 @@ func (pf *privateFunctions) GetString(name string, module string) *murexFuncDeta } func (pf *privateFunctions) Exists(name string, fileRef *ref.File) bool { + pf.mutex.RLock() + defer pf.mutex.RUnlock() + if pf.module[fileRef.Source.Module] == nil { return false } @@ -54,6 +69,9 @@ func (pf *privateFunctions) Exists(name string, fileRef *ref.File) bool { } func (pf *privateFunctions) ExistsString(name string, module string) bool { + pf.mutex.RLock() + defer pf.mutex.RUnlock() + if pf.module[module] == nil { return false } @@ -62,6 +80,9 @@ func (pf *privateFunctions) ExistsString(name string, module string) bool { } func (pf *privateFunctions) BlockString(name string, module string) ([]rune, error) { + pf.mutex.RLock() + defer pf.mutex.RUnlock() + if pf.module[module] == nil { return nil, ErrPrivateNotFound(module) } @@ -70,6 +91,9 @@ func (pf *privateFunctions) BlockString(name string, module string) ([]rune, err } func (pf *privateFunctions) Summary(name string, fileRef *ref.File) (string, error) { + pf.mutex.RLock() + defer pf.mutex.RUnlock() + if pf.module[fileRef.Source.Module] == nil { return "", ErrPrivateNotFound(fileRef.Source.Module) } @@ -78,6 +102,9 @@ func (pf *privateFunctions) Summary(name string, fileRef *ref.File) (string, err } func (pf *privateFunctions) Undefine(name string, fileRef *ref.File) error { + pf.mutex.Lock() + defer pf.mutex.Unlock() + if pf.module[fileRef.Source.Module] == nil { return ErrPrivateNotFound(fileRef.Source.Module) } @@ -85,8 +112,11 @@ func (pf *privateFunctions) Undefine(name string, fileRef *ref.File) error { return pf.module[fileRef.Source.Module].Undefine(name) } -func (pf *privateFunctions) Dump() interface{} { - dump := make(map[string]map[string]interface{}) +func (pf *privateFunctions) Dump() any { + pf.mutex.RLock() + defer pf.mutex.RUnlock() + + dump := make(map[string]map[string]any) for name, module := range pf.module { path := strings.SplitN(name, "/", 2) if len(path) != 2 { @@ -94,7 +124,7 @@ func (pf *privateFunctions) Dump() interface{} { } if len(dump[path[0]]) == 0 { - dump[path[0]] = make(map[string]interface{}) + dump[path[0]] = make(map[string]any) } dump[path[0]][path[1]] = module.Dump() diff --git a/version.svg b/version.svg index efeb860b6..f740bf72b 100644 --- a/version.svg +++ b/version.svg @@ -1 +1 @@ -Version: 6.3.4235Version6.3.4235 +Version: 6.3.4237Version6.3.4237 From 16d55e692910c2704617f287a7ac18eabc496e9d Mon Sep 17 00:00:00 2001 From: Laurence Morgan Date: Wed, 25 Sep 2024 20:07:02 +0100 Subject: [PATCH 09/28] #492 undefine private support --- app/app.go | 4 ++-- builtins/core/structs/function.go | 8 ++++---- builtins/core/structs/function_doc.yaml | 6 +++--- docs/commands/private.md | 6 +++--- version.svg | 2 +- 5 files changed, 13 insertions(+), 13 deletions(-) diff --git a/app/app.go b/app/app.go index ef1f5cadf..0673f5acc 100644 --- a/app/app.go +++ b/app/app.go @@ -16,9 +16,9 @@ const Name = "murex" const ( Major = 6 Minor = 3 - Revision = 4237 + Revision = 4238 Branch = "develop" - BuildDate = "2024-09-25 20:02:12" + BuildDate = "2024-09-25 20:07:04" ) // Copyright is the copyright owner string diff --git a/builtins/core/structs/function.go b/builtins/core/structs/function.go index f5e44c010..b75fc13a4 100644 --- a/builtins/core/structs/function.go +++ b/builtins/core/structs/function.go @@ -19,7 +19,7 @@ func init() { lang.DefineFunction("function", cmdFunc, types.Null) lang.DefineFunction("!function", cmdUnfunc, types.Null) lang.DefineFunction("private", cmdPrivate, types.Null) - //lang.DefineFunction("!private", cmdUnprivate, types.Null) + lang.DefineFunction("!private", cmdUnprivate, types.Null) lang.DefineFunction("method", cmdMethod, types.Null) defaults.AppendProfile(` @@ -185,14 +185,14 @@ func cmdPrivate(p *lang.Process) error { } } -/*func cmdUnprivate(p *lang.Process) error { +func cmdUnprivate(p *lang.Process) error { name, err := p.Parameters.String(0) if err != nil { return err } - return lang.PrivateFunctions.Undefine(name) -}*/ + return lang.PrivateFunctions.Undefine(name, p.FileRef) +} func cmdMethod(p *lang.Process) error { fn, err := p.Parameters.String(0) diff --git a/builtins/core/structs/function_doc.yaml b/builtins/core/structs/function_doc.yaml index 0db650ccd..b6cc1724b 100644 --- a/builtins/core/structs/function_doc.yaml +++ b/builtins/core/structs/function_doc.yaml @@ -388,15 +388,15 @@ ### Undefining a private - Because private functions are fixed to the source file that declares them, - there isn't much point in undefining them. Thus at this point in time, it - is not possible to do so. + Like all other definable states in Murex, you can delete a function with + the bang prefix `!private` ### Order of preference {{ include "gen/includes/order-of-precedence.inc.md" }} Synonyms: - private + - "!private" Related: - function - alias diff --git a/docs/commands/private.md b/docs/commands/private.md index 464d6b589..46bd1f53f 100644 --- a/docs/commands/private.md +++ b/docs/commands/private.md @@ -54,9 +54,8 @@ preference below). ### Undefining a private -Because private functions are fixed to the source file that declares them, -there isn't much point in undefining them. Thus at this point in time, it -is not possible to do so. +Like all other definable states in Murex, you can delete a function with +the bang prefix `!private` ### Order of preference @@ -93,6 +92,7 @@ You can override this order of precedence via the `fexec` and `exec` builtins. ## Synonyms * `private` +* `!private` ## See Also diff --git a/version.svg b/version.svg index f740bf72b..909eef113 100644 --- a/version.svg +++ b/version.svg @@ -1 +1 @@ -Version: 6.3.4237Version6.3.4237 +Version: 6.3.4238Version6.3.4238 From 421772eec55cffecd919601d1940ed8c7458ff01 Mon Sep 17 00:00:00 2001 From: Laurence Morgan Date: Wed, 25 Sep 2024 20:12:29 +0100 Subject: [PATCH 10/28] #429 add tests --- app/app.go | 4 ++-- builtins/core/structs/function_test.go | 16 +++++++++++++++- builtins/docs/summaries.go | 2 ++ version.svg | 2 +- 4 files changed, 20 insertions(+), 4 deletions(-) diff --git a/app/app.go b/app/app.go index 0673f5acc..286a42eb6 100644 --- a/app/app.go +++ b/app/app.go @@ -16,9 +16,9 @@ const Name = "murex" const ( Major = 6 Minor = 3 - Revision = 4238 + Revision = 4239 Branch = "develop" - BuildDate = "2024-09-25 20:07:04" + BuildDate = "2024-09-25 20:12:31" ) // Copyright is the copyright owner string diff --git a/builtins/core/structs/function_test.go b/builtins/core/structs/function_test.go index 1331004b4..694d9b9a6 100644 --- a/builtins/core/structs/function_test.go +++ b/builtins/core/structs/function_test.go @@ -93,7 +93,7 @@ func TestPrivate(t *testing.T) { fn := "TestPrivate" mod := fmt.Sprintf("GoTest-%d", rand.Int()) - count.Tests(t, 2) + count.Tests(t, 4) lang.InitEnv() @@ -118,4 +118,18 @@ func TestPrivate(t *testing.T) { if !lang.PrivateFunctions.ExistsString(fn, mod) { t.Fatalf("Expecting '%s/%s' to be created, it did not", mod, fn) } + + err = lang.PrivateFunctions.Undefine(fn, p.FileRef) + if err != nil { + t.Fatalf("Expecting no errors. Got %v", err) + } + + if lang.PrivateFunctions.ExistsString(fn, mod) { + t.Fatalf("Expecting '%s/%s' to be removed, it was not", mod, fn) + } + + err = lang.PrivateFunctions.Undefine(fn, p.FileRef) + if err == nil { + t.Fatalf("Expecting and error") + } } diff --git a/builtins/docs/summaries.go b/builtins/docs/summaries.go index 3e8f5e752..ca2f68b82 100644 --- a/builtins/docs/summaries.go +++ b/builtins/docs/summaries.go @@ -559,6 +559,7 @@ func init() { "pretty": "pretty", "struct-keys": "struct-keys", "private": "private", + "!private": "private", "time": "time", "function": "function", "!function": "function", @@ -801,6 +802,7 @@ func init() { "commands/pretty": "commands/pretty", "commands/struct-keys": "commands/struct-keys", "commands/private": "commands/private", + "commands/!private": "commands/private", "commands/time": "commands/time", "commands/function": "commands/function", "commands/!function": "commands/function", diff --git a/version.svg b/version.svg index 909eef113..fadebe6bd 100644 --- a/version.svg +++ b/version.svg @@ -1 +1 @@ -Version: 6.3.4238Version6.3.4238 +Version: 6.3.4239Version6.3.4239 From 8ffaf03e87adb573130131301f9fb937c129e9c7 Mon Sep 17 00:00:00 2001 From: Laurence Morgan Date: Wed, 25 Sep 2024 20:16:35 +0100 Subject: [PATCH 11/28] move license file out of license.go (makes github got a little weird) --- app/app.go | 6 +++--- docs.go | 8 ++++++++ license.go | 14 -------------- version.svg | 2 +- 4 files changed, 12 insertions(+), 18 deletions(-) delete mode 100644 license.go diff --git a/app/app.go b/app/app.go index 286a42eb6..9761ee72a 100644 --- a/app/app.go +++ b/app/app.go @@ -15,10 +15,10 @@ const Name = "murex" // Format of version string should be "$(Major).$(Minor).$(Revision) ($Branch)" const ( Major = 6 - Minor = 3 - Revision = 4239 + Minor = 4 + Revision = 101 Branch = "develop" - BuildDate = "2024-09-25 20:12:31" + BuildDate = "2024-09-25 20:16:38" ) // Copyright is the copyright owner string diff --git a/docs.go b/docs.go index d764f9863..56b11b922 100644 --- a/docs.go +++ b/docs.go @@ -9,6 +9,7 @@ import ( "fmt" "strings" + "github.com/lmorg/murex/app" "github.com/lmorg/murex/builtins/docs" "github.com/lmorg/murex/debug" ) @@ -48,3 +49,10 @@ func docsImport(path string) []byte { b, _ := docsEmbeded.ReadFile(path) return b } + +//go:embed LICENSE +var license string + +func init() { + app.SetLicenseFull(license) +} diff --git a/license.go b/license.go deleted file mode 100644 index b504c3645..000000000 --- a/license.go +++ /dev/null @@ -1,14 +0,0 @@ -package main - -import ( - _ "embed" - - "github.com/lmorg/murex/app" -) - -//go:embed LICENSE -var license string - -func init() { - app.SetLicenseFull(license) -} diff --git a/version.svg b/version.svg index fadebe6bd..d92ea86a5 100644 --- a/version.svg +++ b/version.svg @@ -1 +1 @@ -Version: 6.3.4239Version6.3.4239 +Version: 6.4.0101Version6.4.0101 From 9ab0803731ca4c02d524cbe2fa002dee658c3a1e Mon Sep 17 00:00:00 2001 From: Laurence Morgan Date: Fri, 27 Sep 2024 23:13:06 +0100 Subject: [PATCH 12/28] readline: MaxTabCompleterRows defaults to granular increments --- app/app.go | 4 ++-- utils/readline/preview.go | 16 ++++++++++++---- version.svg | 2 +- 3 files changed, 15 insertions(+), 7 deletions(-) diff --git a/app/app.go b/app/app.go index bacb86b07..11b3d62f4 100644 --- a/app/app.go +++ b/app/app.go @@ -16,9 +16,9 @@ const Name = "murex" const ( Major = 6 Minor = 4 - Revision = 200 + Revision = 204 Branch = "develop" - BuildDate = "2024-09-25 20:16:38" + BuildDate = "2024-09-27 23:13:09" ) // Copyright is the copyright owner string diff --git a/utils/readline/preview.go b/utils/readline/preview.go index 368842108..d390defad 100644 --- a/utils/readline/preview.go +++ b/utils/readline/preview.go @@ -121,12 +121,20 @@ func (rl *Instance) getPreviewXY() (*PreviewSizeT, error) { func (rl *Instance) previewAutocompleteHeight(height int) { switch { - case height < 40: - rl.MaxTabCompleterRows = noLargerThan(rl.MaxTabCompleterRows, 4) - case height < 30: - rl.MaxTabCompleterRows = noLargerThan(rl.MaxTabCompleterRows, 3) case height < 20: rl.MaxTabCompleterRows = noLargerThan(rl.MaxTabCompleterRows, 2) + case height < 25: + rl.MaxTabCompleterRows = noLargerThan(rl.MaxTabCompleterRows, 3) + case height < 28: + rl.MaxTabCompleterRows = noLargerThan(rl.MaxTabCompleterRows, 4) + case height < 31: + rl.MaxTabCompleterRows = noLargerThan(rl.MaxTabCompleterRows, 5) + case height < 34: + rl.MaxTabCompleterRows = noLargerThan(rl.MaxTabCompleterRows, 6) + case height < 37: + rl.MaxTabCompleterRows = noLargerThan(rl.MaxTabCompleterRows, 7) + case height < 40: + rl.MaxTabCompleterRows = noLargerThan(rl.MaxTabCompleterRows, 8) } } diff --git a/version.svg b/version.svg index d92ea86a5..f46037039 100644 --- a/version.svg +++ b/version.svg @@ -1 +1 @@ -Version: 6.4.0101Version6.4.0101 +Version: 6.4.0204Version6.4.0204 From 615f76dd891a23fe93b4d8aa5a999cc22bcd1908 Mon Sep 17 00:00:00 2001 From: Laurence Morgan Date: Sun, 29 Sep 2024 21:52:03 +0100 Subject: [PATCH 13/28] core: line numbering bugfix --- app/app.go | 4 ++-- lang/expressions/ast.go | 39 +++++++++++++++++++++++++++++++-------- version.svg | 2 +- 3 files changed, 34 insertions(+), 11 deletions(-) diff --git a/app/app.go b/app/app.go index 11b3d62f4..e55c37502 100644 --- a/app/app.go +++ b/app/app.go @@ -16,9 +16,9 @@ const Name = "murex" const ( Major = 6 Minor = 4 - Revision = 204 + Revision = 265 Branch = "develop" - BuildDate = "2024-09-27 23:13:09" + BuildDate = "2024-09-29 21:52:05" ) // Copyright is the copyright owner string diff --git a/lang/expressions/ast.go b/lang/expressions/ast.go index 2494d52bf..034633c91 100644 --- a/lang/expressions/ast.go +++ b/lang/expressions/ast.go @@ -41,27 +41,50 @@ type ParserT struct { _expandGlob interface{} } -func (tree *ParserT) nextChar() rune { - if tree.charPos+1 >= len(tree.expression) { +// prevChar returns the current character, performing bounds checks in the +// process. +// +// If `charPos` is out of bounds, then `prevChar()` returns `0`. +func (tree *ParserT) prevChar() rune { + if tree.charPos < 1 { return 0 } - return tree.expression[tree.charPos+1] + return tree.expression[tree.charPos-1] } -func (tree *ParserT) prevChar() rune { - if tree.charPos < 1 { +// currentChar returns the current character, performing bounds checks in the +// process. If bounds checking is not required, then please use +// +// tree.expression[tree.charPos] +// +// instead because it saves on a branching instruction inside hot code. +// +// If `charPos` is out of bounds, then `currentChar()` returns `0`. +func (tree *ParserT) currentChar() rune { + if tree.charPos >= len(tree.expression) { return 0 } - return tree.expression[tree.charPos-1] + return tree.expression[tree.charPos] +} + +// nextChar returns the current character, performing bounds checks in the +// process. +// +// If `charPos` is out of bounds, then `nextChar()` returns `0`. +func (tree *ParserT) nextChar() rune { + if tree.charPos+1 >= len(tree.expression) { + return 0 + } + return tree.expression[tree.charPos+1] } func (tree *ParserT) crLf() { tree.endRow++ - tree.endCol = tree.charPos + tree.endCol = 0 //tree.charPos } func (tree *ParserT) GetColumnN() int { return tree.charOffset - tree.startCol + 2 } -func (tree *ParserT) GetLineN() int { return tree.startRow } +func (tree *ParserT) GetLineN() int { return tree.endRow } func (tree *ParserT) appendAst(key symbols.Exp, value ...rune) { tree.ast = append(tree.ast, &astNodeT{ diff --git a/version.svg b/version.svg index f46037039..584091d73 100644 --- a/version.svg +++ b/version.svg @@ -1 +1 @@ -Version: 6.4.0204Version6.4.0204 +Version: 6.4.0265Version6.4.0265 From d0480bd123291310a758ef8fb9e2611fdcbebed6 Mon Sep 17 00:00:00 2001 From: Laurence Morgan Date: Sun, 29 Sep 2024 21:52:40 +0100 Subject: [PATCH 14/28] core: add support for `(expression)` syntax in variable dot notation --- app/app.go | 4 +-- lang/expressions/parse_array.go | 2 +- lang/expressions/parse_function.go | 2 +- lang/expressions/parse_object.go | 2 +- lang/expressions/parse_quotes.go | 2 +- lang/expressions/parse_statement.go | 2 +- lang/expressions/parse_switch.go | 2 +- lang/expressions/parse_vars.go | 41 ++++++++++++++++++++++++++--- version.svg | 2 +- 9 files changed, 46 insertions(+), 13 deletions(-) diff --git a/app/app.go b/app/app.go index e55c37502..258b39265 100644 --- a/app/app.go +++ b/app/app.go @@ -16,9 +16,9 @@ const Name = "murex" const ( Major = 6 Minor = 4 - Revision = 265 + Revision = 266 Branch = "develop" - BuildDate = "2024-09-29 21:52:05" + BuildDate = "2024-09-29 21:52:42" ) // Copyright is the copyright owner string diff --git a/lang/expressions/parse_array.go b/lang/expressions/parse_array.go index e3ef5df71..d49b17027 100644 --- a/lang/expressions/parse_array.go +++ b/lang/expressions/parse_array.go @@ -139,7 +139,7 @@ func (tree *ParserT) parseArray(exec bool) ([]rune, *primitives.DataType, error) } default: - _, v, _, err := tree.parseVarScalar(exec, varAsValue) + _, v, _, err := tree.parseVarScalar(exec, exec, varAsValue) if err != nil { return nil, nil, err } diff --git a/lang/expressions/parse_function.go b/lang/expressions/parse_function.go index f427fdb57..85bd7f3ae 100644 --- a/lang/expressions/parse_function.go +++ b/lang/expressions/parse_function.go @@ -205,7 +205,7 @@ func (tree *ParserT) parseFunctionParameters(cmd []rune) ([]rune, error) { } default: // start scalar - _, _, _, err := tree.parseVarScalar(false, varAsString) + _, _, _, err := tree.parseVarScalar(false, false, varAsString) if err != nil { return nil, raiseError(tree.expression, nil, tree.charPos, err.Error()) } diff --git a/lang/expressions/parse_object.go b/lang/expressions/parse_object.go index 2b9352cc8..71eed8387 100644 --- a/lang/expressions/parse_object.go +++ b/lang/expressions/parse_object.go @@ -154,7 +154,7 @@ func (tree *ParserT) parseObject(exec bool) ([]rune, *primitives.DataType, error default: // inline scalar strOrVal := varFormatting(o.stage) - scalar, val, _, err := tree.parseVarScalar(exec, strOrVal) + scalar, val, _, err := tree.parseVarScalar(exec, exec, strOrVal) if err != nil { return nil, nil, err } diff --git a/lang/expressions/parse_quotes.go b/lang/expressions/parse_quotes.go index 19fbe9f65..d07c811b5 100644 --- a/lang/expressions/parse_quotes.go +++ b/lang/expressions/parse_quotes.go @@ -137,7 +137,7 @@ func (tree *ParserT) parseStringInfix(qEnd rune, exec bool) ([]rune, error) { } default: // inline scalar - scalar, v, _, err := tree.parseVarScalar(exec, varAsString) + scalar, v, _, err := tree.parseVarScalar(exec, exec, varAsString) if err != nil { return nil, err } diff --git a/lang/expressions/parse_statement.go b/lang/expressions/parse_statement.go index fa9e4f6b3..3963bb816 100644 --- a/lang/expressions/parse_statement.go +++ b/lang/expressions/parse_statement.go @@ -429,7 +429,7 @@ func (tree *ParserT) parseStatement(exec bool) error { var tokenise bool tokenise = tree.tokeniseScalar() execScalar := exec && tokenise - value, v, _, err := tree.parseVarScalar(execScalar, varAsString) + value, v, _, err := tree.parseVarScalar(exec, execScalar, varAsString) if err != nil { return raiseError(tree.expression, nil, tree.charPos, err.Error()) } diff --git a/lang/expressions/parse_switch.go b/lang/expressions/parse_switch.go index 429eaaedb..94ba8ff1b 100644 --- a/lang/expressions/parse_switch.go +++ b/lang/expressions/parse_switch.go @@ -228,7 +228,7 @@ func (tree *ParserT) parseSwitch() (int, error) { tree.statement.canHaveZeroLenStr = true case isBareChar(tree.nextChar()): // start scalar - _, v, _, err := tree.parseVarScalar(true, varAsString) + _, v, _, err := tree.parseVarScalar(true, true, varAsString) if err != nil { return 0, err } diff --git a/lang/expressions/parse_vars.go b/lang/expressions/parse_vars.go index 360395259..b02760050 100644 --- a/lang/expressions/parse_vars.go +++ b/lang/expressions/parse_vars.go @@ -11,7 +11,7 @@ import ( ) func (tree *ParserT) parseVarScalarExpr(exec, execScalars bool) ([]rune, interface{}, string, primitives.FunctionT, error) { - runes, v, mxDt, err := tree.parseVarScalar(execScalars, varAsValue) + runes, v, mxDt, err := tree.parseVarScalar(exec, execScalars, varAsValue) if exec { fn := func() (*primitives.Value, error) { @@ -35,7 +35,7 @@ func (tree *ParserT) parseVarScalarExpr(exec, execScalars bool) ([]rune, interfa return runes, v, mxDt, nil, err } -func (tree *ParserT) parseVarScalar(execScalars bool, strOrVal varFormatting) ([]rune, interface{}, string, error) { +func (tree *ParserT) parseVarScalar(exec, execScalars bool, strOrVal varFormatting) ([]rune, interface{}, string, error) { if tree.nextChar() == '(' { tree.charPos++ return tree.parseVarParenthesis(execScalars, strOrVal) @@ -46,11 +46,44 @@ func (tree *ParserT) parseVarScalar(execScalars bool, strOrVal varFormatting) ([ return []rune{'$'}, "$", types.String, nil } + startPos := tree.charPos tree.charPos++ - value := tree.parseBareword() + var value []rune - if tree.charPos < len(tree.expression) && tree.expression[tree.charPos] == '[' { +parseBareword: + value = append(value, tree.parseBareword()...) + + switch tree.currentChar() { + case 0: + break + + case '[': return tree.parseVarIndexElement(execScalars, '$', value, strOrVal) + + case '(': + if tree.prevChar() != '.' { + break + } + + v, err := tree.parseSubExpression(exec) + if err != nil { + return tree.expression[startPos : tree.charPos-1], nil, "", err + } + tree.charPos++ + if execScalars { + s, err := types.ConvertGoType(v, types.String) + if err != nil { + return tree.expression[startPos : tree.charPos-1], nil, "", err + } + value = append(value, []rune(s.(string))...) + + } else { + value = append(value, []rune(v.(string))...) + } + + if tree.currentChar() == '.' { + goto parseBareword + } } tree.charPos-- diff --git a/version.svg b/version.svg index 584091d73..b64e1ef4b 100644 --- a/version.svg +++ b/version.svg @@ -1 +1 @@ -Version: 6.4.0265Version6.4.0265 +Version: 6.4.0266Version6.4.0266 From 15c338881986a1f4687a2385c14cd32b9da8d5a1 Mon Sep 17 00:00:00 2001 From: Laurence Morgan Date: Mon, 30 Sep 2024 20:02:55 +0100 Subject: [PATCH 15/28] negative indexes in elements --- app/app.go | 4 +- lang/define_element_object.go | 8 ++- lang/define_element_object_test.go | 82 +++++++++++++++++++++++++++--- lang/exit.go | 1 - lang/expressions/ast.go | 1 + version.svg | 2 +- 6 files changed, 85 insertions(+), 13 deletions(-) diff --git a/app/app.go b/app/app.go index 258b39265..e2e146191 100644 --- a/app/app.go +++ b/app/app.go @@ -16,9 +16,9 @@ const Name = "murex" const ( Major = 6 Minor = 4 - Revision = 266 + Revision = 269 Branch = "develop" - BuildDate = "2024-09-29 21:52:42" + BuildDate = "2024-09-30 20:03:22" ) // Copyright is the copyright owner string diff --git a/lang/define_element_object.go b/lang/define_element_object.go index b15bf4a6e..f3aa80a05 100644 --- a/lang/define_element_object.go +++ b/lang/define_element_object.go @@ -123,11 +123,15 @@ func isValidElementIndex(key string, length int) (int, error) { } if i < 0 { - return 0, fmt.Errorf("negative keys are not allowed for arrays: %s", key) + i += length + if i < 0 { + return 0, fmt.Errorf("element is an array however key (%s -> %d) is greater than the length (%d)", key, i, length) + } + return i, nil } if i >= length { - return 0, fmt.Errorf("element is an array however key is greater than the length: %s", key) + return 0, fmt.Errorf("element is an array however key (%s) is greater than the length (%d)", key, length) } return i, nil diff --git a/lang/define_element_object_test.go b/lang/define_element_object_test.go index 0c1958a30..07a6493f0 100644 --- a/lang/define_element_object_test.go +++ b/lang/define_element_object_test.go @@ -1,9 +1,8 @@ -package lang_test +package lang import ( "testing" - "github.com/lmorg/murex/lang" "github.com/lmorg/murex/test/count" "github.com/lmorg/murex/utils/json" ) @@ -35,8 +34,8 @@ func TestElementLookup(t *testing.T) { Object: []string{ "foo", "bar", }, - Path: ".-1", - Error: true, + Path: ".-1", + Expected: "bar", }, ///// @@ -59,8 +58,8 @@ func TestElementLookup(t *testing.T) { Object: []interface{}{ "foo", "bar", }, - Path: ".-1", - Error: true, + Path: ".-1", + Expected: "bar", }, ///// @@ -186,7 +185,7 @@ func TestElementLookup(t *testing.T) { for i, test := range tests { expected := json.LazyLogging(test.Expected) - v, err := lang.ElementLookup(test.Object, test.Path) + v, err := ElementLookup(test.Object, test.Path) actual := json.LazyLogging(v) if (err != nil) != test.Error || actual != expected { @@ -200,3 +199,72 @@ func TestElementLookup(t *testing.T) { } } } + +func TestIsValidElementIndex(t *testing.T) { + tests := []struct { + Key string + Length int + Index int + Error bool + }{ + { + Key: "-", + Length: 0, + Index: 0, + Error: true, + }, + { + Key: "0", + Length: 0, + Index: 0, + Error: true, + }, + { + Key: "0", + Length: 1, + Index: 0, + Error: false, + }, + { + Key: "3", + Length: 2, + Index: 0, + Error: true, + }, + { + Key: "-1", + Length: 7, + Index: 6, + Error: false, + }, + { + Key: "-10", + Length: 7, + Index: 0, + Error: true, + }, + { + Key: "-5", + Length: 5, + Index: 0, + Error: false, + }, + } + + count.Tests(t, len(tests)) + + for i, test := range tests { + index, err := isValidElementIndex(test.Key, test.Length) + + if index != test.Index || (err != nil) != test.Error { + t.Errorf("Unexpected return in test %d: ", i) + t.Logf(" Key: '%s'", test.Key) + t.Logf(" Length: %d", test.Length) + t.Logf(" exp index: %d", test.Index) + t.Logf(" act index: %d", index) + t.Logf(" exp err: %v", test.Error) + t.Logf(" act err: %v", err) + } + + } +} diff --git a/lang/exit.go b/lang/exit.go index 4ce42d060..4301c2266 100644 --- a/lang/exit.go +++ b/lang/exit.go @@ -15,6 +15,5 @@ func Exit(exitNum int) { ProfMemCleanUp() ProfTraceCleanUp() - //cache.CloseDb() os.Exit(exitNum) } diff --git a/lang/expressions/ast.go b/lang/expressions/ast.go index 034633c91..f4f65f637 100644 --- a/lang/expressions/ast.go +++ b/lang/expressions/ast.go @@ -80,6 +80,7 @@ func (tree *ParserT) nextChar() rune { func (tree *ParserT) crLf() { tree.endRow++ + tree.startCol = 0 tree.endCol = 0 //tree.charPos } diff --git a/version.svg b/version.svg index b64e1ef4b..da0bab3cb 100644 --- a/version.svg +++ b/version.svg @@ -1 +1 @@ -Version: 6.4.0266Version6.4.0266 +Version: 6.4.0269Version6.4.0269 From 94215bb40fc45eb44b7411cb66a129c9eff9edf8 Mon Sep 17 00:00:00 2001 From: Laurence Morgan Date: Mon, 30 Sep 2024 22:35:18 +0100 Subject: [PATCH 16/28] element: add support for unicode separators --- app/app.go | 4 ++-- builtins/core/element/element_doc.yaml | 19 +++++++------------ docs/parser/element.md | 19 +++++++------------ lang/define_element_object.go | 10 ++++++++-- lang/define_element_object_test.go | 15 +++++++++++++++ version.svg | 2 +- 6 files changed, 40 insertions(+), 29 deletions(-) diff --git a/app/app.go b/app/app.go index e2e146191..a8ee750a1 100644 --- a/app/app.go +++ b/app/app.go @@ -16,9 +16,9 @@ const Name = "murex" const ( Major = 6 Minor = 4 - Revision = 269 + Revision = 275 Branch = "develop" - BuildDate = "2024-09-30 20:03:22" + BuildDate = "2024-09-30 22:38:12" ) // Copyright is the copyright owner string diff --git a/builtins/core/element/element_doc.yaml b/builtins/core/element/element_doc.yaml index 6b9b6377c..2e715d5e3 100644 --- a/builtins/core/element/element_doc.yaml +++ b/builtins/core/element/element_doc.yaml @@ -62,31 +62,26 @@ » config -> [[ \|shell\|syntax-highlighting\|Data-Type ]] bool - » config -> [[ >shell>syntax-highlighting>Data-Type ]] + » config -> [[ 😅shell😅syntax-highlighting😅Data-Type ]] bool ``` However there are a few of caveats: - 1. Currently **element** does not support unicode separators. All separators - must be 1 byte characters. This limitation is highlighted as a bug, albeit - a low priority one. If this limitation does directly affect you then raise - an issue on GitHub to get the priority bumped up. - - 2. Any shell tokens (eg pipe `|`, `;`, `}`, etc) will need to be escaped. For + 1. Any shell tokens (eg pipe `|`, `;`, `}`, etc) will need to be escaped. For readability reasons it is recommended not to use such characters even though it is technically possible to. - ``` + ``` # Would fail because the semi-colon is an unescaped / unquoted shell token config -> [[ ;shell-syntax-highlighting;Data-Type ]] - ``` + ``` - 3. Please also make sure you don't use a character that is also used inside + 2. Please also make sure you don't use a character that is also used inside key names because keys _cannot_ be escaped. For example both of the following would fail: - ``` + ``` # Would fail because 'syntax-highlighting' and 'Data-Type' both also contain # the separator character config -> [[ -shell-syntax-highlighting-Data-Type ]] @@ -94,7 +89,7 @@ # Would fail because you cannot escape key names (escaping happens at the # shell parser level rather than command parameter level) config -> [[ -shell-syntax\-highlighting-Data\-Type ]] - ``` + ``` ### Quoting parameters diff --git a/docs/parser/element.md b/docs/parser/element.md index 4eb6dc58a..2c2e50e0b 100644 --- a/docs/parser/element.md +++ b/docs/parser/element.md @@ -65,31 +65,26 @@ bool » config -> [[ \|shell\|syntax-highlighting\|Data-Type ]] bool -» config -> [[ >shell>syntax-highlighting>Data-Type ]] +» config -> [[ 😅shell😅syntax-highlighting😅Data-Type ]] bool ``` However there are a few of caveats: -1. Currently **element** does not support unicode separators. All separators - must be 1 byte characters. This limitation is highlighted as a bug, albeit - a low priority one. If this limitation does directly affect you then raise - an issue on GitHub to get the priority bumped up. - -2. Any shell tokens (eg pipe `|`, `;`, `}`, etc) will need to be escaped. For +1. Any shell tokens (eg pipe `|`, `;`, `}`, etc) will need to be escaped. For readability reasons it is recommended not to use such characters even though it is technically possible to. -``` + ``` # Would fail because the semi-colon is an unescaped / unquoted shell token config -> [[ ;shell-syntax-highlighting;Data-Type ]] -``` + ``` -3. Please also make sure you don't use a character that is also used inside +2. Please also make sure you don't use a character that is also used inside key names because keys _cannot_ be escaped. For example both of the following would fail: -``` + ``` # Would fail because 'syntax-highlighting' and 'Data-Type' both also contain # the separator character config -> [[ -shell-syntax-highlighting-Data-Type ]] @@ -97,7 +92,7 @@ However there are a few of caveats: # Would fail because you cannot escape key names (escaping happens at the # shell parser level rather than command parameter level) config -> [[ -shell-syntax\-highlighting-Data\-Type ]] -``` + ``` ### Quoting parameters diff --git a/lang/define_element_object.go b/lang/define_element_object.go index f3aa80a05..4137452cc 100644 --- a/lang/define_element_object.go +++ b/lang/define_element_object.go @@ -13,7 +13,13 @@ func ElementLookup(v interface{}, path string) (interface{}, error) { return nil, fmt.Errorf("invalid path for element lookup: `%s` is too short", path) } - pathSplit := strings.Split(path, path[0:1]) + var separator string + for _, r := range path { + separator = string(r) + break + } + + pathSplit := strings.Split(path, separator) obj := v for i := 1; i < len(pathSplit); i++ { @@ -21,7 +27,7 @@ func ElementLookup(v interface{}, path string) (interface{}, error) { if i == len(pathSplit)-1 { break } else { - return nil, fmt.Errorf("path element %d is a zero length string: '%s'", i-1, strings.Join(pathSplit, "/")) + return nil, fmt.Errorf("path element %d is a zero length string: '%s'", i-1, strings.Join(pathSplit, separator)) } } diff --git a/lang/define_element_object_test.go b/lang/define_element_object_test.go index 07a6493f0..5d6fc2521 100644 --- a/lang/define_element_object_test.go +++ b/lang/define_element_object_test.go @@ -16,6 +16,21 @@ type testElementLookupT struct { func TestElementLookup(t *testing.T) { tests := []testElementLookupT{ + { + Object: map[string]any{ + "one": map[string]any{"two": map[string]any{"three": "four"}}, + }, + Path: "/one/two/three", + Expected: "four", + }, + { + Object: map[string]any{ + "one": map[string]any{"two": map[string]any{"three": "four"}}, + }, + Path: "😅one😅two😅three", + Expected: "four", + }, + { Object: []string{ "foo", "bar", diff --git a/version.svg b/version.svg index da0bab3cb..a02ee2b52 100644 --- a/version.svg +++ b/version.svg @@ -1 +1 @@ -Version: 6.4.0269Version6.4.0269 +Version: 6.4.0275Version6.4.0275 From 4fb351c9841dc78997c56aca19232e5a94ab9ec6 Mon Sep 17 00:00:00 2001 From: Laurence Morgan Date: Tue, 1 Oct 2024 22:15:54 +0100 Subject: [PATCH 17/28] readline: `a` + `A` should move the cursor right --- app/app.go | 4 ++-- utils/readline/unicode.go | 4 ++-- utils/readline/vim.go | 8 ++++---- version.svg | 2 +- 4 files changed, 9 insertions(+), 9 deletions(-) diff --git a/app/app.go b/app/app.go index a8ee750a1..207a23400 100644 --- a/app/app.go +++ b/app/app.go @@ -16,9 +16,9 @@ const Name = "murex" const ( Major = 6 Minor = 4 - Revision = 275 + Revision = 283 Branch = "develop" - BuildDate = "2024-09-30 22:38:12" + BuildDate = "2024-10-01 22:15:56" ) // Copyright is the copyright owner string diff --git a/utils/readline/unicode.go b/utils/readline/unicode.go index 246e922c0..dd4ef2d0b 100644 --- a/utils/readline/unicode.go +++ b/utils/readline/unicode.go @@ -35,8 +35,8 @@ func (u *UnicodeT) _offByOne(i int) int { if len(u.value) == 0 { return 0 } - if u.rl != nil && u.rl.modeViMode != vimInsert && i == len(u.value) { - i = len(u.value) - 1 + if i == len(u.value) && (u.rl == nil || u.rl.modeViMode != vimInsert) { + i = len(u.value) } return i } diff --git a/utils/readline/vim.go b/utils/readline/vim.go index 1c1b57eec..7129e1d75 100644 --- a/utils/readline/vim.go +++ b/utils/readline/vim.go @@ -32,8 +32,8 @@ func (rl *Instance) vi(r rune) string { switch r { case 'a': if rl.line.CellLen() > 0 { - output = moveCursorForwardsStr(1) - rl.line.SetRunePos(rl.line.RunePos()) + //output = moveCursorForwardsStr(1) + rl.line.SetRunePos(rl.line.RunePos() + 1) } rl.modeViMode = vimInsert rl.viIteration = "" @@ -41,8 +41,8 @@ func (rl *Instance) vi(r rune) string { case 'A': if rl.line.RuneLen() > 0 { - output = moveCursorForwardsStr(rl.line.CellLen() - rl.line.CellPos()) - rl.line.SetRunePos(rl.line.RuneLen()) + //output = moveCursorForwardsStr(rl.line.CellLen() - rl.line.CellPos()) + rl.line.SetRunePos(rl.line.RuneLen() + 1) } rl.modeViMode = vimInsert rl.viIteration = "" diff --git a/version.svg b/version.svg index a02ee2b52..b665a6825 100644 --- a/version.svg +++ b/version.svg @@ -1 +1 @@ -Version: 6.4.0275Version6.4.0275 +Version: 6.4.0283Version6.4.0283 From adde27ea1643e193e2a3655653681e13f077b4b0 Mon Sep 17 00:00:00 2001 From: Laurence Morgan Date: Tue, 1 Oct 2024 23:21:54 +0100 Subject: [PATCH 18/28] #867 better autocompletion support for `kill` --- app/app.go | 4 ++-- integrations/signals_posix.mx | 11 +++++++++++ version.svg | 2 +- 3 files changed, 14 insertions(+), 3 deletions(-) diff --git a/app/app.go b/app/app.go index 207a23400..9632dfa1d 100644 --- a/app/app.go +++ b/app/app.go @@ -16,9 +16,9 @@ const Name = "murex" const ( Major = 6 Minor = 4 - Revision = 283 + Revision = 287 Branch = "develop" - BuildDate = "2024-10-01 22:15:56" + BuildDate = "2024-10-01 23:21:57" ) // Copyright is the copyright owner string diff --git a/integrations/signals_posix.mx b/integrations/signals_posix.mx index 3e01f3ef4..e93156b91 100644 --- a/integrations/signals_posix.mx +++ b/integrations/signals_posix.mx @@ -18,6 +18,17 @@ test unit private autocomplete.pids %{ } autocomplete set kill %[ + { + Dynamic: %({ kill -l -> jsplit %(\s) -> prefix %(-) }) + FlagsDesc: { + -s: signal_name + -l: "list signals" + } + FlagValues: { + -s: [{ Dynamic: %({ kill -l -> jsplit %(\s) -> prefix %(-) }) }] + } + Optional: true + } { DynamicDesc: '{ autocomplete.pids }' ListView: true diff --git a/version.svg b/version.svg index b665a6825..964aaa3d4 100644 --- a/version.svg +++ b/version.svg @@ -1 +1 @@ -Version: 6.4.0283Version6.4.0283 +Version: 6.4.0287Version6.4.0287 From ffc38623f07c6331a0113913c81e83fe85792d22 Mon Sep 17 00:00:00 2001 From: Laurence Morgan Date: Thu, 3 Oct 2024 23:56:18 +0100 Subject: [PATCH 19/28] datetime: support zero flags + improved errors --- app/app.go | 4 +- builtins/core/typemgmt/datetime.go | 113 ++++++++++++++++++----------- version.svg | 2 +- 3 files changed, 74 insertions(+), 45 deletions(-) diff --git a/app/app.go b/app/app.go index 9632dfa1d..45d7a7a30 100644 --- a/app/app.go +++ b/app/app.go @@ -16,9 +16,9 @@ const Name = "murex" const ( Major = 6 Minor = 4 - Revision = 287 + Revision = 291 Branch = "develop" - BuildDate = "2024-10-01 23:21:57" + BuildDate = "2024-10-03 23:56:20" ) // Copyright is the copyright owner string diff --git a/builtins/core/typemgmt/datetime.go b/builtins/core/typemgmt/datetime.go index 110d1aa97..bbf0e4cee 100644 --- a/builtins/core/typemgmt/datetime.go +++ b/builtins/core/typemgmt/datetime.go @@ -12,40 +12,74 @@ import ( "github.com/lmorg/murex/lang/parameters" "github.com/lmorg/murex/lang/types" "github.com/lmorg/murex/utils" + "github.com/lmorg/murex/utils/escape" ) func init() { lang.DefineMethod("datetime", cmdDateTime, types.Any, types.Any) - defaults.AppendProfile(` - autocomplete: set datetime { - [{ - "FlagsDesc": { - "--value": "(optional) Input data/time string to be parsed", - "--in": "Formatting rules of input data/time string", - "--out": "Formatting rules of output date/time string" - }, - "AllowMultiple": true, - "AnyValue": true - }] - } - `) + defaults.AppendProfile(fmt.Sprintf(` + autocomplete: set datetime %%[{ + FlagsDesc: { + %[3]s: "(optional) Input data/time string to be parsed" + %[1]s: "Formatting rules of input data/time string" + %[2]s: "Formatting rules of output date/time string" + } + AllowMultiple: true + AnyValue: true + }]`, _FLAG_IN, _FLAG_OUT, _FLAG_VAL)) } +const ( + _FLAG_IN = "--in" + _FLAG_OUT = "--out" + _FLAG_VAL = "--value" + + _FORMAT_PYTHON = "{py}" + _FORMAT_GOLANG = "{go}" + _FORMAT_EPOCH = "{unix}" + _FORMAT_NOW = "{now}" + + errTooManyParameters = "too many parameters without flags" +) + func cmdDateTime(p *lang.Process) error { - flags, _, err := p.Parameters.ParseFlags(¶meters.Arguments{ + flags, additional, err := p.Parameters.ParseFlags(¶meters.Arguments{ Flags: map[string]string{ - "--in": types.String, - "--out": types.String, - "--value": types.String, + _FLAG_IN: types.String, + _FLAG_OUT: types.String, + _FLAG_VAL: types.String, }, - AllowAdditional: false, + AllowAdditional: true, }) if err != nil { return err } - if flags["--value"] == "" && flags["--in"] != "{now}" { + var ( + fIn = flags[_FLAG_IN] + fOut = flags[_FLAG_OUT] + fValue = flags[_FLAG_VAL] + ) + + switch len(additional) { + case 0: + break + + case 1: + if fIn != "" || fOut != "" || fValue != "" { + escape.CommandLine(additional) + return fmt.Errorf("%s: %s", errTooManyParameters, strings.Join(additional, " ")) + } + fIn, fOut = _FORMAT_NOW, additional[0] + goto skipFlagValidation + + default: + escape.CommandLine(additional) + return fmt.Errorf("%s: %s", errTooManyParameters, strings.Join(additional, " ")) + } + + if fValue == "" && fIn != _FORMAT_NOW { if err := p.ErrIfNotAMethod(); err != nil { return err } @@ -54,66 +88,61 @@ func cmdDateTime(p *lang.Process) error { if err != nil { return err } - flags["--value"] = string(utils.CrLfTrim(b)) + fValue = string(utils.CrLfTrim(b)) } - in := flags["--in"] - out := flags["--out"] - - if in == "" { - return errors.New("no datatime format provided for input value: `--in` required") + if fIn == "" { + return fmt.Errorf("no datatime format provided for input value: `%s` required", _FLAG_IN) } - if out == "" { - return errors.New("no datatime format provided for output value: `--out` required") + if fOut == "" { + return fmt.Errorf("no datatime format provided for output value: `%s` required", _FLAG_OUT) } +skipFlagValidation: + var datetime time.Time // Parse --in switch { - case strings.HasPrefix(in, "{go}"): - datetime, err = time.Parse(in[4:], flags["--value"]) + case strings.HasPrefix(fIn, _FORMAT_GOLANG): + datetime, err = time.Parse(fIn[4:], fValue) if err != nil { return err } - case strings.HasPrefix(in, "{py}"): - /*datetime, err = time.Parse(in[4:], flags["--value"]) - if err != nil { - return err - }*/ + case strings.HasPrefix(fIn, _FORMAT_PYTHON): return errors.New("TODO! This feature hasn't yet been developed") - case strings.HasPrefix(in, "{now}"): + case strings.HasPrefix(fIn, _FORMAT_NOW): datetime = time.Now() default: - return errors.New("unknown or invalid input parser formatter, expecting `{go}`, `{py}` or `{now}`") + return fmt.Errorf("unknown or invalid input parser formatter, expecting `%s`, `%s` or `%s`", _FORMAT_GOLANG, _FORMAT_PYTHON, _FORMAT_NOW) } // Write --out switch { - case strings.HasPrefix(out, "{go}"): - _, err = p.Stdout.Write([]byte(datetime.Format(out[4:]))) + case strings.HasPrefix(fOut, _FORMAT_GOLANG): + _, err = p.Stdout.Write([]byte(datetime.Format(fOut[4:]))) return err - case strings.HasPrefix(out, "{py}"): - s, err := pyParse(out[4:], datetime) + case strings.HasPrefix(fOut, _FORMAT_PYTHON): + s, err := pyParse(fOut[4:], datetime) if err != nil { return err } _, err = p.Stdout.Write([]byte(s)) return err - case strings.HasPrefix(out, "{unix}"): + case strings.HasPrefix(fOut, _FORMAT_EPOCH): _, err = p.Stdout.Write([]byte(strconv.FormatInt(datetime.Unix(), 10))) return err default: - return errors.New("unknown or invalid output parser formatter, expecting `{go}`, `{py}`, or `{unix}`") + return fmt.Errorf("unknown or invalid output parser formatter, expecting `%s`, `%s`, or `%s`", _FORMAT_GOLANG, _FORMAT_PYTHON, _FORMAT_EPOCH) } } diff --git a/version.svg b/version.svg index 964aaa3d4..5ba9492d0 100644 --- a/version.svg +++ b/version.svg @@ -1 +1 @@ -Version: 6.4.0287Version6.4.0287 +Version: 6.4.0291Version6.4.0291 From cbf080169339fe726c5533c79622bb7db062fb54 Mon Sep 17 00:00:00 2001 From: Laurence Morgan Date: Thu, 10 Oct 2024 23:46:59 +0100 Subject: [PATCH 20/28] refactor: move timedate builtin to time package --- app/app.go | 4 +-- builtins/core/{typemgmt => time}/datetime.go | 2 +- .../core/{typemgmt => time}/datetime_doc.yaml | 1 + builtins/core/time/time.go | 5 +-- docs/apis/lang.IndexTemplateObject.md | 36 ++++++++++++------- docs/commands/datetime.md | 2 +- version.svg | 2 +- 7 files changed, 33 insertions(+), 19 deletions(-) rename builtins/core/{typemgmt => time}/datetime.go (99%) rename builtins/core/{typemgmt => time}/datetime_doc.yaml (99%) diff --git a/app/app.go b/app/app.go index 45d7a7a30..bb4da5e20 100644 --- a/app/app.go +++ b/app/app.go @@ -16,9 +16,9 @@ const Name = "murex" const ( Major = 6 Minor = 4 - Revision = 291 + Revision = 292 Branch = "develop" - BuildDate = "2024-10-03 23:56:20" + BuildDate = "2024-10-10 23:47:06" ) // Copyright is the copyright owner string diff --git a/builtins/core/typemgmt/datetime.go b/builtins/core/time/datetime.go similarity index 99% rename from builtins/core/typemgmt/datetime.go rename to builtins/core/time/datetime.go index bbf0e4cee..44878f597 100644 --- a/builtins/core/typemgmt/datetime.go +++ b/builtins/core/time/datetime.go @@ -1,4 +1,4 @@ -package typemgmt +package time import ( "errors" diff --git a/builtins/core/typemgmt/datetime_doc.yaml b/builtins/core/time/datetime_doc.yaml similarity index 99% rename from builtins/core/typemgmt/datetime_doc.yaml rename to builtins/core/time/datetime_doc.yaml index 1e0ae1e4f..0710d68a9 100644 --- a/builtins/core/typemgmt/datetime_doc.yaml +++ b/builtins/core/time/datetime_doc.yaml @@ -24,6 +24,7 @@ ``` -> datetime --in "format" --out "format" -> ``` + Examples: |- ### Output current date and time diff --git a/builtins/core/time/time.go b/builtins/core/time/time.go index ce0212aa7..4c1e5053e 100644 --- a/builtins/core/time/time.go +++ b/builtins/core/time/time.go @@ -16,7 +16,7 @@ func cmdTime(p *lang.Process) (err error) { p.Stdout.SetDataType(types.Integer) if p.Parameters.Len() == 0 { - return errors.New("Missing parameters") + return errors.New("missing parameters") } block := p.Parameters.StringAll() @@ -27,7 +27,8 @@ func cmdTime(p *lang.Process) (err error) { return } - s := types.FloatToString(time.Now().Sub(start).Seconds()) + s := types.FloatToString(time.Since(start).Seconds()) + _, err = p.Stderr.Write([]byte(s)) return } diff --git a/docs/apis/lang.IndexTemplateObject.md b/docs/apis/lang.IndexTemplateObject.md index 85910f460..31b637af1 100644 --- a/docs/apis/lang.IndexTemplateObject.md +++ b/docs/apis/lang.IndexTemplateObject.md @@ -147,18 +147,30 @@ func itoIndex(p *Process, params []string, object *interface{}, marshaller func( } } else { - - switch { - case v[params[i]] != nil: - obj = v[params[i]] - case v[strings.Title(params[i])] != nil: - obj = v[strings.Title(params[i])] - case v[strings.ToLower(params[i])] != nil: - obj = v[strings.ToLower(params[i])] - case v[strings.ToUpper(params[i])] != nil: - obj = v[strings.ToUpper(params[i])] - default: - return errors.New("key '" + params[i] + "' not found") + var ( + iString int + key string + ok bool + ) + + for { + switch iString { + case 0: + key = params[i] + case 1: + key = strings.Title(params[i]) + case 2: + key = strings.ToLower(params[i]) + case 3: + key = strings.ToUpper(params[i]) + default: + return errors.New("key '" + params[i] + "' not found") + } + obj, ok = v[key] + if ok { + break + } + iString++ } } diff --git a/docs/commands/datetime.md b/docs/commands/datetime.md index a7d5e4bdb..643613ad7 100644 --- a/docs/commands/datetime.md +++ b/docs/commands/datetime.md @@ -96,4 +96,4 @@ required.
-This document was generated from [builtins/core/typemgmt/datetime_doc.yaml](https://github.com/lmorg/murex/blob/master/builtins/core/typemgmt/datetime_doc.yaml). \ No newline at end of file +This document was generated from [builtins/core/time/datetime_doc.yaml](https://github.com/lmorg/murex/blob/master/builtins/core/time/datetime_doc.yaml). \ No newline at end of file diff --git a/version.svg b/version.svg index 5ba9492d0..2f89e7621 100644 --- a/version.svg +++ b/version.svg @@ -1 +1 @@ -Version: 6.4.0291Version6.4.0291 +Version: 6.4.0292Version6.4.0292 From e11da27775127ef60063f4d2f76fd3a698a52f9a Mon Sep 17 00:00:00 2001 From: Laurence Morgan Date: Fri, 11 Oct 2024 11:08:32 +0100 Subject: [PATCH 21/28] support null values in index --- app/app.go | 4 +- docs/apis/lang.IndexTemplateObject.md | 464 +++++++++++--------------- lang/define_index_objects.go | 446 +++++++++++-------------- version.svg | 2 +- 4 files changed, 406 insertions(+), 510 deletions(-) diff --git a/app/app.go b/app/app.go index bb4da5e20..3dd7f77a1 100644 --- a/app/app.go +++ b/app/app.go @@ -16,9 +16,9 @@ const Name = "murex" const ( Major = 6 Minor = 4 - Revision = 292 + Revision = 303 Branch = "develop" - BuildDate = "2024-10-10 23:47:06" + BuildDate = "2024-10-11 16:04:49" ) // Copyright is the copyright owner string diff --git a/docs/apis/lang.IndexTemplateObject.md b/docs/apis/lang.IndexTemplateObject.md index 31b637af1..3641b560e 100644 --- a/docs/apis/lang.IndexTemplateObject.md +++ b/docs/apis/lang.IndexTemplateObject.md @@ -63,9 +63,13 @@ import ( "github.com/lmorg/murex/lang/types" ) +type indexValueT interface { + ~string | ~bool | ~int | ~float64 | any +} + // IndexTemplateObject is a handy standard indexer you can use in your custom data types for structured object types. // The point of this is to minimize code rewriting and standardising the behavior of the indexer. -func IndexTemplateObject(p *Process, params []string, object *interface{}, marshaller func(interface{}) ([]byte, error)) error { +func IndexTemplateObject(p *Process, params []string, object *any, marshaller func(any) ([]byte, error)) error { if p.IsNot { return itoNot(p, params, object, marshaller) } @@ -73,309 +77,247 @@ func IndexTemplateObject(p *Process, params []string, object *interface{}, marsh } // itoIndex allow -func itoIndex(p *Process, params []string, object *interface{}, marshaller func(interface{}) ([]byte, error)) error { - var objArray []interface{} +func itoIndex(p *Process, params []string, object *any, marshaller func(any) ([]byte, error)) error { switch v := (*object).(type) { - case []interface{}: - for _, key := range params { - i, err := strconv.Atoi(key) - if err != nil { - return err - } - if i < 0 { - //i = len(v) + i - i += len(v) - } - if i >= len(v) { - return errors.New("key '" + key + "' greater than number of items in array") - } + case []any: + return itoIndexArray(p, params, v, marshaller) - if len(params) > 1 { - objArray = append(objArray, v[i]) + case map[string]any: + return itoIndexMap(p, params, v, marshaller) + case map[any]any: + return itoIndexMap(p, params, v, marshaller) - } else { - switch v[i].(type) { - case nil: - p.Stdout.SetDataType(types.Null) - case bool: - p.Stdout.SetDataType(types.Boolean) - if v[i].(bool) { - p.Stdout.Write(types.TrueByte) - } else { - p.Stdout.Write(types.FalseByte) - } - case int: - p.Stdout.SetDataType(types.Integer) - s := strconv.Itoa(v[i].(int)) - p.Stdout.Write([]byte(s)) - case float64: - p.Stdout.SetDataType(types.Number) - s := types.FloatToString(v[i].(float64)) - p.Stdout.Write([]byte(s)) - case string: - p.Stdout.SetDataType(types.String) - p.Stdout.Write([]byte(v[i].(string))) - default: - b, err := marshaller(v[i]) - if err != nil { - return err - } - p.Stdout.Writeln(b) - } - } - } - if len(objArray) > 0 { - b, err := marshaller(objArray) - if err != nil { - return err - } - p.Stdout.Writeln(b) - } - return nil - - case map[string]interface{}: - var ( - obj interface{} - err error - ) - - for i := range params { - if len(params[i]) > 2 && params[i][0] == '[' && params[i][len(params[i])-1] == ']' { - obj, err = ElementLookup(v, params[i][1:len(params[i])-1]) - if err != nil { - return err - } - - } else { - var ( - iString int - key string - ok bool - ) - - for { - switch iString { - case 0: - key = params[i] - case 1: - key = strings.Title(params[i]) - case 2: - key = strings.ToLower(params[i]) - case 3: - key = strings.ToUpper(params[i]) - default: - return errors.New("key '" + params[i] + "' not found") - } - obj, ok = v[key] - if ok { - break - } - iString++ - } - } - - if len(params) > 1 { - objArray = append(objArray, obj) + default: + return errors.New("object cannot be indexed") + } +} - } else { - switch obj := obj.(type) { - case nil: - p.Stdout.SetDataType(types.Null) - case bool: - p.Stdout.SetDataType(types.Boolean) - if obj { - p.Stdout.Write(types.TrueByte) - } else { - p.Stdout.Write(types.FalseByte) - } - case int: - p.Stdout.SetDataType(types.Integer) - s := strconv.Itoa(obj) - p.Stdout.Write([]byte(s)) - case float64: - p.Stdout.SetDataType(types.Number) - s := types.FloatToString(obj) - p.Stdout.Write([]byte(s)) - case string: - p.Stdout.SetDataType(types.String) - p.Stdout.Write([]byte(obj)) - default: - b, err := marshaller(obj) - if err != nil { - return err - } - p.Stdout.Writeln(b) - } - } +func itoIndexArray[V indexValueT](p *Process, params []string, v []V, marshaller func(any) ([]byte, error)) error { + var objArray []V + for _, key := range params { + i, err := strconv.Atoi(key) + if err != nil { + return err } - if len(objArray) > 0 { - b, err := marshaller(objArray) - if err != nil { - return err - } - p.Stdout.Writeln(b) + if i < 0 { + i += len(v) + } + if i >= len(v) { + return fmt.Errorf("key '%s' greater than number of items in array", key) } - return nil - - case map[interface{}]interface{}: - for i := range params { - //if v[key] == nil { - // return errors.New("key '" + key + "' not found.") - //} - switch { - case v[params[i]] != nil: - case v[strings.Title(params[i])] != nil: - params[i] = strings.Title(params[i]) - case v[strings.ToLower(params[i])] != nil: - params[i] = strings.ToLower(params[i]) - case v[strings.ToUpper(params[i])] != nil: - params[i] = strings.ToUpper(params[i]) - //case v[strings.ToTitle(params[i])] != nil: - // params[i] = strings.ToTitle(params[i]) - default: - return errors.New("key '" + params[i] + "' not found") - } - if len(params) > 1 { - objArray = append(objArray, v[params[i]]) + if len(params) > 1 { + objArray = append(objArray, v[i]) + continue + } + switch value := any(v[i]).(type) { + case nil: + p.Stdout.SetDataType(types.Null) + case bool: + p.Stdout.SetDataType(types.Boolean) + if value { + p.Stdout.Write(types.TrueByte) } else { - switch v[params[i]].(type) { - case nil: - p.Stdout.SetDataType(types.Null) - case bool: - p.Stdout.SetDataType(types.Boolean) - if v[params[i]].(bool) { - p.Stdout.Write(types.TrueByte) - } else { - p.Stdout.Write(types.FalseByte) - } - case int: - p.Stdout.SetDataType(types.Integer) - s := strconv.Itoa(v[params[i]].(int)) - p.Stdout.Write([]byte(s)) - case float64: - p.Stdout.SetDataType(types.Number) - s := types.FloatToString(v[params[i]].(float64)) - p.Stdout.Write([]byte(s)) - case string: - p.Stdout.SetDataType(types.String) - p.Stdout.Write([]byte(v[params[i]].(string))) - default: - b, err := marshaller(v[params[i]]) - if err != nil { - return err - } - p.Stdout.Writeln(b) - } + p.Stdout.Write(types.FalseByte) } - } - if len(objArray) > 0 { - b, err := marshaller(objArray) + case int: + p.Stdout.SetDataType(types.Integer) + s := strconv.Itoa(value) + p.Stdout.Write([]byte(s)) + case float64: + p.Stdout.SetDataType(types.Number) + s := types.FloatToString(value) + p.Stdout.Write([]byte(s)) + case string: + p.Stdout.SetDataType(types.String) + p.Stdout.Write([]byte(value)) + default: + b, err := marshaller(value) if err != nil { return err } p.Stdout.Writeln(b) } - return nil - - default: - return errors.New("object cannot be indexed") } + if len(objArray) > 0 { + b, err := marshaller(objArray) + if err != nil { + return err + } + p.Stdout.Writeln(b) + } + return nil } -// itoNot requires the indexes to be explicit -func itoNot(p *Process, params []string, object *interface{}, marshaller func(interface{}) ([]byte, error)) error { - switch v := (*object).(type) { - case []interface{}: - var objArray []interface{} - not := make(map[int]bool) - for _, key := range params { - i, err := strconv.Atoi(key) +func itoIndexMap[K comparable, V indexValueT](p *Process, params []string, v map[K]V, marshaller func(any) ([]byte, error)) error { + var ( + objArray []any + obj any + err error + ) + + for i := range params { + if len(params[i]) > 2 && params[i][0] == '[' && params[i][len(params[i])-1] == ']' { + obj, err = ElementLookup(v, params[i][1:len(params[i])-1]) if err != nil { return err } - if i < 0 { - return errors.New("cannot have negative keys in array") - } - if i >= len(v) { - return errors.New("Key '" + key + "' greater than number of items in array") - } - not[i] = true - } + } else { + var ( + iString int + key any + ok bool + ) + + for { + switch iString { + case 0: + key = params[i] + case 1: + key = strings.Title(params[i]) + case 2: + key = strings.ToLower(params[i]) + case 3: + key = strings.ToUpper(params[i]) + default: + return fmt.Errorf("key '%s' not found", params[i]) + } - for i := range v { - if !not[i] { - objArray = append(objArray, v[i]) + obj, ok = v[key.(K)] + if ok { + break + } + iString++ } } - //if len(objArray) > 0 { + if len(params) > 1 { + objArray = append(objArray, obj) + + } else { + switch obj := obj.(type) { + case nil: + p.Stdout.SetDataType(types.Null) + case bool: + p.Stdout.SetDataType(types.Boolean) + if obj { + p.Stdout.Write(types.TrueByte) + } else { + p.Stdout.Write(types.FalseByte) + } + case int: + p.Stdout.SetDataType(types.Integer) + s := strconv.Itoa(obj) + p.Stdout.Write([]byte(s)) + case float64: + p.Stdout.SetDataType(types.Number) + s := types.FloatToString(obj) + p.Stdout.Write([]byte(s)) + case string: + p.Stdout.SetDataType(types.String) + p.Stdout.Write([]byte(obj)) + default: + b, err := marshaller(obj) + if err != nil { + return err + } + p.Stdout.Writeln(b) + } + } + } + if len(objArray) > 0 { b, err := marshaller(objArray) if err != nil { return err } - _, err = p.Stdout.Writeln(b) - //} - return err + p.Stdout.Writeln(b) + } + return nil - case map[string]interface{}: - objMap := make(map[string]interface{}) - not := make(map[string]bool) - for _, key := range params { - not[key] = true - not[strings.Title(key)] = true - not[strings.ToLower(key)] = true - not[strings.ToUpper(key)] = true - //not[strings.ToTitle(key)] = true - } +} - for s := range v { - if !not[s] { - objMap[s] = v[s] - } - } +// itoNot requires the indexes to be explicit +func itoNot(p *Process, params []string, object *any, marshaller func(any) ([]byte, error)) error { + switch v := (*object).(type) { + case []any: + return itoNotArray(p, params, v, marshaller) + + case map[string]any: + return itoNotMap(p, params, v, marshaller) + case map[any]any: + return itoNotMap(p, params, v, marshaller) + + default: + return errors.New("object cannot be !indexed") + } +} - //if len(objMap) > 0 { - b, err := marshaller(objMap) +func itoNotArray[V indexValueT](p *Process, params []string, v []V, marshaller func(any) ([]byte, error)) error { + var objArray []any + not := make(map[int]bool) + for _, key := range params { + i, err := strconv.Atoi(key) if err != nil { return err } - p.Stdout.Writeln(b) - //} - return nil - - case map[interface{}]interface{}: - objMap := make(map[interface{}]interface{}) - not := make(map[string]bool) - for _, key := range params { - not[key] = true - not[strings.Title(key)] = true - not[strings.ToLower(key)] = true - not[strings.ToUpper(key)] = true - //not[strings.ToTitle(key)] = true + if i < 0 { + return errors.New("cannot have negative keys in array") } - - for iface := range v { - s := fmt.Sprint(iface) - if !not[s] { - objMap[iface] = v[iface] - } + if i >= len(v) { + return fmt.Errorf("key '%s' greater than number of items in array", key) } - //if len(objMap) > 0 { - b, err := marshaller(objMap) - if err != nil { - return err + not[i] = true + } + + for i := range v { + if !not[i] { + objArray = append(objArray, v[i]) } - _, err = p.Stdout.Writeln(b) - //} + } + + b, err := marshaller(objArray) + if err != nil { return err + } + _, err = p.Stdout.Writeln(b) - default: - return errors.New("object cannot be !indexed") + return err +} + +func itoNotMap[K comparable, V indexValueT](p *Process, params []string, v map[K]V, marshaller func(any) ([]byte, error)) error { + objMap := make(map[K]any) + not := make(map[K]bool) + var key any + + for _, key = range params { + not[key.(K)] = true + + key = strings.Title(key.(string)) + not[key.(K)] = true + + key = strings.ToLower(key.(string)) + not[key.(K)] = true + + key = strings.ToUpper(key.(string)) + not[key.(K)] = true + } + + for s := range v { + if !not[s] { + objMap[s] = v[s] + } } + + b, err := marshaller(objMap) + if err != nil { + return err + } + p.Stdout.Writeln(b) + + return nil } ``` diff --git a/lang/define_index_objects.go b/lang/define_index_objects.go index bd245cbf5..26a8c312a 100644 --- a/lang/define_index_objects.go +++ b/lang/define_index_objects.go @@ -9,9 +9,13 @@ import ( "github.com/lmorg/murex/lang/types" ) +type indexValueT interface { + ~string | ~bool | ~int | ~float64 | any +} + // IndexTemplateObject is a handy standard indexer you can use in your custom data types for structured object types. // The point of this is to minimize code rewriting and standardising the behavior of the indexer. -func IndexTemplateObject(p *Process, params []string, object *interface{}, marshaller func(interface{}) ([]byte, error)) error { +func IndexTemplateObject(p *Process, params []string, object *any, marshaller func(any) ([]byte, error)) error { if p.IsNot { return itoNot(p, params, object, marshaller) } @@ -19,295 +23,245 @@ func IndexTemplateObject(p *Process, params []string, object *interface{}, marsh } // itoIndex allow -func itoIndex(p *Process, params []string, object *interface{}, marshaller func(interface{}) ([]byte, error)) error { - var objArray []interface{} +func itoIndex(p *Process, params []string, object *any, marshaller func(any) ([]byte, error)) error { switch v := (*object).(type) { - case []interface{}: - for _, key := range params { - i, err := strconv.Atoi(key) - if err != nil { - return err - } - if i < 0 { - //i = len(v) + i - i += len(v) - } - if i >= len(v) { - return errors.New("key '" + key + "' greater than number of items in array") - } + case []any: + return itoIndexArray(p, params, v, marshaller) + + case map[string]any: + return itoIndexMap(p, params, v, marshaller) + case map[any]any: + return itoIndexMap(p, params, v, marshaller) + + default: + return errors.New("object cannot be indexed") + } +} + +func itoIndexArray[V indexValueT](p *Process, params []string, v []V, marshaller func(any) ([]byte, error)) error { + var objArray []V + for _, key := range params { + i, err := strconv.Atoi(key) + if err != nil { + return err + } + if i < 0 { + i += len(v) + } + if i >= len(v) { + return fmt.Errorf("key '%s' greater than number of items in array", key) + } - if len(params) > 1 { - objArray = append(objArray, v[i]) + if len(params) > 1 { + objArray = append(objArray, v[i]) + continue + } + switch value := any(v[i]).(type) { + case nil: + p.Stdout.SetDataType(types.Null) + case bool: + p.Stdout.SetDataType(types.Boolean) + if value { + p.Stdout.Write(types.TrueByte) } else { - switch v[i].(type) { - case nil: - p.Stdout.SetDataType(types.Null) - case bool: - p.Stdout.SetDataType(types.Boolean) - if v[i].(bool) { - p.Stdout.Write(types.TrueByte) - } else { - p.Stdout.Write(types.FalseByte) - } - case int: - p.Stdout.SetDataType(types.Integer) - s := strconv.Itoa(v[i].(int)) - p.Stdout.Write([]byte(s)) - case float64: - p.Stdout.SetDataType(types.Number) - s := types.FloatToString(v[i].(float64)) - p.Stdout.Write([]byte(s)) - case string: - p.Stdout.SetDataType(types.String) - p.Stdout.Write([]byte(v[i].(string))) - default: - b, err := marshaller(v[i]) - if err != nil { - return err - } - p.Stdout.Writeln(b) - } + p.Stdout.Write(types.FalseByte) } - } - if len(objArray) > 0 { - b, err := marshaller(objArray) + case int: + p.Stdout.SetDataType(types.Integer) + s := strconv.Itoa(value) + p.Stdout.Write([]byte(s)) + case float64: + p.Stdout.SetDataType(types.Number) + s := types.FloatToString(value) + p.Stdout.Write([]byte(s)) + case string: + p.Stdout.SetDataType(types.String) + p.Stdout.Write([]byte(value)) + default: + b, err := marshaller(value) if err != nil { return err } p.Stdout.Writeln(b) } - return nil + } + if len(objArray) > 0 { + b, err := marshaller(objArray) + if err != nil { + return err + } + p.Stdout.Writeln(b) + } + return nil +} - case map[string]interface{}: - var ( - obj interface{} - err error - ) +func itoIndexMap[K comparable, V indexValueT](p *Process, params []string, v map[K]V, marshaller func(any) ([]byte, error)) error { + var ( + objArray []any + obj any + err error + ) - for i := range params { - if len(params[i]) > 2 && params[i][0] == '[' && params[i][len(params[i])-1] == ']' { - obj, err = ElementLookup(v, params[i][1:len(params[i])-1]) - if err != nil { - return err - } + for i := range params { + if len(params[i]) > 2 && params[i][0] == '[' && params[i][len(params[i])-1] == ']' { + obj, err = ElementLookup(v, params[i][1:len(params[i])-1]) + if err != nil { + return err + } - } else { + } else { + var ( + iString int + key any + ok bool + ) - switch { - case v[params[i]] != nil: - obj = v[params[i]] - case v[strings.Title(params[i])] != nil: - obj = v[strings.Title(params[i])] - case v[strings.ToLower(params[i])] != nil: - obj = v[strings.ToLower(params[i])] - case v[strings.ToUpper(params[i])] != nil: - obj = v[strings.ToUpper(params[i])] + for { + switch iString { + case 0: + key = params[i] + case 1: + key = strings.Title(params[i]) + case 2: + key = strings.ToLower(params[i]) + case 3: + key = strings.ToUpper(params[i]) default: - return errors.New("key '" + params[i] + "' not found") + return fmt.Errorf("key '%s' not found", params[i]) } - } - if len(params) > 1 { - objArray = append(objArray, obj) - - } else { - switch obj := obj.(type) { - case nil: - p.Stdout.SetDataType(types.Null) - case bool: - p.Stdout.SetDataType(types.Boolean) - if obj { - p.Stdout.Write(types.TrueByte) - } else { - p.Stdout.Write(types.FalseByte) - } - case int: - p.Stdout.SetDataType(types.Integer) - s := strconv.Itoa(obj) - p.Stdout.Write([]byte(s)) - case float64: - p.Stdout.SetDataType(types.Number) - s := types.FloatToString(obj) - p.Stdout.Write([]byte(s)) - case string: - p.Stdout.SetDataType(types.String) - p.Stdout.Write([]byte(obj)) - default: - b, err := marshaller(obj) - if err != nil { - return err - } - p.Stdout.Writeln(b) + obj, ok = v[key.(K)] + if ok { + break } + iString++ } } - if len(objArray) > 0 { - b, err := marshaller(objArray) - if err != nil { - return err - } - p.Stdout.Writeln(b) - } - return nil - - case map[interface{}]interface{}: - for i := range params { - //if v[key] == nil { - // return errors.New("key '" + key + "' not found.") - //} - switch { - case v[params[i]] != nil: - case v[strings.Title(params[i])] != nil: - params[i] = strings.Title(params[i]) - case v[strings.ToLower(params[i])] != nil: - params[i] = strings.ToLower(params[i]) - case v[strings.ToUpper(params[i])] != nil: - params[i] = strings.ToUpper(params[i]) - //case v[strings.ToTitle(params[i])] != nil: - // params[i] = strings.ToTitle(params[i]) - default: - return errors.New("key '" + params[i] + "' not found") - } - if len(params) > 1 { - objArray = append(objArray, v[params[i]]) + if len(params) > 1 { + objArray = append(objArray, obj) - } else { - switch v[params[i]].(type) { - case nil: - p.Stdout.SetDataType(types.Null) - case bool: - p.Stdout.SetDataType(types.Boolean) - if v[params[i]].(bool) { - p.Stdout.Write(types.TrueByte) - } else { - p.Stdout.Write(types.FalseByte) - } - case int: - p.Stdout.SetDataType(types.Integer) - s := strconv.Itoa(v[params[i]].(int)) - p.Stdout.Write([]byte(s)) - case float64: - p.Stdout.SetDataType(types.Number) - s := types.FloatToString(v[params[i]].(float64)) - p.Stdout.Write([]byte(s)) - case string: - p.Stdout.SetDataType(types.String) - p.Stdout.Write([]byte(v[params[i]].(string))) - default: - b, err := marshaller(v[params[i]]) - if err != nil { - return err - } - p.Stdout.Writeln(b) + } else { + switch obj := obj.(type) { + case nil: + p.Stdout.SetDataType(types.Null) + case bool: + p.Stdout.SetDataType(types.Boolean) + if obj { + p.Stdout.Write(types.TrueByte) + } else { + p.Stdout.Write(types.FalseByte) } + case int: + p.Stdout.SetDataType(types.Integer) + s := strconv.Itoa(obj) + p.Stdout.Write([]byte(s)) + case float64: + p.Stdout.SetDataType(types.Number) + s := types.FloatToString(obj) + p.Stdout.Write([]byte(s)) + case string: + p.Stdout.SetDataType(types.String) + p.Stdout.Write([]byte(obj)) + default: + b, err := marshaller(obj) + if err != nil { + return err + } + p.Stdout.Writeln(b) } } - if len(objArray) > 0 { - b, err := marshaller(objArray) - if err != nil { - return err - } - p.Stdout.Writeln(b) + } + if len(objArray) > 0 { + b, err := marshaller(objArray) + if err != nil { + return err } - return nil - - default: - return errors.New("object cannot be indexed") + p.Stdout.Writeln(b) } + return nil + } // itoNot requires the indexes to be explicit -func itoNot(p *Process, params []string, object *interface{}, marshaller func(interface{}) ([]byte, error)) error { +func itoNot(p *Process, params []string, object *any, marshaller func(any) ([]byte, error)) error { switch v := (*object).(type) { - case []interface{}: - var objArray []interface{} - not := make(map[int]bool) - for _, key := range params { - i, err := strconv.Atoi(key) - if err != nil { - return err - } - if i < 0 { - return errors.New("cannot have negative keys in array") - } - if i >= len(v) { - return errors.New("Key '" + key + "' greater than number of items in array") - } + case []any: + return itoNotArray(p, params, v, marshaller) - not[i] = true - } + case map[string]any: + return itoNotMap(p, params, v, marshaller) + case map[any]any: + return itoNotMap(p, params, v, marshaller) - for i := range v { - if !not[i] { - objArray = append(objArray, v[i]) - } - } + default: + return errors.New("object cannot be !indexed") + } +} - //if len(objArray) > 0 { - b, err := marshaller(objArray) +func itoNotArray[V indexValueT](p *Process, params []string, v []V, marshaller func(any) ([]byte, error)) error { + var objArray []any + not := make(map[int]bool) + for _, key := range params { + i, err := strconv.Atoi(key) if err != nil { return err } - _, err = p.Stdout.Writeln(b) - //} - return err - - case map[string]interface{}: - objMap := make(map[string]interface{}) - not := make(map[string]bool) - for _, key := range params { - not[key] = true - not[strings.Title(key)] = true - not[strings.ToLower(key)] = true - not[strings.ToUpper(key)] = true - //not[strings.ToTitle(key)] = true + if i < 0 { + return errors.New("cannot have negative keys in array") } - - for s := range v { - if !not[s] { - objMap[s] = v[s] - } + if i >= len(v) { + return fmt.Errorf("key '%s' greater than number of items in array", key) } - //if len(objMap) > 0 { - b, err := marshaller(objMap) - if err != nil { - return err - } - p.Stdout.Writeln(b) - //} - return nil - - case map[interface{}]interface{}: - objMap := make(map[interface{}]interface{}) - not := make(map[string]bool) - for _, key := range params { - not[key] = true - not[strings.Title(key)] = true - not[strings.ToLower(key)] = true - not[strings.ToUpper(key)] = true - //not[strings.ToTitle(key)] = true - } + not[i] = true + } - for iface := range v { - s := fmt.Sprint(iface) - if !not[s] { - objMap[iface] = v[iface] - } + for i := range v { + if !not[i] { + objArray = append(objArray, v[i]) } + } - //if len(objMap) > 0 { - b, err := marshaller(objMap) - if err != nil { - return err - } - _, err = p.Stdout.Writeln(b) - //} + b, err := marshaller(objArray) + if err != nil { return err + } + _, err = p.Stdout.Writeln(b) - default: - return errors.New("object cannot be !indexed") + return err +} + +func itoNotMap[K comparable, V indexValueT](p *Process, params []string, v map[K]V, marshaller func(any) ([]byte, error)) error { + objMap := make(map[K]any) + not := make(map[K]bool) + var key any + + for _, key = range params { + not[key.(K)] = true + + key = strings.Title(key.(string)) + not[key.(K)] = true + + key = strings.ToLower(key.(string)) + not[key.(K)] = true + + key = strings.ToUpper(key.(string)) + not[key.(K)] = true } + + for s := range v { + if !not[s] { + objMap[s] = v[s] + } + } + + b, err := marshaller(objMap) + if err != nil { + return err + } + p.Stdout.Writeln(b) + + return nil } diff --git a/version.svg b/version.svg index 2f89e7621..97731124f 100644 --- a/version.svg +++ b/version.svg @@ -1 +1 @@ -Version: 6.4.0292Version6.4.0292 +Version: 6.4.0303Version6.4.0303 From b215b50c4a1df475652a6ff88ad3ae505d9483a7 Mon Sep 17 00:00:00 2001 From: Laurence Morgan Date: Tue, 15 Oct 2024 19:19:02 +0100 Subject: [PATCH 22/28] #888 optional parameters with named parameters --- app/app.go | 4 +- builtins/core/structs/function.go | 2 +- lang/functions.go | 96 ++++++++---- lang/functions_test.go | 243 ++++++++++++++++++++++++++---- lang/privates.go | 2 +- version.svg | 2 +- 6 files changed, 292 insertions(+), 57 deletions(-) diff --git a/app/app.go b/app/app.go index 3dd7f77a1..8fdacee6e 100644 --- a/app/app.go +++ b/app/app.go @@ -16,9 +16,9 @@ const Name = "murex" const ( Major = 6 Minor = 4 - Revision = 303 + Revision = 309 Branch = "develop" - BuildDate = "2024-10-11 16:04:49" + BuildDate = "2024-10-15 19:19:06" ) // Copyright is the copyright owner string diff --git a/builtins/core/structs/function.go b/builtins/core/structs/function.go index b75fc13a4..0aa6e03d8 100644 --- a/builtins/core/structs/function.go +++ b/builtins/core/structs/function.go @@ -110,7 +110,7 @@ func cmdUnalias(p *lang.Process) error { } func cmdFunc(p *lang.Process) error { - var dtParamsT []lang.MxFunctionParams + var dtParamsT []lang.MurexFuncParam name, err := p.Parameters.String(0) if err != nil { diff --git a/lang/functions.go b/lang/functions.go index 51279b6c0..b2a1f4bee 100644 --- a/lang/functions.go +++ b/lang/functions.go @@ -21,15 +21,17 @@ type MurexFuncs struct { type murexFuncDetails struct { Block []rune Summary string - Parameters []MxFunctionParams + Parameters []MurexFuncParam FileRef *ref.File } -type MxFunctionParams struct { +type MurexFuncParam struct { Name string DataType string Description string Default string + HasDefault bool + Optional bool } // NewMurexFuncs creates a new table of murex functions @@ -98,6 +100,7 @@ const ( // function parameter error messages fpeEofDefaultRead = "missing closing square bracket on default %d (%d,%d)" fpeParameterNoName = "parameter %d is missing a name" fpeParameterNoDataType = "parameter %d is missing a data type" + fpeUnexpectedMandatory = "mandatory parameters (%d) cannot follow optional parameters (%d)" ) const ( // function parameter contexts @@ -113,11 +116,13 @@ const ( // function parameter contexts ) // Parse the function parameter and data type block -func ParseMxFunctionParameters(parameters string) ([]MxFunctionParams, error) { - /* function example ( - name: str [Bob] "User name", - age: num [100] "How old are you?" - ) {}*/ +func ParseMxFunctionParameters(parameters string) ([]MurexFuncParam, error) { + /* + function example ( + name: str [Bob] "User name", + age: num [100] "How old are you?" + ) { ... } + */ var ( context int @@ -125,7 +130,7 @@ func ParseMxFunctionParameters(parameters string) ([]MxFunctionParams, error) { x, y = 0, 1 ) - mfp := make([]MxFunctionParams, 1) + mfp := make([]MurexFuncParam, 1) for i, r := range parameters { x++ @@ -190,6 +195,7 @@ func ParseMxFunctionParameters(parameters string) ([]MxFunctionParams, error) { mfp[counter].Default += "[" case fpcDescStart, fpcDescEnd: context = fpcDefaultRead + mfp[counter].HasDefault = true } case ']': @@ -210,17 +216,30 @@ func ParseMxFunctionParameters(parameters string) ([]MxFunctionParams, error) { mfp[counter].Default += "," case fpcNameRead: mfp[counter].DataType = types.String - mfp = append(mfp, MxFunctionParams{}) + mfp = append(mfp, MurexFuncParam{}) counter++ context = fpcNameStart case fpcTypeRead, fpcDescEnd, fpcDefaultEnd: - mfp = append(mfp, MxFunctionParams{}) + mfp = append(mfp, MurexFuncParam{}) counter++ context = fpcNameStart default: return nil, fmt.Errorf(fpeUnexpectedComma, i+1, y, x) } + case '!': + switch context { + case fpcNameStart: + context++ + mfp[counter].Optional = true + case fpcDescRead: + mfp[counter].Description += string([]rune{r}) + case fpcDefaultRead: + mfp[counter].Default += string([]rune{r}) + default: + return nil, fmt.Errorf(fpeUnexpectedCharacter, string([]rune{r}), r, i+1, y, x) + } + default: if (r >= 'a' && 'z' >= r) || (r >= 'A' && 'Z' >= r) || @@ -274,6 +293,7 @@ func ParseMxFunctionParameters(parameters string) ([]MxFunctionParams, error) { return nil, fmt.Errorf(fpeEofDefaultRead, len(parameters), y, x) } + var optional bool for i := range mfp { if mfp[i].Name == "" { return nil, fmt.Errorf(fpeParameterNoName, i+1) @@ -281,6 +301,11 @@ func ParseMxFunctionParameters(parameters string) ([]MxFunctionParams, error) { if mfp[i].DataType == "" { return nil, fmt.Errorf(fpeParameterNoDataType, i+1) } + if mfp[i].Optional { + optional = true + } else if optional { + return nil, fmt.Errorf(fpeUnexpectedMandatory, i+1, i) + } } return mfp, nil @@ -294,31 +319,26 @@ func (mfd *murexFuncDetails) castParameters(p *Process) error { return fmt.Errorf("cannot prompt for parameters when a function is run in the background: %s", err.Error()) } - prompt := mfd.Parameters[i].Description - if prompt == "" { - prompt = "Please enter a value for '" + mfd.Parameters[i].Name + "'" - } - if len(mfd.Parameters[i].Default) > 0 { - prompt += " [" + mfd.Parameters[i].Default + "]" + if mfd.Parameters[i].Optional { + if mfd.Parameters[i].HasDefault { + s = mfd.Parameters[i].Default + goto convertType + } + continue } - rl := readline.NewInstance() - rl.SetPrompt(prompt + ": ") - rl.History = new(readline.NullHistory) - s, err = rl.Readline() + s, err = mfd.Parameters[i].promptParameters() if err != nil { return err } - - if s == "" { - s = mfd.Parameters[i].Default - } } + convertType: v, err := types.ConvertGoType(s, mfd.Parameters[i].DataType) if err != nil { return fmt.Errorf("cannot convert parameter %d '%s' to data type '%s'", i+1, s, mfd.Parameters[i].DataType) } + err = p.Variables.Set(p, mfd.Parameters[i].Name, v, mfd.Parameters[i].DataType) if err != nil { return fmt.Errorf("cannot set function variable: %s", err.Error()) @@ -328,8 +348,32 @@ func (mfd *murexFuncDetails) castParameters(p *Process) error { return nil } +func (mfp *MurexFuncParam) promptParameters() (string, error) { + prompt := mfp.Description + if prompt == "" { + prompt = fmt.Sprintf("Please enter a value for '%s'", mfp.Name) + } + if len(mfp.Default) > 0 { + prompt += fmt.Sprintf(" [%s]", mfp.Default) + } + rl := readline.NewInstance() + rl.SetPrompt(prompt + ": ") + rl.History = new(readline.NullHistory) + + s, err := rl.Readline() + if err != nil { + return "", err + } + + if s == "" { + s = mfp.Default + } + + return s, nil +} + // Define creates a function -func (mf *MurexFuncs) Define(name string, parameters []MxFunctionParams, block []rune, fileRef *ref.File) { +func (mf *MurexFuncs) Define(name string, parameters []MurexFuncParam, block []rune, fileRef *ref.File) { summary := funcSummary(block) mf.mutex.Lock() @@ -403,7 +447,7 @@ func (mf *MurexFuncs) Undefine(name string) error { func (mf *MurexFuncs) Dump() interface{} { type funcs struct { Summary string - Parameters []MxFunctionParams + Parameters []MurexFuncParam Block string FileRef *ref.File } diff --git a/lang/functions_test.go b/lang/functions_test.go index 3b5ada5f3..3b9db3930 100644 --- a/lang/functions_test.go +++ b/lang/functions_test.go @@ -10,7 +10,7 @@ import ( type testFuncParseDataTypesT struct { Parameters string Error bool - Expected []MxFunctionParams + Expected []MurexFuncParam } func testFuncParseDataTypes(t *testing.T, tests []testFuncParseDataTypesT) { @@ -43,7 +43,7 @@ func TestFuncParseDataTypes(t *testing.T) { tests := []testFuncParseDataTypesT{ { Parameters: `name, age`, - Expected: []MxFunctionParams{{ + Expected: []MurexFuncParam{{ Name: "name", DataType: "str", }, { @@ -53,7 +53,7 @@ func TestFuncParseDataTypes(t *testing.T) { }, { Parameters: `name: str, age: int`, - Expected: []MxFunctionParams{{ + Expected: []MurexFuncParam{{ Name: "name", DataType: "str", }, { @@ -63,7 +63,7 @@ func TestFuncParseDataTypes(t *testing.T) { }, { Parameters: `name: str "What is your name?", age: int "How old are you?"`, - Expected: []MxFunctionParams{{ + Expected: []MurexFuncParam{{ Name: "name", DataType: "str", Description: "What is your name?", @@ -75,81 +75,91 @@ func TestFuncParseDataTypes(t *testing.T) { }, { Parameters: `name: str [Bob], age: int [100]`, - Expected: []MxFunctionParams{{ - Name: "name", - DataType: "str", - Default: "Bob", + Expected: []MurexFuncParam{{ + Name: "name", + DataType: "str", + Default: "Bob", + HasDefault: true, }, { - Name: "age", - DataType: "int", - Default: "100", + Name: "age", + DataType: "int", + Default: "100", + HasDefault: true, }}, }, { Parameters: `name: str "What is your name?" [Bob], age: int "How old are you?" [100]`, - Expected: []MxFunctionParams{{ + Expected: []MurexFuncParam{{ Name: "name", DataType: "str", Description: "What is your name?", Default: "Bob", + HasDefault: true, }, { Name: "age", DataType: "int", Description: "How old are you?", Default: "100", + HasDefault: true, }}, }, { Parameters: `name: str [Bob]`, - Expected: []MxFunctionParams{{ - Name: "name", - DataType: "str", - Default: "Bob", + Expected: []MurexFuncParam{{ + Name: "name", + DataType: "str", + Default: "Bob", + HasDefault: true, }}, }, { Parameters: `colon: str ":" [:]`, - Expected: []MxFunctionParams{{ + Expected: []MurexFuncParam{{ Name: "colon", DataType: "str", Description: ":", Default: ":", + HasDefault: true, }}, }, { Parameters: `quote: str "'" ["]`, - Expected: []MxFunctionParams{{ + Expected: []MurexFuncParam{{ Name: "quote", DataType: "str", Description: "'", Default: "\"", + HasDefault: true, }}, }, { Parameters: `square: str "[" [[]`, - Expected: []MxFunctionParams{{ + Expected: []MurexFuncParam{{ Name: "square", DataType: "str", Description: "[", Default: "[", + HasDefault: true, }}, }, { Parameters: `square: str "]" [square]`, - Expected: []MxFunctionParams{{ + Expected: []MurexFuncParam{{ Name: "square", DataType: "str", Description: "]", Default: "square", + HasDefault: true, }}, }, { Parameters: `comma: str "," [,]`, - Expected: []MxFunctionParams{{ + Expected: []MurexFuncParam{{ Name: "comma", DataType: "str", Description: ",", Default: ",", + HasDefault: true, }}, }, } @@ -162,7 +172,7 @@ func TestFuncParseDataTypesErrorOutOfOrder(t *testing.T) { { Parameters: `foo baz [bar]`, Error: true, - /*Expected: []MxFunctionParams{{ + /*Expected: []MurexFuncParam{{ Name: "foo", DataType: "baz", Default: "bar", @@ -171,7 +181,7 @@ func TestFuncParseDataTypesErrorOutOfOrder(t *testing.T) { { Parameters: `foo "baz" [bar]`, Error: true, - /*Expected: []MxFunctionParams{{ + /*Expected: []MurexFuncParam{{ Name: "foo", Description: "baz", Default: "bar", @@ -180,7 +190,7 @@ func TestFuncParseDataTypesErrorOutOfOrder(t *testing.T) { { Parameters: `foo [bar]`, Error: true, - /*Expected: []MxFunctionParams{{ + /*Expected: []MurexFuncParam{{ Name: "foo", DataType: "str", Default: "bar", @@ -503,12 +513,13 @@ func TestFuncParseDocExamples(t *testing.T) { tests := []testFuncParseDataTypesT{ { Parameters: `var:datatype [default-value] "description"`, - Expected: []MxFunctionParams{ + Expected: []MurexFuncParam{ { Name: "var", DataType: "datatype", Default: "default-value", Description: "description", + HasDefault: true, }, }, }, @@ -517,18 +528,198 @@ func TestFuncParseDocExamples(t *testing.T) { variable1: data-type [default-value] "description", variable2: data-type [default-value] "description" `, - Expected: []MxFunctionParams{ + Expected: []MurexFuncParam{ { Name: "variable1", DataType: "data-type", Default: "default-value", Description: "description", + HasDefault: true, }, { Name: "variable2", DataType: "data-type", Default: "default-value", Description: "description", + HasDefault: true, + }, + }, + }, + } + + testFuncParseDataTypes(t, tests) +} + +func TestFuncParseOptional(t *testing.T) { + tests := []testFuncParseDataTypesT{ + { + Parameters: `!var:datatype [default-value] "description"`, + Expected: []MurexFuncParam{ + { + Name: "var", + DataType: "datatype", + Default: "default-value", + Description: "description", + HasDefault: true, + Optional: true, + }, + }, + }, + { + Parameters: ` + !variable1: data-type [default-value] "description", + !variable2: data-type [default-value] "description" + `, + Expected: []MurexFuncParam{ + { + Name: "variable1", + DataType: "data-type", + Default: "default-value", + Description: "description", + HasDefault: true, + Optional: true, + }, + { + Name: "variable2", + DataType: "data-type", + Default: "default-value", + Description: "description", + HasDefault: true, + Optional: true, + }, + }, + }, + { + Parameters: ` + variable1: data-type [default-value] "description", + !variable2: data-type [default-value] "description" + `, + Expected: []MurexFuncParam{ + { + Name: "variable1", + DataType: "data-type", + Default: "default-value", + Description: "description", + HasDefault: true, + }, + { + Name: "variable2", + DataType: "data-type", + Default: "default-value", + Description: "description", + HasDefault: true, + Optional: true, + }, + }, + }, + { + Parameters: ` + !variable1: data-type [default-value] "description", + variable2: data-type [default-value] "description" + `, + Error: true, + }, + { + Parameters: `var: !data-type [default-value] "description"`, + Error: true, + }, + { + Parameters: `var: data-!type [default-value] "description"`, + Error: true, + }, + { + Parameters: `var: data-type [!default-value] "description"`, + Expected: []MurexFuncParam{ + { + Name: "var", + DataType: "data-type", + Default: "!default-value", + Description: "description", + HasDefault: true, + }, + }, + }, + { + Parameters: `var: data-type [default-value] "!description"`, + Expected: []MurexFuncParam{ + { + Name: "var", + DataType: "data-type", + Default: "default-value", + Description: "!description", + HasDefault: true, + }, + }, + }, + } + testFuncParseDataTypes(t, tests) +} + +func TestFuncParseHasDefault(t *testing.T) { + tests := []testFuncParseDataTypesT{ + { + Parameters: `!var: data-type "description"`, + Expected: []MurexFuncParam{ + { + Name: "var", + DataType: "data-type", + Default: "", + Description: "description", + HasDefault: false, + Optional: true, + }, + }, + }, + { + Parameters: `!var: data-type`, + Expected: []MurexFuncParam{ + { + Name: "var", + DataType: "data-type", + Default: "", + Description: "", + HasDefault: false, + Optional: true, + }, + }, + }, + ///// + { + Parameters: `!var: data-type "description" []`, + Expected: []MurexFuncParam{ + { + Name: "var", + DataType: "data-type", + Default: "", + Description: "description", + HasDefault: true, + Optional: true, + }, + }, + }, + { + Parameters: `!var: data-type [] "description"`, + Expected: []MurexFuncParam{ + { + Name: "var", + DataType: "data-type", + Default: "", + Description: "description", + HasDefault: true, + Optional: true, + }, + }, + }, + { + Parameters: `!var: data-type []`, + Expected: []MurexFuncParam{ + { + Name: "var", + DataType: "data-type", + Default: "", + Description: "", + HasDefault: true, + Optional: true, }, }, }, diff --git a/lang/privates.go b/lang/privates.go index 3420918dd..c4134fafb 100644 --- a/lang/privates.go +++ b/lang/privates.go @@ -23,7 +23,7 @@ func NewMurexPrivs() *privateFunctions { return pf } -func (pf *privateFunctions) Define(name string, parameters []MxFunctionParams, block []rune, fileRef *ref.File) { +func (pf *privateFunctions) Define(name string, parameters []MurexFuncParam, block []rune, fileRef *ref.File) { pf.mutex.Lock() if pf.module[fileRef.Source.Module] == nil { diff --git a/version.svg b/version.svg index 97731124f..d7428ca0f 100644 --- a/version.svg +++ b/version.svg @@ -1 +1 @@ -Version: 6.4.0303Version6.4.0303 +Version: 6.4.0309Version6.4.0309 From ad1244bfdb995416adb597bb54d2975c65ced058 Mon Sep 17 00:00:00 2001 From: Laurence Morgan Date: Tue, 22 Oct 2024 08:16:47 +0100 Subject: [PATCH 23/28] #889 add framework for Bash-like job IDs --- app/app.go | 4 +- builtins/core/processes/bgfg.go | 4 +- builtins/core/processes/jobs.go | 36 +++-- lang/fork.go | 4 + lang/interpreter.go | 3 +- lang/interpreter_pc.go | 12 +- lang/jobs.go | 105 +++++++++++++ lang/jobs_helper_test.go | 6 + lang/jobs_test.go | 251 ++++++++++++++++++++++++++++++++ lang/process.go | 10 +- lang/process_structs.go | 32 ++-- version.svg | 2 +- 12 files changed, 417 insertions(+), 52 deletions(-) create mode 100644 lang/jobs.go create mode 100644 lang/jobs_helper_test.go create mode 100644 lang/jobs_test.go diff --git a/app/app.go b/app/app.go index 8fdacee6e..8af7e330c 100644 --- a/app/app.go +++ b/app/app.go @@ -16,9 +16,9 @@ const Name = "murex" const ( Major = 6 Minor = 4 - Revision = 309 + Revision = 321 Branch = "develop" - BuildDate = "2024-10-15 19:19:06" + BuildDate = "2024-10-22 08:16:51" ) // Copyright is the copyright owner string diff --git a/builtins/core/processes/bgfg.go b/builtins/core/processes/bgfg.go index c46ae64b3..52e2ded68 100644 --- a/builtins/core/processes/bgfg.go +++ b/builtins/core/processes/bgfg.go @@ -31,13 +31,11 @@ func cmdBackground(p *lang.Process) (err error) { p.Background.Set(true) p.WaitForTermination <- false - fork := p.Fork(lang.F_FUNCTION | lang.F_BACKGROUND) + fork := p.Fork(lang.F_FUNCTION | lang.F_BACKGROUND | lang.F_ASSIGN_JOBID) fork.Name.Set(p.Name.String()) fork.Parameters.CopyFrom(&p.Parameters) go fork.Execute(block) - //lang.GlobalFIDs.WaitOnChildState(p, state.Executing.State()) - return nil } diff --git a/builtins/core/processes/jobs.go b/builtins/core/processes/jobs.go index 5a88b20fd..4a8315cb3 100644 --- a/builtins/core/processes/jobs.go +++ b/builtins/core/processes/jobs.go @@ -2,7 +2,6 @@ package processes import ( "github.com/lmorg/murex/lang" - "github.com/lmorg/murex/lang/state" "github.com/lmorg/murex/lang/types" ) @@ -23,7 +22,8 @@ func cmdJobs(p *lang.Process) error { } b, err := lang.MarshalData(p, dtLine, []interface{}{ - "PID", + "JobID", + "FunctionID", "State", "Background", "Process", @@ -37,23 +37,21 @@ func cmdJobs(p *lang.Process) error { return err } - procs := lang.GlobalFIDs.ListAll() - for _, process := range procs { - if process.Background.Get() || process.State.Get() == state.Stopped { - b, err := lang.MarshalData(p, dtLine, []interface{}{ - process.Id, - process.State.String(), - process.Background.Get(), - process.Name.String(), - getParams(process), - }) - if err != nil { - return err - } - err = aw.Write(b) - if err != nil { - return err - } + for _, job := range lang.Jobs.List() { + b, err := lang.MarshalData(p, dtLine, []interface{}{ + job.JobId, + job.Process.Id, + job.Process.State.String(), + job.Process.Background.Get(), + job.Process.Name.String(), + getParams(job.Process), + }) + if err != nil { + return err + } + err = aw.Write(b) + if err != nil { + return err } } diff --git a/lang/fork.go b/lang/fork.go index c433e3f8a..d5a166924 100644 --- a/lang/fork.go +++ b/lang/fork.go @@ -55,6 +55,9 @@ const ( // F_BACKGROUND this process will run in the background F_BACKGROUND + // F_ASSIGN_JOBID will allow child functions to gain an %n job id for job control + F_ASSIGN_JOBID + // F_CREATE_STDIN will create a new stdin stdio.Io interface F_CREATE_STDIN @@ -106,6 +109,7 @@ func (p *Process) Fork(flags int) *Fork { fork.State.Set(state.MemAllocated) fork.Background.Set(flags&F_BACKGROUND != 0 || p.Background.Get()) + fork._hasJobId = flags&F_ASSIGN_JOBID != 0 || p._hasJobId fork.IsMethod = p.IsMethod fork.OperatorLogicAnd = p.OperatorLogicAnd diff --git a/lang/interpreter.go b/lang/interpreter.go index 6bc7cce53..51ef9d528 100644 --- a/lang/interpreter.go +++ b/lang/interpreter.go @@ -96,12 +96,13 @@ func compile(tree *[]functions.FunctionT, parent *Process) (*[]Process, int) { procs[i].OperatorLogicAnd = (*tree)[i].Properties.LogicAnd() procs[i].OperatorLogicOr = (*tree)[i].Properties.LogicOr() procs[i].Background.Set(parent.Background.Get()) + procs[i]._hasJobId = parent._hasJobId procs[i].Parent = parent procs[i].Scope = parent.Scope procs[i].WaitForTermination = make(chan bool) procs[i].WaitForStopped = make(chan bool) procs[i].HasStopped = make(chan bool) - procs[i].RunMode = rm //parent.RunMode + procs[i].RunMode = rm procs[i].Config = parent.Config procs[i].Tests = parent.Tests procs[i].Variables = parent.Variables diff --git a/lang/interpreter_pc.go b/lang/interpreter_pc.go index 403148538..0427e6c1e 100644 --- a/lang/interpreter_pc.go +++ b/lang/interpreter_pc.go @@ -36,9 +36,7 @@ func runModeNormal(procs *[]Process) (exitNum int) { ((*procs)[i].OperatorLogicOr && (*procs)[prev].ExitNum == 0) || (skipPipeline && ((*procs)[i].OperatorLogicAnd || (*procs)[i].OperatorLogicOr)) { - (*procs)[i].hasTerminatedM.Lock() - (*procs)[i].hasTerminatedV = true - (*procs)[i].hasTerminatedM.Unlock() + (*procs)[i].SetTerminatedState(true) (*procs)[i].ExitNum = (*procs)[prev].ExitNum skipPipeline = true } else { @@ -76,9 +74,7 @@ func runModeTry(procs *[]Process, tryErr bool) (exitNum int) { if next < len(*procs) { if exitNum < 1 && (*procs)[next].OperatorLogicOr { i++ - (*procs)[i].hasTerminatedM.Lock() - (*procs)[i].hasTerminatedV = true - (*procs)[i].hasTerminatedM.Unlock() + (*procs)[i].SetTerminatedState(true) (*procs)[i].Stdout.Close() (*procs)[i].Stderr.Close() GlobalFIDs.Deregister((*procs)[i].Id) @@ -125,9 +121,7 @@ func runModeTryPipe(procs *[]Process, tryPipeErr bool) (exitNum int) { if next < len(*procs) { if exitNum < 1 && (*procs)[next].OperatorLogicOr { i++ - (*procs)[i].hasTerminatedM.Lock() - (*procs)[i].hasTerminatedV = true - (*procs)[i].hasTerminatedM.Unlock() + (*procs)[i].SetTerminatedState(true) (*procs)[i].Stdout.Close() (*procs)[i].Stderr.Close() GlobalFIDs.Deregister((*procs)[i].Id) diff --git a/lang/jobs.go b/lang/jobs.go new file mode 100644 index 000000000..63a07b63e --- /dev/null +++ b/lang/jobs.go @@ -0,0 +1,105 @@ +package lang + +import ( + "fmt" + "sync" +) + +type jobs struct { + mutex sync.Mutex + jobs []*Process +} + +func NewJobs() *jobs { + return &jobs{} +} + +func (j *jobs) Add(p *Process) { + j.mutex.Lock() + j.jobs = append(j.jobs, p) + j.mutex.Unlock() +} + +func (j *jobs) GarbageCollect() { + j.mutex.Lock() + + var ( + last = -1 + running bool + ) + + for i := len(j.jobs) - 1; i >= 0; i-- { + if j.jobs[i] != nil { + if j.jobs[i].HasTerminated() { + j.jobs[i] = nil + } else { + running = true + } + } + + if j.jobs[i] == nil && !running { + last = i + } + } + + switch last { + case -1: + break + /*case 0: + j.jobs = nil*/ + default: + j.jobs = j.jobs[:last] + } + + j.mutex.Unlock() +} + +func (j *jobs) Get(jobId int) (*Process, error) { + j.mutex.Lock() + defer j.mutex.Unlock() + + if jobId < 1 { + return nil, fmt.Errorf("invalid job ID '%d': job ID cannot be less than 1", jobId) + } + + if jobId > len(j.jobs) { + return nil, fmt.Errorf("invalid job ID '%d': job ID was greater than number of jobs (%d)", jobId, len(j.jobs)) + } + + i := jobId - 1 + + if j._hasTerminated(i) { + return nil, fmt.Errorf("job '%d' has already terminated", jobId) + } + + return j.jobs[i], nil +} + +type JobT struct { + JobId string + Process *Process +} + +func (j *jobs) List() []*JobT { + var s []*JobT + + j.mutex.Lock() + + for i := range j.jobs { + if j._hasTerminated(i) { + continue + } + s = append(s, &JobT{ + JobId: fmt.Sprintf("%%%d", i+1), + Process: j.jobs[i], + }) + } + + j.mutex.Unlock() + + return s +} + +func (j *jobs) _hasTerminated(i int) bool { + return j.jobs[i] == nil || j.jobs[i].HasTerminated() +} diff --git a/lang/jobs_helper_test.go b/lang/jobs_helper_test.go new file mode 100644 index 000000000..66bf684f0 --- /dev/null +++ b/lang/jobs_helper_test.go @@ -0,0 +1,6 @@ +package lang + +// Raw is only used for testing +func (j *jobs) Raw() []*Process { + return j.jobs +} diff --git a/lang/jobs_test.go b/lang/jobs_test.go new file mode 100644 index 000000000..0306eb8e1 --- /dev/null +++ b/lang/jobs_test.go @@ -0,0 +1,251 @@ +package lang_test + +import ( + "strings" + "testing" + + "github.com/lmorg/murex/lang" + "github.com/lmorg/murex/test/count" + "github.com/lmorg/murex/utils/json" +) + +func convertProcessSliceToStringSlice(processes []*lang.Process) []string { + s := make([]string, len(processes)) + + for i, p := range processes { + switch { + case p == nil: + s[i] = "nil" + + case p.HasTerminated(): + s[i] = "terminated" + + default: + s[i] = "active" + } + } + + return s +} + +func TestJobsGarbageCollect(t *testing.T) { + tests := []struct { + PreState []*lang.Process + PostState []*lang.Process + Kill []bool + }{ + { + PreState: []*lang.Process{}, + PostState: []*lang.Process{}, + }, + { + PreState: []*lang.Process{nil}, + PostState: []*lang.Process{}, + }, + { + PreState: []*lang.Process{nil, nil, nil}, + PostState: []*lang.Process{}, + }, + ///// + { + PreState: []*lang.Process{ + lang.NewTestProcess(), + lang.NewTestProcess(), + nil, + }, + PostState: []*lang.Process{ + lang.NewTestProcess(), + lang.NewTestProcess(), + }, + }, + { + PreState: []*lang.Process{ + lang.NewTestProcess(), + nil, + lang.NewTestProcess(), + }, + PostState: []*lang.Process{ + lang.NewTestProcess(), + nil, + lang.NewTestProcess(), + }, + }, + { + PreState: []*lang.Process{ + nil, + lang.NewTestProcess(), + lang.NewTestProcess(), + }, + PostState: []*lang.Process{ + nil, + lang.NewTestProcess(), + lang.NewTestProcess(), + }, + }, + ///// + { + PreState: []*lang.Process{ + lang.NewTestProcess(), + nil, + lang.NewTestProcess(), + }, + PostState: []*lang.Process{ + lang.NewTestProcess(), + }, + Kill: []bool{false, false, true}, + }, + { + PreState: []*lang.Process{ + lang.NewTestProcess(), + lang.NewTestProcess(), + lang.NewTestProcess(), + }, + PostState: []*lang.Process{}, + Kill: []bool{true, true, true}, + }, + { + PreState: []*lang.Process{ + lang.NewTestProcess(), + lang.NewTestProcess(), + lang.NewTestProcess(), + }, + PostState: []*lang.Process{ + nil, + nil, + lang.NewTestProcess(), + }, + Kill: []bool{true, true, false}, + }, + } + + count.Tests(t, len(tests)) + + for i, test := range tests { + jobs := lang.NewJobs() + for pi, p := range test.PreState { + jobs.Add(p) + if p != nil && test.Kill != nil { + p.SetTerminatedState(test.Kill[pi]) + } + } + + jobs.GarbageCollect() + + expState := convertProcessSliceToStringSlice(test.PostState) + actState := convertProcessSliceToStringSlice(jobs.Raw()) + + expJson := json.LazyLogging(expState) + actJson := json.LazyLogging(actState) + + if expJson != actJson { + + t.Errorf("Unexpected state in test %d:", i) + t.Logf("PreState: %s", json.LazyLogging(convertProcessSliceToStringSlice(test.PreState))) + t.Logf("Expected: %s", expJson) + t.Logf("Actual: %s", actJson) + } + } +} + +func TestJobsList(t *testing.T) { + reformat := func(s string) string { + s = strings.ReplaceAll(s, `\u003c`, "<") + return strings.ReplaceAll(s, `\u003e`, ">") + } + + tests := []struct { + PreState []*lang.Process + Count int + Kill []bool + }{ + { + PreState: []*lang.Process{}, + Count: 0, + }, + { + PreState: []*lang.Process{nil}, + Count: 0, + }, + { + PreState: []*lang.Process{nil, nil, nil}, + Count: 0, + }, + ///// + { + PreState: []*lang.Process{ + lang.NewTestProcess(), + lang.NewTestProcess(), + nil, + }, + Count: 2, + }, + { + PreState: []*lang.Process{ + lang.NewTestProcess(), + nil, + lang.NewTestProcess(), + }, + Count: 2, + }, + { + PreState: []*lang.Process{ + nil, + lang.NewTestProcess(), + lang.NewTestProcess(), + }, + Count: 2, + }, + ///// + { + PreState: []*lang.Process{ + lang.NewTestProcess(), + nil, + lang.NewTestProcess(), + }, + Kill: []bool{false, false, true}, + Count: 1, + }, + { + PreState: []*lang.Process{ + lang.NewTestProcess(), + lang.NewTestProcess(), + lang.NewTestProcess(), + }, + Kill: []bool{true, true, true}, + Count: 0, + }, + { + PreState: []*lang.Process{ + lang.NewTestProcess(), + lang.NewTestProcess(), + lang.NewTestProcess(), + }, + Kill: []bool{true, true, false}, + Count: 1, + }, + } + + count.Tests(t, len(tests)) + + for i, test := range tests { + jobs := lang.NewJobs() + for pi, p := range test.PreState { + jobs.Add(p) + if p != nil && test.Kill != nil { + p.SetTerminatedState(test.Kill[pi]) + } + } + + list := jobs.List() + + actJson := json.LazyLogging(list) + + if len(list) != test.Count { + t.Errorf("Unexpected state in test %d:", i) + t.Logf("PreState: %s", json.LazyLogging(convertProcessSliceToStringSlice(test.PreState))) + t.Logf("Expected: %d", test.Count) + t.Logf("Actual: %d", len(list)) + t.Logf("act json: %s", reformat(actJson)) + } + } +} diff --git a/lang/process.go b/lang/process.go index 1e27e006f..1dd51d64b 100644 --- a/lang/process.go +++ b/lang/process.go @@ -50,7 +50,7 @@ var ( // GlobalAliases is a table of global aliases GlobalAliases = NewAliases() - // GlobalPipes is a table of named pipes + // GlobalPipes is a table of named pipes GlobalPipes = pipes.NewNamed() // GlobalFIDs is a table of running murex processes @@ -62,6 +62,9 @@ var ( // ForegroundProc is the murex FID which currently has "focus" ForegroundProc = newForegroundProc() + // Jobs is used for job control to store processes manually backgrounded + Jobs = NewJobs() + // ShellExitNum is for when running murex in interactive shell mode ShellExitNum int ) @@ -275,7 +278,9 @@ executeProcess: ansititle.Tmux([]byte(name)) } - //ansititle.Write([]byte(name)) + } + if p._hasJobId { + Jobs.Add(p) } // execution mode: @@ -446,5 +451,6 @@ func deregisterProcess(p *Process) { go func() { p.State.Set(state.AwaitingGC) GlobalFIDs.Deregister(p.Id) + Jobs.GarbageCollect() }() } diff --git a/lang/process_structs.go b/lang/process_structs.go index 6ca2ba4c4..8099e5a10 100644 --- a/lang/process_structs.go +++ b/lang/process_structs.go @@ -56,6 +56,7 @@ type Process struct { hasTerminatedV bool State state.State Background process.Background + _hasJobId bool RunMode runmode.RunMode Config *config.Config Tests *Tests @@ -73,21 +74,21 @@ type Process struct { func (p *Process) Dump() map[string]any { return map[string]any{ - "Id": p.Id, - "Cache": p.cache != nil && p.cache.use, - "Raw": string(p.raw), - "Name": p.Name.String(), - "Parameters": p.Parameters.Dump(), - "NamedPipes": p.namedPipes, - "Stdin": statsToStruct(p.Stdin), - "Stdout": statsToStruct(p.Stdout), - "Stderr": statsToStruct(p.Stderr), - "stdoutOldPtr": statsToStruct(p.stdoutOldPtr), - "ExitNum": p.ExitNum, - "Forks": p.Forks.GetForks(), - "IsFork": p.IsFork, - //"Done": p.Done, - //"Kill": p.Kill, + "Id": p.Id, + "Cache": p.cache != nil && p.cache.use, + "Raw": string(p.raw), + "Name": p.Name.String(), + "Parameters": p.Parameters.Dump(), + "NamedPipes": p.namedPipes, + "Stdin": statsToStruct(p.Stdin), + "Stdout": statsToStruct(p.Stdout), + "Stderr": statsToStruct(p.Stderr), + "stdoutOldPtr": statsToStruct(p.stdoutOldPtr), + "ExitNum": p.ExitNum, + "Forks": p.Forks.GetForks(), + "IsFork": p.IsFork, + "Done": p.cache != nil, + "Kill": p.cache != nil, "SysProcId": _jsonfySysProc(p), "ScopeFid": p.Scope.Id, "ParentFid": p.Parent.Id, @@ -105,6 +106,7 @@ func (p *Process) Dump() map[string]any { //"HasStopped": p.hasCancelledStopped(), "State": p.State.String(), "Background": p.Background.Get(), + "HasJobId": p._hasJobId, "RunMode": p.RunMode.String(), "RunModeStrict": p.RunMode.IsStrict(), //Config *config.Config diff --git a/version.svg b/version.svg index d7428ca0f..616758a96 100644 --- a/version.svg +++ b/version.svg @@ -1 +1 @@ -Version: 6.4.0309Version6.4.0309 +Version: 6.4.0321Version6.4.0321 From 2164bcbf9a410b110e22207189a8906236b5dd1c Mon Sep 17 00:00:00 2001 From: Laurence Morgan Date: Thu, 24 Oct 2024 17:46:39 +0100 Subject: [PATCH 24/28] #889 smarter parameter handling for `fg` & `bg` + new job control routines --- app/app.go | 4 +- builtins/core/processes/bgfg_unix.go | 48 +++++++++++++++++----- builtins/core/processes/fid-list.go | 14 +++---- builtins/types/columns/marshal.go | 4 +- lang/fork.go | 2 +- lang/interpreter.go | 2 +- lang/jobs.go | 35 +++++++++++++++- lang/process.go | 6 ++- lang/process/background.go | 16 ++++---- lang/process/background_test.go | 4 +- lang/process_structs.go | 6 +-- shell/session/session_unix.go | 14 ------- shell/signal_handler/sigfns/sigfns_unix.go | 5 +++ version.svg | 2 +- 14 files changed, 105 insertions(+), 57 deletions(-) diff --git a/app/app.go b/app/app.go index 8af7e330c..97aeb4745 100644 --- a/app/app.go +++ b/app/app.go @@ -16,9 +16,9 @@ const Name = "murex" const ( Major = 6 Minor = 4 - Revision = 321 + Revision = 338 Branch = "develop" - BuildDate = "2024-10-22 08:16:51" + BuildDate = "2024-10-24 17:46:44" ) // Copyright is the copyright owner string diff --git a/builtins/core/processes/bgfg_unix.go b/builtins/core/processes/bgfg_unix.go index 960eb5d76..308b7ac7c 100644 --- a/builtins/core/processes/bgfg_unix.go +++ b/builtins/core/processes/bgfg_unix.go @@ -5,6 +5,10 @@ package processes import ( "errors" + "fmt" + "regexp" + "strconv" + "strings" "syscall" "github.com/lmorg/murex/debug" @@ -13,13 +17,39 @@ import ( "github.com/lmorg/murex/lang/types" ) -func mkbg(p *lang.Process) error { - fid, err := p.Parameters.Uint32(0) - if err != nil { - return errors.New("invalid parameters. Expecting either a code block or FID of a stopped process") +var rxJobId = regexp.MustCompile(`^%[0-9]+$`) + +func getProcess(s string) (*lang.Process, error) { + s = strings.TrimSpace(s) + + // empty command. Get latest + if len(s) == 0 { + return lang.Jobs.GetLatest() + } + + // is a number, likely function ID + i, err := strconv.Atoi(s) + if err == nil { + return lang.GlobalFIDs.Proc(uint32(i)) + } + + // has a % prefix, likely a job ID + if rxJobId.MatchString(s) { + i, err = strconv.Atoi(s[1:]) + if err != nil { + return nil, fmt.Errorf("cannot get job ID from '%s': %s", s, err.Error()) + } + + return lang.Jobs.Get(i) } - f, err := lang.GlobalFIDs.Proc(fid) + // lets just match it to the most recent command + return lang.Jobs.GetFromCommandLine(s) +} + +func mkbg(p *lang.Process) error { + s := p.Parameters.StringAll() + f, err := getProcess(s) if err != nil { return err } @@ -47,12 +77,8 @@ func mkbg(p *lang.Process) error { func cmdForeground(p *lang.Process) error { p.Stdout.SetDataType(types.Null) - fid, err := p.Parameters.Uint32(0) - if err != nil { - return err - } - - f, err := lang.GlobalFIDs.Proc(fid) + s := p.Parameters.StringAll() + f, err := getProcess(s) if err != nil { return err } diff --git a/builtins/core/processes/fid-list.go b/builtins/core/processes/fid-list.go index 13f77fe2b..085b9ceb1 100644 --- a/builtins/core/processes/fid-list.go +++ b/builtins/core/processes/fid-list.go @@ -28,15 +28,11 @@ func init() { } func getParams(p *lang.Process) string { - params := p.Parameters.StringArray() - if len(params) == 0 && len(p.Parameters.PreParsed) > 0 { - params = make([]string, len(p.Parameters.PreParsed)) - for i := range p.Parameters.PreParsed { - params[i] = string(p.Parameters.PreParsed[i]) - } - } - - return strings.Join(params, " ") + s := string(p.Parameters.GetRaw()) + s = strings.ReplaceAll(s, "\r", "") + s = strings.ReplaceAll(s, "\n", "") + s = strings.TrimSpace(s) + return s } func cmdFidList(p *lang.Process) error { diff --git a/builtins/types/columns/marshal.go b/builtins/types/columns/marshal.go index 684b3396d..ac876ac89 100644 --- a/builtins/types/columns/marshal.go +++ b/builtins/types/columns/marshal.go @@ -1,6 +1,7 @@ package columns import ( + "bytes" "fmt" "github.com/lmorg/murex/lang" @@ -11,6 +12,7 @@ func marshal(_ *lang.Process, iface interface{}) (b []byte, err error) { switch v := iface.(type) { case []string: marshalSliceStr(v, b) + b = bytes.TrimRight(b, "\t") return /*case [][]string: @@ -23,6 +25,7 @@ func marshal(_ *lang.Process, iface interface{}) (b []byte, err error) { for i := range v { b = append(b, iface2str(&v[i])...) } + b = bytes.TrimRight(b, "\t") return /*case map[string]string: @@ -62,7 +65,6 @@ func marshalSliceStr(a []string, b []byte) { func iface2str(v *interface{}) (b []byte) { return []byte(fmt.Sprintf("%v\t", *v)) - } /*func unmarshal(p *lang.Process) (interface{}, error) { diff --git a/lang/fork.go b/lang/fork.go index d5a166924..c0fa9ac2c 100644 --- a/lang/fork.go +++ b/lang/fork.go @@ -109,7 +109,7 @@ func (p *Process) Fork(flags int) *Fork { fork.State.Set(state.MemAllocated) fork.Background.Set(flags&F_BACKGROUND != 0 || p.Background.Get()) - fork._hasJobId = flags&F_ASSIGN_JOBID != 0 || p._hasJobId + fork.HasJobId.Set(flags&F_ASSIGN_JOBID != 0 || p.HasJobId.Get()) fork.IsMethod = p.IsMethod fork.OperatorLogicAnd = p.OperatorLogicAnd diff --git a/lang/interpreter.go b/lang/interpreter.go index 51ef9d528..93e468750 100644 --- a/lang/interpreter.go +++ b/lang/interpreter.go @@ -96,7 +96,7 @@ func compile(tree *[]functions.FunctionT, parent *Process) (*[]Process, int) { procs[i].OperatorLogicAnd = (*tree)[i].Properties.LogicAnd() procs[i].OperatorLogicOr = (*tree)[i].Properties.LogicOr() procs[i].Background.Set(parent.Background.Get()) - procs[i]._hasJobId = parent._hasJobId + procs[i].HasJobId.Set(parent.HasJobId.Get()) procs[i].Parent = parent procs[i].Scope = parent.Scope procs[i].WaitForTermination = make(chan bool) diff --git a/lang/jobs.go b/lang/jobs.go index 63a07b63e..fd9f66820 100644 --- a/lang/jobs.go +++ b/lang/jobs.go @@ -1,7 +1,9 @@ package lang import ( + "errors" "fmt" + "strings" "sync" ) @@ -45,8 +47,6 @@ func (j *jobs) GarbageCollect() { switch last { case -1: break - /*case 0: - j.jobs = nil*/ default: j.jobs = j.jobs[:last] } @@ -75,6 +75,37 @@ func (j *jobs) Get(jobId int) (*Process, error) { return j.jobs[i], nil } +func (j *jobs) GetLatest() (*Process, error) { + j.mutex.Lock() + defer j.mutex.Unlock() + + for i := len(j.jobs) - 1; i >= 0; i-- { + if j._hasTerminated(i) { + continue + } + return j.jobs[i], nil + } + + return nil, errors.New("no running commands are in job control") +} + +func (j *jobs) GetFromCommandLine(s string) (*Process, error) { + j.mutex.Lock() + defer j.mutex.Unlock() + + for i := len(j.jobs) - 1; i >= 0; i-- { + if j._hasTerminated(i) { + continue + } + if strings.Contains(j.jobs[i].Name.String(), s) || + strings.Contains(string(j.jobs[i].Parameters.GetRaw()), s) { + return j.jobs[i], nil + } + } + + return nil, fmt.Errorf("no command in job control contains the string '%s'", s) +} + type JobT struct { JobId string Process *Process diff --git a/lang/process.go b/lang/process.go index 1dd51d64b..9bf50e670 100644 --- a/lang/process.go +++ b/lang/process.go @@ -279,7 +279,7 @@ executeProcess: } } - if p._hasJobId { + if p.HasJobId.Get() { Jobs.Add(p) } @@ -451,6 +451,8 @@ func deregisterProcess(p *Process) { go func() { p.State.Set(state.AwaitingGC) GlobalFIDs.Deregister(p.Id) - Jobs.GarbageCollect() + if p.HasJobId.Get() { + Jobs.GarbageCollect() + } }() } diff --git a/lang/process/background.go b/lang/process/background.go index 883ea2120..43ff3a492 100644 --- a/lang/process/background.go +++ b/lang/process/background.go @@ -2,25 +2,25 @@ package process import "sync/atomic" -type Background struct { +type AtomicBool struct { i int32 } -func (bg *Background) Get() bool { - return atomic.LoadInt32(&bg.i) != 0 +func (ab *AtomicBool) Get() bool { + return atomic.LoadInt32(&ab.i) != 0 } -func (bg *Background) Set(v bool) { +func (ab *AtomicBool) Set(v bool) { if v { - atomic.StoreInt32(&bg.i, 1) + atomic.StoreInt32(&ab.i, 1) return } - atomic.StoreInt32(&bg.i, 0) + atomic.StoreInt32(&ab.i, 0) } -func (bg *Background) String() string { - if bg.Get() { +func (ab *AtomicBool) String() string { + if ab.Get() { return "yes" } diff --git a/lang/process/background_test.go b/lang/process/background_test.go index a46b00d73..012d7bd20 100644 --- a/lang/process/background_test.go +++ b/lang/process/background_test.go @@ -7,10 +7,10 @@ import ( "github.com/lmorg/murex/test/count" ) -func TestBackground(t *testing.T) { +func TestAtomicBoolean(t *testing.T) { count.Tests(t, 6) - bg := new(process.Background) + bg := new(process.AtomicBool) bg.Set(false) if bg.Get() { diff --git a/lang/process_structs.go b/lang/process_structs.go index 8099e5a10..02181a351 100644 --- a/lang/process_structs.go +++ b/lang/process_structs.go @@ -55,8 +55,8 @@ type Process struct { hasTerminatedM sync.Mutex `json:"-"` hasTerminatedV bool State state.State - Background process.Background - _hasJobId bool + Background process.AtomicBool + HasJobId process.AtomicBool RunMode runmode.RunMode Config *config.Config Tests *Tests @@ -106,7 +106,7 @@ func (p *Process) Dump() map[string]any { //"HasStopped": p.hasCancelledStopped(), "State": p.State.String(), "Background": p.Background.Get(), - "HasJobId": p._hasJobId, + "HasJobId": p.HasJobId.Get(), "RunMode": p.RunMode.String(), "RunModeStrict": p.RunMode.IsStrict(), //Config *config.Config diff --git a/shell/session/session_unix.go b/shell/session/session_unix.go index 69ef3be25..93431ab55 100644 --- a/shell/session/session_unix.go +++ b/shell/session/session_unix.go @@ -46,24 +46,10 @@ func UnixCreateSession() { debug.Log("!!! Entering UnixSetSid()") var err error - //pid := os.Getpid() - - // create a new group - /*err = unix.Setpgid(pid, os.Getpid()) - if err != nil { - debug.Logf("!!! UnixSetSid()->unix.Setpgid():1 failed: %s", err.Error()) - }*/ // Create a new session unixSid, err = unix.Setsid() if err != nil { debug.Logf("!!! UnixSetSid()->syscall.Setsid():1 failed: %s", err.Error()) } - - /*pgid, err := unix.Getpgid(pid) - debug.Logf("pid: %d, ppid: %d, pgid: %d, err: %v", pid, os.Getppid(), pgid, err) - pgrp := syscall.Getpgrp() - debug.Logf("pid: %d, ppid: %d, pgid: %d, err: --", pid, os.Getppid(), pgrp) - sid, err := syscall.Getsid(pid) - debug.Logf("pid: %d, ppid: %d, sid: %d, err: %v", pid, os.Getppid(), sid, err)*/ } diff --git a/shell/signal_handler/sigfns/sigfns_unix.go b/shell/signal_handler/sigfns/sigfns_unix.go index 9a0fc5f3a..5b1af0aca 100644 --- a/shell/signal_handler/sigfns/sigfns_unix.go +++ b/shell/signal_handler/sigfns/sigfns_unix.go @@ -43,6 +43,11 @@ func returnFromSigtstp(p *lang.Process) { stopStatus(p) } + if !p.HasJobId.Get() { + p.HasJobId.Set(true) + lang.Jobs.Add(p) + } + lang.ShowPrompt <- true p.HasStopped <- true diff --git a/version.svg b/version.svg index 616758a96..c5d0138d2 100644 --- a/version.svg +++ b/version.svg @@ -1 +1 @@ -Version: 6.4.0321Version6.4.0321 +Version: 6.4.0338Version6.4.0338 From 9fb0ab53f7c32556fc54f66b977048d2f1563b64 Mon Sep 17 00:00:00 2001 From: Laurence Morgan Date: Wed, 6 Nov 2024 21:45:55 +0000 Subject: [PATCH 25/28] add support for mxtty csv --- app/app.go | 4 ++-- builtins/pipes/term/out.go | 18 ++++++++++++++++++ builtins/types/csv/marshal.go | 16 +--------------- shell/shell.go | 1 + utils/readline/write.go | 1 + version.svg | 2 +- 6 files changed, 24 insertions(+), 18 deletions(-) diff --git a/app/app.go b/app/app.go index 97aeb4745..3ef0a7d12 100644 --- a/app/app.go +++ b/app/app.go @@ -16,9 +16,9 @@ const Name = "murex" const ( Major = 6 Minor = 4 - Revision = 338 + Revision = 349 Branch = "develop" - BuildDate = "2024-10-24 17:46:44" + BuildDate = "2024-11-06 21:45:58" ) // Copyright is the copyright owner string diff --git a/builtins/pipes/term/out.go b/builtins/pipes/term/out.go index 8da6cf968..8248f224f 100644 --- a/builtins/pipes/term/out.go +++ b/builtins/pipes/term/out.go @@ -15,6 +15,7 @@ import ( // Out is the Stdout interface for term type Out struct { term + dataType string } func (t *Out) File() *os.File { @@ -51,6 +52,23 @@ func (t *Out) SetDataType(dt string) { OutSetDataTypeIPC = false f.Close()*/ + + if os.Getenv("MXTTY") != "true" { + return + } + + switch t.dataType { + case "csv": + t.File().WriteString("\x1b_end;csv\x1b\\") + } + + t.dataType = dt + + switch t.dataType { + case "csv": + t.File().WriteString("\x1b_begin;csv;{\"Headings\":true}\x1b\\") + } + } // Write is the io.Writer() interface for term diff --git a/builtins/types/csv/marshal.go b/builtins/types/csv/marshal.go index 963de09b1..f29ff8aa9 100644 --- a/builtins/types/csv/marshal.go +++ b/builtins/types/csv/marshal.go @@ -5,7 +5,6 @@ import ( enc "encoding/csv" "fmt" "io" - "os" "reflect" "github.com/lmorg/murex/lang" @@ -70,20 +69,7 @@ func marshal(p *lang.Process, iface interface{}) ([]byte, error) { w.Flush() err = w.Error() - if err != nil { - return buf.Bytes(), err - } - - var table []byte - if p.Stdout.IsTTY() && os.Getenv("MXTTY") == "true" { - table = []byte("\x1b_begin;table;{\"Format\":\"csv\"}\x1b\\") - } - table = append(table, buf.Bytes()...) - if p.Stdout.IsTTY() && os.Getenv("MXTTY") == "true" { - table = append(table, []byte("\x1b_end;table\x1b\\")...) - } - - return table, nil + return buf.Bytes(), err } func unmarshal(p *lang.Process) (interface{}, error) { diff --git a/shell/shell.go b/shell/shell.go index 73a237bf7..4ced680d4 100644 --- a/shell/shell.go +++ b/shell/shell.go @@ -282,6 +282,7 @@ func showPrompt() { fork.CCEvent = lang.ShellProcess.CCEvent fork.CCExists = lang.ShellProcess.CCExists lang.ShellExitNum, err = fork.Execute(expanded) + lang.ShellProcess.Stdout.SetDataType("") // reset data type if err != nil { fmt.Fprintln(os.Stdout, ansi.ExpandConsts(fmt.Sprintf("{RED}%v{RESET}", err))) diff --git a/utils/readline/write.go b/utils/readline/write.go index dac973a09..cdf0d106f 100644 --- a/utils/readline/write.go +++ b/utils/readline/write.go @@ -211,5 +211,6 @@ func (rl *Instance) updateHelpersStr() string { } output := rl.clearHelpersStr() output += rl.renderHelpersStr() + return output } diff --git a/version.svg b/version.svg index c5d0138d2..8588dc3c7 100644 --- a/version.svg +++ b/version.svg @@ -1 +1 @@ -Version: 6.4.0338Version6.4.0338 +Version: 6.4.0349Version6.4.0349 From c3a236d4236c52a833faa189048eb7ca371581df Mon Sep 17 00:00:00 2001 From: Laurence Morgan Date: Wed, 6 Nov 2024 22:43:17 +0000 Subject: [PATCH 26/28] #889 smarter handling of raw parameters and `exec` function names --- app/app.go | 4 ++-- builtins/core/processes/fid-list.go | 4 ++-- lang/expressions/parse_block.go | 4 +++- lang/jobs.go | 3 +-- lang/parameters/tokens.go | 5 ++++- lang/process.go | 1 + lang/process_structs.go | 15 +++++++++++++++ version.svg | 2 +- 8 files changed, 29 insertions(+), 9 deletions(-) diff --git a/app/app.go b/app/app.go index 3ef0a7d12..fd9822bb3 100644 --- a/app/app.go +++ b/app/app.go @@ -16,9 +16,9 @@ const Name = "murex" const ( Major = 6 Minor = 4 - Revision = 349 + Revision = 366 Branch = "develop" - BuildDate = "2024-11-06 21:45:58" + BuildDate = "2024-11-06 22:58:43" ) // Copyright is the copyright owner string diff --git a/builtins/core/processes/fid-list.go b/builtins/core/processes/fid-list.go index 085b9ceb1..7bc14d343 100644 --- a/builtins/core/processes/fid-list.go +++ b/builtins/core/processes/fid-list.go @@ -28,9 +28,9 @@ func init() { } func getParams(p *lang.Process) string { - s := string(p.Parameters.GetRaw()) + s := string(p.GetRawParameters()) s = strings.ReplaceAll(s, "\r", "") - s = strings.ReplaceAll(s, "\n", "") + s = strings.ReplaceAll(s, "\n", " ") s = strings.TrimSpace(s) return s } diff --git a/lang/expressions/parse_block.go b/lang/expressions/parse_block.go index c19010cb9..7394a8ce0 100644 --- a/lang/expressions/parse_block.go +++ b/lang/expressions/parse_block.go @@ -40,7 +40,9 @@ func ExpressionParser(expression []rune, offset int, exec bool) (int, error) { // way of parsing function parameters func StatementParametersParser(expression []rune, p *lang.Process) (string, []string, error) { if p.Name.String() == lang.ExpressionFunctionName { - return p.Name.String(), []string{string(p.Parameters.GetRaw())}, nil + // Added to pass TestExpressionsMultipleParams(): + // builtins/core/expressions/expressions_test.go:220 + return lang.ExpressionFunctionName, []string{string(p.Parameters.GetRawishExpression())}, nil } tree := NewParser(nil, expression, 0) diff --git a/lang/jobs.go b/lang/jobs.go index fd9f66820..fb6900ad4 100644 --- a/lang/jobs.go +++ b/lang/jobs.go @@ -97,8 +97,7 @@ func (j *jobs) GetFromCommandLine(s string) (*Process, error) { if j._hasTerminated(i) { continue } - if strings.Contains(j.jobs[i].Name.String(), s) || - strings.Contains(string(j.jobs[i].Parameters.GetRaw()), s) { + if strings.Contains(j.jobs[i].GetRaw(), s) { return j.jobs[i], nil } } diff --git a/lang/parameters/tokens.go b/lang/parameters/tokens.go index 0fdf9b896..161a8d983 100644 --- a/lang/parameters/tokens.go +++ b/lang/parameters/tokens.go @@ -30,7 +30,10 @@ func (param *Parameters) Dump() interface{} { return dump } -func (param *Parameters) GetRaw() []rune { +// GetRawishExpression should only be used for expressions. +// Added to pass TestExpressionsMultipleParams(): +// builtins/core/expressions/expressions_test.go:220 +func (param *Parameters) GetRawishExpression() []rune { var r []rune for i := range param.PreParsed { diff --git a/lang/process.go b/lang/process.go index 9bf50e670..ddb5ee772 100644 --- a/lang/process.go +++ b/lang/process.go @@ -343,6 +343,7 @@ executeProcess: // shell execute p.Parameters.Prepend([]string{name}) p.Name.Set("exec") + p._execAdded = true // Don't put the following here: p.State.Set(state.Executing) // That should be done in the `exec` builtin instead err = GoFunctions["exec"](p) diff --git a/lang/process_structs.go b/lang/process_structs.go index 02181a351..119109a45 100644 --- a/lang/process_structs.go +++ b/lang/process_structs.go @@ -3,6 +3,7 @@ package lang import ( "context" "fmt" + "strings" "sync" "time" @@ -23,6 +24,7 @@ type Process struct { Id uint32 cache *cacheT raw []rune + _execAdded bool Name process.Name Parameters parameters.Parameters namedPipes []string @@ -124,6 +126,19 @@ func (p *Process) Dump() map[string]any { } } +func (p *Process) GetRaw() string { + return string(p.raw) +} + +func (p *Process) GetRawParameters() string { + if p._execAdded { + return string(p.raw) + } + + params := string(p.raw[len(p.Name.String()):]) + return string(strings.TrimLeft(params, " ")) +} + func _jsonfySysProc(p *Process) any { if !p.SystemProcess.External() { return nil diff --git a/version.svg b/version.svg index 8588dc3c7..99ba2b724 100644 --- a/version.svg +++ b/version.svg @@ -1 +1 @@ -Version: 6.4.0349Version6.4.0349 +Version: 6.4.0366Version6.4.0366 From 59b5468bf29bae77d952c0b7cf2528cbab7f048f Mon Sep 17 00:00:00 2001 From: Laurence Morgan Date: Fri, 8 Nov 2024 20:42:16 +0000 Subject: [PATCH 27/28] #889 core: raw parameters missing cmd + whitespaces --- app/app.go | 4 ++-- builtins/core/processes/fid-list.go | 2 +- lang/expressions/parse_block.go | 2 +- lang/parameters/set.go | 3 +++ lang/parameters/tokens.go | 6 ++---- lang/process.go | 1 - lang/process_structs.go | 11 ----------- version.svg | 2 +- 8 files changed, 10 insertions(+), 21 deletions(-) diff --git a/app/app.go b/app/app.go index fd9822bb3..e4cb143d4 100644 --- a/app/app.go +++ b/app/app.go @@ -16,9 +16,9 @@ const Name = "murex" const ( Major = 6 Minor = 4 - Revision = 366 + Revision = 372 Branch = "develop" - BuildDate = "2024-11-06 22:58:43" + BuildDate = "2024-11-08 20:42:19" ) // Copyright is the copyright owner string diff --git a/builtins/core/processes/fid-list.go b/builtins/core/processes/fid-list.go index 7bc14d343..e1c151d8a 100644 --- a/builtins/core/processes/fid-list.go +++ b/builtins/core/processes/fid-list.go @@ -28,7 +28,7 @@ func init() { } func getParams(p *lang.Process) string { - s := string(p.GetRawParameters()) + s := string(p.Parameters.Raw()) s = strings.ReplaceAll(s, "\r", "") s = strings.ReplaceAll(s, "\n", " ") s = strings.TrimSpace(s) diff --git a/lang/expressions/parse_block.go b/lang/expressions/parse_block.go index 7394a8ce0..e543959ab 100644 --- a/lang/expressions/parse_block.go +++ b/lang/expressions/parse_block.go @@ -42,7 +42,7 @@ func StatementParametersParser(expression []rune, p *lang.Process) (string, []st if p.Name.String() == lang.ExpressionFunctionName { // Added to pass TestExpressionsMultipleParams(): // builtins/core/expressions/expressions_test.go:220 - return lang.ExpressionFunctionName, []string{string(p.Parameters.GetRawishExpression())}, nil + return lang.ExpressionFunctionName, []string{string(p.Parameters.Raw())}, nil } tree := NewParser(nil, expression, 0) diff --git a/lang/parameters/set.go b/lang/parameters/set.go index d483404e4..342f53bb1 100644 --- a/lang/parameters/set.go +++ b/lang/parameters/set.go @@ -11,6 +11,9 @@ func (p *Parameters) DefineParsed(params []string) { func (p *Parameters) Prepend(params []string) { p.mutex.Lock() p.params = append(params, p.params...) + for i := range params { + p.PreParsed = append([][]rune{[]rune(params[i])}, p.PreParsed...) + } p.mutex.Unlock() } diff --git a/lang/parameters/tokens.go b/lang/parameters/tokens.go index 161a8d983..792fe0c65 100644 --- a/lang/parameters/tokens.go +++ b/lang/parameters/tokens.go @@ -30,13 +30,11 @@ func (param *Parameters) Dump() interface{} { return dump } -// GetRawishExpression should only be used for expressions. -// Added to pass TestExpressionsMultipleParams(): -// builtins/core/expressions/expressions_test.go:220 -func (param *Parameters) GetRawishExpression() []rune { +func (param *Parameters) Raw() []rune { var r []rune for i := range param.PreParsed { + r = append(r, ' ') r = append(r, param.PreParsed[i]...) } diff --git a/lang/process.go b/lang/process.go index ddb5ee772..9bf50e670 100644 --- a/lang/process.go +++ b/lang/process.go @@ -343,7 +343,6 @@ executeProcess: // shell execute p.Parameters.Prepend([]string{name}) p.Name.Set("exec") - p._execAdded = true // Don't put the following here: p.State.Set(state.Executing) // That should be done in the `exec` builtin instead err = GoFunctions["exec"](p) diff --git a/lang/process_structs.go b/lang/process_structs.go index 119109a45..cd2420c93 100644 --- a/lang/process_structs.go +++ b/lang/process_structs.go @@ -3,7 +3,6 @@ package lang import ( "context" "fmt" - "strings" "sync" "time" @@ -24,7 +23,6 @@ type Process struct { Id uint32 cache *cacheT raw []rune - _execAdded bool Name process.Name Parameters parameters.Parameters namedPipes []string @@ -130,15 +128,6 @@ func (p *Process) GetRaw() string { return string(p.raw) } -func (p *Process) GetRawParameters() string { - if p._execAdded { - return string(p.raw) - } - - params := string(p.raw[len(p.Name.String()):]) - return string(strings.TrimLeft(params, " ")) -} - func _jsonfySysProc(p *Process) any { if !p.SystemProcess.External() { return nil diff --git a/version.svg b/version.svg index 99ba2b724..88e8ec103 100644 --- a/version.svg +++ b/version.svg @@ -1 +1 @@ -Version: 6.4.0366Version6.4.0366 +Version: 6.4.0372Version6.4.0372 From 73287191b6cbafeaecea32ed70176fcfdc8f9f28 Mon Sep 17 00:00:00 2001 From: Laurence Morgan Date: Fri, 22 Nov 2024 22:18:35 +0000 Subject: [PATCH 28/28] add changelog --- app/app.go | 4 +- docs/changelog/README.md | 5 ++ docs/changelog/v6.4.md | 100 ++++++++++++++++++++++++++++++++++++ gen/changelog/v6.4.inc.md | 73 ++++++++++++++++++++++++++ gen/changelog/v6.4_doc.yaml | 17 ++++++ version.svg | 2 +- 6 files changed, 198 insertions(+), 3 deletions(-) create mode 100644 docs/changelog/v6.4.md create mode 100644 gen/changelog/v6.4.inc.md create mode 100644 gen/changelog/v6.4_doc.yaml diff --git a/app/app.go b/app/app.go index e4cb143d4..34aa9f324 100644 --- a/app/app.go +++ b/app/app.go @@ -16,9 +16,9 @@ const Name = "murex" const ( Major = 6 Minor = 4 - Revision = 372 + Revision = 373 Branch = "develop" - BuildDate = "2024-11-08 20:42:19" + BuildDate = "2024-11-22 22:18:43" ) // Copyright is the copyright owner string diff --git a/docs/changelog/README.md b/docs/changelog/README.md index 2a72d476e..2df51e7a0 100644 --- a/docs/changelog/README.md +++ b/docs/changelog/README.md @@ -4,6 +4,11 @@ Track new features, any breaking changes, and the release history here. ## Articles +### 22.11.2024 - [v6.4](../changelog/v6.4.md) + +This change brings a number of ergonomic improvements to job control, `datetime` and working with structures. + + ### 18.09.2024 - [v6.3](../changelog/v6.3.md) This is a massive release ahead of the v7.0. This brings notifications of new deprecations, new builtins, new flags, improved CI/CD flow, and changes to the website. Unfortunately it also carries 3 breaking changes. diff --git a/docs/changelog/v6.4.md b/docs/changelog/v6.4.md new file mode 100644 index 000000000..f3e753904 --- /dev/null +++ b/docs/changelog/v6.4.md @@ -0,0 +1,100 @@ +# v6.4 + +This change brings a number of ergonomic improvements to job control, `datetime` and working with structures. + +## Deprecation Warnings + +Please read out [compatibility commitment](https://murex.rocks/compatibility.html) to understand how features are deprecated. + +* the `?` pipe will be deprecated to make way for a the ternary operator. You can achieve the same result with ` `, eg `command parameters... | next-command ...` + +* the `=` and `let` builtins are now officially deprecated. They've been marked as deprecated in the documentation for a couple of years but you'll now receive a deprecation warning when using them. This warning will not impact any functions that call them (they bypass the stdout and stderr pipes and write directly to your TTY) but it is still recommended that you update any existing code not to use it. The change is very simple, Murex supported expressions as first class primitives, so you can simply drop the `=` and `let` command names from your expressions + +* `tread` has been deprecated for a while due to `read` supporting all `tread`'s use cases. `tread` will officially be removed in the next release + +* `@[]` syntax for ranging has been deprecated for a while. It will be officially removed in the next release in favour of `[]` + +* backtick strings (````) has been an undocumented hack for several years. This release officially sees that hack as deprecated and will be removed in the next release + +* `die` has been deprecated because it just adds a feature for no purpose. It was original borrowed from Perl but realistically you can do the same with `exit 1` so this removal is to bring the language complexity down. + +## Breaking Changes + +None + +## Features + +* IO redirection: smarter file pipes which solve the race condition seen in traditional shells where you try to write to the same file you're reading from (EXPERIMENTAL) ([issue #851](https://github.com/lmorg/murex/issues/851)) + +* private functions: these can now be undefined ([issue #429](https://github.com/lmorg/murex/issues/429)) + +* core: `(expressions)` are supported in dot notation. eg: + ``` + » bob = %[foo bar baz] + » out $bob.(1+1) + baz + ``` + +* elements: negative values are supported for counting backwards. eg: + ``` + » bob = %[foo bar baz] + » out $(bob.-1) + baz + ``` + +* core: support added for Bash-like job IDs (`%n`) in job control ([issue #889](https://github.com/lmorg/murex/issues/889)) + +* `bg` / `fg`: support for command line parameters, as well as Bash-like job IDs ([issue #889](https://github.com/lmorg/murex/issues/889)) + +* function: support for optional parameters when using named parameters ([issue #888](https://github.com/lmorg/murex/issues/888)) + +* autocomplete: improved autocompletion for `kill` ([issue #867](https://github.com/lmorg/murex/issues/867)) + +* `datetime`: numerous ergonomic improvements ([read more](https://murex.rocks/commands/datetime.html), [commit](https://github.com/lmorg/murex/pull/887/commits/ffc38623f07c6331a0113913c81e83fe85792d22)) + +* readline: `MaxTabCompleterRows` is now more granular ([commit](https://github.com/lmorg/murex/pull/887/commits/9ab0803731ca4c02d524cbe2fa002dee658c3a1e)) + +* mxtty: updated `csv` support ([commit](https://github.com/lmorg/murex/pull/887/commits/9fb0ab53f7c32556fc54f66b977048d2f1563b64), [read more](https://github.com/lmorg/mxtty)) + +## Bug Fixes + +* private functions: mutex added to mitigate a potential race condition ([issue #883](https://github.com/lmorg/murex/issues/883)) + +* core: line numbering bugfix ([commit](https://github.com/lmorg/murex/pull/887/commits/615f76dd891a23fe93b4d8aa5a999cc22bcd1908)) + +* readline: when in Vim mode, `a` and `A` should move the cursor right ([commit](https://github.com/lmorg/murex/pull/887/commits/4fb351c9841dc78997c56aca19232e5a94ab9ec6)) + +* index: support for null values ([commit](https://github.com/lmorg/murex/pull/887/commits/e11da27775127ef60063f4d2f76fd3a698a52f9a)) + +* core: improvements with how raw command line parameters are passed to internal routines, which fixes a number of reporting bugs in process management tools + +## Special Thanks + +Thank yous for this release goes to [tiymat](https://github.com/tiymat), [atagen](https://github.com/atagen) for your testing and feedback. + +Also thank you to everyone in the [discussions group](https://github.com/lmorg/murex/discussions) and all who raise bug reports. + +You rock! + +
+ +Published: 22.11.2024 at 21:32 + +## See Also + +* [Background Process (`bg`)](../commands/bg.md): + Run processes in the background +* [Date And Time Conversion (`datetime`)](../commands/datetime.md): + A date and/or time conversion tool (like `printf` but for date and time values) +* [Foreground Process (`fg`)](../commands/fg.md): + Sends a background process into the foreground +* [Get Nested Element (`[[ Element ]]`)](../parser/element.md): + Outputs an element from a nested structure +* [How To Contribute](../Murex/CONTRIBUTING.md): + Murex is community project. We gratefully accept contributions +* [index](../parser/item-index.md): + Outputs an element from an array, map or table + +
+ +This document was generated from [gen/changelog/v6.4_doc.yaml](https://github.com/lmorg/murex/blob/master/gen/changelog/v6.4_doc.yaml). \ No newline at end of file diff --git a/gen/changelog/v6.4.inc.md b/gen/changelog/v6.4.inc.md new file mode 100644 index 000000000..2018b21c8 --- /dev/null +++ b/gen/changelog/v6.4.inc.md @@ -0,0 +1,73 @@ +## Deprecation Warnings + +Please read out [compatibility commitment](https://murex.rocks/compatibility.html) to understand how features are deprecated. + +* the `?` pipe will be deprecated to make way for a the ternary operator. You can achieve the same result with ` `, eg `command parameters... | next-command ...` + +* the `=` and `let` builtins are now officially deprecated. They've been marked as deprecated in the documentation for a couple of years but you'll now receive a deprecation warning when using them. This warning will not impact any functions that call them (they bypass the stdout and stderr pipes and write directly to your TTY) but it is still recommended that you update any existing code not to use it. The change is very simple, Murex supported expressions as first class primitives, so you can simply drop the `=` and `let` command names from your expressions + +* `tread` has been deprecated for a while due to `read` supporting all `tread`'s use cases. `tread` will officially be removed in the next release + +* `@[]` syntax for ranging has been deprecated for a while. It will be officially removed in the next release in favour of `[]` + +* backtick strings (````) has been an undocumented hack for several years. This release officially sees that hack as deprecated and will be removed in the next release + +* `die` has been deprecated because it just adds a feature for no purpose. It was original borrowed from Perl but realistically you can do the same with `exit 1` so this removal is to bring the language complexity down. + +## Breaking Changes + +None + +## Features + +* IO redirection: smarter file pipes which solve the race condition seen in traditional shells where you try to write to the same file you're reading from (EXPERIMENTAL) ([issue #851](https://github.com/lmorg/murex/issues/851)) + +* private functions: these can now be undefined ([issue #429](https://github.com/lmorg/murex/issues/429)) + +* core: `(expressions)` are supported in dot notation. eg: + ``` + » bob = %[foo bar baz] + » out $bob.(1+1) + baz + ``` + +* elements: negative values are supported for counting backwards. eg: + ``` + » bob = %[foo bar baz] + » out $(bob.-1) + baz + ``` + +* core: support added for Bash-like job IDs (`%n`) in job control ([issue #889](https://github.com/lmorg/murex/issues/889)) + +* `bg` / `fg`: support for command line parameters, as well as Bash-like job IDs ([issue #889](https://github.com/lmorg/murex/issues/889)) + +* function: support for optional parameters when using named parameters ([issue #888](https://github.com/lmorg/murex/issues/888)) + +* autocomplete: improved autocompletion for `kill` ([issue #867](https://github.com/lmorg/murex/issues/867)) + +* `datetime`: numerous ergonomic improvements ([read more](https://murex.rocks/commands/datetime.html), [commit](https://github.com/lmorg/murex/pull/887/commits/ffc38623f07c6331a0113913c81e83fe85792d22)) + +* readline: `MaxTabCompleterRows` is now more granular ([commit](https://github.com/lmorg/murex/pull/887/commits/9ab0803731ca4c02d524cbe2fa002dee658c3a1e)) + +* mxtty: updated `csv` support ([commit](https://github.com/lmorg/murex/pull/887/commits/9fb0ab53f7c32556fc54f66b977048d2f1563b64), [read more](https://github.com/lmorg/mxtty)) + +## Bug Fixes + +* private functions: mutex added to mitigate a potential race condition ([issue #883](https://github.com/lmorg/murex/issues/883)) + +* core: line numbering bugfix ([commit](https://github.com/lmorg/murex/pull/887/commits/615f76dd891a23fe93b4d8aa5a999cc22bcd1908)) + +* readline: when in Vim mode, `a` and `A` should move the cursor right ([commit](https://github.com/lmorg/murex/pull/887/commits/4fb351c9841dc78997c56aca19232e5a94ab9ec6)) + +* index: support for null values ([commit](https://github.com/lmorg/murex/pull/887/commits/e11da27775127ef60063f4d2f76fd3a698a52f9a)) + +* core: improvements with how raw command line parameters are passed to internal routines, which fixes a number of reporting bugs in process management tools + +## Special Thanks + +Thank yous for this release goes to [tiymat](https://github.com/tiymat), [atagen](https://github.com/atagen) for your testing and feedback. + +Also thank you to everyone in the [discussions group](https://github.com/lmorg/murex/discussions) and all who raise bug reports. + +You rock! diff --git a/gen/changelog/v6.4_doc.yaml b/gen/changelog/v6.4_doc.yaml new file mode 100644 index 000000000..16034c977 --- /dev/null +++ b/gen/changelog/v6.4_doc.yaml @@ -0,0 +1,17 @@ +- DocumentID: v6.4 + Title: >- + v6.4 + CategoryID: changelog + DateTime: 2024-11-22 21:32 + Summary: >- + This change brings a number of ergonomic improvements to job control, + `datetime` and working with structures. + Description: |- + {{ include "gen/changelog/v6.4.inc.md" }} + Related: + - CONTRIBUTING + - bg + - fg + - datetime + - index + - element diff --git a/version.svg b/version.svg index 88e8ec103..a57949af6 100644 --- a/version.svg +++ b/version.svg @@ -1 +1 @@ -Version: 6.4.0372Version6.4.0372 +Version: 6.4.0373Version6.4.0373