Skip to content

Releases: viash-io/viash

Viash 0.6.4

30 Nov 12:57
Compare
Choose a tag to compare

This release adds features related to managing Viash projects and
allows for better runtime introspection of Nextflow VDSL3 modules.

The most notable changes are:

  • You can switch versions of Viash using the VIASH_VERSION
    environment variable! Example:

    VIASH_VERSION=0.6.0 viash --version

    More importantly, you can specify the version of Viash you want
    in a project config. See below for more info.

  • Introducing Viash project config files as an experimental feature.
    It allows storing project-related settings in a _viash.yaml
    config file which you should store at the root of your repository.
    Example:

    viash_version: 0.6.4
    source: src
    target: target
    config_mods: |
      .platforms[.type == 'docker'].target_registry := 'ghcr.io'
      .platforms[.type == 'docker'].target_organization := 'viash-io'
      .platforms[.type == 'docker'].namespace_separator := '/'
      .platforms[.type == 'docker'].target_image_source := 'https://github.com/viash-io/viash'
  • It's now possible to specify in which order Viash will merge
    Viash configs. Example:

    functionality:
      name: foo
      arguments:
        - __merge__: obj_input.yaml
          name: "--one"
        - __merge__: [., obj_input.yaml]
          name: "--two"
        - __merge__: [obj_input.yaml, .]
         name: "--three"

Please take note of the following breaking changes:

  • Passing non-existent paths to a Viash component will cause the
    component to generate an error when no file or folder is found.
    Set must_exist to false to revert to the previous behaviour.

  • The arguments --write_meta/-w and --meta/-m no longer exist,
    because every viash build/run/test run will generate a
    .config.vsh.yaml meta file.

BREAKING CHANGES

  • Config: Viash configs whose filenames start with a . are ignored (#291).

  • viash build: --write_meta/-m and --meta/-m arguments have been removed.
    Instead, the .config.vsh.yaml file is always created when building Viash components (#293).

  • FileArgument: Default setting of must_exist was changed from false to true.
    As such, the component will throw an error by default if an input file or output file
    is missing (#295).

  • Config merging: __inherits__ has been renamed to __merge__.

NEW FUNCTIONALITY

  • You can switch versions of Viash using the VIASH_VERSION
    environment variable (#304)! Example:

    VIASH_VERSION=0.6.0 viash --version
  • Traceability: Running viash build and viash test creates a .config.vsh.yaml file
    by default, which contains the processed config of the component. As a side effect,
    this allows for reading in the .config.vsh.yaml from within the component to learn
    more about the component being tested (#291 and #293).

  • FileArgument: Added create_parent option, which will check if the directory of an output
    file exists and create it if necessary (#295).

MINOR CHANGES

  • viash run, viash test: When running or testing a component, Viash will add an extension
    to the temporary file that is created. Before: /tmp/viash-run-wdckjnce,
    now: /tmp/viash-run-wdckjnce.py (#302).

  • NextflowPlatform: Add DataflowHelper.nf as a retrievable resource in Viash (#301).

  • NextflowPlatform: During a stubrun, argument requirements are turned off and
    the publishDir, cpus, memory, and label directives are also removed
    from the process (#301).

  • NextflowPlatform: Added a filter processing argument to filter the incoming channel after
    the map, mapData, mapId and mapPassthrough have been applied (#296).

  • NextflowPlatform: Added the Viash config to the Nextflow module for later introspection (#296).
    For example:

    include { foo } from "$targetDir/path/foo/main.nf"
    
    foo.run(filter: { tup ->
      def preferredNormalization = foo.config.functionality.info.preferred_normalization
      tup.normalization_id == preferredNormalization
    })

BUG FIXES

  • BashWrapper: Don't overwrite meta values when trailing arguments are provided (#295).

EXPERIMENTAL FEATURES

  • Viash Project: Viash will automatically search for a _viash.yaml file in the directory of
    a component and its parent directories (#294).

    Contents of _viash.yaml:

    source: src
    target: target
    config_mods: |
      .platforms[.type == 'docker'].target_registry := 'ghcr.io'
      .platforms[.type == 'docker'].target_organization := 'viash-io'
      .platforms[.type == 'docker'].namespace_separator := '/'
      .platforms[.type == 'docker'].target_image_source := 'https://github.com/viash-io/viash'
  • Config merging: Allow specifying the order in which Viash will merge configs (#289).
    If . is not in the list of inherited objects, it will be added at the end.

    Contents of config.vsh.yaml:

    functionality:
      name: foo
      arguments:
        - __merge__: obj_input.yaml
          name: "--one"
        - __merge__: [., obj_input.yaml]
          name: "--two"
        - __merge__: [obj_input.yaml, .]
          name: "--three"

    Contents of obj_input.yaml:

    type: file
    name: --input
    description: A h5ad file

    Output of viash config view config.vsh.yaml (stripped irrelevant bits):

    functionality:
      arguments:
      - type: "file"
        name: "--one"
        description: "A h5ad file"
      - type: "file"
        name: "--input"
        description: "A h5ad file"
      - type: "file"
        name: "--three"
        description: "A h5ad file"

Viash 0.6.3

09 Nov 15:24
Compare
Choose a tag to compare

MAJOR CHANGES

  • Config: Made major internal changes w.r.t. how config files are read and at which point a platform (native, docker, nextflow)
    is applied to the functionality script. The only visible side effect is that
    viash ns list will output each config only once instead of multiple times.

  • Functionality: Structured annotation can be added to a functionality and its arguments using the info field. Example:

    functionality:
      name: foo
      info:
        site: https://abc.xyz
        tags: [ one, two, three ]
      arguments:
        - name: --foo
          type: string
          info:
            foo: bar
            a:
              b:
                c

MINOR CHANGES

  • BashWrapper: Allow printing the executor command by adding ---verbose ---verbose to a viash run or an executable.

  • Testbenches: Rework MainBuildAuxiliaryNativeParameterCheck to create stimulus files and loop over the file from bash instead of looping natively.
    This prevents creating thousands of new processes which would only test a single parameter.
    Note this still calls the main script for each stimulus separately, but that was the case anyway, only much much worse.

  • Testbenches: Split some grouped test benches into slightly smaller test benches that group tested functionality better.

  • Annotations: Complete the config schema annotations.
    Make sure all arguments are documented.
    Added an annotation internalFunctionality and undocumented for arguments that should not be documented.
    Added a testbench that verifies that all arguments are in fact annotated, skipping those that are not in the class constructor.
    Adds a hierarchy field in the __this__ member to list the relation of the own and parent classes.

  • viash ns exec: Allow choosing whether the {platform} field should be filled in, based on the --apply_platform parameter.

BUG FIXES

  • DockerPlatform: Remove duplicate auto-mounts (#257).

  • NextflowVDSL3Platform: Fix 'Module scriptPath has not been defined yet' error when Nextflow>=22.10 (#269).

  • config inject: Doesn't work when must_exist == true (#273).

  • RScript: Fix compatibility issue where the new character escaping in r_script required R>=4.0 (#275). Escaping is now handled without
    using the new r'(foo)' notation.

DEPRECATION

  • DockerRequirements: The resources: setting has been deprecated and will be removed in Viash 0.7.0. Please use copy: instead.

  • DockerRequirements: The privileged: setting has been deprecated and will be removed in Viash 0.7.0. Please use run_args: "--privileged" instead.

EXPERIMENTAL FUNCTIONALITY

  • Config: Any part of a Viash config can use inheritance to fill data (#259). For example:
    Contents of src/test/config.vsh.yaml:
    __inherits__: ../api/base.yaml
    functionality:
      name: test
      resources:
        - type: bash_script
          path: script.sh
          text: |
            echo Copying $par_input to $par_output
            cp $par_input $par_output
    Contents of src/api/base.yaml:
    functionality:
      arguments:
        - name: "--input"
          type: file
        - name: "--output"
          type: file
          direction: output
    The resulting yaml will be:
    functionality:
      name: test
      arguments:
        - name: "--input"
          type: file
        - name: "--output"
          type: file
          direction: output
      resources:
        - type: bash_script
          path: script.sh
          text: |
            echo Copying $par_input to $par_output
            cp $par_input $par_output

Viash 0.6.2

11 Oct 05:45
47b12f6
Compare
Choose a tag to compare

BUG FIXES

  • Git: Strip credentials from remote repositories when retrieving the path.

  • VDSL3: Allow optional output files to be null.

Viash 0.6.1

03 Oct 13:38
4e18fe9
Compare
Choose a tag to compare

This release contains mostly minor improvements of functionality released in Viash 0.6.0. Most notably:

  • Support was added for type: long arguments

  • meta["n_proc"] has been renamed to meta["cpus"]. meta["cpus"] is now an integer, whereas meta["memory_*"] are now longs.

  • viash ns exec is able to recognise {platform} and {namespace} fields.

  • And various bug fixes and improvements to documentation and unit testing.

BREAKING CHANGES

  • Deprecated usage resources_dir variable inside scripts, use meta["resources_dir"] instead (or $meta_resources_dir in Bash, or meta$resources_dir in R).

  • Deprecated meta["n_proc"] in favour for meta["cpus"].

NEW FUNCTIONALITY

  • viash ns exec: Added two more fields:

    • {platform}: the platform name (if applicable)
    • {namespace}: the namespace of the component
  • LongArgument: Added support for 64-bit integers with type: long as opposed to type: integer which are 32-bit integers.

MAJOR CHANGES

  • Allow passing integers/doubles/booleans to string parameters (#225). Removed the 'Version' helper class.

MINOR CHANGES

  • meta["cpus"] is now an integer, meta["memory_*"] are now longs (#224).

  • DockerPlatform: Only store author names in the authors metadata.

  • NextflowPlatform: Only store author names in the authors metadata.

  • Argument[_]: Turn multiple_sep from Char into String.

INTERNAL CHANGES

  • All meta[...] variables are now processed similar to Argument[_]s, instead of using custom code to convert object types and detect Docker mounts.

  • Escaper: Make more generic Escaper helper class.

DOCUMENTATION

  • Hardcoded URLs pointing to viash.io in the documentation annotations were replaced with a new keyword system.

  • Replaced references to "DSL" with "Dynamic Config Modding" in the --help output.

  • Added an example for Ruby based Docker setups.

BUG FIXES

  • viash ns: Reverse exit code outputs, was returning 1 when everything was OK and 0 when errors were detected (#227).

  • viash config inject: Fix processing of arguments when argument groups are defined (#231).

  • Fixed a few typos in the CLI.

  • Fixed the formatting of ns exec documentation.

  • VDSL3: Fix stub functionality.

  • VDSL3: Fix error during error message.

  • viash test: Fix issue where VIASH_TEMP could not be a relative directory when running viash test (#242).

  • BashScript, CSharpScript, JavaScriptScript, PythonScript, RScript, ScalaScript: Fix quoting issues of certain characters (#113).

DEPRECATION

  • NextflowPlatform: Deprecate --param_list_format parameter.

TESTING

  • BashScript, CSharpScript, JavaScriptScript, PythonScript, RScript, ScalaScript: Implement more rigorous testing of which characters are escaped.

  • BashWrapper: Escape usage of multiple_sep. This fixes various checks and transformations not working when when multiple_sep is set to ";" (#235).

Viash 0.6.0

07 Sep 14:22
ed14990
Compare
Choose a tag to compare

The first (major) release this year! The biggest changes are:

  • Nextflow VDSL3 is now the default Nextflow platform, whereas the legacy Nextflow platform has been deprecated.
  • Support for tracking memory and cpu requirements more elegantly
  • Grouping arguments in groups more concisely.
  • The addition of a viash ns exec command, to be able to execute commands on Viash components more easily.

BREAKING CHANGES

  • NextflowPlatform: variant: vdsl3 is now the default NextflowPlatform. variant: legacy has been deprecated.

  • Functionality: Fields .inputs and .outputs has been deprecated. Please use .argument_groups instead (#186).
    Before:

    functionality:
      inputs:
        - name: "--foo"
      outputs:
        - name: "--bar"

    Now:

    functionality:
      argument_groups:
        - name: Inputs
          arguments:
            - name: "--foo"
              type: file
        - name: Outputs
          arguments:
            - name: "--bar"
              type: file
              direction: output
  • Passing strings to an argument group's arguments has been deprecated. Please simply copy the argument itself into the argument group (#186).
    Before:

    functionality:
      arguments:
        - name: "--foo"
          type: file
        - name: "--bar"
          type: file
          direction: output
      argument_groups:
        - name: Inputs
          arguments: [ foo ]
        - name: Outputs
          arguments: [ bar ]

    Now:

    functionality:
      argument_groups:
        - name: Inputs
          arguments:
            - name: "--foo"
              type: file
        - name: Outputs
          arguments:
            - name: "--bar"
              type: file
              direction: output

NEW FUNCTIONALITY

  • Allow setting the number of processes and memory limit from within the Viash config,
    as well as a list of required commands. Example:

    functionality:
    name: foo
    requirements:
      n_proc: 10
      memory: 10G
      commands: [ bash, r, perl ]

    You can override the default requirements at runtime:

    • ./foo ---n_proc 4 ---memory 100pb (for NativePlatform or DockerPlatform)
    • By adding process.cpus = 4 and process.memory "100 PB" to a nextflow.config (for NextflowPlatform)

    This results the following meta variables to be injected into a script:

    • meta_n_proc (in Bash) or meta["n_proc"] (in any other language): Number of processes the script is allowed to spawn.
    • meta_memory_b (in Bash) or meta["memory_b"] (in any other language): Amount of memory the script is allowed to allocate, in bytes.
    • meta_memory_kb (in Bash) or meta["memory_kb"] (in any other language): Same but in kilobytes, rounded up.
    • meta_memory_mb (in Bash) or meta["memory_mb"] (in any other language): Same but in megabytes, rounded up.
    • meta_memory_gb (in Bash) or meta["memory_gb"] (in any other language): Same but in gigabytes, rounded up.
    • meta_memory_tb (in Bash) or meta["memory_tb"] (in any other language): Same but in terabytes, rounded up.
    • meta_memory_pb (in Bash) or meta["memory_pb"] (in any other language): Same but in petabytes, rounded up.
  • viash ns exec: Added a command for executing arbitrary commands for all found Viash components.
    The syntax of this command is inspired by find . -exec echo {} \;.

    The following fields are automatically replaced:

    • {} | {path}: path to the config file
    • {abs-path}: absolute path to the config file
    • {dir}: path to the parent directory of the config file
    • {abs-dir}: absolute path to the directory of the config file
    • {main-script}: path to the main script (if any)
    • {abs-main-script}: absolute path to the main script (if any)
    • {functionality-name}: name of the component

    A command suffixed by \; (or nothing) will execute one command for each
    of the Viash components, whereas a command suffixed by + will execute one
    command for all Viash components.

  • ConfigMod: Added a del(...) config mod to be able to delete a value from the yaml. Example: del(.functionality.version).

MAJOR CHANGES

  • Folder structure: Adjusted the folder structure to correctly reflect the the namespace change of viash from com.dataintuitive.viash to io.viash.

  • Functionality: Reworked the enabled field from boolean to a status field which can have the following statusses: enabled, disabled and deprecated.
    When parsing a config file which has the status field set to deprecated a warning message is displayed on stderr.
    Backwards for enabled is provided where enabled: true => status: enabled and enabled: false => status: false. The enabled field is marked deprecated.

MINOR CHANGES

  • Resources: Handle edge case when no resources are specified in the vsh.yaml config file and display a warning message.

  • BashWrapper: Add a warning when an argument containing flags (e.g. --foo) is not recognized and will be handled as a positional argument as this is likely a mistake.

  • Functionality: Add check to verify there are no double argument names or short names in the config vsh.yaml declarations.

  • BashWrapper: Add check to verify a parameter isn't declared twice on the CLI, except in the case multiple: true is declared as then it's a valid use case.

  • BashWrapper: For int min/max checking: use native bash functionality so there is no dependency to bc.
    For double min/max checking: add fallback code to use awk in case bc is not present on the system (most likely to happen when running tests in a docker container).

  • viash ns list/viash config view: Allow viewing the post-processed argument groups by passing the --parse_argument_groups parameter.

TESTING

  • ConfigMod: Added unit tests for condition config mods.

  • MainTestDockerSuite: Derive config alternatives from the base vsh.yaml instead of adding the changes in separate files.
    This both reduces file clutter and prevents having to change several files when there are updates in the config format.

  • GitTest: Added unit tests for Git helper (#216).

BUG FIXES

  • csharp_script, javascript_script, python_script, r_script, scala_script: Make meta fields for memory and n_proc optional.

  • NextflowVdsl3Platform: Don't generate an error when --publish_dir is not defined and -profile no_publish is used.

  • Viash run: Viash now properly returns the exit code from the executed script.

  • Git: Fix incorrect metadata when git repository is empty (#216).

Viash 0.5.15

14 Jul 09:26
b7967f4
Compare
Choose a tag to compare

BREAKING CHANGES

  • WorkflowHelper::helpMessage: Now only takes one argument, namely the config.

MAJOR CHANGES

  • Namespace: Changed the namespace of viash from com.dataintuitive.viash to io.viash.

MINOR CHANGES

  • Testbenches: Add a testbench framework to test lots of character sequences, single or repeating to be tested in the yaml config. This can be used to later extend to other tests.

  • Testbenches::vdsl3: Add testbenches to verify functionality:

    • Vdsl3's param_list (yamlblob, yaml, json, csv).
    • NextFlow's own params-file.
    • Vdsl3's recalculating resource file paths to be relative to the param_list file instead of the workflow file (only available for yaml, json, csv).
    • Vdsl3's wrapping of modules to run these as a separate workflow automagically out of the box.
  • Main: Added viash --schema_export which outputs a schema of the Viash config file
    to console. This is to be used to automate populating the documentation website.

  • Helper: Split help message by argument group.

  • Helper: Remove unneeded arguments.

  • Functionality: Add default groups Inputs, Outputs and Arguments for all arguments missing from user-defined argument_groups.

  • WorkflowHelper::helpMessage: Rewrite to bring on par with Viash's help message.

  • BooleanArguments: Renamed internal class names for BooleanArguments to be better in line with how they are named in the config yaml.
    BooleanArgumentRegular -> BooleanArgument (in line with boolean)
    BooleanArgumentTrue -> BooleanTrueArgument (in line with boolean_true)
    BooleanArgumentFalse -> BooleanFalseArgument (in line with boolean_false)

BUG FIXES

  • NextflowVdsl3Platform: Change how --id is processed when a VDSL3 module is called from the CLI.

  • NextflowVdsl3Platform: Fix error when param_list is null.

  • NextflowVdsl3Platform: Fix error when optional, multiple arguments are set to null.

  • Testbenches: Better capture expected error messages while running testbenches again. Code changes right before previous release re-introduced some of the messages.

  • NextflowVdsl3Platform: Fix issue where optional parameters aren't removed when .run(args: [optarg: null]).

  • WorkflowHelper::readCsv: Treat empty values as undefined instead of throwing an error.

  • NextflowVdsl3Platform: Use $NXF_TEMP or $VIASH_TEMP as temporary directory if the container engine is not set to docker, podman or charlieengine, else set to /tmp.

  • Resources: When adding a resource folder, allow a trailing / at the end of the path.
    Previously this caused the target folder to be erased and the content of the resource folder to be written directly into the target folder.

Viash 0.5.14

30 Jun 08:23
6a959f6
Compare
Choose a tag to compare

NEW FUNCTIONALITY

  • Functionality: Allow specifying argument groups. Example:

    functionality:
      ...
      argument_groups:
        - name: First group
          arguments: [foo, bar]
          description: Description
  • NextflowVdsl3Platform: Use --param_list to initialise a Nextflow channel with multiple parameter sets.
    Possible formats are csv, json, yaml, or simply a yaml_blob.
    A csv should have column names which correspond to the different arguments of this pipeline.
    A json or a yaml file should be a list of maps, each of which has keys corresponding to the arguments of the pipeline.
    A yaml blob can also be passed directly as a parameter.
    Inside the Nextflow pipeline code, params.params_list can also be used to directly a list of parameter sets.
    When passing a csv, json or yaml, relative path names are relativized to the location of the parameter file.

    Examples:

    nextflow run "target/foo/bar/main.nf" --param_list '[{"id": "foo", "input": "/path/to/bar"}]'
    nextflow run "target/foo/bar/main.nf" --param_list "params.csv" --reference "/path/to/ref"

MAJOR CHANGES

  • NextflowVdsl3Platform: The functionality is now slurped from a json instead of manually
    taking care of the formatting in Groovy.

  • NextflowVdsl3Platform: The --help is auto-generated from the config.

MINOR CHANGES

  • NextflowVdsl3Platform: Allow both --publish_dir and --publishDir when auto.publish = true.

  • NextflowVdsl3Platform: Allow passing parameters with multiplicity > 1 from Nextflow CLI.

  • Main: Added viash --cli_export which outputs the internal cli construction information
    to console. This is to be used to automate populating the documentation website.

  • viash ns: Display success and failure summary statistics, printed to stderr.

  • DataObject: .alternatives is now a OneOrMore[String] instead of List[String], meaning
    you can now specify { type: string, name: "--foo", alternatives: "-f" } instead of
    { type: string, name: "--foo", alternatives: [ "-f" ] }

  • BashWrapper: Added metadata field meta_executable, which is a shorthand notation for
    meta_executable="$meta_resources_dir/$meta_functionality_name"

INTERNAL CHANGES

  • Arguments: Internal naming of functionality.arguments is changed from DataObject to Arguments. Change is also applied to child classes, e.g. StringObject -> StringArgument.

  • Script: Allow more control over where injected code ends up.

  • Restructure type system to allow type-specific arguments.

BUG FIXES

  • DockerPlatform: Change org.opencontainers.image.version annotation to functionality.version when set.
    Additionally fixed retrieving the git tag possibly returning fatal: No names found, cannot describe anything. or similar.

  • viash config inject: Fix config inject when .functionality.inputs or .functionality.outputs is used.

  • BashWrapper: Don't add bc as dependency. Only perform integer/float min/max checks when bc is available, otherwise ignore.

  • DockerPlatform: Fix inputs & outputs arguments being present twice.

  • viash ns test: Silently skip Nextflow platforms as these don't support tests and will always fail.

  • Testbenches: Better capture expected error messages while running testbenches. Having these show on the console could be confusing.

  • NextflowVdsl3Platform: Fix issue when running multiple VDSL3 modules concurrently on the same channel.

Viash 0.5.13

10 Jun 08:57
Compare
Choose a tag to compare

NEW FUNCTIONALITY

  • NextflowVdsl3Platform: Allow overriding the container registry of all Viash components by
    setting the params.override_container_registry value. Only works for auto-derived image names.

MINOR CHANGES

  • Functionality and viash ns: Added .enabled in functionality, set to true by default.
    Filter for disabled components in namespace commands.

  • DockerPlatform: Add org.opencontainers.image annotations to built docker images.

  • Functionality: when defining text resources, permit defining path instead of dest.
    If both dest and path are unset, use a default file name depending on the resource type, such as script.sh or text.txt.

  • viash build: Errors are printed in red.

BUG FIXES

  • NextflowVdsl3Platform: Undefined input files should not inject a VIASH_PAR_* variable when multiple: true.

  • NextflowVdsl3Platform: Make injected resources dir absolute.

  • NextflowVdsl3Platform: Fix escaping of triple single quotes.

  • NextflowVdsl3Platform: Also apply auto.simplifyInput to Lists.

Viash 0.5.12

24 May 20:11
2f0de92
Compare
Choose a tag to compare

MINOR CHANGES

  • --help: Don't print "my_component " when no version is specified,
    but instead simply "my_component".

  • NextflowVdsl3Platform: Set mode=copy for auto.publish and auto.transcript.

  • NextflowVdsl3Platform: When a module is used multiple times in the same workflow,
    don't throw an error anymore, instead simply generate a warning.

  • NextflowVdsl3Platform: Throw an error when an input file was not found.

  • viash build: Indent auto-generated code according the indentation of VIASH START when found.

  • Main: Handle not finding the config file or resources in a config file better.
    Display a more helpful message instead of a stack trace.

  • BashWrapper: Add checks on parameters for valid integer, double and boolean values.

  • BashWrapper: Add option to limit string and integer values to specific choice values.

  • BashWrapper: Add option to set min and max values for integer and double values.

  • Dependencies:

    • Scala was upgraded from 2.12.10 to 2.12.15
    • sbt was upgraded from 1.3.4 to 1.6.1
    • sbt-scoverage was upgraded from 1.5.1 to 1.9.3

BUG FIXES

  • viash_test: Add back --no_cache parameter to viash_test.

  • viash_test: Fix --append parameter for viash_test, was not getting passed through.

  • viash ns test: Fix --append parameter, actually start from a clean file if append is false.

  • viash_push: Fix component not being built during a release of Viash.

  • PythonRequirements: Fix packages being mentioned twice in a Dockerfile.

  • Main: Added support spaces in filenames of config files and resources

  • BashWrapper: Display a message when the last parsed argument would require more values than are still available.
    Now display a message that values are missing, used to silently crash the wrapper.

  • viash config inject: Fix error when file argument is must_exist: true.

Viash 0.5.11

09 May 11:38
2ec0a1e
Compare
Choose a tag to compare

MAJOR CHANGES

  • Functionality: Now also accepts 'inputs' and 'outputs' in addition to 'arguments'. For inputs and outputs, any specified arguments will have default type: file and direction: input or direction: output respectively.

MINOR CHANGES

  • DockerPlatform: Move description labels to the end of the Dockerfile to improve cross-component caching.

  • Functionality: Arguments where .multiple is true can now have lists as default and example.

  • viash_build: Added unit test for this component.

  • viash_test: Added unit test for this component.

  • PythonRequirements: Allow upgrading dependencies. Example: [ type: python. pypi: anndata, upgrade: true ].

  • NextflowLegacyPlatform: Remove annoying messages when building Nxf modules.

  • ConfigMods: Expanded the DSL to allow specifying at which point to apply a config mod.
    This functionality was necessary to allow for setting fields which alter the way configs are parsed.
    Example of when this is useful: <preparse> .platforms[.type == "nextflow"].variant := "vdsl3".
    Updating workflow of parsing a config file is:

    • read Yaml from file
    • apply preparse config mods
    • parse resulting Json as Config, thereby instantiating default values etc.
    • convert Config back to Json
    • apply postparse config mods (original config mods)
    • convert final Json back to Config

BETA FUNCTIONALITY

  • NextflowVdsl3Platform: A beta implementation of the next-generation Viash+Nextflow platform. See #82 for more information. You can access the previous Nextflow platform by using the variant parameter:
    - type: nextflow
      variant: legacy
      separate_multiple_outputs: false

BUG FIXES

  • viash_build and viash_test: The query_name and query_namespace arguments were switched around. These arguments are now passed correctly.

  • BashScript, JavaScriptScript, PythonScript, RScript: Correctly escape ' (#113). Update unit tests accordingly.

  • CSharpScript, ScalaScript: Correctly escape " (#113). Update unit tests accordingly.

  • viash_build, viash_test, viash_push: Don't try to remove log files if they don't exist.

INTERNAL CHANGES

  • DataObject:

    • Renamed otype to flags.
    • Renamed oType to type
    • Deprecated tag (unused feature).
  • All abstract / inherited classes: Renamed oType to type.

DEPRECATION

  • Functionality: Deprecated function_type and add_resources_to_path. These should be
    unused features, by now.