godot-tester

Godot Tester - Godot 4 & GUT 9 Support

What's Changed

• chore: Use Node 20 and checkout@v4 by @jeremyckahn in #40

New Contributors

• @jeremyckahn made their first contribution in #40
• @baabcantcode made their first contribution in #30

Full Changelog: v4...v5.1

Additions from previous releases

• Rewrote the action to use Node JS.
• Added download support for Godot 4 & QA tested with GUT 9.
• Removed input: ignore-errors.
  • Test result parsing is no longer done via STDOUT, it is done via the exported .XML file provided by GUT.
• Increased the default import timeout from 1 second to 15 seconds
• Updated Dockerfile to download .NET dependencies
• customizing the test directory path (e.g. tests instead of test)
• support for alternative release types, previously only stable was supported
• can set maximum failures with max-fails
• can check assert pass / fail counts instead of test pass / fail counts
• can specify a custom download URL for the godot binary
• GUT's XML test results are now being parsed

Example Use:

name: Godot testing

on: [ push ]

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2
    - uses: croconut/[email protected]
      with:
        # required
        version: "4.1"
        # the type of release of godot that the tests should be run with
        release_type: "rc2"
        is-mono: "true"
        # the folder with your project.godot file in it
        path: "tester"
        # how long to spend importing assets before tests are run
        import-time: "5"
        # how long tests can run in seconds
        test-timeout: "45"
        # the ratio of tests that must pass for this action to pass
        # e.g. 0.6 means 60% of your tests must pass
        minimum-pass: "0.6"
        # the directory containing Gut tests
        test-dir: "res://test"
        # instead of running GUT's command line tool, 
        # you can run a test scene if you have one
        # set up a scene like in this repo --> located at /tester_GUT_v9.0.1/test/alt_mode/tests.tscn
        # set up a script like in this repo --> located at /tester_GUT_v9.0.1/test/alt_mode/cli_test.gd
        # ensure that the script exits on test completion
        # uses relative path from your godot project directory
        direct-scene: "tester_GUT_v9.0.1/test/alt_mode/tests.tscn" 
        # default is false, set true to count asserts instead of tests
        assert-check: "true" 
        # not checked by default, set to a number to limit the 
        # maximum amount of failed tests for a passing test suite
        max-fails: 3  
        # default is GUTs default: 'res://.gutconfig.json'; set this to load a different config file
        config-file: "res://.myconfig.json" 
        # designate a custom url to download the godot binary from
        custom-godot-dl-url: ""
        # relative path to the xml file to read / write GUT's results from, recommended
        # for direct-scene users to check this file if you have issues
        result-output-file: "test_results.xml"

godot-tester

Godot 4 & GUT 9 Support + Node JS Rewrite

Summary

• Rewrote the action to use Node JS.
• Added download support for Godot 4 & QA tested with GUT 9.
• Removed input: ignore-errors.
  • Test result parsing is no longer done via STDOUT, it is done via the exported .XML file provided by GUT.
• Increased the default import timeout from 1 second to 15 seconds
• Updated Dockerfile to download .NET dependencies

What's Changed

• Port to Node JS by @RGonzalezTech in #31
• Support Godot 4 w/ 3.x compatibility by @RGonzalezTech in #27
• add a job matrix, fix v4 mono by @baabcantcode in #30

New Contributors

• @baabcantcode made their first contribution in #30

Full Changelog: v4.1...v5

Additions from previous releases

• customizing the test directory path (e.g. tests instead of test)
• support for alternative release types, previously only stable was supported
• can set maximum failures with max-fails
• can check assert pass / fail counts instead of test pass / fail counts
• can specify a custom download URL for the godot binary
• GUT's XML test results are now being parsed

Example Use:

name: Godot testing

on: [ push ]

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2
    - uses: croconut/godot-tester@v5
      with:
        # required
        version: "4.1"
        # the type of release of godot that the tests should be run with
        release_type: "rc2"
        is-mono: "true"
        # the folder with your project.godot file in it
        path: "tester"
        # how long to spend importing assets before tests are run
        import-time: "5"
        # how long tests can run in seconds
        test-timeout: "45"
        # the ratio of tests that must pass for this action to pass
        # e.g. 0.6 means 60% of your tests must pass
        minimum-pass: "0.6"
        # the directory containing Gut tests
        test-dir: "res://test"
        # instead of running GUT's command line tool, 
        # you can run a test scene if you have one
        # set up a scene like in this repo --> located at /tester_GUT_v9.0.1/test/alt_mode/tests.tscn
        # set up a script like in this repo --> located at /tester_GUT_v9.0.1/test/alt_mode/cli_test.gd
        # ensure that the script exits on test completion
        # uses relative path from your godot project directory
        direct-scene: "tester_GUT_v9.0.1/test/alt_mode/tests.tscn" 
        # default is false, set true to count asserts instead of tests
        assert-check: "true" 
        # not checked by default, set to a number to limit the 
        # maximum amount of failed tests for a passing test suite
        max-fails: 3  
        # default is GUTs default: 'res://.gutconfig.json'; set this to load a different config file
        config-file: "res://.myconfig.json" 
        # designate a custom url to download the godot binary from
        custom-godot-dl-url: ""
        # relative path to the xml file to read / write GUT's results from, recommended
        # for direct-scene users to check this file if you have issues
        result-output-file: "test_results.xml"

godot-tester

Godot 4 & GUT 9 Support

Summary

• Added download support for Godot 4 & QA tested with GUT 9.
• Removed input: ignore-errors.
  • Test result parsing is no longer done via STDOUT, it is done via the exported .XML file provided by GUT.
• Increased the default import timeout from 1 second to 15 seconds
• Updated Dockerfile to download .NET dependencies

What's Changed

• Support Godot 4 w/ 3.x compatibility by @RGonzalezTech in #27
• add a job matrix, fix v4 mono by @baabcantcode in #30

New Contributors

• @baabcantcode made their first contribution in #30

Full Changelog: v3...v4.1

Additions from previous releases

• customizing the test directory path (e.g. tests instead of test)
• support for alternative release types, previously only stable was supported
• can set maximum failures with max-fails
• can check assert pass / fail counts instead of test pass / fail counts
• can specify a custom download URL for the godot binary
• GUT's XML test results are now being parsed

Example Use:

name: Godot testing

on: [ push ]

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2
    - uses: croconut/[email protected]
      with:
        # required
        version: "4.0.3"
        # the type of release of godot that the tests should be run with
        release_type: "rc2"
        is-mono: "true"
        # the folder with your project.godot file in it
        path: "tester"
        # how long to spend importing assets before tests are run
        import-time: "5"
        # how long tests can run in seconds
        test-timeout: "45"
        # the ratio of tests that must pass for this action to pass
        # e.g. 0.6 means 60% of your tests must pass
        minimum-pass: "0.6"
        # the directory containing Gut tests
        test-dir: "res://test"
        # instead of running GUT's command line tool, 
        # you can run a test scene if you have one
        # set up a scene like in this repo --> located at /tester_GUT_v9.0.1/test/alt_mode/tests.tscn
        # set up a script like in this repo --> located at /tester_GUT_v9.0.1/test/alt_mode/cli_test.gd
        # ensure that the script exits on test completion
        # uses relative path from your godot project directory
        direct-scene: "tester_GUT_v9.0.1/test/alt_mode/tests.tscn" 
        # default is false, set true to count asserts instead of tests
        assert-check: "true" 
        # not checked by default, set to a number to limit the 
        # maximum amount of failed tests for a passing test suite
        max-fails: 3  
        # default is GUTs default: 'res://.gutconfig.json'; set this to load a different config file
        config-file: "res://.myconfig.json" 
        # designate a custom url to download the godot binary from
        custom-godot-dl-url: ""
        # relative path to the xml file to read / write GUT's results from, recommended
        # for direct-scene users to check this file if you have issues
        result-output-file: "test_results.xml"

godot-tester

Godot 4 & GUT 9 Support

Summary

• Added download support for Godot 4 & QA tested with GUT 9.
• Removed input: ignore-errors.
  • Test result parsing is no longer done via STDOUT, it is done via the exported .XML file provided by GUT.
• Increased the default import timeout from 1 second to 15 seconds

What's Changed

• Support Godot 4 w/ 3.x compatibility by @RGonzalezTech in #27

Full Changelog: v3...v4

Additions from previous releases

• customizing the test directory path (e.g. tests instead of test)
• support for alternative release types, previously only stable was supported
• can set maximum failures with max-fails
• can check assert pass / fail counts instead of test pass / fail counts
• can specify a custom download URL for the godot binary
• GUT's XML test results are now being parsed

Example Use:

name: Godot testing

on: [ push ]

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2
    - uses: croconut/godot-tester@v4
      with:
        # required
        version: "4.0.3"
        # the type of release of godot that the tests should be run with
        release_type: "rc2"
        is-mono: "true"
        # the folder with your project.godot file in it
        path: "tester"
        # how long to spend importing assets before tests are run
        import-time: "5"
        # how long tests can run in seconds
        test-timeout: "45"
        # the ratio of tests that must pass for this action to pass
        # e.g. 0.6 means 60% of your tests must pass
        minimum-pass: "0.6"
        # the directory containing Gut tests
        test-dir: "res://test"
        # instead of running GUT's command line tool, 
        # you can run a test scene if you have one
        # set up a scene like in this repo --> located at /tester_GUT_v9.0.1/test/alt_mode/tests.tscn
        # set up a script like in this repo --> located at /tester_GUT_v9.0.1/test/alt_mode/cli_test.gd
        # ensure that the script exits on test completion
        # uses relative path from your godot project directory
        direct-scene: "tester_GUT_v9.0.1/test/alt_mode/tests.tscn" 
        # default is false, set true to count asserts instead of tests
        assert-check: "true" 
        # not checked by default, set to a number to limit the 
        # maximum amount of failed tests for a passing test suite
        max-fails: 3  
        # default is GUTs default: 'res://.gutconfig.json'; set this to load a different config file
        config-file: "res://.myconfig.json" 
        # designate a custom url to download the godot binary from
        custom-godot-dl-url: ""
        # relative path to the xml file to read / write GUT's results from, recommended
        # for direct-scene users to check this file if you have issues
        result-output-file: "test_results.xml"

godot-tester

for running godot tests made with GUT (and now potentially without GUT)

new additions

• updated to support GUT 7.3+ (if you are on an older version, you may want to use v2.5)
• can specify a custom download URL for the godot binary
• GUT's XML test results are now being parsed

additions from the previous releases

• can ignore SCRIPT ERRORs
• general improvements to the asset importing process
• can specify gut config file

You can also run your own testing framework if you have a scene that can run tests on load and then exit.

You would use the 'direct-scene' option for that and you'll want the output for that scene to be similar to GUT's to help this action's pass / fail to work properly.

Refer to the folder in this action's repo "/tester/test/alt_mode" to see how to up an override test scene for GUT (recommended for applications with heavy duty test setups). Direct scene users should pay careful attention to set the action input parameter "result-output-file" the same as the XML output location they use in their testing scene.

example setup

name: Godot testing

on: [ push ]

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2
    - uses: croconut/godot-tester@v3
      with:
        # required
        version: "3.2.2"
        # the type of release of godot that the tests should be run with
        release_type: "rc2"
        is-mono: "true"
        # the folder with your projects.godot file in it
        path: "tester"
        # how long to spend importing assets before tests are run
        import-time: "5"
        # how long tests can run in seconds
        test-timeout: "45"
        # the ratio of tests that must pass for this action to pass
        # e.g. 0.6 means 60% of your tests must pass
        minimum-pass: "0.6"
        # the directory containing Gut tests
        test-dir: "res://test"
        # instead of running GUT's command line tool, you can run a GUT test scene if you have one
        # set up correctly
        # helps enable things like mouse mocking and current scene to work without any extra coding steps
        # set up a scene like in this repo --> located at /tester/test/alt_mode/tests.tscn
        # set up a script like in this repo --> located at /tester/test/alt_mode/cli_plugin.gd
        # ensure cli_plugin.gd inherits from the GUT plugin
        # and exits on test completion
        # ensure tests.tscn uses your modified plugin script and check yes for run on load
        # uses relative path from your godot project directory
        direct-scene: "test/alt_mode/tests.tscn" 
        # default is false, set true to count asserts instead of tests
        assert-check: "true" 
        # not checked by default, set to a number to limit the 
        # maximum amount of failed tests for a passing test suite
        max-fails: 3  
        # to ensure all SCRIPT ERRORs dont contribute to your failure rate        
        ignore-errors: "true" 
        # default is GUTs default: 'res://.gutconfig.json'; set this to load a different config file
        config-file: "res://.myconfig.json" 
        # designate a custom url to download the godot binary from
        custom-godot-dl-url: ""
        # relative path to the xml file to read / write GUT's results from, recommended
        # for direct-scene users to check this file if you have issues
        result-output-file: "test_results.xml"        

What's Changed

• add option for custom godot builds by @you-win in #23
• fix for GUT 7.3.0 by @db0 in #21
• Leverage GUT XML Output by @RGonzalezTech in #24

New Contributors

• @db0 made their first contribution in #21
• @RGonzalezTech made their first contribution in #24

Full Changelog: v2.5...v3.0

godot-tester

for running godot tests made with GUT (and now potentially without GUT)

new additions

• can ignore SCRIPT ERRORs
• general improvements to the asset importing process
• can specify gut config file

additions from the previous releases

• customizing the test directory path (e.g. tests instead of test)
• support for alternative release types, previously only stable was supported
• can set maximum failures with max-fails
• can check assert pass / fail counts instead of test pass / fail counts

You can also run your own testing framework if you have a scene that can run tests on load and then exit.

You would use the 'direct-scene' option for that and you'll want the output for that scene to be similar to GUT's to help this action's pass / fail to work properly.

Refer to the folder in this action's repo "/tester/test/alt_mode" to see how to up an override test scene for GUT (recommended for applications with heavy duty test setups).

## example setup

name: Godot testing

on: [ push ]

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2
    - uses: croconut/[email protected]
      with:
        # required
        version: "3.2.2"
        # the type of release of godot that the tests should be run with
        release_type: "rc2"
        is-mono: "true"
        # the folder with your projects.godot file in it
        path: "tester"
        # how long to spend importing assets before tests are run
        import-time: "5"
        # how long tests can run in seconds
        test-timeout: "45"
        # the ratio of tests that must pass for this action to pass
        # e.g. 0.6 means 60% of your tests must pass
        minimum-pass: "0.6"
        # the directory containing Gut tests
        test-dir: "res://test"
        # instead of running GUT's command line tool, you can run a GUT test scene if you have one
        # set up correctly
        # helps enable things like mouse mocking and current scene to work without any extra coding steps
        # set up a scene like in this repo --> located at /tester/test/alt_mode/tests.tscn
        # set up a script like in this repo --> located at /tester/test/alt_mode/cli_plugin.gd
        # ensure cli_plugin.gd inherits from the GUT plugin
        # and exits on test completion
        # ensure tests.tscn uses your modified plugin script and check yes for run on load
        # uses relative path from your godot project directory
        direct-scene: "test/alt_mode/tests.tscn" 
        # default is false, set true to count asserts instead of tests
        assert-check: "true" 
        # not checked by default, set to a number to limit the 
        # maximum amount of failed tests for a passing test suite
        max-fails: 3  
        # to ensure all SCRIPT ERRORs dont contribute to your failure rate        
        ignore-errors: "true" 
        # default is GUTs default: 'res://.gutconfig.json'; set this to load a different config file
        config-file: "res://.myconfig.json" 

What's Changed

• swap test to pending by @croconut in #14
• handle custom configuration files, improves import process, enables SCRIPT ERROR ignoring by @croconut in #18

Full Changelog: v2.4...v2.5

godot-tester

for running godot tests made with GUT (and now potentially without GUT)

new additions

• can set maximum failures with max-fails
• can check assert pass / fail counts instead of test pass / fail counts

additions from the previous releases

• customizing the test directory path (e.g. tests instead of test)
• support for alternative release types, previously only stable was supported

You can also run your own testing framework if you have a scene that can run tests on load and then exit.

You would use the 'direct-scene' option for that and you'll want the output for that scene to be similar to GUT's to help this action's pass / fail to work properly.

Refer to the folder in this action's repo "/tester/test/alt_mode" to see how to up an override test scene for GUT (recommended for applications with heavy duty test setups).

example setup

name: Godot testing

on: [ push ]

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2
    - uses: croconut/[email protected]
      with:
        # required
        version: "3.2.2"
        # the type of release of godot that the tests should be run with
        release_type: "rc2"
        is-mono: "true"
        # the folder with your projects.godot file in it
        path: "tester"
        # how long to spend importing assets before tests are run
        import-time: "5"
        # how long tests can run in seconds
        test-timeout: "45"
        # the ratio of tests that must pass for this action to pass
        # e.g. 0.6 means 60% of your tests must pass
        minimum-pass: "0.6"
        # the directory containing Gut tests
        test-dir: "res://test"
        # instead of running GUT's command line tool, you can run a GUT test scene if you have one
        # set up correctly
        # helps enable things like mouse mocking and current scene to work without any extra coding steps
        # set up a scene like in this repo --> located at /tester/test/alt_mode/tests.tscn
        # set up a script like in this repo --> located at /tester/test/alt_mode/cli_plugin.gd
        # ensure cli_plugin.gd inherits from the GUT plugin
        # and exits on test completion
        # ensure tests.tscn uses your modified plugin script and check yes for run on load
        direct-scene: "test/alt_mode/tests.tscn" # uses relative path from your godot project directory
        assert-check: "true" # default is false, set true to count asserts instead of tests
        max-fails: 3  # not checked by default, set to a number to limit the 
                      # maximum amount of failed tests for a passing test suite

What's Changed

• partial test passing, test timeouts, GUT output parsing by @croconut in #3
• bugfix - getting correct number of test failures by @croconut in #5
• add support for non stable godot releases by @Zutatensuppe in #7
• make test dir configurable by @you-win in #8
• Alternate run mode by @croconut in #10
• assert option, max fail option & bugfixes by @croconut in #13

New Contributors

• @Zutatensuppe made their first contribution in #7
• @you-win made their first contribution in #8

Full Changelog: v1.0.2...v2.4

godot-tester

for running godot tests made with GUT (and now potentially without GUT)

new option: overriding GUT's CLI

You can also run your own testing framework if you have a scene that can run tests on load and then exit.

You would use the 'direct-scene' option for that and you'll want the output for that scene to be similar to GUT's to help this action's pass / fail to work properly.

Refer to the folder in this action's repo "/tester/test/alt_mode" to see how to up an override test scene for GUT (recommended for applications with heavy duty test setups).

additions from the previous release

• customizing the test directory path (e.g. tests instead of test)
• support for alternative release types, previously only stable was supported

example setup

name: Godot testing

on: [ push ]

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2
    - uses: croconut/[email protected]
      with:
        # required
        version: "3.2.2"
        # the type of release of godot that the tests should be run with
        release_type: "rc2"
        is-mono: "true"
        # the folder with your projects.godot file in it
        path: "tester"
        # how long to spend importing assets before tests are run
        import-time: "5"
        # how long tests can run in seconds
        test-timeout: "45"
        # the ratio of tests that must pass for this action to pass
        # e.g. 0.6 means 60% of your tests must pass
        minimum-pass: "0.6"
        # the directory containing Gut tests
        test-dir: "res://test"
        # instead of running GUT's command line tool, you can run a GUT test scene if you have one
        # set up correctly
        # helps enable things like mouse mocking and current scene to work without any extra coding steps
        # set up a scene like in this repo --> located at /tester/test/alt_mode/tests.tscn
        # set up a script like in this repo --> located at /tester/test/alt_mode/cli_plugin.gd
        # ensure cli_plugin.gd inherits from the GUT plugin
        # and exits on test completion
        # ensure tests.tscn uses your modified plugin script and check yes for run on load
        direct-scene: "test/alt_mode/tests.tscn" # uses relative path from your godot project directory

What's Changed

• partial test passing, test timeouts, GUT output parsing by @croconut in #3
• bugfix - getting correct number of test failures by @croconut in #5
• add support for non stable godot releases by @Zutatensuppe in #7
• make test dir configurable by @you-win in #8
• Alternate run mode by @croconut in #10

New Contributors

• @Zutatensuppe made their first contribution in #7
• @you-win made their first contribution in #8

Full Changelog: v1.0.2...v2.3

godot-tester

Added in handling for:

• alternate test directories
• any release version of Godot (not just stable)

example setup:

name: Godot testing

on: [ push ]

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2
    - uses: croconut/[email protected]
      with:
        # required
        version: "3.2.2"
        # the type of release of godot that the tests should be run with
        release_type: "rc2"
        is-mono: "true"
        # the folder with your projects.godot file in it
        path: "tester"
        # how long to spend importing assets before tests are run
        import-time: "5"
        # how long tests can run in seconds
        test-timeout: "45"
        # the ratio of tests that must pass for this action to pass
        # e.g. 0.6 means 60% of your tests must pass
        minimum-pass: "0.6"
        # the directory containing Gut tests
        test-dir: "res://test"

Thanks to: @you-win @Zutatensuppe

godot-tester

A small bugfix to fix inconsistent pass rates when having test functions fail multiple asserts.

Still does everything v2 and v2.1 do.

Example usage

name: Godot testing

on: [ push ]

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2
    - uses: croconut/[email protected]
      with:
        # required
        version: "3.2.2"
        is-mono: "true"
        # the folder with your projects.godot file in it
        path: "tester"
        # how long to spend importing assets before tests are run
        import-time: "5"
        # how long GUT can take to run all your tests in seconds
        # action fails on timeout
        test-timeout: "45"
        # the ratio of tests that must pass for this action to pass
        # e.g. 0.6 means 60% of your tests must pass
        minimum-pass: "0.6"