Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add smoke tests #54

Merged
merged 1 commit into from
Aug 13, 2023
Merged

Add smoke tests #54

merged 1 commit into from
Aug 13, 2023

Conversation

ErikSchierboom
Copy link
Member

This PR adds smoke tests for some common scenarios.

The goal of the smoke tests is to increase confidence that the Docker image that is produced works as
intended.
The repo contains other tests, which are great, but what this PR adds is that it runs an integration
test using the following setup:

  1. Build the Docker image
  2. Run the Docker image with the tests directory mapped into the container
  3. Run the Docker image's normal entrypoint (bin/run.sh) on each of the test cases defined in the
    tests directory
  4. Verify that the output of those test runs matches the expected output (which is stored inside the
    tests directories

If successful, we then know that:

  1. The Docker image can be built successfully
  2. The bin/run.sh entrypoint inside the Docker image is setup correctly
  3. The test runner generates the correct output

@iHiD
Copy link
Member

iHiD commented Aug 11, 2023

(This is red btw, @ErikSchierboom)

@ErikSchierboom ErikSchierboom force-pushed the smoke-tests branch 2 times, most recently from e5829cc to 824540a Compare August 11, 2023 14:22
@verdammelt
Copy link
Member

Should we replace the lisp test files used with something minimal?

As for expected behavior - what is the expected behavior of the 'empty file'. The behavior captured here is not because an empty file is itself bad - but because the empty file does not define any of the symbols that the test file expect to have been defined.

@ErikSchierboom
Copy link
Member Author

Should we replace the lisp test files used with something minimal?

Maybe in a follow up PR? This was just an easy way to get the required test cases

@ErikSchierboom
Copy link
Member Author

As for expected behavior - what is the expected behavior of the 'empty file'. The behavior captured here is not because an empty file is itself bad - but because the empty file does not define any of the symbols that the test file expect to have been defined.

It is a standard test case as some tracks have issues dealing with empty files.

@ErikSchierboom
Copy link
Member Author

@verdammelt Are you happy enough with this PR or would you like to do the simplification first?

@verdammelt
Copy link
Member

@verdammelt Are you happy enough with this PR or would you like to do the simplification first?

Happy enough, merge when you want...

@ErikSchierboom ErikSchierboom merged commit e622754 into main Aug 13, 2023
1 check passed
@ErikSchierboom ErikSchierboom deleted the smoke-tests branch August 13, 2023 16:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants