-
Notifications
You must be signed in to change notification settings - Fork 708
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Investigate a way to manually define test specs (specs.json) #616
Comments
That sounds like a great idea! The tests could use a lot of work. I also open #542 to allow more flexibility when testing. Since I didn't write the lib, the lack of effective tests makes it especially hard to review PRs. I'd love to see this implemented. |
I just want to note that I'm also seeing that tests pass when they shouldn't. You can tell that the test shouldn't pass because the specs.json file is changed. |
What about merging the small PR #762 as a quick fix for this issue? This PR disables the re-building of the specs.json files during the build so the tests actually do something useful (Comparing the new results with the previously committed results). In case something changes and these changes are correct then you can run the grunt task I just tested this PR and it works fine. I sabotaged typedoc by changing And don't be alarmed by the unit test failures reported for this PR. They are there because the unit tests now actually do something. So these errors are normal and can be fixed by committing the updated spec files once. |
Yep, I'm working on updating the specs now. I do think that there are some bugs to fix though so I'm held up on that. If I merged #762 now, I'm concerned any PRs would try to change unrelated specs. Thanks for testing it out! |
While investigating this I've noticed that the tests don't run consistently if you run a converter test individually. More investigation is needed to understand that issue. |
I see #762 has been merged. Is this issue now resolved or does more work need to be done before tests can be relied on? |
Yeah, I think it can be. 👍 |
I'm wondering what the tests comparing the specs.json are good for and if there is any case of failing?
Reason: the specs.json files are generated by the same well-functioning or malfunctioning program that generates the output to compare with (=my locally generated typedoc build).
gruntfile.js
test/converter.ts
The tests always succeed since the specs.json is just the written-to-a-file output that is generated a second time for comparison although the output could be wrong regarding its content.
Example:
The isExported flag of "internalVar" is true although it should not.
I would like us to investigate a way to manuelly define specs saying "isExported flag of internalVar must not be true" or things like that.
The text was updated successfully, but these errors were encountered: