-
Notifications
You must be signed in to change notification settings - Fork 152
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Any plans to add Warning/Info status? #191
Comments
Hi @vrbcntrl, Could you please explain the use case for this ? Currently we have 2 different messages, which one of them can be seen as INFO a bit. They are As long as there is a valid use case aligned with the tool's vision, anything is possible. :) |
Hi @eerkunt , my use case is like this ... when a condition is not met, the test should not fail, i want the test status to be returned as a warning instead of FAILURE. currently my CICD pipeline will break if there are any test FAILURES, so I don't want to break the pipeline when my test is of type WARNING not PASS|FAILURE type. I hope that will give some idea what I am trying to achieve. Thanks. |
Ah I see, you want to just run terraform-compliance, but not failing. Well, then I think you can use “—wip” argument ? You can also create tags within your tests and mark those ones (like “test_should_always_fail”) and run them with —tags argument, just to structure it better. I hope I understood your case correctly. Please let me know if this wont solve your case |
thanks for the understanding. |
It is a Whole functional end-to-end tests are running like that, have a look on https://github.com/eerkunt/terraform-compliance/tree/master/tests/functional It may be a bit hard to understand but it is incorporated in https://github.com/eerkunt/terraform-compliance/blob/9d8ecb55e3fd6fcd15c463cb78f1c2f206918011/tests/functional/run_functional_tests.py#L39 |
I just found this link : https://behave.readthedocs.io/en/latest/behave.html#cmdoption-w |
when you use |
hmmm...let me actually try to use this |
Hi @vrbcntrl , Did it work for you ? |
Hi @eerkunt , Sorry for the delay in my response. my feature file with rule
Test results with out the and the Test results with the and the in both cases the test FAILED and thrown a However, my requirement is that when the test is run, I want terraform-compliance to return WARNING/INFO message something like below even though the condition is not met Example WARNING/INFO message:
do you think this is possible? Thanks in advance! |
I think this is possible but requires #170 to be completed first, since listing all failures would be logical if we are not going to fail the tests. Will also do this on Xmas break. |
Awesome!!! Happy holidays :) |
Added This issue will be considered as fixed after #196 is merged. |
Hi @vrbcntrl, Have a try with Thanks! 🎉 |
Hi @eerkunt , Thanks for the update. I have tested my use case with v 1.1.3 and below is my output so, like I mentioned in the screen shot, I want the status to be displayed as "Warning" instead of "Failure". please let me know if that is feasible? may be if we mark the rule/test with a tag something like "@warning", would it be possible to return the Warning status for the rules that are marked with @warning tags? please let me know. Thanks in advance! |
Hi @vrbcntrl, I thought, the main use case you have asked is to continue processing and show all failures that exists and do not fail the whole execution. Currently it runs exactly like that, but as I understand instead of I also saw other problems, it looks like terminal colours and some new lines gone mad in Windows Systems. :( By the way, yes technically it is possible to change |
Thanks @eerkunt for the quick update. I have tried with below 2 rules on 1.1.14
and I got the below output for a rule with @warning tag and with out the @warning tag rule However, my requirement is to throw a warning message for rules that are marked with @warning tags. To answer your question on the reasoning:
I hope I am able to explain my use cases better this time :) let me know if you have additional questions. |
forgot to mention that I had created this env variable on my machine before running the above test
|
Hi @vrbcntrl, There was no It looks like, you actually wanted to tag the Scenarios for I think we can do it either via preconditions (and a new step) or tags, I will have a look. |
Thank you @eerkunt , I appreciate all your efforts on this. |
I think this time I did what you have asked. Please have a look here. 1.1.5 is coming in few. |
Yep, 1.1.5 is out, have a try! Please read the doc before you use :) |
AFAIK, that is not possible :) That is internal to radish. I will have a look though. |
Thank you @eerkunt |
Just had a look there might be a way but it will make things quite dirty. Due to changes in Closing the issue, please let me know if you need any further assistance. |
This issue's conversation is now locked. If you want to continue this discussion please open a new issue. |
Feature Request
Hi eerkunt,
I hope everything is going fine at your end.
Feature description :
do you have any plans to add the test status : Warning or Info?
currently when a test is executed it either Pass or FAIL, however I am wondering if you have any plans to add the status like Warning/Info for cases where it is not necessary to FAIL a test, but return Warnings/Info messages to the consumer?
Thanks in advance!
The text was updated successfully, but these errors were encountered: