Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Traceback for malformed test metadata #1881

Closed
lukaszachy opened this issue Mar 1, 2023 · 2 comments · Fixed by #1905
Closed

Traceback for malformed test metadata #1881

lukaszachy opened this issue Mar 1, 2023 · 2 comments · Fixed by #1905
Assignees
@idorax
Labels
step | discover Stuff related to the discover step
Milestone
1.25

Comments

@lukaszachy
Copy link
Collaborator

I messed up test metadata but tmt should not traceback.

$ cat t.fmf
test: false
result: xfail

$ tmt run -avvv provision -h local
/plans/default
    discover
        how: fmf
        order: 50
        directory: /tmp/r_083846_xMJ
                warn: /t:test - False is not of type 'string'
    finish
        prune: Prune plan workdir '/var/tmp/tmt/run-027/plans/default'.
        summary: 0 tasks completed
Traceback (most recent call last):
  File "/usr/bin/tmt", line 20, in <module>
    tmt.cli.main()
  File "/usr/lib/python3.11/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/click/core.py", line 1689, in invoke
    return _process_result(rv)
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/click/core.py", line 1626, in _process_result
    value = ctx.invoke(self._result_callback, value, **ctx.params)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/tmt/cli.py", line 351, in finito
    click_context.obj.run.go()
  File "/usr/lib/python3.11/site-packages/tmt/base.py", line 2608, in go
    plan.go()
  File "/usr/lib/python3.11/site-packages/tmt/base.py", line 1611, in go
    step.go()
  File "/usr/lib/python3.11/site-packages/tmt/steps/discover/__init__.py", line 267, in go
    phase.go()
  File "/usr/lib/python3.11/site-packages/tmt/steps/discover/fmf.py", line 519, in go
    self._tests = tree.tests(
                  ^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/tmt/base.py", line 2079, in tests
    selected_tests = [
                     ^
  File "/usr/lib/python3.11/site-packages/tmt/base.py", line 2080, in <listcomp>
    Test(
  File "/usr/lib/python3.11/site-packages/tmt/base.py", line 835, in __init__
    super().__init__(
  File "/usr/lib/python3.11/site-packages/tmt/base.py", line 492, in __init__
    super().__init__(node=node, logger=logger, parent=parent, name=node.name, **kwargs)
  File "/usr/lib/python3.11/site-packages/tmt/utils.py", line 3760, in __init__
    super().__init__(node=node, logger=logger, **kwargs)
  File "/usr/lib/python3.11/site-packages/tmt/utils.py", line 4155, in __init__
    self._load_keys(node.get(), node.name, logger)
  File "/usr/lib/python3.11/site-packages/tmt/utils.py", line 4128, in _load_keys
    value = dataclass_normalize_field(self, keyname, value, logger)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/tmt/utils.py", line 3794, in dataclass_normalize_field
    value = normalize_callback(raw_value, logger)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/tmt/base.py", line 721, in _normalize_test
    return ShellScript(value)
           ^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/tmt/utils.py", line 320, in __init__
    self._script = textwrap.dedent(script)
                   ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/textwrap.py", line 435, in dedent
    text = _whitespace_only_re.sub('', text)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: expected string or bytes-like object, got 'bool'
@lukaszachy lukaszachy added the step | discover Stuff related to the discover step label Mar 1, 2023
@idorax idorax self-assigned this Mar 3, 2023
@idorax
Copy link
Contributor

idorax commented Mar 9, 2023

The root cause is if we call ShellScript(value), arg value must be a string. e.g.

$ python3 -m pdb ~/.virtualenvs/tmt/bin/tmt run -avvv provision -h local
...<snip>...
(Pdb) b tmt/base.py:754
Breakpoint 1 at /home/huanli/.virtualenvs/tmt/lib/python3.10/site-packages/tmt/base.py:754
(Pdb) b tmt/utils.py:320
Breakpoint 2 at /home/huanli/.virtualenvs/tmt/lib/python3.10/site-packages/tmt/utils.py:320
...<snip>..........................
(Pdb) c
/var/tmp/tmt/run-001
Found 1 plan.

/plans/foo
summary: Basic smoke test
    discover
        how: fmf
        order: 50
        directory: /home/huanli/dev/1881/foo
                warn: /tests/foo:test - False is not of type 'string'
> /home/huanli/.virtualenvs/tmt/lib/python3.10/site-packages/tmt/base.py(754)_normalize_test()
-> return ShellScript(value)
(Pdb) l
749  	        # TODO: mandatory schema validation would remove the need for Optional...
750  	        # `test` is mandatory, must exist, so how to initialize if it's missing :(
751  	        if value is None:
752  	            return ShellScript('')
753  	
754 B->	        return ShellScript(value)
755  	
756  	    def _normalize_require(
757  	            self,
758  	            value: Optional[_RawRequire],
759  	            logger: tmt.log.Logger) -> List[Require]:
(Pdb) p type(value)
<class 'bool'>
(Pdb) c
> /home/huanli/.virtualenvs/tmt/lib/python3.10/site-packages/tmt/utils.py(320)__init__()
-> self._script = textwrap.dedent(script)
(Pdb) p script
False

To fix it, we should make sure value is a string before invoking ShellScript(value).

@idorax idorax mentioned this issue Mar 9, 2023
Merged
@idorax
Copy link
Contributor

idorax commented Jun 6, 2023

With the patches from PR #1869, traceback looks like:

$ cat main.fmf 
summary: Concise summary describing what the test does
test: false 
framework: beakerlib

$ tmt run -avvv provision -h local
/var/tmp/tmt/run-044
Found 1 plan.

/plans/default
    discover
        how: fmf
        order: 50
        directory: /home/huanli/dev/1881/foo
                warn: /tests/foo:test - False is not of type 'string'
    finish
    
    Prune '/plans/default' plan workdir '/var/tmp/tmt/run-044/plans/default'.
        summary: 0 tasks completed
Traceback (most recent call last):
  File "/home/huanli/.virtualenvs/tmt/bin/tmt", line 14, in <module>
    tmt.cli.main()
  File "/home/huanli/.virtualenvs/tmt/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/home/huanli/.virtualenvs/tmt/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/home/huanli/.virtualenvs/tmt/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/huanli/.virtualenvs/tmt/lib/python3.10/site-packages/click/core.py", line 1689, in invoke
    return _process_result(rv)
  File "/home/huanli/.virtualenvs/tmt/lib/python3.10/site-packages/click/core.py", line 1626, in _process_result
    value = ctx.invoke(self._result_callback, value, **ctx.params)
  File "/home/huanli/.virtualenvs/tmt/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/home/huanli/.virtualenvs/tmt/lib/python3.10/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/huanli/.virtualenvs/tmt/lib/python3.10/site-packages/tmt/cli.py", line 360, in finito
    click_context.obj.run.go()
  File "/home/huanli/.virtualenvs/tmt/lib/python3.10/site-packages/tmt/base.py", line 3003, in go
    plan.go()
  File "/home/huanli/.virtualenvs/tmt/lib/python3.10/site-packages/tmt/base.py", line 1934, in go
    step.go()
  File "/home/huanli/.virtualenvs/tmt/lib/python3.10/site-packages/tmt/steps/discover/__init__.py", line 286, in go
    phase.go()
  File "/home/huanli/.virtualenvs/tmt/lib/python3.10/site-packages/tmt/steps/discover/fmf.py", line 526, in go
    self._tests = tree.tests(
  File "/home/huanli/.virtualenvs/tmt/lib/python3.10/site-packages/tmt/base.py", line 2468, in tests
    selected_tests = [
  File "/home/huanli/.virtualenvs/tmt/lib/python3.10/site-packages/tmt/base.py", line 2469, in <listcomp>
    Test(
  File "/home/huanli/.virtualenvs/tmt/lib/python3.10/site-packages/tmt/base.py", line 1034, in __init__
    super().__init__(
  File "/home/huanli/.virtualenvs/tmt/lib/python3.10/site-packages/tmt/base.py", line 610, in __init__
    super().__init__(node=node, logger=logger, parent=parent, name=node.name, **kwargs)
  File "/home/huanli/.virtualenvs/tmt/lib/python3.10/site-packages/tmt/utils.py", line 4233, in __init__
    super().__init__(node=node, logger=logger, **kwargs)
  File "/home/huanli/.virtualenvs/tmt/lib/python3.10/site-packages/tmt/utils.py", line 4676, in __init__
    self._load_keys(node.get(), node.name, logger)
  File "/home/huanli/.virtualenvs/tmt/lib/python3.10/site-packages/tmt/utils.py", line 4649, in _load_keys
    value = dataclass_normalize_field(self, key_address, keyname, value, logger)
  File "/home/huanli/.virtualenvs/tmt/lib/python3.10/site-packages/tmt/utils.py", line 4268, in dataclass_normalize_field
    value = normalize_callback(key_address, raw_value, logger)
  File "/home/huanli/.virtualenvs/tmt/lib/python3.10/site-packages/tmt/base.py", line 915, in <lambda>
    None if raw_value is None else ShellScript(raw_value))
  File "/home/huanli/.virtualenvs/tmt/lib/python3.10/site-packages/tmt/utils.py", line 355, in __init__
    self._script = textwrap.dedent(script)
  File "/usr/lib64/python3.10/textwrap.py", line 438, in dedent
    text = _whitespace_only_re.sub('', text)
TypeError: expected string or bytes-like object

@psss psss changed the title traceback for malformed test metadata Traceback for malformed test metadata Jun 20, 2023
@psss psss added this to the 1.25 milestone Jun 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@idorax idorax

Labels
step | discover Stuff related to the discover step
Projects
None yet
Milestone
1.25
Development

Successfully merging a pull request may close this issue.

Raise error if malformed test metadata is given teemtee/tmt
3 participants
@lukaszachy @psss @idorax