Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Expose "key address" to normalization callbacks #1869

Merged
merged 5 commits into from
Jun 6, 2023

Conversation

happz
Copy link
Collaborator

@happz happz commented Feb 22, 2023

Keys belong to various objects like step phases or fmf ids, and tmt can do better job when reporting unexpected input values. Another step is to expose the location of the key to normalization callbacks and a dedicated exception for reporting the violations in a unified manner.

The key address is already used by schema validation when reporting validation errors, and while it's often subpar, it enhances error messages a bit.

In the follow-up patches, improving the addresses will happen - for example, sometimes a class name is printed instead of a fmf node name, usually because the node name is not available.

@happz happz added the code | logging Changes related to debug logging label Feb 22, 2023
@happz happz added this to the 1.23 milestone Feb 22, 2023
@happz happz force-pushed the normalize-include-key-address branch from bec564b to 2912b56 Compare March 22, 2023 08:09
@psss psss requested review from frantisekz and thrix as code owners April 4, 2023 14:31
@idorax
Copy link
Contributor

idorax commented Apr 17, 2023

Hi @happz, do you plan get this PR merged soon? PR #1905 of mine has a dependency on this one, I'll update it once your PR is merged, thanks!

@happz
Copy link
Collaborator Author

happz commented Apr 17, 2023

Hi @happz, do you plan get this PR merged soon? PR #1905 of mine has a dependency on this one, I'll update it once your PR is merged, thanks!

Hopefully Soon (TM), depends on reviews :)

@happz happz force-pushed the normalize-include-key-address branch from 2912b56 to 2b3de71 Compare April 17, 2023 07:37
@idorax
Copy link
Contributor

idorax commented Apr 17, 2023

Hi @happz, your PR looks good to me.

And I also tested your patch by updating tmt/base.py because of PR #1905, the test result looks good.

  • Patch
(tmt) huanli@kvm-01-guest01:dev$ git branch
* dev.huanli.1881.20230417
  main
  normalize-include-key-address
(tmt) huanli@kvm-01-guest01:dev$ git diff
diff --git a/tmt/base.py b/tmt/base.py
index 7347dc4..7f316a3 100644
--- a/tmt/base.py
+++ b/tmt/base.py
@@ -766,7 +766,11 @@ class Test(Core, tmt.export.Exportable['Test']):
         # TODO: mandatory schema validation would remove the need for Optional...
         # `test` is mandatory, must exist, so how to initialize if it's missing :(
         if value is None:
-            return ShellScript('')
+            raise tmt.utils.NormalizationError(key_address, value, 'string')
+
+        # Raise an error if value is not a string, e.g. False or True
+        if not isinstance(value, str):
+            raise tmt.utils.NormalizationError(key_address, value, 'string')
 
         return ShellScript(value)
  • Test result
(tmt) huanli@kvm-01-guest01:foo$ pwd 
/home/huanli/dev/1001869/foo/tests/foo
(tmt) huanli@kvm-01-guest01:foo$ cat main.fmf 
summary: Concise summary describing what the test does
test: false 
framework: beakerlib
(tmt) huanli@kvm-01-guest01:foo$ tmt run -avvv provision -h local
/var/tmp/tmt/run-001
Found 1 plan.

/plans
summary: Basic smoke test
    discover
        how: fmf
        order: 50
        directory: /home/huanli/dev/1001869/foo
                warn: /tests/foo:test - False is not of type 'string'
    finish
        prune: Prune plan workdir '/var/tmp/tmt/run-001/plans'.
        summary: 0 tasks completed

Field "/tests/foo:test" can be string, "bool" found.
# ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  Looks good !!!

@happz happz force-pushed the normalize-include-key-address branch from 2b3de71 to 46e7b0c Compare May 10, 2023 10:25
@psss psss modified the milestones: 1.23, 1.24 May 11, 2023
@happz happz force-pushed the normalize-include-key-address branch 3 times, most recently from eecdddc to 85b182a Compare May 16, 2023 11:35
Copy link
Collaborator

@lukaszachy lukaszachy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just typos found.

tests/unit/test_base.py Outdated Show resolved Hide resolved
tmt/base.py Outdated Show resolved Hide resolved
@happz happz force-pushed the normalize-include-key-address branch from e9bfa79 to 9529a77 Compare May 16, 2023 21:44
@lukaszachy
Copy link
Collaborator

Fix for testing-farm:fedora-38-x86_64:internal-plugins in progress

@happz happz force-pushed the normalize-include-key-address branch from 9529a77 to 55c5982 Compare May 18, 2023 07:18
tests/core/fmf-id/test.sh Outdated Show resolved Hide resolved
tests/core/fmf-id/test.sh Outdated Show resolved Hide resolved
tmt/base.py Outdated Show resolved Hide resolved
@happz happz force-pushed the normalize-include-key-address branch 2 times, most recently from ede7100 to 8b0c81c Compare May 26, 2023 13:05
@happz happz force-pushed the normalize-include-key-address branch from 8b0c81c to 9295641 Compare June 1, 2023 09:21
@thrix thrix self-requested a review June 6, 2023 08:51
@thrix thrix requested a review from janhavlin June 6, 2023 08:51
Copy link
Collaborator

@thrix thrix left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, ty!

happz added 5 commits June 6, 2023 11:01
Keys belong to various objects like step phases or fmf ids, and tmt can
do better job when reporting unexpected input values. Another step is to
expose the location of the key to normalization callbacks and a
dedicated exception for reporting the violations in a unified manner.

The key address is already used by schema validation when reporting
validation errors, and while it's often subpar, it enhances error
messages a bit.

In the follow-up patches, improving the addresses will happen - for
example, sometimes a class name is printed instead of a fmf node name,
usually because the node name is not available.
@happz happz force-pushed the normalize-include-key-address branch from 2b951f1 to 75bfbbd Compare June 6, 2023 09:01
@happz happz enabled auto-merge (squash) June 6, 2023 09:05
@happz happz merged commit c6f68a1 into main Jun 6, 2023
@happz happz deleted the normalize-include-key-address branch June 6, 2023 09:52
@@ -522,7 +521,7 @@ class PrepareInstallData(tmt.steps.prepare.PrepareStepData):
)


@tmt.steps.provides_method('install')
@ tmt.steps.provides_method('install')
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was this change intentional?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops, nope, that does not look like an intention. It shouldn't have any effect, but it's not supposed to happen.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, we can fix with some other patch.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's the fix: #2128

4N0body5 pushed a commit that referenced this pull request Jun 7, 2023
Keys belong to various objects like step phases or fmf ids, and tmt can
do a better job when reporting unexpected input values. Another step is to
expose the location of the key to normalization callbacks and a
dedicated exception for reporting the violations in a unified manner.

The key address is already used by schema validation when reporting
validation errors, and while it's often subpar, it enhances error
messages a bit.

In the follow-up patches, improving the addresses will happen - for
example, sometimes a class name is printed instead of an fmf node name,
usually, because the node name is not available.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
code | logging Changes related to debug logging
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants