Missing YAML blocks in the output

[lcapriotti]

Information included in YAML blocks of TAP 13 files, such as the first example on

https://testanything.org/tap-version-13-specification.html

are not contained in the converted file.

[thomasboni]

Hello, I have the same issue, messages are ignored. Example:

Input TAP file (version 13):

TAP version 13
1..20
ok 1 - /mnt/.gitlab-ci.yml
ok 2 - /mnt/conf/values/secrets.staging.yaml
ok 3 - /mnt/conf/values/secrets.production.yaml
ok 4 - /mnt/conf/values/frontend.review.yaml
not ok 5 - /mnt/charts/grid-api/templates/secrets.yaml
  ---
  message: /mnt/charts/grid-api/templates/hook_jobs.yaml\n  1 1       warning  missing document start "---"  (document-
start)\n  1 3       error    syntax error  expected the node content, but found '-' (syntax)\n  5 44      warning  too 
many spaces inside braces  (braces)\n  6 28      warning  too many spaces inside braces  (braces)\n
  ...
not ok 6 - /mnt/charts/grid-api/templates/configmap.yaml
  ---
  message: /mnt/charts/grid-api/templates/configmap.yaml\n  1 1       warning  missing document start "---"  (document-
start)\n  4 11      warning  too many spaces inside braces  (braces)\n  4 41      warning  too many spaces inside brace
s  (braces)\n  7 1       error    syntax error  could not find expected ' ' (syntax)\n
  ...
ok 7 - /mnt/charts/grid-api/Chart.yaml
ok 8 - /mnt/charts/grid-api/values.yaml
ok 9 - /mnt/mkdocs.yml

Output:

# cat yaml-check.tap | tap-junit
<?xml version="1.0"?>
<testsuites tests="20" name="Tap-Junit" failures="7" errors="0">
 <testsuite tests="20" failures="7" errors="0" name="/mnt/.gitlab-ci.yml">
   <testcase name="#1 /mnt/.gitlab-ci.yml"/>
   <testcase name="#2 /mnt/conf/values/secrets.staging.yaml"/>
   <testcase name="#3 /mnt/conf/values/secrets.production.yaml"/>
   <testcase name="#4 /mnt/conf/values/frontend.review.yaml"/>
   <testcase name="#5 /mnt/charts/grid-api/templates/secrets.yaml">
     <failure message="not ok 5 - /mnt/charts/grid-api/templates/secrets.yaml"/>
   </testcase>
   <testcase name="#6 /mnt/charts/grid-api/templates/configmap.yaml">
     <failure message="not ok 6 - /mnt/charts/grid-api/templates/configmap.yaml"/>
   </testcase>
   <testcase name="#7 /mnt/charts/grid-api/Chart.yaml"/>
   <testcase name="#8 /mnt/charts/grid-api/values.yaml"/>
   <testcase name="#9 /mnt/mkdocs.yml"/>
 </testsuite>
</testsuites>

[dhershman1]

Hello all, I guess github didn't want to notify me of this, or it was when I went on my small hiatus and the notification expired.

Fear not, for I shall look into this, my only fear is that it may be a problem with the parser I am using as a dependency, so we shall see.

Edit:
@thomasboni I'm surprised that worked at all for you... because the --- or the ... is causing the tap-out parser I use to just die instantly. I can't get a running example of the examples you posted here to save my life.

So... That's interesting! It may be time to find a new tap parser possibly switching to tap-parser, which is built by the tape team and I like tape... Or build my own, which I am unsure if I want to do.

Edit 2:
The problem here is that tap-parser I lose the ability to keep tests separated. so for example if you're using tape, and have multiple test functions with asserts in them, all the asserts get jumbled together.

The structure of the junit xml becomes:

<testsuites tests="2" name="Tap-Junit" failures="2" errors="0">
  <testcase name="#1 test is equal"/>
  <testcase name="#2 test skip extra # SKIP">
    <skipped/>
  </testcase>
  <testcase name="#3 should not be equal"/>
  <testcase name="#4 should be equal"/>
</testsuites>

Instead of

<testsuites tests="4" name="suite-name" failures="0" errors="0">
  <testsuite tests="3" failures="0" errors="0" name="1 === 1">
    <testcase name="#1 test is equal"/>
    <testcase name="#2 test skip extra # SKIP">
      <skipped/>
    </testcase>
    <testcase name="#3 should not be equal"/>
  </testsuite>
  <testsuite tests="1" failures="0" errors="0" name="2 === 2">
    <testcase name="#4 should be equal"/>
  </testsuite>
  <testsuite tests="0" failures="0" errors="0" name="SKIP skipped test"/>
</testsuites>

Which I am currently unsure if this breaks standard junit practice as I've fallen out of familiarity with the tool

[thomasboni]

Hello @dhershman1, I didn't get notification about your mention. Thanks for the work on this issue !

It solves the issue for several situations but not all. Here an example using the new version v4.0.0 inside an instance of the docker image github/super-linter:v3 (I installed tap-junit using npm):

bash-5.0# cat super-linter-PYTHON_PYLINT.tap 
TAP version 13
1..13
ok 1 - /mnt/manage.py
ok 2 - /mnt/grid/grid.py
ok 3 - /mnt/grid/views.py
ok 4 - /mnt/grid/models.py
ok 5 - /mnt/grid/utils.py
ok 6 - /mnt/grid/__init__.py
ok 7 - /mnt/grid/settings.py
ok 8 - /mnt/tests/unit/test_grid.py
not ok 9 - /mnt/tests/functional/test_api.py
  ---
  message: ************* Module test_api\ntests/functional/test_api.py 12 19  E0602  Undefined variable 'app' (undefined-variable)\ntests/functional/test_api.py 18 12  E0602  Undefined variable 'db' (undefined-variable)\ntests/functional/test_api.py 24 12  E0602  Undefined variable 'db' (undefined-variable)\ntests/functional/test_api.py 25 12  E0602  Undefined variable 'db' (undefined-variable)\ntests/functional/test_api.py 43 25  E0602  Undefined variable 'json' (undefined-variable)\n
  ...
ok 10 - /mnt/migrations/env.py
ok 11 - /mnt/migrations/versions/c80a72291fce_.py
ok 12 - /mnt/migrations/versions/461d24fd37a8_.py
ok 13 - /mnt/run.py
bash-5.0# 
bash-5.0# 
bash-5.0# cat super-linter-PYTHON_PYLINT.tap | tap-junit -p > /tmp/super-linter-PYTHON_PYLINT.xml
bash-5.0# 
bash-5.0# 
bash-5.0# cat /tmp/super-linter-PYTHON_PYLINT.xml 
<?xml version="1.0"?>
<testsuites tests="13" name="Tap-Junit" failures="1">
  <testsuite tests="13" skipped="0" failures="1">
    <testcase name="/mnt/manage.py" id="1"/>
    <testcase name="/mnt/grid/grid.py" id="2"/>
    <testcase name="/mnt/grid/views.py" id="3"/>
    <testcase name="/mnt/grid/models.py" id="4"/>
    <testcase name="/mnt/grid/utils.py" id="5"/>
    <testcase name="/mnt/grid/__init__.py" id="6"/>
    <testcase name="/mnt/grid/settings.py" id="7"/>
    <testcase name="/mnt/tests/unit/test_grid.py" id="8"/>
    <testcase name="/mnt/tests/functional/test_api.py" id="9">
      <failure message="" type="fail">
</failure>
    </testcase>
    <testcase name="/mnt/migrations/env.py" id="10"/>
    <testcase name="/mnt/migrations/versions/c80a72291fce_.py" id="11"/>
    <testcase name="/mnt/migrations/versions/461d24fd37a8_.py" id="12"/>
    <testcase name="/mnt/run.py" id="13"/>
  </testsuite>
</testsuites>

[thomasboni]

I do some tests and the issues disappear when I remove ************* at the beginning of the message

[dhershman1]

@thomasboni I'm not sure if the asterisks are messing with the parser or not. I'd have to test that a bit

[thomasboni]

Hello @dhershman1, I am not sure this is the right place to discuss this, but I would like to let you know how we are using your tool in our company: we provide a hub of open-source plug and play CI/CD jobs for Gitlab runner : https://R2Devops.io

Your tool permit us to convert the tap output of some jobs into junit xml to integrates jobs reports in Gitlab, so thank you very much for that 😉