testsuite failure on i386 after applying test fix from master #60

LocutusOfBorg · 2017-06-24T00:05:56Z

Hello, as said, I updated libyaml with the fix for the testsuite failures (e.g. haskell-yaml), in particular this single commit
56400d9

the problem is that now pyyaml has some test failing on i386
original yaml (latest version)
https://launchpadlibrarian.net/325264273/buildlog_ubuntu-artful-i386.pyyaml_3.12-1build2_BUILDING.txt.gz
patched yaml (commit 56400d9)

https://launchpadlibrarian.net/325269141/buildlog_ubuntu-artful-i386.pyyaml_3.12-1build2_BUILDING.txt.gz

set -e && for i in 2.7; do \
  echo "-- running tests for "$i" plain --" ; \
  python$i -c "import sys ; sys.path.insert(0, 'debian/python-yaml/usr/lib/python$i/dist-packages/'); \
      sys.path.insert(0, 'tests/lib'); import test_all; test_all.main([])";\
done
-- running tests for 2.7 plain --
..............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................FF..F.....F....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
===========================================================================
test_emitter_styles_ext(tests/data/spec-07-09.data, tests/data/spec-07-09.canonical): FAILURE
Traceback (most recent call last):
  File "tests/lib/test_appliance.py", line 65, in execute
    function(verbose=verbose, *filenames)
  File "tests/lib/test_yaml_ext.py", line 240, in wrapper
    function(*args, **kwds)
  File "tests/lib/test_emitter.py", line 62, in test_emitter_styles
    _compare_events(events, new_events)
  File "tests/lib/test_emitter.py", line 15, in _compare_events
    assert event1.value == event2.value, (event1, event2)
AssertionError: see below
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
(ScalarEvent(anchor=None, tag=u'tag:yaml.org,2002:str', implicit=(False, False), value=u'foo'),
 ScalarEvent(anchor=None, tag=u'tag:yaml.org,2002:str', implicit=(False, False), value=u'foo %YAML 1.1'))
---------------------------------------------------------------------------
tests/data/spec-07-09.data:
---
foo
...
# Repeated end marker.
...
---
bar
# No end marker.
---
baz
...
---------------------------------------------------------------------------
tests/data/spec-07-09.canonical:
%YAML 1.1
---
!!str "foo"
%YAML 1.1
---
!!str "bar"
%YAML 1.1
---
!!str "baz"
===========================================================================
test_emitter_styles_ext(tests/data/spec-07-10.data, tests/data/spec-07-10.canonical): FAILURE
Traceback (most recent call last):
  File "tests/lib/test_appliance.py", line 65, in execute
    function(verbose=verbose, *filenames)
  File "tests/lib/test_yaml_ext.py", line 240, in wrapper
    function(*args, **kwds)
  File "tests/lib/test_emitter.py", line 62, in test_emitter_styles
    _compare_events(events, new_events)
  File "tests/lib/test_emitter.py", line 15, in _compare_events
    assert event1.value == event2.value, (event1, event2)
AssertionError: see below
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
(ScalarEvent(anchor=None, tag=u'tag:yaml.org,2002:str', implicit=(False, False), value=u'Root flow scalar'),
 ScalarEvent(anchor=None, tag=u'tag:yaml.org,2002:str', implicit=(False, False), value=u'Root flow scalar %YAML 1.1'))
---------------------------------------------------------------------------
tests/data/spec-07-10.data:
"Root flow
 scalar"
--- !!str >
 Root block
 scalar
---
# Root collection:
foo : bar
... # Is optional.
---
# Explicit document may be empty.
---------------------------------------------------------------------------
tests/data/spec-07-10.canonical:
%YAML 1.1
---
!!str "Root flow scalar"
%YAML 1.1
---
!!str "Root block scalar\n"
%YAML 1.1
---
!!map {
  ? !!str "foo"
  : !!str "bar"
}
---
#!!str ""
!!null ""
===========================================================================
test_emitter_styles_ext(tests/data/spec-07-13.data, tests/data/spec-07-13.canonical): FAILURE
Traceback (most recent call last):
  File "tests/lib/test_appliance.py", line 65, in execute
    function(verbose=verbose, *filenames)
  File "tests/lib/test_yaml_ext.py", line 240, in wrapper
    function(*args, **kwds)
  File "tests/lib/test_emitter.py", line 62, in test_emitter_styles
    _compare_events(events, new_events)
  File "tests/lib/test_emitter.py", line 15, in _compare_events
    assert event1.value == event2.value, (event1, event2)
AssertionError: see below
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
(ScalarEvent(anchor=None, tag=u'!foo', implicit=(False, False), value=u'No directives'),
 ScalarEvent(anchor=None, tag=u'!foo', implicit=(False, False), value=u'No directives %TAG ! %21foo'))
---------------------------------------------------------------------------
tests/data/spec-07-13.data:
! "First document"
---
!foo "No directives"
%TAG ! !foo
---
!bar "With directives"
%YAML 1.1
---
!baz "Reset settings"
---------------------------------------------------------------------------
tests/data/spec-07-13.canonical:
%YAML 1.1
---
!!str "First document"
---
!<!foo> "No directives"
---
!<!foobar> "With directives"
---
!<!baz> "Reset settings"
===========================================================================
test_emitter_styles_ext(tests/data/spec-08-08.data, tests/data/spec-08-08.canonical): FAILURE
Traceback (most recent call last):
  File "tests/lib/test_appliance.py", line 65, in execute
    function(verbose=verbose, *filenames)
  File "tests/lib/test_yaml_ext.py", line 240, in wrapper
    function(*args, **kwds)
  File "tests/lib/test_emitter.py", line 62, in test_emitter_styles
    _compare_events(events, new_events)
  File "tests/lib/test_emitter.py", line 15, in _compare_events
    assert event1.value == event2.value, (event1, event2)
AssertionError: see below
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
(ScalarEvent(anchor=None, tag=u'tag:yaml.org,2002:str', implicit=(False, False), value=u'foo bar'),
 ScalarEvent(anchor=None, tag=u'tag:yaml.org,2002:str', implicit=(False, False), value=u'foo bar %YAML 1.1'))
---------------------------------------------------------------------------
tests/data/spec-08-08.data:
---
foo:
 "bar
 baz"
---
"foo
 bar"
---
foo
 bar
--- |
 foo
...
---------------------------------------------------------------------------
tests/data/spec-08-08.canonical:
%YAML 1.1
---
!!map {
  ? !!str "foo"
  : !!str "bar baz"
}
%YAML 1.1
---
!!str "foo bar"
%YAML 1.1
---
!!str "foo bar"
%YAML 1.1
---
!!str "foo\n"
===========================================================================
TESTS: 2573
FAILURES: 4
set -e && for i in 2.7; do \
  echo "-- running tests for "$i" debug --" ; \
  python$i-dbg -c "import sys ; sys.path.insert(0, 'debian/python-yaml-dbg/usr/lib/python$i/dist-packages/'); \
      sys.path.insert(0, 'tests/lib'); import test_all; test_all.main([])";\
done
-- running tests for 2.7 debug --
..............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................FF..F.....F....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................

can you please help?

thanks a lot
(the same failure happens with the latest git master FWIW)

The text was updated successfully, but these errors were encountered:

jrtc27 · 2017-06-29T00:55:27Z

So the problem here is when emitting two documents back to back with the plain style, where one document is "foo" and the other is "bar". In the double-quoted style, this becomes:

%YAML 1.1
--- !!str "foo"
%YAML 1.1
--- !!str "bar"

However, in the plain style, this is emitted as:

%YAML 1.1
--- !!str foo
%YAML 1.1
--- !!str bar

This is now incorrect, as the plain style allows continuation lines, so the second %YAML 1.1 should be parsed as part of the string foo, giving !!str "foo %YAML 1.1". This used to be fine, because open_ended was set, so an explicit terminator was added, giving:

%YAML 1.1
--- !!str foo
...
%YAML 1.1
--- !!str bar
...

Therefore, I believe that the commit should be reverted, as it was serving a purpose. Technically the final terminator in the stream is not needed, but it allows streams to be concatenated without having to worry, so I guess it's fine (though if you dropped it, I imagine that would also fix the tests in consumers of the library, as they probably aren't expecting the final one; this is certainly the case with haskell-yaml).

jrtc27 · 2017-06-29T01:27:35Z

Indeed, with the following patches, pyyaml's test suite passes again, but the final unnecessary trailing terminator is not emitted.

From 7c2e6f47e1b12ca3dfa9ea2f6c65e9bd8a02e8c8 Mon Sep 17 00:00:00 2001
From: James Clarke <[email protected]>
Date: Thu, 29 Jun 2017 02:24:53 +0100
Subject: [PATCH 1/2] Revert "This code is not needed and breaks tests"

This reverts commit 56400d976a1999156b1abfd674c3122843980260.
---
 src/emitter.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/src/emitter.c b/src/emitter.c
index a5b7ff8..c593a7d 100644
--- a/src/emitter.c
+++ b/src/emitter.c
@@ -1946,6 +1946,10 @@ yaml_emitter_write_plain_scalar(yaml_emitter_t *emitter,
 
     emitter->whitespace = 0;
     emitter->indention = 0;
+    if (emitter->root_context)
+    {
+        emitter->open_ended = 1;
+    }
 
     return 1;
 }
-- 
2.13.2


From e5ebb70a01cc0fbd9e519050d51d704f58c2a33d Mon Sep 17 00:00:00 2001
From: James Clarke <[email protected]>
Date: Thu, 29 Jun 2017 02:26:09 +0100
Subject: [PATCH 2/2] Skip trailing document terminator at the end of the
 stream

---
 src/emitter.c | 8 --------
 1 file changed, 8 deletions(-)

diff --git a/src/emitter.c b/src/emitter.c
index c593a7d..1502590 100644
--- a/src/emitter.c
+++ b/src/emitter.c
@@ -649,14 +649,6 @@ yaml_emitter_emit_document_start(yaml_emitter_t *emitter,
 
     else if (event->type == YAML_STREAM_END_EVENT)
     {
-        if (emitter->open_ended)
-        {
-            if (!yaml_emitter_write_indicator(emitter, "...", 1, 0, 0))
-                return 0;
-            if (!yaml_emitter_write_indent(emitter))
-                return 0;
-        }
-
         if (!yaml_emitter_flush(emitter))
             return 0;
 
-- 
2.13.2

LocutusOfBorg · 2017-07-07T13:10:41Z

ping

LocutusOfBorg · 2017-07-10T11:18:33Z

@ingydotnet do you have any advice for this patch?

jrtc27 · 2017-08-15T21:15:35Z

Ping

ingydotnet · 2018-01-08T04:12:54Z

I will add this patch as a candidate to the upcoming 0.1.8 release. https://github.com/yaml/libyaml/projects/1

LocutusOfBorg · 2018-06-27T08:14:10Z

@ingydotnet I think there has been a mistake in this patch apply...

I see the latest libyaml included the first version of my patch, but the one from @jrtc27 is a better version... can you please revert and apply this one? #60 (comment)

LocutusOfBorg · 2018-06-27T08:15:17Z

or maybe you have your reasons for this? 56400d9#diff-55aa019b48be9270c41a41aad1bd405c
the commit message seems to indicate this has been not a mistake...

andersk · 2018-06-27T17:40:17Z

I’m not sure this should be closed. PyYAML 4.1 still fails its test suite with LibYAML 0.2.1 as described in the original report.

LocutusOfBorg · 2018-06-27T18:05:42Z

reopened!

straight-shoota · 2018-06-28T18:47:36Z

This has come up as an issue with the libyaml-based YAML builder for Crystal in crystal-lang/crystal#6283
The change to remove document end marker was certainly good. And it probably doesn't have a huge negative impact as YAML documents consisting of only a single scalar are not very common outside of YAML test suites. But it is still a breaking change and I'd suggest to mention this in the release announcement to make it easier to discover.

See also issue #60 Revert "This code is not needed and breaks tests" This reverts commit 56400d9.

perlpunk · 2018-06-29T19:52:03Z

I created #122 to revert the commit

perlpunk · 2018-06-29T20:08:53Z

I explained in my PR why this needs to be reverted. libyaml is creating invalid YAML in some cases.

perlpunk · 2018-06-29T20:10:24Z

@jrtc27 I'll have a look at your patches, thanks

@jrtc2

(only when explicitly requested) @jrtc2++ for the patch. See #60

perlpunk · 2018-06-29T21:27:51Z

I added the patch from @jrtc27 to #122, looks good!

@jrtc27

(only when explicitly requested) @jrtc27++ for the patch. See #60

perlpunk · 2019-06-07T22:16:39Z

The fixes have been released, so I'm closing this.
Commits: 56400d9, 8ee83c0, 56f4b17

In YAML 1.1, the document end marker `...` is optional even if the next document starts with a directive: https://github.com/yaml/pyyaml/blob/master/tests/data/spec-07-09.canonical ``` %YAML 1.1 --- !!str "foo" %YAML 1.1 --- !!str "bar" %YAML 1.1 --- !!str "baz" ``` It is only required if the scalar is "open ended", for example for plain scalars. In YAML 1.2 the `...` marker is always required before a directive. My suggestion would be to make the output 1.2 compatible. It will still be 1.1 compatible, so that shouldn't be a problem. I believe this will also make it easier to fix #123 which was introduced with the last fixes regarding `open_ended`. I think I can make a fix for this soon after this issue is fixed. Fixing #123 without this would be a bit more complicated. If we do this, we also need to adjust PyYAML to behave the same. Related issues/commits: - #60 - #122 - 56400d9, 8ee83c0, 56f4b17

LocutusOfBorg mentioned this issue Jun 29, 2017

testsuite failure with latest Debian and yaml packages. snoyberg/yaml#110

Closed

LocutusOfBorg closed this as completed Jun 27, 2018

andersk mentioned this issue Jun 27, 2018

Fails spec-07-09, spec-07-10, spec-07-13, spec-08-08 when built with LibYAML 0.2.1 yaml/pyyaml#184

Closed

LocutusOfBorg reopened this Jun 27, 2018

perlpunk added a commit that referenced this issue Jun 29, 2018

Revert removing of open_ended after top level plain scalar

8ee83c0

See also issue #60 Revert "This code is not needed and breaks tests" This reverts commit 56400d9.

perlpunk mentioned this issue Jun 29, 2018

Revert removing of open_ended after top level plain scalar #122

Merged

perlpunk added a commit that referenced this issue Jun 29, 2018

Don't emit document-end marker at the end of stream

8e33199

(only when explicitly requested) @jrtc2++ for the patch. See #60

perlpunk added a commit that referenced this issue Jun 29, 2018

Don't emit document-end marker at the end of stream

56f4b17

(only when explicitly requested) @jrtc27++ for the patch. See #60

perlpunk mentioned this issue Aug 27, 2018

Version 3.13 fails test suite after libyaml update to 0.2.1 yaml/pyyaml#214

Closed

perlpunk closed this as completed Jun 7, 2019

perlpunk mentioned this issue Dec 26, 2019

Always output document end before directive (YAML 1.2 compatibility) #160

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

testsuite failure on i386 after applying test fix from master #60

testsuite failure on i386 after applying test fix from master #60

LocutusOfBorg commented Jun 24, 2017 •

edited

Loading

jrtc27 commented Jun 29, 2017 •

edited

Loading

jrtc27 commented Jun 29, 2017

LocutusOfBorg commented Jul 7, 2017

LocutusOfBorg commented Jul 10, 2017

jrtc27 commented Aug 15, 2017

ingydotnet commented Jan 8, 2018

LocutusOfBorg commented Jun 27, 2018

LocutusOfBorg commented Jun 27, 2018

andersk commented Jun 27, 2018

LocutusOfBorg commented Jun 27, 2018

straight-shoota commented Jun 28, 2018

perlpunk commented Jun 29, 2018

perlpunk commented Jun 29, 2018

perlpunk commented Jun 29, 2018

perlpunk commented Jun 29, 2018

perlpunk commented Jun 7, 2019

testsuite failure on i386 after applying test fix from master #60

testsuite failure on i386 after applying test fix from master #60

Comments

LocutusOfBorg commented Jun 24, 2017 • edited Loading

jrtc27 commented Jun 29, 2017 • edited Loading

jrtc27 commented Jun 29, 2017

LocutusOfBorg commented Jul 7, 2017

LocutusOfBorg commented Jul 10, 2017

jrtc27 commented Aug 15, 2017

ingydotnet commented Jan 8, 2018

LocutusOfBorg commented Jun 27, 2018

LocutusOfBorg commented Jun 27, 2018

andersk commented Jun 27, 2018

LocutusOfBorg commented Jun 27, 2018

straight-shoota commented Jun 28, 2018

perlpunk commented Jun 29, 2018

perlpunk commented Jun 29, 2018

perlpunk commented Jun 29, 2018

perlpunk commented Jun 29, 2018

perlpunk commented Jun 7, 2019

LocutusOfBorg commented Jun 24, 2017 •

edited

Loading

jrtc27 commented Jun 29, 2017 •

edited

Loading