Fix #70; Consume all tokens in simplemrs.deserialize

CHANGELOG.md

@@ -2,6 +2,10 @@
 
 ## [Unreleased][unreleased]
 
+### Fixed
+
+* Fixed a regression in reading multiple SimpleMRSs (#70)
+
 ## [v0.4.1][]
 
 This release fixes a number of bugs and adds some minor features. The

delphin/__about__.py

@@ -3,7 +3,7 @@
 # the warehouse project:
 #   https://github.com/pypa/warehouse/blob/master/warehouse/__about__.py
 
-__version__ = '0.4.1'
+__version__ = '0.4.2-pre.1'
 __version_info__ = __version__.replace('.', ' ').replace('-', ' ').split()
 
 __title__ = 'pyDelphin'

delphin/mrs/simplemrs.py

@@ -218,7 +218,7 @@ def _read_mrs(tokens, version, errors):
         rels = _read_rels(tokens, vars_)
         hcons = _read_cons(tokens, 'HCONS', vars_)
         icons = _read_cons(tokens, 'ICONS', vars_)
-        _read_literals(']')
+        _read_literals(tokens, ']')
         # at this point, we could uniquify proplists in vars_, but most
         # likely it isn't necessary, and might night harm things if we
         # leave potential dupes in there. let's see how it plays out.

tests/simplemrs_test.md

@@ -137,6 +137,36 @@ Lnk values can take 4 forms in SimpleMRS:
 
 ```
 
+Multiple MRSs can be parsed with `simplemrs.loads()`:
+
+"It rains." and "It snows."
+
+```python
+>>> mrs_gen = simplemrs.loads('''[ LTOP: h0
+... INDEX: e2 [ e SF: prop TENSE: pres MOOD: indicative PROG: - PERF: - ]
+... RELS: < [ "_rain_v_1_rel"<3:9> LBL: h1 ARG0: e2 ] >
+... HCONS: < h0 qeq h1 > ] [ LTOP: h0
+... INDEX: e2 [ e SF: prop TENSE: pres MOOD: indicative PROG: - PERF: - ]
+... RELS: < [ "_snow_v_1_rel"<3:9> LBL: h1 ARG0: e2 ] >
+... HCONS: < h0 qeq h1 > ]''')
+>>> next(mrs_gen)  # doctest: +ELLIPSIS
+<Xmrs object (rain) at ...>
+>>> next(mrs_gen)  # doctest: +ELLIPSIS
+<Xmrs object (snow) at ...>
+
+```
+
+But requesting more than is available causes a `StopIteration` exception:
+
+```python
+>>> try:
+...     next(mrs_gen)
+... except StopIteration:
+...     print('StopIteration caught!')
+StopIteration caught!
+
+```
+
 "It rains", SimpleMRS 1.1 format without surface forms or a top LNK value.
 
 ```python