UnicodeDecodeError when utilizing filter rules.

[LexiconCode]

Thanks @kendonB for reporting the bug. The exact simplified filter example from the docs produces:

A bit of research

kendonB reported the error below :

'ascii' codec can't decode byte 0x83 in position 52: ordinal not in range(128)
<traceback object at 0x1127C2D8>
  File "C:\Users\kendon\Documents\caster\caster\lib\dfplus\merge\ccrmerger.py", line 354, in _run_filters
    filter_fn(merge_pair)

  File "C:\Users\kendon\Documents\caster\caster\lib\dfplus\merge\gfilter.py", line 187, in spec_override_from_config
    new_choice = Choice(extra.name, choices)

  File "C:\Python27\lib\site-packages\dragonfly-0.6.6b1-py2.7.egg\dragonfly\grammar\elements_compound.py", line 323, in __init__
    child = Compound(spec=k, value=v, extras=extras)

  File "C:\Python27\lib\site-packages\dragonfly-0.6.6b1-py2.7.egg\dragonfly\grammar\elements_compound.py", line 270, in __init__
    element = self._parser.parse(spec)

  File "C:\Python27\lib\site-packages\dragonfly-0.6.6b1-py2.7.egg\dragonfly\parser.py", line 52, in parse
    for result in generator:

  File "C:\Python27\lib\site-packages\dragonfly-0.6.6b1-py2.7.egg\dragonfly\parser.py", line 413, in parse
    try: path[-1].next()

  File "C:\Python27\lib\site-packages\dragonfly-0.6.6b1-py2.7.egg\dragonfly\parser.py", line 472, in parse
    try: path[-1].next()

  File "C:\Python27\lib\site-packages\dragonfly-0.6.6b1-py2.7.egg\dragonfly\parser.py", line 413, in parse
    try: path[-1].next()

  File "C:\Python27\lib\site-packages\dragonfly-0.6.6b1-py2.7.egg\dragonfly\parser.py", line 547, in parse
    for result in child.parse(state):

  File "C:\Python27\lib\site-packages\dragonfly-0.6.6b1-py2.7.egg\dragonfly\parser.py", line 413, in parse
    try: path[-1].next()

  File "C:\Python27\lib\site-packages\dragonfly-0.6.6b1-py2.7.egg\dragonfly\parser.py", line 708, in parse
    while not state.finished() and state.peek(1) in self._set:

Filter function 'spec_override_from_config' failed.
Error loading _caster from C:\Users\kendon\Documents\caster\_caster.py
Traceback (most recent call last):
  File "C:\NatLink\NatLink\MacroSystem\core\natlinkmain.py", line 317, in loadFile
    imp.load_module(modName,fndFile,fndName,fndDesc)
  File "C:\Users\kendon\Documents\caster\_caster.py", line 163, in <module>
    _NEXUS.merger.merge(MergeInf.BOOT)
  File "C:\Users\kendon\Documents\caster\caster\lib\dfplus\merge\ccrmerger.py", line 257, in merge
    else: base = self._compatibility_merge(mp, base, rule)
  File "C:\Users\kendon\Documents\caster\caster\lib\dfplus\merge\ccrmerger.py", line 191, in _compatibility_merge
    base = rule if base is None else base.merge(rule)
  File "C:\Users\kendon\Documents\caster\caster\lib\dfplus\merge\mergerule.py", line 112, in merge
    mcontext=context)
  File "C:\Users\kendon\Documents\caster\caster\lib\dfplus\merge\mergerule.py", line 65, in __init__
    MappingRule.__init__(self, name, mapping, extras, defaults, exported)
  File "C:\Python27\lib\site-packages\dragonfly-0.6.6b1-py2.7.egg\dragonfly\grammar\rule_mapping.py", line 146, in __init__
    c = Compound(spec, elements=self._extras, value=value)
  File "C:\Python27\lib\site-packages\dragonfly-0.6.6b1-py2.7.egg\dragonfly\grammar\elements_compound.py", line 270, in __init__
    element = self._parser.parse(spec)
  File "C:\Python27\lib\site-packages\dragonfly-0.6.6b1-py2.7.egg\dragonfly\parser.py", line 52, in parse
    for result in generator:
  File "C:\Python27\lib\site-packages\dragonfly-0.6.6b1-py2.7.egg\dragonfly\parser.py", line 413, in parse
    try: path[-1].next()
  File "C:\Python27\lib\site-packages\dragonfly-0.6.6b1-py2.7.egg\dragonfly\parser.py", line 472, in parse
    try: path[-1].next()
  File "C:\Python27\lib\site-packages\dragonfly-0.6.6b1-py2.7.egg\dragonfly\parser.py", line 413, in parse
    try: path[-1].next()
  File "C:\Python27\lib\site-packages\dragonfly-0.6.6b1-py2.7.egg\dragonfly\parser.py", line 547, in parse
    for result in child.parse(state):
  File "C:\Python27\lib\site-packages\dragonfly-0.6.6b1-py2.7.egg\dragonfly\parser.py", line 413, in parse
    try: path[-1].next()
  File "C:\Python27\lib\site-packages\dragonfly-0.6.6b1-py2.7.egg\dragonfly\parser.py", line 708, in parse
    while not state.finished() and state.peek(1) in self._set:
UnicodeDecodeError: 'ascii' codec can't decode byte 0x83 in position 52: ordinal not in range(128)

[Versatilus]

Unless I'm mistaken, that error comes from deep inside of Dragonfly. I'm not sure how much we can directly do about that. Something to try as an experiment is making sure you have the python-future module installed (pip install future) and opening _caster.py and adding from builtins import str and from __future__ import unicode_literals to the top.

[LexiconCode]

Good suggestion that makes it reproducible without filter rules. The error occurs now on start up of caster.

`Error loading _caster from C:\NatLink\NatLink\MacroSystem\_caster.py
Traceback (most recent call last):
  File "C:\NatLink\NatLink\MacroSystem\core\natlinkmain.py", line 322, in loadFile
    imp.load_module(modName,fndFile,fndName,fndDesc)
  File "C:\NatLink\NatLink\MacroSystem\_caster.py", line 69, in <module>
    class MainRule(MergeRule):
  File "C:\NatLink\NatLink\MacroSystem\_caster.py", line 131, in MainRule
    "disable": False
  File "C:\Python27\lib\site-packages\dragonfly-0.6.6b1-py2.7.egg\dragonfly\grammar\elements_compound.py", line 323, in __init__
    child = Compound(spec=k, value=v, extras=extras)
  File "C:\Python27\lib\site-packages\dragonfly-0.6.6b1-py2.7.egg\dragonfly\grammar\elements_compound.py", line 270, in __init__
    element = self._parser.parse(spec)
  File "C:\Python27\lib\site-packages\dragonfly-0.6.6b1-py2.7.egg\dragonfly\parser.py", line 52, in parse
    for result in generator:
  File "C:\Python27\lib\site-packages\dragonfly-0.6.6b1-py2.7.egg\dragonfly\parser.py", line 413, in parse
    try: path[-1].next()
  File "C:\Python27\lib\site-packages\dragonfly-0.6.6b1-py2.7.egg\dragonfly\parser.py", line 472, in parse
    try: path[-1].next()
  File "C:\Python27\lib\site-packages\dragonfly-0.6.6b1-py2.7.egg\dragonfly\parser.py", line 413, in parse
    try: path[-1].next()
  File "C:\Python27\lib\site-packages\dragonfly-0.6.6b1-py2.7.egg\dragonfly\parser.py", line 547, in parse
    for result in child.parse(state):
  File "C:\Python27\lib\site-packages\dragonfly-0.6.6b1-py2.7.egg\dragonfly\parser.py", line 413, in parse
    try: path[-1].next()
  File "C:\Python27\lib\site-packages\dragonfly-0.6.6b1-py2.7.egg\dragonfly\parser.py", line 708, in parse
    while not state.finished() and state.peek(1) in self._set:
UnicodeDecodeError: 'ascii' codec can't decode byte 0x83 in position 52: ordinal not in range(128)`

[LexiconCode]

Now posted within dictation-toolbox/dragonfly#12

[Versatilus]

I had hoped it would fix it…

[Versatilus]

I just tried adding those lines myself and had no problems whatsoever. My Dragonfly installation has been hacked together, though.

[LexiconCode]

Hmm, I'm using Danesprite/dragonfly fork.

[LexiconCode]

@kendonB
Could you try updating your Danesprite/dragonfly install and see if it fixes your issue?

git clone --recursive https://github.com/Danesprite/dragonfly.git
git submodule foreach python setup.py install

[drmfinlay]

I just verified this is definitely fixed in my fork and closed dictation-toolbox/dragonfly#12.

[LexiconCode]

Tested using the example in the documentation for Rule Filters Simplified. Thank you @Danesprite

[kendonB]

Can confirm this is fixed using the latest dragonfly. Installed using pip install dragonfly2