unicode control characters are not allowed as TextareaWidget value

[petschki]

today I had a strange traceback:

...
 Module plone.app.widgets.dx, line 863, in render
 Module plone.app.widgets.base, line 306, in __init__
 Module plone.app.widgets.base, line 320, in _set_value
 Module lxml.etree, line 904, in lxml.etree._Element.text.__set__ (src/lxml/lxml.etree.c:37197)
 Module lxml.etree, line 721, in lxml.etree._setNodeText (src/lxml/lxml.etree.c:16865)
 Module lxml.etree, line 1369, in lxml.etree._utf8 (src/lxml/lxml.etree.c:22147)
ValueError: All strings must be XML compatible: Unicode or ASCII, no NULL bytes or control characters

turned out, that a user inserted text with hidden unicode control characters in a richtextwidget. I quick-fixed this with the following patch:

class TextareWidget(BaseWidget):
...
    def _set_value(self, value):
        uincode_cc = dict.fromkeys(range(32))
        clean_value = value.translate(uincode_cc)
        self.el.text = clean_value
...

but this is dirty as hell ...

had anybody else problem with this?
BTW ... this is Plone 4.3, p.a.widgets -> 1.x branch

[vangheem]

woah. I've never seen this.

Perhaps this could be fixed in tinymce as well?

[ginetsl]

Also happening in Plone 5.0.6+plone.app.widgets 2.0.6 and Plone 5.1.2.1 + plone.app.widgets 2.3

[petschki]

Still happening in Plone 5.2/py3.7 ...

Traceback (innermost last):
  Module ZPublisher.WSGIPublisher, line 162, in transaction_pubevents
  Module ZPublisher.WSGIPublisher, line 359, in publish_module
  Module ZPublisher.WSGIPublisher, line 262, in publish
  Module ZPublisher.mapply, line 85, in mapply
  Module ZPublisher.WSGIPublisher, line 63, in call_object
  Module z3c.form.form, line 239, in __call__
  Module z3c.form.form, line 162, in render
  Module Products.Five.browser.pagetemplatefile, line 61, in __call__
  Module zope.pagetemplate.pagetemplate, line 135, in pt_render
  Module Products.PageTemplates.engine, line 367, in __call__
  Module z3c.pt.pagetemplate, line 176, in render
  Module chameleon.zpt.template, line 307, in render
  Module chameleon.template, line 192, in render
  Module bd341d9857b25dfa95456b377ff5ce2a, line 198, in render
  Module 8bb8f4a4288f3f378e7daa886d328cd6, line 860, in render_master
  Module 8bb8f4a4288f3f378e7daa886d328cd6, line 1526, in render_content
  Module bd341d9857b25dfa95456b377ff5ce2a, line 179, in __fill_main
  Module 068db699a09e272a02e9cd87c9e3ee84, line 921, in render_titlelessform
  Module 068db699a09e272a02e9cd87c9e3ee84, line 1227, in render_fields
  Module 068db699a09e272a02e9cd87c9e3ee84, line 1764, in render_widget_rendering
  Module 068db699a09e272a02e9cd87c9e3ee84, line 1871, in render_field
  Module zope.tales.expressions, line 250, in __call__
  Module Products.PageTemplates.Expressions, line 196, in _eval
  Module Products.PageTemplates.Expressions, line 126, in render
  Module zope.browserpage.simpleviewclass, line 41, in __call__
  Module zope.browserpage.viewpagetemplatefile, line 81, in __call__
  Module zope.browserpage.viewpagetemplatefile, line 49, in __call__
  Module zope.pagetemplate.pagetemplate, line 135, in pt_render
  Module Products.PageTemplates.engine, line 367, in __call__
  Module z3c.pt.pagetemplate, line 176, in render
  Module chameleon.zpt.template, line 307, in render
  Module chameleon.template, line 214, in render
  Module chameleon.utils, line 75, in raise_with_traceback
  Module chameleon.template, line 192, in render
  Module 7aa697e77773e9cb7c4bc2c0617a2016, line 586, in render
  Module 7aa697e77773e9cb7c4bc2c0617a2016, line 456, in render_widget_wrapper
  Module zope.tales.expressions, line 250, in __call__
  Module Products.PageTemplates.Expressions, line 196, in _eval
  Module Products.PageTemplates.Expressions, line 126, in render
  Module plone.app.z3cform.widget, line 769, in render
  Module plone.app.widgets.base, line 334, in __init__
  Module plone.app.widgets.base, line 348, in _set_value
  Module lxml.etree, line 1024, in lxml.etree._Element.text.__set__
  Module lxml.etree, line 747, in lxml.etree._setNodeText
  Module lxml.etree, line 735, in lxml.etree._createTextNode
  Module lxml.etree, line 1540, in lxml.etree._utf8
ValueError: All strings must be XML compatible: Unicode or ASCII, no NULL bytes or control characters

 - Expression: "widget/render"
 - Filename:   ... rm-3.2.1-py3.7.egg/plone/app/z3cform/templates/widget.pt
 - Location:   (line 39: col 46)
 - Source:     ... xt" tal:replace="structure widget/render"
                                              ^^^^^^^^^^^^^
 - Expression: "widget/@@ploneform-render-widget"
 - Filename:   ... rm-3.2.1-py3.7.egg/plone/app/z3cform/templates/macros.pt
 - Location:   (line 99: col 81)
 - Source:     ... place="structure widget/@@ploneform-render-widget"/>
                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 - Expression: "context/main_template/macros/master"
 - Filename:   ... form-3.2.1-py3.7.egg/plone/app/z3cform/templates/form.pt
 - Location:   (line 6: col 23)
 - Source:     ... etal:use-macro="context/main_template/macros/master">
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 - Arguments:  template: <zope.browserpage.viewpagetemplatefile.ViewPageTemplateFile object at 0x7f84cf1ddcd0>
               options: {}
               args: ()
               nothing: None
               modules: <zope.pagetemplate.engine.TraversableModuleImporter object at 0x7f8509667390>
               request: <WSGIRequest, URL=https://www.erinnern.at/bundeslaender/kaernten/termine/erweiterung-des-denkmals-fuer-die-ns-opfer-in-greifenburg/@@edit>
               view: <Products.Five.browser.metaconfigure.RenderWidget object at 0x7f84da8c0790>
               context: <RichTextWidget 'form.widgets.IRichTextBehavior.text'>
               views: <zope.browserpage.viewpagetemplatefile.ViewMapper object at 0x7f84da8c0050>
               default: <DEFAULT>
               repeat: <Products.PageTemplates.engine.RepeatDictWrapper object at 0x7f84cd53b960>
               loop: {}
               target_language: None
               translate: <function BaseTemplate.render.<locals>.translate at 0x7f84d0ac4200>

[bizuyghur]

Did not have issue saving and viewing the portal type but getting same error while trying to edit it. (Plone 5.2.4 /Python 3.8.10)

Traceback (innermost last):
Module ZPublisher.WSGIPublisher, line 162, in transaction_pubevents
Module ZPublisher.WSGIPublisher, line 371, in publish_module
Module ZPublisher.WSGIPublisher, line 266, in publish
Module ZPublisher.mapply, line 85, in mapply
Module ZPublisher.WSGIPublisher, line 63, in call_object
Module plone.z3cform.layout, line 63, in call
Module plone.z3cform.layout, line 57, in update
Module z3c.form.form, line 162, in render
Module zope.browserpage.viewpagetemplatefile, line 46, in call
Module zope.pagetemplate.pagetemplate, line 133, in pt_render
Module Products.PageTemplates.engine, line 378, in call
Module z3c.pt.pagetemplate, line 176, in render
Module chameleon.zpt.template, line 302, in render
Module chameleon.template, line 192, in render
Module 03b1c025e387bf927f0bbf26a830e098, line 115, in render
Module 8e5deec4d7b9da3c24b6d0312a7c849b, line 921, in render_titlelessform
Module 8e5deec4d7b9da3c24b6d0312a7c849b, line 1227, in render_fields
Module 8e5deec4d7b9da3c24b6d0312a7c849b, line 1764, in render_widget_rendering
Module 8e5deec4d7b9da3c24b6d0312a7c849b, line 1871, in render_field
Module zope.tales.expressions, line 250, in call
Module Products.PageTemplates.Expressions, line 196, in _eval
Module Products.PageTemplates.Expressions, line 126, in render
Module zope.browserpage.simpleviewclass, line 41, in call
Module zope.browserpage.viewpagetemplatefile, line 81, in call
Module zope.browserpage.viewpagetemplatefile, line 46, in call
Module zope.pagetemplate.pagetemplate, line 133, in pt_render
Module Products.PageTemplates.engine, line 378, in call
Module z3c.pt.pagetemplate, line 176, in render
Module chameleon.zpt.template, line 302, in render
Module chameleon.template, line 214, in render
Module chameleon.utils, line 53, in raise_with_traceback
Module chameleon.template, line 192, in render
Module d76fdd108eb0c0612168f4eab86d6105, line 586, in render
Module d76fdd108eb0c0612168f4eab86d6105, line 456, in render_widget_wrapper
Module zope.tales.expressions, line 250, in call
Module Products.PageTemplates.Expressions, line 196, in _eval
Module Products.PageTemplates.Expressions, line 126, in render
Module plone.app.z3cform.widget, line 769, in render
Module plone.app.widgets.base, line 334, in init
Module plone.app.widgets.base, line 348, in _set_value
Module lxml.etree, line 1024, in lxml.etree._Element.text.set
Module lxml.etree, line 747, in lxml.etree._setNodeText
Module lxml.etree, line 735, in lxml.etree._createTextNode
Module lxml.etree, line 1540, in lxml.etree._utf8
ValueError: All strings must be XML compatible: Unicode or ASCII, no NULL bytes or control characters

• Expression: "widget/render"
• Filename: ... rm-3.2.2-py3.8.egg/plone/app/z3cform/templates/widget.pt
• Location: (line 39: col 46)
• Source: ... xt" tal:replace="structure widget/render"
  ^^^^^^^^^^^^^
• Expression: "widget/@@ploneform-render-widget"
• Filename: ... rm-3.2.2-py3.8.egg/plone/app/z3cform/templates/macros.pt
• Location: (line 99: col 81)
• Source: ... place="structure widget/@@ploneform-render-widget"/>
  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
• Expression: "context/@@ploneform-macros/titlelessform"
• Filename: ... 1.3-py3.8.egg/plone/z3cform/pagetemplates/wrappedform.pt
• Location: (line 1: col 22)
• Source: ... e use-macro="context/@@ploneform-macros/titlelessform" />
  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
• Arguments: template: <zope.browserpage.viewpagetemplatefile.ViewPageTemplateFile object at 0x7fa0d886f640>
  options: {}
  args: ()
  nothing: None
  modules: <zope.pagetemplate.engine.TraversableModuleImporter object at 0x7fa105328b20>
  request: <WSGIRequest, URL=XXXXXX/@@edit>
  view: <Products.Five.browser.metaconfigure.RenderWidget object at 0x7fa0ab8af190>
  context: <RichTextWidget 'form.widgets.IRichTextBehavior.text'>
  views: <zope.browserpage.viewpagetemplatefile.ViewMapper object at 0x7fa0ab8af970>
  default:
  repeat: <Products.PageTemplates.engine.RepeatDictWrapper object at 0x7fa09b3ba840>
  loop: {}
  target_language: None
  translate: <function BaseTemplate.render..translate at 0x7fa0b4c0b1f0>