MarkupSafe 3.0.0 throws when used with SimpleLazyObjects

[dhendo]

If you attempt to escape a SimpleLazyObject using Markup.escape, it will throw as below. Confirmed working in MarkupSafe 2.1.5.

from django.utils.functional import SimpleLazyObject
from markupsafe import Markup

a = "Normal String"
b = SimpleLazyObject( lambda: "Lazy String")

print(a)
print(b)

print(Markup.escape(a))
print(Markup.escape(b))

Output:

Normal String
Lazy String
Normal String
Traceback (most recent call last):
  File "/snap/pycharm-professional/418/plugins/python-ce/helpers/pydev/pydevconsole.py", line 364, in runcode
    coro = func()
           ^^^^^^
  File "<input>", line 1, in <module>
  File "/snap/pycharm-professional/418/plugins/python-ce/helpers/pydev/_pydev_bundle/pydev_umd.py", line 197, in runfile
    pydev_imports.execfile(filename, global_vars, local_vars)  # execute the script
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/snap/pycharm-professional/418/plugins/python-ce/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "/home/davidh/dev/TriggeredMessagingV1/test.py", line 11, in <module>
    print(Markup.escape(b))
          ^^^^^^^^^^^^^^^^
  File "python3.11/site-packages/markupsafe/__init__.py", line 233, in escape
    rv = escape(s)
         ^^^^^^^^^
  File "python3.11/site-packages/markupsafe/__init__.py", line 38, in escape
    return Markup(_escape_inner(s))
                  ^^^^^^^^^^^^^^^^
SystemError: <built-in function _escape_inner> returned NULL without setting an exception

Environment:

• Python version: 3.11.8
• MarkupSafe version: 3.0.0
• Jinja2: 3.1.2, 3.1.4
• Django: 3.2.20

[davidism]

You've cut off the traceback.

[dhendo]

Thanks, updated

[davidism]

I'm guessing this is some weird side effect of the proxy object. isinstance(SimpleLazyObject(lambda: "a"), str) returns true, but it's not actually a str, it's a SimpleLazyObject. escape passes strings through without a cast/copy now, as an optimization, which means that a non-string object is passed to C code expecting a string.

[davidism]

I'm not particularly familiar with these Django utils, but I had a look and it seems like there's a django.utils.functional.lazystr that's meant specifically for strings. That does work when passed to escape. Is this something you should fix in your code?

[dhendo]

The wider context - we're getting to escape via a Jinja tag, on django's csrf_token tag

window.FRUI.set("CSRF", '{{ csrf_token }}');

Which is natively returned as a SimpleLazyObject by the django:

https://github.com/django/django/blob/main/django/template/context_processors.py#L33

So I'm guessing this may be an issue others run into in some situations, and isn't directly in custom code

[davidism]

It looks like adding a PyObject_Str(s) call to the C code doesn't change the performance, so we'll fix it here.