Object of type Theme is not JSON serializable #356

TheBitShepherd · 2024-01-26T18:46:16Z

Python version
3.10.11

Django version
4.0.10

Package version
0.28.3

Current behavior (bug description)
On startup the project encounters an error

Desired behavior
Project should start without error

Hi there! I'm really interested in using this package for my team's project, but on startup I'm receiving the following stacktrace. It looks like it may not be playing nicely with some of the dependency packages our project has, as this package doesn't encounter this error when I install it on a fresh project.

Has this issue been encountered by anyone before? Is there a known remedy? Thanks in advance!

Traceback (most recent call last):
  File ".../.venv/lib/python3.10/site-packages/django/core/handlers/exception.py", line 56, in inner
    response = get_response(request)
  File ".../.venv/lib/python3.10/site-packages/django/core/handlers/base.py", line 220, in _get_response
    response = response.render()
  File ".../.venv/lib/python3.10/site-packages/sentry_sdk/integrations/django/views.py", line 38, in sentry_patched_render
    return old_render(self)
  File ".../.venv/lib/python3.10/site-packages/django/template/response.py", line 114, in render
    self.content = self.rendered_content
  File ".../.venv/lib/python3.10/site-packages/sentry_sdk/integrations/django/templates.py", line 74, in rendered_content
    return real_rendered_content.fget(self)
  File ".../.venv/lib/python3.10/site-packages/django/template/response.py", line 92, in rendered_content
    return template.render(context, self._request)
  File ".../.venv/lib/python3.10/site-packages/django/template/backends/django.py", line 61, in render
    return self.template.render(context)
  File ".../.venv/lib/python3.10/site-packages/django/template/base.py", line 175, in render
    return self._render(context)
  File ".../.venv/lib/python3.10/site-packages/django/test/utils.py", line 111, in instrumented_test_render
    return self.nodelist.render(context)
  File ".../.venv/lib/python3.10/site-packages/django/template/base.py", line 1000, in render
    return SafeString("".join([node.render_annotated(context) for node in self]))
  File ".../.venv/lib/python3.10/site-packages/django/template/base.py", line 1000, in <listcomp>
    return SafeString("".join([node.render_annotated(context) for node in self]))
  File ".../.venv/lib/python3.10/site-packages/django/template/base.py", line 958, in render_annotated
    return self.render(context)
  File ".../.venv/lib/python3.10/site-packages/django/template/loader_tags.py", line 157, in render
    return compiled_parent._render(context)
  File ".../.venv/lib/python3.10/site-packages/django/test/utils.py", line 111, in instrumented_test_render
    return self.nodelist.render(context)
  File ".../.venv/lib/python3.10/site-packages/django/template/base.py", line 1000, in render
    return SafeString("".join([node.render_annotated(context) for node in self]))
  File ".../.venv/lib/python3.10/site-packages/django/template/base.py", line 1000, in <listcomp>
    return SafeString("".join([node.render_annotated(context) for node in self]))
  File ".../.venv/lib/python3.10/site-packages/django/template/base.py", line 958, in render_annotated
    return self.render(context)
  File ".../.venv/lib/python3.10/site-packages/django/template/loader_tags.py", line 157, in render
    return compiled_parent._render(context)
  File ".../.venv/lib/python3.10/site-packages/django/test/utils.py", line 111, in instrumented_test_render
    return self.nodelist.render(context)
  File ".../.venv/lib/python3.10/site-packages/django/template/base.py", line 1000, in render
    return SafeString("".join([node.render_annotated(context) for node in self]))
  File ".../.venv/lib/python3.10/site-packages/django/template/base.py", line 1000, in <listcomp>
    return SafeString("".join([node.render_annotated(context) for node in self]))
  File ".../.venv/lib/python3.10/site-packages/django/template/base.py", line 958, in render_annotated
    return self.render(context)
  File ".../.venv/lib/python3.10/site-packages/django/template/loader_tags.py", line 157, in render
    return compiled_parent._render(context)
  File ".../.venv/lib/python3.10/site-packages/django/test/utils.py", line 111, in instrumented_test_render
    return self.nodelist.render(context)
  File ".../.venv/lib/python3.10/site-packages/django/template/base.py", line 1000, in render
    return SafeString("".join([node.render_annotated(context) for node in self]))
  File ".../.venv/lib/python3.10/site-packages/django/template/base.py", line 1000, in <listcomp>
    return SafeString("".join([node.render_annotated(context) for node in self]))
  File ".../.venv/lib/python3.10/site-packages/django/template/base.py", line 958, in render_annotated
    return self.render(context)
  File ".../.venv/lib/python3.10/site-packages/django/template/loader_tags.py", line 157, in render
    return compiled_parent._render(context)
  File ".../.venv/lib/python3.10/site-packages/django/test/utils.py", line 111, in instrumented_test_render
    return self.nodelist.render(context)
  File ".../.venv/lib/python3.10/site-packages/django/template/base.py", line 1000, in render
    return SafeString("".join([node.render_annotated(context) for node in self]))
  File ".../.venv/lib/python3.10/site-packages/django/template/base.py", line 1000, in <listcomp>
    return SafeString("".join([node.render_annotated(context) for node in self]))
  File ".../.venv/lib/python3.10/site-packages/django/template/base.py", line 958, in render_annotated
    return self.render(context)
  File ".../.venv/lib/python3.10/site-packages/django/template/loader_tags.py", line 63, in render
    result = block.nodelist.render(context)
  File ".../.venv/lib/python3.10/site-packages/django/template/base.py", line 1000, in render
    return SafeString("".join([node.render_annotated(context) for node in self]))
  File ".../.venv/lib/python3.10/site-packages/django/template/base.py", line 1000, in <listcomp>
    return SafeString("".join([node.render_annotated(context) for node in self]))
  File ".../.venv/lib/python3.10/site-packages/django/template/base.py", line 958, in render_annotated
    return self.render(context)
  File ".../.venv/lib/python3.10/site-packages/django/template/library.py", line 239, in render
    output = self.func(*resolved_args, **resolved_kwargs)
  File ".../.venv/lib/python3.10/site-packages/admin_interface/templatetags/admin_interface_tags.py", line 64, in get_admin_interface_theme
    set_cached_active_theme(theme)
  File ".../.venv/lib/python3.10/site-packages/admin_interface/cache.py", line 18, in set_cached_active_theme
    app_cache().set("admin_interface_theme", theme)
  File ".../.venv/lib/python3.10/site-packages/debug_toolbar/panels/cache.py", line 40, in wrapped
    value = method(self, *args, **kwargs)
  File ".../.venv/lib/python3.10/site-packages/debug_toolbar/panels/cache.py", line 95, in set
    return self.cache.set(*args, **kwargs)
  File ".../.venv/lib/python3.10/site-packages/django_redis/cache.py", line 31, in _decorator
    return method(self, *args, **kwargs)
  File ".../.venv/lib/python3.10/site-packages/django_redis/cache.py", line 80, in set
    return self.client.set(*args, **kwargs)
  File ".../.venv/lib/python3.10/site-packages/django_redis/client/default.py", line 143, in set
    nvalue = self.encode(value)
  File ".../.venv/lib/python3.10/site-packages/django_redis/client/default.py", line 457, in encode
    value = self._serializer.dumps(value)
  File ".../.venv/lib/python3.10/site-packages/django_redis/serializers/json.py", line 13, in dumps
    return json.dumps(value, cls=self.encoder_class).encode()
  File ".../.asdf/installs/python/3.10.11/lib/python3.10/json/__init__.py", line 238, in dumps
    **kw).encode(obj)
  File ".../.asdf/installs/python/3.10.11/lib/python3.10/json/encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File ".../.asdf/installs/python/3.10.11/lib/python3.10/json/encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
  File ".../.venv/lib/python3.10/site-packages/django/core/serializers/json.py", line 106, in default
    return super().default(o)
  File ".../.asdf/installs/python/3.10.11/lib/python3.10/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '

Exception Type: TypeError at /admin/login/
Exception Value: Object of type Theme is not JSON serializable

Upvote & Fund

We're using Polar.sh so you can upvote and help fund this issue.
We receive the funding once the issue is completed & confirmed by you.
Thank you in advance for helping prioritize & fund our backlog.

The text was updated successfully, but these errors were encountered:

merwok · 2024-01-26T20:15:11Z

This is strange, I also use redis_cache and don’t have this problem.
Can you copy your project to make a minimal reproduction example and share that?

TheBitShepherd · 2024-01-26T20:56:13Z

I can work on setting up a minimal repro 👍

TheBitShepherd · 2024-01-26T21:54:11Z

@merwok - The most minimal repro I've been able to come up with is the following in a django shell after installing this package:

$ theme = Theme.objects.get()
$ theme
 <Theme: Django>
$ from json import JSONEncoder
$ chunks = JSONEncoder().iterencode(theme, _one_shot=True)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
----> 1 chunks = JSONEncoder().iterencode(theme, _one_shot=True)

~/.asdf/installs/python/3.10.11/lib/python3.10/json/encoder.py in iterencode(self, o, _one_shot)
    255                 self.key_separator, self.item_separator, self.sort_keys,
    256                 self.skipkeys, _one_shot)
--> 257         return _iterencode(o, 0)
    258
    259 def _make_iterencode(markers, _default, _encoder, _indent, _floatstr,

~/.asdf/installs/python/3.10.11/lib/python3.10/json/encoder.py in default(self, o)
    177
    178         """
--> 179         raise TypeError(f'Object of type {o.__class__.__name__} '
    180                         f'is not JSON serializable')
    181

TypeError: Object of type Theme is not JSON serializable

I can work on making a smaller project to replicate but that may take some time.

fabiocaccamo · 2024-01-26T22:41:44Z

@TheBitShepherd Does this happen also on a fresh new installation without other third-party requirements installed?

TheBitShepherd · 2024-01-26T22:44:35Z

@fabiocaccamo - If I install this package in a fresh install without third-party dependencies everything works as expected. The issue is definitely with dajngo-redis.

I was able to get things working by specifying a local memory cache for the admin_interface with the following in settings:

CACHES = {
    "default": {
        "BACKEND": "django_redis.cache.RedisCache",
        "LOCATION": config("REDIS_URL", default="redis://"),
        "OPTIONS": {
            "CLIENT_CLASS": "django_redis.client.DefaultClient",
            "SERIALIZER": "django_redis.serializers.json.JSONSerializer",
        },
    },
    "admin_interface": {
        "BACKEND": "django.core.cache.backends.locmem.LocMemCache",
    }
}

fabiocaccamo · 2024-01-26T22:51:36Z

And the django-redis JSONSerializer doesn't do anything special: https://github.com/jazzband/django-redis/blob/master/django_redis/serializers/json.py

This is very strange...

merwok · 2024-01-26T23:24:22Z

The most minimal repro

That’s only showing the surface cause! We already knew from the original traceback that this particular call is a problem. The thing I wanted to see is: why does the combination of django-redis and django-admin-interface causes the call to happen 🙂 → a repro of minimal django project (dependencies and settings)

merwok · 2024-01-26T23:29:48Z

In my project, django-redis 5.3.0 uses a pickle serializer by default.
Maybe you have a different version, or you are defining settings to use a json serializer.

TheBitShepherd · 2024-01-26T23:57:18Z

Maybe you have a different version, or you are defining settings to use a json serializer.

My project is using django_redis==4.2 and does configure it with a JSONSerializer:

"SERIALIZER": "django_redis.serializers.json.JSONSerializer"

I'm seeing the same behavior even when I update it to 5.4.0.

merwok · 2024-01-27T00:30:47Z

Alright then! Can you configure a different cache for this package? See https://github.com/fabiocaccamo/django-admin-interface/blob/main/admin_interface/cache.py#L6

TheBitShepherd · 2024-01-27T16:20:21Z

@merwok - Yes. As I mentioned above configuring a different cache for this package was one way I found to alleviate the error. 👍

fabiocaccamo · 2024-01-27T16:34:38Z

@TheBitShepherd could you try to manually get the default theme instance and serialize it using the django-redis JSONEncoder?

TheBitShepherd · 2024-01-27T21:08:17Z

@fabiocaccamo - It gives the same stacktrace, so not sure that's much additional help. The django_redis.serializers.json.JSONSerializer simply uses the json.dumps method and doesn't inherit from any type of Django serializer.

>>> from admin_interface.models import Theme
>>> from django_redis.serializers.json import JSONSerializer
>>> theme = Theme.objects.get()
>>> theme
<Theme: Django>
>>> JSONSerializer({}).dumps(theme)
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File ".../.venv/lib/python3.12/site-packages/django_redis/serializers/json.py", line 13, in dumps
    return json.dumps(value, cls=self.encoder_class).encode()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../.asdf/installs/python/3.12.0/lib/python3.12/json/__init__.py", line 238, in dumps
    **kw).encode(obj)
          ^^^^^^^^^^^
  File ".../.asdf/installs/python/3.12.0/lib/python3.12/json/encoder.py", line 200, in encode
    chunks = self.iterencode(o, _one_shot=True)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../.asdf/installs/python/3.12.0/lib/python3.12/json/encoder.py", line 258, in iterencode
    return _iterencode(o, 0)
           ^^^^^^^^^^^^^^^^^
  File ".../.venv/lib/python3.12/site-packages/django/core/serializers/json.py", line 106, in default
    return super().default(o)
           ^^^^^^^^^^^^^^^^^^
  File ".../.asdf/installs/python/3.12.0/lib/python3.12/json/encoder.py", line 180, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type Theme is not JSON serializable

fabiocaccamo · 2024-01-27T21:22:13Z

So the issue can be reproduced easily using json.dumps directly.

I'm pretty sure the same error would be raised with any other model.

merwok · 2024-01-28T18:43:22Z

I think the options here are:

close wontfix
document the cache settings in detail (default cache can’t use json serializer, or admin_interface cache mush be defined)
store a dict in the cache, not a Theme – with some code to dump/load Theme instance to/from dict (so basically pickle but json 😉)
use functools.lru_cache rather than django cache!

fabiocaccamo · 2024-01-29T07:24:40Z

@merwok considering that this is not a bug of this library I would opt for improving the caching documentation.

TheBitShepherd · 2024-01-29T16:02:39Z

@fabiocaccamo @merwok - If you'd like I'd be happy to open an MR to enhance the caching documentation with what we found here.

fabiocaccamo · 2024-01-29T16:03:28Z

@TheBitShepherd that would be great, thanks!

TheBitShepherd · 2024-01-29T16:35:12Z

@fabiocaccamo - #357

fabiocaccamo · 2024-01-29T17:06:44Z

@TheBitShepherd thanks!

TheBitShepherd added the bug Something isn't working label Jan 26, 2024

TheBitShepherd assigned fabiocaccamo Jan 26, 2024

fabiocaccamo added this to Open Source Jan 29, 2024

fabiocaccamo moved this to Todo in Open Source Jan 29, 2024

fabiocaccamo added invalid This doesn't seem right and removed bug Something isn't working labels Jan 29, 2024

TheBitShepherd mentioned this issue Jan 29, 2024

Update Caching Documentation #357

Merged

3 tasks

fabiocaccamo closed this as completed Jan 29, 2024

github-project-automation bot moved this from Todo to Done in Open Source Jan 29, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Object of type Theme is not JSON serializable #356

Object of type Theme is not JSON serializable #356

TheBitShepherd commented Jan 26, 2024 •

edited by polar-sh bot

Loading

merwok commented Jan 26, 2024

TheBitShepherd commented Jan 26, 2024

TheBitShepherd commented Jan 26, 2024

fabiocaccamo commented Jan 26, 2024 •

edited

Loading

TheBitShepherd commented Jan 26, 2024

fabiocaccamo commented Jan 26, 2024 •

edited

Loading

merwok commented Jan 26, 2024 •

edited

Loading

merwok commented Jan 26, 2024

TheBitShepherd commented Jan 26, 2024

merwok commented Jan 27, 2024

TheBitShepherd commented Jan 27, 2024

fabiocaccamo commented Jan 27, 2024

TheBitShepherd commented Jan 27, 2024

fabiocaccamo commented Jan 27, 2024

merwok commented Jan 28, 2024

fabiocaccamo commented Jan 29, 2024

TheBitShepherd commented Jan 29, 2024

fabiocaccamo commented Jan 29, 2024

TheBitShepherd commented Jan 29, 2024

fabiocaccamo commented Jan 29, 2024

Object of type Theme is not JSON serializable #356

Object of type Theme is not JSON serializable #356

Comments

TheBitShepherd commented Jan 26, 2024 • edited by polar-sh bot Loading

Upvote & Fund

merwok commented Jan 26, 2024

TheBitShepherd commented Jan 26, 2024

TheBitShepherd commented Jan 26, 2024

fabiocaccamo commented Jan 26, 2024 • edited Loading

TheBitShepherd commented Jan 26, 2024

fabiocaccamo commented Jan 26, 2024 • edited Loading

merwok commented Jan 26, 2024 • edited Loading

merwok commented Jan 26, 2024

TheBitShepherd commented Jan 26, 2024

merwok commented Jan 27, 2024

TheBitShepherd commented Jan 27, 2024

fabiocaccamo commented Jan 27, 2024

TheBitShepherd commented Jan 27, 2024

fabiocaccamo commented Jan 27, 2024

merwok commented Jan 28, 2024

fabiocaccamo commented Jan 29, 2024

TheBitShepherd commented Jan 29, 2024

fabiocaccamo commented Jan 29, 2024

TheBitShepherd commented Jan 29, 2024

fabiocaccamo commented Jan 29, 2024

TheBitShepherd commented Jan 26, 2024 •

edited by polar-sh bot

Loading

fabiocaccamo commented Jan 26, 2024 •

edited

Loading

fabiocaccamo commented Jan 26, 2024 •

edited

Loading

merwok commented Jan 26, 2024 •

edited

Loading