Caching with default pickle version crashes on cache.get()

[mlissner]

Using DRF 3.3.1, Django 1.8.7, and Python 2.7, with the default Django cache settings (LocMemCache), I discovered that my API endpoints failed when they were loaded from cache, with an error like:

Internal Server Error: /api/rest/v3/dockets/
Traceback (most recent call last):
  File "/home/mlissner/.virtualenvs/courtlistener/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 108, in get_response
    response = middleware_method(request)
  File "/home/mlissner/.virtualenvs/courtlistener/local/lib/python2.7/site-packages/django/middleware/cache.py", line 134, in process_request
    response = self.cache.get(cache_key, None)
  File "/home/mlissner/.virtualenvs/courtlistener/local/lib/python2.7/site-packages/django/core/cache/backends/locmem.py", line 54, in get
    return pickle.loads(pickled)
TypeError: __new__() takes exactly 3 arguments (2 given)

What happened was that the first time I loaded the page it worked, but the second time, it failed because it was loading from cache.

Asking on StackOverflow, I learned that there's an issue with version 2 of pickle that can cause this issue, and so I took the opportunity to switch my cache from LocMemCache to RedisCache, which allows setting a custom PICKLE_VERSION.

So....I set that value to 1 instead of the default of 2, and sure enough, the problem went away.

This is all a bit over my head, but my conclusion is that DRF, using the default pickle version....fails...sometimes. I assume this must have something to do with my code, but I don't have anything too interesting in there. This feels like a documentation problem, or a bug in Django, or even, in my code. I don't know, but it took a lot of investigation, so figured I should see if other more enlightened folks have thoughts/fixes/etc.

Thanks!

[mlissner]

Following up. I thought this might have something to do with my code since nobody else seems to have this issue, but I doubt it now. Here's the code that's causing this issue:

views.py:

class CourtViewSet(viewsets.ModelViewSet):
    queryset = Court.objects.all()
    serializer_class = serializers.CourtSerializer

serializers.py:

class CourtSerializer(serializers.HyperlinkedModelSerializer):
    class Meta:
        model = search_models.Court
        exclude = ('notes',)

Court is a very boring object with a variety of standard django fields.

urls.py (shortened):

router = DefaultRouter()
router.register(r'dockets', views.DocketViewSet)
router.register(r'courts', views.CourtViewSet)
urlpatterns = [
    url(r'^api/rest/(?P<version>[v3]+)/', include(router.urls))
]

Seems pretty plain jane, all in all.

[xordoquy]

This should be fixed by #3701

[mlissner]

Yup! Just found it literally a second before your response, after days of looking for a solution! The secret was googling for getnewargs django. Do you know when we can expect this beautiful work to land in pip?