Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

QuerySetNoCache iterator protocol violation #1316

Open
gmacon opened this issue Jun 17, 2016 · 0 comments
Open

QuerySetNoCache iterator protocol violation #1316

gmacon opened this issue Jun 17, 2016 · 0 comments

Comments

@gmacon
Copy link
Contributor

gmacon commented Jun 17, 2016

I believe that the QuerySetNoCache object violates the iterator protocol because iter is not idempotent. I discovered this with code like the following:

from itertools import *
from mongoengine import *

class Foo(Document):
    pass

def chunks(iterator, n):
    it = iter(iterator)
    while True:
        r = tuple(islice(it, n))
        if not r:
            return
        yield r

connect('test')

Foo.objects.delete()
Foo().save()

qs = Foo.objects.no_cache()

for c in chunks(qs, 10):
    print c

This is an infinite loop because islice calls iter on its argument, but calling iter on the QuerySetNoCache rewinds the cursor. This can be repaired by changing the body of the __iter__ method to just return self, but that's not backwards compatible as it no longer clones the query set when iter is called on the original object multiple times. Perhaps the solution is something along the lines of

def __iter__(self):
    qs = self.clone()
    qs.rewind()
    while 1:
        yield qs.next()

This would fix the issue because the __iter__ method now never returns self, and the generator properly does. On the other hand, this looks odd to me, and I wouldn't submit it in a PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant