Django QuerySetSequence

The QuerySetSequence wrapper helps to deal with disparate QuerySet classes, while treating them as a single QuerySet.

Supported Features

Listed below are features of Django's QuerySets that QuerySetSequence implements. The behavior should match that of QuerySet, but applied across multiple QuerySets:

• Methods that take a list of fields (e.g. filter(), exclude(), get(), order_by()) must use fields that are common across all sub-QuerySets.
• Relationships across related models work (e.g. 'foo__bar', 'foo', or 'foo_id'). syntax).
• The sub-QuerySets are evaluated as late as possible (e.g. during iteration, slicing, pickling, repr()/len()/list()/bool() calls).
• Public QuerySet API methods that are untested/unimplemented raise NotImplementedError. AttributeError is raised on attributes not explictly inherited from QuerySet.

QuerySet API implemented by QuerySetSequence

Method	Implemented?	Notes
filter()	✓	See [1] for information on the QuerySet lookup: '#'.
exclude()	✓	See [1] for information on the QuerySet lookup: '#'.
annotate()	✗
order_by()	✓	Does not support random ordering (e.g. order_by('?')). See [1] for information on the QuerySet lookup: '#'.
reverse()	✓
distinct()	✗
values()	✗
values_list()	✗
dates()	✗
datetimes()	✗
none()	✓
all()	✓
select_related()	✓
prefetch_related()	✓
extra()	✗
defer()	✗
only()	✗
using()	✗
select_for_update()	✗
raw()	✗
get()	✓	See [1] for information on the QuerySet lookup: '#'.
create()	✗	Cannot be implemented in QuerySetSequence.
get_or_create()	✗	Cannot be implemented in QuerySetSequence.
update_or_create()	✗	Cannot be implemented in QuerySetSequence.
bulk_create()	✗	Cannot be implemented in QuerySetSequence.
count()	✓
in_bulk()	✗	Cannot be implemented in QuerySetSequence.
iterator()	✓
latest()	✗
earliest()	✗
first()	✗
last()	✗
aggregate()	✗
exists()	✓
update()	✗	Cannot be implemented in QuerySetSequence.
delete()	✓
as_manager()	✓

[1]

(1, 2, 3, 4) QuerySetSequence supports a special field lookup that looks up the index of the QuerySet, this is represented by '#'. This can be used in any of the operations that normally take field lookups (i.e. filter(), exclude(), and get()), as well as order_by(). A few examples are below: # Order first by QuerySet, then by the value of the 'title' field. QuerySetSequence(...).order_by('#', 'title') # Filter out the first QuerySet. QuerySetSequence(...).filter(**{'#__gt': 0}) Note Ordering first by QuerySet allows for a more optimized code path when iterating over the entries. Warning Not all lookups are supported when using '#' (some lookups simply don't make sense; others are just not supported). The following are allowed: exact iexact contains icontains in gt gte lt lte startswith istartswith endswith iendswith range

Requirements

• Python (2.7, 3.4, 3.5)
• Django (1.8, 1.9, 1.10)
• (Optionally) Django REST Framework (3.2, 3.3, 3.4)

QuerySetSequence versions with support for Django/Django REST Framework

	Django 1.8	Django 1.9	Django 1.10
Django REST Framework 3.2	0.7	✗	✗
Django REST Framework 3.3	0.7	0.7	✗
Django REST Framework 3.4	0.7	0.7	0.7

Installation

Install the package using pip.

pip install --upgrade django-querysetsequence

Usage

# Import QuerySetSequence
from queryset_sequence import QuerySetSequence

# Create QuerySets you want to chain.
from .models import SomeModel, OtherModel

# Chain them together.
query = QuerySetSequence(SomeModel.objects.all(), OtherModel.objects.all())

# Use query as if it were a QuerySet! E.g. in a ListView.

You can also provide a model keyword argument if you need to specify the QuerySet Model, e.g. for compatibility with some third-party applications that check the model field for equality

Example

class Author(models.Model):
    name = models.CharField(max_length=50)

    class Meta:
        ordering = ['name']

    def __str__(self):
        return self.name


class Article(models.Model):
    title = models.CharField(max_length=100)
    author = models.ForeignKey(Author)

    def __str__(self):
        return "%s by %s" % (self.title, self.author)


class Book(models.Model):
    title = models.CharField(max_length=50)
    author = models.ForeignKey(Author)
    release = models.DateField(auto_now_add=True)

    def __str__(self):
        return "%s by %s" % (self.title, self.author)

# Create some data.
alice = Author.objects.create(name='Alice')
article = Article.objects.create(title='Dancing with Django', author=alice)

bob = Author.objects.create(name='Bob')
article = Article.objects.create(title='Django-isms', author=bob)
article = Book.objects.create(title='Biography', author=bob)

# Create some QuerySets.
books = Book.objects.all()
articles = Article.objects.all()

# Combine them into a single iterable.
published_works = QuerySetSequence(books, articles)

# Find Bob's titles.
bob_works = published_works.filter(author=bob)
# Still an iterable.
print([w.title for w in bob_works])  # prints: ['Biography', 'Django-isms']

# Alphabetize the QuerySet.
published_works = published_works.order_by('title')
print([w.title for w in published_works])  # prints ['Biography', 'Dancing with Django', 'Django-isms']

Django REST Framework integration

django-querysetsequence comes with a custom CursorPagination class that helps integration with Django REST Framework. It is optimized to iterate over a QuerySetSequence first by QuerySet and then by the normal ordering configuration. This uses the optimized code-path for iteration that avoids interleaving the individual QuerySets. For example:

from queryset_sequence.pagination import SequenceCursorPagination

class PublicationPagination(SequenceCursorPagination):
    ordering = ['author', 'title']

class PublicationViewSet(viewsets.ModelViewSet):
    pagination_class = PublicationPagination

    def get_queryset(self):
        # This will return all Books first, then all Articles. Each of those
        # is individually ordered by ``author``, then ``title``.
        return QuerySetSequence(Book.objects.all(), Article.objects.all())

Attribution

This is based on a few DjangoSnippets that had been going around:

• Originally from https://www.djangosnippets.org/snippets/1103/
• Modified version from https://djangosnippets.org/snippets/1253/
• Upgraded version from https://djangosnippets.org/snippets/1933/
• Updated version from django-ko-demo from The Atlantic

Contribute

• Check for open issues or open a fresh issue to start a discussion around a feature idea or a bug.
• Fork the repository on GitHub to start making your changes.
• Write a test which shows that the bug was fixed or that the feature works as expected.
• Send a pull request and bug the maintainer until it gets merged and published.