Merge pull request #9 from miki725/python-filtering

Plain Filtering + Custom callable filters

.gitignore

@@ -23,6 +23,7 @@ lib64
 pip-log.txt
 
 # Unit test / coverage reports
+.cache
 .coverage
 .tox
 nosetests.xml

.travis.yml

@@ -1,7 +1,7 @@
 language: python
 
 python:
-  - "3.4"
+  - "3.5"
   - "2.7"
   - "pypy"
 

Makefile

@@ -45,7 +45,7 @@ lint:
 	flake8 .
 
 test:
-	py.test -sv --cov=url_filter --cov-report=term-missing tests/
+	py.test -sv --cov=url_filter --cov-report=term-missing --doctest-modules tests/ url_filter/
 
 test-all:
 	tox

docs/api/url_filter.backends.plain.rst

@@ -0,0 +1,7 @@
+url_filter.backends.plain module
+================================
+
+.. automodule:: url_filter.backends.plain
+    :members:
+    :undoc-members:
+    :show-inheritance:

docs/api/url_filter.backends.rst

@@ -13,5 +13,6 @@ Submodules
 
    url_filter.backends.base
    url_filter.backends.django
+   url_filter.backends.plain
    url_filter.backends.sqlalchemy
 

docs/api/url_filter.filtersets.plain.rst

@@ -0,0 +1,7 @@
+url_filter.filtersets.plain module
+==================================
+
+.. automodule:: url_filter.filtersets.plain
+    :members:
+    :undoc-members:
+    :show-inheritance:

docs/api/url_filter.filtersets.rst

@@ -13,5 +13,6 @@ Submodules
 
    url_filter.filtersets.base
    url_filter.filtersets.django
+   url_filter.filtersets.plain
    url_filter.filtersets.sqlalchemy
 

docs/big_picture.rst

@@ -10,8 +10,8 @@ Basics
 In order to filter any data, this library breaks the process
 to 3 phases:
 
-1. Parse the URL querystring into ``LookupConfig``
-2. Loop throught all the configs and generate ``FilterSpec`` when possible
+1. Parse the URL querystring into :class:`.LookupConfig`
+2. Loop throught all the configs and generate :class:`.FilterSpec` when possible
 3. Use the list of specs to actually filter data
 
 And here is a bit more information about each phase.
@@ -20,7 +20,7 @@ Parsing
 +++++++
 
 Fundamentally a querystring is a collection of key-pairs.
-As such, this data is natually flat and is usually represented
+As such, this data is naturally flat and is usually represented
 as a simple dictionary::
 
     ?foo=bar&happy=rainbows => {
@@ -35,7 +35,7 @@ as a simple dictionary::
   are present.
 
 The filtering however is not flat. Each querystring key can be nested
-when using nested ``FilterSet`` and in addition it can optionally
+when using nested :class:`.FilterSet` and in addition it can optionally
 contain lookup. For example::
 
     ?foo=bar
@@ -66,10 +66,10 @@ nested dictionaries. For example::
         }
     }
 
-That is essentially what ``LookupConfig`` stores. Since these dictionaries
+That is essentially what :class:`.LookupConfig` stores. Since these dictionaries
 are flat (each dictionaty has at most one key), it also provides some utility
 properties for dealing with such data. You can refer to the
-:py:class:`url_filter.utils.LookupConfig` API documentation for more
+:class:`.LookupConfig` API documentation for more
 information.
 
 Filter Specification
@@ -78,31 +78,32 @@ Filter Specification
 As mentioned in :doc:`README <index>`, Django URL Filter decouples parsing
 of querystring and filtering. It achieves that by constructing filter
 specifications which have all necessary information to filter data
-without actually filtering data. Thats what ``FilterSpec`` is.
+without actually filtering data. Thats what :class:`.FilterSpec` is.
 It stores 3 required pieces of information on how to filter data:
 
 * Which attribute to filter on. Since models can be related by attributes
-  related models, this actually ends up being a list of attributes which
-  we call components.
+  of related models, this actually ends up being a list of attributes which
+  we call ``components``.
 * Lookup to use to filter data. This specifies how the value should be
   compared while doing filtering. Example is ``exact``, ``contains``.
-  Currenlty only lookups from Django ORM are supported.
+  By default only lookups from Django ORM are supported however custom
+  :class:`.CallableFilter` can be used to define custom lookups.
 * If the filter is negated. For example to filter when username is ``'foo'``
-  to filter when username is not ``'foo'``.
+  or filter when username is not ``'foo'``.
 
 Filtering
 +++++++++
 
-Since filtering is decoupled from the ``FilterSet``, the filtering honors
+Since filtering is decoupled from the :class:`.FilterSet`, the filtering honors
 all go to a specified filter backend. The backend is very simple.
 It takes a list of filter specifications and a data to filter and its
 job is to filter that data as specified in the specifications.
 
 .. note::
-  Currently we only support Django ORM and SQLAlchemy filter backends
+  Currently we only support a handful of backends such as Django ORM,
+  SQLAlchemy and plain Python interables filter backends
   but you can imagine that any backend can be implemented.
-  Eventually filter backends can be added for flat data-structures
-  like filtering a vanilla Python lists or even more exotic sources
+  Eventually filter backends can be added for more exotic sources
   like Mongo, Redis, etc.
 
 Steps
@@ -111,62 +112,62 @@ Steps
 Above information hopefully puts things in perspective and here is more
 detailed step-by-step guide what Django URL Filter does behind the scenes:
 
-#. ``FilterSet`` is instantiated with querystring data as well as
-   querystring to filter.
-#. ``FilterSet`` is asked to filter given data via
-   :py:meth:`filter <url_filter.filtersets.base.FilterSet.filter>` method
+#. :class:`.FilterSet` is instantiated with querystring data as well as
+   queryset to filter.
+#. :class:`.FilterSet` is asked to filter given data via
+   :meth:`filter <url_filter.filtersets.base.FilterSet.filter>` method
    which kicks in all the steps below.
-#. ``FilterSet`` finds all filters it is capable of Filtering
-   via :py:meth:`get_filters <url_filter.filtersets.base.FilterSet.get_filters>`.
+#. :class:`.FilterSet` finds all filters it is capable of Filtering
+   via :meth:`get_filters <url_filter.filtersets.base.FilterSet.get_filters>`.
    This is where custom filtersets can hook into to do custom stuff like
    extracting filters from a Django model.
-#. ``FilterSet`` binds all child filters to itself via
-   :py:meth:`bind <url_filter.filters.Filter.bind>`.
-   This practically sets :py:attr:`parent <url_filter.filters.Filter.parent>`
-   and :py:attr:`name <url_filter.filters.Filter.name>`.
-#. Root ``FilterSet`` loops through all querystring pairs and generates
-   ``LookupConfig`` for all of them.
-#. Root ``FilterSet`` loops through all generated configs and attemps to
+#. :class:`.FilterSet` binds all child filters to itself via
+   :meth:`bind <url_filter.filters.BaseFilter.bind>`.
+   This practically sets :attr:`parent <url_filter.filters.BaseFilter.parent>`
+   and :attr:`name <url_filter.filters.BaseFilter.name>` attributes.
+#. Root :class:`.FilterSet` loops through all querystring pairs and generates
+   :class:`.LookupConfig` for all of them.
+#. Root :class:`.FilterSet` loops through all generated configs and attemps to
    find appropriate filter to use to generate a spec fo the given config.
-   The matching happens by the first key in the ``LookupConfig`` dict.
-   If that key is found in available filters, that filer is used and
+   The matching happens by the first key in the :class:`.LookupConfig` dict.
+   If that key is found in available filters, that filter is used and
    otherwise that config is skipped. This is among the reasons why
-   ``LookupConfig`` is used since it allows this step to be very simple.
+   :class:`.LookupConfig` is used since it allows this step to be very simple.
 #. If appropriate filter is found, it is passed nested config to the child
    filter which then goes through very similar process as in previous step
    until it gets to a leaf filter.
-#. Leaf ``Filter`` gets the config. In then checks if the config is still
+#. Leaf :class:`.Filter` gets the config. In then checks if the config is still
    nested. For example if the config is simply a value (e.g. ``'bar'``)
    or is still a dictionary (e.g. ``{'contains': 'bar'}``).
    If the config is just a value, it then uses a default lookup for that
    filter as provided in ``default_lookup`` parameter when instantiating
-   ``Filter``. If the config is a dictionary, it makes sure that it is a
+   :class:`.Filter`. If the config is a dictionary, it makes sure that it is a
    valid lookup config (e.g. its not ``{'rainbows': {'contains': 'bar'}}``
    since it would not know what to do with ``rainbows`` since it is not a
    valid lookup value).
-#. Now that ``Filter`` validated the lookup itself, it cleans the actual
+#. Now that :class:`.Filter` validated the lookup itself, it cleans the actual
    filter value by using either ``form_field`` as passed as parameter
-   when instantiating ``Filter`` or by using loopup overwrite.
+   when instantiating :class:`.Filter` or by using lookup overwrite.
    Overwrites are necessary for more exotic lookups like ``in`` or ``year``
    since they need to validate data in a different way.
-#. If the value is valid, then the leaf filter constructs a ``FilterSpec``
+#. If the value is valid, then the leaf filter constructs a :class:`.FilterSpec`
    since it has all the necessary information to do that - 1) all filter
    component names from all ancestors (e.g. all attributes which
    should be accessed on the queryset to get to the value to be filtered on);
    2) the actual filter value and 3) if the filter is negated.
-#. At this point, root ``FilterSpec`` will get the ``FilterSpec`` as
+#. At this point, root :class:`.FilterSet` will get the :class:`.FilterSpec` as
    bubbled up from the leaf filter. If any ``ValidationError`` exceptions
-   are raised, then depending on ``strict_mode``, it will either ignores
+   are raised, then depending on ``strict_mode``, it will either ignore
    errors or will propagate them up to the caller of the filterset.
 #. Once all specs are collected from all the querystring key-value-pairs,
-   root ``FilterSet`` instantiates a filter backend and passes it
+   root :class:`.FilterSet` instantiates a filter backend and passes it
    all the specs.
-#. Finally root ``FilterSet`` uses the filter backend to filter
+#. Finally root :class:`.FilterSet` uses the filter backend to filter
    given queryset and returns the results to the user.
 
 Some important things to note:
 
-* Root ``FilterSet`` does all the looping over querystring data and
+* Root :class:`.FilterSet` does all the looping over querystring data and
   generated configurations.
-* Children filters of a root ``FilterSet`` are only responsible for
-  generating ``FilterSpec`` and in the process validating the data.
+* Children filters of a root :class:`.FilterSet` are only responsible for
+  generating :class:`.FilterSpec` and in the process validating the data.

docs/usage.rst

@@ -4,7 +4,7 @@ Usage
 Vanilla
 -------
 
-In its simplest form, Django URL Filter usage resolves around ``FilterSet``.
+In its simplest form, Django URL Filter usage revolves around :class:`.FilterSet`.
 They can be used manually::
 
     from django import forms
@@ -30,19 +30,19 @@ They can be used manually::
 
 Notable things to mention from above:
 
-* ``FilterSet`` can be used as a ``Filter`` within another ``FilterSet``
+* :class:`.FilterSet` can be used as a :class:`.Filter` within another :class:`.FilterSet`
   hence allowing filtering by related models.
 * ``form_field`` is used to validate the filter value.
   Each lookup however can overwrite validation. For example ``year``
   lookup will use ``IntegerField`` rather then ``DateField``.
-* ``Filter`` can restrict allowed lookups for that field by
-  using ``lookup`` parameter
+* :class:`.Filter` can restrict allowed lookups for that field by
+  using ``lookups`` parameter
 
 Django
 ------
 
-Instead of manually creating ``FilterSet``, Django URL Filter comes with
-``ModelFilterSet`` which greatly simplifies the task::
+Instead of manually creating :class:`.FilterSet`, Django URL Filter comes with
+:class:`.ModelFilterSet` which greatly simplifies the task::
 
 
     from django import forms
@@ -56,7 +56,7 @@ Instead of manually creating ``FilterSet``, Django URL Filter comes with
 Notable things:
 
 * ``fields`` can actually be completely omitted. In that case
-  ``FilterSet`` will use all fields available in the model, including
+  :class:`.FilterSet` will use all fields available in the model, including
   related fields.
 * filters can be manually overwritten when custom behavior is required::
 
@@ -71,7 +71,8 @@ SQLAlchemy
 ----------
 
 `SQLAlchemy <http://www.sqlalchemy.org/>`_ works very similar to how Django
-backend works. For example::
+backend works. Additionally :class:`.SQLAlchemyModelFilterSet` is available to be able
+to easily create filter sets from SQLAlchemy models. For example::
 
     from django import forms
     from url_filter.backend.sqlalchemy import SQLAlchemyFilterBackend
@@ -89,25 +90,70 @@ backend works. For example::
 
 Notable things:
 
-* this works exactly same as ``ModelFitlerSet`` so refer above for some of
+* this works exactly same as :class:`.ModelFilterSet` so refer above for some of
   general options.
 * ``filter_backend_class`` **must** be provided since otherwise
-  ``DjangoFilterBackend`` will be used which will obviously not work
+  :class:`.DjangoFilterBackend` will be used which will obviously not work
   with SQLAlchemy models.
 * ``queryset`` given to the queryset should be SQLAlchemy query object.
 
+Plain Filtering
+---------------
+
+In addition to supporting regular ORMs ``django-url-filter`` also allows to
+filter plain Python lists of either objects or dictionaries. This feature
+is primarily meant to filter data-sources without direct filtering support
+such as lists of data in redis. For example::
+
+    from django import forms
+    from url_filter.backend.plain import PlainFilterBackend
+    from url_filter.filtersets.plain import PlainModelFilterSet
+
+    class UserFilterSet(PlainModelFilterSet):
+        filter_backend_class = PlainFilterBackend
+
+        class Meta(object):
+            # the filterset will generate fields from the
+            # primitive Python data-types
+            model = {
+                'username': 'foo',
+                'password': bar,
+                'joined': date(2015, 1, 2),
+                'profile': {
+                    'preferred_name': 'rainbow',
+                }
+            }
+
+    fs = UserFilterSet(data=QueryDict(), queryset=[{...}, {...}, ...])
+    fs.filter()
+
 Integrations
 ------------
 
 Django URL Filters tries to be usage-agnostic and does not assume
-how ``FilterSet`` is being used in the application. It does however
+how :class:`.FilterSet` is being used in the application. It does however
 ship with some common integrations to simplify common workflows.
 
+Django Class Based Views
+++++++++++++++++++++++++
+
+:class:`.FilterSet` or related classes can directly be used within Django class-based-views::
+
+    class MyFilterSet(ModelFilterSet):
+        class Meta(object):
+            model = MyModel
+
+    class MyListView(ListView):
+        queryset = MyModel.objects.all()
+        def get_queryset(self):
+            qs = super(MyListView, self).get_queryset()
+            return MyFilterSet(data=self.request.GET, queryset=qs).filter()
+
 Django REST Framework
 +++++++++++++++++++++
 
 Django URL Filter can rather easily be integrated with DRF.
-For that, a DRF filter backend is implemented and can be used in settings::
+For that, a DRF-specific filter backend :class:`.DjangoFilterBackend` is implemented and can be used in settings::
 
     # settings.py
     REST_FRAMEWORK = {
@@ -126,7 +172,7 @@ or manually set in the viewset::
 
 Note in the example above, fields to be filtered on are explicitly
 specified in the ``filter_fields`` attribute. Alternatively if more
-control over ``FilterSet`` is required, it can be set explicitly::
+control over :class:`.FilterSet` is required, it can be set explicitly::
 
     class MyFilterSet(FilterSet):
         pass
@@ -137,15 +183,4 @@ control over ``FilterSet`` is required, it can be set explicitly::
         filter_backends = [DjangoFilterBackend]
         filter_class = MyFilterSet
 
-Backends
---------
-
-``FilterSet`` by itself is decoupled from the actual filtering
-of the queryset. Backend can be swapped by using ``filter_backend_class``::
-
-    class FooFilterSet(FilterSet):
-        filter_backend_class = MyFilterBackend
-
-.. note::
-  Currently only ``DjangoFilterBackend`` is implemented which uses
-  Django ORM however more backends are planned for.
+For more available options, please refer to :class:`.DjangoFilterBackend` documentation.