Skip to content

Commit

Permalink
Merge pull request #4 from indirectlylit/channel-switching
Browse files Browse the repository at this point in the history
sync with master
  • Loading branch information
christianmemije authored Aug 11, 2016
2 parents 27e2553 + b2198c9 commit 624ea6e
Show file tree
Hide file tree
Showing 83 changed files with 2,242 additions and 393 deletions.
5 changes: 4 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -85,4 +85,7 @@ kolibri/plugins/learn/assets/src/demo/

# ignore strange OSX icon files. (Note there are special characters after 'Icon')
# see http://stackoverflow.com/a/30755378
Icon
Icon

#ignore added files with DS_Store
.DS_Store
19 changes: 19 additions & 0 deletions docs/dev/content.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
Content
======================================

This is a core module found in ``kolibri/Content``.

.. toctree::
:maxdepth: 1

content/concepts_and_definitions
content/implementation
content/api_methods
content/api_endpoints


Models
------

.. automodule:: kolibri.content.models
:members:
22 changes: 22 additions & 0 deletions docs/dev/content/api_endpoints.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
API endpoints
-------------

request specific content:

>>> localhost:8000/api/content/<channel_id>/contentnode/<content_id>

search content:

>>> localhost:8000/api/content/<channel_id>/contentnode/?search=<search words>

request specific content with specified fields:

>>> localhost:8000/api/content/<channel_id>/contentnode/<content_id>/?fields=pk,title,kind

request paginated contents

>>> localhost:8000/api/content/<channel_id>/contentnode/?page=6&page_size=10

request combines different usages

>>> localhost:8000/api/content/<channel_id>/contentnode/?fields=pk,title,kind,instance_id,description,files&page=6&page_size=10&search=wh
5 changes: 5 additions & 0 deletions docs/dev/content/api_methods.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
API Methods
-----------

.. automodule:: kolibri.content.api
:members:
36 changes: 36 additions & 0 deletions docs/dev/content/concepts_and_definitions.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
Concepts and Definitions
========================

ContentNode
-----------

High level abstraction for prepresenting different content kinds, such as Topic, Video, Audio, Exercise, Document, and can be easily extended to support new content kinds. With multiple ContentNode objects, it supports grouping, arranging them in tree structure, and symmetric and asymmetric relationship between two ContentNode objects.

File
----

A Django model that is used to store details about the source file, such as what language it supports, how big is the size, which format the file is and where to find the source file.

ContentDB Diagram
-----------------
.. image:: ../img/content_distributed_db.png
.. Source: https://www.draw.io/#G0B5xDzmtBJIQlNlEybldiODJqUHM
**PK = Primary Key
**FK = Foreign Key
**M2M = ManyToManyField
ContentTag
----------

This model is used to establish a filtering system for all ContentNode objects.


ChannelMetadata
---------------

A Django model in each content database that stores the database readable names, description and author for each channel.

ChannelMetadataCache
--------------------
This class stores the channel metadata cached/denormed into the default database.
115 changes: 115 additions & 0 deletions docs/dev/content/implementation.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,115 @@
Implementation Details and Workflows
====================================

To achieve using separate databases for each channel and being able to switch channels dynamically, the following data structure and utility functions have been implemented.

ContentDBRoutingMiddleware
--------------------------

This middleware will be applied to every request, and will dynamically select a database based on the channel_id.
If a channel ID was included in the URL, it will ensure the appropriate content DB is used for the duration of the request. (Note: `set_active_content_database` is thread-local, so this shouldn't interfere with other parallel requests.)

For example, this is how the client side dynamically requests data from a specific channel:

>>> localhost:8000/api/content/<channel_1_id>/contentnode

this will respond with all the contentnode data stored in database <channel_1_id>.sqlite3

>>> localhost:8000/api/content/<channel_2_id>/contentnode

this will respond with all the contentnode data stored in database <channel_2_id>.sqlite3

get_active_content_database
---------------------------

A utility function to retrieve the temporary thread-local variable that `using_content_database` sets

set_active_content_database
---------------------------

A utility function to set the temporary thread-local variable

using_content_database
----------------------

A decorator and context manager to do queries on a specific content DB.

Usage as a context manager:

.. code-block:: python
from models import ContentNode
with using_content_database("nalanda"):
objects = ContentNode.objects.all()
return objects.count()
Usage as a decorator:

.. code-block:: python
from models import ContentNode
@using_content_database('nalanda')
def delete_all_the_nalanda_content():
ContentNode.objects.all().delete()
ContentDBRouter
---------------

A router that decides what content database to read from based on a thread-local variable.

ContentNode
-----------

``ContentNode`` is implemented as a Django model that inherits from two abstract classes, MPTTModel and ContentDatabaseModel.
`django-mptt's MPTTModel <http://django-mptt.github.io/django-mptt/>`_, which
allows for efficient traversal and querying of the ContentNode tree.
``ContentDatabaseModel`` is used as a marker so that the content_db_router knows to query against the content database only if the model inherits from ContentDatabaseModel.

The tree structure is established by the ``parent`` field that is a foreign key pointing to another ContentNode object. You can also create a symmetric relationship using the ``related`` field, or an asymmetric field using the ``is_prerequisite`` field.

File
----

The ``File`` model also inherits from ``ContentDatabaseModel``.

To find where the source file is located, the class method ``get_url`` uses the ``checksum`` field and ``settings.CONTENT_STORAGE_DIR`` to calculate the file path. Every source file is named based on its MD5 hash value (this value is also stored in the ``checksum`` field) and stored in a namespaced folder under the directory specified in ``settings.CONTENT_STORAGE_DIR``. Because it's likely to have thousands of content files, and some filesystems cannot handle a flat folder with a large number of files very well, we create namespaced subfolders to improve the performance. So the eventual file path would look something like:

``/home/user/.kolibri/content/storage/9/8/9808fa7c560b9801acccf0f6cf74c3ea.mp4``

As you can see, it is fine to store your content files outside of the kolibri project folder as long as you set the ``settings.CONTENT_STORAGE_DIR`` accordingly.

The front-end will then use the ``extension`` field to decide which content player should be used. When the ``supplementary`` field's value is ``True``, that means this File object isn't necessary and can display the content without it. For example, we will mark caption (subtitle) file as supplementary.

Content Constants
-----------------

A Python module that stores constants for the ``kind`` field in ContentNode model and the ``preset`` field and ``extension`` field in File model.

.. automodule:: kolibri.content.constants.content_kinds
.. automodule:: kolibri.content.constants.extensions
.. automodule:: kolibri.content.constants.presets

Workflows
---------

There are two workflows we currently designed to handle content UI rendering and content playback rendering

- Content UI Rendering

1. Start with a ContentNode object.
2. Get the associated File object that has the ``thumbnail`` field being True.
3. Get the thumbnail image by calling this File's ``get_url`` method.
4. Determine the template using the ``kind`` field of this ContentNode object.
5. Renders the template with the thumbnail image.


- Content Playback Rendering

1. Start with a ContentNode object.
2. Retrieve a queryset of associated File objects that are filtered by the preset.
3. Use the ``thumbnail`` field as a filter on this queryset to get the File object and call this File object's ``get_url`` method to get the source file (the thumbnail image)
4. Use the ``supplementary`` field as a filter on this queryset to get the "supplementary" File objects, such as caption (subtitle), and call these File objects' ``get_url`` method to get the source files.
5. Use the ``supplementary`` field as a filter on this queryset to get the essential File object. Call its ``get_url`` method to get the source file and use its ``extension`` field to choose the content player.
6. Play the content.
Binary file added docs/dev/img/content_distributed_db.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 1 addition & 0 deletions docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ Getting Started

user/index
dev/index
dev/content
cli
changelog
contributing
Expand Down
1 change: 1 addition & 0 deletions frontend_build/src/webpack.config.base.js
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,7 @@ var config = {
resolve: {
alias: {
'kolibri_module': path.resolve('kolibri/core/assets/src/kolibri_module'),
'core-constants': path.resolve('kolibri/core/assets/src/constants'),
'core-base': path.resolve('kolibri/core/assets/src/vue/core-base'),
'nav-bar-item': path.resolve('kolibri/core/assets/src/vue/nav-bar/nav-bar-item'),
'nav-bar-item.styl': path.resolve('kolibri/core/assets/src/vue/nav-bar/nav-bar-item.styl'),
Expand Down
54 changes: 51 additions & 3 deletions kolibri/auth/api.py
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
from __future__ import absolute_import, print_function, unicode_literals

from django.contrib.auth import get_user
from django.contrib.auth import authenticate, get_user, login, logout
from django.contrib.auth.models import AnonymousUser
from rest_framework import filters, permissions, viewsets
from rest_framework import filters, permissions, status, viewsets
from rest_framework.response import Response

from .models import Classroom, DeviceOwner, Facility, FacilityUser, LearnerGroup, Membership, Role
Expand Down Expand Up @@ -106,7 +106,7 @@ def list(self, request):
elif type(logged_in_user) is AnonymousUser:
return Response(Facility.objects.all().values_list('id', flat=True))
else:
return Response(logged_in_user.facility)
return Response(logged_in_user.facility_id)


class ClassroomViewSet(viewsets.ModelViewSet):
Expand All @@ -123,3 +123,51 @@ class LearnerGroupViewSet(viewsets.ModelViewSet):
serializer_class = LearnerGroupSerializer

filter_fields = ('parent',)


class SessionViewSet(viewsets.ViewSet):

def create(self, request):
username = request.data.get('username', '')
password = request.data.get('password', '')
facility_id = request.data.get('facility', None)
user = authenticate(username=username, password=password, facility=facility_id)
if user is not None and user.is_active:
# Correct password, and the user is marked "active"
login(request, user)
# Success!
return Response(self.get_session(request))
else:
# Respond with error
return Response("User credentials invalid!", status=status.HTTP_401_UNAUTHORIZED)

def destroy(self, request, pk=None):
logout(request)
return Response([])

def retrieve(self, request, pk=None):
return Response(self.get_session(request))

def get_session(self, request):
user = get_user(request)
if isinstance(user, AnonymousUser):
return {'id': None, 'username': '', 'full_name': '', 'user_id': None, 'facility_id': None, 'kind': 'ANONYMOUS', 'error': '200'}

session = {'id': 'current', 'username': user.username,
'full_name': user.full_name,
'user_id': user.id}
if isinstance(user, DeviceOwner):
session.update({'facility_id': None, 'kind': 'SUPERUSER', 'error': '200'})
return session
else:
roles = Role.objects.filter(user_id=user.id)
if len(roles) is not 0:
session.update({'facility_id': user.facility_id, 'kind': [], 'error': '200'})
for role in roles:
if role.kind == 'admin':
session['kind'].append('ADMIN')
else:
session['kind'].append('COACH')
else:
session.update({'facility_id': user.facility_id, 'kind': 'LEARNER', 'error': '200'})
return session
4 changes: 3 additions & 1 deletion kolibri/auth/api_urls.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
from rest_framework import routers

from .api import (
ClassroomViewSet, CurrentFacilityViewSet, DeviceOwnerViewSet, FacilityUserViewSet, FacilityViewSet, LearnerGroupViewSet, MembershipViewSet, RoleViewSet
ClassroomViewSet, CurrentFacilityViewSet, DeviceOwnerViewSet, FacilityUserViewSet, FacilityViewSet, LearnerGroupViewSet, MembershipViewSet, RoleViewSet,
SessionViewSet
)

router = routers.SimpleRouter()
Expand All @@ -12,6 +13,7 @@
router.register(r'role', RoleViewSet)
router.register(r'facility', FacilityViewSet)
router.register(r'currentfacility', CurrentFacilityViewSet, base_name='currentfacility')
router.register(r'session', SessionViewSet, base_name='session')
router.register(r'classroom', ClassroomViewSet)
router.register(r'learnergroup', LearnerGroupViewSet)

Expand Down
13 changes: 6 additions & 7 deletions kolibri/auth/backends.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,14 +21,13 @@ def authenticate(self, username=None, password=None, facility=None):
:param facility: a Facility
:return: A FacilityUser instance if successful, or None if authentication failed.
"""
try:
user = FacilityUser.objects.get(username=username, facility=facility)
users = FacilityUser.objects.filter(username=username)
if facility:
users = users.filter(facility=facility)
for user in users:
if user.check_password(password):
return user
else:
return None
except FacilityUser.DoesNotExist:
return None
return None

def get_user(self, user_id):
"""
Expand All @@ -48,7 +47,7 @@ class DeviceOwnerBackend(object):
A class that implements authentication for DeviceOwners.
"""

def authenticate(self, username=None, password=None):
def authenticate(self, username=None, password=None, **kwargs):
"""
Authenticates the user if the credentials correspond to a DeviceOwner.
Expand Down
Loading

0 comments on commit 624ea6e

Please sign in to comment.