Problem running collectstatic_schemas: Files from MULTITENANT_STATICFILES_DIRS are not collected.

[loctimize]

Hi everybody,
When running python manage.py collectstatic_schemas --schema=schema_name, files in MULTITENANT_STATICFILES_DIRS are not collected. I am running django v.2.2.4 and django_tenants v.2.2.3 on Python 3.6.7.

My relevant settings in settings.py are:

DEFAULT_FILE_STORAGE = "django_tenants.files.storage.TenantFileSystemStorage"

STATICFILES_FINDERS = [
    "django_tenants.staticfiles.finders.TenantFileSystemFinder",
    "django.contrib.staticfiles.finders.FileSystemFinder",
    "django.contrib.staticfiles.finders.AppDirectoriesFinder",
    "compressor.finders.CompressorFinder",
]

MULTITENANT_STATICFILES_DIRS = os.path.realpath(os.path.join(BASE_DIR, '..', "tenants/%s/static/"))

STATICFILES_DIRS = []
STATICFILES_STORAGE = "django_tenants.staticfiles.storage.TenantStaticFilesStorage"

MULTITENANT_RELATIVE_STATIC_ROOT = ""
STATIC_ROOT = os.path.join(BASE_DIR, '..', "staticfiles/") 
STATIC_URL = '/static/'

When print out the path variable in the listdir method in line 311 in /site-packages/django/core/files/storage.py that lists all dirs that are going to be collected, the first paths are:

>>> /Users/x/PycharmProjects/APPDIR/staticfiles/schema_name
>>> /Users/x/environments/venv36/lib/python3.6/site-packages/dal/static
>>> /Users/x/environments/venv36/lib/python3.6/site-packages/dal/static/autocomplete_light
>>> /Users/x/environments/venv36/lib/python3.6/site-packages/dal_select2/static

Strange thing is that the first path lists the target static path for the tenant and not the files from MULTITENANT_STATICFILES_DIRS where I have 3 subdirectories named css, img, js.

Any idea what is happening here? Any additional information you need?

Many thanks!

[jcass77]

I think you still need to provide a value for STATICFILES_DIRS, for example:

# https://docs.djangoproject.com/en/dev/ref/contrib/staticfiles/#std:setting-STATICFILES_DIRS
STATICFILES_DIRS = [str(APPS_DIR / "static")]

There is also a pending PR at #266 that addresses a few known issues - perhaps try checking out that branch and let us know if that, and the config changes suggested above, resolves your issue?

[loctimize]

I added this to STATICFILES_DIRS but it didn't solve the problem:

STATICFILES_DIRS = [
    os.path.join(BASE_DIR, '..', "static"),  # dev only after collecting static schemas
]

The static files in this dir are collected but still the files in MULTITENANT_STATICFILES_DIRS are ignored.

[jcass77]

Ok, maybe have a look at #265 and #266 as well.

[loctimize]

Thanks @jcass77! I check out jcass77:fix/staticfiles (https://github.com/tomturner/django-tenants/tree/89bcc6b4c64eaf31a3bbafa40116265affd5b9b5) and this did the trick for me. Now static files from tenants are collected. Also if no

STATICFILES_DIRS = [
    os.path.join(BASE_DIR, '..', "static"),  # dev only after collecting static schemas
]

is specified in settings.py.

Anything else I should / could do to help? Am quite new to the whole error reporting / fixing thing ;-)

[jcass77]

@loctimize, I received a few GitHub email notifications containing comments about issues that you were experiencing, but can't find them linked to this issue anymore?

@jcass77 : When using jcass77:fix/staticfiles (https://github.com/tomturner/django-tenants/tree/89bcc6b4c64eaf31a3bbafa40116265affd5b9b5) collectstatic_schemas is working fine. Problem is only that MULTITENANT_RELATIVE_MEDIA_ROOT is not working and media files are uploaded into the main media directory.

At any rate, while looking over #266 again I realised that a bunch of the tests cases in that PR weren't actually being run, which uncovered a handful of regressions that I since fixed via 4b03611.

Could you perhaps check out a fresh copy of #266 and try again?

[loctimize]

@jcass77 : I checked out https://github.com/tomturner/django-tenants/tree/68938b6f04a4904ec7d593a8d3c986996b5a0f6c and ran the tests encountering 2 errors. Environment: MacOSX, Python 3.6.7, Django 2.2.1 and 2.2.4

Here is my test output:

======================================================================
ERROR: test_get_template_based_on_tenant (django_tenants.tests.test_static_files.TemplateLoaderTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/x/.pyenv/versions/3.6.7/envs/venv36/lib/python3.6/site-packages/django_tenants/tests/test_static_files.py", line 78, in test_get_template_based_on_tenant
    template = self.engine.get_template("index.html")
  File "/Users/x/.pyenv/versions/3.6.7/envs/venv36/lib/python3.6/site-packages/django/template/engine.py", line 143, in get_template
    template, origin = self.find_template(template_name)
  File "/Users/x/.pyenv/versions/3.6.7/envs/venv36/lib/python3.6/site-packages/django/template/engine.py", line 129, in find_template
    raise TemplateDoesNotExist(name, tried=tried)
django.template.exceptions.TemplateDoesNotExist: index.html

======================================================================
ERROR: test_get_template (django_tenants.tests.test_static_files.TenantCachedLoaderTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/x/.pyenv/versions/3.6.7/envs/venv36/lib/python3.6/site-packages/django_tenants/tests/test_static_files.py", line 312, in test_get_template
    template = self.engine.get_template("index.html")
  File "/Users/x/.pyenv/versions/3.6.7/envs/venv36/lib/python3.6/site-packages/django/template/engine.py", line 143, in get_template
    template, origin = self.find_template(template_name)
  File "/Users/x/.pyenv/versions/3.6.7/envs/venv36/lib/python3.6/site-packages/django/template/engine.py", line 129, in find_template
    raise TemplateDoesNotExist(name, tried=tried)
django.template.exceptions.TemplateDoesNotExist: index.html

----------------------------------------------------------------------
Ran 46 tests in 6.880s

FAILED (errors=2)
Destroying test database for alias 'default'...

Update: When running collectstatic_schemas for a tenant, files from the tenant-specific dirs defined in MULTITENANT_STATICFILES_DIRS are not copied.

[jcass77]

I can't really think of a reason why the tests would fail - they're all green both when I run them locally and on Travis:

You might have to do a bit of debugging to point us in the right direction :(

[loctimize]

@jcass77 I am trying ;-)
Back to the original problem in my report: When I run collectstatic_schemas on the newly checked out version, static files are copied from STATICFILES_DIRS instead of MULTITENANT_STATICFILES_DIRS. All my media files for the tenant are copied to staticfiles, too! I guess, this is not intended.

I do have a version of django_tenants based on https://github.com/tomturner/django-tenants/tree/89bcc6b4c64eaf31a3bbafa40116265affd5b9b5 that I fixed late at night some days ago. This one works as expected for collecting the static files and tests pass ok. I uploaded it here. I just changed files/storages.py and staticfiles/storage.py as far as I remember. Repo: https://github.com/loctimize/django_tenants/commits/master

I am still learning the whole git workflow thing so sorry for not being more efficient here.

[jcass77]

Thanks for the feedback @loctimize.

When I run collectstatic_schemas on the newly checked out version, static files are copied from STATICFILES_DIRS instead of MULTITENANT_STATICFILES_DIRS.

What we are aiming for here is that any available files in MULTITENANT_STATICFILES_DIRS will override those in STATICFILES_DIRS.

Extract from the docs:

absolute/path/to/your_project_dir
    static              # System-wide static files
    # Tenant-specific files below will override pre-existing system-wide files with same name.
    tenants
        tenant_1        # Static files for tenant_1
            static
        tenant_2        # Static files for tenant_2
            static
    staticfiles         # Created automatically when collectstatic_schemas is run
        tenant_1
        tenant_2

...so we can then have the following scenarios:

1. only STATICFILES_DIRS are provided: all tenants will share the same set of system-wide static files.

2. only MULTITENANT_STATICFILES_DIRS are provided: each tenant has to provide its own complete set of static files.

3. both STATICFILES_DIRS and MULTITENANT_STATICFILES_DIRS are provided. Each tenant only has to provide the static files that are unique to, or customised for, that tenant. These files will then override the system-wide defaults.

All my media files for the tenant are copied to staticfiles, too! I guess, this is not intended.

This is a bug - I might have broken something when I tried to get all of this to work with django-sendfile2 so that files in /media can be protected from unauthorised access - I'll look into it.

[jcass77]

I've pushed d8140ae to #266 which should resolve the issue of media files being collected as well when ``collectstatic_schemas` is run.

[delegate-it]

I am using the fix and it's working great. It collects specific images and css from the tenants MULTITENANT_STATICFILES_DIRS. Thank you, for the fix