Serving two Django sites with WSGI results in internal server error for only one of the sites

Question

I have a Django site and a Django CMS site which I am serving from the same ubuntu 14.04 server running MySQL 5.6, Apache2 2.4.7 and Django 1.8 via mod_wsgi version 4.5.7 using virtual hosts. Locally (on my Linux PC) I have managed to accomplish this with both sites working perfectly and hence decided to migrate to the server.

After the migration, taking care that everything has the same version, the situation is such that the Django site is working properly while the Django CMS site is giving me and internal server error 500. These are my virtual hosts' .conf files, and the wsgi.py file for the broken site.

# wsgi.py
import os
from django.core.wsgi import get_wsgi_application
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "bsg.settings")
application = get_wsgi_application()

# Django_cms_site.conf
<VirtualHost *:80>
    ServerName site.com
    ServerAdmin webmaster@localhost.com

    Alias /static/ /home/bsg/cms/static/
    Alias /media/ /home/bsg/cms/media/

    WSGIScriptAlias / /home/bsg/cms/bsg/wsgi.py
    WSGIDaemonProcess bsgcms python-path=/home/hicklin/bsg/cms:/home/venv-bsg/lib/python2.7/site-packages
    WSGIProcessGroup bsgcms

    <Directory /home/bsg/cms>
        Require all granted
    </Directory>

    LogLevel warn
    ErrorLog /var/log/apache2/cms-error.log
    CustomLog /var/log/apache2/cms-access.log combined
</VirtualHost>

# Django_site.conf
<VirtualHost *:80>
    ServerName django.site.com
    ServerAdmin webmaster@localhost.com

    Alias /static/ /home/bsg/admin/site/static/

    WSGIScriptAlias / /home/bsg/admin/site/wsgi.py
    WSGIDaemonProcess bsgadmin python-path=/home/bsg/admin:/home/hicklin/venv-bsg/lib/python2.7/site-packages
    WSGIProcessGroup bsgadmin

    <Directory /home/bsg/admin>
        Require all granted
    </Directory>

    LogLevel warn
    ErrorLog /var/log/apache2/admin-error.log
    CustomLog /var/log/apache2/admin-access.log combined
</VirtualHost>

As can be noted, I am using the same virtual environment for both sites. The relevant logs give the following error.

[Tue Oct 11 22:39:43.416901 2016] [wsgi:error] [pid 19799] [remote 77.71.226.73:64984] mod_wsgi (pid=19799): Target WSGI script '/home/bsg/cms/bsg/wsgi.py' cannot be loaded as Python module.
[Tue Oct 11 22:39:43.416942 2016] [wsgi:error] [pid 19799] [remote 77.71.226.73:64984] mod_wsgi (pid=19799): Exception occurred processing WSGI script '/home/bsg/cms/bsg/wsgi.py'.
[Tue Oct 11 22:39:43.416977 2016] [wsgi:error] [pid 19799] [remote 77.71.226.73:64984] Traceback (most recent call last):
[Tue Oct 11 22:39:43.417014 2016] [wsgi:error] [pid 19799] [remote 77.71.226.73:64984]   File "/home/bsg/cms/bsg/wsgi.py", line 16, in <module>
[Tue Oct 11 22:39:43.417067 2016] [wsgi:error] [pid 19799] [remote 77.71.226.73:64984]     application = get_wsgi_application()
[Tue Oct 11 22:39:43.417093 2016] [wsgi:error] [pid 19799] [remote 77.71.226.73:64984]   File "/home/venv-bsg/lib/python2.7/site-packages/django/core/wsgi.py", line 14, in get_wsgi_application
[Tue Oct 11 22:39:43.417134 2016] [wsgi:error] [pid 19799] [remote 77.71.226.73:64984]     django.setup()
[Tue Oct 11 22:39:43.417158 2016] [wsgi:error] [pid 19799] [remote 77.71.226.73:64984]   File "/home/venv-bsg/lib/python2.7/site-packages/django/__init__.py", line 18, in setup
[Tue Oct 11 22:39:43.417194 2016] [wsgi:error] [pid 19799] [remote 77.71.226.73:64984]     apps.populate(settings.INSTALLED_APPS)
[Tue Oct 11 22:39:43.417217 2016] [wsgi:error] [pid 19799] [remote 77.71.226.73:64984]   File "/home/venv-bsg/lib/python2.7/site-packages/django/apps/registry.py", line 78, in populate
[Tue Oct 11 22:39:43.417255 2016] [wsgi:error] [pid 19799] [remote 77.71.226.73:64984]     raise RuntimeError("populate() isn't reentrant")
[Tue Oct 11 22:39:43.417289 2016] [wsgi:error] [pid 19799] [remote 77.71.226.73:64984] RuntimeError: populate() isn't reentrant

After modifying the wsgi.py to get the real error as suggested by Dirk Eschler in the thread, the error in /var/log/apache2/cms-error.log changes to

[Tue Oct 11 21:36:06.087723 2016] [wsgi:error] [pid 21584] handling WSGI exception
[Tue Oct 11 21:36:06.087811 2016] [wsgi:error] [pid 21584] Traceback (most recent call last):
[Tue Oct 11 21:36:06.087854 2016] [wsgi:error] [pid 21584]   File "/home/bsg/cms/bsg/wsgi.py", line 9, in <module>
[Tue Oct 11 21:36:06.087975 2016] [wsgi:error] [pid 21584]     application = get_wsgi_application()
[Tue Oct 11 21:36:06.087996 2016] [wsgi:error] [pid 21584]   File "/home/venv-bsg/lib/python2.7/site-packages/django/core/wsgi.py", line 14, in get_wsgi_application
[Tue Oct 11 21:36:06.088067 2016] [wsgi:error] [pid 21584]     django.setup()
[Tue Oct 11 21:36:06.088093 2016] [wsgi:error] [pid 21584]   File "/home/venv-bsg/lib/python2.7/site-packages/django/__init__.py", line 17, in setup
[Tue Oct 11 21:36:06.088164 2016] [wsgi:error] [pid 21584]     configure_logging(settings.LOGGING_CONFIG, settings.LOGGING)
[Tue Oct 11 21:36:06.088190 2016] [wsgi:error] [pid 21584]   File "/home/venv-bsg/lib/python2.7/site-packages/django/conf/__init__.py", line 48, in __getattr__
[Tue Oct 11 21:36:06.088366 2016] [wsgi:error] [pid 21584]     self._setup(name)
[Tue Oct 11 21:36:06.088388 2016] [wsgi:error] [pid 21584]   File "/home/venv-bsg/lib/python2.7/site-packages/django/conf/__init__.py", line 42, in _setup
[Tue Oct 11 21:36:06.088417 2016] [wsgi:error] [pid 21584]     % (desc, ENVIRONMENT_VARIABLE))
[Tue Oct 11 21:36:06.088442 2016] [wsgi:error] [pid 21584] ImproperlyConfigured: Requested setting LOGGING_CONFIG, but settings are not configured. You must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings.
[Tue Oct 11 21:36:08.591950 2016] [wsgi:error] [pid 21584] [remote 77.71.226.73:3851] mod_wsgi (pid=21584): Target WSGI script '/home/bsg/cms/bsg/wsgi.py' does not contain WSGI application 'application'.

Please note that I have managed to run, access and work the cms site via python manage.py runserver and it works without a hitch. Following the latest error did not result in anything fruitful yet. Any help or hints are greatly appreciated.

Graham Dumpleton · Answer 1 · 2016-10-12T00:24:18.273

This is caused by your database or some other resource needed during initialisation of Django not being available, or Django initialisation otherwise failing in some other way the first time. Back in time the initialisation of Django was reentrant and could be called a second time if the first time it failed. This is no longer the case so if initialisation fails the first time, you are forced to restart the process.

As you are using a recent version of mod_wsgi, you should be able to add an additional option to WSGIDaemonProcess to deal with this issue.

startup-timeout=15

What will happen is that if mod_wsgi can't successfully load the WSGI script file within 15 seconds of the first attempt to do so, then the mod_wsgi daemon process will be automatically restarted. It will keep doing this if it keeps failing because of some service required during initialisation not being available.

BTW, there should have been an earlier error message in the error log than that which gave the real reason why initialisation failed. That message you quote is from the subsequent failures after the first. I don't trust that the other message you give is correct, so go back and look for the first message after Apache restart.

Finally, best practice is not to use:

WSGIDaemonProcess bsgadmin python-path=/home/bsg/admin:/home/hicklin/venv-bsg/lib/python2.7/site-packages

but:

WSGIDaemonProcess bsgadmin python-home=home/hicklin/venv-bsg python-path=/home/bsg/admin

That is, use python-home option to specify location of Python virtual environment, do not add site-packages to python-path. See:

http://blog.dscpl.com.au/2014/09/using-python-virtual-environments-with.html

Sorry for the late reply. It took some time to find a solution for this. Firstly I'd like to thank you for your swift reply. — William Hicklin, Oct 15 '16 at 23:30

score 0 · Answer 2 · answered Oct 16 '16 at 00:04

Adding startup-timeout=15 to WSGIDaemonProcess in the virtual hosts config file did not solve my issue.

As for the "earlier error message in the error log" with the modified wsgi.py to get the real error, the only errors present in the /var/log/apache2/cms-error.log seem to be related to the fact that the "Target WSGI script does not contain WSGI application 'application'". As indicated in the question. Inspection of the apache2 error log during restart and subsequent attempts of loading the site showed the typical output.

[Wed Oct 12 16:16:01.372050 2016] [mpm_prefork:notice] [pid 23564] AH00169: caught SIGTERM, shutting down
[Wed Oct 12 16:16:02.292367 2016] [mpm_prefork:notice] [pid 1169] AH00163: Apache/2.4.7 (Ubuntu) mod_wsgi/4.5.7 Python/2.7 PHP/5.5.9-1ubuntu4.20 configured -- resuming normal operations
[Wed Oct 12 16:16:02.292455 2016] [core:notice] [pid 1169] AH00094: Command line: '/usr/sbin/apache2'

Finally, what worked was to format the server and start all over again. The mod_WSGI is now working perfectly with both sites. I have also upgraded to MySQL 5.7 and ubuntu 16.04.

I would like to point out that the final suggestion about how to set the python-path and python-home did not work with the old system, however, it worked without any issues on the new system. This leads me to think that I somehow installed a modified mod-WSGI or it was corrupted during installation. I am not sure of this.

Serving two Django sites with WSGI results in internal server error for only one of the sites

2 Answers2