Multi Tenant Application using Django – Connect and manage multiple databases to the single application

May 21, 2017May 21, 2017 tushar

Today I am going to talk about multi-tenancy. What is multi-tenancy and how to make a multi-tenant architecture based application. Multi-tenancy refers to single instance of the application which server multiple tenants. Tenants refer to the group of users who have common access with specific privileges to the software resource. Suppose that there is a simple application connected to the multiple databases, and each database belongs to the specific users or tenant. Today, I am going to show you how to make a Django Applicaion connect to the multiple database, switching the databases at run time, get data from the multiple databases and also managing the django admin of the each database site. By the database site we refer to the URL where particular database can be managed. Each database has its own corresponding URL or site, eg: testdb1 has admin uri say – http://localhost:8000/testdb1_admin/ or testdb2 has admin uri say – http://localhost:8000/testdb2_admin/. If you are only interested in the source code, it is hosted in my github account here – https://github.com/tusharcoder/multiDbApp. To set up the project:-

# multiDbApp
```bash
#clone the project
git clone https://github.com/tusharcoder/multiDbApp.git

#make virtual environment
virtualenv venv -p python3
#activate the environment
source venv/bin/activate

cd multiDbApp

#install dependencies
pip install -r requirements.txt

#login to mysql and create databases user - root, password - root
mysql -u root -p
root

create database testdb1;
create database testdb2;
create database testdb3;
create database testdb4;
create database testdb5;
#create default database
create database testdbdefault;

#run migrations
python manage.py migrate core testdb1 
python manage.py migrate core testdb2
python manage.py migrate core testdb3
python manage.py migrate core testdb4
python manage.py migrate core testdb5  
python manage.py migrate default

#create super user
python manage.py createsuperuser

#run command to populate the database, testdb1, testdb2.... are the arguments
python manage.py populate_db testdb1 
python manage.py populate_db testdb2
python manage.py populate_db testdb3
python manage.py populate_db testdb4
python manage.py populate_db testdb5  

#run the server
python manage.py runserver
#go to browser to see the test data from multiple databases
#http://localhost:8000/test/ 
#to go to admin backend of testdb1
#http://localhost:8000/testdb1_admin/
#enter crerdentials
#user:- admin
#password:- qazplmq1w2e3r4
#for admin of testdb2
#http://localhost:8000/testdb1_admin/ 
#and so on....
```

# multiDbApp

```bash

#clone the project

git clone https://github.com/tusharcoder/multiDbApp.git

#make virtual environment

virtualenv venv -p python3

#activate the environment

source venv/bin/activate

cd multiDbApp

#install dependencies

pip install -r requirements.txt

#login to mysql and create databases user - root, password - root

mysql -u root -p

root

create database testdb1;

create database testdb2;

create database testdb3;

create database testdb4;

create database testdb5;

#create default database

create database testdbdefault;

#run migrations

python manage.py migrate core testdb1

python manage.py migrate core testdb2

python manage.py migrate core testdb3

python manage.py migrate core testdb4

python manage.py migrate core testdb5

python manage.py migrate default

#create super user

python manage.py createsuperuser

#run command to populate the database, testdb1, testdb2.... are the arguments

python manage.py populate_db testdb1

python manage.py populate_db testdb2

python manage.py populate_db testdb3

python manage.py populate_db testdb4

python manage.py populate_db testdb5

#run the server

python manage.py runserver

#go to browser to see the test data from multiple databases

#http://localhost:8000/test/

#to go to admin backend of testdb1

#http://localhost:8000/testdb1_admin/

#enter crerdentials

#user:- admin

#password:- qazplmq1w2e3r4

#for admin of testdb2

#http://localhost:8000/testdb1_admin/

#and so on....

```

In the […]

Integration of Elastic Search and Django(A Python Framework) – Part three

March 13, 2017March 13, 2017 tushar

In the previous tutorials I made a simple django application, and made a custom command(seed_db) that populate the database with the dummy data, also take count as argument of how many products will be created. Now In this tutorial we will learn, how to index our database with the elastic search. Elasticsearch, in simple words it is like a search engine provide capabilities like full text searching, filtering, faceting and many more with fast speed on non relational JSON like documents. So its like a private search for your web application. Upto this level we had a simple django based application consist of the five models, one is built in model, i.e. User model from django.contrib.auth.models, others are Product, Category, Subcategory and the UserProfile model. We also implemented a custom command to populate database with the dummy data. So we can have any number of dummy records in our db, to make it indexed in elasticsearch. So thats the reason to make seed_db command. Now I am going to make one more custom command i.e. index_db, to index the database […]

Integration of Elastic Search and Django(A Python Framework) – Part Two

March 13, 2017March 13, 2017 tushar

In the previous tutorial I made a simple django application, now in this tutorial I will make a custom command that will populate the database with the dummy data. We will use extra modules of the django listed as follows:- Model Mommy Faker django-extensions activate the virtual environnent

source venv/bin/activate

1	source venv/bin/activate

install model mommy

pip install model-mommy

1	pip install model-mommy

same, install faker

pip install faker

1	pip install faker

install django-extensions

pip install django-extensions

1	pip install django-extensions

add django_extensions to the Installed apps in the settings.py. Make a sample command

python manage.py create_command core

1	python manage.py create_command core

This will make a folder management with the sample command (sample.py) in the core folder. Rename this file sample.py to say seed_db.py. Update the code of the file as:-

from django.core.management.base import BaseCommand
from model_mommy import mommy
import random
from core.models import *
from faker import Factory
from django.contrib.auth.models import User

faker = Factory.create()
def random_with_N_digits(n):
    """utility function to generate the random number of the length n"""
    range_start = 10**(n-1)
    range_end = (10**n)-1
    return random.randint(range_start, range_end)
class Command(BaseCommand):
    help = "My shiny new management command."

    def add_arguments(self, parser):
        parser.add_argument('count', nargs=1, type=int)

    def handle(self, *args, **options):

        print("cleaning the database")
        Product.objects.all().delete()
        Category.objects.all().delete()
        Subcategory.objects.all().delete()
        UserProfile.objects.all().delete()
        users=User.objects.exclude(is_staff=True).delete()
        self.create_category()
        self.create_subcategpries()
        self.create_users()
        self.createProducts(options)
        print("production finished cheers:)")

    def createProducts(self,options):
        """function to create the products"""
        print( "finally, creating the products...")
        self.products=[]
        for i in range(options['count'][0]):
            price = random_with_N_digits(3)
            category=random.choice(self.categories)
            name='{}{} product'.format(faker.name(),category.name)
            subcategory=random.choice(list(filter(lambda x : x.category.name==category.name, self.subcategories)))
            seller=random.choice(self.userprofiles)
            product_type=random.choice(['downloadable_product','shipped_product'])
            product=mommy.prepare(Product,name=name,price=price,category=category,subcategory=subcategory,seller=seller,product_type=product_type)
            self.products.append(product)
        Product.objects.bulk_create(self.products)

    def create_category(self):
        """function to create the categories"""
        print( "categories...")
        _list=['books','toys','kitchenware','computer and accessories','mobile and tablets','clothing and Accessories','watches and eyeware']
        self.categories=[]
        for param in _list:
            category=mommy.make(Category,name=param)
            self.categories.append(category)
    def create_subcategpries(self):
        """function to create the sub categories"""
        print( "subcategories...")
        self.subcategories=[]
        for category in self.categories:
            sub_cat_name='{} {}'.format(faker.name(),category.name)
            subcategory=mommy.make(Subcategory, name=sub_cat_name, category=category)
            self.subcategories.append(subcategory)
    def create_users(self):
        """function to create the users"""
        print( "users...")
        self.userprofiles=[]
        domains=['gmail','outlook','yahoo','yahoo.co.in']
        for r in range(0,100):
            email = '{}{}@{}'.format(faker.name(),str(r),random.choice(domains))
            password=random_with_N_digits(10)
            user=User.objects.create_user(email,email,password)
            userprofile=mommy.make(UserProfile,user=user,user_type='vendor')
            self.userprofiles.append(userprofile)

from django.core.management.base import BaseCommand

from model_mommy import mommy

import random

from core.models import *

from faker import Factory

from django.contrib.auth.models import User

faker = Factory.create()

def random_with_N_digits(n):

"""utility function to generate the random number of the length n"""

range_start = 10**(n-1)

range_end = (10**n)-1

return random.randint(range_start, range_end)

class Command(BaseCommand):

help = "My shiny new management command."

def add_arguments(self, parser):

parser.add_argument('count', nargs=1, type=int)

def handle(self, *args, **options):

print("cleaning the database")

Product.objects.all().delete()

Category.objects.all().delete()

Subcategory.objects.all().delete()

UserProfile.objects.all().delete()

users=User.objects.exclude(is_staff=True).delete()

self.create_category()

self.create_subcategpries()

self.create_users()

self.createProducts(options)

print("production finished cheers:)")

def createProducts(self,options):

"""function to create the products"""

print( "finally, creating the products...")

self.products=[]

for i in range(options['count'][0]):

price = random_with_N_digits(3)

category=random.choice(self.categories)

name='{}{} product'.format(faker.name(),category.name)

subcategory=random.choice(list(filter(lambda x : x.category.name==category.name, self.subcategories)))

seller=random.choice(self.userprofiles)

product_type=random.choice(['downloadable_product','shipped_product'])

product=mommy.prepare(Product,name=name,price=price,category=category,subcategory=subcategory,seller=seller,product_type=product_type)

self.products.append(product)

Product.objects.bulk_create(self.products)

def create_category(self):

"""function to create the categories"""

print( "categories...")

_list=['books','toys','kitchenware','computer and accessories','mobile and tablets','clothing and Accessories','watches and eyeware']

self.categories=[]

for param in _list:

category=mommy.make(Category,name=param)

self.categories.append(category)

def create_subcategpries(self):

"""function to create the sub categories"""

print( "subcategories...")

self.subcategories=[]

for category in self.categories:

sub_cat_name='{} {}'.format(faker.name(),category.name)

subcategory=mommy.make(Subcategory, name=sub_cat_name, category=category)

self.subcategories.append(subcategory)

def create_users(self):

"""function to create the users"""

print( "users...")

self.userprofiles=[]

domains=['gmail','outlook','yahoo','yahoo.co.in']

for r in range(0,100):

email = '{}{}@{}'.format(faker.name(),str(r),random.choice(domains))

password=random_with_N_digits(10)

user=User.objects.create_user(email,email,password)

userprofile=mommy.make(UserProfile,user=user,user_type='vendor')

self.userprofiles.append(userprofile)

As you see in the above code its is a simple python code to generate the dummy data. I used django-extensions for making the sample command like this one seed_db. model_mommy to prepare and create the objects in the database and faker to generate fake data, like names, email etc. To populate db, save this command file and run the command:-

python manage.py seed_db 100

1	python manage.py seed_db 100

100 is the argument as I want to make 100 dummy products. you can […]

Integration of Elastic Search and Django(A Python Framework) – Part One

March 13, 2017March 13, 2017 tushar

In this tutorial I am going to show you the Integration of Elastic Search and Django. I am using Django 1.10, ElasticSearch 5.2.0, and Ubuntu 16.04 OS for the application. Lets Have a look on the model we are going to build. A simple application that have five simple models : – Product Model, Category Model, SubCategory Model, UserProfile, and the User Model. This is the part One of the tutorial, in this part I am going to build an application at Django Part. To install Django on your system refer here. Open terminal and type command

 django-admin.py startproject elasticdjango

1	django-admin.py startproject elasticdjango

make virtual environment

virtualenv -p python3 venv

1	virtualenv -p python3 venv

activate the virtual environment

source venv/bin/activate

1	source venv/bin/activate

install django 1.10 in the virtual environment using pip

pip install django==1.10

1	pip install django==1.10

make a app named core that will contain our models

django-admin startapp core

1	django-admin startapp core

Here is the directory structure for our application go to the elasticdjango/settings.py file and add the core application to the Installed apps:- add following code to the core/models.py file

from django.db import models
from django.contrib.auth.models import User
import django.db.models.options as options
options.DEFAULT_NAMES=options.DEFAULT_NAMES+('es_index_name','es_type_name','es_mapping')

# Create your models here.
class Product(models.Model):
    """Product Model"""
    CHOICES=(('downloadable_product','DOWNLOADABLE PRODUCT'),('shipped_product','SHIPPED PRODUCT'))
    name=models.CharField(max_length=150)
    price=models.PositiveIntegerField(default=0)
    category=models.ForeignKey('Category')
    subcategory=models.ForeignKey('Subcategory')
    seller=models.ForeignKey('UserProfile')
    product_type=models.CharField(choices=CHOICES,max_length=50,default='downloadable_field')
    def __unicode__(self):
        return self.name
    def es_repr(self):
        data={
            "name":self.name,
            "price":self.price,
            "category":{
                "name": self.category.name,
            },
            "subcategory":{
                "name": self.subcategory.name,
            },
            "seller":{
                "email": self.seller.user.email,
            },
            "product_type":self.product_type
        }
        return data

    class Meta:
        es_index_name='ecommerce'
        es_type_name='products'
        es_mapping={
            'properties':{
                'name':{'type':'text'},
                'price':{'type':"long"},
                'category':{
                        'properties':{
                            'name':{'type':'keyword'},
                        }
                    },
                'subcategory':{
                        'properties':{
                            'name':{'type':'keyword'},
                        }
                    },
                'seller':{
                    'properties':{
                        'email':{'type':'text'}
                        }
                    },
                'product_type':{'type':"keyword"},
                }
        }

class Category(models.Model):
    """Category Model"""
    name=models.CharField(max_length=150)
    def __unicode__(self):
        return self.name

class Subcategory(models.Model):
    """Category Model"""
    name=models.CharField(max_length=150)
    category=models.ForeignKey(Category)
    def __unicode__(self):
        return self.name

class UserProfile(models.Model):
    """User Profile model"""
    CHOICES=(('vendor','VENDOR'),('consumer','CONSUMER'))
    user_type=models.CharField(choices=CHOICES,default='consumer',max_length=150)
    user=models.ForeignKey(User)
    def __unicode__(self):
        return self.user_type

from django.db import models

from django.contrib.auth.models import User

import django.db.models.options as options

options.DEFAULT_NAMES=options.DEFAULT_NAMES+('es_index_name','es_type_name','es_mapping')

# Create your models here.

class Product(models.Model):

"""Product Model"""

CHOICES=(('downloadable_product','DOWNLOADABLE PRODUCT'),('shipped_product','SHIPPED PRODUCT'))

name=models.CharField(max_length=150)

price=models.PositiveIntegerField(default=0)

category=models.ForeignKey('Category')

subcategory=models.ForeignKey('Subcategory')

seller=models.ForeignKey('UserProfile')

product_type=models.CharField(choices=CHOICES,max_length=50,default='downloadable_field')

def __unicode__(self):

return self.name

def es_repr(self):

data={

"name":self.name,

"price":self.price,

"category":{

"name": self.category.name,

"subcategory":{

"name": self.subcategory.name,

"seller":{

"email": self.seller.user.email,

"product_type":self.product_type

}

return data

class Meta:

es_index_name='ecommerce'

es_type_name='products'

es_mapping={

'properties':{

'name':{'type':'text'},

'price':{'type':"long"},

'category':{

'properties':{

'name':{'type':'keyword'},

}

'subcategory':{

'properties':{

'name':{'type':'keyword'},

}

'seller':{

'properties':{

'email':{'type':'text'}

}

'product_type':{'type':"keyword"},

}

class Category(models.Model):

"""Category Model"""

name=models.CharField(max_length=150)

def __unicode__(self):

return self.name

class Subcategory(models.Model):

"""Category Model"""

name=models.CharField(max_length=150)

category=models.ForeignKey(Category)

def __unicode__(self):

return self.name

class UserProfile(models.Model):

"""User Profile model"""

CHOICES=(('vendor','VENDOR'),('consumer','CONSUMER'))

user_type=models.CharField(choices=CHOICES,default='consumer',max_length=150)

user=models.ForeignKey(User)

def __unicode__(self):

return self.user_type

Don’t worry if you are not able to understand this time the whole code. This in simple words contain the models definitions […]

Devtech

Talk about technologies

Author: tushar

Multi Tenant Application using Django – Connect and manage multiple databases to the single application

Integration of Elastic Search and Django(A Python Framework) – Part three

Integration of Elastic Search and Django(A Python Framework) – Part Two

Integration of Elastic Search and Django(A Python Framework) – Part One