Open in app

Sign up

Sign in

Write

Sign up

Sign in

How to Build and Containerize a Simple Live Notification System

Yiqing Lan
Dev Genius
Published in
9 min readAug 9, 2022

--

In the previous post, we’ve gone through the planning process by using stories mapping. In this article, I am going to walk you through the actual development (execution) process. Please note that you will need some basic knowledge of Django to understand the discussion. Let’s get started.

Dependencies

  • Python 3.8
  • Django 3.2
  • Channels 3.0.4
  • See other Python dependencies in the requirements.txt file
  • Docker

Step 1. The base project

We are going to start from a base project. It’s attached in the tag below. Please download and unzip the file mysite.zip.

https://github.com/slow999/DjangoAndLiveNotification/releases/tag/0.0.0

After setting up the virtual env and running the server, we should see this page on http://127.0.0.1:8000/.

The page of base project

Step 2. Set up channels

In the requirements.txt file, add channels and channels-redis. It should look like the below.

channels==3.0.4
channels-redis==3.4.0

Create an asgi.py file in the mysite folder. The content is as below.

"""
ASGI config for mysite project.It exposes the ASGI callable as a module-level variable named ``application``.For more information on this file, see
https://docs.djangoproject.com/en/3.2/howto/deployment/asgi/
"""import os
from channels.auth import AuthMiddlewareStack
from channels.routing import ProtocolTypeRouter, URLRouter
from django.core.asgi import get_asgi_applicationos.environ.setdefault('DJANGO_SETTINGS_MODULE', 'mysite.settings')application = ProtocolTypeRouter({
    'http': get_asgi_application()
})

In the settings.py file, add channels library into INSTALLED_APPS. It’s as below.

INSTALLED_APPS = [
    …    'channels',
]

In the settings.py file, set up the ASGI application and channel layer. It’s as below.

ASGI_APPLICATION = 'mysite.asgi.application'
CHANNEL_LAYERS = {
    'default': {
        'BACKEND': 'channels_redis.core.RedisChannelLayer',
        'CONFIG': {
            'hosts': [{'address': ('redis', 6379), 'password': 'mypassword'}],
        }
    }
}

Note that we use Redis as the channel layer. Since we will containerize the build, the service is called ‘redis’ as well. If you would use Redis service outside of the project, the service name is an URL. For example, ‘https://myredisservice.domain.com’. On top of that, we implement basic authentication for the Redis service.

Step 3 Build in-application publisher

This step involves building application views and templates of the publisher.

Create a new app sms by using the command below.

python manage.py startapp sms

In the settings.py file, add sms app to the INSTALLED_APPS. It looks like the below.

INSTALLED_APPS = [
    …    'channels', # we add in the previous step
    'sms.apps.SmsConfig'
]

Create a forms.py file in the sms app. The content is as follows.

from django import formsclass MessageForm(forms.Form):
    content = forms.CharField(widget=forms.Textarea, label='Your content')

In the views.py file in the sms app, add the followings.

import time
import datetime
from django.shortcuts import render, HttpResponse
from channels.layers import get_channel_layer
from asgiref.sync import async_to_sync
from sms.forms import MessageFormdef send_msg(request):
    if request.method == 'GET':
        return render(request, 'sms/send_msg.html', {'form': MessageForm()})
    elif request.method == 'POST':
        form = MessageForm(request.POST)
        if form.is_valid():
            print('Sending sms...')
            channel_layer = get_channel_layer()
            async_to_sync(channel_layer.group_send)('new_message', {'type': 'notify_client', 'message': f'email is sent.{datetime.datetime.now()}'})
            print('Message is sent.')
        else:
            print('Something is wrong')
            print(form.errors)
        return render(request, 'sms/send_msg.html', {'form': MessageForm()})
    else:
        return HttpResponse('Wrong method.')

Note that message will be sent to a subscriber group called new_message. The type of action that proceeds this message is called notify_client. We will develop it in the subscriber section later.

Create a templates folder in the sms app. In the templates folder, create a sms folder. In the sms folder that we just created, create a send_msg.html file. The content is as follows.

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>Title</title>
</head>
    <body>
        <form action="." method="post">
            {% csrf_token %}
            {{ form }}
            <br>
            <input type="submit" value="Submit">
        </form>
    </body>
</html>

Create an urls.py file in the sms app. Add a new URL as follows.

# chat/urls.py
from django.urls import pathfrom . import viewsurlpatterns = [
    path('send-msg/', views.send_msg, name='send_msg'),
]

In the mysite/urls.py file, connect URLs of sms as follows.

urlpatterns = [
    …    path('sms/', include('sms.urls')),
]

Let’s run the server and check out. We should see a basic form below at http://127.0.0.1:8000/sms/send-msg/.

The user interface of in-application publisher

Step 4. Build out-application publisher

This is going to be a standalone Python script. We can run it anywhere. In order to provide the materials in this article in one repository, I would like to add them to this project.

Create a folder outsider in the root directory.

In the outsider folder. Create a publisher.py file. The content is as follows.

import datetime
from channels.layers import get_channel_layer
from asgiref.sync import async_to_syncchannel_layer = get_channel_layer()
async_to_sync(channel_layer.group_send)\
    ('new_message', {'type': 'notify_client', 'message': f'email is sent.{datetime.datetime.now()}'})

Create a project folder in the outsider folder. Within project folder, create a __init__.py file and settings.py file. The content of settings.py file is as follows.

CHANNEL_LAYERS = {
    'default': {
        'BACKEND': 'channels_redis.core.RedisChannelLayer',
        'CONFIG': {
            'hosts': [{'address': ('127.0.0.1', 6666), 'password': 'mypassword'}],
        }
    }
}

The intention of settings.py is to simulate the runtime environment of Django. Because it seems that channels would assume that it runs under the Django environment and look up the CHANNEL_LAYERS variable to connect Redis service. On top of that, note that we use ‘127.0.0.1’ rather than ‘redis’. Because the publisher runs outside of this project. It needs to connect Redis through the host’s URL.

The file structure should look like the below.

The file structure of out-application publisher

Step 5. Build a notification interface

In the sms/views.py file, create a view function called notification. It is as follows.

def notification(request):
    return render(request, 'sms/notification.html', {'unread_count': 9})

Note that our notification starts from 9 unreads.

In the sms/urls.py file, create a notification URL in the urlpatterns. It’s as follows.

urlpatterns = [
    …
    path('new-msg/', views.notification, name='notification'),
]

In the sms/templates/sms/ folder, create a notification.html file. The content is as follows.

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>Message</title>
    <link href="https://cdn.jsdelivr.net/npm/bootstrap@5.1.3/dist/css/bootstrap.min.css" rel="stylesheet" integrity="sha384-1BmE4kWBq78iYhFldvKuhfTAU6auU8tT94WrHftjDbrCEXSU1oBoqyl2QvZ6jIW3" crossorigin="anonymous"
</head>
<body>
    <div class="container" style="margin-top:50px;margin-left:50px;">
        <button type="button" class="btn btn-primary position-relative">
          Inbox
          <span id="unread-content" class="position-absolute top-0 start-100 translate-middle badge rounded-pill bg-danger">
            {{ unread_count }}
          </span>
        </button>
    </div>
</body>
</html>

Let's run the server and check it out at this point. We should see a notification “Inbox” box that starts from 9 unreads at http://127.0.0.1:8000/sms/new-msg/. It’s as below.

The user interface of notification

Step 6. Build an async consumer (subscriber)

In the sms app, create a consumers.py file. The content is as follows.

import json
import asyncio
from channels.generic.websocket import AsyncWebsocketConsumerclass NotificationConsumer(AsyncWebsocketConsumer):
    async def connect(self):
        self.group_name = 'new_message'await self.channel_layer.group_add(
            self.group_name,
            self.channel_name
        )await self.accept()async def disconnect(self, close_code):
        await self.channel_layer.group_discard(
            self.group_name,
            self.channel_name
        )async def receive(self, text_data):
        passasync def notify_client(self, event):
        print(f'Notification consumer ins id {id(self)}')
        print(event)
        message = event['message']
        await self.send(text_data=json.dumps({
            'message': message
        }))

In the sms app, create a routing.py file. The content is as follows.

from django.urls import re_path
from . import consumerswebsocket_urlpatterns = [
    re_path(r'ws/sms/notification/$', consumers.NotificationConsumer.as_asgi()),
]

In the mysite/asgi.py file, add websocket setup and import routing file in the application variable. It’s as the below.

import sms.routingapplication = ProtocolTypeRouter({
    'http': get_asgi_application(), # we added previously
    'websocket': AuthMiddlewareStack(
        URLRouter(
            sms.routing.websocket_urlpatterns
        )
    )
})

Step 7. Build client code that updates the number

In the sms/templates/sms/notification.html file, add some codes in the <body> element as follows.

<body>
…{{ unread_count|json_script:"unread-count" }}
    <script>
        var unread_count = JSON.parse(document.getElementById('unread-count').textContent);const notifySocket = new WebSocket(
            'ws://'
            + window.location.host
            + '/ws/sms/notification'
            + '/'
        );notifySocket.onmessage = function(e) {
            const data = JSON.parse(e.data);
            console.log(data);
            unread_count += 1;
            document.querySelector('#unread-content').textContent = unread_count;
        };notifySocket.onclose = function(e) {
            console.log('Web socket is closed.')
        };
    </script></body>

At this point, the publisher and the subscriber are done. However, we won’t be able to review it until we complete the production practice section. Take a short break. Let’s move forward to the next step.

Step 8. Containerize application

In the root directory, add a gunicorn.app.conf.py file. Its content is as follows.

wsgi_app = 'mysite.wsgi:application'
bind = 'unix:/tmp/gunicorn/gunicorn-app.sock'
worker_class = 'sync' # default
worker = 4
accesslog = '-'

In the root directory, add a gunicorn.channel.conf.py file. Its content is as follows.

wsgi_app = 'mysite.asgi:application'
worker_class = 'uvicorn.workers.UvicornWorker'
workers = 2
bind = 'unix:/tmp/gunicorn/gunicorn-channel.sock'
accesslog = '-'

In the root directory, add a Dockerfile file. Its content is as follows.

FROM python:3.8-alpineRUN apk add python3-dev build-base linux-headers pcre-dev libffi-devRUN mkdir /codes/
WORKDIR codesCOPY requirements.txt requirements.txt
RUN pip install gunicorn uvicorn[standard]
RUN pip install -r requirements.txtCOPY mysite mysite
COPY sms sms
COPY Dockerfile Dockerfile
COPY manage.py manage.py
COPY db.sqlite3 db.sqlite3
COPY gunicorn.app.conf.py gunicorn.app.conf.py
COPY gunicorn.channel.conf.py gunicorn.channel.conf.pyRUN python3 manage.py collectstaticEXPOSE 8000

Step 9. Containerize Redis

In the root directory, add a docker-compose.yml file. The content is as follows.

version: "3.9"
services:
  redis:
    image: redis
    command: redis-server --requirepass mypassword
    ports:
      - 6666:6379
    restart: unless-stopped

Note the password and port that we mentioned in the previous step.

Step 10. Containerize Nginx

In the root directory, create a nginx folder. Within the folder, create Dockerfile, mysite.nginx.conf, and nginx.conf file. Their contents are as follows.

Dockerfile:

FROM nginx:latestCOPY nginx.conf /etc/nginx/nginx.conf
COPY mysite.nginx.conf /etc/nginx/sites-available/mysite.nginx.conf
RUN mkdir /etc/nginx/sites-enabled
RUN ln -s /etc/nginx/sites-available/mysite.nginx.conf /etc/nginx/sites-enabled/CMD ["nginx", "-g", "daemon off;"]

mysite.nginx.conf:

upstream app-backend {
    # fail_timeout=0 means we always retry an upstream even if it failed
    # to return a good HTTP response    # for UNIX domain socket setups
    server unix:/tmp/gunicorn/gunicorn-app.sock fail_timeout=0;# for a TCP configuration
    # server 192.168.0.7:8000 fail_timeout=0;
}upstream channel-backend {server unix:/tmp/gunicorn/gunicorn-channel.sock fail_timeout=0;# for a TCP configuration
    # server web_channel:8001 fail_timeout=0;
}server {
    listen      80;
    server_name 127.0.0.1;
    charset     utf-8;    location /static {
        alias /var/www/mysite/static/;
    }    location / {
        proxy_pass http://app-backend;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header Host $http_host;
        # we don't want nginx trying to do something clever with
        # redirects, we set the Host: header above already.
        proxy_redirect off;
        proxy_buffering off;    }    location /ws/ {
        proxy_pass http://channel-backend;        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";        proxy_redirect off;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Host $server_name;
    }
}

nginx.conf:

user  root;
worker_processes  auto;error_log  /var/log/nginx/error.log notice;
pid        /var/run/nginx.pid;events {
    worker_connections  1024;
}http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';    access_log  /var/log/nginx/access.log  main;    sendfile        on;
    #tcp_nopush     on;    keepalive_timeout  65;    #gzip  on;    #include /etc/nginx/conf.d/*.conf;
    include /etc/nginx/sites-enabled/*;
}

Step 11. Productionize the build

Let’s put everything together into the docker-compose.yml file. The final result should be as follows.

version: "3.9"
services:
  web_channel:
    build: .
    command: gunicorn --config=gunicorn.channel.conf.py
    volumes:
      - socket_data:/tmp/gunicorn/
    restart: unless-stopped  web_app:
    build: .
    command: gunicorn --config=gunicorn.app.conf.py
    volumes:
      - static_data:/var/www/mysite/static/
      - socket_data:/tmp/gunicorn/
    restart: unless-stopped  nginx:
     depends_on:
       - web_app
       - web_channel
     build: nginx/
     ports:
       - 8888:80
     volumes:
       - static_data:/var/www/mysite/static/:ro
       - socket_data:/tmp/gunicorn/
     restart: unless-stopped  redis:
    image: redis
    command: redis-server --requirepass mypassword
    ports:
      - 6666:6379
    restart: unless-stoppedvolumes:
  static_data:
  socket_data:

Note that we are going to visit the site by using port 8888.

At this point, we have completed all tasks. Let’s compose up the build and check out. To start the project, run the following commands.

docker compose build
docker compose up

If everything is all right, we should see the prints in the console as below.

The print of console

Testing

In-application testing

Open http://127.0.0.1:8888/sms/send-msg/ and http://127.0.0.1:8888/sms/new-msg/ in the browser. Type anything in the form and click submit button. The notification number should increase from 9 to 10.

Pages of in-application publisher and subscriber

Out-application testing

Let the containers run and leave http://127.0.0.1:8888/sms/new-msg/ open.

Open a new terminal. Activate the virtual environment. Go to the outsider folder. Set environment variable DJANGO_SETTINGS_MODULE by using one of the commands as follows.

# Windows
$env:DJANGO_SETTINGS_MODULE="project.settings"# Linux or Mac
export DJANGO_SETTINGS_MODULE=project.settings

Run the publisher by the following command. We should see the notification number increase.

python publisher.py

Recap

In this article, we went through all the steps that put together a simple live notification system. Looking back at the previous article, you would notice that we developed it exactly in the order of user story maps. If you are interested in the codebase, I’ve uploaded it to the Github repository as follows. Please feel free to leave your comments. Thanks for reading. Stay tuned.

https://github.com/slow999/DjangoAndLiveNotification

Django
Notifications
Python
Redis
Docker

--

--

Yiqing Lan
Dev Genius
Follow

Written by Yiqing Lan

82 Followers

吾生也有涯，而知也无涯

Follow

Help

Status

About

Careers

Press

Blog

Privacy

Terms

Text to speech

Teams