Redis is and advanced key-value database that allows you to save different types of data and is extremely fast on I/O operations. Redis stores everything in memory, but the data can be persisted by dumping the dataset to disk every once in a while or by adding each command to a log. Redis is very versatile compared to other key-value stores: It provides a set of powerful commands and supports diverse data structures such as strings, hashes, lists, sets, ordered sets, and even bitmaps or HyperLogLogs.
While SQL is best suitable for schema-defined persistent data storage, Redis offers numerous advantages when dealing with rapidly changing data, volatile storage, or when a quick cache is needed. Let's see how Redis can be used for building new functionality into our project.
Download the latest Redis version from http://redis.io/download. Unzip the tar.gz
file, enter the redis
directory and compile Redis using the make
command as follows:
cd redis-3.0.4 make
After installing it use the following shell command to initialize the Redis server:
src/redis-server
You should see an output that ends with the following lines:
# Server started, Redis version 3.0.4 * DB loaded from disk: 0.001 seconds * The server is now ready to accept connections on port 6379
By default, Redis runs on port 6379
, but you can also specify a custom port using the --port
flag, for example redis-server --port 6655
. When your server is ready, you can open the Redis client in another shell using the following command:
src/redis-cli
You should see the Redis client shell like the following:
127.0.0.1:6379>
The Redis client allows you to execute Redis commands directly from the shell. Let's try some commands. Enter the SET
command in the Redis shell to store a value in a key:
127.0.0.1:6379> SET name "Peter" OK
The previous command creates a name
key with the string value "Peter"
in the Redis database. The OK
output indicates that the key has been saved successfully. Then, retrieve the value using the GET
command as follows:
127.0.0.1:6379> GET name "Peter"
You can also check if a key exists by using the EXISTS
command. This command returns 1
if the given key exists, 0
otherwise:
127.0.0.1:6379> EXISTS name (integer) 1
You can set the time for a key to expire using the EXPIRE
command, which allows you to set time to live in seconds. Another option is using the EXPIREAT
command that expects a Unix timestamp. Key expiration is useful to use Redis as a cache or to store volatile data:
127.0.0.1:6379> GET name "Peter" 127.0.0.1:6379> EXPIRE name 2 (integer) 1
Wait for 2 seconds and try to get the same key again:
127.0.0.1:6379> GET name (nil)
The (nil)
response is a null response and means no key has been found. You can also delete any key using the
DEL
command as follows:
127.0.0.1:6379> SET total 1 OK 127.0.0.1:6379> DEL total (integer) 1 127.0.0.1:6379> GET total (nil)
These are just basic commands for key operations. Redis includes a large set of commands for other data types such as strings, hashes, sets, ordered sets, and so on. You can take a look at all Redis commands at http://redis.io/commands and all Redis data types at http://redis.io/topics/data-types.
We need Python bindings for Redis. Install redis-py
via pip
using the command:
pip install redis==2.10.3
You can find the redis-py docs at http://redis-py.readthedocs.org/.
The redis-py
offers two classes for interacting with Redis: StrictRedis
and Redis
. Both offer the same functionality. The StrictRedis
class attempts to adhere to the official Redis command syntax. The Redis
class extends StrictRedis
overriding some methods to provide backwards compatibility. We are going to use the StrictRedis
class since it follows the Redis command syntax. Open the Python shell and execute the following code:
>>> import redis >>> r = redis.StrictRedis(host='localhost', port=6379, db=0)
This code creates a connection with the Redis database. In Redis, databases are identified by an integer index instead of a database name. By default, a client is connected to database 0
. The number of available Redis databases is set to 16, but you can change this in the redis.conf
file.
Now set a key using the Python shell:
>>> r.set('foo', 'bar') True
The command returns True
indicating that the key has been successfully created. Now you can retrieve the key using the get()
command:
>>> r.get('foo') 'bar'
As you can see, the methods of StrictRedis
follow the Redis command syntax.
Let's integrate Redis into our project. Edit the settings.py
file of the bookmarks project and add the following settings to it:
REDIS_HOST = 'localhost' REDIS_PORT = 6379 REDIS_DB = 0
These are the settings for the Redis server and the database that we will use for our project.
Let's store the total number of times an image has been viewed. If we did this using the Django ORM, it would involve an SQL UPDATE
statement every time an image is displayed. Using Redis instead, we just need to increment a counter stored in memory, resulting in much better performance.
Edit the views.py
file of the images
application and add the following code to it:
import redis from django.conf import settings # connect to redis r = redis.StrictRedis(host=settings.REDIS_HOST, port=settings.REDIS_PORT, db=settings.REDIS_DB)
Here we establish the Redis connection in order to use it in our views. Edit the image_detail
view and make it look as follows:
def image_detail(request, id, slug): image = get_object_or_404(Image, id=id, slug=slug) # increment total image views by 1 total_views = r.incr('image:{}:views'.format(image.id)) return render(request, 'images/image/detail.html', {'section': 'images', 'image': image, 'total_views': total_views})
In this view, we use the INCR
command that increments the value of a key by 1
and sets the value to 0
before performing the operation if the key does not exist. The incr()
method returns the value of the key after performing the operation and we store it in the total_views
variable. We build the Redis key using a notation like object-type:id:field
(for example image:33:id
).
Edit the image/detail.html
template and add the following code to it after the existing <span class="count">
element:
<span class="count"> <span class="total">{{ total_views }}</span> view{{ total_views|pluralize }} </span>
Now open an image detail page in your browser and load it several times. You will see that each time the view is executed the total views displayed are incremented by 1
. See the following example:
You have successfully integrated Redis into your project to store item counts.
Let's built some more functionality with Redis. We are going to create a ranking of the most viewed images in our platform. For building this ranking we will use Redis sorted sets. A sorted set is a non-repeating collection of strings in which every member is associated with a score. Items are sorted by their score.
Edit the views.py
file of the images
application and make the image_detail
view look as follows:
def image_detail(request, id, slug): image = get_object_or_404(Image, id=id, slug=slug) # increment total image views by 1 total_views = r.incr('image:{}:views'.format(image.id)) # increment image ranking by 1 r.zincrby('image_ranking', image.id, 1) return render(request, 'images/image/detail.html', {'section': 'images', 'image': image, 'total_views': total_views})
We use the zincrby()
command to store image views in a sorted set with the key image:ranking
. We are storing the image id
, and a score of 1
that will be added to the total score of this element in the sorted set. This will allow us to keep track of all image views globally and have a sorted set ordered by the total number of views.
Now create a new view to display the ranking of the most viewed images. Add the following code to the views.py
file:
@login_required def image_ranking(request): # get image ranking dictionary image_ranking = r.zrange('image_ranking', 0, -1, desc=True)[:10] image_ranking_ids = [int(id) for id in image_ranking] # get most viewed images most_viewed = list(Image.objects.filter( id__in=image_ranking_ids)) most_viewed.sort(key=lambda x: image_ranking_ids.index(x.id)) return render(request, 'images/image/ranking.html', {'section': 'images', 'most_viewed': most_viewed})
This is the image_ranking
view. We use the zrange()
command to obtain the elements in the sorted set. This command expects a custom range by lowest and highest score. By using 0
as lowest and -1
as highest score we are telling Redis to return all elements in the sorted set. We also specify desc=True
to retrieve the elements ordered by descending score. Finally, we slice the results using [:10]
to get the first 10 elements with highest scores. We build a list of returned image IDs and we store it in the image_ranking_ids
variable as a list of integers. We retrieve the Image
objects for those IDs and force the query to be executed by using the list()
function. It is important to force the queryset execution because next we use the sort()
list method on it (at this point we need a list of objects instead of a queryset). We sort the Image
objects by their index of appearance in the image ranking. Now we can use the most_viewed
list in our template to display the 10 most viewed images
.
Create a new image/ranking.html
template file and add the following code to it:
{% extends "base.html" %} {% block title %}Images ranking{% endblock %} {% block content %} <h1>Images ranking</h1> <ol> {% for image in most_viewed %} <li> <a href="{{ image.get_absolute_url }}"> {{ image.title }} </a> </li> {% endfor %} </ol> {% endblock %}
The template is pretty straightforward, as we just iterate over the Image
objects contained in the most_viewed
list.
Finally create an URL pattern for the new view. Edit the urls.py
file of the images application and add the following pattern to it:
url(r'^ranking/$', views.image_ranking, name='create'),
Open http://127.0.0.1:8000/images/ranking/
in your browser. You should be able to see an image ranking as follows:
Redis is not a replacement for your SQL database but a fast in-memory storage that is more suitable for certain tasks. Add it to your stack and use it when you really feel it's needed. The following are some scenarios in which Redis suits pretty well:
incr()
and incrby()
for counting stuff.lpush()
and rpush()
. Remove and return first/last element using lpop()
/ rpop()
.You can trim the list length using ltrim()
to maintain its length.expire()
and expireat()
allows you to use Redis as a cache. You can also find third-party Redis cache backends for Django.