This project demonstrates the integration of Elasticsearch with Django and Django-Rest-Framework (DRF) to build a simple search API. Elasticsearch is a powerful search engine widely used by companies. This project is a basic implementation of Elasticsearch aims to simplify the process of integrating Elasticsearch with Django and provide a comprehensive guide for building search functionality.
The project consists of a Django app with two models:
- Category: Represents a category with a title.
- Article: Represents an article with a title and a category (related to the Category model).
- Allow users to search articles by title using Elasticsearch.
- Enable users to filter articles by category using Elasticsearch.
- Create an auto-complete API for articles.
Before diving into Django, make sure you have an Elasticsearch instance running locally. The preferred method is using Docker and Docker-Compose. The project includes a Dockerfile for Django and a docker-compose.yml file with two services: web and elasticsearch.
The project consists of a demo-data.json
with some dummy data to populate the database. Run the following command to load the dummy data in the database.
python manage.py load_data
docker-compose up -d
Visit localhost:8000
to ensure everything is working.
DocumentViewSet
automatically creates different APIs:
- List and retrieve records.
- Normal functionality (search, filter, etc.).
- Auto-complete functionality.
- Functional suggestions.
-
List:
127.0.0.1:8000/article-search
127.0.0.1:8000/article-search
-
Search:
127.0.0.1:8000/article-search?search=sports
127.0.0.1:8000/article-search?search=<term>
- If you want to search on multiple terms, you can do it using:
localhost:8000/article-search?search=<term1>&search=<term2>
-
Filter:
127.0.0.1:8000/article-search?category=2
127.0.0.1:8000/article-search?category=<id>
- If you want to filter by multiple categories:
127.0.0.1:8000/article-search?category=id1__id2__id3
-
Auto-Suggest:
127.0.0.1:8000/article-search/suggest?title__completion=how
127.0.0.1:8000/article-search/suggest?title__completion=<some_text>
In the documents.py
file, we define the structure of the Elasticsearch document (ArticleDocument
) that corresponds to our Django model (Article
). This file outlines how the data from Django should be stored in Elasticsearch.
-
Document Definition:
- We create a class named
ArticleDocument
that extendsDocument
. This class represents the schema of the document that will be stored in Elasticsearch.
- We create a class named
-
Field Mapping:
- For each field in our Django model that we want to index in Elasticsearch, such as
title
andcategory
, we specify how it should be stored in the Elasticsearch document. - For the
title
field:- It is defined as a text field in Elasticsearch.
- Sub-fields include
'raw'
for regular search and'suggest'
for auto-complete.
- For the
category
field:- As it is a relation to another model, it is defined as an object field in Elasticsearch.
- Sub-fields within the object include
'id'
(an integer) and'title'
(a text field with a raw keyword field for search).
- For each field in our Django model that we want to index in Elasticsearch, such as
-
Index Configuration:
- We specify that the Elasticsearch index for storing this document is named 'articles'. This naming convention helps organize data within Elasticsearch.
-
Django Model Association:
- We associate our Elasticsearch document (
ArticleDocument
) with our Django model (Article
). This linkage ensures that changes in the Django model are automatically reflected in the corresponding Elasticsearch document.
- We associate our Elasticsearch document (
In the signals.py
file, we implement Django signals to automate the synchronization of data between our Django models and Elasticsearch indexes. This ensures that changes to the models trigger updates in the corresponding Elasticsearch documents.
-
Post-Save and Post-Delete Signals:
- We utilize two key signals:
- post_save Signal:
- Triggered after a model instance is saved.
- We use this signal to update the Elasticsearch index when a new record is added or an existing record is modified.
- post_delete Signal:
- Triggered after a model instance is deleted.
- This signal is employed to update the Elasticsearch index when a record is removed.
- post_save Signal:
- We utilize two key signals:
-
Signal Implementation:
- The
update_document
anddelete_document
functions are connected to the respective signals. They perform the following actions:-
update_document:
- Retrieves the app label, model name, and instance from the signal.
- Checks if the signal pertains to the
articles
app and thearticle
model. -If true, iterates through related instances and updates the corresponding Elasticsearch documents.
-
delete_document:
- Similar to
update_document
but handles the deletion event. - Removes the Elasticsearch documents corresponding to the deleted model instances.
- Similar to
-
- The
-
Automation of Index Updates:
- By employing these signals, we automate the process of updating Elasticsearch indexes whenever changes occur in the associated Django models. This ensures consistency between the Django application's data and the Elasticsearch search index.
- In essence,
signals.py
enhances the integration between Django and Elasticsearch, providing a mechanism for real-time updates and synchronization of data between the two environments.