/es-tmdb

Elasticsearch TMDB examples

Primary LanguagePython

Elasticsearch Index for the The Movie Database.

This repository is part of the Think Like a Relevancy Engineer training provided by OpenSource Connections.

Steps to get up and running:

  • Download this repo
  • Install the software (using either Docker or installing manually)
  • Index the TMDB movie data
  • Confirm Elasticsearch has the data
  • Install Postman (optional)

Download this repo

Download the zip from https://github.com/o19s/es-tmdb/archive/master.zip, and you will get the file es-tmdb-master.zip. Unzip this file, resulting in the directory es-tmdb-master.

After you have this download, change into the newly created directory.

Install Elasticsearch w/ Dependencies

Docker option (recommended)

If you have Docker installed and running.

Make sure you have at least 4gb of memory available for Docker, the default is 2gb. See Docker's Preferences >>> Resources-tab, to adjust.

Linux/Windows/Mac:

docker-compose up

Give it a minute to fully boot up then use a browser to go to http://localhost:9200 and http://localhost:5601 to confirm Elasticsearch and Kibana are running.

Local option

Elasticsearch

  1. Download Elasticsearch 8.9.2
  2. Unzip to where you'd like to run Elasticsearch
  3. Add the following to config/elasticsearch.yml
http.cors.allow-origin: "/https?:\\/\\/(.*?\\.)?(quepid\\.com|splainer\\.io)/"
http.cors.enabled: true
indices.query.bool.max_clause_count: 10240
  1. Run Elasticsearch
bin/elasticsearch

  1. In your browser, navigate to http://localhost:9200 to confirm Elasticsearch is running

  2. Install the plugins for LTR and Querqy

bin/elasticsearch-plugin install -b "https://github.com/o19s/elasticsearch-learning-to-rank/releases/download/v1.5.8-es8.9.2/ltr-plugin-v1.5.8-es8.9.2.zip"

bin/elasticsearch-plugin install -b "https://repo1.maven.org/maven2/org/querqy/querqy-elasticsearch/1.7.es892.0/querqy-elasticsearch-1.7.es892.0.zip"

Kibana

  1. Download Kibana 8.9.2

  2. Unzip to where you'd like to run Kibana

  3. Run Kibana

bin/kibana
  1. In your browser, navigate to http://localhost:5601 to confirm Kibana is running with the Analyze plugin included

Index TMDB movies

Once Elasticsearch and Kibana are ready go, we need to create our example search index.

Linux/Mac:

./index.sh

Windows:

powershell index.ps1

If you don't have admin permissions to run PowerShell scripts, then open up the PowerShell ISE application as an administrator. Then switch to the downloaded code directory and cut and paste the contents of index.ps1 into the console to work around this issue.

Confirm Elasticsearch has TMDB movies

Run a wildcard search and confirm you get results.

Postman

Postman helps manage API requests. The examples from the TLRE slides exist here too as a Postman Collection (es-postman-collection.json). We like using Postman because it makes tinkering with query parameters nicer.

We will mainly Kibana's DevTools for tinkering, but if you already familiar with Postman you can use that and get similar milage.

If you want to use Postman during the TLRE class:

  1. Download Postman for your OS
  2. Open Postman and Import (top-menu >> File) es-postman-collection.json
  3. Define a global variable (grey eye icon in the upper-right) es_host to point to your running Elasticsearch instance (default is localhost:9200)
  4. Tinker with the base URL, Params or JSON Body (optional)
  5. Press 'Send' (blue rectangle button right of URL bar) to Search!