nohanaga/search-ranking-tutorial

Python learning to rank tutorial using Azure Cognitive Search

page_type	languages	products	description	urlFragment
sample	python	Azure Cognitive Search	This is a learning to rank tutorial in Python that showcases reranking on top of Azure Cognitive Search	search-ranking-tutorial

Add machine learning to search relevance - Azure Cognitive Search

This tutorial demonstrates the adoption of Learning To Rank to improve search relevance in search applications backed by Azure Cognitive Search. This tutorial highlights how to use the new featuresMode parameter to train a ranking model.

This tutorial is for developers who are looking to improve relevance in their Azure Cognitive Search applications. Azure Cognitive Search provides different ways to control search relevance including scoring profiles and query term boosting. These techniques work well in scenarios where indexed content and user query patterns are relatively static and well understood. In applications where this is not true, machine learning based techniques can be used to tune relevance dynamically.

Why machine learning for ranking?

Machine learned ranking models are highly effective, especially in applications that handle a lot of data and user traffic, such as Bing, Google, Facebook, Twitter, and Netflix. Ranking models are suitable for applications where a notion of what's relevant can be defined and observed. Machine learning based approaches to tune search relevance allow ever-changing information about user behavior and preferences to be injected into the search experience.

Training and serving a ranking model involves lots of "gotchas". This tutorial describes a simple pattern for doing this with Azure Cognitive Search as the retrieval engine where reranking happens on the application side.

Getting Started

If you just want to read the code, skip the "Setup" section.

• Setup
• Part 1: Data Engineering
• Part 2: Training and Testing a Ranking Model
• Conclusion

Setup

Prerequisites

• An existing Azure Cognitive Search service
• Anaconda with Jupyter Notebooks and Python 3.7

Optional

• Prior background in machine learning is helpful. For a hands-on, introductory tutorial, check out Machine learning crash course. Andrew Ng's Machine Learning course is also a great option if you have more time.

Installation

1. Download and install the latest version of Anaconda or Miniconda.
2. Clone this repository to your local machine.
   • On Windows, make sure to open this repo with an Anaconda command prompt.
   • On Linux or OSX, if you didn't add Anaconda to your system PATH variable, you'll have to source the Anaconda environment manually.
3. Install the conda environment with conda env create -f environment.yml.
4. Activate the environment with conda activate azs-l2r.
5. Run Jupyter with your choice of jupyter notebook or jupyter lab. Navigate to the tutorial at l2r_part1_data_eng.ipynb and l2r_part2_experiment.ipynb.

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.

When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.