codefuse-ai/ModelCache

[Feature: Ranking ability] Add ranking model to refine the order of data after embedding recall

Closed this issue · 1 comments

This issue is created to better track my PRs for Todo List [Rank ability]

Background

Efficiently retrieving relevant results from large-scale datasets plays a crucial role in software development and Q&A tasks, including code recommendation and vulnerability code clone discovery. Existing search approaches suffer the difficulty of being scaled up, while maintaining the effectiveness of retrieved results.

Prototype

In this paper, we propose a two-stage search framework to address these limitations and achieve both effectiveness and efficiency in the search Scenario. We will add ranking model to refine the order of data after embedding recall.

Screenshot 2024-05-28 at 10 37 53

Task

  • Add ranking model to refine the order of data after embedding recall.

Due to certain reasons, we have closed this issue, and the ranking model will be implemented in a future iteration.