An AI Search Engine inspired by Search with Lepton. Replaced the backend with SpringBoot + LangChain4J, and made some adjustments to the information processing flow.
Thanks to the great work of Lepton for demystifying RAG search, enabling open-source version of Perplexity
Compared with Search With Lepton, we provide two information processing flows.
- Search keywords with search engine APIs and obtain search results.
- Aggregate the snippets of the search results as context and integrate it into the prompt.
- Submit the prompt to the LLM and obtain the results.
- Search keywords with search engine APIs and obtain search results.
- Store the webpage content of the top two search results and the snippets of the remaining search results in a vector database.
- Retrieve the most similar texts from the vector database and integrate them into the prompt as contextual information.
- Submit the prompt to the LLM and obtain the results.
- How to convert users' search keywords or even language to obtain higher-quality search results
- How to exclude the interference of other irrelevant content in HTML and extract the most accurate content of the main text
- How to retrieve more relevant and accurate content from the vector database
- How to allow the LLM to handle more contextual information
We believe open-source is not just about open source code, but also about open documentation. We are planning a series of documentations to explain the principles of machine learning and RAG development used in this project,
Below is the first article:
If you are interested in this, please follow us on Twitter to receive the latest information.
https://twitter.com/vlinx_soft
- OpenAI API
- Google Search API
- Java 21
Fill in the corresponding information in application-dev.properties
./gradlew build -x testjava -jar build/libs/infinite-search-$version.jarOpen browser and visit http://localhost:8605