Multimodal AI-Powered Search

A modern search interface built with Next.js and Meilisearch, showcasing multimodal AI search capabilities. This demo allows users to search products using both text queries and image uploads, powered by Meilisearch's hybrid search with semantic embeddings.

✨ Features

🔍 Hybrid Search: Combines full-text and semantic search with adjustable AI ratio
🖼️ Image-to-Image Search: Upload images to find visually similar products
🔤 Text-to-Image Search: Describe what you’re looking for and instantly retrieve matching images
📊 Ranking Scores: Visible relevancy scores for transparency

🚀 Getting Started

Prerequisites

Node.js 20+
A running Meilisearch instance
Products indexed with multimodal embeddings (e.g., using Voyage embedder)

Installation

Clone the repository:

git clone <repository-url>
cd demo-multimodal

Install dependencies:

npm install

Configure environment variables:

# .env.local
NEXT_PUBLIC_MEILISEARCH_URL=meilisearch_url
NEXT_PUBLIC_MEILISEARCH_READ_API_KEY=your_api_key
NEXT_PUBLIC_DEFAULT_INDEX=products_multimodal

Run the development server:

npm run dev

Open http://localhost:3000 in your browser.

🔧 Configuration

Search Configuration

The search interface supports extensive configuration through constants in search-interface.tsx:

// Display configuration - controls how results are rendered
const DISPLAY_CONFIGS = {
  products_multimodal: {
    primaryText: 'name',
    secondaryText: 'brand',
    imageUrl: 'image_url',
    additionalFields: [...]
  }
}

// Facet configuration - controls which filters are shown
const FACET_CONFIGS = {
  products_multimodal: {
    visibleFacets: ['color.original_name', 'category_page_id', 'brand'],
    facetDisplayNames: {...},
    facetOrder: [...],
    rangeFilters: {}
  }
}

// Sort configuration
const SORT_CONFIGS = {
  products_multimodal: {
    visibleSorts: ['price.value:asc', 'reviews.bayesian_avg:desc', ...],
    sortDisplayNames: {...},
    defaultSort: 'relevance'
  }
}

Parameters

Key parameters can be adjusted:

RESULTS_LIMIT: Number of results per page (default: 12)
SEARCH_DEBOUNCE_MS: Debounce delay for queries (default: 150ms)
RANKING_SCORE_THRESHOLD: Minimum score for results (default: 0.65)

🎨 Tech Stack

Framework: Next.js 15 with App Router
Language: TypeScript 5
Search Engine: Meilisearch
Styling: Tailwind CSS 4
UI Components: shadcn/ui + Radix UI
Icons: Lucide React
Multi-modal AI model: VoyageAI voyage-multimodal-3

🛠️ Development

Build for Production

npm run build
npm start

Linting

npm run lint

🔗 Resources

🤖 Meilisearch embedder configuration

{
    "default": {
        "source": "rest",
        "apiKey": "pa-iVXXXXXX...",
        "dimensions": 1024,
        "binaryQuantized": false,
        "url": "https://api.voyageai.com/v1/multimodalembeddings",
        "indexingFragments": {
            "picture": {
                "value": {
                    "content": [
                        {
                            "type": "image_url",
                            "image_url": "{{doc.image_url}}"
                        }
                    ]
                }
            },
            "text": {
                "value": {
                    "content": [
                        {
                            "type": "text",
                            "text": "A {% if doc.color and doc.color.original_name %} {{ doc.color.original_name }}{% endif %} {%- if doc.product_type %} {{ doc.product_type }}{% else %} product{% endif %} {% if doc.brand %}by {{ doc.brand }}{% endif %} {% if doc.name %}called {{ doc.name }}{% endif %} {% if doc.gender %}for {{ doc.gender }}{% endif %}. {%- if doc.description and doc.description != \"\" %}. {{ doc.description }}{% endif %}."
                        }
                    ]
                }
            }
        },
        "searchFragments": {
            "image": {
                "value": {
                    "content": [
                        {
                            "type": "image_base64",
                            "image_base64": "data:{{media.image.mime}};base64,{{media.image.data}}"
                        }
                    ]
                }
            },
            "text": {
                "value": {
                    "content": [
                        {
                            "type": "text",
                            "text": "{{q}}"
                        }
                    ]
                }
            }
        },
        "request": {
            "inputs": [
                "{{fragment}}",
                "{{..}}"
            ],
            "model": "voyage-multimodal-3"
        },
        "response": {
            "data": [
                {
                    "embedding": "{{embedding}}"
                },
                "{{..}}"
            ]
        },
        "headers": {}
    }
}

🤝 Contributing

Contributions, issues, and feature requests are welcome!

meilisearch/demo-multimodal