/OpenwebUI-Tooling-Setup

Improve the OpenwebUI experience by adding better and autonomous tool calling,

Primary LanguagePythonGNU General Public License v2.0GPL-2.0

OpenWebUI Auto Tool Selector Suite

Version Python OpenWebUI License Stars

Intelligent tool routing and autonomous AI capabilities for OpenWebUI

πŸ“¦ OpenWebUI Marketplace: Auto Tool Router | Exa Search Router

δΈ­ζ–‡ Readme | Quick Start | Configuration Guide

Overview

The OpenWebUI Auto Tool Selector Suite transforms your local AI models from passive chat interfaces into intelligent, autonomous assistants. Version 1.1 introduces enhanced debugging, flexible architecture, and improved accessibility for both vision and non-vision models.

Key Features

Feature Description
πŸ€– Autonomous Tool Selection Automatically routes user queries to the most appropriate tool without manual intervention
πŸ” Multi-Modal Search Three search modes: Crawl (specific URLs), Standard (quick research), Complete (deep analysis)
🎨 Intelligent Image Generation Auto-optimizes prompts and seamlessly integrates generated images into conversations
πŸ’» Dual Code Execution Support for both Jupyter notebooks and basic Python code interpretation
πŸ‘οΈ Universal Vision Non-vision models gain image understanding through automatic transcription
πŸ”§ Advanced Debugging Comprehensive logging system for troubleshooting and optimization
⚑ Real-Time Status Live progress updates throughout tool execution

Architecture

The suite consists of two main components:

  1. Auto Tool Selector (Function): Master router that analyzes queries and selects appropriate tools
  2. Exa Search Router (Tool): Advanced search capabilities with fallback to native OpenWebUI search

Architecture Diagram

Installation & Setup

Prerequisites

Ensure you have Docker access to your OpenWebUI instance.

Step 1: Install Dependencies

docker exec -it open-webui bash
pip install exa_py
exit
docker restart open-webui

Step 2: Add the Components

  1. Install Auto Tool Selector (Function):

    • Go to Admin Settings β†’ Functions β†’ New Function
    • Copy and paste the contents of auto_tool_selector.py
    • Save the function
    • If you plan to use Jupyter Lab as the code interpreter, please download the uploader.py and place it in the home directory of Jupyter.

  2. Install Exa Search Router (Tool) (Optional):

    • Go to Workspace β†’ Tools β†’ New Tool
    • Copy and paste the contents of exa_router_search.py
    • Important: Set Tool ID to exa_router_search
    • Save the tool

Step 3: Configure Settings

All configuration is now done through the UI settings - no manual file editing required!

  1. Enable the Function:

    • In your model settings, enable only the Auto Tool Selector function
    • Do not enable the Exa Search Router tool directly

  2. Configure Your Preferences:

    • Access function settings through the model interface
    • Configure API keys, models, and behavior options as needed

Configuration

Auto Tool Selector Settings

Setting Purpose Recommendation
helper_model Decides which tool to use for queries GPT-4o-mini, Claude-3-haiku
vision_model Analyzes images for non-vision models GPT-4o, Gemini-2.0-flash
vision_injection_models List of non-vision models to enhance Add your model IDs (comma-separated)
use_exa_router_search Enable advanced Exa search vs native search true (if Exa tool is installed)
debug_enabled Enable detailed debug logging false (enable for troubleshooting)
use_jupyter_code_interpreter Use Jupyter vs basic code execution true (recommended)

Exa Search Router Settings (If Installed)

Setting Purpose Recommendation
exa_api_key Required: Your Exa.ai API key Get yours at exa.ai
router_model Chooses search strategy (Crawl/Standard/Complete) GPT-4o-mini
quick_search_model Handles standard search operations GPT-4o-mini
complete_agent_model Powers deep research analysis GPT-4o, Claude-3-sonnet
complete_summarizer_model Creates final comprehensive summaries GPT-4o, Gemini-2.0-flash
debug_enabled Enable search operation debugging false (enable for troubleshooting)

Usage Examples

Autonomous Tool Selection

User: "What's the latest news about AI developments today?"
β†’ Automatically routes to web search, finds current articles, synthesizes response

User: "Create a logo for my coffee shop called 'Morning Brew'"
β†’ Automatically routes to image generation, optimizes prompt, generates and displays image

User: "Analyze this sales data and create a visualization"
β†’ Automatically routes to code interpreter, processes data, creates charts

Search Modes

  • Crawl Mode: "What does this article say? https://example.com/article"
  • Standard Mode: "What are the benefits of renewable energy?" (default for most queries)
  • Complete Mode: "Do a deep research comparison of React vs Vue.js frameworks" (requires explicit request)

Vision Enhancement

Non-vision models can now process images when you include them in your messages. The system automatically describes images and provides that context to the model.

Troubleshooting

Enable Debug Mode

Set debug_enabled to true in your function/tool settings to see detailed logs in your Docker container:

docker logs open-webui -f

Common Issues

Tool not activating: Check that only the Auto Tool Selector function is enabled in model settings, not the individual tools.

Search failing: If using Exa search, verify your API key is set correctly. The system will fall back to native search if Exa is unavailable.

Vision not working: Ensure vision_model is set and your model ID is listed in vision_injection_models.

Update Log

Version 1.1 (Current)

  • New: Enhanced debugging system with color-coded logging
  • New: Vision model integration for non-vision models
  • New: Modular search architecture with native OpenWebUI fallback
  • New: Choice between Jupyter and basic code interpreters
  • New: Settings-based configuration (no more manual file management)
  • Improved: More robust error handling and retry mechanisms
  • Improved: Better status updates and user feedback

Version 1.0

  • Initial release with autonomous tool routing
  • Basic search, image generation, and code interpretation
  • Manual configuration through separate files

FAQ

Q: Do I need the Exa Search Router tool? A: No, it's optional. The Auto Tool Selector will fall back to OpenWebUI's native search if Exa is not available.

Q: Why use Jupyter over basic code interpreter? A: Jupyter provides a full notebook environment with file persistence, better for complex analysis and data work.

Q: Can I use this with any OpenWebUI model? A: Yes, the Auto Tool Selector works with any model. Vision enhancement works best with models that support tool calling.

Q: How do I know if tools are working? A: Enable debug mode and check Docker logs. You'll see detailed information about tool selection and execution.

License & Support

  • Author: ShaoRou459
  • GitHub: OpenwebUI-Tooling-Setup
  • Issues: Report bugs and request features via GitHub Issues
  • Version: 1.1.0

Transform your AI from reactive to proactive with intelligent tool routing and autonomous capabilities.