/ai-workflow-orchestrator-docs

A lightweight, Spring Boot–friendly plugin framework** to orchestrate complex AI workflows — integrating multiple LLMs, tools, and [MCP (Model Context Protocol)](https://modelcontextprotocol.io/) services with zero boilerplate

MIT LicenseMIT

🤖 AI Workflow Orchestrator

A lightweight, Spring Boot–friendly plugin framework to orchestrate complex AI workflows — integrating multiple LLMs, tools, and MCP (Model Context Protocol) services with zero boilerplate.

🚀 Why Use AI Orchestrator?

  • 🪶 Lightweight Architecture – No external AI frameworks. Uses custom HTTP clients instead of heavy solutions like LangChain4j.
  • 📦 Smaller JAR Size – Reduced by 15–20 MB due to minimal dependencies.
  • ⚡ Faster Startup – No auto-configuration overhead, ensuring quick boot times.
  • 🔧 Direct API Control – Fully optimized request/response handling with Java HttpClient.
  • 🌱 Spring Boot Friendly – Plug-and-play integration with any Spring Boot application.
  • 🚀 High Performance – Efficient, low-latency orchestration with native HTTP API calls.

📚 Table of Contents

🚀 Features

  • 🧠 Multi-LLM Support: OpenAI, Anthropic, and Ollama with custom HTTP clients
  • ⚙️ Dynamic Workflow Planning: AI-driven step orchestration
  • 🔌 MCP Protocol Support: HTTP, WebSocket & Async clients
  • 🛠️ Tool Integration: Register & invoke custom tools seamlessly
  • 🔄 Schema Transformation: JSON schema-based data conversion
  • 🖥️ CLI & Web Modes: Dual execution interfaces
  • 🧩 Intelligent Transformations: LLM-powered data format conversions
  • 🪶 Lightweight Architecture: No external AI framework dependencies
  • ⚡ High Performance: Direct HTTP API calls with Java HttpClient

📁 Project Structure

SpringAIOrchestrator/
├── ai-orchestrator-bom/           # Bill of Materials & Core Framework
│   └── ai-orchestrator-core/      # Core orchestration engine
├── examples/                      # Example workflow implementations
│   ├── ai-travel-workflow/        # Travel planning workflow example
│   └── ai-resume_builder-workflow/ # Resume building workflow example
│   └── ai-document_summarizer-workflow/ # Summaring the document and classification
├── LICENSE                        # MIT License
├── README.md                      # This documentation
└── USAGE.md                       # Detailed usage guide

🛠️ Quick Start

✅ Prerequisites

  • ☕ Java 21+
  • 📦 Maven 3.8+
  • 🔑 API Keys (OpenAI / Anthropic / Ollama)
  • 🌐 Internet connection for LLM API calls
  • 🧠Local Ollama Installation

🧠 Local LLM Setup with Ollama

The AI Orchestrator Framework can run entirely offline using Ollama — a lightweight local runtime for open-source LLMs.
This allows you to test workflows locally without API keys, network dependencies, or cloud costs.

🛠️ Step 1: Install Ollama

Install Ollama on your system:

MacOS:

brew install ollama

Linux

curl -fsSL https://ollama.com/install.sh | sh

Windows

  • Download the latest installer from Ollama Download
  • Run the installer and follow the on-screen instructions.

🛠️ Step 2: Using Ollama For local (free) models

  • The core requirement for using any Ollama-based model in your models.yml is having the Ollama server running locally on the default host and port: http://localhost:11434.
  • Verify Service: Open your terminal and run ollama. The service typically starts automatically in the background. You can verify it's running by checking http://localhost:11434 in your browser.
  • Using Quantized Models Locally : Quantized models are compressed versions of larger LLMs (e.g., Llama 2, Mistral, Gemma), optimized for performance on consumer hardware with less VRAM or RAM. Ollama models are quantized by default (typically to 4-bit, or q4). You use the model tag to select the specific quantization level if needed, but for most models, the default tag is already a quantized version.

📦 Common Ollama Model Pull Commands & Configuration

Goal 🧰 Ollama CLI Command 🧠 models.yml Configuration
Pull Default (usually q4) bash<br>ollama pull llama2<br> yaml<br>modelName: "llama2"<br>
Pull Specific (e.g., 3-bit) bash<br>ollama pull llama2:7b-q3_K_L<br> yaml<br>modelName: "llama2:7b-q3_K_L"<br>
Pull Latest Version bash<br>ollama pull llama3<br> yaml<br>modelName: "llama3"<br>
Pull Specialized Coder Model bash<br>ollama pull qwen2.5-coder:14b-instruct-q3_K_L<br> yaml<br>modelName: "qwen2.5-coder:14b-instruct-q3_K_L"<br>
Pull Lightweight Model bash<br>ollama pull mistral<br> yaml<br>modelName: "mistral"<br>

🧩 Configuration local ollama models Example (models.yml)

The local alias uses the default quantized llama2 model pulled from Ollama:

models:
  - alias: "local"
    provider: "ollama"
    modelName: "llama2"
    baseUrl: "http://localhost:11434"
    enabled: true

☁️ Step 3: Using Ollama Cloud Models

For models that are too large to run locally on your personal machine, Ollama Cloud Models provide a seamless solution. They utilize remote, high-performance GPU hardware while still being accessed through your local http://localhost:11434 endpoint — so your application code doesn’t need any changes.

🔐 Sign In to Ollama Cloud

Cloud models require an Ollama account. Sign in from your terminal:

ollama signin

☁️ Pull / Use Cloud Models

# Example: Pulling a cloud model
ollama pull qwen3-coder:480b-cloud

🧩 Configuration Example (models.yml)

The example below shows how to configure the default alias to use the cloud-offloaded qwen3-coder:480b-cloud model.

models:
- alias: "default"
  provider: "ollama"
  modelName: "qwen3-coder:480b-cloud"  # This model is offloaded to Ollama Cloud
  baseUrl: "http://localhost:11434"
  enabled: true

- alias: "transformer"
  provider: "ollama"
  modelName: "qwen3-coder:480b-cloud"
  baseUrl: "http://localhost:11434"
  enabled: true

⚙️ Code Installation

  1. Clone the repository
git clone <repository-url>
cd SpringAIOrchestrator/spring-ai-orchestrator
  1. Build the framework
cd ai-orchestrator-bom
mvn clean install
  1. Set environment variables
# Required LLM API Keys
export OPENAI_API_KEY="your-openai-key"
export ANTHROPIC_API_KEY="your-anthropic-key"
export OLLAMA_BASE_URL="http://localhost:11434"

# Application Configuration
export SERVER_PORT=8282
export CLI_MODE=false
export APP_NAME="ai-orchestrator"

# Optional: Planner Configuration
export PLANNER_FALLBACK=true
export PLANNER_MAX_ITERATIONS=5
export PLANNER_TARGET_RANK=10
  1. Run an example workflow
cd ../examples/ai-travel-workflow
mvn spring-boot:run

# Or run resume workflow
cd ../ai-resume_builder-workflow
mvn spring-boot:run

🌐 API Usage

🗺️ Run Travel Workflow

# Plan a trip to Tokyo
curl -X POST "http://localhost:8282/api/workflows/run/travel-planner" \
  -H "Content-Type: application/x-www-form-urlencoded" \
  -d "prompt=Plan a 5-day trip to Tokyo in March with budget of $3000"

# Business trip planning
curl -X POST "http://localhost:8282/api/workflows/run/travel-planner" \
  -H "Content-Type: application/x-www-form-urlencoded" \
  -d "prompt=Business trip to London for 3 days, need flights and hotel recommendations"

📄 Run Resume Workflow

# Create Java developer resume
curl -X POST "http://localhost:8282/api/workflows/run/resume-builder" \
  -H "Content-Type: application/x-www-form-urlencoded" \
  -d "prompt=Create resume for Senior Java Developer with 5 years Spring Boot experience"

# Create data scientist resume
curl -X POST "http://localhost:8282/api/workflows/run/resume-builder" \
  -H "Content-Type: application/x-www-form-urlencoded" \
  -d "prompt=Build resume for Data Scientist role with Python and ML expertise"

📋 List All Workflows

curl -X GET "http://localhost:8282/api/workflows/list"

# Response example:
{
  "travel-planner": "travel-planner - AI-powered travel planning with weather, flights, and cost analysis",
  "resume-builder": "AI Resume Builder - Create a professional resume from user inputs and job descriptions"
}

🏗️ Architecture

The AI Orchestrator Framework uses a lightweight, custom architecture:

  • 🧠 Planner LLM – Generates and ranks intelligent execution plans
  • 🔄 Transformer LLM – Handles format conversions between steps
  • 🔗 Custom HTTP Clients – Direct API integration without external AI frameworks
  • 🪶 Zero Dependencies – No LangChain4j or similar heavy frameworks
┌─────────────────────────────────────────────────────────────────────────┐
│                        AI Orchestrator Framework                        │
└─────────────────────────────────────────────────────────────────────────┘

┌──────────────┐  ┌──────────────┐  ┌──────────────┐
│ User Input   │  │ REST API     │  │ CLI Mode     │
└──────┬───────┘  └──────┬───────┘  └──────┬───────┘
       │                 │                 │
       └─────────────────┼─────────────────┘
                         │
               ┌─────────▼─────────┐
               │ Workflow Controller│
               └─────────┬─────────┘
                         │
              ┌──────────▼──────────┐
              │ Dynamic Engine (AI) │
              └──────────┬──────────┘
                         │
           ┌─────────────▼─────────────┐
           │  🧠 Planner + 🔄 Transformer │
           └─────────────┬─────────────┘
                         │
           ┌─────────────▼─────────────┐
           │    Step Execution Engine   │
           └────────────────────────────┘

🔌 Workflow Types & Step Types

Workflow Types

  • 🔁 Sequential – Predefined step order
  • 🧠 Dynamic – AI-planned execution order

Step Types

  • 🤖 LLM – Language model interactions
  • 🔧 Tool – Custom annotated methods
  • 🌐 MCP – External service calls (HTTP, WebSocket, Async)

⚡ Plug & Play Workflow Creation

Create a new AI workflow in 5 minutes 🚀

  1. Add Dependency
<dependency>
  <groupId>codetumblr.net.ai.orchestrator</groupId>
  <artifactId>ai-orchestrator-core</artifactId>
  <version>1.0-SNAPSHOT</version>
</dependency>
  1. Create Application
@SpringBootApplication
@ComponentScan({"codetumblr.net.ai.orchestrator.core", "your.package"})
public class MyWorkflowApplication {
  public static void main(String[] args) {
    SpringApplication.run(MyWorkflowApplication.class, args);
  }
}
  1. Define Models (models.yml)
models:
  - alias: "default"
    provider: "openai"
    model: "gpt-4"
    apiKey: "${OPENAI_API_KEY}"
  1. Create Workflow (workflows/my-workflow.yaml)
id: "document-analyzer"
executionMode: "dynamic"
steps:
  - id: "extract_content"
    type: "tool"
    parameters:
      tool: "document_extractor"
  - id: "analyze_sentiment"
    type: "llm"
    parameters:
      model: "default"
      prompt: "Analyze sentiment: {input}"
  1. Add Tools
@Component
public class DocumentTools {
  @AiTool(name = "document_extractor")
  public String extractText(String filePath) {
    return "Extracted content...";
  }
}
  1. Run the Workflow
mvn spring-boot:run

📊 Real-World Plug-and-Play Workflow Examples

The AI Workflow Orchestrator comes with ready-to-use, production-grade workflow examples. These demonstrate how to combine LLMs, tools, and MCP services into powerful pipelines with minimal setup.

🚀 Built-in Examples

  • 🛫 Travel Planner – Automatically plan detailed, multi-day travel itineraries with flights, weather, and cost analysis.
  • 📄 Resume Builder – Generate tailored, professional resumes directly from a job description or skill set.
  • 📜 Document Summarizer – Extract, summarize, classify, and store metadata from documents (PDF, DOCX, etc.) using a Dynamic Workflow.

🧩 Upcoming Examples (Planned)

  • 🧠 Intelligent Chat Agent – Build a context-aware, tool-invoking assistant.
  • 📈 Financial Report Analyzer – Parse CSVs, extract key metrics, and generate business summaries.
  • 🧪 Code Review Assistant – Analyze pull requests and suggest improvements using code-aware LLMs.

💡 Tip: Each example is fully modular and can be customized or extended by simply editing the workflow YAML file — no code changes required.

🖥️ UI / Web Integration - Document Summarizer

The framework supports typical Spring Boot web controllers to enable simple file upload and dynamic prompt generation for complex workflows like Document Summarizer.

How it Works User uploads a file (PDF, DOCX, etc.) via the web interface.

A Spring Controller reads the file, Base64-encodes the binary content.

The Controller dynamically constructs a detailed Planner Prompt containing the Base64 content and file metadata.

The user can review/edit the prompt before it's submitted to the Orchestrator's API for execution. The LLM Planner then uses the Base64 content to route to the initial MCP step (extract_text).

Run the Web Example To run the web application for the Document Summarizer:

cd examples/ai-document_summarizer-workflow
mvn spring-boot:run

Access the application in your browser: http://localhost:8383/

Key Controller Endpoints

Endpoint Method Description
/ GET Displays the document upload form.
/generate-prompt POST Handles file upload, Base64-encodes the file, and builds the LLM Planner prompt. Forwards to a review page.
/submit-prompt POST Submits the final generated prompt to the core AI Orchestrator API (/api/workflows/run/document-summarizer) for execution.

⚙️ Advanced Configuration Options

Complete Application Configuration

# application.yml
server:
  port: ${SERVER_PORT:8282}
  
spring:
  application:
    name: ${APP_NAME:ai-orchestrator-travel_planner}
  shell:
    interactive:
      enabled: ${CLI_MODE:false}
    command:
      quit:
        enabled: true
    history:
      enabled: false
    script:
      enabled: false
  main:
    allow-circular-references: true
    web-application-type: servlet
  devtools:
    restart:
      enabled: false
    livereload:
      enabled: false
  boot:
    admin:
      enabled: false

logging:
  config: classpath:log4j2.xml

ai:
  orchestrator:
    workflows:
      path: ${WORKFLOWS_PATH:classpath*:workflows/*-workflow.yaml}
    models:
      description-truncate-length: ${DESC_TRUNCATE_LENGTH:30}
      path: classpath:${MODELS_PATH:/models.yml}
    planner:
      fallback-enabled: ${PLANNER_FALLBACK:true}
      step-separator: ${STEP_SEPARATOR:,}
      max-iterations: ${PLANNER_MAX_ITERATIONS:5}
      target-rank: ${PLANNER_TARGET_RANK:10}
      prompt:
        path: ${PLANNER_PROMPT_PATH:}
    transformer:
      prompt:
        path: ${TRANSFORMER_PROMPT_PATH:}
    mcp:
      clients:
        path: classpath:${MCP_CLIENTS_PATH:/mcp-clients.yml}

MCP Client Configuration

# mcp-clients.yml
clients:
  - name: "weather-services-api"
    endpoint: "${MCP_WEATHER_ENDPOINT:http://localhost:8282}"
    service: "weather_forecast"
    protocol: "${MCP_WEATHER_PROTOCOL:http}"
    timeout: "${MCP_WEATHER_TIMEOUT:5000}"  # Configurable timeout
    auth:
      type: "${MCP_WEATHER_AUTH_TYPE:none}"  # none, basic, oauth2, api-key, jwt
      apiKey: "${MCP_WEATHER_API_KEY:}"
      apiKeyHeader: "${MCP_WEATHER_API_KEY_HEADER:X-API-Key}"
    enabled: "${MCP_WEATHER_ENABLED:true}"

  - name: "flight-services-api"
    endpoint: "${MCP_FLIGHT_ENDPOINT:http://localhost:8282}"
    service: "search_flights"
    protocol: "${MCP_FLIGHT_PROTOCOL:http}"
    timeout: "${MCP_FLIGHT_TIMEOUT:5000}"   # Configurable timeout
    auth:
      type: "${MCP_FLIGHT_AUTH_TYPE:none}"
    enabled: "${MCP_FLIGHT_ENABLED:true}"

Model Configuration

# models.yml
config:
  defaultMaxTokens: 1000
  promptTruncateLength: 50
  descriptionTruncateLength: 30
  autoRegisterForPlugins: true

models:
  - alias: "default"
    provider: "ollama"
    modelName: "qwen3-coder:480b-cloud"
    baseUrl: "http://localhost:11434"
    temperature: 0.7
    maxTokens: 2000
    enabled: true

  - alias: "openai"
    provider: "openai"
    modelName: "gpt-4o-mini"
    apiKey: "${OPENAI_API_KEY}"
    temperature: 0.7
    maxTokens: 4000
    enabled: true

  - alias: "planner"
    provider: "ollama"
    modelName: "qwen2.5-coder:14b-instruct-q3_K_L"
    baseUrl: "http://localhost:11434"
    temperature: 0.3
    maxTokens: 1500
    enabled: true

  - alias: "transformer"
    provider: "ollama"
    modelName: "qwen3-coder:480b-cloud"
    baseUrl: "http://localhost:11434"
    temperature: 0.1
    maxTokens: 1000
    enabled: true

🧪 CLI Mode

Enable interactive CLI execution:

export CLI_MODE=true
mvn spring-boot:run
  or
CLI_MODE=true mvn spring-boot:run
  or
java -DCLI_MODE=true -jar ai-travel-workflow-1.0-SNAPSHOT.jar

Available Commands

# List workflows
lw

# Interactive workflow selection
interactive
i
run-interactive

# Run specific workflow
run --workflow-id travel-planner --prompt "Plan weekend trip to Paris"
run-workflow --workflow-id resume-builder --prompt "Create resume for DevOps engineer"

# Quick run by number
quick --number 1
q --number 2

# Show help
help
h
?

# Logging info
log-info
logs
debug-help

# Exit CLI
exit
quit
bye

🧰 Logging Configuration

The framework uses log4j2.xml for logging configuration. Logging is configured via:

logging:
  config: classpath:log4j2.xml

Log files are written to the logs/ directory in the application root.

Configurable Timeouts

# Planner Configuration (All Configurable)
ai:
  orchestrator:
    planner:
      max-iterations: 5        # Maximum planning iterations
      target-rank: 10          # Target plan quality rank
      fallback-enabled: true   # Enable fallback when planning fails

# MCP Client Timeouts (Per Client)
clients:
  - name: "weather-services-api"
    timeout: "5000"           # 5 seconds (configurable)
  - name: "flight-services-api"
    timeout: "5000"           # 5 seconds (configurable)

Hardcoded Timeouts - Scope of Improvement(config based)

  • LLM Planning Timeout: 120 seconds
  • Planning Session Timeout: 60 seconds
  • LLM Client Timeouts: 5 minutes request, 30 seconds connection
  • Schema Caching: Automatic using ConcurrentHashMap
  • HTTP Client: Java 21 HttpClient with connection pooling

🔧 Environment Variables Reference

Core Application

export SERVER_PORT=8282
export CLI_MODE=false
export APP_NAME="ai-orchestrator"

LLM API Keys

export OPENAI_API_KEY="sk-your-openai-key"
export ANTHROPIC_API_KEY="sk-ant-your-anthropic-key"
export OLLAMA_BASE_URL="http://localhost:11434"

Planner Configuration

export PLANNER_FALLBACK=true
export PLANNER_MAX_ITERATIONS=5
export PLANNER_TARGET_RANK=10
export STEP_SEPARATOR=","
export PLANNER_PROMPT_PATH=""
export TRANSFORMER_PROMPT_PATH=""

File Paths

export WORKFLOWS_PATH="classpath*:workflows/*-workflow.yaml"
export MODELS_PATH="/models.yml"
export MCP_CLIENTS_PATH="/mcp-clients.yml"

Logging Levels

export PLANNER_LOG_LEVEL=INFO
export ENGINE_LOG_LEVEL=INFO
export WORKFLOW_LOG_LEVEL=INFO
export MCP_SERVICE_LOG_LEVEL=INFO
export SCHEMA_LOG_LEVEL=INFO

🆘 Troubleshooting

Common Issues & Solutions

Issue Solution
Schema not found Verify schema files exist in resources/schemas/
LLM client not available Check API keys and network connectivity
Workflow not found Ensure workflow YAML is in resources/workflows/
Tool execution failed Verify tool parameters and implementation
MCP service timeout Check MCP client configuration and endpoints
Planning failed Enable planner fallback: PLANNER_FALLBACK=true

Health Checks

# Check workflow status
curl -X GET "http://localhost:8282/api/workflows/list"

# Verify environment variables
echo $OPENAI_API_KEY
echo $ANTHROPIC_API_KEY
echo $OLLAMA_BASE_URL

Debug Steps

  1. Enable Debug Logging 
    export PLANNER_LOG_LEVEL=DEBUG
export ENGINE_LOG_LEVEL=DEBUG
export WORKFLOW_LOG_LEVEL=DEBUG

Refer log4j2.xml - For different component logging levels


2. **Check Log Files**
```bash
tail -f logs/ai-orchestrator-**.log
  1. Validate Configuration 
    # Check YAML syntax
cat src/main/resources/models.yml
cat src/main/resources/mcp-clients.yml

🤝 Contributing

We ❤️ contributions!

  1. Fork the repo
  2. Create a feature branch
  3. Add Javadoc & tests
  4. Submit a PR 🚀

📄 License

This project is licensed under the MIT License. See the LICENSE file for details.

👨‍💻 Author

Ravinderjeet Singh Nagpal
Building next-generation AI orchestration systems.

📊 Performance Features

🪶 Lightweight Architecture

  • No External AI Frameworks - Custom HTTP clients instead of LangChain4j
  • Smaller JAR Size - Reduced by ~15-20MB without heavy dependencies
  • Faster Startup - No auto-configuration overhead
  • Direct API Control - Optimized request/response handling

Support Resources

  • 🔑 Verify environment variable configuration
  • 📜 Validate YAML and schema syntax
  • 📖 See USAGE.md for detailed examples
  • 🐛 Enable debug logging for detailed traces

🔗 Related Projects

⭐ Star this repo if you find it helpful! Your support keeps this project growing 🚀