🎮 3D Scene Asset Generator

Our participation to the 2025 Gradio Agent MCP Hackathon

https://huggingface.co/spaces/Agents-MCP-Hackathon/3D-Game-Environment-Builder

Transform player biographies into personalized 3D environments using LLM-powered analysis and 3D asset generation models pipelines.

🌟 Project Overview

This hackathon project creates a 3D scene generator that analyzes player biographies and automatically generates personalized 3D environments. By combining the power of LLM analysis with generation models (FLUX + Trellis), we create unique, contextual 3D assets that reflect each player's personality, interests, and background.

✨ Key Features

🤖 AI-Powered Analysis: LLM analyzes player biographies to understand personality and interests
🎨 3D Generation: FLUX + Trellis pipeline generates high-quality, contextual 3D assets
🌐 Interactive Web Interface: Gradio interface with real-time generation and examples
🎮 3D Game Integration: Godot game client that connects to MCP server for immersive 3D environment visualization
🔧 MCP Integration: Supports Model Context Protocol for enhanced interactions
⚡ Optimized Pipeline: Uses GGUF quantization and LoRA models for fast, efficient generation
📱 User-Friendly: Simple input → AI analysis → 3D asset generation → game environment workflow

🏗️ Architecture

Thank you to gokaygokay for the GGUF

Technology Stack

Frontend: Gradio with custom CSS styling
Game Client: Godot Engine 4.4 for 3D environment visualization
AI Analysis: Anthropic Claude Sonnet 4
3D Generation: FLUX + Trellis on Modal
MCP Protocol: Model Context Protocol for client-server communication
Output Format: GLB (3D models compatible with most engines)

🚀 Quick Start

Prerequisites

Python 3.8+
Anthropic API key
Modal account
Godot Engine 4.4+ (for game client)
mcptools CLI (for MCP communication)

Installation

Clone the repository

git clone https://github.com/castlebbs/gradio-mcp-hackathon.git
cd gradio-mcp-hackathon

Set up the Gradio application

cd gradio
pip install -r requirements.txt

Configure API keys

export ANTHROPIC_API_KEY="your-anthropic-api-key"

Set up Modal
```
modal setup
```

Deploy the Modal function

cd ../modal
modal deploy flux-trellis-GGUF-text-to-3d.py

Run the application
```
cd ../gradio
python app.py
```
Set up the Godot game client

Install mcptools for MCP communication. Check your OS install instruction on: https://github.com/f/mcptools/blob/master/README.md
Open the Godot project
In Godot Engine, open: godot/project.godot
Run the game scene to start the 3D environment

💡 Usage Example

Web Interface

Input Biography:

"Marcus is a tech enthusiast and gaming streamer who loves mechanical keyboards and collecting vintage arcade games. He's also a coffee connoisseur who roasts his own beans and enjoys late-night coding sessions."

Generated 3D Assets:

Vintage arcade cabinet with classic game artwork
Premium mechanical keyboard with RGB backlighting
Professional coffee roasting station with custom setup
Gaming chair with LED accents and streaming equipment
Retro-futuristic desk lamp with adjustable lighting

Godot Game Client

The Godot game provides an immersive 3D environment where:

Player Input: Enter your biography through the in-game UI
MCP Communication: Game connects to the Gradio MCP server via mcptools
Real-time Generation: 3D assets are generated and sent back to the game
Environment Building: Assets are automatically placed in the 3D scene
Interactive Exploration: Walk around and explore your personalized environment

📁 Project Structure

gradio-mcp-hackathon/
├── gradio/                    # Main Gradio application
│   ├── app.py                # Core application logic
│   ├── requirements.txt      # Python dependencies
│   ├── README.md            # Detailed app documentation
│   └── images/              # UI assets and examples
├── modal/                    # Modal cloud functions
│   ├── flux-trellis-GGUF-text-to-3d.py  # 3D generation pipeline
│   └── README.md            # Modal setup documentation
├── godot/                    # Godot game client
│   ├── project.godot        # Godot project configuration
│   ├── mcp.sh              # MCP communication script (Unix/macOS)
│   ├── mcp.bat             # MCP communication script (Windows)
│   ├── scenes/             # Game scenes (main, player, UI)
│   ├── scripts/            # GDScript files for game logic
│   │   ├── main.gd         # Main scene controller
│   │   ├── ui.gd           # User interface logic
│   │   ├── mcp.gd          # MCP client communication
│   │   └── 3Dgeneration.gd # 3D asset handling and placement
│   ├── assets/             # Generated 3D assets storage
│   └── models/             # Base 3D models and textures
├── LICENSE                   # MIT License
└── README.md                # This file

🔧 Technical Details

AI Pipeline

Text Analysis: Claude Sonnet processes biographical text to extract personality traits and interests
Prompt Generation: AI creates detailed, contextual prompts for 3D asset generation
Asset Creation: FLUX + Trellis pipeline generates high-quality 3D models

Optimizations

GGUF Quantization: Reduces model size while maintaining quality
LoRA Models: Hyper FLUX 8Steps for faster inference, Game Assets LoRA for better 3D results
Modal Scaling: Automatic scaling for concurrent requests

🏆 Hackathon Team

castlebbs@ - Gradio, Modal
stargarnet@ - Godot
zinkenite@ - 3D work

Built with ❤️ for the 2025 Gradio Agent MCP Hackathon

castlebbs/gradio-mcp-hackathon