/CircuitMind

The code about TC-Bench and CircuitMind

Primary LanguagePythonApache License 2.0Apache-2.0

CircuitMind Logo
 

CircuitMind: Achieving Optimal Circuit Generation through Multi-Agent Collaboration

 

English | 简体中文

 
Hugging Face Dataset TC-Bench Dataset GitHub GitHub Stars
 

License Python Version

 

📚 Framework README | 🤖 Multi-Agent Framework | 💾 TC-Bench Dataset README | 📝 Paper | 🐛 Report Issues

Table of Contents

Introduction

CircuitMind is a multi-agent framework designed to overcome the efficiency limitations of Large Language Models (LLMs) in gate-level hardware design. While LLMs excel at code generation, their application in circuit design often results in gate counts far exceeding those of human expert designs. This "Boolean optimization barrier" stems from the limitations of LLMs in structured reasoning and global optimization required for efficient gate-level design.

Our work proposes CircuitMind, a novel hierarchical multi-agent system that achieves efficiency comparable to human experts through three key innovations:

  1. Syntax Locking (SL): Constrains generation to fundamental logic gates (AND, OR, NOT, XOR, NAND), forcing true Boolean optimization at the netlist level.
  2. Retrieval-Augmented Generation (RAG): Enables knowledge-driven design by retrieving and reusing optimized sub-circuit patterns from a dynamic knowledge base.
  3. Dual-Reward Optimization (DR): Balances functional correctness with physical efficiency (gate count, delay) through targeted feedback loops.

To evaluate our approach, we introduce TC-Bench, the first gate-level benchmark leveraging collective intelligence from the TuringComplete game. Based on thousands of competitively optimized human designs, TC-Bench provides metrics and performance tiers aligned with human-level proficiency.

Key Achievements

  • CircuitMind enables a significant portion (55.6%) of model implementations to achieve or surpass the efficiency levels of top human experts on TC-Bench.
  • Our framework demonstrates its ability to elevate mid-sized models to outperform larger models and match top-tier human expert performance without specialized training.
  • Significant improvements in Solution Efficiency Index (SEI) are observed across various LLMs, demonstrating the effectiveness of our collaborative approach in overcoming the Boolean optimization barrier.

Repository Contents

This repository contains the open-source code, models, and data for CircuitMind and TC-Bench. It primarily consists of two parts:

TC-Bench Test Suite & Evaluation Tools

This directory contains the complete TC-Bench benchmark dataset for evaluation.

  • The dataset is organized into three difficulty levels: Easy, Medium, and Hard, with subfolders for each problem (problem description, test code, reference solutions).
  • Notably, this directory also includes performance evaluation tools, allowing you to compare the performance of generated circuits (using metrics like SEI) against established human expert tiers.
  • Detailed information about the dataset structure and the use of evaluation tools can be found in the README file within the TC-Bench directory.
  • Click here to browse the TC-Bench dataset and tools

CircuitMind Framework

This directory contains the source code for the CircuitMind multi-agent framework.

Multi-Agent Implementation

We have developed an advanced multi-agent implementation that enhances the original CircuitMind approach with sophisticated agent collaboration and RAG-enhanced generation.

  • Agent Architecture: Features specialized agents (CoderAgent, Reviewer, Executor, UserProxy, Summarizer) working in coordinated workflows
  • RAG Integration: Knowledge-driven design with retrieval-augmented generation using local Ollama models
  • Robust Execution: Comprehensive error handling, timeout protection, and automated debugging
  • Flexible Configuration: YAML-based configuration supporting multiple LLM providers

📚 Multi-Agent Framework Documentation | 🌐 中文文档

Open Source Plan (To-Do List)

  • ✅ TC-Bench Test Suite & Evaluation Tools - Released!
  • ✅ CircuitMind Multi-Agent Framework - Released! Advanced implementation with sophisticated agent collaboration

Read Our Paper

Read our paper for more details: Paper

Citation

If this work is useful for your research, please cite our paper:

@article{qin2025towards,
  title={Towards Optimal Circuit Generation: Multi-Agent Collaboration Meets Collective Intelligence},
  author={Qin, Haiyan and Feng, Jiahao and Feng, Xiaotong and Xing, Wei W and Kang, Wang},
  journal={arXiv preprint arXiv:2504.14625},
  year={2025}
}

License

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.