| English | 中文 |
|---|
pyExecuter is a high-performance Python script execution program based on devchat-ai/gopool. It leverages Go's parallel processing capabilities and Python's flexibility to provide a robust, efficient, and scalable solution for executing Python scripts.
- Utilizes devchat-ai/gopool's Goroutines pool for efficient concurrent task management
- Implements intelligent task scheduling based on CPU and memory resources
- Dynamically allocates Python script tasks to the worker pool for maximum resource utilization
- Supports priority-based task queues for urgent task execution
- Offers both FIFO and LIFO queue modes to suit different task processing needs
- Includes automatic retry mechanism for failed tasks
- Provides flexible Python script execution modes, supporting both multi-threading and multi-processing
- Automatically selects the appropriate execution mode based on task type (e.g., CPU-intensive vs I/O-intensive)
- Implements efficient load balancing to distribute system resources evenly
- Dynamically adjusts concurrent task numbers based on current system resource usage
- Utilizes devchat-ai/gopool's built-in mechanisms to manage Goroutines within reasonable limits
- Offers detailed task monitoring for real-time tracking of Python script execution status, runtime, and resource consumption
- Records comprehensive task execution logs, including start/end times, output, and errors
- Provides a web-based dashboard for visualizing task status and resource utilization
- Automatically captures exceptions and records detailed error information for failed tasks
- Supports configurable task retries with customizable retry counts and intervals
- Implements task recovery mechanisms to resume execution from previous states after unexpected crashes
- Supports distributed environments for executing tasks across multiple nodes
- Provides inter-node communication and synchronization mechanisms
- Utilizes lightweight RPC or message queues (e.g., Redis, RabbitMQ) for task distribution and result collection
- Sets execution timeout for each Python task to prevent resource hogging
- Automatically terminates and cleans up resources for timed-out tasks
- Automatically collects and stores task results in structured formats (e.g., databases or files)
- Offers a unified result aggregation interface for easy retrieval and processing
- Supports callback functions for custom result processing logic
- Provides a flexible Python execution interface with dynamic parameter passing
- Supports dynamic execution of Python code passed as strings
- Integrates with Python virtual environments for dependency isolation
- Offers REST API for task submission, status querying, and result retrieval
- Implements WebSocket for real-time task status monitoring
- Provides configurable execution parameters (e.g., concurrency level, max resource usage, log levels)
- Supports plugin-based extensions for adding new task types, monitoring tools, etc.
- Executes tasks in isolated environments to prevent system threats from malicious scripts
- Utilizes sandboxing technologies (e.g., Docker or virtual environments) for task isolation
- Implements authentication mechanisms to ensure only authorized users can submit and manage tasks
- Persists task states (pending, running, completed, failed) for recovery after program restarts
- Supports periodic saving of intermediate states for long-running tasks
- Supports task dependency definitions
- Manages task dependencies using Directed Acyclic Graphs (DAG) for efficient, ordered execution
[Instructions on how to install pyExecuter]
[Basic usage examples and code snippets]
[Details on how to configure pyExecuter]
We welcome contributions to pyExecuter! Please see our CONTRIBUTING.md file for details on how to get started.
[Your chosen license]
[Your contact information or project maintainer details]
- devchat-ai/gopool for providing the foundational Goroutines pool implementation.
- [Any other libraries or resources you want to acknowledge]