code-executer: A Go repository from sudeep162002

Executioner: A Scalable Code Execution Platform for Everyone

Introduction

Welcome to Executioner, your one-stop shop for seamless and efficient code execution! Inspired by platforms like LeetCode, we offer a highly scalable and user-friendly environment to execute your code with ease.

Unmatched Scalability:

Built for Performance: Engineered to handle millions of code execution requests concurrently, ensuring smooth operation even under massive workloads.
Containerized with Docker: Leverages containerization for flexible deployment, efficient resource utilization, and effortless scaling.
Master-Worker Architecture: Employs a Kafka-powered master-worker architecture where the master distributes tasks to worker containers for parallel execution.

Tech Stack and Tools:

Golang: Enables efficient container orchestration using https://pkg.go.dev/runtime.
TypeScript: Drives worker containers, tailored for code execution with specific languages.
Node.js and Express: Forms the foundation for the API gateway, facilitating interaction with the platform.
Go Gin: Powers the API gateway, providing a high-performance framework for building robust APIs in Golang.
Kafka: Acts as the central hub for message queuing, handling task distribution and ensuring efficient communication between components.
MySQL Database: Stores code execution results and user data securely.
Server-Sent Events (SSE): Enables real-time result delivery to the frontend, enhancing user experience.

Multithreading for Maximum Efficiency:

Maximizes CPU utilization by employing multithreading within worker containers, ensuring optimal performance for code execution.

Why Golang?

Efficient Docker Management: Golang excels at handling the Docker engine daemon effectively, contributing to the platform's smooth operation.
Concurrency and Performance: Well-suited for building concurrent and high-performance applications.

Local Development Setup

Prerequisites:

Golang: Download and install Golang from https://go.dev/. Ensure you have set the GOPATH and GOBIN environment variables correctly.
Docker: Download and install Docker from https://www.docker.com/.

Steps:

Clone the Repository:

git clone https://github.com/sudeep162002/code-executer

Define Port: Create a file named .env in the base directory of your project. Add a line specifying the port you want to use for the API gateway:
```
API_PORT=8080
```
Replace 8080 with your desired port number.
Install Dependencies: Open a terminal in the project directory and run:
```
go get
```
This command downloads all the necessary Go dependencies listed in the go.mod file.
Run the Platform: Start all components using Docker Compose:
```
docker-compose up
```
This command builds and runs the Docker images for your platform, including Kafka, MySQL, and other dependencies.

Troubleshooting:

If you encounter any issues during setup, you can use the following commands to check the status of your Kafka queues:

Open a Kafka container shell:
```
docker exec -it kafka bash
```
Use kafkacat to check queue contents:
```
kafkacat -b kafka:29092 -t topic-you-want-to-check -C
```
Replace topic-you-want-to-check with the actual name of the topic you want to inspect.

Sample and Usage

Sample Input Body:

{
  "id": "69896",
  "userName": "sudeep162002",
  "Language": "java",
  "output": "..."  // Paste your code here
}

Code Deparser and Parser:

This platform utilizes code deparser and parser functionalities to prepare code for execution and display the results.

Code Deparser and Parser (TypeScript):

// Code Deparser: Transforms code for execution
const deparser = (inputString: string): string => {
  return inputString
    .replace(/\\n/g, '\n')   // Restore actual line breaks
    .replace(/^   /gm, '')   // Remove leading indentation
    .replace(/\\"/g, '"');   // Replace escaped double quotes
};

// Code Parser: Formats code for display
const parser = (normalCode: string): string => {
  return normalCode
    .replace(/"/g, '\\"')    // Escape double quotes
    .split('\n')           // Split into lines
    .map(line => `   ${line}`) // Indent each line
    .join('\\n\\n');       // Join lines with "\n\n"
};

Usage:

Deparsing Code for Execution:

const preparedCode = deparser(inputCode);
// Send preparedCode to worker for execution

Parsing Results for Display:

const formattedOutput = parser(executionResult);
// Display formattedOutput to the user

Execution:

Send a POST request to the API endpoint (http://localhost:<API_PORT>/execute) with the sample input body containing your code.
The platform uses Server-Sent Events (SSE) to push execution results to the frontend in real-time.

Remember:

Replace placeholders like <API_PORT> with appropriate values.
Ensure your code is properly formatted and adheres to the supported language's syntax.

Contributing to the Project

We believe in fostering a collaborative environment! Your contributions are valued! Feel free to explore the project, share your ideas, and submit pull requests to shape the future of Executioner.

Roadmap (To-Do):

Frontend Development: Create a user-friendly interface for interacting with the platform. This includes features like code submission, execution monitoring, and result display.
Prometheus Monitoring: Integrate with Prometheus for detailed infrastructure and application monitoring, providing valuable insights into platform performance and resource usage.
CI/CD Pipeline: Establish a continuous integration and continuous delivery (CI/CD) pipeline utilizing GitHub Actions. This will automate tasks like building and pushing Docker images for supported languages to a Docker hub repository, streamlining the deployment process and ensuring consistent updates.
Language Support Expansion: Gradually expand support for additional programming languages, catering to a wider developer audience.

APIs

Execution API:

Method: POST
URL: http://localhost:5000/run-code
Body:

{
  "id": "69896",
  "userName": "sudeep162002",
  "Language": "java",
  "output": "..."  // Paste your code here
}

Execution Status and Output API:

Method: GET
URL: http://localhost:5000/{reqId}/{username}

Replace {reqId} with the ID of the execution request and {username} with the username for whom the request was made.

Response:

{
  "status": "pending" | "running" | "completed" | "failed",
  "output": "..." (only present if status is "completed")
}

Note:

Replace placeholders like <API_PORT> with appropriate values.
Ensure your code is properly formatted and adheres to the supported language's syntax.