Code_Parser

This project provides a tool to parse the python script from user specific directory and generate corresponding Python and shell scripts for execution on SLURM systems.

Getting Started

These instructions will guide you through setting up the project on your local machine or on a JupyterHub environment for development and testing purposes.

Prerequisites

Python 3.8 and above
pm4py
pickle
Additional libraries or tools as necessary for the project.

Installation

Set up on JupyterHub: To facilitate easy setup and management of the project, it is recommended to use Uni's JupyterHub environment. Access the JupyterHub using this link: RWTH Aachen JupyterHub.
- After successful login, create a new directory to serve as the project's root directory.
- Within this directory, create a virtual environment by executing:
```
python -m venv /path/to/venv
```
- Activate the virtual environment using the following command:
```
source /path/to/venv/bin/activate
```
Clone the Repository:
- Use the Git command to clone the repository into your JupyterHub environment or local machine:
```
git clone <repository-url>
```
- Navigate to the project directory:
```
cd <cloned-project-directory>
```
Install Dependencies:
- While the virtual environment is activated, install the required libraries using pip:
```
pip install pm4py pickle
```
- Ensure all dependencies are installed correctly by checking their installation:
```
pip freeze
```
Prepare the Environment:
- Upload the parse.py and feedbackMechanism.py scripts to the project's root directory.
- Create directories named test and result within the root directory. The test directory will hold the scripts generated by the parser, and the result directory will store logs from script executions.

Usage

Prepare Test Files:
- Access the source_code_for_testing folder on the GitHub project page and download the five test files provided.
- Upload these files to your project’s root directory on the slurm system.
Execute the Parse Script:
- Execute the parse.py script by specifying the test file's path. This script divides the test file into manageable subtasks and automatically stores the generated scripts in the test folder.
- To execute the script:
```
python parse.py
```
Run the Scripts on SLURM:
- Submit the job to SLURM using the following command:
```
sbatch test/trigger_slurm_script.py
```
- Each subtask's output is saved in the root directory, while logs are stored in subdirectories within the results directory, named after their respective test files.
Feedback Mechanism:
- After the execution, run feedbackMechanism.py to integrate run information from different subtasks into a single CSV file:
```
python feedbackMechanism.py
```
- This script also performs process mining and saves the process model images in the specified output directory.

Additional Information

The parser currently only supports static for loops formatted as for var in [0.1, 0.2, 0.3]. The second line must specify the data type of var (currently supports only primitive data types).
Due to the complexity of dynamically analyzing and parsing the abstract syntax tree (AST) for dynamic content, further development is needed to support more complex code structures.

Development

The parse.py script utilizes the Abstract Syntax Tree (AST) module to analyze and transform the Python script into a structured format. This helps in generating executable scripts that are compatible with SLURM execution environments.
The project is designed to be flexible and extendable, allowing for future enhancements like support for more complex loop structures and dynamic code segments.