- This project's importance rests in its capacity to streamline and simplify the data analysis process. Data preprocessing and model selection are typically labor-intensive manual processes that call for the creation of bespoke scripts or the usage of numerous tools by researchers.
- I hope to save researchers considerable time and energy by combining these duties into a single platform, allowing them to concentrate more on the fundamental analysis and interpretation of results.
- This project's goal of enabling academics and data scientists to quickly preprocess their datasets and use a variety of machine learning models, independent of their programming experience, is another key driving force behind it.
- The platform promotes collaboration and knowledge-sharing among academics, thereby boosting the field of data science, by offering a user-friendly interface and including well-liked preprocessing methods and ML algorithms.
Several design principles and architectural approaches were followed to ensure a robust and scalable system:
- Separation of Concerns: The frontend and backend components are decoupled, with each responsible for distinct functionalities. This separation allows for independent development and deployment, facilitating testing, scalability, and maintenance.
- RESTful API-based Framework: The project follows REST principles for communication between the frontend and backend, promoting clear separation of concerns, standardized API endpoints, and scalability.
- Component-Driven Architecture: The frontend follows a component-driven approach, creating reusable and independent components for UI elements, data processing, and result visualization. This design encourages code reuse, maintainability, and scalability.
- Best Practices: The codebase emphasizes best practices such as error handling, input validation, and security measures to ensure reliability and security.
- The frontend operates within a client-server architecture, communicating with the Flask backend through API endpoints.
- It manages state, and component organization to ensure an efficient and organized structure.
- React components represent individual UI elements, while conditional rendering define different views of the application.
- The frontend is built using React, a versatile and component-based JavaScript framework known for its reusability and robust ecosystem.
- Chakra-UI is utilized to expedite UI development, providing pre-styled components and responsive design features.
- Additionally, the HTML2PDF library is integrated to enable users to download reports and figures generated by the frontend.
- The CSVReader module is used to extract the names of all columns(attributes) from the uploaded CSV File.
- Data Upload and Attribute Selection: Users can upload CSV files containing their datasets via the frontend. The uploaded file is then passed to the Flask backend for processing. Users are prompted to select the output attribute from the available columns, a critical step for subsequent analysis and model training.
- Preprocessing Techniques: The frontend provides various preprocessing methods, including handling categorical variables, identifying and eliminating outliers, and using dimensionality reduction techniques like Recursive Feature Elimination (RFE) and Principal Component Analysis (PCA). Users can select the specific preprocessing techniques that suit their needs.
- Machine Learning Model Selection: Users can choose from a variety of machine learning models, including Gaussian naïve bayes, support vector machines, decision trees, random forests, and linear regression. These models can be selected from a list of options for further analysis.
- Model Training and Evaluation: After selecting the preferred machine learning models, users instruct the Flask backend to train these models using the preprocessed data. The models are trained on appropriate training and validation splits, producing predictions for the selected output attribute. Evaluation measures, including accuracy, precision, recall, F1-score, and mean squared error, are computed to assess model effectiveness.
- Results Presentation and Download: Users have access to project outcomes in an easily understandable format. Evaluation metrics, such as accuracy and error measurements, are displayed for each chosen model. Visualizations, such as predicted versus actual value graphs, provide deeper insights into model performance. Users can download the results, including model assessment metrics, forecasts, and visualizations, in PDF format for preservation and distribution.
The frontend development process was structured and iterative, focusing on the following key stages:
- Initial Interface: The development began with creating an intuitive user interface for uploading JSON objects, which would be transformed into DataFrames in the backend.
- CSVReader Module: The CSVReader module was integrated to facilitate output attribute selection. It extracted column names from uploaded CSV files, empowering users to make informed choices.
- Core Functionalities: The initial phase included implementing essential frontend functionalities, such as file upload, output attribute selection, and result generation using machine learning models.
- Chart.js Integration: A significant effort was dedicated to integrating and configuring Chart.js for result visualization. Scatter plots comparing predicted and actual values were created for enhanced data insights.
- Preprocessing Techniques: Additional preprocessing techniques, including categorical variable handling and outlier detection, were integrated into the frontend.
- Feature Selection and Dimensionality Reduction: The frontend was further enhanced with feature selection techniques and dimensionality reduction methods like Recursive Feature Elimination (RFE) and Principal Component Analysis (PCA).
- Download Functionality: Implementing download functionality using Chart.js was a crucial challenge. Data encoding and formatting were carefully managed to enable users to download charts and results for offline access.
- User Experience Refinement: The frontend received finishing touches to enhance user interaction and visual appeal. Borders and loading spinners were added for a more polished and user-friendly interface.
To set up the front-end for the MAPML project, follow these steps:
- Make sure you have Yarn installed on your machine.
- Open the terminal and navigate to the
front_end/directory. - Run the following command to install the required dependencies:
yarn install
To run the front-end application:
- Open the terminal and navigate to the
front_end/directory. - Run the following command:
yarn start - The front-end application will start running on
http://localhost:3000.
src/: Contains the source code for the React.js application.src/components/: Includes various components used in the application.
This repository's back-end is found here.
Contributions to the MAPML project are welcome! If you would like to contribute to the front-end code, please follow these steps:
- Fork the repository.
- Create a new branch for your feature or bug fix.
- Make your changes and commit them.
- Push your changes to your forked repository.
- Submit a pull request, describing your changes in detail.
I express my heartfelt gratitude to all the contributors and researchers who have provided valuable guidance and support throughout the development of the MAPML project. Your insights and feedback have been instrumental in shaping this application and advancing its capabilities. This project was carried out under the direct supervision of Dr. Manoj Semwal.
This project is licensed under the MIT License.