wurzy/DataGen

JSON/XML/CSV Dataset Generator.

DataGen

A JSON, XML and CSV dataset generator, featuring a Domain Specific Language (DSL) that allows for automatic creation of RESTful APIs on the generated datasets.
Back-End · Front-End · Strapi

DataGen From Schemas (v2)

New version capable of producing datasets in JSON/XML from both JSON and XML schemas, taking advantage of DataGen's workflow.
Navigate here

Table of Contents

1. About The Project
   • Built With
2. Getting Started
   • Prerequisites
   • Installation
3. License
4. Contacts
5. Useful Links

About The Project

DataGen is a versatile and powerful tool that allows for quick prototyping and testing of software applications, since currently too few solutions offer both the complexity and scalability necessary to generate adequate datasets in order to feed a data API or a more complex APP, enabling their testing with appropriate data volume and complexity.

DataGen’s core is a Domain Specific Language (DSL) that was created to specify datasets. This language suffered several updates: repeating fields (with no limit), fuzzy fields (statistically generated), lists, high order functions over lists, custom made transformation functions. The final result is a diversified algebra that allows the generation of very complex datasets coping with very convoluted requirements. Throughout the paper, several examples of the possibilities will be given.

After generating a dataset, DataGen gives the user the possibility to generate a RESTFul data API with it, creating a running prototype.

DataGen is available online for public use, try it now: https://datagen.di.uminho.pt/

Built With

The front end of the application is built mainly using:

• Bootstrap
• JQuery
• Vue

The back end is built using the following frameworks and tools:

• MongoDB
• Node.js
• PEG.js
• Strapi

Getting Started

The installation process of the application is relatively simple, assuming you meet the requirements.

Prerequisites

• docker
• docker-compose

Installation

1. Clone the repository

   $ git clone https://github.com/wurzy/DataGen.git

2. Navigate to the repository
3. Create the service and the containers

   $ docker-compose up -d --build

4. Open http://localhost:12080/ (default port on the docker-compose file)
5. (Recommended) Create an admin account on the website
   1. Register a new account using the UI
   2. Open the mongo shell on the container

      $ docker exec -it mongo bash

      $ mongo

   3. Select the 'LEI2021' collection

      $ use LEI2021

   4. Change the 'nivel' field from 'user' to 'admin' on the created account

      $ db.users.updateOne({"email": "your_email_here"}, {$set: {"nivel": "admin"}});

   5. Exit the container and its bash

      $ exit

      $ exit

6. The application is now ready to use

License

Distributed under the MIT License. See LICENSE for more information.

Contacts

• Filipa Santos - LinkedIn - GitHub
• Hugo Cardoso - LinkedIn - GitHub
• João Cunha - LinkedIn - GitHub
• Válter Carvalho - LinkedIn - GitHub

And the supervisor of the project:

• José Ramalho - LinkedIn - GitHub

Useful Links

Localhost Build Link: https://github.com/Abjiri/DataGen-From-Schemas

Live Build Link: https://github.com/wurzy/DataGen

Official University of Minho Repository Entry: http://repositorium.sdum.uminho.pt/handle/1822/73506

Official OASIcs paper (Volume 94, SLATE 2021): https://drops.dagstuhl.de/opus/volltexte/2021/14423/

DataGen is available online for public use, try it now: https://datagen.di.uminho.pt/