/csv-consumer

A simple app that consumes CSV's in a specified format and allows the data to be queried

Primary LanguageJavaScript

CSV Consumer

Build Status Maintenance License: MIT

This app is a backend over HTTP that consumes CSV's and allows users to retrieve CSV entity history based on timestamp.

Pre-requisities:

Some key things to set up / understand to use this app:

Downloading

$ git clone https://github.com/mayankamencherla/csv-consumer.git

Setup Locally

To get the app working locally, or to run test cases, follow the instructions below. After setting up the app, details on each API and how to use it can be found below in the API's available on this app section. If any of the commands below are denied due to a permission error, please prepend a sudo to the command.

  1. Navigate to the app's root directory

  2. Run the following command to install all the dependencies:

$ npm install
  1. Create a .env file in the root directory
$ touch .env

  1. Change the values of the environment variables in .env: and ensure that the MONGODB_URI is set to your MONGODB DB URI.

  2. Run the app on localhost by typing the following command:

$ npm start

  1. Head over to localhost:3000 on your browser to see Success = true

  2. To use this app on heroku, please head over to heroku

Environment variables

Environment variables are picked up from the .env file, which must be created in the app's root directory.

Some key environment variables are listed and explained below:

  1. MONGODB_URI: The URI that points to your MongoDB installation

CSV Format

Each Row is comprised of:
 - object_id: is a unique identifier for a specific object.
 - object_type: denotes the object type.
 - timestamp: needs no explanation
 - object_changes: serialized json comprised of properties that changed at **timestamp**, and their accompanying values.

Example of the accepted format of the CSV

object_id,object_type,timestamp,object_changes
1,Order,1484730554,"{\"customer_name\":\"Jack\",\"customer_address\":\"Trade St.\",\"status\":\"unpaid\"}"
2,Order,1484730623,"{\"customer_name\":\"Sam\",\"customer_address\":\"Gecko St.\",\"status\":\"unpaid\"}"
1,Product,1484731172,"{\"name\":\"Laptop\",\"price\":2100,\"stock_levels\":29}"
1,Order,1484731481,"{\"status\":\"paid\",\"ship_date\":\"2017-01-18\",\"shipping_provider\":\"DHL\"}"
2,Product,1484731671,"{\"name\":\"Microphones\",\"price\":160,\"stock_levels\":1500}"
1,Invoice,1484731920,"{\"order_id\":7,\"product_ids\":[1,5,3],\"status\":\"unpaid\",\"total\":2500}"
1,Invoice,1484732821,"{\"status\":\"paid\"}"

API's available on this app

This app supports 2 API's currently

  1. POST /csv

    • Allows your to upload your CSV via to be stored in the DB
    • The file needs to be uploaded via the file post parameter

  2. GET /query