CHAOSS-GSoC-2020: Implementing GitLab worker & binding the GitLab API, Github API & Augur schema

Repository for Augur Microtasks. It contains a brief information about my background and experiences in the field of Data Engineering and worker development along with the list of Microtasks completed.

About Me

I am an amateur Founder skilled at Data Engineering, Android Development & Competitive Programming, currently pursuing Dual Degree (Bachelors + Masters) in Computer Science and Engineering at Indian Institute of Technology (IIT), Dhanbad. Following are my work experiences.

  • Data Engineer Intern-19 at Zendrive, Bangalore
  • Data Engineer Intern-18 at FedEx Express, Mumbai
  • Big Data Developer Intern-17 at InfoGrains, Indore
  • Founder & Full Stack Android Developer at Night Foodler, Dhanbad

Experience with Data Collector Workers

I have previously developed data collector workers for:

  • Jira
  • Salesforce
  • Elasticsearch
  • Redshift
  • Sage Intacct

with an architecture similar to that suggested in my proposal. These workers would run on a daily basis at a specific time and fetch all the data added after the script’s previous run and later push it into Google Big Query.

Link: https://github.com/abhinavbajpai2012/Data-Collector-Workers

MicroTasks

  • Microtask 0: Familiarize yourself with augur by downloading and configuring the dev branch.
  • Microtask 1: Familiarize yourself with the GitLab API and write a python program that successfully authenticates and pulls data from the API into JSON or other data files stored locally.

      Link: https://github.com/chaoss/augur/tree/dev/workers/abhinavbajpai2012

  • Microtask 2: Make a pull request to update the dev branch with the work from Microtask 1.

     Link: chaoss/augur#621