kochalex/tap-gitlab

tap-gitlab

This is a Singer tap that produces JSON-formatted data following the Singer spec.

This tap:

• Pulls raw data from GitLab's REST API
• Extracts the following resources from GitLab:
  • Branches
  • Commits
  • Issues
  • Projects
  • Project milestones
  • Users
  • Groups
  • Group Milestones
• Outputs the schema for each resource
• Incrementally pulls data based on the input state

Quick start

1. Install

   > pip install tap-gitlab

2. Get your GitLab access token

   • Login to your GitLab account
   • Navigate to your profile page
   • Create an access token

3. Create the config file

   Create a JSON file called config.json containing:

   • Access token you just created
   • API URL for your GitLab account. If you are using the public gitlab.com this will be https://gitlab.com/api/v3
   • Groups to track (space separated)
   • Projects to track (space separated)

   Notes:

   • either groups or projects need to be provided
   • filling in 'groups' but leaving 'projects' empty will sync all group projects.
   • filling in 'projects' but leaving 'groups' empty will sync selected projects.
   • filling in 'groups' and 'groups' will sync selected projects of those groups.

   {"api_url": "https://gitlab.com/api/v3",
    "private_token": "your-access-token",
   "groups": "myorg mygroup", 
   "projects": "myorg/repo-a myorg/repo-b",
    "start_date": "2018-01-01T00:00:00Z"}

4. [Optional] Create the initial state file

   You can provide JSON file that contains a date for the API endpoints to force the application to only fetch data newer than those dates. If you omit the file it will fetch all GitLab data

   {"branches": "2017-01-17T00:00:00Z",
   "commits": "2017-01-17T00:00:00Z",
   "issues": "2017-01-17T00:00:00Z",
   "projects": "2017-01-17T00:00:00Z",
   "project_milestones": "2017-01-17T00:00:00Z", 
   "users": "2017-01-17T00:00:00Z",
   "group_milestones": "2017-01-17T00:00:00Z"}

   Note:

   • currently, groups don't have a date field which can be tracked

5. Run the application

   tap-gitlab can be run with:

   tap-gitlab --config config.json [--state state.json]

---

Copyright © 2018 Stitch