GrimoireLab
GrimoireLab is a CHAOSS toolset for software development analytics. It includes a coordinated set of tools to retrieve data from systems used to support software development (repositories), store it in databases, enrich it by computing relevant metrics and making it easy to run analytics and visualizations on it.
You can learn more about GrimoireLab in the GrimoireLab tutorial, or visit the GrimoireLab website.
Metrics available in GrimoireLab are, in part, developed in the CHAOSS project. For more information regarding CHAOSS metrics, see the latest release at: https://chaoss.community/metrics/
Getting started
GrimoireLab is a set of tools, and to ease starting playing we are providing a default setup to analyze git activity for this repository. Given such set up, there are several options to run GrimoireLab:
docker-compose
Using Requirements:
- Software: git, docker client and docker compose. An example of working configuration:
root@test-68b8628f:~# git --version
git version 2.17.1
root@test-68b8628f:~# docker --version
Docker version 19.03.1, build 74b1e89
root@test-68b8628f:~# docker-compose --version
docker-compose version 1.22.0, build f46880fe
- Hardware: 2 CPUs, 8GB memory RAM and enough virtual memory for Elasticsearch
Steps:
- Clone this project:
foo@bar:~$ git clone https://github.com/chaoss/grimoirelab
- Go to
docker-compose
folder and run the following command:
foo@bar:~$ cd grimoirelab/docker-compose
foo@bar:~/grimoirelab/docker-compose$ docker-compose up -d
Your dashboard will be ready after a while at http://localhost:5601
. The waiting time depends on the amount of data to fetch from a repo, for small repositories you can expect your data to be visible in the dashboard after 10-15 minutes.
More details in the docker-compose folder.
docker run
Using Requirements:
- Software: git and docker client. An example of working configuration:
root@test-68b8628f:~# git --version
git version 2.17.1
root@test-68b8628f:~# docker --version
Docker version 19.03.1, build 74b1e89
- Hardware: 2 CPUs, 8GB memory RAM and set
Steps:
- Clone this project:
$ git clone https://github.com/chaoss/grimoirelab
- Go to the project folder and run the following command:
foo@bar:~$ cd grimoirelab
foo@bar:~/grimoirelab $ docker run -p 127.0.0.1:5601:5601 \
-v $(pwd)/default-grimoirelab-settings/projects.json:/projects.json \
-v $(pwd)/default-grimoirelab-settings/setup.cfg:/setup.cfg \
-t grimoirelab/full
Your dashboard will be ready after a while at http://localhost:5601
. The waiting time depends on the amount of data to fetch from a repo, for small repositories you can expect your data to be visible in the dashboard after 10-15 minutes.
More details in the docker folder.
GrimoireLab components
Currently, GrimoireLab toolkit is organized in the following repositories:
- Data retrieval related components:
- Perceval: retrieval of data from data sources
- Graal: source data analysis with external tools
- KingArthur: batch processing for massive retrieval
- Data enrichment related components:
- GrimoireElk: storage and enrichment of data
- SortingHat: identity management
- Data consumption related components:
- Kibiter: dashboard, downstream version of Kibana
- Sigils: visualizations and dashboards
- Manuscripts: reporting
- Platform management, orchestration, and common utils:
- Mordred: orchestration
- GrimoireLab Toolkit: common utilities
- Bestiary: web-based user interface to manage repositories and projects for Mordred
- Hatstall: web-based user interface to manage SortingHat identities
There are also some components built by the GrimoreLab community, which can be useful for you. Other related repositories are:
- GrimoireLab Tutorial
- GrimoireLab as a whole (this repository)
Contents of this repository
This repository is for stuff relevant to GrimoireLab as a whole. For example:
-
Issues for new features or bug reports that affect more than one GrimoireLab module. In this case, let's open an issue here, and when implementing the fix or the feature, let´s comment about the specific tickets in the specific modules that are used. For example, when supporting a new datasource, we will need patches (at least) in
Perceval
,GrimoireELK
and panels. We would open here the feature request (or the user story) for the whole case, an issue (and later a pull request) inPerceval
for the data retriever, same forGrimoireELK
for the enriching code, and same forpanels
for the Kibiter panels. -
Information about "coordinated releases" for most of GrimoireLab components (directory releases). Coordinated releases are snapshots (specific commits) of most of the GrimoireLab components that are expected to work together. See more information in the releases README.md file.
-
Utils (directory utils) for doing stuff relevant to GrimoireLab as a whole. Includes a script to produce Python packages for a coordinated release, etc.
-
Docker containers for showcasing GrimoireLab (directory docker). Includes dockerfiles and configuration files for the GrimoireLab containers that can be used to demo the technology, and can be the basis for real deployments. See more information in the docker README.md file.
-
If you feel more comfortable with
docker-compose
, the docker-compose folder includes instrucctions and configuration files to deploy GrimoireLab usingdocker-compose
command. -
How releases of GrimoireLab are built and tested: Building