/metatron-discovery

Powerful & Easy way for big data discovery

Primary LanguageTypeScriptApache License 2.0Apache-2.0

metatron-discovery-logo

[NOTICE] Respecting the community's voice, repository reset has been completed and Stars have been initialized. But Watch and Fork have also been initialized with Star, so please re-register it if you need.

Metatron Discovery is an end-to-end big data self discovery solution. To learn more about it, visit our web site. Check our blog for upcoming events and development news.

Strengths

  • Able to process HUGE data sets super FAST, powered by (optimized) Apache Druid.
  • Single solution for data analytics from easy data preparation to fast visualization.
  • Easy data analytics for everyone.

Key Features

Metatron provides:

  • Interactive dashboards with numerous preloaded charts.
  • Query(SQL) based data exploration and GUI based data wrangling.
  • Various data source connections (e.g. DBs, HIVE, or Kafka streams).
  • Geo data analysis with geospatial operations.
  • Job and data usage monitoring.
  • Metadata management.
  • 3rd party data analytics tool integration(e.g. Apache Zeppelin).
  • Fine grain access control of users and workspaces.
  • Overcomes Druid weaknesses(e.g. no data join function) without performance degradation using Metatron optimized Druid.
  • Full API support, enabling easy integration into your environment/software.
  • Multiple languages according to browser setting(Currently supporting English, Korean, Chinese).
  • Available on AWS, Azure('19.3Q).
  • Docker support for distributed version deploy('19.3Q).
  • Monitoring Metatron engine performance('19.3Q).

Screenshots

Create Dashboards


Create Charts


Analyze with Geospatial Data


Use Datasources


Manage Datasources


Manage Engine Performance (Coming Soon)


Connect Database


Prepare Origin Data


Query Monitoring


SQL in Druid


Installation

Requirements

Install the Customized Druid

Here is the link for downloading the archive.

To install the Metatron distributed Druid, simply untar the downloaded archive. And start | stop the druid with the following commands.

 $ start-single.sh | stop-single.sh

Build from source

Git clone this project.

$ git clone https://github.com/metatron-app/metatron-discovery.git

Build through Maven 3.3+.

$ mvn clean install -DskipTests

Building the whole project takes some time especially for the "discovery-frontend". Please wait a few minutes.

If the build succeeds, you can find an archive file under "discovery-distribution/target"

Start up the Metatron Discovery

Untar the archive binary file of Metatron Discovery.

$ tar zxf metatron-discovery-{VERSION}-{TIMESTAMP}-bin.tar.gz

Configuration (optional)

Metatron Discovery loads its configuration from the files under “/conf” directory by default. We already wrote some frequent configurations in the template files. For your own configuration of Metatron Discovery application, you should create a new configuration file with reference to the pre-distributed template file as belows. In the generated setting file, refer to the configuration guide and specify detailed setting information.

$ cp ./conf/application-config.templete.yaml ./conf/application-config.yaml

To configure the environment in which the server is running, you need to configure server memory or classpath settings by editing the “metatron-env.sh” file.

$ cp ./conf/metatron-env.sh.templete ./conf/metatron-env.sh

For example, if you want to use MySQL and increase the memory, you should set it as below. See the comments in the file “metatron-env.sh.templete” for more information.

export METATRON_JAVA_OPTS=-Xms4g -Xmx4g
export METATRON_DB_TYPE=mysql

Run Metatron Discovery

Initialize and run with the following command.

$ bin/metatron.sh --init start

⚠️ Cautions! --init option initialize whole data.
Add this argument only the first time or when you want to reset your development environment.

Running options are provided as well.

$ bin/metatron.sh [--config=directory] [--init] [--management] [--debug=port] {start|stop|restart|status}

To access Metatron Discovery, go to http://localhost:8180. (The default admin user account is provided as Username: admin, PW: admin.)

Using REST API

Metatron support RESTful APIs. Please refer to the following details how to using the REST API

Problems & Suggestions

This project welcomes contributions and suggestions. If you encounter any bugs or want to request new features, feel free to open an GitHub Issue in the repo so that the community can find resolutions for it. Or reports bug to our discussion forum. Although, please check before you raise an issue. That is please make sure someone else hasn’t already created an issue for the same topic.

Question

Need help using Metatron Discovery? Check our FAQs or ask away on our discussion forum! Our fellow community members will be glad to help you out! Or you can check out our user manual on our website as well.

License

Metatron Discovery is available under the Apache License V2.