/fastdata

One-stop annotation tool. Build with Springcloud(Java) and FastAPI(Python), Spark and Flink for real-time computation.

Primary LanguageJava

Fastdata Annotation tool

Table of Contents

  1. About the Project
  2. Project Status
  3. Getting Started
    1. Getting the Source
  4. Authors

About the Project

  • Microservices of one-stop annotation tool
  • Multiple programming languages (Python, Scala, Java)
  • Spark / Flink real-time computation injection

Back to top

Project Status

Initialize project

Back to top

Getting Started

Just clone the source

Getting the Source

git clone https://github.com/lucky0604/fastdata.git

build a development environment by manual

  1. fastdata-common folder
cd fastdata-common
mvn install
  1. fastdata-auth folder
cd fastdata-auth/fastdata-auth-client 
mvn install
  1. docker-compose folder
# first , create an network
docker network create fastdata-net
docker-compose -f docker-compose.yml up -d mysql
docker-compose -f docker-compose.yml up -d redis
docker-compose -f docker-compose.yml up -d rabbitmq
  1. fastdata-center folder
cd fastdata-center/fastdata-bus
mvn package && mvn docker:build
  1. docker-compose folder
cd docker-compose
docker-compose -f docker-compose.yml -f docker-compose.nacos.yml up -d nacos
  1. fastdata-gateway folder
cd fastdata-gateway/fastdata-gateway-web
mvn package && mvn docker:build
cd fastdata-gateway/fastdata-gateway-admin
mvn package && mvn docker:build
  1. docker-compose folder
    init applications' database
docker-compose up mysql-init
  1. docker-compose folder
docker-compose -f docker-compose.yml -f docker-compose.spring-gateway.yml up -d fastdata-gateway-web
docker-compose -f docker-compose.yml -f docker-compose.spring-gateway.yml up -d fastdata-gateway-admin

issues

build bus server

  • Exception caught: java.util.concurrent.ExecutionException: com.spotify.docker.client.shaded.javax.ws.rs.ProcessingException: com.spotify.docker.client.shaded.org.apache.http.conn.HttpHostConnectException: Connect to localhost:2375 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection refused: connect -> [Help 1]

    update the docker-maven-plugin to the latest version

License

Copyright (c) 2021 Lucky

Back to top

Authors