Pinned Repositories
avro_sample_serde
A sample project that AVRO serializes and deserializes a record. Some logic from this will later be used in tigstep/KafkaConnectSpooldir_Avro_Kafka.
connect_transform
docker_kafka_cluster
This is the upgraded version of gten/docker-kafka-cluster (kafka version is changed from 0.9 to 2.1.0 and the start-kafka.sh is changed to work with kafka2.1.0 specific server.properties)
ecs_airflow
A project that sets up an AWS ECS infrastructure and hosts an Apachi Airflow running inside a docker container.
file_obfuscator_importer_pipeline
A simple file obfuscation/ingestion pipeline that runs on AWS with automated infrastructure deployment and execution.
kafka
Mirror of Apache Kafka
kafka_connect_spooldir_pipeline
A dockerized Confluent Kafka Cluster running on AWS EC2 instances that uses the Spooldir Connector(https://github.com/jcustenborder/kafka-connect-spooldir) to spool a directory, Avro serialize the data using the Schema Registry and publish to Kafka Brokers. This project also makes the Confluent Control Center available for visualization.
kafka_misc
This repo containes various modules/classes related to Apache Kafka development efforts
kafka_storm_pipeline
A real-time, dockerized Kafka event processor pipeline (utilizing an Apache Storm topology). The project is running in AWS with automated infrastructure deployment and execution, using Ansible.
loan_dafault_predictor
tigstep's Repositories
tigstep/kafka_storm_pipeline
A real-time, dockerized Kafka event processor pipeline (utilizing an Apache Storm topology). The project is running in AWS with automated infrastructure deployment and execution, using Ansible.
tigstep/ecs_airflow
A project that sets up an AWS ECS infrastructure and hosts an Apachi Airflow running inside a docker container.
tigstep/avro_sample_serde
A sample project that AVRO serializes and deserializes a record. Some logic from this will later be used in tigstep/KafkaConnectSpooldir_Avro_Kafka.
tigstep/connect_transform
tigstep/docker_kafka_cluster
This is the upgraded version of gten/docker-kafka-cluster (kafka version is changed from 0.9 to 2.1.0 and the start-kafka.sh is changed to work with kafka2.1.0 specific server.properties)
tigstep/file_obfuscator_importer_pipeline
A simple file obfuscation/ingestion pipeline that runs on AWS with automated infrastructure deployment and execution.
tigstep/kafka
Mirror of Apache Kafka
tigstep/kafka_connect_spooldir_pipeline
A dockerized Confluent Kafka Cluster running on AWS EC2 instances that uses the Spooldir Connector(https://github.com/jcustenborder/kafka-connect-spooldir) to spool a directory, Avro serialize the data using the Schema Registry and publish to Kafka Brokers. This project also makes the Confluent Control Center available for visualization.
tigstep/kafka_misc
This repo containes various modules/classes related to Apache Kafka development efforts
tigstep/loan_dafault_predictor
tigstep/visitor_count
A sample Spark exercise to count the visitors based on given conditions.