xubo245/CarbonDataLearning

Apache CarbonData Learning

CarbonDataLearning

Apache CarbonData Learning, collect related learning materials, learning CarbonData code, and analyze CarbonData source code and so on.

1. Learning materials

• CarbonData Learning Materials.

2.source code Analysis

• CarbonData load data的源码分析
• Load支持Global_sort源码分析
• CarbonData创建表的源码分析
• Compaction
  • compaction获取segment文件大小代码分析
  • Major compaction 按大小合并的源码分析
  • Minor compaction 按segments数量合并的源码分析
• supportBoolean

汇总：https://github.com/xubo245/CarbonDataLearning/tree/master/docs/sourceCodeAnalysis

3. Code learning

• Example: Run CarbonData with Spark-shell
• 使用华为云maven镜像库
• preaggregate
• mv

汇总：https://github.com/xubo245/CarbonDataLearning/tree/master/docs/codeLearning

4.Performance Test

• concurrency Test

汇总： https://github.com/xubo245/CarbonDataLearning/tree/master/docs/performance

5. Problem

• Problem conclusion
  • CarbonData遇到的问题1之Invalid bad records location和TIMESTAMP is not a valid TIMESTAMP type
  • DEFAULT_ENABLE_AUTO_LOAD_MERGE为true时单独运行BooleanDataTypesInsertTest测试用例遇到问题
  • Compaction的minor默认compact数变了导致test错误

汇总： https://github.com/xubo245/CarbonDataLearning/tree/master/docs/problems