datafold/data-diff

Add support for Clickzetta

idling11 opened this issue · 6 comments

Hello everyone, I am an engineer from Yunqi China. Yunqi Lakehouse is a fully managed data management and analysis service with a lake warehouse architecture. The Yunqi Lakehouse uses a single engine to meet the needs of data warehouse construction, data lake exploration and analysis, offline and real-time data processing and analysis, business reports and other scenarios. It meets the scalability requirements of enterprises at different stages through the separation of storage and computing, Serverless elastic computing power, and intelligent optimization features. , cost and performance needs. Built-in data integration, development and operation and maintenance, data asset management and other ready-to-use services greatly simplify data development and management work and accelerate enterprise data insights and value realization.

We now have many customers in automobile companies, securities, retail and other fields. In the next step, we will also accelerate our pace of going out of China. In our self-developed clickzetta-migration, data-diff is deeply integrated for cross-database data verification. At present, we have forked a repo from the master, developed and integrated the clickzetta driver ourselves, and verified the availability and stability of our driver in the production environment. In my opinion data-diff is a powerful and efficient python library. If the community can accept and merge the clickzetta driver, it will be a special honor for us.

If possible, I will follow the guidelines of the official documentation and submit a pull request for clickzetta driver.
Our official website link is here: https://www.singdata.com
Thank you all again and look forward to community responses.

Submit a pull request and show me the tests for your fork, and we'll be open to supporting it! We'll need a demo instance to pass our integration tests. Can you dockerize Clickzeta to make running these tests easier?

Submit a pull request and show me the tests for your fork, and we'll be open to supporting it! We'll need a demo instance to pass our integration tests. Can you dockerize Clickzeta to make running these tests easier?

Thanks for your reply. I will sort out the current code later and attach the corresponding UT. Make a pull request to git

Submit a pull request and show me the tests for your fork, and we'll be open to supporting it! We'll need a demo instance to pass our integration tests. Can you dockerize Clickzeta to make running these tests easier?

clickzetta itself is cloud native and consists of many components, such as Ingestion service, meta service, coodinator, etc. We will open an external test account and keep it open for a long time. All tests will be conducted based on this account.

I have push the pr to git. Link is #828 . BTW, Our public account is still under preparation and will be provided later, thank you

This issue has been marked as stale because it has been open for 60 days with no activity. If you would like the issue to remain open, please comment on the issue and it will be added to the triage queue. Otherwise, it will be closed in 7 days.

Although we are closing this issue as stale, it's not gone forever. Issues can be reopened if there is renewed community interest. Just add a comment and it will be reopened for triage.