amundsen-io/amundsen

Feature Proposal: Building Bigquery views lineage extractor

sagar-raythatha opened this issue · 2 comments

sagar-raythatha commented

Expected Behavior or Use Case

Currently lineage is not being extracted for the tools which doesn't provide lineage inherently including BigQuery. If we can extract the lineage information from views created in BigQuery, it will enable us to view the lineage in Amundsen.

Service or Ingestion ETL

Databuilder - BigQuery extractor

Possible Implementation

We can use sqllineage to get the lineage information from SQL views.

Example Screenshots:

Screen Shot 2022-11-16 at 11 34 06 AM

Context

I have built the extractor which is working fine for my organization. However, I am not sure if everyone wants that as a feature in the tool. This would be an optional extractor which can be used only the customers who wants to use this.

stale commented

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

Golodhros commented

Hi @sagar-raythatha, thanks for opening this issue!

We would love to have this as part of Amundsen, please send us a PR and we will review it timely!