kedro-org/kedro-plugins

Improve the documentation of ManagedTableDataset

Opened this issue · 1 comments

Description

Is your feature request related to a problem? A clear and concise description of what the problem is: "I'm always frustrated when ..."

Hello Kedro community,
I have currently a problem on how my databricks.ManagedTableDataset is created (Problem of type, precision of DecimalType.
To avoid that, I want to define schema in my yaml file in order define schema should have the ManagedTableDataset in databricks.
Would you have some yaml example on how to create this schema ? (with the DecimalType if possible 🙂 ). I did not find any example, and IntegerType (a spark type) did not match anything for example.
Thanks and have a good day !

Context

Why is this change important to you? How would you use it? How can it benefit other users?

Users are struggling to use this dataset as there are limited example. I also found that the docstring has weird reference. The schema field requires user dumping a big json schema in YAML format which I find suboptimal as well.

It's not easy to figure out how to put that JSON format in yaml, so this is what I suggested temporarily to put this on catalgo.

image

Comments from the user:
image