machine-learning-exchange/mlx

Current limitations for auto-generated pipelines for component

Tomcli opened this issue · 2 comments

Is your feature request related to a problem? Please describe.
Here is the list of limitations for the current code generation:

  1. Description is required in MLX, but optional in KFP
  2. MLX only can pass parameters as string type. If KFP component has non-string type, the pipeline will have type errors in the arguments.
  3. Component parameter names need to be sanitized as Python variable. (https://github.com/kubeflow/pipelines/blob/master/components/git/clone/component.yaml)
  4. MLX converted double quote " into single quote ' for all the parameter values. This doesn't work for some components when users need to pass in JSON string with double quote around their keys and values. (e.g. https://github.com/kubeflow/pipelines/blob/master/components/contrib/json/Combine_lists/component.yaml)

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

I think 3 and 4 should be easier to fix. 1 and 2 is nice to have but could be difficult to cover all the edge cases.