sky-uk/kfp-operator

KFP Operator Support for TFX 1.15

Closed this issue · 1 comments

Describe the bug
On trying to use tfx >= 1.15 compile pipeline workflow is fine and passes but on creating a run or runconfiguration against the pipeline that workflow fails with

message: invalid character '#' looking for beginning of value

This seems to be when the pipeline is loaded from pipeline storage bucket.

To Reproduce
Us TFX 1.15.1 and deploy pipeline and runconfiguration resource

Expected behavior
Should both deploy successfully

Version and configuration
If applicable, add configuration details to help explain your problem.

Logs
Workflow has (not much else):

message: invalid character '#' looking for beginning of value

I can see on looking at the pipeline file produced the contents produced in TFX 1.14 is json but in 1.15 (change here it changes to yaml.
The code in the VAI provider in enrichJobWithSpecFromTemplateUri does err = json.Unmarshal(buf.Bytes(), &raw) which fails as the file contents is now yaml.

I think it is yaml because we say to produce output file resource.yaml here

It may just be a case we need to change the resource.yaml to be resource.json and it will work