KFP Operator Support for TFX 1.15
Closed this issue · 1 comments
Describe the bug
On trying to use tfx >= 1.15 compile pipeline workflow is fine and passes but on creating a run or runconfiguration against the pipeline that workflow fails with
message: invalid character '#' looking for beginning of value
This seems to be when the pipeline is loaded from pipeline storage bucket.
To Reproduce
Us TFX 1.15.1 and deploy pipeline and runconfiguration resource
Expected behavior
Should both deploy successfully
Version and configuration
If applicable, add configuration details to help explain your problem.
Logs
Workflow has (not much else):
message: invalid character '#' looking for beginning of value
I can see on looking at the pipeline file produced the contents produced in TFX 1.14 is json but in 1.15 (change here it changes to yaml.
The code in the VAI provider in enrichJobWithSpecFromTemplateUri
does err = json.Unmarshal(buf.Bytes(), &raw)
which fails as the file contents is now yaml.
I think it is yaml because we say to produce output file resource.yaml here
It may just be a case we need to change the resource.yaml to be resource.json and it will work