ORC output/input support

Question

ORC output/input support

Closed this issue 5 years ago · 6 comments

tooptoop4 commented 5 years ago

Ability to write output in ORC file format

Answer 1 · 2019-02-23T20:03:12.000Z

Not planned at the moment, if anyone wishes to help write this it can be very helpful.

Answer 2 · 2019-04-01T16:28:26.000Z

It's possible now with the new generic file input

Answer 3 · 2019-04-04T11:32:10.000Z

readme does not mention anything about ORC

Answer 4 · 2019-04-04T18:39:30.000Z

Add to your metric config file:

output:
  - dataFrameName: df
    outputType: File
    format: orc
    outputOptions:
      saveMode: Overwrite
      path: file.orc
     ....

Check out metric full sample config file

Answer 5 · 2019-04-12T19:54:21.000Z

its not in the metric full sample config file?

Answer 6 · 2019-04-13T15:27:35.000Z

We haven't documented all formats possible but the file output type allows sending arbitrary formats and parameters to spark as long as it's in your classpath.
You are welcome to add an orc example + documentation it would be great!