How to make this script produce 1 file
kfkhalili opened this issue · 1 comments
kfkhalili commented
Hello sskaje,
I'd like to use your very handy script but would like to output only one merged file. My table is ORC format, and partitioned by month (int).
Is this possible?
Regards,
Khalid
sskaje commented
@kfkhalili sorry but I can't help you now, this code was written when I was working for my last company, I don't have a hadoop cluster for testing.
You can read my code, it is just recreate a table/partition and insert data to that table/partition with HiveQL.
I remember that I had a better solution handling that many files, but I can't find out what it was now :(