How to merge the four parts bounding_box_partx.json files in the Google Drive into a singe json file?

[patrolli]

I tried to merge these four json files by using this command cat part1.json part2.json part3.json part4.json > annotation.json. But when I run the train.py code, an error raised : "JSONDecodeError: Extra data: line 1 column 1118071028 (char 1118071027)".
So could some guys tell me how to concatenate these four parts json files?? Thansks a lot!!

[meifish]

When you concatenate by cat, wouldn't the extra brackets overlapping in the end of the previous file and the start of the next file? Have you tried read them into Python, and merge the data, and then save into one file? (I haven't tried out training.py yet, but just guessing based on your description.)

[patrolli]

Yes! Your guessing is right. Just using the cat command to concatenate these json files is not right, and I find a solution by using this command line:
jq -s '.[0] * .[1] * .[2] *.[3]' bounding_box_smthsmth_part1.json bounding_box_smthsmth_part2.json bounding_box_smthsmth_part3.json bounding_box_smthsmth_part4.json > bounding_box.json

[meifish]

@patrolli : That looks cool! jq seems very light-weight and perfect to the use case like this. I just tried using Python to merge them, and it took a long while to process. Thanks for the tip!

[turtleman99]

Yes! Your guessing is right. Just using the cat command to concatenate these json files is not right, and I find a solution by using this command line:
jq -s '.[0] * .[1] * .[2] *.[3]' bounding_box_smthsmth_part1.json bounding_box_smthsmth_part2.json bounding_box_smthsmth_part3.json bounding_box_smthsmth_part4.json > bounding_box.json

Thanks for your answer! I just used the command line to concatenate the four parts json files. But the shell raised an error which goes like the following

parse error: Expected separator between values at line 1, column 317718537

Thanks for any help!

[turtleman99]

Yes! Your guessing is right. Just using the cat command to concatenate these json files is not right, and I find a solution by using this command line:
jq -s '.[0] * .[1] * .[2] *.[3]' bounding_box_smthsmth_part1.json bounding_box_smthsmth_part2.json bounding_box_smthsmth_part3.json bounding_box_smthsmth_part4.json > bounding_box.json

Thanks for your answer! I just used the command line to concatenate the four parts json files. But the shell raised an error which goes like the following

parse error: Expected separator between values at line 1, column 317718537

Thanks for any help!

Fixed. The command works, thanks!

[tjinjin95]

hi, Why do I use the above method, and the result is killed?

[meifish]

@tjinjin95 : That is an indicator that the process used up all your memory.