barasher/go-exiftool

Does not work with big amount of files

asannikov opened this issue · 5 comments

asannikov commented

I'm looping big ~10k files using this call for each iteration:
fileInfos := et.ExtractMetadata(file)

After the ~7k loops the program hangs. I debugged a bit and found that it hangs in https://github.com/barasher/go-exiftool/blob/master/exiftool.go#L121 on the line: fmt.Fprintln(e.stdin, executeArg)

i tried to run the same file in 10k loops. Works fine. With the different files does not work.

Can it be that e.stdin is overflowed?

barasher commented

I'll try to reproduce the bug at home, this weekend.
I'll have some holidays starting next week, for a month, so I'm unfortunately not sure that it will be fixed "quickly" :(

asannikov commented

@barasher, thank you for your quick response.
This is not a problem, i'll add another interesting statistic in the meanwhile. have a good holidays!

asannikov commented

@barasher i solved that issue in the MR #9
it does nothing with big amount of files. The problem is in the overflowed token buffer.

barasher commented

@asannikov , thanks a lot !
I'm back tuesday, I'll have a look and gladly merge it then.

barasher commented

Feature added in commit 8b99a50, released in v1.2.0 tag
Once more, thank you for your contribution @asannikov !
Could you please check if this version still fix the bug you encountered ?

  • 👍1