[bug] freezer panics on missing folder
Opened this issue · 6 comments
From we can tell it looks like it's either a bug that there's a rogue folder in the data directory or there's a race condition where it's trying to read the directory before it has been fully created.
The error gets returned from here:
https://github.com/uw-labs/freezer/blob/master/dirs.go#L47
This occurs on startup so once the directory is in this state the only way to fix it is to delete the folder.
Any idea how to reproduce this?
Or failing that, an example of the error output, which might hint why it happened.
Erroring (but not panicing) I think is correct behaviour if some file already exists in there and it isn't a directory.
The other suggestion of a rogue folder shouldn't hit this because IsDir would return true.
Am I missing something?
This only occurs on startup, there is no error when the previous instance of this service exits prior to getting this error.
the error message looks like this: creating sink with URL freezer+s3://s3-bucket-name/folder?sse=aes256: '/folder' does not contain enough folders
I'm not sure how to best get replicate this, writing large numbers of events and restarting the service seems to be when we get this most.
The code is recursively checking the to the depth of 6 but if the we are say 3 folders deep and there are no more sub-folders then we will hit that condition here https://github.com/uw-labs/freezer/blob/master/dirs.go#L42
Thanks. I was looking in the wrong place, I also now suspect it's a timing issue.
One more question - Does this ever happen on startup after it's previously run okay? In other words, once the initial directory tree has been created okay?