n-mam/offset

Streaming to stdout

Closed this issue · 1 comments

Hello,
I want to create/build a "poor-man" backup solution with deduplication capabilities. I thought about connecting Offset with something that can directly accept input stream i.e Restic, Backy2, and can do deduplication on-the-fly. Additional features would be encryption and plenty of remote destinations.

Would it be possible to implement "stdout" output and some kind of CLI, so unattended backups would be possible?

BR!

n-mam commented

yes it should be possible since the image format can be streamed without the need to stage the image locally first. Currently streaming is supported only for ftp. Internally logic is more or less based on observer pattern where the subject can both be an event emitter as well as a listener. These subjects then can be chained together. At the lowest level we have a platform specific IO event loop which triggers async io completion notifications (socket and file based). We just need to support the dynamics of a new subject which in this case would feed the input stream to any arbitrary data ingester engine without engaging the local io event loop. This functionality should be fairly straightforward to expose. Ill check this aspect.