fwaris/FsStorm

Improve testability with local F# runner

Closed this issue · 4 comments

Just like Java impl has.

I agree we need testability and CI. Maybe use the F# project scaffolding see https://github.com/fsprojects/ProjectScaffold

This is what I'm talking about:
https://storm.apache.org/documentation/Local-mode.html

I find that any non-trivial component will have a lot of issues only surfacing when you try to run it. There's nothing in FsStorm at the moment to help with that by either making the components unit-testable (Configuration argument alone for the Local.func is really not enough) or integration-testable with some kind of F# runtime to execute the topology inside the test.

I'm going to explore the latter approach in my own solution and if looks good merge it in here.

I have used a local storm single node cluster and attached a debugger to the spout/bolt instance to debug. But you are right that it is still cumbersome. A simple unit test framework will be very helpful.

I think the StreamParse python library has something like this. It emulates the IO messages that storm generates / consumes to unit test topology components.

I've tweaked things a bit and added samples of unit tests for components.