ConduitIO/conduit-connector-sdk

Acceptance Tests: Generate structured data

lovromazgon opened this issue · 0 comments

Acceptance tests right now only generate raw data. We should support generating both raw and structured data. Raw data should contain random bytes (not necessarily a valid string as it is now), while structured data should ideally contain a few different types (e.g. int, string, float, boolean). Think about what the default behavior should be (e.g. generate raw or structured data, or both?), make the behavior configurable and let the user choose if they want to generate structured or raw data only.

Also, think about how this will impact destination tests since we are by default using the source connector to fetch the written records. Most source connectors only produce one kind of data (e.g. only raw data), so the assertion will fail if we are trying to write the other kind of data to the destination (e.g. structured data). Maybe that's just a limitation we have to live with, we can only assert that the destination is able to write the same kind of data that the source is producing.

See this comment for more info: #14 (comment)

Depends on #16.