obecto/perper

Add sample for unit testing Perper applications

branimirangelov opened this issue ยท 4 comments

Introduce a library for unit testing of Perper application. One approach is with creating pre-defined data for simulating stream / module interactions. This data can be generated in a pattern as VCR gem:

  1. Live mode execution -> in this mode the test has direct access to remote ignite cluster (e.g. dev cluster) and interacts with it. During the interaction, recording is generated for the notifications and the ignite thin client cache responses. This recordings are saved for later use.

  2. Playback mode execution -> in this mode the test executes without any remote access. All notifications and responses are simulated (played back) from the recordings in step 1.

Proposal after discussing the topic:

  1. Unit testing.
    Mocks of all needed interfaces such as IState, IContext, etc. so that the unit tester can pass them to the tested function.
    Mock IContext implementation allows for registering additional functions for CallFunctionAsync and StreamFunctionAsync.
    Mock IStream implementation does not necessarily need to support filtering and replaying in the first version.
    Could allow iterating created streams/agents.
    Recording of results from a real system in order to replay them in the unittested environment could be added at a later point.

    Sample code:
    public async Task Test_Sample()
    {
        var context = new MockContext();
        context.RegisterFunction("CalledFunction", (object? _params) => 10);
    
        var parameters = PerperHelper.ConvertParameters<dynamic>(new { Count = 5 });
    
        var result = await TestedClass.TestedFunction(parameters, context, default(CancellationToken));
    
        Assert.AreEqual(result, 5 * 10);
    }
    // Elsewhere
    [FunctionName("TestedFunction")]
    public async Task<int> TestedFunction(dynamic parameters, IContext context, CancellationToken cancellationToken)
    {
        var sum = 0;
        for (var i = 0; i < parameters.Count; i ++)
        {
            sum += await context.CallFunctionAsync("CalledFunction", null);
        }
        return sum;
    }
  2. Integration testing.
    Provides unmocked instances of IContext, IState, etc., and runs a WebJobs host, so that the testing framework can call into Perper.
    Might allow for registering fake agents and functions and iterating created streams or agents, similar to unit tests above.
    Could easily replace the current entrypoint system that uses PERPER_ROOT_AGENT.

    Sample code:
    public async Task IntegrationTest_Sample()
    {
        var environment = await PerperEnvironment.StartWithWebHost();
    
        var result = await environment.Context.CallFunctionAsync("TestedFunction", new { Count = 5 });
    
        Assert.AreEqual(result, 5 * 10);
    }
    // TestedFunction as in previous sample, real CalledFunction implementation happens to be same as one in the unit test

Note: While (1) was implemented in 0.6, the code was subsequently dropped in the 0.7 refactor, and is still missing in 0.8.
However, (2) should be pretty much implemented with 1308db7.

@branimirangelov @aph5nt ~ I think I spotted some discussions in the Apocryph Discord that (1)--unit testing with mocked state, stream, and context--might not be actually necessary, due to the interaction with Ignite being the main thing one would like to test. Am I right in thinking that?

@bojidar-bg - this is correct. For Perper apps it seems that component integration test approach using fabric as a medium is a better fit.

Sweet! In this case we pretty much have all that's needed implemented for 0.8 with the PerperStartup changes.
Tagging for documentation as we would want to have a sample for this eventually. ๐Ÿ˜ƒ