add target cleanup steps that run regardless of pass/fail

[mimir-d]

Current state: when a target fails a test step, it is removed from the run queue and the following step plugins do not receive this target. This also implies that there is no way to clean things up on failure.

Proposal: add another (optional) list of cleanup steps that get access to the same run context as the main step group (incl step outputs). These cleanup steps will be run regardless of the execution outcome of the target test steps and failure is treated as a warning (reporting TBD).

"TestDescriptors": [
    {
        "TargetManagerName": "TargetList",
        "TargetManagerAcquireParameters": {
            "Targets": [...]
        },
        "TargetManagerReleaseParameters": {},
        "TestFetcherName": "literal",
        "TestFetcherFetchParameters": {
            "TestName": "Literal test",
            "Steps": [
                {
                    "name": "gathercmd",
                    ...
                }
            ],
            "CleanSteps": [
                {
                    "name": "gathercmd",
                    ...
                }
            ]
        }
    }
]

Concepts naming still TBD.

[mimir-d]

design iteration: make a "CleanupFetcher" that can use the same fetcher plugins as the "TestFetcher", but shares the output vars context

"TestDescriptors": [
    {
        "TargetManagerName": "TargetList",
        "TargetManagerAcquireParameters": {
            "Targets": [...]
        },
        "TargetManagerReleaseParameters": {},
        "TestFetcherName": "literal",
        "TestFetcherFetchParameters": {
            "TestName": "cleanup",
            "Steps": [
                {
                    "name": "gathercmd",
                    ...
                }
            ]
        },
        "CleanupFetcherName": "literal",
        "CleanupFetcherFetchParameters": {
            "TestName": "Literal test",
            "Steps": [
                {
                    "name": "gathercmd",
                    ...
                }
            ]
        }
    }
]

[mimir-d]

@tfg13 @xaionaro ping for opinions about the last snippet

[tfg13]

Yeah I like this, it seems more flexible and cleaner.
Can you elaborate a tiny bit on the "shares the output vars context" part?

[mimir-d]

@tfg13 yeah, so @rihter007 added the step outputs thing some time ago and it would be useful to have those available between the test steps and cleanup steps (prob needs some refactoring to make the object "test" scoped).

This would allow for stuff like make a temp folder and delete it at the end (or some target specific cleanup, etc):

"TestDescriptors": [
    {
        "TargetManagerName": "TargetList",
        "TargetManagerAcquireParameters": {
            "Targets": [...]
        },
        "TargetManagerReleaseParameters": {},
        "TestFetcherName": "literal",
        "TestFetcherFetchParameters": {
            "TestName": "test",
            "Steps": [
                {
                    "name": "gathercmd",
                    "label": "gather0",
                    "parameters": [{"binary": "mktemp"}]
                }
            ]
        },
        "CleanupFetcherName": "literal",
        "CleanupFetcherFetchParameters": {
            "TestName": "cleanup",
            "Steps": [
                {
                    "name": "gathercmd",
                    "label": "clean0",
                    "parameters": [{"binary": "rm", "args": ["-rf", "{{ strvar gather0.stdout }}"]}]
                }
            ]
        }
    }
]

note how gather0.stdout needs to be available in the cleanup block.

[tfg13]

right, yeah that makes a lot of sense.

Yeah this proposal looks good to me