[Question] Impact of WorkflowRegistry storing all definitions in memory.

[Julian-Robinson]

Hi team

We're evaluating whether Workflow-Core is suitable option for us as a workflow engine to drive an end user automation feature to allow our users to create their own simple automation steps (eg, when x happens, then do y then z). Our approach so far is that when a user sets up a workflow, we create and register a workflow definition to represent it and it seems to be working. But with this model, there will likely be many thousand Workflow Definitions as users set up their own automation rules.

One thing we've noticed is in the WorkflowRegistry implementation (below) it loads all definitions on startup and stores them in a concurrent dictionary in memory. We're wondering whether there may be a ceiling we hit here if we have a lot of definitions.

workflow-core/src/WorkflowCore/Services/WorkflowRegistry.cs

Lines 11 to 35 in ab97ce9

	public class WorkflowRegistry : IWorkflowRegistry
	{
	private readonly IServiceProvider _serviceProvider;
	private readonly ConcurrentDictionary<string, WorkflowDefinition> _registry = new ConcurrentDictionary<string, WorkflowDefinition>();
	private readonly ConcurrentDictionary<string, WorkflowDefinition> _lastestVersion = new ConcurrentDictionary<string, WorkflowDefinition>();
	public WorkflowRegistry(IServiceProvider serviceProvider)
	{
	_serviceProvider = serviceProvider;
	}
	public WorkflowDefinition GetDefinition(string workflowId, int? version = null)
	{
	if (version.HasValue)
	{
	if (!_registry.ContainsKey($"{workflowId}-{version}"))
	return default;
	return _registry[$"{workflowId}-{version}"];
	}
	else
	{
	if (!_lastestVersion.ContainsKey(workflowId))
	return default;
	return _lastestVersion[workflowId];
	}

So I guess my questions are:

• What kind of scale in terms of a rough number of definitions, would you expect Workflow core to support?
• Is our approach something that you would consider suitable for Workflow-Core?

Thanks, and we really appreciate your time looking over this.

[danielgerlag]

You could create an alternative implementation of IWorkflowRegistry that persists to a datastore and swap it out with the in-memory one.