americanexpress/unify-flowret

Add capability to switch off persistence

harry12345 opened this issue · 4 comments

Add capability to switch off persistence totally.
It is useful if we would like to use the flow engine for service orchestration purposes (where all services are read services).
In this case no persistence is required but performance is very important.

Thanks for your comments. In Flowret, persistence is called out for the following scenarios:

  1. At start of process, when the journey / sla files are written
  2. At the end of execution of each step
  3. At the time that the process pends or completes
  4. In addition to the above, there is also an audit trail that is persisted at the end of each step

If I understand you correctly, you would not like to have any persistence as we go along i.e. point 2 and you would not like to have any audit logging i.e. point 4. Can be done but be warned that if this is done, there will not be a record of which steps were executed and in case of a JVM failure, the process will not start from the previous / current step but from the last recorded position that may be several steps back. In your case, should not matter as you are only calling read services i.e. in other words idempotent services I assume.

Will add it to the back log. Thanks.

Yes, I would not like any persistence (point 1,2,3,4) in some cases. If there are only read service calls in the flow.
Eg: There is a separated book service and a separated review service. For example we need a query which gives back the name and author of the book and the reviews. In this case we can use a parallel flow definition which gets data from both services in the same time and generates the composed result.

I think, this persistence on/off switch should be somewhere in the flow definition because it can be different flow by flow.
For example if there is an orchestration flow which executes commands and queries in this case persistence is required.
However if there is a flow definition which executes queries only in this case persistence is not required.

With this solution it could be possible to use Flowret as a high performance API Composer engine as well.

Thank you for adding the ticket to the backlog.

Have been thinking a bit more on this. First, the persistence we are talking about is that of the orchestrator. This persistence as outlined in points 1, 2, 3, 4 is to do with saving the state of the process as we go along. The primary objective of this persistence is to be able to recover from a crash and to build an audit trail. Also to determine if the process has already been started or not. Typically this kind of persistence is not specified in the flow definition.

The other kind of persistence is that which is required on the application side. Lets say that as part of running various steps, instead of the application persisting data to a database after each step, it decides to persist when the process pends. Or it wants to persist somewhere in between. For the former, a flag at Flowret level will work while for the latter we already have a special step known as Persist which can be specified as part of flow definition. This is a hint to the application to persist the data. This kind of persistence is totally on the application side and Flowret has nothing to do with this.

Second, if you wanted no persistence for points 1, 2, 3 and 4, a really simple way would be to return void from the write method of the implementation of the Dao interface.

Using flags at Flowret level, I will implement in a few days, just running a bit busy now. The option to return void from the Dao is available to you even today. Hope this clarifies.

Have added two options. First is to disable audit logging. Second is to write process info file at pend or completion. In case you want no persistence, how that can be achieved is described in the previous post. Have updated readme as well. With this am closing the issue.