Allow asynchronous task-based handling of lengthy transfer operations

[bjester]

Background

After a client pushes data to a server during a sync, the data requires dequeuing to reconcile the pushed data with the server's database. This dequeuing process can take a significant amount of time depending on several factors such as how much data was synced and the size of the server's database. If the request that finalizes the sync, and dequeues the data on the server, timeouts out due to the lengthy operations, the sync can appear to have failed from the client's perspective.

User story

As user utilizing Morango's synchronization features within Kolibri, I would like to sync large amounts of data while able to ensure that the process has completed successfully by avoiding false positive errors arising from lengthy syncing operations on the server.
As a developer, I would to be able to queue asynchronous tasks for lengthy operations that can be run in the background.

Deliverables

• There should exist a middleware-like architecture that handles lengthy sync operations like serialization, queuing, dequeuing, and deserialization
• The middleware architecture should utilize the Chain of responsibility OOP pattern
• The middleware should be customizable through Morango configuration
• The default middleware configuration should handle operations synchronously
• The middleware should provide enough context to the middleware handlers such that operations can be tasked asynchronously if possible
• API endpoints should call the middleware to initiate transfer operations
• API endpoints should return 200 OK response if the transfer stage matches the expected completed state after initiating operations, otherwise return 202 Accepted