Transaction support

Question

Transaction support

Ramblurr opened this issue 4 years ago · 4 comments

A library like this isn't complete without some support for transactions.

One first step, that might also be sufficient, is to expose a with-transaction macro that wraps the next.jdbc's with-transaction macro.

Reading:

Answer 1 · 2020-08-25T12:41:24.000Z

From Slack:

kwrooijen
6:21 PM
I think we need to do the following things. what are your thoughts?

Think of a nice structure for aggregating the executions in data. If a more functional way is desired we can create honeysql-like wrapper functions to build the structure

We will need a transaction-connection dynamic definition which will take precedence over any other connection (for the binding in transact)

In the exception handler we need to check if transaction is set, if it is then we need to throw the data instead of returning it

Create a transaction function in factory. This function will take the datastructure and execute each one in sequence. If any fail, extract the data with ex-data and return that with the pairing identifier key. (edited)

Casey
6:36 PM
Sounds good

6:37
I find it helpful to write an example of what I think the user api would look like, I suppose we could do that with the elixir account transfer example

Answer 2 · 2020-08-26T09:20:59.000Z

Going straight for the complicated case: running multiple step pipelines in a transaction.

I modeled this example after the Elixir Ecto article linked above.

It represents a multi step pipeline in a domain where each step could produce its own errors. It represents the stages, results, and errors as data with functions, no macros necessary.

;; given this model 
(def model-account
  [:map
   [:account/id {:primary-key true} int?]
   [:account/balance int?]
   ])

;; define the functions in the pipeline
(defn retrieve-accounts [a1-id a2-id]
  (fn []
    (let [accounts {a1-id (q/find-by :account/id a1-id)
                    a2-id (q/find-by :account/id a2-id)}]
      (if (every? some? accounts)
        (vals accounts)
        {:error [:account-not-found (mapv first (filter (fn [[_ v]] (nil? v)) accounts))]}))))

(defn verify-balances [transfer-amount]
  ;; each pipeline fn is passed a map of steps executed thus far
  (fn [{[sender recipient] :retrieve-accounts}]
    (if (< (:account/balance sender) transfer-amount)
      [sender recipient transfer-amount]
      {:error [:balance-too-low sender]})))

(defn subtract-from-sender [{[sender recipient verified-amount] :verify-balances}]
  (q/save! (changeset sender
                      (update sender :account/balance #(- % verified-amount)))))

(defn add-to-recipient [{[sender recipient verified-amount] :verify-balances}]
  (q/save! (changeset recipient
                      (update recipient :account/balance #(+ % verified-amount)))))

(defn transfer-money [sender-id recipient-id amount]
  [[:retrieve-accounts (retrieve-accounts sender-id recipient-id)]
   [:verify-balances (verify-balances amount)]
   [:subtract-from-sender subtract-from-sender]
   [:add-to-recipient add-to-recipient]])

;; execute the pipeline in a transaction
(exec-tx-pipeline (transfer-money 1 2 100))

;; => returns an ok map with a vector containing the results of each step in order
{:ok [[:retrieve-accounts [{:account/id 1 :account/balance 500} {:account/id 2 :account/balance 10}]]
      [:verify-balances [{:account/id 1 :account/balance 500} {:account/id 2 :account/balance 10} 100]]
      [:subtract-from-sender ...]
      [:add-to-recipient ...]]}


;; => unless an error occurs, in which case the pipeline short circuits and returns a map containing the error, and the results of all previous steps
{:error [:verify-balances :balance-too-low]
 :results [:retrieve-accounts [{:account/id 1 :account/balance 50} {:account/id 2 :account/balance 10}]]}

As you can see the convention is that the step functions return their result or on error a map keyed with :error. Except for the functions that return the (q/save!) directly, on error here the :changeset/error will be present which the pipeline executor should handle.

I'm not totally pleased with the data format here, but I do think vectors make more sense than maps considering order is important. If you've used re-frame on the front end, you'll see a similarity between the result vectors and re-frame event vectors.

Answer 3 · 2020-08-26T09:41:03.000Z

For the simpler case, i.e., execute this single function in a transaction with no special error handling ceremony we can wrap next.jdbc's transact function. To the user it would look something like:

(defn handle-create-user [user-name email]
  (let [result] (q/transact
                 (fn []
                   (-> (changeset {:user/name name :user/email email})
                       q/save!
                       send-welcome-email!)))
       (if (some? (:tx/rollback result))
         ;; handle error
         ;; success!
         )))

For this to work q/save would throw on :changeset/errors and on sql execution errors.

Answer 4 · 2020-08-26T10:20:39.000Z

Fram Slack:

kwrooijen 11:45 AM
Then I think it's a good setup. I agree that vectors are a bit strange instead of maps, but as you pointed out we need to hold on to the order of execution. The only thing I would change is making the keywords qualified. e.g. :transaction/error :transaction/results (and I would replace :ok with :transaction/results as well, just don't include :transaction/error)

Casey 11:53 AM
{:transaction/results {:retrieve-accounts [{:account/id 1 ...}...], :verify-balances ... }
:transaction/steps [:retrieve-accounts :verify-balances :subtract-from-sender ...]}

kwrooijen 11:54 AM
That's also a nice option actually
Kind of reminds me of the changesets

Casey 11:55 AM
just had the same thought hah 😀

kwrooijen 11:55 AM
Maybe rename steps to pipeline ?

Allso the simple use case looks good too. All of these functions should be defined in gungnir.transaction. We can look at function naming later.