Pre-load price data
Closed this issue · 3 comments
Currently, when a dynamically priced order is created, Mostro fetches a quote from Yadio to check that the provided fiat amount yields an amount in sats within the values set in the config file as max_order_amount and min_payment_amount. This procedure requires a network call, which, based on my local measurements, takes around 2 seconds to complete. Since both limits of the range are checked, this delay is further duplicated with range orders, resulting in approximately 4 seconds for this processing alone.
Here are a couple of measurement samples:
2024-07-07T17:35:22.958910Z INFO mostrod::app::order: >> Quote at 0: 8790 in 2.065589375s
2024-07-07T17:35:25.007952Z INFO mostrod::app::order: >> Quote at 1: 179318 in 2.048965875s
2024-07-07T17:36:23.608319Z INFO mostrod::app::order: >> Quote at 0: 8790 in 2.01362875s
2024-07-07T17:36:25.557908Z INFO mostrod::app::order: >> Quote at 1: 35160 in 1.949496833s
We could improve this with range orders by getting a quote only once, extracting the price and using it to check against the opposite limit.
Or we could also introduce a new table which we'd use to store the price of BTC in all fiat currencies tracked by Yadio. There's an API endpoint for that at https://api.yadio.io/exrates/BTC. Here's a sample response:
{"BTC":{"BTC":1,"AED":211246.58,"ALL":5333413.47,"ANG":103694.46,"AOA":50102764.87,"ARS":80873960.91,"AUD":85451.86,"AZN":97772.71,"BDT":6760554.97,"BGN":103978.12,"BHD":21675.87,"BIF":165605558.36,"BMD":57513.36,"BOB":397573.36,"BRL":313303.86,"BWP":805277.93,"BYN":188291.79,"BZD":115974.14,"CAD":78487.56,"CDF":163691664.89,"CHF":51624.4,"CLP":53525416.94,"CNY":418265.92,"COP":230188367.19,"CRC":30760722.56,"CUP":19554542.91,"CZK":1348517.05,"DJF":10244507.82,"DKK":396806.94,"DOP":3401433.49,"DZD":7723431.24,"EGP":2764632.78,"ETB":5809331.65,"EUR":53190.94,"GBP":45032.1,"GEL":158281.89,"GHS":886047.86,"GNF":495443979.97,"GTQ":446858,"HKD":449253.55,"HNL":1425054.4,"HUF":20985075.92,"IDR":941424279.81,"ILS":214542.84,"INR":4802380.64,"IRR":34067464553.37,"IRT":3406746455.34,"ISK":7930517.42,"JMD":9019034.74,"JOD":40759.72,"JPY":9283438.27,"KES":7376088.61,"KGS":4925576.56,"KRW":81317513.09,"KZT":27636272.45,"LBP":5147445854.62,"LKR":17491518.62,"MAD":566762.83,"MGA":258695911.56,"MLC":66286.56,"MRU":2278491.77,"MXN":1031463.95,"MYR":271304.44,"NAD":1043175.51,"NGN":88489355.83,"NIO":2118122.78,"NOK":610540.45,"NPR":7685069.83,"NZD":93793.88,"PAB":57513.36,"PEN":218870.38,"PHP":3369413.9,"PKR":16018203.67,"PLN":229333.61,"PYG":433831076.53,"QAR":209793.44,"RON":266352.49,"RSD":6225763.87,"RUB":5139947.26,"RWF":75276353.08,"SAR":215720.71,"SEK":607061.01,"SGD":77612.79,"SYP":848322082.19,"THB":2096573.56,"TND":179572.53,"TRY":1891484.02,"TTD":389585.91,"TWD":1887128.88,"TZS":152757563.06,"UAH":2650804.46,"UGX":213193898.76,"USD":57513.36,"UYU":2304576.73,"UZS":726042065.5,"VES":2342114.49,"VND":1519222319.22,"XAF":34891045.71,"XAG":1869.04,"XAU":24.36,"XOF":34891045.71,"XPT":58.5,"ZAR":1068724.1},"base":"BTC","timestamp":1720543203583}
This response is ~1.6KB. If we schedule this to run every 5 minutes that would result in a total daily data transfer of 476 KB. Which is acceptable IMO (provided that the API doesn't rate limit us) and would yield in a ~4 second reduction in wait time for the client when posting an order.
I totally agree that this is not a good practice and it slows down the order creation, I'm in favor of your proposal of saving the fiat prices in our database and update it let's say each 5 minutes.
This database values are good to be used as a reference on order creation to check limits, but no when the taker takes the order because we need real time price, so the take order part shouldn't be changed.
Ok this has been implemented in PR 329. The original idea of introducing a database table for this has been ditched for several reasons:
- A price data table wouldn't have a relationship with anything else stored in the database
- We don't need (at this point) to keep a long history of price data, which is something that a db would be good for
- Keeping data in memory is even better in terms of speed
With this in place, new measurements of the quote retrieval procedure for ranged orders look like the following:
2024-07-14T03:05:58.508829Z INFO mostrod::app::order: >> Quote at 0: 1683 in 40.5µs
2024-07-14T03:05:58.508868Z INFO mostrod::app::order: >> Quote at 1: 3366 in 13.167µs
2024-07-14T03:06:47.208367Z INFO mostrod::app::order: >> Quote at 0: 3366 in 59.458µs
2024-07-14T03:06:47.208414Z INFO mostrod::app::order: >> Quote at 1: 5050 in 16.5µs
Closed by #329