/quantfury-ipfs-verification

Primary LanguageClojure

QF Data Verification

This project aims to check spread computation against IPFS of trades published by Quantfury.

Usage

Nix will automatically handle all dependencies for you. To set it up, you can follow the instructions on the official Nix website. After the installation, use the following command to run the script:

clojure -M -m quantfury-ipfs-verification.app

The CIDs and passwords of each monthly trading data archive are listed in the TRADING DATA part of the QTF & QDT Dapp Portal. The script verifies the data against the Number of trades and Spread revenue listings on the STATISTICS tab of QDT TOKENS, as they appear on the Dapp portal.

The trade data archives to be verified should be defined by adding the respective CIDs, passwords and expected results into verification_info.edn.

Note

Currently, although the IPFS archive is downloaded through a IPFS node and not simply the QTF IPFS Gateway, the expected results are still taken from the Dapp Portal and not directly by checking the Eheterum transactions made for the QDT token.

Deprecation Note

The verify.sh bash implementation of this script is now deprecated in favor of the Clojure version.

A Babashka version also exists:

bb -m quantfury-ipfs-verification.app

But please note that the Babashka version is significantly slower to run, and only left as an example of how code can be reused between Clojure and Babashka by leveraging reader conditionals.

Identified discrepancy

A discrepancy has been identified in the published trading data for April 2023.

https://etherscan.io/tx/0x2f5bdaf3ccd877f37dcc01ec6197c5a841301569396f3c1d6f54baa1e8941bc7

#	Name	Type	Data
0	ipfsHash	string	QmUW2b6ai5Bm5P18RuTCf4FnPKkWBV64EizjSJBWar9xPK
1	epochTime	uint256	1682888400
2	weiAmount	uint256	879617026970000000000
3	tokenAmount	uint256	235679400000000

https://etherscan.io/tx/0x0a5d7d1f869ab25fd700794f5ed27a1a11baaae7cbe40096c4a7e2cddef41059

#	Name	Type	Data
0	epoch	uint256	46
1	password	string	gvC,olWGO,a7Ik50pY(h

This discrepancy manifests in the form of a difference in the expected and obtained results:

FAILED: QmUW2b6ai5Bm5P18RuTCf4FnPKkWBV64EizjSJBWar9xPK Expected: {:num-trades 2356794 :sum-spread 1643828.3} Got: {:num-trades 2299968, :sum-spread 1619084.05071113M}

The root cause appears to be missing data for the last day of the month in the downloaded zip archive for April 2023:

tail qf20230501.csv -n 5
2023-04-30 00:44:55.1930,ATOM/USD,11.73455184,11.74002685,,MANUAL,102.214417,0.27981267
2023-04-30 00:44:55.3076,ETH/USDT,1901.46,1901.47,,MANUAL,4.03900161,0.02019501
2023-04-30 00:44:59.2205,ATOM/USDT,11.731,11.732,,MANUAL,4.26221124,0.00213111
2023-04-30 00:45:06.5808,FIL/USD,5.55167613,5.55838602,,MANUAL,357.45557053,1.19924407
2023-04-30 00:45:10.6554,MATIC/USDT,1,1.0001,,ORDER,1111.99364575,0.05559968

It is expected to have data until 23:59:XX, as observed in more recent files:

tail qf20230601.csv -n 5
2023-05-31 23:59:23.7890,MANA/USDT,0.463,0.4633,,ORDER,8268.73385013,1.24031008
2023-05-31 23:59:32.6360,ETH/USDT,1873.62,1873.63,,MANUAL,0.52836069,0.0026418
2023-05-31 23:59:40.0300,FIL/USDT,4.692,4.693,,MANUAL,23.58996354,0.01179498
2023-05-31 23:59:44.2975,BTC/USDT,27209.05,27209.06,,MANUAL,0.00073481,0.00000367
2023-05-31 23:59:48.7760,FTM/USDT,0.3123,0.3124,,MANUAL,647.66839378,0.03238342

This issue has been reported to QTF Support, as a result a new archive QmXb1QDbQeE4qexbE5FXD4bgNvmYupzWtr2HPSqb5SHkFG has been uploaded and linked on the QTF Portal.