pip install dunebuggyTo retrieve a query, all we'll need is the query_id for the public query we're interested in. In the below example we can take a look at the popular "Custom NFT Floor Tracker" query by @smaroo (The query_id below can be found in the URL).
from dunebuggy import Dune
dune = Dune()
query = dune.fetch_query(83579)query here is a DuneQuery object, we can get the pandas DataFrame for the query output bf calling df on the object
print(query.df.head())| Floor (Approx) | Time Interval | |
|---|---|---|
| 0 | 0.122649 | 2021-06-01T00:00:00+00:00 |
| 1 | 0.130000 | 2021-06-02T00:00:00+00:00 |
| 2 | 0.193455 | 2021-06-03T00:00:00+00:00 |
| 3 | 0.189000 | 2021-06-04T00:00:00+00:00 |
| 4 | 0.189930 | 2021-06-05T00:00:00+00:00 |
We can also take a look at some basic information about the returned query with query.info
print(query.info){'name': 'Custom NFT Floor Tracker',
'author': '@smaroo',
'length': 264,
'query_id': 83579,
'result_id': UUID('e5aef8a0-1453-44d1-a27b-f576ea2b3ba2'),
'job_id': UUID('ec680fa9-217f-44c5-b223-56730cd07473'),
'columns': ['Time Interval', 'Floor (Approx)']}
Some queries in Dune are "parameterized", meaning that the author exposes certain variables for the user to enter custom values. The example query (83579) happens to be parameterized, we can verify this by inspecting query.parameters
print(query.parameters)[QueryParameter(key='Enter NFT Contract Address', type='text', value='xc3f733ca98e0dad0386979eb96fb1722a1a05e69', enumOptions=None),
QueryParameter(key='Floor Time Interval', type='enum', value='Day', enumOptions=['Day', 'Hour']),
QueryParameter(key='Start Date', type='datetime', value='2021-06-01 00:00:00', enumOptions=None)]
If you'd like to run this query with your own custom parameters, all we'll need to do is take the parameters from from the initial query, change the values to what we want, and re-fetch the query.
Below we are replacing the old NFT contract address param with a new one (the contract address for BAYC)
params = query.parameters
# Replacing with contract address for BAYC
params[0].value = 'xBC4CA0EdA7647A8aB7C2061c2E118A18a936f13D'
custom_query = dune.fetch_query(83579, parameters=params)Note: You can also create a fresh set of parameters by importing QueryParameter from dunebuggy.models.query and adding the values to the new object.
from dunebuggy.models.query import QueryParameter
param_to_change = QueryParameter(
key='Enter NFT Contract Address',
value='xBC4CA0EdA7647A8aB7C2061c2E118A18a936f13D',
)
params[0] = param_to_change
custom_query = dune.fetch_query(83579, parameters=params)print(custom_query.info){'name': 'Custom NFT Floor Tracker',
'author': '@smaroo',
'length': 265,
'query_id': 83579,
'result_id': UUID('42a3c13d-5fbd-42bd-86c0-acc9adcdc803'),
'job_id': UUID('9051ebe7-862f-46d0-9999-b4645659ca56'),
'columns': ['Time Interval', 'Floor (Approx)']}
Note that the result_id and job_id here are different, this is because we ran the query with our changed params
print(custom_query.parameters)[QueryParameter(key='Enter NFT Contract Address', type='text', value='xBC4CA0EdA7647A8aB7C2061c2E118A18a936f13D', enumOptions=None),
QueryParameter(key='Floor Time Interval', type='enum', value='Day', enumOptions=['Day', 'Hour']),
QueryParameter(key='Start Date', type='datetime', value='2021-06-01 00:00:00', enumOptions=None)]
print(custom_query.df.head())| Floor (Approx) | Time Interval | |
|---|---|---|
| 0 | 0.8000 | 2021-06-01T00:00:00+00:00 |
| 1 | 0.8518 | 2021-06-02T00:00:00+00:00 |
| 2 | 0.8260 | 2021-06-03T00:00:00+00:00 |
| 3 | 0.7400 | 2021-06-04T00:00:00+00:00 |
| 4 | 0.8499 | 2021-06-05T00:00:00+00:00 |
dunebuggy also allows you to create a new using an existing Dune.com account.To login just need to pass in your username/password into the Dune object.
You can verify your login by inspecting your Dune user_id
import os
username = os.environ.get('DUNE_USERNAME')
password = os.environ.get('DUNE_PASSWORD')
dune = Dune(username=username, password=password)
# print(dune.user_id)To create a query now, all we need to do is pass in a name, query_string and dataset_id
We can construct the SQL query by using a raw sql string
query_string = "select * from ethereum.transactions\nLIMIT 100\n"Or we could use a fancy ORM-style library like pypika
from pypika import Database, Query
ethereum = Database('ethereum')
q = Query.from_(ethereum.transactions).select('*').limit(100)
query_string = q.get_sql(quote_char=None)
print(query_string)'SELECT * FROM ethereum.transactions LIMIT 100'
Dune requires us to specify a dataset_id for each of their supported blockchain datasets upon query creation. The currently supported datasets are the following:
| Blockchain Dataset | Id |
|---|---|
| ETHEREUM | 4 |
| XDAI | 6 |
| POLYGON | 7 |
| OPTIMISM_1 | 8 |
| OPTIMISM_2 | 10 |
| BINANCE | 9 |
| SOLANA | 1 |
We can access these integer codes via the DatasetId enum
from dunebuggy.models.constants import DatasetId
created_query = dune.create_query("My Query's Name", query_string, DatasetId.ETHEREUM)Our created query can be accessed like any other, you can also log into your Dune account as see it there as well!
print(created_query.df.head())| access_list | block_hash | block_number | block_time | data | from | gas_limit | gas_price | gas_used | hash | index | max_fee_per_gas | max_priority_fee_per_gas | nonce | priority_fee_per_gas | success | to | type | value | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | None | \x887c665b0c52ccace092d817e984e2e828ef59079295... | 47287 | 2015-08-07T08:50:01+00:00 | None | \xdb312d1d6a2ccc64dd94a3892928bac82b4e8c15 | 21000 | 100000000000 | 21000 | \xd3e6a2fc34066d20bb83020b1ee95b9dc7919fd242bd... | 0 | None | None | 0 | None | None | \x34bb6978c5a1ad68777ad388c6787df53903430c | None | 1000000000000000000 |
| 1 | None | \x4869e218b0a8f5784f16193ac66cbf35c4510ace0c9b... | 48698 | 2015-08-07T15:29:53+00:00 | None | \x48040276e9c17ddbe5c8d2976245dcd0235efa43 | 90000 | 57550496008 | 21000 | \x8ba39f908731171fe96ee4e700e71d170ef8e651fac7... | 0 | None | None | 0 | None | None | \xd8d0549637b65d58e7fb6cbdd11530b399d1ddac | None | 100000000000000000000 |
| 2 | None | \xab9491b62b16bd928b281a83db82483584c22aeebc0d... | 49051 | 2015-08-07T17:03:48+00:00 | None | \x8686578c4f7c75246f548299d6ffdac3b67b5cd1 | 90000 | 57178423039 | 21000 | \x57f8ba638903d6335e211eb470159587c73316788880... | 0 | None | None | 0 | None | None | \x87abffa6b80f712c852a9558120ba6611f0b5e46 | None | 45150000000000000000 |
| 3 | None | \x1f9adc2190701ca3085b28252e4f1f467d980f763dad... | 49174 | 2015-08-07T17:41:03+00:00 | None | \x18e4ce47483b53040adbab35172c01ef64506e0c | 90000 | 58589751415 | 21000 | \xb8280da44f8d35011c3f431f7d1a82213477a4e742de... | 2 | None | None | 0 | None | None | \xfb26ae2d3621829472555fbd11bb2a324b7a5c57 | None | 10000000000000000000 |
| 4 | None | \xf1f392fd197a149afe9f8843d7ba759d1a9f79d1ef62... | 49938 | 2015-08-07T21:06:21+00:00 | None | \xc6bf5b6558f2ee21f2e43d9ff9b5408a0cb89413 | 90000 | 71214529679 | 21000 | \x538e1664c12c55287c98dc5dd248f60c642cbbbd7a18... | 0 | None | None | 4 | None | None | \x33a3f479f6c3e7f91128348490d1f7e8d2a0fab5 | None | 5000000000000000000 |
To save a query to a CSV, we can take advantage of the to_csv method on our df
created_query.df.to_csv('my_test_data.csv')- Cleanup punding TODO comments
- Add support for embedding Dune graphs/ plotting w/ Dune style colors
- Add tests (lol)
- Add support for query updating
- Investigate whether dashboard support makes sense?
- Investigate whether there is a max row limit for data returned, if so, query in batches?
- Better formatting for certain returned columns (links etc..)
- Add Documentation (Sphinx or something else)
This project was inspired by the itzemstar's duneanalytics repo
README image is from the IAN Symbols dataset