/webb-subql

Primary LanguageTypeScriptApache License 2.0Apache-2.0

Webb Logo Webb Logo

Webb SubQuery

🚀 SubQuery implementation for the Tangle Network 🚀

License Apache 2.0 Twitter Telegram Discord

Getting Started 🎉

Tangle Network is an DKG protocol that governs zk applications in a decentralized manner. This Subgraph ingests the data of the protocol and chain.

A SubQuery package defines which data The SubQuery node will index data from the Substrate blockchain, store it, and make the data available in a graphQL API.

For additional information, please refer to the Webb Official Documentation 📝. Have feedback on how to improve the webb-subql subgraph? Or have a specific question to ask? Checkout the Webb Dapp Feedback Discussion 💬.

Prerequisites

This repository makes use of yarn, Docker, nodejs, and requires version node v16. To install node.js binaries, installers, and source tarballs, please visit https://nodejs.org/en/download/.

Once node.js is installed you may proceed to install yarn:

npm install --global yarn

Great! Now your Node environment is ready! 🚀🚀

To install Docker for various machine setups please visit the Official Docker Documentation for instructions. 🐳

If you are not making use of the Docker setup you will need to install the SubQuery Node CLI:

npm install -g @subql/node

Great! Now your environment is ready! 🚀🚀

Quickstart ⚡

You can run a local SubQuery node with incredible ease and speed with Docker.

Install dependencies:

yarn install

Generate the project types:

yarn codegen

Build the project:

yarn build

Start the Docker container:

yarn start:docker

You should see you the following services starting up in the Docker container:

  • postgres
  • subquery-node
  • graphql-engine

Once initialized the container will begin syncing and ingesting the data of the archive node.

You can now visit localhost:3000 to view the graphQL playground and make queries to fetch data. For example,

query {
  blocks(first: 10, orderBy: NUMBER_ASC) {
    nodes {
      id
      number
      hash
      timestamp
      parentHash
      specVersion
      stateRoot
      extrinsicsRoot
    }
  }
}

Great! You have officially setup, started, and queried data from the Tangle Standalone testnet using SubQuery!! 🚀🚀

Development

Running Tangle Node Locally

During the development process you may want to run your own local Tangle Network to test. To do so you first need to follow the prerequisites and installation instructions outlined on the Tangle repo.

You may also make use of the scripts to automate the setup further. Please refer to the scripts directory.

Once completed, to start the local test network run:

RUST_LOG=dkg=trace ./target/release/tangle-standalone --base-path /tmp/standalone/alice --alice   --pruning=archive --rpc-cors=all &
RUST_LOG=dkg=trace ./target/release/tangle-standalone --base-path /tmp/standalone/bob --bob --pruning=archive --rpc-cors=all &
RUST_LOG=dkg=trace ./target/release/tangle-standalone --base-path /tmp/standalone/charlie --charlie --pruning=archive --rpc-cors=all

Note: Notice the usage of --pruning=archive flag this is required in order for SubQuery node to index all the chains data.

Check the connection by using the Polkadot UI app.

Great! You have officially setup, and started a 3 node local test network!

Troubleshooting

If you get the error when starting the nodes, please make sure to remove the existing blockchain data using:

rm -rf /tmp/standalone/alice/chains/local_testnet/db/full/ &
rm -rf /tmp/standalone/bob/chains/local_testnet/db/full/ &
rm -rf /tmp/standalone/charlie/chains/local_testnet/db/full/

Running SubQuery with Local Network

Now that you have your local testnet running we need to setup, and start a SubQuery node. We are going to follow the same steps mentioned in the Quickstart but before we do, we are going to update the project.yaml file to reflect the local networks chainID and endpoint. You will need update these fields under the network section.

To find the chainId of your local network, navigate to Polkadot UI app, > Settings > Metadata. The value you need is labelled Genesis Hash. Copy that value and update the project.yaml file. Next update the endpoint to either your local Docker container or local machine.

network:
  # Use Tangle network endpoint for Arana Alpha testnet see: https://polkadot.js.org/apps/?rpc=wss%3A%2F%2Fstats-dev.api.webb.tools%2Fpublic-ws#/settings/metadata
  chainId: "0x18c48b6ec39f7f3384c3b003853c40dd6f9b1501929889ae8429b9a259e7e74a"

  # using testnet archive node endpoint
  endpoint: "wss://stats-dev.api.webb.tools/public-ws"

  # if you are using docker, uncomment the below line
  # endpoint: "ws://host.docker.internal:9944"

  # if you are not using Docker and want to run locally, uncomment the below line
  # endpoint: "ws://127.0.0.1:9944"

Once updated proceed to start the SubQuery node:

Install dependencies:

yarn install

Generate the project types:

yarn codegen

Build the project:

yarn build

Start the Docker container:

yarn start:docker

Great! You have officially setup, and started your local SubQuery environment with a local test network.

Without Docker

If you do not wish to utilize Docker, you may proceed to follow these steps here.

If you do not use Docker, and want to start the graphQL playground you have to run the Query Service via subql-query to have the playground running. Then you can open the playground at localhost:3000 and use the below graphql query to test.

query {
  blocks(first: 10, orderBy: NUMBER_ASC) {
    nodes {
      id
      number
      hash
      timestamp
      parentHash
      specVersion
      stateRoot
      extrinsicsRoot
    }
  }
}

Tips for development:

Sometimes, when you make changes to the code, make sure to run yarn build to re-build the project, then your change will be applied in the Docker container. Otherwise, you won't get the latest changes.

If you get errors regarding the Postgres service in the Docker container, remove the .data folder or you can try to add the --force-clean in the command section of the subquery-node service in docker-compose file.

Deployment

You can deploy your SubQuery project to the decentralized network with the following steps:

  1. Create a new SubQuery project
  2. Run:
    • yarn install
    • yarn codegen
    • yarn build
  3. Get a refresh token from the SubQuery project UI - see detail instructions here.
  4. Publish the project and copy the CID
subql publish

This will return the IPFS CID

Uploading SupQuery project to IPFS
SubQuery Project uploaded to IPFS: QmZ3q7YZSmhwBiot4PQCK3c7Z6HkteswN2Py58gkkZ8kNd  //CID
  1. Copy the CID and input it in the managed project UI - see detail instructions here.
  2. Provide the archive node endpoint from the project.yml and click deploy

Existing Deployments

You can query the existing deployed SubQuery instance at GraphQL Playground. This instance is deployed against the the Arana Alpha Testnet.

License

Licensed under Apache 2.0 license.

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in this crate by you, as defined in the Apache 2.0 license, shall be licensed as above, without any additional terms or conditions.