Bittensor

Internet-scale Neural Networks

Bittensor is a mining network, similar to Bitcoin, that includes built-in incentives designed to encourage computers to provide access to machine learning models in an efficient and censorship-resistant manner. These models can be queried by users seeking outputs from the network, for instance; generating text, audio, and images, or for extracting numerical representations of these input types. Under the hood, Bittensor’s economic market, is facilitated by a blockchain token mechanism, through which producers (miners) and the verification of the work done by those miners (validators) are rewarded. Miners host, train or otherwise procure machine learning systems into the network as a means of fulfilling the verification problems defined by the validators, like the ability to generate responses from prompts i.e. “What is the capital of Texas?.

The token based mechanism under which the miners are incentivized ensures that they are constantly driven to make their knowledge output more useful, in terms of speed, intelligence and diversity. The value generated by the network is distributed directly to the individuals producing that value, without intermediaries. Anyone can participate in this endeavour, extract value from the network, and govern Bittensor. The network is open to all participants, and no individual or group has full control over what is learned, who can profit from it, or who can access it.

To learn more about Bittensor, please read our paper.

Install

There are three ways to install Bittensor

Through the installer:

$ /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/opentensor/bittensor/master/scripts/install.sh)"

With pip:

$ pip3 install bittensor

From source:

$ git clone https://github.com/opentensor/bittensor.git
$ python3 -m pip install -e bittensor/

Using Conda (recommended for Apple M1):

$ conda env create -f ~/.bittensor/bittensor/scripts/environments/apple_m1_environment.yml
$ conda activate bittensor

To test your installation, type:

$ btcli --help

or using python

import bittensor

CUDA

If you anticipate using PoW registration for subnets or the faucet (only available on staging), please install cubit as well for your version of python. You can find the Opentensor cubit implementation and instructions here.

For example with python 3.10:

pip install https://github.com/opentensor/cubit/releases/download/v1.1.2/cubit-1.1.2-cp310-cp310-linux_x86_64.whl

Wallets

Wallets are the core ownership and identity technology around which all functions on Bittensor are carried out. Bittensor wallets consists of a coldkey and hotkey where the coldkey may contain many hotkeys, while each hotkey can only belong to a single coldkey. Coldkeys store funds securely, and operate functions such as transfers and staking, while hotkeys are used for all online operations such as signing queries, running miners and validating.

Wallets can be created in two ways.

Using the python-api

import bittensor
wallet = bittensor.wallet()
wallet.create_new_coldkey()
wallet.create_new_hotkey()
print (wallet)
"Wallet (default, default, ~/.bittensor/wallets/)"

Or using btcli

Use the subcommand wallet or it's alias w:

$ btcli wallet new_coldkey
    Enter wallet name (default):      

    IMPORTANT: Store this mnemonic in a secure (preferably offline place), as anyone who has possession of this mnemonic can use it to regenerate the key and access your tokens. 
    The mnemonic to the new coldkey is:
    **** *** **** **** ***** **** *** **** **** **** ***** *****
    You can use the mnemonic to recreate the key in case it gets lost. The command to use to regenerate the key using this mnemonic is:
    btcli w regen_coldkey --mnemonic post maid erode shy captain verify scan shoulder brisk mountain pelican elbow

$ btcli wallet new_hotkey
    Enter wallet name (default): d1
    Enter hotkey name (default): 

    IMPORTANT: Store this mnemonic in a secure (preferably offline place), as anyone who has possession of this mnemonic can use it to regenerate the key and access your tokens. 
    The mnemonic to the new hotkey is:
    **** *** **** **** ***** **** *** **** **** **** ***** *****
    You can use the mnemonic to recreate the key in case it gets lost. The command to use to regenerate the key using this mnemonic is:
    btcli w regen_hotkey --mnemonic total steak hour bird hedgehog trim timber can friend dry worry text

In both cases you should be able to view your keys by navigating to ~/.bittensor/wallets or viewed by running btcli wallet list

$ tree ~/.bittensor/
    .bittensor/                 # Bittensor, root directory.
        wallets/                # The folder containing all bittensor wallets.
            default/            # The name of your wallet, "default"
                coldkey         # You encrypted coldkey.
                coldkeypub.txt  # Your coldkey public address
                hotkeys/        # The folder containing all of your hotkeys.
                    default     # You unencrypted hotkey information.

Your default wallet Wallet (default, default, ~/.bittensor/wallets/) is always used unless you specify otherwise. Be sure to store your mnemonics safely. If you lose your password to your wallet, or the access to the machine where the wallet is stored, you can always regenerate the coldkey using the mnemonic you saved from above.

$ btcli wallet regen_coldkey --mnemonic **** *** **** **** ***** **** *** **** **** **** ***** *****

Using the cli

The Bittensor command line interface (btcli) is the primary command line tool for interacting with the Bittensor network. It can be used to deploy nodes, manage wallets, stake/unstake, nominate, transfer tokens, and more.

Basic Usage

To get the list of all the available commands and their descriptions, you can use:

btcli --help

usage: btcli <command> <command args>

bittensor cli v{bittensor.__version__}

commands:
  subnets (s, subnet) - Commands for managing and viewing subnetworks.
  root (r, roots) - Commands for managing and viewing the root network.
  wallet (w, wallets) - Commands for managing and viewing wallets.
  stake (st, stakes) - Commands for staking and removing stake from hotkey accounts.
  sudo (su, sudos) - Commands for subnet management.
  legacy (l) - Miscellaneous commands.

Example Commands

Viewing Senate Proposals

btcli root proposals

Viewing Senate Members

btcli root list_delegates

Viewing Proposal Votes

btcli root senate_vote --proposal=[PROPOSAL_HASH]

Registering for Senate

btcli root register

Leaving Senate

btcli root undelegate

Voting in Senate

btcli root senate_vote --proposal=[PROPOSAL_HASH]

Miscellaneous Commands

btcli legacy update
btcli legacy faucet

Managing Subnets

btcli subnets list
btcli subnets create

Managing Wallets

btcli wallet list
btcli wallet transfer

Note

Please replace the subcommands and arguments as necessary to suit your needs, and always refer to btcli --help or btcli <command> --help for the most up-to-date and accurate information.

For example:

btcli subnets --help

usage: btcli <command> <command args> subnets [-h] {list,metagraph,lock_cost,create,register,pow_register,hyperparameters} ...

positional arguments:
  {list,metagraph,lock_cost,create,register,pow_register,hyperparameters}
                        Commands for managing and viewing subnetworks.
    list                List all subnets on the network.
    metagraph           View a subnet metagraph information.
    lock_cost           Return the lock cost to register a subnet.
    create              Create a new bittensor subnetwork on this chain.
    register            Register a wallet to a network.
    pow_register        Register a wallet to a network using PoW.
    hyperparameters     View subnet hyperparameters.

options:
  -h, --help            show this help message and exit

Post-Installation Steps

To enable autocompletion for Bittensor CLI, run the following commands:

btcli --print-completion bash >> ~/.bashrc  # For Bash
btcli --print-completion zsh >> ~/.zshrc    # For Zsh
source ~/.bashrc  # Reload Bash configuration to take effect

The Bittensor Package

The bittensor package contains data structures for interacting with the bittensor ecosystem, writing miners, validators and querying the network. Additionally, it provides many utilities for efficient serialization of Tensors over the wire, performing data analysis of the network, and other useful utilities.

Wallet: Interface over locally stored bittensor hot + coldkey styled wallets.

import bittensor
# Bittensor's wallet maintenance class.
wallet = bittensor.wallet() 
# Access the hotkey
wallet.hotkey 
# Access the coldkey
wallet.coldkey ( requires decryption )
# Sign data with the keypair.
wallet.coldkey.sign( data )

Subtensor: Interfaces with bittensor's blockchain and can perform operations like extracting state information or sending transactions.

import bittensor
# Bittensor's chain interface.
subtensor = bittensor.subtensor() 
# Get the chain block
subtensor.get_current_block()
# Transfer Tao to a destination address.
subtensor.transfer( wallet = wallet, dest = "xxxxxxx..xxxxx", amount = 10.0)
# Register a wallet onto a subnetwork
subtensor.register( wallet = wallet, netuid = 1 )

Metagraph: Encapsulates the chain state of a particular subnetwork at a specific block.

import bittensor
# Bittensor's chain state object.
metagraph = bittensor.metagraph( netuid = 1 ) 
# Resync the graph with the most recent chain state
metagraph.sync()
# Get the list of stake values
print ( metagraph.S )
# Get endpoint information for the entire subnetwork
print ( metagraph.axons )
# Get the hotkey information for the miner in the 10th slot
print ( metagraph.hotkeys[ 10 ] )
# Sync the metagraph at another block
metagraph.sync( block = 100000 )
# Save the metagraph
metagraph.save()
# Load the same
metagraph.load()

Synapse: Responsible for defining the protocol definition between axon servers and dendrite clients

class Topk( bittensor.Synapse ):
    topk: int = 2  # Number of "top" elements to select
    input: bittensor.Tensor = pydantic.Field(..., allow_mutation=False)  # Ensure that input cannot be set on the server side. 
    v: bittensor.Tensor = None
    i: bittensor.Tensor = None

def topk( synapse: Topk ) -> Topk:
    v, i = torch.topk( synapse.input.deserialize(), k = synapse.topk ) 
    synapse.v = bittensor.Tensor.serialize( v )
    synapse.i = bittensor.Tensor.serialize( i )
    return synapse

# Attach the forward function to the axon and start.
axon = bittensor.axon().attach( topk ).start()

Axon: Serves Synapse protocols with custom blacklist, priority and verify functions.

import bittensor

class MySynapse( bittensor.Synapse ):
    input: int = 1
    output: int = None

# Define a custom request forwarding function
def forward( synapse: MySynapse ) -> MySynapse:
    # Apply custom logic to synapse and return it
    synapse.output = 2
    return synapse

# Define a custom request verification function
def verify_my_synapse( synapse: MySynapse ):
    # Apply custom verification logic to synapse
    # Optionally raise Exception

# Define a custom request blacklist function
def blacklist_my_synapse( synapse: MySynapse ) -> bool:
    # Apply custom blacklist 
    # return False ( if non blacklisted ) or True ( if blacklisted )

# Define a custom request priority function
def prioritize_my_synape( synapse: MySynapse ) -> float:
    # Apply custom priority
    return 1.0 

# Initialize Axon object with a custom configuration
my_axon = bittensor.axon(config=my_config, wallet=my_wallet, port=9090, ip="192.0.2.0", external_ip="203.0.113.0", external_port=7070)

# Attach the endpoint with the specified verification and forwarding functions  
my_axon.attach(
    forward_fn = forward_my_synapse, 
    verify_fn=verify_my_synapse,
    blacklist_fn = blacklist_my_synapse,
    priority_fn = prioritize_my_synape
).start()

Dendrite: Inheriting from PyTorch's Module class, represents the abstracted implementation of a network client module designed to send requests to those endpoints to receive inputs.

Example:

dendrite_obj = dendrite( wallet = bittensor.wallet() )
# pings the axon endpoint
await d( <axon> )
# ping multiple axon endpoints
await d( [<axons>] ) 
# Send custom synapse request to axon.
await d( bittensor.axon(), bittensor.Synapse() ) 
# Query all metagraph objects.
await d( meta.axons, bittensor.Synapse() )

Setting weights on root network

Use the root subcommand to access setting weights on the network across subnets.

btcli root weights --wallet.name <coldname> --wallet.hotkey <hotname>
Enter netuids (e.g. 0, 1, 2 ...):
# Here enter your selected netuids to set weights on
1, 2

>Enter weights (e.g. 0.09, 0.09, 0.09 ...): 
# These do not need to sum to 1, we do normalization on the backend.
# Values must be > 0
0.5, 10

Normalized weights: 
        tensor([ 0.5000, 10.0000]) -> tensor([0.0476, 0.9524])

Do you want to set the following root weights?:
  weights: tensor([0.0476, 0.9524])
  uids: tensor([1, 2])? [y/n]: 
y

⠏ 📡 Setting root weights on test ...

Bittensor Subnets API

This guide provides instructions on how to extend the Bittensor Subnets API, a powerful interface for interacting with the Bittensor network across subnets. The Bittensor Subnets API facilitates querying across any subnet that has exposed API endpoints to unlock utility of the Bittensor decentralized network.

The Bittensor Subnets API consists of abstract classes and a registry system to dynamically handle API interactions. It allows developers to implement custom logic for storing and retrieving data, while also providing a straightforward way for end users to interact with these functionalities.

Core Components

APIRegistry: A central registry that manages API handlers. It allows for dynamic retrieval of handlers based on keys.
SubnetsAPI (Abstract Base Class): Defines the structure for API implementations, including methods for querying the network and processing responses.
StoreUserAPI & RetrieveUserAPI: Concrete implementations of the SubnetsAPI for storing and retrieving user data.

Implementing Custom Subnet APIs

To implement your own subclasses of bittensor.SubnetsAPI to integrate an API into your subnet.

Inherit from SubnetsAPI: Your class should inherit from the SubnetsAPI abstract base class.
Implement Required Methods: Implement the prepare_synapse and process_responses abstract methods with your custom logic.

That's it! For example:

import bittensor

class CustomSubnetAPI(bittensor.SubnetsAPI):
    def __init__(self, wallet: "bittensor.wallet"):
        super().__init__(wallet)
        # Custom initialization here

    def prepare_synapse(self, *args, **kwargs):
        # Custom synapse preparation logic
        pass

    def process_responses(self, responses):
        # Custom response processing logic
        pass

Release

The release manager should follow the instructions of the RELEASE_GUIDELINES.md document.

Contributions

Please review the contributing guide for more information before making a pull request.

License

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Acknowledgments

learning-at-home/hivemind

crazydevlegend/bittensor