/tree-sitter-language-pack

A tree-sitter language pack

Primary LanguagePythonOtherNOASSERTION

Tree Sitter Language Pack

This package bundles a comprehensive collection of tree-sitter languages as both source distribution and pre-built wheels.

Installation

pip install tree-sitter-language-pack

Important Notes:

  • This package started life as a maintained and updated fork of tree-sitter-languages by Grant Jenks, and it incorporates code contributed by ObserverOfTime (see this PR).
  • This package is MIT licensed and the original package of which this is a fork has an Apache 2.0 License. Both licenses are available in the LICENSE file.
  • All languages bundled by this package are licensed under permissive open-source licenses (MIT, Apache 2.0 etc.) only - no GPL licensed languages are included.

Features

  • 100+ Languages: Support for all major programming languages and many domain-specific languages
  • Pre-built Wheels: Easy installation with no compilation required
  • Type-Safe: Full typing support for better IDE integration and code safety
  • Zero GPL Dependencies: All bundled languages use permissive licenses (MIT, Apache 2.0, etc.)

Usage

This library exposes two functions get_language and get_parser.

from tree_sitter_language_pack import get_binding, get_language, get_parser

python_binding = get_binding('python')  # this is a pycapsule object pointing to the C binding
python_lang = get_language('python')  # this is an instance of tree_sitter.Language
python_parser = get_parser('python')  # this is an instance of tree_sitter.Parser

See the list of available languages below to get the name of the language you want to use.

Available Languages:

Each language below is identified by the key used to retrieve it from the get_language and get_parser functions.

Contribution

This library is open to contribution. Feel free to open issues or submit PRs. It's better to discuss issues before submitting PRs to avoid disappointment.

Local Development

  1. Clone the repo
  2. Install the system dependencies
  3. Install the full dependencies with uv sync --no-install-project
  4. Install the pre-commit hooks with: 
    pre-commit install && pre-commit install --hook-type commit-msg
  5. Clone the vendors with uv run --no-sync scripts/clone_vendors.py
  6. Build the local extensions with PROJECT_ROOT=. uv run setup.py build_ext --inplace

Running Tests

To run the tests, execute the following command:

PROJECT_ROOT=. uv run --no-sync pytest tests

Adding a new language

Install

Some bindings are installed via UV and are added to the package dependencies in the pyproject.toml file.

To add an installed package follow these steps:

  1. Install the bindings with uv add <bindings_package_name> --no-install-project.
  2. Execute the cloning script with uv run --no-sync scripts/clone_vendors.py.
  3. Update both the literal type InstalledBindings and the installed_bindings_map dictionary in the __init .py _ file.
  4. Update the code in the init file as necessary.
  5. Build the bindings by executing: uv run --no-sync setup.py build_ext --inplace.
  6. Execute the tests (see above).
  7. If the tests pass, commit your changes and open a pull request.

Adding a Binary Wheel Language

  1. Add the language to the sources/language_definitions.json file at the repository's root. This file contains a mapping of language names to their respective repositories: 
    {
  "name": {
    "repo": "https://github.com/...",
    "branch": "master", // not mandatory
    "directory": "sub-dir/something", // not mandatory
    "generate": true, // not mandatory
  },
}
    • repo is the URL of the tree-sitter repository. This value is mandatory
    • branch the branch of the repository to check out. You should specify this only when the branch is not called main ( i.e. for master or other names, specify this).
    • directory is the directory under which there is a src folder. This should be specified only in cases where the src folder is not immediately under the root folder.
    • generate is a flag that dictates whether the tree-sitter-cli generate command should be executed in the given repository / directory combo. This should be specified only if the binding needs to be built in the repository.
  2. Update the SupportedLanguage literal type in the init.py file.
  3. Install the dev dependencies with uv sync --no-install-project -v
  4. Execute the cloning script with uv run --no-sync scripts/clone_vendors.py.
  5. Build the bindings by executing: PROJECT_ROOT=. uv run setup.py build_ext --inplace.
  6. Execute the tests (see above).
  7. If the tests pass, commit your changes and open a pull request.