crwilcox/synthtool

SynthTool (for client libraries)

This tool helps to generate and layout cloud client libraries. Synthtool runs the GAPIC (Generated API Client) Generator via Google API Artifact Manager (artman).

Installation

This tool requires Python 3.6. Either install it from python.org or use pyenv to get 3.6.

If you are using GAPIC generator synthtool.gcp.GAPICGenerator or DiscoGAPIC generator synthtool.gcp.DiscoGAPICGenerator, install Docker.

Install latest

python3 -m pip install --user --upgrade git+https://github.com/googleapis/synthtool.git

Install stable

python3 -m pip install --user --upgrade gcp-synthtool

Basic usage

To start the process of generation, clone the destination repository.

git clone git@github.com:googleapis/google-cloud-python.git
cd google-cloud-python/

Navigate to the destination directory to generate the library.

cd tasks/

Running synthtool

If a synth.py script is not present, create a new one.

You can create one from scratch or copy one from another library.

• e.g. the synth.py for the Cloud Tasks API for Python, Java, Node.js, PHP, or Ruby.

Run synthtool:

python3 -m synthtool

After synthtool runs successfully:

• Investigate the changes it made
• Run the library tests
• Commit and push the changes to a branch and open a Pull Request

Find examples below in different programming languages (Cloud Tasks API used as an example).

Python

• Clone the destination repository:

  git clone git@github.com:googleapis/google-cloud-python.git
  cd google-cloud-python/
  

• Navigate to the destination directory to generate the library:

  cd tasks/
  

• Run synthtool to generate using the existing synth.py file for the Python Client for Cloud Tasks API:

  python3 -m synthtool
  

• See the Python Contributing Guide or instructions to install dependencies, run tests, and submit a contribution.

Java

• Clone the destination repository:

  git clone git@github.com:googleapis/google-cloud-java.git
  cd google-cloud-java/
  

• Navigate to the destination directory to generate the library:

  cd google-cloud-clients/google-cloud-tasks/
  

• Run synthtool to generate using the existing synth.py file for the Google Cloud Java Client for Cloud Tasks:

  python3 -m synthtool
  

• See the Java Contributing Guide or instructions to install dependencies, run tests, and submit a contribution.

Node.js

• Clone the destination repository:

  git clone git@github.com:googleapis/nodejs-tasks.git
  cd nodejs-tasks/
  

• Run synthtool to generate using the existing synth.py file for the Google Cloud Tasks Node.js Client:

  python3 -m synthtool
  

• See the Node.js Contributing Guide or instructions to install dependencies, run tests, and submit a contribution.

PHP

• Clone the destination repository:

  git clone git@github.com:googleapis/google-cloud-php.git
  cd google-cloud-php/
  

• Navigate to the destination directory to generate the library:

  cd Tasks/
  

• Run synthtool to generate using the existing synth.py file for the Google Cloud Tasks client for PHP:

  python3 -m synthtool
  

• See the PHP Contributing Guide or instructions to install dependencies, run tests, and submit a contribution.

Ruby

• Clone the destination repository:

  git clone git@github.com:googleapis/google-cloud-ruby.git
  cd google-cloud-ruby/
  

• Navigate to the destination directory to generate the library:

  cd google-cloud-tasks/
  

• Run synthtool to generate using the existing synth.py file for the Ruby Client for Cloud Tasks API:

  python3 -m synthtool
  

• See the Ruby Contributing Guide or instructions to install dependencies, run tests, and submit a contribution.

Features

Templating

SynthTool supports template files using Jinja.

Templates are found in subdirectories of synthtool/gcp/templates/ for each language,

• e.g. the template directories for Python, Node.js, PHP, or Ruby.

You can generate and copy templates using gcp.CommonTemplates in your synth.py:

common_templates = gcp.CommonTemplates()

templates = common_templates.node_library()
s.copy(templates)

You can provide variables to templates as keyword arguments to the library generation method:

common_templates = gcp.CommonTemplates()

templates = common_templates.node_library(version=5, show_version=True, previous_versions=[1,2,3,4])

s.copy(templates)

Template files can access any values provided, e.g.

• README.md.j2

  {% if show_version %}
  The version is {{ version }}
  
  {% if previous versions is defined %}
  Previous versions:
    {% for ver in previous_versions %}
    - {{ ver }}
    {% endfor %}
  {% endif %}
  {% endif %}

You can learn more about Jinga templating in the Template Designer Documentation.

googleapis-private

SynthTool supports generation from googleapis/googleapis-private.

gapic = gcp.GAPICGenerator()

library = gapic.node_library('speech', 'v1', private=True)

2FA is required to clone a private repo.

• Using SSH: Before running Synthtool, set the environment variable AUTOSYNTH_USE_SSH to true.

The repo will be cloned using SSH.

• Using HTTPS: Generate a GitHub Personal Access Token with scope repo. Run synthtool.

When GitHub prompts for your GitHub password, provide the access token instead.

synthtool > Cloning googleapis-private.
Username for 'https://github.com': busunkim96
Password for 'https://busunkim96@github.com':

Artman Version

SynthTool uses the latest version of the Artman Docker image. You can change this by setting the environment variable SYNTHTOOL_ARTMAN_VERSION to the desired version tag.

export SYNTHTOOL_ARTMAN_VERSION=0.16.2

Local Googleapis

SynthTool supports generation from a local copy of googleapis. Specify the path togoogleapis in the environment variable SYNTHTOOL_GOOGLEAPIS.

export SYNTHTOOL_GOOGLEAPIS=path/to/local/googleapis

Local GAPIC Generator

SynthTool supports generation from a local copy of gapic-generator. Specify the path togapic-generator in the environment variable SYNTHTOOL_GENERATOR.

export SYNTHTOOL_GENERATOR=path/to/local/gapic-generator

Don't forget to compile gapic-generator before running SynthTool.

cd path/to/local/gapic-generator
./gradlew fatJar

Include .proto files

SynthTool supports copying .proto API definition files from googleapis.

gapic = gcp.GAPICGenerator()

library = gapic.node_library('speech', 'v1', include_protos=True)

Tracking obsolete files

SynthTool automatically determines which files were generated by examining the last modified time (mtime) of all the files in your directory. New files are recorded in the metadata.

When a file appears in one generation but not the next, it is automatically removed.

To disable tracking obsolete files, add this code to your synth.py:

import synthtool as s

s.metadata.set_track_obsolete_files(False)

Helpful tips

Where does the generated code go?

SynthTool will run Artman which will create generated code that can be found at ~/.cache/synthtool/googleapis<-private>/artman_genfiles. This is useful for figuring out what it is you need to copy for your specific library.