LinkML Project Cookiecutter

A Cookiecutter template for projects using LinkML.

Prerequisites

The following are required and recommended tools for using this cookiecutter and the LinkML project that it generates. This is all one-time setup, so if you have already done it skip to the next section!

Python >= 3.8

LinkML tools are mainly written in Python, so you will need a recent Python interpreter to run this generator and to use the generated project.
pipx

pipx is a tool for managing isolated Python-based applications. It is the recommended way to install Poetry and cruft. To install pipx follow the instructions here: https://pypa.github.io/pipx/installation/
Poetry

Poetry is a Python project management tool. You will use it in your generated project to manage dependencies and build distribution files. If you have pipx installed you can install Poetry by running:
```
pipx install poetry
```
For other installation methods see: https://python-poetry.org/docs/#installation
Poetry behind firewalls

In sandboxed environments (proxy or internal repositories), you must configure poetry source in ~/.config/pypoetry/pyproject.toml to allow software installation, illustrated below:
```
[[tool.poetry.source]]
name = "myproxy"
url = "https://repo.example.com/repository/pypi-all/simple"
priority = "default"
```
Poetry Dynamic Versioning Plugin:

This plugin automatically updates certain version strings in your generated project when you publish it. Your generated project will automatically be set up to use it. Install it by running:
```
poetry self add "poetry-dynamic-versioning[plugin]"
```
cruft

cruft is a tool for generating projects based on a cookiecutter (like this one!) as well as keeping those projects updated if the original cookiecutter changes. Install it with pipx by running:
```
pipx install cruft
```
You may also choose to not have a persistent installation of cruft, in which case you would replace any calls to the cruft command below with pipx run cruft.

Creating a new project

Step 1: Generate the project files

To generate a new LinkML project run the following:

cruft create https://github.com/linkml/linkml-project-cookiecutter

Alternatively, to add linkml project files to pre-existing directory, (perhaps an existing non-linkml project), pass -f option:

cruft create -f https://github.com/linkml/linkml-project-cookiecutter

You will be prompted for a few values. The defaults are fine for most projects, but do name your project something that makes sense to you! The interactive session will guide you through the process:

project_name: Name of the project, use kebab-case with no spaces. Suggestions:
- patient-observation-schema
- sample-collection-metadata
- resume-standard
github_org: Your github username or organization name. This is used to construct links to documentation pages.
project_description: Description of the project.
- A single brief sentence is recommended
- Can easily be modified later
full_name: Your name
email: Your email
license: Choose a license for the project. If your desired license is not listed you can update or remove the LICNSE file in the generated project.
main_schema_class:
- This is used to generate an example schema which you can edit
- The value of this field is ignored if this is a schemasheets project
- You can always change this later
- Examples: Person, Observation, Sample, Entry, Gene, Event
create_python_project
- If "Yes", set this up as a python project
- Select Yes if you want to make data classes that can be used by developers
use_schemasheets
- If "Yes", set this to obtain your schema from schemasheets
google_sheet_id
- Ignore/use default value if answer to previous question was "No"
- If you are using schemasheets then enter your google doc ID here
- If you like you can leave the default value, and this will use the demo schema
google_sheet_tabs
- Ignore/use default value if not using schemasheets
- If you are using schemasheets, enter a space-separated list of tabs
- your tabs in your sheet MUST NOT have spaces in them
github_token_for_pypi_deployment:
- The github token name which aligns with your autogenerated PyPI token for making releases.
- This helps automated releases to PyPI
- This should be ignored if this is not a python project
- Even if this is a python project, you can leave blank and fill in later

Step 2: Set up the LinkML project

Change to the folder your generated project is in.

Optionally customize your project if needed:

pass arguments to linkml generators via 'config.yaml' configuration file;
pass supported environment variables via '.env.public' configuration file;

configure proxy server via project configuration file (as illustrated):

git init
poetry source add --priority=default myproxy https://repo.example.com/repository/pypi-all/simple

Setup your project

cd my-awesome-schema  # using the folder example above
make setup

Step 3: Edit the schema

Edit the schema (the .yaml file) in the src/my_awesome_schema/schema folder

nano src/my_awesome_schema/schema/my_awesome_schema.yaml

Step 4: Validate the schema

make test

Step 5: Generate documentation locally

LinkML generates schema documentation automatically. The template comes with GitHub Actions that generate and publish the documentation when you push schema changes to GitHub. The published documentation can be found at a URL like this one: https://{github-user-or-organization}.github.io/{project-name}/

You can also preview the documentation locally before pushing to GitHub by running:

make serve

Step 6: Create a GitHub project

Go to https://github.com/new and follow the instructions, being sure to NOT add a README or .gitignore file (this cookiecutter template will take care of those files for you)

Add the remote to your local git repository

git remote add origin https://github.com/{github-user-or-organization}/{project-name}.git
git branch -M main
git push -u origin main

Configure your repository for deploying the documentation as GitHub pages

Under Settings > Actions > General in section "Workflow Permissions" mark "Read repository and packages permission".
Under Pages in section "Build and Deployment":
- Under "Source" select "Deploy from a branch"
- Under "Branch" select "gh-pages" and "/ (root)"

Step 7: Register the schema

See How to register a schema

Making releases

See How to Manage Releases of your LinkML Schema

Keeping your project up to date

In order to be up-to-date with the template, first check if there is a mismatch between the project's boilerplate code and the template by running:

cruft check

This indicates if there is a difference between the current project's boilerplate code and the latest version of the project template. If the project is up-to-date with the template:

SUCCESS: Good work! Project's cruft is up to date and as clean as possible :).

Otherwise, it will indicate that the project's boilerplate code is not up-to-date by the following:

FAILURE: Project's cruft is out of date! Run `cruft update` to clean this mess up.

For viewing the difference, run cruft diff. This shows the difference between the project's boilerplate code and the template's latest version.

After running cruft update, the project's boilerplate code will be updated to the latest version of the template.

noelmcloughlin/linkml-project-cookiecutter