scanoss/scanoss-winnowing.py

Fast implementation of SCANOSS algorithm in C for Python

SCANOSS Winnowing Python Library

The SCANOSS Winnowing python package provides fast implementation of the winnowing algorithm. This is achieved using a C code implementation to generate the snippet IDs.

Installation

To install (from pypi.org), please run:

pip3 install scanoss_winnowing

Usage

The scanoss_winnowing package can be used in Python projects/scripts. A good example of how to consume it can be found here.

The scanoss-py package leverages this package. It can be installed using:

pip3 install scanoss[fast_winnowing]

Development

Before starting with development of this project, please read our CONTRIBUTING and CODE OF CONDUCT.

Requirements

Python 3.7 or higher.

The dependencies can be found in the requirements.txt and requirements-dev.txt files.

To install dependencies, run:

pip3 install -r requirements.txt
pip3 install -r requirements-dev.txt

Package Development

More details on Python packaging/distribution can be found here, here, and here.

It is good practice to set up a Virtual Env (venv) to isolate and simplify development/testing. If using PyCharm, please follow these instructions.

In order to develop/test a Python package, it is necessary to register the package locally. This can be done using the following command:

pip3 install -e .

There is also a Makefile in the repository, which provide helpers to achieve this:

make dev_setup

Package Deployment

Packaging the library for deployment is done using setup.

Versioning

The version of the package is defined in the init file. Please update this version before packaging/releasing an update.

Packaging

To package the library, please run:

make dist

Deployment

Deployment - GitHub Action

There are a number of GitHub Actions configured to help with the release of the scanoss_winnowing package:

• Test Release to TestPyPI - Manually triggered from the GitHub UI
  • Source
  • Windows
  • Linux
  • Mac
• Official Release to PyPI - Triggered by pushing a tagged version
  • Multi-OS & Source

The test deployments have to be triggered manually, while the official release can also be triggered by pushing a tagged version.

Deployment - Desktop

There are also local packaging command to aid local development/testing.

These use twine to upload packages to pypi.org. In order to run twine, a user needs to be registered with both TestPyPI and PyPI. Details for using TestPyPI can be found here and PyPI here.

Once the credentials have been stored in $HOME/.pypirc, the following command can be run:

make publish_test

This will deploy the package to TestPyPI. Run some tests to verify everything is ok.

Then deploy to prod:

make publish

This will deploy the package to PyPI.

The package will then be available to install using:

pip3 install scanoss_winnowing

Bugs/Features

To request features or alert about bugs, please do so here.

Changelog

Details of major changes to the library can be found in CHANGELOG.md.

Background

Details about the Winnowing algorithm used for scanning can be found here.