/diffgram

Adding custom webhooks for data annotation

Primary LanguagePythonOtherNOASSERTION

Docs • Diffgram.com • Request Slack Invite • Enterprise

Support & Community

  1. Open an issue (Technical, bugs, etc)
  2. 😍 Join us on slack!

Open Source Training Data Platform

Modern Training Data platform for machine learning delivered as a single application.

Open Source Data Labeling, Workflow, Automation, Exploring, Streaming, and so much more!

Watch a high level video explanation.

New 1st Class AI/ML Integrations

Integrate and share your machine learning. Event driven backed by RabbitMQ. Learn more Workflow

Hugging Face

Hugging Face Zero Shot

Deepchecks

Deepchecks Image Properties Outliers

Build your Own

Custom Actions

Annotate Anything - Images, Video, 3D, Text, Geo, Audio And more

Images

Box, Polygons, Lines, Keypoints, Classification Tags, Quadratic Curves, Cuboids, Segmentation, and More

Video

Long, High Frame Rate, High Resolution Videos.

3D

3D Labeling Docs

Text

Text Labeling docs

Named Entity Recognition, Part of Speech Tagging, Coreference Resolution, Dependency Parsing

Diffgram Text Interface

Audio

Audio Labeling docs

Annotate Audio Regions available now,

Diffgram Audio Interface

Audios Transcription (coming soon)

Geospatial & Tiled Imagery

Support for COG (Cloud Optimized GeoTIFF), streaming, multi-layer, standard and cloud-optimized.

Alpha Release: Geospatial labeling docs

Diffgram Geospatial Interface

Documents

More

Build your own UI or contact us. Our intent is to build and cover all major media types in 2022, including timeseries, DICOM, and more.

Manage all of your training data

Manage multiple Schemas, Users, Datasets, Process, and so much more.

Customize Everything

With Diffgram you can get the exact branded experience you want through the what-you-see-is-what-you-get editor. Whitelabel UI Layout & Branding, Automations, Schema, Geometry, Processes, Pipelines, Queries, and More. Diffgram is the most customizable training data platform. Training Data Customization

Cybersecurity

How secure is your training data? Learn more about Cybersecurity for Training Data
Security Policies

Migration

Labelbox to Diffgram

Are you getting great value from Labelbox? Labelbox vs Diffgram

One Click Migration from Labelbox

Labelstudio to Diffgram

Learn about upgrading to Diffgram

SuperAnnotate to Diffgram

Contact us to request prioritization of the automatic migration.

What is Training Data?

Training Data is the art of supervising machines through data. This includes the activities of annotation, which produces structured data; ready to be consumed by a machine learning model. Annotation is required because raw media is considered to be unstructured and not usable without it. That’s why training data is required for many modern machine learning use cases including computer vision, natural language processing and speech recognition.

What is Diffgram?

Diffgram is multiple training data tools in one single application. What is Diffgram

Diffgram is Open Source and optionally Client Installed. Quickstart

Who is Diffgram for?

Data Engineers, Machine Learning Leaders, AI Experts, Software Engineers, Data Scientists, Data Annotators and Subject Matter Experts.

New to Training Data?

Learn more about the general concepts with the Training Data Book.

Why Diffgram?

Diffgram brings the functions of a complex toolchain directly into one application. Providing multiple tools with one single integrated application.

Enterprise Questions? Please contact us.

Security issues: Do not create a public issue. Email security@diffgram.com with the details. Docs

Online Playground

Try Diffgram Online (Hosted Service, No Setup.)

Install

Install Diffgram

Read also our Docker compose commands cheat-sheet

Bugs and Issues

If you see any missing features, bugs etc please report them ASAP to diffgram/issues.

Contributing

See Contribution Guide for more. More on Understanding Diffgram High Level

Cloud

Cloud logos

Full support for Amazon AWS, Google Cloud, Microsoft Azure, and MinIO.

Run Diffgram on and access data from any of the clouds.

What is Diffgram a drop in replacement for?

Diffgram is a drop in replacement for the following systems: Labelbox, CVAT, SuperAnnotate, Label Studio (Heartex), V7 Labs (Darwin), BasicAI, SuperbAI, Kili-Technology, Cord, HastyAI, Dataloop, Keymakr, Scale Nucleus.

Please see the roadmap and talk with us if you see a missing feature.

How much does this cost? What's your business model?

Compare Diffgram Versions

Premium Support

Learn more.

Enterprise

Enterprise Edition.

Features

This is an ACTIVE project. We are very open to feedback and encourage you to create Issues and help us grow!

User Friendly

  • NEW Streamlined Annotation UI suitable both from "First Time" Subject Matter Experts, and powerful options for Professional Full Time Annotators

Standard Features

  • Many User Labeling - Designed for many users from Day 1.
  • Scale to Mega Projects with sophisticated organizational concepts.
  • Fully configurable - customize labels, attributes, and more.

Ingest

Ingest prediction data without writing extra scripts.

  • NEW Import Wizard saves you hours having to map your data (pre-labels, QA, debug etc.).
  • All-Cloud Integrated File Browser
  • Scalable pipeline for massive ingestion - we have tested to 600+ hardware nodes
  • Integrated pipeline hooks - newly added data auto creates tasks and more

Store

Collaboration across teams between machine learning, product, ops, managers, and more.

  • Store virtually any scale of dataset and instantly access slices of the data to avoid having to download/unzip/load.
  • Fast access to datasets from multiple machines. Have multiple Data Scientists working on the same data.
  • Integrates with your tools and 3rd party workforces. Integrations It's a database for your training data, both metadata and access of raw BLOB data (over top of your storage choice).

QA & Human Tasks

Manage Annotation Workflow, Tasks, Quality Assurance and more. Task features can be used as modules within Workflow

QA Features including:

  • QA Slideshow: Reduce Costly Errors
  • Reduce Context Switching Costs with Discussions & Issue Tracking
  • Get New Team Members Certified with Training and Exams
  • Hold People Accountable with Per User Reporting
  • Reduce Human Errors with Human Centered Tasks

Learn more -> Quality Assurance Features

  • Automatic Per Task Review Routing, with configurable review chance
  • Human Task Pipelines.
  • Webhooks with Actions
  • Easily annotate a single dataset, or scale to hundreds of projects with thousands of subdivided task sets. Includes easy search and filtering.
  • Fully integrated customizable Annotation Reporting.
  • Continually upgrade your data, including easily adding more depth to existing partially annotated sets.

Annotation

Annotation Docs

Schema (Ontology): Diffgram supports all popular attributes and spatial types including Custom Spatial types. (Best Data Annotation for AI/ML)

Annotation Automation

Run models instantly with Javascript or make API calls to any language of your choice.

General purpose automation language, solve any annotation automation challenge. Less annotation and automation costs.

Stream to Training

Easier and faster for data science. Less compute cost. More privacy controls. Load streaming data from Diffgram directly into pytorch and tensorflow with one line (alpha release live!)

Explore

Skip downloading and unzipping massive datasets. Explore data instantly through the browser.

Debug

Use your models to debug the human. Visually see errors.

Diffgram is an amazing way to access, view, compare, and collaborate on datasets to create the highest quality models. Because these features are fully integrated with the Annotation Tooling, it's absolutely seamless to go from spotting an issue, to creating a labeling campaign, updating schema, etc to correct it.

  • Uncover bad data and edge cases
  • Curate data and send for labeling with one click
  • Automatic error highlighting (Coming Soon)

Secure and Private

  • Runs on your local system or cloud. Less lag, more secure, more control.
  • Enforce PII & RBAC automatically across life-cycle of training data from ingest to dataset to model predictions and back again

Tested and Stable Core

Fully integrated automatic test suite, with comprehensive End to End tests and many unit tests.

Flexible & Scaleable

  1. Flexible deploy and many integrations - run Diffgram anywhere in the way you want.
  2. Scale every aspect - from volume of data, to number of supervisors, to ML speed up approaches.
  3. Fully featured - 'batteries included'.

Docs

Docs

Vision

Vision

Speed Ups & AI

Latest AI + More

Ecosystem

Ecosystem

Contributing

We welcome contributions! Please see our contributing documentation.

Architecture & Design Docs

We plan to release more internal architecture docs over time. Please see the general docs in the mean time.