inference-engine

There are 222 repositories under inference-engine topic.

  • FedML-AI/FedML

    FEDML - The unified and scalable ML library for large-scale distributed training, model serving, and federated learning. FEDML Launch, a cross-cloud scheduler, further enables running any AI jobs on any GPU cloud or on-premise cluster. Built on this library, TensorOpera AI (https://TensorOpera.ai) is your generative AI platform at scale.

    Language:Python4.1k114323769

  • hyperjumptech/grule-rule-engine

    Rule engine implementation in Golang

    Language:Go2.1k58167326

  • zjhellofss/KuiperInfer

    带你从零实现一个高性能的深度学习推理库，支持大模型 llama2 、Unet、Yolov5、Resnet等模型的推理。Implement a high-performance deep learning inference library step by step

    Language:C++2.1k2124231

  • janhq/cortex

    Drop-in, local AI alternative to the OpenAI stack. Multi-engine (llama.cpp, TensorRT-LLM). Powers 👋 Jan

    Language:C++1.7k1427885
  • onediff

    siliconflow/onediff

    OneDiff: An out-of-the-box acceleration library for diffusion models.

    Language:Python1.3k3833779

  • Tencent/FeatherCNN

    FeatherCNN is a high performance inference engine for convolutional neural networks.

    Language:C++1.2k10144285

  • PaddlePaddle/Paddle.js

    Paddle.js is a web project for Baidu PaddlePaddle, which is an open source deep learning framework running in the browser. Paddle.js can either load a pre-trained model, or transforming a model from paddle-hub with model transforming tools provided by Paddle.js. It could run in every browser with WebGL/WebGPU/WebAssembly supported. It could also run in Baidu Smartprogram and WX miniprogram.

    Language:JavaScript94992140134

  • Adlik/Adlik

    Adlik: Toolkit for Accelerating Deep Learning Inference

    Language:C++7882918082

  • msnh2012/Msnhnet

    🔥 (yolov3 yolov4 yolov5 unet ...)A mini pytorch inference framework which inspired from darknet.

    Language:C++7292623143

  • PygmalionAI/aphrodite-engine

    PygmalionAI's large-scale inference engine

    Language:Python6601311280
  • Forward

    Tencent/Forward

    A library for high performance deep learning inference on NVIDIA GPUs.

    Language:C++546222867

  • PaddlePaddle/Anakin

    High performance Cross-platform Inference-engine, you could run Anakin on x86-cpu,arm, nv-gpu, amd-gpu,bitmain and cambricon devices.

    Language:C++5305787135

  • pylint-dev/astroid

    A common base representation of python source code for pylint and other projects

    Language:Python51425801265

  • HoloClean/holoclean

    A Machine Learning System for Data Enrichment.

    Language:Python5123032129

  • insight-platform/Savant

    Python Computer Vision & Video Analytics Framework With Batteries Included

    Language:Python4941138539

  • ulfurinn/wongi-engine

    A rule engine written in Ruby.

    Language:Ruby481266342

  • buguroo/pyknow

    PyKnow: Expert Systems for Python

    Language:Python4603948141

  • quic/ai-hub-models

    The Qualcomm® AI Hub Models are a collection of state-of-the-art machine learning models optimized for performance (latency, memory etc.) and ready to deploy on Qualcomm® devices.

    Language:Python282114239

  • gottingen/kumo-search

    docs for search system and ai infra

    Language:C++24226

  • BMW-InnovationLab/BMW-TensorFlow-Inference-API-CPU

    This is a repository for an object detection inference API using the Tensorflow framework.

    Language:Python18615249
  • MIVisionX

    ROCm/MIVisionX

    MIVisionX toolkit is a set of comprehensive computer vision and machine intelligence libraries, utilities, and applications bundled into a single toolkit. AMD MIVisionX also delivers a highly optimized open-source implementation of the Khronos OpenVX™ and OpenVX™ Extensions.

    Language:C++1842422772

  • midea-ai/Aidget

    Ai edge toolbox，专门面向边端设备尤其是嵌入式RTOS平台，AI模型部署工具链，包括模型推理引擎和模型压缩工具

    Language:Python15315632

  • HolmesShuan/CNN-Inference-Engine-Quick-View

    A quick view of high-performance convolution neural networks (CNNs) inference engines on mobile devices.

    15012218

  • haobosang/TinyTensor

    TinyTensor is a tool for running already trained NN (Neural Network) models to be able to use them for inference of various tasks such as image classification, semantic segmentation, etc.

    Language:C++145389

  • nilp0inter/experta

    Expert Systems for Python

    Language:Python139103036

  • solidglue/Recommender_System_Inference_Services

    Large scale recommender system inference Microservices and APIs （Dubbo 、gRPC and REST ） with Golang.

    Language:Go100502
  • openvino_contrib

    openvinotoolkit/openvino_contrib

    Repository for OpenVINO's extra modules

    Language:C++973078139

  • nrl-ai/daisykit

    DaisyKit is an easy AI toolkit with face mask detection, pose detection, background matting, barcode detection, face recognition and more. - with NCNN, OpenCV, Python wrappers

    Language:C++9661117
  • deep-vision-processing

    cansik/deep-vision-processing

    Deep computer-vision algorithms for the Processing framework.

    Language:Java8851921

  • Torsion-Audio/nn-inference-template

    Neural network inference template for real-time cricital audio environments - presented at ADC23

    Language:C++88424

  • HoloClean/HoloClean-Legacy-deprecated

    A Machine Learning System for Data Enrichment.

    Language:Python752217522

  • T-head-Semi/csi-nn2

    An optimized neural network operator library for chips base on Xuantie CPU.

    Language:C759632

  • CoderLSF/fast-llama

    Runs LLaMA with Extremely HIGH speed

    Language:C++74466

  • BMW-InnovationLab/BMW-IntelOpenVINO-Detection-Inference-API

    This is a repository for a No-Code object detection inference API using the OpenVINO. It's supported on both Windows and Linux Operating systems.

    Language:Python73313

  • Media-Smart/cheetahinfer

    A C++ inference SDK based on TensorRT

    Language:C++696215

  • RubixML/Server

    A standalone inference server for trained Rubix ML estimators.

    Language:PHP618312