Home / C/C++ / Machine Learning

DNN Inference with CPU, C++, ONNX support: Instant

WARNING: Instant is deprecated. Please use Menoh instead. WARNING: Instant is deprecated. Please use Menoh instead. WARNING: Instant is deprecated. Please use Menoh instead. Instant WARNING: Ins
Information
Category: C/C++ / Machine Learning
Watchers: 7
Star: 56
Fork: 3
Last update: Sep 21, 2023
Resource links
https://github.com/okdshin/instant

WARNING: Instant is deprecated. Please use Menoh instead.

WARNING: Instant is deprecated. Please use Menoh instead.

WARNING: Instant is deprecated. Please use Menoh instead.

Instant

Build Status

WARNING: Instant is deprecated. Please use Menoh instead.

Instant is DNN inference library written in C++.

Instant is released under MIT Licence.

Goal

  • DNN Inference with CPU
  • C++
  • ONNX support
  • Easy to use.

Requirement

  • MKL-DNN Library
  • ProtocolBuffers

Build

Execute below commands in root directory.

sh retrieve_data.sh
mkdir build && cd build
cmake ..
make

Installation

Execute below command in build directory created at Build section.

make install

Run VGG16 example

Execute below command in root directory.

./example/vgg16_example

Current supported nodes

  • Conv (2D)
  • Relu
  • MaxPool
  • Reshape (nchw -> nc)
  • FC
  • Dropout
  • Softmax
1 Open More issues
Closed

Impl batch norm

opened Dec 29, 2017 by okdshin 1
Closed

Travis

opened Dec 22, 2017 by okdshin 0
Closed

Refactoring

opened Dec 18, 2017 by okdshin 0
Closed

Onnx viewer

opened Dec 15, 2017 by okdshin 0
Closed

Refactoring

opened Dec 15, 2017 by okdshin 0
Open

FC layer

I am curious to know how you got the FC layer in the VGG16.onnx? It is listed as experimental operator in ONNX doc. When using CNTK+ONNX, the FC layer was exported as a combination of Add and MatMul nodes. Thanks!

enhancement question 
opened Jan 3, 2018 by SGShen 3

Contents

    Microsoft ONNX.js: run ONNX models using JavaScript
    microsoft A flexible and efficient deep neural network (DNN) compiler that generates high-performance executable from a DNN model description.
    onnx Convert scikit-learn models and pipelines to ONNX
    pfnet-research Menoh: fast DNN inference library with multiple programming language support
    mravanelli pytorch-kaldi is a project for developing state-of-the-art DNN/RNN hybrid speech recognition systems. The DNN part is managed by pytorch, while feature extraction, label computation, and decoding are performed with the kaldi toolkit.
    intel Intel® Optimization for Chainer*, a Chainer module providing numpy like API and DNN acceleration using MKL-DNN.
    patil-suraj Accelerated NLP pipelines for fast inference on CPU. Built with Transformers and ONNX runtime.
    NVIDIA-ISAAC-ROS Hardware-accelerated DNN model inference ROS2 packages using NVIDIA Triton/TensorRT for both Jetson and x86_64 with CUDA-capable GPU.
    onnx Caffe2 implementation of Open Neural Network Exchange (ONNX)
    onnx Tutorials for creating and using ONNX models
    onnx A collection of pre-trained, state-of-the-art models in the ONNX format
    microsoft ONNX.js: run ONNX models using JavaScript
    onnx ONNX-TensorRT: TensorRT backend for ONNX
    xboot A lightweight, portable pure C99 onnx inference engine for embedded devices with hardware acceleration support.
    Microsoft ONNX Runtime: cross-platform, high performance ML inferencing and training accelerator
    microsoft ONNX Runtime: cross-platform, high performance ML inferencing and training accelerator
    alrevuelta Pure C ONNX runtime with zero dependancies for embedded devices
    yixuan Tiny yet Powerful Deep Neural Networks
    santiblanko vue-instant - Vue instant allows you to easily create custom search controls with auto suggestions for your vue 2 applications.
    dlunion TensorRT ONNX Plugin、Inference、Compile
    yangsheng6810 Org-latex-instant-preview provides instant preview of LaTeX snippets via MathJax outputed SVG.
    Share this repo

    Related Repos


    Machine Learning
    114
    OpenVINO™ integration with TensorFlow
    openvinotoolkit English | 简体中文 OpenVINO™ integration with TensorFlow This repository contains the source code of OpenVINO™ integration with TensorFlow, designed for T
     
    Machine Learning
    7.8k
    Vowpal Wabbit is a machine learning system which pushes the frontier of machine learning
    VowpalWabbit Vowpal Wabbit is a machine learning system which pushes the frontier of machine learning with techniques such as online, hashing, allreduce, reductions, learning2search, active, and interactive learning
     
    Machine Learning
    3k
    High performance, easy-to-use, and scalable machine learning (ML) package
    aksnzhy High performance, easy-to-use, and scalable machine learning (ML) package, including linear model (LR), factorization machines (FM), and field-aware factorization machines (FFM) for Python and CLI interface.
     
    Machine Learning
    62
    Sunnet is a light deep learning library
    Tyill Sunnet is a light deep learning library. Linux/Windows License ResNet cpp-example for Win Compare with Tensorflow, inference ResNet50. PC: i5-2400, GF
     
    Machine Learning
    769
    Gesture recognition toolkit
    nickgillian The Gesture Recognition Toolkit (GRT) is a cross-platform, open-source, C++ machine learning library designed for real-time gesture recognition.
     
    Machine Learning
    79
    FANN (Fast Artifical Neural Network) binding in Crystal
    NeuraLegion crystal-fann Crystal bindings for the FANN C lib Installation Add this to your application's shard.yml: dependencies: crystal-fann: github: Neur
     
    Machine Learning
    5
    Serve pytorch / torch models using Drogon
    SABER-labs C++ Torch Server Serve torch models as rest-api using Drogon, example included for resnet18 model for Imagenet. Benchmarks show improvement of ~6-10x
     
    Machine Learning
    187
    FANN (Fast Artificial Neural Network Library) bindings for Node.js
    rlidwka node-fann node-fann is a FANN bindings for Node.js. FANN (Fast Artificial Neural Network Library) is a free open source neural network library, which
     
    Machine Learning
    162.1k
    An Open Source Machine Learning Framework for Everyone
    tensorflow Documentation TensorFlow is an end-to-end open source platform for machine learning. It has a comprehensive, flexible ecosystem of tools, libraries, a
     
    Machine Learning
    6
    This repository is a tensorrt deployment of the onsets and frames model, which is implemen...
    xk-wang Onsets and Frames TensorRT inference This repository is a tensorrt deployment of the onsets and frames model, which is implemented using pytorch (http
     
    Machine Learning
    27
    SigLib Digital Signal Processing and Machine Learning Library
    Numerix-DSP siglib SigLib Digital Signal Processing and Machine Learning Library What is SigLib? SigLib is a highly portable DSP and ML library that supports many
     
    Machine Learning
    10
    a naive NCNN implementation of GFPGAN aims at developing Practical Algorithms for Real-wor...
    FeiGeChuanShu GFPGAN-ncnn a naive ncnn implementation of GFPGAN aims at developing Practical Algorithms for Real-world Face Restoration model support: 1.GFPGANClean
     
    Machine Learning
    4
    use c++ train yolov5
    jedi007 LibTorch-YoLoV5-train-detection use c++ train yolov5 Just for learning!!! Just for learning!!! Just for learning!!! Dependencies Windows10 Qt5.1
     
    Machine Learning
    317
    OpenMMLab Model Deployment Framework
    open-mmlab Introduction English | 简体中文 MMDeploy is an open-source deep learning model deployment toolset. It is a part of the OpenMMLab project. Major features F
     
    Machine Learning
    4
    YoloX for a Jetson Nano 4 using ncnn.
    Qengineering YoloX Jetson Nano YoloX with the ncnn framework. Paper: https://arxiv.org/pdf/2107.08430.pdf Special made for a Jetson Nano, see Q-engineering deep le