Skip to content

Latest commit

 

History

History
124 lines (85 loc) · 6.64 KB

nnstreamer.adoc

File metadata and controls

124 lines (85 loc) · 6.64 KB
Raw

NNStreamer Project Proposal

Name of the project: NNStreamer

Requested maturity level: Incubation stage

Description: NNStreamer is a set of GStreamer plugins that allow various neural network models and their frameworks along with tensor-related operations in GStreamer pipelines. NNStreamer also provides GStreamer plugins to attach other frameworks (ROS, Tizen Sensor F/W, Linux IIO, and so on) to GStreamer pipelines, which were generally not considered in conventional GStreamer applications, but are useful for neural network applications in general.

NNStreamer promotes easier and efficient development of on-device AI systems by allowing to describe general systems with various input, outputs, processors, and neural networks easily with the pipe-and-filter architecture. It provides easy-to-use APIs with corresponding SDKs as well: C-APIs (all platforms), Tizen.NET (C#), and Android (Java) along with a wide range of nerual network frameworks and software platforms (Ubuntu, macOS, OpenEmbedded). NNStreamer has started as an open source project since 2018 and is under active developoment with Tizen project and a wide range of consumer electronic devices.

More information is available at

Statement on alignment with LF AI’s mission: Machine Learning applications often process online stream input data in real-time, which may include multiple pre/post-processors, neural networks, input and output nodes, and complex data paths. Such a complex system can be easily represented and efficiently executed as a stream pipeline by NNStreamer along with conventional stream pipeline libraries (GStreamer and its plugins).

Have you identified possible collaboration opportunities with current LF AI hosted projects (https://lfai.foundation/projects/)? Please explain.:

  • Acumos AI: import deployed AI models to NNStreamer pipelines or let models be updated via ACumos.

  • Marquez: import data from Marquez pipelines as inputs of NNStreamer pipelines for neural network training.

  • ONNX: NNStreamer APIs may need to support ONNX model files for ONNX-compliant frameworks.

  • Sparklyr: import data from Sparklyr streams to NNStreamer pipelines.

License: LGPL-2.1

Source control: Github

Issue tracker: Github Issues

Collaboration tools: Github Issues, Github Wiki, Gitter.im

External dependencies:

  • Mandatory Dependencies

    • GStreamer (LGPL-2.1)

    • GLib (LGPL-2.1)

  • Optional Dependencies

    • Protocol Buffers (BSD 3-clause)

    • Python (PSF)

  • Plugins of NNStreamer, which may be chosen at build time or added later after the build.

    • ROS (BSD)

    • TensorFlow/TensorFlow-Lite (Apache-2.0)

    • PyTorch/Caffe2 (BSD-"style")

    • nnfw-runtime (Apache-2.0)

    • Movidius-NCSDK2 (Apache-2.0)

    • edge-TPU runtime (Apache-2.0)

    • openVINO (Apache-2.0)

    • Arm NN (MIT)

    • Note that users may write their own proprietary plugins of NNStreamer and attach them in run-time.

Initial committers:

The roles of contributor, committer, maintainer, etc.: MAINTAINERS.md

Total number of contributors to the project including their affiliations.:

  • Samsung Research: 11 (active) / 2 (inactive / casual)

  • Hobbyist/Student: 5 (casual)

Release methodology: RELEASES.md

Code of conduct: CODE_OF_CONDUCT.md

Do you have any specific infrastructure requests needed as part of hosting the project in the LF AI?: None so far. Infrastructure we currently use include:

  • Per-PR build and test infrastructure in AWS, sponsored by Samsung Research.

  • Tizen OBS build and release infrastructure

  • Ubuntu Launchpad/PPA build and release infrastructure

  • JFrog/JCenter Android Studio extension release infrastructure

  • Coverity static code analyzer

Project website:

  • We do not have a web site other than out Github main

  • We do not have a domain reserved.

  • We would like to have one with demos, documents, and example apps.

Project governance: contributing.md

Social media accounts: None

Existing sponsorship: Samsung Research, Samsung (LF Platinum member)