Introduction

The NNTool is a graph manipulation tool capable of reading a high-level computational graph description mixing DSP and NN operators and mapping it to a set of Autotiler nodes (named Generators, provided in the GAP SDK, look at the autotiler docs for more details) that it will use to generate optimized GAP C code.

NNTool is a Python library with a numpy backend to run the provided computational graph in both floating and fixed point arithmetic, bit accurate wrt GAP fixed point inference.

The process that the NNTool applies to a provided Neural Network is the following:

  1. Model Import: take the model description (.onnx or .tflite) and map it to NNTool graph nodes

  2. Topology optimizations: minimize the number of computational nodes and fuse together concatenation of nodes which can be implemented in a single step by the SW backend

  3. Quantization: collect tensors statistics accross the graph through full precision inference over a calibration dataset (or load tensors statistics from an already quantized graph), and apply specified quantization options. Look at Quantization Spec document for a detailed explanation of quantization and how it works in the GAPflow

  4. Validation: run the deployable model with bit accurate python backend wrt model running on GAP

  5. Generate the Autotiler Model: map the quantized NNTool nodes to the correct Autotiler code generators and the connections between the nodes

From a user perspective, NNTool can be used both interactively in Command Line Usage (DEPRECATED) or directly importing it as a package in a Python script with NNTool Python API Introduction. The graphs provided to NNTool must be TFLite or ONNX models.

NOTE: Not all the operands from the two formats are supported, for a complete list open the nntool command line and type:

help onnx or help tflite