Tflite backend

[neil-tan]

Install

Using a virtual environment is a good idea.

git clone [email protected]:uTensor/utensor_cgen.git
cd utensor_cgen
git checkout tflite_backend
pip install -e .[dev]

Running Test

pytest -s tests/tflm

This is semi-working for:

input0 = [1,1,1,1].tranpose #set at the run time
input1 = [2,4,6,8].tranpose
weight = [10,20,30,40]
bias = [7]

add_output = ADD(input0, input1)
out = FC(add_output, weight, bias)

out == 707

Notes

Unexplained behaviors: when the first tensor in the subgraph, input0, is not set at the runtime, the tensor is then initialized to undefined values instead of values contained in FlatBuffers. See here.

@dboyliao Can use some input on the python coding style/design patterns. Keep in mind, I barely got this to work. Perhaps using registry/factory pattern for additional op supports? Imports are a mess.

The files to look at are:

• tests/tflm/tflite_export/conftest.py
• tests/tflm/tflite_export/test_write.py
• uTensor_cgen/transformer/tflite_exporter.py

There are 3 types of quantization flows in TFLM:

• None. (inputs and outputs are floating points)
• Hybrid (one of the inputs is of floating point, and quantized for others)
• Offline (everything is quantized, including activation. Activation ranged determined offline, see the API and the Colab)

Hybrid quantization seems broken atm. uTensor-cli require additional work to support offline quantization. So, no quantization is the way to go.

Useful Docs:

• Interpreter's Python binding
• Python code snippet for inferencing
• How to get the FlatBuffers TFLM Python binding