Converting ONNX to TFLite, TensorFlow, Keras, TFJS, and CoreML

My first public article in about three light years.

1. My original tool: onnx2tf

Since there are too many features implemented, please refer to the README for conversion options not covered in this article. I intended to support one-shot conversion for Transformers, STT, TTS, and many others. It can generate models with higher inference efficiency than onnx-tensorflow. Also, it supports more types of operations than onnx-tensorflow.

https://github.com/PINTO0309/onnx2tf

Contributors

Code volume (code = lines)

cloc .

     419 text files.
     414 unique files.                                          
     174 files ignored.

----------------------------------------
Language    files  blank  comment   code
----------------------------------------
Python        300   4820     6637  37182
JSON           27    109        0   3107
Markdown        5    343        0   2061
YAML            7     42       79    586
Dockerfile      1      6        3     38
----------------------------------------
SUM:          340   5320     6719  42974
----------------------------------------

2. Usage

• onnx==1.13.1
• onnxruntime==1.13.1
• onnx-simplifier==0.4.17
• onnx_graphsurgeon
• simple_onnx_processing_tools
• tensorflow==2.13.0rc0

2-1. Installation and downloading sample models

• docker

docker run --rm -it \
-v `pwd`:/workdir \
-w /workdir \
ghcr.io/pinto0309/onnx2tf:latest

• pip

pip install onnx2tf -U

• Model

wget https://github.com/PINTO0309/onnx2tf/releases/download/0.0.2/resnet18-v1-7.onnx

2-2. Basics: Part 1

Output TFLite.

onnx2tf -i resnet18-v1-7.onnx

2-3. Basics: Part 2

TFLite conversion + outputting a complete saved_model.

onnx2tf -i resnet18-v1-7.onnx -osd

2-4. Basics: Part 3

TFLite conversion + accuracy check for all OPs.

onnx2tf -i resnet18-v1-7.onnx -cotof

2-5. Keras

Since .h5 has various issues, outputting in the current latest keras_v3 format.

onnx2tf -i resnet18-v1-7.onnx -okv3

When generating .h5.

onnx2tf -i resnet18-v1-7.onnx -oh5

2-6. Customizing TFLite Input/Output Names

# Custom flatc binary for Ubuntu 20.04+
# https://github.com/PINTO0309/onnx2tf/issues/196
wget https://github.com/PINTO0309/onnx2tf/releases/download/1.7.3/flatc.tar.gz \
  && tar -zxvf flatc.tar.gz \
  && sudo chmod +x flatc \
  && sudo mv flatc /usr/bin/

# Custom flatc binary for Windows
# Set the environment variable paths appropriately on your own.
# https://github.com/PINTO0309/onnx2tf/issues/196
https://github.com/PINTO0309/onnx2tf/releases/download/1.7.3/flatc.exe

onnx2tf -i resnet18-v1-7.onnx -coion

2-7. INT8 Quantization (per-channel)

onnx2tf -i resnet18-v1-7.onnx -oiqt -qt per-channel

2-8. INT8 Quantization (per-tensor)

onnx2tf -i resnet18-v1-7.onnx -oiqt -qt per-tensor

2-9. TFJS Conversion

onnx2tf -i resnet18-v1-7.onnx -osd

pip install tensorflowjs

tensorflowjs_converter \
--input_format tf_saved_model \
--output_format tfjs_graph_model \
saved_model \
tfjs_model

2-10. CoreML Conversion

onnx2tf -i resnet18-v1-7.onnx -osd -k data

pip install coremltools

import coremltools as ct

FOLDER_PATH = 'saved_model'

model = ct.convert(
    model=FOLDER_PATH,
    source='tensorflow',
)
model.save(f'{FOLDER_PATH}/model.mlmodel')

3. Supported Layers

OP	Status
Abs	✅
Acosh	✅
Acos	✅
Add	✅
And	✅
ArgMax	✅
ArgMin	✅
Asinh	✅
Asin	✅
Atanh	✅
Atan	✅
AveragePool	✅
BatchNormalization	✅
Bernoulli	✅
BitShift	✅
BitwiseAnd	Help wanted
BitwiseNot	Help wanted
BitwiseOr	Help wanted
BitwiseXor	Help wanted
Cast	✅
Ceil	✅
Celu	✅
CenterCropPad	Help wanted
Clip	✅
Col2Im	Help wanted
Compress	✅
ConcatFromSequence	✅
Concat	✅
ConstantOfShape	✅
Constant	✅
Conv	✅
ConvTranspose	✅
Cosh	✅
Cos	✅
CumSum	✅
DepthToSpace	✅
Det	✅
DequantizeLinear	✅
DFT	Help wanted
Div	✅
Dropout	✅
DynamicQuantizeLinear	✅
Einsum	✅
Elu	✅
Equal	✅
Erf	✅
Expand	✅
Exp	✅
EyeLike	✅
Flatten	✅
Floor	✅
FusedConv	✅
GatherElements	✅
GatherND	✅
Gather	✅
Gemm	✅
GlobalAveragePool	✅
GlobalLpPool	✅
GlobalMaxPool	✅
GreaterOrEqual	✅
Greater	✅
GridSample	✅
GroupNormalization	Help wanted
GRU	✅
Hardmax	✅
HardSigmoid	✅
HardSwish	✅
Identity	✅
If	✅
Input	✅
InstanceNormalization	✅
Inverse	✅
IsInf	✅
IsNaN	✅
LayerNormalization	✅
LeakyRelu	✅
LessOrEqual	✅
Less	✅
Log	✅
LogSoftmax	✅
Loop	Help wanted
LpNormalization	✅
LRN	✅
LSTM	✅
MatMul	✅
MatMulInteger	✅
MaxPool	✅
Max	✅
MaxRoiPool	Help wanted
MaxUnpool	✅
Mean	✅
MeanVarianceNormalization	✅
MelWeightMatrix	Help wanted
Min	✅
Mish	✅
Mod	✅
Mul	✅
Multinomial	✅
Neg	✅
NonMaxSuppression	✅
NonZero	✅
Optional	Help wanted
OptionalGetElement	Help wanted
OptionalHasElement	Help wanted
Not	✅
OneHot	✅
Or	✅
Pad	✅
Pow	✅
PRelu	✅
QLinearAdd	✅
QLinearConcat	✅
QLinearConv	✅
QLinearLeakyRelu	✅
QLinearMatMul	✅
QLinearMul	✅
QLinearSigmoid	✅
QLinearSoftmax	✅
QuantizeLinear	✅
RandomNormalLike	✅
RandomNormal	✅
RandomUniformLike	✅
RandomUniform	✅
Range	✅
Reciprocal	✅
ReduceL1	✅
ReduceL2	✅
ReduceLogSum	✅
ReduceLogSumExp	✅
ReduceMax	✅
ReduceMean	✅
ReduceMin	✅
ReduceProd	✅
ReduceSum	✅
ReduceSumSquare	✅
Relu	✅
Reshape	✅
Resize	✅
ReverseSequence	✅
RNN	✅
RoiAlign	✅
Round	✅
ScaleAndTranslate	✅
Scatter	✅
ScatterElements	✅
ScatterND	✅
Scan	Help wanted
Selu	✅
SequenceAt	✅
SequenceConstruct	✅
SequenceEmpty	✅
SequenceErase	✅
SequenceInsert	✅
SequenceLength	✅
Shape	✅
Shrink	✅
Sigmoid	✅
Sign	✅
Sinh	✅
Sin	✅
Size	✅
Slice	✅
Softmax	✅
Softplus	✅
Softsign	✅
SpaceToDepth	✅
Split	✅
SplitToSequence	✅
Sqrt	✅
Squeeze	✅
STFT	Help wanted
StringNormalizer	Help wanted
Sub	✅
Sum	✅
Tanh	✅
Tan	✅
TfIdfVectorizer	Help wanted
ThresholdedRelu	✅
Tile	✅
TopK	✅
Transpose	✅
Trilu	✅
Unique	✅
Unsqueeze	✅
Upsample	✅
Where	✅
Xor	✅