Forging Clean Float32 Models by Dequantizing TFLite Sparse Tensors and the Densify Operation

tensorflow-onnx did not support the conversion of Sparse models using Densify, so I implemented it in my own dequantization tool. Incidentally, Densify causes the TensorFlow Lite runtime to abort with a Segmentation Fault when interpreter.get_tensor is executed, meaning information cannot be read through normal means. Therefore, I temporarily parsed the .tflite binary into JSON, processed all information (including weights) into a readable state, and then converted the Sparse tensor back into a Dense tensor.

https://github.com/PINTO0309/tflite2tensorflow

To compress model size, Densify removes all values close to zero from the FlatBuffer. It appears that if you try to read it as-is at runtime, the structural tensor size does not match the byte size recorded in the .tflite, leading to an abort.

I convert the weights parsed into JSON into a Float16 byte array and interpolate the zeros that were removed during sparsification.

elif op_type == 'DENSIFY':
    json_tensor_info = searh_json_tensor_detail(
	interpreter._get_tensor_details(
	    op['outputs'][0]
	)['name'][:-1]
    )
    output_detail = interpreter._get_tensor_details(
	op['outputs'][0]
    )

    shape = json_tensor_info['shape']
    dtype = cast_type_np[json_tensor_info['type']]
    dim_metadata = json_tensor_info['sparsity']['dim_metadata']
    traversal_order = json_tensor_info['sparsity']['traversal_order']
    array_segments = None
    array_indices = None
    for dict_data in dim_metadata:
	if 'format' in dict_data:
	    if dict_data['format'] == 'SPARSE_CSR':
		array_segments = dict_data['array_segments']['values']
		array_indices = dict_data['array_indices']['values']

    denj = full_json['buffers'][output_detail['index']]['data']
    b = np.array(denj).astype(np.uint8).tobytes()

    dense_list = []
    for i in range(len(b))[::2]:
	dense_list.append(struct.unpack_from('0e', bytes(b[i:i+2]))[0])

    starting_point = 0
    dense_shape = np.asarray(shape)
    groups = dense_shape[-1]
    total_number_of_elements = dense_shape[0]
    for i in dense_shape[1:]:
	total_number_of_elements *= i

    densify_values = np.zeros((total_number_of_elements))
    sidx = 0
    aidx = 0
    didx = 0
    addition = 0

    for idx in range(total_number_of_elements):
	if array_segments[sidx] == aidx:
	    addition = sidx * groups
	    sidx += 1
	if array_indices[aidx] + addition == idx:
	    densify_values[idx] = dense_list[didx]
	    didx += 1
	    if aidx < len(array_indices) - 1:
		aidx += 1

    densify_values = densify_values.reshape(dense_shape)
    output_tensor = densify_values.astype(dtype)

    tensors[output_detail['index']] = output_tensor

Using tflite2tensorflow, you can reverse-convert models and process them into saved_model or Float32 tflite without needing to be aware of the internal implementation, so it is not particularly difficult. This is just a memorandum regarding the implementation.