NEConvolutionLayer \ NEGEMMConvolutionLayer: F32 dequantized output for `QASYMM8` \ `QASYMM8_SIGNED` inputs

[alvoron]

Is it possible to support F32 dequantized output for QASYMM8 \ QASYMM8_SIGNED inputs in NEConvolutionLayer
/ NEGEMMConvolutionLayer?

[morgolock]

Hi @alvoron

Could you please share more details about the use case for this? Is there a specific model that requires this configuration?

An alternative would be to use NEDequantizationLayer after NEGEMMConvolutionLayer to get F32. Would this approach work?

Hope this helps

[alvoron]

@morgolock It seems that calling NEDequantizationLayer after NEGEMMConvolutionLayer \ NEConvolutionLayer solves the issue, but I'm wondering whether it's possible to support within one layer how you've done here: #1152

[morgolock]

Hi @alvoron

It would help if you could share a small test demonstrating the exact configuration you would like us to support.

[alvoron]

@morgolock
Do I understand correctly that ACL is using 32-bit accumulator if input and weights are 8 bit? And then ACL quantizes 32-bit value into 8 bit using QuantizationInfo of destination tensor?

In mobilenetv2-SSD-int8 there are no quantization operations after the last convolution, so we want to use fp32/fp16 after it.

[morgolock]

Hi @alvoron

Could you please clarify the workload configuration you would like ACL to support ? It would be good to understand the types of the input, weights, bias and output.

NEGEMMLowpMatrixMultiplyCore already supports F32 output. Would this work for you?

[alvoron]

@morgolock sure, we're looking for the following workload:
src: u8 / i8
wei: i8
bias: f16 / f32 / i32
dst: u8 / i8 / f16 / f32 / i32

Unfortunately NEGEMMLowpMatrixMultiplyCore does not work for us, since it's suitable for the very limited scope of convolution operations.