Select how weights are quantized for the final model:
- Per-Tensor — Uses a single scale for an entire layer. Minimizes memory usage and slightly simplifies inference, but may reduce accuracy.
- Per-Channel — Uses one scale per output neuron (fully connected layers) or output filter (convolutional layers). Adds only a small metadata overhead, and although it requires a different requantization multiplier per output channel, the inference overhead is minimal — typically justified by the improved accuracy.