Quantization Explorer

LAB03: TensorFlow Lite and Quantization

Interactive Quantization Visualization

Explore how different bit widths affect model accuracy and size.

Concept from LAB03

See Section 3.2: Quantization Types in the PDF book.

Interactive Quantization Simulator

Bit Width Comparison

Accuracy vs Model Size Trade-off

Quantization Levels Visualization

Quantization Error Distribution

Weight Distribution Comparison

Code 
import numpy as np
import matplotlib.pyplot as plt

# Simulate FP32 weights (normal distribution)
np.random.seed(42)
fp32_weights = np.random.randn(10000) * 0.5

# Quantize to different bit widths
def quantize(weights, bits):
    min_val, max_val = weights.min(), weights.max()
    scale = (max_val - min_val) / (2**bits - 1)
    quantized = np.round((weights - min_val) / scale) * scale + min_val
    return quantized

fig, axes = plt.subplots(2, 2, figsize=(12, 8))

# FP32
axes[0, 0].hist(fp32_weights, bins=100, alpha=0.7, color='blue')
axes[0, 0].set_title('FP32 (32-bit float)')
axes[0, 0].set_xlabel('Weight Value')

# INT8
int8_weights = quantize(fp32_weights, 8)
axes[0, 1].hist(int8_weights, bins=100, alpha=0.7, color='green')
axes[0, 1].set_title('INT8 (8-bit integer)')
axes[0, 1].set_xlabel('Weight Value')

# INT4
int4_weights = quantize(fp32_weights, 4)
axes[1, 0].hist(int4_weights, bins=16, alpha=0.7, color='orange')
axes[1, 0].set_title('INT4 (4-bit integer)')
axes[1, 0].set_xlabel('Weight Value')

# INT2
int2_weights = quantize(fp32_weights, 2)
axes[1, 1].hist(int2_weights, bins=4, alpha=0.7, color='red')
axes[1, 1].set_title('INT2 (2-bit integer)')
axes[1, 1].set_xlabel('Weight Value')

plt.tight_layout()
plt.show()
Figure 23.1: Effect of quantization on weight distribution

Size vs Accuracy Trade-off

Code 
# Simulated data based on typical results
bit_widths = [32, 16, 8, 4, 2]
model_sizes = [100, 50, 25, 12.5, 6.25]  # Relative size (%)
accuracies = [97.5, 97.4, 97.2, 95.8, 89.3]  # Typical MNIST accuracy

fig, ax1 = plt.subplots(figsize=(10, 5))

# Model size bars
bars = ax1.bar(range(len(bit_widths)), model_sizes, color='steelblue', alpha=0.7)
ax1.set_xticks(range(len(bit_widths)))
ax1.set_xticklabels([f'{b}-bit' for b in bit_widths])
ax1.set_ylabel('Model Size (%)', color='steelblue')
ax1.tick_params(axis='y', labelcolor='steelblue')

# Accuracy line
ax2 = ax1.twinx()
ax2.plot(range(len(bit_widths)), accuracies, 'ro-', linewidth=2, markersize=10)
ax2.set_ylabel('Accuracy (%)', color='red')
ax2.tick_params(axis='y', labelcolor='red')
ax2.set_ylim(85, 100)

plt.title('Quantization: Size vs Accuracy Trade-off')
plt.tight_layout()
plt.show()
Figure 23.2: Model size vs accuracy for different quantization levels

Quantization Error Analysis

Code 
errors = []
for bits in [2, 4, 8, 16]:
    quantized = quantize(fp32_weights, bits)
    error = np.abs(fp32_weights - quantized)
    errors.append(error)

fig, ax = plt.subplots(figsize=(10, 5))
bp = ax.boxplot(errors, labels=['2-bit', '4-bit', '8-bit', '16-bit'])
ax.set_ylabel('Absolute Error')
ax.set_xlabel('Quantization Level')
ax.set_title('Quantization Error Distribution')
ax.set_yscale('log')
plt.grid(True, alpha=0.3)
plt.show()
/tmp/ipykernel_9954/781551588.py:8: MatplotlibDeprecationWarning: The 'labels' parameter of boxplot() has been renamed 'tick_labels' since Matplotlib 3.9; support for the old name will be dropped in 3.11.
  bp = ax.boxplot(errors, labels=['2-bit', '4-bit', '8-bit', '16-bit'])
Figure 23.3: Quantization error by bit width

Key Insights

Bit Width Size Reduction Typical Accuracy Loss Use Case
16-bit 2x < 0.1% GPU inference
8-bit 4x < 0.5% Edge devices (recommended)
4-bit 8x 1-3% Extreme resource constraints
2-bit 16x 5-10% Research only
Recommendation

INT8 quantization provides the best balance for edge deployment: - 4x size reduction - Minimal accuracy loss - Hardware acceleration on most devices

Try It Yourself

  1. Open the LAB03 notebook
  2. Train a model on CIFAR-10
  3. Apply different quantization methods
  4. Compare file sizes and accuracy