Multi-Object Tracking under Quantization on Embedded Hardware

Master’s thesis project (M.Eng. Electrical Engineering, THI Ingolstadt).

Repository: github.com/Arash-Barabadi/MOT-Quantization

What this project asks

When an AI model is made smaller and faster (a process called quantization, going from FP32 to INT8), it becomes less precise. This project asks a specific question:

Does this loss of precision hurt the tracker’s ability to keep people’s identities straight more than it hurts its ability to find people?

The idea: lower precision makes the boxes drawn around people jitter slightly. That jitter may confuse the part of the tracker that decides “this person in frame 2 is the same as that person in frame 1” — causing the tracker to swap their IDs. This project measures whether that identity confusion is worse than the drop in plain detection would predict.

The hardware and tools

• NVIDIA Jetson Orin Nano (Super) — a small embedded AI computer
• Runs headless (no monitor), controlled over SSH
• All measurements taken in maximum-performance mode with locked clock speeds, so timing results are consistent
• Everything runs inside a Docker container for reproducibility

The pipeline

• YOLOv8n — finds people in each video frame (detection)
• ByteTrack — links those detections across frames so each person keeps a consistent ID (tracking)
• MOT17 — a standard dataset of seven street videos, with every person already labeled by hand (the answer key)
• TrackEval — the official tool that scores the tracker against the answer key

The plan

1. Run the full-precision (FP32) pipeline and measure it — the “before” picture.
2. Shrink the detector to INT8 and run the exact same test.
3. Compare: do the identity scores drop more than the detection scores?
4. Later: recover the lost accuracy, and integrate into a ROS 2 robotics pipeline.

Phase 0 — Full-precision baseline (done)

The complete FP32 pipeline was run and scored on all seven videos. Combined results:

These numbers are the reference point. Every quantized version gets compared against them.

The scores look low on purpose: YOLOv8n is a small, fast model chosen to fit a low-power device. This project is about how much the scores drop under quantization — not about hitting the highest possible score.

Interesting starting point: at full precision, keeping identities straight (AssA 42.8) scores higher than finding people (DetA 31.2). The key question for the next phase was whether INT8 flips this — dragging identity scores down harder than detection scores.

Phase 1 — INT8 quantization: go/no-go test (done)

The detector was exported to ONNX and compiled into three TensorRT engines covering the full precision ladder: FP32, FP16, and INT8 (post-training quantization, calibrated on 700 frames sampled across all seven videos). Each engine ran the identical pipeline and was scored with TrackEval.

Building the ladder this way isolates two separate effects. Comparing the original PyTorch FP32 against the TensorRT FP32 engine checks whether the conversion to TensorRT changes anything on its own. Comparing FP32 → FP16 → INT8 then shows what each step of precision loss costs, with the engine held constant.

Combined results (all seven videos)

What the numbers say

The TensorRT conversion is harmless. PyTorch FP32 and TensorRT FP32 are identical within noise. The Phase 0 baseline stands.

FP16 is essentially free. Half precision costs almost nothing — every metric sits on top of FP32.

INT8 degrades sharply, but not where the hypothesis predicted. The original expectation was that identity scores (AssA) would drop harder than detection scores (DetA). The data shows the opposite: detection collapsed (DetA 31.7 → 20.6, a 35% relative drop) while association barely moved (AssA 42.3 → 39.6, a 6% drop).

The cause is visible in the sub-metrics: INT8 became recall-starved. Detection recall fell (33.9 → 21.3) while detection precision slightly rose (73.9 → 75.7). In plain terms, INT8 stopped finding people rather than inventing false ones. On a detector that already found only a third of people at full precision, losing recall is devastating.

The apparent stability of association is partly a mirage: with far fewer detections, there are fewer identity decisions to get wrong. ID switches even dropped (610 → 400) — not because tracking improved, but because you cannot swap the identity of a person you never detected.

What this means

The hypothesis — that INT8 hurts identity association disproportionately via box jitter — is not supported in this post-training-quantization regime. Naive INT8 PTQ primarily destroys detection recall, and that detection collapse dominates and masks any association-specific effect.

This is a result, not a dead end. The association effect may still exist, but it is buried under the detection failure and cannot be seen while detection is breaking. Separating the two requires a detection-controlled experiment: feed the tracker identical full-precision detections while quantizing only the internal arithmetic, and test whether association degrades on its own.

Caveat: this is one PTQ calibration, one tracker (ByteTrack), one weak detector. The honest headline is “an association-specific degradation was not observed in this regime,” not “the hypothesis is false.”

Repository structure

The dataset, model files, TensorRT engines, and visualization videos are not stored in the repo (excluded via .gitignore) — they are large and regenerable.

• scripts/ — the pipeline: tracking runner, calibration sampler, INT8 engine builder, and the batch-evaluation loop
• results/ — tracking outputs and TrackEval metrics for each precision (fp32, trt_fp32, fp16, int8, each with a matching _eval folder)

Status

• Phase 0 — Full-precision baseline
• Phase 1 — INT8 quantization (go/no-go test)
• Phase 2 — ROS 2 integration
• Phase 3 — Accuracy recovery (QAT / mixed precision)
• Phase 4 — Writing