MAI5063 Advances in Deep Learning · Lab 5 · Week 4b companion lab

Finding objects: IoU, NMS and two-stage detectors

Every detector depends on two small tools: a way to measure how well two boxes overlap, and a way to remove duplicate boxes. Build both from scratch, see how they shape precision and recall, then run a pretrained Faster R-CNN in PyTorch and TensorFlow.

Platform Colab, PyTorch and TensorFlow Runtime GPU recommended Data One sample image, COCO classes

By the end you can

Compute IoU between two boxes and explain why 0.5 is a common threshold.
Write greedy Non-Max Suppression and explain how its threshold trades duplicates against merged neighbours.
Relate IoU, precision, recall and mAP.
Run a pretrained Faster R-CNN and read its boxes, labels and scores.
Explain how the score threshold moves a detector along the precision–recall curve.

Before the lab

Review classification versus localisation versus detection, sliding windows, region proposals, IoU, NMS, mAP and R-CNN to Faster R-CNN. Open colab.research.google.com and start a notebook. Tasks 1 to 4 need only NumPy and run on a CPU. Runtime › Change runtime type › GPU for Tasks 5 to 7. Pretrained weights download on first use. Run cells in order. Write answers in the reflection section.

Lecture link · From classification to detection

Three questions about an image

Detection answers all three at once, for every object. The sliding-window calculator shows why brute force is too slow, which is why proposals and then learned detectors were invented.

Sliding-window cost

Tasks 2 · Intersection over Union

How well does a predicted box match the truth?

The green box is the ground truth. Move and resize the red prediction and watch the overlap.

IoU = area(B_p ∩ B_gt) / area(B_p ∪ B_gt)

Tasks 3 and 4 · Non-Max Suppression

Keep the best box, drop its near-copies

A detector fires many overlapping boxes on one object. NMS sorts them by score, keeps the best, discards any box that overlaps it by more than the threshold τ, and repeats. The seven boxes below are two objects plus a smaller third object next to the second.

keptsuppressed

What the algorithm does

Task 7 · Score threshold, precision and recall

One detector, many operating points

Each detection has a confidence score, and IoU has already decided whether it is a true positive. A threshold keeps the detections above it. This toy set has 24 detections and 16 real objects. The AP is computed exactly from this list.

Precision = TP / (TP + FP) Recall = TP / (TP + FN) AP = area under the precision–recall curve

precision–recall curveyour threshold

Tasks 5 and 6 · Two-stage detectors

Propose first, classify second

Faster R-CNN has a first stage that proposes regions that might hold an object, and a second stage that classifies each region and refines its box. The three generations differ in how much work is shared.

Tasks 5 to 7 · Reading the output

Pixels or fractions: two box formats

torchvision returns pixel boxes as [x1, y1, x2, y2]. TF-Hub returns fractions of the image as [y1, x1, y2, x2]. Convert a TF-Hub box and see why normalised boxes survive resizing.

TF-Hub box → pixels

Hands-on · Google Colab

The Colab tasks

Copy each cell into Colab and run them in order. Tasks 1 to 4 use NumPy only. Task 5 uses PyTorch and Task 6 uses TensorFlow, so you can do either in one runtime and restart for the other. Comments in the code explain each step.

Reflection

Write it in your own words

Answers are saved in this browser only. Open a hint after you have tried.