Advanced

26 min read

#Computer Vision#Object Detection#YOLO#Segmentation#U-Net#Mask R-CNN#Tracking

13. Computer Vision Tasks (Detection, Segmentation, Tracking)

End-to-end computer vision: classification, object detection (YOLO, Faster R-CNN), semantic & instance segmentation (U-Net, Mask R-CNN), and mAP/IoU evaluation.

Computer Vision: Complete Notes (Beginner to Advanced)

Introduction#

Computer Vision (CV) is a field of artificial intelligence that enables computers to understand and analyze visual information such as images and videos.

In deep learning, computer vision commonly uses Convolutional Neural Networks (CNNs), detection architectures, and segmentation networks to perform tasks such as:

Classifying an image
Locating an object
Detecting multiple objects
Separating different regions of an image
Identifying individual object instances

The major tasks covered in this topic can be viewed as increasing levels of visual understanding:

Architecture & Data Flow
Image
  |
  +--> What is in the image?
  |       |
  |       +--> Image Classification
  |
  +--> Where is the object?
  |       |
  |       +--> Image Localization
  |
  +--> What objects are present and where?
  |       |
  |       +--> Object Detection
  |
  +--> Which pixels belong to each class?
  |       |
  |       +--> Semantic Segmentation
  |
  +--> Which pixels belong to each individual object?
          |
          +--> Instance Segmentation

1. Image Classification

Image Classification is the task of assigning one or more class labels to an entire image.

The model receives an image and predicts what category it belongs to.

Example#

Suppose the model receives:

›Image → [picture of a dog]

The output could be:

text
Dog: 0.95
Cat: 0.03
Horse: 0.02

The predicted class is:

›Dog

Classification Does Not Provide Location#

Classification answers:

›"What is present?"

It does not answer:

›"Where is it?"

For example:

Architecture & Data Flow
Input image
     |
     v
+----------------+
|      🐕        |
|                |
|                |
+----------------+

Prediction:
Dog

There is no bounding box around the dog.

Binary Classification#

There are two possible classes.

Example:

text
Cat vs Dog
Spam vs Not Spam
Defective vs Non-defective

A model may produce:

Mathematical Formulation
P(Dog) = 0.87

Multiclass Classification#

There are multiple possible classes, and typically one class is selected.

Example:

text
Cat
Dog
Horse
Bird

The probabilities may be:

Mathematical Formulation
Cat   = 0.05
Dog   = 0.82
Horse = 0.08
Bird  = 0.05

Prediction:

›Dog

Multilabel Classification#

An image can belong to multiple classes simultaneously.

Example:

›Image → [car + person + road]

Output:

Mathematical Formulation
Car    = 0.94
Person = 0.91
Road   = 0.97

Unlike multiclass classification, multiple labels can be positive.

Typical Deep Learning Pipeline#

text
Image
  ↓
Preprocessing
  ↓
CNN / Vision Backbone
  ↓
Feature Extraction
  ↓
Classification Head
  ↓
Class Probabilities

2. Image Localization

Image Localization identifies an object's class and its location within an image.

The location is commonly represented using a bounding box.

Classification vs Localization#

text
Classification:
Image → "Dog"

Localization:
Image → "Dog + Where the dog is"

Example:

Architecture & Data Flow
+--------------------------------+
|                                |
|       +-------------+          |
|       |     Dog     |          |
|       +-------------+          |
|                                |
+--------------------------------+

The model predicts:

Mathematical Formulation
Class = Dog
Bounding Box = (x, y, width, height)

Single-Object Localization#

Traditional image localization generally focuses on locating one primary object in the image.

For example:

text
Input
  ↓
CNN
  ↓
Features
  ↓
Classification Head → object class
Bounding Box Head    → object location

The model therefore has two prediction components:

text
Object identity
+
Object location

Bounding Box Coordinates#

A box can be represented using:

›(x_min, y_min, x_max, y_max)

or:

›(x_center, y_center, width, height)

These representations describe the same rectangular region using different parameterizations.

3. Object Detection

Object Detection combines object classification and localization for multiple objects in the same image.

The model determines:

What objects are present
Where each object is located

Example#

Architecture & Data Flow
+--------------------------------------+
|                                      |
|     +---------+                      |
|     |  Dog    |                      |
|     +---------+                      |
|                         +--------+   |
|                         |  Car   |   |
|                         +--------+   |
|                                      |
+--------------------------------------+

Output:

›Dog → bounding box → confidence
Car → bounding box → confidence

Detection Output#

Each detected object can be represented as:

›(class, confidence, bounding_box)

Example:

›Dog   0.96  [50, 80, 250, 300]
Car   0.91  [400, 180, 700, 400]

Detection Pipeline#

A simplified detection pipeline is:

text
Image
  ↓
Feature Extraction
  ↓
Object Detection Head
  ↓
Candidate Predictions
  ↓
Confidence Filtering
  ↓
Non-Maximum Suppression
  ↓
Final Detections

Classification vs Localization vs Detection#

Task	Classification	Localization	Detection
Classifies objects	Yes	Yes	Yes
Finds location	No	Yes	Yes
Multiple objects	No/limited	Usually single object	Yes
Bounding boxes	No	Yes	Yes

4. Semantic Segmentation

Semantic Segmentation assigns a class label to every pixel in an image.

Instead of predicting only a box, the model predicts a pixel-level mask.

Example#

Suppose an image contains:

text
Person
Road
Car
Sky

Semantic segmentation produces a label for each pixel:

›Pixel → class

Conceptually:

Architecture & Data Flow
+----------------------+
| Sky Sky Sky Sky      |
| Sky Person Sky       |
| Road Person Road     |
| Road Car   Road      |
+----------------------+

Every pixel belongs to one semantic category.

Important Characteristic#

Semantic segmentation does not distinguish between separate objects of the same class.

For example:

›Person A
Person B

Both can simply receive:

›Person

The model knows:

Mathematical Formulation
these pixels = person

but not necessarily:

Mathematical Formulation
this group = Person A
this group = Person B

Output#

For an image of size:

›H × W

a semantic segmentation model can produce a pixel-level class prediction for each of the:

›H × W

locations.

For C classes, the network commonly produces:

›H × W × C

class scores before converting them into pixel labels.

5. Instance Segmentation

Instance Segmentation performs pixel-level segmentation while distinguishing individual objects.

For example, if an image contains three cars:

text
Car 1
Car 2
Car 3

instance segmentation produces separate masks:

text
Mask 1 → Car 1
Mask 2 → Car 2
Mask 3 → Car 3

Semantic vs Instance Segmentation#

Semantic segmentation:

›Car Car Car Car
Car Car Car Car

All car pixels have the same class.

Instance segmentation:

text
Car 1 → Mask A
Car 2 → Mask B
Car 3 → Mask C

Each object instance receives its own mask.

Detection vs Instance Segmentation#

Object detection:

›Object → Bounding Box

Instance segmentation:

›Object → Bounding Box + Pixel Mask

The mask follows the actual shape of the object more closely than a rectangular bounding box.

6. Bounding Boxes

A Bounding Box is a rectangle used to represent the approximate location of an object.

The box normally surrounds the object.

Example:

Architecture & Data Flow
+-------------------------------+
|                               |
|       +-------------+         |
|       |             |         |
|       |    Object   |         |
|       |             |         |
|       +-------------+         |
|                               |
+-------------------------------+

Common Bounding Box Representations#

1. Corner Coordinates

›(x_min, y_min, x_max, y_max)

Where:

text
x_min → left
y_min → top
x_max → right
y_max → bottom

2. Center Coordinates

›(x_center, y_center, width, height)

Conversion#

From corner coordinates:

Mathematical Formulation
width  = x_max - x_min
height = y_max - y_min

x_center = (x_min + x_max) / 2
y_center = (y_min + y_max) / 2

Normalized Bounding Boxes#

Some models use values normalized to [0, 1].

For an image with width W and height H:

Mathematical Formulation
x_center_normalized = x_center / W
y_center_normalized = y_center / H

width_normalized  = width / W
height_normalized = height / H

Normalization makes the representation less dependent on the absolute image resolution.

7. IoU

IoU stands for Intersection over Union.

It measures how much two regions overlap.

It is heavily used in object detection and segmentation evaluation.

Formula#

Mathematical Formulation
IoU = Area of Intersection / Area of Union

For two bounding boxes A and B:

Mathematical Formulation
             Area(A ∩ B)
IoU = -----------------------------
             Area(A ∪ B)

The IoU value ranges from:

›0 → no overlap
1 → perfect overlap

Example#

Suppose:

Mathematical Formulation
Intersection Area = 40
Union Area        = 100

Then:

Mathematical Formulation
IoU = 40 / 100
    = 0.40

Visual Interpretation#

Architecture & Data Flow
Low IoU:

+-------+
|   A   |
|       |
+-------+

       +-------+
       |   B   |
       |       |
       +-------+

Small overlap → Low IoU


High IoU:

+-------------+
|    A + B    |
|             |
+-------------+

Large overlap → High IoU

IoU in Detection#

During evaluation, a predicted bounding box is compared with the ground-truth box.

For example:

›IoU ≥ threshold

may be considered a sufficiently good localization depending on the evaluation protocol.

IoU thresholds are also used by NMS.

IoU in Segmentation#

IoU can also compare:

text
Predicted Mask
vs
Ground-Truth Mask

So IoU is not restricted to bounding boxes.

8. Non-Maximum Suppression (NMS)

Object detectors can produce multiple overlapping predictions for the same object.

For example:

text
Dog: 0.95 → Box A
Dog: 0.91 → Box B
Dog: 0.72 → Box C

All three boxes may refer to the same dog.

NMS removes redundant overlapping predictions.

Why NMS Is Needed#

Without NMS:

Architecture & Data Flow
        +-----------+
        |           |
     +--|--- Dog ---|--+
     |  |           |  |
     |  +-----------+  |
     +-----------------+

Multiple boxes for one object

After NMS:

Architecture & Data Flow
        +-----------+
        |    Dog    |
        |           |
        +-----------+

One final box

NMS Algorithm#

Given predicted boxes and confidence scores:

text
1. Select the box with the highest confidence.
2. Keep it.
3. Calculate IoU between this box and remaining boxes.
4. Remove boxes whose IoU exceeds the chosen threshold.
5. Select the next highest-confidence remaining box.
6. Repeat.

Example#

Suppose:

text
Box A → 0.95
Box B → 0.90
Box C → 0.70

If:

Mathematical Formulation
IoU(A, B) = 0.80

and the NMS threshold is:

›0.50

then Box B is removed because it overlaps heavily with the higher-confidence Box A.

If:

Mathematical Formulation
IoU(A, C) = 0.20

Box C can remain because the overlap is below the threshold.

NMS Flow#

text
Predictions
    ↓
Sort by confidence
    ↓
Highest-confidence box
    ↓
Keep it
    ↓
Calculate IoU
    ↓
Suppress highly overlapping boxes
    ↓
Repeat
    ↓
Final detections

Confidence Threshold vs NMS Threshold#

These are different concepts.

Confidence threshold:

›"Is this prediction confident enough to keep?"

NMS IoU threshold:

›"Does this prediction overlap too much with a better prediction?"

9. Mean Average Precision (mAP)

Mean Average Precision (mAP) is a commonly used metric for evaluating object detection systems.

It combines precision-recall performance across classes.

To understand mAP, first understand:

text
Precision
Recall
Average Precision (AP)
Mean Average Precision (mAP)

Precision#

Precision measures how many predicted positives are actually correct.

Mathematical Formulation
Precision = TP / (TP + FP)

Where:

Mathematical Formulation
TP = True Positives
FP = False Positives

Recall#

Recall measures how many actual positives were successfully detected.

Mathematical Formulation
Recall = TP / (TP + FN)

Where:

Mathematical Formulation
FN = False Negatives

Average Precision#

For a class, the detector generates predictions at different confidence levels.

The Precision-Recall relationship is summarized using Average Precision (AP).

Conceptually:

Mathematical Formulation
AP = area under the Precision-Recall curve

The exact AP calculation depends on the evaluation protocol.

Mean Average Precision#

Suppose there are three classes:

Mathematical Formulation
AP(car)    = 0.80
AP(person) = 0.90
AP(dog)    = 0.70

Then:

Mathematical Formulation
mAP = (0.80 + 0.90 + 0.70) / 3
    = 0.80

So:

Mathematical Formulation
mAP = mean of AP values across classes

Role of IoU in mAP#

For object detection, whether a prediction counts as a correct detection depends on both:

text
Class correctness
+
Bounding-box overlap

IoU is used to determine whether the predicted box sufficiently matches the ground-truth box.

Different benchmarks can report metrics at different IoU thresholds.

For example:

›AP@0.50

means AP evaluated using an IoU threshold of 0.50.

A common notation is:

›mAP@0.50

For protocols that average AP across multiple IoU thresholds, notation such as:

›mAP@0.50:0.95

is used.

Detection Evaluation Flow#

text
Predictions
    ↓
Confidence Ranking
    ↓
Match Predictions with Ground Truth
    ↓
IoU Criterion
    ↓
Precision / Recall
    ↓
Precision-Recall Curve
    ↓
AP per Class
    ↓
Mean Across Classes
    ↓
mAP

10. YOLO

YOLO stands for You Only Look Once.

It is a family of object detection architectures designed for fast object detection.

The central idea is to perform object detection using a neural network in a unified pipeline rather than relying on a separate traditional region-proposal stage.

Basic YOLO Concept#

text
Image
  ↓
Neural Network
  ↓
Object Predictions
  ↓
Post-processing
  ↓
Detected Objects

A detector predicts information such as:

text
Bounding Box
Class
Confidence

for objects in the image.

Why YOLO Became Important#

YOLO made real-time object detection practical and popular.

It is commonly associated with:

Fast inference
End-to-end detection
Single-stage detection
Real-time applications

Single-Stage Detection#

YOLO belongs to the family of single-stage detectors.

Conceptually:

text
Image
  ↓
Backbone
  ↓
Feature Processing
  ↓
Detection Head
  ↓
Boxes + Classes + Scores

The term "single-stage" refers to the detection process being performed directly by the detection network rather than using a separate region proposal stage as in traditional two-stage detectors.

YOLO Evolution#

The YOLO family has evolved substantially.

A simplified historical progression is:

text
YOLO
 ↓
YOLOv2
 ↓
YOLOv3
 ↓
YOLOv4
 ↓
YOLOv5
 ↓
YOLOv6 / YOLOv7
 ↓
YOLOv8
 ↓
Later YOLO-family variants

Different YOLO versions use different architectures and training strategies, so "YOLO" should be understood as a family rather than one fixed architecture.

YOLO Detection Flow#

text
Input Image
     ↓
Backbone
     ↓
Feature Maps
     ↓
Detection Head
     ↓
Candidate Boxes
     ↓
Class Scores
     ↓
Confidence Filtering
     ↓
NMS / Post-processing
     ↓
Final Bounding Boxes

Example#

Input:

text
Image containing:
2 people
1 car
1 dog

Output:

text
Person → Box → 0.96
Person → Box → 0.91
Car    → Box → 0.94
Dog    → Box → 0.88

11. R-CNN

R-CNN stands for Regions with Convolutional Neural Network Features.

It is an early region-based object detection architecture.

The main idea is:

text
Image
  ↓
Generate Region Proposals
  ↓
Extract CNN Features for Each Region
  ↓
Classify Each Region
  ↓
Refine Bounding Boxes

R-CNN Pipeline#

Mathematical Formulation
Input Image
    ↓
Region Proposal Algorithm
    ↓
~Many Candidate Regions
    ↓
CNN Feature Extraction
    ↓
Classifier
    ↓
Bounding Box Regression
    ↓
Final Detections

Key Idea#

Instead of applying classification to the entire image once, R-CNN first identifies candidate regions that might contain objects.

Each region is then processed by the CNN.

Limitation#

The original R-CNN was computationally expensive because:

text
Many region proposals
        +
CNN computation for each region
        ↓
Very expensive

This motivated improvements such as Fast R-CNN and Faster R-CNN.

12. Fast R-CNN

Fast R-CNN improves R-CNN by computing convolutional features for the entire image only once.

R-CNN vs Fast R-CNN#

Original R-CNN:

text
Image
 ↓
Region proposals
 ↓
CNN separately for each region

Fast R-CNN:

text
Image
 ↓
CNN once
 ↓
Shared feature map
 ↓
Region of Interest (RoI) features
 ↓
Classification + Bounding Box Regression

Fast R-CNN Pipeline#

Architecture & Data Flow
Input Image
     ↓
CNN
     ↓
Shared Feature Map
     ↓
External Region Proposals
     ↓
RoI Pooling
     ↓
Fully Connected Layers
     ↓
+----------------------+
| Class Scores         |
| Bounding Box Deltas  |
+----------------------+

RoI Pooling#

RoI Pooling extracts a fixed-size feature representation from a region of interest on the feature map.

This allows regions of different sizes to be processed by subsequent fully connected layers that expect a fixed-size representation.

Main Improvement#

The expensive CNN computation is shared:

text
R-CNN:
CNN × many regions

Fast R-CNN:
CNN × 1 image

This makes Fast R-CNN substantially faster and more efficient than R-CNN.

Remaining Problem#

Fast R-CNN still relies on an external region proposal method.

Therefore, the complete pipeline is still not fully learned end-to-end.

This motivated Faster R-CNN.

13. Faster R-CNN

Faster R-CNN replaces the external region proposal mechanism with a learned Region Proposal Network (RPN).

Main Idea#

text
Image
 ↓
CNN Backbone
 ↓
Shared Feature Map
 ↓
Region Proposal Network
 ↓
Candidate Regions
 ↓
RoI Feature Extraction
 ↓
Classification + Bounding Box Regression
 ↓
Final Detections

Region Proposal Network (RPN)#

The RPN operates on the shared feature map and predicts candidate object regions.

Conceptually:

text
Feature Map
     ↓
RPN
     ↓
Objectness Scores
+
Bounding Box Proposals

The proposals are then passed to the detection stage.

Faster R-CNN Architecture#

Architecture & Data Flow
                  Input Image
                       |
                       v
                CNN Backbone
                       |
                       v
                Feature Map
                  /         \
                 /           \
                v             v
              RPN        RoI Features
                |             |
                v             v
        Region Proposals   Detection Head
                              |
                     +--------+--------+
                     |                 |
                     v                 v
                  Classes          Box Refinement

R-CNN Family Progression#

The evolution can be understood as:

Architecture & Data Flow
R-CNN
  |
  | CNN repeated for each region
  v
Fast R-CNN
  |
  | CNN shared across image
  | external region proposals
  v
Faster R-CNN
  |
  | learned Region Proposal Network
  v
More efficient two-stage detection

Single-Stage vs Two-Stage#

YOLO:

›Single-stage detector

Faster R-CNN:

text
Two-stage detector

Stage 1 → Region proposals
Stage 2 → Detection and box refinement

Strengths#

Faster R-CNN is known for strong detection accuracy and precise localization.

Trade-off#

Compared with many single-stage detectors, two-stage detection can involve greater computational cost and latency.

14. U-Net

U-Net is a convolutional neural network architecture designed primarily for image segmentation.

It was originally developed for biomedical image segmentation.

Its defining structure is an encoder-decoder architecture with skip connections between corresponding encoder and decoder levels.

Basic Architecture#

Architecture & Data Flow
                 Encoder
                   ↓
Input → [Conv] → [Conv] → [Conv]
          ↓        ↓        ↓
          |        |        |
          |        |        |
          |        |        |
          +--------+--------+
                   ↓
                Bottleneck
                   ↓
          Decoder / Upsampling
                   ↑
        Skip Connections
                   |
                   ↓
             Segmentation
                 Mask

A more detailed conceptual representation:

Architecture & Data Flow
Input
  |
  v
[Encoder Block]
  |
  +----------------------------+
  |                            |
  v                            |
[Encoder Block]                |
  |                            |
  +----------------------+     |
  |                      |     |
  v                      |     |
[Bottleneck]             |     |
  |                      |     |
  v                      |     |
[Decoder Block] <--------+     |
  |                            |
  v                            |
[Decoder Block] <--------------+
  |
  v
Output Segmentation Mask

Encoder#

The encoder progressively extracts higher-level features while reducing spatial resolution.

Conceptually:

text
Large image
   ↓
Smaller feature map
   ↓
Smaller feature map
   ↓
Deep semantic representation

It captures information such as:

text
Edges
Textures
Shapes
Higher-level structures

Bottleneck#

The bottleneck is the deepest part of the network.

It contains a compact, high-level representation of the input.

text
Encoder
   ↓
Bottleneck
   ↓
Decoder

Decoder#

The decoder progressively increases spatial resolution.

text
Low-resolution features
        ↓
Upsampling
        ↓
Higher resolution
        ↓
Higher resolution
        ↓
Segmentation mask

Skip Connections#

U-Net passes feature maps from the encoder directly to corresponding decoder stages.

Architecture & Data Flow
Encoder Feature Map
        |
        +--------------------+
                             |
                             v
                       Decoder Block

These skip connections help preserve fine spatial details that can be lost during downsampling.

Why U-Net Is Useful for Segmentation#

Segmentation needs both:

text
High-level semantic information
+
Fine spatial information

The encoder provides strong semantic representations.

The skip connections preserve spatial details.

The decoder combines these to reconstruct a detailed segmentation output.

U-Net Data Flow#

text
Input Image
    ↓
Encoder
    ↓
Downsampling
    ↓
Deep Features
    ↓
Bottleneck
    ↓
Upsampling
    ↑
Skip Connections
    ↓
Decoder
    ↓
Pixel-level Prediction

U-Net vs Classification CNN#

A normal classification network generally ends with:

text
Feature Map
   ↓
Global/Flattened Representation
   ↓
Fully Connected / Classification Head
   ↓
Class

U-Net instead preserves and reconstructs spatial information:

text
Image
 ↓
Encoder
 ↓
Bottleneck
 ↓
Decoder
 ↓
Pixel-level Mask

U-Net vs Object Detection#

Object detection generally predicts:

›Class + Bounding Box

U-Net predicts:

›Pixel-level segmentation

Therefore:

›Detection → "Where is the object?"
U-Net     → "Which pixels belong to the target?"

15. Computer Vision Task Comparison

Task	Main Output	Spatial Detail	Multiple Objects
Image Classification	Class label	Very low	Usually not explicitly localized
Image Localization	Class + Bounding Box	Object-level	Usually one primary object
Object Detection	Classes + Bounding Boxes	Object-level	Yes
Semantic Segmentation	Class per pixel	Pixel-level	Same-class instances not separated
Instance Segmentation	Instance masks + classes	Pixel-level	Yes, individually separated

16. Detection Architecture Comparison

Architecture	Type	Region Proposals	Main Idea
R-CNN	Two-stage	External	CNN applied to proposed regions
Fast R-CNN	Two-stage	External	Shared CNN feature map + RoI features
Faster R-CNN	Two-stage	Learned RPN	Learned region proposals + detection
YOLO	Single-stage	No separate proposal stage	Direct unified object detection

17. Segmentation Comparison

Method	Output	Separates Same-Class Instances?
Semantic Segmentation	Class mask	No
Instance Segmentation	Individual object masks	Yes
U-Net	Pixel-level segmentation output	Depends on the task/output design

18. Important Relationships

These concepts are strongly connected:

text
Bounding Box
     ↓
IoU
     ↓
NMS
     ↓
Object Detection Evaluation
     ↓
mAP

And the major detection architectures connect as:

text
R-CNN
  ↓
Fast R-CNN
  ↓
Faster R-CNN

while another major branch is:

text
Single-stage Detection
        ↓
       YOLO

For segmentation:

Architecture & Data Flow
Segmentation
    |
    +--> Semantic Segmentation
    |
    +--> Instance Segmentation
    |
    +--> U-Net

19. End-to-End Computer Vision Flow

A modern computer vision system can be understood as:

Architecture & Data Flow
Input Image
     ↓
Preprocessing
     ↓
Vision Backbone
     ↓
Feature Extraction
     ↓
Task-specific Head
     |
     +--------------------+
     |                    |
     v                    v
Classification        Detection
                          |
                          v
                    Bounding Boxes
                          |
                          v
                         IoU
                          |
                          v
                         NMS
                          |
                          v
                         mAP

For segmentation:

Architecture & Data Flow
Input Image
     ↓
Encoder / Backbone
     ↓
Feature Representation
     ↓
Decoder / Segmentation Head
     ↓
Pixel-level Predictions
     |
     +--> Semantic Segmentation
     |
     +--> Instance Segmentation

20. Summary

Concept	Purpose
Image Classification	Identify what an image contains
Image Localization	Identify an object and locate it
Object Detection	Find and classify multiple objects
Semantic Segmentation	Assign a class to every pixel
Instance Segmentation	Assign pixels to individual object instances
Bounding Box	Rectangular representation of object location
IoU	Measures overlap between regions
NMS	Removes redundant overlapping detections
mAP	Evaluates object detection performance
YOLO	Fast single-stage object detection family
R-CNN	Region-proposal-based object detection
Fast R-CNN	Shared CNN features with RoI processing
Faster R-CNN	Uses learned RPN for region proposals
U-Net	Encoder-decoder architecture for segmentation

21. Quick Recap

text
Classification
→ What is in the image?

Localization
→ What is it + where is it?

Detection
→ What objects are present + where is each one?

Semantic Segmentation
→ Which class does each pixel belong to?

Instance Segmentation
→ Which individual object does each pixel belong to?

Bounding Box
→ Rectangle around an object.

IoU
→ How much do two regions overlap?

NMS
→ Remove duplicate overlapping detections.

mAP
→ Measure object detection performance across classes.

YOLO
→ Single-stage object detection family.

R-CNN
→ Region proposals + CNN per region.

Fast R-CNN
→ Shared CNN features + RoI processing.

Faster R-CNN
→ Learned RPN + RoI-based detection.

U-Net
→ Encoder-decoder segmentation network with skip connections.

Final Mental Model

Architecture & Data Flow
                    COMPUTER VISION
                          |
          +---------------+----------------+
          |               |                |
          v               v                v
    Classification    Detection       Segmentation
          |               |                |
          v               v          +-----+-----+
      Image Class    Bounding Boxes   |           |
                                      v           v
                                  Semantic     Instance
                                      |           |
                                      +-----+-----+
                                            |
                                            v
                                          U-Net

Detection
    |
    +--> Bounding Boxes
    |
    +--> IoU
    |
    +--> NMS
    |
    +--> mAP
    |
    +--> YOLO
    |
    +--> R-CNN
            |
            +--> Fast R-CNN
            |
            +--> Faster R-CNN

Knowledge Checkpoint

13. Computer Vision Tasks Checkpoint

Q1.What is the key difference between Semantic Segmentation and Instance Segmentation?

ASemantic segmentation classifies each pixel by category (all 'person' pixels share one label), while instance segmentation detects and separates individual distinct object instances.

BSemantic segmentation only works on black and white images.

CInstance segmentation does not predict pixel masks.

DSemantic segmentation requires 3D bounding boxes.

Q2.How does YOLO (You Only Look Once) achieve real-time object detection speed?

AIt frames object detection as a single regression problem, predicting bounding boxes and class probabilities directly from full images in a single forward pass.

BIt crops thousands of proposal regions and runs a separate CNN on each region sequentially.

CIt replaces deep learning with classical OpenCV edge detection.

DIt processes frames only every 10 seconds.

Q3.What metric is standard for evaluating object detection bounding box overlap against ground truth?

AIntersection over Union (IoU) = Area(Overlap) / Area(Union).

BMean Squared Error (MSE) of image pixels.

CCosine similarity of RGB channels.

DBleu score of bounding box coordinates.

Track Your Learning

Finished studying this notebook?

Mark this guide as completed to update your course progress roadmap.

Previous Lesson

12. Landmark CNN Architectures (LeNet to ResNet & ConvNeXt)

Next Lesson

14. Sequence Models & Recurrent Neural Networks (RNN)