INTELLIGENCE X← MAI5063 resources
MAI5063 Advances in Deep Learning · Lab 4b · Week 4 companion lab

A CNN as a feature extractor

Use a pretrained VGG16 to turn each image into a vector of numbers, then hand those vectors to a classic classifier such as an SVM or a Random Forest. No deep network is trained at all. This lab shows what a pretrained network knows, and where classical machine learning fits in.

Platform Colab + Keras + scikit-learn Runtime GPU helpful, not required Data Fashion-MNIST subset, 12,000 images

By the end you can

  • Load VGG16 or VGG19 without its classifier and turn images into fixed-length feature vectors.
  • Train an SVM and a Random Forest on those vectors and on raw pixels.
  • Say why features must be scaled for an SVM, and fitted on training data only.
  • Compare features from early, middle and late layers.
  • Decide when features plus classical ML is a better choice than fine-tuning.

Before the lab

The whole lab in one picture

Images in, features out, classifier on top

Select a stage to see its shape and what happens to it. Only the last stage is trained, and it is trained in seconds.

Tasks 4, 5 and 10 · Choose the features

How big is the feature vector?

The output of any VGG block can serve as features. Global Average Pooling turns its maps into one number per channel, so the vector length is the channel count, whatever the image size.

Extractor settings

Network
Take features after

Each block ends in a 2×2 MaxPool, so the size halves (rounded down). VGG needs at least 32×32 input.

What you get

GAP(A_c) = (1 / H·W) · Σᵢⱼ A_c(i, j)

Cost of the extraction pass

Convolution parameters up to each block. These are frozen: they are only used in a forward pass, and the pass runs once per image.

Tasks 6 to 8 · The classifier

Four classic choices on the same features

Pick a classifier to see how it treats a 512-number feature vector.


    

  

  

    
Scaling and data leakage

    
z = (f − μ_train) / σ_train      fit on the training set only

    

    

      

    

    

  










  
Tasks 7 and 8 · Hyperparameters

  
Two knobs that matter: SVM C and gamma, and the forest size

  
Both charts are illustrations of typical behaviour. Your grid search and forest will give different numbers, but the shape should match.

  

    

      
SVM (RBF): C against gamma

      
Select a cell. Validation accuracy of the whole grid, standardised features.

      

      

      

    

    

      
Random Forest: trees and depth

      

      
max_depth 5max_depth 15no limit

      
More trees reduce variance and then level off, so past a point they only cost time. Depth controls how closely each tree fits the training data.

    

  










  
Tasks 6 to 10 · Predict the ranking

  
Do CNN features beat raw pixels?

  
These are typical ranges for this setup, not measurements. Fashion-MNIST pixels are already a good representation, so the gain from VGG features here can be small. On cluttered colour photos the gap is much larger. Decide your prediction first, then run the tasks.

  

    

      

      
Bars show the middle of each range. Training time for the SVM grows quickly with the number of samples, while a forest can use all CPU cores.

    

  










  
Connect to Lab 4

  
Features plus classifier, or fine-tuning?

  
Both reuse a pretrained network. They differ in what is allowed to change.

  

    

      
Features + SVM or forest

      
    
        
  • The network never changes, so there is no risk of forgetting.
    • 
        
  • Features are extracted once. Trying many classifiers or settings takes seconds.
    • 
        
  • Works with a CPU, small data and existing scikit-learn pipelines.
    • 
        
  • Cannot adapt to a very different domain.
    • 
      

    

    

      
Fine-tuning (Lab 4)

      
    
        
  • Top layers adapt to your data, which usually gives the highest accuracy.
    • 
        
  • Needs a GPU, a low learning rate and care against overfitting.
    • 
        
  • Every experiment means training the network again.
    • 
        
  • Needs more labelled data to be worthwhile.
    • 
      

    

  










  
Hands-on · Google Colab

  
The Colab tasks

  
Copy each cell into Colab and run them in order. Comments in the code explain each step. Feature extraction runs once, and everything after it uses the saved arrays.

  










  
Reflection

  
Write it in your own words

  
Answers are saved in this browser only. Open a hint after you have tried.