Key Word(s): Convolutional Neural Network, Neural Networks, CNNs, MPL

Download Notebook

Title :

Pooling Mechanics

Description :

The aim of this exercise is to understand the tensorflow.keras implementation of:

  1. Max Pooling
  2. Average Pooling

Instructions :

First, implement Max Pooling by building a model with a single MaxPooling2D layer. Print the output of this layer by using model.predict() to show the output.

Next, implement Average Pooling by building a model with a single AvgPooling2D layer. Print the output of this layer by using model.predict() to show the output.

Hints:

tf.keras.layers.MaxPooling2D()Max pooling operation for 2D spatial data.

tf.keras.layers.AveragePooling2D()Average pooling operation for spatial data.

np.squeeze()Remove single-dimensional entries from the shape of an array.

np.expand_dims()Add single-dimensional entries from the shape of an array.

Example: np.expand_dims (img, axis=(0,3))

In [2]:
# Import necessary libraries 
import matplotlib.pyplot as plt
import numpy as np
import tensorflow as tf
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import MaxPool2D,AveragePooling2D,Input
from helper import plot_pool

---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
 in 
      2 import matplotlib.pyplot as plt
      3 import numpy as np
----> 4 import tensorflow as tf
      5 from tensorflow.keras.models import Sequential
      6 from tensorflow.keras.layers import MaxPool2D,AveragePooling2D,Input

ModuleNotFoundError: No module named 'tensorflow'
In [20]:
# Load the 7x7 mnist image 
img = np.load('3.npy')
plt.imshow(img,cmap = 'bone', alpha=0.5);
plt.axis('off');
plt.title('MNIST image of 3',fontsize=20);

⏸ Consider an input of size $(7,7)$ pixels.What will be the dimensions of the output if you use pool_size=2, strides = 1 & padding='valid'?

A. $(5,5)$

B. $(6,6)$

C. $(4,4)$

D. $(7,7)$

In [0]:
### edTest(test_chow1) ###
# Submit an answer choice as a string below (eg. if you choose option C, put 'C')
answer1 = '___'

Max Pooling

In [16]:
# Specify the variables for pooling 
pool_size = ___
strides = ___

# Padding parameter can be 'valid', 'same', etc.
padding = '___'

# Build the model to perform maxpooling operation 
model_1 = Sequential(name = 'MaxPool')
model_1.add(Input(shape = np.expand_dims(img,axis=2).shape))
model_1.add(MaxPool2D(pool_size = pool_size,strides=strides, padding=padding))

# Take a look at the summary to see the output shape
model_1.summary()

Model: "MaxPool"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
max_pooling2d_3 (MaxPooling2 (None, 6, 6, 1)           0         
=================================================================
Total params: 0
Trainable params: 0
Non-trainable params: 0
_________________________________________________________________
In [17]:
# Output the image using the model above 
# Remember to use np.expand_dims to change input image dimensions
# to 4-d tensor because model_1.predict will not work on 2-d tensor
pooled_img = model_1.predict(___)


# Use the helper code to visualize the pooling operation 
# np.squeeze() is used to bring the image to 2-dimension 
# to use matplotlib to plot it
pooled_img = pooled_img.squeeze()

# plot_pool is a function that will return 3 plots to help visualize
# the pooling operation
plot_pool(img,pooled_img)

WARNING:tensorflow:5 out of the last 5 calls to .predict_function at 0x1442bb700> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has experimental_relax_shapes=True option that relaxes argument shapes that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/tutorials/customization/performance#python_or_tensor_args and https://www.tensorflow.org/api_docs/python/tf/function for  more details.

⏸ What if your stride is larger than your pool size?

A. Operation is invalid

B. Operation is valid but you will have an output larger than the input

C. Operation is valid but you will miss out on some pixels

D. Operation is valid but you will have an output as the same size as the input

In [0]:
### edTest(test_chow2) ###
# Submit an answer choice as a string below 
# (eg. if you choose option C, put 'C')
answer2 = '___'

Average Pooling

In [18]:
# Specify the variables for pooling 
pool_size = ___
strides = ___

# Padding parameter can be 'valid', 'same', etc.
padding = '___'

# Build the model to perform average pooling operation 
model_2 = Sequential(name = 'AveragePool')
model_2.add(Input(shape = np.expand_dims(img,axis=2).shape))
model_2.add(AveragePooling2D(pool_size = pool_size,strides=strides, padding=padding))
model_2.summary()

Model: "AveragePool"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
average_pooling2d_1 (Average (None, 2, 2, 1)           0         
=================================================================
Total params: 0
Trainable params: 0
Non-trainable params: 0
_________________________________________________________________
In [19]:
# Output the image using the model above 
# Remember to use np.expand_dims to change input image dimensions
# to 4-d tensor because model_1.predict will not work on 2-d tensor
pooled_img = model_2.predict(___)


# Use the helper code to visualize the pooling operation 
pooled_img = pooled_img.squeeze()
plot_pool(img,pooled_img)

WARNING:tensorflow:6 out of the last 6 calls to .predict_function at 0x1091618b0> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has experimental_relax_shapes=True option that relaxes argument shapes that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/tutorials/customization/performance#python_or_tensor_args and https://www.tensorflow.org/api_docs/python/tf/function for  more details.

⏸ Which among the following 2 pooling operation activates the input image more? Answer based on your results above.

A. Average pooling

B. Max pooling

In [1]:
### edTest(test_chow3) ###
# Submit an answer choice as a string below 
# (eg. if you choose option A, put 'a')
answer3 = '___'