PA3 - Digit Recogniztion with Convolutional Neural Networks

Partners

This assignment may be completed individually or in pairs. If you are doing this in pairs, you must notify me at the beginning of the project. My expectation for pairs is that both members are actively involved, and take full responsibility for all aspects of the project. In other words, I expect that you are either sitting or virtually together to work, and not that you are splitting up tasks to be completed separately. If both members of the group are not able to fully explain the code to me, then this does not meet this expectation.

Learning Objectives

After completing this activity, students should be able to:

• Build a NN that utilizes both convolution and dense layers
• Implement a classifier that codes output in one-hot encoding
• Evaluate different network architectures
• Analyze and plot computational cost versus accuracy

Implementation

You will need to build 2 Keras models: m3_cnn.py and m4_cnn.py. Both models must utlize convolution and one should include image augmentation. Examples of both of these techniques are provided in the lecture slides.

The two models must use different hyperparameters. The hyperparameters that you should explore are:

• number of convolutional kernels, including varying their size and number
• number of convolutional layers
• number of maxpool layers and dropout layers between convolutional layers
• the effects of adding image segmentation to reduce overfitting
• number of nodes in the dense
• activitation functions employed

For each model, you will justify the hyperparameters selected and why you think this will perform well. Justification involves generating supportive evidence from running your model (and not speculation). For example, plots of the training/validation loss and/or accuracy comparing the impact of adding nodes and/or layers. Above all, for each model, you must make an argument (with supporting evidence) that you are not overfitting.

Support Files

• keras_pre.py – A template for your preprocessing class. This function is responsible for recoding and reshaping your data as required for your model. You will need a preprocessing file for each of the 2 models you create (named m3_pre.py, m4_pre.py).
• keras_test_digit_model.py – Unit test program (same as used by Gradescope). Your programs MUST work when called from this program.
• MNIST_X_train.npy – training data stored as a numpy array
• MNIST_y_train.npy – training data labels stored as a numpy array
• MNIST_X_test.npy – test data stored as a numpy array
• MNIST_y_test.npy – test data labels stored as a numpy array

Compare Model Performance on Test Data

Run your two models on the test data and create:

• a single plot that compares their accuracies (call this Plot_1)
• a table (call this Table_1) that shows for each model the:
  • number of parameters
  • number of dropout layers
  • number of maxpooling layers
  • a flag indicating if image augmentation was utilized
  • accuracy

Submission

You will need to submit the following files to gradescope:

• for your best performing model, save an h5 model file and name it cs445_mnist_cnn_best.h5. You will need to include the pre-processing model with it (named appropriately).
• the source code for both models and both preprocessing files (m3_cnn.py, m3_pre.py, m4_cnn.py, m4_pre.py)
• a PDF which contains
  • Plot_1
  • Table_1
  • Description of each model (w/hyperparameters) and justification for this model

Grading