PA2 - Digit Recogniztion with Neural Networks

Partners

This assignment may be completed individually or in pairs. If you are doing this in pairs, you must notify me at the beginning of the project. My expectation for pairs is that both members are actively involved, and take full responsibility for all aspects of the project. In other words, I expect that you are either sitting or virtually together to work, and not that you are splitting up tasks to be completed separately. If both members of the group are not able to fully explain the code to me, then this does not meet this expectation.

Learning Objectives

After completing this activity, students should be able to:

• Build a simple Neural Network in Keras
• Implement a classifier that codes output in one-hot encoding
• Evaluate different network architectures
• Analyze and plot computational cost versus accuracy

Implementation

You will need to build 2 Keras models: m1_dense.py and m2_dense.py. The two models must use different hyperparameters. The hyperparameters that you should explore are:

• number of nodes
• number of layers
• activitation functions employed

For each model, you will justify the hyperparameters selected and why you think this will perform well. Justification involves generating supportive evidence from running your model (and not speculation). For example, plots of the training/validation loss and/or accuracy comparing the impact of adding nodes and/or layers. Above all, for each model, you must make an argument (with supporting evidence) that you are not overfitting.

Support Files

• keras_model_template.py – Template file for each of your Keras models
• keras_pre.py – A template for your preprocessing class. This function is responsible for recoding and reshaping your data as required for your model. You will need a preprocessing file for each of the 2 models you create (named m1_pre.py, m2_pre.py).
• keras_test_digit_model.py – Unit test program (same as used by Gradescope). Your programs MUST work when called from this program.
• MNIST_X_train.npy – training data stored as a numpy array
• MNIST_y_train.npy – training data labels stored as a numpy array
• MNIST_X_test.npy – test data stored as a numpy array
• MNIST_y_test.npy – test data labels stored as a numpy array

Compare Model Performance on Test Data

Run your two models on the test data and create:

• a single plot that compares their accuracy (call this Plot_1)
• a table (call this Table_1) that shows for each model the:
  • number of parameters
  • accuracy

Submission

You will need to submit the following files to gradescope:

• for your best performing model, save an h5 model file and name it cs445_mnist_dense_best.h5. You will need to include the pre-processing model with it (named cs445_mnist_dense_best_pre.py).
• the source code for both models (m1_dense.py, m1_pre.py, m2_dense.py, m2_pre.py)
• a PDF (named cs445_mnist_dense_summary.pdf )which contains
  • Plot_1
  • Table_1
  • Description of each model (w/hyperparameters) and justification for this model

Grading