Neural Network Programming Assignment

Learning Objectives

After completing this assignment, students should be able to:

• Implement a multi-layer neural network using low-level TensorFlow

Part 1: Implementation

Complete the stubbed-out MLP classifier class in tf_classifiers.py:

• tf_classfiers.py

This module contains some utility methods for generating datasets that can be used to test your implementation:

• datasets.py

This module contains some sample invocations of the classifiers in tf_classifier.py:

• tf_classifier_examples.py

Hints/Suggestions

• For logistic regression, weight initialization isn't particularly important. Since the loss function is convex, we are guaranteed to find the minima regardless of the initial weights. This is not the case for multi-layer neural networks. Good weight initialization can make a big difference in avoiding bad local minima during training. You will probably need to do some research into weight initialization or at least experiment with different approaches.

Part 2: Analysis

Submit a completed version of cifar_challenge.py along with a short document describing the steps that you used to maximize performance. There should be enough information in the document for a reader to replicate your network and training procedure. The document should include tensorboard screenshots illustrating your experiments and should include your final testing performance.

Partners

This assignment may be completed individually or in pairs. My expectation for pairs is that both members are actively involved, and take full responsibility for, all aspects of the project. In other words, I expect that you are sitting down together to work, not that you are splitting up tasks to be completed separately.

If you intend to work with a partner, you must inform me before you start working on the project.

Grading

Grades will be calculated according to the following distribution.

• Readability/Style 10%

  Your code should follow PEP8 conventions. It should be well documented and well organized.

• Basic Functionality 60%

  Your submission should illustrate that your MLP is able to reliably learn the noisy_xor dataset defined in datasets.py.

• Refactoring 10%

  The starter implementation requires separate training loops for Logistic regression and MLP. This functionality should be re-factored into the superclass so that any appropriately structured subclass can be trained using the same logic.

• Part 2 Submission 20%