CIFAR Challenge!

Instructions

• CIFAR-10 is a small computer vision dataset that was an important benchmark for some of the seminal early work in deep-learning for computer vision.

• Download cifar_challenge.py and cifar_test.py.

• The provided model has:

  • 3072 input dimensions (32x32 images with three color channels)
  • 16 hidden units
  • 10 output units (one for each class)

  How many total parameters (weights and biases) will this network have? Once you have calculated your answer by hand, run cifar_challenge.py to check your work.

• Let the model run for the full 10 epochs, then start tensorboard to examine your model:

  tensorboard --logdir runs

  Examine the contents of the “Time Series” tab and the “Graphs” tab.

  Does it look like 10 epochs of training was enough? Too much? Are we continuing to see improvement in the validation loss and accuracy?

• Repeat the previous steps for the following two models:

  • Larger three layer model:
    • 3072 inputs
    • 1024 hidden units
    • 10 output units.
  • Deeper model:
    • 3072 inputs
    • 512 hidden units
    • 512 hidden units
    • 512 hidden units
    • 10 output units.

  For each network:

  • Change run_name in main() so you can tell the runs apart.
  • Run the model for 10 epochs.
  • Look at the results in Tensorboard.

  Which models are overfitting? How can you tell?

  For the two larger models, compare the validation loss with the validation accuracy. How is it possible for the validation loss to get worse while the validation accuracy stays about the same?

• Tune the model to get the best possible testing performance before the end of class. See the header comments in cifar_challenge.py for advice and PyTorch syntax.

  Keep a short log as you go: for each run, write down what you changed and the best validation accuracy you observed.

• Once you have run the testing code (once!), post your test accuracy and a one-sentence description of your network to the class leaderboard.