{ "cells": [ { "cell_type": "markdown", "metadata": { "deletable": false, "editable": false, "nbgrader": { "cell_type": "markdown", "checksum": "b6538b44d75745eb0d472cbc3a8dc8fe", "grade": false, "grade_id": "cell-e06c8e0fb875a17c", "locked": true, "schema_version": 3, "solution": false, "task": false } }, "source": [ "# Data Sets And Numpy\n", "**Enter the names of all group members in the cell below**. If you are submitting this individually, enter your name, along with the names of anyone you worked with while completing these exercises.\n" ] }, { "cell_type": "markdown", "metadata": { "deletable": false, "nbgrader": { "cell_type": "markdown", "checksum": "dc7ac7a1b3b40de7873711d34e89eb77", "grade": true, "grade_id": "Names", "locked": false, "points": 1, "schema_version": 3, "solution": true, "task": false } }, "source": [ "YOUR ANSWER HERE" ] }, { "cell_type": "markdown", "metadata": { "deletable": false, "editable": false, "nbgrader": { "cell_type": "markdown", "checksum": "bc5a9812db90a09513e0e28349257f6d", "grade": false, "grade_id": "cell-6febd7c31efe7fae", "locked": true, "schema_version": 3, "solution": false, "task": false } }, "source": [ "## Data Organization\n", "A learning example is comprised of a set of *features* (or *attributes*, or *dimensions*). In the case where the features are real-valued, each example can be considered a *vector* or a *point*. Let $x$ represent an individual example. Let $d$ be the dimensionality of $x$ Thus, each example $x$ is a single point where $x \\in \\mathbb{R}^{d}$.\n", "\n", "In Python, example data is typically stored in matrix form. In ML, this data can be\n", "stored 2 ways, in rows, as shown on the left, and in columns, as is shown on the right. \n", "\n", "$\\begin{bmatrix} \n", "-- & x^{(1)} & -- \\\\\n", "-- & x^{(2)} & -- \\\\ \n", "-- & ... & -- \\\\\n", "-- & x^{(m)} & -- \\\\\n", "\\end{bmatrix}$ \n", "or\n", "$\\begin{bmatrix} \n", "| & | & | & | \\\\\n", "x^{(1)} & x^{(2)}& ... & x^{(m)} \\\\\n", "| & | & | & | \\\\\n", "\\end{bmatrix}$" ] }, { "cell_type": "markdown", "metadata": { "deletable": false, "editable": false, "nbgrader": { "cell_type": "markdown", "checksum": "2e64ee09126a7e9d7a642c12394a54c3", "grade": false, "grade_id": "cell-6bb0d9f3aaeb1b38", "locked": true, "schema_version": 3, "solution": false, "task": false } }, "source": [ "### QUESTION\n", "* Consider two points, $x^{(1)}$ = (1,3) and $x^{(2)}$ = (2,5).\n", "Calculate, by hand, the Euclidean distance between these two points.\n", "Recall that the the formula for Euclidean distance is $$\\sqrt { \\sum_{i=1}^{d}{(x_{i}^{(1)} - x_{i}^{(2)})^2}}$$. \n" ] }, { "cell_type": "markdown", "metadata": { "deletable": false, "nbgrader": { "cell_type": "markdown", "checksum": "b3f815fb85ffd35495e811b15331db29", "grade": true, "grade_id": "ManualDistance", "locked": false, "points": 1, "schema_version": 3, "solution": true, "task": false } }, "source": [ "YOUR ANSWER HERE" ] }, { "cell_type": "markdown", "metadata": { "deletable": false, "editable": false, "nbgrader": { "cell_type": "markdown", "checksum": "8d394feb7f0af31fd1987f8f56164264", "grade": false, "grade_id": "cell-7059d45fd3b4485d", "locked": true, "schema_version": 3, "solution": false, "task": false } }, "source": [ "---\n", "## PART 1 - Processing Numpy Arrays Using Loops\n", "\n", "Until now, you have primarily used *for* loops for performing mathematical operations\n", "on arrays of numbers. As an example of doing this in Python, you can use a loop to work \n", "with a numpy array:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import numpy as np\n", "np.set_printoptions(precision=3) # Keeps output from getting cluttered.\n", "\n", "\n", "x = np.random.rand(10)\n", "\n", "sum_x = 0\n", "\n", "for i in range(x.shape[0]):\n", " sum_x += x[i]\n", "\n", "print(sum_x)\n" ] }, { "cell_type": "markdown", "metadata": { "deletable": false, "editable": false, "nbgrader": { "cell_type": "markdown", "checksum": "99be7a68c5bcd44c15424623de7de5d7", "grade": false, "grade_id": "cell-d1cb05ecf502f535", "locked": true, "schema_version": 3, "solution": false, "task": false } }, "source": [ "## Activity 1\n", "\n", "In the cell below, complete the `distance_loop` function using a for loop. You can test your implementation by executing the next cell. (To be clear: using loops is *not* a good idea here. We'll develop a better implementation later.)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "deletable": false, "nbgrader": { "cell_type": "code", "checksum": "8b773b8a06c410f2f677b5d3a1832550", "grade": false, "grade_id": "cell-7462c972b77c842e", "locked": false, "schema_version": 3, "solution": true, "task": false } }, "outputs": [], "source": [ "# Write the distance_loop method\n", "\n", "def distance_loop(x1, x2):\n", " \"\"\" Returns the Euclidean distance between the 1-d numpy arrays x1 and x2\"\"\"\n", " \n", " # YOUR CODE HERE\n", " raise NotImplementedError()" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "deletable": false, "editable": false, "nbgrader": { "cell_type": "code", "checksum": "aa60b8f56bb72d741415c470f9e31d68", "grade": true, "grade_id": "act_1_test", "locked": true, "points": 5, "schema_version": 3, "solution": false, "task": false } }, "outputs": [], "source": [ "# Tests for the distance_loop method.\n", "\n", "import numpy as np\n", "import math\n", "\n", "a = np.array([1.0, 2.0, 3.0])\n", "b = np.array([0.0, 1.0, 4.0])\n", "\n", "test_distance = distance_loop(a, b) \n", "expected_answer = 1.7320508075688772\n", "\n", "np.testing.assert_allclose(test_distance, expected_answer)\n", "np.testing.assert_allclose(distance_loop(b, a), expected_answer)\n", "np.testing.assert_allclose(distance_loop(a, a), 0)\n" ] }, { "cell_type": "markdown", "metadata": { "deletable": false, "editable": false, "nbgrader": { "cell_type": "markdown", "checksum": "409630edd85e88b6af0e5beb2286da64", "grade": false, "grade_id": "cell-95ae67b6e242c685", "locked": true, "schema_version": 3, "solution": false, "task": false } }, "source": [ "## Activity 2\n", "Very soon we will study the *K-Nearest Neighbors* algorithm. The simplest version is 1-Nearest Neighbors, which classifies a target point by searching through the training data to find the single point that is **nearest** to the target, and using the class of that point as a prediction. Using a **for loop**, and your completed `distance_loop` function, complete the `nearest_loop` function in the cell below (which would be useful for implementing 1-Nearest Neighbors classifier). \n", "\n", "Here are some potentially helpful reminders:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# The shape property returns a tuple containing the shape of a numpy array:\n", "\n", "X = np.random.random((5,3))\n", "print(X)\n", "print(X.shape)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Slicing can be used to extract a single row or column from a numpy array. \n", "# This extracts the middle column from X:\n", "\n", "print(X[:, 1])" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "deletable": false, "nbgrader": { "cell_type": "code", "checksum": "8e675e4b9ac7ab59cb9c1f30dbc3b9b2", "grade": false, "grade_id": "cell-fee525bf642aedb2", "locked": false, "schema_version": 3, "solution": true, "task": false }, "pycharm": { "is_executing": true } }, "outputs": [], "source": [ "# ##\n", "# Exercise: Complete nearest_loop\n", "# ##\n", "\n", "def nearest_loop(X, target):\n", " \"\"\" Return the index of the nearest point in X to target.\n", " Arguments:\n", " X - An m x d numpy array storing m points of dimension d\n", " target - a length-d numpy array\n", " \"\"\"\n", " # YOUR CODE HERE\n", " raise NotImplementedError()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Execute the cell below to test your implementation." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "deletable": false, "editable": false, "nbgrader": { "cell_type": "code", "checksum": "6dbab71ecccd9a6bc5259ffb4e6e5294", "grade": true, "grade_id": "act_2_test", "locked": true, "points": 5, "schema_version": 3, "solution": false, "task": false } }, "outputs": [], "source": [ "# Tests for nearest_loop\n", "\n", "data = np.array([[0.71, 0.74, 0.06],\n", " [0.86, 0.5 , 0.18],\n", " [0.35, 0.47, 0.03],\n", " [0.57, 0.32, 0.65],\n", " [0.3 , 0.17, 0.21],\n", " [0.15, 1. , 0.74]])\n", "\n", "assert(nearest_loop(data, np.array([.6, .3, .7])) == 3)\n", "assert(nearest_loop(data, np.array([.1, 1.0, .7])) == 5)\n", "assert(nearest_loop(data, np.array([.7, .7, .0])) == 0) " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# PART 2 - Processing Numpy Arrays WITHOUT Loops\n", "\n", "Loops in Python are *sllloooowwww*. The key to writing fast numerical programs in Python is to avoid loops by taking advantage of numpy operators and library calls.\n", "\n", "## Activity 3\n", "\n", "Complete the `distance` function in the cell below. The functionality should be exactly the same as `distance_loop`, but your implementation *should not contain any loops* (or list comprehensions). Execute the following cell to test your implementation and compare the speed to our previous version." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "deletable": false, "nbgrader": { "cell_type": "code", "checksum": "90a0a689740d2ae352f5a55165f09b26", "grade": false, "grade_id": "cell-cd69ff25922adf20", "locked": false, "schema_version": 3, "solution": true, "task": false } }, "outputs": [], "source": [ "def distance(x1, x2):\n", " \"\"\" Returns the Euclidean distance between the 1-d numpy arrays x1 and x2\"\"\"\n", " # Note: This should be a one-liner!\n", " # YOUR CODE HERE\n", " raise NotImplementedError()" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "deletable": false, "editable": false, "nbgrader": { "cell_type": "code", "checksum": "5e94e555e57156ee6039fc612d66d250", "grade": true, "grade_id": "act_3_test", "locked": true, "points": 5, "schema_version": 3, "solution": false, "task": false }, "pycharm": { "is_executing": true } }, "outputs": [], "source": [ "a = np.array([1.0, 2.0, 3.0])\n", "b = np.array([0.0, 1.0, 4.0])\n", "\n", "dist_a_b = distance(a, b)\n", "expected_dist = 1.7320508075688772\n", "\n", "np.testing.assert_allclose(dist_a_b, expected_dist)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "pycharm": { "is_executing": true } }, "outputs": [], "source": [ "# TEST SPEED\n", "dim = 10000 # Try changing this value to see how it impacts the running times.\n", "a = np.random.random(dim)\n", "b = np.random.random(dim)\n", "print('time for distance_loops')\n", "\n", "%timeit distance_loop(a, b)\n", "\n", "print('--------\\n\\nTime for distance without loops')\n", "%timeit distance(a, b)" ] }, { "cell_type": "markdown", "metadata": { "deletable": false, "editable": false, "nbgrader": { "cell_type": "markdown", "checksum": "08b2b17f5718c1e6bf5ab45ca0d6a345", "grade": false, "grade_id": "cell-a37d57086287e042", "locked": true, "schema_version": 3, "solution": false, "task": false } }, "source": [ "## Activity 4\n", "\n", "Complete a method named `nearest` who's functionality should be exactly the same as `nearest_loop`, but your implementation should **not** contain any loops (or list comprehensions). Execute the following cells to test your implementation and compare the speed to the earlier version. Note that your updated implementation should *not* call your `distance` function... doing so would require you to loop through all of the rows of `X`.\n", "\n", "Here are a few useful examples of using numpy that may prove helpful when writing your code." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "pycharm": { "is_executing": true } }, "outputs": [], "source": [ "# \"Broadcasting\" can be used to apply the same \n", "# operation across all rows/columns of an array:\n", "\n", "X = np.random.random((5, 3))\n", "a = np.array([10, 20, 30])\n", "\n", "print(\"X:\\n{}\".format(X))\n", "print(\"a:\\n{}\".format(a))\n", "print(\"\\nX + a:\\n{}\".format(X + a)) # Broadcasting!" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# argmin can be used to find the index of the smallest element in an array:\n", "\n", "b = np.array([5, 5, 1, 5])\n", "print(np.argmin(b))" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "deletable": false, "nbgrader": { "cell_type": "code", "checksum": "f6f276aba75c55d0240ecfcdb4c92a3a", "grade": false, "grade_id": "cell-aed3f8d83b95cc5a", "locked": false, "schema_version": 3, "solution": true, "task": false }, "pycharm": { "is_executing": true } }, "outputs": [], "source": [ "def nearest(X, target):\n", " \"\"\" Return the index of the nearest point in X to target.\n", " Arguments:\n", " X - An m x d numpy array storing m points of dimension d\n", " target - a length-d numpy array\n", " \"\"\"\n", " # ADVICE: \n", " # - Use a properly broadcast subtraction operation to subtract the target point\n", " # from each point in X using one statement.\n", " # - Square all of the resulting differences using a single **2 operation.\n", " # - use np.sum along with an appopriate axis argument to sum the squared\n", " # values across the rows.\n", " # - You can take the sqrt of the results, but it isn't necessary (Why?)\n", " # - use np.argmin to find the index of the smallest distance.\n", " \n", " # YOUR CODE HERE\n", " raise NotImplementedError()" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "deletable": false, "editable": false, "nbgrader": { "cell_type": "code", "checksum": "043ebd2b49604819c922615487656631", "grade": true, "grade_id": "act_4_test", "locked": true, "points": 5, "schema_version": 3, "solution": false, "task": false }, "pycharm": { "is_executing": true } }, "outputs": [], "source": [ "# Tests for nearest\n", "\n", "data = np.array([[0.71, 0.74, 0.06],\n", " [0.86, 0.5 , 0.18],\n", " [0.35, 0.47, 0.03],\n", " [0.57, 0.32, 0.65],\n", " [0.3 , 0.17, 0.21],\n", " [0.15, 1. , 0.74]])\n", "\n", "assert(nearest(data, np.array([.6, .3, .7])) == 3)\n", "assert(nearest(data, np.array([.1, 1.0, .7])) == 5)\n", "assert(nearest(data, np.array([.7, .7, .0])) == 0) " ] }, { "cell_type": "code", "execution_count": null, "metadata": { "pycharm": { "is_executing": true } }, "outputs": [], "source": [ "# TEST SPEED\n", "dim = 3\n", "num = 100000\n", "data1 = np.random.random((num, dim))\n", "point1 = np.random.random(dim)\n", "\n", "print('timing nearest_loop')\n", "%timeit nearest_loop(data1, point1)\n", "print('-------\\n')\n", "\n", "print('timing nearest')\n", "%timeit nearest(data1, point1)" ] }, { "cell_type": "markdown", "metadata": { "deletable": false, "editable": false, "nbgrader": { "cell_type": "markdown", "checksum": "e6953cb24ebff0b43d7878bd505f9be6", "grade": false, "grade_id": "cell-e280a48306dc6233", "locked": true, "schema_version": 3, "solution": false, "task": false } }, "source": [ "# Part 3 - Numpy Masking\n", "\n", "\n", "The cell below loads a tiny data set of handwritten digits and displays the first digit as an image." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "deletable": false, "editable": false, "nbgrader": { "cell_type": "code", "checksum": "9a1babe8a807d4d6e27e5895ce20952e", "grade": false, "grade_id": "cell-74a0e8de3d46fe52", "locked": true, "schema_version": 3, "solution": false, "task": false }, "pycharm": { "is_executing": true } }, "outputs": [], "source": [ "from sklearn.datasets import load_digits\n", "digits, labels = load_digits(return_X_y=True)\n", "\n", "\n", "print(digits.shape)\n", "print(labels.shape)\n", "\n", "# Show the fist and second digits in the data set:\n", "import matplotlib.pyplot as plt \n", "plt.gray() \n", "plt.matshow(digits[0, :].reshape(8, 8)) ## make into a square matrix for matshow\n", "plt.show()\n", "plt.matshow(digits[20, :].reshape(8, 8)) ## make into a square matrix for matshow\n", "plt.show() " ] }, { "cell_type": "markdown", "metadata": { "deletable": false, "editable": false, "nbgrader": { "cell_type": "markdown", "checksum": "3b008b838b6c954f603bd9c285aa39f2", "grade": false, "grade_id": "cell-af9b3f5d0a48adaa", "locked": true, "schema_version": 3, "solution": false, "task": false } }, "source": [ "## Activity 5\n", "\n", "(Note that this activity is only worth 1 point. I encourage you to complete it if you have time, but don't stress about it if you need to move on to something else.) \n", "\n", "Use numpy masking and slicing to create a smaller data set that contains only the the 1's and 7's from the data set imported above. Store the images in an array named `digits_subset` and the labels in an array named `labels_subset`\n", "\n", "Numpy masking provides a fast and convient way of accessing array entries that satisfy logical conditions (see [section 1.4.1.7 Fancy indexing](https://scipy-lectures.org/intro/numpy/array_object.html#using-boolean-masks) in the reading material **scipy lectures** for a brief introduction). Here is an example:\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Use numpy operators to pull out all columns \n", "# with indices that are multiples of 3 or 7\n", "\n", "X = np.random.random((3, 10))\n", "print(\"X:\\n{}\".format(X))\n", "\n", "all_cols = np.arange(0, X.shape[1])\n", "\n", "print(\"\\nAll column indices: {}\".format(all_cols))\n", "\n", "# Which are multiples of three?\n", "cols_mult_3 = all_cols % 3 == 0\n", "\n", "print(\"\\nMultiples of three:\")\n", "print(cols_mult_3)\n", "\n", "# Which are multiples of seven?\n", "cols_mult_7 = all_cols % 7 == 0\n", "\n", "print(\"\\nMultiples of seven:\")\n", "print(cols_mult_7)\n", "\n", "# We can use logical operators to combine boolean numpy arrays:\n", "col_mask = cols_mult_3 | cols_mult_7\n", "\n", "print(\"\\nFinal mask:\")\n", "print(col_mask)\n", "\n", "X_reduced = X[:, col_mask]\n", "\n", "print(\"\\nX_reduced:\\n{}\".format(X_reduced))" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "deletable": false, "nbgrader": { "cell_type": "code", "checksum": "55ad81caf60d9051586e1ba70552be63", "grade": false, "grade_id": "cell-45eabab9b82a7b8d", "locked": false, "schema_version": 3, "solution": true, "task": false }, "pycharm": { "is_executing": true } }, "outputs": [], "source": [ "# Write code here to initialize the variables digits_subset and labels_subset.\n", "\n", "# YOUR CODE HERE\n", "raise NotImplementedError()\n", "\n", "print(digits_subset.shape) # should be (361, 64)\n", "print(labels_subset.shape) # should be (361,)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "deletable": false, "editable": false, "nbgrader": { "cell_type": "code", "checksum": "7678088525b0a61c693ef021788f4075", "grade": true, "grade_id": "act_5_test", "locked": true, "points": 1, "schema_version": 3, "solution": false, "task": false } }, "outputs": [], "source": [ "assert(digits_subset.shape == (361,64))\n", "assert(labels_subset.shape == (361,))\n", "assert(labels_subset[0] == 1)\n", "assert(labels_subset[-1] == 7)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "pycharm": { "is_executing": true } }, "outputs": [], "source": [ "# Look at some of the digits, assuming the variables have been initialized correctly.\n", "plt.gray() \n", "plt.matshow(digits_subset[0,:].reshape(8,8)) # should be a \"1\"\n", "plt.figure()\n", "plt.matshow(digits_subset[180,:].reshape(8,8)) # should be \"7\"\n", "plt.show() " ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.12.11" } }, "nbformat": 4, "nbformat_minor": 4 }