{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Make sure you fill in any place that says YOUR CODE HERE or \"YOUR ANSWER HERE\", as well as your name below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "NAME = \"王纪宝\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-e0f9b2de18190d9d",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "# Homework 2-1: Predicting Housing Prices\n",
    "\n",
    "### Collaboration Policy\n",
    "\n",
    "Data science is a collaborative activity. While you may talk with others about the homework, we ask that you **write your solutions individually**. If you do discuss the assignments with others please **include their names** at the top of your solution.\n",
    "\n",
    "## Introduction\n",
    "In this homework, we will go through the iterative process of specifying, fitting, and refining a model.  \n",
    "\n",
    "In the first portion of the assignment, we will guide you through some basic EDA, laying out the thought process that leads to certain modeling decisions.  We will then specify and fit two linear models, providing an example of the type of code we expect from you in the open-response.\n",
    "\n",
    "The second part of the assignment is purposefully left open-ended.  You will be allowed to build a linear model of your choice.\n",
    "\n",
    "After this homework, you should feel comfortable with the following:\n",
    "\n",
    "1. Working with a messy data set that requires a moderate amount of cleaning and wrangling\n",
    "1. Using sklearn to build models\n",
    "1. Using several different transformations on your data\n",
    "1. Building a data pipeline using pandas\n",
    "1. Using cross-validation for model selection\n",
    "\n",
    "### You may find some standoffish empty cells with only a comment in them in this assignment.  They are there on purpose; please do not attempt to modify them.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-03936654c721e35b",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "## Short Proofs\n",
    "\n",
    "We have data $(x_1, y_1) ... (x_n, y_n)$ drawn independently from the same distribution $P(x, y)$.  Our goal is to use a linear model to describe the relationship between $x$ and $y$.\n",
    "\n",
    "$$ f_\\theta(x) = \\theta_1 + \\theta_2 x $$\n",
    "\n",
    "We fit $\\hat{\\theta}_1$ and $\\hat{\\theta}_2$ by minimizing average $L_2$ loss $$ \\hat\\theta = \\text{argmin}_{\\theta_1, \\theta_2}\\frac{1}{n}\\sum_{i=1}^n (y_i - f_\\theta(x_i))^2$$.\n",
    "\n",
    "##### Part a <a name=\"q1a\"></a>\n",
    "The fitted/predicted y-values are given by:\n",
    "\n",
    "$$ \\hat{y}_i = f_\\hat\\theta(x_i) = \\hat\\theta_1 + \\hat\\theta_2 x_i $$\n",
    "\n",
    "The **residuals** are defined as:\n",
    "\n",
    "$$ e_i = \\text{observed } y_i - \\text{predicted } y_i = y_i - \\hat y_i $$\n",
    "\n",
    "Then the sum of the residuals is equal to 0.  In other words, \n",
    "\n",
    "$$\\sum_{i=1}^n e_i = 0$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-62cfd21463535cac",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "import pandas as pd\n",
    "from pandas.api.types import CategoricalDtype\n",
    "#魔法函数，功能是可以内嵌绘图，并且可以省略掉plt.show()这一步\n",
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "\n",
    "from sklearn import linear_model as lm\n",
    "from sklearn.model_selection import train_test_split  # 切分数据集为训练集和验证集\n",
    "from sklearn.model_selection import KFold   # K折叠交叉验证\n",
    "\n",
    "# Plot settings\n",
    "plt.rcParams['figure.figsize'] = (12, 9)\n",
    "plt.rcParams['font.size'] = 12"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-f68729731e7fe39d",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "# The Data\n",
    "\n",
    "The Ames data set consists of 2930 records taken from the Ames Assessor’s Office.  The data set has 23 nominal, 23 ordinal, 14 discrete, and 20 continuous variables (and 2 additional observation identifiers) --- 82 features in total.  An explanation of each variable can be found in the included `codebook.txt` file.  The information was used in computing assessed values for individual residential properties sold in Ames, Iowa from 2006 to 2010.  **Some noise has been added to the actual sale price, so prices will not match official records.**\n",
    "\n",
    "The data are split into training and test sets with 2000 and 930 observations, respectively.  The actual sale price is withheld from you in the test set.  In addition, the test data are further split into public and private test sets.  When you upload a test set prediction onto Kaggle for validation, the score you receive will be calculated using the public test set.  The private test set will be used in a final evaluation of this homework assignment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-e8fea30adc9d489b",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "training_data = pd.read_csv(\"ames_train.csv\")\n",
    "test_data = pd.read_csv(\"ames_test.csv\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-9d6d509b6e854e10",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "As a good sanity check, we should at least verify that the data shape matches the description."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-c841a2de55691502",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "# 2000 observations and 82 features in training data\n",
    "assert training_data.shape == (2000, 82)\n",
    "# 930 observations and 81 features in test data\n",
    "assert test_data.shape == (930, 81)\n",
    "# SalePrice is hidden in the test data\n",
    "assert 'SalePrice' not in test_data.columns.values\n",
    "# Every other column in the test data should be in the training data\n",
    "assert len(np.intersect1d(test_data.columns.values, \n",
    "                          training_data.columns.values)) == 81"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-ce9acc2f62c96e59",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "The next order of business is getting a feel for the variables in our data.  The Ames data set contains information that typical homebuyers would want to know.  A more detailed description of each variable is included in `codebook.txt`.  **You should take some time to familiarize yourself with the codebook before moving forward.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-4e60a7a0cda5eecf",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['Order', 'PID', 'MS_SubClass', 'MS_Zoning', 'Lot_Frontage',\n",
       "       'Lot_Area', 'Street', 'Alley', 'Lot_Shape', 'Land_Contour',\n",
       "       'Utilities', 'Lot_Config', 'Land_Slope', 'Neighborhood',\n",
       "       'Condition_1', 'Condition_2', 'Bldg_Type', 'House_Style',\n",
       "       'Overall_Qual', 'Overall_Cond', 'Year_Built', 'Year_Remod/Add',\n",
       "       'Roof_Style', 'Roof_Matl', 'Exterior_1st', 'Exterior_2nd',\n",
       "       'Mas_Vnr_Type', 'Mas_Vnr_Area', 'Exter_Qual', 'Exter_Cond',\n",
       "       'Foundation', 'Bsmt_Qual', 'Bsmt_Cond', 'Bsmt_Exposure',\n",
       "       'BsmtFin_Type_1', 'BsmtFin_SF_1', 'BsmtFin_Type_2', 'BsmtFin_SF_2',\n",
       "       'Bsmt_Unf_SF', 'Total_Bsmt_SF', 'Heating', 'Heating_QC',\n",
       "       'Central_Air', 'Electrical', '1st_Flr_SF', '2nd_Flr_SF',\n",
       "       'Low_Qual_Fin_SF', 'Gr_Liv_Area', 'Bsmt_Full_Bath',\n",
       "       'Bsmt_Half_Bath', 'Full_Bath', 'Half_Bath', 'Bedroom_AbvGr',\n",
       "       'Kitchen_AbvGr', 'Kitchen_Qual', 'TotRms_AbvGrd', 'Functional',\n",
       "       'Fireplaces', 'Fireplace_Qu', 'Garage_Type', 'Garage_Yr_Blt',\n",
       "       'Garage_Finish', 'Garage_Cars', 'Garage_Area', 'Garage_Qual',\n",
       "       'Garage_Cond', 'Paved_Drive', 'Wood_Deck_SF', 'Open_Porch_SF',\n",
       "       'Enclosed_Porch', '3Ssn_Porch', 'Screen_Porch', 'Pool_Area',\n",
       "       'Pool_QC', 'Fence', 'Misc_Feature', 'Misc_Val', 'Mo_Sold',\n",
       "       'Yr_Sold', 'Sale_Type', 'Sale_Condition', 'SalePrice'],\n",
       "      dtype=object)"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "training_data.columns.values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-ba0f6926b0dafefb",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "# Guided Modeling\n",
    "\n",
    "In the first portion of the assignment, we will take you step-by-step through one cycle of the modeling process.  Along the way, we will provide commentary to give you a sense of the thought process that goes into building a model.\n",
    "\n",
    "## EDA\n",
    "Naturally, the first thing we want to do is get a feel for our data.  In this section, we will make a series of exploratory visualizations.  The plots we ask you to reproduce here are far from exhaustive.  **When you build your own model in the second part of this assignment, you will want to delve deeper into the data.**\n",
    "\n",
    "Note that we will perform EDA on the **training data** so that information from the test data does not influence our modeling decisions.\n",
    "\n",
    "### Sale Price\n",
    "We begin by examining a [raincloud plot](https://micahallen.org/2018/03/15/introducing-raincloud-plots/amp/?__twitter_impression=true) (yet another name for a combination of a KDE, a boxplot, and a boxplot all-in-one) of our target variable `SalePrice`.  At the same time, we also take a look at some descriptive statistics of this variable."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-15d483a695655cea",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 48844 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, axs = plt.subplots(nrows=2)  #建立画布和 axs（编号行优先）\n",
    "\n",
    "sns.distplot(\n",
    "    training_data['SalePrice'], \n",
    "    ax=axs[0],\n",
    ")\n",
    "sns.stripplot(\n",
    "    training_data['SalePrice'], \n",
    "    jitter=0.4, \n",
    "    size=3,\n",
    "    ax=axs[1],\n",
    "    alpha=0.3\n",
    ")\n",
    "sns.boxplot(\n",
    "    training_data['SalePrice'],\n",
    "    width=0.3, \n",
    "    ax=axs[1],\n",
    "    showfliers=False,\n",
    ")\n",
    "\n",
    "# Align axes\n",
    "spacer = np.max(training_data['SalePrice']) * 0.05\n",
    "xmin = np.min(training_data['SalePrice']) - spacer\n",
    "xmax = np.max(training_data['SalePrice']) + spacer\n",
    "axs[0].set_xlim((xmin, xmax))   # x轴上下限设定\n",
    "axs[1].set_xlim((xmin, xmax))\n",
    "\n",
    "# Remove some axis text\n",
    "axs[0].xaxis.set_visible(False)\n",
    "axs[0].yaxis.set_visible(False)\n",
    "axs[1].yaxis.set_visible(False)\n",
    "\n",
    "# Put the two plots together\n",
    "plt.subplots_adjust(hspace=0)\n",
    "\n",
    "# Adjust boxplot fill to be white\n",
    "axs[1].artists[0].set_facecolor('white')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-45e5037c06db70f0",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "count      2000.000000\n",
       "mean     180775.897500\n",
       "std       81581.671741\n",
       "min        2489.000000\n",
       "25%      128600.000000\n",
       "50%      162000.000000\n",
       "75%      213125.000000\n",
       "max      747800.000000\n",
       "Name: SalePrice, dtype: float64"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "training_data['SalePrice'].describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-592d5f41ebd67ee2",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "#### Question 1 <a name=\"q2\"></a>\n",
    "To check your understanding of the graph and summary statistics above, answer the following `True` or `False` questions:\n",
    "\n",
    "1. The distribution of `SalePrice` in the training set is left-skew.\n",
    "1. The mean of `SalePrice` in the training set is greater than the median.\n",
    "1. 75% of the houses in the training set sold for less than \\$213,125.00."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "q1-answer",
     "locked": false,
     "schema_version": 1,
     "solution": true
    }
   },
   "outputs": [],
   "source": [
    "# These should be True or False\n",
    "q1statement1 = False \n",
    "q1statement2 = True\n",
    "q1statement3 = True\n",
    "\n",
    "# YOUR CODE HERE\n",
    "#raise NotImplementedError()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "nbgrader": {
     "grade": true,
     "grade_id": "q1-tests",
     "locked": true,
     "points": 2,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "# Two of these statements are True.\n",
    "assert sum([q1statement1, q1statement2, q1statement3]) == 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "nbgrader": {
     "grade": true,
     "grade_id": "cell-843b895e8c775560",
     "locked": true,
     "points": 4,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "# Yes, this is a cell with just a comment."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-9e22aac9b45f88e3",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "### SalePrice vs Gr_Liv_Area\n",
    "\n",
    "Next, we examine `SalePrice` vs `Gr_Liv_Area`.  Now it's not entirely obvious what `Gr_Liv_Area` should be, so we'll need to consult `codebook.txt`.  We find:\n",
    "\n",
    "```\n",
    "Gr Liv Area (Continuous): Above grade (ground) living area square feet\n",
    "```\n",
    "\n",
    "Ok, so this variable represents the square footage of the house excluding anything built underground.  Some additional research (into real estate conventions) reveals that this value also excludes the garage space."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-02a467f8950ee680",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<seaborn.axisgrid.JointGrid at 0x240e95613c8>"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "[encoded data, 45068 characters]\n",
      "text/plain": [
       "<Figure size 432x432 with 3 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.jointplot(\n",
    "    x='Gr_Liv_Area', \n",
    "    y='SalePrice', \n",
    "    data=training_data,\n",
    "    stat_func=None,\n",
    "    kind=\"reg\",\n",
    "    ratio=4,\n",
    "    space=0,\n",
    "    scatter_kws={\n",
    "        's': 3,\n",
    "        'alpha': 0.25\n",
    "    },\n",
    "    line_kws={\n",
    "        'color': 'black'\n",
    "    }\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-e69fbfdd6101f836",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "We notice that there is a plausible linear relationship between house size and sale price, but the spread is wider at larger sale prices and above grade living areas.  There seem to be two particularly suspicious houses above 5000 square feet.\n",
    "\n",
    "#### Question 2 <a name=\"q3\"></a>\n",
    "What are the Parcel Indentification Numbers for the two houses with `Gr_Liv_Area` greater than 5000 sqft?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-eb0c9f329767dfc2",
     "locked": false,
     "schema_version": 1,
     "solution": true
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1014    908154235\n",
       "1471    908154195\n",
       "Name: PID, dtype: int64"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Hint: You can answer this question purely through pandas\n",
    "# q2house1 and q2house2 should be integers\n",
    "q2house1 = 908154235\n",
    "q2house2 = 908154195\n",
    "\n",
    "# YOUR CODE HERE\n",
    "#raise NotImplementedError()\n",
    "training_data['PID'][training_data['Gr_Liv_Area']>5000]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "nbgrader": {
     "grade": true,
     "grade_id": "cell-c54dffdb6bbe776f",
     "locked": true,
     "points": 2,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "assert isinstance(q2house1, int)\n",
    "assert isinstance(q2house2, int)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "nbgrader": {
     "grade": true,
     "grade_id": "cell-b28c0d0507c81282",
     "locked": true,
     "points": 4,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "# Yes, this is a cell with just a comment. Watcha gonna do?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-e6273150cc398987",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "#### Question 3 <a name=\"q4b\"></a>\n",
    "\n",
    "Write a function `remove_outliers` that removes outliers from a data set based off a threshold value of a variable.  For example, `remove_outliers(training_data, 'Gr_Liv_Area', upper=5000)` should return a data frame with only observations that satisfy `Gr_Liv_Area` less than or equal to 5000."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-9186ec2ca053d0aa",
     "locked": false,
     "schema_version": 1,
     "solution": true
    }
   },
   "outputs": [],
   "source": [
    "def remove_outliers(data, variable, lower=-np.inf, upper=np.inf):\n",
    "    \"\"\"\n",
    "    Input:\n",
    "      data (data frame): the table to be filtered\n",
    "      variable (string): the column with numerical outliers\n",
    "      lower (numeric): observations with values lower than this will be removed\n",
    "      upper (numeric): observations with values higher than this will be removed\n",
    "    \n",
    "    Output:\n",
    "      a winsorized data frame with outliers removed\n",
    "    \"\"\"\n",
    "        \n",
    "    # YOUR CODE HERE\n",
    "    #raise NotImplementedError()\n",
    "    data = data.loc[(data[variable] > lower) & (data[variable] < upper)]\n",
    "    return data\n",
    "\n",
    "training_data = remove_outliers(training_data, 'Gr_Liv_Area', upper=5000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "nbgrader": {
     "grade": true,
     "grade_id": "cell-1b16eb9d9ed74f3e",
     "locked": true,
     "points": 1,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "# Make sure that two observations were removed\n",
    "assert training_data.shape[0] == 1998"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "nbgrader": {
     "grade": true,
     "grade_id": "cell-f1aeeb0f6c025fe4",
     "locked": true,
     "points": 2,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "# No admittance.  Except on party business."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-7b287f5749ce8d38",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "### Neighborhood vs Sale Price"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-692437a76f5a0651",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 40440 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, axs = plt.subplots(nrows=2)\n",
    "\n",
    "sns.boxplot(\n",
    "    x='Neighborhood',\n",
    "    y='SalePrice',\n",
    "    data=training_data.sort_values('Neighborhood'),\n",
    "    ax=axs[0]\n",
    ")\n",
    "\n",
    "sns.countplot(\n",
    "    x='Neighborhood',\n",
    "    data=training_data.sort_values('Neighborhood'),\n",
    "    ax=axs[1]\n",
    ")\n",
    "\n",
    "# Draw median price\n",
    "axs[0].axhline(\n",
    "    y=training_data['SalePrice'].median(), \n",
    "    color='red',\n",
    "    linestyle='dotted'\n",
    ")\n",
    "\n",
    "# Label the bars with counts\n",
    "for patch in axs[1].patches:\n",
    "    x = patch.get_bbox().get_points()[:, 0]\n",
    "    y = patch.get_bbox().get_points()[1, 1]\n",
    "    axs[1].annotate(f'{int(y)}', (x.mean(), y), ha='center', va='bottom')\n",
    "    \n",
    "# Format x-axes\n",
    "axs[1].set_xticklabels(axs[1].xaxis.get_majorticklabels(), rotation=90)\n",
    "axs[0].xaxis.set_visible(False)\n",
    "\n",
    "# Narrow the gap between the plots\n",
    "plt.subplots_adjust(hspace=0.01)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-b1a7c4a82cbad2d7",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "From the plot above, it becomes clear that there is quite some variation in prices across neighborhoods.  Moreover, the amount of data available is not uniformly distributed among neighborhoods.  North Ames, for example, comprises almost 15% of the training data while Green Hill has a scant（不足的） 2 observations in this data set.\n",
    "\n",
    "One way we can deal with the lack of data from some neighborhoods is to create a new feature that bins neighborhoods together.  Let's dichotomize（对分） our neighborhoods in a very crude way: we'll take the top 3 neighborhoods measured by median `SalePrice` and identify them as \"rich neighborhoods\"; the other neighborhoods are not marked.\n",
    "\n",
    "#### Question 4 <a name=\"q5\"></a>\n",
    "\n",
    "Write a function that returns list of the top n most pricy neighborhoods as measured by our choice of aggregating function.  For example, in the setup above, we would want to call `find_rich_neighborhoods(training_data, 3, np.median)` to find the top 3 neighborhoods measured by median `SalePrice`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-0b6e60ab464a87ca",
     "locked": false,
     "schema_version": 1,
     "solution": true
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['StoneBr', 'NridgHt', 'NoRidge']"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def find_rich_neighborhoods(data, n=3, metric=np.median):\n",
    "    \"\"\"\n",
    "    Input:\n",
    "      data (data frame): should contain at least a string-valued Neighborhood\n",
    "        and a numeric SalePrice column\n",
    "      n (int): the number of top values desired\n",
    "      metric (function): function used for aggregating the data in each neighborhood.\n",
    "        for example, np.median for median prices\n",
    "    \n",
    "    Output:\n",
    "      a list of the top n richest neighborhoods as measured by the metric function\n",
    "    \"\"\"\n",
    "    # groupby先进行分组，然后利用agg在每个分组的各个列上运行函数\n",
    "    neighborhoods = data.groupby(\"Neighborhood\").agg(metric).sort_values(by=[\"SalePrice\"],ascending=False).iloc[:n].index.values.tolist()\n",
    "    \n",
    "   # YOUR CODE HERE\n",
    "    #raise NotImplementedError()\n",
    "    \n",
    "    return neighborhoods\n",
    "\n",
    "rich_neighborhoods = find_rich_neighborhoods(training_data, 3, np.median)\n",
    "rich_neighborhoods"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "nbgrader": {
     "grade": true,
     "grade_id": "cell-56545f1d10f9699e",
     "locked": true,
     "points": 2,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "# Check to see if the n argument works\n",
    "assert len(find_rich_neighborhoods(training_data, 5, np.median)) == 5\n",
    "assert isinstance(rich_neighborhoods, list)\n",
    "# Check to see if the list contains only strings\n",
    "assert all([isinstance(neighborhood, str) for neighborhood in rich_neighborhoods])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "nbgrader": {
     "grade": true,
     "grade_id": "cell-f58f1e4866635bf7",
     "locked": true,
     "points": 4,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "# Shoo."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Missing Data\n",
    "\n",
    "#### Question 5 <a name=\"q6\"></a>\n",
    "Let's see if our data set has any missing values.  Create a Series object containing the counts of missing values in each of the columns of our data set sorted from greatest to least.  The Series should be indexed by the variable names.  For example, `missing_counts['Fireplace_Qu']` should return 975."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-95890767879a9a12",
     "locked": false,
     "schema_version": 1,
     "solution": true
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Order                0\n",
       "PID                  0\n",
       "MS_SubClass          0\n",
       "MS_Zoning            0\n",
       "Lot_Frontage       352\n",
       "Lot_Area             0\n",
       "Street               0\n",
       "Alley             1865\n",
       "Lot_Shape            0\n",
       "Land_Contour         0\n",
       "Utilities            0\n",
       "Lot_Config           0\n",
       "Land_Slope           0\n",
       "Neighborhood         0\n",
       "Condition_1          0\n",
       "Condition_2          0\n",
       "Bldg_Type            0\n",
       "House_Style          0\n",
       "Overall_Qual         0\n",
       "Overall_Cond         0\n",
       "Year_Built           0\n",
       "Year_Remod/Add       0\n",
       "Roof_Style           0\n",
       "Roof_Matl            0\n",
       "Exterior_1st         0\n",
       "Exterior_2nd         0\n",
       "Mas_Vnr_Type        18\n",
       "Mas_Vnr_Area        18\n",
       "Exter_Qual           0\n",
       "Exter_Cond           0\n",
       "                  ... \n",
       "Bedroom_AbvGr        0\n",
       "Kitchen_AbvGr        0\n",
       "Kitchen_Qual         0\n",
       "TotRms_AbvGrd        0\n",
       "Functional           0\n",
       "Fireplaces           0\n",
       "Fireplace_Qu       975\n",
       "Garage_Type        113\n",
       "Garage_Yr_Blt      114\n",
       "Garage_Finish      114\n",
       "Garage_Cars          0\n",
       "Garage_Area          0\n",
       "Garage_Qual        114\n",
       "Garage_Cond        114\n",
       "Paved_Drive          0\n",
       "Wood_Deck_SF         0\n",
       "Open_Porch_SF        0\n",
       "Enclosed_Porch       0\n",
       "3Ssn_Porch           0\n",
       "Screen_Porch         0\n",
       "Pool_Area            0\n",
       "Pool_QC           1991\n",
       "Fence             1607\n",
       "Misc_Feature      1922\n",
       "Misc_Val             0\n",
       "Mo_Sold              0\n",
       "Yr_Sold              0\n",
       "Sale_Type            0\n",
       "Sale_Condition       0\n",
       "SalePrice            0\n",
       "Length: 82, dtype: int64"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "missing_counts = training_data.isnull().sum()\n",
    "missing_counts\n",
    "# YOUR CODE HERE\n",
    "#raise NotImplementedError()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "nbgrader": {
     "grade": true,
     "grade_id": "cell-93d71894d6084ce9",
     "locked": true,
     "points": 2,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "# Make sure your answer is a Series\n",
    "assert isinstance(missing_counts, pd.Series)\n",
    "# Make sure all columns are represented\n",
    "assert missing_counts.size == 82\n",
    "# Make sure your index values match column names\n",
    "assert set(missing_counts.index.values) == set(training_data.columns.values)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "nbgrader": {
     "grade": true,
     "grade_id": "cell-bec86d2c1e1fede0",
     "locked": true,
     "points": 4,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "# Well go on now.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-f1045c22c76ca88f",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "It turns out that if we look at the codebook carefully, some of these \"missing values\" aren't missing at all! The Assessor's Office just used `NA` to denote a special value or that the information was truly not applicable for one reason or another.  One such example is the `Fireplace_Qu` variable.\n",
    "```\n",
    "FireplaceQu (Ordinal): Fireplace quality\n",
    "\n",
    "       Ex\tExcellent - Exceptional Masonry Fireplace\n",
    "       Gd\tGood - Masonry Fireplace in main level\n",
    "       TA\tAverage - Prefabricated Fireplace in main living area or Masonry Fireplace inbasement\n",
    "       Fa\tFair - Prefabricated Fireplace in basement\n",
    "       Po\tPoor - Ben Franklin Stove\n",
    "       NA\tNo Fireplace\n",
    "```\n",
    "\n",
    "#### Question 6 <a name=\"q7\"></a>\n",
    "\n",
    "An `NA` here actually means that the house had no fireplace to rate.  Let's fix this in our data set.  Write a function that replaces the missing values in `Fireplace_Qu` with `'No Fireplace'`.  In addition, it should replace each abbreviated condition with its full word.  For example, `'TA'` should be changed to `'Average'`.  Hint: the [DataFrame.replace](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.replace.html) method may be useful here.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-0fc3d70040e13894",
     "locked": false,
     "schema_version": 1,
     "solution": true
    }
   },
   "outputs": [],
   "source": [
    "def fix_fireplace_qu(data):\n",
    "    \"\"\"\n",
    "    Input:\n",
    "      data (data frame): a data frame containing a Fireplace_Qu column.  Its values\n",
    "                         should be limited to those found in the codebook\n",
    "    Output:\n",
    "      data frame identical to the input except with a refactored Fireplace_Qu column\n",
    "    \"\"\"\n",
    "    \n",
    "    # YOUR CODE HERE\n",
    "    #raise NotImplementedError()\n",
    "    return data.replace({'Fireplace_Qu' : { 'Ex' : 'Excellent', 'Gd' : 'Good',\n",
    "                                           'TA' : 'Average', 'Fa' : 'Fair',\n",
    "                                          'Po' : 'Poor', np.NaN : 'No Fireplace'}})\n",
    "    \n",
    "training_data = fix_fireplace_qu(training_data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "nbgrader": {
     "grade": true,
     "grade_id": "cell-37e22ea673b500be",
     "locked": true,
     "points": 2,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "# Make sure you've replaced all the missing values with 'No Fireplace'\n",
    "assert sum(training_data['Fireplace_Qu'] == 'No Fireplace') == 975"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "nbgrader": {
     "grade": true,
     "grade_id": "cell-2ecd79a12548f1f7",
     "locked": true,
     "points": 4,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "# Bah humbug.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-03bb29004f9a9837",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "It turns out that simply fixing these missing values isn't sufficient for using `Fireplace_Qu` in our model.  Since `Fireplace_Qu` is a categorical/nominal variable, we will have to one-hot-encode the data.  Notice in the example code below that we have to pre-specify the categories.  Why? Imagine what would happen if we automatically generated the categories only from the training data.  What would happen if the testing data contained a category not found in the training set?  For more information on categorical data in pandas, refer to this [link](https://pandas-docs.github.io/pandas-docs-travis/categorical.html).  **Note that `get_dummies` removes the original column.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-990396e0b4792eab",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "#One-Hot-Encode编码某一个类别类型的列，必然一列分成了好几列\n",
    "def ohe_fireplace_qu(data):\n",
    "    \"\"\"\n",
    "    One-hot-encodes fireplace quality.  New columns are of the form fpq_QUALITY\n",
    "    \"\"\"\n",
    "    cats = [\n",
    "        'Excellent',\n",
    "        'Good',\n",
    "        'Average',\n",
    "        'Fair',\n",
    "        'Poor',\n",
    "        'No Fireplace'\n",
    "    ]\n",
    "    \n",
    "    cat_type = CategoricalDtype(categories=cats)\n",
    "    \n",
    "    data['Fireplace_Qu'] = data['Fireplace_Qu'].astype(cat_type)\n",
    "    data = pd.get_dummies(data,\n",
    "                          prefix='fpq',\n",
    "                          columns=['Fireplace_Qu'], \n",
    "                          drop_first=True)\n",
    "    \n",
    "    return data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-d77d7326fbf4aa85",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>fpq_Good</th>\n",
       "      <th>fpq_Average</th>\n",
       "      <th>fpq_Fair</th>\n",
       "      <th>fpq_Poor</th>\n",
       "      <th>fpq_No Fireplace</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   fpq_Good  fpq_Average  fpq_Fair  fpq_Poor  fpq_No Fireplace\n",
       "0         1            0         0         0                 0\n",
       "1         0            0         0         0                 1\n",
       "2         0            0         0         0                 1\n",
       "3         0            1         0         0                 0\n",
       "4         0            1         0         0                 0\n",
       "5         1            0         0         0                 0\n",
       "6         0            0         0         0                 1\n",
       "7         0            1         0         0                 0\n",
       "8         0            0         0         0                 1\n",
       "9         1            0         0         0                 0"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "training_data = ohe_fireplace_qu(training_data)\n",
    "training_data.filter(regex='fpq').head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-63f48a31637fa052",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "## Feature Engineering\n",
    "\n",
    "In this section we will create a new feature out of existing ones through a simple data transformation.  When you move on to create your own model, you may want to try out more complex transformations.\n",
    "\n",
    "### Bathrooms\n",
    "\n",
    "We will create a groundbreaking new feature.  Due to recent advances in Universal WC Enumeration Theory, we now know that Total Bathrooms can be calculated as:\n",
    "\n",
    "$$ TotalBathrooms=(BsmtFullBath + FullBath) + \\dfrac{1}{2}(BsmtHalfBath + HalfBath)$$\n",
    "\n",
    "The actual proof is beyond the scope of this class, but we will use the result in our model.\n",
    "#### Question 7 <a name=\"q8\"></a>\n",
    "\n",
    "Write a function `add_total_bathrooms(data)` that returns the input data frame with a new column called `total_bathrooms` as calculated above.  **Treat missing values as 0s**.  Remember that you can make use of vectorized code here."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-116902ce0d5e9f37",
     "locked": false,
     "schema_version": 1,
     "solution": true
    }
   },
   "outputs": [],
   "source": [
    "def add_total_bathrooms(data):\n",
    "    \"\"\"\n",
    "    Input:\n",
    "      data (data frame): a data frame containing at least 4 numeric columns \n",
    "            Bsmt_Full_Bath, Full_Bath, Bsmt_Half_Bath, and Half_Bath\n",
    "    Output:\n",
    "      data frame identical to the input with the addition of a total_bathrooms column\n",
    "    \"\"\"\n",
    "    \n",
    "   # YOUR CODE HERE\n",
    "    #raise NotImplementedError()\n",
    "    a = data.loc[:,\"Bsmt_Full_Bath\"].fillna(value=0)\n",
    "    b = data.loc[:,\"Full_Bath\"].fillna(value=0)\n",
    "    c = 0.5*data.loc[:,\"Bsmt_Half_Bath\"].fillna(value=0)\n",
    "    d = 0.5*data.loc[:,\"Half_Bath\"].fillna(value=0)\n",
    "    data[\"total_bathrooms\"] = a+b+c+d\n",
    "    return data\n",
    "\n",
    "training_data = add_total_bathrooms(training_data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "nbgrader": {
     "grade": true,
     "grade_id": "cell-77e71ef4c3305b6c",
     "locked": true,
     "points": 2,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "# Check that missing values are dealt with\n",
    "assert ~training_data['total_bathrooms'].isnull().any()\n",
    "# Check that the values are as expected\n",
    "assert training_data['total_bathrooms'].sum() == 4421.5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "nbgrader": {
     "grade": true,
     "grade_id": "cell-f3794a6cf09068dc",
     "locked": true,
     "points": 4,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "# **leers**\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-7f7419ac0a98e696",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "### Rich Neighborhoods\n",
    "\n",
    "#### Question 8 <a name=\"q9\"></a>\n",
    "From before, we have a list of neighborhoods we've deemed as richer than others.  Let's use that information to make a new variable `in_rich_neighborhood`.  Write a function `add_rich_neighborhood` that adds an indicator variable which takes on the value 1 if the house is part of `rich_neighborhoods` (question 4) and the value 0 otherwise."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-c48789abe9e04e3a",
     "locked": false,
     "schema_version": 1,
     "solution": true
    }
   },
   "outputs": [],
   "source": [
    "def add_in_rich_neighborhood(data, neighborhoods):\n",
    "    \"\"\"\n",
    "    Input:\n",
    "      data (data frame): a data frame containing a 'Neighborhood' column with values\n",
    "        found in the codebook\n",
    "      neighborhoods (list of strings): strings should be the names of neighborhoods\n",
    "        pre-identified as rich\n",
    "    Output:\n",
    "      data frame identical to the input with the addition of a binary\n",
    "      in_rich_neighborhood column\n",
    "    \"\"\"\n",
    "\n",
    "    # YOUR CODE HERE\n",
    "    #raise NotImplementedError()\n",
    "    data['in_rich_neighborhood'] = data['Neighborhood'].isin(set(neighborhoods))\n",
    "    return data\n",
    "\n",
    "rich_neighborhoods = find_rich_neighborhoods(training_data, 3, np.median)\n",
    "training_data = add_in_rich_neighborhood(training_data, rich_neighborhoods)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "nbgrader": {
     "grade": true,
     "grade_id": "cell-5ebcf403adaba07b",
     "locked": true,
     "points": 2,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "# Check to see if you have identified the correct number of rich neighborhoods\n",
    "assert sum(training_data['in_rich_neighborhood']) == 191\n",
    "# Check to see if you've introduced any missing values\n",
    "assert sum(training_data['in_rich_neighborhood'].isnull()) == 0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "nbgrader": {
     "grade": true,
     "grade_id": "cell-aea0ebbf97f0c3be",
     "locked": true,
     "points": 4,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "# Repello Muggletum\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-5ffdfab3f8801658",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "## Modeling\n",
    "\n",
    "We've finally gotten to a point where we can specify a simple model.  But first, we need to create perform a test-train split of our data.  We begin by loading a fresh copy of the data in at this point just in case our code above produced any undesired side-effects.  At this point, we will begin to treat `ames_train.csv` as our complete data set.  We will use `train_test_split` from `sklearn` to split the data into `test` and `train` sets.\n",
    "\n",
    "Remember: The reason we have to do a train-test split on `ames_train.csv` here is because we want to evaluate how well our model might perform on future data (`ames_test.csv`), but that data set does not have `SalePrice` in it!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-700027ec3c0adc57",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "# Load a fresh copy of the data\n",
    "full_data = pd.read_csv(\"ames_train.csv\")\n",
    "\n",
    "# This makes the train-test split in this section reproducible across different runs \n",
    "# of the notebook.  You do not need this line to run train_test_split in general\n",
    "np.random.seed(1337) \n",
    "\n",
    "# Split the data \n",
    "train, test = train_test_split(full_data, test_size=0.2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-8006b45a1ad7725b",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "# Something has gone awry in the cell above if these do not pass\n",
    "assert test.shape == (400, 82)\n",
    "assert train.shape == (1600, 82)\n",
    "assert train.loc[887, 'PID'] == 902402260"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-acdc861fd11912e9",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "### Reusable Pipeline\n",
    "\n",
    "Throughout this assignment, you should notice that your data flows through a single processing pipeline several times.  From a software engineering perspective, this should be sufficient motivation to abstract parts of our code into reusable functions/methods.  We will now encapsulate our entire pipeline into a single function `process_data_gm`.  gm is shorthand for \"guided model\"."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-2fe1d82b2c19d1fa",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "def select_columns(data, *columns):\n",
    "    return data.loc[:, columns]\n",
    "\n",
    "def process_data_gm1(data):\n",
    "    # Clean Data\n",
    "    data = remove_outliers(data, 'Gr_Liv_Area', upper=5000)\n",
    "    data = fix_fireplace_qu(data)\n",
    "    \n",
    "    # Transform Data\n",
    "    data = add_total_bathrooms(data)\n",
    "    data = add_in_rich_neighborhood(data, rich_neighborhoods)\n",
    "    data = select_columns(data, \n",
    "                          'SalePrice', \n",
    "                          'Gr_Liv_Area', \n",
    "                          'total_bathrooms',\n",
    "                          'Fireplace_Qu',\n",
    "                          'in_rich_neighborhood'\n",
    "                         )\n",
    "    data = ohe_fireplace_qu(data)\n",
    "    \n",
    "    # Return predictors and response variables separately\n",
    "    X = data.drop(['SalePrice'], axis = 1)\n",
    "    y = data['SalePrice']\n",
    "    \n",
    "    return X, y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-dcea887f3db333c0",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "An alternative way of writing the same code above explicitly allows us to think about our data flowing through a [pipeline](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.pipe.html) where the output of one function is the input of the next.  Carefully thought out function names make the code self-documenting: you can just read off the intended high-level processing steps from top to bottom.\n",
    "\n",
    "You are not required to use this style of coding.  We just wanted to point out that it exists."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-d7de4796ff0ab97d",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "def process_data_gm1(data):\n",
    "\n",
    "    data = (\n",
    "        data\n",
    "        # Clean Data\n",
    "        .pipe(remove_outliers, 'Gr_Liv_Area', upper=5000)\n",
    "        .pipe(fix_fireplace_qu)\n",
    "        \n",
    "        # Transform data\n",
    "        .pipe(add_total_bathrooms)\n",
    "        .pipe(add_in_rich_neighborhood, rich_neighborhoods)\n",
    "        .pipe(select_columns, \n",
    "              'SalePrice',           \n",
    "              'Gr_Liv_Area',            \n",
    "              'total_bathrooms',             \n",
    "              'Fireplace_Qu',            \n",
    "              'in_rich_neighborhood'\n",
    "             )\n",
    "        .pipe(ohe_fireplace_qu)\n",
    "    ) \n",
    "    \n",
    "    # Return predictors and response variables separately\n",
    "    X = data.drop(['SalePrice'], axis = 1)\n",
    "    y = data['SalePrice']\n",
    "    \n",
    "    return X, y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-41994ca25b31660e",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "### Fitting our first model\n",
    "\n",
    "We are finally going to fit a model!  This part is slightly unceremonious since we did much of the heavy lifting in the previous sections.  The model we will fit can be written as follows (with the caveat that one of the fireplace qualities is actually removed to avoid collinearity):\n",
    "\n",
    "$$\\begin{align} SalePrice = &\\theta_0 + \\theta_1 \\times GrLivArea + \\theta_2 \\times TotalBathrooms \\\\\n",
    "&+ \\theta_3 \\times InRichNeighborhood + \\sum_{quality \\in FireplaceQuality} \\theta_{quality} \\times Quality\n",
    "\\end{align}$$\n",
    "\n",
    "#### Question 9a <a name=\"q10a\"></a>\n",
    "Remove the commenting and fill in the ellipses `...` below with `X_train`, `y_train`, `X_test`, or `y_test`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-1be99eea86f6cf57",
     "locked": false,
     "schema_version": 1,
     "solution": true
    }
   },
   "outputs": [],
   "source": [
    "# Pre-process our training and test data in exactly the same way\n",
    "# Our functions make this very easy!\n",
    "X_train, y_train = process_data_gm1(train)\n",
    "X_test, y_test = process_data_gm1(test)\n",
    "guidedmodel1 = lm.LinearRegression(fit_intercept=True)\n",
    "\n",
    "# Fill in the ... below with X_train, y_train, X_train, or X_test.\n",
    "# Remember to uncomment\n",
    "guidedmodel1.fit(X_train, y_train)\n",
    "y_fitted = guidedmodel1.predict(X_train)\n",
    "y_predicted = guidedmodel1.predict(X_test)\n",
    "\n",
    "# YOUR CODE HERE\n",
    "#raise NotImplementedError()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "nbgrader": {
     "grade": true,
     "grade_id": "cell-374e010658179ed3",
     "locked": true,
     "points": 3,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "assert 181100 <= y_fitted.mean() <= 181400\n",
    "assert 177700 <= y_predicted.mean() <= 177800"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-e963e93c7b4f9bad",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "In this assignment, we will use Root-Mean-Square Error (RMSE) to measure the quality of our models.  As a reminder, this quantity is defined as:\n",
    "\n",
    "$$RMSE = \\sqrt{\\dfrac{\\sum_{\\text{houses in test set}}(\\text{actual price of house} - \\text{predicted price of house})^2}{\\text{# of houses in test set}}}$$\n",
    "\n",
    "#### Question 9b <a name=\"q10b\"></a>\n",
    "\n",
    "Write a function `rmse` that calculates the RMSE of a model.  Again, make sure you are taking advantage of vectorized code.  This can be solved without any iteration."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-96600fa98a6c2e97",
     "locked": false,
     "schema_version": 1,
     "solution": true
    }
   },
   "outputs": [],
   "source": [
    "def rmse(actual, predicted):\n",
    "    \"\"\"\n",
    "    Calculates RMSE from actual and predicted values\n",
    "    Input:\n",
    "      actual (1D array-like): vector of actual values\n",
    "      predicted (1D array-like): vector of predicted/fitted values\n",
    "    Output:\n",
    "      a float, the root-mean square error\n",
    "    \"\"\"\n",
    "    \n",
    "    # YOUR CODE HERE\n",
    "    #raise NotImplementedError()\n",
    "    rmse = np.sqrt(np.sum((actual - predicted)**2)/len(actual))\n",
    "    \n",
    "    return rmse"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "nbgrader": {
     "grade": true,
     "grade_id": "cell-0684404647cc924f",
     "locked": true,
     "points": 3,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "assert 43000 <= rmse(y_test, y_predicted) <= 44000"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-a359da2dda38fcdd",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "### Residual Plots\n",
    "\n",
    "One way of diagnosing a model is through a residual plot.  Here we plot the actual sale prices against the residuals of the model.  Ideally, we would see a horizontal line of points at 0 (perfect prediction!).  The next best thing would be a homogenous set of points centered at 0.  But alas, our simple model is probably too simple.  We notice that we are really underfitting the more expensive homes in the test set.  In fact, it looks like our model tends to underprice the more expensive homes!  You will probably want to address this in your own work."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-4d79f42d60b94fca",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0, 0.5, 'Residuals (Actual Price - Predicted Price)')"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "[encoded data, 31284 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "gm1_residuals = y_test - y_predicted\n",
    "ax = sns.regplot(y_test, gm1_residuals)\n",
    "ax.set_xlabel('Sale Price (Test Data)')\n",
    "ax.set_ylabel('Residuals (Actual Price - Predicted Price)')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-4638dca2c51b40f0",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "### Regularizing our model\n",
    "\n",
    "Ok, so let's make our modeling a little more fancy by regularizing the coefficients.  This second model will use the Lasso, but you are free to use Ridge or ElasticNet in your work.  First, we need to add a step to our pre-processing.  In order for regularization to be fair to all the variables in our model, we need to standardize our predictor columns (otherwise it would unfairly penalize variables with inherently small values).\n",
    "\n",
    "#### Question 10 <a name=\"q11\"></a>\n",
    "\n",
    "Write a function that standardizes the columns of a data frame containing only numeric columns.  Be sure to make use of vectorization."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-7331e38e38968cb9",
     "locked": false,
     "schema_version": 1,
     "solution": true
    }
   },
   "outputs": [],
   "source": [
    "def standardize_columns(data):\n",
    "    '''\n",
    "    Input:\n",
    "      data (data frame): contains only numeric columns\n",
    "    Output:\n",
    "      data frame, the same data, except each column is standardized \n",
    "      to have 0-mean and unit variance\n",
    "    '''\n",
    "    standardized_data = (data - data.mean()) / data.std()\n",
    "    \n",
    "    # YOUR CODE HERE\n",
    "    #raise NotImplementedError()\n",
    "    \n",
    "    return standardized_data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "nbgrader": {
     "grade": true,
     "grade_id": "cell-af48dec574258015",
     "locked": true,
     "points": 6,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "test_standardize_df = standardize_columns(training_data[['SalePrice', 'Gr_Liv_Area']])\n",
    "# Make sure the mean is correct\n",
    "assert -0.001 < test_standardize_df.mean().sum() < 0.001\n",
    "# Make sure the standard deviation is correct\n",
    "assert 1.9 < test_standardize_df.std().sum() < 2.1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-2b0b86e3a1743a83",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "We'll now use this function to describe our new processing for the regularized model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-50de09444fbd962d",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "def process_data_gm2(data):\n",
    "\n",
    "    data = (\n",
    "        data\n",
    "        # Clean Data\n",
    "        .pipe(remove_outliers, 'Gr_Liv_Area', upper=5000)\n",
    "        .pipe(fix_fireplace_qu)\n",
    "        \n",
    "        # Transform data\n",
    "        .pipe(add_total_bathrooms)\n",
    "        .pipe(add_in_rich_neighborhood, rich_neighborhoods)\n",
    "        .pipe(select_columns, \n",
    "              'SalePrice',           \n",
    "              'Gr_Liv_Area',            \n",
    "              'total_bathrooms',             \n",
    "              'Fireplace_Qu',            \n",
    "              'in_rich_neighborhood'\n",
    "             )\n",
    "        .pipe(ohe_fireplace_qu)\n",
    "    ) \n",
    "    \n",
    "    # Return predictor and response variables separately\n",
    "    X = standardize_columns(data).drop(['SalePrice'], axis = 1)\n",
    "    y = data['SalePrice']\n",
    "    \n",
    "    return X, y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-d15ce882bb659e94",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "It may be instructive to see the cross-validation procedure explicitly once.  You should be able to understand what each part of the code is doing below, but we do not expect you to use this code for your own model (use [LassoCV](http://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LassoCV.html) instead)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-093735333c641eab",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The validation RMSE for this model with alpha=56.2 is 43520.03610313791.\n"
     ]
    }
   ],
   "source": [
    "# Process our data\n",
    "X_train, y_train = process_data_gm2(train)\n",
    "X_test, y_test = process_data_gm2(test)\n",
    "\n",
    "# Specify our model\n",
    "guidedmodel2 = lm.Lasso(copy_X=True)\n",
    "\n",
    "# Specify CV method and alpha grid\n",
    "five_fold_cv = KFold(n_splits = 5)  #将样本划分成5份，每次取其中一份做验证集，其余4份做训练集\n",
    "alphas = np.arange(0.1, 200.1, .1)\n",
    "rmses = np.zeros(len(alphas))\n",
    "\n",
    "# Grid search over alphas\n",
    "for i, alpha in enumerate(alphas):\n",
    "    guidedmodel2.set_params(alpha=alpha)  #网格调参，寻找合适的alpha\n",
    "    model_rmse = 0\n",
    "    \n",
    "    # Fit each fold using the other four as training data\n",
    "    for train_index, test_index in five_fold_cv.split(X_train):\n",
    "        X_fold_train = X_train.iloc[train_index]\n",
    "        y_fold_train = y_train.iloc[train_index]\n",
    "        X_fold_test = X_train.iloc[test_index]\n",
    "        y_fold_test = y_train.iloc[test_index]\n",
    "        \n",
    "        guidedmodel2.fit(X_fold_train, y_fold_train)\n",
    "        y_fold_predicted = guidedmodel2.predict(X_fold_test)\n",
    "        model_rmse += rmse(y_fold_test, y_fold_predicted)\n",
    "    \n",
    "    # Average RMSE over the five folds for alpha_i\n",
    "    rmses[i] = model_rmse / 5\n",
    "\n",
    "optimal_alpha = alphas[rmses == np.min(rmses)]\n",
    "guidedmodel2.set_params(alpha=optimal_alpha)\n",
    "guidedmodel2.fit(X_train, y_train)\n",
    "y_predicted = guidedmodel2.predict(X_test)\n",
    "\n",
    "print(f'The validation RMSE for this model with '\n",
    "      f'alpha={float(optimal_alpha)} is {rmse(y_test, y_predicted)}.')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-7e699b2561e846e5",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "### Lasso Path\n",
    "\n",
    "Let's take a look at how RMSE varied across different choices of the regularization hyperparameter ($\\lambda$ in lecture, `alpha` in `sklearn`).  This is often called the Lasso or Regularization Path.  The dashed red line marks the alpha that minimizes RMSE."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-031b47ffdfd586a3",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0, 0.5, 'RMSE')"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "[encoded data, 19384 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(alphas, rmses)\n",
    "plt.axvline(x=optimal_alpha, color='red', linestyle='dashed')\n",
    "ax = plt.gca() #Get Current Axes\n",
    "ax.set_title('Average LASSO RMSE Path')\n",
    "ax.set_xlabel('Regularization Level')\n",
    "ax.set_ylabel('RMSE')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-1a2d8b21df5a43ff",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "### A simplified approach to Lasso and CV"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-29756b33edf57bc8",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "Here we perform another L1-regularized regression but using `LassoCV`.  This is more in line with what we expect from you code-wise.  That being said, you should still understand the concepts from the previous section."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-e7875ce8a7c2435a",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The validation RMSE for this model with alpha=55.8 is 43520.1.\n"
     ]
    }
   ],
   "source": [
    "# Process the data\n",
    "X_train, y_train = process_data_gm2(train)\n",
    "X_test, y_test = process_data_gm2(test)\n",
    "\n",
    "# Specify the model, alphas, and number of folds for CV\n",
    "alphas = np.arange(0.1, 200.1, .1)\n",
    "guidedmodel2 = lm.LassoCV(alphas=alphas, cv=5)  #Lasso回归，要求系数不要太大\n",
    "\n",
    "# Fit and predict\n",
    "guidedmodel2.fit(X_train, y_train)\n",
    "y_predicted = guidedmodel2.predict(X_test)\n",
    "\n",
    "print(f'The validation RMSE for this model with '\n",
    "      f'alpha={round(float(guidedmodel2.alpha_), 2)} is '\n",
    "      f'{round(rmse(y_test, y_predicted), 2)}.')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-d2b63d56e68f43c1",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "Again, we can map out the path that the Lasso algorithm took."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-9a316c221f111bc4",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0, 0.5, 'RMSE')"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "[encoded data, 18224 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(guidedmodel2.alphas_, np.sqrt(np.apply_along_axis(np.mean, 1, guidedmodel2.mse_path_)))\n",
    "plt.axvline(x=optimal_alpha, color='red', linestyle='dashed')\n",
    "ax = plt.gca()\n",
    "ax.set_title('Average LASSO RMSE Path')\n",
    "ax.set_xlabel('alpha')\n",
    "ax.set_ylabel('RMSE')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-a56a300a011993b3",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "### Lasso Residual Plot\n",
    "\n",
    "Looking at the residual plot for our L1 regularized linear model, it's clear the regularization did not solve the problems we saw in the simple model.  It seems you have your work cut out for you!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-aaa612b22458ba79",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0, 0.5, 'Residuals (Actual Price - Predicted Price)')"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "[encoded data, 32376 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "gm2_residuals = y_test - y_predicted\n",
    "ax = sns.regplot(y_test, gm2_residuals)\n",
    "ax.set_xlabel('Sale Price (Test Data)')\n",
    "ax.set_ylabel('Residuals (Actual Price - Predicted Price)')"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Create Assignment",
  "kernelspec": {
   "display_name": "Python [conda env:datasci]",
   "language": "python",
   "name": "conda-env-datasci-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
