{ "cells": [ { "cell_type": "code", "execution_count": 1, "metadata": { "hide_input": true, "internals": {}, "slideshow": { "slide_type": "skip" }, "tags": [ "hide-cell" ] }, "outputs": [], "source": [ "import numpy as np\n", "import scipy as sp\n", "import matplotlib.pyplot as plt\n", "import pandas as pd\n", "import seaborn as sns\n", "import matplotlib as mp\n", "import sklearn\n", "from IPython.display import Image, HTML\n", "\n", "import laUtilities as ut\n", "\n", "%matplotlib inline\n", "\n", "import statsmodels.api as sm\n", "\n", "def centerAxes(ax):\n", " ax.spines['left'].set_position('zero')\n", " ax.spines['right'].set_color('none')\n", " ax.spines['bottom'].set_position('zero')\n", " ax.spines['top'].set_color('none')\n", " ax.xaxis.set_ticks_position('bottom')\n", " ax.yaxis.set_ticks_position('left')\n", " bounds = np.array([ax.axes.get_xlim(), ax.axes.get_ylim()])\n", " ax.plot(bounds[0][0],bounds[1][0],'')\n", " ax.plot(bounds[0][1],bounds[1][1],'')" ] }, { "cell_type": "markdown", "metadata": { "hide_input": true, "internals": { "slide_type": "subslide" }, "slideshow": { "slide_type": "slide" } }, "source": [ "# Linear Regression" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" }, "tags": [ "hide-input" ] }, "source": [ "\"Figure\"" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "hide_input": true, "tags": [ "hide-input" ] }, "outputs": [ { "data": { "text/html": [ "Sir Francis Galton by Charles Wellington Furse by Charles Wellington Furse (died 1904) - National Portrait Gallery: NPG 3916" ], "text/plain": [ "" ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "HTML(u'Sir Francis Galton by Charles Wellington Furse by Charles Wellington Furse (died 1904) - National Portrait Gallery: NPG 3916')" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "\"Figure\"" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In 1886 Francis Galton published his observations about how random factors affect outliers.\n", "\n", "This notion has come to be called \"regression to the mean\" because unusually large or small phenomena, after the influence of random events, become closer to their mean values (less extreme)." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "\"Figure\"" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Galton fit a straight line to this effect, and the fitting of lines or curves to data has come to be called regression as well." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "The most common form of machine learning is __regression__, which means constructing an equation that describes the relationships among variables.\n", "\n", "It is a form of supervised learning: whereas __classification__ deals with predicting categorical features (labels or classes), __regression__ deals with predicting continuous features (real values)." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "For example, we may look at these points and decide to model them using a line." ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "hide_input": false, "tags": [ "hide-input" ] }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZoAAAGKCAYAAAA10If4AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAhA0lEQVR4nO3dfYxU9b3H8c/srnCvrWDURZ5EHRb2CZZ1mQp/tBoNILTiAxqrsakp2G3UpmltNe0fpg+RotakWjWYTa8Nyq3EJlZYQWpBbYzV4ICJFa4rsNDwWBYLQaGwDPu7fwyz7uzOzs7Mnt+c3znn/UrIsjNnmN8eds9nf0/fEzPGCAAAWyr8bgAAINwIGgCAVQQNAMAqggYAYBVBAwCwqqrE17FUDZE0f/58rV+/3u9mAK6K5XqQHg1QhMOHD/vdBCBwCBoAgFUEDQDAKoIGAGAVQQMAsIqgAQBYRdAAAKwiaAAAVhE0AACrCBoAgFUEDQDAKoIGAGAVQYPI++1vf6vGxkZNmzZNd9xxh06ePOl3k4BQIWgQafv27dPvfvc7JZNJffTRRzpz5oxWrVrld7OAUCFoEHmpVEr/+c9/lEqldOLECY0fP97vJgGhQtAg0iZMmKCf/OQnmjRpksaNG6fRo0dr3rx5Wce0tbUpkUgokUioq6vLp5YCFnR2So2NUlVV+mNnp5W3IWgQaUeOHNHq1au1a9cu7d+/X8ePH9fKlSuzjmltbVUymVQymVR1dbVPLQUsWLhQ+vhj6cyZ9MeFC628DUGDSNuwYYMuv/xyVVdX65xzztGiRYv097//3e9mAeXR0SH19KT/3tOT/twCggaRNmnSJL333ns6ceKEjDHauHGj6uvr/W4WUB61tVLF2RioqEh/bgFBg0ibNWuWbr31VrW0tGj69Onq6elRa2ur380CyqO9Xaqrkyor0x/b2628TcwYU8rrSnoREHSJRELJZNLvZgCuiuV6kB4NAMAqggYAYBVBAwBhUqa9McUgaAAgTMq0N6YYBA0AhEmZ9sYUg6ABgDAp096YYhA0ABAmZdobU4wqvxsAAPBQPC5t3ep3K7LQowEAWEXQAACsImgAIIwc2k9D0ABAGDm0n4agAYAwcmg/DUEDAGHk0H4aggYAwsih/TTsowGAMHJoPw09GgCAVQQNAMAqggYAYBVBAwCwiqABAFhF0AAArCJoAABWETQAAKsIGgCAVQQNAMAqggYAYBVBAwCwiqABAFhF0AAArCJoAABWETQAAKsIGgCAVQQNAMAqggYAYBVBAwCwiqABAFhF0ABAUHV2So2NUlVV+mNnp98tyomgAYCgWrhQ+vhj6cyZ9MeFC/1uUU4EDQAEVUeH1NOT/ntPT/pzBxE0ABBUtbVSxdnLeEVF+nMHETSIvKNHj+rWW29VXV2d6uvr9e677/rdJKAw7e1SXZ1UWZn+2N4+9Gt8mNeJGWNKeV1JLwJcdNddd+lrX/ua7r77bnV3d+vEiRM6//zzcx6bSCSUTCbL20DAS42N6fmcnp50L6iuTtq61at/PZbzQYIGUXbs2DHNmDFDnZ2disVy/oxkIWgQeFVV6cUDGZWVUirl1b+e84eIoTNEWmdnp6qrq/Wd73xHV1xxhe6++24dP34865i2tjYlEgklEgl1dXX51FLAIz7M6xA0iLRUKqUtW7bonnvu0QcffKAvfelLeuSRR7KOaW1tVTKZVDKZVHV1tU8tBTxSyrzOMBE0iLSJEydq4sSJmjVrliTp1ltv1ZYtW3xuFWBRPJ6ek0ml0h/jcetvSdAg0saOHatLLrlEHWf3H2zcuFENDQ0+twoIlyq/GwD47amnntKdd96p7u5uxeNx/eEPf/C7SUCoEDSIvObmZlaSoTSdnemyLx0d6Un19vayDEUFDUNnAFCqctYaC0gBzVwIGgAoVTlrjQWkgGYuBA0AlKrQPSle9EYCUkAzF4IGAEpV6J4UL3ojASmgmQuLAQCgVJk9KUPxojfS3j5w4UFAEDQAYFttbXYhy1J6I4WGmoMYOgMA23wo++ISggYAbMv0Rj75ROruliZPlmIxacqU4hYGBHSJM0EDAOWycKG0Y8cXn+/YUdzCgIAucSZoAKBcci0CKGZhQECXOBM0AFAuuRYBFLMwIKBLnAkaACiX9nappuaLz2tqilsYENBFBSxvBhAerhe5jMel7duH9/oALnGmRwMgPAI6WR52BA2A8AjoZHnYETQAwiOgk+VhR9AACI+ATpaHHYsBAIRHQCfLw44eDQDAKoIGAGAVQQMAfgpoocxiEDQA4KcI7P0haACERxB7B17t/XH4aydoAIRHEHsHXu39cfhrJ2gAhEcQKwN4tffH4a+dfTQAwqO2Nv3bfE9PcCoDeLX3x+GvnR4NgPCIcmUAh792ejQAwiPKlQEc/trp0QAArCJoAABWETQA4BWH97L4iaABAK84vJfFTwQNAHil/16WbdukWEyaMiXSvRuCBgC80neXf187dkS6d0PQAIBXMntZcnFop365ETQA4JXMXpaGhoHPObRTv9wIGgDwWnu7VFPzxec1NcXv1A/RCraYMaaU15X0IiDoEomEksmk381AFDQ2Ztcuq6tzdud/H7FcD9KjAQAXOVyNuVgEDQC4yKv71DiAoAEQLUGZ+3C4GnOxCBpE3pkzZ3TFFVfo+uuv97spKEWxwTGc3fvlDKnMCrZUKv0xHg9OSPZD0CDynnzySdXX1/vdjOBx5aJXbHAMZ+7D7xIzfr9/iQgaRNrevXu1du1a3X333X43xV+lhIYrF71ig2M4cx9+T9D7/f4lImgQaT/84Q/12GOPqSJX2ZCz2tralEgklEgk1NXVVcbWlVEpoeHKRa+Q4OgbpN3d6WGoUuY+/J6g9/v9S0TQILJeffVVjRkzRjNnzsx7XGtrq5LJpJLJpKqrq8vUujIrJTRcuegVMmneN0g7O6URI7LnPrx8L5v8fv8SsWETkfWzn/1ML7zwgqqqqnTy5EkdO3ZMixYt0sqVKwd9TWg3bJayObCzM30B7+hIh0x7e3EX7XKqqkqHTEZlZTpo4LWcGzYJGkDSW2+9pccff1yvvvpq3uNCGzRBCo1SBHOXfRDlDJqqcrcCgIMyS2nDqr19YJCibOjRAEUIbY8G8Aa1zgAA5UfQAACsImgAAFYRNAAAqwgaAIBVBA2iyZWCkEAEEDSIJlcKQgIRQNAgmlwpCInB0esMDYIG0eRKQchCRfGi27fXuW2bNHlydL72kCFoEE1Bq4IbxaG+vr3OjKh87SFD0CCact0m12X5hvqK6e0EqWfUt9eZwTBnIBE0QBDkG+orprcTpJ5RptfZX2Wl2wGJAQgaIAjyDfUVs7AhSIsgMr3OnTvTNyrLOH3a7YDEANwmAAiCfGX8a2uz77WSb2FDMce6Ih7PvmmZMW4HJAagRwMEUd+5lu7u9MW4kIUNQVsEkRG0VYLIQo8GCKLMXEtPTzp06uoKuzVxUG9wxo3LAo2gAYIoSHMtXghqQEISQ2dAMDGUhAAhaID+grDXJKhzLcgWhO81D8SMMaW8rqQXAYHQ2Ji9MquurnfYJpFIKJlM+txAhEae77WAiuV6kB4N0F/U5j/gn4h8rxE0QH/Mf6BcIvK9RtAA/TH/Aak88ycR+V5jjgYoAnM0ERK++ZNyYI4GAAoWkfmTciBoAGSLyJLbIUVk/qQcCBoA2YJ0KwGbIjJ/Ug6UoAGQjSGjNMreeIYeDYBsDBnBYwQNgGwMGcFjDJ0ByMaQETxGjwYAYBVBAwCwiqABkB/7ajBMBA2A/NhXg2EiaADkx74aDBNBAyA/9tVgmAgaIGjKPWdSyL4a5nGQB7cJAIrgxG0CXCxf72Kb4AduEwCEgotzJi62Cc4gaICgcXHOxMU2wRkEDSJtz549uuaaa1RfX6/GxkY9+eST/jSk0DmOzk6pu/uL3kM87kYtMuqjIQ/maBBpBw4c0IEDB9TS0qLPPvtMM2fO1CuvvKKGhoacx1uboyl0joO5ELiNORqgv3HjxqmlpUWSdN5556m+vl779u0rf0MKneNgLgQBRNAAZ+3evVsffPCBZs2aVf43L3SOg7kQBBBBA0j6/PPPdcstt+iJJ57QqFGjsp5ra2tTIpFQIpFQV1eXnQYUOsfBXAgCiDkaRN7p06d1/fXX67rrrtP999+f91gn9tEA7mKOBujPGKMlS5aovr5+yJABUBqCBpH2zjvv6IUXXtAbb7yh5uZmNTc3a926dX43CwgVbuWMSPvqV7+qEoeP8+vsTJfT7+hIT9i3t6f3vAARRI8GsIF7uAC9CBqUV1Sq/LLfBehF0KC8wvSbfr7QDOt+l6j8ogBPETQorzD9pp8vNF3Z7+J1MITpFwWUDUGD8grTb/r5QjMeT9cg++ST9OdTp/rTA/A6GML0iwLKhqBBebnym74XCglNv3sAXgdDmH5RQNkQNCivzG/6qVT6Y5CX/BYSmn73ALwOhjD9ooCyYR8NUKpMaOZTW5td1r/cPYD29oH7eYajkK8Z6IegAWzy+kJfLIIBDiBoAJu40APM0QAA7CJoAABWETQAAKsIGgCAVQQNUIhMKZfNm6nxBRSJoIEbXC/WmNnhL3m7w9/1rxvwQKzEmz5ZuFMUIq2xMXtjY12dW8uCq6qkM2eUkJSU0jvjU6nh/7uuf91AcWK5HqRHAzfYLNVSSK9hqGNs1fjyu0RNf/SwYAFBAzfYLNZYSGHLoY7J1PiSvK3x5VqRSr+LgCKUGDqDGzo7B5Zq8arg5tlhr165hr0KOUZSIpFQMpn0pl2S3a+7FAWeB2AQOYfOKEEDN9gs1VJIYUu/il+6VqLG7yKgCCWGzhB+hZS2p/x9GucBFjB0BhTB86EzIFxYdQYAKD+CBgBgFUEDALCKoEFwsbkQCASCBsHF5kIgEAgaBFcx5Vvo/QC+IWgQXMWUb6H3A/iGoEFwFbO50LXilUCEUIIGwVVM+RZKqwC+oUeDaKC0CuAbejSIBteKVwIRQo8GAGAVQQOUA8urEWEEDYIpaBdullcjwggaBFPQLtwsr0aEETQIpqBduIvZXAqEDEGDYArahZvl1YgwljcjmNrb08NlHR3pkHH9ws3yakQYPZowC9qEeTEyF+5UKv0xHve7RQAGQdCEWdAmzH2yfv161dbWqqamRo888ojfzQFCh6AJs6BNmA+3B1bC68+cOaP77rtPr732mrZt26YXX3xR27ZtK/ELAJALQRNmQZswH24PrITXb9q0STU1NYrH4xoxYoRuv/12rV69usQvAEAuMWNM0S+aP3++OXz48JDHdXV1qbq6upR2+SZUbT51StqxQzp5Uvqv/5JqaqSRI8vfwEEMaPfmzQMPmjmz8H+whNcfOXJEx44d06WXXipJ+vTTT3X8+HFNmjQpq52Z7/dTp06pubm58DY5IlTf144LYru9avPmzZv/YoyZP+AJY0wpfwoyc+bMQg91Bm0ug507jWloMCnJmIaG9OfGpP9eUWGMlP7Y0FDcv1vC61966SWzZMmS3s+ff/558/3vf3/Q488999zi2uSIwH2PmGC22ZhgttvDNufMDIbOUH5nh7gqpewhruHuNSnh9RMnTtSePXt6P9+7d6/Gjx9f3PsCyIt9NCi/wRYpDHevSQmv/8pXvqLt27dr165dmjBhglatWqU//vGPpbcBwABWezStra02/3kraHMZOLRIoaqqSk8//bSuu+461dfX67bbblNjY+Ogx1900UVlbJ13Avc9omC2WQpmu223uaTFAJJKehEgKb3suP+u/oBsuEwkEkomk343A3BVLNeDDJ2h/CjHAkQKiwEQTYVs7gxzCR+gjIYVNH/605/U2NioioqKAcMJy5YtU01NjWpra/WXv/wl5+v//e9/a+7cuZoyZYrmzp2rI0eODKc5JfnmN7+p5uZmNTc367LLLht0j8Rll12m6dOnq7m5WYlEoryN7OcXv/iFJkyY0NvudevW5TzOmdIqZy/YZyoqtH3kSC2oq9PNN9+so0eP5jy8pHNdbCgMsblz/fr12l5frzPbtuU8xhijH/zgB6qpqVFTU5O2bNlSWDst2rNnj6655hrV19ersbFRTz755IBj3nrrLY0ePbr3e+dXv/qVDy3NNtT/t4vnuqOjo/ccNjc3a9SoUXriiSeyjnHhXC9evFhjxozRtGnTeh8r9Lrr6fVjsHXPQ/wxxhizbds28/HHH5urr77avP/++70Lqbdu3WqamprMyZMnTWdnp4nH4yaVSg1YcP3AAw+YZcuWGWOMWbZsmXnwwQe9Wstdkvvvv9/88pe/zPncpZdearq6usrcotx+/vOfm9/85jd5j0mlUiYej5udO3eaU6dOmaamJrN169YytbCfPvtbes7ub3nwwQe/+P8+u6/GVFYa09Bgvjp+fPHnutg9NJWV6WMzfyore5/KnLueHMdk9husXbvWzJ8/3/T09Jh3333XXHnllcW114L9+/ebzZs3G2OMOXbsmJkyZcqA//M333zTfOMb3/CjeYMa6mfLxXPdVyqVMhdffLHZvXt31uMunOu//e1vZvPmzaaxsbH3sUKuu8O4fni/j6a+vl61OVYMrV69WrfffrtGjhypyy+/XDU1Ndq0aVPO4+666y5J0l133aVXXnllOM0ZFmOMXnrpJd1xxx2+tcFLTpVW6bOcOXZ2OfPs2bO1d+/e9PP9ehf/09U1rPfoXTKdr5eTZ+Vb5tzF+hzTE4tlHbN69Wp9+9vfViwW0+zZs3X06FEdOHCg+HZ7aNy4cWppaZEknXfeeaqvr9e+fft8bZMXXDzXfW3cuFGTJ0/urS7hkquuukoXXHBB1mOFXHe9vn5YmaPZt2+fLrnkkt7PJ06cmPMb/l//+pfGjRsnKf1DcujQIRvNGVyfC9GJeFwt55+vKVOm5Dw0Fotp3rx5mjlzptra2srbzhyefvppNTU1afHixTm7voX+H5RFjov6c889pwULFqQf6xcS8dOniz/XuYIj3/BYns2dvefu7DE9FRX61/nn5z7mLF/Pbw67d+/WBx98oFmzZg147t1339WMGTO0YMECbXVgUcZQP1uun+tVq1YN+guqa+daKuy66/U5H3LVWSwW2yBpbN/HGhsbtXTpUt144405X2NyLJmOxXKueiuLOXPm6ODBgwMe//uxYxq1b5/U06P//uc/9T9jxgz6b7zzzjsaP368Dh06pLlz56qurk5XXXVV2du8dOlS3XPPPXrooYcUi8X00EMP6cc//rGee+65rOP8+j/I1e6Jp0/rpQkTNGr/fqm2Vs/Mn6+qzk7deeed6QNqa9NB0NMjVVTI1NRoy5YtxZ3rXDdCmzp18OrVeVa+9Z67s8f87wsvaNOmTXqqzxJs177H+/r88891yy236IknntCoUaOynmtpadE///lPffnLX9a6det00003afv27T61NG2ony2Xz3V3d7fWrFmjZcuWDXjOxXNdKK/P+ZBBY4yZk+vhfK8ptKzHxRdfrAMHDmjcuHE6cOCAxuS50A/Hhg0bcj9RVdV7IaowRhfmKRSaaf+YMWN08803a9OmTVaDZtA29/Pd735X119//YDH/SqtMlS7V6xYoZXPPquNGzd+8Y3bLyTOOdtzKOpc5wqOfgFW6MbQQs6dq6VrTp8+rVtuuUV33nmnFi1aNOD5vsHz9a9/Xffee68OHz7s60bUoX62XD3XkvTaa6+ppaVFF1988YDnXDzXUmHXXa/PuZWhsxtuuEGrVq3SqVOntGvXLm3fvl1XXnllzuNWrFghKX0BGqyHZE2f4ZYzUnpMPofjx4/rs88+6/3766+/nrWKo9z6jk//+c9/HtiWzk7NXrJEa19/Xd1Tp6r744+1atUq3XDDDd42pMiVXuvXr9ejjz6qNWvW6Nxzz/3iiT53yzy+aZM+O1tFdtjnusTaaX3L0nR3d+c8dzfccIOef/55GWP03nvvafTo0b3DEX4xxmjJkiWqr6/X/fffn/OYgwcP9v62umnTJvX09OjCCy8sZzOzFPKz5eK5znjxxRcHHTZz7VxnFHLdLeRnoCiDrRIY4o8xxpiXX37ZTJgwwYwYMcKMGTPGzJs3r3fpwcMPP2zi8biZOnWqWbduXe/jS5Ys6V2hdvjwYXPttdeampoac+2115pPP/20kFUN3slUEY7FzKfjxn1RRdgYs2/fPrNgwYKzh+00TU1NpqmpyTQ0NJiHH364vO3s51vf+paZNm2amT59ulm4cKHZv3+/MaZPm/uswEpJpuOcc+y0uciVXpMnTzYTJ040M2bMMDNmzDDf+973sttt3DnXa9euNVOmTDHxeLy3DcuXLzeTJk0yxhjT09Nj7r33XhOPx820adOyVl365e233zaSzPTp03vP8dq1a83y5cvN8uXLjTHGPPXUU6ahocE0NTWZWbNmmXfeecfXNg/2/923zS6ea2OMOX78uLngggvM0aNHex9z7VzffvvtZuzYsaaqqspMmDDB/P73vx/0utv359CY3D8DBciZGZSgCaOqqvQEeEZlpZRKDTxuuKVgCn2fEKEEDZBXzokcKgOEUaFFK4d7R0uHimMCcBdBE0aFzk0MVq7f6/cZCqVegFAjaMKoz+S6tm4dfDhsuD2SQt9nKMPtWWUQWICTCJoo86pHMlzD7VlleBVYADzFbQKizJVy/SXudxnAq8AC4Cl6NPCfVz0rFicATqJHA/951bPKVYYGgO8IGoSHK0OBALIwdAZ7WAUGQAQNbGIVGAARNLCJVWAARNDAJlaBARBBEw6uzoW4siEUgK+o3hwGjY3ZGx7r6lh9ZQnVm4G8qN4cWsyFAHAYQRMGzIUAcBhBEwbMhQBwGJUBwoAd8QAcRo8GAGAVQQMAsIqgAQBYRdAAAKwiaGxwdac+APiAoLFh4ULp//4vXbV42zapvp6wARBZBI0NHR1S39I+3d2UyAcQWQSNDbl25lMWBkBEETTDlWs+pr1dGjHii2MoCwMgwgia4cp1F8l4PD1H09AQzLIwLGYA4CFuEzBcVVXpkMmorJRSKf/a4wVuOzAobhMA5MVtAqwIY+VkbjsAwEMEzXC5VjnZi2GvMIZnDg888IDq6urU1NSkm2++WUePHvW7SUAoETTDlamcnEqlP8bj/rYn15xRsVwLT0vmzp2rjz76SB9++KGmTp2qZcuW+d0kIJSiEzRRmeD2YtjLtfC0ZN68eaqqSt8pY/bs2dq7d6/PLQLCKTpB48Vv+kEQkWEvrz333HNasGBBzufa2tqUSCSUSCTU1dVV5pYBwRedVWdhXB2WS2dnOkQ7OtIh094e2h5JIebMmaODBw8OeHzp0qW68cYbe/+eTCb18ssvKxbLuWimF6vOgLxy/gBF5w6btbXZS3bD+ps+d9vMsmHDhrzPr1ixQq+++qo2btw4ZMgAKI17Q2e25lIiMsGNwq1fv16PPvqo1qxZo3PPPdfv5gCh5d7QGZsFUSY1NTU6deqULrzwQknpBQHPPvts3tcwdAbkFZChsyBvFmR+JFB27NjhdxOASHBv6CyIq6Yyw32TJ6fvPxP2lW0AUAT3giaIcymZpdN9Ba03BgCWuBc0rm8WzLVYoe9wX0ZQemMAYJl7QeO6XBs/+w73ZQSlNwYAlhE0hcr0ZLZtG7hYoe9wX0ODtHOnm70xAPCBe6vOXJVrHiYzPMYmSQAYFD2aQuWah2F4DACGRNAUqv+y64YGhscAoAAETaFKWXYdlVsTAEAe7pWgCRPK6YQOJWiAvHKWoKFHY1OQy+kAgEcIGi8MNkQWxHI6AOAxgsYLg929M4jldADAY+yj8cJgQ2TsrwEAejSeYIgMAAZF0HiBITIAGBRDZ15giAwABkWPBoVh8ymAEhE0KMxgK+sAYAgEDQrD5lMAJSJoChX1oSNW1gEoEUFTqKgPHbGyDkCJ3AwaF3sPUR86yqysS6W4PQKAorgZNC72Hhg6AoCSuBk0LvYennkm3cOS0h+fecbf9gBAQLgZNC72Hu67Lz1sJKU/3nefv+0BgIBwM2hcnHh2sZcFAAHgZgkaF0u61NZm3y3ThV4WAASAmz0aF7nYywKAAHCzR+MiF3tZABAA9GgAAFYRNAAAqwgaAIBVBA0AwCqCBgBgFUEDALCKoAEAWEXQAACsImj6c/FeOAAQYG4HjR8XfRfvhQMAAeZ20Phx0adKMwB4yu2g8eOi7+K9cGDV448/rlgspsOHD/vdFCCU3A4aPy76VGmOlD179uivf/2rJk2a5HdTgNByO2j8uOhnqjSnUumP8bj994RvfvSjH+mxxx5TLBbzuylAaLl9mwBK88OiNWvWaMKECZoxY0be49ra2tTW1iZJ6urqKkfTgFCJGWNKeV1JLwLKbc6cOTp48OCAx5cuXapf//rXev311zV69GhddtllSiaTuuiii/L+e4lEQslk0lZzgaDLOTTgdo8GGKYNGzbkfPwf//iHdu3a1dub2bt3r1paWrRp0yaNHTu2nE0EQo+gQSRNnz5dhw4d6v280B4NgOK5vRgAABB49GgASbt37/a7CUBo0aMBAFjlT9BQuBIAIsOfoKFwJQBEhj9BQ+FKAIgMf4Km2BpmDLUBQGD5EzTF1jBjqA0AAsuf5c3F1jBjqA0AAisYy5u5RwwABFYwgoZ7xABAYAWjMgC3CwCAwApGjwYAEFgEDQDAKoIGAGAVQQMAsIqgAQBYRdAAAKwiaAAAVhE0AACrCBoAgFUEDQDAKoIGAGAVQQMAsIqgAQBYFayg4ZbOABA4wQoabukMAIETrKDhls4AEDjBChpu6QwAgROsoOGWzgAQOMG4lXMGt3QGgMAJVo8GABA4BA0AwCqCBgBgFUEDALCKoAEAWEXQAACsImgAAFYRNAAAqwgaAIBVBA0AwCp/gob7ygBAZPgTNNxXBgAiw5+g4b4yABAZ/gQN95WBI5566inV1taqsbFRDz74oN/NAULJn9sEtLenh8s6OtIhw31l4IM333xTq1ev1ocffqiRI0fq0KFDfjcJCCV/gob7ysABy5cv109/+lONHDlSkjRmzBifWwSEE8ubEVmffPKJ3n77bc2aNUtXX3213n///ZzHtbW1KZFIKJFIqKurq8ytBIIvWHfYBIo0Z84cHTx4cMDjS5cuVSqV0pEjR/Tee+/p/fff12233abOzk7FYrGsY1tbW9Xa2ipJSiQSZWk3ECYEDUJtw4YNgz63fPlyLVq0SLFYTFdeeaUqKip0+PBhVVdXl7GFQPgxdIbIuummm/TGG29ISg+jdXd366KLLvK5VUD40KNBZC1evFiLFy/WtGnTNGLECK1YsWLAsBmA4SNoEFkjRozQypUr/W4GEHoMnQEArCJoAABWETQAAKsIGgCAVQQNAMAqggYAYBVBAwCwKmaM8bsNQGDEYrH1xpj5frcDCBKCBgBgFUNnAACrCBoAgFUEDQDAKoIGAGAVQQMAsOr/AVyrmV3SVzmyAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "ax = plt.figure(figsize = (7, 7)).add_subplot()\n", "centerAxes(ax)\n", "line = np.array([1, 0.5])\n", "xlin = -10.0 + 20.0 * np.random.random(100)\n", "ylin = line[0] + (line[1] * xlin) + np.random.randn(100)\n", "ax.plot(xlin, ylin, 'ro', markersize = 4);" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "We may look at these points and decide to model them using a quadratic function." ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "hide_input": true, "tags": [ "hide-input" ] }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZcAAAGKCAYAAADALgxIAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAgSUlEQVR4nO3df2xV9f3H8ddtS9l3PzA623oFq15b29tCQXon/jOn1gJutkwxTINZM50sc8YtTo3/LNkWlW7LMpjz6zdkamrMVzISB1aRTogszmnYRWMiBSwUHGDlh8DXiUJp7+f7x+GW29t7by/t5957zj3PR9KU+6v3w+ntfd3Pr/cJGGMEAIBNJYVuAACg+BAuAADrCBcAgHWECwDAOsIFAGBd2Tncl2Vl8LWFCxdqw4YNhW4G4CaBdDfQcwGydOTIkUI3AfAMwgUAYB3hAgCwjnABAFhHuAAArCNcAADWES4AAOsIFwCAdYQLAMA6wgUAYB3hAgCwjnABAFhHuAAArCNcAADWES4AAOsIFwDwi/5+qbFRKitzvvf35+ypCBcA8Iu2NmnHDml42Pne1pazpyJcAMAvdu6UYjHn37GYczlHCBcUtT/84Q9qbGzUzJkzdccdd+jkyZM6evSoWltbVVtbq9bWVh07dqzQzQTyo65OKjnztl9S4lzOEcIFRevAgQP64x//qGg0qvfff1/Dw8NavXq1Ojs71dLSor6+PrW0tKizs7PQTQXyo7tbqq+XSkud793dOXsqwgVFbWhoSF988YWGhob0+eef6+KLL9a6devU0dEhSero6NDatWsL20ggX0Ihads2aWjI+R4K5eypCBcUrenTp+vBBx9UdXW1gsGgzjvvPM2fP18HDx5UMBiUJAWDQR06dCjtz1i1apUikYgikYgOHz6cr6YDnke4oGgdO3ZM69at0549e/TRRx/pxIkTev7558/pZyxbtkzRaFTRaFQVFRU5ailQfAgXFK2NGzfq8ssvV0VFhaZMmaJbb71V//znP1VVVaWBgQFJ0sDAgCorKwvcUqD4EC4oWtXV1Xr77bf1+eefyxijTZs2KRwOq729XV1dXZKkrq4uLVq0qMAtBYpPWaEbAOTKvHnzdNttt2nu3LkqKyvTVVddpWXLlumzzz7TkiVL9PTTT6u6ulpr1qwpdFOBohMwxmR736zvCBSjSCSiaDRa6GYAbhJIdwPDYgAA6wgXAIB1hAsAwDrCBQBgHeECALCOcAEAWEe4AACsI1wAANYRLgAA6wgXAIB1hAsAwDrCBQBgHeECALCOcAEAWEe4AACsI1wAANYRLgAA6wgXAIB1hAsAwDrCBQC8avNmaepUKRBwvm/eXOgWjSBcAMCrFiyQBgedfw8OOpddgnABAK+KB0u6ywVEuACAV5WXZ75cQIQLAHhVT8/ZQCkvdy67RFmhGwAAmKDrrpNOnSp0K1Ki5wIAsI5wAQBYR7gAAKwjXAAA1hEuAADrCBcA8Ir+fqmxUSorc7739xe6RWkRLgDgFW1t0o4d0vCw872trdAtSotwAQCv2LlTisWcf8dizmWXIlwAwCvq6qSSM2/bJSXOZZciXADAK7q7pfp6qbTU+d7dXegWpUX5FwDwilBI2rat0K3ICj0XAIB1hAsAwDrCBQBgHeECALCOcAEAWEe4AICbeajkSyLCBQDczEMlXxIRLgDgZh4q+ZKIcAEAN/NQyZdEhAsAuJmHSr4kovwLALiZh0q+JKLnAgCwjnABADfy6BLkOMIFANzIo0uQ4wgXAHAjjy5BjiNcAMCNPLoEOY5wAQA38ugS5DiWIgOAG3l0CXIcPRcAgHWECwDAOsIFAGAd4QIAsI5wAQA38PiO/GSECwC4gcd35CcjXADADZJ35Pf2err3QrgAgBuk2oHv4d4L4QIAbpBqB77H6oklIlwAwA1CIamhwdP1xBIRLgDgFh6vJ5aI2mIA4BYeryeWiJ4LAMA6wgUAYB3hAgCwjnABgHwqsjIv6RAuAJBPRVbmJR3CBUXv+PHjuu2221RfX69wOKy33npLR48eVWtrq2pra9Xa2qpjx44Vupnwi+QyLx7eKJkJ4YKi99Of/lQLFy7Ujh079N577ykcDquzs1MtLS3q6+tTS0uLOjs7C91M+EVdXdFslMwkYIzJ9r5Z3xFwi08//VSzZ89Wf3+/AoHAyPV1dXXavHmzgsGgBgYGdN1112nnOJ8gI5GIotForpuMYtff7wyF7dzpBEt3t7O/xZsC6W5gEyWKWn9/vyoqKvSDH/xA7733npqbm7Vy5UodPHhQwWBQkhQMBnXo0KECtxS+UUQbJTNhWAxFbWhoSO+8845+/OMf691339VXvvKVcxoCW7VqlSKRiCKRiA4fPpzDlgLFhXBBUZsxY4ZmzJihefPmSZJuu+02vfPOO6qqqtLAwIAkaWBgQJWVlSkfv2zZMkWjUUWjUVVUVOSt3YDXES4oahdddJEuueSSkfmUTZs2qaGhQe3t7erq6pIkdXV1adGiRYVsJlB0mHNB0XviiSe0dOlSDQ4OKhQK6dlnn1UsFtOSJUv09NNPq7q6WmvWrCl0M4GiwmoxIEusFgPGSLtajGExAIB1hAsAwDrCBQBgHeECALCOcAGAXPNJmf1EhAsA5JpPyuwnIlwAINd8UmY/EeECALnmkzL7iQgXAMi17m6pvl4qLXW+d3cXukU5R/kXAMg1n5TZT0TPBQBgHeECALCOcAEAWEe4AACsI1wAANYRLgAA6wgXALDFhzXE0iFcAMAWH9YQS4dwAQBbfFhDLB3CBQBs8WENsXQIFwCwxYc1xNKhthgA2OLDGmLp0HMBAFhHuAAArCNcAGAi2NOSEeECABOxYIHU2+vsaentdS5jBOECABOxa1fmyz5HuAAArCNcAGAiamoyX/Y5wgUAJqKnR2pocDZMNjQ4lzGCTZQAMBFsmMyIngsAwDrCBQBgHeECALCOcAEAWEe4AACsI1wAANYRLgAA6wgXAIB1hAsAwDrCBQBgHeECALCOcAEAWEe4AACsI1wAANYRLgAA6wgXAIB1hAsApNLfLzU2SmVlzvf+/kK3yFMIFwBIpa1N2rFDGh52vre1FbpFnkK4AEAqO3dKsZjz71jMuYysES4AkEpdnVRy5i2ypMS5jKwRLgCQSne3VF8vlZY637u7C90iTykrdAMAwJVCIWnbtkK3wrPouQDwN1aF5QThAsDfWBWWE4QLAH9jVVhOEC4A/I1VYTlBuADwN1aF5QSrxQD4G6vCcoKeCwDAOsIFAGAd4QIAsI5wAQBYR7gAAKwjXAAA1hEuAADrCBcA/kKhyrwgXAD4S3KhynCYoMkBwgWAvyQXqhwcpCJyDhAuAPwlsVBlIioiW0W4APCXxEKV5eVSIOBcT0VkqwgXAP4SL1Q5NCRt3+7MuVAR2TrCBUVveHhYV111lW6++WZJ0tGjR9Xa2qra2lq1trbq2LFjBW4hCiYxaLZtcy7DCsIFRW/lypUKh8Mjlzs7O9XS0qK+vj61tLSos7OzgK0DihPhgqK2f/9+vfLKK/rhD384ct26devU0dEhSero6NDatWsL1DqgeBEuKGo/+9nP9Nvf/lYlCauDDh48qGAwKEkKBoM6dOhQ2sevWrVKkUhEkUhEhw8fznl7gWJBuKBovfzyy6qsrFRzc/OEf8ayZcsUjUYVjUZVUVFhsXVAceM0xyhab775pl566SWtX79eJ0+e1Keffqo777xTVVVVGhgYUDAY1MDAgCorKwvdVKDo0HNB0Vq+fLn279+vvXv3avXq1brhhhv0/PPPq729XV1dXZKkrq4uLVq0qMAtBYoP4QLfeeSRR/Taa6+ptrZWr732mh555JFCNwkoOgFjTLb3zfqOQDGKRCKKRqOFbgbgJoF0N9BzAQBYR7gAKE6ct6WgCBcAxSn5vC2U088rwgVAcUo+bwvl9POKcAFQnBLP20I5/bwjXAAUp8TztlBOP+/YoQ+gOMXL6aMg6LkAAKwjXAAA1hEuAADrCBcAgHWECwDAOsIFAGAd4QIAsI5wAQBYR7gAAKwjXAAA1hEuAADrCBcAgHWECwDAOsIFgHtxqmLPIlwAuBenKvYswgWAe3GqYs8iXAC4F6cq9izCBYB7capiz+I0xwDci1MVexY9FwCAdYQLAMA6wgWAO7HHxdMIFwDuxB4XTyNcALhLvMfS28seFw8jXAC4y4IFTrAkKy1laMxDCBcA7rJrV+rrh4YYGvMQwgWANzA05imECwB3qalJfT3lXzyFcAHgLj09UkODM8dSU+N8Uf7Fcyj/AsBdKPlSFOi5AACsI1wAuBM79D2NcAHgTuzQ9zTCBYA7cRZKTyNcALgTZ6H0NMIFgDtxFkpPYykyAHdiSbKn0XMBAFhHuADIL5YY+wLhAiC/WGLsC4QLgPxiibEvEC4A8oslxr5AuADIL5YY+wLhAiD3+vul2lopEJCuuEIaHJQ++MBZahwKFbp1yAHCBUDutbWNPn3xrl1M5Bc5wgVA7qWatGciv6gRLgByL9WkPRP5RY1wAZB73d3O6YrjamqYyC9y1BYDkHuhkNTXV+hWII/ouQAArCNcAADWES4AAOsIFwCAdYQLAMA6wgWAfZyzxfcIFwD2cc4W3yNcANjHOVt8j3ABYB/nbPE9wgXAxKWbW+GcLb4XMMZke9+s7wgUo0gkomg0WuhmuEtjozOnEos5PZT6euccLfCLQLob6LkAmDjmVpAG4QJgYvr7nWGvOOZWkIBwATAxbW3S6dNnL5eVMbeCEYQLgInZuVNKnLMdHnZK6wMiXFDk9u3bp+uvv17hcFiNjY1auXKlJOno0aNqbW1VbW2tWltbdezYsQK31INYbowM8hMulIJAgZSVlen3v/+9tm/frrfffltPPvmkent71dnZqZaWFvX19amlpUWdnZ2Fbqr3sNwYGeRnKTLLFeESixYt0n333af77rtPmzdvVjAY1MDAgK677jrtHGelE0uRgTHSLkXOT7iUlTnjsXGlpdLQ0IR/HDARe/fu1bXXXqv3339f1dXVOn78+Mht559/fsqhsVWrVmnVqlWSpMOHD+vDDz/MV3MBLyjwPhfGZlFgn332mRYvXqwVK1Zo2rRpWT9u2bJlikajikajqqioyGELgeKSn3BhbBYFdPr0aS1evFhLly7VrbfeKkmqqqrSwMCAJGlgYECVlZWFbKK3MaeKFPITLqGQEyh1dc7yxbY2XoDIC2OM7r77boXDYT3wwAMj17e3t6urq0uS1NXVpUWLFhWqid5HeX2kkL/aYkzqowD+8Y9/6Jvf/KZmzZqlkjNDs48//rjmzZunJUuW6N///reqq6u1Zs0aXXDBBRl/FhP6aTCn6mcFntCXeAHC8wiXNPjg6GcuKFzJpD5QnJhTRQr5CxdegIB3nMskfSjk9FSGhpzvlICBOJ8LkDVfDYsx1IXsuGBYDIB3cJ4WTBLhAmAs5kgxSYQLgLGYI8UklRW6AQBcKD5JD0wQPRcAgHWEC4CxqBeGSSJcAIxFvTBMEuEC+F2qXgpLkTFJhAvgd6l6KSxFxiS5P1wY+wVyK1UvhaXImCT3hwtjv0BupeqlUC8Mk+T+cGHsF8gteinIAfdvoqyrG11Aj7FfwC42TCIH3N9z4VMVAHiO+3sufKoCAM9xf88lESvHAMATvBUurBwDJib+way0VJo6lQ9oyDlvhQsrx4CJiX8wi8WkwUE+oCHnvBUu7BoGJibxg1kcH9CQQ94KF1aOAROT+MEsjg9oyCH3rxZLxMox4Nz09ztDXzt3OvMsp09LU6Y4w2J1dXxAQ854K1wAnJvEuRZjpHCYD2jIC28NiwHITnx1WG/v6EUwO3YUtl3wjeILF/bCAGd7LMnKGKxAfngzXBIDpLbW+YqHyYIFZ/fC9PZKV1xByMB/Uq0Ok5y/CyAPvBkuiZspd+1yvuLr9nftGvtH1dvrjDUTMPALVoehwLwZLuk+lcWvS/6jkpyNY2wYQzFL7NEPDjqrK0tKpPJylu8j77w5AJtYhj9RSYnzB1Ve7vRWkrFhDMUscWVYf78TJgyDoUC82XNJ3ExZU+N8xT+Z9fQ4Sy1373ZCJo4hARQ7yiPBRbzZc8lmM2UoJG3ffnYDGRvGUOw4sR5cxJvhki129MNPurv5MAXXKO5wAfyED1NwEW/OuQAAXI1wAbyMihRwKcIFcLtMAbJggbPsPl6RYsGCwrUTSEC4AG6X6fTeu3aNvm/yZaBA/BUuqT4BMqwAt0vev9Lby2sVrhcwxmR736zv6DrxEyYl7tovKXE2XUqj9wbU17PiBilFIhFFo9H8P3Fj49iKFPHX6uDg6N5KTY3U15f/NsKvAulu8EfPJVX58fgnwOTzXbCrGW4Tr0iRKP5a7emRGhqcChUNDc5lwAX8ES7pCl0mY1cz3CB5qFZyetMNDWeLssZfq/G9LUNDzvdQqHDtBhL4I1xSlR9PRtVYuEW6CfzEmnplZc5tzL3ApfwRLol/lA0Nzrh04ifAhgY++cE90hWgjPdS6uqc12v8tMWcSgIu5I9wSR466Ok5Gzb0VuA2iT3tVEO1VD+GB/gjXJIxTg036+4++5qMxZwVYYlDX+OFD+AC/gwXwM3iJ7yLB0h8KX1c4jAvPW+4FFWRATfKNPRF9WN4AD0XwI0Y+oLHES6AGzH0BY9jWAxwI4a+4HH0XAAA1hEuQL4klnWZOtUZ8mKHPYoU4QLkS2JZl8FBdtijqBEuQL6kKqDKDnsUKcIFyJdUBVTjy4w5aR2KDOEC5Evi8uL4Dvz4MuNMpzIGPIilyEC+ZFpeTDFKFBl6LpPBUAZsYUc+igzhMhkMZcAWduSjyBAu2UrVS2Eow39ytVeF00CgyASMMdneN+s7FqXGRqd3EoudnYiVxl5HyY6iFYlEFP3ii7O/8zh+9/CvQLob6LlkK1UvhXOa+w97VYCsEC7ZSjXhyjnN/SfTXhUAIwiXbGWacGXuxT8y7VWRWEEInMGciw2p5mMSx9/jp6ndudP5hJt4jnR4RiQSUTQazXyn8V4LQHFhziWnxltGypJl/6AXC0hih74d453YiTcc/6irG91zYS4GPkXPJR/Yfe0fbIYEJNFzyY94YcLEORcUJ05PDEii55Jb8ZVDV17pXP7gg8y7r8dbacRKJKs2bNiguro61dTUqLOzs9DNAYoK4ZJLiRP527dL4XDmYBhv4j+bhQGZAohwGjE8PKyf/OQnevXVV9Xb26sXXnhBvb29hW4WUDQIl1xKnMg3xjm1baZgGG/iP5uFAZkCqNhWrU0kLM88pqS8XJsOHlRIUnl5uW6//XatW7cu500G/CLrfS4LFy40R44cOacffvjwYVVUVEykXQVjtc3btkknT6a/vbk58/2/9CXnTfPUKWnXrrE/68zto9q8dWv658l0W55ZOc7pjtcEHvPJJ5/oxIkTqq6uHtPO/zt0SJecOqV+YxT+0pekmhqnaKUH+P5vME/82uatW7f2GGMWprzRGJPt1zlrbm6eyMMKymqbd+82pqHBmNJSY8rLjSkpMUZyvjc0ZL5/Q4Nz2Rjn3/HHxr8Sbh/V5sT7Jj9P8m01NamfLw+sHOfS0tHHpLR0wo957rnnzH333Zf6MWeOW3Om351L+f5vME983Oa0mcGwWC4lllHfvn38Jarpyq6nKpaYSn+/M/QWv28oNPp5kpfJSt4eJpvIEu+ExwzHL0vav3+/Lr744tSPYZ8ScM4Il3yZyPk64nMKw8Njb0s34R+fdygpcWpfZXqePXu8/aY5kT0lZx5jSku1e8oU7fvv/9bg4KBWr16t9vb21I9hnxJwznIaLsuWLcvlj88JV7U5PgGfSkIYjLR5vE/YyRP6paWZ3zRzuLqsYMf5TMgHhoa0a+1atdxzj8LhsJYsWaLGdPM18RCTPLcx0lWv5yzR5vzIdZspXOlmZWVjey0lJemLIiYXTYz3WnbtSv3z4z8jXUFNtxdhzHP7sipcCfgLhSs9KXk4pqYm8zBQqjmV8YIl01Cdm+ca+vul3l73tg/wOcLFzZLDoqcndRikqwSwZ0/qn5vtHIXb5hoSh+nC4bG3F7p9AEZMKlzWrFmjxsZGlZSUjBkuWL58uWpqalRXV6eenp6Ujz969KhaW1tVW1ur1tZWHTt2bDLNOWff+973NGfOHM2ZM0eXXXaZ5syZk/J+l112mWbNmqU5c+YoEonkr4EpFgH88pe/1PTp00favX79+pSbIzds2KC+0tKxY5kNDdkvKrBUhPGhhx5SfX29mpqadMstt+j48eMp7zfucU78fw4Opm7vJI1XEsYYo/vvv181NTVqamrSO++8M+nnnIx9+/bp+uuvVzgcVmNjo1auXDnmPps3b9Z555038pr59a9/XYCWjjbe79ptx3nnzp0jx2/OnDmaNm2aVqxYMeo+bjnOd911lyorKzVz5syR67J9r7VaEinTOmUzzj6X3t5es2PHDvOtb33L/Otf/xq5ftu2baapqcmcPHnS9Pf3m1AoZIaGhsY8/qGHHjLLly83xhizfPly8/DDD9tYdz0hDzzwgPnVr36V8rZLL73UHD58OM8tOiNp78uK++83v/vd70bfJ2nvRqy01IRCIfPh66+b4SuuMDHJxCRnX0se97LE9fT0mNOnTxtjjHn44YfT/p7HPc7Je1TiX5b2ngwNDZlQKGR2795tTp06ZZqamsy2bdtGbm9ubjavvPKKWbhwoYnFYuatt94yV1999aSfdzI++ugjs3XrVmOMMZ9++qmpra0d1WZjjHn99dfNd77znUI0L63xftduO86JhoaGTFVVldm7d++o691ynP/+97+brVu3msbGxpHrsnmvHe/1n0Zu9rmEw2HVpRiKWLdunW6//XZNnTpVl19+uWpqarRly5aU9+vo6JAkdXR0aO3atZNpzoQZY/SXv/xFd9xxR0GeP6OkXskdL7ww9j5Jw1cng0FtOnhQ1TfeqJKpU/U/Dz6ozscfl/r6cnMGzHFWlc2fP19lZU4B7muuuUb79++f2PMk/j8DAWeptcXS9lu2bFFNTY1CoVDakjDr1q3T97//fQUCAV1zzTU6fvy4BgYGJv3cExUMBjV37lxJ0te+9jWFw2EdOHCgYO2xxW3HOdGmTZt0xRVX6NJLLy10U1K69tprdcEFF4y6Lpv32mxe/+ciJ3MuBw4c0CWXXDJyecaMGSlf8AcPHlQwGJTk/JEcOnQoF80Z1xtvvKGqqirV1tamvD0QCGj+/Plqbm7WqlWr8tu4pEn1C48c0Z/+9Cc1NTXprrvucrq3ScNXp4eGdMmJE2cD6X//d2JvONkuRT6HmmXPPPOMbrrpppS3jXucE/+f4bCzMfVc9g2NI5vXbbav7ULYu3ev3n33Xc2bN2/MbW+99ZZmz56tm266SdtcsOJvvN+1m4/z6tWr034Qddtxjsvmvdb2MR/3fC6BQGCjpIuS9wA89thjWrRoUcrHmBTLmwOBtCvWcurGG2/Uxx9/POb6xPa/8MILGXstb775pi6++GIdOnRIra2tqq+v17XXXpuXNq8tK9Plw8MqlRQLBBSrrdXu3l4FAgH94he/0M9//nM988wzo5bgfrW09OynhlhM0z7+eGLHPx4asdjZ0Ej1B5MUgEO9vZpzZrw38Tg/9thjKisr09KlS1M+3bjHOcfnSsnmdeum13aizz77TIsXL9aKFSs0bdq0UbfNnTtXH374ob761a9q/fr1+u53v6u+vr4CtdQx3u/arcd5cHBQL730kpYvXz7mNjce53Nh+5iPGy7GmBvj/8z2h86YMUP79u0buZyutEZVVZUGBgYUDAY1MDCgysrKbJ8iaxs3bsx4+9DQkF588UVtTVXU8Yx42ysrK3XLLbdoy5YtOQ2XUW3u7x850VhJXZ1KurudT+6S7rnnHt18881jHn+yulpT9+5VqSSVlOiTCy9MX9okk2yXIied2resvl7vv//+qLt0dXXp5Zdf1qZNm9K+YPN9nJNl87rN9rWdT6dPn9bixYu1dOlS3XrrrWNuTwybb3/727r33nt15MgRXXjhhfls5ijj/a7deJwl6dVXX9XcuXNVVVU15jY3Hue4bN5rbR/znAyLtbe3a/Xq1Tp16pT27Nmjvr4+XX311Snv19XVJcl580nXE8qljRs3qr6+XjNmzEh5+4kTJ/Sf//xn5N9/+9vfRq3CyLmkFWMD//VfIzf99a9/TdmW8p4e7Z4yRaa0VLG6OnWcf3760iaZZLsUOXlV2ZNPjhpO+/uzz+o3v/mNXnrpJX35y19O+SMKfpwlfeMb31BfX5/27NmTtiRMe3u7nnvuORlj9Pbbb+u8884bGW4oBGOM7r77boXDYT3wwAMp7/Pxxx+PfCrdsmWLYrGYvv71r+ezmaNk87t223GOyzTK4bbjnCib99psXv/nJNNsf9LXGC+++KKZPn26KS8vN5WVlWb+/Pkjtz366KMmFAqZK6+80qxfv37k+rvvvntkZdmRI0fMDTfcYGpqaswNN9xgPvnkk/FWJljX0dFhnnrqqVHXHThwwNx0003GGGN2795tmpqaTFNTk2loaDCPPvpo3tuY6M477zQzZ840s2bNMm1tbeajjz4yxoxuszHOapva2loTCoUm3uZ0VZrHk1R9+YMpU8yMGTPM7NmzzezZs82PfvSjMW12y3FOddyeeuop89RTT5nm5mYTi8XMvffea0KhkJk5c+aoVZKF8MYbbxhJZtasWSPH95VXXhlpszHGPPHEE6ahocE0NTWZefPmmTfffLOgbU73u05ss9uOszHGnDhxwlxwwQXm+PHjI9e58Tjffvvt5qKLLjJlZWVm+vTp5s9//nPa91oL7xtpM4PyL7AjYfhuTMma0lKn5+VxlH8Bxkg7KTPunAswrv5+Z/VWqo2NbtjZDyDvCBdMXltb6mApLT1bEBOAr1BbDGdNtMR+qlVk51JmBkDRIVxw1jlshhwlcVWZ5Oycp7cC+BrhgrMmWmI/cSlyQ4Ozc57eCuBrzLngrKTNkFlPxOd45zwA76HngrMsldgHAHouOIseCABL6LkAAKwjXAAA1hEuAADrCBcAgHWECwDAOsIFAGAd4QIAsI5wAQBYR7gAAKwjXAAA1hEuAADrCBcAgHWECwDAuoAxptBtADwhEAhsMMYsLHQ7AC8gXAAA1jEsBgCwjnABAFhHuAAArCNcAADWES4AAOv+H708OBig61OOAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "ax = plt.figure(figsize = (7, 7)).add_subplot()\n", "centerAxes(ax)\n", "quad = np.array([1, 3, 0.5])\n", "xquad = -10.0 + 20.0 * np.random.random(100)\n", "yquad = quad[0] + (quad[1] * xquad) + (quad[2] * xquad * xquad) + np.random.randn(100)\n", "ax.plot(xquad, yquad, 'ro', markersize = 4);" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "And we may look at these points and decide to model them using a logarithmic function." ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "hide_input": true, "tags": [ "hide-input" ] }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaIAAAGKCAYAAABO9EU0AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAgsklEQVR4nO3df2xc5b3n8c9jTy01TUgvNCExDg1e5zqJDQ7N8GN1AYlwkzqGmA2wVVApKT8UgYKg261071UltEKsGvWqWyhEzfV2ywV1N/6De29dkx/lR8OC2kTeaUhviYkxOKiJExKnEIWG0sT2s38cOx5PZuyZzDnznB/vl2RNxnN8zjNDez7zfJ/nPMdYawUAgCtVrhsAAEg2gggA4BRBBABwiiACADhFEAEAnEoFtF9fpuK1trZq586dfuwKAOCWKfRCqHtEJ06ccN0EAEDAQh1EAID4I4gAAE4RRAAApwgiAIBTBBEAwCmCCADgFEEEAHCKIAIAOEUQAQCcIogAAE4RRAAAp4pe9NQY81NJt0k6bq1tHvvdP0paI+mMpPcl3WetPZn7twsXLtSsWbNUXV2tVCqlTCbjS+MBANFXSo/onyW15vzuFUnN1tqrJL0r6R8K/fGuXbu0b98+QggAMEnRQWStfUPSRzm/e9laOzz2dI+kOh/bBgBIAD/HiO6XtCPfC8YYrVq1SsuXL1dHR8eUO+no6FA6nVY6ndbQ0JCPzQMAlGRgQGpqklIp73FgIJDDGGuLv4edMWahpJfGx4iyfv9dSWlJd1hvh5N2euTIEdXW1ur48eNauXKlnnnmGd10003THi+dTlPKAwBXmpqkAwek0VGpqkpavFjav/9C9xbcjfGMMevlTWL4ui2QarW1tZKkuXPnau3aterp6Sn3sACAoPX1eSEkeY99fYEcpqwgMsa0Svo7Se3W2k/zbXP69Gl98skn5/798ssvq7m5Od+mAIAwaWz0ekKS99jYGMhhig4iY8xWSbslNRpjDhtjHpD0rKRZkl4xxuwzxmyRvFJcW1ubJOnYsWO64YYb1NLSomuvvVa33nqrWltzJ98BAEKnu9srx1VXe4/d3YEcpqQxohL4slPGiAAgNoIbIwIARFSFZsVNhyACgKRas8abFTcy4j2uWeOkGQQRACRVhWbFTYcgAoCkqtCsuOkQRACQVBWaFTedolffBgDETH19OSsl+IYeEQDAKYIIAOAUQQQAcIogAgA4RRABAJwiiAAAThFEAACnCCIAgFMEEQDAKYIIAOAUQQQAcIogAoCkCckN8cYRRAAQJX6ESEhuiDeOIAKAKPEjREJyQ7xxBBGAaAlZWakist9zb2/5IVJdPfXzCiOIAERLyMpKFZH9nrNl31W1lIAeHp76eYURRACiJWRlpYrIfs/jcu+qWkpAL148+RbhixcH0+4iEUQAoqWxcfJJdLxHEGe573npUq8Xs3+/d5dVqfiAHhiQzpyZ2La+3tktwscRRACipbvb+waf2yOIs2Lec7EBvWbNRNmuqkqqqZkIM0eMtTaI/fqy03Q6rUwm48euACDeBga8kOnrk664wvvdwYNeIHV3T4RNKjV5rKm6ulJjRKbQC/SIACAO6uu9Ut3wsNfLGRjIP14UwtImQQQAcTPVeFEIS5sp1w0AAPissdHrCY2Ont/rGe85hQg9IgCImxD2eqZCjwgA4iaEvZ6p0CMCEB9hXv4nzG1zjCACEB9hXv4nzG1zjCACEB9hXv4nzG1zjCACEB8hvEbmnDC3zTGCCEB8hHm2WJjb5hiz5oA4yF7eJXdJlyQJ82yxMLfNMXpEQBwwEI4II4iAOGAgPPqCnN4d8qnjBBEQBwyER1+QvdqQ95iLDiJjzE+NMceNMW9n/e5iY8wrxpj+sce/yve3O3fuVGNjoxoaGrRp0yY/2g0gGwPhbvnR4wiyVxvyHnMpPaJ/ltSa87u/l/SatXaRpNfGnk8yMjKijRs3aseOHert7dXWrVvV29t7wQ0GkEf2LQCy79qJyvCjxxFkrzbkPeaig8ha+4akj3J+fbuk58f+/byk/5T7dz09PWpoaFB9fb1qamq0bt06dXV1XWh7ASB8/OhxBNmrDXmPudwxokuttUclaexxbu4Gg4ODWrBgwbnndXV1GhwcLLjDjo4OpdNppdNpDQ0Nldk8AJEX8oF2SRN3RC30vBhB9mpD3mMOfLJCvluRG1PwjrHasGGDMpmMMpmM5syZE2TTAERByAfaUb5yg+iYMWa+JI09Hs/doK6uTocOHTr3/PDhw6qtrS3zsAASI+QD7ZKkgwenfo4plRtEv5C0fuzf6yWdN/hzzTXXqL+/XwcPHtSZM2fU2dmp9vb2Mg8LIBaKKbsFOdB+IWW/fH8T8skAoWetLepH0lZJRyWdlXRY0gOSLpE3W65/7PFia60GBwft6tWr7bht27bZRYsW2fr6evvkk0/aYi1fvrzobQFE0NKl1lZVWSt5j0uXnr/N++97v6+u9h7ff7+yx89thzTxM/43QbbRT27bWTBfjM0zhuNHvvmxk3Q6rUwm48euAIRRKuWN/YyrrvYG1MN4/KYmb4xqvExYzN+ETfZ7qKryZtBVbv27gpMDWFkBgDuuS1qlHD97rGpc1MpwIR1vI4gAuOP6+pZSjp8dWuNCeE3OlFwHfwGU5gCgGHG41Ybb91CwNMf9iABEXyVOsHG4n1BI3wOlOQDRx0WvkUYQAShdsdffVGp5npAOwvsmCssclYEgApLA7xNZdg+kt1dasiT/PivVUwnpILxvYt7jY7ICkAR+Xz+Se/2NJC1dev4+K3WdUBwmEkzF9fVW/uA6IiDR/C5d5etx5NtnpXoqIV9dumy5U8dHRmJVoiOIgCTwOxC6u6WamonnxuTfZ1DXCcV8zOQ8459jthiV6CjNAUkQROnKZTnM7VI17kS7RFewNEcQAYieaJ+QL1y0A5gxIgAxEvdZcoW4XhIpIKysACB6urvPLwsmQUhXRigXQQQgemJ6Qk4qSnNA0sRpxlmc3kuCEURA0sTpKv04vZcLFYMwJoiApInyumy5J93sO6ZG7b34JQZhTBABSePXjDMX38RzT7qpVDJnz2WL8heLMQQRkDR+TQGe6pt4UCGVe9IdGYnldOaSxGAqO0EEJMV4OPz1X3vP3323vHXZpvomHlS5KN9JN85rzBUjBtcWEURAUvgdDlN9Ew+qXFTMSTcGg/clicGCrwQRkBR+h8NUoRBUuaiYk25YBu+TFohlIIiApPA7HKYKBZflorAM3oclECOAIALiLPtb+ZkzXlhUIhxclovCMngflkCMAIIIiLPsb+UDA949hCI8llCUsAzehyUQI4AgAsqR3eNYtMj7KWdMwO9xhSR+Kw/L4H1YAjECuB8RUI7cq/vHXei9Yvy+30y071+DeOF+REAgsnsc2S6091HJmW1ASBBEQDmyxwGyFTsmkFuKu+KKys1sw/SYgl0RBBFQjuweR0OD91NK7yN3iq9EDyZMmIJdEYwRAS6lUt5Jblx1tdd7QTjw38dPjBEBocQU32CVW1rjv09FEERAJeWeGDdvphQXpHJLa0z2qAhKc0AllTKdemDAO3H29XnfxLu7L3yygZ/7ihJKa2FCaQ4IhVKmZ/s5UJ7UQXdKa5FAEAGVVMqJ0c9rikrZV5ymLFNaiwSCCKikUk6Mfn6bL2VfYeo9lRuKXEcVCYwRAWHlaowoTOMqLFEUJ8GNERljGo0x+7J+Tj311FOTtnn99dc1e/ZsLVu2TMuWLdMTTzxR7mGB+PPz23wp+wrTuEoSF21NoLKDyFrbZ61dZq1dJmm5pE/Xrl173nY33nij9u3bp3379unxxx8v97BAsgU5jhOmcZUwhSIC4/cY0S2S3v/yl7/s824BTBLkOE6YxlXCFIoIjN9BtE7S1nwv7N69Wy0tLVq9erX2T1Hj7ejoUDqdVjqd1tDQkM/NA2IiKSWrMIUiAuPbZAVjTI2kI5KarLUfZr926tQpVVVVaebMmdq+fbsee+wx9ff3T7tPJisABcRtED+pF9wmS0UuaF0taa+19ljuCxdddJFmzpwpSWpra9PZs2d14sQJHw8NJEzcSlZhmjKOikv5uK+7VaAs9+GHH+rSSy+VMUY9PT0aHR3VJZdc4uOhgYQZL1nFRVJKjcjLlx6RMWaGpJWS/nX8d1u2bNGWLVskSS+++KKam5vV0tKiRx99VJ2dnTKmYC8NQNDCtnoCs+MSjQtaEU2MKZQnbGNM/PdMgoK9D4II0RS2E2nUhGn1BCQFq28jZhhTKA+lMIQIQYRo8uNEGrZxkkqK26w7RBqlOUSTH2MKlPeASipYmvNz+jZQOX5MX6a8B4QCpTkEK8zlL8ZJgFAgiBCsMF8xzzjJZGH+0oBYY4wIwWKacHQwZoZgMX0bjlD+ig7GzOAIQYRgRan8lfTSFF8a4AilOWBc0ktTLLODYDF9G5hW0ktTcVvRG5FBaQ4YR2kKcIIgAsZFaTwLiBGCCNHn1ySD8dLU8LD3yPgIUBEEEaIvqItmkz6LDqgQggjRF9QkgzCvCgHECEGE6AtqkkHSZ9EBFUIQIfqCmmTALDqgIriOCNEX1PUv3d3nX+AJwHf0iBBericLMIsOqAiCCOHFZAEgEQgihBeTBYBEIIgQXkwWiC7XZVVECkGE8MqdDbd5Mye3qKCsihIQRAiv3MkCGzcWd3Lj27h7lFVRAoII0VHsyY1v4+5RVkUJCKIoSfo3/WJPbnwbd4+VzFECgihKkv5NP/vkVl8vnTmTP5T5Nu4e12ChBARRlCT9m372ya2mxguffKHMt3EgUgiiKOGb/oSpQplv48VJeqkXoUEQRUkcvun7dfIjlMuX9FIvQsNYa4PYry87TafTymQyfuwKYdHU5J30Rke9AFm8+MIWLB0YOH9BUno+pUmlvBAaV13t9SKBYJhCL7D6NirLr3GuoFbcTpLGxslfCuhVwhFKc8gvqPEDSmrhEYdSL2KB0hzy86uElouSGpBUlOZQoqCmilNSA5CD0hzyc1lCY1oxkCgEEfJzOX7AtGIgUXwJImPMB8aY3xtj9hljzhvUsdbq0UcfVUNDg6666irt3bvXj8MiSC4vCk36ChJAwvjZI7rZWrvMWpvOfWHHjh3q7+9Xf3+/Ojo69PDDD/t4WMQOM+uARKlIaa6rq0v33nuvjDG6/vrrdfLkSR09erQSh4YUvTEXphUDieLXrDkr6WVjjJX0T7lTwgcHB7VgwYJzz+vq6jQ4OKj58+eft6OOjg51dHRIkoaGhnxqXsKNj7mMjk6MuYR55hoz64BE8atH9DfW2q9IWi1p4xtvvDHpxXzXKhmTf0r5hg0blMlklMlkNGfOHJ+al3DFjrlEred0oZLyPoGI8CWIrLVHxh6PS/q3np6eSa/X1dXp0KFD554fPnxYtbW1fhwaxSh2zCUps9WS8j6BiCg7iIwxXzDGzBr/t6RVzc3Nk7Zpb2/XCy+8IGut9uzZo9mzZ+ctyyEgxY65JGW2WlLeJxARfowRXSrp38ZKbSlJ/6e1tfU/btmyRZL00EMPqa2tTdu3b1dDQ4NmzJih5557zofDomjFjrkkZRHMpLxPICJYay7Jctd927xZ2rgx/uvAsd4d4ELBteYIoiQLamFTADhfwSBiiZ8kY6wEQAgQREnGCgYAQoAgSjJWMAAQAtyPKMlYwQBACNAjAkrBqgyA7wgioBSsygD4jiACSsFMQ8B3BBFQCmYaAr4jiIBSMNMQ8B1BhPCIwkQAl7dQB2KKIAqr3JPy66+H/yRdLiYCAInEWnNhlbsOXCrlfQuP87pwqZQXQuOqq733DCAOWGsucnJnZ505E//ZWkwEABKJIAqr3JNyTU38T9JMBAASiSAKq9yT8i9/6f9JOmyTA5gIACQSY0RJxv2IAFQOY0TIg1UCAIQAQZRkTA4AEAIEUZIxOQBACBBEURDUpAImBwAIAYIoCoJYcSBsM+YAJBZBFAVBTCpgOR0AIUEQhcF0vZMgJhUwYw5ASBBEQSi17DVd7ySISQW54VZdTZkOgBNc0OqngQEvRHp7J35XzIWiLhb7HG9rX593vLNnJWu5sBVAULigtSLGezbZiil7ubieJ3vG3MiIF0ISZToAFUcQ+Sl73GVcMcHi+noeLmwF4BBB5KfsE/q4YoLF9fU8roMQQKKlXDcgVrq7J8ZdGhu951G4SHQ8CAHAAYLIT5zQAaBklOYAAE4RRAAApwgiAIBTBBEAwCmCCADgFEEEAHCKIAIAOEUQAQCcKjuIjDELjDG7jDHvGGP2G2Mey93m9ddf1+zZs7Vs2TItW7ZMTzzxRLmHBQDEhB8rKwxL+q/W2r3GmFmSftvb26ulS5dO2ujGG2/USy+95MPhAABxUnaPyFp71Fq7d+zfn0h6Z3BwsOyGxVqpN84DgBjzdYzIGLNQ0tXXXXfdea/t3r1bLS0tWr16tfZPsR5bR0eH0um00um0hoaG/GxeeEx3R1YASBDf7tBqjJkp6f9K+u/W2n/Jfu3UqVOqqqrSzJkztX37dj322GPq7++fdp+Ru0NrsVzckRUA3Ar2Dq3GmM9J+hdJ/9ta+6+5r1900UWaOXOmJKmtrU1nz57ViRMn/Dh05RVTVptuG25EBwDn+DFrzkj6X5Lesdb+j3zbfPjhhxrvefX09Gh0dFSXXHJJuYd2o5iy2nTbcCM6ADjHj1lzfyPpG5J+b4zZJ0nbtm3TH/7wB0nSQw89pBdffFE//vGPlUql9PnPf16dnZ3y8iuCsm8HPjrqPS91G+5bBADn+DZGlMOXnYZyjKipyevljI56ZbXFi88PlWK2AYBkCXaMKFGKKatRegOAonGr8FIVU1aj9AYARaNHBABwiiACADhFEAWN5XwAYEoEUdBYzgcApkQQBa2Y644AIMEIImn68lk55TWW8wGAKRFE0vTls3LKa5s3ewEmeY+bN/vXbgCIAYJIOr981ts7uedTTnlt48aJlbWHh73nAIBzCCJpcvlsXHbPp5zyGmNEADAlgkiaWJInW3ZolLNkD2NEADAlgkiaWJJn6dL8oTH++vCw91hfX/y+WXcOAKbEWnPZuru9clxfnxdCfoQG684BwJQIomyEBgBUHKU5AIBTBBEAwCmCCADgFEEEAHCKIAIAOEUQAQCcIogAAE4RRAAApwgiAIBTBBEAwCmCCADgFEEEAHCKICpkYEBatEgyxvtZtGjijq0AAN8QRIWsWSO9997E8/fem7hjKwDANwRRIflu6c1tvgHAdwRRIflu6V3Kbb4HBqSmJimV8h4p6wFAXgRRId3dUkPDxPOGhtLu2LpmjXTggDQy4j1S1gOAvLhDayH19VJ//4X/fV+fNDrq/Xt0lLIeABRAjygojY1S1djHW1VVWlkPABKEIApKd7e0eLFUXe09llLWA4AEoTQXlPp6af9+160AgNCjRwQAcIogAgA4RRABAJzyJYiMMa3GmD5jzHvGmL/Pfd1aq0cffVQNDQ266qqrtHfvXj8OCwCIgbKDyBhTLWmzpNWSlkq6u7e3d9I2O3bsUH9/v/r7+9XR0aGHH3643MMCAGLCjx7RtZLes9YOWGvPSOrs6uqatEFXV5fuvfdeGWN0/fXX6+TJkzp69KgPhwYARJ2x1pa3A2PuktRqrX1w7Pk3Lr/88hfmzJlzbpv33ntP8+bN08yZMyVJ7777rurq6jRjxozz9jc0NKQTJ05Ikv7yl79o2bJlZbUvroaGhpT9GWMCn01hfDaF8dkU5sdn89vf/vaX1trWvC9aa8v6kfSfJf0k6/k3HnnkEZutra3Nvvnmm+eer1ixwmYyGTudGTNmTLtNUi1fvtx1E0KLz6YwPpvC+GwK8+mzKZgjfpTmDktakPW8rra2dtIGdXV1OnTo0MQfHD6s3G0AAMnkRxD9P0mLjDFXGGNqJK1rb2+ftEF7e7teeOEFWWu1Z88ezZ49W/Pnz/fh0ACAqCt7iR9r7bAx5hFJv5RULemnTU1NV23ZskWS9NBDD6mtrU3bt29XQ0ODZsyYoeeee66ofX/pS18qt3mxtWHDBtdNCC0+m8L4bArjsyks6M+m7MkKBfiy03Q6rUwm48euAABumUIvsLICAMApgggA4FQog2jnzp1qbGzU22+/rU2bNrluTmgcOnRIN998s5YsWaKmpiY9/fTTrpsUOiMjI7r66qt12223uW5KqJw8eVJ33XWXFi9erCVLlmj37t2umxQaP/zhD9XU1KTm5mbdfffd+uyzz1w3yZn7779fc+fOVXNz87nfffTRR1q5cqUWLVqklStX6uOPP/b9uKELopGREW3cuFE7duxQU1OTtm7dqtwlg5IqlUrpBz/4gd555x3t2bNHmzdv5rPJ8fTTT2vJkiWumxE6jz32mFpbW3XgwAH97ne/4zMaMzg4qB/96EfKZDJ6++23NTIyos7OTtfNcuab3/ymdu7cOel3mzZt0i233KL+/n7dcsstgXQOQhdEPT09amhoUH19vYwxWrdunXKXDEqq+fPn6ytf+YokadasWVqyZIkGBwcdtyo8Dh8+rG3btunBBx903ZRQOXXqlN544w098MADkqSamhp98YtfdNuoEBkeHtaf//xnDQ8P69NPP030NY433XSTLr744km/6+rq0vr16yVJ69ev189//nPfjxu6IBocHNSCBRPXx9bV1XGyzeODDz7QW2+9peuuu851U0LjW9/6lr7//e+rqip0/7N2amBgQHPmzNF9992nq6++Wg8++KBOnz7tulmhcNlll+k73/mOLr/8cs2fP1+zZ8/WqlWrXDcrVI4dO3buus/58+fr+PHjvh8jdP+PzTed3JiCs/4S6U9/+pPuvPNOPfXUU7roootcNycUXnrpJc2dO1fLly933ZTQGR4e1t69e/Xwww/rrbfe0he+8AXGXsd8/PHH6urq0sGDB3XkyBGdPn1aP/vZz1w3K3FCF0QsBzS1s2fP6s4779TXv/513XHHHa6bExq//vWv9Ytf/EILFy7UunXr9Ktf/Ur33HOP62aFQl1dnerq6s71nu+66y7uCTbm1Vdf1RVXXKE5c+boc5/7nO644w795je/cd2sULn00kvP3S3h6NGjmjt3ru/HCF0QXXPNNerv79fBgwdlrVVnZ6dylwxKKmutHnjgAS1ZskTf/va3XTcnVL73ve/p8OHD+uCDD9TZ2akVK1bwzXbMvHnztGDBAvX19UmSXnvtNS1dutRxq8Lh8ssv1549e/Tpp5/KWqvXXnuNiRw52tvb9fzzz0uSnn/+ed1+++3+H2SqFVHL+CnLtm3b7KJFi2xNTY198skny91dbLz55ptWkr3yyittS0uLbWlpsdu2bXPdrNDZtWuXvfXWW103I1Teeustu3z5cnvllVfa22+/3X700UeumxQajz/+uG1sbLRNTU32nnvusZ999pnrJjmzbt06O2/ePJtKpexll11mf/KTn9gTJ07YFStW2IaGBrtixQr7xz/+8UJ3XzAzWOIHAFAJLPEDAAgngggA4BRBBABwiiACADhFEAEAnCKIAABOEUQAAKcIIgCAUwQRAMApgggA4BRBBABwiiACADhFEAEAnCKIAABOEUQAAKcIIgCAUwQRAMApgggA4BRBBABwiiACADhFEAEAnCKIAABOEUQAAKcIIgCAUwQRAMApgggA4BRBBABwiiACADiVKuePjTH/KGmNpDOS3pd0n7X2ZO52Cxcu1KxZs1RdXa1UKqVMJlPOYQEAMVJuj+gVSc3W2qskvSvpHwptuGvXLu3bt899CA0MSE1NUirlPQ4MuG0PACRcWUFkrX3ZWjs89nSPpLrymxSwNWukAwekkRHvcc0a1y0CgETzc4zofkk78r1gjNGqVau0fPlydXR0TLmTjo4OpdNppdNpDQ0N+di8MX190uio9+/RUe85AMAZY62degNjXpU0L89L37XWdo1t811JaUl3WG+Hk3Z65MgR1dbW6vjx41q5cqWeeeYZ3XTTTdM2Lp1O+1/Ka2ryekKjo1JVlbR4sbR/v7/HAADkMoVemHaygrX2b6fcszHrJd0m6RZbINVqa2slSXPnztXatWvV09NTVBAForvbK8f19UmNjd5zAIAzZZXmjDGtkv5OUru19tN825w+fVqffPLJuX+//PLLam5uLuew5amv93pAw8PeY329u7YAAMoeI3pW0ixJrxhj9hljtkheKa6trU2SdOzYMd1www1qaWnRtddeq1tvvVWtra1lHhYAEBfTjhFdIF92GsgYEQDAhYJjRKysAABwiiACADhFEAEAnCKIAABOEUQAAKcIIgCAUwQRAMApgggA4BRBBABwiiACADhFEAEAnCKIAABOEUQAAKcIIgCAUwQRAMApgggA4BRBBABwiiACADhFEAEAnCKIAABOEUQAAKcIIgCAUwQRAMApgggA4BRBBABwiiACADhFEAEAnCKIAABOEUQAAKcIIgCAUwQRAMApgggA4BRBBABwiiACADhFEAEAnCKIAABOEUQAAKcIIgCAUwQRAMApgggA4FRZQWSM+W/GmEFjzL6xn7Z82+3cuVONjY1qaGjQpk2byjkkACBm/OgR/dBau2zsZ3vuiyMjI9q4caN27Nih3t5ebd26Vb29vT4cFgAQB4GX5np6etTQ0KD6+nrV1NRo3bp16urqCvqwAICI8COIHjHG/Lsx5qfGmL/KfXFwcFALFiw497yurk6Dg4MFd9bR0aF0Oq10Oq2hoSEfmgcACLNpg8gY86ox5u08P7dL+rGk/yBpmaSjkn6Q+/fW2nz7LHi8DRs2KJPJKJPJaM6cOcW/EwBAJKWm28Ba+7fF7MgY8z8lvZT7+7q6Oh06dOjc88OHD6u2traUNgIAYqzcWXPzs56ulfR27jbXXHON+vv7dfDgQZ05c0adnZ1qb28v57AAgBgpd4zo+8aY3xtj/l3SzZL+iyQdOXJEbW3eTO5UKqVnn31WX/3qV7VkyRJ97WtfU1NTU5mHBQDEhck3huMDX3aaTqeVyWT82BUAwK2CkwNYWQEA4BRBBABwKhpBNDAgNTVJqZT3ODDgukUAAJ9EI4jWrJEOHJBGRrzHNWtctwgA4JNoBFFfnzQ66v17dNR7DgCIhWgEUWOjVDXW1Koq7zkAIBaiEUTd3dLixVJ1tffY3e26RQAAn0y7xE8o1NdL+/e7bgUAIADR6BEBAGKLIAIAOEUQAQCcIogAAE4RRAAApwgiAIBTBBEAwCmCCADgFEEEAHCKIAIAOEUQAQCcIogAAE4RRAAAp4y11nUbCjLG7LTWtrpuBwAgOKEOIgBA/FGaAwA4RRABAJwiiAAAThFEAACnCCIAgFP/H2nqwv3UirghAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "ax = plt.figure(figsize = (7, 7)).add_subplot()\n", "centerAxes(ax)\n", "log = np.array([1, 4])\n", "xlog = 10.0 * np.random.random(100)\n", "ylog = log[0] + log[1] * np.log(xlog) + np.random.randn(100)\n", "ax.plot(xlog, ylog, 'ro', markersize=4);" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "Clearly, none of these datasets agrees perfectly with the proposed model. So the question arises:\n", "\n", "How do we find the __best__ linear function (or quadratic function, or logarithmic function) given the data?" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "__Framework.__\n", "\n", "This problem has been studied extensively in the field of statistics. Certain terminology is used:\n", "\n", "* Some values are referred to as \"independent,\" and\n", "* Some values are referred to as \"dependent.\"" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "The basic regression task is: \n", "* given a set of independent variables \n", "* and the associated dependent variables, \n", "* estimate the parameters of a model (such as a line, parabola, etc) that describes how the dependent variables are related to the independent variables." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "The independent variables are collected into a matrix $X,$ which is called the __design matrix.__\n", "\n", "The dependent variables are collected into an __observation__ vector $\\mathbf{y}.$\n", "\n", "The parameters of the model (for any kind of model) are collected into a __parameter__ vector $\\mathbf{\\beta}.$" ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "hide_input": true, "slideshow": { "slide_type": "skip" }, "tags": [ "hide-input" ] }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZcAAAGKCAYAAADALgxIAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAuuUlEQVR4nO3deXzU9Z3H8fcviUHxRg45FAxICOGIEMX1AA+CQItlxWsfWqGgUK2tLh4Pu2t3t1sRa6vCqsXNo4rYrge2uhBRBBQVXSwGxQMkIhHlLAmFRXAlJvnuH8MkmfvIb37HzOv5ePgI85vrM5Nx3vmeP8sYIwAA7JTndgEAgOxDuAAAbEe4AABsR7gAAGxHuAAAbFeQwm2ZVoacNnbsWC1dutTtMgAvsWJdQcsFSFJ9fb3bJQC+QbgAAGxHuAAAbEe4AABsR7gAAGxHuAAAbEe4AABsR7gAAGxHuAAAbEe4AABsR7gAAGxHuAAAbEe4AABsR7gAAGxHuAAAbEe4AEAm1dZKpaVSQUHgZ22t2xU5gnABgEyaMEHauFFqagr8nDDB7YocQbgAQCbV1EjNzYF/NzcHLucAwgUAMqm4WMo7/FWblxe4nAMIFwDIpKoqacAAKT8/8LOqyu2KHFHgdgEAkNWKiqT1692uwnG0XAAAtiNcAAC2I1wAALYjXAAAtiNcAAC2I1wAALYjXAAAtiNcAAC2I1wAALYjXAAAtiNckNP27dunyy+/XAMGDFBJSYlWr17tdklAVmBvMeS0W265RWPHjtWf/vQnNTQ06JtvvnG7JCAr0HJBztq/f7/eeustTZs2TZJUWFioE044wd2i4K4cPWtkJhAuyFm1tbXq0qWLfvSjH+mMM87Q9ddfr4MHD4bcprKyUuXl5SovL1ddXZ1LlcIxOXDWyAMHpJ//XNq5M7PPQ7ggZzU2Nur999/XjTfeqA8++EBHH3207rvvvpDbTJ8+XdXV1aqurlaXLl1cqhSOyfKzRr78snTssdJ990n/8z+ZfS7CBTmrV69e6tWrl0aMGCFJuvzyy/X++++7XBVc5aWzRtrcRVdRIX3ve4F//+hH0qRJNtQYB+GCnHXyySfrlFNOUc3hv05fe+01DRw40OWq4CovnTXSpi667dsly5JWrAhcXr1aeuIJG+uMwTLGJHvbpG8I+MW6det0/fXXq6GhQUVFRZo/f75OPPHEqLctLy9XdXW1wxUiZxUUBIIlKD9famxM6SGGDpU++ijw7yOOCIy3FBbaWKNkxbqCqcjIaWVlZQQGvKm4ONBiaW5OuYuuoUHq0KH18i9/Kf3Lv2SgxjjoFgMAL0qzi+6pp0KDZcUK54NFouUCAN5UVCStX5/SXaywTqrm5shjqq0NjN/U1ARaQ1VVgeeyGS0XAHCDjbPBNm8ODZErr5SMiRIskmNreRjQB5LEgD5sVVoaOqYyYEDKLRVJ6t9f2rSp9XJ1tTR8eJw72DBRoA0G9AHAU2xYsBneMkmqrdCOiQKpoFsMANzQjgWbixaFBssVVyQZLJJja3louQCAG6qqIgfWkxDeWtmzR+rUKYXnTWOiQDoIFwBwQ4pf8gcPSsccE3os+SFz59EtBgAeN3ZsaLD85396O1gkWi4A4Gnh3WCNjYHhEq+j5QIAHrRiRfTZYH4IFolwAQDPsazAFvlBzz0nmc3+Oksm3WIA4CEx166UTmhdnxJcWe/ArK900XIBAA+4/voEiyJ9dpZMWi4A4LLwUNm0SerXL+xGDq2stwstFwC2n1IXydmzJ3prJSJYJG+dJTMJbFwJJCmrN660aRNFJC88VHr1krZudaeWdmDjSgBx+Kw/3+/Cg+W77wKNxmxCtxiAdm2iiOQ9/XT0brBsCxaJcAEg+a4/348sS7rmmtbLd9xh8xYuHhs3Y8wFSFJWj7kgo9I670qq3Bk3iznmQssFADKkZ0+HgkXy3LgZ4QLAP9zo+knzOS1L2rGj9fK772Z4J2OPjZsRLgD8Y8LhLVCamlq3QPHYc+7YEb21MmJEBmuUPDduxpgLkCTGXDygoCDwJR+Unx/Yg94jzxkeKlISrZXa2sgzUhYVpV+vsxhzAZAF3Oj6ifacUbrKwoPl0KEku8Ey1RpzefYY4QLAP5zq+mn7xdzQEGhJtH3ONoEw99MKWX1DWxrGSIWFST5Xpgbi3ehCbINuMSBJdIvlkETTeg93lVlhX4sTJ0ovvmjzc6XLmS5EusUAIGmJWhPFxRHBYkwawSJlrjXm8uwxwgUAwsX5YrYsydoQ2rIwm9sxnlFUFGipNDYGfto1mO/y7DHCBQDCxfhiDh+0X7bs8KC9F2d3ZSq0kpSF26UBQDsFv5gP27ZNOsWplfZZgnABgDjSWrsCusUAIJbwYPn2W4IlWYQLAIS57bboW7h06OBOPX5EuABAG5YlPfhg6+WSEpdbKx47T0uyCBcAOCxaa2XDBndqaeHySvt0ES4AclObFoFlOXjelVR57DwtySJcAOSWYKj07Stt2CCrKXRLlKef9lCwSK6vtE8XU5EB5I7a2sAgSkODNqmf+mtTyNWeCpWg4EaZbbfk9wHCBUDumDBBamiI2BdMkszAUkkZP+d86sIWdPoF4QIgd9TURATLQXVUx4Gn+aZF4BeEC4CccNVV0sKw8RVT2EH69FNv7g3mc4QLgKwXdQuXgaVSFcGSKYQLclpTU5PKy8vVs2dPvfTSS26XgwyIPcXYf+MYfsJUZOS0uXPnqqSkxO0ykAGeXruSAwgX5Kxt27ZpyZIluv76690uBTYLD5V58wgWp9Ethpx166236v7779fXX38d8zaVlZWqrKyUJNXV1TlVGtK0cWNgGUtbhIo7aLkgJ7300kvq2rWrhg8fHvd206dPV3V1taqrq9WlSxeHqkM6LItg8RLCBTnpnXfe0eLFi9WnTx9dffXVev3113Xttde6XZb3xduh18Xde8O7wfbvJ1jcZpnkfwP8qpCV3njjDf32t79NOFusvLxc1dXVDlXlUaWlgb6n5ubAPlcDBrSuHo93XTJqayO3OUkwTXjiRGnRotBjhIqjokzyDqDlAiB58XboDb9uw4bUWjEpbi1vWQSLl9FyAZJEy0XJt1zaSrYVU1AQCJag/HypsTHqTZli7Bm0XADYoKoqEBT5+YGfbffjantdW21bOPHGZZLYWt6xtSvtGT/y6Zkj7UbLBUgSLZcUxGrhxGv5JBhzCQ+VBx6QZs50uP5M39d/YrZcCBcgSYRLCmIFRQpdX0FprV1JY3JAiDTqtOW+/kO3GAAHBc9B0tgY+Bn8Yk/xrIppr11p73nn23P2R5+eOdJuhAsA58QbswkT3g22b18K4yvtPe98CnXaet8sQrcYkCS6xZwxfrz0yiuhx1IetM+tcQ830S2WaYcOHdLs2bM1ePBgHXXUUTrqqKNUVlamxx9/3O3SEvJz7cgCbWZXWZYNwSLRevAAWi42aGhoUEVFhd566y2VlZXpggsu0P/93//pmWee0f79+7V06VJdcsklbpcZlZ9rdxotlww53MqwmptCDrd8NbV3cB6ZFLPlImNMsv8hhvvvv99IMjNmzDDNzc0tx//rv/7LSDJ33323i9XF5+fanTZ8+HC3S8hKgRgJ/S/EwIHG5OUFrsjLC1yGV8TMDLrFbPDYY4+pY8eOeuCBB2S1GYUsKAic0eCkk07K2HM/+eSTsixLb7zxRlr3d7N2ZLEkFxKGD9r/yvpF4PTDbbV3cB6u8FW4BL8AH3jggajX19TUqEOHDho5cqRjNX355Zeqra3VxRdfrKOPPjrkuoULF0qSLrroIsfqSUWmah8zZowsy9ILL7wQctwYoylTpsiyLN11113pFw7vSzAVeOPGKCvt8wt0d8kLkeMjTO31JV+Fy3nnnSdJevfdd6Ne/9Of/lRNTU165JFHHKsp2Ac/YsSIlmPGGM2dO1d//vOfNXr0aA0ZMsSxelKRqdp/85vfKC8vT3fffbea2iwmu/3227VgwQLdcMMNuu+++9r/AhCfm9uQxGltRF27srk2ck1MEIPzvuSrM1EOGzZMRx11lP7yl79EXPf8889r+fLl+tnPfhb3C3HOnDnat29f0s9ZVlamiRMnxrx+7dq1kqThw4dr5cqVevrpp7Vq1SrV1NRo6NCh+uMf/5j0czktU7UPHTpUP/zhD7VgwQL94Q9/0JQpU3TvvffqwQcf1JVXXqnHHnvMzpeBWIKth+bm1taDU9Nxi4tDpwIfbm2Et1b+phN1Yt5+aUKcqcLBBZnwl3gDMsaDA/ojR440ksz27dtbjh04cMD06tXLdO3a1ezbty/u/Xv37m0UmPmW1H+TJ0+O+3hjxowxkkxdXZ256qqrQu571VVXhdSZCfPnzzeSzMqVK1O+byZr37p1qznyyCNN7969zcMPP2wkmUsuucQcOnQo7cd0m+8G9PPzQ0fJ8/Ode+7NmwMD7/n5xgwcaIaWfBs5aN/e2sKew2zebPvLQEIxM8N34fLzn//cSDJ//vOfW47deeedRpKZP3++4/V07tzZ9O7d2xhjTGNjo6mvrzdvvPGGmTRpkpFkBg8eHHGfRx991PTp08d06NDBDBs2zLz11ltJPZfdwZhq7W+++aaZMGGC6dGjR1Lv91133dVSyznnnGMOHjyY1Ov0Kt+Fi0dmWUWdDWZHbR55fTkuZmb4qltMks4991xJ0l/+8hdddtll2rhxox566CH93d/9nSZPnuxoLV9++aXq6+s1atQoSVJ+fr5OOukkjRo1SqNGjVJZWZk+/PBD1dbWquhwP/Jzzz2nW265Rb/73e903nnn6Xe/+53GjRunDRs26NRTT437fLfeemtEl966deu0aNEiTZ48WX369Am5rqyszNbaDxw4oEGDBum6667Tddddl/D9aXvO+ccff1wdO3ZMeB/YqKoqcn2Iw2Juj19rQ23MIvM034XLOeecI8uyWgb1b775ZjU1NenRRx8NmUobi51jLsExi/Ly8qjXn3jiiZKkY489tuXYgw8+qClTpuiGG26QJD388MNaunSp5s2bp9mzZ8et5dZbb4049uSTT2rRokWaMmWKLrjgggSvpn21jx8/XuPHj5ckTZkyJe7jP/PMM7r99tt18skna9euXZo7d67mzZuXdH1oh/BFh5995viiw2j/K4as17ZjHCXGuA48Il6zxniwW8wYYwYOHGg6duzYstDvxhtvTPq+dnYt/dM//ZORZJYtWxZx3Z49e8wRRxwR0rV06NAhk5+fbxYuXBhy25tuusmMHDky6dfQVrpjLqnWHu7oo4+O2S22ZMkSc8QRR5hBgwaZ3bt3mwEDBpiCggLz6aefplSj1/imW8zl7qLwLrCbbsrQEzHm4gXZtYjyvPPO0zfffKMZM2aoc+fOuueee5K+75YtW1IJVD355JMxHyv41/9zzz0XGMA6rKGhQTNmzNB3332n2267reV4fX29mpqa1K1bt5DH6datm3bt2pX0a7BDqrUn6+2339bll1+uXr16admyZerSpYt+9atfqbGxkbUtTnGpuyjq2hUjPfpohp4w1rb+8ATfdYtJgXGXyspKHThwQA899JA6derkSh3BL+jHH39c69at00UXXaSvv/5ay5YtU21traZMmRJ1HCi8+84Yk1SXnp3SrT2eDz/8UN///vd1/PHHa/ny5erevbsk6fLLL1d5ebkWLVqkVatW6fzzz7f99aANF7qLEnaDIef4suVy2mmnSZLOPPNMTZs2zZUavvrqK9XX16uiokJXXHGFtmzZogcffFALFy5UUVGRFi5cqPnz54fcp3PnzsrPz49opezevTuiNeO12hP5/PPPdckll8iyLL366qvq27dvyPXB8aQ77rjDtteBGBxedBgeLPX1BAt82nIJrgBPdhA/E4J/+VdUVCT9hVlYWKjhw4dr+fLluuKKK1qOL1++XJMmTUqrjilTpiQcXA+XTu2J9OvXL27X3ujRo0O635BBDi06PP986e23Q4/xK0aQ78Ll6aefVlVVlX7yk5/ozDPPdK2O4Bf0sGHDUrrfzJkz9cMf/lBnnXWWzj33XD322GPasWOHfvzjH2eizKjSrf3AgQP6/PPPJUnNzc366quvtG7dOnXq1CnhNGpkF7rBkFAKg9uu+fLLL83s2bPN9ddfbwoLC01paanrC/LGjh1rJJk9e/akfN9HH33U9O7d2xQWFpphw4aZN998MwMVxpZu7StXrkxrsWa28M1ssQyLuz0+ck3MzPDFycIqKys1Y8YMnXDCCaqoqNCcOXPUo0cPt8qRJHXt2lUdO3bUli1bXK0jHX6u3U25frIwWiuIwt+nOZ4+fbqMMdq7d68WLlzoerBIgUF4v345+7n2nOPmzsZthAfLzTe3I1g88pqQWb5ouQBe4ErL5fApgFumFQ+Is3twBtTUBJ6yrXa3Vlx+TbCVv1suQM6yc0Fkii0Gy8pAsEjsCZYjCBfAy+w8C2OCs0O2Fd4NVldn4/gKZ5bMCYQL4GV2LohMosVwwQXRt3Dp3Dn9p43AmSVzgu/WuQA5xc4FkQm2hXFsNhhnlswJtFyAXBGnxRCttcI0Y7QHLRcgV0RpMbB2BZlCywXIUeHBMnWqQ8HCOpecQMsFyDEZWbuSiuCstebm1llrjMFkHcIFyCGe6AZjnUtOoFsMyBHhwbJrl0vjK6xzyQmEC5DlLrww+mwwB89PF4p1LjmBbjEgi3miGywc61xyAi0XIEtFtFby8mUGlrpTDHIO4QJkGcuKEiyyEg+eM0UYNiJcgCwSHirTTviTTF5+4EKiwfMUNrYEEiFcgCywaVP0Qfvfrx2W/OA5U4RhI8IFOWvr1q268MILVVJSotLSUs2dO9ftktJiWVL//qHHWgbtg4PnjY2Bn0VFsR+IKcKwEeGCnFVQUKAHHnhAn376qd599109+uij2rBhg9tlpSS8tbJzZztmgzFFGDZiKjJyVvfu3dW9e3dJ0rHHHquSkhJt375dAwcOdLmyxMaMkZYvDz3W7inGTBGGjQgXQNKWLVv0wQcfaMSIESHHKysrVVlZKUmqq6tzo7QInly7AoSxTPKfSj6+yEoHDhzQqFGj9M///M+67LLLYt6uvLxc1dXVDlYWKdqgPeCiKH/qBNByQU777rvvNGnSJF1zzTVxg8VttFbgNwzoI2cZYzRt2jSVlJRo5syZbpcTU3iwTJ58OFhY9AgPo1sMOevtt9/W+eefr8GDByvv8BTce++9V+PHj496e6e7xTZtijPFWAoESvC8KHl5gRleDMjDWTG7xQiXbFZbG1hlXVMTWLNQVRV/nQPicjJckuoGKygIrKYPys8PrGdxAp8tBMQMF7rFshnbefhSeLBs2xZjfMXNRY98tpAA4ZLN2M7Dee0YB7ngguizwXr2jHEHNxc98tlCAswWy2bFxaF98mznkXlpnh8+rdlgbi565LOFBGi5ZDO283BeGn/RR2utZHyacXtnmvHZQgIM6ANJSmpAP4UZXK6uXWGmGezBgD7giCT/og8PlkmTHF4UyZgJMowxF8BOCcZBamulvn1Dj7my0p4xE2QYLRfAIZblkWCRGDNBxtFyARwQ3g321VfSKae4U4skttdHxtFyATLo/POjzwZzNVgAB9ByATKEnYyRywgXIAM47wpyHeEC2IjWChDAmAtgk/BgGTOGYEHuouUCtJNn1q4AHkK4AO1ANxgQHd1i8B+PnN43PFi2bCFYgCDCBf7j8omqYp13pXdvR8sAPI1wQSiPtArisnvTxRRes2VJb74ZeozWChCJLfcRyg9bsdtdY5KPZ1nlklq33CdUALbcR7L8sBW73ZsuRnvNbVoylpVgUaQfWnuAw2i5IJQfWi52C3/NBQVSY6PU3Cwr5GNfrgsvrNbrrye4fy68Z0AALRckyYmt2L32l374a25s1JbmU8KCRRo+XJHBIvmjtQc4jHBBqOBW7I2NgZ9FRbFvm25IuDzbK0LYa7aam3SatoTcJG4Dv7g40GKR0j/xltcCF2gnwgXpSzckPPyXfvjYyhf9KmQ2J/iit6O157XABdqJMRekr6Ag8GUYlJ8f+Os/EQ+OUZxzjrR6deix8P81ysvLVV1drYxI970E3MWYCzIg3e4gj51i17ISB0vG2dG1BngI4YL0pRISbccUJkwI3DaZcZ0MizbF2JX1Kx4LXKC96BaDM5zoCqutDQRXTU3gL/+qqpjBlc6GkxntFgP8iW4xuMyJQfwkB8XDg2XQIFbbA3YjXOAMJ8YUEgRYbW30brCPP7a/FCDXES5wRrrjM6ms+YgTYJbFCb0AJxEucEYqizPTXfMRI8DCWyuffUawAJnGmSjhPemOzwQD7LARI6Q1a0JvQqgAziBc4D3FxaEzy9IYn+H0w4C76BaD97RzzYdn1q4AOYxwgfvCB/Cl5Mdn2kh43hUAjiFc4D4bNm0MD5WSkgTBwi7EQEYRLnBfOxZYbtkSvbWyYUOCO7ILMZBRhAvcl+YCS8uSTjst9FjS3WBOb/tPSwk5hnCB+9IYwA9vrWzcmOL4itO7ENNSQo4hXOC+FBZYDh8evRss5WxwehdiD58gDcgE1rnAN2xduxK24DLjbFi7A/gJLRf4gu/XrnC+FuQYwgWelum1K0uXLlVxcbH69eun++67z74HDpfK3mpAFiBc4FnhodK5s73B0tTUpJ/85Cd65ZVXtGHDBj3zzDPakHAOM4BkEC7ZzKfTX7/4Inprpa7O3udZs2aN+vXrp6KiIhUWFurqq6/WokWL7H0SIEclfZrjsWPHmvr6+pQevK6uTl26dEmnLtdkVc3r10vfftt6+cgjW7dXcVmsmteujbzt8OGZqWHv3r3av3+/evfuLUnas2ePDh48qFNPPTWkzuDn/tChQyorK8tMMRmUVZ9pD8vFmteuXfuqMWZs1CuNMcn+l7Lhw4enczdXZVXN+fnBce/Af/n5zhYWR7Sa25YqGfPRR5mtYeHChWbatGktl5966ilz8803x7x9x44dM1tQhmTVZ9rDcrTmmJlBt1g2c3qhYJrOOCN6N9jgwZl93l69emnr1q0tl7dt26YePXpk9kmBHEG4ZDOvT3+trZVlSevWhR52aorxmWeeqU2bNumLL75QQ0ODnn32WV166aXOPDmQ5TK6iHL69OmZfPiMyKqanV4omILp06fL6hs6HdcMLHW03oKCAj3yyCO65JJL1NTUpKlTp6o0zphU586dHavNTln1mfYwag6V9IC+JD8tWYOHRV1pLyvQwmpsdL6gJJWXl6u6utrtMgAvifJ/cwDdYl7h02nDqQoPluO1LxAsHh4TApA6wsUrsnzX3C+/jDJov7lW+wae690xIQBpa1e4PP/88yotLVVeXl5Ed8Hs2bPVr18/FRcX69VXX416/7/97W+qqKjQ6aefroqKCu3du7c95aTlqquuUllZmcrKytSnT5+Y6xj69OmjwYMHq6ysTOXl5fYXksKuuf/2b/+mnj17ttT98ssvR72dY1ubJGBZUp8+ocf27t0XdUuUjL/PSUj0vhlj9LOf/Uz9+vXTkCFD9P7777tQZautW7fqwgsvVElJiUpLSzV37tyI27zxxhs6/vjjWz4z//7v/+5CpZES/b699l7X1NS0vIdlZWU67rjjNGfOnJDbeOG9njp1qrp27apBgwa1HEv2+9a2741485RNgnUuGzZsMBs3bjSjRo0y7733Xsvx9evXmyFDhphvv/3W1NbWmqKiItPY2Bhx/zvuuMPMnj3bGGPM7NmzzZ133tneOdftMnPmTPPLX/4y6nW9e/c2dXV1mXvygQONycsLLPDIywtcjuFf//VfzW9+85u4D9fY2GiKiorM5s2bzaFDh8yQIUPM+vXr7a46ofC1K+vWGXPnnXfG/F1n/H1OIN77FlwTsGTJEjN27FjT3NxsVq9ebc466yzX6jXGmB07dpi1a9caY4zZv3+/Of300yN+1ytXrjTf+9733CgvrkS/b6+91201Njaabt26mS1btoQc98J7/eabb5q1a9ea0tLSlmPJfN+m8b2RmXUuJSUlKo7ST75o0SJdffXV6tChg0477TT169dPa9asiXq7yZMnS5ImT56s//7v/25POe1ijNHChQv1D//wD+4UYPO0Ybe3Nikvj752ZehQ6eyzz9a2bdscqyUVybxvixYt0nXXXSfLsnT22Wdr37592rlzp0sVS927d9ewYcMkSccee6xKSkq0fft21+qxk9fe67Zee+019e3bt2WHBy8ZOXKkOnXqFHIsme9bO783MjLmsn37dp1yyiktl3v16hX1w/7Xv/5V3bt3lxT4H2T37t2ZKCcpq1atUrdu3XT66adHvd6yLI0ZM0bDhw9XZWWl/QWkuGvuI488oiFDhmjq1KlRm7fJ/g4ywbIit3FpOynxiSee0Lhx42LcN8PvcwLJvG9uvreJbNmyRR988IFGjBgRcd3q1as1dOhQjRs3Tus9MkU90e/by+/1s88+G/OPUS++18l839r5fidc52JZ1gpJJ4fP/581a5Z+8IMfRL2PiTK92Yo2/9Qho0eP1q5duyKOt30NzzzzTNxWyzvvvKMePXpo9+7dqqio0IABAzRy5EhXar7xxhv1i1/8QpZl6Re/+IVuu+02PfHEEyG3c+N3MHr0aL322oqQY6WlgzRr1ixJgfd51qxZKigo0DXXXBP1MZx+n8Ml87557fMddODAAU2aNElz5szRcccdF3LdsGHD9OWXX+qYY47Ryy+/rIkTJ2rTpk0uVdoq0e/bq+91Q0ODFi9erNmzZ0dc59X3Ohl2vt8Jw8UYMzr4z2QfNNltNbp166adO3eqe/fu2rlzp7p27ZrsU6RkxYoV0a84PP3X1NToVknHvflmzMcI1t+1a1f9/d//vdasWZPRL72YNYe54YYb9P3vfz/iuNNbmwQ+f6E1Bz6nn7RcXrBggV566SW99tprMT+wTr/P4ZJ537y4bcx3332nSZMm6ZprrtFll10WcX3bsBk/frxuuukm1dfXu74wNNHv24vvtSS98sorGjZsmLp16xZxnVff62S+b+18vzPSLXbppZfq2Wef1aFDh/TFF19o06ZNOuuss6LebsGCBZICXzyxWkIZc3j6r9XUpH5NTeoeY7XqwYMH9fXXX7f8e9myZSGzMJzWts/5xRdfjFqLk1ubJHP64aVLl+rXv/61Fi9erI4dO0Z9HC+8z8m8b5deeqmeeuopGWP07rvv6vjjj2/pbnCDMUbTpk1TSUmJZs6cGfU2u3btavmrdM2aNWpubtZJJ53kZJkRkvl9e+29DorX0+HF91pK7vvW1u+NeKP9Yf9FeOGFF0zPnj1NYWGh6dq1qxkzZkzLdffcc48pKioy/fv3Ny+//HLL8WnTprXMLKuvrzcXXXSR6devn7nooovMnj174s1KsF+cXYO3b99uxo0bZ4wxZvPmzWbIkCFmyJAhZuDAgeaee+5xts4w1157rRk0aJAZPHiwmTBhgtmxY4cxJrRms3mz2X/qqeY7ydQccYR5ZOZM2+v46qvI2WCx9O3b1/Tq1csMHTrUDB061MyYMSOiZq+8z0uWLDGnn366KSoqaqlh3rx55tRTTzXGGNPc3GxuuukmU1RUZAYNGhQyU9INq1atMpLM4MGDW97fJUuWmHnz5pl58+YZY4x5+OGHzcCBA82QIUPMiBEjzDvvvONqzcbE/n23rdtr77Uxxhw8eNB06tTJ7Nu3r+WY197rq6++2px88smmoKDA9OzZ0/z+97+P+X0b8r1hon/+44iZGbm9/UtpaWDBYnNzYIX4gAGp721VWxtoAdXUBFaYV1UFBuNjHXeKHa8tjmRaK9mG7V+ACGz/EpUd039jrax3e8V9CosyUxUeLB984FKw5MiWOYAf5XbLxQ4FBYEACQpuvhjruFMy0HIZMUIKX67kamslw62zcLRcgAi0XDIm1gm53D5Rl82LMi3LY8EiZbR1BqB9CJf2ivUl7vaJulJclBlPtJX2rgeL5H6AA4iJbjHE5PlBe4cnTdAtBkSI2S2W0TNRwr/Cg8WT5/Hy8Jk2gVxHtxhC7NwZvRvMc8ECwNNouaCF57vBAPgGLZds0o51H+HB8uGHBAuA9BEufhUtSNJYuHnxxdG7wYYMyVDdAHIC3WJ+FQyS5ubWIElx3QfdYAAyhZaLX0ULkhTWfXh27QqArEC4+FW0IEli4aZlRQ8WALAT4eJX0YIkwar88FA56iiCBUBmEC5ekspsrxS2d/nrX6O3Vr75xsZ6vCxbXgfgI2z/4gXBmV4bNrQes2mX33YN2ju863DG2PQ62P4FiMCuyJ4WnPnVlg27/IYHy7p1KXaDZcuuw9nyOgAfIVzSZWdXS9svv6B27PI7dWr0brChQ1N8oGzZdThbXgfgI4RLuuw802TbL7+gNLfptyxp/vzQY2kP2rt92gC7ZMvrAHyEMZd02XmmSZu2jmeKcWYx5gJEYMzFdnZ2tbTzxF6ur11hNhaAMIRLujzS1RIeKsXFLrRY7OwijIUAA3yFbjGfqq+XunQJPeZaN5idXYSxeGBaNN1iQAS6xbKJZXkoWCRnZmMxnRjwFcJF8lWXS3g32EcfeWDg3okuQqYTA75CuEjOjBm0009/Gn3QfvBgd+oJ0c4JCUnxyBgXgORwPhfJ810unHdFrQEGwBdouUie7nLhvCsA/IhwkTzZ5ZLRtSvRxph8NO4EwPuYiuxB4aFSWip98omNTxBtWq/k+lRfr2MqMhAh5lRkxlw8ZO9eqVOn0GMZ6QKLNcbk4XEnAP5Ct5id2tG1ZFkOBYsUfYzJw+NOAPyHcLFTmlOaw7vBPv44A8HSNvgaGgKzr9qOMXlw3AmAf9EtZqcUpzQ/8IB0++2hxzLWWgkGX3NzIGgGDIjcoiWHxljuuOMOVVVVqbCwUH379tX8+fN1wgknuF0WkDVoudgpha4ly3IwWCTPr+VxWkVFhT755BN99NFH6t+/v2bPnu12SUBWIVzslGTXkitrVxhTCTFmzBgVFAQa7meffba2bdvmckVAdiFc7BAcz+jfP3D5s8+iboPi6nlXGFOJ6YknntC4ceOiXldZWany8nKVl5errq7O4coA/2Kdix2S2A4+PFQmTpRefNG5EnPR6NGjtWvXrojjs2bN0g9+8IOWf1dXV+uFF16QFW2fnTZY5wJEYJ1LRsUZzzh4UDrmmNCbs32LM1asWBH3+gULFuill17Sa6+9ljBYAKSGbjE7xBjPsCyCxauWLl2qX//611q8eLE6duzodjlA1iFc7BBlPCP8D+HNmwkWL7n55pv19ddfq6KiQmVlZfrxj3/sdklAVqFbzA5ttoNfuFC6qm/o1Z4JldrawHqXmppA66qqKjPnXvGBzz//3O0SgKxGuNjI8+ddabuQMriDQA4tnATgHMLFJq5NMU4FCykBOIQxl3YaP94nwSKxkBKAYwiXdrAs6ZVXWi/Pnh0nWLxwMi4WUgJwCIso03DokHTkkaHHEr6NSSy0hLexiBKIwCJKu6Q9aM94B4AcQrdYCsKDZceOFMZXGO8AkEMIlySsXBl90L579xQeJFfGOzI9tuSFsSsACTHmkkB4qHToIH37rTu1+EKmx5ZcHLtizAWIwJhLOnwzxdhLMj22xNgV4At0i0UxYQLBkrZMjy0xdgX4gnPh4pO+csuSXnqp9fJ//AfBkpJMjy3lytgV4HPOjbl4fJ1HWmtXkFMYcwEieGDMxcN95Z7fcBIAfMa5brFM9pW3o8stPFi2bydYAKC9nAuXTPaVB7eSb2pq3Uo+gU8+iT5o36OHfWVF8Mm4EwC0V3ascykoCARLUH6+1NgY8+bhoXLxxVKC063bw+PjToiPMRcgQswxl+yYipxCl1u01kpGg6Vta2XDBs+OOwGAnbIjXJLocrv3XpfWrrTtsmuLNRoAslh2rNBvcw77aMJDZfHipIZl7NF2llxQfn7rOewBIAtlR7jE0NgoHXFE6DHHZ4IVFzPOAiDnZEe3WBRnnOGBYJFYUQ4gJ2VlyyW8G2zvXumEE1wpJWGXHQBko6xquWzaFH3Q3rVgAYAclTXhYllS//6tl2fMSLIbzI6FjSyOBIAQWREu0Vorjz2W5J3TWN1vy2MQSACymK/D5bnnbFi7YseGmuk8hh2hBgAe5dtwsSzp6qtbL69eneZsMDs21EznMTy8SzQAtJfvwqWpKXpr5eyz03xAO6YKp/MYnFERQBbzVbjcfntgiCKoe3cb1q4Epwo3NgZ+FhU58xisfwGQxXyzziW8tfK//ysdd5w7tdiC9S8AspjnWy67dkXvBvN1sABAlvN0uNxzT6DrK+ixxzhLJAD4gWe7xVzZHh8AYAvPtVxqa0OD5dZbCRYA8BtPtVyuvFJ6/vnWy7t2Sd26uVcPACA9nggXY1qXfLQ9BgDwJ9e7xVasCA2WP/yBYAEAv3M1XE48UaqoaL186JB07bVx7uCXzR79UicAZIgr4XLgQGDQft++wOUzzgi0VgoLE9zRL5s9JlsnIQQgSzkeLg89JB17bOvl996T3n8/yTv7ZbPHZOv0S1gCQIocHdCvqAiMsQSlPLZSXBz4Em5u9vZmj8nW6ZewBIAUOdZyaWhoDZZbbklz0N4vmz0mWyc7IwPIUo6FS2FhYEjh4EFpzpw0H8SOHYydkGydfgnLLPbb3/5WlmWpvr7e7VKArOJot9hppzn5bD7Azsiu2rp1q5YvX65TTz3V7VKArOP6OhfALf/4j/+o+++/X1b4RnYA2s0TK/QBpy1evFg9e/bU0KFD496usrJSlZWVkqS6ujonSgOygmWSH1l3dt18bW1gam5NTWCgu6rKu2Ms8KTRo0dr165dEcdnzZqle++9V8uWLdPxxx+vPn36qLq6Wp07d477eOXl5aqurs5UuYAfxWz2ezdcSktDp/MOGMD4BGzx8ccf6+KLL1bHjh0lSdu2bVOPHj20Zs0anXzyyTHvR7gAEXwYLgUFgcWFQfn5gdlXgM1ouQBpixku3h3QZw0IAPiWd8OFNSBwyJYtWxK2WgCkxruzxVgDAgC+5d2Wi53YfRgAHJUb4cLuwwDgqNwIF3YfBgBH5Ua4MPMMAByVG+HCzDMAcJR3Z4vZiZlnAOCo3Gi5AAAcRbgAAGxHuAAAbEe4AABsR7gAAGzn7XBh2xYA8CVvhwvbtgCAL3k7XNi2BQB8ydvhwrYtAOBL3g4Xtm0BAF/y9vYvbNsCAL7k7ZYLAMCXCBcAgO2cCxfWrABAznAuXFizAgA5w7lwYc0KAOQM58KFNSsAkDOcCxfWrABAznBunQtrVgAgZzAVGQBgO8IFAGA7wgUAYDvCBQBgO8IFAGA7wgUAYDvCBQBgO8IFAGA7wgUAYDvCBQBgO8IFAGA7wgUAYDvCBQBgO8IFAGA7wgUAYDvCBQBgO8IFAGA7wgUAYDvCBTnr4YcfVnFxsUpLS3XnnXe6XQ6QVQrcLgBww8qVK7Vo0SJ99NFH6tChg3bv3u12SUBWoeWCnDRv3jzddddd6tChgySpa9euLlcEZBfCBTnps88+06pVqzRixAiNGjVK7733XtTbVVZWqry8XOXl5aqrq3O4SsC/6BZD1ho9erR27doVcXzWrFlqbGzU3r179e677+q9997TlVdeqdraWlmWFXLb6dOna/r06ZKk8vJyR+oGsgHhgqy1YsWKmNfNmzdPl112mSzL0llnnaW8vDzV19erS5cuDlYIZC+6xZCTJk6cqNdff11SoIusoaFBnTt3drkqIHvQckFOmjp1qqZOnapBgwapsLBQCxYsiOgSA5A+wgU5qbCwUH/84x/dLgPIWnSLAQBsR7gAAGxHuAAAbEe4AABsR7gAAGznz3CprZVKS6WCgsDP2lq3KwIAtOHPcJkwQdq4UWpqCvycMMHtigAAbfgzXGpqpObmwL+bmwOXAQCe4c9wKS6W8g6XnpcXuAwA8Ax/hktVlTRggJSfH/hZVeV2RQCANvy5/UtRkbR+vdtVAABi8GfLBQDgaYQLAMB2hAsAwHaECwDAdoQLAMB2hAsAwHaECwDAdoQLAMB2hAsAwHaECwDAdoQLAMB2hAsAwHaECwDAdpYxxu0aAF+wLGupMWas23UAfkC4AABsR7cYAMB2hAsAwHaECwDAdoQLAMB2hAsAwHb/Dzm3hD5gGKYwAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "ax = plt.figure(figsize = (7, 7)).add_subplot()\n", "centerAxes(ax)\n", "line = np.array([1, 0.5])\n", "xlin = -10.0 + 20.0 * np.random.random(100)\n", "ylin = line[0] + (line[1] * xlin) + np.random.randn(100)\n", "ax.plot(xlin, ylin, 'ro', markersize = 4)\n", "ax.plot(xlin, line[0] + line[1] * xlin, 'b-')\n", "plt.text(-9, 3, r'$y = \\beta_0 + \\beta_1x$', size=20);" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "## Least-Squares Lines" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "The first kind of model we'll study is a linear equation, $y = \\beta_0 + \\beta_1 x.$\n", "\n", "Experimental data often produce points $(x_1, y_1), \\dots, (x_n,y_n)$ that seem to lie close to a line. \n", "\n", "We want to determine the parameters $\\beta_0, \\beta_1$ that define a line that is as \"close\" to the points as possible." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "Suppose we have a line $y = \\beta_0 + \\beta_1 x$. For each data point $(x_j, y_j),$ there is a point $(x_j, \\beta_0 + \\beta_1 x_j)$ that is the point on the line with the same $x$-coordinate." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "\"Figure\"" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Image from Lay, _Linear Algebra and its Applications,_ 4th edition" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "We call \n", "* $y_j$ the __observed__ value of $y$ and \n", "* $\\beta_0 + \\beta_1 x_j$ the __predicted__ $y$-value. \n", "\n", "The difference between an observed $y$-value and a predicted $y$-value is called a __residual__." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "There are several ways of measure how \"close\" the line is to the data. \n", "\n", "The usual choice is to sum the squares of the residuals. " ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "The __least-squares line__ is the line $y = \\beta_0 + \\beta_1x$ that minimizes the sum of squares of the residuals. \n", "\n", "The coefficients $\\beta_0, \\beta_1$ of the line are called __regression coefficients.__" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "__A least-squares problem.__\n", "\n", "If the data points were on the line, the parameters $\\beta_0$ and $\\beta_1$ would satisfy the equations\n", "\n", "$$\\beta_0 + \\beta_1 x_1 = y_1 $$\n", "$$\\beta_0 + \\beta_1 x_2 = y_2 $$\n", "$$\\beta_0 + \\beta_1 x_3 = y_3 $$\n", "$$ \\vdots$$\n", "$$\\beta_0 + \\beta_1 x_n = y_n $$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "We can write this system as \n", "\n", "$$X\\mathbf{\\beta} = \\mathbf{y}$$\n", "\n", "where \n", "\n", "$$X=\\begin{bmatrix}1&x_1\\\\1&x_2\\\\\\vdots&\\vdots\\\\1&x_n\\end{bmatrix},\\;\\;\\mathbf{\\beta} = \\begin{bmatrix}\\beta_0\\\\\\beta_1\\end{bmatrix},\\;\\;\\mathbf{y}=\\begin{bmatrix}y_1\\\\y_2\\\\\\vdots\\\\y_n\\end{bmatrix}$$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "Of course, if the data points don't actually lie exactly on a line, \n", "\n", "... then there are no parameters $\\beta_0, \\beta_1$ for which the predicted $y$-values in $X\\mathbf{\\beta}$ equal the observed $y$-values in $\\mathbf{y}$, \n", "\n", "... and $X\\mathbf{\\beta}=\\mathbf{y}$ has no solution." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "Now, since the data doesn't fall exactly on a line, we have decided to seek the $\\beta$ that minimizes the sum of squared residuals, ie,\n", "\n", "$$\\sum_i (\\beta_0 + \\beta_1 x_i - y_i)^2$$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "$$=\\Vert X\\beta -\\mathbf{y}\\Vert^2$$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "This is key: __the sum of squares of the residuals__ is __exactly__ the __square of the distance between the vectors $X\\mathbf{\\beta}$ and $\\mathbf{y}.$__" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "Computing the least-squares solution of $X\\beta = \\mathbf{y}$ is equivalent to finding the $\\mathbf{\\beta}$ that determines the least-squares line." ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "hide_input": true, "tags": [ "hide-input" ] }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZUAAAGKCAYAAADE29x1AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAuE0lEQVR4nO3de3RV5YH+8WcnERVFWpQ7CkaQXCBEEsCqC1sLCFasVludpRVGQdQ61mrrOG39TZ2qcbTtSFGxjDd0Wu9UiFoUrRe0WhpUWknJokQoVwErK4AVSPL+/jic5Nxyci7vPnvvc76ftbJg73N72Qn7yXt3jDECAMCGIq8LAADIH4QKAMAaQgUAYA2hAgCwhlABAFhT0s3jDA0DJE2dOlVLly71uhiAXzhdPUBNBUjBzp07vS4CEAiECgDAGkIFAGANoQIAsIZQAQBYQ6gAAKwhVAAA1hAqAABrCBUAgDWECgDAGkIFAGANoQIAsIZQAQBYQ6gAAKwhVAAA1hAqABBUzc1SZaVUUhL6s7nZ6xIRKgAQOOEwOeEEqbFRamuT1qyRpk/3umSECgAEzvTpoRCJ1N4uNTV5U54IhAoABE1TUyhEIhUVSSNHelOeyGJ4XQAAQJpGjgyFSKSyMqm+3pvyRCBUACBo6utDIVJcLFVUSOvWSatXS6WliZ+fww59xxiT7PGkDwKFora2Vg0NDV4XA8hMZWWoD6a9PVTDKSsLhVDmnK4eoKYCAPkusg/G5Q59QgUA8l1kH4zLHfqECgDku8g+GJc79AkVAIA1hAoA5LvwZMkczLwnVAAg39FRDwCwho56AIA1OeyoL3HtnQEA/lBamu1kx5RRUwEAWEOoAACsIVQAANYQKgAAawgVAIA1hAoAwBpCBQBgDaGCgrRr1y5dcMEFKisrU3l5ud555x2viwTkBUIFBem73/2upk6dqjVr1mjVqlUqLy/3ukiAO3K4lbDEdsIoQC0tLRozZoyam5vlOF3uihqF7YQRWBFbCRunSE551lsJS2wnDHRqbm5W37599a//+q866aSTNGvWLO3du9frYgHuaGrSB+2j5cioyLTJrHFvhWKJUEEBam1t1XvvvaerrrpK77//vo444gjdcccdcc9bsGCBamtrVVtbqx07dnhQUiB7xzkbdZI+6Dh2ytxboVgiVFCAhgwZoiFDhmjChAmSpAsuuEDvvfde3POuuOIKNTQ0qKGhQX379s11MYGsOY60sXVgx7GpqHR1hWKJUEEBGjBggI499lg1Hdyo6NVXX1VFRYXHpQLs2bMnFChhX/uaZIxCfSmlpa5+NkvfoyDNmzdPF198sfbv36/S0lI9/PDDXhcJsOLBB6VZszqP339fqq7O3ecTKihI1dXVjOZC3okdzNjeHn/ObYQKAOSB2PBIPlvEPfSpAECAbd0aHSjXXeddoEjUVAAgsMrLQ/MawzZskI47zrvySIQKAASSX5q7YtH8BQAB49dAkQgVAEhPjhdojLRoUXSglJb6K1Akmr8AID3Tp3cs0Kg1a0LH2S/Q2K3Y2slHH0nDhrn+sWkjVAAgHU1NoUCRQn82ubtAo+Tv5q5YNH8BQDpGjpSKDt46i4pCxy5pbQ1WoEiECgCkp75eKiuTiotDf7q0QONtt0mHHNJ5fPHF/g8UieYvAEhPaanrfSixtZOWFqlXL1c/0hpCBQB8JGjNXbFo/gIAH/jHP9IMFA+HNidDqACAx6qqpKOP7jz+4Q9TqKGEhza3tXUObfYBmr8AxGtuDt2kmppCo5vq613f3KlQxdZOWltDYwC65cHQ5lRQUwEQz6e/BVvho2ajRM1dKQWKlNOhzekgVADE8+lvwVb4IDA/+MBCh3yOhjani1ABEM+nvwVbkSgwc1h7cRzppJM6jx94IMMRXuGhza2tOdl7PlWECoB4Pv0t2IpEgZmj2kui2snll7vyUZ6hox5AvBxM8PNMfX38IIQTT3S9uS/o809SRU0FQGFJ1GzkYnPfgw8WTqBIhAoAuNbc5zjSrFmdx3/4Q34HikTzFwC40txXSLWTSNRUAMCi9vbCDRSJUAEAa2bPjp+8WEiBItH8BQBWxNZO1q+Xhg71pCieIlQAIEuF3NwVi+YvALnhozW3bGlpIVBiESoAcsMHa27ZNHSo1Lt35/G4cZYCJeDh65jkV6HAMxcIqa2tVUNDg9fFCLaSklCghBUXhyYgBlBs7eSzz6TDD7f05pWVodBtbw9NxCwr8+PqBk5XD1BTAZAbebJIZaLmLmuBIgV+hWhCBUBuBHWRyoPNUR8Wj8lN/0nAw5dQAZAbPl2qvVvTp8tpXK3R7as6Ts2Z42KHfFDD9yD6VIAU0KdSuGJrJ+1FJXLagtkXZBF9KgCQrrjmrqJiOWXBao7KNUIFAGIsXJggUIpLAtkclWvMqAeACLFh8qtfSVdcIUkF3+SVEmoqAHBQotFdoUDJQsAnM6aLUAEAubjcSp6tJNAdQgVAQZs61eX1uwI+mTFdhAqAguU40ksvdR6//bYL808CPpkxXYQKgIKUqHZyyikufNC994b6U6TQn/fe68KH+AehAiA/ddFB/tlnOV6u/jvf6Vw4s7U1dJzHCBUA+SlBB7njSEccEf001/c/oU8FAPJAzM3caYxePn7LlhxtqEWfCgDkgYibuROzjKEx0sCBOSpHwBeITBehAiA/1ddrzfHTEgZKTgV1deYMsUwLgLzknFAq6fnOY6ezNQzuoaaCgtXW1qaTTjpJZ599ttdFgWWxo7sOHPBxoOTZMi6ECgrW3LlzVV5e7nUxYFmi4cIlfm6TybNlXAgVFKRNmzbphRde0KxZs7wuCiz59a9zPP/EljwbckyooCBdd911uvPOO1VU1PV/gQULFqi2tla1tbXasWNHDkvnkQA3wziOdMklnceXXBKQQJHybsgxoYKC8/zzz6tfv36qqalJ+rwrrrhCDQ0NamhoUN++fXNUOg8FtBkmUe3ksce8KUtG8mzIsZ9bGgFXvP3221qyZIlefPFFff7552ppadEll1yi//u///O6aN4KYDNMIJu7YoWHHOcJaiooOHV1ddq0aZPWr1+vJ554QmeccQaBIgWqGebSS/MkUPIQoQIgJCDNMI4T3bz16KMEip84Jvl3g28VIKm2tlYNDQ1eF8N/mptDfS9NTaGaTX29qzPGqZ34htPVA9RUAGQuR537ra0ESlDQUQ8gczno3C8ujp8NT6D4FzUVAJlzuXM/dr2u1asJFL8jVABkzsXO/UTNXRUV1t4eLqH5C0DmXJhjsXWrNGhQ9DlqJ8FBqADwjdjaiUSgBA3NXwB8ITZQ9uwhUIKImgoAzzFcOH9QU7Fg3759qqur0+jRo3X44Yfr8MMPV3V1tR588EGvi9atIJcdwff73xMo+YYZ9Vnav3+/Jk+erDfffFPV1dX68pe/rH/+8596/PHH1dLSoqVLl+rMM8/0upgJBbnsucaMevtiw2T8eOmPf/SmLEhblzPqaf7K0ty5c/Xmm29qzpw5mj9/vpyD/1MmTpyoiy++WG+99ZZvb8xBLjuSyPHSKZmgdpK/aP7K0v3336+ePXvq5z//ecdNWZJKDu5fevTRR7v22Y888ogcx9Hrr7+e0eu9LDtc5PN9UQiU/BaYUAnf+H7+858nfLypqUmHHnqoJk6cmLMybdiwQc3NzfrqV7+qI444Iuqxp556SpJ0xhln5Kw86XCr7FOmTJHjOFq0aFHUeWOMZs6cKcdxdNNNN2VecHTPp/uiXHstgVIIAhMqp512miTp3XffTfj4v/3bv6mtrU333HNPzsoUbmOfMGFCxzljjObOnatnn31WkyZNUlVVVc7Kkw63yn7XXXepqKhIP/7xj9XW1tZx/vvf/74WLlyo2bNn64477sj+H4Cu+XBfFMeR5s3rPP5//49AyVeB6VMZO3asDj/8cP0xQU/e008/rWXLlunaa69NeiO8++67tWvXrpQ/s7q6Wueee26Xj69cuVKSVFNTo9dee02/+c1vtHz5cjU1NWnMmDG+3vjJrbKPGTNG3/72t7Vw4UI99thjmjlzpm6//Xb94he/0Le+9S3df//9Nv8ZSKS+Pr5PxUPUTgqMMSbZl69MnDjRSDKbN2/uOLdnzx4zZMgQ069fP7Nr166krx86dKhRaERbSl8zZsxI+n5TpkwxksyOHTvMhRdeGPXaCy+8MKqcbnj44YeNJPPaa6+l/Vo3y75x40Zz2GGHmaFDh5p58+YZSebMM880+/bty/g9vVZTU+N1EQKnvd2YUIR0fiFvdJkbgQqV//iP/zCSzLPPPttx7sYbbzSSzMMPP5zz8hxzzDFm6NChxhhjWltbzc6dO83rr79uzj//fCPJjB49Ou419957rxk2bJg59NBDzdixY82bb76Z0mfZDsR0y/7GG2+Y6dOnm0GDBqV0vW+66aaOspxyyilm7969Kf07/YpQSU/v3gRKnusyNwLT/CVJp556qiTpj3/8o77xjW9ozZo1+p//+R996Utf0owZM3Jalg0bNmjnzp06/fTTJUnFxcU6+uijdfrpp+v0009XdXW1Vq1apebmZpUeHM755JNP6rvf/a7uu+8+nXbaabrvvvs0bdo0NTY26rjjjkv6edddd11c090HH3ygxYsXa8aMGRo2bFjUY9XV1VbLvmfPHo0aNUqXXnqpLr300m6vT9++fTv+/uCDD6pnz57dvgb5Iba563e/k6ZO9aYsyL1Ahcopp5wix3E6OuuvueYatbW16d57740aEtsVm30q4T6J2trahI9/8YtflCT16tWr49wvfvELzZw5U7Nnz5YkzZs3T0uXLtX8+fNVV1eXtCzXXXdd3LlHHnlEixcv1syZM/XlL3+5m39NdmU/66yzdNZZZ0mSZs6cmfT9H3/8cX3/+9/XgAEDtG3bNs2dO1fz589PuXwIrpT6T/w2j8Zv5Qm6ZNUYTypV3aioqDA9e/Y0v/71r40kc9VVV6X8WptNSD/84Q+NJPPyyy/HPfbJJ5+YQw45JKoJad++faa4uNg89dRTUc+9+uqrzcSJE1P+N0TKtE8l3bLHOuKII7ps/nrhhRfMIYccYkaNGmW2b99uysrKTElJifnrX/+aVhn9huav5D79NI3mrooKY4qKQk8qKgode8lv5QmGLnMjMEOKw0477TR99tlnmjNnjo455hjdeuutKb92/fr13YVo1NcjjzzS5XuFf9t/8sknQ51TB+3fv19z5szRgQMHdMMNN3Sc37lzp9ra2tS/f/+o9+nfv7+2bduW8r/BhnTLnqq33npLF1xwgYYMGaKXX35Zffv21U9/+lO1trYyNyWPOY50sHLbIekIL7/No/FbeQIucKES7lfZs2eP6urq1KdPH0/KEb4xP/jggxo3bpxuvPFGXXXVVSovL9czzzyjmTNnJuzniW2mM8ak1HRnU6ZlT2bVqlU6++yz1bt3by1btkwDBw6UJF1wwQWqra3V4sWLtXz5cuv/FsRobpYqK6WSktCfzc2uvk/sj+6mTSkMGfbbPBq/lSfouvlt3XfefPNNI8mMGzfOtLe3e1KGDRs2GElm8uTJ5pvf/KY5+uijTXFxsenTp4+ZNGlSXBOXMf5p/sqk7LFim7/Wrl1r+vfvb77whS+YVatWxT1/2bJlRpKZMGFCyuX0m8A0f0U25YS/KiqMWbcu8/fpokko49Fd69aF3q+4OLOy2ea38gRDfgwpNsaY6dOnm6KiIrNixQrPyrBo0SIjydx5551pvW78+PFm9uzZUedGjBhhbrrpJpvFSyrTskdK1qeSr3wXKl3dCIuL4+/2mfQTxL5PcXHHQ2+9xXBh5MmQ4t/85jeqr6/Xd77zHY0bN86zcoSbj8aOHZvW666//np9+9vf1vjx43Xqqafq/vvv15YtW3TllVe6UcyEMi37nj179Le//U2S1N7err///e/64IMP1KdPn26HQ8MF4UUj29s7F41cvTrUdBM+H5ZJP0Hk+0Q0CbHdL7qVLHE8yb8YGzZsMHV1dWbWrFmmR48eprKy0vOJdFOnTjWSzCeffJL2a++9914zdOhQ06NHDzN27FjzxhtvuFDCrmVa9tdeey2jSZb5wnc1la5qEuEaTLY1lQQ1odjaSWur/X8WAqPL3PD9Jl0LFizQnDlz9IUvfEGTJ0/W3XffrUGDBnlapn79+qlnz55av369p+XIRJDL7iXfbdJVWRldkygrC9VUwizPvWD9LsTocnSR70MF8APfhUqOJuzdcov0k59EnyNQIHZ+BPJMaWl0zcQFsbWTL31J+sMfXP1I5AFCBUAcmruQqcBNfgTgLgIF2SBUAEiSjjqKQEH2CBXA72wtvZKE40i7d3ce33UXgYLM0KcC+FV4hFdjY+e5yImOllA7gU3UVAC/Cs+aj2RxFd19+wgU2EeoAH4S2dTV2Bi93IpkbRVdx5EOOyz6HIECGwgVwE/CtZO2tsSPl5WFJjpmIbZ28qc/ESiwhz4VwE8iN4wKKy62Nmue5i64jZoK4CexG0ZVVEitraGO+SwCZc0aAgW5QagAflJfH2riKi620tQlhcKkvDz6HIECt9D8BfiJ5TW9YmsnLS1Sr17W3h6IQ00FyFOJmrusBUoOJmQimAgVIIiS3NR/+csc9J9EjlILT8gExH4qQEp8t59KF5t05Wy735KS6GHPxcWhAQUoFF3up0JNBQiiyKHHB2fZJ6qduNYhHztKLTwhk2axgkeoAEEUc1N32qJrCa6P7upqlBrNYgWPUAGC6OBNfYw+kNMePfs+J8OFw6PUYufQJKhBobAQKkAQlZbKaVytP2tMx6lzz/XB/JOumsVQMAgVFJyNGzfqK1/5isrLy1VZWam5c+e692Eu9TEk6j/57W+tvHV2XJi8iWBh9BcKztatW7V161aNHTtWu3fvVk1NjZ577jlVVFR0+ZqMR391MUorU8Z0VgQizwE5xugvIGzgwIEaO3asJKlXr14qLy/X5s2b3fkwi30MjkOgwP8IFRS09evX6/3339eECRPiHluwYIFqa2tVW1urHTt2ZPYBlvoYYpu7Fi4kUOBPNH+hYO39y1+0bfx4Hb9/v4rC7f9drASccfNXeEvgpqaMl69ndWH4UJfNX4QKCtKBAwe0+Ytf1HGffaaicEdFkv4OL2bUb9kiDR4cfY5AgU/QpwKEGWN0+eWX69hwoEi+m1PhOAQKgolQQcF5++239dhjj+mjHj0UnjbY7ji+mVMR29y1fj2BguAgVFBwTjvtNBljNLyxUcUVFVJxsYrKy30xpyJR/8nQod6UBcgEm3ShcFneECsbzz0nnXde9DlqJwgiaipANizMmHccAgX5g1ABspFsVd4UAie2uautjUBBsBEqQDaSzZjvZhn4RP0nsTPmgaDhRxjIRrIZ810EzkUXMaER+YtQySfsupd7yVblTRA4jiM9+WTnU/r1I1CQX5hRn08sr4iLThnNqI9ZosVpjP5eECYIMGbUFwR23fOXiN0RfRUo1GjhIkIln7Drnn3hG/DKlRndgB3Hh/0n7CMPFxEq+YRd9+wL34CltG/AsWFyy3X/kKnwQQ2BGi1cRJ8KkExJidTWplpJDVIosFtbu31ZwtqJX/q8/FIOBBl9KkBGUm1SPNhMtrf4qK6bu/xSQ6BGCxcRKkAy4RuwlPwGPH26nMbVOrK9Jep0VEOAX/q8IgYQaPXqtDcNA5IhVIBkwjfgmpqkN+DY0V3vFJ0a3yFPDQEFgFWKgSzFNXcVFXfWbiL5aFVkwC3UVIAMrViRIFCKSwqvFsK8F0Rg9BfQneZm1VZWquHAgVA/SH29nBPim8E8n3/iFUaTFaIuR38RKkB3KitV29gYGlJcVCSnvS3q4d27pSOP9KRk/nBw2HWHFIddI9AYUgxkLGLob2ygGFPggSL5Z1QbfIFQQWFLpT9g5Eht1iA5MRX3gm3uisWoNkSg+QuFLYX+gFBnfMecekkECgoefSpAQt30B3SO7gqFCmECSKJPBQUhk6GtSfoDfLe6MBAAhAryRyZLuifoDxgwID5QamrcKTKQbwgV+EtkbePQQ0M3+1RrHZks2BizDpZzQqk+/rjz4QsuoIYCpINlWuAv4dpGe3tnX0e41tHdhLqRI6M73dMc2kpzF5A9airwl8jaRliqtY4Mh7a2txMogC3UVOAvkbWNsFRrHRks2BgbJhKBAmSDmgr8JbK20aNH59wRFybUxQbKs89mESgsqghIYp4KClS6zV21tbVqaGjo+gksqojCwjwVQJI2bXKp/8QvWwUDHiNUUDAcRzr22Ohz1vpPWFQRkESooEDE1k62bLHcIc+iioAkRn+hAORkuDBbBQOSqKkgjz3zTAqBwqgtwCpqKshLKc8/iZzBn+rMfQBdoqaCvBMbKG1tSZq8/DRqi1oT8gChUggK6GaVqLmrKNlPuZ9GbWWyyjLgM4RKIcjnm9XBwPyq8/vMOuT9NGrLT7UmIEP0qRSCfL5ZTZ8upzG6D6RfP0UtX5+Un0ZtZbnKMuAH1FQKgZ+aeCyLDRRTXJJ6oPiNn2pNQIaoqRSC+vpQk1dTUyhQ8uRmFdfcVVQsjSzzpjA2+KnWBGSImkohiNndUKWlmb+XDzr9HSdBoBSX8Ns94AOECtLjcad/bJj8tO8vQ4Fy/PHS/v3SiSeGwu711z0PP6AQsfQ90lNS0rnNrxRq/29tzclHx9VOKirjN/SSQv1GJSWhcllair7bpe+BwsLS97DEg07/lpYulltJtPWwFDq3f3/+jngDfIxQQXpyPELJcaTevaPPdVSuIwMuUlFR566R4eM8GvEG+BmhgvTY7PTvRmzt5L33YiY0Rgbc8OGhr3DYvfRS0vBbunSpRo4cqeHDh+uOO+5Ir2A+GKwA+BV9KvAlN5erb2tr04knnqhly5ZpyJAhGjdunB5//HFVVFR0+ZqoPhW2DgboU0GWcvTb+fLl7u9/smLFCg0fPlylpaXq0aOHLrroIi1evDj1N8jnFQqALCWtqUydOtXs3Lkz4zffsWOH+vbtm/Hrcy1o5ZVyWObVq6XPP+88PuywULhkoKsyr1wZ/9yamow+IqlPP/1ULS0tGjp0qCTpk08+0d69e3XcccfFlTP8879v3z5VV1eHHrB4LdzCz3JuBK3Mtsq7cuXKl4wxUxM+aIxJ9pWVmpqabN8ip4JWXmNyWObiYmNClYbQV3Fxxm+VqMyRby0Z889/ZlPY5J566ilz+eWXdxw/+uij5pprrkn6mp49e3YerFtnTEVF6BpUVISOfYaf5dwIWpktlrfL3GCZFqTGxcUOc7Ldb4QhQ4Zo48aNHcebNm3SoEGDUn8DllMBukSfClLjwlDiG27IfaBI0rhx47R27Vp99NFH2r9/v5544gmdc8457n8wUABcralcccUVbr69dUErr5TDMlv87fyKK65IfbtfF5SUlOiee+7RmWeeqba2Nl122WWq7KZP5JhjjslN4SzhZzk3glbmXJSXIcXITHNz/MrHKc5Z8aJ2ki2WaQGiMKQYWUg0nDjDhSXTChQmGQKBQ00F3auslP76184E6NEjFCZpLCyZUXOXjyYZUlMBorhTU3n66adVWVmpoqKiuP9wdXV1Gj58uEaOHKmXXnop4ev/8Y9/aPLkyRoxYoQmT56sTz/9NJvipO3CCy9UdXW1qqurNWzYsM55CDGGDRum0aNHq7q6WrW1tTktY6yf/OQnGjx4cEe5X3zxxYTPy2oZklhNTdEJsH9/KERSXFsrNlDOP39/wkCJu845nmTY3TUzxujaa6/V8OHDVVVVpffee8/V8nRn48aN+spXvqLy8nJVVlZq7ty5cc95/fXX1bt3746fl//6r//yoKTRuvv/5Lfr3NTU1HH9qqurddRRR+nuu++Oeo7X1/myyy5Tv379NGrUqI5zqd5frd4rpOzmqTQ2Npo1a9aY008/3fzpT3/qOL969WpTVVVlPv/8c9Pc3GxKS0tNa2tr3Ot/8IMfmLq6OmOMMXV1debGG2/MfNR0lq6//npzyy23JHxs6NChZseOHTkuUWL/+Z//ae66666kz2ltbTWlpaVm3bp1Zt++faaqqsqsXr068w+tqIifSFJUlNJcjdiX3XjjjV1+n+Ouc0VF6HMiP88l3V2zmpoa88ILL5ipU6ea9vZ2884775jx48e7Vp5UbNmyxaxcudIYY0xLS4sZMWJE3Pf5tddeM1/72te8KF6Xuvv/5LfrHKm1tdX079/frF+/Puq819f5jTfeMCtXrjSVlZUd51K5v2Zxr+gyN7KqqZSXl2tkgt9QFy9erIsuukiHHnqojj/+eA0fPlwrVqxI+LwZM2ZIkmbMmKHnnnsum+JkzBijp556Sv/yL//iyed3sNSHkPUyJLHq60NNXmGRTVFdLCzZ2pq4/+Tkk0/Wpk2bUv/cHK2InMo1W7x4sS699FI5jqOTTz5Zu3bt0tatW10rU3cGDhyosWPHSpJ69eql8vJybd682bPy2OK36xzp1Vdf1QknnNCxGoNfTJw4UX369Ik6l8r91fq9Qi511G/evFnHHntsx/GQIUMS/rB//PHHGjhwoKTQf5Dt27e7UZxuLV++XP3799eIESMSPu44jqZMmaKamhotWLDAvYKk2Pl9zz33qKqqSpdddlnCKm2q1z9lpaWhPpWKipRu8I4jHXJI9Llwc9dDDz2kadOmdfG6mOucwxWRU7lm1q+rRevXr9f777+vCRMmxD32zjvvaMyYMZo2bZpW+2DSZnf/n/x8nZ944okuf/n023VO5f7qxrXudp7KpEmTtG3btrjzt912m77+9a8nfI1J0GDuJOqpzYFUyv/4448nraW8/fbbGjRokLZv367JkyerrKxMEydOtF/YmD6E1sZGVUe0kd5222266qqrdPPNN8txHN1888264YYb9NBDD0W9jSvXv5t5KuHrvHr1h1Hn//3fV+iOO8Z3lL+kpEQXX3xxwvfI2XVOIJVr5qef60h79uzR+eefr7vvvltHHXVU1GNjx47Vhg0bdOSRR+rFF1/Uueeeq7Vr13pU0pDuvs9+vc779+/XkiVLVFdXF/eYH69zKty41t2GyiuvvJL2m6a6DEb//v21detWDRw4UFu3blW/fv3S/qzudFf+1tZWLVq0SCsTrWZ4ULjs/fr103nnnacVK1a4c7OLWQqlpKxMH374YZdPnz17ts4+++y481kvQ5KBV155JfF2vz9rkupH6tmZM/X888/r1Vdf7fKHNmfXOYFUrpkX17U7Bw4c0Pnnn6+LL75Y3/jGN+IejwyZs846S1dffbV27tzp6WTO7r7PfrzOkvS73/1OY8eOVf/+/eMe8+N1TuX+6sa1dqX565xzztETTzyhffv26aOPPtLatWs1fvz4hM9buHChJGnhwoVd1nzc9Morr6isrExDhgxJ+PjevXu1e/fujr+//PLLUSMsrEqhDyGybfm3v/1twrLkehmStWuT7B/f1qb2v/5VY378Yy1ZskQ9e/ZM+B45vc4JpHLNzjnnHD366KMyxujdd99V7969O5oXvGCM0eWXX67y8nJdf/31CZ+zbdu2jt9GV6xYofb2dh199NG5LGaUVL7PfrvOYclaNPx2naXU7q+u3CuS9eJ31/2/aNEiM3jwYNOjRw/Tr18/M2XKlI7Hbr31VlNaWmpOPPFE8+KLL3acv/zyyztGiu3cudOcccYZZvjw4eaMM84wn3zySSqjDqyaMWOGmT9/ftS5zZs3m2nTphljjFm3bp2pqqoyVVVVpqKiwtx66605L2OkSy65xIwaNcqMHj3aTJ8+3WzZssUYE11mY0IjaEaMGGFKS0tdLXPs6C6Ff2piVjU+IJkxY8aYMWPGmDlz5sSV2Q/XOdE1mz9/vpk/f76pqakx7e3t5uqrrzalpaVm1KhRUSMevbB8+XIjyYwePbrj2r7wwgsdZTbGmHnz5pmKigpTVVVlJkyYYN5++21Py9zV9zmyzH67zsYYs3fvXtOnTx+za9eujnN+us4XXXSRGTBggCkpKTGDBw82DzzwQJf3V0v3ii5zo7AnP2ax1AjiayfbtkkdLQM+mrhoA5MfgSgs05JQhkuNIPFw4aimZpvDgVmuBQiMwq6plJSktdQIpF/9Srryyuhzri8I6YNaDzUVIAo1lYRGjkx5qRHXBeC3ccfxIFAk9oQHAqSwQyWHM7a7ZaspzqVwim3uam/P4ZL1fgp/AEkVdvOXn9hqinOhqcjz/U98MKCC5i8gCs1fvmfrt3GLTUUTJ/ogUKScLtcCIDuEil/YaoqzFE6OIy1f3nk8ZkwwdmgE4C1X96hHGmztAV9fH99UlCZf1E4ABBI1lWQCMCIrThZNRcYQKACyQ6gkU0CTIx2ns9UsjEABkC5CJZkCmR8RWzu55x4CBUBm6FNJJmYp+nycH0FzFwCbqKkk46fJkZbt2kWgALCPmkoytkZk+UyiPbIIFAA2UFMpMLGBsmYNgQLAHmoqBYTmLgBuo6ZSAN58k0ABkBvUVPIc/ScAcomaSh6LDZR9+yICJYirBQDwPUIlTyVq7urRI+JEAa0WACB3CJU8c/PNKfaf5Gq1AGpEQEEhVGzwyY3TcaRbb+08PuKIJP0nbuymmOg6UCMCCgo7P9rgwm6L6Up7dJcbuykmug5NTXZ2tPQYOz8CUbrc+ZFQscHWVsAZ8s1w4UTXIXb9NA8C1wZCBYjCdsKucqMpKQWlpT4KFCnxdcjj9dMAxCNUbPDgxuk40kcfdR7fcosP5p8kug7sLw8UFCY/2pBs4UkX+i58VTuJlKcLcAJIHTUVt1kc/XTggI8DBQBETcV9luaDsNwKgCCgpuI2C534sYHyhz8QKAD8iVDJVKoTHrPsxE/U3PWlL2VYZgBwGaGSqVT7SjIc/bRpUxr9Jz6Z0Q8AhRkqNm7CLq6d5TjSscdGn0va3MVSKAB8ojBDxcZN2KUJj7G1k927U+g/ydXikADQjWCEiu3mHRs3YRcmPCZq7jryyBRe6NGMfgCIFYxQsd28Y+MmbHGm+NNPZzn/hKVQAPhEMELFdvOOVzfhBDUux5G+9a3Op/TqlcFwYZZCAeATwQgVm807biz5nqqYGpdzQvTnGiO1tOSmKIXqBz/4gcrKylRVVaXzzjtPu3bt8rpIQF4JRqjYrFl4OVIqosbltLdFPcRkxtyYPHmyPvzwQ/35z3/WiSeeqLq6Oq+LBOSVYISKzeYdL0dKjRyp2fpfOTHb1BAouTNlyhSVlIRWJzr55JO1adMmj0sE5JdghIpNHo6UchpX6wHN6jie9a2W1AKFyY2ueOihhzRt2rQuH1+wYIFqa2tVW1urHTt25LBkQHAV3s6PHvWpZDW6a8QI6W9/6zwePlxau9ZKufLRpEmTtG3btrjzt912m77+9a93/L2hoUGLFi2Sk2i1zhjs/AhEYTthrxjTWTGKPJcWlii2auHChbr//vv16quvqmfPnim9hlABorCdsBeGDrUQKLBq6dKl+u///m8tWbIk5UABkDpCxSWOI/39753Hr7ySZqBE9qMcckj0Y8OHWyljIbrmmmu0e/duTZ48WdXV1bryyiu9LhKQV9ikywVWdmcMD31ubw999egRGgYd7gdCRv4W2TcFwDpCxaLdu6Wjjoo+l3FzV+TQZ2NCgdLamlX5AMBtNH/FynD4ruNYDBSJRSIBBBKhEityxn1jo3TCCd2GS2xz18cfW+iQD68iUFQUCrimJuaoAPA9hhTHKikJBUqkoqLQDX716rinW+k/SaaysrNvJUk54C6GFANRGFKcsshmp7AEy7k0NeUgUMIfxAZcAAKCUIkVbnaKFNOn4TjRTzn2WBfnn9C3AiBACJVY4cUr162TKiriVkaOrZ20tUXPR+lW7ECA119PPjCADbgABAh9KmmIa+4qLkl//bDYPpKSktBQ4Wz7TLzcJ6YA0KcCRKFPJRvPPJMgUIqKM9uTJbaPZP9+O30mXu4TAwAHESrdcBzpm9/sPP7Rjw7WUDINgtg+kh497PSZ2OrQZ5l9AFkozFBJ8caZaHTXrbcqu87z2D6Sl16y02diq0OfGg+ALBRmn0oKcz+SDhf2Y/+FrTLFztMpLmZ5GNGnAsRgP5UoSW6ct98eauKKVFDL1TPZMiFCBYhCR32ULpqKHCc6UNJerj4fMIQZQBYKc5Xi+vq4pqKczI4PgvA8HQDIQGGGSsSNs60t1BoWqWADBQCyVJjNXwf96OpPCRQAsKgwayoKj+76YsfxVmeQBpR/URJNPwCQqYIMlbj+EzmhcW5N2z0pDwDki4Jq/vr88y6WW5FYARgALCiYUJk3Tzr88M7j+++XzLpmhs8CgEUF0fwVWztpbQ3liMTwWQCwKe9DhfknAJA7edv8tWNHdKAcfzyBAgBuy8tQ+d73pH79Oo/ffTfJCu4s9Q4A1uRd81fazV3hpd7b2zuXeqefBQAyklc1lYz6T2xtbgUAyI9Q+fvfowPlRz9Ko//E1uZWAIDgh8r3vicNHdp5vKVoiG79bRp9Iyz1DgDWBLpPJeHs+PZ2ac3W1PtGWOodAKwJZE3FmASBUlxC3wgAeCxwofLBB51dIJL0v/97sP+EvhEA8FygQuXss6WTTuo8bmmRZs06eEDfCAB4LjB9Kt0OF6ZvBAA85/uaSltbdKAMHsxyKwDgV74OlVWrovePf+89adMm78oDAEjOt6Hyne9I1dWdx62t0f0pAAD/8WWoOI50332dx8aE9z9xCYtKAoAVvgqV2O1+f/rTHPWfhBeVbGvrXFQSAJA234z+eu016YwzOo/XrQsN6MoJFpUEACt8UVOZPj06UNrbcxgoEhMnAcASz0PFcaTnnw/9vV+/xEuwuI6JkwBghWehsmtXdHg88ID08cceFSY8cbK1NfRnTqtJAJA/POlTeeop6cILO4+3b5f69vWiJAAAm3IeKpWVUmNj5zGz4wEgf+S0+ctxOgPl1FPzNFCY8wKggOUkVLZsie4/qa+X3norF5/sAea8AChgrofK738fWgQybPfu0BL2eYs5L4Hws5/9TI7jaOfOnV4XBcgrrofKL3/Z+XdjpCOPdPsTPcacF9/buHGjli1bpuOOO87rogB5x/VQeeYZ6Z//zNP+k0SY8+J73/ve93TnnXfKyfmEKCD/uR4qJSXSYYe5/SkRvO4oZ86Lry1ZskSDBw/WmDFjun3uggULVFtbq9raWu3YsSMHpQOCzzHJqxDBq19UVoY6yNvbQ81PZWXsCFlgJk2apG3btsWdv+2223T77bfr5ZdfVu/evTVs2DA1NDTomGOO6fY9a2tr1dDQ4EZxgSDqspqff6FSUhIaeRVWXByqNaDg/eUvf9FXv/pV9ezZU5K0adMmDRo0SCtWrNCAAQOSvpZQAaJ0GSq+WaXYmpEjo2sqdJTjoNGjR2v79u0dx+nUVACkxvMFJa2joxwAPJN/NZVwRznQjfXr13tdBCDv5F9NBQDgGUIFAGANoQIAsIZQAQBYQ6gAAKwhVAAA1hAqAABrCBUAgDWECgDAGkIFAGANoQIAsIZQAQBY426oeL0LIwAgp9wNlenTQ3ubtLWF/pw+3dWPAwB4y91QaWoKbZYlhf5sanL14wAA3nI3VEaODO2+KLELIwAUAHdDhV0YAaCguLvzI7swAkBBYUgxAMAaQgUAYA2hAgCwhlABAFhDqAAArCFUAADWECoAAGsIFQCANYQKAMAaQgUAYA2hAgCwhlABAFhDqAAArGE7YQCANWwnDACwhu2EAQDWsJ0wAMAathMGAFjDdsIAAGsYUgwAsIZQAQBYQ6gAAKwhVAAA1hAqAABrCBUAgDWECgDAGkIFAGANoQIAsIZQAQBYQ6gAAKwhVAAA1hAqAABrHGOM12UAfM9xnKXGmKlelwPwO0IFAGANzV8AAGsIFQCANYQKAMAaQgUAYA2hAgCw5v8D63+24aQprnEAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "ax = plt.figure(figsize = (7, 7)).add_subplot()\n", "centerAxes(ax)\n", "line = np.array([1, 0.5])\n", "xlin = -10.0 + 20.0 * np.random.random(100)\n", "ylin = line[0] + (line[1] * xlin) + np.random.randn(100)\n", "ax.plot(xlin, ylin, 'ro', markersize = 4)\n", "ax.plot(xlin, line[0] + line[1] * xlin, 'b-')\n", "plt.text(-9, 3, r'$y = \\beta_0 + \\beta_1x$', size=20);" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "Now, to obtain the least-squares line, find the least-squares solution to $X\\mathbf{\\beta} = \\mathbf{y}.$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "From linear algebra we know that the least squares solution of $X\\mathbf{\\beta} = \\mathbf{y}$ is given by the solution of the __normal equations__:\n", "\n", "$$X^TX\\mathbf{\\beta} = X^T\\mathbf{y}$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We also know that the normal equations __always__ have at least one solution.\n", "\n", "And if $X^TX$ is invertible, there is a unique solution that is given by:\n", " \n", "$$\\hat{\\mathbf{\\beta}} = (X^TX)^{-1} X^T\\mathbf{y}$$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "## The General Linear Model" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "Another way that the inconsistent linear system is often written is to collect all the residuals into a __residual vector.__ \n", "\n", "Then an exact equation is\n", "\n", "$$y = X\\mathbf{\\beta} + {\\mathbf\\epsilon}$$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "Any equation of this form is referred to as a __linear model.__ \n", "\n", "In this formulation, the goal is to find the $\\beta$ so as to minimize the length of $\\epsilon$, ie, $\\Vert\\epsilon\\Vert.$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "In some cases, one would like to fit data points with something other than a straight line. \n", "\n", "In cases like this, the matrix equation is still $X\\mathbf{\\beta} = \\mathbf{y}$, but the specific form of $X$ changes from one problem to the next." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "## Least-Squares Fitting of Other Models" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "In model fitting, the parameters of the model are what is unknown. \n", "\n", "A central question for us is whether the model is _linear_ in its parameters." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "For example, the model \n", "\n", "$$y = \\beta_0 e^{-\\beta_1 x}$$\n", "\n", "is __not__ linear in its parameters. \n", "\n", "The model \n", "\n", "$$y = \\beta_0 e^{-2 x}$$\n", "\n", "__is__ linear in its parameters." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "For a model that is linear in its parameters, an observation is a linear combination of (arbitrary) known functions." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "In other words, a model that is linear in its parameters is\n", "\n", "$$y = \\beta_0f_0(x) + \\beta_1f_1(x) + \\dots + \\beta_nf_n(x)$$\n", "\n", "where $f_0, \\dots, f_n$ are known functions and $\\beta_0,\\dots,\\beta_k$ are parameters." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "__Example.__ \n", "\n", "Suppose data points $(x_1, y_1), \\dots, (x_n, y_n)$ appear to lie along some sort of parabola instead of a straight line. " ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "hide_input": false, "tags": [ "hide-input" ] }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZQAAAGKCAYAAAArGbdLAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAfVElEQVR4nO3df2xV9f3H8ddta/W7TXFutF6pVS8t7W2hIFxl/8xfXQE3CxsaptGsmT9YdMu2uM2Y72I2Fze6LWY6JSZsbt8aE4kkDqwymRoxzmlY0SyxBawUFGqFIhAFB+X2fr5/HG57295fbT/3nnPPfT4Scnvvubf9eLznvu77fH6cgDFGAABMV4nbDQAA+AOBAgCwgkABAFhBoAAArCBQAABWlGXYzhAwQNKyZcv0wgsvuN0MwCsCyR6kQgGycOjQIbebAHgegQIAsIJAAQBYQaAAAKwgUAAAVhAoAAArCBQAgBUECgDACgIFAGAFgQIAsIJAAQBYQaAAAKwgUAAAVhAoAAArCBQAgBUECgD4XV+f1NgolZU5t319OfkzBAp87Q9/+IMaGxs1d+5c3XTTTTpx4oQOHz6slpYW1dbWqqWlRUeOHHG7mUButbZKO3dKw8PObWtrTv4MgQLf6u/v1x//+Ed1dXXpnXfe0fDwsNavX6/29nY1Nzert7dXzc3Nam9vd7upQG7t2iXFYs7PsZhzPwcIFPhaNBrVf//7X0WjUX322We64IILtGnTJrW1tUmS2tratHHjRncbCeRaXZ1UcvrjvqTEuZ8DBAp8a9asWfrpT3+q6upqBYNBzZgxQ0uWLNGBAwcUDAYlScFgUAcPHnS5pUCOdXZK9fVSaalz29mZkz9DoMC3jhw5ok2bNmnPnj368MMPdfz4cT355JNZv37dunWKRCKKRCIaHBzMYUuBHAuFpO5uKRp1bkOhnPwZAgW+9dJLL+mSSy7RzJkzdcYZZ2jlypX617/+pcrKSg0MDEiSBgYGVFFRkfT1q1evVldXl7q6ujRz5sx8Nh0oSAQKfKu6ulpvvvmmPvvsMxlj9PLLLyscDmv58uXq6OiQJHV0dGjFihUutxTwhzK3GwDkyuLFi3XDDTdo4cKFKisr06WXXqrVq1fr2LFjWrVqlR5//HFVV1drw4YNbjcV8IWAMSbd9rQbgWIRiUTU1dXldjMArwgke5BTXgAAKwgUAIAVBAoAwAoCBQBgBYECALCCQAEAWEGgAACsIFAAAFYQKAAAKwgUAIAVBAoAwAoCBQBgBYECALCCQAEAWEGgAACsIFAAAFYQKAAAKwgUAIAVBAoAwAoCBQBgBYECALCCQAEAWEGgAACsIFAAAFYQKAAAKwgUAIAVBAoAwAoCBQBgBYECALCCQAEAWEGgAACsIFAAAFYQKAAAKwgUAIAVBAoAwAoCBQBgBYECALCCQAGAQtDXJzU2SmVlzm1fn9stmoBAAYBCsHSp1NMjDQ87t0uXut2iCQgUACgE772X/r4HECgAACsIFAAoBDU16e97AIECAIVgyxapoUEqLXVut2xxu0UTlLndAABAFkIhqbvb7VakRYUCALCCQAEAWEGgAACsIFAAAFYQKAAAKwgUAIAVBAoAwAoCBQBgBYECAF5TAEvVJ0OgAIDXtLZKO3c6S9Xv3OncLwAECgB4za5dUizm/ByLOfcLAIECAF4RP9U1PDz6WEmJVFfnXpsmgcUhAcAL+vqkcFgaGhr7eH291NnpTpsmiUABAC9obZ0YJqWlnl9hOBGnvADAC5L1kxTIqa44AgUAvKCuzukviSsvL5hTXXEECgB4QWen018SvyLjjh3ORbUKCH0oAOAFBXBFxkyoUAAAVhAoAAArCBQAgBUECgDACgIFAGAFgQIAsIJAAQBYQaAAAKwgUAAAVhAo8LWjR4/qhhtuUH19vcLhsN544w0dPnxYLS0tqq2tVUtLi44cOeJ2MwFfIFDgaz/60Y+0bNky7dy5U//5z38UDofV3t6u5uZm9fb2qrm5We3t7W43E/CFgDEm3fa0GwEv++STTzR//nz19fUpEAiMPF5XV6etW7cqGAxqYGBAV111lXZluMRqJBJRV1dXrpsMFIpAsgepUOBbfX19mjlzpr773e/q0ksv1e23367jx4/rwIEDCgaDkqRgMKiDBw8mff26desUiUQUiUQ0ODiYz6YDBYlAgW9Fo1G99dZbuvPOO/X222/r85///KROb61evVpdXV3q6urSzJkzc9hSwB8IFPhWVVWVqqqqtHjxYknSDTfcoLfeekuVlZUaGBiQJA0MDKiiosLNZgK+QaDAt84//3xdeOGFI/0jL7/8shoaGrR8+XJ1dHRIkjo6OrRixQo3mwn4BhfYgq898sgjuvnmmzU0NKRQKKS//vWvisViWrVqlR5//HFVV1drw4YNbjcT8AVGeQFZYJQXMAajvAAAuUOgAACsIFAAAFYQKAAAKwgUAIAVBAoAwAoCBQBs6+uTGhulsjLntq/P7RblBYECALa1tko7d0rDw85ta6vbLcoLAgUAbNu1S4rFnJ9jMed+ESBQAMC2ujqp5PTHa0mJc78IECgAYFtnp1RfL5WWOrednW63KC9YHBIAbAuFpO5ut1uRd1QoAAArCBQAgBUECgDACgIFAGAFgQIAsIJAAQCbinTZFYlAAQC7inTZFYlAAYDpGV+RFOmyKxKBAgDTM74iKS0tymVXJAIFAKZnfEUSjRblsisSS68AwPTU1TmVSSzmVCT19UW57IpEhQIA01OkC0EmQ4UCANNRpAtBJkOFAgCwgkABAFhBoAAArCBQAABWECgAACsIFACAFQQKAMAKAgUAslXES9Nng0ABgGwV8dL02SBQACBbRbw0fTYIFADIVl1d0S5Nnw0CBQAy2bpVOvNMqafHqUwCgaJfCDIZFocEgEyWLpWGhkbvn3EGC0ImQYUCAJkkhkmy+5BEoABAZuXl6e9DEoECAJlt2TIaIuXlzn1MQB8KAGRy1VXSyZNut8LzqFAAAFYQKAAAKwgUAIAVBAoAwAoCBQASsaLwlBEoAJCIFYWnjEABgESsKDxlBAoAJGJF4SkjUAAgUWens5JwaSkrCk8SM+UBIFEoxErCU0SFAgCwgkABAFhBoAAoXsw5sYpAAVC8mHNiFYECoHiNn3PS00O1Mg0ECoDilTjnJI5qZcoIFADFK3HOSSJmyE8JgQKgeMXnnESjUkMDM+SniUABAIkZ8hYQKACKT7LhwonVSne3cx+TQqAAKD4MF84JAgVA8WGJ+pwgUAAUH5aozwkCBUDxoQM+J1i+HkDxYYn6nKBCAQBYQaAAAKwgUAAAVhAoAAArCBT43vDwsC699FJdd911kqTDhw+rpaVFtbW1amlp0ZEjR1xuIeAPBAp87+GHH1Y4HB65397erubmZvX29qq5uVnt7e0utg7wDwIFvrZ//349//zzuv3220ce27Rpk9ra2iRJbW1t2rhxo0utA/yFQIGv/fjHP9bvfvc7lSRcROnAgQMKBoOSpGAwqIMHDyZ97bp16xSJRBSJRDQ4OJiX9gKFjECBbz333HOqqKjQokWLpvT61atXq6urS11dXZo5c6bl1gH+w0x5+Nbrr7+uZ599Vps3b9aJEyf0ySef6JZbblFlZaUGBgYUDAY1MDCgiooKt5sK+AIVCnxrzZo12r9/v/bu3av169frmmuu0ZNPPqnly5ero6NDktTR0aEVK1a43FLAHwgUFJ17771XL774ompra/Xiiy/q3nvvdbtJgC8EjDHptqfdCBSLSCSirq4ut5sBeEUg2YNUKAAAKwgUAP6S7HrxyAsCBYC/cL141xAoAPyF68W7hkAB4C9cL941BAoAf+nsdC7xKzkVytAQ/Sh5QqAA8JdQSCovH61S+vroR8kTAgWA/9CP4goCBYD/0I/iCgIFgP90dkr19VJpqXPb2el2i4oCqw0D8J9QSOrudrsVRYcKBQBgBYECoHCxzIqnECgAChfLrHgKgQKgcDE82FMIFACFi+HBnkKgAChcDA/2FIYNAyhcDA/2FCoUAIAVBAoAwAoCBQBgBYECALCCQAEAWEGgACgcLLXiaQQKgMLBUiueRqAAKBwsteJpBAqAwsFSK55GoAAoHCy14mksvQKgcLDUiqdRoQAArCBQAHgPw4MLEoECwHuWLpV6epzhwT09zn14HoECwHveey/9fXgSgQIAsIJAAeA9NTXp78OTCBQA3rNli9TQ4Mw3aWhw7sPzmIcCwHuYb1KQqFAAAFYQKAAAKwgUAIAVBAoAwAoCBQBgBYECwD2s2eUrBAoA93BJX18hUAC4h0v6+gqBAsA9XNLXVwgUAO7hkr6+wtIrANzDEiu+QoUCALCCQAEAWEGgAMgP5pz4HoECID+Yc+J7BAqA/GDOie8RKADygzknvkegAMiN8X0ma9cy58TnmIcCIDfifSaxmHP7/e8z58TnqFAA5AZ9JkWHQAGQG/SZFB0CBUBusE5X0aEPBUBusE5X0aFCAQBYQaAAAKwgUAAAVhAoAAArCBQAgBUECgB7WKK+qBEoAOxhifqiRqAAsIflVooagQJgctKd1mK5laJGoACYnHSntVhupagRKPCtffv26eqrr1Y4HFZjY6MefvhhSdLhw4fV0tKi2tpatbS06MiRIy63tMCkO60VX24lGnVuQyF32ghXECjwrbKyMj344IPasWOH3nzzTa1du1Y9PT1qb29Xc3Ozent71dzcrPb2drebWlg4rYUUchsoDCGEi4LBoBYuXChJOvvssxUOh9Xf369Nmzapra1NktTW1qaNGze62MoCtHatc0xLzu3ate62B54RMMak2552Y0aNjaNXbCspcc6psvooXLB3715dccUVeuedd1RdXa2jR4+ObPviF7+Y9LTXunXrtG7dOknS4OCg3n///Xw119s4riEFkj6Y00ApK3M67hLt3s15VeTVsWPHdOWVV+rnP/+5Vq5cqXPPPTerQEkUiUTU1dWV45YWiPHHdWmp02eCYpI0UHJ7yivZuVUmOiGPTp06peuvv14333yzVq5cKUmqrKzUwMCAJGlgYEAVFRVuNrHw0IeCFHIbKMmGDDLRCXlijNFtt92mcDisu+++e+Tx5cuXq6OjQ5LU0dGhFStWuNXEwsTQYKSQ21NeEudb4Zp//vOf+upXv6p58+ap5PQ36t/85jdavHixVq1apQ8++EDV1dXasGGDzjvvvLS/i1NewBgu9KFIzsiu1lanMrnkEuexPXtS/1xX53zjoZ8FHkKgAGO4FCiJEquVVKhi4EEECjCGC53y4yXOsE2FBeUAb2E+GbKU30BJHB2SCqNGAG+IB8ns2VJPD0vSI6P8Bkri6JCaGuff+J8ZNQK4J7EaCYedAEnEGQSkUZbXvxZfOA6AN8VXEo7FJk5KljiDgLRYHBLAqEz9nJxBQBreCxQ6AAH31NVJgXEDeEpKpIYGZ9kklqRHGt4LFK5JDbins1M644zR+4nD+AkSZOC9QOGa1IA74pOQh4ZGH+MYxCR4L1BYeA5wR/zsQCKOQUyCdwIl3neya5fTfxIvtekABPIjWYc8xyAmIb/DhtNJHK5oDMuvAPlWV8dCrpgW71Qo9J0A7mJZekyTdyqU8d+OOG8L5BcTjzFN3qlQ+HYEAAXNOxUK344AoKB5p0IBABS0wggUlmMBAM8rjEBhORYA8LzCCBSGFAP2UfnDMu8GSuKbvbR0dAVUhhQD2UsXGlT+sMy7gZL4Zo9GnRVQGVIMTE660KDyh2XeDZTxb/Z4sMSHFlOqA5mlCw0WYoVl3g2UdG92SnUgO+mOIyYTwzLvBkq6NzulOjAqXT9JuuMoPpk4XvlzAS1Mk3cDJd2bnVIdGJWuYo8fR+++69yfM4fTxMgZ7wZKOpTqwKhsKnZOEyMPCjNQklUvjKlHsRpfsZeWTjwOOE2MPCjMQEmGb2AoVokVe1mZdOrUxOOA08TIA/8ECt/AUKwSK/bhYeeKp9LY44DTxMgD7yxfP11coAtIfRxweQjkgX8qFL6BARwHcJV/KhS+gQEcB3CVfyoUiZFeAOAifwUKI70AwDX+ChRGegGAa/wVKIy1BwDX+CtQGOECAK7xzygviREuAOAif1UoQLFgRCM8qPgChQMRfsCIRnhQ8QUKByL8gBGN8KDiCxQORPgBIxrhQcUXKByI8ANGNMKD/DXKKxudnc5prl27nDDhQEQhYkQjPKj4AoUDEQByovhOeY3HqC8AsIJAYdQXAFhBoDDqCwCsIFAY9QUAVhAoDL8EACsIlPior2h0dEgxHfTIJwaGwCcIlER00CMXMgUG7zv4BIGSKJsOer5NIlt9fVJtrTR7ttTT4wTGjh0TA4OBIfAJAiVRNh30fJtEtlpbpffeG/uYMRMDg4Eh8AkCJVE2HfR8m0S2Ur03xgcGA0PgE8W39Eo62SzLUlfnVCaxGN8mkV5dnXOqK1F5+cTAYDkg+AQVymT09UlDQ6MVSijEt0mk1tkp1dSM3q+pcfpQQiH32gTkEBXKZLS2jnbCl5Q43zb5cEAqoZDU2+t2K4C8oUKZjKn0nzAqDECRIFAmYyqjcRgVBqBIECiTMZXROIwK8w+qTSAtAmUyEpdp6e7Orv+EOQb+kW21SfCgSBEoucYcA//IttrkNCeKFIGSa1OpauCuVBVGttUmpzlRpAgUYLxUFUa21SanOVGkAsaYdNvTbgR8qazMCZO40lJFFixQV1dXdq/v63NCaNcuJ0w6O6lM4TeBZA8ysREYb7rL67CUCooUp7yA8RhIAUwJFQowHhUGMCVUKG7KNF+B+QwACgiB4qZM8xWYz5AzL7zwgurq6lRTU6P29vaxGwlyYEoY5eWmJKOJFI2m3i5JDQ2MGpqKhJFXZs4cXX3smP6ydauqqqp02WWX6amnnlJDQ4Pz3MbGsZ3y9fWK/M//ZD/KC/C/pKO8qFDclGm+QuL2uJ4e5xrlfHOenIRqz+zcqf87fFihUEjl5eW68cYbtWnTptHnMjERmJK0FcqyZcvMoUOHpvzLBwcHNXPmzCm/3g15bfPJk841x0+ckM46y7kA05lnJt+ezFlnabCign2cje3bJz62aJEk6eOPP9bx48dVXV3tPN7dPWafnwwEtKOkRAsWLMhDQ+3h+MuPYmzz9u3btxhjlk3YYIxJ929aFi1aNN1fkXeebHNDgzElJcZIY/+VlnqzvRnkrc27dzv7rrTUmPLykX04HAiY/eeeO/K0J554wvzgBz9I/rqGBmN27zaf+9zn8tNmi3hv5EeRtjlpZnDKqxDE50UkYkmPifr6pNpaKRBw/tXXO5fcHR6WTp1y+qRKS3Xioov0v3Pnjrxs//79uuCCC0Z/D+uvAVNCoBSC+Afc7t1Op3ymCXfFOkqptdU5RRh36pRTy0nO7fCwFI2qvLdXr/X3a8+ePRoaGtL69eu1fPlyd9oM+EhOA2X16tW5/PU54ek2J/nmnLS9Nocb5yCccraP03WeJ1R0ZWVlevTRR7V06VKFw2GtWrVKjY2NaX/1l7/8ZZstzQtPv5dToM35kbM2pzoXZiz0oSBLSc7ZT+s1paUT+lqmLLH/pqTEue+WTPupoWFiP1N5+djnT2Vfm8I8Tw7kEH0onjWViiLda2wunz7VIbTJKpvpVjuJ/83Jhk93djoj5eJqapw+lMS+ECaLArmTKmkMFUr+TKWiSPeabL+FZ/O8ZBXKVF+XbLTaJKqECf/NU6mapli9UaEAY9ivUDZs2KDGxkaVlJRMmEW8Zs0a1dTUqK6uTlu2bEn6+sOHD6ulpUW1tbVqaWnRkSNHptOcSfv2t7+tBQsWaMGCBbr44otTzjO4+OKLNW/ePC1YsECRSMR+QyZRUfzyl7/UrFmztLusTCNz6Me/JhTSCw8+qLrZs1Vz8qTan346+S/L5tt6spV3s3nduMom2tOjaE/P6GNxCa9PuZ/jlc34VQNO/+5JTTzMsK/TLski5wvYD3/4Q9XU1KipqUlvvfVW9n87B/bt26err75a4XBYjY2Nevjhhyc8Z+vWrZoxY8bIe/1Xv/qVCy0dK9Mx5bX9vGvXrpH9t2DBAp1zzjl66KGHxjzHC/v51ltvVUVFheYmjGLM9nM203s/K6mSxmRRofT09JidO3eaK6+80vz73/8eeby7u9s0NTWZEydOmL6+PhMKhUw0Gp3w+p/97GdmzZo1xhhj1qxZY+65555pReZ03H333eb+++9Puu2iiy4yg4ODufvjkziv/4tf/ML8/ve/H/uamhrn3+nXR99914RCIbN7925z8uRJ09TUZLq7uyf+vSTzWjK2r7w8u9clVCPDgYCJhcPGNDSY4UAg5etT7udU83CmUqGk2dfRaDTlfotXKM8//7xZtmyZicVi5o033jCXX3559n87Bz788EOzfft2Y4wxn3zyiamtrR37/9oY88orr5hvfOMbbjQvpUzHlNf2c6JoNGoqKyvN3r17xzzuhf386quvmu3bt5vGxsaRx7L5nE333k/BfoUSDodVl+Tb9KZNm3TjjTfqzDPP1CWXXKKamhpt27Yt6fPa2tokSW1tbdq4ceN0mjNlxhg9/fTTuummm1z5+1Oa95D4mvJy51v86Yrh5JIlqqmpSb20SLzCSDT+23pif0c4PDqfY2go9esSXzM05LSxtFQl4bACzz0nrV0rU1o6doG4bPp4EqsdyamWshk+nUyafb1t27b0+03Oe/Y73/mOAoGAvvKVr+jo0aMaGBjI/u9bFgwGtXDhQknS2WefrXA4rP7+ftfaY4vX9nOil19+WbNnz9ZFF13kdlMmuOKKK3TeeeeNeSybz9ls3vvZyEmnfH9/vy688MKR+1VVVUnf5AcOHFAwGJTkHBgHDx7MRXMyeu2111RZWana2tqk2wOBgJYsWaJFixZp3bp1eW7dRI8++qiampp06623OuXruNNLZ33wQfr9P/4DWpr4wZx4WmtoaHQ+x3j19dLatU6QzJ7tdJYPDzvhUl4+9oP7+9+XotGxq8rFYs7v7+tLvZ+TnabKwcTDbN632b633bB37169/fbbWrx48YRtb7zxhubPn69rr71W3R641kumY8rL+3n9+vUpv3x6bT9L2X3O2trfGS+w9bWvfU0fffTRhMd//etfa8WKFUlfY5J8+AQCSRenzLls2v/UU0+lrU5ef/11XXDBBTp48KBaWlpUX1+vK664wpU233nnnbrvvvsUCAR033336Sc/+Yn+Mu6StZ8mzvo+bcz+H3+J2/r6iReUShY6qdxxh4bfe0+liY/FYjI9Pfq0ulrnbN0qhUKK7dgx9jlxp1cCTrmf4/02iddoz4Fs3rdeem8nOnbsmK6//no99NBDOuecc8ZsW7hwod5//3194Qtf0ObNm/XNb35Tvb29LrXUkemY8up+Hhoa0rPPPqs1a9ZM2ObF/ZwtW/s7Y6C89NJLk/6lVVVV2rdv38j9CUtbnFZZWamBgQEFg0ENDAyooqJi0n8rk0ztj0ajeuaZZ7Q92eKBp8XbXlFRoW9961vatm1bTgMl231+xx136Lrrrpvwgbv7/vu1709/GnnehP2fzQf0+NCJL6VfWjr2tNfp5yQLioCkc/r7pdZWddxzj7561lm65ORJBcYH1emO9ZT7OU9XUMzmfZvtezufTp06peuvv14333yzVq5cOWF7YsB8/etf11133aVDhw65Olkz0zHlxf0sSX//+9+1cOFCVVZWTtjmxf0sZfc5a2t/5+SU1/Lly7V+/XqdPHlSe/bsUW9vry6//PKkz+vo6JAkdXR0pKx4cumll15SfX29qqqqkm4/fvy4Pv3005Gf//GPf4wZQZFvieeR//a3vzltGdcv0HT6m1HKpUWy6bMZP7orPp/j5Ennsbh4OIxfZj9he2znTv32t7/VjFdfVSD+O8vLx5zGitXWur6fL7vssvT7Tc579oknnpAxRm+++aZmzJgxcjrBDcYY3XbbbQqHw7r77ruTPuejjz4a+Qa6bds2xWIxfelLX8pnM8fI5pjy2n6OS3c2w2v7OS6bz9ls3vtZSdVbb7IY5fXMM8+YWbNmmfLyclNRUWGWLFkysu2BBx4woVDIzJkzx2zevHnk8dtuu21kRNihQ4fMNddcY2pqasw111xjPv7440x/0rq2tjbz2GOPjXmsv7/fXHvttcYYY3bv3m2amppMU1OTaWhoMA888EDe25jolltuMXPnzjXz5s0zra2t5sMPPzTGjG2zMc4omdraWhMKhey3efwck5qasSOnamrGbH/3jDNMVVWVmT9/vpk/f7753ve+Z8zu3WZozhwTDQSMaWgw77/yysT9PMVZ7dORbL899thjprq62hhjTCwWM3fddZcJhUJm7ty5Y0Y3uuG1114zksy8efNG9u/zzz9vHnvssZH39SOPPGIaGhpMU1OTWbx4sXn99dddbXOqYyqxzV7bz8YYc/z4cXPeeeeZo0ePjjzmtf184403mvPPP9+UlZWZWbNmmT//+c8pP2en+ZmRNDO4YiMmL+HqhyOnzBKrnEzbs5Xkyon5OPWVTCQS4YqNwKikHSwECrwr0yWS84hAAcbgEsCYIreWw7e5JhmAnCNQkJlbCyomW/YFgGdlHDYMTHnF4enK03BhAHZQoSAzTj0ByAKBgsw49QQgCwQKMpvM4pXFej17AAQKLOOKiEDRIlBgl1sd+ABcR6DALjrwgaJFoMAuOvCBosU8FNjF3BGgaFGhAACsIFAAAFYQKAAAKwgUAIAVBAoAwAoCBQBgBYECALCCQAEAWEGgAACsIFAAAFYQKAAAKwgUAIAVBAoAwIqAMcbtNgCeFwgEXjDGLHO7HYCXESgAACs45QUAsIJAAQBYQaAAAKwgUAAAVhAoAAAr/h+YBqizQ5Cn6wAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "ax = plt.figure(figsize = (7, 7)).add_subplot()\n", "centerAxes(ax)\n", "quad = np.array([1, 3, 0.5])\n", "xquad = -10.0 + 20.0 * np.random.random(100)\n", "yquad = quad[0] + (quad[1] * xquad) + (quad[2] * xquad * xquad) + np.random.randn(100)\n", "ax.plot(xquad, yquad, 'ro', markersize = 4);" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "As a result, we wish to approximate the data by an equation of the form\n", "\n", "$$y = \\beta_0 + \\beta_1x + \\beta_2x^2.$$\n", "\n", "Let's describe the linear model that produces a \"least squares fit\" of the data by the equation." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "__Solution.__ The ideal relationship is $y = \\beta_0 + \\beta_1x + \\beta_2x^2.$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "Suppose the actual values of the parameters are $\\beta_0, \\beta_1, \\beta_2.$ Then the coordinates of the first data point satisfy the equation\n", "\n", "$$y_1 = \\beta_0 + \\beta_1x_1 + \\beta_2x_1^2 + \\epsilon_1$$\n", "\n", "where $\\epsilon_1$ is the residual error between the observed value $y_1$ and the predicted $y$-value." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "Each data point determines a similar equation:\n", "\n", "$$y_1 = \\beta_0 + \\beta_1x_1 + \\beta_2x_1^2 + \\epsilon_1$$\n", "$$y_2 = \\beta_0 + \\beta_1x_2 + \\beta_2x_2^2 + \\epsilon_2$$\n", "$$\\vdots$$\n", "$$y_n = \\beta_0 + \\beta_1x_n + \\beta_2x_n^2 + \\epsilon_n$$" ] }, { "cell_type": "markdown", "metadata": { "collapsed": true, "slideshow": { "slide_type": "fragment" } }, "source": [ "Clearly, this system can be written as $\\mathbf{y} = X\\mathbf{\\beta} + \\mathbf{\\epsilon}.$\n", "\n", "$$\\begin{bmatrix}y_1\\\\y_2\\\\\\vdots\\\\y_n\\end{bmatrix} = \\begin{bmatrix}1&x_1&x_1^2\\\\1&x_2&x_2^2\\\\\\vdots&\\vdots&\\vdots\\\\1&x_n&x_n^2\\end{bmatrix} \\begin{bmatrix}\\beta_0\\\\\\beta_1\\\\\\beta_2\\end{bmatrix} + \\begin{bmatrix}\\epsilon_1\\\\\\epsilon_2\\\\\\vdots\\\\\\epsilon_n\\end{bmatrix}$$" ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [], "source": [ "#\n", "# Input data are in the vectors xquad and yquad\n", "#\n", "# estimate the parameters of the linear model\n", "#\n", "m = np.shape(xquad)[0]\n", "X = np.array([np.ones(m), xquad, xquad**2]).T\n", "beta = np.linalg.inv(X.T @ X) @ X.T @ yquad" ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "hide_input": true, "slideshow": { "slide_type": "fragment" }, "tags": [ "hide-input" ] }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAADxCAYAAAAwXvePAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAq20lEQVR4nO3deXiTVdr48e+ThLIJKEJr2QmtSZtSqlQR3xFHpQ6oFZXXhUFFQVHR0Z+KDK+Ojjqj4LzjCAqj9hWUQcdtXDCyjQuOijq1KCOENmADIlBZBNQC0qY5vz9O96Z7mifL/bmuXtmeJnefJndPz3IfQymFEEKI2GIxOwAhhBChJ8ldCCFikCR3IYSIQZLchRAiBklyF0KIGCTJXQghYpAkdxE1DMMYaBjGGsMwCg3D8BiGcVvl/b0Nw3jHMIwtlZfHmR2rEGYzZJ67iBaGYSQDyUqpLwzD6AGsAy4CrgH2K6XmGoYxGzhOKfVb8yIVwnzSchdRQylVopT6ovL6T0Ah0B+YACypPGwJOuELEdc6NLmrdJeqMKwKw1DKYlW4XAqQL/lq99e2bdvUwIEDL/zhhx/+2atXrxOVUrsApZTadeyxx54Y7Hvy8vJUdna2ys7OVq54fS/6fGprN5fyGzZ1cIBL4fOZH5N81f8KiY7tlrHZFBUVNbetVvD7O+71RFwoLS3lzDPP5J577uGSSy7h2GOP5eDBg9WPH3fccRw4cKDJ58jOzqagoKCDI408ZSe6sG4pwkoAZbFgOJ3g8ZgdlqjLCMWTdGy3jMNBwNAvUYEF5XB06MuJ2FdeXs7EiROZPHkyl1xyCQBJSUmUlJQAUFJSQmJiopkhRjTb116sBAAwAgHwek2OSHSUjk3ubjdGmhM/Vopw8uUD7g59ORHblFJMmzaNtLQ07rjjjur7L7zwQpYs0V3uS5YsYcKECWaFGPG2JjioqPrYWywgDa6Y1dGzZRQ+H9+dkkuf/V729naQ/Lkb7PaOfE0Roz7++GPOOOMMhg8fjsWiE9TDDz/MqFGjuOyyy9i+fTuDBg3i1VdfpXfv3k0+Vzx2yxQWwvnpPpZbcnEaXgyHA9zyeYxAIemW6fjk7nIRKCzCogJUYMGS5sTYJH18wlzxmNzvuw/+8Ae49lpYvNjsaEQToqDPHcDrxaJ0H58V6eMTwgxKwYsv6uuTJpkbiwiPjk/uDofu20MPqpb0kj4+IcJt3Tr4+mtITISzzjI7GhEOHZ/c3W5wOlFWPah6IW6ZDSlEmFW12i+/HGw2c2MR4dHxyd1u1/Noy/38t9PDugN23n+/w19VCFEpEICXX9bXpUsmfoSt/IBhwI3n+tiIi3PG2cDlAp8vXC8vRNz66CPYuROGDIHTTjM7GhEuYa0tc8PyXJwUYVUVqMJCSEvT/yN27qxXr0rCFyLkqrpkrrhCN7JEfAhrcu+yrdbqOKWgrAwqKvRlIABFRZCbG86QhIhp5eXwj3/o69IlE1/CWxWyVjmCoLPrZTm0ECH1zjvw/feQng7Dh5sdjQin8CZ3t5uKFF2O4CgJKEu9l5fl0EKE1Lt5epxrQ6ENI0O6PeNJeJO73U6nzR7OPctPOoUcTHLqvvaEBJ3YnU49dVII0T4+H4E0F48uG0Y6m7CoCun2jDOmbNZx1VWwFTsXpXp0CeCjR3Xfu8cjdS6ECIXcXPAWYVBrLbt0e8YVU5L7xInQtSt8+CFs22ZGBELEuFplP6pJt2dcMSW59+wJF12kr7/wghkRCBHbKlJqSvtWT16Qbs+4YtoeqlddpS+XLtVFjYQQIeDzgcuFsbmIcmxUYMVIT4fiYun2jDOmJfecHEhK0l2An39uVhRCxJjcXCjSJbY74een/g5J6nHKtORus9Usqli61KwohIgxXq8eOEWX2O71nQygxivTkjvUdM289JJeSSeEaKd6+xYbMoAat0xN7iedpFfO7dsHq1ZR3V+ITQqLCdEWgWVuvrbqhYI/D5YB1HhmanI3DLj6an196VKq+wupkAUXQrTFB9vtjPO78XVy0G2HV3+GpJEUl0xN7gCTJ+sk/9ZboGr1F8qCCyFab8kScJNLir8IQxpJcc305D5ggN726+hROJhYsyWfLLgQonVKS+G118BBrQVM0kiKW6Ynd6gZWL1pgN6SD6tVFlwI0Uqrn/Tx70MurFTU3CmNpLhlqI5dQdSiJ//pJz3n/cgR2LpV7xgjREfKzs6moKDA7DBCalt3FwMPF1XvmQDoGQtut8xzjy4h2VIlIlruPXrUlCN4/nlTQxEiKn3zDQw47K2b2K1WWcAUxyIiuQPckKPrTs++14aSaZBCtMrSpeClpp6MdMeIiEnuY/5X769qowJVKCP8QrSUUvC3v0Eubg4PkjErodnMDqCKsbnmX0qLkhF+IVrq009hyxZITrbTzecBq9kRiUgQMS13HI7qbfcqsFCRKv9SirqmTp1KYmIiGRkZ1ffdf//99O/fn6ysLLKyslixYoWJEZpjyRJ9eeWVutEuBERScne7MZxOKrBShJPXr5F/KUVd11xzDatWrWpw/+2338769etZv3495513ngmRmefIEXj5ZX19yhRzYxGRJXKSu90OHg9/W+wnAw/z3TLCL+oaM2YMvXv3NjuMiOJ2ww8/wMiRuhyTEFUiJ7lXuvRSOOYYWLsWCgvNjkZEgwULFpCZmcnUqVM5cOCA2eGE1XPP6UtptYv6Ii65H3NMTZ33xYvNjUVEvptuuoni4mLWr19PcnIyd955Z6PH5uXlkZ2dTXZ2Nnv37g1jlB1j+3ZYvRoSEmo+M0JUibjkDjBtmr5csgTKysyNRUS2pKQkrFYrFouF66+/nvz8/EaPnT59OgUFBRQUFNC3b98wRtkxFi/WpWMuuQT69DE7GhFpIjK5n3oqZGTA3r3w9ttmRyMiWUlJSfX1N954o85MmlhWUQGLFunr119vbiwiMkVkcjeMmtb7M8+YG4uIHJMmTWL06NF4vV4GDBjAokWLmDVrFsOHDyczM5M1a9bw2GOPmR1mWKxaBTt2QEoK/PKXZkcjIlFEFA4LZt8+6N8f/H7Ytg0GDgxhVCLuRXvhsIsugmXLYO5c+O1vzY5GhFjsFA4Lpk8fmD7Wx1cBF/0Hy7Z7QlTZtUt3V9pscM01ZkcjIlXEJneAOR5db8aiKlCyo4wQADz7rO5znzBBl8oWIpiIqS0TTPcdXozKejOG7CgjBIFAzTjU9OnmxiIiW0S33A2Hg4BRU29GSpiKePfuu3oMasgQGDvW7GhEJIvo5I7bjT/FiR8rXpwc+JvUmxHxLS9PX153HVi2+fRYlE3GpERDkZ3c7XYSNnvIHefHhYela6XejIhfu3frGTJWK1x7LXoMqqhId8DLmJSoJ/KSu69ha6T2nPeOnbkpRISp9XmwZroY6Pdx/vnQrx96DCpQua2ejEmJeiIvuQdpjVx4IfTtCxs2wGefmR2gEGFU6/Nw3J4i3OTWDKQ6HHo7PZBt9UQDkZfcg7RGEhJqVqwuXGheaEKEXa3Pg5UADryMG1f5mNutt9OTbfVEEJGX3Btpjdx4oy5L8OqrsGePifEJEU61Pg8VWNjf11Gz21LlHgj4/frSLmNSokbkJfdGWiODB8O0s3x8UeaizwkyO0DECbcbf6qeMVaEk/LXpXUuWiZia8sEUzrYRdftRVgJoCwWDKdTt1iEaKVoqi3zl7/AnXfCeefB8uVmRyPCICS1ZSJ6hWp93XfKilURXwIB+Otf9XVZkSpaI/K6ZZogK1ZFvFm5EoqLdbfkBReYHY2IJlGV3HG7USfW9D9ufVz6H0Vse+IJfTljBjUDqUK0QHQld7sda5GHG6f5ycDDfLfMDhAxIsjiPa9X75HapUvlVOAgxwjRmOhK7pVuvllfPvccHDpkaihChEaQxXtVazomT4bjjw9+jBCNicrkftJJMHo0/PADvPCC2dEIEQL1Fu8pr5fnntM3f/Ob4MfIhALRlKhM7lDTel+4UOrNiBhQb/He/r4OfvoJzjgDRowIfoxMKBBNidrk/t//revNfPUVfPKJ2dEI0U61Fu8pp5PLu+rJArfeGvwYKTcgmhO1yb1zZ7j+ehiKj8HnySCTiHK1Sgl8PMvN/K25lGNj4n213tNSbkC0QlStUK1v+3b4abALJ3rVKhaLbtHIqlXRjEheobq9h4v+pfKejmPxt0K1vkGDoMLwYlUyyCRig88Hg0q9OrGDvKdFm0Vtt0yVI4McerUqoGSQSUS5hQvBS817WgZORVtFfXLv9q6brQl61eoPyTLIJKJXaSksWgS5uCmzy8CpaJ+oT+6WFDsfLPTQCT9n9fWghsogk4hOzz+v126cMNpO12IZOBXtE/XJHeDKKyExEdavhzVrzI5GiNZTChYs0NerFy0J0Q4xkdy7dIFbbtHXH33U3FiEaIs1a3Qj/YQTYOJEs6MRsSAmkjvATTdB166wYgVs2mR2NKIjTJ06lcTERDIyMqrv279/Pzk5OaSmppKTk8OBAwdMjLDtqholN90ECQnmxiJiQ8wk9z59YMoUff0vfzE3FtExrrnmGlatWlXnvrlz53LOOeewZcsWzjnnHObOnWtSdG23YYNulHTtqkv7ChEKMZPcAW6/XW+ivXQp7N5tdjQi1MaMGUPv3r3r3Lds2TKmVP5VnzJlCm+++aYJkbXPn/+sL6dN040UIUIhNpJ7ZZ3rE9NtbDvGRf8yX3W5VBHbdu/eTXJyMgDJycns2bOn0WPz8vLIzs4mOzubvXv3hivEJn37Lfz973o6+x13mB2NiCWxkdxr1bkeUFqEm1z++lc4fNjswEQkmT59OgUFBRQUFNC3b1+zwwFg3jw94/Gyy2DoULOjEbEkNpJ7rTrXFhXAiZfvv4clS0yOS3S4pKQkSkpKACgpKSExMdHkiFru4EHIy9PX77rL1FBEDIqN5F6vznXpAL1c+7HHavY2ELHpwgsvZEnlX/ElS5YwYcIEkyNquaee0qtSzzkHTj7Z7GhErImN5F6vznX399wMHgxbtsjK7VgyadIkRo8ejdfrZcCAASxatIjZs2fzzjvvkJqayjvvvMPs2bPNDrNFfv4Z5s/X12fNMjcWEZuiuuRvU+bN07NnzjgDPvzQrChEpDK75O8zz+j9CEaMgC+/1LO8hKgUkndDbLTcg5g2DXr1go8+gvx8s6MRokYgAP/7v/r6rFmS2EXHiNnk3qMHTJ+ur1d9kISIBG+9BZs3w+DBcOmlZkcjYlXMJneA227TS7lfew0KC82ORghdIOyRR/T1O+6ATp3MjUfErphO7v37w7XX6g/UQw+ZHY0QsHYtfPYZ9O6tuw6F6CgxndwBZs/We2e/+KKePSOEmf70J305YwZ0361XVsvm7qIjxHxyHzIErrpKD2LNmWN2NCKeeTx6am7nzpU122utrKaoSN8WIkRiPrkD3H03DDN8zHzWhZJWkjDJH/+oL6dO1ZvL1F5ZLRthi1CLi+SekgIf9MzFQRGGtJKECTweePllPcD/P/9TeWe9ldWyEbYIpbhI7gD9S71YkVaSMMcf/qAH9q+7DgYOrLyz3spqWU4tQslmdgDhYjgcVGwqwkqAgGHBIq0kESYeD7zyim6116mOYLfrB4XoAHHTcsftptzuxI+VIpzsWSStJBEeQVvtQnSw+Enudjtdij1cepEfl/LwyKt2syMScaB2q726r12IMIif5F7pd7/Tl089BRGyGY+IYQ8+qFvt118PAwaYHY2IJ3GX3EeOhPPP17s0yUbaoiNt3Aivvhqkr12IMIi75A5w7736csEC2L/f3FhE7Krqa5dWuzBDXCb3UaMgJ0fvgvPYY2ZHI2JR7Va79LULM8Rlcge4/359+dhjsHu3qaGIGFTV1z59ui5gJ0S4xWdy9/k4/XoXfsPGvw+5+OtMKUUgQqeq1d65s/S1C/PEZ3KvLNhkVRU4KeLy53MpLjY7KBErHnxQX0qrXZgpPpN7rYJNVgKciLd6kFWI9vjyy5pW+29/a3Y0Ip7FZ3KvVbBJWSxsxsGLL+oPphBtpRTcdZe+fvPN0moX5orP5F6rYJPhdPL6tboUgcxqEO2xejW89x4ceyzcc4/Z0Yh4F5/Jvapgk98PHg83/slOz576w7lmjdnBiWhUUVHTav/d7/Q2ekKYKT6Tez19+tR8MGfP1v9eC9EaS5boWTKDB+suGSHMJsm90u23Q1IS5OfD66+bHY2IJocP16x6fvhh6NLF3HiEAEnu1bp3h/vu09fvuUf32AjREo89Brt26bpFV1xhdjRCaJLcfTU70N+4wMWZA314vfDss2YHJqLBnj3wyCP6+hO3+7AM1+8l2adXmM1QHdvBHPm91y6X3lM1EACLhR+SnRy700O/frBlC3TrZnaAoiWGDBlCjx49sFqt2Gw2CgoKmjw+Ozu72WNa4pZbYOFCXWn07a1130s4nbLTkmgLIxRPIi33ejvQ9/zOy0kn6X+zpSRwdFmzZg3r168PSdIOqtZ/ebhcbH3Px9NP6zz+yCM0eC/JPr3CTJLc6+1AbzgcPPqovvnww7B9u3mhiQji80FaGmzapOc9FhZivTgXvx+mTtU5v/57CdmnV5hIknuQHejPOgsuvRSOHIE/3Vi3tSb9qJHJMAzOPfdcRo4cSV5eXtBj8vLyyM7OJjs7m72t3YbrV7+CsrKa20rR7ycv3brV1JIJ9l4SwizS596I7dv15/PzIy7SjCIsSvpRI9muXbvo168fe/bsIScnhyeeeIIxY8Y0enyr+9yNut2gCthEOq/c5+GBB9oYtBDBSZ97Rxo0CO6+Gxx4dWIH6UeNYP369QMgMTGRiy++mPz8/A5/zal93Myc2eEvI0SbSHJvwsyZ4OvkoALpR41khw4d4qeffqq+/s9//pOMjIzQvkhKSvVVBWwhhdvm2+nRI7QvI0SoSHJvQpcusH2BmyKc+LHiT5F+1Ei0e/dufvGLXzBixAhOPfVUzj//fMaNGxfaF1m9GtLTqTCsbCKdB05bzaRJoX0JIUJJ+tyboRScdx6sWqVnRSxaZHZEIhTaMs/9s8/g9NP12Pp//qMnzwjRAaTPPRwMA+bPh06dYPFiXXtGxJ+KCpgxQ/+xv/NOSewi8klyb0rlopUT021829PFUHzcckvNOhURP558Um/mMmiQLukrRKST5N6Uyr1Wqagg8UARK225fP45PPec2YGJcPruu5rNN+bP10XmhIh0ktybUms5uREIkBrQ0yBnz4YDB8wMTITTzJnw44+6fsyECWZHI0TLSHJvSv3SBE4HZ5wBe/fqflcR+9asgRde0DOnHn+8wVomISKWJPem1FtObrjd5OXpne2ffVbPoKnDJ6UKYklZWc2uSnffrXdnFCJayFTINnjkEd01M3Cg3lqtZ8/KB1xS8jVatGQq5Ny5etP01FT46ivZYUmEjUyFNMudd0J2Nnz7LcyaVesBKfkaMzZtgvvv19cXLJDELqKPJPc2sNl0t0ynTvD00/D++5UPSMnXmFBeDlOmwNGjcO21cO65ZkckROtJcm+jjIyaTZGvuw5KS5GSr9GmkTGSuXOhoEDPaZ83z9wQhWgr6XNvh/JyOPVUWL8ebr1Vz4EW0SE7O5uCI0cajJF8sdTDqFF6g/T33oOzzzY7UhGHpM/dbFUlCWw2eOIJ+PhjsyMSrVJvjER5vVx9tU7sv/lNZWKXGVAiSklyb6eTTtIzZ5TShcWOHDE7ItFi9cZI9hzrwOPRs2Pmzq08ptYqZYqK9G0hooAk9xD43e90o27LFrjvPrOjES1Wa4zk8CAn/7XfjcUCS5ZAt26Vx8gMKBGlJLmHQNWiJosFHn201uwZEdnsdvB4OPSDnxE2D8XKzqxZMHp0rWNkBpSIUpLcQ+SUU3RxKaVg8mTYvdvsiES1ZvrNZ8+Gr7+G4cNr5rZXkxlQIkrJbJkQ8vvhnHPgww8hJ0eXJ7DIn0/zBVk5nN21KwUFBbz7rv5d2Wzw+eeQlWV2sELIbJmIY7PB3/8OffrAO+/UGpQT5mqk33zXLv1fFsDvfy+JXcQWabl3gFWrYPx43Uj84AM44wyzI4pzQVruI7t0pWvXAtau1f9trV6te16EiADSco9U48bpftxAACZNgn37zI4ozgXpN9+xA9auhf799X9bkthFrJGWewcpL4df/hI++US34t9+W/rfI8Wrr8Jll2VjsxXw4Yf1ZscIYT5puUeyTp3gpZegd29YuVJPkRTmKyrSi81A/04ksYtYJS33Dvb223pRo9UKH30kycRMpaUwapQu53vccdl8/32B7KwkIpG03KPBBRfo+u8VFXDppXqGhgg/pWD6dJ3Y09JgyBDZMk/ENknuYfDww/CLX8DOnTrZl5YGOUgKVHWohQvhxRehe3d47TUZ/xCxT97iYZCQAG+8AcOGwZdfwq9/rVvydUiBqg7z8cdwxx36+qJFuuUuRKyT5B4mfX704TFclGNjjtvFnOvrtcylQFWH2LQJLrxQz1667Ta4/HKzIxIiPCS5h0tuLp19RdiowEkRFz+by8KFtR6XAlXtsmrVKhwOBykpKcytXBq8c6dec3DsAR/fHOPisQXS5SXih8yWCRebrU5fjB8rnS1+3G447zx0wsnN1S12h0MvvLHbzYs3GlSeM+X1ssViocs//8kJp5/OKaecwtNPv8z06U42bIDiri6GHi3CCFJbRogIJLNlokq9lvn+Pg4CAd1N8J//oBO5262P83p1opcWZtMqxymMigqGlZcz6OabSUhIYOLEyfz6113ZsEEvSB1a5tWJHaTLS8SNDm25u1wu1bVr1w57/lDYu3cvffv27fgXOnpU15X9+Wfo0gVSUti6qzP79+sFT2lp0GmzRz9epUsX3Y0QzjjbKaxxrlvX8L6RIyksLOPw4QQ6ddLJPWFL3fN61DAotFjIioJKYfJ7D61oiHPdunUepVRGu59IKdVhXyNHjlSRzswYjxxR6r/+SylQyuVSKmC16hu1v9LTlSouNjXO1ghrnOnpSlksSoHygwqkp6tbb9WnLSHhsFq/vvK44mJ9rNVafT67desWvjjbQX7voRUNcQIFKgT5V7plTNSli54imZYGHg8UWx2o+hOwZVpk4yoLgimLhW+7dyfvAjePPw5Wq5+rrnqdESMqj6vccQm/X1/KWIaIA5LcTda3L6xZA+npcG6Zm2Kbs+4otPQRN64yaVccPcrIzndx45900u7f/25uvz3L3NiEMFmHJvfp06d35NOHRCTEmJSkE3w3l53UMg/FCek1LfjKaZGREGdLhDtOpWDOHBv79/8egOOPv5/p03vhqhyraEyfPn3CEV67ye89tKIkzrxQPIlMhYwge/fqjSNKN/hYnZBLSoUXI96nRfp88Ktf6cFogMGD9Qj01q0oh4M5o93cs8iOYcD//R9Mm9ayp83OzpapkCJShWQqpCT3CLNvn07wX30Fqam6Rd+/v9lRmcjl0stMgwhgoRAnWTYPL7wAl13W8qeV5C4imMxzj0V9+sD77+v9PLds0Rt+bN/eyMHxUGysifEGCwEceFm2rHWJXYh40K7kbhjGpYZheAzDCBiGkV3vsf9JSUnB4XCwevXqoN+/f/9+cnJySE1NJScnhwMHDrQnnBa5/PLLycrKIisriyFDhjQ613nIkCEMHz6crKwssrOzgx7TUY4/HnJyHqFTpw18/TUMG7aPP//5k4YH5uYSKCyEigoqNm1irwnF4u+66y6cTieZmZlcfPHFHDx4MOhxbT6f9cowKGr+HazAwtHBDr3CtxHByhLUeT6luPXWW0lJSSEzM5Mvvvii5bGFyLfffstZZ51FWloaLpeL+fPnNzjmgw8+oFevXtXv3QcffDDscTb3O4yEc+n1eqvPUVZWFj179mTevHl1jjHrXE6dOpXExEQyMmqmsLc0BxqGMc4wDK9hGF8bhjG7RS/YnnmUQBrgAD4Asmvdnw785+eff1Y+n0/Z7Xbl9/sbzOe866671Jw5c5RSSs2ZM0fNmjUrdJNFW+COO+5QDzzwQNDHBg8erPbu3RvWeGr7/e9/rx544Al11llV87aVWrSo7jH158WXg/J4PGGNc/Xq1aq8vFwppdSsWbMa/R22+XwWFyuVkqIUqACoXZ0HKy8pqhyrOjxUz1lvjN/vV3a7XRUXF6ujR4+qzMzM6vNTNd95+fLlaty4cSoQCKhPP/1UnXrqqa2PsZ127dql1q1bp5RS6scff1SpqakNfo9r1qxR559/fthjq62532EknMva/H6/SkpKUtu2batzv1nn8l//+pdat26dcrlc1fc1kgPr51krUAzYgQTgP0B6/ePqf7Wr5a6UKlRKBfu/eQLwUufOnRk6dCgpKSnk5+c3OGjZsmVMmTIFgClTpvDmm2+2J5xWUUrxyiuvMGnSpLC9Zmt16/Yzq1fDLbdAWZkeLLztNj1dG+DIwIFUV6uxWDiQmMiyZcta9yLt7No599xzsdlsAJx22mns2LGjda/fHLsdtmzhk7WKgf0V/Y5u45fJW/Bu9NPV1/Sc9fz8fFJSUrDb7SQkJHDFFVc0OD/Lli3j6quvxjAMTjvtNA4ePEhJSUlof4ZmJCcnc/LJJwPQo0cP0tLS2LlzZ1hjCIVIOJe1vffeewwbNozBgwebFkNtY8aMoXfv3nXua2EOPBX4WinlU0qVAS+hc2yTOqrPvT/wbdWNAQMGBH2z7t69m+TkZEC/wffs2dNB4TT00UcfkZSURGpqatDHDcPg3HPPZeTIkeTlhWRmktaKZLpgwQJGjszk0KGpzJt3iE6d4PHH9eSR77+Hf82cyXfHHqv38HM6+XjWrNYnhRDWkV+8eDHjx48P+lhbz6dSMG8enHmmrvJ4+umQn19dlaFJO3fuZODAgdW3g70PW3JMOG3bto0vv/ySUaNGNXjs008/ZcSIEYwfPx6PxxP22Jr7HUbauXzppZcabbyZfS6rtDAH1smnwI7K+5pka+4AwzDeBU4I8tA9SqnGmokNRnuNMO5pNnbsWL777rsG9z/00ENMmKD/4L344otNttrXrl1Lv3792LNnDzk5OTidTsaMGdP+4KqSaSBAxaZNbE1P56KUlAZx3nTTTdx7770YW7eyd/Ro+jz3HNcNTuOsn9y8/76dU0+FGTNSeG3iRJ555hkASpcuxdi2rXXxtKCOfEvO50MPPYTNZmPy5MlBX6bJ89lIRcwff9SbWb/2mj7sjjtg7lw9E7KORr5fBZkJVv992JJjwqW0tJSJEycyb948evbsWeexk08+mW+++YZjjjmGFStWcNFFF7Fly5awxtfcZyKSzmVZWRlvvfUWc+bMafBYJJzLVgp2EpudidhscldKjW1DMDuA6j/hO3bsoF+/fg0OSkpKoqSkhOTkZEpKSkhMTGzDSzX07rvvNvm43+/n9ddfZ12wwlOVquJNTEzk4osvJj8/PzTJvVYytQIpfj8bN25s/PgJE0jcvx9DKbpvL2LtsFxOG+zhiy/gvvvGMmRIIUrp/UAbO89Ncjiq/9g0Vkc+6PmsSqgTJ3IgKYkvEhN5Ye3aRj/MTZ7PWn/wqjY5LXqjkAv/n50tW6BnT1i8GCZObORnqP39Vf99eDwMGDCAb7+tafAEOz8tOSYcysvLmThxIpMnT+aSSy5p8HjtZH/eeecxY8YM9u3bF9bFWM19JiLlXAKsXLmSk08+maSkpAaPRcK5rNLCHFgnnwIDgOZ3Y26uU74lXzQcUHVRa0B16NChQQdUZ86cWWcw4a677mrPeEWLrVy5Uo0ZM6bRx0tLS9WPP/5YfX306NFq5cqVoXnxWsWulMWibwexa9cufaV+MTGrVR06pNSkSTV3jRp1WG3YoAcMN27c2Lp4ghTVCnpfEz+HH1T5kCGNfk+z57PezxgA5THSFSiVmanU5s3N/AxBzpFSSpWXl6uhQ4cqn89XPaBadX6qBlTffvvtOoOAp5xySuvOXwgEAgF11VVXqdtuu63RY0pKSlQgEFBKKfXvf/9bDRw4sPp2OLTkMxEJ57LK5ZdfrhYvXhz0MTPP5datW+sMqDaSA+vnVxvgA4ZSM6Dqqn9cg+9r7oAmvxkurvyrchTYDayu9dg9drtdnXjiiWrFihXVP8y0adPU559/rpRSat++ferss89WKSkp6uyzz1bff/99qM9lUFOmTFFPPvlknft27typxo8fr5RSqri4WGVmZqrMzEyVnp6u/vjHP4buxVuSOJVSV155pcrIyFBbOndW/qqkZbGoshNPVOPHj1eBgFIvvqhUr14/K1DKMH5WOTnvq7KyEMTYkj9AQRKyv97tbd27K1Vc3Pz5TE+vm5xBlWNVU6cqdfhw++Jdvny5Sk1NVXa7vfp1n3zySTVo0CCllE6sM2bMUHa7XWVkZFS/N8Ppo48+UoAaPny4GjFihBoxYoRavny5evLJJ6vfp0888YRKT09XmZmZatSoUWrt2rVhjbGx32HtGCPhXCql1KFDh1Tv3r3VwYMHq++LhHN5xRVXqBNOOEHZbDbVv39/9cwzzzSWAwH6AStUTT49D9iMnjVzj2pJfm7JQe34Eu3VzB+DffuUmjKlJi9mZiqVn9+y721UIy3hOuon1Pqlipv5z6TOz5BfrMosCSpQ+X1+DHWwf/PfV60NP2c0lH4VcSsk+VfKD8SId96BG26ArVt11/lvfgN/XunC9nWt/nSnU5e8bY7LVbcfPtj31R/ELCvT91UNzlaxWmvmbtbj3+zjhzNz6fWdFx9DMQC7sRUcDqzLO7aejpQfEBFMyg/EnSamUebkwIYNcOed+vZb831YN29qdiZM0OdduFAn9MoplrjdDb+nfo301av1sbU1MkBbUQFvvw3bMnM59ju9afgwfAwaloA14MdaKDXXhWgvablHk5a0qIEvvoA+Z7oYWLqpugmggJ/t6XQtDtJyr/28VdLT21aNsomNvvfu1bNennoKtm2DcmzYapZhNdnKDzVpuYsIJi33uNOCOekAJ58Mg454G7xDXD43OTmwfHndrVrrPG+V+guaWrr4ql6LXg218+mncNVVMGAAzJ6tE/vQoXAwydGgbr0QIjQkuUcTh0MnQWg+GdY6Vlks7O6dzt5j7Lz7LlxwAfTuDdPO9rE/2YWqqGj4/fX/eDS3krVW8g+kufjkeR8PPKD/0Jx+Ojz/PJSX69desUKXZ+/ziRujue4fIUSbSLdMNGmiy6Mlxx44zs5TT8E//qG7bjbiwkkRVgJ1flEG+g8CDifGpspuHJtNJ/YqlV0oSuka9J1HujhmRxEWFaACg3I6YaMCLw6mHOcm5wY706frFnskkG4ZEcFksw7RdiUlkDTAhiVQk7D9WPHiwEER/sr+cJ/NwePOhTy26VfYAmU68Vce77Wkk4ubrwP2Bv3nCv0ODRh6bMCyybz6HcFIchcRTPrcRdslJ4PFWdN1EzAsfH+8g99d5OGbLk464dezWPxF/Hnjr7AEyuskdgNIDRTxZiCX7t3h224OncipSewAFhXAslk2+BYi3CS5xzO3u3rKoyXNSVK+mzfegGHlXqxU1b8J0JkyrJVp3aAmcVsJkG71UloKQze4saTp5zISElo+NiCE6BCS3ONZ/bnqVf339QZu6yTr2iwWvYF3/ecqLGx+nrwQokNJchcN1WrR43TWLFCyWiElRX+1ZoGTLEgSIuyaLfkr4lBVcq7NxA0NhBCtJy13EX7t3NpPCNE8Se4i/EK4tZ8QIjhJ7iL8WlhGQQjRdpLcRfi1poyCEKJNJLmL8Ks/G6cdUyXvv/9++vfvT1ZWFllZWaxYsSKEgQoRvWS2jGhdzZpQCDYbpx1uv/12Zs6cGbLnEyIWSMtdyACnEDFIkruI+gHOBQsWkJmZydSpUzlw4IDZ4QgREaQqpGjxDk9mGTt2LN99912D+x966CFOO+00+vTpg2EY3HvvvZSUlLB48eKgz5OXl0deXh4Ae/fu5ZtvvunQuIVoIyn5K0Ik3H3uHWTbtm1ccMEFbNy4sdljpeSviGAhSe4yoCpCPsAZTiUlJSQnJwPwxhtvkJGRYXJEQkQGSe4iqs2aNYv169djGAZDhgzh6aefNjskISKCJHcR1ZYuXWp2CEJEJJktI4QQMUiSu2g5qeYoRNSQ5C5aThY7CRE1JLmLlovyxU5CxBNJ7qLlpJqjEFFDkrtouRBWcxRCdCyZCilaLooXOwkRb6TlLoQQMUiSuxBCxCBJ7kIIEYMkuQshRAyS5C6EEDFIkrsQQsQgSe5CCBGDJLkLIUQMkuQuhBAxSJK7EELEIEnuQggRgyS5CyFEDJLkLoQQMUiSuxBCxCBJ7kIIEYMkuQshRAyS5C6EEDFIkrsQQsQgSe5CCBGDJLkLIUQMkuQuhBAxSJK7iAqvvvoqLpcLi8VCQUFBncfmzJlDSkoKDoeD1atXmxShEJHFZnYAQrRERkYGr7/+OjfccEOd+zdt2sRLL72Ex+Nh165djB07ls2bN2O1Wk2KVIjIIC13ERXS0tJwOBwN7l+2bBlXXHEFnTt3ZujQoaSkpJCfn29ChEJEFkMpZXYMQrSYYRgfADOVUgWVtxcAnymlnq+8vQhYqZT6R5DvnQ5Mr7zZRSmVEZ6ohQg/6ZYREcMwjHeBE4I8dI9Sallj3xbkvqAtFqVUHpDXxvCEiCqS3EXEUEqNbcO37QAG1ro9ANgVmoiEiF7S5y6i3VvAFYZhdDYMYyiQCkinu4h7ktxFVDAM42LDMHYAo4HlhmGsBlBKeYBXgE3AKuBmpVSFeZEKERlkQFUIIWKQtNyFECIGSXIXQogYJMldCCFikCR3IYSIQZLchRAiBklyF0KIGCTJXQghYtD/B8sPDbXe0V4dAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "#\n", "# plot the results\n", "#\n", "ax = ut.plotSetup(-10, 10, -10, 20)\n", "ut.centerAxes(ax)\n", "xplot = np.linspace(-10, 10, 50)\n", "yestplot = beta[0] + beta[1] * xplot + beta[2] * xplot**2\n", "ax.plot(xplot, yestplot, 'b-', lw=2)\n", "ax.plot(xquad, yquad, 'ro', markersize=4);" ] }, { "cell_type": "code", "execution_count": 11, "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [], "source": [ "#\n", "# Input data are in the vectors xlog and ylog\n", "#\n", "# estimate the parameters of the linear model\n", "#\n", "m = np.shape(xlog)[0]\n", "X = np.array([np.ones(m), np.log(xlog)]).T\n", "beta = np.linalg.inv(X.T @ X) @ X.T @ ylog" ] }, { "cell_type": "code", "execution_count": 12, "metadata": { "hide_input": true, "slideshow": { "slide_type": "fragment" }, "tags": [ "hide-input" ] }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAADxCAYAAAAwXvePAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAjsklEQVR4nO3dfXRTVb438O9JShBEQARKoWIJqWmTUnppEcY7wyBS3iQgukZhdNRVBljoXK/vz8x4XaKjC+eZmTv4iNdFx3EG7+PIg/eCWEWqeEUdxKlFmaWFltJQx0IVCnRABtsm2c8fO2nS9LR56UlOcvL9rNWVJjk5/fW0/WZ3n733UYQQICIiYzHpXQAREWmP4U5EZEAMdyIiA2K4ExEZEMOdiMiAGO5ERAbEcKe0oSjKC4qinFAU5fOQx9YpinJMUZQD/o9FetZIlCoY7pRO/ghggcrjvxVClPg/dia5JqKUxHCntCGEeB/Aab3rIEoHSoJnqHL6K2mqubkZixcvxuefy56ZdevW4Y9//COGDx+OsrIy/OY3v8Gll16q+trKykpUVlYCAC5cuIC6urqk1U0UA0WLnbDlTmlt7dq1aGpqwoEDB5CTk4P777+/z21Xr16N2tpa1NbWYsiQIUmskij5GO6U1rKzs2E2m2EymbBq1SrU1NToXRJRSmC4U1prbW3t/nz79u0oKirSsRqi1JGldwFE0VqxYgX27NmDtrY25Obm4rHHHsOePXtw4MABKIqCvLw8bNq0Se8yiVICT6hSRiorK0Ntba3eZRCp4QlVIiJSx3AnIjIghjsRkQEx3ImIDIjhTkRkQAx3IiIDYrgTERkQw52IyIAY7kREBsRwJyIyIIY7EZEBMdyJiAyI4U5EZEAMdyIiA2K4ExEZEMOdiCgVuN2A0wkoikeL3THciYhSgcsF1NcDgFmL3THciYj0EmitZ2UBBw8CPp9mu+Y1VImI9BJorWsY6gFsuRMR6aWhQS3YvVrsmuFORKQXux0w+WPYZAIcDkAITXpUGO5ERHqpqgIKCgCzWd5WVWm2a/a5ExHpxWoF6uoSsmu23ImIDIjhTkRkQAx3IiI9hI5xdzrlfQ0x3InIuBIcoAP62vPny4lLXq+8nT9f0y/PcCci4wpMEvJ65a3LlfivGQj1yZOD4X3oEFBY2DPojxzp+brw+wPE0TJEZFyhk4R8Pnk/0YJrxAQJAXR2ys/r6zVvpathy52IjCt8kpDdnvivqT7rNMjnU2+l22yalsFwJyLjSuAkoT6FvqEEWCyRX1ddrWkZDHciMq7AJCGPR95arYn/mqFvKA4H0NQk+9z7Elh2QOPaGO5ElDm0Gj3T337U3lCsVhngoV1EFktC/6NguBNR5tBq9Ew8+wnvIjp0KKH/UTDciSi59Bx7rtXomXj2k+QuIoY7pY2KigqMHTsWRUVF3Y+dPn0a5eXlyM/PR3l5Oc6cOaNjhRQVPcaeB2g1eiZ8P5MmRf+GFfbm5m1047PPgN/9Dli5Mr5y1DDcKW3ccccd2LVrV4/HnnrqKVx77bVobGzEtddei6eeekqn6ihqeow9D4h39Ez4fxvPPttzP0DUb1iehS74DsltvQfrcdjuQnExsHo18MILGnyPfooQQru99ZbQnVPmaW5uxuLFi/H5558DAOx2O/bs2YOcnBy0trZi9uzZaIgiLMrKylBbW5vockmN0xm8tJzJJMMxEcveut0yZBsaZEu7qir+rpD8/J5j0202oLExeD8rSwZ7gNkMeDzo6AA+/RT4y1+Ajz6SH43NWcgKudiSB2bk53kwcyYwYwZwzz1Q4isyjBAikR9Emjp69KhwOp3d90eMGNHj+ZEjR/b52k2bNonS0lJRWloqJk6cmKgSKZKmJiEcDiHMZnnb1JSY/dpsQphMQgDy1uGIvZ7Ac3KOac+PUA6H8Pm/llcxieOjHGLGDCEslt4vO6g4hAfBbbuu7FWXJvnLljullfCW+8iRI9He3t79/KWXXhpVvztb7gYU/h9B+CxRf2s64usC/0m43XI9mMCyAWG8HoEDB4D33wca3nTj3v9xYbK3AQ2ww4UqHIUViiJ3MXNm8MNxkRvmRfOD/wnYbHICU/C/Ck1a7lxbhtJadnY2Wltbu7tlxo4dq3dJpJfwvnwgGPL9nTzt6xyAy6Ua7ALA8aE2OEYBZ88GHrViE+owerTsWqnwB/n06cCIEeF7sMox7oHaAt1HGndN8YQqpbUlS5Zg8+bNAIDNmzdj6dKlOldEugkfwWKz9Tzp+eyz6iNa+hhBI8LO3Qj/x0E48L1/VOPsWdnYvuMO4A9/kF3wJ04Ar78O/Nu/AXPnqgW7XzJOKmvVv9PHB5Fmli9fLsaNGyeysrLEhAkTxPPPPy/a2trEnDlzhM1mE3PmzBGnTp2Kal+lpaUJrpY09e67wQ5si0XeD+8rf/fd/vvyHQ71Pnj/fnxmszh3hUP8+0+axPe/L0RdSN+4DxAXYBFzrU1i9WohXnpJiC+/7KPWaM4p9FWLpEn+MtwpIzHcU0g0YRh+ZtJiiRSQvZnNPfbhM5vF3r1CPPqoEFdfLURWVs8vMQlNonGwQ3gUs2jPdYiTf4nyxK/DIYSi9Kw1/Hvq/3vmCVWiePGEagqJZmikonKO0WxWHX7YS6BP++BBCMizlV6YcNhUAIcv+HVMJuCf/gn4/veBWbOA730PGDUqju8nfFgkINeVib5PnSdUicgAoul/tlh6nty0WGSfeuibgsoJ03PnADHbhWEt9d0nGAWAehTA5auC3Q7MmweUl8tQHz5cg+/HbpdXYAr/HpOM4U5E+rLbgyENyBa4291zwlF1tbx6UWenDPbqamDixF6TlLxe4JNPgLfekh8ffghc8DTABLlvBYBXMePDTXX4n3IgLy8B309VVc8hlMm6SEgYdstQRmK3TAoJH08eGBweZTfG2bPAm28Cr70mM//UqeBzNpMbdaIQg0QnFMhAUiwWuSJjIhfuCp0dO2mSfOzo0WhnymrSLcOhkESkv9AuFyHUuzFC1nfpmpSP9jH58Jqy0DLCiZ8td+NPf5LBnpcHrFkDbNsG1Oe7YFE83WmpAEBXV+IXKwtdAdJikbUneaE0ttwpI7HlnkKczt591GEnIIUAOmxOWI7WwyR83cESODn6t6EF2P6LOrhcsiu++/yr2snNALM5vjVnYl2zpo91Z/qhScud4U4ZieGeQtQCuKkJXZdb8cEHsrtlx47eC271EM3SAmriWbgs1oXPYl8ojd0yRKQDt1uukqgo8iM/v/f65dFckCOwTUiwC5MJ5yY6cMfjVmRnA9deCzz9NNDcDDSZ7fD6I6tHq7G/E5ahS/wGpvyHimd2aKyzS/W4SDfYcqcMxZb7AETRjRJVazVkm0BQNJgdWOSVi24B8mVLl8qPGWPcMC2N+yRlbLUl8vWRsVuGKF4M9wFQ60YJdIuETBhSfd5PCLkfxddzXfNB8KCgAFixArjppuB1MDQVTZ95f9touU68OoY7UbwY7gPQX8tdrY87pHV76BDw8svAli3A9kYnClAPM3zwwoS2ywrQursOU6eqT0jVVKSATtYFRdQx3InixXAfALdbTigKX48cACZP7rW5ANA+2oZbR1djZ30wQKdf5sYO4cK49gYgywzF4wn2SSf44tERwzv2ES5a4glVItKB1SrXtw0sjdXYKB8LG78dWCJXATC8zY3/Xe/CiBFARQXw9tvAh19ZkXOqDkqBXQa7zxccBx7NCdn+RHp9pJOiWl1IW0dsuVNGYss9AcJau4FgD/CZzOj6hweDB/f/uu7x54k86Rnp+cT3q/eHLXci0l9np5wN+sVFwaGKXpjQCQt8SrD1a7JOwuBpUV4sY6AXs4j0+kjDE0NnmNbVJTPYNcNwJ6K4HDsG/PznwOWXAzfeCFxzvgoNKIBXMePCFQUY9E41TIUhAQrI1nL4NHy1oI23W0Rl7Lzq6w0Q3pGwW4YyErtl4ldTA2zYALzySvAco9MJ/PjHwK23AqNH9/HCWE5SxtstojZax+FIdrfKQLFbhojiEMfJSo8H2LoVuPpqeQHol1+W51J/8APgz38GPvsMuOeefoIdiNwaD63L5ZKBHGvLOrQ7BpBvIAZtmUfCcCfKNC6XevdIQEjIeguc2PS/3LBagZtvBvbtA0aOBB56SG62dSvwz/8c5bj0SP3ckeqKhgFGuWiF3TKUkQzZLRNtV0ak7hGnE6K+HopPTi6qRwGKUAe7HfjXfwVuuw24+OIE1K/F2HJ9R7lohd0yRBQi2pZvP63bL74AvIcaoPi7NszwoUBpwM6dclLq2rUJCvbwugD5fcQ6xj0DTpRGi+FOZBTRDh9U6R5papInRG02oF70XH3RLLxY+IATpuYYJxLFKlBXqCRe3MJoGO5ERhFtf3NI6/bw9jrc8bgVdjvw+9/L94TfL61C12QZst39A8kI2UBdZnPwsXjGuBMAhjuRccSwbnjL+260jHTCas/Cg5udmCTcuOMOmeH//qoVFx3RMWR5UlQTDHcio4iiv7m9XY50OTvbhZy/1yMLXhSiHnVWF/7wB3ndjW5a9IHHQ6eLWxgNR8tQRjLkaJl+dHYC//EfwC9+AZw+DXQh7JJ1aiNT1NZmT/7yt5mIo2WIqH9CADs3utE8zImf3JuF9087sWKGG12Touj6YB94WmO4ExnUp5/KCUZX/IsLk7v8XTBKPV4658KQ3TF0fbAPPC0x3IkM5tw54N57gbIyOaPUjgaYIYdImoQPysGDsU3vT8U+8IGu954BGO5EBiEEsH27XCdrwwb52D33AEpB2IlRILaLYqTixCAtliowOIY7UbIloNX5xRfA0qXADTcALS2y1f7xx8BvfwuY31CZHOTzyROlhYXpGZIDXe89AzDciZJNw1anzwc8/XRwVdspF7txapwTNZ9mYdqP/G8cgZa3w9G7Bd/ZmTohGcubHs8DRMRwJ0q2eFqdKsHX0gLMmye7Xv7xD+Cmm4D9E1wYdaIeitobh9r0/lB6h2Qsb3qpeB4gxXCcO2UkXce5R7p+ZxSv+XtOAfLO16G9HRgzBvjd72S3TFQrK4buS1GAQYPka/ReRVGLVSGNgePciQLy8vIwZcoUlJSUoKysTO9y+hdPqzOstX/xsQa0twOLF8sLZSxd6t8uvLti0qTeXR2hX7+wEDh0KDVOlrKrRVNsuZMh5OXloba2FqP7vRRQUNrNUHU64TtUD5OQa6wfVgrw5011+PGPwy6UEb6eeWenfCyW/xL0Yoy12LXAljuR4aj0rQsB/OGGKtSLAnhgxhdDCjBkdxVWrVK5AlL4sMWjR1PnhGkkqTjkMo0x3MkQFEXBvHnzUFpaisrKStVtKisrUVZWhrKyMpw8eTLJFUYp7KSir3w+WkY68aMnroQA8OtVhzHxbB3y5kQZfLF2dXBykGGwW4YM4fjx4xg/fjxOnDiB8vJyPPPMM5g1a1af26dst0zYScXAH5ACwKeYYCqMsVsl1q6OeE72ktbYLUMUMH78eADA2LFjsWzZMtTU1OhcUZxCWtoC8q888JduEnF0q8Ta1cHJQYbBcKe0d/78eZw7d67787feegtFRUU6VxWnqiqczi7oDvYekjGChCNWDIPhTmnv66+/xne/+11MnToVV111Fa677josWLBA77JiJgTw8+etuKy1DgfhgE8J+/NMxmSd/oZpsj8+rbDPnTJSqvW5d3YCFRXASy/JXP3TE27c9J8pNiyQ/fHJwj53IiP45htg0SIZ7MOGAa+/Dtz0Uw2GBWrd0mZ/fFphuBMlS2jYDh4MmM3wFjpRcY0b77wDjBsHvPceoFmPktbL4rI/Pq0w3ImSJTRsA6sx1tfj0VoXcnOBDz4Apk3T8Otp3dLmYl1pheFOlCyhYetnhg92NOC99wCbLWz7gXaraN3S5gzStMJwJ0qWsDHsAOCFCb58u3pODrRbhS3tjJaldwFEGaOqCr5586E0HQEgA96bZ4VlVx+hO9BulUBLmzISW+5EiRTStSJcLnz9NeCDSY51M5lgGWpR795wu2WLO0BReAKTYsJwJ9KC2w3k58sQVhT5eWBdF3/XijhUj3HfHIEZsjWu9Ncad7mArq7gfSGAZ59NwjdCRsFwJ9KCywUcORK8f+RIcMEuf9eKScjb7pmn/Z3kbGiQgR7qrru0rpoMjOFOpAW1Frh/dmkgzL0w4dw4m1zZMdJJTrXQ56QhigHDnUgLamFst+Po/6lCgyIvsnFqTAGG762ObjhhVRVgsQTvs8+dYsRwJ9JCVVXPgeo2G755uQqL77bC4avDD673YPRXMYwNt1rltU0djuC1TjmUkWLAoZBEWrBagcbG7rtCAHf8ADh4UObziy8G5xPFtE8OZaQ4seVOlACbNgH//d/AiBHAq68Cl1yid0WUaRjuRBo7dAi47z75+aZNclQkUbIx3Ik01NEBrFgBXLgAPHCDGzc/zotbkD4Y7kTxCl/Ya88enJngRO1fs3B4kBNPHZiv7ZK7RDHglZgoI2lyJaawKxP5zFkQXR6Y4YNPMXVPWupmNsshkET90+RKTBwtQxSvsIW9FF9n97/C3cFuMgUvS8dx6pRE7JYhilfIEr4+xYQOWOBFyNICNhuX3CXdMNyJ4uVfL12YzahHAeajGt+O909SCrToq6p4cQvSBcOdKBKVE6dwOoErrwQArJl9GE5Rh7zbZuPioSGvO3IEmD9fl5KJeEKVMlJMJ1SdTjl4PfRvxd+X7lNMOCQKMHNYHRobgXE5KufCEvs3RsajyQlVttyJIlFbfjdkGV87GvCznwHjxulQG1EfGO5EkfQzysULE9xZdtx7r/+B8Ktc97rqNVFyMNyJIlFZfldYLPBAnkg99KsqDBnif666OriSo8Mh7xPpgOFOFInK8rsvPXwIg+DB8qI6uO629tw2mvXaiRKMk5iIohGy/O633wI/9fe2rFsXx1K+REnAX0uiGD3/PHDsGFBcDCxbpnc1ROoY7kQx8HiAX/9afv7oo2y1U+riryZRDN5/fA/qvxgMHxQsWzFYTmgiSkEMd6IoCQF894n5GIxOKACUzs6eM1DDZ7Jy/XbSEcOdKEp79wKDRGfP6YOdncHPXS6u304pg+FOFKVnnwU6YOm5pkbo+PewJYDR0JDM8oh6YLgTReHkSWDbNmABqiEG+QPdYuk5SSlkCWCu3056Y7iTIezatQt2ux02mw1PPfWU5vt/8UXZA3PxotkwdXbIDviODmD27OBG/iWAuX47pQJOYqK05/V6cdddd+Htt99Gbm4upk+fjiVLlsDhcGiyfyGAykr5+Zo1/WwYMtGJSG9suVPaq6mpgc1mg9VqhcViwfLly7Fjxw7N9v/RR8Dhw0BODrBokWa7JUqohK7n7nQ6xZDuFZVS08mTJzFmzBi9y4iIdfbtzJkzOHv2LK644goAwKlTp3D+/HlMnDixV21tbW0AgI6ODpSUlES1/7/9Tfa5Z2cDubmalh4Rf+7aSoc69+/fXyeEKBrwjoQQCfsoLS0VqS4dahSCdfZn69atYuXKld33X3zxRfGTn/yk39cMHTo0qn13dgoxerQQgBCffDKgMuPCn7u20qFOALVCg/xltwylvdzcXHz55Zfd91taWjB+/HhN9v3220BbG1BYCETZ0CdKCQx3SnvTp09HY2Mjjh49is7OTmzZsgVLlizRZN+vvCJvf/hDQNHk4mdEyZHQ0TKrV69O5O41kQ41AqyzP1lZWdi4cSPmz58Pr9eLiooKOJ3Ofl8zevToiPv1eIKjGW+8UYtKY8efu7bSpM5KLXbCC2RTRormAtnvvgvMmSPnItXXJ6kwIl4gmyixtm2Tt1yzndIRw51IhRDA66/Lz6+/XtdSiOIyoHBXFOUHiqLUKYriUxSlLOy5n9lsNtjtdlT3cZHg06dPo7y8HPn5+SgvL8eZM2cGUk5Ubr75ZpSUlKCkpAR5eXl9jnXOy8vDlClTUFJSgrKyMtVtEmndunWYMGFCd607d+5U3S7R0+4jefDBB1FQUIDi4mIsW7YM7e3tqtvpdTwjHR8hBO6++27YbDYUFxfjk08+ASAnLTU3A6NHA9OnJ7bGL7/8Etdccw0KCwvhdDrx9NNP99pmz549GDFiRPfvw+OPP57YolRE+hn2dSyTqaGhofsYlZSUYPjw4diwYUOPbfQ6lhUVFRg7diyKioJD2KPNQEVRFiiK0qAoyhFFUX4a1RccyDhKAIUA7AD2ACgLedwB4K/ffvutcLvdwmq1Co/H02s854MPPijWr18vhBBi/fr14qGHHtJwtGhk9913n3jsscdUn7viiivEyZMnk1pPqEcffVT86le/6ncbj8cjrFaraGpqEh0dHaK4uFjU1dUlqUKpurpadHV1CSGEeOihh/r8GepxPPs7PoHxzm+88YZYsGCB8Pl8Yt++feKqq64SQgixYYMc2/7DHya+zuPHj4v9+/cLIYQ4e/asyM/P7/VzfPfdd8V1112X+GL6Eeln2Nex1IvH4xHZ2dmiubm5x+N6Hcv33ntP7N+/Xzidzu7H+sjA8Jw1A2gCYAVgAfBXAI7w7cI/BtRyF0IcEkKorWu6FMCWwYMHY9KkSbDZbKipqem10Y4dO3D77bcDAG6//Xa8+uqrAyknJkIIbN26FStWrEja19RaoqfdR2PevHnIypKDrmbOnImWlpakfv3+RHN8duzYgdtuuw2KomDmzJlob29Ha2srdu2Szy9YkPg6c3JyMG3aNADAJZdcgsLCQhw7dizxX1hjfR1LvbzzzjuYPHly98xlvc2aNQujRo3q8ViUGXgVgCNCCLcQohPAFsiM7Vei+twnAOieVZKbm6v6y/r1118jJycHgPwFP3HiRILK6e2DDz5AdnY28vPzVZ9XFAXz5s1DaWkpKis1GZkUs40bN6K4uBgVFRWq/64dO3YMl19+eff9vo5zsrzwwgtYuHCh6nN6HM9ojo/aNs3Nx/Hee/J+eXlSSu3W3NyMTz/9FDNmzOj13L59+zB16lQsXLgQdTosUBbpZ5hqv49btmzps/Gm97EMiDIDe+QpgBb/Y/2KOM5dUZTdAMapPPWwEKKvZmKvoTxKEmeAzJ07F1999VWvx5988kksXSrf8F5++eV+W+179+7F+PHjceLECZSXl6OgoACzZs1KWp1r167FI488AkVR8Mgjj+D+++/HCy+80GM7oTKMNRHHOZrj+eSTTyIrKwu33HKL6j6ScTzDRXN81LY5ePBiXLgAOBzAOLXf/AT55ptvcOONN2LDhg0YPnx4j+emTZuGL774AsOGDcPOnTtx/fXXo7GxMXnFIfLPMFm/j9Ho7OzEa6+9hvXr1/d6LhWOZYzUDmLEYeYRw10IMTeOYloAdL+F9zUdPDs7G62trcjJyUFrayvGjh0bx5fqbffu3f0+7/F4sG3bNuzfv7/PbQL1jh07FsuWLUNNTY3mYRSpzoBVq1Zh8eLFvR5P5LT7UJHq3Lx5M15//XW88847ff4xJ+N4hovm+Khtc/iwbBSFLtWeaF1dXbjxxhtxyy234IYbbuj1fGjYL1q0CHfeeSfa2tqimoyllUg/w2T9PkbjzTffxLRp05Cdnd3ruVQ4lgFRZmCPPAWQC+B4pH0nqlvmNQDLOzo6cPToUTQ2NuKqq67qtdGSJUuwefNmADIgAq3ARNu9ezcKCgqQ28cSf+fPn8e5c+e6P3/rrbd6nOFOhtC+yu3bt6t+/UROu4/Wrl278Mtf/hKvvfYahg4dqrqNXsczmuOzZMkSvPjiixBC4KOPPsKIESOwf/8lAJIX7kIIrFy5EoWFhbjvvvtUt/nqq6+6W8Y1NTXw+Xy47LLLklMgovsZqh3LQJdDsvX3n7nexzJUlBn4MYB8RVEmKYpiAbAcMmP7F+mMa38fAJZBvqt0APgaQHXIcw9brVZx5ZVXip07d3afHV65cqX4+OOPhRBCtLW1iTlz5gibzSbmzJkjTp06NdAT0lG5/fbbxXPPPdfjsWPHjomFCxcKIYRoamoSxcXFori4WDgcDvHEE08kpa5Qt956qygqKhJTpkwRLpdLHD9+vFedQsgRCvn5+cJqtepS5+TJk0Vubq6YOnWqmDp1qlizZk2vOvU8nmrH57nnnhMTJ04UQgjh8/nEnXfeKaxWqygqKhL79n0shg6VI2W++io5NX7wwQcCgJgyZUr3cXzjjTfEc8891/17+swzzwiHwyGKi4vFjBkzxN69e5NTnF9fP8PQGsOPZeDvPNnOnz8vRo0aJdrb27sfS4VjuXz5cjFu3DiRlZUlJkyYIJ5//vm+MhAAxgPYKYJ5ugjAYchRMw+LKPKZyw9QRupr+YFPPgFKS4HJk4EjR3QojIjLDxBpb98+eXv11frWQTRQDHeiEB9+KG+/8x196yAaKIY7UYjAXLuZM/Wtg2igGO5Efn//u+xnt1iAoiIAbjfgdAJZWfLW7da7RKKoMdyJ/ALrXBUXA4MGAXC55ELuXq+8dbl0rY8oFgx3Ir/AnLbSUv8DDQ2Azyc/9/nkfaI0wXAn8jtwQN761/CSl2Ay+f9ETCZ5nyhNMNyJ/D77TN4WF/sfqKoCCgoAs1neBi6oSpQGEnqBbKJ00dUFHDokP++eVW+1AjquGEg0EGy5E0F2p3d1yTwfNkzvaogGjuFOhGADPcnrwxElDMOdCHKkIwAUFupbB5FWGO5ECPa3M9zJKBjuRAi23AsK9K2DSCsMd8p4QgCHD8vPOZSdjILhThmvtRW4cAEYPRoYOVLvaoi0wXCnjBe4KIfNpm8dRFpiuFPGY7iTETHcKeMFVvKdPFnfOoi0xHCnjNfcLG/z8vSsgkhbDHfKeAx3MiKGO2W8o0flLcOdjIThThmts1MOhTSZgAkT9K6GSDsMd0pr69atw4QJE1BSUoKSkhLs3LkzptcfPy4nMeXk+C+tR2QQXM+d0t69996LBx54IK7XtrTI28sv17AgohTAljtltC+/lLe5ufrWQaQ1hjulvY0bN6K4uBgVFRU4c+ZMn9tVVlairKwMZWVlOHnyJADg2DH5HMOdjIbhTilv7ty5KCoq6vWxY8cOrF27Fk1NTThw4ABycnJw//3397mf1atXo7a2FrW1tRgzZgwAeTIVkH3uREbCPndKebt3745qu1WrVmHx4sUx7TsQ7uPHx1oVUWpjy53SWmsgnQFs374dRdFeJ6+jA3A68Z8vZ+FzODFJuBNUIZE+FCFEIvef0J0T/ehHP8KBAwegKAry8vKwadMm5ETRx1I2ZAhqOzsBnw9emNA1uQAXHalLQsVEESma7IThTpmoTFFQG3JfmM1QPB7d6iEKoUm4s1uGMtNFF0GY5K+/FyZegokMh+FOmclmg2dyATww40hWAZSqKr0rItIUw50y0+DB+Pz/1WEQPLjJUQdYrXpXRKQphjtlrLY2eesf8k5kKAx3yliBcB89Wt86iBKB4U4Zi+FORsZwp4x16pS8vewyfesgSgSGO2Ws06fl7ahR+tZBlAgMd8pYDHcyMoY7ZSyGOxkZw50yViDcL71U3zqIEoHhThmrvV3eMtzJiBjulLEC4T5ypJ5VECUGw50yFsOdjIzhThlJCHm9jqws4KKL9K6GSHsMd8pIXq+8HTECUDRZPZsotTDcKSMFwn34cH3rIEoUhjtlJIY7GR3DnTJSINwvuUTfOogSheFOGcnnk7cMdzIqhjtlJLbcyegY7pSR2HIno2O4U0YKtNyHDdO3DqJEYbhTRgq03C++WN86iBKF4U4ZieFORsdwp4wU6JZhuJNRMdwpI7HlTkbHcKeMxHAno2O4U0YKhPvQofrWQZQoDHfKSAx3MjqGO2WkQLgPGaJvHUSJwnCnjMRwJ6NjuFNGYriT0THcKSMJIW8Z7mRUDHfKSIGWO6+fSkbFcKe08Morr8DpdMJkMqG2trbHc+vXr4fNZoPdbkd1dXVU+2O3DBldlt4FEEWjqKgI27Ztw5o1a3o8fvDgQWzZsgV1dXU4fvw45s6di8OHD8NsNve7v0C3DFvuZFRsuVNaKCwshN1u7/X4jh07sHz5cgwePBiTJk2CzWZDTU1Nv/vy+YLhbrEkoloi/Ski8FtOlAYURdkD4AEhRK3//kYAHwkh/q///u8BvCmE+C+V164GsNp/9yIhRFFyqiZKPnbLUMpQFGU3gHEqTz0shNjR18tUHlNtsQghKgFUxlkeUVphuFPKEELMjeNlLQAuD7mfC+C4NhURpS/2uVO6ew3AckVRBiuKMglAPoD+O92JMgDDndKCoijLFEVpAfAdAG8oilINAEKIOgBbARwEsAvAXUIIr36VEqUGnlAlIjIgttyJiAyI4U5EZEAMdyIiA2K4ExEZEMOdiMiAGO5ERAbEcCciMqD/DwgkP1vJ3oeFAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "# \n", "# plot the results\n", "#\n", "ax = ut.plotSetup(-10,10,-10,15)\n", "ut.centerAxes(ax)\n", "xplot = np.logspace(np.log10(0.0001),1,100)\n", "yestplot = beta[0]+beta[1]*np.log(xplot)\n", "ax.plot(xplot,yestplot,'b-',lw=2)\n", "ax.plot(xlog,ylog,'ro',markersize=4);" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "## Multiple Regression" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "Suppose an experiment involves two independent variables -- say, $u$ and $v$, -- and one dependent variable, $y$. A simple equation for predicting $y$ from $u$ and $v$ has the form\n", "\n", "$$y = \\beta_0 + \\beta_1 u + \\beta_2 v$$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "Since there is more than one independent variable, this is called __multiple regression.__" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "A more general prediction equation might have the form\n", "\n", "$$y = \\beta_0 + \\beta_1 u + \\beta_2 v + \\beta_3u^2 + \\beta_4 uv + \\beta_5 v^2$$\n", "\n", "A least squares fit to equations like this is called a __trend surface.__" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "In general, a linear model will arise whenever $y$ is to be predicted by an equation of the form\n", "\n", "$$y = \\beta_0f_0(u,v) + \\beta_1f_1(u,v) + \\cdots + \\beta_kf_k(u,v)$$\n", "\n", "with $f_0,\\dots,f_k$ any sort of known functions and $\\beta_0,...,\\beta_k$ unknown weights." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "Let's take an example. Here are a set of points in $\\mathbb{R}^3$:" ] }, { "cell_type": "code", "execution_count": 13, "metadata": { "hide_input": true, "scrolled": false, "tags": [ "hide-input" ] }, "outputs": [ { "data": { "application/javascript": [ "/* Put everything inside the global mpl namespace */\n", "/* global mpl */\n", "window.mpl = {};\n", "\n", "mpl.get_websocket_type = function () {\n", " if (typeof WebSocket !== 'undefined') {\n", " return WebSocket;\n", " } else if (typeof MozWebSocket !== 'undefined') {\n", " return MozWebSocket;\n", " } else {\n", " alert(\n", " 'Your browser does not have WebSocket support. ' +\n", " 'Please try Chrome, Safari or Firefox ≥ 6. ' +\n", " 'Firefox 4 and 5 are also supported but you ' +\n", " 'have to enable WebSockets in about:config.'\n", " );\n", " }\n", "};\n", "\n", "mpl.figure = function (figure_id, websocket, ondownload, parent_element) {\n", " this.id = figure_id;\n", "\n", " this.ws = websocket;\n", "\n", " this.supports_binary = this.ws.binaryType !== undefined;\n", "\n", " if (!this.supports_binary) {\n", " var warnings = document.getElementById('mpl-warnings');\n", " if (warnings) {\n", " warnings.style.display = 'block';\n", " warnings.textContent =\n", " 'This browser does not support binary websocket messages. ' +\n", " 'Performance may be slow.';\n", " }\n", " }\n", "\n", " this.imageObj = new Image();\n", "\n", " this.context = undefined;\n", " this.message = undefined;\n", " this.canvas = undefined;\n", " this.rubberband_canvas = undefined;\n", " this.rubberband_context = undefined;\n", " this.format_dropdown = undefined;\n", "\n", " this.image_mode = 'full';\n", "\n", " this.root = document.createElement('div');\n", " this.root.setAttribute('style', 'display: inline-block');\n", " this._root_extra_style(this.root);\n", "\n", " parent_element.appendChild(this.root);\n", "\n", " this._init_header(this);\n", " this._init_canvas(this);\n", " this._init_toolbar(this);\n", "\n", " var fig = this;\n", "\n", " this.waiting = false;\n", "\n", " this.ws.onopen = function () {\n", " fig.send_message('supports_binary', { value: fig.supports_binary });\n", " fig.send_message('send_image_mode', {});\n", " if (fig.ratio !== 1) {\n", " fig.send_message('set_dpi_ratio', { dpi_ratio: fig.ratio });\n", " }\n", " fig.send_message('refresh', {});\n", " };\n", "\n", " this.imageObj.onload = function () {\n", " if (fig.image_mode === 'full') {\n", " // Full images could contain transparency (where diff images\n", " // almost always do), so we need to clear the canvas so that\n", " // there is no ghosting.\n", " fig.context.clearRect(0, 0, fig.canvas.width, fig.canvas.height);\n", " }\n", " fig.context.drawImage(fig.imageObj, 0, 0);\n", " };\n", "\n", " this.imageObj.onunload = function () {\n", " fig.ws.close();\n", " };\n", "\n", " this.ws.onmessage = this._make_on_message_function(this);\n", "\n", " this.ondownload = ondownload;\n", "};\n", "\n", "mpl.figure.prototype._init_header = function () {\n", " var titlebar = document.createElement('div');\n", " titlebar.classList =\n", " 'ui-dialog-titlebar ui-widget-header ui-corner-all ui-helper-clearfix';\n", " var titletext = document.createElement('div');\n", " titletext.classList = 'ui-dialog-title';\n", " titletext.setAttribute(\n", " 'style',\n", " 'width: 100%; text-align: center; padding: 3px;'\n", " );\n", " titlebar.appendChild(titletext);\n", " this.root.appendChild(titlebar);\n", " this.header = titletext;\n", "};\n", "\n", "mpl.figure.prototype._canvas_extra_style = function (_canvas_div) {};\n", "\n", "mpl.figure.prototype._root_extra_style = function (_canvas_div) {};\n", "\n", "mpl.figure.prototype._init_canvas = function () {\n", " var fig = this;\n", "\n", " var canvas_div = (this.canvas_div = document.createElement('div'));\n", " canvas_div.setAttribute(\n", " 'style',\n", " 'border: 1px solid #ddd;' +\n", " 'box-sizing: content-box;' +\n", " 'clear: both;' +\n", " 'min-height: 1px;' +\n", " 'min-width: 1px;' +\n", " 'outline: 0;' +\n", " 'overflow: hidden;' +\n", " 'position: relative;' +\n", " 'resize: both;'\n", " );\n", "\n", " function on_keyboard_event_closure(name) {\n", " return function (event) {\n", " return fig.key_event(event, name);\n", " };\n", " }\n", "\n", " canvas_div.addEventListener(\n", " 'keydown',\n", " on_keyboard_event_closure('key_press')\n", " );\n", " canvas_div.addEventListener(\n", " 'keyup',\n", " on_keyboard_event_closure('key_release')\n", " );\n", "\n", " this._canvas_extra_style(canvas_div);\n", " this.root.appendChild(canvas_div);\n", "\n", " var canvas = (this.canvas = document.createElement('canvas'));\n", " canvas.classList.add('mpl-canvas');\n", " canvas.setAttribute('style', 'box-sizing: content-box;');\n", "\n", " this.context = canvas.getContext('2d');\n", "\n", " var backingStore =\n", " this.context.backingStorePixelRatio ||\n", " this.context.webkitBackingStorePixelRatio ||\n", " this.context.mozBackingStorePixelRatio ||\n", " this.context.msBackingStorePixelRatio ||\n", " this.context.oBackingStorePixelRatio ||\n", " this.context.backingStorePixelRatio ||\n", " 1;\n", "\n", " this.ratio = (window.devicePixelRatio || 1) / backingStore;\n", "\n", " var rubberband_canvas = (this.rubberband_canvas = document.createElement(\n", " 'canvas'\n", " ));\n", " rubberband_canvas.setAttribute(\n", " 'style',\n", " 'box-sizing: content-box; position: absolute; left: 0; top: 0; z-index: 1;'\n", " );\n", "\n", " // Apply a ponyfill if ResizeObserver is not implemented by browser.\n", " if (this.ResizeObserver === undefined) {\n", " if (window.ResizeObserver !== undefined) {\n", " this.ResizeObserver = window.ResizeObserver;\n", " } else {\n", " var obs = _JSXTOOLS_RESIZE_OBSERVER({});\n", " this.ResizeObserver = obs.ResizeObserver;\n", " }\n", " }\n", "\n", " this.resizeObserverInstance = new this.ResizeObserver(function (entries) {\n", " var nentries = entries.length;\n", " for (var i = 0; i < nentries; i++) {\n", " var entry = entries[i];\n", " var width, height;\n", " if (entry.contentBoxSize) {\n", " if (entry.contentBoxSize instanceof Array) {\n", " // Chrome 84 implements new version of spec.\n", " width = entry.contentBoxSize[0].inlineSize;\n", " height = entry.contentBoxSize[0].blockSize;\n", " } else {\n", " // Firefox implements old version of spec.\n", " width = entry.contentBoxSize.inlineSize;\n", " height = entry.contentBoxSize.blockSize;\n", " }\n", " } else {\n", " // Chrome <84 implements even older version of spec.\n", " width = entry.contentRect.width;\n", " height = entry.contentRect.height;\n", " }\n", "\n", " // Keep the size of the canvas and rubber band canvas in sync with\n", " // the canvas container.\n", " if (entry.devicePixelContentBoxSize) {\n", " // Chrome 84 implements new version of spec.\n", " canvas.setAttribute(\n", " 'width',\n", " entry.devicePixelContentBoxSize[0].inlineSize\n", " );\n", " canvas.setAttribute(\n", " 'height',\n", " entry.devicePixelContentBoxSize[0].blockSize\n", " );\n", " } else {\n", " canvas.setAttribute('width', width * fig.ratio);\n", " canvas.setAttribute('height', height * fig.ratio);\n", " }\n", " canvas.setAttribute(\n", " 'style',\n", " 'width: ' + width + 'px; height: ' + height + 'px;'\n", " );\n", "\n", " rubberband_canvas.setAttribute('width', width);\n", " rubberband_canvas.setAttribute('height', height);\n", "\n", " // And update the size in Python. We ignore the initial 0/0 size\n", " // that occurs as the element is placed into the DOM, which should\n", " // otherwise not happen due to the minimum size styling.\n", " if (fig.ws.readyState == 1 && width != 0 && height != 0) {\n", " fig.request_resize(width, height);\n", " }\n", " }\n", " });\n", " this.resizeObserverInstance.observe(canvas_div);\n", "\n", " function on_mouse_event_closure(name) {\n", " return function (event) {\n", " return fig.mouse_event(event, name);\n", " };\n", " }\n", "\n", " rubberband_canvas.addEventListener(\n", " 'mousedown',\n", " on_mouse_event_closure('button_press')\n", " );\n", " rubberband_canvas.addEventListener(\n", " 'mouseup',\n", " on_mouse_event_closure('button_release')\n", " );\n", " rubberband_canvas.addEventListener(\n", " 'dblclick',\n", " on_mouse_event_closure('dblclick')\n", " );\n", " // Throttle sequential mouse events to 1 every 20ms.\n", " rubberband_canvas.addEventListener(\n", " 'mousemove',\n", " on_mouse_event_closure('motion_notify')\n", " );\n", "\n", " rubberband_canvas.addEventListener(\n", " 'mouseenter',\n", " on_mouse_event_closure('figure_enter')\n", " );\n", " rubberband_canvas.addEventListener(\n", " 'mouseleave',\n", " on_mouse_event_closure('figure_leave')\n", " );\n", "\n", " canvas_div.addEventListener('wheel', function (event) {\n", " if (event.deltaY < 0) {\n", " event.step = 1;\n", " } else {\n", " event.step = -1;\n", " }\n", " on_mouse_event_closure('scroll')(event);\n", " });\n", "\n", " canvas_div.appendChild(canvas);\n", " canvas_div.appendChild(rubberband_canvas);\n", "\n", " this.rubberband_context = rubberband_canvas.getContext('2d');\n", " this.rubberband_context.strokeStyle = '#000000';\n", "\n", " this._resize_canvas = function (width, height, forward) {\n", " if (forward) {\n", " canvas_div.style.width = width + 'px';\n", " canvas_div.style.height = height + 'px';\n", " }\n", " };\n", "\n", " // Disable right mouse context menu.\n", " this.rubberband_canvas.addEventListener('contextmenu', function (_e) {\n", " event.preventDefault();\n", " return false;\n", " });\n", "\n", " function set_focus() {\n", " canvas.focus();\n", " canvas_div.focus();\n", " }\n", "\n", " window.setTimeout(set_focus, 100);\n", "};\n", "\n", "mpl.figure.prototype._init_toolbar = function () {\n", " var fig = this;\n", "\n", " var toolbar = document.createElement('div');\n", " toolbar.classList = 'mpl-toolbar';\n", " this.root.appendChild(toolbar);\n", "\n", " function on_click_closure(name) {\n", " return function (_event) {\n", " return fig.toolbar_button_onclick(name);\n", " };\n", " }\n", "\n", " function on_mouseover_closure(tooltip) {\n", " return function (event) {\n", " if (!event.currentTarget.disabled) {\n", " return fig.toolbar_button_onmouseover(tooltip);\n", " }\n", " };\n", " }\n", "\n", " fig.buttons = {};\n", " var buttonGroup = document.createElement('div');\n", " buttonGroup.classList = 'mpl-button-group';\n", " for (var toolbar_ind in mpl.toolbar_items) {\n", " var name = mpl.toolbar_items[toolbar_ind][0];\n", " var tooltip = mpl.toolbar_items[toolbar_ind][1];\n", " var image = mpl.toolbar_items[toolbar_ind][2];\n", " var method_name = mpl.toolbar_items[toolbar_ind][3];\n", "\n", " if (!name) {\n", " /* Instead of a spacer, we start a new button group. */\n", " if (buttonGroup.hasChildNodes()) {\n", " toolbar.appendChild(buttonGroup);\n", " }\n", " buttonGroup = document.createElement('div');\n", " buttonGroup.classList = 'mpl-button-group';\n", " continue;\n", " }\n", "\n", " var button = (fig.buttons[name] = document.createElement('button'));\n", " button.classList = 'mpl-widget';\n", " button.setAttribute('role', 'button');\n", " button.setAttribute('aria-disabled', 'false');\n", " button.addEventListener('click', on_click_closure(method_name));\n", " button.addEventListener('mouseover', on_mouseover_closure(tooltip));\n", "\n", " var icon_img = document.createElement('img');\n", " icon_img.src = '_images/' + image + '.png';\n", " icon_img.srcset = '_images/' + image + '_large.png 2x';\n", " icon_img.alt = tooltip;\n", " button.appendChild(icon_img);\n", "\n", " buttonGroup.appendChild(button);\n", " }\n", "\n", " if (buttonGroup.hasChildNodes()) {\n", " toolbar.appendChild(buttonGroup);\n", " }\n", "\n", " var fmt_picker = document.createElement('select');\n", " fmt_picker.classList = 'mpl-widget';\n", " toolbar.appendChild(fmt_picker);\n", " this.format_dropdown = fmt_picker;\n", "\n", " for (var ind in mpl.extensions) {\n", " var fmt = mpl.extensions[ind];\n", " var option = document.createElement('option');\n", " option.selected = fmt === mpl.default_extension;\n", " option.innerHTML = fmt;\n", " fmt_picker.appendChild(option);\n", " }\n", "\n", " var status_bar = document.createElement('span');\n", " status_bar.classList = 'mpl-message';\n", " toolbar.appendChild(status_bar);\n", " this.message = status_bar;\n", "};\n", "\n", "mpl.figure.prototype.request_resize = function (x_pixels, y_pixels) {\n", " // Request matplotlib to resize the figure. Matplotlib will then trigger a resize in the client,\n", " // which will in turn request a refresh of the image.\n", " this.send_message('resize', { width: x_pixels, height: y_pixels });\n", "};\n", "\n", "mpl.figure.prototype.send_message = function (type, properties) {\n", " properties['type'] = type;\n", " properties['figure_id'] = this.id;\n", " this.ws.send(JSON.stringify(properties));\n", "};\n", "\n", "mpl.figure.prototype.send_draw_message = function () {\n", " if (!this.waiting) {\n", " this.waiting = true;\n", " this.ws.send(JSON.stringify({ type: 'draw', figure_id: this.id }));\n", " }\n", "};\n", "\n", "mpl.figure.prototype.handle_save = function (fig, _msg) {\n", " var format_dropdown = fig.format_dropdown;\n", " var format = format_dropdown.options[format_dropdown.selectedIndex].value;\n", " fig.ondownload(fig, format);\n", "};\n", "\n", "mpl.figure.prototype.handle_resize = function (fig, msg) {\n", " var size = msg['size'];\n", " if (size[0] !== fig.canvas.width || size[1] !== fig.canvas.height) {\n", " fig._resize_canvas(size[0], size[1], msg['forward']);\n", " fig.send_message('refresh', {});\n", " }\n", "};\n", "\n", "mpl.figure.prototype.handle_rubberband = function (fig, msg) {\n", " var x0 = msg['x0'] / fig.ratio;\n", " var y0 = (fig.canvas.height - msg['y0']) / fig.ratio;\n", " var x1 = msg['x1'] / fig.ratio;\n", " var y1 = (fig.canvas.height - msg['y1']) / fig.ratio;\n", " x0 = Math.floor(x0) + 0.5;\n", " y0 = Math.floor(y0) + 0.5;\n", " x1 = Math.floor(x1) + 0.5;\n", " y1 = Math.floor(y1) + 0.5;\n", " var min_x = Math.min(x0, x1);\n", " var min_y = Math.min(y0, y1);\n", " var width = Math.abs(x1 - x0);\n", " var height = Math.abs(y1 - y0);\n", "\n", " fig.rubberband_context.clearRect(\n", " 0,\n", " 0,\n", " fig.canvas.width / fig.ratio,\n", " fig.canvas.height / fig.ratio\n", " );\n", "\n", " fig.rubberband_context.strokeRect(min_x, min_y, width, height);\n", "};\n", "\n", "mpl.figure.prototype.handle_figure_label = function (fig, msg) {\n", " // Updates the figure title.\n", " fig.header.textContent = msg['label'];\n", "};\n", "\n", "mpl.figure.prototype.handle_cursor = function (fig, msg) {\n", " var cursor = msg['cursor'];\n", " switch (cursor) {\n", " case 0:\n", " cursor = 'pointer';\n", " break;\n", " case 1:\n", " cursor = 'default';\n", " break;\n", " case 2:\n", " cursor = 'crosshair';\n", " break;\n", " case 3:\n", " cursor = 'move';\n", " break;\n", " }\n", " fig.rubberband_canvas.style.cursor = cursor;\n", "};\n", "\n", "mpl.figure.prototype.handle_message = function (fig, msg) {\n", " fig.message.textContent = msg['message'];\n", "};\n", "\n", "mpl.figure.prototype.handle_draw = function (fig, _msg) {\n", " // Request the server to send over a new figure.\n", " fig.send_draw_message();\n", "};\n", "\n", "mpl.figure.prototype.handle_image_mode = function (fig, msg) {\n", " fig.image_mode = msg['mode'];\n", "};\n", "\n", "mpl.figure.prototype.handle_history_buttons = function (fig, msg) {\n", " for (var key in msg) {\n", " if (!(key in fig.buttons)) {\n", " continue;\n", " }\n", " fig.buttons[key].disabled = !msg[key];\n", " fig.buttons[key].setAttribute('aria-disabled', !msg[key]);\n", " }\n", "};\n", "\n", "mpl.figure.prototype.handle_navigate_mode = function (fig, msg) {\n", " if (msg['mode'] === 'PAN') {\n", " fig.buttons['Pan'].classList.add('active');\n", " fig.buttons['Zoom'].classList.remove('active');\n", " } else if (msg['mode'] === 'ZOOM') {\n", " fig.buttons['Pan'].classList.remove('active');\n", " fig.buttons['Zoom'].classList.add('active');\n", " } else {\n", " fig.buttons['Pan'].classList.remove('active');\n", " fig.buttons['Zoom'].classList.remove('active');\n", " }\n", "};\n", "\n", "mpl.figure.prototype.updated_canvas_event = function () {\n", " // Called whenever the canvas gets updated.\n", " this.send_message('ack', {});\n", "};\n", "\n", "// A function to construct a web socket function for onmessage handling.\n", "// Called in the figure constructor.\n", "mpl.figure.prototype._make_on_message_function = function (fig) {\n", " return function socket_on_message(evt) {\n", " if (evt.data instanceof Blob) {\n", " var img = evt.data;\n", " if (img.type !== 'image/png') {\n", " /* FIXME: We get \"Resource interpreted as Image but\n", " * transferred with MIME type text/plain:\" errors on\n", " * Chrome. But how to set the MIME type? It doesn't seem\n", " * to be part of the websocket stream */\n", " img.type = 'image/png';\n", " }\n", "\n", " /* Free the memory for the previous frames */\n", " if (fig.imageObj.src) {\n", " (window.URL || window.webkitURL).revokeObjectURL(\n", " fig.imageObj.src\n", " );\n", " }\n", "\n", " fig.imageObj.src = (window.URL || window.webkitURL).createObjectURL(\n", " img\n", " );\n", " fig.updated_canvas_event();\n", " fig.waiting = false;\n", " return;\n", " } else if (\n", " typeof evt.data === 'string' &&\n", " evt.data.slice(0, 21) === 'data:image/png;base64'\n", " ) {\n", " fig.imageObj.src = evt.data;\n", " fig.updated_canvas_event();\n", " fig.waiting = false;\n", " return;\n", " }\n", "\n", " var msg = JSON.parse(evt.data);\n", " var msg_type = msg['type'];\n", "\n", " // Call the \"handle_{type}\" callback, which takes\n", " // the figure and JSON message as its only arguments.\n", " try {\n", " var callback = fig['handle_' + msg_type];\n", " } catch (e) {\n", " console.log(\n", " \"No handler for the '\" + msg_type + \"' message type: \",\n", " msg\n", " );\n", " return;\n", " }\n", "\n", " if (callback) {\n", " try {\n", " // console.log(\"Handling '\" + msg_type + \"' message: \", msg);\n", " callback(fig, msg);\n", " } catch (e) {\n", " console.log(\n", " \"Exception inside the 'handler_\" + msg_type + \"' callback:\",\n", " e,\n", " e.stack,\n", " msg\n", " );\n", " }\n", " }\n", " };\n", "};\n", "\n", "// from http://stackoverflow.com/questions/1114465/getting-mouse-location-in-canvas\n", "mpl.findpos = function (e) {\n", " //this section is from http://www.quirksmode.org/js/events_properties.html\n", " var targ;\n", " if (!e) {\n", " e = window.event;\n", " }\n", " if (e.target) {\n", " targ = e.target;\n", " } else if (e.srcElement) {\n", " targ = e.srcElement;\n", " }\n", " if (targ.nodeType === 3) {\n", " // defeat Safari bug\n", " targ = targ.parentNode;\n", " }\n", "\n", " // pageX,Y are the mouse positions relative to the document\n", " var boundingRect = targ.getBoundingClientRect();\n", " var x = e.pageX - (boundingRect.left + document.body.scrollLeft);\n", " var y = e.pageY - (boundingRect.top + document.body.scrollTop);\n", "\n", " return { x: x, y: y };\n", "};\n", "\n", "/*\n", " * return a copy of an object with only non-object keys\n", " * we need this to avoid circular references\n", " * http://stackoverflow.com/a/24161582/3208463\n", " */\n", "function simpleKeys(original) {\n", " return Object.keys(original).reduce(function (obj, key) {\n", " if (typeof original[key] !== 'object') {\n", " obj[key] = original[key];\n", " }\n", " return obj;\n", " }, {});\n", "}\n", "\n", "mpl.figure.prototype.mouse_event = function (event, name) {\n", " var canvas_pos = mpl.findpos(event);\n", "\n", " if (name === 'button_press') {\n", " this.canvas.focus();\n", " this.canvas_div.focus();\n", " }\n", "\n", " var x = canvas_pos.x * this.ratio;\n", " var y = canvas_pos.y * this.ratio;\n", "\n", " this.send_message(name, {\n", " x: x,\n", " y: y,\n", " button: event.button,\n", " step: event.step,\n", " guiEvent: simpleKeys(event),\n", " });\n", "\n", " /* This prevents the web browser from automatically changing to\n", " * the text insertion cursor when the button is pressed. We want\n", " * to control all of the cursor setting manually through the\n", " * 'cursor' event from matplotlib */\n", " event.preventDefault();\n", " return false;\n", "};\n", "\n", "mpl.figure.prototype._key_event_extra = function (_event, _name) {\n", " // Handle any extra behaviour associated with a key event\n", "};\n", "\n", "mpl.figure.prototype.key_event = function (event, name) {\n", " // Prevent repeat events\n", " if (name === 'key_press') {\n", " if (event.key === this._key) {\n", " return;\n", " } else {\n", " this._key = event.key;\n", " }\n", " }\n", " if (name === 'key_release') {\n", " this._key = null;\n", " }\n", "\n", " var value = '';\n", " if (event.ctrlKey && event.key !== 'Control') {\n", " value += 'ctrl+';\n", " }\n", " else if (event.altKey && event.key !== 'Alt') {\n", " value += 'alt+';\n", " }\n", " else if (event.shiftKey && event.key !== 'Shift') {\n", " value += 'shift+';\n", " }\n", "\n", " value += 'k' + event.key;\n", "\n", " this._key_event_extra(event, name);\n", "\n", " this.send_message(name, { key: value, guiEvent: simpleKeys(event) });\n", " return false;\n", "};\n", "\n", "mpl.figure.prototype.toolbar_button_onclick = function (name) {\n", " if (name === 'download') {\n", " this.handle_save(this, null);\n", " } else {\n", " this.send_message('toolbar_button', { name: name });\n", " }\n", "};\n", "\n", "mpl.figure.prototype.toolbar_button_onmouseover = function (tooltip) {\n", " this.message.textContent = tooltip;\n", "};\n", "\n", "///////////////// REMAINING CONTENT GENERATED BY embed_js.py /////////////////\n", "// prettier-ignore\n", "var _JSXTOOLS_RESIZE_OBSERVER=function(A){var t,i=new WeakMap,n=new WeakMap,a=new WeakMap,r=new WeakMap,o=new Set;function s(e){if(!(this instanceof s))throw new TypeError(\"Constructor requires 'new' operator\");i.set(this,e)}function h(){throw new TypeError(\"Function is not a constructor\")}function c(e,t,i,n){e=0 in arguments?Number(arguments[0]):0,t=1 in arguments?Number(arguments[1]):0,i=2 in arguments?Number(arguments[2]):0,n=3 in arguments?Number(arguments[3]):0,this.right=(this.x=this.left=e)+(this.width=i),this.bottom=(this.y=this.top=t)+(this.height=n),Object.freeze(this)}function d(){t=requestAnimationFrame(d);var s=new WeakMap,p=new Set;o.forEach((function(t){r.get(t).forEach((function(i){var r=t instanceof window.SVGElement,o=a.get(t),d=r?0:parseFloat(o.paddingTop),f=r?0:parseFloat(o.paddingRight),l=r?0:parseFloat(o.paddingBottom),u=r?0:parseFloat(o.paddingLeft),g=r?0:parseFloat(o.borderTopWidth),m=r?0:parseFloat(o.borderRightWidth),w=r?0:parseFloat(o.borderBottomWidth),b=u+f,F=d+l,v=(r?0:parseFloat(o.borderLeftWidth))+m,W=g+w,y=r?0:t.offsetHeight-W-t.clientHeight,E=r?0:t.offsetWidth-v-t.clientWidth,R=b+v,z=F+W,M=r?t.width:parseFloat(o.width)-R-E,O=r?t.height:parseFloat(o.height)-z-y;if(n.has(t)){var k=n.get(t);if(k[0]===M&&k[1]===O)return}n.set(t,[M,O]);var S=Object.create(h.prototype);S.target=t,S.contentRect=new c(u,d,M,O),s.has(i)||(s.set(i,[]),p.add(i)),s.get(i).push(S)}))})),p.forEach((function(e){i.get(e).call(e,s.get(e),e)}))}return s.prototype.observe=function(i){if(i instanceof window.Element){r.has(i)||(r.set(i,new Set),o.add(i),a.set(i,window.getComputedStyle(i)));var n=r.get(i);n.has(this)||n.add(this),cancelAnimationFrame(t),t=requestAnimationFrame(d)}},s.prototype.unobserve=function(i){if(i instanceof window.Element&&r.has(i)){var n=r.get(i);n.has(this)&&(n.delete(this),n.size||(r.delete(i),o.delete(i))),n.size||r.delete(i),o.size||cancelAnimationFrame(t)}},A.DOMRectReadOnly=c,A.ResizeObserver=s,A.ResizeObserverEntry=h,A}; // eslint-disable-line\n", "mpl.toolbar_items = [[\"Home\", \"Reset original view\", \"fa fa-home icon-home\", \"home\"], [\"Back\", \"Back to previous view\", \"fa fa-arrow-left icon-arrow-left\", \"back\"], [\"Forward\", \"Forward to next view\", \"fa fa-arrow-right icon-arrow-right\", \"forward\"], [\"\", \"\", \"\", \"\"], [\"Pan\", \"Left button pans, Right button zooms\\nx/y fixes axis, CTRL fixes aspect\", \"fa fa-arrows icon-move\", \"pan\"], [\"Zoom\", \"Zoom to rectangle\\nx/y fixes axis, CTRL fixes aspect\", \"fa fa-square-o icon-check-empty\", \"zoom\"], [\"\", \"\", \"\", \"\"], [\"Download\", \"Download plot\", \"fa fa-floppy-o icon-save\", \"download\"]];\n", "\n", "mpl.extensions = [\"eps\", \"jpeg\", \"pgf\", \"pdf\", \"png\", \"ps\", \"raw\", \"svg\", \"tif\"];\n", "\n", "mpl.default_extension = \"png\";/* global mpl */\n", "\n", "var comm_websocket_adapter = function (comm) {\n", " // Create a \"websocket\"-like object which calls the given IPython comm\n", " // object with the appropriate methods. Currently this is a non binary\n", " // socket, so there is still some room for performance tuning.\n", " var ws = {};\n", "\n", " ws.binaryType = comm.kernel.ws.binaryType;\n", " ws.readyState = comm.kernel.ws.readyState;\n", " function updateReadyState(_event) {\n", " if (comm.kernel.ws) {\n", " ws.readyState = comm.kernel.ws.readyState;\n", " } else {\n", " ws.readyState = 3; // Closed state.\n", " }\n", " }\n", " comm.kernel.ws.addEventListener('open', updateReadyState);\n", " comm.kernel.ws.addEventListener('close', updateReadyState);\n", " comm.kernel.ws.addEventListener('error', updateReadyState);\n", "\n", " ws.close = function () {\n", " comm.close();\n", " };\n", " ws.send = function (m) {\n", " //console.log('sending', m);\n", " comm.send(m);\n", " };\n", " // Register the callback with on_msg.\n", " comm.on_msg(function (msg) {\n", " //console.log('receiving', msg['content']['data'], msg);\n", " var data = msg['content']['data'];\n", " if (data['blob'] !== undefined) {\n", " data = {\n", " data: new Blob(msg['buffers'], { type: data['blob'] }),\n", " };\n", " }\n", " // Pass the mpl event to the overridden (by mpl) onmessage function.\n", " ws.onmessage(data);\n", " });\n", " return ws;\n", "};\n", "\n", "mpl.mpl_figure_comm = function (comm, msg) {\n", " // This is the function which gets called when the mpl process\n", " // starts-up an IPython Comm through the \"matplotlib\" channel.\n", "\n", " var id = msg.content.data.id;\n", " // Get hold of the div created by the display call when the Comm\n", " // socket was opened in Python.\n", " var element = document.getElementById(id);\n", " var ws_proxy = comm_websocket_adapter(comm);\n", "\n", " function ondownload(figure, _format) {\n", " window.open(figure.canvas.toDataURL());\n", " }\n", "\n", " var fig = new mpl.figure(id, ws_proxy, ondownload, element);\n", "\n", " // Call onopen now - mpl needs it, as it is assuming we've passed it a real\n", " // web socket which is closed, not our websocket->open comm proxy.\n", " ws_proxy.onopen();\n", "\n", " fig.parent_element = element;\n", " fig.cell_info = mpl.find_output_cell(\"
\");\n", " if (!fig.cell_info) {\n", " console.error('Failed to find cell for figure', id, fig);\n", " return;\n", " }\n", " fig.cell_info[0].output_area.element.on(\n", " 'cleared',\n", " { fig: fig },\n", " fig._remove_fig_handler\n", " );\n", "};\n", "\n", "mpl.figure.prototype.handle_close = function (fig, msg) {\n", " var width = fig.canvas.width / fig.ratio;\n", " fig.cell_info[0].output_area.element.off(\n", " 'cleared',\n", " fig._remove_fig_handler\n", " );\n", " fig.resizeObserverInstance.unobserve(fig.canvas_div);\n", "\n", " // Update the output cell to use the data from the current canvas.\n", " fig.push_to_output();\n", " var dataURL = fig.canvas.toDataURL();\n", " // Re-enable the keyboard manager in IPython - without this line, in FF,\n", " // the notebook keyboard shortcuts fail.\n", " IPython.keyboard_manager.enable();\n", " fig.parent_element.innerHTML =\n", " '';\n", " fig.close_ws(fig, msg);\n", "};\n", "\n", "mpl.figure.prototype.close_ws = function (fig, msg) {\n", " fig.send_message('closing', msg);\n", " // fig.ws.close()\n", "};\n", "\n", "mpl.figure.prototype.push_to_output = function (_remove_interactive) {\n", " // Turn the data on the canvas into data in the output cell.\n", " var width = this.canvas.width / this.ratio;\n", " var dataURL = this.canvas.toDataURL();\n", " this.cell_info[1]['text/html'] =\n", " '';\n", "};\n", "\n", "mpl.figure.prototype.updated_canvas_event = function () {\n", " // Tell IPython that the notebook contents must change.\n", " IPython.notebook.set_dirty(true);\n", " this.send_message('ack', {});\n", " var fig = this;\n", " // Wait a second, then push the new image to the DOM so\n", " // that it is saved nicely (might be nice to debounce this).\n", " setTimeout(function () {\n", " fig.push_to_output();\n", " }, 1000);\n", "};\n", "\n", "mpl.figure.prototype._init_toolbar = function () {\n", " var fig = this;\n", "\n", " var toolbar = document.createElement('div');\n", " toolbar.classList = 'btn-toolbar';\n", " this.root.appendChild(toolbar);\n", "\n", " function on_click_closure(name) {\n", " return function (_event) {\n", " return fig.toolbar_button_onclick(name);\n", " };\n", " }\n", "\n", " function on_mouseover_closure(tooltip) {\n", " return function (event) {\n", " if (!event.currentTarget.disabled) {\n", " return fig.toolbar_button_onmouseover(tooltip);\n", " }\n", " };\n", " }\n", "\n", " fig.buttons = {};\n", " var buttonGroup = document.createElement('div');\n", " buttonGroup.classList = 'btn-group';\n", " var button;\n", " for (var toolbar_ind in mpl.toolbar_items) {\n", " var name = mpl.toolbar_items[toolbar_ind][0];\n", " var tooltip = mpl.toolbar_items[toolbar_ind][1];\n", " var image = mpl.toolbar_items[toolbar_ind][2];\n", " var method_name = mpl.toolbar_items[toolbar_ind][3];\n", "\n", " if (!name) {\n", " /* Instead of a spacer, we start a new button group. */\n", " if (buttonGroup.hasChildNodes()) {\n", " toolbar.appendChild(buttonGroup);\n", " }\n", " buttonGroup = document.createElement('div');\n", " buttonGroup.classList = 'btn-group';\n", " continue;\n", " }\n", "\n", " button = fig.buttons[name] = document.createElement('button');\n", " button.classList = 'btn btn-default';\n", " button.href = '#';\n", " button.title = name;\n", " button.innerHTML = '';\n", " button.addEventListener('click', on_click_closure(method_name));\n", " button.addEventListener('mouseover', on_mouseover_closure(tooltip));\n", " buttonGroup.appendChild(button);\n", " }\n", "\n", " if (buttonGroup.hasChildNodes()) {\n", " toolbar.appendChild(buttonGroup);\n", " }\n", "\n", " // Add the status bar.\n", " var status_bar = document.createElement('span');\n", " status_bar.classList = 'mpl-message pull-right';\n", " toolbar.appendChild(status_bar);\n", " this.message = status_bar;\n", "\n", " // Add the close button to the window.\n", " var buttongrp = document.createElement('div');\n", " buttongrp.classList = 'btn-group inline pull-right';\n", " button = document.createElement('button');\n", " button.classList = 'btn btn-mini btn-primary';\n", " button.href = '#';\n", " button.title = 'Stop Interaction';\n", " button.innerHTML = '';\n", " button.addEventListener('click', function (_evt) {\n", " fig.handle_close(fig, {});\n", " });\n", " button.addEventListener(\n", " 'mouseover',\n", " on_mouseover_closure('Stop Interaction')\n", " );\n", " buttongrp.appendChild(button);\n", " var titlebar = this.root.querySelector('.ui-dialog-titlebar');\n", " titlebar.insertBefore(buttongrp, titlebar.firstChild);\n", "};\n", "\n", "mpl.figure.prototype._remove_fig_handler = function (event) {\n", " var fig = event.data.fig;\n", " if (event.target !== this) {\n", " // Ignore bubbled events from children.\n", " return;\n", " }\n", " fig.close_ws(fig, {});\n", "};\n", "\n", "mpl.figure.prototype._root_extra_style = function (el) {\n", " el.style.boxSizing = 'content-box'; // override notebook setting of border-box.\n", "};\n", "\n", "mpl.figure.prototype._canvas_extra_style = function (el) {\n", " // this is important to make the div 'focusable\n", " el.setAttribute('tabindex', 0);\n", " // reach out to IPython and tell the keyboard manager to turn it's self\n", " // off when our div gets focus\n", "\n", " // location in version 3\n", " if (IPython.notebook.keyboard_manager) {\n", " IPython.notebook.keyboard_manager.register_events(el);\n", " } else {\n", " // location in version 2\n", " IPython.keyboard_manager.register_events(el);\n", " }\n", "};\n", "\n", "mpl.figure.prototype._key_event_extra = function (event, _name) {\n", " var manager = IPython.notebook.keyboard_manager;\n", " if (!manager) {\n", " manager = IPython.keyboard_manager;\n", " }\n", "\n", " // Check for shift+enter\n", " if (event.shiftKey && event.which === 13) {\n", " this.canvas_div.blur();\n", " // select the cell after this one\n", " var index = IPython.notebook.find_cell_index(this.cell_info[0]);\n", " IPython.notebook.select(index + 1);\n", " }\n", "};\n", "\n", "mpl.figure.prototype.handle_save = function (fig, _msg) {\n", " fig.ondownload(fig, null);\n", "};\n", "\n", "mpl.find_output_cell = function (html_output) {\n", " // Return the cell and output element which can be found *uniquely* in the notebook.\n", " // Note - this is a bit hacky, but it is done because the \"notebook_saving.Notebook\"\n", " // IPython event is triggered only after the cells have been serialised, which for\n", " // our purposes (turning an active figure into a static one), is too late.\n", " var cells = IPython.notebook.get_cells();\n", " var ncells = cells.length;\n", " for (var i = 0; i < ncells; i++) {\n", " var cell = cells[i];\n", " if (cell.cell_type === 'code') {\n", " for (var j = 0; j < cell.output_area.outputs.length; j++) {\n", " var data = cell.output_area.outputs[j];\n", " if (data.data) {\n", " // IPython >= 3 moved mimebundle to data attribute of output\n", " data = data.data;\n", " }\n", " if (data['text/html'] === html_output) {\n", " return [cell, data, j];\n", " }\n", " }\n", " }\n", " }\n", "};\n", "\n", "// Register the function which deals with the matplotlib target/channel.\n", "// The kernel may be null if the page has been refreshed.\n", "if (IPython.notebook.kernel !== null) {\n", " IPython.notebook.kernel.comm_manager.register_target(\n", " 'matplotlib',\n", " mpl.mpl_figure_comm\n", " );\n", "}\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# %matplotlib qt\n", "%matplotlib nbagg\n", "ax = ut.plotSetup3d(-7, 7, -7, 7, -10, 10, figsize = (7, 7))\n", "v = [4.0, 4.0, 2.0]\n", "u = [-4.0, 3.0, 1.0]\n", "npts = 50\n", "# set locations of points that fall within x,y\n", "xc = -7.0 + 14.0 * np.random.random(npts)\n", "yc = -7.0 + 14.0 * np.random.random(npts)\n", "A = np.array([u,v]).T\n", "# project these points onto the plane\n", "P = A @ np.linalg.inv(A.T @ A) @ A.T\n", "coords = P @ np.array([xc, yc, np.zeros(npts)])\n", "coords[2] += np.random.randn(npts)\n", "ax.plot(coords[0], coords[1], 'ro', zs=coords[2], markersize = 6);" ] }, { "cell_type": "markdown", "metadata": { "collapsed": true, "slideshow": { "slide_type": "slide" } }, "source": [ "__Example.__ \n", "\n", "In geography, local models of terrain are constructed from data $(u_1, v_1, y_1), \\dots, (u_n, v_n, y_n)$ where $u_j, v_j$, and $y_j$ are latitude, longitude, and altitude, respectively." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "Let's describe the linear models that gives a least-squares fit to such data. The solution is called the least-squares _plane._" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "__Solution.__ We expect the data to satisfy these equations:\n", "\n", "$$y_1 = \\beta_0 + \\beta_1 u_1 + \\beta_2 v_1 + \\epsilon_1$$\n", "$$y_1 = \\beta_0 + \\beta_1 u_2 + \\beta_2 v_2 + \\epsilon_2$$\n", "$$\\vdots$$\n", "$$y_1 = \\beta_0 + \\beta_1 u_n + \\beta_2 v_n + \\epsilon_n$$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "This system has the matrix for $\\mathbf{y} = X\\mathbf{\\beta} + \\epsilon,$ where\n", "\n", "$$\\mathbf{y} = \\begin{bmatrix}y_1\\\\y_1\\\\\\vdots\\\\y_n\\end{bmatrix},\\;\\;X = \\begin{bmatrix}1&u_1&v_1\\\\1&u_2&v_2\\\\\\vdots&\\vdots&\\vdots\\\\1&u_n&v_n\\end{bmatrix},\\;\\;\\mathbf{\\beta}=\\begin{bmatrix}\\beta_0\\\\\\beta_1\\\\\\beta_2\\end{bmatrix},\\;\\;\\epsilon = \\begin{bmatrix}\\epsilon_1\\\\\\epsilon_2\\\\\\vdots\\\\\\epsilon_n\\end{bmatrix}$$" ] }, { "cell_type": "code", "execution_count": 14, "metadata": { "hide_input": true, "slideshow": { "slide_type": "fragment" }, "tags": [ "hide-input" ] }, "outputs": [ { "data": { "application/javascript": [ "/* Put everything inside the global mpl namespace */\n", "/* global mpl */\n", "window.mpl = {};\n", "\n", "mpl.get_websocket_type = function () {\n", " if (typeof WebSocket !== 'undefined') {\n", " return WebSocket;\n", " } else if (typeof MozWebSocket !== 'undefined') {\n", " return MozWebSocket;\n", " } else {\n", " alert(\n", " 'Your browser does not have WebSocket support. ' +\n", " 'Please try Chrome, Safari or Firefox ≥ 6. ' +\n", " 'Firefox 4 and 5 are also supported but you ' +\n", " 'have to enable WebSockets in about:config.'\n", " );\n", " }\n", "};\n", "\n", "mpl.figure = function (figure_id, websocket, ondownload, parent_element) {\n", " this.id = figure_id;\n", "\n", " this.ws = websocket;\n", "\n", " this.supports_binary = this.ws.binaryType !== undefined;\n", "\n", " if (!this.supports_binary) {\n", " var warnings = document.getElementById('mpl-warnings');\n", " if (warnings) {\n", " warnings.style.display = 'block';\n", " warnings.textContent =\n", " 'This browser does not support binary websocket messages. ' +\n", " 'Performance may be slow.';\n", " }\n", " }\n", "\n", " this.imageObj = new Image();\n", "\n", " this.context = undefined;\n", " this.message = undefined;\n", " this.canvas = undefined;\n", " this.rubberband_canvas = undefined;\n", " this.rubberband_context = undefined;\n", " this.format_dropdown = undefined;\n", "\n", " this.image_mode = 'full';\n", "\n", " this.root = document.createElement('div');\n", " this.root.setAttribute('style', 'display: inline-block');\n", " this._root_extra_style(this.root);\n", "\n", " parent_element.appendChild(this.root);\n", "\n", " this._init_header(this);\n", " this._init_canvas(this);\n", " this._init_toolbar(this);\n", "\n", " var fig = this;\n", "\n", " this.waiting = false;\n", "\n", " this.ws.onopen = function () {\n", " fig.send_message('supports_binary', { value: fig.supports_binary });\n", " fig.send_message('send_image_mode', {});\n", " if (fig.ratio !== 1) {\n", " fig.send_message('set_dpi_ratio', { dpi_ratio: fig.ratio });\n", " }\n", " fig.send_message('refresh', {});\n", " };\n", "\n", " this.imageObj.onload = function () {\n", " if (fig.image_mode === 'full') {\n", " // Full images could contain transparency (where diff images\n", " // almost always do), so we need to clear the canvas so that\n", " // there is no ghosting.\n", " fig.context.clearRect(0, 0, fig.canvas.width, fig.canvas.height);\n", " }\n", " fig.context.drawImage(fig.imageObj, 0, 0);\n", " };\n", "\n", " this.imageObj.onunload = function () {\n", " fig.ws.close();\n", " };\n", "\n", " this.ws.onmessage = this._make_on_message_function(this);\n", "\n", " this.ondownload = ondownload;\n", "};\n", "\n", "mpl.figure.prototype._init_header = function () {\n", " var titlebar = document.createElement('div');\n", " titlebar.classList =\n", " 'ui-dialog-titlebar ui-widget-header ui-corner-all ui-helper-clearfix';\n", " var titletext = document.createElement('div');\n", " titletext.classList = 'ui-dialog-title';\n", " titletext.setAttribute(\n", " 'style',\n", " 'width: 100%; text-align: center; padding: 3px;'\n", " );\n", " titlebar.appendChild(titletext);\n", " this.root.appendChild(titlebar);\n", " this.header = titletext;\n", "};\n", "\n", "mpl.figure.prototype._canvas_extra_style = function (_canvas_div) {};\n", "\n", "mpl.figure.prototype._root_extra_style = function (_canvas_div) {};\n", "\n", "mpl.figure.prototype._init_canvas = function () {\n", " var fig = this;\n", "\n", " var canvas_div = (this.canvas_div = document.createElement('div'));\n", " canvas_div.setAttribute(\n", " 'style',\n", " 'border: 1px solid #ddd;' +\n", " 'box-sizing: content-box;' +\n", " 'clear: both;' +\n", " 'min-height: 1px;' +\n", " 'min-width: 1px;' +\n", " 'outline: 0;' +\n", " 'overflow: hidden;' +\n", " 'position: relative;' +\n", " 'resize: both;'\n", " );\n", "\n", " function on_keyboard_event_closure(name) {\n", " return function (event) {\n", " return fig.key_event(event, name);\n", " };\n", " }\n", "\n", " canvas_div.addEventListener(\n", " 'keydown',\n", " on_keyboard_event_closure('key_press')\n", " );\n", " canvas_div.addEventListener(\n", " 'keyup',\n", " on_keyboard_event_closure('key_release')\n", " );\n", "\n", " this._canvas_extra_style(canvas_div);\n", " this.root.appendChild(canvas_div);\n", "\n", " var canvas = (this.canvas = document.createElement('canvas'));\n", " canvas.classList.add('mpl-canvas');\n", " canvas.setAttribute('style', 'box-sizing: content-box;');\n", "\n", " this.context = canvas.getContext('2d');\n", "\n", " var backingStore =\n", " this.context.backingStorePixelRatio ||\n", " this.context.webkitBackingStorePixelRatio ||\n", " this.context.mozBackingStorePixelRatio ||\n", " this.context.msBackingStorePixelRatio ||\n", " this.context.oBackingStorePixelRatio ||\n", " this.context.backingStorePixelRatio ||\n", " 1;\n", "\n", " this.ratio = (window.devicePixelRatio || 1) / backingStore;\n", "\n", " var rubberband_canvas = (this.rubberband_canvas = document.createElement(\n", " 'canvas'\n", " ));\n", " rubberband_canvas.setAttribute(\n", " 'style',\n", " 'box-sizing: content-box; position: absolute; left: 0; top: 0; z-index: 1;'\n", " );\n", "\n", " // Apply a ponyfill if ResizeObserver is not implemented by browser.\n", " if (this.ResizeObserver === undefined) {\n", " if (window.ResizeObserver !== undefined) {\n", " this.ResizeObserver = window.ResizeObserver;\n", " } else {\n", " var obs = _JSXTOOLS_RESIZE_OBSERVER({});\n", " this.ResizeObserver = obs.ResizeObserver;\n", " }\n", " }\n", "\n", " this.resizeObserverInstance = new this.ResizeObserver(function (entries) {\n", " var nentries = entries.length;\n", " for (var i = 0; i < nentries; i++) {\n", " var entry = entries[i];\n", " var width, height;\n", " if (entry.contentBoxSize) {\n", " if (entry.contentBoxSize instanceof Array) {\n", " // Chrome 84 implements new version of spec.\n", " width = entry.contentBoxSize[0].inlineSize;\n", " height = entry.contentBoxSize[0].blockSize;\n", " } else {\n", " // Firefox implements old version of spec.\n", " width = entry.contentBoxSize.inlineSize;\n", " height = entry.contentBoxSize.blockSize;\n", " }\n", " } else {\n", " // Chrome <84 implements even older version of spec.\n", " width = entry.contentRect.width;\n", " height = entry.contentRect.height;\n", " }\n", "\n", " // Keep the size of the canvas and rubber band canvas in sync with\n", " // the canvas container.\n", " if (entry.devicePixelContentBoxSize) {\n", " // Chrome 84 implements new version of spec.\n", " canvas.setAttribute(\n", " 'width',\n", " entry.devicePixelContentBoxSize[0].inlineSize\n", " );\n", " canvas.setAttribute(\n", " 'height',\n", " entry.devicePixelContentBoxSize[0].blockSize\n", " );\n", " } else {\n", " canvas.setAttribute('width', width * fig.ratio);\n", " canvas.setAttribute('height', height * fig.ratio);\n", " }\n", " canvas.setAttribute(\n", " 'style',\n", " 'width: ' + width + 'px; height: ' + height + 'px;'\n", " );\n", "\n", " rubberband_canvas.setAttribute('width', width);\n", " rubberband_canvas.setAttribute('height', height);\n", "\n", " // And update the size in Python. We ignore the initial 0/0 size\n", " // that occurs as the element is placed into the DOM, which should\n", " // otherwise not happen due to the minimum size styling.\n", " if (fig.ws.readyState == 1 && width != 0 && height != 0) {\n", " fig.request_resize(width, height);\n", " }\n", " }\n", " });\n", " this.resizeObserverInstance.observe(canvas_div);\n", "\n", " function on_mouse_event_closure(name) {\n", " return function (event) {\n", " return fig.mouse_event(event, name);\n", " };\n", " }\n", "\n", " rubberband_canvas.addEventListener(\n", " 'mousedown',\n", " on_mouse_event_closure('button_press')\n", " );\n", " rubberband_canvas.addEventListener(\n", " 'mouseup',\n", " on_mouse_event_closure('button_release')\n", " );\n", " rubberband_canvas.addEventListener(\n", " 'dblclick',\n", " on_mouse_event_closure('dblclick')\n", " );\n", " // Throttle sequential mouse events to 1 every 20ms.\n", " rubberband_canvas.addEventListener(\n", " 'mousemove',\n", " on_mouse_event_closure('motion_notify')\n", " );\n", "\n", " rubberband_canvas.addEventListener(\n", " 'mouseenter',\n", " on_mouse_event_closure('figure_enter')\n", " );\n", " rubberband_canvas.addEventListener(\n", " 'mouseleave',\n", " on_mouse_event_closure('figure_leave')\n", " );\n", "\n", " canvas_div.addEventListener('wheel', function (event) {\n", " if (event.deltaY < 0) {\n", " event.step = 1;\n", " } else {\n", " event.step = -1;\n", " }\n", " on_mouse_event_closure('scroll')(event);\n", " });\n", "\n", " canvas_div.appendChild(canvas);\n", " canvas_div.appendChild(rubberband_canvas);\n", "\n", " this.rubberband_context = rubberband_canvas.getContext('2d');\n", " this.rubberband_context.strokeStyle = '#000000';\n", "\n", " this._resize_canvas = function (width, height, forward) {\n", " if (forward) {\n", " canvas_div.style.width = width + 'px';\n", " canvas_div.style.height = height + 'px';\n", " }\n", " };\n", "\n", " // Disable right mouse context menu.\n", " this.rubberband_canvas.addEventListener('contextmenu', function (_e) {\n", " event.preventDefault();\n", " return false;\n", " });\n", "\n", " function set_focus() {\n", " canvas.focus();\n", " canvas_div.focus();\n", " }\n", "\n", " window.setTimeout(set_focus, 100);\n", "};\n", "\n", "mpl.figure.prototype._init_toolbar = function () {\n", " var fig = this;\n", "\n", " var toolbar = document.createElement('div');\n", " toolbar.classList = 'mpl-toolbar';\n", " this.root.appendChild(toolbar);\n", "\n", " function on_click_closure(name) {\n", " return function (_event) {\n", " return fig.toolbar_button_onclick(name);\n", " };\n", " }\n", "\n", " function on_mouseover_closure(tooltip) {\n", " return function (event) {\n", " if (!event.currentTarget.disabled) {\n", " return fig.toolbar_button_onmouseover(tooltip);\n", " }\n", " };\n", " }\n", "\n", " fig.buttons = {};\n", " var buttonGroup = document.createElement('div');\n", " buttonGroup.classList = 'mpl-button-group';\n", " for (var toolbar_ind in mpl.toolbar_items) {\n", " var name = mpl.toolbar_items[toolbar_ind][0];\n", " var tooltip = mpl.toolbar_items[toolbar_ind][1];\n", " var image = mpl.toolbar_items[toolbar_ind][2];\n", " var method_name = mpl.toolbar_items[toolbar_ind][3];\n", "\n", " if (!name) {\n", " /* Instead of a spacer, we start a new button group. */\n", " if (buttonGroup.hasChildNodes()) {\n", " toolbar.appendChild(buttonGroup);\n", " }\n", " buttonGroup = document.createElement('div');\n", " buttonGroup.classList = 'mpl-button-group';\n", " continue;\n", " }\n", "\n", " var button = (fig.buttons[name] = document.createElement('button'));\n", " button.classList = 'mpl-widget';\n", " button.setAttribute('role', 'button');\n", " button.setAttribute('aria-disabled', 'false');\n", " button.addEventListener('click', on_click_closure(method_name));\n", " button.addEventListener('mouseover', on_mouseover_closure(tooltip));\n", "\n", " var icon_img = document.createElement('img');\n", " icon_img.src = '_images/' + image + '.png';\n", " icon_img.srcset = '_images/' + image + '_large.png 2x';\n", " icon_img.alt = tooltip;\n", " button.appendChild(icon_img);\n", "\n", " buttonGroup.appendChild(button);\n", " }\n", "\n", " if (buttonGroup.hasChildNodes()) {\n", " toolbar.appendChild(buttonGroup);\n", " }\n", "\n", " var fmt_picker = document.createElement('select');\n", " fmt_picker.classList = 'mpl-widget';\n", " toolbar.appendChild(fmt_picker);\n", " this.format_dropdown = fmt_picker;\n", "\n", " for (var ind in mpl.extensions) {\n", " var fmt = mpl.extensions[ind];\n", " var option = document.createElement('option');\n", " option.selected = fmt === mpl.default_extension;\n", " option.innerHTML = fmt;\n", " fmt_picker.appendChild(option);\n", " }\n", "\n", " var status_bar = document.createElement('span');\n", " status_bar.classList = 'mpl-message';\n", " toolbar.appendChild(status_bar);\n", " this.message = status_bar;\n", "};\n", "\n", "mpl.figure.prototype.request_resize = function (x_pixels, y_pixels) {\n", " // Request matplotlib to resize the figure. Matplotlib will then trigger a resize in the client,\n", " // which will in turn request a refresh of the image.\n", " this.send_message('resize', { width: x_pixels, height: y_pixels });\n", "};\n", "\n", "mpl.figure.prototype.send_message = function (type, properties) {\n", " properties['type'] = type;\n", " properties['figure_id'] = this.id;\n", " this.ws.send(JSON.stringify(properties));\n", "};\n", "\n", "mpl.figure.prototype.send_draw_message = function () {\n", " if (!this.waiting) {\n", " this.waiting = true;\n", " this.ws.send(JSON.stringify({ type: 'draw', figure_id: this.id }));\n", " }\n", "};\n", "\n", "mpl.figure.prototype.handle_save = function (fig, _msg) {\n", " var format_dropdown = fig.format_dropdown;\n", " var format = format_dropdown.options[format_dropdown.selectedIndex].value;\n", " fig.ondownload(fig, format);\n", "};\n", "\n", "mpl.figure.prototype.handle_resize = function (fig, msg) {\n", " var size = msg['size'];\n", " if (size[0] !== fig.canvas.width || size[1] !== fig.canvas.height) {\n", " fig._resize_canvas(size[0], size[1], msg['forward']);\n", " fig.send_message('refresh', {});\n", " }\n", "};\n", "\n", "mpl.figure.prototype.handle_rubberband = function (fig, msg) {\n", " var x0 = msg['x0'] / fig.ratio;\n", " var y0 = (fig.canvas.height - msg['y0']) / fig.ratio;\n", " var x1 = msg['x1'] / fig.ratio;\n", " var y1 = (fig.canvas.height - msg['y1']) / fig.ratio;\n", " x0 = Math.floor(x0) + 0.5;\n", " y0 = Math.floor(y0) + 0.5;\n", " x1 = Math.floor(x1) + 0.5;\n", " y1 = Math.floor(y1) + 0.5;\n", " var min_x = Math.min(x0, x1);\n", " var min_y = Math.min(y0, y1);\n", " var width = Math.abs(x1 - x0);\n", " var height = Math.abs(y1 - y0);\n", "\n", " fig.rubberband_context.clearRect(\n", " 0,\n", " 0,\n", " fig.canvas.width / fig.ratio,\n", " fig.canvas.height / fig.ratio\n", " );\n", "\n", " fig.rubberband_context.strokeRect(min_x, min_y, width, height);\n", "};\n", "\n", "mpl.figure.prototype.handle_figure_label = function (fig, msg) {\n", " // Updates the figure title.\n", " fig.header.textContent = msg['label'];\n", "};\n", "\n", "mpl.figure.prototype.handle_cursor = function (fig, msg) {\n", " var cursor = msg['cursor'];\n", " switch (cursor) {\n", " case 0:\n", " cursor = 'pointer';\n", " break;\n", " case 1:\n", " cursor = 'default';\n", " break;\n", " case 2:\n", " cursor = 'crosshair';\n", " break;\n", " case 3:\n", " cursor = 'move';\n", " break;\n", " }\n", " fig.rubberband_canvas.style.cursor = cursor;\n", "};\n", "\n", "mpl.figure.prototype.handle_message = function (fig, msg) {\n", " fig.message.textContent = msg['message'];\n", "};\n", "\n", "mpl.figure.prototype.handle_draw = function (fig, _msg) {\n", " // Request the server to send over a new figure.\n", " fig.send_draw_message();\n", "};\n", "\n", "mpl.figure.prototype.handle_image_mode = function (fig, msg) {\n", " fig.image_mode = msg['mode'];\n", "};\n", "\n", "mpl.figure.prototype.handle_history_buttons = function (fig, msg) {\n", " for (var key in msg) {\n", " if (!(key in fig.buttons)) {\n", " continue;\n", " }\n", " fig.buttons[key].disabled = !msg[key];\n", " fig.buttons[key].setAttribute('aria-disabled', !msg[key]);\n", " }\n", "};\n", "\n", "mpl.figure.prototype.handle_navigate_mode = function (fig, msg) {\n", " if (msg['mode'] === 'PAN') {\n", " fig.buttons['Pan'].classList.add('active');\n", " fig.buttons['Zoom'].classList.remove('active');\n", " } else if (msg['mode'] === 'ZOOM') {\n", " fig.buttons['Pan'].classList.remove('active');\n", " fig.buttons['Zoom'].classList.add('active');\n", " } else {\n", " fig.buttons['Pan'].classList.remove('active');\n", " fig.buttons['Zoom'].classList.remove('active');\n", " }\n", "};\n", "\n", "mpl.figure.prototype.updated_canvas_event = function () {\n", " // Called whenever the canvas gets updated.\n", " this.send_message('ack', {});\n", "};\n", "\n", "// A function to construct a web socket function for onmessage handling.\n", "// Called in the figure constructor.\n", "mpl.figure.prototype._make_on_message_function = function (fig) {\n", " return function socket_on_message(evt) {\n", " if (evt.data instanceof Blob) {\n", " var img = evt.data;\n", " if (img.type !== 'image/png') {\n", " /* FIXME: We get \"Resource interpreted as Image but\n", " * transferred with MIME type text/plain:\" errors on\n", " * Chrome. But how to set the MIME type? It doesn't seem\n", " * to be part of the websocket stream */\n", " img.type = 'image/png';\n", " }\n", "\n", " /* Free the memory for the previous frames */\n", " if (fig.imageObj.src) {\n", " (window.URL || window.webkitURL).revokeObjectURL(\n", " fig.imageObj.src\n", " );\n", " }\n", "\n", " fig.imageObj.src = (window.URL || window.webkitURL).createObjectURL(\n", " img\n", " );\n", " fig.updated_canvas_event();\n", " fig.waiting = false;\n", " return;\n", " } else if (\n", " typeof evt.data === 'string' &&\n", " evt.data.slice(0, 21) === 'data:image/png;base64'\n", " ) {\n", " fig.imageObj.src = evt.data;\n", " fig.updated_canvas_event();\n", " fig.waiting = false;\n", " return;\n", " }\n", "\n", " var msg = JSON.parse(evt.data);\n", " var msg_type = msg['type'];\n", "\n", " // Call the \"handle_{type}\" callback, which takes\n", " // the figure and JSON message as its only arguments.\n", " try {\n", " var callback = fig['handle_' + msg_type];\n", " } catch (e) {\n", " console.log(\n", " \"No handler for the '\" + msg_type + \"' message type: \",\n", " msg\n", " );\n", " return;\n", " }\n", "\n", " if (callback) {\n", " try {\n", " // console.log(\"Handling '\" + msg_type + \"' message: \", msg);\n", " callback(fig, msg);\n", " } catch (e) {\n", " console.log(\n", " \"Exception inside the 'handler_\" + msg_type + \"' callback:\",\n", " e,\n", " e.stack,\n", " msg\n", " );\n", " }\n", " }\n", " };\n", "};\n", "\n", "// from http://stackoverflow.com/questions/1114465/getting-mouse-location-in-canvas\n", "mpl.findpos = function (e) {\n", " //this section is from http://www.quirksmode.org/js/events_properties.html\n", " var targ;\n", " if (!e) {\n", " e = window.event;\n", " }\n", " if (e.target) {\n", " targ = e.target;\n", " } else if (e.srcElement) {\n", " targ = e.srcElement;\n", " }\n", " if (targ.nodeType === 3) {\n", " // defeat Safari bug\n", " targ = targ.parentNode;\n", " }\n", "\n", " // pageX,Y are the mouse positions relative to the document\n", " var boundingRect = targ.getBoundingClientRect();\n", " var x = e.pageX - (boundingRect.left + document.body.scrollLeft);\n", " var y = e.pageY - (boundingRect.top + document.body.scrollTop);\n", "\n", " return { x: x, y: y };\n", "};\n", "\n", "/*\n", " * return a copy of an object with only non-object keys\n", " * we need this to avoid circular references\n", " * http://stackoverflow.com/a/24161582/3208463\n", " */\n", "function simpleKeys(original) {\n", " return Object.keys(original).reduce(function (obj, key) {\n", " if (typeof original[key] !== 'object') {\n", " obj[key] = original[key];\n", " }\n", " return obj;\n", " }, {});\n", "}\n", "\n", "mpl.figure.prototype.mouse_event = function (event, name) {\n", " var canvas_pos = mpl.findpos(event);\n", "\n", " if (name === 'button_press') {\n", " this.canvas.focus();\n", " this.canvas_div.focus();\n", " }\n", "\n", " var x = canvas_pos.x * this.ratio;\n", " var y = canvas_pos.y * this.ratio;\n", "\n", " this.send_message(name, {\n", " x: x,\n", " y: y,\n", " button: event.button,\n", " step: event.step,\n", " guiEvent: simpleKeys(event),\n", " });\n", "\n", " /* This prevents the web browser from automatically changing to\n", " * the text insertion cursor when the button is pressed. We want\n", " * to control all of the cursor setting manually through the\n", " * 'cursor' event from matplotlib */\n", " event.preventDefault();\n", " return false;\n", "};\n", "\n", "mpl.figure.prototype._key_event_extra = function (_event, _name) {\n", " // Handle any extra behaviour associated with a key event\n", "};\n", "\n", "mpl.figure.prototype.key_event = function (event, name) {\n", " // Prevent repeat events\n", " if (name === 'key_press') {\n", " if (event.key === this._key) {\n", " return;\n", " } else {\n", " this._key = event.key;\n", " }\n", " }\n", " if (name === 'key_release') {\n", " this._key = null;\n", " }\n", "\n", " var value = '';\n", " if (event.ctrlKey && event.key !== 'Control') {\n", " value += 'ctrl+';\n", " }\n", " else if (event.altKey && event.key !== 'Alt') {\n", " value += 'alt+';\n", " }\n", " else if (event.shiftKey && event.key !== 'Shift') {\n", " value += 'shift+';\n", " }\n", "\n", " value += 'k' + event.key;\n", "\n", " this._key_event_extra(event, name);\n", "\n", " this.send_message(name, { key: value, guiEvent: simpleKeys(event) });\n", " return false;\n", "};\n", "\n", "mpl.figure.prototype.toolbar_button_onclick = function (name) {\n", " if (name === 'download') {\n", " this.handle_save(this, null);\n", " } else {\n", " this.send_message('toolbar_button', { name: name });\n", " }\n", "};\n", "\n", "mpl.figure.prototype.toolbar_button_onmouseover = function (tooltip) {\n", " this.message.textContent = tooltip;\n", "};\n", "\n", "///////////////// REMAINING CONTENT GENERATED BY embed_js.py /////////////////\n", "// prettier-ignore\n", "var _JSXTOOLS_RESIZE_OBSERVER=function(A){var t,i=new WeakMap,n=new WeakMap,a=new WeakMap,r=new WeakMap,o=new Set;function s(e){if(!(this instanceof s))throw new TypeError(\"Constructor requires 'new' operator\");i.set(this,e)}function h(){throw new TypeError(\"Function is not a constructor\")}function c(e,t,i,n){e=0 in arguments?Number(arguments[0]):0,t=1 in arguments?Number(arguments[1]):0,i=2 in arguments?Number(arguments[2]):0,n=3 in arguments?Number(arguments[3]):0,this.right=(this.x=this.left=e)+(this.width=i),this.bottom=(this.y=this.top=t)+(this.height=n),Object.freeze(this)}function d(){t=requestAnimationFrame(d);var s=new WeakMap,p=new Set;o.forEach((function(t){r.get(t).forEach((function(i){var r=t instanceof window.SVGElement,o=a.get(t),d=r?0:parseFloat(o.paddingTop),f=r?0:parseFloat(o.paddingRight),l=r?0:parseFloat(o.paddingBottom),u=r?0:parseFloat(o.paddingLeft),g=r?0:parseFloat(o.borderTopWidth),m=r?0:parseFloat(o.borderRightWidth),w=r?0:parseFloat(o.borderBottomWidth),b=u+f,F=d+l,v=(r?0:parseFloat(o.borderLeftWidth))+m,W=g+w,y=r?0:t.offsetHeight-W-t.clientHeight,E=r?0:t.offsetWidth-v-t.clientWidth,R=b+v,z=F+W,M=r?t.width:parseFloat(o.width)-R-E,O=r?t.height:parseFloat(o.height)-z-y;if(n.has(t)){var k=n.get(t);if(k[0]===M&&k[1]===O)return}n.set(t,[M,O]);var S=Object.create(h.prototype);S.target=t,S.contentRect=new c(u,d,M,O),s.has(i)||(s.set(i,[]),p.add(i)),s.get(i).push(S)}))})),p.forEach((function(e){i.get(e).call(e,s.get(e),e)}))}return s.prototype.observe=function(i){if(i instanceof window.Element){r.has(i)||(r.set(i,new Set),o.add(i),a.set(i,window.getComputedStyle(i)));var n=r.get(i);n.has(this)||n.add(this),cancelAnimationFrame(t),t=requestAnimationFrame(d)}},s.prototype.unobserve=function(i){if(i instanceof window.Element&&r.has(i)){var n=r.get(i);n.has(this)&&(n.delete(this),n.size||(r.delete(i),o.delete(i))),n.size||r.delete(i),o.size||cancelAnimationFrame(t)}},A.DOMRectReadOnly=c,A.ResizeObserver=s,A.ResizeObserverEntry=h,A}; // eslint-disable-line\n", "mpl.toolbar_items = [[\"Home\", \"Reset original view\", \"fa fa-home icon-home\", \"home\"], [\"Back\", \"Back to previous view\", \"fa fa-arrow-left icon-arrow-left\", \"back\"], [\"Forward\", \"Forward to next view\", \"fa fa-arrow-right icon-arrow-right\", \"forward\"], [\"\", \"\", \"\", \"\"], [\"Pan\", \"Left button pans, Right button zooms\\nx/y fixes axis, CTRL fixes aspect\", \"fa fa-arrows icon-move\", \"pan\"], [\"Zoom\", \"Zoom to rectangle\\nx/y fixes axis, CTRL fixes aspect\", \"fa fa-square-o icon-check-empty\", \"zoom\"], [\"\", \"\", \"\", \"\"], [\"Download\", \"Download plot\", \"fa fa-floppy-o icon-save\", \"download\"]];\n", "\n", "mpl.extensions = [\"eps\", \"jpeg\", \"pgf\", \"pdf\", \"png\", \"ps\", \"raw\", \"svg\", \"tif\"];\n", "\n", "mpl.default_extension = \"png\";/* global mpl */\n", "\n", "var comm_websocket_adapter = function (comm) {\n", " // Create a \"websocket\"-like object which calls the given IPython comm\n", " // object with the appropriate methods. Currently this is a non binary\n", " // socket, so there is still some room for performance tuning.\n", " var ws = {};\n", "\n", " ws.binaryType = comm.kernel.ws.binaryType;\n", " ws.readyState = comm.kernel.ws.readyState;\n", " function updateReadyState(_event) {\n", " if (comm.kernel.ws) {\n", " ws.readyState = comm.kernel.ws.readyState;\n", " } else {\n", " ws.readyState = 3; // Closed state.\n", " }\n", " }\n", " comm.kernel.ws.addEventListener('open', updateReadyState);\n", " comm.kernel.ws.addEventListener('close', updateReadyState);\n", " comm.kernel.ws.addEventListener('error', updateReadyState);\n", "\n", " ws.close = function () {\n", " comm.close();\n", " };\n", " ws.send = function (m) {\n", " //console.log('sending', m);\n", " comm.send(m);\n", " };\n", " // Register the callback with on_msg.\n", " comm.on_msg(function (msg) {\n", " //console.log('receiving', msg['content']['data'], msg);\n", " var data = msg['content']['data'];\n", " if (data['blob'] !== undefined) {\n", " data = {\n", " data: new Blob(msg['buffers'], { type: data['blob'] }),\n", " };\n", " }\n", " // Pass the mpl event to the overridden (by mpl) onmessage function.\n", " ws.onmessage(data);\n", " });\n", " return ws;\n", "};\n", "\n", "mpl.mpl_figure_comm = function (comm, msg) {\n", " // This is the function which gets called when the mpl process\n", " // starts-up an IPython Comm through the \"matplotlib\" channel.\n", "\n", " var id = msg.content.data.id;\n", " // Get hold of the div created by the display call when the Comm\n", " // socket was opened in Python.\n", " var element = document.getElementById(id);\n", " var ws_proxy = comm_websocket_adapter(comm);\n", "\n", " function ondownload(figure, _format) {\n", " window.open(figure.canvas.toDataURL());\n", " }\n", "\n", " var fig = new mpl.figure(id, ws_proxy, ondownload, element);\n", "\n", " // Call onopen now - mpl needs it, as it is assuming we've passed it a real\n", " // web socket which is closed, not our websocket->open comm proxy.\n", " ws_proxy.onopen();\n", "\n", " fig.parent_element = element;\n", " fig.cell_info = mpl.find_output_cell(\"
\");\n", " if (!fig.cell_info) {\n", " console.error('Failed to find cell for figure', id, fig);\n", " return;\n", " }\n", " fig.cell_info[0].output_area.element.on(\n", " 'cleared',\n", " { fig: fig },\n", " fig._remove_fig_handler\n", " );\n", "};\n", "\n", "mpl.figure.prototype.handle_close = function (fig, msg) {\n", " var width = fig.canvas.width / fig.ratio;\n", " fig.cell_info[0].output_area.element.off(\n", " 'cleared',\n", " fig._remove_fig_handler\n", " );\n", " fig.resizeObserverInstance.unobserve(fig.canvas_div);\n", "\n", " // Update the output cell to use the data from the current canvas.\n", " fig.push_to_output();\n", " var dataURL = fig.canvas.toDataURL();\n", " // Re-enable the keyboard manager in IPython - without this line, in FF,\n", " // the notebook keyboard shortcuts fail.\n", " IPython.keyboard_manager.enable();\n", " fig.parent_element.innerHTML =\n", " '';\n", " fig.close_ws(fig, msg);\n", "};\n", "\n", "mpl.figure.prototype.close_ws = function (fig, msg) {\n", " fig.send_message('closing', msg);\n", " // fig.ws.close()\n", "};\n", "\n", "mpl.figure.prototype.push_to_output = function (_remove_interactive) {\n", " // Turn the data on the canvas into data in the output cell.\n", " var width = this.canvas.width / this.ratio;\n", " var dataURL = this.canvas.toDataURL();\n", " this.cell_info[1]['text/html'] =\n", " '';\n", "};\n", "\n", "mpl.figure.prototype.updated_canvas_event = function () {\n", " // Tell IPython that the notebook contents must change.\n", " IPython.notebook.set_dirty(true);\n", " this.send_message('ack', {});\n", " var fig = this;\n", " // Wait a second, then push the new image to the DOM so\n", " // that it is saved nicely (might be nice to debounce this).\n", " setTimeout(function () {\n", " fig.push_to_output();\n", " }, 1000);\n", "};\n", "\n", "mpl.figure.prototype._init_toolbar = function () {\n", " var fig = this;\n", "\n", " var toolbar = document.createElement('div');\n", " toolbar.classList = 'btn-toolbar';\n", " this.root.appendChild(toolbar);\n", "\n", " function on_click_closure(name) {\n", " return function (_event) {\n", " return fig.toolbar_button_onclick(name);\n", " };\n", " }\n", "\n", " function on_mouseover_closure(tooltip) {\n", " return function (event) {\n", " if (!event.currentTarget.disabled) {\n", " return fig.toolbar_button_onmouseover(tooltip);\n", " }\n", " };\n", " }\n", "\n", " fig.buttons = {};\n", " var buttonGroup = document.createElement('div');\n", " buttonGroup.classList = 'btn-group';\n", " var button;\n", " for (var toolbar_ind in mpl.toolbar_items) {\n", " var name = mpl.toolbar_items[toolbar_ind][0];\n", " var tooltip = mpl.toolbar_items[toolbar_ind][1];\n", " var image = mpl.toolbar_items[toolbar_ind][2];\n", " var method_name = mpl.toolbar_items[toolbar_ind][3];\n", "\n", " if (!name) {\n", " /* Instead of a spacer, we start a new button group. */\n", " if (buttonGroup.hasChildNodes()) {\n", " toolbar.appendChild(buttonGroup);\n", " }\n", " buttonGroup = document.createElement('div');\n", " buttonGroup.classList = 'btn-group';\n", " continue;\n", " }\n", "\n", " button = fig.buttons[name] = document.createElement('button');\n", " button.classList = 'btn btn-default';\n", " button.href = '#';\n", " button.title = name;\n", " button.innerHTML = '';\n", " button.addEventListener('click', on_click_closure(method_name));\n", " button.addEventListener('mouseover', on_mouseover_closure(tooltip));\n", " buttonGroup.appendChild(button);\n", " }\n", "\n", " if (buttonGroup.hasChildNodes()) {\n", " toolbar.appendChild(buttonGroup);\n", " }\n", "\n", " // Add the status bar.\n", " var status_bar = document.createElement('span');\n", " status_bar.classList = 'mpl-message pull-right';\n", " toolbar.appendChild(status_bar);\n", " this.message = status_bar;\n", "\n", " // Add the close button to the window.\n", " var buttongrp = document.createElement('div');\n", " buttongrp.classList = 'btn-group inline pull-right';\n", " button = document.createElement('button');\n", " button.classList = 'btn btn-mini btn-primary';\n", " button.href = '#';\n", " button.title = 'Stop Interaction';\n", " button.innerHTML = '';\n", " button.addEventListener('click', function (_evt) {\n", " fig.handle_close(fig, {});\n", " });\n", " button.addEventListener(\n", " 'mouseover',\n", " on_mouseover_closure('Stop Interaction')\n", " );\n", " buttongrp.appendChild(button);\n", " var titlebar = this.root.querySelector('.ui-dialog-titlebar');\n", " titlebar.insertBefore(buttongrp, titlebar.firstChild);\n", "};\n", "\n", "mpl.figure.prototype._remove_fig_handler = function (event) {\n", " var fig = event.data.fig;\n", " if (event.target !== this) {\n", " // Ignore bubbled events from children.\n", " return;\n", " }\n", " fig.close_ws(fig, {});\n", "};\n", "\n", "mpl.figure.prototype._root_extra_style = function (el) {\n", " el.style.boxSizing = 'content-box'; // override notebook setting of border-box.\n", "};\n", "\n", "mpl.figure.prototype._canvas_extra_style = function (el) {\n", " // this is important to make the div 'focusable\n", " el.setAttribute('tabindex', 0);\n", " // reach out to IPython and tell the keyboard manager to turn it's self\n", " // off when our div gets focus\n", "\n", " // location in version 3\n", " if (IPython.notebook.keyboard_manager) {\n", " IPython.notebook.keyboard_manager.register_events(el);\n", " } else {\n", " // location in version 2\n", " IPython.keyboard_manager.register_events(el);\n", " }\n", "};\n", "\n", "mpl.figure.prototype._key_event_extra = function (event, _name) {\n", " var manager = IPython.notebook.keyboard_manager;\n", " if (!manager) {\n", " manager = IPython.keyboard_manager;\n", " }\n", "\n", " // Check for shift+enter\n", " if (event.shiftKey && event.which === 13) {\n", " this.canvas_div.blur();\n", " // select the cell after this one\n", " var index = IPython.notebook.find_cell_index(this.cell_info[0]);\n", " IPython.notebook.select(index + 1);\n", " }\n", "};\n", "\n", "mpl.figure.prototype.handle_save = function (fig, _msg) {\n", " fig.ondownload(fig, null);\n", "};\n", "\n", "mpl.find_output_cell = function (html_output) {\n", " // Return the cell and output element which can be found *uniquely* in the notebook.\n", " // Note - this is a bit hacky, but it is done because the \"notebook_saving.Notebook\"\n", " // IPython event is triggered only after the cells have been serialised, which for\n", " // our purposes (turning an active figure into a static one), is too late.\n", " var cells = IPython.notebook.get_cells();\n", " var ncells = cells.length;\n", " for (var i = 0; i < ncells; i++) {\n", " var cell = cells[i];\n", " if (cell.cell_type === 'code') {\n", " for (var j = 0; j < cell.output_area.outputs.length; j++) {\n", " var data = cell.output_area.outputs[j];\n", " if (data.data) {\n", " // IPython >= 3 moved mimebundle to data attribute of output\n", " data = data.data;\n", " }\n", " if (data['text/html'] === html_output) {\n", " return [cell, data, j];\n", " }\n", " }\n", " }\n", " }\n", "};\n", "\n", "// Register the function which deals with the matplotlib target/channel.\n", "// The kernel may be null if the page has been refreshed.\n", "if (IPython.notebook.kernel !== null) {\n", " IPython.notebook.kernel.comm_manager.register_target(\n", " 'matplotlib',\n", " mpl.mpl_figure_comm\n", " );\n", "}\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "%matplotlib nbagg\n", "ax = ut.plotSetup3d(-7, 7, -7, 7, -10, 10, figsize = (7, 7))\n", "v = [4.0, 4.0, 2.0]\n", "u = [-4.0, 3.0, 1.0]\n", "# plotting the span of v\n", "ut.plotSpan3d(ax,u,v,'Green')\n", "npts = 50\n", "ax.plot(coords[0], coords[1], 'ro', zs = coords[2], markersize=6);" ] }, { "cell_type": "markdown", "metadata": { "hide_input": true, "slideshow": { "slide_type": "fragment" } }, "source": [ "This example shows that the linear model for multiple regression has the same form as the model for the simple regression in the earlier examples." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "We can see that there the general principle is the same across all the different kinds of linear models." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "Once $X$ is defined properly, the normal equations for $\\mathbf{\\beta}$ have the same matrix form, no matter how many variables are involved.\n", "\n", "Thus, for any linear model where $X^TX$ is invertible, the least squares estimate $\\hat{\\mathbf{\\beta}}$ is given by $(X^TX)^{-1}X^T\\mathbf{y}$." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "## Measuring the fit of a regression model: $R^2$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "Given any $X$ and $\\mathbf{y}$, the above algorithm will produce an output $\\hat{\\beta}$.\n", "\n", "But how do we know whether the data is in fact well described by the model?" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "The most common measure of fit is $R^2$.\n", "\n", "$R^2$ measures the __fraction of the variance__ of $\\mathbf{y}$ that can be explained by the model $X\\hat{\\beta}$." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "The variance of $\\mathbf{y}$ is \n", "\n", "$$\\text{Var}(\\mathbf{y}) =\\frac{1}{n} \\sum_{i=1}^n \\left(y_i-\\overline{y}\\right)^2$$\n", "\n", "where $\\overline{y}=\\frac{1}{n}\\sum_{i=1}^ny_i$." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "For any given $n$, we can equally work with just \n", "$$\\sum_{i=1}^n \\left(y_i-\\overline{y}\\right)^2$$\n", "which is called the __Total Sum of Squares__ (TSS)." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "Now to measure the quality of fit of a model, we break TSS down into two components. \n", "\n", "For any given $\\mathbf{x}_i$, the prediction made by the model is $\\hat{y_i} = \\mathbf{x}_i^T\\beta$.\n", "\n", "Therefore, \n", "* the residual $\\epsilon$ is $y_i - \\hat{y_i}$, and \n", "* the part that the model \"explains\" is $\\hat{y_i} - \\overline{y_i}.$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "Se we define Residual Sum of Squares (RSS) as:\n", "\n", "$$\\text{RSS} = \\sum_{i=1}^n \\left(y_i-\\hat{y_i}\\right)^2,$$\n", "\n", "and Explained Sum of Squares (ESS) as:\n", "\n", "$$\\text{ESS} = \\sum_{i=1}^n \\left(\\hat{y_i}-\\overline{y}\\right)^2,$$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "Then it turns out that the total sum of squares is exactly equal to the sum of squares of the residuals plus the sum of squares of the explained part.\n", "\n", "In other words:\n", "\n", "$$\\text{TSS} = \\text{RSS} + \\text{ESS}$$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "Now, a good fit is one in which the model explains a large part of the variance of $\\mathbf{y}$. " ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "So the measure of fit $R^2$ is defined as:\n", "\n", "\\begin{eqnarray}\n", "R^2 & = & \\frac{\\text{ESS}}{\\text{TSS}} = 1-\\frac{\\text{RSS}}{\\text{TSS}}\n", "\\end{eqnarray}" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "As a result, $0\\leq R^2\\leq 1$.\n", "\n", "The closer the value of $R^2$ is to $1$ the better the fit of the regression:\n", "* small values of RSS imply that the residuals are small and therefore we have a better fit." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$R^2$ is called the __coefficient of determination.__ \n", "\n", "It tells us \"how well does the model predict the data?\"" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "__WARNING -- WARNING -- WARNING__\n", "\n", "Do __not__ confuse $R^2$ with Pearson's $r$, which is the __correlation coefficient.__\n", "\n", "(To make matters worse, sometimes people talk about $r^2$... very confusing!)\n", "\n", "The correlation coefficient tells us whether two variables are __correlated__. \n", "\n", "However, just because two variables are correlated does not mean that one is a good __predictor__ of the other!\n", "\n", "To compare ground truth with predictions, we always use $R^2$." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "## OLS in Practice" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "First, we'll look at a test case on synthetic data." ] }, { "cell_type": "code", "execution_count": 15, "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [], "source": [ "from sklearn import datasets\n", "X, y = datasets.make_regression(n_samples=100, n_features=20, n_informative=5, bias=0.1, noise=30, random_state=1)" ] }, { "cell_type": "code", "execution_count": 16, "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " OLS Regression Results \n", "=======================================================================================\n", "Dep. Variable: y R-squared (uncentered): 0.969\n", "Model: OLS Adj. R-squared (uncentered): 0.961\n", "Method: Least Squares F-statistic: 123.8\n", "Date: Mon, 01 Nov 2021 Prob (F-statistic): 1.03e-51\n", "Time: 14:03:54 Log-Likelihood: -468.30\n", "No. Observations: 100 AIC: 976.6\n", "Df Residuals: 80 BIC: 1029.\n", "Df Model: 20 \n", "Covariance Type: nonrobust \n", "==============================================================================\n", " coef std err t P>|t| [0.025 0.975]\n", "------------------------------------------------------------------------------\n", "x1 12.5673 3.471 3.620 0.001 5.659 19.476\n", "x2 -3.8321 2.818 -1.360 0.178 -9.440 1.776\n", "x3 -2.4197 3.466 -0.698 0.487 -9.316 4.477\n", "x4 2.0143 3.086 0.653 0.516 -4.127 8.155\n", "x5 -2.6256 3.445 -0.762 0.448 -9.481 4.230\n", "x6 0.7894 3.159 0.250 0.803 -5.497 7.076\n", "x7 -3.0684 3.595 -0.853 0.396 -10.224 4.087\n", "x8 90.1383 3.211 28.068 0.000 83.747 96.529\n", "x9 -0.0133 3.400 -0.004 0.997 -6.779 6.752\n", "x10 15.2675 3.248 4.701 0.000 8.804 21.731\n", "x11 -0.2247 3.339 -0.067 0.947 -6.869 6.419\n", "x12 0.0773 3.546 0.022 0.983 -6.979 7.133\n", "x13 -0.2452 3.250 -0.075 0.940 -6.712 6.222\n", "x14 90.0179 3.544 25.402 0.000 82.966 97.070\n", "x15 1.6684 3.727 0.448 0.656 -5.748 9.085\n", "x16 4.3945 2.742 1.603 0.113 -1.062 9.851\n", "x17 8.7918 3.399 2.587 0.012 2.028 15.556\n", "x18 73.3771 3.425 21.426 0.000 66.562 80.193\n", "x19 -1.9139 3.515 -0.545 0.588 -8.908 5.080\n", "x20 -1.3206 3.284 -0.402 0.689 -7.855 5.214\n", "==============================================================================\n", "Omnibus: 5.248 Durbin-Watson: 2.018\n", "Prob(Omnibus): 0.073 Jarque-Bera (JB): 4.580\n", "Skew: 0.467 Prob(JB): 0.101\n", "Kurtosis: 3.475 Cond. No. 2.53\n", "==============================================================================\n", "\n", "Notes:\n", "[1] R² is computed without centering (uncentered) since the model does not contain a constant.\n", "[2] Standard Errors assume that the covariance matrix of the errors is correctly specified.\n" ] } ], "source": [ "model = sm.OLS(y, X)\n", "results = model.fit()\n", "print(results.summary())" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "The $R^2$ value is very good. We can see that the linear model does a very good job of predicting the observations $y_i$.\n", "\n", "However, some of the independent variables may not contribute to the accuracy of the prediction. " ] }, { "cell_type": "code", "execution_count": 17, "metadata": { "hide_input": true, "slideshow": { "slide_type": "fragment" }, "tags": [ "hide-input" ] }, "outputs": [ { "data": { "application/javascript": [ "/* Put everything inside the global mpl namespace */\n", "/* global mpl */\n", "window.mpl = {};\n", "\n", "mpl.get_websocket_type = function () {\n", " if (typeof WebSocket !== 'undefined') {\n", " return WebSocket;\n", " } else if (typeof MozWebSocket !== 'undefined') {\n", " return MozWebSocket;\n", " } else {\n", " alert(\n", " 'Your browser does not have WebSocket support. ' +\n", " 'Please try Chrome, Safari or Firefox ≥ 6. ' +\n", " 'Firefox 4 and 5 are also supported but you ' +\n", " 'have to enable WebSockets in about:config.'\n", " );\n", " }\n", "};\n", "\n", "mpl.figure = function (figure_id, websocket, ondownload, parent_element) {\n", " this.id = figure_id;\n", "\n", " this.ws = websocket;\n", "\n", " this.supports_binary = this.ws.binaryType !== undefined;\n", "\n", " if (!this.supports_binary) {\n", " var warnings = document.getElementById('mpl-warnings');\n", " if (warnings) {\n", " warnings.style.display = 'block';\n", " warnings.textContent =\n", " 'This browser does not support binary websocket messages. ' +\n", " 'Performance may be slow.';\n", " }\n", " }\n", "\n", " this.imageObj = new Image();\n", "\n", " this.context = undefined;\n", " this.message = undefined;\n", " this.canvas = undefined;\n", " this.rubberband_canvas = undefined;\n", " this.rubberband_context = undefined;\n", " this.format_dropdown = undefined;\n", "\n", " this.image_mode = 'full';\n", "\n", " this.root = document.createElement('div');\n", " this.root.setAttribute('style', 'display: inline-block');\n", " this._root_extra_style(this.root);\n", "\n", " parent_element.appendChild(this.root);\n", "\n", " this._init_header(this);\n", " this._init_canvas(this);\n", " this._init_toolbar(this);\n", "\n", " var fig = this;\n", "\n", " this.waiting = false;\n", "\n", " this.ws.onopen = function () {\n", " fig.send_message('supports_binary', { value: fig.supports_binary });\n", " fig.send_message('send_image_mode', {});\n", " if (fig.ratio !== 1) {\n", " fig.send_message('set_dpi_ratio', { dpi_ratio: fig.ratio });\n", " }\n", " fig.send_message('refresh', {});\n", " };\n", "\n", " this.imageObj.onload = function () {\n", " if (fig.image_mode === 'full') {\n", " // Full images could contain transparency (where diff images\n", " // almost always do), so we need to clear the canvas so that\n", " // there is no ghosting.\n", " fig.context.clearRect(0, 0, fig.canvas.width, fig.canvas.height);\n", " }\n", " fig.context.drawImage(fig.imageObj, 0, 0);\n", " };\n", "\n", " this.imageObj.onunload = function () {\n", " fig.ws.close();\n", " };\n", "\n", " this.ws.onmessage = this._make_on_message_function(this);\n", "\n", " this.ondownload = ondownload;\n", "};\n", "\n", "mpl.figure.prototype._init_header = function () {\n", " var titlebar = document.createElement('div');\n", " titlebar.classList =\n", " 'ui-dialog-titlebar ui-widget-header ui-corner-all ui-helper-clearfix';\n", " var titletext = document.createElement('div');\n", " titletext.classList = 'ui-dialog-title';\n", " titletext.setAttribute(\n", " 'style',\n", " 'width: 100%; text-align: center; padding: 3px;'\n", " );\n", " titlebar.appendChild(titletext);\n", " this.root.appendChild(titlebar);\n", " this.header = titletext;\n", "};\n", "\n", "mpl.figure.prototype._canvas_extra_style = function (_canvas_div) {};\n", "\n", "mpl.figure.prototype._root_extra_style = function (_canvas_div) {};\n", "\n", "mpl.figure.prototype._init_canvas = function () {\n", " var fig = this;\n", "\n", " var canvas_div = (this.canvas_div = document.createElement('div'));\n", " canvas_div.setAttribute(\n", " 'style',\n", " 'border: 1px solid #ddd;' +\n", " 'box-sizing: content-box;' +\n", " 'clear: both;' +\n", " 'min-height: 1px;' +\n", " 'min-width: 1px;' +\n", " 'outline: 0;' +\n", " 'overflow: hidden;' +\n", " 'position: relative;' +\n", " 'resize: both;'\n", " );\n", "\n", " function on_keyboard_event_closure(name) {\n", " return function (event) {\n", " return fig.key_event(event, name);\n", " };\n", " }\n", "\n", " canvas_div.addEventListener(\n", " 'keydown',\n", " on_keyboard_event_closure('key_press')\n", " );\n", " canvas_div.addEventListener(\n", " 'keyup',\n", " on_keyboard_event_closure('key_release')\n", " );\n", "\n", " this._canvas_extra_style(canvas_div);\n", " this.root.appendChild(canvas_div);\n", "\n", " var canvas = (this.canvas = document.createElement('canvas'));\n", " canvas.classList.add('mpl-canvas');\n", " canvas.setAttribute('style', 'box-sizing: content-box;');\n", "\n", " this.context = canvas.getContext('2d');\n", "\n", " var backingStore =\n", " this.context.backingStorePixelRatio ||\n", " this.context.webkitBackingStorePixelRatio ||\n", " this.context.mozBackingStorePixelRatio ||\n", " this.context.msBackingStorePixelRatio ||\n", " this.context.oBackingStorePixelRatio ||\n", " this.context.backingStorePixelRatio ||\n", " 1;\n", "\n", " this.ratio = (window.devicePixelRatio || 1) / backingStore;\n", "\n", " var rubberband_canvas = (this.rubberband_canvas = document.createElement(\n", " 'canvas'\n", " ));\n", " rubberband_canvas.setAttribute(\n", " 'style',\n", " 'box-sizing: content-box; position: absolute; left: 0; top: 0; z-index: 1;'\n", " );\n", "\n", " // Apply a ponyfill if ResizeObserver is not implemented by browser.\n", " if (this.ResizeObserver === undefined) {\n", " if (window.ResizeObserver !== undefined) {\n", " this.ResizeObserver = window.ResizeObserver;\n", " } else {\n", " var obs = _JSXTOOLS_RESIZE_OBSERVER({});\n", " this.ResizeObserver = obs.ResizeObserver;\n", " }\n", " }\n", "\n", " this.resizeObserverInstance = new this.ResizeObserver(function (entries) {\n", " var nentries = entries.length;\n", " for (var i = 0; i < nentries; i++) {\n", " var entry = entries[i];\n", " var width, height;\n", " if (entry.contentBoxSize) {\n", " if (entry.contentBoxSize instanceof Array) {\n", " // Chrome 84 implements new version of spec.\n", " width = entry.contentBoxSize[0].inlineSize;\n", " height = entry.contentBoxSize[0].blockSize;\n", " } else {\n", " // Firefox implements old version of spec.\n", " width = entry.contentBoxSize.inlineSize;\n", " height = entry.contentBoxSize.blockSize;\n", " }\n", " } else {\n", " // Chrome <84 implements even older version of spec.\n", " width = entry.contentRect.width;\n", " height = entry.contentRect.height;\n", " }\n", "\n", " // Keep the size of the canvas and rubber band canvas in sync with\n", " // the canvas container.\n", " if (entry.devicePixelContentBoxSize) {\n", " // Chrome 84 implements new version of spec.\n", " canvas.setAttribute(\n", " 'width',\n", " entry.devicePixelContentBoxSize[0].inlineSize\n", " );\n", " canvas.setAttribute(\n", " 'height',\n", " entry.devicePixelContentBoxSize[0].blockSize\n", " );\n", " } else {\n", " canvas.setAttribute('width', width * fig.ratio);\n", " canvas.setAttribute('height', height * fig.ratio);\n", " }\n", " canvas.setAttribute(\n", " 'style',\n", " 'width: ' + width + 'px; height: ' + height + 'px;'\n", " );\n", "\n", " rubberband_canvas.setAttribute('width', width);\n", " rubberband_canvas.setAttribute('height', height);\n", "\n", " // And update the size in Python. We ignore the initial 0/0 size\n", " // that occurs as the element is placed into the DOM, which should\n", " // otherwise not happen due to the minimum size styling.\n", " if (fig.ws.readyState == 1 && width != 0 && height != 0) {\n", " fig.request_resize(width, height);\n", " }\n", " }\n", " });\n", " this.resizeObserverInstance.observe(canvas_div);\n", "\n", " function on_mouse_event_closure(name) {\n", " return function (event) {\n", " return fig.mouse_event(event, name);\n", " };\n", " }\n", "\n", " rubberband_canvas.addEventListener(\n", " 'mousedown',\n", " on_mouse_event_closure('button_press')\n", " );\n", " rubberband_canvas.addEventListener(\n", " 'mouseup',\n", " on_mouse_event_closure('button_release')\n", " );\n", " rubberband_canvas.addEventListener(\n", " 'dblclick',\n", " on_mouse_event_closure('dblclick')\n", " );\n", " // Throttle sequential mouse events to 1 every 20ms.\n", " rubberband_canvas.addEventListener(\n", " 'mousemove',\n", " on_mouse_event_closure('motion_notify')\n", " );\n", "\n", " rubberband_canvas.addEventListener(\n", " 'mouseenter',\n", " on_mouse_event_closure('figure_enter')\n", " );\n", " rubberband_canvas.addEventListener(\n", " 'mouseleave',\n", " on_mouse_event_closure('figure_leave')\n", " );\n", "\n", " canvas_div.addEventListener('wheel', function (event) {\n", " if (event.deltaY < 0) {\n", " event.step = 1;\n", " } else {\n", " event.step = -1;\n", " }\n", " on_mouse_event_closure('scroll')(event);\n", " });\n", "\n", " canvas_div.appendChild(canvas);\n", " canvas_div.appendChild(rubberband_canvas);\n", "\n", " this.rubberband_context = rubberband_canvas.getContext('2d');\n", " this.rubberband_context.strokeStyle = '#000000';\n", "\n", " this._resize_canvas = function (width, height, forward) {\n", " if (forward) {\n", " canvas_div.style.width = width + 'px';\n", " canvas_div.style.height = height + 'px';\n", " }\n", " };\n", "\n", " // Disable right mouse context menu.\n", " this.rubberband_canvas.addEventListener('contextmenu', function (_e) {\n", " event.preventDefault();\n", " return false;\n", " });\n", "\n", " function set_focus() {\n", " canvas.focus();\n", " canvas_div.focus();\n", " }\n", "\n", " window.setTimeout(set_focus, 100);\n", "};\n", "\n", "mpl.figure.prototype._init_toolbar = function () {\n", " var fig = this;\n", "\n", " var toolbar = document.createElement('div');\n", " toolbar.classList = 'mpl-toolbar';\n", " this.root.appendChild(toolbar);\n", "\n", " function on_click_closure(name) {\n", " return function (_event) {\n", " return fig.toolbar_button_onclick(name);\n", " };\n", " }\n", "\n", " function on_mouseover_closure(tooltip) {\n", " return function (event) {\n", " if (!event.currentTarget.disabled) {\n", " return fig.toolbar_button_onmouseover(tooltip);\n", " }\n", " };\n", " }\n", "\n", " fig.buttons = {};\n", " var buttonGroup = document.createElement('div');\n", " buttonGroup.classList = 'mpl-button-group';\n", " for (var toolbar_ind in mpl.toolbar_items) {\n", " var name = mpl.toolbar_items[toolbar_ind][0];\n", " var tooltip = mpl.toolbar_items[toolbar_ind][1];\n", " var image = mpl.toolbar_items[toolbar_ind][2];\n", " var method_name = mpl.toolbar_items[toolbar_ind][3];\n", "\n", " if (!name) {\n", " /* Instead of a spacer, we start a new button group. */\n", " if (buttonGroup.hasChildNodes()) {\n", " toolbar.appendChild(buttonGroup);\n", " }\n", " buttonGroup = document.createElement('div');\n", " buttonGroup.classList = 'mpl-button-group';\n", " continue;\n", " }\n", "\n", " var button = (fig.buttons[name] = document.createElement('button'));\n", " button.classList = 'mpl-widget';\n", " button.setAttribute('role', 'button');\n", " button.setAttribute('aria-disabled', 'false');\n", " button.addEventListener('click', on_click_closure(method_name));\n", " button.addEventListener('mouseover', on_mouseover_closure(tooltip));\n", "\n", " var icon_img = document.createElement('img');\n", " icon_img.src = '_images/' + image + '.png';\n", " icon_img.srcset = '_images/' + image + '_large.png 2x';\n", " icon_img.alt = tooltip;\n", " button.appendChild(icon_img);\n", "\n", " buttonGroup.appendChild(button);\n", " }\n", "\n", " if (buttonGroup.hasChildNodes()) {\n", " toolbar.appendChild(buttonGroup);\n", " }\n", "\n", " var fmt_picker = document.createElement('select');\n", " fmt_picker.classList = 'mpl-widget';\n", " toolbar.appendChild(fmt_picker);\n", " this.format_dropdown = fmt_picker;\n", "\n", " for (var ind in mpl.extensions) {\n", " var fmt = mpl.extensions[ind];\n", " var option = document.createElement('option');\n", " option.selected = fmt === mpl.default_extension;\n", " option.innerHTML = fmt;\n", " fmt_picker.appendChild(option);\n", " }\n", "\n", " var status_bar = document.createElement('span');\n", " status_bar.classList = 'mpl-message';\n", " toolbar.appendChild(status_bar);\n", " this.message = status_bar;\n", "};\n", "\n", "mpl.figure.prototype.request_resize = function (x_pixels, y_pixels) {\n", " // Request matplotlib to resize the figure. Matplotlib will then trigger a resize in the client,\n", " // which will in turn request a refresh of the image.\n", " this.send_message('resize', { width: x_pixels, height: y_pixels });\n", "};\n", "\n", "mpl.figure.prototype.send_message = function (type, properties) {\n", " properties['type'] = type;\n", " properties['figure_id'] = this.id;\n", " this.ws.send(JSON.stringify(properties));\n", "};\n", "\n", "mpl.figure.prototype.send_draw_message = function () {\n", " if (!this.waiting) {\n", " this.waiting = true;\n", " this.ws.send(JSON.stringify({ type: 'draw', figure_id: this.id }));\n", " }\n", "};\n", "\n", "mpl.figure.prototype.handle_save = function (fig, _msg) {\n", " var format_dropdown = fig.format_dropdown;\n", " var format = format_dropdown.options[format_dropdown.selectedIndex].value;\n", " fig.ondownload(fig, format);\n", "};\n", "\n", "mpl.figure.prototype.handle_resize = function (fig, msg) {\n", " var size = msg['size'];\n", " if (size[0] !== fig.canvas.width || size[1] !== fig.canvas.height) {\n", " fig._resize_canvas(size[0], size[1], msg['forward']);\n", " fig.send_message('refresh', {});\n", " }\n", "};\n", "\n", "mpl.figure.prototype.handle_rubberband = function (fig, msg) {\n", " var x0 = msg['x0'] / fig.ratio;\n", " var y0 = (fig.canvas.height - msg['y0']) / fig.ratio;\n", " var x1 = msg['x1'] / fig.ratio;\n", " var y1 = (fig.canvas.height - msg['y1']) / fig.ratio;\n", " x0 = Math.floor(x0) + 0.5;\n", " y0 = Math.floor(y0) + 0.5;\n", " x1 = Math.floor(x1) + 0.5;\n", " y1 = Math.floor(y1) + 0.5;\n", " var min_x = Math.min(x0, x1);\n", " var min_y = Math.min(y0, y1);\n", " var width = Math.abs(x1 - x0);\n", " var height = Math.abs(y1 - y0);\n", "\n", " fig.rubberband_context.clearRect(\n", " 0,\n", " 0,\n", " fig.canvas.width / fig.ratio,\n", " fig.canvas.height / fig.ratio\n", " );\n", "\n", " fig.rubberband_context.strokeRect(min_x, min_y, width, height);\n", "};\n", "\n", "mpl.figure.prototype.handle_figure_label = function (fig, msg) {\n", " // Updates the figure title.\n", " fig.header.textContent = msg['label'];\n", "};\n", "\n", "mpl.figure.prototype.handle_cursor = function (fig, msg) {\n", " var cursor = msg['cursor'];\n", " switch (cursor) {\n", " case 0:\n", " cursor = 'pointer';\n", " break;\n", " case 1:\n", " cursor = 'default';\n", " break;\n", " case 2:\n", " cursor = 'crosshair';\n", " break;\n", " case 3:\n", " cursor = 'move';\n", " break;\n", " }\n", " fig.rubberband_canvas.style.cursor = cursor;\n", "};\n", "\n", "mpl.figure.prototype.handle_message = function (fig, msg) {\n", " fig.message.textContent = msg['message'];\n", "};\n", "\n", "mpl.figure.prototype.handle_draw = function (fig, _msg) {\n", " // Request the server to send over a new figure.\n", " fig.send_draw_message();\n", "};\n", "\n", "mpl.figure.prototype.handle_image_mode = function (fig, msg) {\n", " fig.image_mode = msg['mode'];\n", "};\n", "\n", "mpl.figure.prototype.handle_history_buttons = function (fig, msg) {\n", " for (var key in msg) {\n", " if (!(key in fig.buttons)) {\n", " continue;\n", " }\n", " fig.buttons[key].disabled = !msg[key];\n", " fig.buttons[key].setAttribute('aria-disabled', !msg[key]);\n", " }\n", "};\n", "\n", "mpl.figure.prototype.handle_navigate_mode = function (fig, msg) {\n", " if (msg['mode'] === 'PAN') {\n", " fig.buttons['Pan'].classList.add('active');\n", " fig.buttons['Zoom'].classList.remove('active');\n", " } else if (msg['mode'] === 'ZOOM') {\n", " fig.buttons['Pan'].classList.remove('active');\n", " fig.buttons['Zoom'].classList.add('active');\n", " } else {\n", " fig.buttons['Pan'].classList.remove('active');\n", " fig.buttons['Zoom'].classList.remove('active');\n", " }\n", "};\n", "\n", "mpl.figure.prototype.updated_canvas_event = function () {\n", " // Called whenever the canvas gets updated.\n", " this.send_message('ack', {});\n", "};\n", "\n", "// A function to construct a web socket function for onmessage handling.\n", "// Called in the figure constructor.\n", "mpl.figure.prototype._make_on_message_function = function (fig) {\n", " return function socket_on_message(evt) {\n", " if (evt.data instanceof Blob) {\n", " var img = evt.data;\n", " if (img.type !== 'image/png') {\n", " /* FIXME: We get \"Resource interpreted as Image but\n", " * transferred with MIME type text/plain:\" errors on\n", " * Chrome. But how to set the MIME type? It doesn't seem\n", " * to be part of the websocket stream */\n", " img.type = 'image/png';\n", " }\n", "\n", " /* Free the memory for the previous frames */\n", " if (fig.imageObj.src) {\n", " (window.URL || window.webkitURL).revokeObjectURL(\n", " fig.imageObj.src\n", " );\n", " }\n", "\n", " fig.imageObj.src = (window.URL || window.webkitURL).createObjectURL(\n", " img\n", " );\n", " fig.updated_canvas_event();\n", " fig.waiting = false;\n", " return;\n", " } else if (\n", " typeof evt.data === 'string' &&\n", " evt.data.slice(0, 21) === 'data:image/png;base64'\n", " ) {\n", " fig.imageObj.src = evt.data;\n", " fig.updated_canvas_event();\n", " fig.waiting = false;\n", " return;\n", " }\n", "\n", " var msg = JSON.parse(evt.data);\n", " var msg_type = msg['type'];\n", "\n", " // Call the \"handle_{type}\" callback, which takes\n", " // the figure and JSON message as its only arguments.\n", " try {\n", " var callback = fig['handle_' + msg_type];\n", " } catch (e) {\n", " console.log(\n", " \"No handler for the '\" + msg_type + \"' message type: \",\n", " msg\n", " );\n", " return;\n", " }\n", "\n", " if (callback) {\n", " try {\n", " // console.log(\"Handling '\" + msg_type + \"' message: \", msg);\n", " callback(fig, msg);\n", " } catch (e) {\n", " console.log(\n", " \"Exception inside the 'handler_\" + msg_type + \"' callback:\",\n", " e,\n", " e.stack,\n", " msg\n", " );\n", " }\n", " }\n", " };\n", "};\n", "\n", "// from http://stackoverflow.com/questions/1114465/getting-mouse-location-in-canvas\n", "mpl.findpos = function (e) {\n", " //this section is from http://www.quirksmode.org/js/events_properties.html\n", " var targ;\n", " if (!e) {\n", " e = window.event;\n", " }\n", " if (e.target) {\n", " targ = e.target;\n", " } else if (e.srcElement) {\n", " targ = e.srcElement;\n", " }\n", " if (targ.nodeType === 3) {\n", " // defeat Safari bug\n", " targ = targ.parentNode;\n", " }\n", "\n", " // pageX,Y are the mouse positions relative to the document\n", " var boundingRect = targ.getBoundingClientRect();\n", " var x = e.pageX - (boundingRect.left + document.body.scrollLeft);\n", " var y = e.pageY - (boundingRect.top + document.body.scrollTop);\n", "\n", " return { x: x, y: y };\n", "};\n", "\n", "/*\n", " * return a copy of an object with only non-object keys\n", " * we need this to avoid circular references\n", " * http://stackoverflow.com/a/24161582/3208463\n", " */\n", "function simpleKeys(original) {\n", " return Object.keys(original).reduce(function (obj, key) {\n", " if (typeof original[key] !== 'object') {\n", " obj[key] = original[key];\n", " }\n", " return obj;\n", " }, {});\n", "}\n", "\n", "mpl.figure.prototype.mouse_event = function (event, name) {\n", " var canvas_pos = mpl.findpos(event);\n", "\n", " if (name === 'button_press') {\n", " this.canvas.focus();\n", " this.canvas_div.focus();\n", " }\n", "\n", " var x = canvas_pos.x * this.ratio;\n", " var y = canvas_pos.y * this.ratio;\n", "\n", " this.send_message(name, {\n", " x: x,\n", " y: y,\n", " button: event.button,\n", " step: event.step,\n", " guiEvent: simpleKeys(event),\n", " });\n", "\n", " /* This prevents the web browser from automatically changing to\n", " * the text insertion cursor when the button is pressed. We want\n", " * to control all of the cursor setting manually through the\n", " * 'cursor' event from matplotlib */\n", " event.preventDefault();\n", " return false;\n", "};\n", "\n", "mpl.figure.prototype._key_event_extra = function (_event, _name) {\n", " // Handle any extra behaviour associated with a key event\n", "};\n", "\n", "mpl.figure.prototype.key_event = function (event, name) {\n", " // Prevent repeat events\n", " if (name === 'key_press') {\n", " if (event.key === this._key) {\n", " return;\n", " } else {\n", " this._key = event.key;\n", " }\n", " }\n", " if (name === 'key_release') {\n", " this._key = null;\n", " }\n", "\n", " var value = '';\n", " if (event.ctrlKey && event.key !== 'Control') {\n", " value += 'ctrl+';\n", " }\n", " else if (event.altKey && event.key !== 'Alt') {\n", " value += 'alt+';\n", " }\n", " else if (event.shiftKey && event.key !== 'Shift') {\n", " value += 'shift+';\n", " }\n", "\n", " value += 'k' + event.key;\n", "\n", " this._key_event_extra(event, name);\n", "\n", " this.send_message(name, { key: value, guiEvent: simpleKeys(event) });\n", " return false;\n", "};\n", "\n", "mpl.figure.prototype.toolbar_button_onclick = function (name) {\n", " if (name === 'download') {\n", " this.handle_save(this, null);\n", " } else {\n", " this.send_message('toolbar_button', { name: name });\n", " }\n", "};\n", "\n", "mpl.figure.prototype.toolbar_button_onmouseover = function (tooltip) {\n", " this.message.textContent = tooltip;\n", "};\n", "\n", "///////////////// REMAINING CONTENT GENERATED BY embed_js.py /////////////////\n", "// prettier-ignore\n", "var _JSXTOOLS_RESIZE_OBSERVER=function(A){var t,i=new WeakMap,n=new WeakMap,a=new WeakMap,r=new WeakMap,o=new Set;function s(e){if(!(this instanceof s))throw new TypeError(\"Constructor requires 'new' operator\");i.set(this,e)}function h(){throw new TypeError(\"Function is not a constructor\")}function c(e,t,i,n){e=0 in arguments?Number(arguments[0]):0,t=1 in arguments?Number(arguments[1]):0,i=2 in arguments?Number(arguments[2]):0,n=3 in arguments?Number(arguments[3]):0,this.right=(this.x=this.left=e)+(this.width=i),this.bottom=(this.y=this.top=t)+(this.height=n),Object.freeze(this)}function d(){t=requestAnimationFrame(d);var s=new WeakMap,p=new Set;o.forEach((function(t){r.get(t).forEach((function(i){var r=t instanceof window.SVGElement,o=a.get(t),d=r?0:parseFloat(o.paddingTop),f=r?0:parseFloat(o.paddingRight),l=r?0:parseFloat(o.paddingBottom),u=r?0:parseFloat(o.paddingLeft),g=r?0:parseFloat(o.borderTopWidth),m=r?0:parseFloat(o.borderRightWidth),w=r?0:parseFloat(o.borderBottomWidth),b=u+f,F=d+l,v=(r?0:parseFloat(o.borderLeftWidth))+m,W=g+w,y=r?0:t.offsetHeight-W-t.clientHeight,E=r?0:t.offsetWidth-v-t.clientWidth,R=b+v,z=F+W,M=r?t.width:parseFloat(o.width)-R-E,O=r?t.height:parseFloat(o.height)-z-y;if(n.has(t)){var k=n.get(t);if(k[0]===M&&k[1]===O)return}n.set(t,[M,O]);var S=Object.create(h.prototype);S.target=t,S.contentRect=new c(u,d,M,O),s.has(i)||(s.set(i,[]),p.add(i)),s.get(i).push(S)}))})),p.forEach((function(e){i.get(e).call(e,s.get(e),e)}))}return s.prototype.observe=function(i){if(i instanceof window.Element){r.has(i)||(r.set(i,new Set),o.add(i),a.set(i,window.getComputedStyle(i)));var n=r.get(i);n.has(this)||n.add(this),cancelAnimationFrame(t),t=requestAnimationFrame(d)}},s.prototype.unobserve=function(i){if(i instanceof window.Element&&r.has(i)){var n=r.get(i);n.has(this)&&(n.delete(this),n.size||(r.delete(i),o.delete(i))),n.size||r.delete(i),o.size||cancelAnimationFrame(t)}},A.DOMRectReadOnly=c,A.ResizeObserver=s,A.ResizeObserverEntry=h,A}; // eslint-disable-line\n", "mpl.toolbar_items = [[\"Home\", \"Reset original view\", \"fa fa-home icon-home\", \"home\"], [\"Back\", \"Back to previous view\", \"fa fa-arrow-left icon-arrow-left\", \"back\"], [\"Forward\", \"Forward to next view\", \"fa fa-arrow-right icon-arrow-right\", \"forward\"], [\"\", \"\", \"\", \"\"], [\"Pan\", \"Left button pans, Right button zooms\\nx/y fixes axis, CTRL fixes aspect\", \"fa fa-arrows icon-move\", \"pan\"], [\"Zoom\", \"Zoom to rectangle\\nx/y fixes axis, CTRL fixes aspect\", \"fa fa-square-o icon-check-empty\", \"zoom\"], [\"\", \"\", \"\", \"\"], [\"Download\", \"Download plot\", \"fa fa-floppy-o icon-save\", \"download\"]];\n", "\n", "mpl.extensions = [\"eps\", \"jpeg\", \"pgf\", \"pdf\", \"png\", \"ps\", \"raw\", \"svg\", \"tif\"];\n", "\n", "mpl.default_extension = \"png\";/* global mpl */\n", "\n", "var comm_websocket_adapter = function (comm) {\n", " // Create a \"websocket\"-like object which calls the given IPython comm\n", " // object with the appropriate methods. Currently this is a non binary\n", " // socket, so there is still some room for performance tuning.\n", " var ws = {};\n", "\n", " ws.binaryType = comm.kernel.ws.binaryType;\n", " ws.readyState = comm.kernel.ws.readyState;\n", " function updateReadyState(_event) {\n", " if (comm.kernel.ws) {\n", " ws.readyState = comm.kernel.ws.readyState;\n", " } else {\n", " ws.readyState = 3; // Closed state.\n", " }\n", " }\n", " comm.kernel.ws.addEventListener('open', updateReadyState);\n", " comm.kernel.ws.addEventListener('close', updateReadyState);\n", " comm.kernel.ws.addEventListener('error', updateReadyState);\n", "\n", " ws.close = function () {\n", " comm.close();\n", " };\n", " ws.send = function (m) {\n", " //console.log('sending', m);\n", " comm.send(m);\n", " };\n", " // Register the callback with on_msg.\n", " comm.on_msg(function (msg) {\n", " //console.log('receiving', msg['content']['data'], msg);\n", " var data = msg['content']['data'];\n", " if (data['blob'] !== undefined) {\n", " data = {\n", " data: new Blob(msg['buffers'], { type: data['blob'] }),\n", " };\n", " }\n", " // Pass the mpl event to the overridden (by mpl) onmessage function.\n", " ws.onmessage(data);\n", " });\n", " return ws;\n", "};\n", "\n", "mpl.mpl_figure_comm = function (comm, msg) {\n", " // This is the function which gets called when the mpl process\n", " // starts-up an IPython Comm through the \"matplotlib\" channel.\n", "\n", " var id = msg.content.data.id;\n", " // Get hold of the div created by the display call when the Comm\n", " // socket was opened in Python.\n", " var element = document.getElementById(id);\n", " var ws_proxy = comm_websocket_adapter(comm);\n", "\n", " function ondownload(figure, _format) {\n", " window.open(figure.canvas.toDataURL());\n", " }\n", "\n", " var fig = new mpl.figure(id, ws_proxy, ondownload, element);\n", "\n", " // Call onopen now - mpl needs it, as it is assuming we've passed it a real\n", " // web socket which is closed, not our websocket->open comm proxy.\n", " ws_proxy.onopen();\n", "\n", " fig.parent_element = element;\n", " fig.cell_info = mpl.find_output_cell(\"
\");\n", " if (!fig.cell_info) {\n", " console.error('Failed to find cell for figure', id, fig);\n", " return;\n", " }\n", " fig.cell_info[0].output_area.element.on(\n", " 'cleared',\n", " { fig: fig },\n", " fig._remove_fig_handler\n", " );\n", "};\n", "\n", "mpl.figure.prototype.handle_close = function (fig, msg) {\n", " var width = fig.canvas.width / fig.ratio;\n", " fig.cell_info[0].output_area.element.off(\n", " 'cleared',\n", " fig._remove_fig_handler\n", " );\n", " fig.resizeObserverInstance.unobserve(fig.canvas_div);\n", "\n", " // Update the output cell to use the data from the current canvas.\n", " fig.push_to_output();\n", " var dataURL = fig.canvas.toDataURL();\n", " // Re-enable the keyboard manager in IPython - without this line, in FF,\n", " // the notebook keyboard shortcuts fail.\n", " IPython.keyboard_manager.enable();\n", " fig.parent_element.innerHTML =\n", " '';\n", " fig.close_ws(fig, msg);\n", "};\n", "\n", "mpl.figure.prototype.close_ws = function (fig, msg) {\n", " fig.send_message('closing', msg);\n", " // fig.ws.close()\n", "};\n", "\n", "mpl.figure.prototype.push_to_output = function (_remove_interactive) {\n", " // Turn the data on the canvas into data in the output cell.\n", " var width = this.canvas.width / this.ratio;\n", " var dataURL = this.canvas.toDataURL();\n", " this.cell_info[1]['text/html'] =\n", " '';\n", "};\n", "\n", "mpl.figure.prototype.updated_canvas_event = function () {\n", " // Tell IPython that the notebook contents must change.\n", " IPython.notebook.set_dirty(true);\n", " this.send_message('ack', {});\n", " var fig = this;\n", " // Wait a second, then push the new image to the DOM so\n", " // that it is saved nicely (might be nice to debounce this).\n", " setTimeout(function () {\n", " fig.push_to_output();\n", " }, 1000);\n", "};\n", "\n", "mpl.figure.prototype._init_toolbar = function () {\n", " var fig = this;\n", "\n", " var toolbar = document.createElement('div');\n", " toolbar.classList = 'btn-toolbar';\n", " this.root.appendChild(toolbar);\n", "\n", " function on_click_closure(name) {\n", " return function (_event) {\n", " return fig.toolbar_button_onclick(name);\n", " };\n", " }\n", "\n", " function on_mouseover_closure(tooltip) {\n", " return function (event) {\n", " if (!event.currentTarget.disabled) {\n", " return fig.toolbar_button_onmouseover(tooltip);\n", " }\n", " };\n", " }\n", "\n", " fig.buttons = {};\n", " var buttonGroup = document.createElement('div');\n", " buttonGroup.classList = 'btn-group';\n", " var button;\n", " for (var toolbar_ind in mpl.toolbar_items) {\n", " var name = mpl.toolbar_items[toolbar_ind][0];\n", " var tooltip = mpl.toolbar_items[toolbar_ind][1];\n", " var image = mpl.toolbar_items[toolbar_ind][2];\n", " var method_name = mpl.toolbar_items[toolbar_ind][3];\n", "\n", " if (!name) {\n", " /* Instead of a spacer, we start a new button group. */\n", " if (buttonGroup.hasChildNodes()) {\n", " toolbar.appendChild(buttonGroup);\n", " }\n", " buttonGroup = document.createElement('div');\n", " buttonGroup.classList = 'btn-group';\n", " continue;\n", " }\n", "\n", " button = fig.buttons[name] = document.createElement('button');\n", " button.classList = 'btn btn-default';\n", " button.href = '#';\n", " button.title = name;\n", " button.innerHTML = '';\n", " button.addEventListener('click', on_click_closure(method_name));\n", " button.addEventListener('mouseover', on_mouseover_closure(tooltip));\n", " buttonGroup.appendChild(button);\n", " }\n", "\n", " if (buttonGroup.hasChildNodes()) {\n", " toolbar.appendChild(buttonGroup);\n", " }\n", "\n", " // Add the status bar.\n", " var status_bar = document.createElement('span');\n", " status_bar.classList = 'mpl-message pull-right';\n", " toolbar.appendChild(status_bar);\n", " this.message = status_bar;\n", "\n", " // Add the close button to the window.\n", " var buttongrp = document.createElement('div');\n", " buttongrp.classList = 'btn-group inline pull-right';\n", " button = document.createElement('button');\n", " button.classList = 'btn btn-mini btn-primary';\n", " button.href = '#';\n", " button.title = 'Stop Interaction';\n", " button.innerHTML = '';\n", " button.addEventListener('click', function (_evt) {\n", " fig.handle_close(fig, {});\n", " });\n", " button.addEventListener(\n", " 'mouseover',\n", " on_mouseover_closure('Stop Interaction')\n", " );\n", " buttongrp.appendChild(button);\n", " var titlebar = this.root.querySelector('.ui-dialog-titlebar');\n", " titlebar.insertBefore(buttongrp, titlebar.firstChild);\n", "};\n", "\n", "mpl.figure.prototype._remove_fig_handler = function (event) {\n", " var fig = event.data.fig;\n", " if (event.target !== this) {\n", " // Ignore bubbled events from children.\n", " return;\n", " }\n", " fig.close_ws(fig, {});\n", "};\n", "\n", "mpl.figure.prototype._root_extra_style = function (el) {\n", " el.style.boxSizing = 'content-box'; // override notebook setting of border-box.\n", "};\n", "\n", "mpl.figure.prototype._canvas_extra_style = function (el) {\n", " // this is important to make the div 'focusable\n", " el.setAttribute('tabindex', 0);\n", " // reach out to IPython and tell the keyboard manager to turn it's self\n", " // off when our div gets focus\n", "\n", " // location in version 3\n", " if (IPython.notebook.keyboard_manager) {\n", " IPython.notebook.keyboard_manager.register_events(el);\n", " } else {\n", " // location in version 2\n", " IPython.keyboard_manager.register_events(el);\n", " }\n", "};\n", "\n", "mpl.figure.prototype._key_event_extra = function (event, _name) {\n", " var manager = IPython.notebook.keyboard_manager;\n", " if (!manager) {\n", " manager = IPython.keyboard_manager;\n", " }\n", "\n", " // Check for shift+enter\n", " if (event.shiftKey && event.which === 13) {\n", " this.canvas_div.blur();\n", " // select the cell after this one\n", " var index = IPython.notebook.find_cell_index(this.cell_info[0]);\n", " IPython.notebook.select(index + 1);\n", " }\n", "};\n", "\n", "mpl.figure.prototype.handle_save = function (fig, _msg) {\n", " fig.ondownload(fig, null);\n", "};\n", "\n", "mpl.find_output_cell = function (html_output) {\n", " // Return the cell and output element which can be found *uniquely* in the notebook.\n", " // Note - this is a bit hacky, but it is done because the \"notebook_saving.Notebook\"\n", " // IPython event is triggered only after the cells have been serialised, which for\n", " // our purposes (turning an active figure into a static one), is too late.\n", " var cells = IPython.notebook.get_cells();\n", " var ncells = cells.length;\n", " for (var i = 0; i < ncells; i++) {\n", " var cell = cells[i];\n", " if (cell.cell_type === 'code') {\n", " for (var j = 0; j < cell.output_area.outputs.length; j++) {\n", " var data = cell.output_area.outputs[j];\n", " if (data.data) {\n", " // IPython >= 3 moved mimebundle to data attribute of output\n", " data = data.data;\n", " }\n", " if (data['text/html'] === html_output) {\n", " return [cell, data, j];\n", " }\n", " }\n", " }\n", " }\n", "};\n", "\n", "// Register the function which deals with the matplotlib target/channel.\n", "// The kernel may be null if the page has been refreshed.\n", "if (IPython.notebook.kernel !== null) {\n", " IPython.notebook.kernel.comm_manager.register_target(\n", " 'matplotlib',\n", " mpl.mpl_figure_comm\n", " );\n", "}\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "%matplotlib nbagg\n", "ax = ut.plotSetup3d(-2, 2, -2, 2, -200, 200)\n", "# try columns of X with large coefficients, or not\n", "ax.plot(X[:, 0], X[:, 13], 'ro', zs=y, markersize = 4);" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "Note that each parameter of an independent variable has an associated confidence interval. \n", "\n", "If a coefficient is not distinguishable from zero, then we cannot assume that there is any relationship between the independent variable and the observations.\n", "\n", "In other words, if the confidence interval for the parameter includes zero, the associated independent variable may not have any predictive value." ] }, { "cell_type": "code", "execution_count": 18, "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Confidence Intervals: [[ 5.65891465 19.47559281]\n", " [ -9.44032559 1.77614877]\n", " [ -9.31636359 4.47701749]\n", " [ -4.12661379 8.15524508]\n", " [ -9.4808662 4.22965424]\n", " [ -5.49698033 7.07574692]\n", " [-10.22359973 4.08684835]\n", " [ 83.74738375 96.52928603]\n", " [ -6.77896356 6.75226985]\n", " [ 8.80365396 21.73126149]\n", " [ -6.86882065 6.4194618 ]\n", " [ -6.97868351 7.1332267 ]\n", " [ -6.71228582 6.2218515 ]\n", " [ 82.96557061 97.07028228]\n", " [ -5.74782503 9.08465366]\n", " [ -1.06173893 9.85081724]\n", " [ 2.02753258 15.5561241 ]\n", " [ 66.56165458 80.19256546]\n", " [ -8.90825108 5.0804296 ]\n", " [ -7.85545335 5.21424811]]\n", "Parameters: [ 1.25672537e+01 -3.83208841e+00 -2.41967305e+00 2.01431564e+00\n", " -2.62560598e+00 7.89383294e-01 -3.06837569e+00 9.01383349e+01\n", " -1.33468527e-02 1.52674577e+01 -2.24679428e-01 7.72715974e-02\n", " -2.45217158e-01 9.00179264e+01 1.66841432e+00 4.39453916e+00\n", " 8.79182834e+00 7.33771100e+01 -1.91391074e+00 -1.32060262e+00]\n" ] } ], "source": [ "print('Confidence Intervals: {}'.format(results.conf_int()))\n", "print('Parameters: {}'.format(results.params))" ] }, { "cell_type": "code", "execution_count": 19, "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [ { "data": { "text/plain": [ "array([False, True, True, True, True, True, True, False, True,\n", " False, True, True, True, False, True, True, False, False,\n", " True, True])" ] }, "execution_count": 19, "metadata": {}, "output_type": "execute_result" } ], "source": [ "CIs = results.conf_int()\n", "notSignificant = (CIs[:,0] < 0) & (CIs[:,1] > 0)\n", "notSignificant" ] }, { "cell_type": "code", "execution_count": 20, "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [ { "data": { "text/plain": [ "(100, 6)" ] }, "execution_count": 20, "metadata": {}, "output_type": "execute_result" } ], "source": [ "Xsignif = X[:,~notSignificant]\n", "Xsignif.shape" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "By eliminating independent variables that are not significant, we help avoid overfitting." ] }, { "cell_type": "code", "execution_count": 21, "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " OLS Regression Results \n", "=======================================================================================\n", "Dep. Variable: y R-squared (uncentered): 0.965\n", "Model: OLS Adj. R-squared (uncentered): 0.963\n", "Method: Least Squares F-statistic: 437.1\n", "Date: Mon, 01 Nov 2021 Prob (F-statistic): 2.38e-66\n", "Time: 14:03:54 Log-Likelihood: -473.32\n", "No. Observations: 100 AIC: 958.6\n", "Df Residuals: 94 BIC: 974.3\n", "Df Model: 6 \n", "Covariance Type: nonrobust \n", "==============================================================================\n", " coef std err t P>|t| [0.025 0.975]\n", "------------------------------------------------------------------------------\n", "x1 11.9350 3.162 3.775 0.000 5.657 18.213\n", "x2 90.5841 2.705 33.486 0.000 85.213 95.955\n", "x3 14.3652 2.924 4.913 0.000 8.560 20.170\n", "x4 90.5586 3.289 27.535 0.000 84.028 97.089\n", "x5 8.3185 3.028 2.747 0.007 2.307 14.330\n", "x6 71.9119 3.104 23.169 0.000 65.749 78.075\n", "==============================================================================\n", "Omnibus: 9.915 Durbin-Watson: 2.056\n", "Prob(Omnibus): 0.007 Jarque-Bera (JB): 11.608\n", "Skew: 0.551 Prob(JB): 0.00302\n", "Kurtosis: 4.254 Cond. No. 1.54\n", "==============================================================================\n", "\n", "Notes:\n", "[1] R² is computed without centering (uncentered) since the model does not contain a constant.\n", "[2] Standard Errors assume that the covariance matrix of the errors is correctly specified.\n" ] } ], "source": [ "model = sm.OLS(y, Xsignif)\n", "results = model.fit()\n", "print(results.summary())" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "## Real Data: House Prices in Ames, Iowa" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "Let's see how powerful multiple regression can be on a real-world example.\n", "\n", "A typical application of linear models is predicting house prices. Linear models have been used for this problem for decades, and when a municipality does a value assessment on your house, they typically use a linear model." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "We can consider various measurable attributes of a house (its \"features\") as the independent variables, and the most recent sale price of the house as the dependent variable." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "For our case study, we will use the features:\n", "* Lot Area (sq ft), \n", "* Gross Living Area (sq ft), \n", "* Number of Fireplaces, \n", "* Number of Full Baths, \n", "* Number of Half Baths, \n", "* Garage Area (sq ft), \n", "* Basement Area (sq ft)" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "So our design matrix will have 8 columns (including the constant for the intercept):\n", "\n", "$$ X\\beta = \\mathbf{y}$$\n", "\n", "and it will have one row for each house in the data set, with $y$ the sale price of the house." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "We will use data from housing sales in Ames, Iowa from 2006 to 2009:" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\"Figure\"\n", "\n", "

\n", "Ames, Iowa

\n", "

\n", "Tim Kiser (w:User:Malepheasant), CC BY-SA 2.5, via Wikimedia Commons

" ] }, { "cell_type": "code", "execution_count": 22, "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [], "source": [ "df = pd.read_csv('data/ames-housing-data/train.csv')" ] }, { "cell_type": "code", "execution_count": 23, "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
LotAreaGrLivAreaFireplacesFullBathHalfBathGarageAreaTotalBsmtSFSalePrice
084501710021548856208500
1960012621204601262181500
2112501786121608920223500
395501717110642756140000
41426021981218361145250000
\n", "
" ], "text/plain": [ " LotArea GrLivArea Fireplaces FullBath HalfBath GarageArea \\\n", "0 8450 1710 0 2 1 548 \n", "1 9600 1262 1 2 0 460 \n", "2 11250 1786 1 2 1 608 \n", "3 9550 1717 1 1 0 642 \n", "4 14260 2198 1 2 1 836 \n", "\n", " TotalBsmtSF SalePrice \n", "0 856 208500 \n", "1 1262 181500 \n", "2 920 223500 \n", "3 756 140000 \n", "4 1145 250000 " ] }, "execution_count": 23, "metadata": {}, "output_type": "execute_result" } ], "source": [ "df[['LotArea', 'GrLivArea', 'Fireplaces', 'FullBath', 'HalfBath', 'GarageArea', 'TotalBsmtSF', 'SalePrice']].head()" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "Some things to note here:\n", "* House prices are in dollars\n", "* Areas are in square feet\n", "* Rooms are in counts" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "Do we have scaling concerns here? \n", "\n", "No, because each feature will get its own $\\beta$, which will correct for the scaling differences between different units of measure." ] }, { "cell_type": "code", "execution_count": 24, "metadata": { "slideshow": { "slide_type": "slide" } }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/Users/crovella/opt/anaconda3/lib/python3.7/site-packages/statsmodels/tsa/tsatools.py:142: FutureWarning: In a future version of pandas all arguments of concat except for the argument 'objs' will be keyword-only\n", " x = pd.concat(x[::order], 1)\n" ] } ], "source": [ "X_no_intercept = df[['LotArea', 'GrLivArea', 'Fireplaces', 'FullBath', 'HalfBath', 'GarageArea', 'TotalBsmtSF']]\n", "X_intercept = sm.add_constant(X_no_intercept)\n", "y = df['SalePrice'].values" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "skip" } }, "source": [ "```{note}\n", "Note that removing the intercept will cause the $R^2$ to go up, which is counter-intuitive. The reason is explained here -- but amounts to the fact that the formula for R2 with/without an intercept is different.\n", "https://stats.stackexchange.com/questions/26176/removal-of-statistically-significant-intercept-term-increases-r2-in-linear-mo/26205#26205\n", "```" ] }, { "cell_type": "code", "execution_count": 25, "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [], "source": [ "from sklearn import utils, model_selection\n", "X_train, X_test, y_train, y_test = model_selection.train_test_split(\n", " X_intercept, y, test_size = 0.5, random_state = 0)" ] }, { "cell_type": "code", "execution_count": 26, "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " OLS Regression Results \n", "==============================================================================\n", "Dep. Variable: y R-squared: 0.759\n", "Model: OLS Adj. R-squared: 0.757\n", "Method: Least Squares F-statistic: 325.5\n", "Date: Mon, 01 Nov 2021 Prob (F-statistic): 1.74e-218\n", "Time: 14:03:54 Log-Likelihood: -8746.5\n", "No. Observations: 730 AIC: 1.751e+04\n", "Df Residuals: 722 BIC: 1.755e+04\n", "Df Model: 7 \n", "Covariance Type: nonrobust \n", "===============================================================================\n", " coef std err t P>|t| [0.025 0.975]\n", "-------------------------------------------------------------------------------\n", "const -4.285e+04 5350.784 -8.007 0.000 -5.34e+04 -3.23e+04\n", "LotArea 0.2361 0.131 1.798 0.073 -0.022 0.494\n", "GrLivArea 48.0865 4.459 10.783 0.000 39.332 56.841\n", "Fireplaces 1.089e+04 2596.751 4.192 0.000 5787.515 1.6e+04\n", "FullBath 1.49e+04 3528.456 4.224 0.000 7977.691 2.18e+04\n", "HalfBath 1.56e+04 3421.558 4.559 0.000 8882.381 2.23e+04\n", "GarageArea 98.9856 8.815 11.229 0.000 81.680 116.291\n", "TotalBsmtSF 62.6392 4.318 14.508 0.000 54.163 71.116\n", "==============================================================================\n", "Omnibus: 144.283 Durbin-Watson: 1.937\n", "Prob(Omnibus): 0.000 Jarque-Bera (JB): 917.665\n", "Skew: 0.718 Prob(JB): 5.39e-200\n", "Kurtosis: 8.302 Cond. No. 6.08e+04\n", "==============================================================================\n", "\n", "Notes:\n", "[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.\n", "[2] The condition number is large, 6.08e+04. This might indicate that there are\n", "strong multicollinearity or other numerical problems.\n" ] } ], "source": [ "model = sm.OLS(y_train, X_train)\n", "results = model.fit()\n", "print(results.summary())" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "We see that we have:\n", "* $\\beta_0$: Intercept of -\\$42,850\n", "* $\\beta_1$: Marginal value of one square foot of Lot Area: \\$0.23 \n", " * but __NOTICE__ - this coefficient is not statistically different from zero!\n", "* $\\beta_2$: Marginal value of one square foot of Gross Living Area: \\$48 \n", "* $\\beta_3$: Marginal value of one additional fireplace: \\$10,890\n", "* $\\beta_4$: Marginal value of one additional full bath: \\$14,900\n", "* $\\beta_5$: Marginal value of one additional half bath: \\$15,600\n", "* $\\beta_6$: Marginal value of one square foot of Garage Area: \\$99\n", "* $\\beta_7$: Marginal value of one square foot of Basement Area: \\$62" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "Is our model doing a good job? \n", "\n", "There are many statistics for testing this question, but we'll just look at the predictions versus the ground truth.\n", "\n", "For each house we compute its predicted sale value according to our model:\n", "\n", "$$\\hat{\\mathbf{y}} = X\\hat{\\beta}$$" ] }, { "cell_type": "code", "execution_count": 27, "metadata": { "hide_input": true, "slideshow": { "slide_type": "fragment" }, "tags": [ "hide-input" ] }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAuoAAAFRCAYAAAAigoiNAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAACK8ElEQVR4nO29eXxU9b3//3yfSQjIppEgmxACilVxYdNalcW2WkrRLrdSUWzt1V7F7d669tZqa23t99fbVkSpWq0iuLXWihS1loCV1iVAiyiKhBh2JBoEgwSSmc/vj7PkzJkzk8kyyczk/Xw88sjMmbN8zpnkfV7n/XkvYoxBURRFURRFUZTswursASiKoiiKoiiKkogKdUVRFEVRFEXJQlSoK4qiKIqiKEoWokJdURRFURRFUbIQFeqKoiiKoiiKkoWoUFcURVEURVGULESFuqIoiqIoiqJkISrUFUVpEyLyWRF5VUReFpHHRaSws8ekKIrSVVAbnN+oUFcUpa1sAqYYYyYCVcC5nTweRVGUroTa4DxGhbrSIkRklIj8S0Q+EZGrO3s8qRCRahH5fGePI4iIPCwiP3Vevy0ik1q5n1Zv254YY7YbY/Y7bxuBWLJ1s2XMiqJkHrXBHYPa4PxGhXoOIiLfFpG1IvKpiOwUkXkicmgLtm+L8bwBWG6M6W2MmROy79NF5J8iskdEakXkHyIyvpXH6hSc67NfROpE5AMR+b2I9MrEsYwxxxljlqc5prjvLN1t24qIHCYixrken4rIJhH5bsh6w4EvAYuT7as1YxaRYhF5RkT2Oce+II1tZojIO842G0XkDGf5lSKyUkQOiMjD7XEsRckErbXTaoNbhtrgtI6ftl10xuj/iYrI3c5ny0Wk3vfZet92nxGRcufvtlJEvtqSMeYzKtRzDBH5PvAL4HqgL3AqMAx4SUS6dcAQhgFvJxlbH2wDcTdQDAwGfgwc6IBxtTdfMcb0AsYA44EfBlcQkYIOH1XncBLwoTGmlzHmEOBm4D4R6eeu4Hz3jwAXGWMOtvPx7wEOAkcAM4F5InJcspVF5AvY/yPfAXoDZ2JPBwNsB34KPNQex1KUbKKr2WDoMnb4JHLEBjtj7OV8d0cA+4E/+Fa50rfOKGfsBcCz2H+7xcBlwAIRObqdzyM3McboT478AH2AOuCbgeW9gF3AJc57A4z0ff4w8FPn9aPY02L7nX3dEHKczwDLgY+xRfl0Z3k5EAXqnW2PDmw3Dvi4mXO4CdgIfAKsA77q+6wa+wHkTWAf8CD2P/rzzvp/Aw4LrH+zs5/dwO+B7oHPP++8HgQ8DdQA7wNXpxijt53z/v8DFvs+u9EZ4wGgoLl9AycDq51zeBJ4wvd9BI91JPAnZ18fAXOTfWeB8wv9zgLndJ0z7j3OOLonuwaBbf8beNH3fjD239hRzvsC4C/YMZLN7cs/5huBbc51WQ+cFbJ+T+wbxNG+ZY8Cd6Y4xj+B7zYzjp8CD7f1WPqjP5n6CfyvpPX/Sxewwb7P/XZ4aKp9ozY42d9VRmywb72LsZ0k4rxfDvxnyHrHO9dVfMv+Ctze2f+H2fCjHvXc4jSgO7YR8TDG1GEb0i80twNjzEXAZhxvhTHm//k/Fztb/Dnsf5L+wFXAQhEZZYyZArxC0xPxe4HdvwdEReQREfmSiBwWMoSNwBnYswE/xn5qHuj7/OvOeRwNfMU5rx8A/bBngIJx8TOBs4ERzjZhnm/LOac12AbuLOBaETk7yWXyb3skMBX4l2/xt4AvA4diG++k+3ZmOf6MbdiKsT0LX09yrAi2R2ETUOrs74m2fGeBQ3wTOAcYDpwAfLu583c4GVjlHOtQ4OfO+0rf9TgF+JEztXl+czt0xnYlMN4Y0xv7O6wOWfVoIBr4W1sDhHpznGs4Dihxpk+3ishcEenR7Fm28FiK0sGk8//bVWwwNNnhYuCZZPtWGxxOpmxwgIuB+cZR3g4/F5EPnZCsSe5wwoaILeC7PCrUc4t+2NNfjSGf7XA+byunYnvo7zTGHDTGlGMbrm81t6ExZi9wOvaT/gNAjYgsEpEjfOv8wdiJLzFjzJPABmCCbzd3G2M+MMZsw34oeN0Y8y9jzAFsY3xy4LBzjTFbjDG1wB1JxjkeKDHG/MQ5pypnfDNSnM6fReRjYAXwMvAz32dznGPuT2PfpwKFwG+MMQ3GmD8CFUmOOQHb63S9MWafMabeGLMixRhd0v3O5jjXvhb7pnJSGvvGWe8aEdmL7TXrD5zjGl9jzKPGmH7GmEnOz5Np7DMKFAHHikihMabaGLMxZL1e2N4nP3uwQ1rCOAL7en8DW4ychP03Ezpt3sZjKUpH0uz/bxeywe712IIt5tQGZ48NBkBEhgITscNxXG4EyrAfgO4HnhOREcC72FEB14tIoYh80dn2kDTOI+9RoZ5bfAj0SxKTN9D5vK0MArYYY/xZ45uw/7GaxRjzjjHm28aYIdgGdBDwG/dzEZklIv8WkY8dI3w88Q8YH/he7w95H0wo2hIY56CQYQ0DBrnHdI77A2xRl4zzjDGHGmOGGWOuME0Z9cFjNrfvQcC2gEdhU5JjHglsSvIglop0v7OdvtefkngtExCRIuwp3ROMMX2wBfCpQEMLxxiHMaYSuBa4DdglIk+ISNh3V4cd8uWnD/ZUbRju93S3MWaHMeZD4FfYHrnmaOmxFKUjSev/t4vYYP9x1Qa3ggzaYJdZwApjzPu+Y75ujPnEGHPAGPMI8A9gqjGmATgPe4ZkJ/B94Clga0vPKx9RoZ5bvIodj/c1/0IR6Ymd6b3UWfQp8U+iAwL7MSRnO3CkM1XpMhQ7jq1FGGPexY6PP94Z5zBsT8eVwOHGmEOBtwif9kqXIwPj3B6yzhbgfcfouz+9jTHpiLcw/NevuX3vAAaLiP8chybZ7xZgaJIHsQ75zkI4HvtvrgrAGPM09hRw6NRxSzDGPGaMOR37RmuwE0CDvAcUiMhRvmUnkiSh2RizG9u4p7peyWjRsRQl28ljGwxN/+Nqg1tJJmywj1nEe9NDh4Dzt2eMedMYM9EYc7gx5mxsz/sbaZxG3qNCPYcwxuzBjim8W0TOcaaISrFj7rZix+AB/Bu4QEQiInIO9hSSnw+w/wnCeB07iegGZ/+TsOMUn2hufCJyjIh8X0SGOO+PxJ76e81ZpSf2P2aN8/l3aHsM2mwRGSIixdhelLApvzeAvSJyo4j0cK7L8dI+Jcua2/er2HVtrxaRAhH5GvHTzMF97QDuFJGeItJdRD7nfJaR7wy8msIPJ/n4ZOCtgDdqCTA9nX2nOOYoEZnieIvqsT110eB6xph92DkZP3Guyeewm3k8GlzXx++Bq0SkvxOjey1OuTLnO+gORICIc40L2nAsRcka1AarDU6XTNpgETkNezbhD75lh4rI2a7NFZGZ2BW5XnQ+P8H57BARuQ47SuDhtpxjvqBCPccwdhLLD4BfAnuxDcQW7GxttwTXNdhG4mPsRJ8/B3bzc+CHzjThdYH9H8Q2AF/CDqW5F5jleGaa4xPshJbXRWQf9s3hLexpLIwx64D/wzacHwCjsae+2sJj2Ak8Vc7PT4MrGGOi2NfjJOyKAB8Cv8NOpmoTze3buZ5fw04a2g2cTyAZOGRfI7E9Jlud9SFz3xnYHrFk38NJ2FUK/LwAfMERvK2lCLgTe7w7sWMuf5Bk3SuAHtgxjI8DlxtjPG+OiDwvIv5tb8eOQX0PeAc7Ce0O57MfYt+QbgIudF7749dTHktRshy1wWqD0yWTNvhi4E/GGH94TCH230aNc8yrsMOb3FrqF2E/JO3CTgj+gk/TdGkk/iFNUXIHEanGLvX0t84eS64idkWENdjxj22KeVQUpWuhNrjtqA1WmqMrNApQFCUJjifoM509DkVRlK6I2mClOTT0RVEURVEURVGykIwKdRGpFpG1YpeCWuksKxaRl0Rkg/P7MN/6N4vdpGS9+BohiMhYZz+VIjLHzd4WkSIRedJZ/rqTWOluc7FzjA0icnEmz1PpHIwxpTrlqiiK0jmoDVaUzNMRHvXJxpiTjDHjnPc3AUuNMUdhlxO8CUBEjsVuUHAcdueue8XuEgYwD7gMOMr5OcdZ/l1gtzFmJPBrnNJCTvb5rdhJNROAWyW8Q5uiKIqiKIqiZCWdEfpyLk21NR/BLnLvLn/CKYT/PnZr3AlitzbuY4x51SlPND+wjbuvPwJnOd72s4GXjDG1Tl3ll2gS94qiKIqiKIqS9WRaqBvgryKySkQuc5YdYYzZAeD87u8sH0x8h7OtzrLBxHencpfHbeN0EtsDHJ5iX4qiKIqiKIqSE2S66svnjDHbRaQ/8JKIpKopGtYZzaRY3tptmg5oPzxcBtCzZ8+xxxxzTIrhKYqitDMHDsD69WAMHH009OjR6l2tWrXqQ2NMSTuOLitRu60oSqcRi8GGDVBXB8OHQ3Fxq3eVrs3OqFA3xmx3fu8SkWew48U/EJGBxpgdTljLLmf1rcS3Ih6C3ZZ3q/M6uNy/zVanu2BfoNZZPimwzfKQ8d0P3A8wbtw4s3Llylafq6IoSouorIRJk6BvX1i6FE44oU27E5FN7TOw7EbttqIoncK+fTB1Knz6KTz+OMyY0abdpWuzMxb64rSa7e2+Br6I3SFtEXbXKpzfzzqvFwEznEouw7GTRt9wwmM+EZFTnfjzWYFt3H19Ayh34thfBL4oIoc5SaRfdJYpiqJ0Pq5IP3CgXUS6oiiKkkFckb5iBSxc2GaR3hIy6VE/AnjGqaRYADxmjHlBRCqAp0Tku9gtev8DwBjztog8BawDGoHZTjtfgMuBh7Fb2D7v/AA8CDwqIpXYnvQZzr5qRcRtIw7wE2NMbQbPVVEUJT1UpCuKouQOnSjSIYNC3RhTBZwYsvwj4Kwk29wB3BGyfCVwfMjyehyhH/LZQ8BDLRu1oihKBlGRriiKkjt0skgH7UyqKIrSMahIVxRFyR2yQKSDCnVFUZTMoyJdURQld8gSkQ4q1BVFUTKLinRFUZTcIYtEOqhQVxRFyRwq0hVFUXKHLBPpoEJdURQlM6hIVxRFyR2yUKSDCnVFUZT2R0W6oihK7pClIh1UqCuKorQvKtIVRVFyhywW6aBCXVEUpf1Qka4oipI7ZLlIBxXqiqIo7YOKdEVRlNwhB0Q6qFBXFEVpOyrSFUVRcoccEemgQl1RFKVtqEhXFEXJHXJIpIMKdUVRlNajIl1RFCV3yDGRDirUFUVRWoeKdEVRlNwhB0U6qFBXFEVpOSrSFUVRcoccFemgQl1RFKVlqEhXFEXJHXJYpIMKdUVRlPRRka4oipI75LhIBxXqiqIo6aEiXVEUJXfIA5EOKtQVRVGaR0W6oihK7pAnIh1UqCuKoqRGRbqiKErukEciHVSoK4qiJEdFuqIoSu6QZyIdVKgriqKEoyJdURQld8hDkQ4q1BVFURJRka4oipI75KlIBxXqiqIo8ahIVxRFyR3yWKSDCnVFUZQmVKQriqLkDnku0kGFuqIoio2KdEVRlNyhC4h0UKGuKIqiIl1RFCWX6CIiHVSoK4rS1VGRriiKkjt0IZEOKtQVRenKqEhXFEXJHbqYSAcV6oqidFVUpCuKouQOXVCkgwp1RVG6IirSFUVRcocuKtJBhbqiKF0NFemKoii5QxcW6aBCXVGUroSKdEVRlNyhi4t0UKGuKEpXQUW6oihK7qAiHVChrihKV0BFuqIoSu6gIt1DhbqiKPmNinRFUZTcwSfSP5j3IE+OOI2qmrrOHlWnUdDZA1AURckYKtIVRVFyh4BIn7x1AGbLOkRg8VWnU1bSq7NH2OGoR11RlPxERbqiKEruEAh3WX7yFIyB/Q1RjIGK6trOHmGnoB51RVHyDxXpiqIouUNITPr4mjpEoEdhBBEYX1rc2aPsFFSoK4qSX6hIVxRFyR2SJI6WlfRi8VWnU1Fdy/jS4i4Z9gIq1BVFySdUpCuKouQOzVR3KSvp1WUFuovGqCuKkh+oSFcURckdtARjWqhQVxQl91GRriiKkjuoSE8bFeqKouQ2KtIVRVFyBxXpLUKFuqIouYuKdEVRlNxBRXqLUaGuKEpuoiJdURQld1CR3ioyLtRFJCIi/xKRxc77YhF5SUQ2OL8P8617s4hUish6ETnbt3ysiKx1PpsjIuIsLxKRJ53lr4tIqW+bi51jbBCRizN9noqidCAq0hVFUXIHFemtpiM86tcA7/je3wQsNcYcBSx13iMixwIzgOOAc4B7RSTibDMPuAw4yvk5x1n+XWC3MWYk8GvgF86+ioFbgVOACcCt/gcCRVFyGBXpiqIouYOK9DaRUaEuIkOALwO/8y0+F3jEef0IcJ5v+RPGmAPGmPeBSmCCiAwE+hhjXjXGGGB+YBt3X38EznK87WcDLxljao0xu4GXaBL3iqLkKirSFUVRcgcV6W0m0x713wA3ADHfsiOMMTsAnN/9neWDgS2+9bY6ywY7r4PL47YxxjQCe4DDU+xLUZRcRUW6oihK7pCFIr2qpo4nKzZTVVPX2UNJm4x1JhWRacAuY8wqEZmUziYhy0yK5a3dxj/Gy7BDahg6dGgaQ1QUpVNQka44qN1WlBwgS0X6tLtXYAyIwOKrTs+JrqeZ9Kh/DpguItXAE8AUEVkAfOCEs+D83uWsvxU40rf9EGC7s3xIyPK4bUSkAOgL1KbYVxzGmPuNMeOMMeNKSkpaf6aKomQOFemKD7XbipLlZKFIB6iorsUY2N8QxRj7fTKyyfOeMaFujLnZGDPEGFOKnSRaboy5EFgEuFVYLgaedV4vAmY4lVyGYyeNvuGEx3wiIqc68eezAtu4+/qGcwwDvAh8UUQOc5JIv+gsUxQll1CRriiKkjtkqUgHGF9ajAj0KIwgYr8Pw/W837ZoHdPuXtHpYj1joS8puBN4SkS+C2wG/gPAGPO2iDwFrAMagdnGmKizzeXAw0AP4HnnB+BB4FERqcT2pM9w9lUrIrcDFc56PzHGJH90UhQl+1CRriiKkjtksUgHKCvpxeKrTqeiupbxpcVJw178nvcehREqqms7NUSmQ4S6MWY5sNx5/RFwVpL17gDuCFm+Ejg+ZHk9jtAP+ewh4KHWjllRlE5ERbqiKErukOUi3aWspFezojtdz3tH0RkedUVRlOSoSFcURckdckSkp0u6nveOQoW6oijZg4p0RVGU3CHPRLpLOp73jqIjOpMqiqI0j4p0RVGU3CFPRXq2oUJdUZTOR0W6oihK7qAivcNQoa4oHUQ21WXNKlSkK4qi5A4q0jsUjVFXlA4gVzuiZRwV6YqiKLmDivQORz3qitIBtKQjWpdBRbqiKEruoCK9U1ChrigdQLbVZe10VKQriqLkDirSOw0NfVGUDiDb6rJ2KirSFUVRcgcV6Z2KCnVF6SCyqS5rp6EiXVEUJXdQkd7paOiLoigdg4p0RVGU3EFFelagQl1RlMyjIl1RFCV3UJGeNahQVxQls6hIVxSli5KT/TMyLNJz8pp0IhqjrihK5lCRrihKFyVV/4yqmrrsLC7QASI9X3qKdNR3qEJdUZTM0AqRnrU3L0VRlBbi75/RozBCRXUtZSW9slesdkC4S7Jrkmt05HeoQl1RlPanlSI9K29eiqIorSBZ/4ysFKsdFJOeLz1FOvI7VKGuKEr70spwl6y8eSmKorSSZP0zsk6sdmDiaL70FOnI71CFuqIo7UcbYtKz7ualKIrSRsL6Z2SVWO2E6i7Z1FOkteGWHfkdqlBXFKV9aKVI9xvKrLl5KYqiZJCsEKtdvARjW8MtO+o7VKGuKErbaYNIdw2lwXDl5JFMHT2w829giqIo+UwXF+mQO+GWWkddUZS20YZwF7+hrG+IcdfSDUy7e4XW11UURckUKtKB3Am3VI+6oiitp4110l1DWRgRGqKGhqihwCJrPRuKoijNkdVlZlWke2RVrkAKVKgritI62qmZ0RWTRlC77yCPvbEZQbLas6EoipKKrC4zmwMivaqmjiVrdwB0SBhkVuQKNIMKdUVRWk47iPTgDe2+C8eyc299h3k2strrpShKTpLpuOdW260cEelT57xCfUMMgLnLKlly9Rld3j6rUFcUpUVsfn0N/aafQ1G0gUh5eas96cEb2s699Zw/fmg7jzacrPZ6KYqSs2Qy7rnVdisHRDrY94RozHjvozGjYZBoMqmiKC1g8+tr6PaFs6iv+5Svf/12qgaWtXpfnZnI439IMMZ+ryiK0lbcuOfbph/b7g6AVtmtdhLpVTV1PFmxOaOJ/uNLi4lY4r2PWKJhkKhHXVGUdKmspN/0c6iPNnDB+Xewqf/wNnk7OjORJ1ey/RVFyT0yFffcYrvVjiK9I2Ygy0p6seTqMzo0Rj0XUKGuKErzODHpRdEGLrjwTjb1H94uArezEnlyJdtfURTFpUV2qx3DXTqy3nhZSS+unHJURvadq6hQVxQlNb7E0Uh5Ob8aWJYXAjcXsv0VRVH8pGW32jkmXWcgOxcV6oqiJCekuksZqMBVFEXJRjKQOKozkJ2LCnVFUcJJowSjljhUFEXJEjJY3UVnIDsPFeqKoiSSpkjvyiUO9SFFUZSsIUdKMCotR4W6oijxonPPzrSaGXVkglG20dUfUhRFySKyXKSrU6NtqFBXuixqPGz8onPY7u0s/tMtFDQcbLbjaDYlGHX0d9mVH1IURckickCkq1OjbahQV7okajyacEVn/11bePjxm2mMGApeXtZsx9HWJBhlQlC35bts7Xiy6SFFUZTcptV2MctFOqhToz1Qoa50SdR4NDG+tJhhu7fz8OM30y3awIeLXmRIMyLdpSUJRu31cBS8qfm/y8KIsGTtjrTq8LZlPFoFQVGU9qDVdigHRDp0jFMj32fHVagrXZIBfboTjcUoKrC6vEe0bM9OFv/pFhojxhbpk04NXa+txrA9Ho7CbmrjS4sxGAAaooa5yyrT6mjX1vFoFQRFUdpKq+xQjoh0yLxToyvMjludPQBF6Wiqauq4fOFqRARjYN7MMRn/x66qqePJis1U1dRl9DgtxqnuUtBwkO4vL0sp0qfdvYLbFq1j2t0rWnUe7eFZ8d/UjMG7qV05eSSFEQFAECqqaztkPIqiKG2hxXYoh0S6S1lJL84fPzQj99mwe0K+oR51pcvh/mMfaIzRozDCzr31GT1eZ8RQp0UaJRhd2sMb3h6elWQ3tamjB3Lv8o0UWKQtujV8RVG6LtkSLtEiO5SDIr21pPv9BO8JA/p058mKzZ3+vbYnKtSVLkcmPalhxqW1IjejU3oBkV41sIyKFMatva5ZW8NFkt3UWiu6NXxFUboe2RYu4b9X+N/H0cVEerrfj9/2D+jTncsXrs6a77W9SFuoi0hPY8y+TA5GUTqCTHlSkxmX1orcjCW8hoj05oxitnifU3lZVHQripIO2VZMoFlh2oVEOrT8+3Ft/5MVm7Pqe20vmhXqInIa8DugFzBURE4EvmeMuSLTg1OUTJEJUZfMuLRW5GbE8x8S7lKRpnHrbCGcbV4wRVFyk2S2tS3hMKm2bW6/KYVpG0R6W8N7Ois8qLX3vnzNO0rHo/5r4GxgEYAxZo2InJnRUSlKDpLKSLRG5La7FztJTHquGLdMecGyJVZVUZSOIcy2tjWXKNm26ew3qQ1uo0hvi2OjMx0jbQllzIaZ3/YmrdAXY8wWEfEvimZmOIqSuwRj5VLGG7Zgn5kId/EnjuaKccvEA4V66RWlaxK0rW3JJbpnWSXRmPEKFPi3TWe/oTa4jeEubXVsdHZ4UGvvfZ0985sJ0hHqW5zwFyMi3YCrgXcyOyxFyU1cA5FV4i+N6i5hxi3M09yZ3udMPFB09s1IUZTsoDWOAPdB3xXpYX050t1vnA1uh5j0tjo2cmWmtSuQjlD/L+AuYDCwFfgrMDuTg1KUXCarxF8LSjD6CfM0Q+c/gLS3t0RvRoqSfXSGQ6A1jgB/qd+iAotpJwxk9uSRcdu6+12ydkd6A2mnxNFk4T3pxtG3xTGi4YTtS7NC3RjzITCzpTsWke7A34Ei5zh/NMbcKiLFwJNAKVANfNMYs9vZ5mbgu9ihNVcbY150lo8FHgZ6AEuAa4wxRkSKgPnAWOAj4HxjTLWzzcXAD53h/NQY80hLz0FRWkPWiL/KShrPnEjj/no+fOYvDElTpEP4wwaQPQ8g7USuhP0oSlehs2OjW3KsoK0PinQ/9y7fiDH276TnlESkJxO+zQli//m0Jo6+ueuRbNa1sx06+UY6VV8ewRbGHzvvDwP+zxhzSTObHgCmGGPqRKQQWCEizwNfA5YaY+4UkZuAm4AbReRYYAZwHDAI+JuIHG2MiQLzgMuA17CF+jnA89iifrcxZqSIzAB+AZzvPAzcCowDDLBKRBa5DwSKkkmyQvw5In3vnjq+M/NONpR/zOLj6tpccSYrHkDamXyMaVSUXCWrZiSbIV1bn9Y5BUR61VnTqKjY7NUGj8YMxsADs8YycVT/FgviVGNozTVPdvxc+v5yhXRCX05wRTqAMWa3iJzc3EbGGAO4fcYLnR8DnAtMcpY/AiwHbnSWP2GMOQC8LyKVwAQRqQb6GGNeBRCR+cB52EL9XOA2Z19/BOaKnfV6NvCSMabW2eYlbHH/eBrnqyitxu9hOH/80M4ZhBPu0ri/nu/MvJM1xUPp4bRWbmv2fKc/gCiKktdkzYxkmrgP+lU1dUk7YjZ7Tvv2sf8LZ1P0+qu8+//dS3n/scyd8wqCEI3FAOFgNAbApfNX8sK1Z7ZYEKcaQ2uuebLj59r3lwukI9QtETnMF55SnOZ2iEgEWAWMBO4xxrwuIkcYY3YAGGN2iEh/Z/XB2B5zl63OsgbndXC5u80WZ1+NIrIHONy/PGQbRckIWTHl5wt3eeuRP7Jh5QF6OONpaa3gME9ze3qfNY5RUZQgWTEj2UKas/0pz8kR6d1ee5Xvn3s9z+waQsHfNtAYMwAUFVjeawAR8fbTEkGcagytuebJjp+L31+2k47g/j/gnyLyR+f9fwB3pLNzJ2zlJBE5FHhGRI5PsbqELDMplrd2m6YDilyGHVLD0KGd5P1U8oa2TPm1RrQmbBMMd1l5gHkzx7Bzb3271ApuT7JlHEruoXY7/8m1cLR0SzAmC3cpev1Vrj/vep45+gwAT5gXRoSIJfxk+nHc8uxbiNjvXXveUkGc6rq29Jo3J/xz6fvLdtJJJp0vIiuBKdgC+GvGmHUtOYgx5mMRWY4dfvKBiAx0vOkDgV3OaluBI32bDQG2O8uHhCz3b7NVRAqAvkCts3xSYJvlIeO6H7gfYNy4cQlCXslPMuXJbe2UX2tEa3CbF748kKFf/3JCuMvOvfVxIThL1u6gIRqjIWo6LH4w7HprHKPSWtRu5y75OovWEtvvXoMJJUUMv/ibsGIFNfMe5PmtAygKlHm8cvJIpo4eaIeUDC9OuHatSfZsT1SQdwxJhbqI9DHG7HVCXXYCj/k+K3bjv1NsXwI0OCK9B/B57GTPRcDFwJ3O72edTRYBj4nIr7CTSY8C3jDGREXkExE5FXgdmAXc7dvmYuBV4BtAuVMN5kXgZ07iK8AXgZvTuyRKPpNJT25LPRyuEa355ECLRatf6I765AP6Tf8OxBr58Jm/sKH844RwF/d4c5dV0hC1tY3BZDx+MNn11jhGRela5PIsWjpVV9Kx/e416H6gnvueuIXSreuQhQs5YsYMFjv7GtCne9wsqEtLRHFVTR1L1u5g7rJKBAm93pkS8fn6MNaZpPKoPwZMw44x93stxHlf1sy+BwKPOHHqFvCUMWaxiLwKPCUi3wU2Y4fSYIx5W0SeAtYBjcBsJ3QG4HKayjM+7/wAPAg86iSe1mJXjcEYUysitwMVzno/ae7BQukaZNqTm64x9d+0jPPv1RLR6grdUZ98wCOP3kBRN6C8nCEnnMDi48INZUV1LeJEhRVYMPGokrTG2Rajm+x6t+ShRg2/ouQ+6djebPxfT/aAEba8uQICFdW1dD9Qz7zHb2HM1nW8evtdnBaok35k8SFMHNU/yR6av0buuNyZUyDhemfqoSmXH8aymaRC3RgzzamgMtEYs7mlOzbGvAkkVIcxxnwEnJVkmzsIiX83xqwEEuLbjTH1OEI/5LOHgIdaNmqls+goA50tntzgTWv25BGU9C5KaXyDzShe+PJA+k3/DkXdIFJe7jUzSvaw4J57UYHFgcYYy9+r4ZXKD5Ma0/Ywuqmud3CcWpNXUfKXZLagyudJvnzh6qz7X0/2gNEap8+EkiLue8IW6Tecdz2zL/s2kLzBXGvsoTsuV6QXRiTB9mbKYVVRXRvXpVVDGtuHlDHqThjJM9gNhRQlI3SkGMuWjPTgTcuNQwwj9Prs2cnQr38ZYo3gE+mpcM/9nmWVLH5zBwcaYymNdHsY83Svt9bkVZT8JswW+P/vo7EYItKsXepokj1gtNjps28fwy/+JqWOJ332Zd9OmrOzZO0Or0FSS+2hf1wGExfnDratrfnkAAbT7g6rAX26c6DRLiN5oDHGgD7d22W/XZ10qr68JiLjjTEVza+qKC2no8VYNiTAtOSBIXh93nllNWVXXwAHDsDSpWmJdP9xZ08eyfNv7aSowCIaS25Mx5cWYzAURqRN8ezpeM61Jq+i5D9BW+D/vy8qsDAm+/7Xk9nqFjl9fM2MZOHChHCXoJ2D8C7Q6djDZOMKxq0DzJ48IqWTqKXs3FvvzdgWFVjs3FvfLvvt6qQj1CcD/+U0HtqHE6NujElfHShKCrqqGPPftFKF/vivz7Dd2zn7ylug4WCLRbr/uPNmjuHS+asQES5fuLrDpplbmlyaLTMgiqK0P8H/+2A52WwhmXMnLadPko6jqUQ/wL3LNzZrD4HQJkthzpGwuPWS3kXtmmA6vrSYiCVd7l6eadIR6l/K+CiULk02i7H2jJ1PVTmguWYZ82aO4R8vvsb3H/ghBdGGVot093iL1mxHxJ6eLIwIS9bu4MopR8Wt5yafNkRjFFjtE2/YmuTSbJgBURSl9SSzfdls+9uFEJGezNYH7Vxz9rCqpo6pc14hGjNELGHJ1WckvX7pxK2nE4L68vpdLFqzneknDgpNeM3777OTSFWesT/wA+yuomuBnxtj9nbUwJSuRTaKsdbWN29OjAfjBsPiE/2JpVU1dfz87sU8PP8G9kUbqPnLiwxpg0ifOucVGqMxnFBCGqKGucsqE6ZAWxv6ku7sQHPJpYqi5D7pOCI6ajavQwVkiEi/Z1mll2zZXJhnc9dlydod1DfYRrwhakKdLS7Nxa1D8yGoL6/fxcW/tyOgn169jUe+Mz6pWFc73r6k8qjPxy7NeDd2mcY5wLc7YExKnpONJbjCaGnsfKobkr/REMBdSzdw7/KNLL7q9AQj6q9963rSH55/A4WNDXznwju5oOcgzm/lOfmNO4AlEDMgSLt4zNvUSltRlLwjGxLCO7x6VBJPejTQ0KiloSH+e2dLSMfuNheCumjN9oT3qcpIKu1HKqE+wBjzv87rF0VkdUcMSMlvcqncXktj55PdkIKNhsD2gBRY9jbnjx/qGdGaTw5wd3kl9Y1RCiPC7b95jgULb7RF+syfUzlgeLvG/UUEigrCz681oS+tbqWtKEpekg05SB36sBAQ6cyYQUXFZozBE+nTThjI7Mkj0xpDsvKVt047Nm690YP7ptxPc3Y3VRJqRXUtE0qLeXr1Nm/96ScOanbsSvuQSqiL09nTyUEm4n+vDYSU1pAN3pV0aan3N9kNKdhoCMdb7g8ncY3oy+t3eeWtBn24jQWP3+yJ9KO+cBq/SdO4J2Pq6IHMdaZfI5Zw34VjkyZvtSb0JRtuyoqiZA/ZMIvWUXbp/eoP6Pm1cylZU4E4Ij3s+C0R6cnKV75RXdvuFVaSJaG6Dwe/+Npo3qiuTRqjrmSGVEK9L3boi/iWuV71dDqTKkoCuSbkWuL9dZM+3WQbd7tgaEvMGIwJ34db3mpAzVae9In0yoFlzYr0dEKKykp6seTqM0K9JmHVA1pKNtyUFUXJLjp7Fq297VKYrX2/+gM+nPh5hm5Zx3XnXc/ss6Z5Iqm1x09VvnL6iYN4/q2dGb2XBh1rCPzfN09q9+MoqUnVmbS0A8ehdBHyWci9vH4Xl85fiYjw/Fs7vbAev4Dv26OQx9/Ywv7GaGg4yfjSYoZ/vIOHHZF+8cyf8V7/Uh6YOabF8fGQ2NkOmvea+JtrtKbqS2fflBVFUYK0xC65NcchsRldqL08ROj5tXMZumUd1077Pn87bhITfPaytXlZ/m7SxsDt5x4HjijviHtprjnW8pV0yjMqSruSj0KuqqaOS+ev4mDUACaufbIt4FchgtfMIpnhK9uzk8V/uoV6olww8+e8dfgwelj2tGYqY59OZzt3veD22mxIURTFxq2M5Sbdz11WGVf6MGgvV7+zlbJbvkfJmgquO+96/nrsREzMeI3k2pKX5Tp5XAfQrc+9zZWTR8Z9Hrav9irYkM+OtVxChbrSZWnP6jMV1bWeCAcwxo7ptgX8SkfA2/Vrp4zqz+ghfRM7wlVWwqRJFDQc5ONFz1NZ/jE9HOM+oE/3lMY+VWe7woiw4LVNPFGxJXR7bTakKEpXxn8vqKiuJRprik1sjBoqqu2UPDep07WX3Q/u57NXzcK8tQpZuJDpJ0/hOUdUu43k/BW/WpOXtXNvPRHLYn9DFIivGJZMpLdnwYZ8dKzlGirUlS5JexsztyObO0X5wKyxlJX04smKzfjD0RuihvL1u3il8kOmjh7Y9IEj0jlwAJYuZcgJJ7D4uPibR9DrDfEe8mDHurnLKr1jzn9tExFfIpK/Ik1FdW3SjoBqpBVFyQeSNesJ3gvmzRyDJQKO5W6MGWr2HuCc3/wdESFiCfNmjuGdyh2MveJCBvhi0ndW13qi2p3Z9Ff8Ciblp+Msch0phRGhIWriKoaFbZNLBRuU9EjV8CjlPLdWfena5Eot9GS0tzFL5n0eX1ocl40NJBragEh3O44GRbIbqxiNGTCEPmj4179y8kjuWrqBhqghIhKXiOR6+3OlVKaiKEprSdWsJ3gv2Lm3nqumjOQ3f9tAY8yuevXrpRscL7sd1ljzQS1fu+V7HB6ISQ+b2XQrfhVGhCt9BQHStb/uvcUV/W6PjWShiMHiBTWfHKCqpk5tew5jpfhsFbDS+V0DvAdscF6vyvzQlGzFNTC3LVrHtLtXUFVT19lDSolb0cQ/zkzEX5eV9OL88UMTPNIPzBpHt4hFYcQ22HHNLpKI9LB9z5s5BmMMInDLs28RjRn2N0SJxgz3LKtM+B6mjh5IYcSiR2GEiCU8MGsst00/Ni5Z1L1BGYPnpVcURckngs16Fr6+yXs9oE93ojETZ5enjh5ItwLbdgoQ8cU1dj+wny/f9J+UrKnghvOu528nTvEEMdiC27WzU0cP9O4zhRErbha1Jfa3rKQXV045iiVXnxFnw5Otu/iq05k9eQQA9yzbmBP3aSU5qaq+DAcQkd8Ci4wxS5z3XwI+3zHDU7KRXJpaS+a1yET8dbJKARNH9eeFa8/w4hvdEJOCqo18Ov0ciqINRMrLk4p0F3+sohti49bRXfzmjrhKM5A6xryqpo6aTw5gMB2WLJrrszCKonQM7WErXHtcu+9gwmcvv/ehJ1wvX7gaEYjFYsw8pRSIt51uoyERi+4H6nl5+S/psfp1WLiQ2WdNY7jj6b5n2UYvdvz88UO9YyWzwWEPCO1FWUkvSnoXIUhO3KeV1KQToz7eGPNf7htjzPMicnsGx6RkOblUDSTVQ0Vr46+DbZxdY/69BauSVgoIHmvz62vo9oWzqI82cMGFd/KrgWVezd1kN6ngdXdLPi5+c0dC7LmLe1x/nXRoCpsBmD15RGJiazujYTaKoqRDS21FmL0MVm4BuwuzAWIGLBHPg+12DAV41Em6D4YSLr7qdFa/s5Uv3/SfnkhnxgzKoFlBnMwGuw8IxhjmzRzbrD1s6XXJpfu0kpp0hPqHIvJDYAH23/mFwEcZHZWS1bTEG93ZXtR0jFVLxug3lsZJNhKEaCyGr1AA0ZhJSNj09l9ZSb/p59gi/fw72NR/eNy6yYxx2HU/sviQZpteBPc5Y/yRcVUISnoXZfy7yaVZGEVROo90EuddUvWA8FducRERiiLxMd4GQ0QgakLyhxzKDhHKbvkevP4qH8x7kD/0HwvlG5g6emDa9xj/OK+YNMJ7QHDj4ltzXVLZUK3alT+kI9S/BdwKPIMt1P/uLFO6MOl4o7PBi9qcsWpujK7IdkNWaj45EFf2EKAhardwFl99l4glDOjTnbnlG+ISgF748kCGfv3LFDme9E39h8cZ9+aMcfC6p2OMg93tHn1tU9IqBJlCvTuKoqRD0FakKk2bqgdExBLPzoEtxAuteA+2G/7iFnkJDUPZtw+mToUVK/hg3oNM3HIE9VXvAfbM6X0XjuWKSXY8eFhzpIrq2rj7Ro/CiHfMltjD1thQrdqVHzQr1J3qLteISC9jjGYjKGmTLV7UVMYq1RhdER+NGQ40xjwjDk0Z9QAFlr38/pljWbttDwCjB/flewtW0dAYw71XHLV3J/2mfwdijUTKy/nVwLLwKjHtbIz9+4zGYl7psWAVgkyi3h1FyU/ae9Y0aCtS2ehUPSCWXH2GF6O+dfenLF//IQcdp4rbQO6eZZUYA40xW6RPO2Egs/020SfSWbiQ5SNOI7rpLW+sjdEYl85fRcSyHTH+ZFF/ozuAmC8eferogUwdPbBF101taNelWaEuIqcBvwN6AUNF5ETge8aYKzI9OCW3yQUvaqoxujcIN37RnaacPXkEJb2L4rzgruF0S37NLd8QFx85bPd25j92MwWFwHI7cbQMEoxtMmPclpthWGKUe76jB/flyYrNcUmumboBqHdHUfKLTM2ahpWmDbPRqcSrWykFbNH813W7ANuOL33nA25d9LZn34sKLCKWpBTpzJjB+Jq6eE+92AI9+BARbHQHUGBJQjx6smuVzN6H2dDODi9VMk86oS+/Bs4GFgEYY9aIyJkZHZWSF+SCByAoYt14SHf61K1d7veou9ObyZKYKqpr4yoNDNu9nSceu5miWAPlc5/inGaquwSNcXvcDIOJUX7R7p8xiFiiiZ6KoqRFR8yaNncfSeUA8IeeuA2DAE+0gy2gJx7dj5u+9JkEkW5WrODVn9zFgLOmeY4V11MPtqPD7/gY0Kc7T1ZspuaTA4ivaRLYjZPSiUdvib3PhvBSJfOk1ZnUGLNF/P3RIZqZ4Sj5Ri54Ud3xhRk8v6j1e5zDutmt3baHOeUbvGoq4BPp0QYuuehOfvWFz3mfpesJyURzprISp2tqyIyBJnoqipIOHTVr2pr7SELif2JuKWAL6L9v+JCbvuQs8In06869niX1I5G7V8RVgnE99RDv+PjeglVEY3avC0vsTtVRY7AEIpJeGcaW2PtsCS9VMks6Qn2LE/5iRKQbcDXwTmaHpSgdSzKDl+wGEUzQvHT+KmLG0OirNFD28Q4WOiL9oR8/wK++PTUh/j2VJ8SfyJqJm2GyGYNsDFFSFCX7yFQFsGTrhpXGTSeJvkdhhIs+O5RHXq0m6kQkWmJXgYnGDNGYYcnaHVx5yiAv3OXVn9zFkvqRzYpg9x4RDHe85HPDGDWgN+NLi9lS+ymL1mxn+omDmj33ljz85EJ4qdJ20hHq/wXcBQwGtgJ/BTQ+XckrwioNuDVvwwyrX+Q2RmNYlsSJ9GG7t/PY4zfTLeZ40n0iHfDKh7kCOXgT8CciRSxh3swx7R5DnmrGQFGUrkM6wjiZXWhJBbBozGCM4YFZ47x8nuCx3JC8oBPDrY0ejRksJzbcraYV5ujw2+hozDDqiN4URiwwdoJ/RIQGx2Y3RA0PvvgW3/3hd+jx+quwcCENJ08hOn9VSgdG8MHBT3HPbpw/fihVNXXe+QSb0iW7nuk+/ORCeKnSdtIR6qOMMTP9C0Tkc8A/MjMkRel4whIuU3m7y0p6MW/mGCfj3+JgNObFQI74eDsLH7+Z4gIo/+1T/OoLn0vYfkCf7nEhJwP6dPc+CyYiuVUK/N3u3PXaaqBzITRJUZTMkWx2rz3jn/2OCYBL56/ihWubGsL5j+WGjgRD8Zas3eHzWBsKLKExljxcr8lGr0REuOXZt2wPuuNPaYjZ9dNjBrofrOf+p2+jaOs6WLiQqrOmcfndK0IbEiV7oJg3cwzdC+2HgoglXgWY1oSntMQuqw3Pf9IR6ncDY9JYpihtpiMy2N3W0hBf9zYYux2WyR+sqR6xxKupPnpwX757RJTJV/yIom4QKS9PSBx197F+5yfOjcZ4Qtyloro2LhHJmERvjiYRKYrSHiQTki1pPNQc40uLMb7kHZH4pkLBUEJjTLPhHEL8DOjc8g1AvE1fu20PBjjozFwa0xTq1y1iO1h6HKzn4T/expit66j57YMcMWMGFb78HX8CaKoHip1761ly9RntUnJXUfwkFeoi8lngNKBERP7H91EfIJLpgSldj44Qn8HW0nOXVbLk6jPijhNmWFPVVHcN/ydvvcvYm25ErBj/mv80Gw8cyviaugSvkd+zBHZ9Xb9H3W3W4d6wHpiV2F5ak4gURWkPkgnJljQeao6ykl48MGtcXDifX7AGjzVv5tiEULypowcyd1mlJ5AvPGUYxT27eT0rgjbdfe1Weokawx3nHg/OuTz2+mb+sWYTD/7xNsZtXcfffvBLzr7s20nP3a3mkuqBIsy73RHhKVqiMb9J5VHvhl07vQDo7Vu+F/hGJgeldE0yKT79Zbr8raWjMRPa/TNoWJNVSJk9eQTvf7iPN5evYv6jN1IYbeAbF/yMt//5KUUFb8eVOwzWZXeJGcOl81fxwKyxTBzVP/T4QUOsXhpFUdqDZELSDR1xkyB37q1vk32eOKo/L1yb6HEOjiFZvoy/NOLcZZU8/sYWROCKSSNCbTrYMeweBn68eB2LrzodgNfe2syDf7BF+rXTvs83LpoZd6ywMrZuBZdUDxTJrnEmZ4h1djW/SSrUjTEvAy+LyMPGmE0dOCYlT2kuYSlT1U2CZbosX1hJ0LPj4jesVTV1ticFk1AhZfTgvvTdtonrF9xIt2gDF8y4g3f7lQKJMZbBKisRsdtaRw1EozEunb+SF649M6HaTDJDrElEiqK0B2FCMpgEOW/mmDbb52THce3Y+NLilKKzrKQXJb2LMAbqG22vtr9nBcTbdBG83KHGmKHQ2A6hyP5PeeCJWxnriPTnj5/E6YH65v5QSP8saGFEmD1lBKMH900q0lN5uNvb+62zq/lPOjHqvxOR/zDGfAwgIocBTxhjzs7oyJS8IkxsAp53xM3ez0R1E78hK4wIF51qT5lCfDxjcLzBhCGAq6aM9Az0gD7d+fndi3l4/g10izbw1J2/p7qmB0W+bnfBttauuMbAD/68Nq62r4jEGVn/LEBLSkcqiqK0laAA3Lm3PmlzuGQ0J0qD94UrJo1oVnQGE/Hnv1qNiFAQEc49cRB9exSyZO0Opo4eyOKrTmfBa5t49LVNXqWYCSVFDLjgAro5Iv25YyfSPRLusIHE+HrLiXlMVnAglYc7E95vnV3Nf9IR6v1ckQ5gjNktIv1TrK8oCQY6aPSXrN3Bvcs30hCNeTGE7s0gWN2krYwvLbYbXmCX4Xrsjc0JcenBsSdLGCrpXeSVFXv00b/x8PwbKGxsYNbMn3HO8aO5b3BfFq3ZzoTSYvDFLbr4a+5GfREwkUDcZkKzDtQQK4rScYQJQNeWpSM20xGlwfsCkFJ0VtXUsWjN9rguo7ZmN3SLWCz69zbcwjBzl1Vy34VjeaJiC5YIxhju++oxDL/4m5jXX+V/zr2Ov4w6k4jABROGevsPCwEKxtcDSR8oUnm4M+H91tnV/CcdoR4TkaHGmM0AIjKMpD2+FCXcQAdr2tbuO4gxeMa2MCJJDXNrDFCwQssFE4by6GubaIgaBElpINOpQLD59TV8YfYMChsbmDnjDt49vJT3yjfYXhskrXq5fj7/mSO48UvHAIQmLU08uh8nDDmU0YP7puXJUhRFaQvJBGDQPt6zrNKLYffn1NzjJH6m6ngcfBiYOnogU0cP9Kpy+XHvK37njp+oMU5MupM8GjMsWrPdywsqNg2M+u63YE0Ff735lzzTeLS9oYGH/1nNgtc3efY7+GARjK8HuHf5xoT7gj9MMuxhI1Peb51dzW/SEer/C6wQkZed92cCl2VuSEquE+Y1OH/80Liato+9sRmwDZbBcOXkkQlhKMGmP8mE78vrd3kJTxNH9U9aoUUQIpadvNlctzeD8R4efjL9eMrf3cXgw3rYK1RW0m/6OdS7Men9hwO2h6UxFqMxRmgTIz/+CgYRSzyR7veix4xdK/hAY4yX3/uQv2/4EEjd5ENRFKW9CBOAwVybRWu28/TqbRQVWF5zNjf5srmOx8kS5+9dvpFozDBnaaWXZO/WYg+K9ALLDke5/dzjufW5t73GcxFLmH7iIJ5/ayfFpoH7nriFEqdO+ob+Y+Gv73n7iBk42GgosAyNMUIfLILXwk20nVBaTEV1LVtqP40Lk5w9eUTCPS3fvd9afSYzNCvUjTEviMgY4FTs0qX/bYz5MOMjU3KWZF6DnXvriViWJ+BnTx5BSe+ipAk3waY/YcL3yTc2c+Of1gLw9OptPPKd8V51An8co1szF2xPy5baT9MyJMbALYve4mCjPY4VL7zGkmd+RFG0gRkzf867xcP8a+MWdAk2MQqeW0V1LfddGF8xwF+/vajAImaapq4ONNoNlQAaosk9VIqiKJnEFZv3LKtk8Zs7Eiph+b3YRQUW004YyOzJI5PaqqAATmyOZCfZ+2PTwZ6FjVgS5+QZP7zY88a7uUT3fXUUo777LUq2rkMWLoQZM5haU8fcZZUcaIjFhweI0KOw6cEiVQEE92HEfUgxxiAicWGSYeecr95vrT6TOVLVUT/GGPOuI9IBtju/hzqhMKszPzwlF0nmNRjQpzvRWJOHJVkiJ6Tf9OcHz6yNW7bw9U3c9KXPxHl8igosz8viXy/Ywjru2AgN0RiFEbvOOcCw3duZ//jNNEYM3V9exhc/7ME7SzfQELW971NG9Wf5ezXeMXcGqgi4Y05mzPwPONGYISLCAefhIiL2+pYIBVZyD5WiKEqmKSvpxezJI3n+rZ0JlbBcL7brqEkl0sOwkzeb3rtJ9tBk0wstu8HcNWcdlWDHS3oXeUUAuh+o9zzprkh3x3/fhWP5z0dW0hBrcgbdf1GT8wSSx+IHS+26529M180l0uozmSOVR/37wKXA/4V8ZoApGRmRkhcEvQauB0JEMMaeNkz1T5xu0x+nx7O3bMhhh8TV/3WTOjF4nneAl9/7kCpfM6LgsV3BbLCTSYfUbOOJx26mMNrAwz/9PV8cWMbUgXacoiUGY2DKMf15pfLDlIa6OWN2xaQRgO0Nunzhau/G5Ir0oBdeUZSuRzaEGKSqfd6W8A47eXOsFybpT7J37wkHGmO8vX0vly9c7QnoYBGAQxrqmffEjxizdR2v3n4Xpzki3WXn3noKIhYNMbsa2FVTRnqhk6mqbUFi+I8/7Ker2metPpM5UtVRv9T5PbnjhqPkK34PhL8lczKCzTbCvN/jS4spjIjX7KIwIlx46rCE+r+uIV//wSc88uomojG7lvqC1zaxZ39Dwv6DN5qCqo0c+uVLaHRj0rcX8Zs5r7Dk6jPi4u5/vHhds4Y6mTEL87QHp5cLI5KRqjiKouQO2RRi0Fxd9NaOy07ePDNhP0Gb6BfQfidI78YD3PfUrYzbuo7rzr2Oqy77dlyBgbXb9tgFDXxJn1NHD0y72laqh5SuSr7H33cmqUJfvpZqQ2PMn9p/OEq+0tKnbX8M4HNrtvPArHEJYr2spKlTHTSF0gRjve9ZVsnsySO58NRhPFGxxfa4mBgP/aMaaIptD4p1gHdeWc3ZV37LFukzf8a7h5cC0NAYY8naHZT0LoqLu3eFdFVNHU9WbPbO02+8mquk4E/AdaeX1UuhKApkd4hBez5EhD0E+ENugjZxfGkxMWPocbCe3/2xqePoc6PO5NT3a/nx4nVxse8A3QqEq6eMDL13NJdHla+x5m1Br0lmSBX68hXnd3/gNKDceT8ZWA6oUFc8mvOitPRpOzGhaBUvXJtY+7yspBdTRw/0Yhjd8lh2xRTbg//cmh08t2YHt597nBdasnbrHl5c94G3n8de38zabXsAW/ADXPHjp3h4/g3sjTZQ9fifqayoxy3SGzVNdXqDDyBhXplgpZbmypSFNUlSL4WiKNkaYpCsJCOQYL+C5XNbattcO+7Pc9pS+ykFn37KQ36RfuxEAB55tTouptzFGOKSPsPKRarNVTqbVKEv3wEQkcXAscaYHc77gcA9HTM8JRdI14vSkqftYDc4EUI9R1U1dUyd84oXzuKGrB9ojOH0pfCqvdz4p7UURoTCiMWt046NE+rl6z/w3s9dVsnsIXjNjL5z4Z1cMHAES64u5hfPv8vSd3fRGLNr9vq79YVVb0m3UksqQd5VvRTZEIerKNlGRz+8p/N/mKwk7oA+3ePuDfNmjmHttj3MKd9ALAaNMePFdwfvG8HjVtXUJXSydsNVlqzdwX1L3gwV6QAbPqhDrMRx++Pfoflrm6oKjIbBKJkinTrqpa5Id/gAODpD41FykEx1Wwt2g/PHc7vGcsnaHdS7reiw6467FV4ChV4AnIZHBgR+8bXRPLlyC/16dWPpO7u8dQbXbOMbd93kNTOqLBnGgD7dKSvpxY1fOoZXKj+k0LnxhHmzgsmoYCeCRmMmaclG95zVuNtkUxyuomQbHWUrwv4PIdFD7s9B8pdk9M+MdotYXDp/JTHTVMYWSIg1B7vs7g+eWQsiFEbsJPrLF65O6GTtdriOfPopv3sqXKSDU/3i6P6Ur99FQ9S+T5x1TH9u/NIxaYe0JLsW7sOD//xT9f1QlJaSjlBfLiIvAo9j/73PAJZldFRKh9BeHstMTcUGu8EFM/tF4EvHD4jbxmC8TPxkHIzGwBAaszhs93YWPH4z3dyOo/2HEzHEVRfwe1wgvISXf50ttZ96Caf+/SjJyeY4XEXpKgT/D11hHLR3wXuAW5JxS+2nnn09GLUT4oOm2RLbbrv29OX1u5oqdBlDNGZ44JWqOJEeETjYGOXNrR9TsP9THnASR12R3lTY16YxZryqXAWWPfYwkd7ctfDPGLjXItgpNVUnVkVpDek0PLpSRL6K3ZEU4H5jzDOZHZaSaVrrsUw2xdceU7FhDw5hzTD8N45P6hvi9vHZssM5texwavcd5JF/VhPSaZqIwBuBOrgAIz7ewR//8EMiJso3fR1HozG7/KJreP1jCiYfha1TUV0bl3CqBrx5sjUOV1HyjVQOm+D/IZDU3vmrdLn72bm3vqn2eUQQmkoaRixxQhbjx7Pw9U0JY/xH5Uee8LbAs+uv/HszD4eEuxhsO2+wZ1YLI8Ib1bVtKp/ob7h0oDFmV40xeCK9wMLrSq02S2lP0vGoA6wGPjHG/E1EDhGR3saYTzI5MCWztMZjGRaH6J/ia6nhC5bL8sce+mvj+m8i40uLMdgNhgyGmacM4+8bPqShMUbUwOvv1/L6+7VMP3EQlgXRaOJxowZwHlDcm8ZRe3fy6GM30acb/Gvhn3j3n5966xdGJNTwuomr/hJfzYXCqAFPD02iVZTM05zDJmwG8d7lG0PLy4aVxHX7Ybjru0J5QJ/uLFqz3SuzKGJYsnYHowf3pfzdmoRx+v0trmulx8H6UJHuIpZQaInnkFn85o64sbUU/0NHUYFFcc9ucSGOV04e6XVDVZultCfNCnURuRS4DCgGRgCDgd8CZzWz3ZHAfGAA9v/W/caYu0SkGHgSKAWqgW8aY3Y729wMfBeIAlcbY150lo8FHgZ6AEuAa4wxRkSKnGOMBT4CzjfGVDvbXAz80BnOT40xjzR7NboQrRGPYd3YUon8VJ6aoOh3vRFAXLWAsJhAP0cWH8J9F47lkodXAsbzbjy9ehsRy/aqhHnVn/7XNn7xtdEgULp7B6Mv+k+KukGkvJxxJ5zALwZs5gd/fgsLu1+S26DJTVyq3XeQx97YjO0jgtmTRyStEJBMdGqyZGo0Zl9RMovfYVMYEZas3cGVU46KWyf4f9hceVn/flzb55bQPbL4EACvGd1fnOUNUcPd5ZXEYrG40BjBtv3BfKNUIj1iCZbARacOY+LRJXEPBM05pZqbXXAfOtzco67c4EjpONLxqM8GJgCvAxhjNohIeO/1eBqB7xtjVotIb2CViLwEfBtYaoy5U0RuAm4CbhSRY7Hj348DBgF/E5GjjTFRYB72w8Jr2EL9HOB5bFG/2xgzUkRmAL8AznceBm4FxmE/jK8SkUXuA4HSOo9lWDe2ZCLfL8SNgQdmjY2rUx4U/a5x9nuvw7z+YJc6bIjGKLDsOMH3P9wX6j0XSOhc6ufJlVv4zdheDL3k6xBrhPJyOOEEb+MCS7zz3Lm3nqqaOs656+8cbIzfX4/CSFyJrzCCNztNllQUpbNxZyjBFstzl1U2W5LQb8v8s6Jh+wE72dKu8mL41V/f8zziT6/extnHHuEleMaMwQSiyw1N5ts15b0bD8TXSXdEekTAsuCU4cW8UV3L429s4YmKLcybOSah7rrrcIGmEozpzi64yaP3LNuotlvpENIR6geMMQfFCVATkQLiZ6JCcSrF7HBefyIi72B7488FJjmrPYJdk/1GZ/kTxpgDwPsiUglMEJFqoI8x5lXn+POB87CF+rnAbc6+/gjMFXugZwMvGWNqnW1ewhb3j6dxvl2Glnos/eK+uTJUiXXQV/LCtWcm1KsNiv4rJ4+Mu1GEef39041utr2dyU9ColLU54qxAp6ZPWvfodtNNxF1POmuSK+qqeON92vjZg4G9OnOkrU7EkR6RJJXf0mFJksqitJW2jorV1bSiysnj+SupRuciliSti0KCtsLJgzl0dc2eftxBe2BxlgyXwkvrfuAwgKLwohdGUss2yPeGDUYY/CbcwH6xg7y9JI7KN26juumX89zx5xBYUTsmVSxf/9z40eOnW9qQhcM35k65xWvWtic8g1c7cwihNnk4DUu6V2EIB1iu3XWVYH0hPrLIvIDoIeIfAG4AniuJQcRkVLgZGyv/BFuuUdjzA6fd34wtsfcZauzrMF5HVzubrPF2VejiOwBDvcvD9lGofUGIF1xb9dBb3ovEn8DSEf0J/P6u8tqPjnA3eX2jSAiUNqvJ5W79nnbx0wgttHAdV84mvL1u9jz5joWLLyZbtEGrrtqDqfWHwoVmxnQpzvfW7DKV/LR9vLv3Fsfep6GprAYl3SurcatK4rSFtprVm7q6IHcu3yjVw0lXVsUdDYU9+xGYcTy9lO772CcHQ0jBhxsjGGAAstwx3mjeaO6luGH9+T//vZenAEvOlDPfX+8jdKt6/if6d/nL8edyXVTjqJ8/S7WbtvjhT26zpgCS7xa7sE+F34HzsFGw11LNxBxslqTNa9zr3FH2W6ddVVc0hHqNwL/CawFvocdevK7dA8gIr2Ap4FrjTF7Xc982Kohy0yK5a3dxj+2y7BDahg6dGiyceUdHWEA7DroY72yhMHGEu466cS2J6tz+/L6XZ7XO2rg/Zp9CfvyExFAYHJkL99wRPoFM+7g3Wg/nnGaIWFMQky7IF6ZxSCFEStOxLek+ZMmSypKy+mqdjtIe83KtdYWhXXxdLtEjy8tZsFridVbwnDNbWMMbn5mLYWRxPK6CTHpx0yEqOGu8g1OKKRJmDGdckwJM08Z5iW5uvY4rJeFPSNrMXvyCEp6F4U2r3Ov8fnjh3aI7dZZV8UlpVAXEQt40xhzPPBAS3cuIoXYIn2hMeZPzuIPRGSg400fCLidZrYCR/o2HwJsd5YPCVnu32arE5LTF6h1lk8KbLM8OD5jzP3A/QDjxo1rNpwnX+goA2DXQT+zRQatJQ8RO/fWxyeLBuPRA99oJCIsevrvzH/0RgqjTXXSXRpCsk4LLTu+vqyklxfT6O1PEjvbteTaarKkorScrmq3g7SnZ7c1tiiVwF+ydgcLQsosHnlYD7bu3p80djYWKJkLqRNHYwaiMXt9Y+zSjZYFBRGLm770maR14C0RCiJw7omD+MvaHXGdTv3nkewap+tkags666q4pBTqxpiYiKwRkaHGmM0t2bETK/4g8I4x5le+jxYBFwN3Or+f9S1/TER+hZ1MehTwhjEmKiKfiMip2KEzs4C7A/t6FfgGUO5Ug3kR+JmIHOas90Xg5paMP59pbwOQyjilcwPwb98SoTugT/c473csUBrAb+4LLPh2/0YumXOj13H0PZ9IDxKxhM/7OtdV1dTx5taP49Y56zP9uelLn0nLsGusoaIo7Uk2zMolS5IPNgFy2bJ7P+Dm9tiJoyl60yUV6W54jf8YTtVdImJ3MgUSyucCXhGDHoURJgwv9jqoJrt/pXuN23umOhu+XyU7SCf0ZSDwtoi8AXixBcaY6c1s9zngImCtiPzbWfYDbIH+lIh8F9gM/Iezv7dF5ClgHXbFmNlOxReAy2kqz/i88wP2g8CjTuJpLXbVGIwxtSJyO1DhrPcTN7FUaV8D0FbjFNx+3swxcXXSkz1EVNXUsWjNdrpFLLvTKKkznIfV7uCSuTd5In3jwOFEECLG9sqc9ZkSyt/d5d00IiJxIt2tYAN27GNBRBJEOhDa+ENjDRVFyQSdMSuXyungOlr8HUSjJqRMrghTjinhhCGH8ubWj3lp3a44+11gCd89uYQLfnwFQ3wi3RL49mmljBrQm/U7P+Hhf1QTpvPXbtvjhbxAU/lcSKwD39w1TPcaZ2KmWmddFUhPqP+4NTs2xqwgPFYcktRgN8bcAdwRsnwlcHzI8nocoR/y2UPAQ+mOt6vRXgagrcbJrqQSozFmKCqwWLttT+h6/nJaowf35fKFq4nGDAejMQosoTFYaNfHsN3bWfj4zXRrtGPSKweUMap/bzbsquNgzK44c8KQQ1lR+RESjRGLGS48ZSj3LKtk+omD2Lm33vPCFBVYTDthINNPHOSViwx6lIKNPzTWUFGUXCWdxnQu/nKPYHvNv37yID7YU8+KjR95y6Mxw7L1NSxf/yG3n3scf9/wYVziadGB/Vzw4ysY9s5q/vmTu3iuzp79jBmYeHQJRxYfwo+efTtUpLtJoX6b6y+fmykvtYaqKJkiqVAXke7AfwEjsRNJHzTGNHbUwJTcoKXGye+NAbs0liuy3bbM/jrprhj2l9MqtATLV+N84tH9WPpuTVwmv0vp7u0884cfYtzE0f7DIWZY/8Fez3t+oDFGSa8i5s0cw6XzV2EJPPTPasCu9XvdF46OO8cJpcVxSbL+m1WYKFcDrihKLhJsTBexxLOzYU6HYLnHxphh0ZrtiNhe9Zgzq2h5YSuGm59Zy2llh3NIUYSl7+yi24F6HvrjbQzZuo51v7yX1SdNIfLSe0RNUwWunXvrifnykSJihyIOOewQint2Y/Tgvh0egqihKkqmSOVRfwS7NOIrwJeAY4FrOmJQSu7QnHEKCnN/CMgVk0bE5X4WWMS1ZfY3PvKL8JgxRBuNPZ0ajTHksEM476RBPL16W9yxh+3ezuOP3Uws2sDFM3/Gu4eX2jcJEuut/+CZtVx8WikRS9gfKCn266UbeOjicV7r60vnr+Kgc5MpKrDiblZholwNuKIouUiwH4bfDkdNLNTpMHpwXwS8vhbBWPVIYJ49ZvC87ScXF3DT3KaY9IP9xrJsaaUXNtMQNazf+QkTjy4hYom378ICK6HCS7BraEdVO1P7rrQ3qYT6scaY0QAi8iDwRscMSck1whKK3KlSv+F0hfn+hihFBRZvbv0YS5o60VnOlKVrYN36twP6dI9bz7P7zu+H/lFNtwKJi1cftns7Tzxml2CcOeMO3j281N7EQJTEmMmogUf+WU0kIk0NNBwiYntxzh8/lCcrNntJSfb+4uPok4lyNeCKouQaA/p0T6jC4hIWbeiG/okIAhRGDLFYvK1tjMGYoYeyevPHcdv2OFgfJ9KfO3YiX+9e4Nl0l0df28QTFVu478KxXqikWxbSP5vp2mwXDUFUcpVUQr3BfeE0E+qA4Si5jt9rEY0ZRJoy7IG4bqTl79ZgWXDJ50rZs7+BRWu2M2dpJQBXTxnJLX9+254ytYSZpwzlkX9WJ9Q4dzEGJpQeyj821iaK9P7DE2rsnnTkofx788dEffuIGrAMXHPWUWBsT3pE7MRRV4yPLy0mYglFBRbGNJVu9BMmyrXqi6Io2UZzdmnn3nrPXhdYto10Z0FjUcM9yyqZMLzY294VwwcaY3SLWKHhiADD+/WME+rB6i5LjpvI108ezM49+xO2dbtQ79xbz5VOR1GXVCGGGoKo5CqphPqJIrLXeS3YnUn3Oq+NMaZPxken5BxL1u7wSnPZYrapNJbbEONHf36LFRs/smPTHWfJc2u2ezGLAL986T1vn90iFtt278cSIZqkF7Ux0LOoIE6k//S/5/ButB+Q6P1Zs20PBQWWnTzq+1wQSnoXcf74oUw9YWCoZ7ylYSxa9UVRlM4imRh/ef2upLk2Lq64LYzY69zxleP4wZ/XEo3Zpvvp1dv487+2U1ggLLn6DC+ZtMCSOE+45bS4MNj76tuj0PvMFenjt63juvOu54XPnEmBCH9Zu4PGgGemwBIvTj7YuCiZbfafv4YgKrlIUqFujIl05ECU3Keqpo65yyq9sBERuP+icQlxgq9Xx1fK3Lr7U6+mbhgHozH+9s4HJNHoADTGDOv/8W9PpM+64A7eifWLWyciTeE1lojn6Z86eoAX334wGvNuAMnCVVoaxqJTroqidAbJnARVNXUJuTZL1u6I68oJsKX2U6IxY4tsA+s/+ITh/XpSuaupC3TUGKINhosfeoMrJ48EiEv0tN83vW6IGvbstyfs/Z70hVfdwZI+J0LMEIvFsCzLKzQQseyQxB9/5ThuefYtRITLF65OeLhIVtfdf/7+cBhFyQXSKc+oKGlRUV2LOBU5CyPClZNHMnFUf8+j4a7jjzePWDDzlGH8fcOHibV2fSRb7hIMd3mvZHiC7v/ZV0eD4MXOu57+4f16etO7RQUWO/fWt+UyJKBTroqidAbJnAQV1bVxuTaxWCyh7CLApfNXeo6XxpjhoX9UJz3Wlt37ufFPa73qLqlYs+XjhHCXFYePoWF/A1GDF1rYozCCwXDl5JFeHHrEstJyelTV1HHPskovGVadJEquokJdSaA18dRVNXUJXeCmjh6Y4NG4ddqx8WW1rKa7heUEkjcnyoOExaT7Rbol8POvjub8CU2eFP8UKCQ2wWgLweunVV8URekMkjkJgrk2F546lMff2ML+hiiFEfG866lmOpORjv3etu2jxI6jn3ppcRgMD8yKn411ScfpESwrWVRgqZNEyVlUqCtxtCae2r8NNHWBKyvpxZMVm+Mqvfzwz2/FJRhFxLLr7Dq104Olu5ojVKT7iFjCqCN6MaBv9yR76JhOrVr1RVGUjiZVFaqgs+KxNzYDdmjK3GWV3HfhWK/meXMe8lS4ZRpdgp70546dmLDN5KP7M3FU/7TPJ4g/qdVtUjd78ki1wUpOYnX2AJTswj9VagxeyEq627jJmGF1xWOxGA0x43WTK7BsMTv9xEFeNZhUcehByj5OLdItsSvPrNvxCRf/voKX1+8CmsT0bYvWMe3uFVTV1FFW0ovzxw8NNeRVNXU8WbGZqpq6Fl2LdK+foihKpkhm2/zL3UZFhY6nRBCv9KE76VloCRGBz40oTtpyPAxj4JLTSrEEejUcaFakA0w5pn9Sm5vKVrv47zsRS1SkKzmNetSVOFoTT51qmy21n3LGyH4MPqwHQFyM49FH9ObGc45h4qj+TlfQlVgWxMLL9sYxYs8O/vDk/2KSiHQgQfQvWrOdiaP6tyi5s6UzDBqPrihKNhFsOpfMGz119EDmLrPL4zbGojy/difRmKExhlOa0RAR4R8bE50PQa+5n6ixOz33aqjnoT/YIv2aJCL92IG9ufizpfx48bo2VcnScEMln1ChrsSRysD5Gxn5YweTbfPy+l1c/PsKb/uvjxlMUYHlGH/Dxpp9XL5wNfNmjnHaTEtCOS4/Xzy2P+Xraxj20TYWPnYzBSbKN5OIdEiMrJx+4iCgZWK6pRVb9AahKEq24Hc0GMci+hNGw+xTNGrPer69w67O7BfhwWouYHvczzrmCMrX76IhaneM/uyIw/nnxo+8kJkeB+t58I+3MXbrOh78r5/wXJ+TvO0FO0SxICLMvWBMu1XJ0nBDJV9Qoa4kkKxhTzA5x197N2ybRWu2x73/87+2UVhgcWpZMa9VfeTt59L5q7zGSO5NISwusnf3Qso+3sH8x26msLGBGUlEejD9SYDvfK7Ui3lsiZhujYdcbxCKomQD/r4WblhLQzS8AoorkIOO8UGH9mDHnvq4bs1+CiMWU47pz4vrPgBsD/qpww/ntSpbqAdj0lcdeSrsaaqsJdj2/sdfOc5zBOmspKI0oUJdSQt/cg40dRsNq73rMv3EQV59cnA6f8YMr278yKsM4NbLPdAYozAixAwURgi9Kawur+AxR6QnC3cB2ztjTFP1mMKIcOGpw+LWSVdMq4dcUZRcJKyvhSVCgRVeAWV8aTEmxGM+e9JIJxQlmhDeYom935q6A3SLWF6ToznlGzj3pMEseW1jQkx6JFD+NgaICLc8+xYRZ2zzZo4JrfiiKF0RFepKWrieZX+9cYNJqL3rN6pHFh/C18cMZs3Wj9n00T4ssTyhD7YnZfIx/Vm+vsbrYlpgSdw6LsN2b+exQOJossJhhvgSYbefe3ybjL16yBVFyTWCfS2unnKUV4s8TACXlfTi9nOP5wfPvIVrWX/mlLUdP7yY6/6whtWbP47bJmbwjuFHROgnDQki3QrplVEYESceXbxwl51767UxkaI4qFBX0sLvWXZj1Gs+OcA9yzZ6xtXvXQeYOucV6hts0V0YgYlH92PpO7s8Q22Av67bZU/JGltQ/3jxOgos8TrSQWIJxg1HDOeS00oBQhtwRAMxMzV1B9r9eiiKomQzwbA9t2RuqsT5Hy9eR4Ezszn9xEHU1B3wqmJdc9ZRcTlH3SJ2+KMIjB7clwtPHcqjr1ZjWRaHNNTz+esuYUwz1V0sgYtOHcbEo0vimtBpuIuiNKFCXUmbsPbMbqOgoHf9ikkjaIw2ecYbovbvsDDHhqihqEDA8coveG2TJ8DD6qR3c0JZykp6UXxIN/7vpffsOEcLIF7kp6I1jZ3ac3tFUZT2xm+XWhK2FwxvdMMW5y6rZMnVZzBxVH8e+c54Fq3ZzvQTB3Fk8SGe4+byhattB4kI3z6xH7N+OpsBWxJFukV8DHxEhMfe2Exxz24a7qIoSVChrrQav5c96F0H7OBFX3DKS+/sSrqvxmiM8nd2MaBPd0YN6E1EYEhtk0i/6IKf8W6/UiJOnCXYN6Q55ZUY5ygRsSdh3fCcAqeSwNTRAxOO15rGTu25vaIoSnsTZpfGlxZ7/RzSSZwPzmhGY4aK6lq21H7KojXbGX54T0+snz9+KE9WbPYSVnscrGfK9y9h4LZ1XHfe9Tx/zJlxVQGMCAVivFj3hpihIWa4a+kGCiOW2lFFCUGFutImXC+737vuToXOOnUY81+tBsSOG0/h6Y4aeHHdB7y47gMuOa00TqTPnHEH7/Yr9dbrht3i+v0P98XtQ8BrO42BN6prmX7ioFDD39YSYO1VQkxRFKW9CNqlJWt3cO/yjWk5FMpKejFv5hguebgi8UNDXNgL2B73R74zngF9unsi3Y1J/+/p3+eFYydCND7fqMASRISIZXvu3epeDVFDgYXaUUUJQYW6kpJ0wzuCMeyXL1yNMVAQsbhgwlBHsKfH0r/8kyceD4a72BUF7MYbMeYuq8QYOBiNEbEES2yRPnFU/ziv0vNv7Qy9ObW1MZE2NlIUJdsI2iWgRQ6FnYGKLABTRvVn6bsfhK6/aM12hvfrmVCC8bljJlJoTFyyf0Tg/ovGcmTxISxZu4M55RuIxeza7EUFTZVoNKRQUeJRoa4A4YI8bBoVCG16BE3e9bnlG5qmQgsj7NnfQMSyaEyj5WhYTHqBBf16dWPX3noMQiwGYjXFUtolxZoqD6Tj7W5r2UUt26goSltpb1EatEtA3Exncw6FAX26J1TSuuCUoSx8fVPo+tNPHMQ7lTsSqrsUOEmm/pwkS4RFa7Yze/JISnoXERGLg7EoRQUW004YyOzJIwG8fh3GGM/5oihdGRXqStJ462TTqP4GGsG4wmDtXoNhQmkxz63Z7jUzcuulB0NhgiL9vSOGU2gJDVHDdrdBhlvnV+z9NEQNMWN7ZS6dv5IXrj0zbW93W8suatlGRVFaS6byXIJ2ye38HAwDDD4kVNXU8b0Fq7yQckvgqycP5sjiQ/iw7mDcMYoPKeTX55/E0CJD6Y3/yZBAdZdZnx1Gcc9uzFla6dVWb4gZnv33Nv6ydgf3XTgWg30PEYHZk0dSVtKLJys2e031AC6dv4oXrj1D7azSpbE6ewBK5+MX5MbgJR6FTaNGY8YT4Q1R4yUa+fflr917wYSh/HjxOuw4deG6LxzNNWcdRSTwlxfmSf/5V0czenDf0DHPOnUY15x1lNdtD+w6vK73fPFVp3Pb9GM1OUlRlKwkmd1tC1U1dTxZsZmqmjrv/eULV7Nk7U4uX7g6bvm0u1dw26J1TLt7hSfa/c6TmLFDW6bdvQIJuNmLe3bD+nQfH078PEPWrY4T6QWWUNyzG2BX4ioqsHDNdGMM6htivPxeje84hiVrd1BVU5fQdMlg2uW6KEouox51JakHOmwadc7SDXHbGmPiPdYGDjbaseSuhI7GjOdV+fXf3uOsz/SnoTF5nfR3+w/HEhg/3N5vsMlGoYVXnnH04L5cOn+lXfnFGAb06e6NXQW6oijZSnvnuYR56MPCAAHuWVbpea7d5eNLi4k4M5gubpJnaUlPVm352Fu+c/tHdDv3qtA66ZYIc8o3eOO4espR1O47GNfzYuvuTxGEBue+cNfSDdy7fCOLrzqd2889nhv/tNY7vmvTFaWrokJdSRlvHRS8D8wax6XzV3nvbz/3OM/4b6n91DOwGFtQL3h9E1Gf4Y8au8mRS5hIB3vadcnaHcxdVhk31sKI8LtZ47yp2p1767n93OO55dm3EYHLF65WL7qiKFlPe+e5hIny4MPAgD7dvRhwt8O0+5BQVtKLJVef4VXUevbf27BnQg19exR6x+lxsJ4HAzHpLpbA+NJD+cfGeC/4hacO47E37LCWiCWcdcwRLF9f45WC9Fd9IdABOyzBVVG6EirUFSB9D/TEUf154dozEqq7iMAZI/vFrdsQM/HdLQIkE+nQVCvdPxUbEbjmrKMSKrtEYwYR4rxDKtQVRcl22nPmL8xDH3wY8Dc18idx+gsCjB7cl7uWbnBqnRssC97b+QlAYnWXQMfRwojldaMOnueSq+PvGyKCOBVfLJG4WYWIJVpRS1EcVKgrLSZZdZde3dP/cwqK9MM/O5ZIVa3duMgSHpg1jiOLD2FOeVOoTWGB5TUv8nuPigosjGPwo7GYTpUqitLlSOahDz4M+MW8K9KraupYsnYHtfsO8uir1fi1dmPUsGLjR0lF+heP7c8JQw61vfD/2sYqX6hitwJh9OC+PFmxmfGlxV6DJPdhoUdhhNmTR1DSuyhuzFpRS1GaUKGutIqqmjruLo+v7jK8X0+v3rnbyCKMhOou/YfDxlrP+V4gcGTxIYDtWS+wbJf9fReO9Yx20Ht0xcSR/HrpBiIiCeEvWpdXUZSuQHMe+jAxX1VTx9Q5r4R6wsG246k86X/f8CEzTxnGnKWV+FKPsAS+csKguFlXt1Oqa7vdSuvNhVwqSldGhbrSKha8tskroQVwwYShTB09kHuXb0TEivvMj1+k//S/57Ah1i9B0Md8lWQEoTEGPQrjYxWDDZYunb+SaMwQxSBieeEvmSqBpiiKkosERXCw2osfS6DoQOpwF+NUh3Erg7nEDCz693YsSzjQGKMwYneUvnLKUSy+6nQvB+meZRu9RFK1zYqSiJZnVDyCpb3CPnt5/S7mlm9I2mn0ikkjGF96WOhnfpF+8cyfYZ14IpYliSuKML60uNmqCGUlvTh//FB27q1HfHcJY5rWzUQJNEVRlGwile1OtV5VTR01nxxIENkA3SIWPZoR6WCHsExwKsbY8eZ45RgtS4g5je4aooa5yyqpqqmjrKQXJb2LEERts6I0g3rUFSB18w33M7dSgJ2pH7/9gtc2O+Jd4urgugTDXTb0G85FpcUsfnMHBHrh3XHu8YBtuOfNHJPQATXIeN9Nwu5mlzxERhOTFEXJJ9zQFbeiypKrwxsEBde778KxTWEpCJ8bcRgD+vbgL2t3IAiF9Z/yQMrEUbuUY5FTi9c/w+kPd5l5SimPvraJhqhBaOp1obZZUdJDhboChJf2co29v1IAQKMzTSrS1CjUrZMeFN0QXt1FjOEHf16LJXa9df9W6z/4hB8vXue0kYYHZo0F8BKSgjeh5spLamKSoij5ypK1O7z48oao8cJLmlvvsdc3ezYf4PX3dyPsZvpJgziqp/C5ay7m2KQi3U76t0Qwxi776L9f+B0sAE9UbLF7a6To06G2WVHCUaGuAKk9z+NLizEYr+ZtUYGFwdAYNSGy3CZiQSwGQ5OUYDRANAbRkG3/veXjuDbS352/kgJLECRpnHmq5CNNTFIUpauw7N1drN26hwtOGeqVsq2oruXNrR/HrTf4sB4YbO96NGY8B8yS1zbyyB9v47it67gmSbgLCGce1Y+l7+zCsuwE/ismjuDXf9uAZdllGv12Ot0+HYqiJKJCXQHCu5C6HmwXEbvc1lVTRiZ0mgty3omDqXrt39ybpE56Kt7c8jHij103hmgMGqJaJ11RFMXP1NEDmbusksaoLbbd8ogvrvuAr48ZzF/W7iDqNBVysQRGHdGbmDHEfImkbnWXsSEifWDfImo+OUBjzPbIu43rolGDMVF++dJ7znuIxaJxnn0V5IrSelSo5zEtLUvoGtNgvPqM8Ud6hr5HYQSAbbv3p9zXyvKKpM2MgvhDaAAiEYuZpwxlwWubELE74xlDXBc9RVEUpamZ0D3LKnnmX9viqmg9vXpb6DbGwA+ffStOvDfXzOiDvQeSltyNBnKWogbmLqtk6uiBKtAVpY1o1Zc8xRXbty1ax7S7V6Ss5BL8bMnaHTREY+xviBKNGS8RCKAxFmXuskqWv1cD2J6ZIKk6jvqxxE5I6hZp+jMssISIJVx46jBeuPZMrpoyEnESVKPGMG/mGDX8iqIoPspKejF78kgKIiEGOQQDoZ70VNVdwkR6YUQojAinjUh0nkRjdry8oihtQ4V6ntJcWcIwIV9VU8fc8g1xjYzc0lousRheYqklcPKQQ+2sf4dkIr0gRNF/+7RSfjdrnNcuunuhxbWft2vsuudQu+8gBxpjNMbsDnlrt+1p1+ukKIqSD5SV9OKFa87k9BGHh5ZbPHZg7zjHiqu7/SL9v7+SLCY9nogl/OJro7nmrKMQgX9stO8vlkChcxB/OUZFUVqPhr7kKc2VvgpWeVmydgf3Lt9IQzTmifTCiHDRqcNY8Nomb7uIZdHoiPeYgVVbPuYXXxtNTd0BzIZK/uPem+kWS/SkN8ZMQrfS4p7dOLL4EK6YNAKA0YP7snNvPVtqP+V7C1Z5VV/8rN26x6vDqyiKki+0RwflspJe/OS845l29woONETxRbbwmYF9qP7oUxqjxq7SFdZx9DPNi3SAs44pYfzwYpas3UHUd5CIJUweVcKy9TUJ5RgVRWkdKtTzlOZKXwWFPNiecr9IL4xYXHjqMCYeXcKl81chYhvi4wccymonYQnguTe387PRPej3o0uwIoavffMO3ukXH+7i1tz1U7vvoBcL79aPsTuRxuLWjdhlemmMwfL3anil8kPtYqcoSt7Qnh2UXdt/5/PveAmfAMP79WTxVafzi+ff5W/vfEC3NJoZJeOv63axbP0uLLHiHgYiljDzlGGsqPwooRyjoiitQ4V6HtNcyUK3jTNASa8iorGYl7B55eSRXiJQWUkvXrj2DE/0L3htU5xQ31axlm7/czP1jQ3M/FaiSAc4pDDCnmij977AEop7dvO8+oURd7o0RjDM8ryTBgOw+M0dHGjUyi+KouQPVTV13LOs0itJ2x72raykFzNPGRYn1Gv3HWRL7ae8vKEmpUi3gFjIPoM0RKHAslV6gSWcdUx/bvzSMVofXVHaGRXqXZx7l2/0bhDdIhbGwP0XjWXiqP5x6/lF/8SjS7zSjMN2b+exx26mMNrABTPu4N2S8MTRPfWNce9nfXYYU0cP5N7lG+lRGPE86gWWRUMsht9NM7xfT6aOHsjzb+3ULnaKouQNwa7P7VnZaufeeooKLK8fxfxXq3nkn9X0ajzI/UlEuuckSdYgI4B/VtYV6aDlGBWlPVGh3oUJdhw96NQp37m3Puk2VTV1LFqzHUvgyNr0qrsEscQW+2G1290W1G6MesQSz7OvXhpFUfIJvw0uKrCYdsJAZk8emWDfWhO/Pr60mIglXthhY8yOSU8m0qHJPxKxABPnL0mgW8TigVljNcFfUTKMCvUuTFjH0TBvjnuTGNCnO5cvXE1DNNYikS7EO2giTic7fxymG4LjivIlV5+RcGNSL42iKPlEMFcomUhvTfy6P7zxVy+9R1FIuEuhJcSMYdQRvVi3s6k6y8WfLWXb7v28tO4DYkCBZddKj9PtYr+7d/lGjLF/a+6QorQ/KtQVROxmQldNGZnQoMJ/k4jGDDETY/BHTSL9wmZEemFEELG717nemYaowZKmGrtzyjdwsNH+cO6ySpZcfYaKckVR8pKgd7y5mcKwCl0uzTUUKivpxdTRA/ntkrU8GBDpRQUWk44u4eUNNWyo2Re33d/fq6HSt6wxZpd3fHfnJ17lrlgMFq3ZHjc2zR1SlPZHhXqeks5UaUV1LYLQEI3Ro9CipHdRwrr+m0S3iBUn0i+e+TPeKylNmnlUGLFLdb2y4SMOmmjcZzFj19gNtraOxowae0VR8pJk3vFU9s7vdTeYpI6NZPz1jY08+NStcSK9wLIdKIMP65Fgg4E4ke6ybscnce8bY4YJpcWaO6QoGUaFeh6S7lRpc7XWg+sM/nAbCx63E0dnzriDqv5lnHfiQP60eltC7pGdlGT49GCUxljUC68pjAgCXHjqMB5/YwsNgd7TEUvU2CuKkpcEvePpOCX8XveaTw5w19IN3meuY8Pdtz/XZ3xpMWWHCF/5waUMCMSkG2OIGVjw2uY4kR7sdZGKogK7bq7mDilKZsmYUBeRh4BpwC5jzPHOsmLgSaAUqAa+aYzZ7Xx2M/BdIApcbYx50Vk+FngY6AEsAa4xxhgRKQLmA2OBj4DzjTHVzjYXAz90hvJTY8wjmTrPTNLaBhhhU6UlvYtC9zNj/JFs272fC04ZGnoM9ybxziur+fwVt7Av2sCsmT9jQ79S/nvKSO4q3xBaICBmIBqFFZUfAWCJoVvE4sJTh1LcsxujB/fliYotnpfoggn28uamchVFUXKVdJwjfvz3gPPHD7W7Ry9r6hxtDLzxfi23LnobQeL6URzSUM+Kv/+SQW+t4trp3+e5Y5oSR6MGTKMh5rPeY4YeyvB+PXl69bZmzyMiTU4VDVNUlMwiJtj6sb12LHImUAfM9wn1/wfUGmPuFJGbgMOMMTeKyLHA48AEYBDwN+BoY0xURN4ArgFewxbqc4wxz4vIFcAJxpj/EpEZwFeNMec7DwMrgXHYuS+rgLHuA0Eyxo0bZ1auXNn+F6KVtKUBhn9bv+H276eqpo6pc16hvsH2aHcvtEKnUKtq6njnldWcfeW3KGg4yAu/WcDst6NERIiaGH6HuACnjSjmjerdCVOpLgURodCyk1bnzRzDzr316olRlHZARFYZY8Z19jg6kmyz2+mQrgMm2T2gqqaOBa9tYv6rm2gMuL/dfhQF+/cz/+nbGLt1Hdefez3PHnMGjc0UR7cECiNN5RwBLjmtlFEDejP/1U28u3Ovl2dUGBF+N2tcQhlfRVHSJ12bbWVqAMaYvwO1gcXnAq53+xHgPN/yJ4wxB4wx7wOVwAQRGQj0Mca8auwnivmBbdx9/RE4S0QEOBt4yRhT64jzl4Bz2vv8Mo3fK24M3vRmOrhe8NumH8uVk0ciSNx+3AYbjT6V3Rg13LOskqqapsz/qpo6rvjxU4z99tfYu6eOlQ8+xdXrokRjdgvqWMDwi8BnR/Sz3TxJMDHjjWXn3nrOHz/Uu/k8WbE57viKoij5RllJL8/uuYTZv2T3gLKSXowa0NvrKO1SGBEilu1Jf+SPtzFmyzpeuvmXLDluUrMiHexZ0GAo4sP/rGb88GL+cs0Z/PcXjm56ELCslGV8FUVpPzo6Rv0IY8wOAGPMDhFxH8cHY3vMXbY6yxqc18Hl7jZbnH01isge4HD/8pBt4hCRy4DLAIYOHdr6s8oALZ0iDeJOR1bV1HlNhURgQJ/uXoMNv/FujBkWv7mD59/ayeKrTgfgicfLeXj+DRQ2NvCdC+9kwMe9iZn93jYRgUafJnef+qafNDjp9GnUkFAGsj3bZyuKkt9ks91uDcnsn1s+tzBih7T47wFujXR35rJbgXD1lKM48bACup83nTFb13H9eddz7kUzkYWrvVrqLgWW7U8JTnxaYpdrdIkBP/rzW/zkvOO9BnUFFpo4qigdSLYkk0rIMpNieWu3iV9ozP3A/WBPoTY/zI6jvRr8BPdTUV0b14l08qgSAJa/V+O1r16ydgeL//RKk0if+XPeO6KUdet3eVOtBQIPfns8a7fu4ZcvvQfYon3O0kosyxbxYdEvYU09WpNgpShK1ySb7XZraG2C6ZKrz4jvP3GIsOuMszh8i504+rfjJnHK3nrmzRzDJQ9XxG0/5Zj+nDDkUH79tw1EHZtuCZx70iD+ufFDduw54K37z6qPmHb3ChZfdbomjipKJ5Cx0JckfOCEs+D83uUs3woc6VtvCLDdWT4kZHncNiJSAPTFDrVJtq+cI2yKtK37GdCne1wn0gtOGcqNXzqGiCWe173vtk1xIv2oL5zGBROGxsedi3Bk8SG8/1F8Ga+D0RiCcOKRhyaMo6jAImIJsyePBPCmets6e6AoipKrJLN/TeVzDYLEhT+6ce5TRw/kyilHUXaIwNSplKyp4IbzruevJ0wmGjMM6NPdDlEJxMmcMORQpo4eSFGBRVGB5YTNwNOrt8WJdHAKA/jK5rbHPUlRlPTpaI/6IuBi4E7n97O+5Y+JyK+wk0mPAt5wkkk/EZFTgdeBWcDdgX29CnwDKHeqwbwI/ExEDnPW+yJwc+ZPLTfYubeeogI7YagwIixas53Zk0cyb+YYFq3ZzjcPPWDHpEftcJfKAcP5zeSRLHhtU9x+DHY30b49CuOWuw2OTjryUFZv/thb/vUxg5kwvNi7CfmneufNHMMVk0YAzTfwUBRFySeSzZ4mE/Avr9/FpfNXImLHpP/lkpMZfvE3YcUKZOFChpeM5eBL7xGx4PKFq7l12rGe1xxsG+3a2cSyj+ETFAcaYwzo0z3j10JRlEQyWZ7xcWAS0E9EtgK3Ygv0p0Tku8Bm4D8AjDFvi8hTwDqgEZhtjNch53KayjM+7/wAPAg8KiKV2J70Gc6+akXkdsCd6/uJMSb9TMw8x41tdMX64jd38Bdn+nRY7Q6uf/QGpBt8+pcXuaDnIO/msG33/rj9CHazDUEojNhd6iyxa6TPmzmGI4sP4bE3NhONGc+L7t6AnqzY7IXfFEaES+evIuI04Jg6emBHXg5FUZROIVj9Jeig8AvpAX26U1Fdy5baT7l0/ioORg1gODR2kJ5fOxfWVMDChbx88hR++Xv71tcYA4nGeKO61rP3FnDRqcMSjjV6cN+E5FQ/hRHR5FFF6SQyJtSNMd9K8tFZSda/A7gjZPlK4PiQ5fU4Qj/ks4eAh9IebBeirKQX82aO4a6lG3hr215PLA+t3c7DC26iW6yBl+Y8xTmTTuV8mhKdXI9MREAsYZbTsGh/Q5TCiGBZ0BA19CiMsHNvPRNH9WfJ1WeExjP6w28aoobCCOxviGl8uqIoXYJ0E+jdZe66DbGYV1Srx8F67v/jbZRsW8c/f3IXA86axqJllXHbx2KG6ScO8pwxMeCxNzZz4anD4vZrsMNrImJCc4sEDUlUlM6io2PUlU6mqqaOyxeu5u3tezkYjVFUYDH84x0sXHgTRbEGLrnwTnqMPdmLH3cTnQ402uued/JgHpw1juKe3TAYuzudsb3pwSnaZPGMbvgNQLeIhaDx6YqidB1aUn53ydodHGyMsr8hSmPUEI0Zehy0SzCO27aO6869nu/Wj2Ta3SsYfnjPuG1/9tXRTBzVnysnj/RKK7rx7v4xRGOGmDFeVa7TRxxOYUToFrHoFrF4YNY4daAoSieRLVVflA4iKLwv7t/ADQ/9COkGL815iv8eezKXL1wdFz/uj5OcfuIg73NjIBaLYTkNjGZPHuGFrjxZsTlpZQA3/Mbd57yZY7XxkaIoXYZ0E+irauqYU74hrpSuX6S/9pO7WFI/kv0NUYoKLOaUV9ItYhE1hp+ddzznT7DLV44e3BdBKCqQuOO5Y3Ab4xU4tvwn59mT2FrhRVE6HxXqXQz/DWLY7u3c8NAtFDQchPJyzjnhBJ6s2BxXKmzttj1xiZ5+L0xhREDEK+tY0rsIoNkp3WDyFKDxj4qidBnSLb9bUV0b11jOFeljt66j5rcPMuCr30DuXkFRgUVj1M4HOhi17bFbqNidRTUYolHDT6aP9o7nH8OW2k9ZtGY7008c5H2uAl1ROh8V6l0M9wbxziurOftKR6QvXQonnMDL63ex9J0PiBnjeVnchFE30dMv9A32dKm/IUc6NYH9SVTQvLBXFEXJN8ISSIMM6NPd613h96TX/PZBjrjs24A962lXgTEcjBo7nDBQ5jEaM1553VuefZvxw5sSWN3GeO5Mqdv0Tu2womQHKtS7IGV7dlJ29QXgiPSqgWUseO5tHvpHtbfO18cMZni/ntyzbGOc6D5//NC4SgTfW7CKqK+TXXNTusEkqismjdBmR4qiKCG4+TzWp5/GhbsM+Oo34tYx4IXHRGOGB2aN9QR4zScH4my0CAl2VpvOKUr2okK9q1FZCZMmwYEDnkifdvcK6huicav9+V/beOjb40NFt+uFebJis9OQI0aBZSUI+bAp3eANAUgrVlNRFKWrMb60mF6NB5j3tB3u8v1zr+eF+pGI0ynUbWLnb0YXsexSin6niCV2iUXLqb0etLPadE5RshcV6l2JgEjnhBOocGLSYyEluXburU8puoPGfUCf7l4S6fnjh4YOIbjN1NEDvdh3TVpSFEVpouwQYenS/0fvLev4n+nX8eyoM8Dn9QZYtGY7hZbQEIuf2Qw6RWZPGUFJ76JQO5tuzLyiKB2PCvWuQohIhybh7DbEcCkssEIbcQSbdPjDYPzVYlLVBQ67IeiNQVEUxce+fTB1Kn1Wv871513Ps0efATR1fx7Qp7vX46IhZucKCXhhL0CCUySVnU0nZl5RlI5HhXpXICTcZcFzb7Nt935GD+7LGSP7MfiwHkw8uoS12/YA4UY9WZMONwwm3RhHvSEoiqKkwBHprFhBzbwHeX7rAHo4jYmunDwyrgKX27Ru9OC+XHPWUUwc1d/bjb9ilxuzrl5zRcktVKjnKa5B/mxsN0O//uU4kX7OXX/nYKM9Tfriug+8bSYeXcKVU46K28/L63d5Jbt27q0PFeNuwpLBaIyjoihKW/CJdBYu5IgZM1gcIrC31H5KNGaL9Iao4e3te/neglVcOXkkowf3jZvhnDp6YNrdUBVFyS5UqOchrkE+8qPtTHz0BqLdIFJe7sWkR8N6RGPHOvq9MS+v38XFv68A4OnV2/jF10YnJBz5jT80NT3SG4CiKEoLCYh0ZswAEmch3XKKIkLUCXtxQxfvWroBAcTX48KNZ9fKLoqSe1idPQCl/amoruXIj7bz8Pwb6BZt4KU5C+Ni0iNOK+kgEwJe8Mde3xz3/o3qWhZfdTq3TT/W88b4E5YEoaR3kRp/RVGUlpJEpIfhD3spsOzY9ALnbt4QNYgIxsQ7VQb06U5j1F4/amKs3/kJc8s3UFVT1zHnpyhKq1CPeh7y2dhuJj56A91iDVxy4Z386guf8z4rK+nFC9ecyYLXNrFt934GH9qDR17bRESEHy9e5zXCqKqpY9l7u+L263as8wtxLeulKIrSRlog0iHR7t467Th++OxbgD21KQL3XzSWnXvrGdCnO0vW7mBO+YamyjAxvL4Zc5dVsuTqM9TBoihZigr1fKOykqFf/zLRbvDSnKf41Rc+F1qK60dfOQ6AueUbsAQORmNErKbp0IrqWiJiAVEiAhefVhoXFuPfl5b1UhRFaTlVNXWsfmcrX77pP+nx+qtJRXowCbSspBfzZo5h4eubGHLYIdTUHaDAsmiIRimMCFdOHsnEUf290MSGaCyu1rqfaMxoGIyiZDEq1PMJX3WXSHk55zjhLsmoqqlj7rJKz4AbjOcRD3psLjx1WNx2wZuGGnlFUZTmce3ngD7d+Z/f/5PfPn4L3bas44PfPsgRSUR6MAkU4HsLVlHfYMelFxVYCaUYoSlEJplIB0IbICmKkj2oUM8XktRJT0VFdS12dCOeF8Zf1zzMU+7eNKIxgzGGB2aNC/W0K4qidDWaK3/oF93d6vfzwFM/YsyWddxw3vVMOHkK54fsM9i4yE0MjfoaHMWM4ZopRyU0NPI7XAyGCyYMpbhnN0YP7puyFK+iKNmDCvV8oBUiHcK7hPoJ85RXVNcSjRmvwsCl81fxwrUa36goStcmnfKHruhm3z4eePo2xjoi/fnjJzE7iVc7WR5QxBLPUx6xJFRwpwpNVAeLouQGKtRznRQivTnvTtCIAzxZsTllrPn40mKMafLkiKDxjYqidHnCPN9Buzi+tJhDGur57dO3MWbLOtb/8l4mnPElZqewucnE9pKrz2DJ2h1Aaq+4hiYqSm6jQj0LSbt7XDMiPZ3mFq4Rb8n6D8wax6XzVyGi8Y2KoiiQXgWsskOEFX//JUVb11Hz2wc59rJvc2wa+w4T22UlvRIa1CmKkn+oUM8y0u4e10y4SzrenZau73+AeOHaM7TSi6IoikOzFbCcEoxudZewxNGOJG2HkKIonYoK9SwjLYFdWUnjmRNp3F/Ph8/8hSEhMenp1jf3VyBItX7YA8T544e223kriqLkOknDTFpYJz3TpO0QUhSl01GhnmU0K7Adkb53Tx3fmXknG8o/ZvFxdS1KInIJGut5M8ewc299aLx6Sz30iqIoClkn0qHlM66KonQeKtSzjJQC2wl3adxfz3dm3sma4qH0MMmTOZtLIgoa65176zl//NBQb4t2IFUURWkhWSDSw0Jc1J4rSu6gQj0LCRXYvpj0D5/5CxvKP6aHI6RbY2Srauqo+eQABpNgrMO8LeePH6odSBVFUUIIjfduR5Hu3z+Qth1OFuKiHaUVJXdQoZ4LBBJHh5xwAouPa30iUFVNHVPnvEI0ZhCESaMOZ+Ypw7zqL8kEvJb5UhRFiSdUDB8i7SLSq2rqWLJ2B3OXVSIIBkM0FiMWg0hEeOGaM1tdJEDtuaLkBirUs50k1V3aYmSXrN3htZ4GQ/m7Nayo/Ih5M8dw+cLVuGXSZ08eoV3rFEVRUhAUw6vf2UrZLd9LKtLT9Y67DwAN0VhTYyMB5yXRRsOC1zbxo68cl3RsGuKiKLmPCvVspgUdR9tSaqsxZig0sGjN9rgbTknvIhXpiqIoKfCL4UMa6vnyTf8JTgnGMJHuet8NtuIWJLTyivsA4Ir0wogQi5m4/W3bvd/bb5j91xAXRcl9VKhnKy0U6S0ptTV19EDmLqukMWpojBkKLHtKdfqJg3j+rZ3qfVEURUkTVwyvfmcrX77pP7066WHhLn7ve2FEAGiIxhLCUoIhiAbDlZNHUtKriBv/tNbb3wWnhCf/B8W6CnRFyV1UqGcjLRDp0PJSW2UlvVhy9RkseG0T81/d5C0/svgQ9b4oiqK0kLJDxA53SSHSId777nrUCywrzjHiF96QGII4oG93Fq3ZzvQTBzFxVH+erNispRYVJY9RoZ5ttFCkQ+vjEBe8tolGZyo1YolX3UWNvKIoSpq0oLpLMBQFEmPUg46XYAjixFH9mTiqv/de49AVJb9RoZ5NtEKkQ+viECuqaxERcLw6xqiBVxRFaRGtKMEYDEUJ2uuWCm+NQ1eU/EaFerbQSpHu0tI4xPGlxUQsoajAwhjDA7PGqoFXFEVJlww1M2qN8NY4dEXJX1SoZwNtFOmtQb0wiqIorSTDHUdVeCuK4qJCvbPpBJHuojcDRVGUFtJKkd6WErqKonRdVKh3Jp0o0hVFUZQW0gaR3pISuoqiKC5WZw+gy6IiXVEUJXdoQ7iLv5KLMfZ7RVGUdFCh3hmoSFcURckd2hiTriUUFUVpLRr60tGoSFcURckd2iFxtD2S9zXGXVG6JirUOxIV6YqiKLlDO1Z3aUvyvsa4K0rXRUNfOgoV6YqiKLlDhkswtgSNcVeUrosK9Y5ARbqiKErukEUiHTTGXVG6Mhr6kmlUpCuKouQOWSbSQRvUKUpXRoV6JlGRriiKkjtkoUh30QZ1itI10dCXTKEiXVEUJXfIYpGuKErXRYV6JlCRriiKkjuoSFcUJUvJa6EuIueIyHoRqRSRmzrkoCrSFUVRcgcV6YqiZDF5K9RFJALcA3wJOBb4logcm9GDqkhXFEXJHVSkK4qS5eStUAcmAJXGmCpjzEHgCeDcjB1NRbqiKEruoCJdUZQcIJ+F+mBgi+/9VmdZ+7N9u4p0RVGUXCEahWnTVKQripL15HN5RglZZuJWELkMuMx5Wyci69t4zH6ceOKHbdxHNtIP0PPKLfL13PL1vKDt5zasvQaSzbSz3e7Ht771Id/6VjuMLKvQ/5PcQ88r9+gQmy3GmObXykFE5LPAbcaYs533NwMYY36ewWOuNMaMy9T+Ows9r9wjX88tX88L8vvcspV8veb5el6Qv+em55V7dNS55XPoSwVwlIgMF5FuwAxgUSePSVEURVEURVHSIm9DX4wxjSJyJfAiEAEeMsa83cnDUhRFURRFUZS0yFuhDmCMWQIs6cBD3t+Bx+pI9Lxyj3w9t3w9L8jvc8tW8vWa5+t5Qf6em55X7tEh55a3MeqKoiiKoiiKksvkc4y6oiiKoiiKouQsKtTbARE5R0TWi0iliNzU2eNxEZGHRGSXiLzlW1YsIi+JyAbn92G+z252zmG9iJztWz5WRNY6n80REXGWF4nIk87y10Wk1LfNxc4xNojIxe18XkeKyDIReUdE3haRa/Lh3ESku4i8ISJrnPP6cT6cl2//ERH5l4gszrPzqnbG9G8RWZlP55aviNpstdntd25qt3PsvCTXbLYxRn/a8IOdqLoRKAO6AWuAYzt7XM7YzgTGAG/5lv0/4Cbn9U3AL5zXxzpjLwKGO+cUcT57A/gsdm3654EvOcuvAH7rvJ4BPOm8LgaqnN+HOa8Pa8fzGgiMcV73Bt5zxp/T5+aMoZfzuhB4HTg118/Ld37/AzwGLM6Xv0XnGNVAv8CyvDi3fPxBbbba7PY9N7XbOXZe5JjN7nTDlOs/zpf0ou/9zcDNnT0u33hKiTf664GBzuuBwPqwcWNXy/mss867vuXfAu7zr+O8LsAu/C/+dZzP7gO+lcFzfBb4Qj6dG3AIsBo4JR/OCxgCLAWm0GTwc/68nH1Wk2j08+Lc8vEHtdmd/rdEHtpsZ99qt3PjvKrJIZutoS9tZzCwxfd+q7MsWznCGLMDwPnd31me7DwGO6+Dy+O2McY0AnuAw1Psq91xppROxvZi5Py5OdOM/wZ2AS8ZY/LivIDfADcAMd+yfDgvsDse/1VEVondNRPy59zykVy7bnn1t5RvNhvUbgfHGBhLNp5XTtnsvC7P2EFIyDLT4aNoO8nOI9X5tWabdkNEegFPA9caY/Y64WGhqyYZT9admzEmCpwkIocCz4jI8SlWz4nzEpFpwC5jzCoRmZTOJknGklXn5eNzxpjtItIfeElE3k2xbq6dWz6SL9ct5/6W8tFmg9ptd5MkY8mq83LIKZutHvW2sxU40vd+CLC9k8aSDh+IyEAA5/cuZ3my89jqvA4uj9tGRAqAvkBtin21GyJSiG3wFxpj/uQszotzAzDGfAwsB84h98/rc8B0EakGngCmiMiCPDgvAIwx253fu4BngAnkybnlKbl23fLibynfbTao3c6R88o9m91eMT9d9Qd7VqIKO8nATUw6rrPH5RtfKfHxjv8f8QkT/895fRzxCRNVNCVMVGAnx7gJE1Od5bOJT5h4ynldDLyPnSxxmPO6uB3PSYD5wG8Cy3P63IAS4FDndQ/gFWBarp9X4Bwn0RTrmPPnBfQEevte/xP7Jp3z55avP6jNVpvdvuemdjuHzosctNmdbpTy4QeYip3FvhH4384ej29cjwM7gAbsJ7nvYsdJLQU2OL+Lfev/r3MO63Gyl53l44C3nM/mgtcoqzvwB6ASO/u5zLfNJc7ySuA77Xxep2NPF70J/Nv5mZrr5wacAPzLOa+3gB85y3P6vALnOIkmg5/z54VdOWSN8/M2zv9/PpxbPv+gNlttdvudm9rtHDovctBma2dSRVEURVEURclCNEZdURRFURRFUbIQFeqKoiiKoiiKkoWoUFcURVEURVGULESFuqIoiqIoiqJkISrUFUVRFEVRFCULUaGuKICIHC4i/3Z+dorINt/7bu10jOUisl5E1ojIP0RkVJL1ficix7bHMRVFUfIRtdlKV0HLMypKABG5DagzxvzSt6zAGNPYxv0uB64zxqwUkcuAacaY6YF1IsZuR60oiqKkgdpsJZ9Rj7qiJEFEHhaRX4nIMuAXInKbiFzn+/wtESl1Xl8oIm843pz7RCTSzO7/Dox0tq0TkZ+IyOvAZx0vzjjns3NEZLXj0VnqLOspIg+JSIWI/EtEzs3A6SuKouQUarOVfESFuqKk5mjg88aY7ydbQUQ+A5wPfM4YcxIQBWY2s9+vAGud1z2xW4afYoxZ4dtvCfAA8HVjzInAfzgf/S9QbowZD0wG/j8R6dniM1MURck/1GYreUVBZw9AUbKcP6QxrXkWMBaoEBGAHsCuJOsuFJH9QDVwlbMsCjwdsu6pwN+NMe8DGGNqneVfBKb7PEXdgaHAO82ejaIoSn6jNlvJK1SoK0pq9vleNxI/C9Xd+S3AI8aYm9PY30xjzMrAsvokNxYBwpJIBNtjsz6N4ymKonQl1GYreYWGvihK+lQDYwBEZAww3Fm+FPiGiPR3PisWkWHtcLxXgYkiMtzdr7P8ReAqcVxBInJyOxxLURQl36hGbbaS46hQV5T0eRooFpF/A5cD7wEYY9YBPwT+KiJvAi8BA9t6MGNMDXAZ8CcRWQM86Xx0O1AIvCkibznvFUVRlHjUZis5j5ZnVBRFURRFUZQsRD3qiqIoiqIoipKFqFBXFEVRFEVRlCxEhbqiKIqiKIqiZCEq1BVFURRFURQlC1GhriiKoiiKoihZiAp1RVEURVEURclCVKgriqIoiqIoShaiQl1RFEVRFEVRspD/H22rCii9qPZ3AAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "%matplotlib inline\n", "from sklearn.metrics import r2_score\n", "\n", "fig, (ax1, ax2) = plt.subplots(1,2,sharey = 'row', figsize=(12, 5))\n", "y_oos_predict = results.predict(X_test)\n", "r2_test = r2_score(y_test, y_oos_predict)\n", "ax1.scatter(y_test, y_oos_predict, s = 8)\n", "ax1.set_xlabel('True Price')\n", "ax1.set_ylabel('Predicted Price')\n", "ax1.plot([0,500000], [0,500000], 'r-')\n", "ax1.axis('equal')\n", "ax1.set_ylim([0, 500000])\n", "ax1.set_xlim([0, 500000])\n", "ax1.set_title(f'Out of Sample Prediction, $R^2$ is {r2_test:0.3f}')\n", "#\n", "y_is_predict = results.predict(X_train)\n", "ax2.scatter(y_train, y_is_predict, s = 8)\n", "r2_train = r2_score(y_train, y_is_predict)\n", "ax2.set_xlabel('True Price')\n", "ax2.plot([0,500000],[0,500000],'r-')\n", "ax2.axis('equal')\n", "ax2.set_ylim([0,500000])\n", "ax2.set_xlim([0,500000])\n", "ax2.set_title(f'In Sample Prediction, $R^2$ is {r2_train:0.3f}');" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "We see that the model does a reasonable job for house values less than about \\$250,000. \n", "\n", "It tends to underestimate at both ends of the price range.\n", "\n", "Note that the $R^2$ on the (held out) test data is 0.610. \n", "\n", "We are not doing as well on test data as on training data (somewhat to be expected).\n", "\n", "For a better model, we'd want to consider more features of each house, and perhaps some additional functions such as polynomials as components of our model." ] } ], "metadata": { "celltoolbar": "Slideshow", "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.10" }, "rise": { "scroll": true, "theme": "beige", "transition": "fade" } }, "nbformat": 4, "nbformat_minor": 1 }