examples/02_model_collaborative_filtering/als_deep_dive.ipynb · shzgamelife/recommenders

加入 Gitee

与超过 1200万开发者一起发现、参与优秀开源项目，私有仓库也完全免费：）

克隆/下载

als_deep_dive.ipynb 36.18 KB

{
 "cells": [
  {
   "cell_type": "markdown",
   "source": [
    "<i>Copyright (c) Microsoft Corporation. All rights reserved.</i>\n",
    "\n",
    "<i>Licensed under the MIT License.</i>"
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "# Spark Collaborative Filtering (ALS) Deep Dive"
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "Spark MLlib provides a collaborative filtering algorithm that can be used for training a matrix factorization model, which predicts explicit or implicit ratings of users on items for recommendations.\n",
    "\n",
    "This notebook presents a deep dive into the Spark collaborative filtering algorithm."
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "## 1 Matrix factorization algorithm\n",
    "\n",
    "### 1.1 Matrix factorization for collaborative filtering problem\n",
    "\n",
    "Matrix factorization is a common technique used in recommendation tasks. Basically, a matrix factorization algorithm tries to find latent factors that represent intrinsic user and item attributes in a lower dimension. That is,\n",
    "\n",
    "$$\\hat r_{u,i} = q_{i}^{T}p_{u}$$\n",
    "\n",
    "where $\\hat r_{u,i}$ is the predicted ratings for user $u$ and item $i$, and $q_{i}^{T}$ and $p_{u}$ are latent factors for item and user, respectively. The challenge to the matrix factorization problem is to find $q_{i}^{T}$ and $p_{u}$. This is achieved by methods such as matrix decomposition. A learning approach is therefore developed to converge the decomposition results close to the observed ratings as much as possible. Furthermore, to avoid overfitting issue, the learning process is regularized. For example, a basic form of such matrix factorization algorithm is represented as below.\n",
    "\n",
    "$$\\min\\sum(r_{u,i} - q_{i}^{T}p_{u})^2 + \\lambda(||q_{i}||^2 + ||p_{u}||^2)$$\n",
    "\n",
    "where $\\lambda$ is a the regularization parameter. \n",
    "\n",
    "In case explict ratings are not available, implicit ratings which are usually derived from users' historical interactions with the items (e.g., clicks, views, purchases, etc.). To account for such implicit ratings, the original matrix factorization algorithm can be formulated as \n",
    "\n",
    "$$\\min\\sum c_{u,i}(p_{u,i} - q_{i}^{T}p_{u})^2 + \\lambda(||q_{i}||^2 + ||p_{u}||^2)$$\n",
    "\n",
    "where $c_{u,i}=1+\\alpha r_{u,i}$ and $p_{u,i}=1$ if $r_{u,i}>0$ and $p_{u,i}=0$ if $r_{u,i}=0$. $r_{u,i}$ is a numerical representation of users' preferences (e.g., number of clicks, etc.). \n",
    "\n",
    "### 1.2 Alternating Least Square (ALS)\n",
    "\n",
    "Owing to the term of $q_{i}^{T}p_{u}$ the loss function is non-convex. Gradient descent method can be applied but this will incur expensive computations. An Alternating Least Square (ALS) algorithm was therefore developed to overcome this issue. \n",
    "\n",
    "The basic idea of ALS is to learn one of $q$ and $p$ at a time for optimization while keeping the other as constant. This makes the objective at each iteration convex and solvable. The alternating between $q$ and $p$ stops when there is convergence to the optimal. It is worth noting that this iterative computation can be parallelised and/or distributed, which makes the algorithm desirable for use cases where the dataset is large and thus the user-item rating matrix is super sparse (as is typical in recommendation scenarios). A comprehensive discussion of ALS and its distributed computation can be found [here](http://stanford.edu/~rezab/classes/cme323/S15/notes/lec14.pdf)."
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "## 2 Spark Mllib implementation\n",
    "\n",
    "The matrix factorization algorithm is available as `ALS` module in [Spark `ml`](https://spark.apache.org/docs/latest/ml-collaborative-filtering.html) for DataFrame or [Spark `mllib`](https://spark.apache.org/docs/latest/mllib-collaborative-filtering.html) for RDD. \n",
    "\n",
    "* The uniqueness of ALS implementation is that it distributes the matrix factorization model training by using \"Alternating Least Square\" method. \n",
    "* In the training method, there are parameters that can be selected to control the model performance.\n",
    "* Both explicit and implicit ratings are supported by Spark ALS model."
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "## 3 Spark ALS based MovieLens recommender\n",
    "\n",
    "In the following code, the MovieLens-100K dataset is used to illustrate the ALS algorithm in Spark."
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "**Note**: This notebook requires a PySpark environment to run properly. Please follow the steps in [SETUP.md](https://github.com/Microsoft/Recommenders/blob/master/SETUP.md#dependencies-setup) to install the PySpark environment."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "source": [
    "# set the environment path to find Recommenders\n",
    "import sys\n",
    "import pandas as pd\n",
    "from matplotlib import pyplot as plt\n",
    "import numpy as np\n",
    "import seaborn as sns\n",
    "import sys\n",
    "import pandas as pd\n",
    "\n",
    "import pyspark\n",
    "from pyspark.sql import SparkSession\n",
    "from pyspark.ml.recommendation import ALS\n",
    "import pyspark.sql.functions as F\n",
    "from pyspark.sql.functions import col\n",
    "from pyspark.ml.tuning import CrossValidator\n",
    "from pyspark.sql.types import StructType, StructField\n",
    "from pyspark.sql.types import FloatType, IntegerType, LongType\n",
    "\n",
    "from recommenders.datasets import movielens\n",
    "from recommenders.utils.spark_utils import start_or_get_spark\n",
    "from recommenders.evaluation.spark_evaluation import SparkRankingEvaluation, SparkRatingEvaluation\n",
    "from recommenders.tuning.parameter_sweep import generate_param_grid\n",
    "from recommenders.datasets.spark_splitters import spark_random_split\n",
    "\n",
    "print(\"System version: {}\".format(sys.version))\n",
    "print(\"Pandas version: {}\".format(pd.__version__))\n",
    "print(\"PySpark version: {}\".format(pyspark.__version__))"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "System version: 3.6.9 (default, Jan 26 2021, 15:33:00) \n",
      "[GCC 8.4.0]\n",
      "Pandas version: 1.1.5\n",
      "PySpark version: 2.4.8\n"
     ]
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "Data column names"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "source": [
    "MOVIELENS_DATA_SIZE = \"100k\"\n",
    "\n",
    "COL_USER = \"UserId\"\n",
    "COL_ITEM = \"MovieId\"\n",
    "COL_RATING = \"Rating\"\n",
    "COL_PREDICTION = \"prediction\"\n",
    "COL_TIMESTAMP = \"Timestamp\""
   ],
   "outputs": [],
   "metadata": {
    "tags": [
     "parameters"
    ]
   }
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "source": [
    "schema = StructType(\n",
    "    (\n",
    "        StructField(COL_USER, IntegerType()),\n",
    "        StructField(COL_ITEM, IntegerType()),\n",
    "        StructField(COL_RATING, FloatType()),\n",
    "        StructField(COL_TIMESTAMP, LongType()),\n",
    "    )\n",
    ")"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "Model hyper parameters - these parameters are selected with reference to the benchmarking results [here](http://mymedialite.net/examples/datasets.html)."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "source": [
    "RANK = 10\n",
    "MAX_ITER = 15\n",
    "REG_PARAM = 0.05"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "Number of recommended items"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "source": [
    "K = 10"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "Initialize a Spark session."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "source": [
    "spark = start_or_get_spark(\"ALS Deep Dive\", memory=\"16g\")"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### 3.1 Load and prepare data"
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "Data is read from csv into a Spark DataFrame."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "source": [
    "dfs = movielens.load_spark_df(spark=spark, size=MOVIELENS_DATA_SIZE, schema=schema)"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stderr",
     "text": [
      "100%|██████████| 4.81k/4.81k [00:00<00:00, 20.5kKB/s]\n"
     ]
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "source": [
    "dfs.show(5)"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "+------+-------+------+---------+\n",
      "|UserId|MovieId|Rating|Timestamp|\n",
      "+------+-------+------+---------+\n",
      "|   196|    242|   3.0|881250949|\n",
      "|   186|    302|   3.0|891717742|\n",
      "|    22|    377|   1.0|878887116|\n",
      "|   244|     51|   2.0|880606923|\n",
      "|   166|    346|   1.0|886397596|\n",
      "+------+-------+------+---------+\n",
      "only showing top 5 rows\n",
      "\n"
     ]
    }
   ],
   "metadata": {
    "scrolled": true
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "Data is then randomly split by 80-20 ratio for training and testing."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "source": [
    "dfs_train, dfs_test = spark_random_split(dfs, ratio=0.75, seed=42)"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### 3.2 Train a movielens model "
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "It is worth noting that Spark ALS model allows dropping cold users to favor a robust evaluation with the testing data. In case there are cold users, Spark ALS implementation allows users to drop cold users in order to make sure evaluations on the prediction results are sound."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "source": [
    "als = ALS(\n",
    "    maxIter=MAX_ITER, \n",
    "    rank=RANK,\n",
    "    regParam=REG_PARAM, \n",
    "    userCol=COL_USER, \n",
    "    itemCol=COL_ITEM, \n",
    "    ratingCol=COL_RATING, \n",
    "    coldStartStrategy=\"drop\"\n",
    ")\n",
    "\n",
    "model = als.fit(dfs_train)"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### 3.3 Prediction with the model\n",
    "\n",
    "The trained model can be used to predict ratings with a given test data."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "source": [
    "dfs_pred = model.transform(dfs_test).drop(COL_RATING)"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "With the prediction results, the model performance can be evaluated."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "source": [
    "evaluations = SparkRatingEvaluation(\n",
    "    dfs_test, \n",
    "    dfs_pred,\n",
    "    col_user=COL_USER,\n",
    "    col_item=COL_ITEM,\n",
    "    col_rating=COL_RATING,\n",
    "    col_prediction=COL_PREDICTION\n",
    ")\n",
    "\n",
    "print(\n",
    "    \"RMSE score = {}\".format(evaluations.rmse()),\n",
    "    \"MAE score = {}\".format(evaluations.mae()),\n",
    "    \"R2 score = {}\".format(evaluations.rsquared()),\n",
    "    \"Explained variance score = {}\".format(evaluations.exp_var()),\n",
    "    sep=\"\\n\"\n",
    ")"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "RMSE score = 0.9726930349322086\n",
      "MAE score = 0.7565710909806911\n",
      "R2 score = 0.24411065820407096\n",
      "Explained variance score = 0.249700271662727\n"
     ]
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "Oftentimes ranking metrics are also of interest to data scientists. Note usually ranking metrics apply to the scenario of recommending a list of items. In our case, the recommended items should be different from those that have been rated by the users. "
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "source": [
    "# Get the cross join of all user-item pairs and score them.\n",
    "users = dfs_train.select(COL_USER).distinct()\n",
    "items = dfs_train.select(COL_ITEM).distinct()\n",
    "user_item = users.crossJoin(items)\n",
    "dfs_pred = model.transform(user_item)\n",
    "\n",
    "# Remove seen items.\n",
    "dfs_pred_exclude_train = dfs_pred.alias(\"pred\").join(\n",
    "    dfs_train.alias(\"train\"),\n",
    "    (dfs_pred[COL_USER] == dfs_train[COL_USER]) & (dfs_pred[COL_ITEM] == dfs_train[COL_ITEM]),\n",
    "    how='outer'\n",
    ")\n",
    "\n",
    "dfs_pred_final = dfs_pred_exclude_train.filter(dfs_pred_exclude_train[\"train.Rating\"].isNull()) \\\n",
    "    .select('pred.' + COL_USER, 'pred.' + COL_ITEM, 'pred.' + \"prediction\")\n",
    "\n",
    "dfs_pred_final.show()"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "+------+-------+----------+\n",
      "|UserId|MovieId|prediction|\n",
      "+------+-------+----------+\n",
      "|     1|    587| 3.2763875|\n",
      "|     1|    869|  1.996331|\n",
      "|     1|   1208| 3.0924819|\n",
      "|     1|   1677| 3.0549564|\n",
      "|     2|     80| 2.2266486|\n",
      "|     2|    303| 3.5071766|\n",
      "|     2|    472| 2.4076686|\n",
      "|     2|    582|  4.137449|\n",
      "|     2|    838| 1.6214753|\n",
      "|     2|    975| 2.7880914|\n",
      "|     2|   1260|  3.155648|\n",
      "|     2|   1325| 1.2494813|\n",
      "|     2|   1381|  3.712147|\n",
      "|     2|   1530|   2.04168|\n",
      "|     3|     22| 2.5458775|\n",
      "|     3|     57| 1.7472819|\n",
      "|     3|     89|   3.85607|\n",
      "|     3|    367| 3.2235723|\n",
      "|     3|   1091| 1.5452085|\n",
      "|     3|   1167| 3.5050836|\n",
      "+------+-------+----------+\n",
      "only showing top 20 rows\n",
      "\n"
     ]
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "source": [
    "evaluations = SparkRankingEvaluation(\n",
    "    dfs_test, \n",
    "    dfs_pred_final,\n",
    "    col_user=COL_USER,\n",
    "    col_item=COL_ITEM,\n",
    "    col_rating=COL_RATING,\n",
    "    col_prediction=COL_PREDICTION,\n",
    "    k=K\n",
    ")\n",
    "\n",
    "print(\n",
    "    \"Precision@k = {}\".format(evaluations.precision_at_k()),\n",
    "    \"Recall@k = {}\".format(evaluations.recall_at_k()),\n",
    "    \"NDCG@k = {}\".format(evaluations.ndcg_at_k()),\n",
    "    \"Mean average precision = {}\".format(evaluations.map_at_k()),\n",
    "    sep=\"\\n\"\n",
    ")"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "Precision@k = 0.03170731707317073\n",
      "Recall@k = 0.012679519170565132\n",
      "NDCG@k = 0.02914424248125332\n",
      "Mean average precision = 0.0033674440032626088\n"
     ]
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### 3.4 Fine tune the model\n",
    "\n",
    "Prediction performance of a Spark ALS model is often affected by the parameters\n",
    "\n",
    "|Parameter|Description|Default value|Notes|\n",
    "|-------------|-----------------|------------------|-----------------|\n",
    "|`rank`|Number of latent factors|10|The larger the more intrinsic factors considered in the factorization modeling.|\n",
    "|`regParam`|Regularization parameter|1.0|The value needs to be selected empirically to avoid overfitting.|\n",
    "|`maxIters`|Maximum number of iterations|10|The more iterations the better the model converges to the optimal point.|\n",
    "\n",
    "It is always a good practice to start model building with default parameter values and then sweep the parameter in a range to find the optimal combination of parameters. The following parameter set is used for training ALS models for comparison study purposes."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "source": [
    "param_dict = {\n",
    "    \"rank\": [10, 15, 20],\n",
    "    \"regParam\": [0.001, 0.1, 1.0]\n",
    "}"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "Generate a dictionary for each parameter combination which can then be fed into model training."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "source": [
    "param_grid = generate_param_grid(param_dict)"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "Train models with parameters specified in the parameter grid. Evaluate the model with, for example, the RMSE metric, and then record the metrics for visualization."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "source": [
    "rmse_score = []\n",
    "\n",
    "for g in param_grid:\n",
    "    als = ALS(        \n",
    "        userCol=COL_USER, \n",
    "        itemCol=COL_ITEM, \n",
    "        ratingCol=COL_RATING, \n",
    "        coldStartStrategy=\"drop\",\n",
    "        **g\n",
    "    )\n",
    "    \n",
    "    model = als.fit(dfs_train)\n",
    "    \n",
    "    dfs_pred = model.transform(dfs_test).drop(COL_RATING)\n",
    "    \n",
    "    evaluations = SparkRatingEvaluation(\n",
    "        dfs_test, \n",
    "        dfs_pred,\n",
    "        col_user=COL_USER,\n",
    "        col_item=COL_ITEM,\n",
    "        col_rating=COL_RATING,\n",
    "        col_prediction=COL_PREDICTION\n",
    "    )\n",
    "\n",
    "    rmse_score.append(evaluations.rmse())\n",
    "\n",
    "rmse_score = [float('%.4f' % x) for x in rmse_score]\n",
    "rmse_score_array = np.reshape(rmse_score, (len(param_dict[\"rank\"]), len(param_dict[\"regParam\"]))) "
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "source": [
    "rmse_df = pd.DataFrame(data=rmse_score_array, index=pd.Index(param_dict[\"rank\"], name=\"rank\"), \n",
    "                       columns=pd.Index(param_dict[\"regParam\"], name=\"reg. parameter\"))"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "source": [
    "fig, ax = plt.subplots()\n",
    "sns.heatmap(rmse_df, cbar=False, annot=True, fmt=\".4g\")"
   ],
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": [
       "<AxesSubplot:xlabel='reg. parameter', ylabel='rank'>"
      ]
     },
     "metadata": {},
     "execution_count": 19
    },
    {
     "output_type": "display_data",
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAEGCAYAAABmXi5tAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAitElEQVR4nO3deXxU1d3H8c9vMlkhLAnIjmjBRxDRyiKIIFoFRFGkAloQcK2i4AIIWhXQtooIVn1wwaosPoJg2cSFpYWKCBIEAVksyKKBgLILJCSZnOePGQKBJKTKZEju9/16zYuZe86993dzX/nOzblnBnPOISIipZ8v0gWIiEjxUOCLiHiEAl9ExCMU+CIiHqHAFxHxCH+kCyjI3i5tNH2ohKo8Y0OkS5BfYf/gVpEuQX6FMk9PsoLadIUvIuIRCnwREY9Q4IuIeIQCX0TEIxT4IiIeocAXEfEIBb6IiEco8EVEPEKBLyLiEQp8ERGPUOCLiHiEAl9ExCMU+CIiHqHAFxHxCAW+iIhHKPBFRDxCgS8i4hEKfBERj1Dgi4h4hAJfRMQjFPgiIh6hwBcR8QgFvoiIRyjwRUQ8QoEvIuIRCnwREY9Q4IuIeIQCX0TEIxT4IiIeocAXEfEIBb6IiEf4I11AaZBw36NEN26B27+PA/1vP6k9uklL4m65A5yDQIDDY/+XwPrVRNWpS8LdD2PxCbicHDKmvkvWF/MBKPv0y1h8AgC+chXI3rieQyOeKNbjKs3atW3DqFFPE+Xz8fY7E3l+xOg87bVr1+DvY0ZRqXISe/fso2fvfmzblsZFF13A6FeeJbFcWQKBAM8+9wpTpswE4Mo2LRk+/EliYqJZvnw1d9/Tn0AgAMAVrVswcuQwoqP97N61h6uuvrnYj7k0iun0R/znXYI7dID00QNPao86vzExV3XFOQc5ATI/GU/O99/iO6cBMe175vbzVarOkSkvE1i/DN+5DYlp2x3MIDODI9New+3ZWZyHFTbmnIt0Dfna26XNmVlYPvz1G+Ey0inzwOP5Bj5x8ZCRDkBU7XMp88hQDjzUE1+1muAcOTu2YRWTKTd8DAce6oU7fDDP6mX6DyMrZRGZn80phqP59SrP2BDpEgrl8/lYt2Yh7TvcSmpqGksWf0yP2/qwbt2xuidNfIOPPp7HhAlTuLJNS3r16kbv2/tRr965OOfYuHEz1apVYemST2jYqA0HDvzMpo1Ladu+Gxs2bGLokAFs3ZrKO2MnUb58ORZ+NoPrru/ODz9sp3LlZH76aXcEfwKF2z+4VaRLKDLf2edDZgaxne/PN/CJiYXMIwBYldrEdX2Q9Ff65+0TX4aEB1/i8Mg+kJVJfL8XyXhvBG7XdvxNr8FXsy6Z014rhqM5Pco8PckKatOQzmmQvW4V7uDPBXcIhT0AcXHBK30gJy2VnB3bAHB7d5Ozfy9WrnzedeMT8De8hMyUz0932Z7VrOlv+e67LWze/D1ZWVlMnjyDGzq2y9Onfv16zJ+/CID5CxZxQ8e2AGzYsImNGzcDkJa2kx9/2k3lyskkJ1ckMzOTDRs2ATBv3md0vqkDALfechPTp3/CDz9sBzijw76kydm6Hpd+qOAOobAHsJjYfLv4GzQnsOFryMoMLXFYXPCva4tLwP289zRVG3kK/GIS3exyyv1tPGUfe45Drw0/qT2q7vmYP5qcndvzLI9pejnZ3yyH9MPFVWqpV71GVX5IPfZzTt2WRvXqVfP0WbVqLTd1uhaATp2upVy5RJKSKubp07TJxcTERPPdd1vYtWsPfr+fxpc0AqBz5+uoWas6APXqnUuFCuX559wpfLnkE3r00HBOcYqq35T4viOJ6z6II9NfP6ndf2ELsld/kfv6yIwxxPUYRHz/0fgvakXWwhnFWW5YKfCLSdbSzznwUE8OPf8E8d3uzNNmFZIo0/dxDr06PPfq/6iYy39H5uf/LM5SBXh00DO0bt2clKWzad2qOampabnj8QBVq57F2LEvc9ddj3B0WLR7jz6MfGEoixfN4uDBQwQCOQD4/VE0vqQRHW/sSYfr/sCfHnuIevXOjchxeVFgXQrpr/QnY+ILxFzVNU+bla2Ar0ptAhtX5i6LbtGBjHeHkz7yfrJXLCCm/W3FXXLYhCXwzay8mT1nZuvNbI+Z7TazdaFlFQpZ7x4zW2Zmy8Zu2l5QtxIte90qfFWqYYmhoZv4BMo+9hzpE98isGFtnr6WWJ6ouueTtXxJBCotvbZv20GtmtVzX9esUY3t23fk6ZOWtpMuXe+mabN2PPlU8C+y/fsPAJCYWJaZM8bz5FPD+XLp8tx1lnz5FW2u6kyLltezcOGS3OGdbdvSmDN3AYcPp7N7914Wfr6ERo0ahPsw5QQ5W9djFc+ChMTcZVENW5C9LgVyQm/mCYn4qp5NTupGALK/WUxUrfMiUW5YhOsKfzKwF2jjnEtyziUDV4aWTS5oJefcGOdcE+dck97nVi+oW4njq1oj93nUOfWw6Gjcz/vB76fswGfI/Pccspb8+6T1optfQdZXi48bW5TTIWXZ19Stew516tQiOjqarl1v5MNZeW+IJydXxCx472vwoL6MHTcJgOjoaP4x5S3effcDpk79KM86lSsnAxATE8PAAfczZswEAGZ+OJuWlzUjKiqK+Pg4mjX7LevXn9k3tksLS6qS+9xXrQ74o+Hwsftt/gsvI3v1omMrZBzCYuOx5GoARP2mETk/bSuucsMuXNMy6zjn8gxUO+d2AMPN7I4w7TNiyjz4JP4LLsYSy1P+9SmkT34HooI/2sy5M4m+tDWxV7TFBQKQeYSDLz4NQEyLK/HXvwhLLE/Mle0BODz6OQJbglcXMS2vImP6e5E5qFIsEAjw4ENP8PFH7xHl8zF23PusXfsfhg4ZwLKvVjJr1lyuuOIy/vLMYzgcCxcuoW+/PwHQpUtHWrW6lKTkivTsGRweuPOuh1m5cg0DHrmPDtddjc/n4403xjN/QTBI1q/fyOw581mxfB45OTm8/fZE1qz5NmLHX5rE3twX3zkNsIRE4vuPJmv+B+CLAiB72Tz8DS7Ff3Gr4O9ediZHJr+Uu65VqIyVTyZny7pjG8zJ4cjMN4m75eHgUF36oXzH/UuqsEzLNLM5wDxgnHNuZ2hZFaA3cI1z7upTbaMkTcuUvM70aZlSuJI0LVNOFolpmd2AZODfoTH8PcACIAnoEqZ9iohIIcIypOOc2wsMCj3yMLPbgXfCsV8RESlYJKZlDovAPkVEPC8sV/hmtqqgJqBKAW0iIhJG4ZqlUwVoR3Aa5vEM+OLk7iIiEm7hCvxZQFnn3NcnNpjZgjDtU0REChGum7Z3FtL2h3DsU0RECqfv0hER8QgFvoiIRyjwRUQ8QoEvIuIRCnwREY9Q4IuIeIQCX0TEIxT4IiIeocAXEfEIBb6IiEco8EVEPEKBLyLiEQp8ERGPUOCLiHiEAl9ExCMU+CIiHqHAFxHxCAW+iIhHKPBFRDxCgS8i4hEKfBERj1Dgi4h4hAJfRMQjFPgiIh7hj3QBBSk7+s1IlyC/1Iw2ka5ARPKhK3wREY9Q4IuIeIQCX0TEIxT4IiIeocAXEfEIBb6IiEco8EVEPEKBLyLiEQp8ERGPUOCLiHiEAl9ExCMU+CIiHqHAFxHxCAW+iIhHKPBFRDxCgS8i4hEKfBERj1Dgi4h4hAJfRMQjFPgiIh6hwBcR8QgFvoiIRyjwRUQ8QoEvIuIRCnwREY9Q4IuIeIQCX0TEI4oU+GYWm8+ypNNfjoiIhEtRr/Cnmln00RdmVg2YG56SREQkHIoa+NOByWYWZWZ1gNnAY+EqSkRETj9/UTo55940sxiCwV8H+KNz7osw1iUiIqdZoYFvZo8c/xKoDXwNNDez5s65UWGsTURETqNTXeEnnvB6agHLPe2JZ//GZ1+kkFSxPNPHv3pS+9IVq+j32J+pUa0KAFe3voz7br81tz0QCNDt7oc5q1Iyrz4/BIBBT49gzfqN+P1RNKx/HkMGPkC0v0h/kEkRtGvbhlGjnibK5+Ptdyby/IjRedpr167B38eMolLlJPbu2UfP3v3Yti2N2rVr8MGUt/D5fERH+xk9+h3GvDkBgEt+eyFvvfUi8XFxfPLpv3j4kacAGP7sE1x3/TVkZmayadNW7rzrEfbvP1Dsx1waxXT6I/7zLsEdOkD66IEntUed35iYq7rinIOcAJmfjCfn+2/xndOAmPY9c/v5KlXnyJSXCaxfhu/chsS07Q5mkJnBkWmv4fbsLM7DChtzzkW6hnxl/bjhzCwsH8u+/oaE+Dge/8uoAgN/7MRpuWF+onGTprHm240cPHQ4t89ni1No1bwJAI8OG0Hjixpyy00dwncQp1F8zTaRLqFQPp+PdWsW0r7DraSmprFk8cf0uK0P69ZtyO0zaeIbfPTxPCZMmMKVbVrSq1c3et/ej+joaMyMzMxMypRJYOWKf9HqihtJS9vJ4kWzeOjhp/hy6XJmzZzA/45+m09nz+eaq1vzr/mLCAQCPPvXxwF47PG/RurwT2n/4FaRLqHIfGefD5kZxHa+P9/AJyYWMo8AYFVqE9f1QdJf6Z+3T3wZEh58icMj+0BWJvH9XiTjvRG4XdvxN70GX826ZE57rRiO5vQo8/QkK6itqNMyzzOzMWY2x8z+dfRx+kos2Zpc3JDy5X7ZHz07ftzFZ4tT+P31bfMsb92iKWaGmXFh/fPY+dOu01GqAM2a/pbvvtvC5s3fk5WVxeTJM7ihY7s8ferXr8f8+YsAmL9gETd0DJ6frKwsMjMzAYiNjcXnC/4KVa16FonlEvly6XIAJvzfB9xwQ3sA5s77jEAgAMCSL5dTo0a18B+kR+RsXY9LP1Rwh1DYA1jMSbPLAfA3aE5gw9eQlRla4rC4hOA6cQm4n/eepmojr6izdKYAK4AngIHHPaSIVq5ZT+feD3DvgCFs3Lw1d/nwl8fwSJ87MF/+b8pZ2dl8OHs+l196SXGVWupVr1GVH1K3575O3ZZG9epV8/RZtWotN3W6FoBOna6lXLlEkpIqAlCzZnWWfzWXLZtSGPHCaNLSdlKjelW2pablrr8tNY0aJ2wT4Pbet/Dp7PnhOCwpQFT9psT3HUlc90Ecmf76Se3+C1uQvfrYHJQjM8YQ12MQ8f1H47+oFVkLZxRnuWFV1MDPds695pxb6pz76ujjdBdjZveY2TIzW/b38ZNO9+YjpsF5dZk75W2mjv1f/vD76+n3+J8BWLBoKUkVK3DB/9QtcN0/j3yVxhdfQOOLGhZXuQI8OugZWrduTsrS2bRu1ZzU1LTcq/TU1O1c0vga/qd+S3re1oWzzqpUpG0+Nrgf2dnZvPfe1FN3ltMmsC6F9Ff6kzHxBWKu6pqnzcpWwFelNoGNK3OXRbfoQMa7w0kfeT/ZKxYQ0/624i45bIoa+B+aWR8zq2ZmSUcfBXU2s/bHPS9vZm+Z2Soze8/MqhS0nnNujHOuiXOuyV09b/kvDuPMVrZMAgkJ8UBwqCY7O8DefftZsXotCxZ9SdsudzBw6PMsXb6KQU+/kLveq++8x959B3j0gbsiVXqptH3bDmrVrJ77umaNamzfviNPn7S0nXTpejdNm7XjyaeGA5x0ozUtbSffrPmWyy+/lG3bd1Cj5rGhmho1q7HtuG32vK0r13W4mtt6PhCOQ5IiyNm6Hqt4FiQcG36NatiC7HUpkBN8MychEV/Vs8lJ3QhA9jeLiap1XiTKDYuiBn4vgkM4XwBfhR7LCul//B2pkUAa0BFIAd7478ss2Xbt3svRm+Or135LTo6jQvlyPHxvb/45dRxzprzNiKGP0uySRgx/agAAH3w4m0VLl/P80IG548RyeqQs+5q6dc+hTp1aREdH07XrjXw4a06ePsnJFTELDrMNHtSXseOCf3HWqFGNuLg4ACpUKE/Lls34z3++Y8eOH/n5wM9c2iw49HZb95v58MPZQHBG0IAB99Gpc2/S0zOK6zAFsKRj15e+anXAHw2Hf85d5r/wMrJXLzq2QsYhLDYeSw6+eUf9phE5P20rrnLDrqgfvDrnV+yjiXPu4tDzF82s16/Y1hlp4NDnSVmxmn37D/C7zr3oc0d3srOzAejWqQNzFnzO+9M/ISrKR1xsLCOGPpobJgV5ZuRoqlU5i+73Bt8ATpzKKb9cIBDgwYee4OOP3iPK52PsuPdZu/Y/DB0ygGVfrWTWrLlcccVl/OWZx3A4Fi5cQt9+fwKg/vl1ef75p3AuOGtv1KjX+eab9QA80Pfx3GmZn86ezyefBuc1vPS3PxMbG8unnwTfNL78cjn3PzA4MgdfysTe3BffOQ2whETi+48ma/4H4IsCIHvZPPwNLsV/cStcIADZmRyZ/FLuulahMlY+mZwt645tMCeHIzPfJO6Wh4MXaemH8h33L6mKPC3TzBoCDYC4o8ucc+ML6JsKjCL4Ya37gd+40I7MbJVzrtGp9leSpmVKXmf6tEwpXEmaliknK2xaZpGu8M1sCNCGYOB/DFwLfA7kG/jAmxz7cNY4oBLwk5lVJfhJXRERKWZF/ejmzcBFwArn3O2hG6/vFtTZOTesgOU7zExz0kREIqCodwMznHM5QLaZlQN+BGr9wn3m+2YgIiLhdcorfAveXVxlZhUIDtV8BRwEFheyzqqCmoACp2WKiEj4nDLwnXPOzJo55/YBr5vZp0A551xBoQ7BUG8HnPiZZCM4tVNERIpZUcfwl5tZU+dcinNuSxH6zwLKOue+PrHBzBYUvTwRETldihr4lwLdzWwrcIjglboraHqlc+7OgjbknPvDf12liIj8akUN/Han7iIiImeyon7Sduupe4mIyJlMX9IiIuIRCnwREY9Q4IuIeIQCX0TEIxT4IiIeocAXEfEIBb6IiEco8EVEPEKBLyLiEQp8ERGPUOCLiHiEAl9ExCMU+CIiHqHAFxHxCAW+iIhHKPBFRDxCgS8i4hEKfBERj1Dgi4h4hAJfRMQjFPgiIh6hwBcR8Qh/pAsokC8q0hWIiJQqusIXEfEIBb6IiEco8EVEPEKBLyLiEQp8ERGPUOCLiHiEAl9ExCMU+CIiHqHAFxHxCAW+iIhHKPBFRDxCgS8i4hEKfBERj1Dgi4h4hAJfRMQjFPgiIh6hwBcR8QgFvoiIRyjwRUQ8QoEvIuIRCnwREY9Q4IuIeIQCX0TEIxT4IiIeocAXEfEIBb6IiEco8EVEPEKBLyLiEQp8ERGPUOCLiHiEAl9ExCP8kS6gNHjir6P4bNFSkipWYPq7r5/UvnT5KvoNHkaNalUBuPqKy7jvju657YFAgG539uOsypV4dcQwAJxzvDxmHHPmf47P56PbTdfRo8uNxXNAHtCubRtGjXqaKJ+Pt9+ZyPMjRudpr127Bn8fM4pKlZPYu2cfPXv3Y9u2NC666AJGv/IsieXKEggEePa5V5gyZSYAV7ZpyfDhTxITE83y5au5+57+BAIB+j9yL7fe2hkAvz+K+ufXo2r1Ruzdu6+4D7vUien0R/znXYI7dID00QNPao86vzExV3XFOQc5ATI/GU/O99/iO6cBMe175vbzVarOkSkvE1i/DN+5DYlp2x3MIDODI9New+3ZWZyHFTbmnIt0DfnK2rXpzCwsH8u+Xk1CfDyPP/NCgYE/duI/csP8ROMmTWXN+g0cPHQ4t8+0j+awdPkq/vKnR/D5fOzeu4/kihXCeRinTXz1VpEuoVA+n491axbSvsOtpKamsWTxx/S4rQ/r1m3I7TNp4ht89PE8JkyYwpVtWtKrVzd6396PevXOxTnHxo2bqVatCkuXfELDRm04cOBnNm1cStv23diwYRNDhwxg69ZU3hk7Kc++r7/uGh7sdzfXtOta3IddZPsHn9nn73i+s8+HzAxiO9+fb+ATEwuZRwCwKrWJ6/og6a/0z9snvgwJD77E4ZF9ICuT+H4vkvHeCNyu7fibXoOvZl0yp71WDEdzepR5epIV1KYhndOgycUXUr5c4i9ad8ePP/HZF0v5fcd2eZa/P+0j7rv9D/h8wVNUUsK+JGjW9Ld8990WNm/+nqysLCZPnsENJ/z869evx/z5iwCYv2ARN3RsC8CGDZvYuHEzAGlpO/nxp91UrpxMcnJFMjMz2bBhEwDz5n1G55s6nLTvbt1uZNL708N4dN6Ss3U9Lv1QwR1CYQ9gMbH5dvE3aE5gw9eQlRla4rC4hOA6cQm4n/eepmojT4FfTFZ+s47Ovfpwb/8n2bhpa+7y4S+9wSN97sQs76n4YVsan/zz33S9ox/39n+SrT9sK+6SS63qNaryQ+r23Nep29KoXr1qnj6rVq3lpk7XAtCp07WUK5dIUlLFPH2aNrmYmJhovvtuC7t27cHv99P4kkYAdO58HTVrVc/TPz4+jnZt2zB12sfhOCwpQFT9psT3HUlc90EcmX7yX+D+C1uQvfqL3NdHZowhrscg4vuPxn9RK7IWzijOcsMqLIFvZuXN7DkzW29me8xst5mtCy2rEI59nska/M9vmPuPcUwd9yp/+H1H+j32NAALFn1JUsUKXHB+vZPWyczKIjYmhslvv8zvO7bnyb++WNxle9qjg56hdevmpCydTetWzUlNTSMQCOS2V616FmPHvsxddz3C0WHR7j36MPKFoSxeNIuDBw8RCOTk2eb117fli8XLNHZfzALrUkh/pT8ZE18g5qq8Q2lWtgK+KrUJbFyZuyy6RQcy3h1O+sj7yV6xgJj2txV3yWETriv8ycBeoI1zLsk5lwxcGVo2uaCVzOweM1tmZsv+Pn5imEorfmXLlCEhIR6A1pc1Izs7m7379rNi1VoWfL6Etr/vxcAhz7H0q5UMGvY8AFUrV+LqK1oCwZu8//luc8TqL222b9tBrZrHrr5r1qjG9u078vRJS9tJl65307RZO558ajgA+/cfACAxsSwzZ4znyaeG8+XS5bnrLPnyK9pc1ZkWLa9n4cIlucM7R3XreoOGcyIoZ+t6rOJZkHBs+DWqYQuy16VATujNPCERX9WzyUndCED2N4uJqnVeJMoNi3AFfh3n3HDnXO5vkXNuh3NuOHB2QSs558Y455o455rc1fPWMJVW/Hbt3pN7Fbh67bfkOEeF8uV4+L7b+ef0d5nzj3GMGDaYZo0vYviQRwG4qnULli4PXnWkrFjN2bVqRKz+0iZl2dfUrXsOderUIjo6mq5db+TDWXPy9ElOrohZ8N7X4EF9GTsuePM1Ojqaf0x5i3ff/YCpUz/Ks07lyskAxMTEMHDA/YwZMyG3rVy5RFq3as7MmbPDeWhyAkuqkvvcV60O+KPh8M+5y/wXXkb26kXHVsg4hMXGY8nVAIj6TSNyfio9w6nhmpa51cweBcY553YCmFkVoDfwQ5j2GTEDhzxHyopV7Nt3gN916kGfO28jOzsbgG43Xcec+Z/z/rSPiPJHERcTw4hhg3PDpCB39ujKoGHPM+H96STExzFs8EPFcCTeEAgEePChJ/j4o/eI8vkYO+591q79D0OHDGDZVyuZNWsuV1xxGX955jEcjoULl9C3358A6NKlI61aXUpSckV69gwOD9x518OsXLmGAY/cR4frrsbn8/HGG+OZv+BYkHS68VrmzvuMw4fTI3LMpVXszX3xndMAS0gkvv9osuZ/AL4oALKXzcPf4FL8F7fCBQKQncmRyS/lrmsVKmPlk8nZsu7YBnNyODLzTeJueTh4kZZ+KN9x/5IqLNMyzawiMBi4EagCOGAnMBMY7pzbc6ptlKRpmZLXmT4tUwpXkqZlyskKm5YZlit859xeM3sHmAsscc4dPNpmZu2BT8OxXxERKVi4Zun0A2YADwDfmNnxHxH9azj2KSIihQvXGP7dQGPn3EEzqwN8YGZ1nHMvAYUPXouISFiEK/B9R4dxnHNbzKwNwdA/GwW+iEhEhGta5k4zu/joi1D4Xw9UAi4M0z5FRKQQ4Qr8nkCeT7I457Kdcz2B1mHap4iIFCJcs3RSC2lbVFCbiIiEj748TUTEIxT4IiIeocAXEfEIBb6IiEco8EVEPEKBLyLiEQp8ERGPUOCLiHiEAl9ExCMU+CIiHqHAFxHxCAW+iIhHKPBFRDxCgS8i4hEKfBERj1Dgi4h4hAJfRMQjFPgiIh6hwBcR8QgFvoiIRyjwRUQ8QoEvIuIRCnwREY9Q4IuIeIQCX0TEI8w5F+kaPMnM7nHOjYl0HfLL6PyVXF4+d7rCj5x7Il2A/Co6fyWXZ8+dAl9ExCMU+CIiHqHAjxxPjiGWIjp/JZdnz51u2oqIeISu8EVEPEKBLyLiEQr808DM2pvZt2a20cwG59Mea2bvh9q/NLM6x7U9Flr+rZm1O27522b2o5l9U0yHIScownltbWbLzSzbzG6ORI2Sv1P9/ljQy6Fzu8rMLinuGiNBgf8rmVkUMBq4FmgA3GpmDU7odiew1zlXF3gRGB5atwFwC3AB0B54NbQ9gLGhZRIBRTyv3wO9gfeKtzopgrEU/vtzLVAv9LgHeK0Yaoo4Bf6v1wzY6Jzb5JzLBCYBN57Q50ZgXOj5B8DvzMxCyyc554445zYDG0Pbwzn3GbCnOA5A8nXK8+qc2+KcWwXkRKJAKVgRfn9uBMa7oCVABTOrVjzVRY4C/9erAfxw3OvU0LJ8+zjnsoH9QHIR15XI0Lkp3Tx5fhX4IiIeocD/9bYBtY57XTO0LN8+ZuYHygO7i7iuRIbOTenmyfOrwP/1UoB6ZnaOmcUQvAk784Q+M4Feoec3A/9ywU+8zQRuCc3iOYfgDaSlxVS3FK4o51VKrplAz9BsnebAfudcWqSLCjd/pAso6Zxz2Wb2ADAbiALeds6tMbOngWXOuZnAW8AEM9tI8EbSLaF115jZZGAtkA3c75wLAJjZRKANUMnMUoEhzrm3ivnwPKso59XMmgLTgIpARzMb5py7IIJlS0h+vz9ANIBz7nXgY6ADwYkSh4HbI1Np8dJXK4iIeISGdEREPEKBLyLiEQp8ERGPUOCLiHiEAl9ExCMU+CJnIDN7PNI1SOmjaZlSYoS+cM6cc2fEl5WZWdTRz02EYdsHnXNlz5R6pHTQFb6c0cysTug76ccD3wC1zGygmaWEvsd82HF9nwz1/dzMJprZgFNsu7eZzTCzBWa2wcyGHNc23cy+MrM1ZnbPccsPmtlIM1sJtDCzp0K1fGNmY0JvSoS2+aKZLTOzdWbW1Mymhvbz5+O218PMlprZ12b2hplFmdlzQHxo2f8V1C+/ek7LD11KL+ecHnqcsQ+gDsGvH24eet2W4H9CbQQvWGYBrYGmwNdAHJAIbAAGnGLbvYE0gt9cGk/wDaVJqC0p9O/R5cmh1w7oetw2ko57PgHoGHq+ABgeev4gsB2oBsQS/GbGZKA+8CEQHer3KtAz9PzgcdstrF+eevTQo7CHvlpBSoKtLvid5RAM/LbAitDrsgS/gygRmOGcywAyzOzDIm57rnNuN4CZTQUuB5YB/czsplCfWqF97AYCwD+OW/9KM3sUSACSgDUEwxmOfffOamCNC31Xi5ltCm3zcqAxkBL6wyAe+DGfGn9XSL8T6xEpkAJfSoJDxz034Fnn3BvHdzCzh37htk+8ieXMrA1wNdDCOXfYzBYQ/MsBIMMd+76jOIJX202ccz+Y2dDj+gEcCf2bc9zzo6/9oWMZ55x77BQ1FtYvtx6RU9EYvpQ0s4E7zKwsgJnVMLOzgEUEv8AsLtR2fRG3d42ZJZlZPNAptJ3yBP9LysNmdj7QvIB1j4b7rtA+/9v/1/afwM2h+gnVcXaoLcvMoovQT6TIdIUvJYpzbo6Z1QcWh4Y3DgI9nHMpZjYTWAXsJDiMsh/AzO4Nrft6PptcSnBIpCbwrnNumZmtBu41s3XAt8CSfNbDObfPzN4kOMa/g+BXKv83x7LWzJ4A5piZD8gC7ge2ErxPscrMljvnuhfST6TINC1TSg0zK+ucO2hmCcBnwD3OueWF9O9NcDjmgeKqUSSSdIUvpckYM2tAcKhlXGFhL+JFusIXEfEI3bQVEfEIBb6IiEco8EVEPEKBLyLiEQp8ERGP+H8bcalIQKGLvQAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     }
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "The calculated RMSE scores can be visualized to comparatively study how model performance is affected by different parameters."
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "It can be seen from this visualization that RMSE first decreases and then increases as rank increases, due to overfitting. When the rank equals 20 and the regularization parameter equals 0.1, the model achieves the lowest RMSE score."
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### 3.5 Top K recommendation"
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "#### 3.5.1 Top k for all users (items)"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "source": [
    "dfs_rec = model.recommendForAllUsers(10)"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "source": [
    "dfs_rec.show(10)"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "+------+--------------------+\n",
      "|UserId|     recommendations|\n",
      "+------+--------------------+\n",
      "|   471|[[814, 3.7504895]...|\n",
      "|   463|[[814, 3.1264873]...|\n",
      "|   833|[[814, 3.3154662]...|\n",
      "|   496|[[814, 3.055388],...|\n",
      "|   148|[[814, 4.03012], ...|\n",
      "|   540|[[814, 3.8661027]...|\n",
      "|   392|[[814, 4.119951],...|\n",
      "|   243|[[814, 3.748784],...|\n",
      "|   623|[[814, 3.9018161]...|\n",
      "|   737|[[814, 3.8507497]...|\n",
      "+------+--------------------+\n",
      "only showing top 10 rows\n",
      "\n"
     ]
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "#### 3.5.2 Top k for a selected set of users (items)"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "source": [
    "users = dfs_train.select(als.getUserCol()).distinct().limit(3)\n",
    "\n",
    "dfs_rec_subset = model.recommendForUserSubset(users, 10)"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "source": [
    "dfs_rec_subset.show(10)"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "+------+--------------------+\n",
      "|UserId|     recommendations|\n",
      "+------+--------------------+\n",
      "|   471|[[814, 3.7504895]...|\n",
      "|   463|[[814, 3.1264873]...|\n",
      "|   148|[[814, 4.03012], ...|\n",
      "+------+--------------------+\n",
      "\n"
     ]
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "#### 3.5.3 Run-time considerations for top-k recommendations\n",
    "\n",
    "It is worth noting that usually computing the top-k recommendations for all users is the bottleneck of the whole pipeline (model training and scoring) of an ALS based recommendation system. This is because\n",
    "* Getting the top k from all user-item pairs requires a cross join which is usually very computationally expensive. \n",
    "* Inner products of user-item pairs are calculated individually instead of leveraging matrix block multiplication features which are available in certain contemporary computing acceleration libraries (e.g., BLAS).\n",
    "\n",
    "More details about possible optimizations of the top k recommendations in Spark can be found [here](https://engineeringblog.yelp.com/2018/05/scaling-collaborative-filtering-with-pyspark.html)."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "source": [
    "# cleanup spark instance\n",
    "spark.stop()"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "## References"
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "1. Yehuda Koren, Robert Bell, and Chris Volinsky, \"Matrix Factorization Techniques for Recommender Systems\n",
    "\", ACM Computer, Vol. 42, Issue 8, pp 30-37, Aug., 2009.\n",
    "2. Yifan Hu, Yehuda Koren, and Chris Volinsky, \"Collaborative Filtering for Implicit Feedback Datasets\n",
    "\", Proc. IEEE ICDM, 2008, Dec, Pisa, Italy.\n",
    "3. Apache Spark. url: https://spark.apache.org/docs/latest/ml-collaborative-filtering.html\n",
    "4. Seaborn. url: https://seaborn.pydata.org/\n",
    "5. Scaling collaborative filtering with PySpark. url: https://engineeringblog.yelp.com/2018/05/scaling-collaborative-filtering-with-pyspark.html\n",
    "6. Matrix Completion via Alternating Least Square (ALS). url: http://stanford.edu/~rezab/classes/cme323/S15/notes/lec14.pdf"
   ],
   "metadata": {}
  }
 ],
 "metadata": {
  "celltoolbar": "Tags",
  "interpreter": {
   "hash": "7ec2189bea0434770dca7423a25e631e1cca9c4e2b4ff137a82f4dff32ac9607"
  },
  "kernelspec": {
   "name": "python3",
   "display_name": "Python 3.6.9 64-bit ('.env': venv)"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}

一键复制编辑原始数据按行查看历史

提交于 2021-09-24 16:56 . Re-render als_deep_dive NB outputs

马建仓 AI 助手

尝试更多

代码解读

代码找茬

代码优化

https://gitee.com/shzgamelife/recommenders.git

git@gitee.com:shzgamelife/recommenders.git

shzgamelife

recommenders

main

shzgamelife / recommenders

简介

发行版

贡献者

近期动态

shzgamelife / recommenders .gitee-modal { width: 500px !important; }

简介

发行版

贡献者

近期动态

搜索帮助

shzgamelife / recommenders